Virtualization 101 - Lightweight Virtualization: Containers & Unikernels

class: center, middle
background-image: url(include/title-background.svg)
# .right[Virtualisation 101]
### .right[Lightweight Virtualisation:  
Containers & Unikernels]
<br>
.right[Pierre Olivier  
.right[[pierre.olivier@manchester.ac.uk](mailto:pierre.olivier@manchester.ac.uk)]
]

---
class: center, middle, inverse

# Lightweight Virtualisation: Motivation & Definition

---
# Lightweight Virtualisation
<div style="text-align:center"><img src="include/lightweight1.svg" width=700 /></div>

---
# Lightweight Virtualisation
<div style="text-align:center"><img src="include/lightweight2.svg" width=700 /></div>

---
# Lightweight Virtualisation
<div style="text-align:center"><img src="include/lightweight3.svg" width=700 /></div>

---
# Lightweight Virtualisation
<div style="text-align:center"><img src="include/lightweight4.svg" width=700 /></div>

---
# Lightweight Virtualisation
<div style="text-align:center"><img src="include/lightweight5.svg" width=700 /></div>

---
# Lightweight Virtualisation
<div style="text-align:center"><img src="include/lightweight6.svg" width=700 /></div>

---
# Lightweight Virtualisation: Definition

- Lightweight virtualisation represent virtualisation solutions aiming at
  providing, compared to traditional virtual machines:
  - **Lower memory footprint**: 0 to a few MBs per virtualised instance
  - **Faster boot times**: in the order of micro/milliseconds
  - Lower disk footprint: in the order of KBs/MBs
--

- These metrics regards the systems software, part of the boot time/memory or disk footprint will be application-specific
--

- Can approach these lightweightness objectives with small traditional Linux VMs: **micro VMs**
- Brief overview of two (quite different) technologies that go one step further: **containers** and
**unikernels**

---
class: middle, center, inverse

# Containers

---
# Introducing Containers

.leftcol[
- **Containers: process-level sandboxing technologies**
  - Enforced by the **operating system**
  - Sometimes called OS-level virtualisation
]

.rightcol[
<div style="text-align:center"><img src="include/intro.svg" width=500 /></div>
<br>
]

---
# Containers: Key Idea

- **The OS restricts the visibility on system resources for a process
  or a set of processes**
  - Filesystem, network interfaces, PIDs, etc.
--

- **The OS also controls hardware resource allocation/usage between such isolated processes**
  - CPU scheduling cycles, memory, disk/network bandwidth, etc.
--

- Conceptually it achieves the same isolation goals as a VM: process(es) are confined in this virtualised environment, **a container**
--

- **A container is much lighter than a traditional VM**
  - Per-container system memory/disk footprint close to 0
  - Boot time is that of spawning a process, i.e. micro seconds
--

- Still containers are not an ideal form of virtualisation
  - **Security issues**

---
# Containers: Use Cases

.left3col[.medium[
- **Software development/testing/deployment**
  - Develop, build and test in a controlled, identical environment
  - Deploy in the same environment as the development one (repeatability)
      - Can be deployed on any machine supporting containers independently of the host configuration
]]

.right3col[
<div style="text-align:center"><img src="include/docker.png" width=200 /></div>
<br>
]

.left3col[.medium[
- **Lightweight (low cost) & elastic virtualisation**
  - Containers consume few resources and can be brought up/destroyed very fast
  - Cloud services such as Gmail and Facebook make extensive use of containers
  - Function as a Service (e.g. AWS Lambda)
]]

.right3col[
<div style="text-align:center"><img src="include/lambda.png" width=100 /></div>
]

---

## Containers: Restricting System Resources Visibility with Namespaces

- **Filesystem/mount points** (~chroot)
  - Container cannot see host/other containers's file systems
  - E.g. can run a Fedora rootfs in a container on a Debian host
--

- **Network stack**
  - Container has its own IP, virtual bridged/routed network
--

- **Processes: PIDs and IPCs**
  - Isolated PID set, cannot see or communicate with host/other containers' processes
--

- **Host and domain name**
--

- **User IDs**
  - Can have root privileges inside container

--
- Achieved with Linux through a feature called **namespaces**

---
## Containers: Controlling Hardware Resources Access with Control Groups

- **Memory**
  - Limits memory and swap usage
--

- **CPU**
  - Limit CPU usage (can be for example 1.5 CPU) and CPU sets
  - Control CFS quotas
--

- **Devices**
  - Enable/disable access to certain devices
--

- **Block I/O**
  - Control throughput
--

- **Network I/O**
  - Control network traffic priority
--

- Achieved under Linux through a feature called **control groups**

---
# Containers: Different Technologies

- **Chroot (1982)**
  - Generally for UNIX-like OS, introduced in 1982 (BSD)
  - Runtime switch to another root file system
  - Goal: testing installation and build system of BSD
  - Chroot isolates only the filesystem, no isolation/control for memory usage,
   network, I/O, PIDs/processes, etc.
- **FreeBSD Jails (2000)**
- **Solaris Zones (2004)**
- **Linux OpenVZ (2005)**
- **In Linux, high-level APIs leveraging namespaces/control groups**:
  - LXC (2008)
  - Docker (2013)

---
# Containers vs. VMs

Pros of containers and VMs:

| Containers | VMs |
| ---------- | --- |
| Low memory/disk usage | OS diversity |
| Fast Boot times | Kernel version |
| Per host density | Performance isolation |
| Nesting | **Security** |

---
# Containers and Security
<div style="text-align:center"><img src="include/secu1.svg" width=800 /></div>

---
# Containers and Security
<div style="text-align:center"><img src="include/secu2.svg" width=800 /></div>

---
# Containers and Security
<div style="text-align:center"><img src="include/secu3.svg" width=800 /></div>

---
# Containers and Security
<div style="text-align:center"><img src="include/secu4.svg" width=800 /></div>

---
# Containers and Security
<div style="text-align:center"><img src="include/secu5.svg" width=800 /></div>

---
# Containers and Security

- **The isolation enforced by a host OS between containers is not trusted to
  be as strong as that enforced between VMs by a hypervisor**
  - Due to the size and complexity of the interface between a container
    and the host OS: the **system call interface**
--
- Attempts at securing containers:
  - Run containers within virtual machines...

<div style="text-align:center"><img src="include/kata.png" width=600 /></div>
--

- **But can we get both lightweightness and security?**

---
class: center, middle, inverse

# Unikernels

---
# Unikernels

---
# Unikernels

---
# Unikernels: Definition

- **Unikernel: application + dependencies + thin OS compiled as a static binary running
on top of a hypervisor**
--

- **Single-***
  - **Single purpose**: run 1 application
      - Want to run multiple applications? Run multiple unikernels
--
  - **Single process**
      - Want to run a multiprocess application? Run multiple unikernels
      - However, SMP (multicores) and multithreading are supported
--
  - **Single binary** and single address space for application + kernel
      - No user/kernel protection needed

.small[
Madhavapeddy et al., “Unikernels: Library Operating Systems for the Cloud”, ASPLOS’13
]

---
# Unikernels: Benefits

.left4col[.medium[
- **A form of lightweight virtualisation**
  - Contain and run only what is absolutely necessary to the application
  - Cost advantage: memory/disk footprint reduction
  - **Considered as a secure alternative to containers**
      - Unikernels are virtual machines!
- **Per-application tailored kernel** (LibOS/Exokernel model)
  - Specialisation for lightweightness but also performance
- **Reduced OS noise, increased performance**
  - Low system call latency: app + kernel in ring 0, system calls are function calls
  - Sub-second boot time
]]

.right4col[
<br>
<div style="text-align:center"><img src="include/lightweight8.svg" width=150 /></div>
]

---
# Unikernels: Application Domains

- Cloud applications: servers, microservices, SaaS
- Embedded virtualisation, Edge computing, IoT
- Network Function Virtualisation, HPC, VM introspection, malware analysis, secure
Desktop applications
- etc.

- **Contrary to containers which are a mature and widespread technology,
  unikernels are still at the stage of research prototypes**

---
# Unikernel Projects

- Unikernels can be classified based on the targeted language for supported applications
  - Pure memory safe languages (OCamL, Erlang, Haskell): [MirageOS](https://mirage.io/),
    [LING](https://github.com/cloudozer/ling), [HalVM](https://github.com/GaloisInc/HaLVM)
  - C/C++, semi-posix API: [Unikraft](https://unikraft.org/),
    [HermiTux](https://ssrg-vt.github.io/hermitux/),
    [HermitCore](https://hermitcore.org/), [OSv](http://osv.io/),
    [Rumprun](https://github.com/rumpkernel/rumprun),
    [Lupine](https://github.com/hckuo/Lupine-Linux)
  - Rust/Go: [Hermit](http://hermit-os.org/),
    [Clive](https://lsub.org/clive/)
  - More: https://unikernelalliance.org/projects/

<div style="text-align:center"><img src="include/uk-logos.png" width=800 /></div>
---
# Unikernels: Compatibility Issues
> Unikernels are lightweight AND secure, why didn't I hear of them?
--

> **Because it's hard to port existing applications!**

---
## Unikernels & Application Compatibility
<div style="text-align:center"><img src="include/porting1.svg" width=800 /></div>

---
## Unikernels & Application Compatibility
<div style="text-align:center"><img src="include/porting2.svg" width=800 /></div>

- **Proprietary software → source code not available**

---
## Unikernels & Application Compatibility
<div style="text-align:center"><img src="include/porting3.svg" width=800 /></div>

- Proprietary software → source code not available
- **Incompatible language**

---
## Unikernels & Application Compatibility
<div style="text-align:center"><img src="include/porting4.svg" width=800 /></div>

- Proprietary software → source code not available
- Incompatible language
- **Unsupported features**

---
## Unikernels & Application Compatibility
<div style="text-align:center"><img src="include/porting5.svg" width=800 /></div>

- Proprietary software → source code not available
- Incompatible language
- Unsupported features
- **Porting is hard, needs knowledge about both application and unikernel**

---
## Unikernels & Application Compatibility
<div style="text-align:center"><img src="include/porting6.svg" width=800 /></div>

- Proprietary software → source code not available
- Incompatible language
- Unsupported features
- Porting is hard, needs knowledge about both application and unikernel
- **Complex build toolchains**

---
## Unikernels Targeting Compatibility

.medium[
Compatibility with existing applications developed for popular OSes (e.g. Linux)
achieved in unikernels in different ways:
- **Source-compatible** (HermitCore, Rumprun)
  - Recompile your application's code with a custom libc + unikernel kernel
  - *Weak form of compatibility*:
      - Libc is not the only interface with the kernel, still requires recompilation
]
--
.medium[
- **Libc-level binary-compatible** (OSv, Lupine):
  - (Sometimes) avoids recompilation, still limited to programs requesting
    OS services only through the libc
]
--
.medium[
- **System call-level binary compatible** (HermiTux, Unikraft)
  - The strongest form of compatibility targeting the standard app/OS interface
  - Runs unmodified Linux apps as unikernels
]

.small[P. Olivier et al., *A Syscall-Level Binary-Compatible Unikernel*, IEEE Transactions on Computers, 2021]

---
# Unikernels: Performance Benefits

- **Kernel + application in protection ring 0, system calls are function calls**
  - Significant decrease in syscall latency

<div style="text-align:center"><img src="include/uk-perf2.png" width=390 /></div>
.center[.medium[Redis throughput under various setups (higher is better)]]

.small[S. Kuenzer et al., *Unikraft: Fast, Specialized Unikernels the Easy Way*, EuroSys'21]

---
class: inverse, center, middle

# Wrapping Up

---
# Wrapping Up

- Heavyweight nature of traditional VMs calls for **lightweight virtualisation**
  in many scenarios
  - Low system memory disk footprint, fast boot times
--

- **Containers** are a form of OS-level lightweight virtualisation
  - Widespread use, in particular for software development/testing/deployment
  - Security concerns
--

- **Unikernels** are single-purpose minimal VMs
  - Lightweight and secure
  - Performance benefits
  - Still at the stage of research prototypes, main issue hindering adoption: compatibility with existing apps