Secure Architectures and Systems - OS Security Concepts Part 1

class: center, middle

### Secure Computer Architecture and Systems
***
# OS Security Concepts

---

# Introduction
- Operating system: foundation of a computer system
- Manages hardware/software resources shared between applications/users
- OS security is critical for the system's correct operation

--
- **Security goals:**
--

- Enforce which **subjects** (e.g. applications, users) can perform which **operations** (e.g. read, write) on which **objects** (e.g. files, system resources)
--

- Apply the **principle of least privilege** and maintain **confidentiality**, **integrity** and **availability**

--
  - Achieving this is often at odds with other key OS objectives:
      - Performance, scalability, resource usage, ease of use/compatibility

---
# Basic Process-level Security Invariants

1. **Inter-process isolation: processes cannot access (R/W/X) each other's state (mostly memory) <u>directly</u>**, i.e. *unmediated by the OS*
  - Enforced with page tables, by giving each process its own address space
--

2. **User/kernel isolation:**
  - **Processes cannot access (R/W/X) the kernel's memory directly**
        - Enforced with the user/supervisor execution modes and the associated bits in page table entries
  - **Processes can only enter the kernel through well-defined entry points**
        - System calls (and interrupts/exceptions) switch to supervisor mode and jump to a kernel-defined entry point

---
# Basic User-level Security Invariants

- **User authentication: only authorised users can access the system**
    - Enforced with an authentication mechanism, e.g. password, fingerprint, facial recognition
--
- **Users can configure whether/how the resources they own are shared with other users**
  - Implemented with file permissions
  - In UNIX/Linux, files abstract many types of system resources
--

- **Only privileged users (administrators) can perform security-critical tasks**
  - E.g. loading kernel code, shutting down the computer, mounting filesystems

---
# Basic Trust Model

.leftlargecol[
<div style="text-align:center"><img src="include/threat-landscape-1.svg" width=600 /></div>
]

.rightsmallcol[
- The entire kernel is trusted
- Local and remote applications and users are not trusted
]

---
# Basic Trust Model

.leftlargecol[
<div style="text-align:center"><img src="include/threat-landscape-2.svg" width=600 /></div>
]

.rightsmallcol[
- Hardware is also assumed to behave correctly
]

---
# Basic Trust Model

.leftlargecol[
<div style="text-align:center"><img src="include/threat-landscape-3.svg" width=600 /></div>
]

.rightsmallcol[
- The system administrator has ambient authority
- Certain apps (e.g. to log a user in or change passwords) are privileged
]

---
# Basic Trust Model

.leftlargecol[
<div style="text-align:center"><img src="include/threat-landscape-4.svg" width=600 /></div>
]

.rightsmallcol[
- BIOS, bootloader and boot process are also trusted
]

---
# Basic Trust Model

.leftlargecol[
<div style="text-align:center"><img src="include/threat-landscape-4.svg" width=600 /></div>
]

.rightsmallcol[
.center[
  **Does this model reflect reality?**
]
]

---
# Realistic Trust Model

.leftlargecol[
<div style="text-align:center"><img src="include/threat-landscape-5.svg" width=600 /></div>
]

.rightsmallcol[
- BIOS/bootloader may have bugs or be compromised
- A local attacker could replace the on-disk kernel image with a malicious one
]

---
# Realistic Trust Model

.leftlargecol[
<div style="text-align:center"><img src="include/threat-landscape-6.svg" width=600 /></div>
]

.rightsmallcol[
- The kernel has bugs
- Particularly problematic in third-party code, e.g. drivers
]

---
# Realistic Trust Model

.leftlargecol[
<div style="text-align:center"><img src="include/threat-landscape-7.svg" width=600 /></div>
]

.rightsmallcol[
- The hardware has vulnerabilities
- E.g. transient execution attacks such as Spectre and Meltdown
]

---
# Realistic Trust Model

.leftlargecol[
<div style="text-align:center"><img src="include/threat-landscape-8.svg" width=600 /></div>
]

.rightsmallcol[
The computer owner or system administrator may not even be trusted in certain scenarios
]

---
# Apps/Kernel Isolation

.leftlargecol[
<div style="text-align:center"><img src="include/threat-landscape-9.svg" width=600 /></div>
]

.rightsmallcol[
For now, let's assume the basic trust model and focus on process-level isolation
]

---
# Process-level Isolation

1. **Processes should be isolated from each other**
  - No memory access, nor other forms of interference
--

2. Isolation is enforced by the kernel, so **the kernel must itself be isolated from applications**
  - Same principle: no memory access, no interference
  - But applications still need to be able to invoke the kernel

.leftcol[
<div style="text-align:center"><img src="include/process-level-1.svg" width=250 /></div>
]

.rightcol[
- Processes cannot access each other's memory directly
- But a process could exploit a kernel bug to interfere with another process
]

---
# User -> Kernel Attacks

- Kernels are mostly written in memory-unsafe languages (e.g. C for Linux)
- They are subject to all the memory safety/UB vulnerabilities we saw previously
- General-purpose kernels are huge: millions of lines of code
  - Cannot guarantee the absence of bugs (their sheer size almost guarantees their presence!)

.leftcol[
<div style="text-align:center"><img src="include/process-level-2.svg" width=250 /></div>
]

.rightcol[
- Main attack surface from user space applications: the **system call interface**
- **Very large** interface, hard to secure
]

---
# Hardening the System Call Interface

- The Linux kernel treats **all data flowing from user space through system calls as untrusted**
  - Syscall parameters + for pointer parameters, the data they point to
--

- Must assume user space may send through the system call interface:
  - Corrupted data structures
  - Bad indexing information
  - NULL or invalid pointers, e.g., to unmapped/unallocated memory
  - References to resources (e.g. files) the calling process is not allowed to access
  - System call sequences invoked in the wrong order
  - Etc.

--
- Requires the kernel to **check** the validity of this data/control flow
  - Hard to get all checks right due to the complexity of the interface

---
# System Calls: Pointer Parameters

- A process often passes the kernel a pointer to some of its memory
  - E.g. `readv`/`writev` system calls
- Validating the data pointed to requires special care

.leftcol[
<div style="text-align:center"><img src="include/tocttou-1.svg" width=400 /></div>
]

---
# System Calls: Pointer Parameters

- A process often passes the kernel a pointer to some of its memory
  - E.g. `readv`/`writev` system calls
- Validating the data pointed to requires special care

.leftcol[
<div style="text-align:center"><img src="include/tocttou-2.svg" width=400 /></div>
]

.rightcol[
- User space can use another thread to modify the data after the check
- **Time of check to time of use** (TOCTTOU) vulnerability, here a **double fetch**
]

---
# System Calls: Pointer Parameters

- Main defence against TOCTTOU: the kernel must **copy** all user data passed by reference into kernel memory
  - Validity checks and subsequent uses are performed on this copy

```c
unsigned long copy_from_user(void *to, const void __user *from, unsigned long n);
unsigned long copy_to_user(void __user *to, const void *from, unsigned long n);
```

---
# User -> Kernel Attacks

By exploiting a kernel vulnerability, an attacker can aim to:
- **Leak/tamper with kernel memory**
  - E.g. to read kernel pointers and break KASLR, or to escalate privileges (e.g. gain root)
--

- **Access other processes' memory**
  - The kernel can access all of the computer's memory
--

- **Execute code**, possibly arbitrary, in the context of the kernel (with full privileges)
  - E.g. to install and hide malicious programs (rootkits)

--
- **Denial of service (DoS)** against the system or other applications
- Etc.

---
# Kernel Vulnerabilities

- Chen et al., **"Linux kernel vulnerabilities: State-of-the-art defenses and open problems"**
--

- Study of one year of Linux kernel vulnerabilities (141 CVEs, 2010), classified as:
  - Missing pointer check
  - Missing permission check
  - Buffer overflow
  - Integer overflow
  - Uninitialised data leaked to user space
  - Memory mismanagement (leak, use-after-free, double free)
  - Miscellaneous: NULL dereference, division by zero, infinite loop, race condition, deadlock

---
# Kernel Vulnerabilities

.center[Vulnerabilities vs. their impact]

---
# Kernel Vulnerabilities

.center[Location of vulnerabilities in the kernel code base]

---
class: middle, center, inverse

# Linux: Runtime Defences

---
# Attack Surface Reduction

- **Strict kernel memory permissions**
  - Kernel executable code and read-only data must not be writable
  - Kernel function pointers and sensitive variables should be read-only where possible
--

- Segregate kernel and user space memory (hardware-dependent, x86 names below; Arm: PXN/PAN):
      - **Supervisor Mode Execution Prevention (SMEP)** prevents the kernel from executing code in user space memory
      - **Supervisor Mode Access Prevention (SMAP)** prevents the kernel from reading/writing user space memory (temporarily lifted during `copy_{to,from}_user`)
      - Protects against the kernel being tricked into using user space code or data pointers (ret2usr attacks)

---
# Attack Surface Reduction

.leftlargecol[
- **Reduce applications' access to system calls: system call filtering**
  - Restrict the system calls an application can invoke to those it legitimately needs
  - Widely used in production to harden multi-tenant and sensitive environments: Docker containers, Android, Flatpak, etc.
- Implemented with **seccomp** on Linux
- Problem: **how to derive precise, per-application system call allowlists/denylists?**
]

.rightsmallcol[
<div style="text-align:center"><img src="include/syscall-filtering.svg" width=200 /></div>
]

---
# Probabilistic Defences

- **Kernel stack canaries**
- **Kernel Address Space Layout Randomisation (KASLR)**
--

- These are not a panacea
  - A single canary value per task, shared by all its stack frames: can be leaked with e.g. a stack buffer overread
  - KASLR is coarse-grained: one random offset per kernel memory region
      - E.g. kernel image, modules, kernel stacks, vmalloc area, physmap
      - A single pointer leak may break KASLR for an entire region

---
# Memory Integrity

.leftlargecol[
- Detect kernel stack overflows/underflows with **guard pages**
  - Unmapped memory pages that fault when accessed
- **Sanity-check heap free lists for corruption** upon allocation/free
- **Detect integer overflows** (e.g. reference counters, size variables)
- Protect return addresses with **shadow stacks** rather than canaries

Some features are optional or not enabled by default (performance cost)
]

.rightsmallcol[
<div style="text-align:center"><img src="include/guard-page.svg" width=200 /></div>
]

---
# Prevent Kernel Information Leaks

- **Avoid exposing kernel pointers to user space**
  - A single pointer leak can break KASLR
  - Avoid copying partially initialised/uninitialised data structures or buffers to user space
  - Kernel logs or files containing pointers should be readable only by the administrator
--

- **Don't use kernel addresses as identifiers visible to user space**: use e.g. atomic counters instead

--
- **Poison/zero out freed memory** to counter reuse attacks (reading uninitialised memory, use-after-free, etc.)
  - Can impact performance

---
class: middle, center, inverse

# Linux: Bug Detection

---
# Dynamic Analysis Techniques

- **Sanitisers:** KASAN, UBSAN, memory leak (kmemleak) and data race (KCSAN) detectors, etc.
- **Lockdep:** tracks lock usage to detect potential deadlocks, double locking, lock order inversion
- **Dynamic tracing/instrumentation tools:** ftrace, perf, eBPF
- **Fuzzing:** Syzkaller

---
# Syzkaller

.leftcol[
- Widely used kernel fuzzer
- Issues malformed system calls from user space in the hope of triggering kernel bugs
- Kernel under test runs in VMs, fuzzing controlled from the host
- Works best with KASAN and coverage collection (KCOV) enabled to guide fuzzing
- For each fuzzing round, a program (a short sequence of syscalls) is generated and executed
]

.rightcol[
<div style="text-align:center"><img src="include/syzkaller.svg" width=340 /></div>
]

---
# Syzkaller: Syzlang

- Syzkaller is grammar-based and has detailed knowledge of the target interface
- **Syzlang** language describes system calls and the data flow between them
  - Syscall arguments and their types
  - Values passed between system calls (e.g. an `fd` created by `open()` can be used by `read()`)
  - Length parameters specifying the size of other parameters

.leftcol[
- Good knowledge of the syscall interface lets Syzkaller optimise program generation/mutation
]

.rightcol[
```xml
resource fd[int32]: 0xffffffff, AT_FDCWD
resource sock[fd]
resource sock_unix[sock]

socket(...) sock
accept(fd sock, ...) sock
listen(fd sock, backlog int32)
```
]

---
# Syzkaller: Syzbot

- **Syzbot:** continuous fuzzing of Linux, Android, FreeBSD, NetBSD, OpenBSD and gVisor on 25 Syzkaller instances (~150-200 VMs in total)
- Has reported thousands of bugs to the Linux kernel mailing lists
- https://syzkaller.appspot.com
- A large proportion (about two thirds) of reported bugs are invalid

---
# Linux Test Project

.leftcol[
- Repository containing thousands of test cases for the kernel
- https://github.com/linux-test-project/ltp/tree/master
- Example categories:
  - Syscalls
  - POSIX conformance
  - Filesystems
  - Networking
  - Memory management
  - Reproducing existing CVEs
  - Etc.
]

.rightcol[
<div style="text-align:center"><img src="include/ltp.png" width=300 /></div>
]

---
# Static Analysis Techniques

- **Pattern-based analysis**
  - Coccinelle, Smatch: describe patterns of programming mistakes
      - E.g. each call to `kmalloc` should be matched by a `kfree`
--
- **Control and data flow analysis tools**
  - Sparse uses source annotations to express properties of pointers and locks
      - E.g. `__user` marks pointers to user space, `__acquires` marks a lock held on function exit but not on entry, etc.
--
- Others: formal verification, symbolic execution, compiler-based techniques
--

- Some downsides of static analysis (scalability, state explosion) are exacerbated for OS kernels due to the size of their code base
  - E.g. Linux has tens of millions of lines of code

---
# Summary

- **Basic OS security invariants:**
  - Processes cannot directly access each other's memory
  - Processes cannot directly access the kernel's memory
- Old trust model: the entire kernel, hardware, sysadmin and privileged apps, BIOS/bootloader/boot process are fully trusted
  - **Does not reflect modern environments**
- The system call interface is the main vector for user -> kernel attacks
- We saw some kernel runtime defences: attack surface reduction, KASLR, etc.
- We also saw some bug detection techniques: fuzzing, testing, static analysis