Software Compartmentalisation Policies

You can access the slides 🖼️ for this lecture.

Here we cover the abstractions used by programmers to implement compartmentalisation policies in applications.

Abstractions

Abstraction is a term commonly used in computer science. An abstraction refers to a simplification layer whose goal is to ease the use of a software or hardware component. The abstraction will hide all the unnecessary internal details of the component in question, and will expose a convenient and clear interface for it to be invoked.

Let us illustrate the concept with an example. We have a complex component that has a lot of different functionalities. We can create an abstraction that somehow handles all of these different functionalities, and exposes them under a higher-level, simpler interface for clients to use:

Here the client can be the programmer, another software or hardware layer, etc. Another concrete example is that of an operating system, e.g., the Linux kernel. The kernel itself implements tons of functionalities, remember that it is made of more than 20 million lines of code. The system call interface abstracts all these functionalities for user space applications under a relatively small interface made of a series of system calls that can be invoked to request any service from the OS. In other words, the system call interface is an abstraction of the operating system (complex layer) for client applications (client).

Abstractions are absolutely everywhere in computer science. We can list a few more examples of abstractions:

• A programming language is an abstraction of assembly instructions, simplifying the development of programs using high-level statements, functions, loops, etc.
• Each system call is an abstraction of relatively complex features implemented by the operating system; for example, the fork UNIX primitive, implemented through the Linux clone system call, abstracts many details of process creation and resource duplication.
• A file is an abstraction of data stored on disk, indexed with a particular path on the filesystem. The operations one can apply to a file, such as reading from or writing to it, abstract away complex mechanisms such as drivers communicating with devices following specific protocols, block allocation, request scheduling, data and metadata caching, etc.

Compartmentalisation Abstractions

A compartmentalisation abstraction is a simplification layer that exposes primitives for the programmer to express compartmentalisation policies within a program. Using such abstractions, the programmer can express many things:

• What part of the application goes into what compartment.
• What and where are the compartment boundaries.
• What data should be shared between compartments, or private to the containing compartment.
• When and how compartments should be created and destroyed.
• How to manage compartments’ permissions.
• Etc.

Main Abstraction Categories

The compartmentalisation abstractions one can use are very different depending on what compartmentalisation approach, framework and tools one decides to use. However, most abstractions will fall within one of the following main categories:

• CREATE and DESTROY for compartment creation and destruction.
• ASSIGN to assign permissions to a compartment.
• CALL and RETURN to invoke the execution of and return from a compartment.

CREATE, DESTROY, ASSIGN Abstractions

The first category is CREATE and DESTROY: the programmer needs a way to express the creation and destruction of compartments in the code. They are illustrated in steps 1 and 2 in the diagram below:

Examples of abstractions belonging to the CREATE and DESTROY categories are calls to fork and exit, if one chooses to use process-based compartmentalisation. Another important category of abstraction is ASSIGN. It corresponds to abstractions that let the programmer assign permissions to compartments. For example, as illustrated in step 3 in the diagram above, we could have a part of the memory be accessible only by the green compartment, and another part of the memory accessible only by the orange one. We’ll see in the next lecture that there are several mechanisms that allow us to enforce these permissions. An example would simply be again to run each compartment within its own process; this way they would have separate address spaces.

CALL/RETURN Abstractions

Because they belong to the same application, at some point compartments need to communicate with each other and transition execution from one to another. These cross-compartment transitions are security domain switches, realised with abstractions belonging to the CALL and RETURN categories, as illustrated in step 4 in the diagram above.

If we zoom in on CALL and RETURN abstractions, they will generally enforce a series of security properties. First, we need some form of control flow integrity between compartments: we cannot let an untrusted caller compartment jump to arbitrary code addresses within another callee compartment; that would be equivalent to letting the caller execute arbitrary code within the context of the callee. Hence, CALL and RETURN abstractions must make sure that cross-compartment transitions only target legitimate locations in the code. For example, a compartment should only be called through the API it exposes. Another important aspect of CALL and RETURN abstractions is that they must switch the CPU state from that of the caller compartment to that of the callee’s, e.g., switch stacks and clear registers, as part of the cross-compartment transitions. Without this, the old stack and register content may contain important information that would leak across compartments.

CALL and RETURN abstractions can be synchronous or asynchronous. With synchronous calls and returns, compartment switches are relatively similar to function calls and returns: the caller compartment blocks waiting for the callee to return. This is illustrated on the left of the diagram below. If the compartments run concurrently, CALLs and RETURNs can also be asynchronous, meaning that the caller does not wait for the callee to finish processing the call. This is illustrated on the right of the diagram. In that case, the application is closer to a distributed system, and transitions between compartments become remote procedure calls involving message-passing communication.

Implicit/Explicit Abstractions

Abstractions can be explicit, meaning they are exposed to the programmer, who must explicitly use them, for example by placing annotations. This involves a certain amount of engineering effort, depending on how easy it is to use the abstractions. Abstractions can also be implicit, requiring no intervention from the programmer because they are applied automatically. For example, you could have a compartmentalisation framework placing each library within its own compartment and automatically handling compartment creation, destruction, and transitions.

Because most existing approaches to compartmentalisation are code-centric, CREATE and DESTROY abstractions are often implicit and managed automatically. For example, when placing a library within its own compartment, that compartment will be created when the application starts, and destroyed when the application exits. The more automated an approach is, the more implicit abstractions it will support, including ASSIGN, CALL and RETURN.

Properties Enforced

Most of the existing software compartmentalisation abstractions will at least enforce integrity, which is a prerequisite to enforce the other properties (confidentiality, availability). Confidentiality is also a property commonly enforced.

Concerning availability, it is not supported by most abstractions. The reason why there is almost no compartmentalisation work targeting availability is rather simple: it is extremely difficult to achieve, and in most cases it requires entirely redesigning the application one wishes to compartmentalise from scratch. Indeed, to be able to preserve availability in the presence of one or more malicious components, the application needs to be reworked into a fault-tolerant distributed system. That requires specific abstractions: compartments need to run concurrently, and CALL/RETURN abstractions need to be asynchronous. We need to enforce performance isolation and bounded resource consumption across compartments, because we can’t let a malicious compartment starve the rest of the application of resources. The TCB as well as the interfaces need to be redesigned to place as much state as possible outside of compartments that may be prone to crashes, because we want to be able to restart them in the case they fail. All of this is obviously extremely complicated, and that complexity needs to be added on top of the difficulty of maintaining confidentiality and integrity, which is already quite hard.

Composing with Other Abstractions

One last important consideration about compartmentalisation abstractions is how they compose with other system abstractions.

Processes and Threads

For example, regarding threads and processes, they can either be orthogonal to compartments, with one thread or one process able to execute multiple compartments, and compartment transitions realised in the context of the thread or process. This is illustrated on the left of the diagram below, with two threads running several compartments, transitioning upon calls and returns:

Processes or threads can also be coupled with the compartments. In that case, each process or thread executes a single compartment only, and upon CALLs and RETURNs the running thread or process needs to be switched. This is illustrated on the right of the diagram above.

CPU privilege levels

When compartmentalising an application, the abstractions used are influenced by the user/kernel interface. In particular, as many compartmentalised systems require a privileged monitor to perform security domain transitions, the kernel can play that role. This is the case implicitly when using process-based compartmentalisation. The monitor can also be placed in user space, which may have some performance benefits or drawbacks, depending on how the monitor is implemented and isolated.

When compartmentalising the kernel or another privileged entity like a virtual machine monitor, things are more complicated because these entities run with full privileges, but we need to reduce some of these privileges for certain compartments. The goal here is to establish a trusted computing base that will play the role of the aforementioned monitor, and to isolate it from the rest of the kernel or the hypervisor.

Monitor-Level Side Channels

The isolation mechanisms used for compartmentalisation, which we will cover in the next lecture, are quite robust, and in most cases, combined with good interface security, they will prevent compartment A from directly accessing the private memory of compartment B. Yet another attack vector is the secure monitor, which in many cases is the kernel. If not properly secured for compartmentalisation, some kernel interfaces may allow a malicious compartment to access memory supposed to be private to other compartments:

Kernel interfaces are well secured when considering process-based compartmentalisation, because the kernel is already designed to provide good isolation between processes. But when using some of the more modern isolation mechanisms, these interfaces may become concerning, because the kernel was not designed with these isolation mechanisms in mind. Examples of such monitor (kernel) level side channels, which may allow the isolation enforced by certain mechanisms to be bypassed, include the system call interface (e.g., a malicious compartment able to mmap memory from another victim compartment), or pseudo-filesystems (e.g., a malicious compartment having access to /dev/mem, which holds the contents of all physical memory).