Trusted Execution Environments

You can access the slides 🖼️ for this lecture.

This is a brief introduction to Trusted Execution Environments. These represent hardware-assisted isolated computing environments, which enforce a specific trust model that is particularly strong.

Introduction

Imagine a server owned by a cloud provider, sitting in a data center and used for cloud computing. This cloud provider executes several virtual machines on top of that server, and each of these machines is rented to a different client. In the context of cloud computing, these clients are named tenants.

Let us reason about the different trust models at play in that scenario. From the cloud provider’s point of view, the hardware is trusted, so is the privileged software handling the virtualisation: that is the virtual machine monitor, some host-level software, and so on. What the cloud provider does not trust is: everything run by tenants within the virtual machines, i.e., tenants’ applications and guest operating systems. The cloud provider has no idea what the tenants will run in their VMs, and needs to assume that software may be malicious.

If we now take the point of view of one of the tenants, they trust their own software. Tenants also trust the cloud provider’s hardware and software such as the virtual machine monitor. Finally, tenants distrust each other: processes running on the same OS, we don’t want a VM belonging to one tenant accessing the memory of a VM belonging to another tenant.

Trusted Execution Environments (TEEs) target the enforcement of a stronger trust model. They still consider that the cloud provider distrusts tenants and that tenants distrust each other. But there is an additional component: tenants do not trust the cloud provider. This makes sense today: a lot of companies are reluctant to move their workloads to the cloud because they don’t want the big cloud providers to get access to their code and data. In that context, the objective of TEEs is to create an isolated execution environment and maintain confidentiality and integrity for software executing in it, in the presence of untrusted privileged software. Here, untrusted privileged software concretely means that we consider a VM that does not trust the hypervisor managing it, or a process that does not trust the operating system it runs on top of. In our cloud scenario, the distrust relationships between provider and tenants can then be illustrated as follows:

What is a TEE?

A TEE is a hardware-enforced isolated execution context in the CPU. The isolation is strong enough so that the privileged layers, namely the OS or the hypervisor, can neither read nor write within that environment. Because they protect against read and write accesses, TEEs enforce confidentiality and integrity. They cannot really enforce availability: that is too difficult in the presence of an attacker strong enough to run privileged software. For example, because the attacker can run OS code in supervisor mode on the CPU, nothing prevents them from simply shutting down the machine to affect availability. A classic TEE threat model assumes attackers may control: all host software outside the TEE, including privileged layers such as the OS or hypervisor, as well as I/O devices.

Key Characteristics

We are going to see next three different implementations of trusted execution environments. They all share the following key characteristics. First, as previously mentioned, they all enforce strong isolation for the software running within the TEE, through some form of isolation mechanism. The outside world, including the privileged software like the OS, cannot read or write the memory allocated to the TEE. Second, this isolation implies controlled interactions between the TEE and the outside world. Indeed, a TEE is not entirely self-sufficient and needs to interact with the OS or the hypervisor to perform I/O. These interactions need to be secured as, contrary to traditional trust models, the OS and the hypervisor are not trusted by the TEE.

TEEs also provide some secure storage for sensitive data such as cryptographic keys or passwords. That secure storage is itself realised with encryption keys that never leave the TEE. Finally, because most TEEs are accessed by their users remotely, such remote users need a guarantee that:

1. The TEE they connect to runs on the right hardware, i.e., a CPU with support for trusted execution environments; and
2. The untrusted owner of that hardware has loaded the right software within the TEE, not a version that has been tampered with, e.g., to confuse the user into entering sensitive data.

Obtaining such guarantees is possible through a mechanism called attestation. We will cover it in more detail toward the end of this lecture.

Use Cases

TEEs are useful in many scenarios. We mentioned cloud computing, where the cloud provider may not be trusted. They also play a key role for secure data storage applications, such as password managers, or personal data storage software, for example applications storing fingerprints or other biometric data on smartphones. More generally, TEEs are useful in scenarios where trusted code needs to run and process trusted data in hostile environments, where the rest of the system is almost entirely untrusted: DRM systems, confidential data processing such as ML training/inference, healthcare applications, and so on.

TEE Implementations

Next, we cover the main TEE implementations available from the principal CPU designers: Intel, AMD, and ARM.

Intel Software Guard Extensions (SGX)

One of the earliest implementations of TEEs was made by Intel, and it is named Software Guard Extensions (SGX). SGX places the trusted code and data in user space within what is called an enclave. At any time, the CPU runs either within or outside of the enclave. The enclave’s memory pages are encrypted and cannot be accessed from outside. Memory decryption is done transparently when the CPU runs within the enclave. A traditional SGX setup is illustrated on the left-hand side of this diagram:

The enclave still needs to go through the untrusted host OS if it wants to perform system calls, for example for I/O. This requires transitions outside the enclave, something that is not only slow, but also represents an attack vector, because according to the threat model of TEEs, the OS could be malicious and return corrupted information. Hence, most of the frameworks that help develop enclave applications will embed within the enclave a small operating system, called a library OS (LibOS), which aims to process as many system calls as possible within the enclave, avoiding some of the costly and dangerous enclave exits. This is illustrated on the right-hand side of the diagram above. Even with a LibOS, for disk and network I/O, the untrusted host OS still needs to be invoked.

AMD Secure Encrypted Virtualisation

A particular issue with Intel SGX is that existing applications need to be modified to run as enclaves:

1. They need to be adapted to connect to an enclave framework that generally includes a small library OS.
2. Within the enclave, the data returned by I/O system calls also needs to be sanity-checked, because it comes from the untrusted host OS.

This is a significant downside, because very few developers would be ready to maintain a specific branch of their application for compatibility with SGX. AMD’s TEE implementation is called Secure Encrypted Virtualisation (SEV), and attempts to address that compatibility issue. SEV places an entire virtual machine, called a confidential VM, within the TEE. The entire VM’s memory is isolated and encrypted, similarly to what SGX does for an enclave’s memory:

The advantage with SEV is that existing applications do not need to be ported; they can run as is. The main drawbacks are twofold. First, a confidential VM integrates a large trusted computing base, which includes the application but also the entire guest operating system. Second, for I/O, the VM still needs to go through the untrusted hypervisor, and that interface has proven very hard to secure.

ARM TrustZone

ARM’s implementation of TEEs is called TrustZone. With TrustZone, the CPU execution mode is dissociated between the normal world (outside the TEE), and the secure world (the TEE). These modes are orthogonal to privilege levels like supervisor or user modes, which are available in both normal and secure worlds. Once again, the hardware prevents the normal world from accessing the memory allocated for the secure world – although this time there is no memory encryption.

In the secure world, a small, secure OS runs, on top of which trusted applications running user code can execute. In the normal world, regular applications can invoke trusted applications:

For example, if you are running Android and use your fingerprint to unlock your screen to access your bank or NHS app, the app in question will invoke the fingerprint check trusted application within a TrustZone environment to verify your identity. This way, even if your phone gets stolen and the thief manages to root it, it is still very difficult to steal your fingerprints.

Remote Attestation

As discussed earlier, in a scenario where a client connects remotely to a TEE application that itself runs on an untrusted host, the client needs to make sure that 1) the application runs in a TEE on the expected hardware (e.g., an SGX enclave on a genuine SGX CPU) and 2) the application in the TEE is the expected one, and it has not been tampered with before being loaded.

This is achieved with a technique called remote attestation. The TEE sends the client some form of measurement (e.g., hashes) uniquely identifying the software running in the TEE and the hardware it executes upon. For example, with Intel SGX, a measurement of the enclave’s initial state is sent to the client. That measurement mostly identifies the enclave’s memory content at load time. This allows the client to certify that the application loaded in the enclave is the expected one. Intel also offers an attestation service that will validate the hardware: