ijlal-loutfi
on 19 December 2023
Ubuntu Confidential VMs on Azure: Introducing Ephemeral OS disks & vTPMs
As the adoption of confidential computing continues to grow, customers expect their confidential workloads to be strongly separated from their underlying cloud providers. To better align with these user requirements, Canonical is excited to announce ephemeral OS disks for Ubuntu confidential VMs (CVMs) on Microsoft Azure – a new solution that enables you to store disks on your VM’s OS cache disk or temp/resource disk, without needing to save them to any other remote Azure Storage.
Empowering customers with a Virtual Trusted Platform Module (vTPM) that doesn’t preserve its state across reboots, this solution lays the foundation for a more meaningful remote attestation solution and reduces dependence on cloud infrastructure.
Before we dig deeper into ephemeral OS disks and vTPMs, let’s take a look at the current state of confidential computing and its limitations.
Beyond silicon attestation for confidential computing
Confidential computing aims to protect end-users’ sensitive workloads by running them within hardware-protected trusted execution environments (TEEs). To achieve this high level of security, silicon technologies such as AMD SEV-SNP and Intel TDX incorporate new CPU security extensions that provide strong confidentiality and integrity guarantees to the code and data that run within the TEE. Let’s explore how this is achieved:
- Confidentiality: CPUs equipped with confidential computing capabilities include an AES-128 hardware encryption engine within their memory controller. This engine encrypts and decrypts memory pages whenever there is a memory read or write operation. The encryption key is secure stored within a hardware root of trust and is inaccessible to any of the platform’s privileged system software (such as the host operating system or hypervisor)
- Integrity: Additional CPU-Based Hardware Access Control Mechanisms introduce new instructions and data structures that enable auditing security-sensitive tasks typically carried out by privileged system software. These tasks encompass memory management and access to platform devices. For example, when reading memory pages mapped to confidential workloads, these new instructions also provide information about the last value written into the page. This feature helps prevent data corruption and replay attacks by detecting unauthorised modifications to memory pages.
Okay, so now your workload is securely running within its own isolated TEE…or is it? How can you verify that your cloud provider has not deployed your workload in the normal non-confidential way? How can you know that it has indeed provisioned your workload into a genuine hardware TEE? And if so, how can you verify that its system software has loaded your application as you intended it to be to the TEE? Do you just take the cloud provider’s word for it?
You don’t have to. Instead, you should leverage the remote attestation capabilities of your hardware TEE before provisioning your secrets into it, and before accepting its results as trustworthy.
Naturally, end-users would want an attestation measurement that comprehensively reflects all of the software components that are running within their confidential VM’s boundary. This includes the CVM’s in-guest firmware, the guest operating system and their workloads.
However, silicon attestation falls short as it offers measurements solely for the software initially loaded into the CVM, which usually only includes the CVM in-guest firmware. Given this limitation, relying solely on silicon attestation creates uncertainties about the loaded guest OS and the user’s workload, which could potentially have been compromised by the cloud provider. This significantly weakens the value proposition of the attestation process and undermines the efficacy of confidential computing.
What is the role of vTPMs?
To close this gap within the attestation process, we need to find a way to extend the measurements beyond the in-guest firmware. To achieve this, two requirements must be fulfilled:
- Devise a secure software solution capable of measuring and extending all software that operates post the CVM firmware. This ensures a comprehensive assessment of the entire software stack, augmenting the attestation process’s integrity.
- Tie these software measurements seamlessly into the underlying hardware silicon attestation is imperative.
In this context, trusted platform modules have emerged as the unifying layer that can achieve both requirements.
Requirements of vTPMs for confidential VMs
A trusted platform module (TPM) is traditionally implemented as a hardware-based security component that resides on the motherboard of a computer. It is a dedicated microcontroller that plays a pivotal role in generating, storing, and managing cryptographic keys and performing various security-related tasks. These keys can be used to authenticate the system, ensure secure communication, and protect sensitive data. Furthermore, TPMs also are equipped with an on-chip non-volatile memory, where they can securely store their persistent state such as cryptographic seeds and keys and users’ persistent data.
With the rise of virtualization, the need arose to extend similar functionalities to virtual machines – enter vTPMs. Execution and storage for vTPMs are often implemented within the cloud infrastructure. For non-confidential computing workloads where the cloud provider is trusted, this setup meets the expected security guarantees, as it protects the vTPM from the guest workload.
However, in the context of confidential computing, where everything beyond the CVM boundary is deemed untrusted, this standard vTPM implementation is unacceptable. In fact, executing vTPMs within any part of the cloud host software, including the hypervisor, doesn’t align with the heightened security demands. Therefore, we need to find a way to protect the vTPM execution and persistent storage from both the host cloud and the CVM guest software.
Isolated execution
To address the question of isolated execution, we can utilize the primitives that the underlying silicon solution provides us. In the context of AMD SEV-SNP , we can situate the vTPM within the guest firmware itself and shielded it utilizing virtual machine privilege levels (VMPLs). This implementation ensures a secure enclave for the vTPM, enhancing its protection from the rest of the CVM software. For Intel TDX, the vTPM can run within a distinct CVM, Trusted Domain (TD).
Persistent storage
Traditionally, software vTPMs store their persistent state in disk-backed files. In order to protect the file from rollback and modification attacks, the file would then need to be encrypted, sealed to a particular CVM state, and only allowed to be decrypted by the vTPM. However, in confidential computing scenarios where the cloud infrastructure isn’t fully trusted, the challenge lies in securely storing the encryption while safeguarding it from both host and guest access. Additionally, establishing a secure, practical protocol for the vTPM associated with that specific CVM during boot-up is crucial.
Enter Ephemeral vTPMs for confidential VMs
Addressing this security challenge while ensuring usability has proven to be a complex task, and solving it has driven our exploration of ephemeral vTPMs for Ubuntu confidential VMs operating on Microsoft Azure.
The solution we propose for Ubuntu confidential VMs offers an ephemeral OS disk that can be stored either on the VM’s OS cache disk or the VM’s temp/resource disk. This feature operates using a stateless ephemeral vTPM that doesn’t retain any persistent state within the host. Instead, it generates fresh cryptographic material each time it boots up.
Who benefits from Ephemeral vTPMs for Ubuntu confidential VMs
Ephemeral OS disks and vTPMs on Ubuntu confidential VMs serve as an excellent solution for customers seeking comprehensive CVM remote attestation that extends beyond in-guest firmware, without needing to trust the cloud provider. However, it is important to note that due to the inability to retain secrets across reboots, not all workloads can leverage this solution. Ephemeral OS disks are especially beneficial for stateless workloads, for applications that don’t rely on persistent storage within the cloud host environment and can tolerate individual VM failures.
Looking ahead
Security is a subtle business where attention to details is crucial. When discussing confidential VMs, it is natural to focus on the software components that the technology helps you remove from your VM’s trust boundary, such as the underlying hypervisor and host OS, because these highlight the innovative and transformative aspect of confidential computing.
However, to provide a comprehensive and transparent evaluation of this technology, it is critical to shine light on the elements that still need trust when deploying workloads as confidential VMs. These trusted components can significantly impact the overall security of the system and should be approached with care and consideration.
In this blog, we explained how providing attestation that covers the entire software stack of the CVM requires a vTPM whose persistent storage requires the customer to go back to trusting the cloud infrastructure. We have then explored how using ephemeral OS disks are vTPMs allow to remove this dependency.
At Canonical, we understand the importance of complete transparency. By actively identifying and mitigating the challenges that stand in the way of making confidential computing truly emancipated from the underlying hosting infrastructure, we’re developing a comprehensive roadmap to enhance CVM attestation. Ephemeral vTPMs mark a foundational step, and we’re enthusiastic about sharing more features as they evolve.
