Skip to main content
Version: 1.4

On-premises MarbleRun deployment

This guide walks you through setting up MarbleRun for your on-premises deployment.

Prerequisites​

Hardware and firmware​

CPU​

To deploy MarbleRun with Intel SGX, the machine or VM has to support Intel SGX. Particularly, MarbleRun requires support for the SGX Data Center Attestation Primitives (DCAP). You can verify if your CPU supports DCAP.

BIOS​

BIOS support is required for Intel SGX to provide the capability to enable and configure the Intel SGX feature in the system. Currently, most of the SGX-capable systems have SGX disabled by default in the BIOS. This default setting might change, but for now, you need to manually enable it if it's not already enabled.

Updates​

As with any modern technology, Intel SGX has been affected by security vulnerabilities. Intel addresses these vulnerabilities by updating the microcode of CPUs, changing the hardware of new CPUs, and updating the system software. Each microcode update that patches an SGX vulnerability requires a BIOS update. During remote attestation, it's checked that the microcode of the CPU, which is deployed by the BIOS, is up to date. The microcode and platform enclaves are commonly called the platform Trusted Computing Base (TCB).

If your BIOS/firmware is outdated, you will see errors as Platform TCB (2) is not up-to-date (oe_result_t=OE_TCB_LEVEL_INVALID) during remote attestation procedures.

Hypervisor​

If you are using VMs for your MarbleRun deployment, you need to make sure your hypervisor has SGX enabled. Most of the popular hypervisors support SGX:

Driver​

You need to install the DCAP SGX Driver. Azure provides instructions on how to install this driver that you can use for your on-premises machines.

SGX Data Center Attestation Primitives (DCAP)​

DCAP is the new attestation mechanism for SGX, replacing EPID. You can find an overview of DCAP in the official Intel docs. MarbleRun only supports DCAP and requires DCAP libraries installed and configured on your system.

From the perspective of MarbleRun and your workloads, DCAP is accessed with a Quote Generation Library (QGL) and a Quote Verification Library (QVL) for generating and verifying quotes respectively. The QGL and QVL libraries need to be configured to talk to a Provisioning Certificate Caching Service (PCCS). You currently have two options regarding PCCS for your on-premises machines and clusters:

  1. Use a public PCCS service by configuring your QGL and QVL to point to the public endpoints. Currently, Azure and Alibaba Cloud provide such a service but require using infrastructure by these providers to make full use of the service.

  2. Run your own PCCS and expose it to your machine or cluster. See Intel's demo reference implementation and design guide for more information.

    Follow these steps to set up your machines for your PCCS:

    The PCCS is a cache, so you need to make sure it stays up to date. In case your cache is outdated, you might see error messages as:

    coordinator-enclave.signed:mbedtls_x509_crt_verify failed with The CRL is expired (flags=0x20) (oe_result_t=OE_VERIFY_CRL_EXPIRED)

    You can inspect the Intel Root CA CRL of your PCCS:

    curl --insecure --request GET --url https://<YOUR_PCCS_DOMAIN>:<YOUR_PCCS_PORT>/sgx/certification/v3/rootcacrl > rootca.crl
    openssl crl -inform DER -text -noout -in rootca.crl

    You can refresh all SGX collaterals for your PCCS:

    curl --insecure --request GET -H "admin-token: <my password>" --url https://<YOUR_PCCS_DOMAIN>:<YOUR_PCCS_PORT>/sgx/certification/v3/refresh

    If refreshing CRL fails, you can manually delete the pckcache.db database (default location /opt/intel/sgx-dcap-pccs/pckcache.db) and restart your PCCS.

The docker image for the MarbleRun Coordinator comes with the default quote provider library by Intel. Mount the desired configuration to /etc/sgx_default_qcnl.conf.

Deploy MarbleRun​

You have made sure your hardware supports SGX, updated all firmware, installed the SGX driver, and configured DCAP on all your machines and VMs? Great! Now it's time to install MarbleRun and get going.

You can either use MarbleRun in standalone mode or install it in your Kubernetes cluster.