Skip to main content
Version: Next

End-to-end prompt encryption

Continuum uses end-to-end encryption to protect user data from being accessed along the supply chain. Prompts are encrypted on the client side, decrypted within runtime-encrypted Continuum workers, and re-encrypted before being returned to the client. This encryption uses a symmetric key with Authenticated Encryption implemented through AES-GCM.

Encryption workflow​

Prompts are encrypted by the client-side continuum-proxy and decrypted by server-side encryption proxy hosted on the Continuum worker. Replies are handled accordingly where encryption is done by the server-side encryption proxy and decryption of the replies is performed by the client-side continuum-proxy. The steps for the encryption between the client and the proxy are:

  1. Key Exchange: The server-side encryption proxy and the client-side continuum-proxy exchange a symmetric AES key, facilitated by the attestation service (AS). The AES keys are also referred to as inference secrets in this documentation. The AS handles key uploads from clients and grants access to verified worker nodes. Each key has a corresponding ID which is encoded in the encrypted data to let the decoder know which key was used.
  2. Key Synchronization: The server-side proxy monitors for key updates to stay in sync with the client.
  3. Request Encryption: The client only encrypts the request field with the prompt text, keeping other details like token length accessible to the service provider. The encrypted field encodes the key ID which maps to the used key.
  4. Request Decryption: The server-side proxy decodes the prompt field with the key that maps to the encoded key ID.
  5. Sandbox Forwarding: The decrypted prompt is securely transmitted to the sandboxed inference server via a UNIX domain socket.
  6. Response Encryption: The response from the inference server is returned through the same socket. The server-side proxy then encrypts the response and sends it back to the user.

Future enhancements​

Future enhancements will include support for Retrieval Augmented Generation (RAG) use cases, expanding Continuum's encryption framework capabilities.