Hi Folks, We are looking at implementing a new encryption feature in libRBD as explained below. Feedback and thoughts are welcome. Best, Danny Harnik The problem: There is a growing need from multiple industries (e.g. In the finance industry) to encrypt data at the host with tenant/user provided encryption keys at a volume granularity. This is driven by regulations and a rising emphasis on security. To date, Ceph RBD does not offer any such solution and the existing alternatives are to add an encryption layer before libRBD. Examples of such solutions are using QEMU LUKS encryption or relying on DM-Crypt. However, using an encryption layer above RBD has limitations when interfacing with storage functions implemented in the RBD layer. Most glaring is cloning that will only work if the parent and child are encrypted with the same encryption key. By moving encryption down into libRBD, we can achieve the flexibility to use Ceph RBD clones, for example, to create encrypted clones of unencrypted golden images. Such a feature exists today with QEMU when using LUKS encryption together with qCOW2 clones. We want to support a similar capability in libRBD, and potentially address some security limitations of existing QEMU based solutions. Key design points and thoughts: - Key management flow will closely mimic that of LUKS, as is implemented in QEMU for qCOW2. The point here is to avoid any security pitfalls that come from key management. QEMU with qCOW2 also has a format in place to handle a chain of clones each with different key/encryption mechanisms and we should follow suit in this design.     - We intend to support two main encryption methods:   1) Standard AES-XTS full disk encryption which is the method recommended in LUKS. We plan to use an encryption  block size of 4KB which is supported by LUKS2 (but not LUKS v1). Note that the encryption in QEMU works at a 512 byte encryption block granularity which is less efficient. 2) A new enhanced security format - while AES-XTS is the state of the art in full disk encryption it is designed to provide the best security under the limitations of a physical disk. An RBD volume is not a physical disk, but rather a virtual one which gives us room to store additional information such as IV and authentication data. Such a "fresh IV" approach will eliminate IV reuse which is a central limitation in physical disks, and would therefore enhance security over the one provided by LUKS encryption today. In the enhanced security format IVs need to be stored along with the data in order to allow future decryption, adding a space and performance overhead which we intend to evaluate. - We intend to add an new Encryption Object Dispatch Layer  in libRBD that will handle encryption and decryption. This layer will be placed right below the Cache layer and above the Journal layer. Thus the data in the cache will be in the clear but the journal data will be encrypted. - The biggest challenge in terms of design is to handle clones correctly and specifically "copyup" of data from the parent needs to be decrypted with the parent key and re-encrypted with the child key. - There is room for expanding the RBD export/import utilities to export data encrypted rather than in the clear. Upon import it can be read once the proper keys are given. Such a feature is straightforward if a single volume is involved but more complex if a chain of clones is involved.