Using CephFS for encrypted Kubernetes volumes
Hi, we are trying to use Ceph (Tentacle 20.2.2) as storage provider for multiple Kubernetes clusters. We are trying to implement the following security measures: * Each Kubernetes cluster should use a dedicated RADOS namespace for its objects. * The volumes should use fscrypt encryption. We can deploy pods successfully as long as we don’t try to enable fscrypt. With encryption enabled the pod shows the following logs: AttachVolume.Attach succeeded for volume "pvc-0923605d-3993-4d99-a4a3-d41749ead831" MountVolume.MountDevice failed for volume "pvc-0923605d-3993-4d99-a4a3-d41749ead831" : rpc error: code = Internal desc = failed to lock volume ID 64ab25ab-9d66-4508-be74-13c02825f40e: rados: ret=-1, Operation not permitted There is a closed issue that mentions a length limitation when creating a lock to initialize the encryption: https://github.com/ceph/ceph-csi/issues/5419 I also tried the mentioned workaround but that does not help. There has also been a discussion that the ’x’ permission is needed in the OSD capabilities, which we have set: mon_caps: 'allow r fsname=root' mgr_caps: 'allow rw' osd_caps: 'allow rwx namespace=k8s-test tag cephfs metadata=root, allow rw namespace=k8s-test tag cephfs data=root' mds_caps: 'allow r fsname=root path=/volumes, allow rws fsname=root path=/volumes/k8s-test' Does anybody have a working setup with encryption and RADOS namespaces? -- Stefan
participants (1)
-
Stefan Möding