Hi, I haven't tried it myself yet so I don't really have an answer. But I noticed that your OSD caps may be incomplete wrt x permissions:

osd_caps: 'allow rwx namespace=k8s-test tag cephfs metadata=root, allow rw namespace=k8s-test tag cephfs data=root'

The second allow statement doesn't have an "x". Maybe that's the issue why the workaround fails? Regards, Eugen Zitat von Stefan Möding via ceph-users <ceph-users@ceph.io>:

Hi,

we are trying to use Ceph (Tentacle 20.2.2) as storage provider for multiple Kubernetes clusters. We are trying to implement the following security measures:

* Each Kubernetes cluster should use a dedicated RADOS namespace for its objects. * The volumes should use fscrypt encryption.

We can deploy pods successfully as long as we don’t try to enable fscrypt. With encryption enabled the pod shows the following logs:

AttachVolume.Attach succeeded for volume "pvc-0923605d-3993-4d99-a4a3-d41749ead831" MountVolume.MountDevice failed for volume "pvc-0923605d-3993-4d99-a4a3-d41749ead831" : rpc error: code = Internal desc = failed to lock volume ID 64ab25ab-9d66-4508-be74-13c02825f40e: rados: ret=-1, Operation not permitted

There is a closed issue that mentions a length limitation when creating a lock to initialize the encryption: https://github.com/ceph/ceph-csi/issues/5419

I also tried the mentioned workaround but that does not help.

There has also been a discussion that the ’x’ permission is needed in the OSD capabilities, which we have set:

mon_caps: 'allow r fsname=root' mgr_caps: 'allow rw' osd_caps: 'allow rwx namespace=k8s-test tag cephfs metadata=root, allow rw namespace=k8s-test tag cephfs data=root' mds_caps: 'allow r fsname=root path=/volumes, allow rws fsname=root path=/volumes/k8s-test'

Does anybody have a working setup with encryption and RADOS namespaces?

-- Stefan _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io