Hi, that sounds a bit impractical to create a new pool for each tenant regarding PG and pool management (adding and possibly removing data pools from the CephFS), depending on how many tenants you have or plan to have. I would recommend to use this client authorization as a template: ceph fs subvolume authorize cephfs subvol-ebl user2 --group_name=subvolgroup1 --access_level=rw This results in these caps: ceph auth get client.user2 [client.user2] key = XXX caps mds = "allow rw path=/volumes/subvolgroup1/subvol-ebl/2596be92-9168-439f-b39f-61eb092e207b" caps mon = "allow r" caps osd = "allow rw pool=cephfs.cephfs.data namespace=fsvolumens_subvol-ebl" I created the subvolume in the dashboard with isolated namespace (which can be done via CLI as well, of course). And then you also need the "rw" caps (which you already have) for the mgr for k8s to be able to utilize the volume module. But I'm not sure if this will suffice to make it work. What I just tested was to manually encrypt a directory in a test cluster with fscrypt, that seems to work as intended. But I don't have anything with k8s to test it at the moment. IIRC, the caps you specified wrt metadata would indeed allow a client to access other tenant's metadata, but I'm not entirely sure. Hopefully someone else can chime in and has better answers for you. Regards, Eugen Zitat von Stefan Möding via ceph-users <ceph-users@ceph.io>:
Hi,
Eugen Block via ceph-users <ceph-users@ceph.io> writes:
I haven't tried it myself yet so I don't really have an answer. But I noticed that your OSD caps may be incomplete wrt x permissions:
osd_caps: 'allow rwx namespace=k8s-test tag cephfs metadata=root, allow rw namespace=k8s-test tag cephfs data=root'
The second allow statement doesn't have an "x". Maybe that's the issue why the workaround fails?
We tried that, but the error remains. We now consider to use the following capabilities in combination to creating a dedicated data pool for each tenant (K8s cluster):
mon_caps: 'allow r fsname=root' mgr_caps: 'allow rw' osd_caps: 'allow rwx pool=cephfs.meta, allow rw pool=cephfs.data, allow rw pool=cephfs.k8s-test' mds_caps: 'allow r fsname=root path=/volumes, allow rws fsname=root path=/volumes/k8s-test'
CephFS is created with cephfs.meta as metadata pool and cephfs.data as default data pool while cephfs.k8s-test is the pool for this tenant.
What are the implications to share the metadata/default data pools and have no namespace? Is the MDS capability sufficient to separate tenants or can everyone access the metadata objects of everyone else?
-- Stefan _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io