Hello all, We're getting a "Operation not permitted" error while trying to create a snapshot on the client. It is somehow related to previously-asked Pacific issue mentioned here: https://www.spinics.net/lists/ceph-users/msg67908.html We are on squid (19.2.1) and the given workaround seems to be only temporary. Anyway, here are some details about the issue: Cephx side: ``` client.mount-update-production key: *** caps: [mds] allow rws fsname=production-cephfs path=/volumes/_nogroup/update-production caps: [mon] allow r fsname=production-cephfs caps: [osd] allow rw tag cephfs data=production-cephfs ``` FS flags: ``` ~> sudo ceph fs lsflags production-cephfs joinable allow_snaps allow_multimds_snaps ``` I got the path via following command: ``` ~> sudo ceph fs subvolume getpath production-cephfs update-production /volumes/_nogroup/update-production/e155ba30-d201-4653-be3f-86533324ee5f ``` And mounted to client with mentioned key: ``` ~> mount | grep ceph mount-update-production@00000000-0000-0000-0000-000000000000.production-cephfs=/volumes/_nogroup/update-production/e155ba30-d201-4653-be3f-86533324ee5f on /opt/aptly/.aptly type ceph (rw,noatime,name=mount-update-production,secret=<hidden>,acl,mon_addr=[multiple mon addresses here]) ``` Everything works flawlessly. Yet, when we want to get a snapshot: ``` /opt/aptly/.aptly/pool/9c/0b ~> sudo mkdir .snap/test mkdir: cannot create directory ‘.snap/test’: Operation not permitted ``` If I mount the whole FS with admin rights and run `setfattr -n ceph.dir.subvolume -v 0` for _all_ directories of following path one by one: /volumes/_nogroup/update-production/e155ba30-d201-4653-be3f-86533324ee5f, then it is solving the issue. But it does not last; not sure what is triggering it again, but if I try snapshotting anything next day, getting the same "Operation not permitted" error again. Is there something I can try, or am I missing something obvious? Thanks, Gürkan