In case I misunderstand this part of your message, something like this getfattr -d -m "ceph.*" dir does not work any more, the xattribs are no longer discoverable. In addition, quota settings were never discoverable. The earlier thread https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/message/3HWU4DITVD… is about this. Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Gregory Farnum <gfarnum(a)redhat.com> Sent: 26 March 2020 18:36 To: Frank Schilder Cc: ceph-users Subject: Re: [ceph-users] How to migrate ceph-xattribs? On Thu, Mar 26, 2020 at 9:24 AM Frank Schilder <frans(a)dtu.dk> wrote: These are all policy decisions that seem pretty cluster-specific to me. We hid them in the first place because exposing ceph xattrs to rsync was breaking it horribly. IIRC, you can still see all these attributes by querying for the "ceph" xattr. Does that not work for you?

Dear Jeff, thank you for your answer. A little bit of time went by since I asked all this and I made a small discovery that might pave the path to a useful solution. This will be a longer text including examples, which might become one or two feature requests. I would be grateful to have your opinion before posting these requests. In my questions I basically raise 2 separate issues: 1) How can one find efficiently all (or an important subset of) directories/files with ceph xattribs set? 2) How can one discover the names of all ceph xattribs, even the ones not set or not documented? The first question lends more or less to this Feature request #1: add dump xattrs to daemon mds commands. It would be really helpful if an MDS could provide information about all settable dirs/files with xattribs set similar to "dump snaps". Most importantly, the quota and file layout attributes are of interest. The command could have the form dump xattrs [[PATH] DEPTH] dumping by default everything at or below "/" and, optionally, at or below PATH. As a further option one could provide a DEPTH argument to allow limiting the output to a certain level at or below PATH. Output could be the same as for dump snaps as it is quite straightforward to find path names. The second question might have an answer as well and I need to add a bit of explanation here. While playing with dir layouts, I discovered something interesting: [root@gnosis hpc]# getfattr -n ceph.dir.layout.pool apps # file: apps ceph.dir.layout.pool="con-fs2-data-ec-ssd" [root@gnosis hpc]# getfattr -n ceph.dir.layout apps # file: apps ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=con-fs2-data-ec-ssd" [root@gnosis hpc]# getfattr -n ceph.dir apps apps: ceph.dir: No such attribute # :( Why not? What this observation lends to is the idea that the ceph attributes are actually a recursive data structure that could be represented with a recursive output format like json. Here my attempt to exemplify what I would consider a really cool feature (kept incomplete on purpose): [root@gnosis hpc]# getfattr -n ceph apps # file: apps ceph={quota:{max_bytes:"1099511627776",max_files:"0"},dir:{rentries:"2770612",...,layout:{stripe_unit:4194304,...,pool:"con-fs2-data-ec-ssd"}}} [root@gnosis hpc]# getfattr -n ceph.quota apps # file: apps ceph.quota={max_bytes:"1099511627776",max_files:"0"} [root@gnosis hpc]# getfattr -n ceph.quota.max_bytes apps # file: apps ceph.quota.max_bytes="1099511627776" What about "unset" attribs? How about: [root@gnosis home]# getfattr -n ceph.quota abc # file: abc ceph.quota={max_bytes:NULL,max_files:NULL} [root@gnosis home]# getfattr -n ceph.quota.max_bytes abc abc: ceph.quota.max_bytes: No such attribute Note that setting attributes still requires the full path all the way down to an individual value just as is the case today with the layout attributes. In this way, the names of *all* ceph attributes are discoverable (including unset ones) without breaking any current functionality. All I have to remember is "ceph", everything else comes by itself. This is even possible in a programmatic way as jq is an excellent json parser for shells. In case there are output length restrictions, one could simply enable pagination by including the standard field "next" with the page to request the next chunk of output, or NULL if this is the last one. No matter how good or bad xattribs are for the purpose they are used for, with the above recursive structure it becomes very easy to find what one is looking for without having to remember anything. Its completely self-documenting. Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Jeff Layton <jlayton(a)redhat.com> Sent: 15 April 2020 13:00 To: Frank Schilder; Gregory Farnum Cc: ceph-users Subject: Re: [ceph-users] Re: How to migrate ceph-xattribs? On Fri, 2020-03-27 at 07:29 +0000, Frank Schilder wrote: We mainly turned them off to follow the existing conventions in the kernel (see commit e09580b343aa in the Linux kernel). Doing otherwise was causing xattr conformance tests to fail, and (like Greg mentioned) that has always been problematic for backup programs. I'm afraid I don't know of a workaround. The big issue is that listxattr() itself doesn't do any filtering. You just get a dump of names and have to filter them in userspace yourself. It also doesn't help that xattrs aren't governed by any sort of standard so the rules for all of this are quite vague. Personally, I find xattrs to be a garbage interface for this sort of thing, but we're sort of stuck with it at this point. One possibility would be to add a well-known "meta" ceph xattr that lists the xattrs that are not set to the defaults. We could also start reporting them to listxattr() again, but that has its own issues. -- Jeff Layton <jlayton(a)redhat.com>