Made a ticket in the Linux Kernel Client tracker: https://tracker.ceph.com/issues/44791 I naively don't think this should be very complicated at all, except that I recall hearing something about locking issues with quota realms in the kclient? But the userspace client definitely doesn't have to do anything tricky with moving things, it's pretty much a normal move with some glue to point at a different quota realm. -Greg On Fri, Mar 27, 2020 at 7:34 AM Jeff Layton <jlayton@redhat.com> wrote:
On Fri, 2020-03-27 at 07:36 -0400, Jeff Layton wrote:
On Thu, 2020-03-26 at 10:32 -0700, Gregory Farnum wrote:
On Thu, Mar 26, 2020 at 9:13 AM Frank Schilder <frans@dtu.dk> wrote:
Dear all,
yes, this is it, quotas. In the structure A/B/ there was a quota set on A. Hence, B was moved out of this zone and this does indeed change mv to be a cp+rm.
The obvious follow-up. What is the procedure for properly moving data as an administrator? Do I really need to unset quotas, do the move and set quotas back again?
Ah-ha, this is a difference between the userspace and kernel clients. :( The kernel client always returns EXDEV if crossing "quota realms" (different quota roots). I'm not sure why it behaves that way as userspace is different: * If fhere is a quota in the target directory, and * If the target directory's existing data, plus the file(s) being moved, exceed the target directory's quota, then userspace returns EXDEV/EDQUOT. This seems like the right kernel behavior as well...
Jeff, is that a known issue for some reason? Should we make a new bug? :)
(cc'ing Luis since he wrote cafe21a4fb3075, which added this check)
In ceph_rename, we have this, which is probably what you're hitting:
/* don't allow cross-quota renames */ if ((old_dir != new_dir) && (!ceph_quota_is_same_realm(old_dir, new_dir))) return -EXDEV;
That does seem to just check whether the realms are the same and doesn't actually check the space in them.
I haven't studied this in detail, but it may be hard to ensure that we won't exceed the quota in a race-free way. How do you ensure that another thread doesn't do something that would exceed the quota just as you issue the rename request?
This is fairly trivial in the userland client since it does everything under the BCM (Big Client Mutex), but won't be in the kernel client.
Opening a bug for this won't hurt, but it may not be simple to implement.
Ok, I gave it a harder look and it doesn't look too bad, given that quotas are sort of "sloppy" anyway. We could just do a check that we won't exceed quota on the destination directory. The catch is that there may be other details to deal with too in moving an existing inode under a different snaprealm.
You're welcome to open a tracker bug for this, but I'm not sure who will take on that project and when.
Thanks, -- Jeff Layton <jlayton@redhat.com>