Moving files around within the namespace never changes the way the file data is represented within RADOS. It’s just twiddling metadata bits. :) -Greg On Thu, Jul 6, 2023 at 3:26 PM Dan van der Ster <dan.vanderster@clyso.com> wrote:
Hi Mathias,
Provided that both subdirs are within the same snap context (subdirs below where the .snap is created), I would assume that in the mv case, the space usage is not doubled: the snapshots point at the same inode and it is just linked at different places in the filesystem.
However, if your cluster and livelihood depends on this being true, I suggest making a small test in a tiny empty cephfs, listing the rados pools before and after mv and snapshot operations to find out exactly which data objects are created.
Cheers, Dan
______________________________________________________ Clyso GmbH | Ceph Support and Consulting | https://www.clyso.com
On Thu, Jun 22, 2023 at 8:54 AM Kuhring, Mathias < mathias.kuhring@bih-charite.de> wrote:
Dear Ceph community,
We want to restructure (i.e. move around) a lot of data (hundreds of terabyte) in our CephFS. And now I was wondering what happens within snapshots when I move data around within a snapshotted folder. I.e. do I need to account for a lot increased storage usage due to older snapshots differing from the new restructured state? In the end it is just metadata changes. Are the snapshots aware of this?
Consider the following examples.
Copying data: Let's say I have a folder /test, with a file XYZ in sub-folder /test/sub1 and an empty sub-folder /test/sub2. I create snapshot snapA in /test/.snap, copy XYZ to sub-folder /test/sub2, delete it from /test/sub1 and create another snapshot snapB. I would have two snapshots each with distinct copies of XYZ, hence using double the space in the FS: /test/.snap/snapA/sub1/XYZ <-- copy 1 /test/.snap/snapA/sub2/ /test/.snap/snapB/sub1/ /test/.snap/snapB/sub2/XYZ <-- copy 2
Moving data: Let's assume the same structure. But now after creating snapshot snapA, I move XYZ to sub-folder /test/sub2 and then create the other snapshot snapB. The directory tree will look the same. But how is this treated internally? Once I move the data, will there be an actually copy created in snapA to represent the old state? Or will this remain the same data (like a link to the inode or so)? And hence not double the storage used for that file.
I couldn't find (or understand) anything related to this in the docs. The closest seems to be the hard-link section here: https://docs.ceph.com/en/quincy/dev/cephfs-snapshots/#hard-links Which unfortunately goes a bit over my head. So I'm not sure if this answers my question.
Thank you all for your help. Appreciate it.
Best Wishes, Mathias Kuhring
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io