CephFS: convert directory into subvolume
Hi, while writing a response to [1] I tried to convert an existing directory within a single cephfs into a subvolume. According to [2] that should be possible, I'm just wondering how to confirm that it actually worked. Because setting the xattr works fine, the directory just doesn't show up in the subvolume ls command. This is what I tried (in Reef and Pacific): # one "regular" subvolume already exists $ ceph fs subvolume ls cephfs [ { "name": "subvol1" } ] # mounted / and created new subdir $ mkdir /mnt/volumes/subvol2 $ setfattr -n ceph.dir.subvolume -v 1 /mnt/volumes/subvol2 # still only one subvolume $ ceph fs subvolume ls cephfs [ { "name": "subvol1" } ] I also tried it directly underneath /mnt: $ mkdir /mnt/subvol2 $ setfattr -n ceph.dir.subvolume -v 1 /mnt/subvol2 But still no subvolume2 available. What am I missing here? Thanks Eugen [1] https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/G4ZWGGUPPFQI... [2] https://www.spinics.net/lists/ceph-users/msg72341.html
well, you should've used the ceph command to create the subvol it's much simpler that way $ ceph fs subvolume create mycephfs subvol2 The above command creates a new subvol (subvol2) in the default subvolume group. So, in your case the actual path to the subvolume would be /mnt/volumes/_nogroup/subvol2/<uuid> On Tue, Aug 22, 2023 at 4:50 PM Eugen Block <eblock@nde.ag> wrote:
Hi,
while writing a response to [1] I tried to convert an existing directory within a single cephfs into a subvolume. According to [2] that should be possible, I'm just wondering how to confirm that it actually worked. Because setting the xattr works fine, the directory just doesn't show up in the subvolume ls command. This is what I tried (in Reef and Pacific):
# one "regular" subvolume already exists $ ceph fs subvolume ls cephfs [ { "name": "subvol1" } ]
# mounted / and created new subdir $ mkdir /mnt/volumes/subvol2 $ setfattr -n ceph.dir.subvolume -v 1 /mnt/volumes/subvol2
# still only one subvolume $ ceph fs subvolume ls cephfs [ { "name": "subvol1" } ]
I also tried it directly underneath /mnt:
$ mkdir /mnt/subvol2 $ setfattr -n ceph.dir.subvolume -v 1 /mnt/subvol2
But still no subvolume2 available. What am I missing here?
Thanks Eugen
[1] https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/G4ZWGGUPPFQI... [2] https://www.spinics.net/lists/ceph-users/msg72341.html
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Milind
Thanks, I understand that. I was just explicitly asking for the conversion of an existing directory (created without subvolume) with xattr as mentioned in the thread [2]. Anyway, apparently it works like Anh Phan stated in his response, moving an existing directory to the subvolumegroup subdir makes it a subvolume. So there's no need for xattr here. Thanks, Eugen Zitat von Milind Changire <mchangir@redhat.com>:
well, you should've used the ceph command to create the subvol it's much simpler that way
$ ceph fs subvolume create mycephfs subvol2
The above command creates a new subvol (subvol2) in the default subvolume group. So, in your case the actual path to the subvolume would be
/mnt/volumes/_nogroup/subvol2/<uuid>
On Tue, Aug 22, 2023 at 4:50 PM Eugen Block <eblock@nde.ag> wrote:
Hi,
while writing a response to [1] I tried to convert an existing directory within a single cephfs into a subvolume. According to [2] that should be possible, I'm just wondering how to confirm that it actually worked. Because setting the xattr works fine, the directory just doesn't show up in the subvolume ls command. This is what I tried (in Reef and Pacific):
# one "regular" subvolume already exists $ ceph fs subvolume ls cephfs [ { "name": "subvol1" } ]
# mounted / and created new subdir $ mkdir /mnt/volumes/subvol2 $ setfattr -n ceph.dir.subvolume -v 1 /mnt/volumes/subvol2
# still only one subvolume $ ceph fs subvolume ls cephfs [ { "name": "subvol1" } ]
I also tried it directly underneath /mnt:
$ mkdir /mnt/subvol2 $ setfattr -n ceph.dir.subvolume -v 1 /mnt/subvol2
But still no subvolume2 available. What am I missing here?
Thanks Eugen
[1] https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/G4ZWGGUPPFQI... [2] https://www.spinics.net/lists/ceph-users/msg72341.html
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Milind
fyi - the xattr is indeed required even if the dir is under a subvolumegroup dir there's some management involved in the way the subvolume dir is created On Thu, Aug 24, 2023 at 5:10 PM Eugen Block <eblock@nde.ag> wrote:
Thanks, I understand that. I was just explicitly asking for the conversion of an existing directory (created without subvolume) with xattr as mentioned in the thread [2]. Anyway, apparently it works like Anh Phan stated in his response, moving an existing directory to the subvolumegroup subdir makes it a subvolume. So there's no need for xattr here.
Thanks, Eugen
Zitat von Milind Changire <mchangir@redhat.com>:
well, you should've used the ceph command to create the subvol it's much simpler that way
$ ceph fs subvolume create mycephfs subvol2
The above command creates a new subvol (subvol2) in the default subvolume group. So, in your case the actual path to the subvolume would be
/mnt/volumes/_nogroup/subvol2/<uuid>
On Tue, Aug 22, 2023 at 4:50 PM Eugen Block <eblock@nde.ag> wrote:
Hi,
while writing a response to [1] I tried to convert an existing directory within a single cephfs into a subvolume. According to [2] that should be possible, I'm just wondering how to confirm that it actually worked. Because setting the xattr works fine, the directory just doesn't show up in the subvolume ls command. This is what I tried (in Reef and Pacific):
# one "regular" subvolume already exists $ ceph fs subvolume ls cephfs [ { "name": "subvol1" } ]
# mounted / and created new subdir $ mkdir /mnt/volumes/subvol2 $ setfattr -n ceph.dir.subvolume -v 1 /mnt/volumes/subvol2
# still only one subvolume $ ceph fs subvolume ls cephfs [ { "name": "subvol1" } ]
I also tried it directly underneath /mnt:
$ mkdir /mnt/subvol2 $ setfattr -n ceph.dir.subvolume -v 1 /mnt/subvol2
But still no subvolume2 available. What am I missing here?
Thanks Eugen
[1] https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/G4ZWGGUPPFQI... [2] https://www.spinics.net/lists/ceph-users/msg72341.html
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Milind
-- Milind
Hello, I'm following this tread and the original. I'm trying to convert directories into subvolumes. Where I'm stuck is how you move a directory into the subvolume root directory. I have a volume 'tank' and it's mounted on the host as '/mnt/tank' I have subfolders '/mnt/tank/database', '/mnt/tank/gitlab', etc... I create a subvolume and getpath gives me: /volumes/_nogroup/database/4a....74 Questions: 1) How do I move /mnt/tank/database into /volumes/_nogroup/database/4a...74 2) Each of the directories have different pools associated with them, do I need to create the sub volume in the same pool? 3) Or can I just move '/mnt/tank/gitlab' --> /volumes/_nogroup/gitlab without first creating the volume? This would skip question 2.. Thx! Jie
Hi, check out this thread [1] as well, where Anh Phan pointed out:
Not really sure what you want, but for simplicity, just move folder to following structure:
/volumes/[Sub Volume Group Name]/[Sub Volume Name]
ceph will recognize it (no extend attr needed), if you use subvolumegroup name difference than "_nogroup", you must provide it in all subvolume command [--group_name <subvol_group_name>]
You'll need an existing group (the _nogroup won't work) where you can move your directory tree to. That worked in my test as expected. Regards, Eugen [1] https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/G4ZWGGUPPFQI... Zitat von jie.zhang7@gmail.com:
Hello,
I'm following this tread and the original. I'm trying to convert directories into subvolumes. Where I'm stuck is how you move a directory into the subvolume root directory.
I have a volume 'tank' and it's mounted on the host as '/mnt/tank' I have subfolders '/mnt/tank/database', '/mnt/tank/gitlab', etc...
I create a subvolume and getpath gives me: /volumes/_nogroup/database/4a....74
Questions: 1) How do I move /mnt/tank/database into /volumes/_nogroup/database/4a...74 2) Each of the directories have different pools associated with them, do I need to create the sub volume in the same pool? 3) Or can I just move '/mnt/tank/gitlab' --> /volumes/_nogroup/gitlab without first creating the volume? This would skip question 2..
Thx!
Jie _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Eugon, Thank you, however I'm am still lost. I can create a subvolume group, that I understand. The issue is '/volume/<group>' isn't a real directory on the host, it's a 'virtual directory' in cephfs. '/mnt/tank/database' is a real folder structure on the host. I can't `mv /mnt/tank/database /volume/<group>` Re-reading the thread, is the answer basically: 1) Create group and subvolume 2) mount the subvolume onto the host and then move the data? Or is there a more direct way to convert '/mnt/tank/database' to '/volume/<group>/database' Thx! Jie
Hi, you'll need to have either both mount points mounted on your host or simply mount the root directory. I don't think there's a different method available to convert directories to subvolumes. Zitat von jie.zhang7@gmail.com:
Eugon,
Thank you, however I'm am still lost. I can create a subvolume group, that I understand. The issue is '/volume/<group>' isn't a real directory on the host, it's a 'virtual directory' in cephfs. '/mnt/tank/database' is a real folder structure on the host. I can't `mv /mnt/tank/database /volume/<group>`
Re-reading the thread, is the answer basically: 1) Create group and subvolume 2) mount the subvolume onto the host and then move the data?
Or is there a more direct way to convert '/mnt/tank/database' to '/volume/<group>/database'
Thx!
Jie _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (3)
-
Eugen Block
-
jie.zhang7@gmail.com
-
Milind Changire