Alright, it's not the bluefs commands (obviously) but the LVM tools. So I just tried this on a test cluster: # starting point tentacle:~ # pvs PV VG Fmt Attr PSize PFree /dev/sda ceph-8e963da9-aaec-4eb1-93bc-8902bf3230c3 lvm2 a-- 10.00g 0 /dev/sdb ceph-ef456936-3aff-498f-ae5f-f66bba1966d8 lvm2 a-- 10.00g 0 /dev/sdc ceph-6fbf767a-b8e5-4cfb-bc4b-b51fa5f6e38e lvm2 a-- 10.00g 0 # resize PV tentacle:~ # pvresize -v /dev/sdc Resizing volume "/dev/sdc" to 31457280 sectors. Resizing physical volume /dev/sdc from 2559 to 3839 extents. Updating physical volume "/dev/sdc" Archiving volume group "ceph-6fbf767a-b8e5-4cfb-bc4b-b51fa5f6e38e" metadata (seqno 5). Physical volume "/dev/sdc" changed Creating volume group backup "/etc/lvm/backup/ceph-6fbf767a-b8e5-4cfb-bc4b-b51fa5f6e38e" (seqno 6). 1 physical volume(s) resized or updated / 0 physical volume(s) not resized # resize LV tentacle:~ # lvextend -l +100%FREE /dev/ceph-6fbf767a-b8e5-4cfb-bc4b-b51fa5f6e38e/osd-block-267fede8-9c40-4cf7-8661-a3d8cd3310c6 Size of logical volume ceph-6fbf767a-b8e5-4cfb-bc4b-b51fa5f6e38e/osd-block-267fede8-9c40-4cf7-8661-a3d8cd3310c6 changed from 10.00 GiB (2559 extents) to 15.00 GiB (3839 extents). Logical volume ceph-6fbf767a-b8e5-4cfb-bc4b-b51fa5f6e38e/osd-block-267fede8-9c40-4cf7-8661-a3d8cd3310c6 successfully resized. # check sizes tentacle:~ # cephadm shell --name osd.1 ... [ceph: root@tentacle /]# ceph-bluestore-tool --data-path /var/lib/ceph/osd/ceph-1/ --command bluefs-bdev-sizes inferring bluefs devices from bluestore path 1 : device size 0x3bfc00000(15 GiB) : using 0x14203e000(5.0 GiB) For some reason the DB assigned itself 5 GB, not sure how and why though. :-D Anyway, osd.1 now shows 15 GB size total, but the crush weight won't be updated, you'd need to do that manually: tentacle:~ # ceph osd df ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS 0 hdd 0.00980 1.00000 10 GiB 33 MiB 4.8 MiB 29 KiB 28 MiB 10 GiB 0.32 0.02 228 up 1 hdd 0.00980 1.00000 15 GiB 5.4 GiB 4.0 MiB 24 KiB 29 MiB 9.6 GiB 36.16 2.31 226 up 2 hdd 0.00980 1.00000 10 GiB 32 MiB 4.0 MiB 30 KiB 28 MiB 10 GiB 0.31 0.02 220 up tentacle:~ # ceph osd crush reweight osd.1 0.015 reweighted item id 1 name 'osd.1' to 0.015 in crush map So as you see, it's all manual work which shouldn't really be necessary in reality. But of course it's nice to know a way around things, who knows, maybe this will help you one day. ;-) Regards, Eugen Zitat von Eugen Block <eblock@nde.ag>:
Hm, maybe I misremember and the expand command isn't for the main device but only for DB/WAL... I'll need to check again, as I wrote, it's been quite a while since I last played with those commands...
Zitat von lejeczek via ceph-users <ceph-users@ceph.io>:
is this...
-> $ systemctl stop ceph-9f4f9dba-72c7-11f0-8052-525400519d29@osd.10.service
-> $ ceph-bluestore-tool bluefs-bdev-sizes --path /var/lib/ceph/9f4f9dba-72c7-11f0-8052-525400519d29/osd.10/ inferring bluefs devices from bluestore path 1 : device size 0x4affc00000(300 GiB) : using 0x36f49bd000(220 GiB)
-> $ ceph-bluestore-tool bluefs-bdev-expand --path /var/lib/ceph/9f4f9dba-72c7-11f0-8052-525400519d29/osd.10/ inferring bluefs devices from bluestore path 1 : device size 0x4affc00000(300 GiB) : using 0x36f49bd000(220 GiB) Expanding DB/WAL... 1 : nothing to do, skipped
not weird(wrong) - would not all agree?
-> $ ceph osd df ... 10 hdd 0.29300 1.00000 300 GiB 219 GiB 218 GiB 567 KiB 632 MiB 81 GiB 72.91 0.99 165 up
-> $ ceph-volume lvm list ... ====== osd.10 ====== ... devices /dev/vdc ... -> $ fdisk -l /dev/vdc Disk /dev/vdc: 400 GiB, 429496729600 bytes, 104857600 sectors
Something, somewhere Ceph is ignoring, not picking up...? From what I can find 'ceph-volume' does not offer re-sizing volumes. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io