It was up and running. I should mention that the situation came about when I was testing OSD rebuild. So I had a previously autogenerated hybrid OSD, with SSD db, and HDD block dev. I then stopped the OSD, wiped HDD, .... did "ceph osd rm" and ceph auth rm. and then rebuilt with ceph-volume.. with the wrong flags. I belive I used: ceph-volume lvm create --block /dev/xxx --journal /dev/blahblah After that, ceph-volume lvm list claimed it was using the ssd again as db device. but ls -l on /var/lib/ceph/osd/XXXX only showed symlink for block dev. This was on nautilus. ----- Original Message ----- From: "Eugen Block" <eblock(a)nde.ag> To: "ceph-users" <ceph-users(a)ceph.io> Sent: Tuesday, April 6, 2021 5:20:40 AM Subject: [ceph-users] Re: which is definitive: /var/lib/ceph symlinks or ceph-volume? Is the OSD up and running? Do you see IO on the dedicated block.db LV when running a 'ceph tell osd.7 bench'? It sounds very strange, I haven't seen that. Zitat von Philip Brown <pbrown(a)medata.com>om>: _______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

Where does it read it from? does it keep it in the block.db lv, or the block dev lv, or both? I removed the vg from the block dev and did wipefs, if I recall. ----- Original Message ----- From: "Eugen Block" <eblock(a)nde.ag> To: "Philip Brown" <pbrown(a)medata.com> Cc: "ceph-users" <ceph-users(a)ceph.io> Sent: Tuesday, April 6, 2021 9:06:50 AM Subject: [BULK] Re: [ceph-users] Re: which is definitive: /var/lib/ceph symlinks or ceph-volume? Did you recreate the OSD with the same UUID? I'm guessing ceph-volume just reads LV tags and thinks there's a block.db for that OSD. Before creating the OSD you should also wipe the LV containing the block.db for the failed OSD. Zitat von Philip Brown <pbrown(a)medata.com>om>: > It was up and running. > > I should mention that the situation came about when I was testing > OSD rebuild. > > So I had a previously autogenerated hybrid OSD, with SSD db, and HDD > block dev. > > I then stopped the OSD, wiped HDD, .... did "ceph osd rm" and ceph auth rm. > and then rebuilt with ceph-volume.. with the wrong flags. > I belive I used: > > ceph-volume lvm create --block /dev/xxx --journal /dev/blahblah > > After that, ceph-volume lvm list > claimed it was using the ssd again as db device. but ls -l on > /var/lib/ceph/osd/XXXX only showed symlink for block dev. > > This was on nautilus. > > > > ----- Original Message ----- > From: "Eugen Block" <eblock(a)nde.ag> > To: "ceph-users" <ceph-users(a)ceph.io> > Sent: Tuesday, April 6, 2021 5:20:40 AM > Subject: [ceph-users] Re: which is definitive: /var/lib/ceph > symlinks or ceph-volume? > > Is the OSD up and running? > Do you see IO on the dedicated block.db LV when running a 'ceph tell > osd.7 bench'? It sounds very strange, I haven't seen that. > > > Zitat von Philip Brown <pbrown(a)medata.com>om>: > >> I am in a situation where I see conflicting information. >> On the one hand, >> ls -l /var/lib/ceph/osd/ceph-7 >> shows a symlink for block device, but no block.db >> >> On the other hand, >> >> ceph-volume lvm list >> claims that there is a separate db device registered for osd 7 >> >> how can I know which one is correct? >> >> (This is currently ceph nautilus) >> >> >> >> >> >> -- >> Philip Brown| Sr. Linux System Administrator | Medata, Inc. >> 5 Peters Canyon Rd Suite 250 >> Irvine CA 92606 >> Office 714.918.1310| Fax 714.918.1325 >> pbrown(a)medata.com| www.medata.com >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io > > > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io