To update, the OSD had data on HDD and DB on SSD. After "ceph orch osd rm 12 --replace --force" and wait till rebalancing is done and daemon is stopped, I ran "ceph orch device zap ceph-osd-2 /dev/sdd" to zap the device. It cleared PV, VG and LV for data device, but not DB device. DB device issue is being discussed in another thread. Eventually, I restart the active mgr, then the device shows up available. Not sure what was stuck in mgr. Thanks! Tony ________________________________________ From: Marc <Marc@f1-outsourcing.eu> Sent: February 10, 2021 12:21 PM To: Philip Brown; Matt Wilder Cc: ceph-users Subject: [ceph-users] Re: Device is not available after zap I had something similar a while ago, can't remember how I solved it sorry, but it is not a lvm bug. Also posted it here. To bad this is still not fixed.
-----Original Message----- Cc: ceph-users <ceph-users@ceph.io> Subject: [ceph-users] Re: Device is not available after zap
ive always run it against the block dev
----- Original Message ----- From: "Matt Wilder" <matt.wilder@bitmex.com> To: "Philip Brown" <pbrown@medata.com> Cc: "ceph-users" <ceph-users@ceph.io> Sent: Wednesday, February 10, 2021 12:06:55 PM Subject: Re: [ceph-users] Re: Device is not available after zap
Are you running zap on the lvm volume, or the underlying block device?
If you are running it against the lvm volume, it sounds like you need to run it against the block device so it wipes the lvm volumes as well. (Disclaimer: I don't run Ceph in this configuration)
On Wed, Feb 10, 2021 at 10:24 AM Philip Brown <pbrown@medata.com> wrote:
Sorry, not much to say other than a "me too". i spent a week testing ceph configurations.. it should have only been 2 days. but a huge amount of my time was wasted because I needed to do a full reboot on the hardware.
on a related note: sometimes "zap" didnt fully clean things up. I had to manually go in and clean up vgs. or pvs. or sometimes wipefs -a
so, in theory, this could be a linux LVM bug. but if I recall, i was doing this with ceph octopus, and centos 7.9
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io