Nice, thanks for the info. Zitat von Zakhar Kirpichenko <zakhar@gmail.com>:
Thank you, Eugen.
It was actually very straightforward. I'm happy to report back that there were no issues with removing and zapping the OSDs whose data devices were unavailable. I had to manually remove stale dm entries, but that was it.
/Z
On Tue, 2 Apr 2024 at 11:00, Eugen Block <eblock@nde.ag> wrote:
Hi,
here's the link to the docs [1] how to replace OSDs.
ceph orch osd rm <OSD_ID> --replace --zap [--force]
This should zap both the data drive and db LV (yes, its data is useless without the data drive), not sure how it will handle if the data drive isn't accessible though. One thing I'm not sure about is how your spec file will be handled. Since the drive letters can change I recommend to use a more generic approach, for example the rotational flags and drive sizes instead of paths. But if the drive letters won't change for the replaced drives it should work. I also don't expect an impact on the rest of the OSDs (except for backfilling, of course).
Regards, Eugen
[1] https://docs.ceph.com/en/latest/cephadm/services/osd/#replacing-an-osd
Zitat von Zakhar Kirpichenko <zakhar@gmail.com>:
Hi,
Unfortunately, some of our HDDs failed and we need to replace these drives which are parts of "combined" OSDs (DB/WAL on NVME, block storage on HDD). All OSDs are defined with a service definition similar to this one:
``` service_type: osd service_id: ceph02_combined_osd service_name: osd.ceph02_combined_osd placement: hosts: - ceph02 spec: data_devices: paths: - /dev/sda - /dev/sdb - /dev/sdc - /dev/sdd - /dev/sde - /dev/sdf - /dev/sdg - /dev/sdh - /dev/sdi db_devices: paths: - /dev/nvme0n1 - /dev/nvme1n1 filter_logic: AND objectstore: bluestore ```
In the above example, HDDs `sda` and `sdb` are not readable and data cannot be copied over to new HDDs. NVME partitions of `nvme0n1` with DB/WAL data are intact, but I guess that data is useless. I think the best approach is to replace the dead drives and completely rebuild each affected OSD. How should we go about this, preferably in a way that other OSDs on the node remain unaffected and operational?
I would appreciate any advice or pointers to the relevant documentation.
Best regards, Zakhar _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io