If your cluster uses mclock you might consider switching to wpq. Or you could also override mclock settings and use the "legacy" recovery/backfill parameters. The docs have more details: https://docs.ceph.com/en/squid/rados/configuration/mclock-config-ref/ https://docs.ceph.com/en/squid/rados/configuration/mclock-config-ref/#steps-... Zitat von Patrick Begou <ceph-users@ceph.io>:
Hi Eugen
Many thanks for this tutorial. I will document it in the management doc of the cluster for my colleagues. Very helpfull even if i do'nt want to break a new HDD to test it quickly 🙂
Just an additionnal basic question about this incident: why remapping pg is so slow in Ceph ? I have 5 nodes, 4 osd each and osd usage is around 25%of their capacity only. Of course OSDs are 14TB SATA HDDs so a few hundreds of MB/s when writing I think and not a lot of iops. I had initialy 5.2% of objects misplaced yesterday. Today (24 hours later) there are still 3.7% of misplaced objects. Recovery is between 70 and 100MB/s. Network is ethernet 25Gb/s. There are only around 3.5TB used per OSD so I do not understand how Ceph manages this and if there are some tunable parameters (the clusters has really a small I/O load from the clients)
[ceph: root@whitaker03-ceph /]# ceph osd metadata 2 { "id": 2, "arch": "x86_64", "back_addr": "[v2:172.30.1.99:6826/1673599336,v1:172.30.1.99:6827/1673599336]", "back_iface": "", "bluefs": "1", "bluefs_dedicated_db": "0", "bluefs_dedicated_wal": "0", "bluefs_single_shared_device": "1", "bluestore_allocation_from_file": "0", "bluestore_bdev_access_mode": "blk", "bluestore_bdev_block_size": "4096", "bluestore_bdev_dev_node": "/dev/dm-8", "bluestore_bdev_devices": "sda", "bluestore_bdev_driver": "KernelDevice", "bluestore_bdev_optimal_io_size": "0", "bluestore_bdev_partition_path": "/dev/dm-8", "bluestore_bdev_rotational": "1", "bluestore_bdev_size": "14000515448832", "bluestore_bdev_support_discard": "1", "bluestore_bdev_type": "hdd", "bluestore_min_alloc_size": "4096", "ceph_release": "squid", "ceph_version": "ceph version 19.2.2 (0eceb0defba60152a8182f7bd87d164b639885b8) squid (stable)", "ceph_version_short": "19.2.2", "ceph_version_when_created": "ceph version 19.2.2 (0eceb0defba60152a8182f7bd87d164b639885b8) squid (stable)", "container_hostname": "whitaker03-ceph", "container_image": "quay.io/ceph/ceph@sha256:8214ebff6133ac27d20659038df6962dbf9d77da21c9438a296b2e2059a56af6", "cpu": "AMD EPYC 7313P 16-Core Processor", "created_at": "2026-09-07T14:27:28.074430Z", "default_device_class": "hdd", "device_ids": "sda=ATA_MB014000GYCJT_K3S04LQP", "device_paths": "sda=/dev/disk/by-path/pci-0000:07:00.0-scsi-0:2:17:0", "devices": "sda", "distro": "centos", "distro_description": "CentOS Stream 9", "distro_version": "9", "front_addr": "[v2:172.30.1.99:6824/1673599336,v1:172.30.1.99:6825/1673599336]", "front_iface": "", "hb_back_addr": "[v2:172.30.1.99:6830/1673599336,v1:172.30.1.99:6831/1673599336]", "hb_front_addr": "[v2:172.30.1.99:6828/1673599336,v1:172.30.1.99:6829/1673599336]", "hostname": "whitaker03-ceph", "journal_rotational": "1", "kernel_description": "#1 SMP PREEMPT_DYNAMIC Sat May 24 12:53:17 EDT 2025", "kernel_version": "5.14.0-570.17.1.el9_6.x86_64", "mem_swap_kb": "10239996", "mem_total_kb": "131106104", "network_numa_unknown_ifaces": "back_iface,front_iface", "objectstore_numa_unknown_devices": "sda", "os": "Linux", "osd_data": "/var/lib/ceph/osd/ceph-2", "osd_objectstore": "bluestore", "osdspec_affinity": "all-available-devices", "rotational": "1" }
Patrick
Le 08/09/2026 à 11:41, Eugen Block a écrit :
Hi,
first, there's a command to manage and unmanage services:
ceph orch set-managed <service> ceph orch set-unmanaged <service>
So you don't need to apply them each time.
The OSD replacement has different aspects, for example if you have a failing drive which the kernel loses access to, you won't be able to drain and/or zap anything. In that case some manual intervention would be required to purge the OSD from the crush tree. If the OSD is responsive and draining works, you can (or should) use the orchestrator as I already mentioned.
Managing/Unmanaging an OSD service can be necessary if you need to zap the drive (with or without a separate DB/WAL device). Because if your service is managed and you zap the OSD it could be immediately redeployed, depending on the Ceph version even as a standalone OSD although your spec requires a separate DB device (a known bug). So here's an example from one of my test clusters:
# I want to replace osd.7 host1:~ # ceph osd metadata 7 | grep osdspec "osdspec_affinity": "osd-hdd-ssd-mix",
# The osd has a separate DB/WAL host1:~ # ceph device ls | grep osd.7 147249cb-9f8b-40c8-b host1:vdb osd.7 4a4167cb-8e42-4e1e-b host1:vdd osd.10 osd.7
# Set service to unmanaged host1:~ # ceph orch set-unmanaged osd.osd-hdd-ssd-mix
# Destroy and zap the drives (incl. DB/WAL) host1:~ # ceph orch osd rm 7 --replace --zap
After draining and zap have completed the drive can be replaced.
# Manage the service host1:~ # ceph orch set-managed osd.osd-hdd-ssd-mix
After some time the destroyed OSD will be recreated:
2026-09-08T09:15:01.280+0000 7f54bb9da640 0 [cephadm INFO cephadm.services.osd] Found osd claims for drivegroup osd-hdd-ssd-mix -> {'host1': ['7']} 2026-09-08T09:15:01.280+0000 7f54bb9da640 0 log_channel(cephadm) log [INF] : Found osd claims for drivegroup osd-hdd-ssd-mix -> {'host1': ['7']} 2026-09-08T09:15:17.676+0000 7f54b89d4640 0 [cephadm INFO cephadm.serve] Deploying daemon osd.7 on host1 2026-09-08T09:15:17.676+0000 7f54b89d4640 0 log_channel(cephadm) log [INF] : Deploying daemon osd.7 on host1
The OSD has been redeployed successfully.
Hope this helps!
Zitat von Patrick Begou <ceph-users@ceph.io>:
Situation is back to normal now.
I've run: ceph orch osd rm stop 2 ceph osd tree -> 2 hdd 12.73340 osd.2 destroyed 1.00000 1.00000 ceph orch ps --daemon_type osd -> no osd.2 was running ceph orch apply osd --all-available-devices --dry-run -> after some amount of time show:
################ OSDSPEC PREVIEWS ################ +---------+-----------------------+-----------------+----------+----+-----+ |SERVICE |NAME |HOST |DATA |DB |WAL | +---------+-----------------------+-----------------+----------+----+-----+ |osd |all-available-devices |whitaker03-ceph |/dev/sda |- |- | +---------+-----------------------+-----------------+----------+----+-----+
ceph orch apply osd --all-available-devices --unmanaged=false ceph orch ps --daemon_type osd -> after some amount of time it shows:
osd.2 whitaker03-ceph starting
Then:
osd.2 whitaker03-ceph running (39s)
and now datas are moved back to the new HDD of osd.2: ceph -s
cluster: id: xxxxxxxxxxxxxxxxxxxxx health: HEALTH_OK
services: mon: xxxxxxxxxxxxxx mgr: mds: xxxxxxxxxxxxxx osd: 20 osds: 20 up (since 25m), 20 in (since 25m); 89 remapped pgs
data: volumes: 2/2 healthy pools: 8 pools, 705 pgs objects: 17.65M objects, 37 TiB usage: 60 TiB used, 194 TiB / 255 TiB avail pgs: 6270145/120559591 objects misplaced (5.201%) 591 active+clean 88 active+remapped+backfill_wait 20 active+clean+scrubbing 5 active+clean+scrubbing+deep 1 active+remapped+backfilling
But I'm not really sure about what is the right procedure for a next HDD replacement....
All is running fine however. Thank you Eugen for your help.
Patrick
Le 07/09/2026 à 15:33, Eugen Block a écrit :
Try this:
ceph orch osd rm stop 2
This should clear the orch state and potentially redeploy the OSD if there's an applicable spec.
Zitat von Patrick Begou <ceph-users@ceph.io>:
Le 07/09/2026 à 14:44, Eugen Block a écrit :
> # cephadm ceph-volume lvm create --osd-id 2 --data /dev/sda
This will not work with cephadm as expected and as you already found out.
It's not a good idea to use ceph-volume to create OSDs, let the orchestrator handle it for you. Either via osd spec or for unmanaged OSDs like this:
ceph orch daemon add osd <host>:/dev/sdX
> Run the `ceph-volume lvm zap` command with `--destroy` against > the VG/LV if you want them to be destroyed.
I created a tracker for this misleading message:
https://tracker.ceph.com/issues/80083
It should not suggest to run ceph-volume directly.
> The dashboard do not show any error or warning (cluster health > is now OK because osd.2 is removed I think) but osd2 remains > in this state since nearly 1 hour. > Should I unset 'ceph orch apply osd --all-available-devices > --unmanaged=true' now to finish ?
Which state exactly? Has the OSD been drained successfully? What's the output of:
ceph orch osd rm status
If the OSD is empty but hasn't been zapped, it should be marked as "destroyed" in the osd tree, correct? In that case you can replace the drive now, cephadm should eventually recognize it as available and then deploy an OSD on it, at least if the osd spec is applicable.
Hi Eugen, this is the status:
Not using image '4892a7ef541bbfe6181ff8fd5c8e03957338f7dd73de94986a5f15e185dacd51' as it's not in list of non-dangling images with ceph=True label [ceph: root@whitaker03-ceph /]# ceph orch osd rm status OSD HOST STATE PGS REPLACE FORCE ZAP DRAIN STARTED AT 2 whitaker03-ceph done, waiting for purge 0 True False True
Ceph show this OSD as destroyed, the HDD has been replaced and zapped:
[ceph: root@whitaker03-ceph /]# ceph osd tree ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -1 254.66789 root default .... -5 50.93358 host whitaker03-ceph 2 hdd 12.73340 osd.2 destroyed 0 1.00000 5 hdd 12.73340 osd.5 up 1.00000 1.00000 12 hdd 12.73340 osd.12 up 1.00000 1.00000 15 hdd 12.73340 osd.15 up 1.00000 1.00000 ....
[ceph: root@whitaker03-ceph /]# ceph health HEALTH_OK
I do not know how to break this failing command "ceph orch osd rm 2 --replace" now.
Patrick
Zitat von Patrick Begou <ceph-users@ceph.io>:
> Hi Eugen, > > thank you for this quick response. > > - Yes it is a cephadm cluster. > - I had launched "cephadm shell" before the "ceph-volume lvm > zap /dev/sda" documented in > https://docs.ceph.com/en/squid/cephadm/services/osd/#replacing-an-osd > but this was not working. > - I saw that /dev/mapper/osvg-rootvol does not exists in the > cephadm container, but exists on the host. But ceph-volume > command is only available in cephadm container. > - your suggestion to launch from the cluster node "cephadm > ceph-volume lvm zap /dev/sda" works ! > - I try the same approach (login root on the node) for: > > # cephadm ceph-volume lvm create --osd-id 2 --data /dev/sda > > but it fails with: > > Inferring fsid aa64f278-3ba8-11f0-b327-303ea701bc10 > Not using image > '4892a7ef541bbfe6181ff8fd5c8e03957338f7dd73de94986a5f15e185dacd51' as > it's not in list of non-dangling images with ceph=True label > Non-zero exit code 1 from /usr/bin/podman run --rm --ipc=host > --stop-signal=SIGTERM --net=host --entrypoint > /usr/sbin/ceph-volume --privileged --group-add=disk --init -e > CONTAINER_IMAGE=quay.io/ceph/ceph:v19 -e > NODE_NAME=whitaker03-ceph -e CEPH_VOLUME_SKIP_RESTORECON=yes > -e CEPH_VOLUME_DEBUG=1 -v > /var/run/ceph/aa64f278-3ba8-11f0-b327-303ea701bc10:/var/run/ceph:z -v > /var/log/ceph/aa64f278-3ba8-11f0-b327-303ea701bc10:/var/log/ceph:z -v > /var/lib/ceph/aa64f278-3ba8-11f0-b327-303ea701bc10/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/aa64f278-3ba8-11f0-b327-303ea701bc10/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpsuhjdojs:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v19 lvm create --osd-id 2 --data > /dev/sda > /usr/bin/podman: stderr Running command: > /usr/bin/ceph-authtool --gen-print-key > /usr/bin/podman: stderr Running command: > /usr/bin/ceph-authtool --gen-print-key > /usr/bin/podman: stderr Running command: /usr/bin/ceph > --cluster ceph --name client.bootstrap-osd --keyring > /var/lib/ceph/bootstrap-osd/ceph.keyring osd tree -f json > /usr/bin/podman: stderr stderr: 2026-09-07T10:22:55.282+0000 > 7f3a7a2f8640 -1 auth: unable to find a keyring on > /var/lib/ceph/bootstrap-osd/ceph.keyring: (2) No such file or > directory > /usr/bin/podman: stderr stderr: 2026-09-07T10:22:55.282+0000 > 7f3a7a2f8640 -1 AuthRegistry(0x7f3a740669a0) no keyring found > at /var/lib/ceph/bootstrap-osd/ceph.keyring, disabling cephx > /usr/bin/podman: stderr stderr: 2026-09-07T10:22:55.282+0000 > 7f3a7a2f8640 -1 auth: unable to find a keyring on > /var/lib/ceph/bootstrap-osd/ceph.keyring: (2) No such file or > directory > /usr/bin/podman: stderr stderr: 2026-09-07T10:22:55.282+0000 > 7f3a7a2f8640 -1 AuthRegistry(0x7f3a7406a450) no keyring found > at /var/lib/ceph/bootstrap-osd/ceph.keyring, disabling cephx > /usr/bin/podman: stderr stderr: 2026-09-07T10:22:55.283+0000 > 7f3a7a2f8640 -1 auth: unable to find a keyring on > /var/lib/ceph/bootstrap-osd/ceph.keyring: (2) No such file or > directory > /usr/bin/podman: stderr stderr: 2026-09-07T10:22:55.283+0000 > 7f3a7a2f8640 -1 AuthRegistry(0x7f3a7a2f6ff0) no keyring found > at /var/lib/ceph/bootstrap-osd/ceph.keyring, disabling cephx > /usr/bin/podman: stderr stderr: 2026-09-07T10:22:55.285+0000 > 7f3a73fff640 -1 monclient(hunting): handle_auth_bad_method > server allowed_methods [2] but i only support [1] > /usr/bin/podman: stderr stderr: 2026-09-07T10:22:55.285+0000 > 7f3a737fe640 -1 monclient(hunting): handle_auth_bad_method > server allowed_methods [2] but i only support [1] > /usr/bin/podman: stderr stderr: 2026-09-07T10:22:55.285+0000 > 7f3a7886e640 -1 monclient(hunting): handle_auth_bad_method > server allowed_methods [2] but i only support [1] > /usr/bin/podman: stderr stderr: 2026-09-07T10:22:55.285+0000 > 7f3a7a2f8640 -1 monclient: authenticate NOTE: no keyring > found; disabled cephx authentication > /usr/bin/podman: stderr stderr: [errno 13] RADOS permission > denied (error connecting to the cluster) > /usr/bin/podman: stderr Traceback (most recent call last): > /usr/bin/podman: stderr File "/usr/sbin/ceph-volume", line > 33, in <module> > /usr/bin/podman: stderr > sys.exit(load_entry_point('ceph-volume==1.0.0', > 'console_scripts', 'ceph-volume')()) > /usr/bin/podman: stderr File > "/usr/lib/python3.9/site-packages/ceph_volume/main.py", line > 54, in __init__ > /usr/bin/podman: stderr self.main(self.argv) > /usr/bin/podman: stderr File > "/usr/lib/python3.9/site-packages/ceph_volume/decorators.py", > line 59, in newfunc > /usr/bin/podman: stderr return f(*a, **kw) > /usr/bin/podman: stderr File > "/usr/lib/python3.9/site-packages/ceph_volume/main.py", line > 166, in main > /usr/bin/podman: stderr terminal.dispatch(self.mapper, > subcommand_args) > /usr/bin/podman: stderr File > "/usr/lib/python3.9/site-packages/ceph_volume/terminal.py", > line 194, in dispatch > /usr/bin/podman: stderr instance.main() > /usr/bin/podman: stderr File > "/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py", > line 47, in main > /usr/bin/podman: stderr terminal.dispatch(self.mapper, self.argv) > /usr/bin/podman: stderr File > "/usr/lib/python3.9/site-packages/ceph_volume/terminal.py", > line 194, in dispatch > /usr/bin/podman: stderr instance.main() > /usr/bin/podman: stderr File > "/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/create.py", > line 84, in main > /usr/bin/podman: stderr self.create() > /usr/bin/podman: stderr File > "/usr/lib/python3.9/site-packages/ceph_volume/decorators.py", > line 16, in is_root > /usr/bin/podman: stderr return func(*a, **kw) > /usr/bin/podman: stderr File > "/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/create.py", > line 31, in create > /usr/bin/podman: stderr self.objectstore.safe_prepare() > /usr/bin/podman: stderr File > "/usr/lib/python3.9/site-packages/ceph_volume/objectstore/lvmbluestore.py", line 118, in > safe_prepare > /usr/bin/podman: stderr self.prepare() > /usr/bin/podman: stderr File > "/usr/lib/python3.9/site-packages/ceph_volume/decorators.py", > line 16, in is_root > /usr/bin/podman: stderr return func(*a, **kw) > /usr/bin/podman: stderr File > "/usr/lib/python3.9/site-packages/ceph_volume/objectstore/lvmbluestore.py", line 131, in > prepare > /usr/bin/podman: stderr self.pre_prepare() > /usr/bin/podman: stderr File > "/usr/lib/python3.9/site-packages/ceph_volume/objectstore/lvmbluestore.py", line 39, in > pre_prepare > /usr/bin/podman: stderr self.osd_id = > prepare_utils.create_id(self.osd_fsid, > /usr/bin/podman: stderr File > "/usr/lib/python3.9/site-packages/ceph_volume/util/prepare.py", line > 142, in create_id > /usr/bin/podman: stderr if osd_id_available(osd_id): > /usr/bin/podman: stderr File > "/usr/lib/python3.9/site-packages/ceph_volume/util/prepare.py", line > 180, in osd_id_available > /usr/bin/podman: stderr raise RuntimeError('Unable check > if OSD id exists: %s' % osd_id) > /usr/bin/podman: stderr RuntimeError: Unable check if OSD id exists: 2 > Traceback (most recent call last): > File "/usr/lib64/python3.9/runpy.py", line 197, in > _run_module_as_main > return _run_code(code, main_globals, None, > File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code > exec(code, run_globals) > File "/usr/sbin/cephadm/__main__.py", line 5581, in <module> > File "/usr/sbin/cephadm/__main__.py", line 5569, in main > File "/usr/sbin/cephadm/__main__.py", line 409, in _infer_config > File "/usr/sbin/cephadm/__main__.py", line 353, in _infer_fsid > File "/usr/sbin/cephadm/__main__.py", line 437, in _infer_image > File "/usr/sbin/cephadm/__main__.py", line 311, in _validate_fsid > File "/usr/sbin/cephadm/__main__.py", line 3314, in > command_ceph_volume > File "/usr/sbin/cephadm/cephadmlib/call_wrappers.py", line > 310, in call_throws > RuntimeError: Failed command: /usr/bin/podman run --rm > --ipc=host --stop-signal=SIGTERM --net=host --entrypoint > /usr/sbin/ceph-volume --privileged --group-add=disk --init -e > CONTAINER_IMAGE=quay.io/ceph/ceph:v19 -e > NODE_NAME=whitaker03-ceph -e CEPH_VOLUME_SKIP_RESTORECON=yes > -e CEPH_VOLUME_DEBUG=1 -v > /var/run/ceph/aa64f278-3ba8-11f0-b327-303ea701bc10:/var/run/ceph:z -v > /var/log/ceph/aa64f278-3ba8-11f0-b327-303ea701bc10:/var/log/ceph:z -v > /var/lib/ceph/aa64f278-3ba8-11f0-b327-303ea701bc10/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/aa64f278-3ba8-11f0-b327-303ea701bc10/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpsuhjdojs:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v19 lvm create --osd-id 2 --data > /dev/sda > [root@whitaker03-ceph ~]# cephadm ceph osd tree > usage: cephadm [-h] [--image IMAGE] [--docker] [--data-dir > DATA_DIR] [--log-dir LOG_DIR] [--logrotate-dir LOGROTATE_DIR] > [--sysctl-dir SYSCTL_DIR] [--unit-dir UNIT_DIR] [--verbose] > [--log-dest {file,syslog}] [--timeout TIMEOUT] > [--retry RETRY] [--env ENV] [--no-container-init] > [--no-cgroups-split] > {version,pull,inspect-image,ls,list-networks,adopt,rm-daemon,rm-cluster,run,shell,enter,ceph-volume,zap-osds,unit,unit-install,logs,bootstrap,deploy,_orch,check-host,prepare-host,add-repo,rm-repo,install,registry-login,gather-facts,host-maintenance,agent,disk-rescan} > ... > cephadm: error: argument > {version,pull,inspect-image,ls,list-networks,adopt,rm-daemon,rm-cluster,run,shell,enter,ceph-volume,zap-osds,unit,unit-install,logs,bootstrap,deploy,_orch,check-host,prepare-host,add-repo,rm-repo,install,registry-login,gather-facts,host-maintenance,agent,disk-rescan}: invalid choice: 'ceph' (choose from 'version', 'pull', 'inspect-image', 'ls', 'list-networks', 'adopt', 'rm-daemon', 'rm-cluster', 'run', 'shell', 'enter', 'ceph-volume', 'zap-osds', 'unit', 'unit-install', 'logs', 'bootstrap', 'deploy', '_orch', 'check-host', 'prepare-host', 'add-repo', 'rm-repo', 'install', 'registry-login', 'gather-facts', 'host-maintenance', 'agent', > 'disk-rescan') > > > So I try with the orchestrator as suggested > > # cephadm shell > [ceph: root@whitaker03-ceph /]# ceph orch osd rm 2 --replace > Scheduled OSD(s) for removal. > VG/LV for the OSDs won't be zapped (--zap wasn't passed). > Run the `ceph-volume lvm zap` command with `--destroy` against > the VG/LV if you want them to be destroyed. > > The dashboard do not show any error or warning (cluster health > is now OK because osd.2 is removed I think) but osd2 remains > in this state since nearly 1 hour. > > Should I unset 'ceph orch apply osd --all-available-devices > --unmanaged=true' now to finish ? > > Patrick > > Le 07/09/2026 à 11:59, Eugen Block a écrit : >> Is it a cephadm managed cluster? In that case don't use >> ceph-volume outside of containers. Instead you'd need to >> execute: >> >> cephadm ceph-volume lvm zap /dev/sda >> >> Or via orchestrator: >> >> ceph orch device zap <host> <device_path> >> >> And if it's orchestrated, you could just use orch as well to >> remove the OSD as described here: >> >> https://docs.ceph.com/en/squid/cephadm/services/osd/#replacing-an-osd >> If it's not cephadm managed, look at the ceph-volume.log to >> find out more about the root cause. But that error message >> reminds me of other user's attempts to use ceph-volume >> outside of containers. >> >> Zitat von Patrick Begou <ceph-users@ceph.io>: >> >>> Better layout of the text: >>> >>> >>> I'm replacing the HDD of osd.2 in my squid cluster following >>> the provided documentation: >>> https://docs.ceph.com/en/squid/rados/operations/add-or-rm-osds/ >>> >>> The OSD was yet out and the daemon down. >>> >>> The osd.2 is destroyed: >>> >>> ceph osd destroy 2 --yes-i-really-mean-it >>> >>> >>> I've stopped the node and replaced the HDD. Ceph (graphic >>> board) see the HDD as available. But I'm unable to go >>> further with the ceph-volume commands: >>> >>> # ceph-volume lvm zap /dev/sda >>> >>> --> RuntimeError: /dev/mapper/osvg-rootvol not found. >>> >>> # ceph-volume lvm prepare --osd-id 2 --data /dev/sda >>> >>> --> RuntimeError: /dev/mapper/osvg-rootvol not found. >>> >>> Something wrong in my cluster or a step is missing in the >>> documentation ? >>> >>> >>> >>> Le 07/09/2026 à 11:38, Patrick Begou a écrit : >>>> Hi >>>> >>>> I'm replacing the HDD of osd.2 in my squid cluster >>>> following the provided documentation: >>>> https://docs.ceph.com/en/squid/rados/operations/add-or-rm-osds/ >>>> >>>> The OSD was yet out and the daemon down. >>>> >>>> The osd.2 is destroyed: >>>> >>>> cephosddestroy2--yes-i-really-mean-it I've stopped the node >>>> and replaced the HDD. Ceph (graphic board) see the HDD as >>>> available. But I'm unable to go further with the >>>> ceph-volume commands: # ceph-volume lvm zap /dev/sda --> >>>> RuntimeError: /dev/mapper/osvg-rootvol not found. # >>>> ceph-volume lvm prepare --osd-id 2 --data /dev/sda --> >>>> RuntimeError: /dev/mapper/osvg-rootvol not found. >>>> >>>> Something wrong in my cluster or a step is missing in the >>>> documentation ? >>>> >>>> Thank >>>> >>>> Patrick >>>> >>>> _______________________________________________ >>>> ceph-users mailing list -- ceph-users@ceph.io >>>> To unsubscribe send an email to ceph-users-leave@ceph.io >>>> >>>> >>>> >>>> >>> >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users@ceph.io >>> To unsubscribe send an email to ceph-users-leave@ceph.io >> >> >> >> _______________________________________________ >> ceph-users mailing list -- ceph-users@ceph.io >> To unsubscribe send an email to ceph-users-leave@ceph.io >> >> >> >> > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io