Sorry about spamming the mailing list... but I have some additional info. Why didn't I think about it earier, I don’t know; but I just removed the services with "ceph orch daemon rm osd.xx --force" Now my cluster is back in Healty state :) But stil I can't create osd's because of the error below. Not manualy and not by ' ceph orch apply osd --all-available-devices'...
-----Oorspronkelijk bericht----- Van: Dominique Ramaekers <dominique.ramaekers@cometal.be> Verzonden: woensdag 22 januari 2025 19:10 Aan: Frédéric Nass <frederic.nass@univ-lorraine.fr> CC: ceph-users <ceph-users@ceph.io> Onderwerp: [ceph-users] Re: osd won't restart
Hi Frédéric,
Thanks a lot for your help... I'm getting a bit desperate
I keep on having the same issue about permissions (see below) but I've checked. All folders in /var/lib/ceph are owned by ceph:ceph.
# ceph orch daemon add osd hvs005:/dev/sde ... /usr/bin/docker: stderr Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-18/keyring /usr/bin/docker: stderr Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-18/ /usr/bin/docker: stderr Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 18 --monmap /var/lib/ceph/osd/ceph- 18/activate.monmap --keyfile - --osdspec-affinity None --osd-data /var/lib/ceph/osd/ceph-18/ --osd-uuid a9a453d1-af41-4889-8bd7- 360db63031af --setuser ceph --setgroup ceph /usr/bin/docker: stderr stderr: 2025-01-22T17:50:07.503+0000 799d28690640 -1 bluestore(/var/lib/ceph/osd/ceph-18//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-18//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator &) decode past end of struct encoding: Malformed input [buffer:3] /usr/bin/docker: stderr stderr: 2025-01-22T17:50:07.503+0000 799d28690640 -1 bluestore(/var/lib/ceph/osd/ceph-18//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-18//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator &) decode past end of struct encoding: Malformed input [buffer:3] /usr/bin/docker: stderr stderr: 2025-01-22T17:50:07.504+0000 799d28690640 -1 bluestore(/var/lib/ceph/osd/ceph-18//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-18//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator &) decode past end of struct encoding: Malformed input [buffer:3] /usr/bin/docker: stderr stderr: 2025-01-22T17:50:07.504+0000 799d28690640 -1 bluestore(/var/lib/ceph/osd/ceph-18/) _read_fsid unparsable uuid /usr/bin/docker: stderr stderr: 2025-01-22T17:50:07.578+0000 799d28690640 -1 bluestore(/var/lib/ceph/osd/ceph-18//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-18//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-22T17:50:07.578+0000 799d28690640 -1 bluestore(/var/lib/ceph/osd/ceph-18//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-18//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-22T17:50:07.578+0000 799d28690640 -1 bluestore(/var/lib/ceph/osd/ceph-18//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-18//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-22T17:50:07.578+0000 799d28690640 -1 bluestore(/var/lib/ceph/osd/ceph-18//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-18//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-22T17:50:07.578+0000 799d28690640 -1 bluestore(/var/lib/ceph/osd/ceph-18//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-18//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-22T17:50:07.578+0000 799d28690640 -1 bluestore(/var/lib/ceph/osd/ceph-18//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-18//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-22T17:50:07.578+0000 799d28690640 -1 bluestore(/var/lib/ceph/osd/ceph-18//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-18//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-22T17:50:08.016+0000 799d28690640 -1 bluestore(/var/lib/ceph/osd/ceph-18//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-18//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-22T17:50:08.016+0000 799d28690640 -1 bdev(0x5702dfb36a80 /var/lib/ceph/osd/ceph-18//block) open open got: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-22T17:50:08.016+0000 799d28690640 -1 OSD::mkfs: ObjectStore::mkfs failed with error (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-22T17:50:08.016+0000 799d28690640 -1 ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph- 18/: (13) Permission denied /usr/bin/docker: stderr --> Was unable to complete a new OSD, will rollback changes ...