You're mixing up non-cephadm with cephadm cluster commands. Don't use ceph-volume directly. Try the 'ceph cephadm osd activate <host>' command instead, then inspect osd logs and ceph-volume.log in case it fails. Zitat von Jacek Rużyczka <jacek.ruzyczka@googlemail.com>:
Hi,
I must have followed the wrong instructions (as often seems to be the case when it comes to Ceph). I gave Ceph another try with --fsid, and then it worked.
A missing fsid file cost me an hour, but the OSD subfolder seems to be OK now:
mixtile@blade3n1:~$ sudo ls -al /var/lib/ceph/8aad3073-39a1-11f1-bf6e-f2704a1efa 9b/osd.2 total 16 drwxr-xr-x 2 ceph ceph 4096 Jun 1 16:50 . drwx------ 14 167 167 4096 Jun 1 13:47 .. lrwxrwxrwx 1 ceph ceph 93 Jun 1 16:44 block -> /dev/ceph-81bc1761-e8f6-446c- 96f3-eb1b8f92628b/osd-block-9f7fd40d-0698-40b9-8718-62942b03e263 -rw-r--r-- 1 ceph ceph 37 Jun 1 16:50 fsid
In the meantime, I've managed to issue this command: sudo ceph-volume lvm activate --all
There's one problem though: I get the feedback that ceph-osd@2 is working, but sudo systemctl status ceph-osd@2 says that the daemon failed to start:
Jun 01 18:57:43 blade3n1 ceph-osd[317772]: 2026-06-01T18:57:43.403+0200 ffffb584 9040 -1 osd.2 7314 init authentication failed: (13) Permission denied
What puzzles me: Why is the OSD daemon started as a normal service and not as a Docker container?
Am So., 31. Mai 2026 um 21:27 Uhr schrieb Eugen Block via ceph-users < ceph-users@ceph.io>:
I don't see the '--fsid' flag in your command, it needs to be part of the bootstrap command. So 'cephadm bootrap --mon-ip IP --fsid FSID' and so on.
4 OSDs istn't that much, it should be doable in a reasonable amount of time. It took me around two hours to bring my test cluster with 3 OSDs back up, I think.
/var/lib/ceph/osd/ceph-2
This is the view from within the cephadm shell, the directory on the filesystem is :
/var/lib/ceph/{FSID}/osd.2
which is then mapped into the container.
Zitat von Jacek Rużyczka <jacek.ruzyczka@googlemail.com>:
No, the data is not inaccessible until you wipe the OSDs.
Ah, that's good to know that at least the data are still there.
How many OSDs are we talking about here?
I've got 4 OSDs, which hold 1.3 TB of data in total. I do have a backup of them on an external HDD, which spit out a handful of warnings during the last restore, so destroying the previous SSD, creating new ones, and trying another restore are things I'd like to avoid.
And regarding the fsid, you can specify it in the bootstrap command,
you won’t be able to change it afterwards.
This is exactly what I tried to do:
sudo cephadm bootstrap --mon-ip $MYIP --config /etc/ceph/ceph.conf~~ --allow-overwrite
And /etc/ceph/ceph.conf~~ contains the old FSID:
# minimal ceph.conf for 8aad3073-39a1-11f1-bf6e-f2704a1efa9b [global] fsid = 8aad3073-39a1-11f1-bf6e-f2704a1efa9b mon_host = [v2:10.20.0.11:3300/0,v1:10.20.0.11:6789/0] [v2: 10.20.0.12:3300/0,v1:10.20.0.12:6789/0] [v2: 10.20.0.13:3300/0,v1:10.20.0.13:6789/0] [v2: 10.20.0.14:3300/0,v1:10.20.0.14:6789/0]
Nevertheless, cephadm ignored the old settings and gave the new cluster a generated ID. So I've gotta tear down the cluster and start again? What about the key pairs stored in /etc/ceph?
You can simply create the OSD directories and create a symbolic link to
the block device. Cephadm simply maps it then.
I've now recreated the dir /var/lib/ceph/osd/ceph-2 and have to symlink /dev/ceph-81bc1761-e8f6-446c-96f3-eb1b8f92628b/o sd-block-9f7fd40d-0698-40b9-8718-62942b03e263 there? Or should ceph-2 be the name of the symlink?
I recommend to hire a professional to help you out, just to be on the safe
side.
I've got a Ceph expert in our hackspace, but I'm not sure whether he can help me.
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io