Hi Marc, Did you have any success with `ceph-volume` for activating your OSD? I am having a similar problem where the command `ceph-bluestore-tool` fails to be able to read a label for a previously created OSD on an LVM partition. I had previously been using the OSD without issues, but after a reboot it fails to load. 1. I had initially created my OSD using Ceph Octopus 15.x with `ceph orch daemon add osd <my hostname>:boot/cephfs_meta` that was able to create an OSD on the LVM partition and bring up an OSD. 2. After a reboot, the OSD fails to come up, with error from `ceph-bluestore-tool` happening inside the container specifically being unable to read the label of the device. 3. When I query the symlinked device /dev/boot/cephfs_meta -> /dev/dm3, with `dmsetup info /dev/dm-3`, I can see the state is active and that it has a UUID, etc. 4. I installed `ceph-osd` CentOS package providing the ceph-bluestore-tool, and tried to manually test and `sudo ceph-bluestore-tool show-label --dev /dev/dm-3` fails to read the label. When I try with other OSD's that were created for entire disks this command is able to read the label and print out information. I am considering submitting a ticket to the ceph issue tracker, as I am unable to figure out why the ceph-bluestore-tool cannot read the labels and it seems either the OSD was initially created incorrectly or there is a bug in ceph-bluestore-tool. One possibility is that I did not have the LVM2 package installed on this host prior to the `ceph orch daemon add ..` command and this caused a particular issue with the LVM partition OSD. -Matt On Sat, Sep 19, 2020 at 9:11 AM Marc Roos <M.Roos@f1-outsourcing.eu> wrote:
[@]# ceph-volume lvm activate 36 82b94115-4dfb-4ed0-8801-def59a432b0a Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-36 Running command: /usr/bin/ceph-authtool /var/lib/ceph/osd/ceph-36/lockbox.keyring --create-keyring --name client.osd-lockbox.82b94115-4dfb-4ed0-8801-def59a432b0a --add-key AQBxA2Zfj6avOBAAIIHqNNY2J22EnOZV+dNzFQ== stdout: creating /var/lib/ceph/osd/ceph-36/lockbox.keyring added entity client.osd-lockbox.82b94115-4dfb-4ed0-8801-def59a432b0a auth(key=AQBxA2Zfj6avOBAAIIHqNNY2J22EnOZV+dNzFQ==) Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-36/lockbox.keyring Running command: /usr/bin/ceph --cluster ceph --name client.osd-lockbox.82b94115-4dfb-4ed0-8801-def59a432b0a --keyring /var/lib/ceph/osd/ceph-36/lockbox.keyring config-key get dm-crypt/osd/82b94115-4dfb-4ed0-8801-def59a432b0a/luks Running command: /usr/sbin/cryptsetup --key-file - --allow-discards luksOpen /dev/ceph-9263e83b-7660-4f5b-843a-2111e882a17e/osd-block-82b94115-4dfb-4 ed0-8801-def59a432b0a I8MyTZ-RQjx-gGmd-XSRw-kfa1-L60n-fgQpCb stderr: Device I8MyTZ-RQjx-gGmd-XSRw-kfa1-L60n-fgQpCb already exists. Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-36 Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/mapper/I8MyTZ-RQjx-gGmd-XSRw-kfa1-L60n-fgQpCb --path /var/lib/ceph/osd/ceph-36 --no-mon-config stderr: failed to read label for /dev/mapper/I8MyTZ-RQjx-gGmd-XSRw-kfa1-L60n-fgQpCb: (2) No such file or directory --> RuntimeError: command returned non-zero exit status: 1
dmsetup ls lists this????
Where is an option to set the weight? As far as I can see you can only set this after peering started?
How can I mount this tmpfs manually to inspect this? Maybe put in the manual[1]?
[1] https://docs.ceph.com/en/latest/ceph-volume/lvm/activate/
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Matt Larson, PhD Madison, WI 53705 U.S.A.