1 Oct
2020
1 Oct
'20
5:53 p.m.
Hi Marc,
Did you have any success with `ceph-volume` for activating your OSD?
I am having a similar problem where the command `ceph-bluestore-tool`
fails to be able to read a label for a previously created OSD on an
LVM partition. I had previously been using the OSD without issues, but
after a reboot it fails to load.
1. I had initially created my OSD using Ceph Octopus 15.x with `ceph
orch daemon add osd <my hostname>:boot/cephfs_meta` that was able to
create an OSD on the LVM partition and bring up an OSD.
2. After a reboot, the OSD fails to come up, with error from
`ceph-bluestore-tool` happening inside the container specifically
being unable to read the label of the device.
3. When I query the symlinked device /dev/boot/cephfs_meta ->
/dev/dm3, with `dmsetup info /dev/dm-3`, I can see the state is active
and that it has a UUID, etc.
4. I installed `ceph-osd` CentOS package providing the
ceph-bluestore-tool, and tried to manually test and `sudo
ceph-bluestore-tool show-label --dev /dev/dm-3` fails to read the
label. When I try with other OSD's that were created for entire disks
this command is able to read the label and print out information.
I am considering submitting a ticket to the ceph issue tracker, as I
am unable to figure out why the ceph-bluestore-tool cannot read the
labels and it seems either the OSD was initially created incorrectly
or there is a bug in ceph-bluestore-tool.
One possibility is that I did not have the LVM2 package installed on
this host prior to the `ceph orch daemon add ..` command and this
caused a particular issue with the LVM partition OSD.
-Matt
On Sat, Sep 19, 2020 at 9:11 AM Marc Roos <M.Roos(a)f1-outsourcing.eu> wrote:
[@]# ceph-volume lvm activate 36 82b94115-4dfb-4ed0-8801-def59a432b0a
Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-36
Running command: /usr/bin/ceph-authtool
/var/lib/ceph/osd/ceph-36/lockbox.keyring --create-keyring --name
client.osd-lockbox.82b94115-4dfb-4ed0-8801-def59a432b0a --add-key
AQBxA2Zfj6avOBAAIIHqNNY2J22EnOZV+dNzFQ==
stdout: creating /var/lib/ceph/osd/ceph-36/lockbox.keyring
added entity client.osd-lockbox.82b94115-4dfb-4ed0-8801-def59a432b0a
auth(key=AQBxA2Zfj6avOBAAIIHqNNY2J22EnOZV+dNzFQ==)
Running command: /usr/bin/chown -R ceph:ceph
/var/lib/ceph/osd/ceph-36/lockbox.keyring
Running command: /usr/bin/ceph --cluster ceph --name
client.osd-lockbox.82b94115-4dfb-4ed0-8801-def59a432b0a --keyring
/var/lib/ceph/osd/ceph-36/lockbox.keyring config-key get
dm-crypt/osd/82b94115-4dfb-4ed0-8801-def59a432b0a/luks
Running command: /usr/sbin/cryptsetup --key-file - --allow-discards
luksOpen
/dev/ceph-9263e83b-7660-4f5b-843a-2111e882a17e/osd-block-82b94115-4dfb-4
ed0-8801-def59a432b0a I8MyTZ-RQjx-gGmd-XSRw-kfa1-L60n-fgQpCb
stderr: Device I8MyTZ-RQjx-gGmd-XSRw-kfa1-L60n-fgQpCb already exists.
Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-36
Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph
prime-osd-dir --dev /dev/mapper/I8MyTZ-RQjx-gGmd-XSRw-kfa1-L60n-fgQpCb
--path /var/lib/ceph/osd/ceph-36 --no-mon-config
stderr: failed to read label for
/dev/mapper/I8MyTZ-RQjx-gGmd-XSRw-kfa1-L60n-fgQpCb: (2) No such file or
directory
--> RuntimeError: command returned non-zero exit status: 1
dmsetup ls lists this????
Where is an option to set the weight? As far as I can see you can only
set this after peering started?
How can I mount this tmpfs manually to inspect this? Maybe put in the
manual[1]?
[1]
https://docs.ceph.com/en/latest/ceph-volume/lvm/activate/
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
--
Matt Larson, PhD
Madison, WI 53705 U.S.A.