I have a seemingly strange situation. I have three OSDs that I created with Ceph Octopus
using the `ceph orch daemon add <host>:device` command. All three were added and
everything was great. Then I rebooted the host. Now the daemon’s won’t start via Docker.
When I attempt to run the `docker` command directly it errors with:
root@balin:/var/lib/ceph/c3d06c94-bb66-4f84-bf78-470a2364b667/osd.12# /usr/bin/docker run
--rm --net=host --privileged --group-add=disk --name
ceph-c3d06c94-bb66-4f84-bf78-470a2364b667-osd.12 -e
CONTAINER_IMAGE=docker.io/ceph/ceph:v15 -e NODE_NAME=balin -v
/var/run/ceph/c3d06c94-bb66-4f84-bf78-470a2364b667:/var/run/ceph:z -v
/var/log/ceph/c3d06c94-bb66-4f84-bf78-470a2364b667:/var/log/ceph:z -v
/var/lib/ceph/c3d06c94-bb66-4f84-bf78-470a2364b667/crash:/var/lib/ceph/crash:z -v
/var/lib/ceph/c3d06c94-bb66-4f84-bf78-470a2364b667/osd.12:/var/lib/ceph/osd/ceph-12:z -v
/var/lib/ceph/c3d06c94-bb66-4f84-bf78-470a2364b667/osd.12/config:/etc/ceph/ceph.conf:z -v
/dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v
/run/lock/lvm:/run/lock/lvm --entrypoint /usr/bin/ceph-osd docker.io/ceph/ceph:v15 -n
osd.12 -f --setuser ceph --setgroup ceph --default-log-to-file=false
--default-log-to-stderr=true --default-log-stderr-prefix="debug "
debug 2020-05-07T22:58:06.258+0000 7f622a161ec0 0 set uid:gid to 167:167 (ceph:ceph)
debug 2020-05-07T22:58:06.258+0000 7f622a161ec0 0 ceph version 15.2.1
(9fd2f65f91d9246fae2c841a6222d34d121680ee) octopus (stable), process ceph-osd, pid 1
debug 2020-05-07T22:58:06.258+0000 7f622a161ec0 0 pidfile_write: ignore empty --pid-file
debug 2020-05-07T22:58:06.258+0000 7f622a161ec0 -1
bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open
/var/lib/ceph/osd/ceph-12/block: (13) Permission denied
debug 2020-05-07T22:58:06.258+0000 7f622a161ec0 -1 ** ERROR: unable to open OSD
superblock on /var/lib/ceph/osd/ceph-12: (2) No such file or directory
The OSDs are able to come back online if I run `ceph-volume lvm activate —all`. Everything
from a usage point of view is fine, even after a reboot, however I now have errors in the
`ceph orch ps` list:
osd.12 balin error 27s ago - <unknown>
docker.io/ceph/ceph:v15 <unknown> <unknown>
This is an Ubuntu 20.04 system, FWIW. I haven’t a clue where to go from here. While things
are technically working since the OSDs are online and functioning, I’d really like to have
them under the `ceph orch` management like the rest of the systems.
~Sean
Show replies by date