In Nautilus (Ubuntu Cloud Archive Train Version) the osd caps profile
rbd-read-only seems broken.
It is impossible to map a RBD if the user has the following caps:
[client.yyy]
key = AQBYL8NdHDpnERAAhk8XOKgFNwhUpCo3EMaW3g==
caps mgr = "profile rbd"
caps mon = "profile rbd"
caps osd = "profile rbd-read-only"
(also tried with 'caps osd = "profile rbd-read-only pool=rbd")
This is what I do and what I get back:
root@maas:~# sudo rbd map rbd/ltsp-01 --read-only --name client.yyy -m
10.101.0.24 --keyfile /etc/ceph/ceph.client.yyy.key
rbd: sysfs write failed
In some cases useful info is found in syslog - try "dmesg | tail".
rbd: map failed: (1) Operation not permitted
I can see no reason in dmesg:
root@maas:~# dmesg | tail
[ 4442.395179] rbd0: p1
[ 4442.396140] rbd: rbd0: capacity 21474836480 features 0x5
[ 4900.499466] libceph: mon0 (1)10.101.0.24:3300 socket closed (con
state CONNECTING)
[ 4900.884945] libceph: mon0 (1)10.101.0.24:3300 socket closed (con
state CONNECTING)
[ 4901.876882] libceph: mon0 (1)10.101.0.24:3300 socket closed (con
state CONNECTING)
[ 4903.892861] libceph: mon0 (1)10.101.0.24:3300 socket closed (con
state CONNECTING)
[ 4903.959079] libceph: mon1 (1)10.101.0.24:6789 session established
[ 4903.959624] libceph: client15954 fsid afdda66a-000c-11ea-824b-00163e31d42c
[ 4923.847328] libceph: mon1 (1)10.101.0.24:6789 session established
[ 4923.847749] libceph: client15969 fsid afdda66a-000c-11ea-824b-00163e31d42c
Using /sys/bus/rbd/add_single_major directly does not work as well:
root@maas:~# echo "10.101.0.24
name=client.yyy,secret=AQBYL8NdHDpnERAAhk8XOKgFNwhUpCo3EMaW3g== rbd
ltsp-01" | sudo tee /sys/bus/rbd/add_single_major
10.101.0.24 name=client.yyy,secret=AQBYL8NdHDpnERAAhk8XOKgFNwhUpCo3EMaW3g==
rbd ltsp-01
tee: /sys/bus/rbd/add_single_major: Operation not permitted
Using a client with 'caps osd = "profile rbd"' does work without any
problems.
System Information:
Ubuntu 18.04.3
ceph version 14.2.2 (4f8fa0a0024755aae7d95567c63f11d6862d55be) nautilus (stable)
Kernel: 5.3.0-19-generic (but also tested with 5.0.0-32-generic and
4.15.0-66-generic)
The cluster was built with MAAS and JUJU from openstack-charmers stable charms:
Ubuntu 18.04.3
ceph version 14.2.2 (4f8fa0a0024755aae7d95567c63f11d6862d55be) nautilus (stable)
Kernel: 4.15.0-66-generic
I wanted to file this as a bug report, but my account (trickkiste) has
not been approved yet.
Probably due to this bug according to jdillaman on IRC:
https://tracker.ceph.com/issues/42667
Regards,
Markus