Not all clients are capable yet to use the new key format. I suggest to rotate the key with --key-type aes and then mount the CephFS with the old key-type. The vulnaribilities are fixed when the core services have been upgraded, so you don't necessarily need to rotate client keyrings yet if your clients can't use them. Zitat von Johan <ceph-users@ceph.io>:
I have a cephadm managed cluster, Tentacle 20.2.4 and Debian 12 & 13.
On the Debian 12 Bookworm servers I can connect to the cluster. id: <ceph-id> health: HEALTH_OK (muted: AUTH_INSECURE_CLIENT_KEY_TYPE(13d AUTH_INSECURE_KEYS_ALLOWED(13d))
services: mon: 3 daemons, quorum lagenhet4,lagenhet3,lagenhet5 (age 117m) [leader: lagenhet4] mgr: lagenhet3.rbkplm(active, since 2h), standbys: lagenhet4.xontzc mds: 1/1 daemons up, 1 standby osd: 8 osds: 8 up (since 2h), 8 in (since 12d)
...
Worth noting is that my osd-disks are encrypted and those keys are not rotated. Keys for the crash services are not rotated either (a number of bootstrap-users are left too, but read in a earlier conversation that those should be safe to delete).
One client that I previously have been using to connect to a cephfs have had the key rotated, but I can't connect to the filesystem. If the keyfile is written as before [client.fsuser] key = <key> or [client.fsuser] key = <key> caps mds = "allow rw fsname=filesystem" caps mon = "allow r fsname=filesystem" caps osd = "allow rw tag cephfs data=filesystem" and only replace the key I get the error adding ceph secret key to kernel failed: Unknown error 524 couldn't append secret option: -524 when I try to mount the filesystem.
If I only have the key itself in the keyfile I get mount error: no mds (Metadata Server) is up. The cluster might be laggy, or you may not be authorised
To test if I have proper authorisation I created a new user sudo ceph fs authorize newuser client.newuser / rw | sudo tee /etc/ceph/ceph.client.newuser.keyring and I still get the same errors.
Looking at the keys in the dashboard it shows the same keys as I'm using.
I tested to create a new filesystem (information regarding previous filesystems are removed)
# ceph osd pool create dummy_data pool 'dummy_data' created # ceph osd pool create dummy_meta pool 'dummy_meta' created
# ceph fs new dummyfs dummy_meta dummy_data Pool 'dummy_data' (id '21') has pg autoscale mode 'on' but is not marked as bulk. Consider setting the flag by running # ceph osd pool set dummy_data bulk true new fs with metadata pool 22 and data pool 21
# ceph fs ls name: dummyfs, metadata pool: dummy_meta, data pools: [dummy_data ]
# sudo ceph orch apply mds dummyfs Scheduled mds.dummyfs update...
# ceph mds stat <deleted>:1 dummyfs:1 {<deleted>:0=<deleted>.lagenhet4.kvscuw=up:active,dummyfs:0=dummyfs.lagenhet5.jpqcdl=up:active} 2 up:standby
# ceph fs authorize dummyfs client.dummyuser / rw | tee /etc/ceph/ceph.client.dummyuser.keyring [client.dummyuser] key = AgBvWpxqEJPpOiAAjr6ziBqhdz23pCcU/2pCeY2ZGHNVKV4pAt13jCeKxr0= caps mds = "allow rw fsname=dummyfs" caps mon = "allow r fsname=dummyfs" caps osd = "allow rw tag cephfs data=dummyfs"
# mount -t ceph dummyuser@.dummyfs=/ /datastorage/dummyfs adding ceph secret key to kernel failed: Unknown error 524 couldn't append secret option: -524
I manually change keyring-file to include only the key.
# mount -t ceph dummyuser@.dummyfs=/ /datastorage/dummyfs mount error: no mds (Metadata Server) is up. The cluster might be laggy, or you may not be authorized
This procedure to create a filesystem has worked before. I also tested this procedure on my main cluster, Tentacle 20.2.1, and there was no problem and didn't have to edit keyring-file.
Anyone have any idea what could be wrong?
/Johan
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io