Authentication looks broken
Hi all! We upgraded a staging environment from 19.2.4 to 19.2.6 and rotated all of the keys without issues for the cluster. But for the Cephfs clients we noticed something strange. We created new keys for the Cephfs (kernel) clients so they could switch-over with just a remount. All went fine and thus we removed the old keys. Still everything kept working. Only then we rebooted one of the nodes to apply a new kernel. But it wasn't accessible after a couple of minutes, so we checked the console output and there where authentication errors for mds and mon. With a reset the node got online again, but without it's cephfs mount. When investigating the issue, we forgot to adjust the client name in the fstab. It was using the old name instead of the new name. So it looks like when mounting a cephfs filesystem, the backend isn't checking if the combination of client name and key is correct. Just if either is correct, you can mount the filesystem. Also, why isn't the umount failing after some time when authentication fails and then reboots the machine? Are others hitting this issue also? I'm going to check this on another system later on, but I thought I share this observation. System info Ceph is deployed via Cephadm Clients are running on RHEL 9 Using kernel mounts Installed ceph-common 19.2.6 on clients Kind regards, Sake
participants (1)
-
Sake