Hi Bartosz, thanks a lot for your thoughts on how to rotate the rbd client key. Even though I'm using Ganeti and not openstack, some general problems remain the same. On Fri, 18 Sep 2026, Bartosz Rabiega wrote:
If we rotate cephxuser key the living VMs will eventually stop working - when either auth_mon_ticket_ttl is exhausted or some connection to mon needs to be re-established librbd will try to authenticate with old key and fail.
That is a fundamental observation. While I'm not able to approve this technically it sounds very reasonable to me. I got the impression that others seem to neglect this?
Potential solutions ------------------- ... B - use new user and new key - new libvirt secret - running VMs will be OK, as long as we keep old user and key, live migration will probably fix the VM and use new user and new key - stopped (SHUTOFF) VMs will be OK when started, but will fail when old user/key is removed. username and libvirt secret uuid is kept in nova database and nothing will update it - new VMs and new attachments will be OK
That option is my favourite with the libvirt-less Ganeti. It seems you are unsure if live migration will switch to the new user? For Ganeti I can tell, that it will not (old user kept in runtime config). Stopped VMs don't have anything persistent in it's config WRT ceph auth in Ganeti. They will pick up the new user/key on startup. So everything I need to do, is to stop and start all VMs :-). Luckily in my environment this seems possible over a long period (~1 year). Other than that, I don't see other options yet. Thanks, Sascha.