Hi Gary, On Thu, Aug 20, 2026 at 2:31 PM Gary Molenkamp <ceph-users@ceph.io> wrote:

Hi Patrick

...

...

One question, though: is it possible to have a client with two keys, aes and aes256? One of my cluster is mainly used for RBD with client.libvirt user, and I have hundreds of running VMs, which I cannot restart at once. So I would like to _add_ an aes256 key to the client.libvirt user, and incrementally restart the qemu processes which use this secret. No, you cannot have two keys. I think a lesson (for everyone) here is that each node should have its own client key. It became prevalent to use the same key everywhere for simplicity but it makes deployments and updates more challenging.

...

Or what is the recommended way of gradually replacing a client key? Create new keys for each node.

A challenge to that approach is that Proxmox PVE defines external RBD pools at the datacenter level using a single user/key pair. This then becomes the storage pool on each individual node in the proxmox cluster. I believe ceph-csi K8 storage class also defines this once cluster wide.

I understand but the only way to change things is to spread the word. If the key is that challenging to rotate, then I would suggest leaving it as aes and push for changes in the product or deployment technology. Rook developers are already aware this should be reworked. -- Patrick Donnelly, Ph.D. He / Him / His Red Hat Partner Engineer IBM, Inc. GPG: 19F28A586F808C2402351B93C3301A3E258DD79D