Hi, since the latest ceph releases introduce a new key type to fix a security vulnerability, the release notes recommend to rotate the client keys and switch to the new key algorithm. This might not be possible for client that use a shared key for multiple hosts, e.g. proxmox, openstack and ceph-csi in k8s clusters. The recommended way to handle this is using host specific keys, but this it currently not an option for the mentioned clients. I would like to collect the various methods for handling the client side of the key rotation in the thread as a reference for other users. We did not upgrade yet, so the following steps are not tested yet. But we did something similar in the past in in different scenario, so it should work [tm]. So: use at own risk - Proxmox and Openstack Both usually use qemu, and qemu has support for performing a data live migration. In the case of proxmox I would propose to define a new rbd pool, using the same setup (crush etc.) as the original data pool. Then define a new storage backend in proxmox using the new pool and a ceph client with the new key algorithm. After testing the pool you should be able to migrate existing disks to the new pool. The migration option is available in the hardware overview of each virtual machine; select the disk, and the "Disk action" menu will allow to move the storage to a different backend. As mentioned above this is a live migration, so the running instance will not be affected (except for slightly higher I/O latencies). Since proxmox offers an API and CLI utilities, you should also be able to automate this process if you are running a large number of instances. Openstack supports live data migration for cinder volumes. You need to define a second volume type, again with a new pool and a new ceph client. The pool has to be configured on all controllers , which should be performed by your deployment tool. The "openstack volume set" command can be used to trigger the migration; I'm not sure whether it supports an option for waiting until the migration is done. In the worst case you'll have to poll the volume state. The might be some restriction, e.g. snapshot presence, which might prevent a volume from being migrated. Automation should also be possible as script or command line. We did a similar migration in the past, migrating from one ceph cluster to another; we had to update some stuff in the cinder database afterwards since they were still referring to the old cluster. You might want to test this first... Advantage: - live migration, no downtime necessary for most instances - automation possible Disadvantage: - physical data migration to new pool aka data is being copied - ceph-csi RBD and cephfs via csi in k8s clusters do not use qemu or other layers of indirection; rbds are mapped directly to the host and mounted, cephfs volume are mounted. CSI supports cloning and volume migration, so similar approaches might be possible. I haven't done this in the past, so I cannot comment on this or recommend any approach. Feedback from experiences users is highly welcome. Best regards, Burkhard Linke