Hi Adam, I tried following your guidance regarding rotating an OSD lockbox key via ceph auth rotate and then redeploying the OSD via ceph orch redeploy in a test cluster. It seems that cephadm does not update the lockbox keyring during a redeploy, which results in the OSDs failing to start with the new keyring because they cannot unlock their dm-crypt volumes. To restore my test cluster, I had to manually update the lockbox.keyring for each OSD with the new key. Is this a bug in cephadm, or am I missing something here? Best regards, Max ________________________________ From: Adam King <ceph-users@ceph.io> Sent: 20 August 2026 15:46 To: Pedro Macedo <pmacedo@pmacedo.com> Cc: ceph-users@ceph.io <ceph-users@ceph.io> Subject: [ceph-users] Re: [CVE] [URGENT] Squid v19.2.6 and Tentacle v20.2.4 released Warning: This email is from an unusual correspondent. Warning: Make sure this is someone you trust. That command got removed for the time being as it used an old way of doing the key rotation that wasn't built to work with the changes for the cve and seemed to be broken for a number of daemon types. Since the name would make people want to use it it was decided to just temporarily disables it so we can fix it and reintroduce it in a new version as an easier way to rotate the keys. For any keys tied to a cephadm deployed daemon that aren't automatically rotated (cephadm handles mgr, mon, osd, and mds keyrings during upgrade) it's a matter of `ceph auth rotate <entity-name> --key-type aes256k` then `ceph orch daemon redeploy <daemon-name>`. For anyone using nfs exports managed through the nfs module the export's keyrings also need to be rotated. `ceph auth rotate <export-keyring-entity> --key-type aes256k`, `ceph nfs export info <nfs-cluster-name> <pseudo-path> > export_file`, `ceph nfs export apply <nfs-cluster-name> -i export_file`. Between rotating the keyrings for the nfs cluster/daemons and rotating the keyring for the individual export(s) those export(s) are unavailable iirc. On Thu, Aug 20, 2026 at 7:11 AM Pedro Macedo <ceph-users@ceph.io> wrote:
Hi,
I upgraded a test cluster to Ceph Squid 19.2.6 via cephadm. The following cephadm managed client keys were not rotated automatically:
client.bootstrap-* (bootstrap-rgw, bootstrap-osd, ...) client.ceph-exporter. client.crash. client.osd-lockbox.. client.rgw.
The Squid docs mention ceph orch daemon rotate-key <name>, but in 19.2.*
On 2026-08-20 12:57 PM, Maximilian Stinsky-Damke wrote: this command does not exist.
Also seeing this on the 20.2.4 image via cephadm - rotate-key is missing there, but present on 20.2.3.
What is the correct way to rotate these client keys via cephadm as part
of the cve fix?
Thanks for any guidance. Best Regards Max
________________________________ From: Patrick Donnelly <ceph-users@ceph.io> Sent: 19 August 2026 18:08 To: ceph-users <ceph-users@ceph.io> Subject: [ceph-users] [CVE] [URGENT] Squid v19.2.6 and Tentacle v20.2.4
released
Warning: This email is from an unusual correspondent. Warning: Make sure this is someone you trust.
Today, August 19th, 2026, the Ceph project has released Tentacle 20.2.4 and Squid 19.2.6 hotfixes for four CVEs across several components.
This is the sixth backport release in the Squid series.
This is the fourth backport release in the Tentacle series.
Additional details will be posted on the Ceph website/blog:
We strongly recommend that all Ceph operators upgrade to one of these releases as soon as possible.
## Critical Upgrade Steps
The fix for CVE-2025-30156 introduces a new CephX key type, aes256k. This is the first time Ceph has introduced a new key type for CephX credentials. Therefore, a new procedure exists for upgrading and rotating Ceph daemon keys as part of this CVE. Before upgrading a Ceph cluster, the operator should familiarize themselves with the procedure documented at:
These steps are for package-based deployments of Ceph. In other sections, the document also includes new information about key rotation procedures, monitor authentication controls, and emergency recovery procedures.
Deployments using cephadm will automate the process except for client keys. You may notice that Cephadm spends more time than normal on the upgrade after updating all daemon images. This is due to a new process rotating the OSD and MDS keys. Please be patient. There are plans to improve this in the future.
Deployments using Rook will also automate rotation of some client keys with some exclusions. Look to the Rook project’s announcement for more details.
Client and kernel upgrades are recommended to support aes256k. Note that upstream Linux kernel client support began in kernel 7.0 and has been backported to CentOS Stream 9 and 10. Check with your distribution vendor for backported key support before rotating client keys used by the kernel.
As part of upgrading a Ceph cluster, six new health warnings and errors will be generated. This is normal. As part of the process to upgrade/rotate entity keys, you will address the warnings and errors as you go. These are documented beginning here:
The CephX documentation has been brought up to date and vetted for accuracy. Those changes are live here:
Regarding the fix for CVE-2026-54330, RGW will now reject Sigv4 requests with `host` and `x-amz-` headers not included in the signed subset. Unfortunately, the REST client used in multisite was generating such improperly signed requests.
If you are running multisite, you must set the ``rgw_sigv4_insecure`` option to true before you begin to upgrade. After all clusters are upgraded, set the option to ``false`` again.
Regarding the fix for CVE-2026-50152, formal guidance on rotating all secrets stored in the Monitor config-key store will be forthcoming. In the meantime, operators of cephadm-managed clusters are strongly encouraged to rotate the cephadm SSH key, as an established process exists:
See:
Rotating other secrets is advised but the process is not yet established. Operators should assess their cluster's potential exposure to determine if immediate Monitor config-key secret rotation is required for their environment.
## CVE Fixes
* CVE-2025-30156 is an authentication bypass in CephX caused by misuse of AES-CBC.
See:
* CVE-2026-39944 shares the unauthenticated-encryption root cause of CVE-2025-30156, but applies it to RGW's STS session tokens resulting in improper verification of a cryptographic signature.
See:
* CVE-2026-50152 is an improper authorization flaw in the Ceph Monitor subscription handler.
See:
* CVE-2026-54330 is a flaw in RGW not properly verifying its SigV4 cryptographic signatures in RGW's SigV4 verifier.
See:
## Other Media
A Cephalocon 2025 talk covers some of the details surrounding the new CephX key upgrade process.
## Getting Ceph ------------ * Source at
* v19.2.6 Tarball at https://urlsand.esvalabs.com/?u=https%3A%2F%2Fdownload.ceph.com%2Ftarballs%2Fceph-19.2.6.tar.gz&e=50525cf7&h=1ff455cf&f=y&p=y&m=4hQBLz2RJPzTh0J * v20.2.4 Tarball at https://urlsand.esvalabs.com/?u=https%3A%2F%2Fdownload.ceph.com%2Ftarballs%2Fceph-20.2.4.tar.gz&e=50525cf7&h=aa19cd0e&f=y&p=y&m=4hQBLz2RJPzTh0J * Containers at https://urlsand.esvalabs.com/?u=https%3A%2F%2Fquay.io%2Frepository%2Fceph%2Fceph&e=50525cf7&h=9878a2f7&f=y&p=y&m=4hQBLz2RJPzTh0J * For packages, see https://urlsand.esvalabs.com/?u=https%3A%2F%2Fdocs.ceph.com%2Fen%2Flatest%2Finstall%2Fget-packages%2F&e=50525cf7&h=a319dcda&f=y&p=y&m=4hQBLz2RJPzTh0J * For upgrade with cephadm, see:
* v19.2.6 Release git sha1: f9fd95b4335bad6a26d7a74468f55d269e8dbef4 * v20.2.4 Release git sha1: 7f793731f1b39eb4f465e960113d2363c311b964
-- Patrick Donnelly, Ph.D. He / Him / His Red Hat Partner Engineer IBM, Inc. GPG: 19F28A586F808C2402351B93C3301A3E258DD79D _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- This message has been checked by Libraesva ESG and is found to be clean. Follow this link to report as spam/bad: https://mx10.wiit.cloud/action/4hQBLz2RJPzTh0J/report-as-bad Follow this link to blocklist sender: https://mx10.wiit.cloud/action/4hQBLz2RJPzTh0J/blocklist
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io -- This message has been checked by Libraesva ESG and is found to be clean. Follow this link to report as spam/bad: https://mx10.wiit.cloud/action/4hQl8t5ktzzTkbn/report-as-bad Follow this link to blocklist sender: https://mx10.wiit.cloud/action/4hQl8t5ktzzTkbn/blocklist