Hi, Yes there are operation that you have to do by yourself. The upgrade only rotate the mon keys automatically. I tried to clarify this on this post : https://pepitedata.com/ceph-tentacle-squid-security-release/ Where you'll find some code snippet to help like: ``` wait_for_active_clean() { local timeout=${1:-1200} waited=0 not_clean while true; do # scrubbing is normal background activity; block only on states that are # not a superset of active+clean (degraded, undersized, peering, ...) not_clean=$(ceph -s -f json | jq '[.pgmap.pgs_by_state[] | select(.state_name | (startswith("active+clean") | not))] | length') [[ "${not_clean}" -eq 0 ]] && { echo " active+clean (scrubbing ignored)."; return 0; } (( waited >= timeout )) && { echo " TIMEOUT after ${timeout}s, ${not_clean} PG group(s) still not active+clean" >&2; return 1; } echo " waiting for active+clean (${not_clean} PG group(s) not clean, ${waited}s elapsed)..." sleep 10 waited=$(( waited + 10 )) done } hostlist=$(ceph orch ps --daemon-type osd -f json | jq -r '.[].hostname' | sort -u) for host in $hostlist; do echo "=== ${host} ===" ceph orch host ok-to-stop ${host} || continue ceph orch host maintenance enter ${host} || continue failed_osds="" for osdid in $(ceph orch ps --hostname=${host} --daemon-type osd -f json | jq -r '.[].daemon_id'); do echo "--- osd.${osdid} ---" newkey=$(ceph auth rotate --key-type=aes256k osd.${osdid} | grep -m1 '^[[:space:]]*key = ' | sed -E 's/^[[:space:]]*key = //') label_written=0 if [[ -n "${newkey}" ]]; then # maintenance enter does not guarantee the OSD container has released the # device yet (bluefs/rocksdb close can lag a few seconds); retry on busy for attempt in 1 2 3 4 5; do if ssh -i /tmp/cephadm_id_rsa -o StrictHostKeyChecking=accept-new ${cephadm_user}@${host} "sudo cephadm --image ${image} shell --name osd.${osdid} -- bash -c 'dev=$(readlink -f /var/lib/ceph/osd/ceph-${osdid}/block); ceph-bluestore-tool --dev $dev set-label-key --key osd_key -v ${newkey}'"; then label_written=1 break fi echo " device busy, retrying osd.${osdid} in 5s (attempt ${attempt})..." >&2 sleep 5 done fi if [[ ${label_written} -eq 0 ]]; then echo " !! FAILED rotating osd.${osdid} (keyring/label mismatch), it will not come back up on its own" >&2 failed_osds="${failed_osds} ${osdid}" fi done if [[ -n "${failed_osds}" ]]; then echo "!! osd(s) left in a bad state on ${host}: ${failed_osds} - fix manually before exiting maintenance" >&2 break fi ceph orch host maintenance exit ${host} || { echo "!! maintenance exit failed on ${host}" >&2; break; } maintenance exit should start every daemon on the host; confirm it did, since a failed rotation above leaves the OSD down for good for osdid in $(ceph orch ps --hostname=${host} --daemon-type osd -f json | jq -r '.[].daemon_id'); do status=$(ceph orch ps --hostname=${host} --daemon-type osd --daemon-id=${osdid} --refresh -f json | jq -r '.[0].status_desc') if [[ "${status}" != "running" ]]; then echo " osd.${osdid} is '${status}' after maintenance exit, forcing start" >&2 ceph orch daemon start osd.${osdid} fi done ceph orch ps --hostname=${host} --daemon-type osd --refresh wait_for_active_clean || break done ``` On 11/2/26 16:04, Robert Sander wrote:
Hi,
after upgrading a cephadm managed cluster to 20.2.4 the keys for MON, MGR, OSD and MDS are rotated to the new aes256k format.
What has to be done for the ceph-exporter, crash, nfs, rgw, smb and other services managed by cephadm and the bootstrap keys?
A "ceph orch redeploy service_type" does not update the keys.
Regards