Issue with encrypted OSDs when upgrading from version 19.2.4 to 19.2.6 using cephadm
When upgrading Ceph cluster from version 19.2.4 to 19.2.6 using cephadm, an issue occurred with encrypted OSDs. 2026-09-02 10:53:28,494 7f24e08141c0 DEBUG Determined image: 'quay.io/ceph/ceph@sha256:9217c31b7a5a900688d485adca40dd0c5c365008c15dd2c6429dfc888a9f5089' 2026-09-02 10:53:28,501 7f24e08141c0 INFO Reconfig daemon osd.36 ... 2026-09-02 10:53:29,885 7f24e08141c0 INFO Stopping osd.36 to update osd_key bluestore label 2026-09-02 10:53:31,670 7f24e08141c0 INFO Opening osd.36 with crypsetup 2026-09-02 10:53:33,955 7f24e08141c0 INFO Non-zero exit code 5 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /tmp/cryptsetup_action.sh --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:9217c31b7a5a900688d485adca40dd0c5c365008c15dd2c6429dfc888a9f5089 -e NODE_NAME=k08r06cph03 -v /dev:/dev -v /tmp/cephadm-osd-36-rotate-helper.sh:/tmp/cryptsetup_action.sh quay.io/ceph/ceph@sha256:9217c31b7a5a900688d485adca40dd0c5c365008c15dd2c6429dfc888a9f5089 2026-09-02 10:53:33,956 7f24e08141c0 INFO /usr/bin/podman: stderr Cannot use device /dev/ceph-d1270182-40c9-49c8-9a52-f29d0475aeba/osd-block-80ae3d20-07df-4db5-96b6-2ad575a592ad which is in use (already mapped or mounted). 2026-09-02 10:53:33,956 7f24e08141c0 ERROR ERROR: Got error rotating osd keyring while using cryptsetup tool Out: Err:Cannot use device /dev/ceph-d1270182-40c9-49c8-9a52-f29d0475aeba/osd-block-80ae3d20-07df-4db5-96b6-2ad575a592ad which is in use (already mapped or mounted). The issue did not occur on all hosts in the cluster (fortunately, it was a test env), but it affected all OSDs on affected host. After error occurred OSDs remained in a down state and had to be up manually. After several attempts, the update was successfully completed, but we had to rotate the OSD keyrings ourselves. A few hours latter when monmap was changed and cephadm scheduled a reconfig this problem happened again, on 2 out of 4 nodes of the cluster all osds goes down due to cephadm reconfig and this same error: "Cannot use device... which is in use (already mapped or mounted)". Has anyone encountered similar problems? Regards Eugeniusz P
participants (1)
-
Eugeniusz Pokora