When upgrading Ceph cluster from version 19.2.4 to 19.2.6 using
cephadm, an issue occurred with encrypted OSDs.
2026-09-02 10:53:28,494 7f24e08141c0 DEBUG Determined image:
'quay.io/ceph/ceph@sha256:9217c31b7a5a900688d485adca40dd0c5c365008c15dd2c6429dfc888a9f5089'
2026-09-02 10:53:28,501 7f24e08141c0 INFO Reconfig daemon osd.36 ...
2026-09-02 10:53:29,885 7f24e08141c0 INFO Stopping osd.36 to update
osd_key bluestore label
2026-09-02 10:53:31,670 7f24e08141c0 INFO Opening osd.36 with
crypsetup
2026-09-02 10:53:33,955 7f24e08141c0 INFO Non-zero exit code 5 from
/usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host
--entrypoint /tmp/cryptsetup_action.sh --privileged --group-add=disk
--init -e
CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:9217c31b7a5a900688d485adca40dd0c5c365008c15dd2c6429dfc888a9f5089
-e NODE_NAME=k08r06cph03 -v /dev:/dev -v
/tmp/cephadm-osd-36-rotate-helper.sh:/tmp/cryptsetup_action.sh
quay.io/ceph/ceph@sha256:9217c31b7a5a900688d485adca40dd0c5c365008c15dd2c6429dfc888a9f5089
2026-09-02 10:53:33,956 7f24e08141c0 INFO /usr/bin/podman: stderr
Cannot use device
/dev/ceph-d1270182-40c9-49c8-9a52-f29d0475aeba/osd-block-80ae3d20-07df-4db5-96b6-2ad575a592ad
which is in use (already mapped or mounted).
2026-09-02 10:53:33,956 7f24e08141c0 ERROR ERROR: Got error rotating
osd keyring while using cryptsetup tool
Out:
Err:Cannot use device
/dev/ceph-d1270182-40c9-49c8-9a52-f29d0475aeba/osd-block-80ae3d20-07df-4db5-96b6-2ad575a592ad
which is in use (already mapped or mounted).
The issue did not occur on all hosts in the cluster (fortunately, it
was a test env), but it affected all OSDs on affected host. After
error occurred OSDs remained in a down state and had to be up
manually.
After several attempts, the update was successfully completed, but
we had to rotate the OSD keyrings ourselves.
A few hours latter when monmap was changed and cephadm scheduled a
reconfig this problem happened again, on 2 out of 4 nodes of the
cluster all osds goes down due to cephadm reconfig and this same
error: "Cannot use device... which is in use (already mapped or
mounted)".
Has anyone encountered similar problems?
Regards
Eugeniusz P