On Jul 24, 2026, at 7:23 AM, Michel Le Cocq via ceph-users <ceph-users@ceph.io> wrote:

The point I wanted to explore is the following:

We have three nodes for the MON MGR and MDS roles.

To bootstrap the cluster, we installed cephadm via apt only on the first node (M1).

Then, we copied the ceph.pub key to the second node (M2) and the third node (M3).

After that, we ran the following commands on M1:

ceph orch host add m2 <IP> ceph orch host add m3 <IP>

Note: cephadm is not installed on M2 and M3.

Eugen covered this, but I'll try to help: The copy of cephadm that you installed on M1 was used *only* to bootstrap the cluster. You don't use that copy ever again. The bootstrap process and "ceph orch host add" install *another*, different copy of cephadm in a different location. You could "apt delete cephadm" on M1 and nothing would change. This admittedly is a bit confusing.

Question:

What happens if M1 fails completely?

Nothing.

In this scenario, we would need the file: /var/lib/ceph/{FSID}/cephadm.{SOME_DIGEST} to be able to launch the cephadm shell and recover the cluster.

That is installed on every cluster node. But you don't strictly need the cephadm shell. What you do want to do is ceph orch host label add m1 _admin ceph orch host label add m2 _admin ceph orch host label add m3_admin and ensure that your orch services are set up to let mon, mgr, prometheus, grafana etc. run on all three nodes. Once you have these additional hosts added to the cluster, M2 and M3 are no different from M1. And assuming that your pools in this scenario are replicated size=3, which is default, if M1 goes up in smoke you don't need to recover the cluster because the cluster and all services will continue serving data. Your redundancy will be degraded until you get a third host back in, but the cluster itself will be fine.

...

...

Is it a good idee to do on host :

# apt purge cephadm

That doesn't hurt, but it isn't necessary unless you want to run the cephadm shell.

...

...

# chmod +x /var/lib/ceph/{FSID}/cephadm.{SOME_DIGEST} # ln -s /var/lib/ceph/{FSID}/cephadm.{SOME_DIGEST} /root/cephadm

Nononono. The orchestrator goes to great lengths to free you from micromanagement.