Thanks Sage, I opted to move to an explicit placement map of candidate hostnames and a replica count rather than using labels. This is a testing cluster of VMs to experiment before updating the production system. The only reason I was starting with 2 on the test cluster is that my production ceph has a few mons that are custom built docker containers that will give some grief in the adoption process. My plan is to remove those mons before converting to cephadm. On further thought, it will probably be safer to add a few normal mons to the OSD servers before adopting to cephadm, to ensure I have more than 3 before the conversion. I like the work done with cephadm and orchestration as compared to the older ceph-deploy. I do have a few questions/challenges on OSD deployment that I will put in a new post. Cheers. Gary On 2021-03-31 5:00 p.m., Sage Weil wrote:
Hi Gary,
It looks like everything you did is fine. I think the "problem" is that cephadm has/had some logic that tried to leave users with an odd number of monitors. I'm pretty sure this is why two of them were removed.
This code has been removed in pacific, and should probably be backported to octopus.
There is nothing wrong with an even number of mons. The only number you might want to avoid is 2 because a failure of either monitor will cause the cluster to lose quorum and become unavailable (quorum requires > N/2, which in a 2-mon case means both mons). As far as availability goes that is probably not ideal, but as far as durability goes, it's extremely useful to have a duplicate copy of the mon data so that losing a single disk doesn't destroy the cluster metadata (and require a complicated recovery process).
In any case, generally speaking, nobody should worry about having an even number of monitors. Focus instead of getting >2 so you can tolerate at least one mon failure and keep the cluster running.
On Wed, Mar 31, 2021 at 10:14 AM Gary Molenkamp <molenkam@uwo.ca> wrote:
A nautilus cluster with two mons (I know this is not correct for quorum), a mgr, and a handful of osds. I went though the adoption Any number of monitors is correct. Less than 3 is not recommended.
sage
-- Gary Molenkamp Computer Science/Science Technology Services Systems Administrator University of Western Ontario molenkam@uwo.ca http://www.csd.uwo.ca (519) 661-2111 x86882 (519) 661-3566