Is there any better solution?

Yes, add more nodes. ;-) Having only two OSD nodes is not the best idea, it's kind of a corner case and I've observed some weird behaviour with corner cases in the past, not even mentioning the 2 replicas. Is this a test environment? Zitat von by morphin <morphinwithyou@gmail.com>:

I've figured out but I'm scared from the result. The solution is "mon_osd_min_down_reporters = 1" Due to "two node" cluster and "replicated 2" with "chooseleaf host" the reporter count should be set to 1 but on a malfunction this could be a serious problem.

Is there any better solution?

by morphin <morphinwithyou@gmail.com>, 20 May 2021 Per, 22:04 tarihinde şunu yazdı:

...

Hello

I have a weird problem on 3 node cluster. "Nautilus 14.2.9" When I try power failure OSD's are not marking as DOWN and MDS do not respond anymore. If I manually set osd down then MDS becomes active again.

BTW: Only 2 node has OSD's. Third node is only for MON.

I've set mon_osd_down_out_interval = 0.3 in ceph.conf at global section and restart all MON's but when I check it with "ceph daemon mon.ID config show" I see mon_osd_down_out_interval: "600". I didn't get it why its still "600" and honestly I don't know even it has any effect on my problem.

Where should I check?

_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

You can define the public_network [1]: ceph config set mon public_network *<mon-cidr-network>* For example: ceph config set mon public_network 10.1.2.0/24 Or is that already defined and it happens anyway? [1] https://docs.ceph.com/en/latest/cephadm/mon/#designating-a-particular-subnet... Zitat von "Harry G. Coin" <hgcoin@gmail.com>:

Is there a way to force '.. orch apply  *' to limit ip address selection to addresses matching the hostname in dns or /etc/hosts, or to a specific address given at 'host add' time?   I've hit a bothersome problem:

On v15, 'ceph orch apply mon ...' appears not to use the dns ip or /etc/hosts when installing a monitor, but instead appears to select one from the current list of interfaces up on the host.  99 of 100 times this is correct as the host has but one address on the public cluster subnet.  

However, this or that host might have a temporary added ip interface up on that subnet (whether for one-of diagnostic purposes or some HA assigned movable address like a time of day server).  Occasionally that second 'unlisted' address is 'up' when the orchestrator decides then would be a good time  to add a mon to that host.   Ceph somehow occasionally prefers this 'unlisted but up' interface address, using that permanently as the host address for the monitor docker image, even though that 'secondary' interface is not in DNS for the host, nor in /etc/hosts anywhere for that host.  

Is this a known issue?  Is there a way to direct the orchestrator to use 'dns resolvable only' or 'preferred' addresses?

Thanks

Harry Coin

_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

The public network is defined, and it happens anyway (the temporary, unlisted interface address is in the public network, but not either forward or reverse resolveable to the host in dns or /etc/hosts).

Okay, then I don't have a solution for that. Maybe someone else has dealt with this already. Zitat von "Harry G. Coin" <hgcoin@gmail.com>:

On 5/21/21 9:49 AM, Eugen Block wrote:

...

You can define the public_network [1]:

ceph config set mon public_network *<mon-cidr-network>*

For example:

ceph config set mon public_network 10.1.2.0/24

Or is that already defined and it happens anyway? The public network is defined, and it happens anyway (the temporary, unlisted interface address is in the public network, but not either forward or reverse resolveable to the host in dns or /etc/hosts).

[1] https://docs.ceph.com/en/latest/cephadm/mon/#designating-a-particular-subnet...