Re: Can't join new mon - lossy channel, failing
Hi, Let's do some serious necromancy here. I just had this exact problem. Turns out that after rebooting all nodes (one at the time of course), the monitor could join perfectly. Why? You tell me. We did not see any traces of the ip address in any dumps that we could get a hold of. I restarted all ceph-mgr beforehand as well. Med vänliga hälsningar Josef Johansson On 10/5/21 15:37, Konstantin Shalygin wrote:
As last resort we've change ipaddr of this host, and mon successfully joined to quorum. When revert ipaddr back - mon can't join, we think there something on switch side or on old mon's side. From old mon's I was checked new mon process connectivity via telnet - all works It's good to make a some reproducer of this network problem to know what exactly message of ceph protocol is broken
k
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi,
On 16 Aug 2023, at 11:30, Josef Johansson <josef@oderland.se> wrote:
Let's do some serious necromancy here.
I just had this exact problem. Turns out that after rebooting all nodes (one at the time of course), the monitor could join perfectly.
Why? You tell me. We did not see any traces of the ip address in any dumps that we could get a hold of. I restarted all ceph-mgr beforehand as well.
What is your release? This deadlock may be fixed via [1] [1] https://tracker.ceph.com/issues/55355 k
Hi, I'm running ceph version 15.2.16 (a6b69e817d6c9e6f02d0a7ac3043ba9cdbda1bdf) octopus (stable), that would mean I am not running the fix. Glad to know that an upgrade will solve the issue! Med vänliga hälsningar Josef Johansson On 8/16/23 12:05, Konstantin Shalygin wrote:
Hi,
On 16 Aug 2023, at 11:30, Josef Johansson <josef@oderland.se> wrote:
Let's do some serious necromancy here.
I just had this exact problem. Turns out that after rebooting all nodes (one at the time of course), the monitor could join perfectly.
Why? You tell me. We did not see any traces of the ip address in any dumps that we could get a hold of. I restarted all ceph-mgr beforehand as well. What is your release? This deadlock may be fixed via [1]
On 16 Aug 2023, at 13:23, Josef Johansson <josef@oderland.se> wrote:
I'm running ceph version 15.2.16 (a6b69e817d6c9e6f02d0a7ac3043ba9cdbda1bdf) octopus (stable), that would mean I am not running the fix.
Glad to know that an upgrade will solve the issue!
I'm not 100% sure that this tracker, exactly fix exactly this [ipaddr deadlock "somewhere"] issue, but looks very similar Thanks! k
participants (2)
-
Josef Johansson
-
Konstantin Shalygin