2 Feb
2023
2 Feb
'23
7:47 p.m.
The reason
this was particularly annoying is that the switch hardware
also runs a lot of our critical services, some of which also store data
on our internal Ceph cluster which was itself also affected by the
network integrity issue (loooooong heartbeats (like, tens of seconds),
presumed-dead mons and osds flapping, etc.). Narrowing down on a
specific cause was not fun.
Were the osds actually flapping or were they just being reported down
by osds that lost connection with them? Curious as I wouldn't expect
them to actually go down, just be reported down (but then maybe that's
what you meant by 'flapping').