Hi, this was the first thing I was thinking about (and yes, there had been some issues, but they are resolved - double checked!). MTU is consistent throughout the whole net and pings in all sizes are handled well. And MTU problems wouldn't probably make a difference between backfills and normal operations, I assume (I use a single network design). And in case of a MTU problem I would suspect that all OSDs on a wrong configured host would go down, or am I wrong? Is ceph sensitive to packet loss? On some VPN links I have up to 20% packet loss on 64k packets but less than 3% on 5k packets in the evenings. Perhaps TCP retransmits trigger some timeouts. Any suggestions how to tune this? I tried setting osd_heartbeat_grace to 120, but this doesn't help... Thanks for any ideas! Lorenz Am 14.08.19 um 13:56 schrieb Paul Emmerich: > MTU issues due to the VPN connection? > > > Paul >

Ok, then Ceph probably doesn't fit for me. I wanted to provide a backup platform for me, my family and my friends. Speed is not relevant, but long-term reliability. So I'm depending on home internet connections and VPN. At the moment I'm mostly using wireguard, but I could switch to openvpn to reduce packet loss (but this will increase latency). But I still don't understand why ceph runs rock-solid as long as there are backfills and recoveries? Lorenz Am 14.08.19 um 17:48 schrieb Wido den Hollander: > > On 8/14/19 5:46 PM, Lorenz Kiefner wrote: >> Hi, >> >> this was the first thing I was thinking about (and yes, there had been >> some issues, but they are resolved - double checked!). >> >> MTU is consistent throughout the whole net and pings in all sizes are >> handled well. And MTU problems wouldn't probably make a difference >> between backfills and normal operations, I assume (I use a single >> network design). And in case of a MTU problem I would suspect that all >> OSDs on a wrong configured host would go down, or am I wrong? >> >> Is ceph sensitive to packet loss? On some VPN links I have up to 20% >> packet loss on 64k packets but less than 3% on 5k packets in the evenings. >> > Yes, it is! 20%, but even 3% is a very, very high number when it comes > to packet loss. This should be almost 0%. > > Ceph is designed to work on low latency networks usually within one > datacenter. > >> Perhaps TCP retransmits trigger some timeouts. Any suggestions how to >> tune this? I tried setting osd_heartbeat_grace to 120, but this doesn't >> help... >> >> >> Thanks for any ideas! >> >> >> Lorenz >> >> >> Am 14.08.19 um 13:56 schrieb Paul Emmerich: >>> MTU issues due to the VPN connection? >>> >>> >>> Paul >>> >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io >> > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io