Hi Dave, On Tue, 2020-08-25 at 15:25 +0100, david.neal wrote:
Hi Momo,
This can be caused by many things apart from the ceph sw.
For example I saw this once with the MTU in openvswitch not fully matching on a few nodes . We realised this using ping between nodes. For a 9000 MTU:
"linux- ping -M do -s 8972 -c 4 <other host ip>
I've performed a test as you suggested and had no hiccups for any node (long ping log is at the bottom of this email for the curious).
Perhaps starting from the ground up and testing might be the way to go?
This cluster is very new, deployed in December and has been kept up to date since. Like I said previously, we didn't do any config customization - everything is as vanilla as possible.
Kind regards,
Dave
<snip> Kind regards, Momo. Log: root@cephosd01:~# ping -M do -s 8972 -c 4 10.179.40.33 PING 10.179.40.33 (10.179.40.33) 8972(9000) bytes of data. 8980 bytes from 10.179.40.33: icmp_seq=1 ttl=64 time=0.020 ms 8980 bytes from 10.179.40.33: icmp_seq=2 ttl=64 time=0.047 ms 8980 bytes from 10.179.40.33: icmp_seq=3 ttl=64 time=0.030 ms 8980 bytes from 10.179.40.33: icmp_seq=4 ttl=64 time=0.027 ms --- 10.179.40.33 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3062ms rtt min/avg/max/mdev = 0.020/0.031/0.047/0.009 ms root@cephosd01:~# ping -M do -s 8972 -c 4 10.179.40.34 PING 10.179.40.34 (10.179.40.34) 8972(9000) bytes of data. 8980 bytes from 10.179.40.34: icmp_seq=1 ttl=64 time=0.141 ms 8980 bytes from 10.179.40.34: icmp_seq=2 ttl=64 time=0.094 ms 8980 bytes from 10.179.40.34: icmp_seq=3 ttl=64 time=0.106 ms 8980 bytes from 10.179.40.34: icmp_seq=4 ttl=64 time=0.139 ms --- 10.179.40.34 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3078ms rtt min/avg/max/mdev = 0.094/0.120/0.141/0.020 ms root@cephosd01:~# ping -M do -s 8972 -c 4 10.179.40.35 PING 10.179.40.35 (10.179.40.35) 8972(9000) bytes of data. 8980 bytes from 10.179.40.35: icmp_seq=1 ttl=64 time=0.113 ms 8980 bytes from 10.179.40.35: icmp_seq=2 ttl=64 time=0.169 ms 8980 bytes from 10.179.40.35: icmp_seq=3 ttl=64 time=0.138 ms 8980 bytes from 10.179.40.35: icmp_seq=4 ttl=64 time=0.081 ms --- 10.179.40.35 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3062ms rtt min/avg/max/mdev = 0.081/0.125/0.169/0.033 ms root@cephosd01:~# ping -M do -s 8972 -c 4 10.179.40.36 PING 10.179.40.36 (10.179.40.36) 8972(9000) bytes of data. 8980 bytes from 10.179.40.36: icmp_seq=1 ttl=64 time=0.147 ms 8980 bytes from 10.179.40.36: icmp_seq=2 ttl=64 time=0.163 ms 8980 bytes from 10.179.40.36: icmp_seq=3 ttl=64 time=0.132 ms 8980 bytes from 10.179.40.36: icmp_seq=4 ttl=64 time=0.077 ms --- 10.179.40.36 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3054ms rtt min/avg/max/mdev = 0.077/0.129/0.163/0.035 ms root@cephosd01:~# ping -M do -s 8972 -c 4 10.179.40.37 PING 10.179.40.37 (10.179.40.37) 8972(9000) bytes of data. 8980 bytes from 10.179.40.37: icmp_seq=1 ttl=64 time=0.095 ms 8980 bytes from 10.179.40.37: icmp_seq=2 ttl=64 time=0.153 ms 8980 bytes from 10.179.40.37: icmp_seq=3 ttl=64 time=0.145 ms 8980 bytes from 10.179.40.37: icmp_seq=4 ttl=64 time=0.122 ms --- 10.179.40.37 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3070ms rtt min/avg/max/mdev = 0.095/0.128/0.153/0.026 ms root@cephosd01:~# ping -M do -s 8972 -c 4 10.179.40.38 PING 10.179.40.38 (10.179.40.38) 8972(9000) bytes of data. 8980 bytes from 10.179.40.38: icmp_seq=1 ttl=64 time=0.132 ms 8980 bytes from 10.179.40.38: icmp_seq=2 ttl=64 time=0.156 ms 8980 bytes from 10.179.40.38: icmp_seq=3 ttl=64 time=0.101 ms 8980 bytes from 10.179.40.38: icmp_seq=4 ttl=64 time=0.143 ms --- 10.179.40.38 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3070ms rtt min/avg/max/mdev = 0.101/0.133/0.156/0.020 ms root@cephosd01:~# ping -M do -s 8972 -c 4 10.179.40.39 PING 10.179.40.39 (10.179.40.39) 8972(9000) bytes of data. 8980 bytes from 10.179.40.39: icmp_seq=1 ttl=64 time=0.140 ms 8980 bytes from 10.179.40.39: icmp_seq=2 ttl=64 time=0.094 ms 8980 bytes from 10.179.40.39: icmp_seq=3 ttl=64 time=0.155 ms 8980 bytes from 10.179.40.39: icmp_seq=4 ttl=64 time=0.155 ms --- 10.179.40.39 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3054ms rtt min/avg/max/mdev = 0.094/0.136/0.155/0.025 ms root@cephosd01:~# root@cephosd02:~# ping -M do -s 8972 -c 4 10.179.40.33 PING 10.179.40.33 (10.179.40.33) 8972(9000) bytes of data. 8980 bytes from 10.179.40.33: icmp_seq=1 ttl=64 time=0.143 ms 8980 bytes from 10.179.40.33: icmp_seq=2 ttl=64 time=0.149 ms 8980 bytes from 10.179.40.33: icmp_seq=3 ttl=64 time=0.137 ms 8980 bytes from 10.179.40.33: icmp_seq=4 ttl=64 time=0.081 ms --- 10.179.40.33 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3080ms rtt min/avg/max/mdev = 0.081/0.127/0.149/0.029 ms root@cephosd02:~# ping -M do -s 8972 -c 4 10.179.40.34 PING 10.179.40.34 (10.179.40.34) 8972(9000) bytes of data. 8980 bytes from 10.179.40.34: icmp_seq=1 ttl=64 time=0.019 ms 8980 bytes from 10.179.40.34: icmp_seq=2 ttl=64 time=0.047 ms 8980 bytes from 10.179.40.34: icmp_seq=3 ttl=64 time=0.048 ms 8980 bytes from 10.179.40.34: icmp_seq=4 ttl=64 time=0.047 ms --- 10.179.40.34 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3066ms rtt min/avg/max/mdev = 0.019/0.040/0.048/0.013 ms root@cephosd02:~# ping -M do -s 8972 -c 4 10.179.40.35 PING 10.179.40.35 (10.179.40.35) 8972(9000) bytes of data. 8980 bytes from 10.179.40.35: icmp_seq=1 ttl=64 time=0.074 ms 8980 bytes from 10.179.40.35: icmp_seq=2 ttl=64 time=0.104 ms 8980 bytes from 10.179.40.35: icmp_seq=3 ttl=64 time=0.113 ms 8980 bytes from 10.179.40.35: icmp_seq=4 ttl=64 time=0.085 ms --- 10.179.40.35 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3075ms rtt min/avg/max/mdev = 0.074/0.094/0.113/0.015 ms root@cephosd02:~# ping -M do -s 8972 -c 4 10.179.40.36 PING 10.179.40.36 (10.179.40.36) 8972(9000) bytes of data. 8980 bytes from 10.179.40.36: icmp_seq=1 ttl=64 time=0.093 ms 8980 bytes from 10.179.40.36: icmp_seq=2 ttl=64 time=0.144 ms 8980 bytes from 10.179.40.36: icmp_seq=3 ttl=64 time=0.109 ms 8980 bytes from 10.179.40.36: icmp_seq=4 ttl=64 time=0.110 ms --- 10.179.40.36 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3069ms rtt min/avg/max/mdev = 0.093/0.114/0.144/0.018 ms root@cephosd02:~# ping -M do -s 8972 -c 4 10.179.40.37 PING 10.179.40.37 (10.179.40.37) 8972(9000) bytes of data. 8980 bytes from 10.179.40.37: icmp_seq=1 ttl=64 time=0.150 ms 8980 bytes from 10.179.40.37: icmp_seq=2 ttl=64 time=0.141 ms 8980 bytes from 10.179.40.37: icmp_seq=3 ttl=64 time=0.120 ms 8980 bytes from 10.179.40.37: icmp_seq=4 ttl=64 time=0.139 ms --- 10.179.40.37 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3068ms rtt min/avg/max/mdev = 0.120/0.137/0.150/0.016 ms root@cephosd02:~# ping -M do -s 8972 -c 4 10.179.40.38 PING 10.179.40.38 (10.179.40.38) 8972(9000) bytes of data. 8980 bytes from 10.179.40.38: icmp_seq=1 ttl=64 time=0.118 ms 8980 bytes from 10.179.40.38: icmp_seq=2 ttl=64 time=0.152 ms 8980 bytes from 10.179.40.38: icmp_seq=3 ttl=64 time=0.100 ms 8980 bytes from 10.179.40.38: icmp_seq=4 ttl=64 time=0.091 ms --- 10.179.40.38 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3071ms rtt min/avg/max/mdev = 0.091/0.115/0.152/0.024 ms root@cephosd02:~# ping -M do -s 8972 -c 4 10.179.40.39 PING 10.179.40.39 (10.179.40.39) 8972(9000) bytes of data. 8980 bytes from 10.179.40.39: icmp_seq=1 ttl=64 time=0.116 ms 8980 bytes from 10.179.40.39: icmp_seq=2 ttl=64 time=0.095 ms 8980 bytes from 10.179.40.39: icmp_seq=3 ttl=64 time=0.143 ms 8980 bytes from 10.179.40.39: icmp_seq=4 ttl=64 time=0.134 ms --- 10.179.40.39 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3061ms rtt min/avg/max/mdev = 0.095/0.122/0.143/0.018 ms root@cephosd02:~# root@cephosd03:~# ping -M do -s 8972 -c 4 10.179.40.33 PING 10.179.40.33 (10.179.40.33) 8972(9000) bytes of data. 8980 bytes from 10.179.40.33: icmp_seq=1 ttl=64 time=0.174 ms 8980 bytes from 10.179.40.33: icmp_seq=2 ttl=64 time=0.150 ms 8980 bytes from 10.179.40.33: icmp_seq=3 ttl=64 time=0.161 ms 8980 bytes from 10.179.40.33: icmp_seq=4 ttl=64 time=0.082 ms --- 10.179.40.33 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3062ms rtt min/avg/max/mdev = 0.082/0.141/0.174/0.038 ms root@cephosd03:~# ping -M do -s 8972 -c 4 10.179.40.34 PING 10.179.40.34 (10.179.40.34) 8972(9000) bytes of data. 8980 bytes from 10.179.40.34: icmp_seq=1 ttl=64 time=0.144 ms 8980 bytes from 10.179.40.34: icmp_seq=2 ttl=64 time=0.111 ms 8980 bytes from 10.179.40.34: icmp_seq=3 ttl=64 time=0.101 ms 8980 bytes from 10.179.40.34: icmp_seq=4 ttl=64 time=0.076 ms --- 10.179.40.34 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3055ms rtt min/avg/max/mdev = 0.076/0.108/0.144/0.024 ms root@cephosd03:~# ping -M do -s 8972 -c 4 10.179.40.35 PING 10.179.40.35 (10.179.40.35) 8972(9000) bytes of data. 8980 bytes from 10.179.40.35: icmp_seq=1 ttl=64 time=0.020 ms 8980 bytes from 10.179.40.35: icmp_seq=2 ttl=64 time=0.048 ms 8980 bytes from 10.179.40.35: icmp_seq=3 ttl=64 time=0.028 ms 8980 bytes from 10.179.40.35: icmp_seq=4 ttl=64 time=0.029 ms --- 10.179.40.35 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3069ms rtt min/avg/max/mdev = 0.020/0.031/0.048/0.011 ms root@cephosd03:~# ping -M do -s 8972 -c 4 10.179.40.36 PING 10.179.40.36 (10.179.40.36) 8972(9000) bytes of data. 8980 bytes from 10.179.40.36: icmp_seq=1 ttl=64 time=0.140 ms 8980 bytes from 10.179.40.36: icmp_seq=2 ttl=64 time=0.149 ms 8980 bytes from 10.179.40.36: icmp_seq=3 ttl=64 time=0.073 ms 8980 bytes from 10.179.40.36: icmp_seq=4 ttl=64 time=0.138 ms --- 10.179.40.36 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3048ms rtt min/avg/max/mdev = 0.073/0.125/0.149/0.030 ms root@cephosd03:~# ping -M do -s 8972 -c 4 10.179.40.37 PING 10.179.40.37 (10.179.40.37) 8972(9000) bytes of data. 8980 bytes from 10.179.40.37: icmp_seq=1 ttl=64 time=0.109 ms 8980 bytes from 10.179.40.37: icmp_seq=2 ttl=64 time=0.178 ms 8980 bytes from 10.179.40.37: icmp_seq=3 ttl=64 time=0.140 ms 8980 bytes from 10.179.40.37: icmp_seq=4 ttl=64 time=0.118 ms --- 10.179.40.37 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3048ms rtt min/avg/max/mdev = 0.109/0.136/0.178/0.027 ms root@cephosd03:~# ping -M do -s 8972 -c 4 10.179.40.38 PING 10.179.40.38 (10.179.40.38) 8972(9000) bytes of data. 8980 bytes from 10.179.40.38: icmp_seq=1 ttl=64 time=0.128 ms 8980 bytes from 10.179.40.38: icmp_seq=2 ttl=64 time=0.134 ms 8980 bytes from 10.179.40.38: icmp_seq=3 ttl=64 time=0.116 ms 8980 bytes from 10.179.40.38: icmp_seq=4 ttl=64 time=0.070 ms --- 10.179.40.38 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3074ms rtt min/avg/max/mdev = 0.070/0.112/0.134/0.025 ms root@cephosd03:~# ping -M do -s 8972 -c 4 10.179.40.39 PING 10.179.40.39 (10.179.40.39) 8972(9000) bytes of data. 8980 bytes from 10.179.40.39: icmp_seq=1 ttl=64 time=0.066 ms 8980 bytes from 10.179.40.39: icmp_seq=2 ttl=64 time=0.196 ms 8980 bytes from 10.179.40.39: icmp_seq=3 ttl=64 time=0.156 ms 8980 bytes from 10.179.40.39: icmp_seq=4 ttl=64 time=0.127 ms --- 10.179.40.39 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3063ms rtt min/avg/max/mdev = 0.066/0.136/0.196/0.048 ms root@cephosd03:~# root@cephosd04:~# ping -M do -s 8972 -c 4 10.179.40.33 PING 10.179.40.33 (10.179.40.33) 8972(9000) bytes of data. 8980 bytes from 10.179.40.33: icmp_seq=1 ttl=64 time=0.110 ms 8980 bytes from 10.179.40.33: icmp_seq=2 ttl=64 time=0.146 ms 8980 bytes from 10.179.40.33: icmp_seq=3 ttl=64 time=0.156 ms 8980 bytes from 10.179.40.33: icmp_seq=4 ttl=64 time=0.111 ms --- 10.179.40.33 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3073ms rtt min/avg/max/mdev = 0.110/0.130/0.156/0.024 ms root@cephosd04:~# ping -M do -s 8972 -c 4 10.179.40.34 PING 10.179.40.34 (10.179.40.34) 8972(9000) bytes of data. 8980 bytes from 10.179.40.34: icmp_seq=1 ttl=64 time=0.143 ms 8980 bytes from 10.179.40.34: icmp_seq=2 ttl=64 time=0.118 ms 8980 bytes from 10.179.40.34: icmp_seq=3 ttl=64 time=0.165 ms 8980 bytes from 10.179.40.34: icmp_seq=4 ttl=64 time=0.110 ms --- 10.179.40.34 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3060ms rtt min/avg/max/mdev = 0.110/0.134/0.165/0.021 ms root@cephosd04:~# ping -M do -s 8972 -c 4 10.179.40.35 PING 10.179.40.35 (10.179.40.35) 8972(9000) bytes of data. 8980 bytes from 10.179.40.35: icmp_seq=1 ttl=64 time=0.108 ms 8980 bytes from 10.179.40.35: icmp_seq=2 ttl=64 time=0.091 ms 8980 bytes from 10.179.40.35: icmp_seq=3 ttl=64 time=0.081 ms 8980 bytes from 10.179.40.35: icmp_seq=4 ttl=64 time=0.170 ms --- 10.179.40.35 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3055ms rtt min/avg/max/mdev = 0.081/0.112/0.170/0.036 ms root@cephosd04:~# ping -M do -s 8972 -c 4 10.179.40.36 PING 10.179.40.36 (10.179.40.36) 8972(9000) bytes of data. 8980 bytes from 10.179.40.36: icmp_seq=1 ttl=64 time=0.022 ms 8980 bytes from 10.179.40.36: icmp_seq=2 ttl=64 time=0.031 ms 8980 bytes from 10.179.40.36: icmp_seq=3 ttl=64 time=0.027 ms 8980 bytes from 10.179.40.36: icmp_seq=4 ttl=64 time=0.047 ms --- 10.179.40.36 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3077ms rtt min/avg/max/mdev = 0.022/0.031/0.047/0.011 ms root@cephosd04:~# ping -M do -s 8972 -c 4 10.179.40.37 PING 10.179.40.37 (10.179.40.37) 8972(9000) bytes of data. 8980 bytes from 10.179.40.37: icmp_seq=1 ttl=64 time=0.114 ms 8980 bytes from 10.179.40.37: icmp_seq=2 ttl=64 time=0.122 ms 8980 bytes from 10.179.40.37: icmp_seq=3 ttl=64 time=0.087 ms 8980 bytes from 10.179.40.37: icmp_seq=4 ttl=64 time=0.118 ms --- 10.179.40.37 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3076ms rtt min/avg/max/mdev = 0.087/0.110/0.122/0.015 ms root@cephosd04:~# ping -M do -s 8972 -c 4 10.179.40.38 PING 10.179.40.38 (10.179.40.38) 8972(9000) bytes of data. 8980 bytes from 10.179.40.38: icmp_seq=1 ttl=64 time=0.120 ms 8980 bytes from 10.179.40.38: icmp_seq=2 ttl=64 time=0.175 ms 8980 bytes from 10.179.40.38: icmp_seq=3 ttl=64 time=0.077 ms 8980 bytes from 10.179.40.38: icmp_seq=4 ttl=64 time=0.194 ms --- 10.179.40.38 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3076ms rtt min/avg/max/mdev = 0.077/0.141/0.194/0.047 ms root@cephosd04:~# ping -M do -s 8972 -c 4 10.179.40.39 PING 10.179.40.39 (10.179.40.39) 8972(9000) bytes of data. 8980 bytes from 10.179.40.39: icmp_seq=1 ttl=64 time=0.081 ms 8980 bytes from 10.179.40.39: icmp_seq=2 ttl=64 time=0.132 ms 8980 bytes from 10.179.40.39: icmp_seq=3 ttl=64 time=0.136 ms 8980 bytes from 10.179.40.39: icmp_seq=4 ttl=64 time=0.152 ms --- 10.179.40.39 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3065ms rtt min/avg/max/mdev = 0.081/0.125/0.152/0.027 ms root@cephosd04:~# root@cephosd05:~# ping -M do -s 8972 -c 4 10.179.40.33 PING 10.179.40.33 (10.179.40.33) 8972(9000) bytes of data. 8980 bytes from 10.179.40.33: icmp_seq=1 ttl=64 time=0.101 ms 8980 bytes from 10.179.40.33: icmp_seq=2 ttl=64 time=0.096 ms 8980 bytes from 10.179.40.33: icmp_seq=3 ttl=64 time=0.109 ms 8980 bytes from 10.179.40.33: icmp_seq=4 ttl=64 time=0.155 ms --- 10.179.40.33 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3064ms rtt min/avg/max/mdev = 0.096/0.115/0.155/0.024 ms root@cephosd05:~# ping -M do -s 8972 -c 4 10.179.40.34 PING 10.179.40.34 (10.179.40.34) 8972(9000) bytes of data. 8980 bytes from 10.179.40.34: icmp_seq=1 ttl=64 time=0.089 ms 8980 bytes from 10.179.40.34: icmp_seq=2 ttl=64 time=0.157 ms 8980 bytes from 10.179.40.34: icmp_seq=3 ttl=64 time=0.166 ms 8980 bytes from 10.179.40.34: icmp_seq=4 ttl=64 time=0.111 ms --- 10.179.40.34 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3056ms rtt min/avg/max/mdev = 0.089/0.130/0.166/0.034 ms root@cephosd05:~# ping -M do -s 8972 -c 4 10.179.40.35 PING 10.179.40.35 (10.179.40.35) 8972(9000) bytes of data. 8980 bytes from 10.179.40.35: icmp_seq=1 ttl=64 time=0.144 ms 8980 bytes from 10.179.40.35: icmp_seq=2 ttl=64 time=0.088 ms 8980 bytes from 10.179.40.35: icmp_seq=3 ttl=64 time=0.123 ms 8980 bytes from 10.179.40.35: icmp_seq=4 ttl=64 time=0.084 ms --- 10.179.40.35 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3072ms rtt min/avg/max/mdev = 0.084/0.109/0.144/0.028 ms root@cephosd05:~# ping -M do -s 8972 -c 4 10.179.40.36 PING 10.179.40.36 (10.179.40.36) 8972(9000) bytes of data. 8980 bytes from 10.179.40.36: icmp_seq=1 ttl=64 time=0.148 ms 8980 bytes from 10.179.40.36: icmp_seq=2 ttl=64 time=0.086 ms 8980 bytes from 10.179.40.36: icmp_seq=3 ttl=64 time=0.152 ms 8980 bytes from 10.179.40.36: icmp_seq=4 ttl=64 time=0.112 ms --- 10.179.40.36 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3048ms rtt min/avg/max/mdev = 0.086/0.124/0.152/0.029 ms root@cephosd05:~# ping -M do -s 8972 -c 4 10.179.40.37 PING 10.179.40.37 (10.179.40.37) 8972(9000) bytes of data. 8980 bytes from 10.179.40.37: icmp_seq=1 ttl=64 time=0.017 ms 8980 bytes from 10.179.40.37: icmp_seq=2 ttl=64 time=0.041 ms 8980 bytes from 10.179.40.37: icmp_seq=3 ttl=64 time=0.042 ms 8980 bytes from 10.179.40.37: icmp_seq=4 ttl=64 time=0.041 ms --- 10.179.40.37 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3057ms rtt min/avg/max/mdev = 0.017/0.035/0.042/0.011 ms root@cephosd05:~# ping -M do -s 8972 -c 4 10.179.40.38 PING 10.179.40.38 (10.179.40.38) 8972(9000) bytes of data. 8980 bytes from 10.179.40.38: icmp_seq=1 ttl=64 time=0.137 ms 8980 bytes from 10.179.40.38: icmp_seq=2 ttl=64 time=0.126 ms 8980 bytes from 10.179.40.38: icmp_seq=3 ttl=64 time=0.148 ms 8980 bytes from 10.179.40.38: icmp_seq=4 ttl=64 time=0.075 ms --- 10.179.40.38 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3048ms rtt min/avg/max/mdev = 0.075/0.121/0.148/0.030 ms root@cephosd05:~# ping -M do -s 8972 -c 4 10.179.40.39 PING 10.179.40.39 (10.179.40.39) 8972(9000) bytes of data. 8980 bytes from 10.179.40.39: icmp_seq=1 ttl=64 time=0.081 ms 8980 bytes from 10.179.40.39: icmp_seq=2 ttl=64 time=0.132 ms 8980 bytes from 10.179.40.39: icmp_seq=3 ttl=64 time=0.130 ms 8980 bytes from 10.179.40.39: icmp_seq=4 ttl=64 time=0.100 ms --- 10.179.40.39 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3073ms rtt min/avg/max/mdev = 0.081/0.110/0.132/0.024 ms root@cephosd05:~# root@cephosd06:~# ping -M do -s 8972 -c 4 10.179.40.33 PING 10.179.40.33 (10.179.40.33) 8972(9000) bytes of data. 8980 bytes from 10.179.40.33: icmp_seq=1 ttl=64 time=0.065 ms 8980 bytes from 10.179.40.33: icmp_seq=2 ttl=64 time=0.140 ms 8980 bytes from 10.179.40.33: icmp_seq=3 ttl=64 time=0.115 ms 8980 bytes from 10.179.40.33: icmp_seq=4 ttl=64 time=0.152 ms --- 10.179.40.33 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3073ms rtt min/avg/max/mdev = 0.065/0.118/0.152/0.033 ms root@cephosd06:~# ping -M do -s 8972 -c 4 10.179.40.34 PING 10.179.40.34 (10.179.40.34) 8972(9000) bytes of data. 8980 bytes from 10.179.40.34: icmp_seq=1 ttl=64 time=0.152 ms 8980 bytes from 10.179.40.34: icmp_seq=2 ttl=64 time=0.152 ms 8980 bytes from 10.179.40.34: icmp_seq=3 ttl=64 time=0.120 ms 8980 bytes from 10.179.40.34: icmp_seq=4 ttl=64 time=0.110 ms --- 10.179.40.34 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3073ms rtt min/avg/max/mdev = 0.110/0.133/0.152/0.022 ms root@cephosd06:~# ping -M do -s 8972 -c 4 10.179.40.35 PING 10.179.40.35 (10.179.40.35) 8972(9000) bytes of data. 8980 bytes from 10.179.40.35: icmp_seq=1 ttl=64 time=0.173 ms 8980 bytes from 10.179.40.35: icmp_seq=2 ttl=64 time=0.110 ms 8980 bytes from 10.179.40.35: icmp_seq=3 ttl=64 time=0.094 ms 8980 bytes from 10.179.40.35: icmp_seq=4 ttl=64 time=0.147 ms --- 10.179.40.35 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3066ms rtt min/avg/max/mdev = 0.094/0.131/0.173/0.030 ms root@cephosd06:~# ping -M do -s 8972 -c 4 10.179.40.36 PING 10.179.40.36 (10.179.40.36) 8972(9000) bytes of data. 8980 bytes from 10.179.40.36: icmp_seq=1 ttl=64 time=0.142 ms 8980 bytes from 10.179.40.36: icmp_seq=2 ttl=64 time=0.137 ms 8980 bytes from 10.179.40.36: icmp_seq=3 ttl=64 time=0.147 ms 8980 bytes from 10.179.40.36: icmp_seq=4 ttl=64 time=0.127 ms --- 10.179.40.36 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3065ms rtt min/avg/max/mdev = 0.127/0.138/0.147/0.011 ms root@cephosd06:~# ping -M do -s 8972 -c 4 10.179.40.37 PING 10.179.40.37 (10.179.40.37) 8972(9000) bytes of data. 8980 bytes from 10.179.40.37: icmp_seq=1 ttl=64 time=0.109 ms 8980 bytes from 10.179.40.37: icmp_seq=2 ttl=64 time=0.123 ms 8980 bytes from 10.179.40.37: icmp_seq=3 ttl=64 time=0.104 ms 8980 bytes from 10.179.40.37: icmp_seq=4 ttl=64 time=0.125 ms --- 10.179.40.37 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3065ms rtt min/avg/max/mdev = 0.104/0.115/0.125/0.011 ms root@cephosd06:~# ping -M do -s 8972 -c 4 10.179.40.38 PING 10.179.40.38 (10.179.40.38) 8972(9000) bytes of data. 8980 bytes from 10.179.40.38: icmp_seq=1 ttl=64 time=0.013 ms 8980 bytes from 10.179.40.38: icmp_seq=2 ttl=64 time=0.037 ms 8980 bytes from 10.179.40.38: icmp_seq=3 ttl=64 time=0.020 ms 8980 bytes from 10.179.40.38: icmp_seq=4 ttl=64 time=0.037 ms --- 10.179.40.38 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3065ms rtt min/avg/max/mdev = 0.013/0.026/0.037/0.012 ms root@cephosd06:~# ping -M do -s 8972 -c 4 10.179.40.39 PING 10.179.40.39 (10.179.40.39) 8972(9000) bytes of data. 8980 bytes from 10.179.40.39: icmp_seq=1 ttl=64 time=0.124 ms 8980 bytes from 10.179.40.39: icmp_seq=2 ttl=64 time=0.143 ms 8980 bytes from 10.179.40.39: icmp_seq=3 ttl=64 time=0.142 ms 8980 bytes from 10.179.40.39: icmp_seq=4 ttl=64 time=0.092 ms --- 10.179.40.39 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3074ms rtt min/avg/max/mdev = 0.092/0.125/0.143/0.022 ms root@cephosd06:~# root@cephosd07:~# ping -M do -s 8972 -c 4 10.179.40.33 PING 10.179.40.33 (10.179.40.33) 8972(9000) bytes of data. 8980 bytes from 10.179.40.33: icmp_seq=1 ttl=64 time=0.148 ms 8980 bytes from 10.179.40.33: icmp_seq=2 ttl=64 time=0.138 ms 8980 bytes from 10.179.40.33: icmp_seq=3 ttl=64 time=0.158 ms 8980 bytes from 10.179.40.33: icmp_seq=4 ttl=64 time=0.152 ms --- 10.179.40.33 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3056ms rtt min/avg/max/mdev = 0.138/0.149/0.158/0.007 ms root@cephosd07:~# ping -M do -s 8972 -c 4 10.179.40.34 PING 10.179.40.34 (10.179.40.34) 8972(9000) bytes of data. 8980 bytes from 10.179.40.34: icmp_seq=1 ttl=64 time=0.133 ms 8980 bytes from 10.179.40.34: icmp_seq=2 ttl=64 time=0.128 ms 8980 bytes from 10.179.40.34: icmp_seq=3 ttl=64 time=0.083 ms 8980 bytes from 10.179.40.34: icmp_seq=4 ttl=64 time=0.126 ms --- 10.179.40.34 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3048ms rtt min/avg/max/mdev = 0.083/0.117/0.133/0.022 ms root@cephosd07:~# ping -M do -s 8972 -c 4 10.179.40.35 PING 10.179.40.35 (10.179.40.35) 8972(9000) bytes of data. 8980 bytes from 10.179.40.35: icmp_seq=1 ttl=64 time=0.166 ms 8980 bytes from 10.179.40.35: icmp_seq=2 ttl=64 time=0.089 ms 8980 bytes from 10.179.40.35: icmp_seq=3 ttl=64 time=0.134 ms 8980 bytes from 10.179.40.35: icmp_seq=4 ttl=64 time=0.103 ms --- 10.179.40.35 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3056ms rtt min/avg/max/mdev = 0.089/0.123/0.166/0.029 ms root@cephosd07:~# ping -M do -s 8972 -c 4 10.179.40.36 PING 10.179.40.36 (10.179.40.36) 8972(9000) bytes of data. 8980 bytes from 10.179.40.36: icmp_seq=1 ttl=64 time=0.135 ms 8980 bytes from 10.179.40.36: icmp_seq=2 ttl=64 time=0.125 ms 8980 bytes from 10.179.40.36: icmp_seq=3 ttl=64 time=0.136 ms 8980 bytes from 10.179.40.36: icmp_seq=4 ttl=64 time=0.139 ms --- 10.179.40.36 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3056ms rtt min/avg/max/mdev = 0.125/0.133/0.139/0.015 ms root@cephosd07:~# ping -M do -s 8972 -c 4 10.179.40.37 PING 10.179.40.37 (10.179.40.37) 8972(9000) bytes of data. 8980 bytes from 10.179.40.37: icmp_seq=1 ttl=64 time=0.148 ms 8980 bytes from 10.179.40.37: icmp_seq=2 ttl=64 time=0.115 ms 8980 bytes from 10.179.40.37: icmp_seq=3 ttl=64 time=0.141 ms 8980 bytes from 10.179.40.37: icmp_seq=4 ttl=64 time=0.118 ms --- 10.179.40.37 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3056ms rtt min/avg/max/mdev = 0.115/0.130/0.148/0.018 ms root@cephosd07:~# ping -M do -s 8972 -c 4 10.179.40.38 PING 10.179.40.38 (10.179.40.38) 8972(9000) bytes of data. 8980 bytes from 10.179.40.38: icmp_seq=1 ttl=64 time=0.130 ms 8980 bytes from 10.179.40.38: icmp_seq=2 ttl=64 time=0.144 ms 8980 bytes from 10.179.40.38: icmp_seq=3 ttl=64 time=0.106 ms 8980 bytes from 10.179.40.38: icmp_seq=4 ttl=64 time=0.143 ms --- 10.179.40.38 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3048ms rtt min/avg/max/mdev = 0.106/0.130/0.144/0.020 ms root@cephosd07:~# ping -M do -s 8972 -c 4 10.179.40.39 PING 10.179.40.39 (10.179.40.39) 8972(9000) bytes of data. 8980 bytes from 10.179.40.39: icmp_seq=1 ttl=64 time=0.017 ms 8980 bytes from 10.179.40.39: icmp_seq=2 ttl=64 time=0.046 ms 8980 bytes from 10.179.40.39: icmp_seq=3 ttl=64 time=0.037 ms 8980 bytes from 10.179.40.39: icmp_seq=4 ttl=64 time=0.038 ms --- 10.179.40.39 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3064ms rtt min/avg/max/mdev = 0.017/0.034/0.046/0.012 ms root@cephosd07:~#