Amudhan, Here is a trick I've used to test and evaluate Jumbo Frames without breaking production traffic: * Open a couple root ssh sessions on each of the two systems you want to test with. o In one window start a continuous ping to the other system. * On both test systems: o Look at the output of 'ip route list'. o Use 'ip route change' to explicitly set the MTU of all of your routes to 1500. o Review the output of 'ip route list' to assure you got them all. o Use 'ip link set' to set the MTU of the interface. + Size of all packets is still 1500 due to the route MTUs + Ping should still be working like normal. * Using 'ip addr add' to create a test logical IP on each of the two systems - use something unrelated to your production IP addresses, like 2.2.2.2/24 and 2.2.2.3/24. o Go back to the procedure above and find any new routes associated with the addresses you just added. + Use 'ip addr change' to set the MTU of these new routes to 8192. o Default route not needed. * Using the test interfaces try 'ping -s 5000' or anything above 1500. o If this works, you have everything, including your network switches, set up correctly, and you're sending packets larger that 1500. o Yet your SSH sessions and all production traffic are still running at MTU 1500. o Since the route MTUs for these spare logical IPs is 8192, you should be able probe up to 'ping -s 8150' or higher, but I think you need to leave space for the IP and ICMP headers. The nice thing about this is that you haven't disrupted your production traffic at this point, and in the worst case you can undo all of these changes by rebooting the two test nodes. If you want to move your production traffic to Jumbo Frames, change the appropriate routes to MTU 8192 on all systems. Then test test test. Lastly, change your network configuration on any effected nodes so the increased MTU will be reinstated after every reboot. -Dave Dave Hall Binghamton University On 5/24/2020 9:53 AM, Suresh Rama wrote:
Ping with 9000 MTU won't get response as I said and it should be 8972. Glad it is working but you should know what happened to avoid this issue later.
On Sun, May 24, 2020, 3:04 AM Amudhan P <amudhan83@gmail.com> wrote:
No, ping with MTU size 9000 didn't work.
On Sun, May 24, 2020 at 12:26 PM Khodayar Doustar <doustar@rayanexon.ir> wrote:
Does your ping work or not?
On Sun, May 24, 2020 at 6:53 AM Amudhan P <amudhan83@gmail.com> wrote:
Yes, I have set setting on the switch side also.
On Sat 23 May, 2020, 6:47 PM Khodayar Doustar, <doustar@rayanexon.ir> wrote:
Problem should be with network. When you change MTU it should be changed all over the network, any single hup on your network should speak and accept 9000 MTU packets. you can check it on your hosts with "ifconfig" command and there is also equivalent commands for other network/security devices.
If you have just one node which it not correctly configured for MTU 9000 it wouldn't work.
On Sat, May 23, 2020 at 2:30 PM sinan@turka.nl <sinan@turka.nl> wrote:
Can the servers/nodes ping eachother using large packet sizes? I guess not.
Sinan Polat
> Op 23 mei 2020 om 14:21 heeft Amudhan P <amudhan83@gmail.com> het volgende geschreven: > In OSD logs "heartbeat_check: no reply from OSD" > >> On Sat, May 23, 2020 at 5:44 PM Amudhan P <amudhan83@gmail.com> wrote: >> Hi, >> >> I have set Network switch with MTU size 9000 and also in my netplan >> configuration. >> >> What else needs to be checked? >> >> >>> On Sat, May 23, 2020 at 3:39 PM Wido den Hollander <wido@42on.com wrote: >>> >>> >>>> On 5/23/20 12:02 PM, Amudhan P wrote: >>>> Hi, >>>> >>>> I am using ceph Nautilus in Ubuntu 18.04 working fine wit MTU size 1500 >>>> (default) recently i tried to update MTU size to 9000. >>>> After setting Jumbo frame running ceph -s is timing out. >>> Ceph can run just fine with an MTU of 9000. But there is probably >>> something else wrong on the network which is causing this. >>> >>> Check the Jumbo Frames settings on all the switches as well to make sure >>> they forward all the packets. >>> >>> This is definitely not a Ceph issue. >>> >>> Wido >>> >>>> regards >>>> Amudhan P >>>> _______________________________________________ >>>> ceph-users mailing list -- ceph-users@ceph.io >>>> To unsubscribe send an email to ceph-users-leave@ceph.io >>>> >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users@ceph.io >>> To unsubscribe send an email to ceph-users-leave@ceph.io >>> > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-leave@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io