[ceph-users] Re: Experience with 100G Ceph in Proxmox

Sounds weird to me. Don't you have some element in the network that is just limited to 5140 and above it, it starts to fix fragmentation or so. I can remember asking the data center to enable 9000 and they never did and also were experimenting with some software defined network. I will bet that peer to peer you should have no issues when testing mtu 9000.

For example, we found that many 10g NICs don't handle full-sized jumbo frames (9000) without a performance hit. We got a huge performance improvement by finding the exact optimal MTU (5139 on a cluster I am looking at - at 5140 it falls off a cliff). Just one example.