fio test on local disk(NVME) and ceph rbd.
I would suggest trying rados bench as well. This will show the basic Ceph object performance level. If you have good rados performance with a 1M object size (which is usually the case in my experience), then you can look at what is happening at the RBD or VM level. Here is an example 30-second write test: rados bench -p volumes 30 write -t 32 -b 1M On Wed, Jul 23, 2025 at 4:53 PM Devender Singh <devender@netskrt.io> wrote:
Hello all
I tried doing some fio test on local disk(NVME) and ceph rbd. Why ceph is having low IO whereas it’s also on all NVME. What to tune to reach equal amount of IO?
root@node01:~/fio-cdm# python3 fio-cdm ./ tests: 5, size: 1.0GiB, target: /root/fio-cdm 6.3GiB/64.4GiB |Name | Read(MB/s)| Write(MB/s)| |------------|------------|------------| |SEQ1M Q8 T1 | 8441.37| 3588.71| |SEQ1M Q1 T1 | 3074.86| 1172.46| |RND4K Q32T16| 723.65| 733.76| |. IOPS | 176671.80| 179141.74| |. latency us| 2892.49| 2839.37| |RND4K Q1 T1 | 71.05| 57.88| |. IOPS | 17347.13| 14131.57| |. latency us| 56.13| 66.40|
### When vm moved to ceph storage... root@node01:~/fio-cdm# python3 fio-cdm ./ tests: 5, size: 1.0GiB, target: /root/fio-cdm 9.3GiB/64.4GiB |Name | Read(MB/s)| Write(MB/s)| |------------|------------|------------| |SEQ1M Q8 T1 | 1681.40| 889.89| |SEQ1M Q1 T1 | 310.74| 852.11| |RND4K Q32T16| 403.04| 274.23| |. IOPS | 98397.32| 66951.49| |. latency us| 5196.98| 7637.10| |RND4K Q1 T1 | 4.69| 45.18| |. IOPS | 1146.17| 11029.39| |. latency us| 869.47| 87.50|
Regards Dev
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io