[ceph-users] Re: Mapped rbd is very slow

Now to go for "apples to apples" either run fio -ioengine=libaio -name=test -bs=4k -iodepth=1 -direct=1 -fsync=1 -rw=randwrite -runtime=60 -filename=/dev/nvmeXXXXXXXXX to compare with the single-threaded RBD random write result (the test is destructive, so use a separate partition without data) ...Or run fio -ioengine=rbd -name=test -bs=4M -iodepth=32 -rw=write -runtime=60 -pool=kube -rbdname=bench to compare with your dd's linear write result. 58 single-threaded random iops for NVMes is pretty sad either way. Are you NVMe's server ones? Do they have capacitors? :) in the response to a likely question about what they are I'll just post my link here again :) https://yourcmc.ru/wiki/Ceph_performance -- Vitaliy Filippov