# fio --ioengine=libaio --direct=1 --bs=16384 --iodepth=128 --rw=randread --norandommap --size=20G --numjobs=1 --runtime=300 --time_based --name=/dev/rbd/rbd.ssd/test /dev/rbd/rbd.ssd/test: (g=0): rw=randread, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=libaio, iodepth=128 fio-3.25 Starting 1 process Jobs: 1 (f=1): [r(1)][100.0%][r=31.3MiB/s][r=2002 IOPS][eta 00m:00s] /dev/rbd/rbd.ssd/test: (groupid=0, jobs=1): err= 0: pid=3825356: Thu Jan 30 19:49:15 2025 read: IOPS=2007, BW=31.4MiB/s (32.9MB/s)(9422MiB/300334msec) slat (usec): min=2, max=500, avg=13.50, stdev= 7.03 clat (usec): min=45, max=3395.6k, avg=63732.62, stdev=241344.46 lat (usec): min=136, max=3395.6k, avg=63746.39, stdev=241344.47 clat percentiles (usec): | 1.00th=[ 204], 5.00th=[ 243], 10.00th=[ 273], | 20.00th=[ 330], 30.00th=[ 429], 40.00th=[ 1090], | 50.00th=[ 6390], 60.00th=[ 9110], 70.00th=[ 14091], | 80.00th=[ 30278], 90.00th=[ 107480], 95.00th=[ 287310], | 99.00th=[1350566], 99.50th=[1820328], 99.90th=[2701132], | 99.95th=[2868904], 99.99th=[3204449] bw ( KiB/s): min= 2560, max=39680, per=100.00%, avg=32371.00, stdev=3270.26, samples=596 iops : min= 160, max= 2480, avg=2023.19, stdev=204.39, samples=596 lat (usec) : 50=0.01%, 250=6.09%, 500=27.41%, 750=4.50%, 1000=1.83% lat (msec) : 2=0.50%, 4=2.27%, 10=20.07%, 20=12.33%, 50=9.59% lat (msec) : 100=4.95%, 250=4.91%, 500=2.41%, 750=0.92%, 1000=0.64% lat (msec) : 2000=1.18%, >=2000=0.40% cpu : usr=1.09%, sys=4.10%, ctx=541464, majf=0, minf=524 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1% issued rwts: total=603021,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=128 Run status group 0 (all jobs): READ: bw=31.4MiB/s (32.9MB/s), 31.4MiB/s-31.4MiB/s (32.9MB/s-32.9MB/s), io=9422MiB (9880MB), run=300334-300334msec