Good evening everyone! Guys, what to expect latency for RBD images in a cluster with only HDD (36 HDDs)? Sometimes I see that the write latency is around 2-5 ms in some images even with very low IOPS and bandwidth while the read latency is around 0.2-0.7 ms. For a cluster with only HDD is this latency expected? Is there any parameter I can study to improve? What do you recommend for tuning in the OS? The latency between machines is always around 0.1ms, all connected via fiber optics. Thanks in advance!
Murilo， The latency of HDD is about 10ms+. and the IO stack in Ceph may spends ~3ms+, so the test result is still in doubt. I guess the rbd test used the ram cache. You can paste more fio outputs here. On 2023/3/17 07:16, Murilo Morais wrote:
Good evening everyone!
Guys, what to expect latency for RBD images in a cluster with only HDD (36 HDDs)?
Sometimes I see that the write latency is around 2-5 ms in some images even with very low IOPS and bandwidth while the read latency is around 0.2-0.7 ms. For a cluster with only HDD is this latency expected? Is there any parameter I can study to improve? What do you recommend for tuning in the OS?
The latency between machines is always around 0.1ms, all connected via fiber optics.
Thanks in advance! _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
If you need to improve performance, moving the journal and RocksDB to an NVMe device can double IOPS and reduce latency. On Thu., Mar. 16, 2023, 7:16 p.m. Murilo Morais, <murilo@evocorp.com.br> wrote:
Good evening everyone!
Guys, what to expect latency for RBD images in a cluster with only HDD (36 HDDs)?
Sometimes I see that the write latency is around 2-5 ms in some images even with very low IOPS and bandwidth while the read latency is around 0.2-0.7 ms. For a cluster with only HDD is this latency expected? Is there any parameter I can study to improve? What do you recommend for tuning in the OS?
The latency between machines is always around 0.1ms, all connected via fiber optics.
Thanks in advance! _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (3)
-
Murilo Morais
-
Nathan Fish
-
Norman