On 23/06/2023 04:18, Work Ceph wrote:

Hello guys,

We have a Ceph cluster that runs just fine with Ceph Octopus; we use RBD for some workloads, RadosGW (via S3) for others, and iSCSI for some Windows clients.

We started noticing some unexpected performance issues with iSCSI. I mean, an SSD pool is reaching 100MB of write speed for an image, when it can reach up to 600MB+ of write speed for the same image when mounted and consumed directly via RBD.

Is that performance degradation expected? We would expect some degradation, but not as much as this one.

Can't say on ceph-iscsi since we use a kernel based rbd backstore, but generally you should change the Windows iSCSI initiator registry setting MaxTransferLength from 256KB -> 4MB (a reboot is required) This will have a large impact for large block writes performance, as the default in Windows is to chop such writes into 256 KB blocks which is too low for distributed systems where latency is higher than traditional SANs. It will also improve smaller sequential write performance that are buffered such as regular Windows file copy, as the Windows page cache will buffer those to 1MB in size.

Also, we have a question regarding the use of Intel Turbo boost. Should we disable it? Is it possible that the root cause of the slowness in the iSCSI GW is caused by the use of Intel Turbo boost feature, which reduces the clock of some cores?

I would not recommend this, best is to set the highest sustained/steady state performance speced for the cpu. Best is to set the governor to performance and disable c-states cpupower idle-set -D 0 cpupower frequency-set -g performance /Maged

Any feedback is much appreciated. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io