On 23/09/2019 11:49, Marc Roos wrote:

And I was just about to upgrade. :) How is this even possible with this change[0] where 50-100% iops lost?

[0] https://github.com/ceph/ceph/pull/28573

-----Original Message----- From: 徐蕴 [mailto:yunxu@me.com] Sent: maandag 23 september 2019 8:28 To: ceph-users@ceph.io Subject: [ceph-users] rados bench performance in nautilus

Hi ceph experts,

I deployed Nautilus (v14.2.4) and Luminous (v12.2.11) on the same hardware, and made a rough performance comparison. The result seems Luminous is much better, which is unexpected.

My setup: 3 servers, each has 3 HDD OSDs, 1 SSD as DB, two separated 1G network for cluster and public. Pool test has 32 pg and pop numbers, replicated size is 3. Using "rados -p bench 80 write” to measure write performance. The result: Luminous: Average IOPS 36 Nautilus: Average IOPS 28

Is the difference considered valid for Nautilus?

Br, Xu Yun _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

The intent of this change is to increase iops on bluestore, it was implemented in 14.2.4 but it is a general bluestore issue not specific to Nautilus. /Maged

On 24/09/2019 10:25, Marc Roos wrote:

...

The intent of this change is to increase iops on bluestore, it was implemented in 14.2.4 but it is a general bluestore issue not specific to Nautilus.

I am confused. Is it not like this that an increase in iops on bluestore = increase in overall iops? It is specific to Nautilus, because afaik it is not backported to Luminous.

Hi Marc, As per my earlier post, i doubt the higher values measured in the test are correct for a 1Gbps link /Maged

Hi Maged, I also noticed that the write bandwidth is about 114MBps, which cloud be limited by 1G network. But why did the same hardware get better performance mark when running Luminous or even Jewel? I ran the test at one server in this cluster, so I assume that about 30% write requests (I have 3 nodes) will be handled by the node itself, which has much higher bandwidth for internal network loop. So it write performance could be higher than 114M. Maybe it’s related to the implementation rados bench itself? Br, Xu Yun

在 2019年9月23日，下午6:30，Maged Mokhtar <mmokhtar@petasan.org> 写道：

On 23/09/2019 08:27, 徐蕴 wrote:

...

Hi ceph experts,

I deployed Nautilus (v14.2.4) and Luminous (v12.2.11) on the same hardware, and made a rough performance comparison. The result seems Luminous is much better, which is unexpected.

My setup: 3 servers, each has 3 HDD OSDs, 1 SSD as DB, two separated 1G network for cluster and public. Pool test has 32 pg and pop numbers, replicated size is 3. Using "rados -p bench 80 write” to measure write performance. The result: Luminous: Average IOPS 36 Nautilus: Average IOPS 28

Is the difference considered valid for Nautilus?

Br, Xu Yun _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

If you ran "rados -p bench 80 write”without specifying the block size -b option, then you will be using default 4MB block sizes, at such sizes you should be looking at Throughput MB/s rather than iops, the 28 iops x 4M will already saturate your 1G network.

/Maged