23 Sep
2019
23 Sep
'19
7:27 a.m.
Hi ceph experts,
I deployed Nautilus (v14.2.4) and Luminous (v12.2.11) on the same hardware, and made a
rough performance comparison. The result seems Luminous is much better, which is
unexpected.
My setup:
3 servers, each has 3 HDD OSDs, 1 SSD as DB, two separated 1G network for cluster and
public.
Pool test has 32 pg and pop numbers, replicated size is 3.
Using "rados -p bench 80 write” to measure write performance.
The result:
Luminous: Average IOPS 36
Nautilus: Average IOPS 28
Is the difference considered valid for Nautilus?
Br,
Xu Yun
23 Sep
23 Sep
10:49 a.m.
And I was just about to upgrade. :) How is this even possible with this
change[0] where 50-100% iops lost?
[0]
https://github.com/ceph/ceph/pull/28573
-----Original Message-----
From: 徐蕴 [mailto:yunxu@me.com]
Sent: maandag 23 september 2019 8:28
To: ceph-users(a)ceph.io
Subject: [ceph-users] rados bench performance in nautilus
Hi ceph experts,
I deployed Nautilus (v14.2.4) and Luminous (v12.2.11) on the same
hardware, and made a rough performance comparison. The result seems
Luminous is much better, which is unexpected.
My setup:
3 servers, each has 3 HDD OSDs, 1 SSD as DB, two separated 1G network
for cluster and public.
Pool test has 32 pg and pop numbers, replicated size is 3.
Using "rados -p bench 80 write” to measure write performance.
The result:
Luminous: Average IOPS 36
Nautilus: Average IOPS 28
Is the difference considered valid for Nautilus?
Br,
Xu Yun
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an
email to ceph-users-leave(a)ceph.io
11:30 a.m.
On 23/09/2019 08:27, 徐蕴 wrote:
Hi ceph experts,
I deployed Nautilus (v14.2.4) and Luminous (v12.2.11) on the same hardware, and made a
rough performance comparison. The result seems Luminous is much better, which is
unexpected.
My setup:
3 servers, each has 3 HDD OSDs, 1 SSD as DB, two separated 1G network for cluster and
public.
Pool test has 32 pg and pop numbers, replicated size is 3.
Using "rados -p bench 80 write” to measure write performance.
The result:
Luminous: Average IOPS 36
Nautilus: Average IOPS 28
Is the difference considered valid for Nautilus?
Br,
Xu Yun
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
If you ran "rados -p bench 80 write”without specifying the block size -b
option, then you will be using default 4MB block sizes, at such sizes
you should be looking at Throughput MB/s rather than iops, the 28 iops x
4M will already saturate your 1G network.
/Maged
11:43 a.m.
Hi Maged,
I also noticed that the write bandwidth is about 114MBps, which cloud be limited by 1G
network. But why did the same hardware get better performance mark when running Luminous
or even Jewel? I ran the test at one server in this cluster, so I assume that about 30%
write requests (I have 3 nodes) will be handled by the node itself, which has much higher
bandwidth for internal network loop. So it write performance could be higher than 114M.
Maybe it’s related to the implementation rados bench itself?
Br,
Xu Yun
在 2019年9月23日，下午6:30，Maged Mokhtar
<mmokhtar(a)petasan.org> 写道：
On 23/09/2019 08:27, 徐蕴 wrote:
Hi ceph experts,
I deployed Nautilus (v14.2.4) and Luminous (v12.2.11) on the same hardware, and made a
rough performance comparison. The result seems Luminous is much better, which is
unexpected.
My setup:
3 servers, each has 3 HDD OSDs, 1 SSD as DB, two separated 1G network for cluster and
public.
Pool test has 32 pg and pop numbers, replicated size is 3.
Using "rados -p bench 80 write” to measure write performance.
The result:
Luminous: Average IOPS 36
Nautilus: Average IOPS 28
Is the difference considered valid for Nautilus?
Br,
Xu Yun
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
If you ran "rados -p bench 80 write”without specifying the block size -b option,
then you will be using default 4MB block sizes, at such sizes you should be looking at
Throughput MB/s rather than iops, the 28 iops x 4M will already saturate your 1G network.
/Maged
11:51 a.m.
On 23/09/2019 11:49, Marc Roos wrote:
And I was just about to upgrade. :) How is this even possible with this
change[0] where 50-100% iops lost?
[0]
https://github.com/ceph/ceph/pull/28573
-----Original Message-----
From: 徐蕴 [mailto:yunxu@me.com]
Sent: maandag 23 september 2019 8:28
To: ceph-users(a)ceph.io
Subject: [ceph-users] rados bench performance in nautilus
Hi ceph experts,
I deployed Nautilus (v14.2.4) and Luminous (v12.2.11) on the same
hardware, and made a rough performance comparison. The result seems
Luminous is much better, which is unexpected.
My setup:
3 servers, each has 3 HDD OSDs, 1 SSD as DB, two separated 1G network
for cluster and public.
Pool test has 32 pg and pop numbers, replicated size is 3.
Using "rados -p bench 80 write” to measure write performance.
The result:
Luminous: Average IOPS 36
Nautilus: Average IOPS 28
Is the difference considered valid for Nautilus?
Br,
Xu Yun
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an
email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
The intent of this change is to increase iops on bluestore, it was
implemented in 14.2.4 but it is a general bluestore issue not specific
to Nautilus. /Maged
24 Sep
24 Sep
9:25 a.m.
The intent of this change is to increase iops on
bluestore, it was
implemented in 14.2.4 but it is a
general bluestore issue not specific to Nautilus.
I am confused. Is it not like this that an increase in iops on bluestore
= increase in overall iops? It is specific to Nautilus, because afaik
it is not backported to Luminous.
12:34 p.m.
On 24/09/2019 10:25, Marc Roos wrote:
Hi Marc,
As per my earlier post, i doubt the higher values measured in the test
are correct for a 1Gbps link
/Maged
The intent of
this change is to increase iops on bluestore, it was
implemented in 14.2.4 but it
is a
general bluestore issue not specific to Nautilus.
I am confused. Is it not like this that an increase in iops on bluestore
= increase in overall iops? It is specific to Nautilus, because afaik
it is not backported to Luminous.
1698
days inactive
1699
days old
6 comments
3 participants
participants (3)
-
Maged Mokhtar -
Marc Roos -
徐蕴