30 Jan
2021
30 Jan
'21
2 p.m.
Hi all,
We are testing our S3 Ceph endpoints and we are not satisfied with its
speed. Our results are something between around 120 - 150 MB/s depending
on small/bigger files. This is good for 1Gbps connection, but not for
10GE or more.
We've tried the most recent versions of the AWS CLI, s3cmd, s4cmd, s3fs
... programs. Of course we are using multipart upload/download which is
precondition for parallel upload/download. Also we tried multi-thread
(25 or more threads) transfer in s4cmd but still we don't get proper
results.
For proof of concept that high speed can be achieved we have written
small script in bash which uses multi-part & parallel transfer and can
saturate at least 10GE without problem.
I would like to ask you, if you know proper program and its parameters,
so we can saturate n x 10GE if needed?
We are using the latest nautilus.
S3 gateways have much more computer power and bandwidth to internet then
it is used right now.
Thank you
Regards
Michal Strnad
8 Feb
8 Feb
4:46 p.m.
Try s5cmd
k
On 30 Jan 2021, at 17:00, michal.strnad(a)cesnet.cz
wrote:
We are testing our S3 Ceph endpoints and we are not satisfied with its
speed. Our results are something between around 120 - 150 MB/s depending
on small/bigger files. This is good for 1Gbps connection, but not for
10GE or more.
We've tried the most recent versions of the AWS CLI, s3cmd, s4cmd, s3fs
... programs. Of course we are using multipart upload/download which is
precondition for parallel upload/download. Also we tried multi-thread
(25 or more threads) transfer in s4cmd but still we don't get proper
results.
For proof of concept that high speed can be achieved we have written
small script in bash which uses multi-part & parallel transfer and can
saturate at least 10GE without problem.
I would like to ask you, if you know proper program and its parameters,
so we can saturate n x 10GE if needed?
We are using the latest nautilus.
S3 gateways have much more computer power and bandwidth to internet then
it is used right now.
9 Feb
9 Feb
12:31 p.m.
Thank you! s5cmd do the trick :-).
--
Michal
On 2/8/21 5:46 PM, Konstantin Shalygin wrote:
Try s5cmd
k
On 30 Jan 2021, at 17:00,
michal.strnad(a)cesnet.cz
<mailto:michal.strnad@cesnet.cz> wrote:
We are testing our S3 Ceph endpoints and we are not satisfied with its
speed. Our results are something between around 120 - 150 MB/s depending
on small/bigger files. This is good for 1Gbps connection, but not for
10GE or more.
We've tried the most recent versions of the AWS CLI, s3cmd, s4cmd, s3fs
... programs. Of course we are using multipart upload/download which is
precondition for parallel upload/download. Also we tried multi-thread
(25 or more threads) transfer in s4cmd but still we don't get proper
results.
For proof of concept that high speed can be achieved we have written
small script in bash which uses multi-part & parallel transfer and can
saturate at least 10GE without problem.
I would like to ask you, if you know proper program and its parameters,
so we can saturate n x 10GE if needed?
We are using the latest nautilus.
S3 gateways have much more computer power and bandwidth to internet then
it is used right now.
8 Feb
8 Feb
5:25 p.m.
You'll want to review your rgw configuration on ceph. How many gateways are
you using? Are you using any sort of load balancer in front of them? Are
you using BEAST or civet?
On Mon, Feb 8, 2021 at 9:13 AM <michal.strnad(a)cesnet.cz> wrote:
Hi all,
We are testing our S3 Ceph endpoints and we are not satisfied with its
speed. Our results are something between around 120 - 150 MB/s depending
on small/bigger files. This is good for 1Gbps connection, but not for
10GE or more.
We've tried the most recent versions of the AWS CLI, s3cmd, s4cmd, s3fs
... programs. Of course we are using multipart upload/download which is
precondition for parallel upload/download. Also we tried multi-thread
(25 or more threads) transfer in s4cmd but still we don't get proper
results.
For proof of concept that high speed can be achieved we have written
small script in bash which uses multi-part & parallel transfer and can
saturate at least 10GE without problem.
I would like to ask you, if you know proper program and its parameters,
so we can saturate n x 10GE if needed?
We are using the latest nautilus.
S3 gateways have much more computer power and bandwidth to internet then
it is used right now.
Thank you
Regards
Michal Strnad
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
--
Steven Pine
*E * steven.pine(a)webair.com | *P * 516.938.4100 x
*Webair* | 501 Franklin Avenue Suite 200, Garden City NY, 11530
webair.com
[image: Facebook icon] <https://www.facebook.com/WebairInc/> [image:
Twitter icon] <https://twitter.com/WebairInc> [image: Linkedin icon]
<https://www.linkedin.com/company/webair>
NOTICE: This electronic mail message and all attachments transmitted with
it are intended solely for the use of the addressee and may contain legally
privileged proprietary and confidential information. If the reader of this
message is not the intended recipient, or if you are an employee or agent
responsible for delivering this message to the intended recipient, you are
hereby notified that any dissemination, distribution, copying, or other use
of this message or its attachments is strictly prohibited. If you have
received this message in error, please notify the sender immediately by
replying to this message and delete it from your computer.
1165
days inactive
1175
days old
3 comments
4 participants
participants (4)
-
Konstantin Shalygin -
Michal Strnad -
michal.strnad＠cesnet.cz
-
Steven Pine