Hi all, We are testing our S3 Ceph endpoints and we are not satisfied with its speed. Our results are something between around 120 - 150 MB/s depending on small/bigger files. This is good for 1Gbps connection, but not for 10GE or more. We've tried the most recent versions of the AWS CLI, s3cmd, s4cmd, s3fs ... programs. Of course we are using multipart upload/download which is precondition for parallel upload/download. Also we tried multi-thread (25 or more threads) transfer in s4cmd but still we don't get proper results. For proof of concept that high speed can be achieved we have written small script in bash which uses multi-part & parallel transfer and can saturate at least 10GE without problem. I would like to ask you, if you know proper program and its parameters, so we can saturate n x 10GE if needed? We are using the latest nautilus. S3 gateways have much more computer power and bandwidth to internet then it is used right now. Thank you Regards Michal Strnad
Try s5cmd k
On 30 Jan 2021, at 17:00, michal.strnad@cesnet.cz wrote:
We are testing our S3 Ceph endpoints and we are not satisfied with its speed. Our results are something between around 120 - 150 MB/s depending on small/bigger files. This is good for 1Gbps connection, but not for 10GE or more.
We've tried the most recent versions of the AWS CLI, s3cmd, s4cmd, s3fs ... programs. Of course we are using multipart upload/download which is precondition for parallel upload/download. Also we tried multi-thread (25 or more threads) transfer in s4cmd but still we don't get proper results.
For proof of concept that high speed can be achieved we have written small script in bash which uses multi-part & parallel transfer and can saturate at least 10GE without problem.
I would like to ask you, if you know proper program and its parameters, so we can saturate n x 10GE if needed?
We are using the latest nautilus. S3 gateways have much more computer power and bandwidth to internet then it is used right now.
Thank you! s5cmd do the trick :-). -- Michal On 2/8/21 5:46 PM, Konstantin Shalygin wrote:
Try s5cmd
k
On 30 Jan 2021, at 17:00, michal.strnad@cesnet.cz <mailto:michal.strnad@cesnet.cz> wrote:
We are testing our S3 Ceph endpoints and we are not satisfied with its speed. Our results are something between around 120 - 150 MB/s depending on small/bigger files. This is good for 1Gbps connection, but not for 10GE or more.
We've tried the most recent versions of the AWS CLI, s3cmd, s4cmd, s3fs ... programs. Of course we are using multipart upload/download which is precondition for parallel upload/download. Also we tried multi-thread (25 or more threads) transfer in s4cmd but still we don't get proper results.
For proof of concept that high speed can be achieved we have written small script in bash which uses multi-part & parallel transfer and can saturate at least 10GE without problem.
I would like to ask you, if you know proper program and its parameters, so we can saturate n x 10GE if needed?
We are using the latest nautilus. S3 gateways have much more computer power and bandwidth to internet then it is used right now.
You'll want to review your rgw configuration on ceph. How many gateways are you using? Are you using any sort of load balancer in front of them? Are you using BEAST or civet? On Mon, Feb 8, 2021 at 9:13 AM <michal.strnad@cesnet.cz> wrote:
Hi all,
We are testing our S3 Ceph endpoints and we are not satisfied with its speed. Our results are something between around 120 - 150 MB/s depending on small/bigger files. This is good for 1Gbps connection, but not for 10GE or more.
We've tried the most recent versions of the AWS CLI, s3cmd, s4cmd, s3fs ... programs. Of course we are using multipart upload/download which is precondition for parallel upload/download. Also we tried multi-thread (25 or more threads) transfer in s4cmd but still we don't get proper results.
For proof of concept that high speed can be achieved we have written small script in bash which uses multi-part & parallel transfer and can saturate at least 10GE without problem.
I would like to ask you, if you know proper program and its parameters, so we can saturate n x 10GE if needed?
We are using the latest nautilus. S3 gateways have much more computer power and bandwidth to internet then it is used right now.
Thank you
Regards Michal Strnad _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Steven Pine *E * steven.pine@webair.com | *P * 516.938.4100 x *Webair* | 501 Franklin Avenue Suite 200, Garden City NY, 11530 webair.com [image: Facebook icon] <https://www.facebook.com/WebairInc/> [image: Twitter icon] <https://twitter.com/WebairInc> [image: Linkedin icon] <https://www.linkedin.com/company/webair> NOTICE: This electronic mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged proprietary and confidential information. If the reader of this message is not the intended recipient, or if you are an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and delete it from your computer.
participants (4)
-
Konstantin Shalygin
-
Michal Strnad
-
michal.strnad＠cesnet.cz
-
Steven Pine