Re-uploading existing s3 objects slower than initial upload
Hi all, I am investigating a performance issue related to uploading s3 objects and would appreciate any insights. I have a bucket with ~200 million objects. I noticed that re-uploading files that already exist in the bucket appears to be slower than uploading them for the first time. To verify this behavior, I created a small test case with a new empty bucket: - 500 files with random sizes between 5 KB and 50 KB. - First time upload took 8 seconds: s3cmd put files/* s3://my-bucket/ - Second upload (same files, no changes) took 10 seconds I had expected the second upload to be faster or at least the same. Is this expected behavior? Any guidance on this would be appreciated. Thanks! Sinan
Hi Sinan, Do you have versioning enabled on the bucket? It may explain that.. ----------------------------------------------------------------- Roberto VALVERDE CAMESELLE IT Storage And Data Management CERN, European Organization for Nuclear Research Esplanade des Particules 1, Geneve (Switzerland) ________________________________ From: Sinan Polat <sinan86polat@gmail.com> Sent: Friday, June 27, 2025 3:59 PM To: ceph-users@ceph.io <ceph-users@ceph.io> Subject: [ceph-users] Re-uploading existing s3 objects slower than initial upload Hi all, I am investigating a performance issue related to uploading s3 objects and would appreciate any insights. I have a bucket with ~200 million objects. I noticed that re-uploading files that already exist in the bucket appears to be slower than uploading them for the first time. To verify this behavior, I created a small test case with a new empty bucket: - 500 files with random sizes between 5 KB and 50 KB. - First time upload took 8 seconds: s3cmd put files/* s3://my-bucket/ - Second upload (same files, no changes) took 10 seconds I had expected the second upload to be faster or at least the same. Is this expected behavior? Any guidance on this would be appreciated. Thanks! Sinan _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (2)
-
Roberto Valverde Cameselle
-
Sinan Polat