S3 Bucket Upload - Boto3 - Disable or Enable Checksum on bucket/rgw
Hello Using boto3 trying to upload files to s3 bucket, when I am using boto version 1.35 it works fine, when upgrading it to 1.36 doesn’t work, facing below error, python3 s3_upload.py Error testing the new bucket: An error occurred (XAmzContentSHA256Mismatch) when calling the PutObject operation: None But works with AWS s3 Bucket. “https://aws.amazon.com/blogs/aws/introducing-default-data-integrity-protections-for-new-objects-in-amazon-s3/“ Is there a way to enable/disable checksum ? Regards Dev
Hi Dev, AWS recently updated their SDKs to enable CRC32 checksums on multiple object operations by default. (boto >= 1.36.0) This is a known issue and the reef backport has merged for v18.2.5 (not released yet). https://tracker.ceph.com/issues/64465 https://github.com/ceph/ceph/pull/58435 Because quincy is EOL it won't receive the backport. Which Ceph version have you running? Cheers Stephan Am Di., 25. Feb. 2025 um 19:25 Uhr schrieb Devender Singh < devender@netskrt.io>:
Hello
Using boto3 trying to upload files to s3 bucket, when I am using boto version 1.35 it works fine, when upgrading it to 1.36 doesn’t work, facing below error,
python3 s3_upload.py Error testing the new bucket: An error occurred (XAmzContentSHA256Mismatch) when calling the PutObject operation: None
But works with AWS s3 Bucket.
“ https://aws.amazon.com/blogs/aws/introducing-default-data-integrity-protecti... “
Is there a way to enable/disable checksum ?
Regards Dev
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
hi Dev, On Tue, Feb 25, 2025 at 1:25 PM Devender Singh <devender@netskrt.io> wrote:
Hello
Using boto3 trying to upload files to s3 bucket, when I am using boto version 1.35 it works fine, when upgrading it to 1.36 doesn’t work, facing below error,
python3 s3_upload.py Error testing the new bucket: An error occurred (XAmzContentSHA256Mismatch) when calling the PutObject operation: None
we're tracking this in https://tracker.ceph.com/issues/63153. that was resolved for squid 19.2.0, and the reef backport is part of the upcoming 18.2.5 release while this doesn't implement the checksum feature, it does resolve the XAmzContentSHA256Mismatch errors and works with sdk versions trying to use checksums
But works with AWS s3 Bucket.
Is there a way to enable/disable checksum ?
https://github.com/aws/aws-cli/issues/9214 describes how to disable checksums where possible
Regards Dev
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (3)
-
Casey Bodley
-
Devender Singh
-
Stephan Hohn