I added Matt as CC as he is the one who implemented the new checksum capabilities I reference below ... On 16.01.25 8:26 AM, Szabo, Istvan (Agoda) wrote:
Amazon released a new version of their cli today https://github.com/aws/aws-cli/tags and seems to break our stuffs with the following error when PUT object happens:
bash-4.2# /usr/local/bin/aws --endpoint=https://endpoint --no-verify-ssl s3 cp online.txt s3://bucket/ upload failed: ./online.txt to s3://bucket/online.txt An error occurred (MissingContentLength) when calling the PutObject operation: Unknown
Wonder is there anything on ceph side that I can do to eliminate this? The amazon workaround response_checksum_validation = when_required did not work. https://github.com/aws/aws-cli/issues/9214
I had issues using their s3-mointpoint (https://github.com/awslabs/mountpoint-s3). This was caused by missing checksum features in RGW, see https://tracker.ceph.com/issues/63153 -> https://github.com/ceph/ceph/pull/54856 While merged, I suppose these have not made it into a release yet. Client using AWS Go SDK v2 had issues because that utilized the newer features. See e.g. terraform https://github.com/hashicorp/terraform/issues/34086 Looking at the changelog of the recent aws-cli release ... * https://github.com/aws/aws-cli/blob/2.23.0/CHANGELOG.rst * https://github.com/aws/aws-cli/blob/1.37.0/CHANGELOG.rst it seems there is a lot more checksumming now being done. Whatever new feature / checksum you find to cause this incompatibility should go as test case into the s3-tests (https://github.com/ceph/s3-tests) to ensure this feature is tested from then on. Regards Christian