4 Sep
2020
4 Sep
'20
4:04 p.m.
We have continued to troubleshoot this issue and have now isolated the
issue to be only with multipart uploads with partsizes (or chunk
sizes) that are 16777217 bytes or larger.
Does anyone know if there is a setting or restriction somewhere that
might be causing rgw to return error 500?
On Wed, Sep 2, 2020 at 3:15 PM shubjero <shubjero(a)gmail.com> wrote:
>
> Good day,
>
> I am having an issue with some multipart uploads to radosgw. I
> recently upgraded my cluster from Mimic to Nautilus and began having
> problems with multipart uploads from clients using the Java AWS SDK
> (specifically 1.11.219). I do NOT have issues with multipart uploads
> with other clients such as the aws2 cli. The error that surfaces is
> http error 500. Small files that are not multipart uploads work fine
> with the Java client.
>
> I enabled debugging for debug_ms = 1 and debug_rgw = 20 and then
> performed a multipart upload with the offending client and this is
> what I see (full log @
> https://object.cancercollaboratory.org:9080/swift/v1/ceph-debug-pub/filtere…
> )
>
> - just a snippet -
> 2020-09-02 18:03:43.339 7f9277474700 1 -- 172.25.13.93:0/458188962
> --> [v2:172.25.12.152:6936/12180,v1:172.25.12.152:6937/12180] --
> osd_op(unknown.0.0:6331 17.325
> 17:a4c2f681:::2020-09-02-18-default.262831163.2-score-rdpc-collab-qa:head
> [append 0~817] snapc 0=[] ondisk+write+known_if_redirected e1493993)
> v8 -- 0x55e9fefb9000 con 0x55e9f3e61f80
> 2020-09-02 18:03:43.339 7f9277474700 2 req 1714 70.432s s3:put_obj op
> status=-35
> 2020-09-02 18:03:43.339 7f9277474700 2 req 1714 70.432s s3:put_obj
> http status=500
>
> haproxy log:
> Sep 2 13:33:36 localhost haproxy[14495]: 206.108.127.16:46088
> [02/Sep/2020:13:32:06.263] radosgw_public_https_11~
> radosgw_public_https_11/storage2-r5 0/0/0/71677/89948 500 477 - - ----
> 12/12/11/1/0 0/0 {} "PUT
>
/score-rdpc-collab-qa/data/1c9e4395-677c-55c1-8437-b6467c806173?uploadId=2~zTiBm6p4sB2SyMu2cJ_dwd3ECMpELh3&partNumber=1&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200902T173202Z&X-Amz-SignedHeaders=host&X-Amz-Expires=518400&X-Amz-Credential=TO1DRTS4H14HJ388I26R%2F20200902%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=225b6f111cc342c97ad48f8a39f34af044647ac04da532ef949d770509e65f76
> HTTP/1.1"
>
> Does anyone know if there may be incompatiblities with Java AWS SDK
> v1? I see that v2 is out so one of our devs is going to run some tests
> using Java AWS SDK v2 but this may take a bit of time.
>
> Thanks all,
>
> Jared Baker
> Ontario Institute for Cancer Research