4 May
2021
4 May
'21
3:35 p.m.
Hi there,
I think I found a bug in the radosgw-admin bucket radoslist command. I'm not 100% sure
so would like to check here first before I fill a bug report.
I have a bucket called bucket3. If I do a multipart upload and stop it halfway for example
and start a new upload with the same name and abort it, I'll get duplicate lines in
the radoslist output.
For example:
root@alpha:~# radosgw-admin bucket radoslist --bucket bucket3
root@alpha:~#
root@alpha:~# s3cmd put -P 100MB.bin s3://bucket3/multipart-obj-fail
upload: '100MB.bin' -> 's3://bucket3/multipart-obj-fail' [part 1 of 7,
15MB] [1 of 1]
15728640 of 15728640 100% in 0s 35.14 MB/s done
upload: '100MB.bin' -> 's3://bucket3/multipart-obj-fail' [part 2 of 7,
15MB] [1 of 1]
15728640 of 15728640 100% in 0s 33.59 MB/s done
upload: '100MB.bin' -> 's3://bucket3/multipart-obj-fail' [part 3 of 7,
15MB] [1 of 1]
15728640 of 15728640 100% in 0s 33.88 MB/s done
upload: '100MB.bin' -> 's3://bucket3/multipart-obj-fail' [part 4 of 7,
15MB] [1 of 1]
65536 of 15728640 0% in 0s 806.10 KB/s^CERROR:
Upload of '100MB.bin' part 4 failed. Use
/usr/bin/s3cmd abortmp s3://bucket3/multipart-obj-fail
2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX
to abort the upload, or
/usr/bin/s3cmd --upload-id 2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX put ...
to continue the
If I now run a radoslist it looks fine:
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.4
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.1_1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.1_2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.1_3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.2_1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.2_2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.2_3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.3_1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.3_2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.3_3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.4
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.4_1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.4_2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.4_3
However, if I do a second upload with the same name:
root@alpha:~# s3cmd put -P 100MB.bin s3://bucket3/multipart-obj-fail
upload: '100MB.bin' -> 's3://bucket3/multipart-obj-fail' [part 1 of 7,
15MB] [1 of 1]
15728640 of 15728640 100% in 0s 23.18 MB/s done
upload: '100MB.bin' -> 's3://bucket3/multipart-obj-fail' [part 2 of 7,
15MB] [1 of 1]
15728640 of 15728640 100% in 1s 13.10 MB/s done
upload: '100MB.bin' -> 's3://bucket3/multipart-obj-fail' [part 3 of 7,
15MB] [1 of 1]
15728640 of 15728640 100% in 0s 29.93 MB/s done
upload: '100MB.bin' -> 's3://bucket3/multipart-obj-fail' [part 4 of 7,
15MB] [1 of 1]
15728640 of 15728640 100% in 0s 39.53 MB/s done
upload: '100MB.bin' -> 's3://bucket3/multipart-obj-fail' [part 5 of 7,
15MB] [1 of 1]
^CERROR:
Upload of '100MB.bin' part 5 failed. Use
/usr/bin/s3cmd abortmp s3://bucket3/multipart-obj-fail
2~yS7Tzru_FSP6rkg4yeO28os207nDtTw
to abort the upload, or
/usr/bin/s3cmd --upload-id 2~yS7Tzru_FSP6rkg4yeO28os207nDtTw put ...
to continue the upload.
See ya!
Note that the IDs are unique, run1: 2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX run2:
2~yS7Tzru_FSP6rkg4yeO28os207nDtTw
But if we look at the radoslist output again:
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.4
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.4
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.4
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.4
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.1_1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.1_2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.1_3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.2_1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.2_2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.2_3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.3_1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.3_2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.3_3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.4_1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.4_2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.4_3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.1_1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.1_2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.1_3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.2_1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.2_2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.2_3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.3_1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.3_2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.3_3
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.4_1
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.4_2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__shadow_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.4_3
the duplicates are(output from radoslist | sort | uniq -c):
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.1
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.2
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.3
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.4
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.1
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.2
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.3
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.4
If I again do a new upload and stop it I get a new upload ID
2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX but again with duplicate entries(output from radoslist |
sort | uniq -c):
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.1
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.2
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.3
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~0GMvfYOGO5yhFppCWjYrUqBAOqQscoX.4
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~iNQha2hAznnKdSLukUAbJT1-4nXoWUy.1
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~iNQha2hAznnKdSLukUAbJT1-4nXoWUy.2
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~iNQha2hAznnKdSLukUAbJT1-4nXoWUy.3
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~iNQha2hAznnKdSLukUAbJT1-4nXoWUy.4
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.1
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.2
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.3
2
646e346c-2355-49df-973f-d8ac2c6349f9.74148.1__multipart_multipart-obj-fail.2~yS7Tzru_FSP6rkg4yeO28os207nDtTw.4
I'm running Ceph Octopus deployed by cephadm:
root@alpha:~# ceph versions
{
"overall": {
"ceph version 15.2.11 (e3523634d9c2227df9af89a4eac33d16738c49cb) octopus
(stable)": 12
}
}
Is this expected behaviour or should I make a bug report for this?
Kind regards,
Rob
1096
days inactive
1096
days old
0 comments
1 participants
participants (1)
-
Rob Haverkamp