Thanks, we are currently scanning our object storage. It looks like we can detect the
missing objects that return “No Such Key” looking at all “__multipart_” objects returned
by radosgw-admin bucket radoslist, and checking if they exist using rados stat. We are
currently not looking at shadow objects as our approach already yields more instances of
this problem.
On 19 Nov 2020, at 09:09, Janek Bevendorff
<janek.bevendorff(a)uni-weimar.de> wrote:
- The head object had a size of 0.
- There was an object with a ’shadow’ in its name, belonging to that path.
That is
normal. What is not normal is if there are NO shadow objects.
On 18/11/2020 10:06, Denis Krienbühl wrote:
> It looks like a single-part object. But we did replace that object last night from
backup, so I can’t know for sure if the lost one was like that.
>
> Another engineer that looked at the Rados objects last night did notice two things:
>
- The head object had a size of 0.
- There was an object with a ’shadow’ in its name, belonging to that path.
>
> I’m not knowledgable about Rados, so I’m not sure this is helpful.
>
>> On 18 Nov 2020, at 10:01, Janek Bevendorff <janek.bevendorff(a)uni-weimar.de>
wrote:
>>
>> Sorry, it's radosgw-admin object stat --bucket=BUCKETNAME --object=OBJECTNAME
(forgot the "object" there)
>>
>> On 18/11/2020 09:58, Janek Bevendorff wrote:
>>>> The object, a Docker layer, that went missing has not been touched in 2
months. It worked for a while, but then suddenly went missing.
>>> Was the object a multipart object? You can check by running radosgw-admin
stat --bucket=BUCKETNAME --object=OBJECTNAME. It should say something "ns":
"multipart" in the output. If it says "ns": "shadow",
it's a single-part object.
>> _______________________________________________
>> ceph-users mailing list -- ceph-users(a)ceph.io
>> To unsubscribe send an email to ceph-users-leave(a)ceph.io