Thanks Frédéric, we’ve done that in the meantime to work around issue #47866.
The error has been reproduced and there’s a PR associated with the issue:
On 23 Nov 2020, at 11:56, Frédéric Nass
<frederic.nass(a)univ-lorraine.fr> wrote:
Hi Denis,
You might want to look at rgw_gc_obj_min_wait from [1] and try increasing the default
value of 7200s (2 hours) to whatever suits your need < 2^64.
Just remind that at some point you'll have to get these objects processed by the gc.
Or manually through the API [2].
One thing that comes to mind regarding the "last night's missing object" is
maybe it was multi-part re-written and the re-write failed somehow and the object was then
enlisted by the gc. But that supposes this particular object sometimes gets re-written
which may not be the case.
Regards,
Frédéric.
[1]
https://docs.ceph.com/en/latest/radosgw/config-ref/
[2]
https://docs.ceph.com/en/latest/dev/radosgw/admin/adminops_nonimplemented/#…
Le 18/11/2020 à 11:27, Denis Krienbühl a écrit :
By the way, since there’s some probability that
this is a GC refcount issue, would it be possible and sane to somehow slow the GC down or
disable it altogether? Is that something we could implement on our end as a stop-gap
measure to prevent dataloss?
On 18 Nov 2020, at 10:46, Denis Krienbühl
<denis(a)href.ch> wrote:
I can now confirm that last night’s missing object was a multi-part file.
> On 18 Nov 2020, at 10:01, Janek Bevendorff <janek.bevendorff(a)uni-weimar.de>
wrote:
>
> Sorry, it's radosgw-admin object stat --bucket=BUCKETNAME --object=OBJECTNAME
(forgot the "object" there)
>
> On 18/11/2020 09:58, Janek Bevendorff wrote:
>>> The object, a Docker layer, that went missing has not been touched in 2
months. It worked for a while, but then suddenly went missing.
>> Was the object a multipart object? You can check by running radosgw-admin stat
--bucket=BUCKETNAME --object=OBJECTNAME. It should say something "ns":
"multipart" in the output. If it says "ns": "shadow",
it's a single-part object.
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io