Hi,
(un)fortunately I can't test it because I managed to repair the pg.
snaptrim and snaptrim_wait have been a part of this particular pg's
status. As I was trying to look deeper into the case I had a watch on
ceph health detail and noticed that snaptrim/snaptrim_wait was suddenly
not a part of the status anymore.
So I gave it another try with ceph pg repair 18.19a and suddenly the
pg's status changed to active+clean+inconsistent+repair. It repaired
successfully.
Is snaptrim somehow blocking repair instructions? I would have thought
that repair instructions will be queued up until they can be performed
but it does not seem to work as I expected it to.
Anyway I'll keep your script in mind and give it a shot if it happens
again. Thank you :)
Daniel
Am 25.05.20 um 17:40 schrieb Dan van der Ster:
> Hi,
>
> Does this help?
>
>
https://github.com/cernceph/ceph-scripts/blob/master/tools/scrubbing/autore…
>
> Cheers, Dan
>
> On Mon, May 25, 2020 at 5:18 PM Daniel Aberger - Profihost AG
> <d.aberger(a)profihost.ag> wrote:
>>
>> Hello,
>>
>> we are currently experiencing problems with ceph pg repair not working
>> on Ceph Nautilus 14.2.8.
>>
>> ceph health detail is showing us an inconsistent pg:
>>
>> [aaaaax-yyyy ~]# ceph health detail
>> HEALTH_ERR 1 scrub errors; Possible data damage: 1 pg inconsistent
>> OSD_SCRUB_ERRORS 1 scrub errors
>> PG_DAMAGED Possible data damage: 1 pg inconsistent
>> pg 18.19a is active+clean+inconsistent+snaptrim_wait, acting
>> [21,15,39,18,0,9]
>>
>> when we try to repair it, nothing happens.
>>
>> [aaaaax-yyyy ~]# ceph pg repair 18.19a
>> instructing pg 18.19as0 on osd.21 to repair
>>
>> There are no new entries in OSD 21's log file.
>>
>> We have no trouble repairing pgs in our other clusters so I assume it
>> might have to be something related to this cluster using Erasure
>> Codings. But this is just a wild guess.
>>
>> I found a similar problem in this mailing list -
>>
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-April/026304.html
>>
>> Unfortunately the solution of waiting more than a week until it fixes
>> itself isn't quite satisfying.
>>
>> Is there anyone who has had similar issues and knows how to repair these
>> inconsistent pgs or what is causing the delay?
>>
>>
>> --
>> Mit freundlichen Grüßen
>> Daniel Aberger
>> Ihr Profihost Team
>>
>> -------------------------------
>> Profihost AG
>> Expo Plaza 1
>> 30539 Hannover
>> Deutschland
>>
>> Tel.: +49 (511) 5151 8181 | Fax.: +49 (511) 5151 8282
>> URL:
http://www.profihost.com | E-Mail: info(a)profihost.com
>>
>> Sitz der Gesellschaft: Hannover, USt-IdNr. DE813460827
>> Registergericht: Amtsgericht Hannover, Register-Nr.: HRB 202350
>> Vorstand: Cristoph Bluhm, Sebastian Bluhm, Stefan Priebe
>> Aufsichtsrat: Prof. Dr. iur. Winfried Huck (Vorsitzender)
>> _______________________________________________
>> ceph-users mailing list -- ceph-users(a)ceph.io
>> To unsubscribe send an email to ceph-users-leave(a)ceph.io