27 May
2020
27 May
'20
7:43 p.m.
On Wed, May 27, 2020 at 5:28 AM Daniel Aberger - Profihost AG <
d.aberger(a)profihost.ag> wrote:
Hi,
(un)fortunately I can't test it because I managed to repair the pg.
snaptrim and snaptrim_wait have been a part of this particular pg's
status. As I was trying to look deeper into the case I had a watch on
ceph health detail and noticed that snaptrim/snaptrim_wait was suddenly
not a part of the status anymore.
So I gave it another try with ceph pg repair 18.19a and suddenly the
pg's status changed to active+clean+inconsistent+repair. It repaired
successfully.
Is snaptrim somehow blocking repair instructions? I would have thought
that repair instructions will be queued up until they can be performed
but it does not seem to work as I expected it to.
Anyway I'll keep your script in mind and give it a shot if it happens
again. Thank you :)
Daniel
Am 25.05.20 um 17:40 schrieb Dan van der Ster:
We expect the repair to sometimes take all night, but it does happen
eventually, and we do not see any client issues while waiting for the
repair.
--
Alex Gorbachev
Intelligent Systems Services Inc.
Hi,
Does this help?
https://github.com/cernceph/ceph-scripts/blob/master/tools/scrubbing/autore…
Cheers, Dan
On Mon, May 25, 2020 at 5:18 PM Daniel Aberger - Profihost AG
<d.aberger(a)profihost.ag> wrote:
>
> Hello,
>
> we are currently experiencing problems with ceph pg repair not working
> on Ceph Nautilus 14.2.8.
>
> ceph health detail is showing us an inconsistent pg:
>
> [aaaaax-yyyy ~]# ceph health detail
> HEALTH_ERR 1 scrub errors; Possible data damage: 1 pg inconsistent
> OSD_SCRUB_ERRORS 1 scrub errors
> PG_DAMAGED Possible data damage: 1 pg inconsistent
> pg 18.19a is active+clean+inconsistent+snaptrim_wait, acting
> [21,15,39,18,0,9]
>
> when we try to repair it, nothing happens.
>
> [aaaaax-yyyy ~]# ceph pg repair 18.19a
> instructing pg 18.19as0 on osd.21 to repair
>
> There are no new entries in OSD 21's log file.
>
> We have no trouble repairing pgs in our other clusters so I assume it
> might have to be something related to this cluster using Erasure
> Codings. But this is just a wild guess.
>
> I found a similar problem in this mailing list -
>
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-April/026304.html
>
> Unfortunately the solution of waiting more than a week until it fixes
> itself isn't quite satisfying.
>
> Is there anyone who has had similar issues and knows how to repair these
> inconsistent pgs or what is causing the delay?
>
>
> --
> Mit freundlichen Grüßen
> Daniel Aberger
> Ihr Profihost Team
>
> -------------------------------
> Profihost AG
> Expo Plaza 1
> 30539 Hannover
> Deutschland
>
> Tel.: +49 (511) 5151 8181 | Fax.: +49 (511) 5151 8282
> URL: http://www.profihost.com | E-Mail: info(a)profihost.com
>
> Sitz der Gesellschaft: Hannover, USt-IdNr. DE813460827
> Registergericht: Amtsgericht Hannover, Register-Nr.: HRB 202350
> Vorstand: Cristoph Bluhm, Sebastian Bluhm, Stefan Priebe
> Aufsichtsrat: Prof. Dr. iur. Winfried Huck (Vorsitzender)
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io