El 2020-10-20 23:17, Anthony D'Atri escribió:
On Oct 20, 2020, at 6:23 PM, Ing. Luis Felipe Domínguez Vega <luis.dominguez@desoft.cu> wrote:
El 2020-10-20 19:33, Anthony D'Atri escribió:
You have a *lot* of peering and recovery going on. Write a script that monitors available memory on the system and restarts the OSD process using the most when it crosses some threshold. Run that on all OSD nodes. OSDs will come up, make some progress, get restarted, but eventually they’ll sync up.
On Oct 20, 2020, at 2:57 PM, Ing. Luis Felipe Domínguez Vega <luis.dominguez@desoft.cu> wrote: Hi, today mi Infra provider has a blackout, then the Ceph was try to recover but are in an inconsistent state because many OSD can recover itself because the kernel kill it by OOM. Even now one OSD that was OK, go down by OOM killed. Even in a server with 32GB RAM the OSD use ALL that and never recover, i think that can be a memory leak, ceph version octopus 15.2.3 In: https://pastebin.pl/view/59089adc You can see that buffer_anon get 32GB, but why?? all my cluster is down because that. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io That's the only solution???
I didn’t say that. If you don’t like it, don’t do it. You can also try eras’ idea. You’ve offered almost no detail about your cluster or hardware.
there is nothing to limit the OSD resource usage on recover?
Sorry, what is the era's idea? And i can send my cluster info, what do you need to try a solution (already i'm running your suggestion of a script that restart the OSD on high memory)