I don't really have a good idea right now, but there was a thread [1] about ssh sessions that are not removed, maybe that could have such an impact? And if you crank up the debug level to 30, do you see anything else? ceph config set mgr debug_mgr 30 [1] https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/I452F3PWBAU4... Zitat von Robert Sander <r.sander@heinlein-support.de>:
On 8/15/23 16:36, Adam King wrote:
with the log to cluster level already on debug, if you do a "ceph mgr fail" what does cephadm log to the cluster before it reports sleeping? It should at least be doing something if it's responsive at all. Also, in "ceph orch ps" and "ceph orch device ls" are the REFRESHED columns reporting that they've refreshed the info recently (last 10 minutes for daemons, last 30 minutes for devices)?
They have been refreshed very recently.
The issue seems to be a bit larger than just the not working upgrade.
We are now not even able to restart a daemon.
When I issue the command
# ceph orch daemon restart crash.cephmon01
these two lines show up in the cephadm log but nothing else happens:
2023-08-16T10:35:41.640027+0200 mgr.cephmon01 [INF] Schedule restart daemon crash.cephmon01 2023-08-16T10:35:41.640497+0200 mgr.cephmon01 [DBG] _kick_serve_loop
The container for crash.cephmon01 does not get restarted.
It looks like the service loop does not get executed.
Can we see what jobs are in this queue and why they do not get executed?
Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin
https://www.heinlein-support.de
Tel: 030 / 405051-43 Fax: 030 / 405051-19
Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io