On 8/15/23 16:36, Adam King wrote:
with the log to cluster level already on debug, if you do a "ceph mgr fail" what does cephadm log to the cluster before it reports sleeping? It should at least be doing something if it's responsive at all. Also, in "ceph orch ps" and "ceph orch device ls" are the REFRESHED columns reporting that they've refreshed the info recently (last 10 minutes for daemons, last 30 minutes for devices)?
They have been refreshed very recently. The issue seems to be a bit larger than just the not working upgrade. We are now not even able to restart a daemon. When I issue the command # ceph orch daemon restart crash.cephmon01 these two lines show up in the cephadm log but nothing else happens: 2023-08-16T10:35:41.640027+0200 mgr.cephmon01 [INF] Schedule restart daemon crash.cephmon01 2023-08-16T10:35:41.640497+0200 mgr.cephmon01 [DBG] _kick_serve_loop The container for crash.cephmon01 does not get restarted. It looks like the service loop does not get executed. Can we see what jobs are in this queue and why they do not get executed? Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin https://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin