Greetings — Have been seeing repeated crashes on my mgr module. Seems to run for about 45 to 50 seconds and then boom. Cephadm setup here. Did (try to) enable a couple modules lately. iostat, stats diskprediction_local, but have toggled them back off - unfortunately it hasn’t fixed the issue. I was seeing “no mgr” with ceph -s — to get around that I’ve tweaked the “StartLimitInterval” setting in the ceph-[CLUSTER-UID]@.service file down to 1m, and have 4x mgr’s setup so that I can get a couple commands run before mgr crashes and another starts. The 30m default there seems… high, imo - I was having intervals with no mgr which makes it tough to do much with the cluster - even “ceph -s” was hanging. Everything else in the cluster seems “normal” - still serving data. One other note — which I think is generally unrelated — I did upgrade one of my cluster nodes from “Plucky Puffin" (25.04) ubuntu, to “Questing Quokka” (25.10) ubuntu. After the upgrade, cephadm managed containers didn’t want to start. I tracked that down to having the ceph user userid in /etc/passwd set at 64045, but the container seeming to want userid 167. Most things under /var/lib/ceph/[CLUSTER UID]/ … appear to be owned by user/group 167:167 — I assume this is a default inside the container. Workaround here was to manually change the UID/GID for ceph in /etc/passwd and /etc/group. I’m going to imagine this is some collision between cephadm managed deployments, and how Ubuntu / apt installs cephadm. The aforementioned mgr Crashes look like: { "assert_condition": "cursor != root", "assert_file": "/ceph/rpmbuild/BUILD/ceph-20.2.0/src/mgr/PyFormatter.h", "assert_func": "virtual void PyFormatter::close_section()", "assert_line": 84, "assert_msg": "/ceph/rpmbuild/BUILD/ceph-20.2.0/src/mgr/PyFormatter.h: In function 'virtual void PyFormatter::close_section()' thread ffff34e55700 time 2026-02-08T13:10:07.526894+0000\n/ceph/rpmbuild/BUILD/ceph-20.2.0/src/mgr/PyFormatter.h: 84: FAILED ceph_assert(cursor != root)\n", "assert_thread_name": "telemetry", "backtrace": [ "__kernel_rt_sigreturn()", "/lib64/libc.so.6(+0x82a78) [0xffff83603a78]", "raise()", "abort()", "(ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x190) [0xffff84039874]", "/usr/bin/ceph-mgr(+0xcf540) [0xaaaacdf2f540]", "(ActivePyModules::get_perf_schema_python(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xf6c) [0xaaaacdf46ec0]", "/usr/bin/ceph-mgr(+0x105528) [0xaaaacdf65528]", "/lib64/libpython3.9.so.1.0(+0xf7bfc) [0xffff84ae9bfc]", "_PyEval_EvalFrameDefault()", "/lib64/libpython3.9.so.1.0(+0xda3f0) [0xffff84acc3f0]", "_PyEval_EvalFrameDefault()", "/lib64/libpython3.9.so.1.0(+0xc47e8) [0xffff84ab67e8]", "_PyFunction_Vectorcall()", "_PyEval_EvalFrameDefault()", "/lib64/libpython3.9.so.1.0(+0xc47e8) [0xffff84ab67e8]", "_PyFunction_Vectorcall()", "_PyEval_EvalFrameDefault()", "/lib64/libpython3.9.so.1.0(+0xc47e8) [0xffff84ab67e8]", "_PyFunction_Vectorcall()", "_PyEval_EvalFrameDefault()", "/lib64/libpython3.9.so.1.0(+0xc47e8) [0xffff84ab67e8]", "_PyFunction_Vectorcall()", "_PyEval_EvalFrameDefault()", "/lib64/libpython3.9.so.1.0(+0xda3f0) [0xffff84acc3f0]", "/lib64/libpython3.9.so.1.0(+0xea93c) [0xffff84adc93c]", "/lib64/libpython3.9.so.1.0(+0xcf304) [0xffff84ac1304]", "/lib64/libpython3.9.so.1.0(+0x197d78) [0xffff84b89d78]", "_PyObject_CallMethod_SizeT()", "(PyModuleRunner::serve()+0x6c) [0xaaaacdfdf6cc]", "(PyModuleRunner::PyModuleRunnerThread::entry()+0x148) [0xaaaacdfdff08]" ], "ceph_version": "20.2.0", "crash_id": "2026-02-08T13:10:07.528739Z_f18e6b74-438b-47db-9438-5a3861fdef2d", "entity_name": "mgr.hc-945901a5cad1b6e3.mtijbv", "os_id": "centos", "os_name": "CentOS Stream", "os_version": "9", "os_version_id": "9", "process_name": "ceph-mgr", "stack_sig": "319d76a0d71a4644f9d65f592f6e621cca918d9205fc759ab7acf6944bc77fdd", "timestamp": "2026-02-08T13:10:07.528739Z", "utsname_hostname": "hc-945901a5cad1b6e3", "utsname_machine": "aarch64", "utsname_release": "6.14.0-1010-raspi", "utsname_sysname": "Linux", "utsname_version": "#10-Ubuntu SMP PREEMPT_DYNAMIC Tue Jul 15 19:09:05 UTC 2025" }