Hi Igor, thanks for the confirmation, I just wanted to make sure I understood correctly. Thanks! Eugen Zitat von Igor Fedotov <igor.fedotov@croit.io>:

Hi Eugen,

that's a know issue - crash on OSD termination. Fixed in v18.2.5, see https://tracker.ceph.com/issues/66148

And your backtrace explicitly states that happened in 18.2.4:

...

    "ceph_version": "18.2.4",

Thanks,

Igor

On 16.04.2025 12:07, Eugen Block wrote:

...

Hi,

I'm currently upgrading the first customer cluster to latest Reef (18.2.5). I haven't seen that in my lab clusters yet, but it might be due to insufficient crash configuration (or the shutdown_timeout is not triggered). I found this backport for 18.2.5:

https://tracker.ceph.com/issues/63527

Am I interpreting correctly that the assert is triggered because the OSDs were shut down on 18.2.4, but this won't happen anymore on 18.2.5?

Thanks! Eugen

---snip--- root@storage01:~# ceph crash info 2025-04-16T08:51:38.757577Z_bd34dfdd-183a-4274-8a40-a6cb3311ca56 {     "archived": "2025-04-16 08:52:50.374393",     "assert_condition": "end_time - start_time_func < cct->_conf->osd_fast_shutdown_timeout",     "assert_file": "/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.2.4/rpm/el9/BUILD/ceph-18.2.4/src/osd/OSD.cc",     "assert_func": "int OSD::shutdown()",     "assert_line": 4585,     "assert_msg": "/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.2.4/rpm/el9/BUILD/ceph-18.2.4/src/osd/OSD.cc: In function 'int OSD::shutdown()' thread 7f313de0a640 time 2025-04-16T08:51:38.751362+0000\n/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.2.4/rpm/el9/BUILD/ceph-18.2.4/src/osd/OSD.cc: 4585: FAILED ceph_assert(end_time - start_time_func < cct->_conf->osd_fast_shutdown_timeout)\n",     "assert_thread_name": "signal_handler",     "backtrace": [         "/lib64/libc.so.6(+0x3e6f0) [0x7f31412d16f0]",         "/lib64/libc.so.6(+0x8b94c) [0x7f314131e94c]",         "raise()",         "abort()",         "(ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x179) [0x56015bef62b2]",         "/usr/bin/ceph-osd(+0x3c9413) [0x56015bef6413]",         "/usr/bin/ceph-osd(+0x3591d7) [0x56015be861d7]",         "/usr/bin/ceph-osd(+0xa1da57) [0x56015c54aa57]",         "/lib64/libc.so.6(+0x89c02) [0x7f314131cc02]",         "/lib64/libc.so.6(+0x10ec40) [0x7f31413a1c40]"     ],     "ceph_version": "18.2.4",     "crash_id": "2025-04-16T08:51:38.757577Z_bd34dfdd-183a-4274-8a40-a6cb3311ca56",     "entity_name": "osd.162",     "os_id": "centos",     "os_name": "CentOS Stream",     "os_version": "9",     "os_version_id": "9",     "process_name": "ceph-osd",     "stack_sig": "434a67dd4e3c173992edcf6eb5dc6014d029ca333c4904eced28762dc1959db1",     "timestamp": "2025-04-16T08:51:38.757577Z",     "utsname_hostname": "storage04",     "utsname_machine": "x86_64",     "utsname_release": "5.15.0-130-generic",     "utsname_sysname": "Linux",     "utsname_version": "#140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024" } ---snip--- _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io