Hello, ceph version is 14.2.22 (ca74598065096e6fcbd8433c8779a2be0c889351) nautilus (stable) and the rbd info command is also slow, some times it needs 6 seconds. rbd snap create command takes 17 seconds. There is another cluster with the same configuration it only takes less than 1 second. crash log -2> 2024-10-09 09:52:50.782 7f30ddcb4700 5 prioritycache tune_memory target: 2147483648 mapped: 1286012928 unmapped: 1695842304 heap: 2981855232 old mem: 1020054731 new mem: 1020054731 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.22/rpm/el7/BUILD/ceph-14.2.22/src/mon/OSDMonitor.cc: 4112: ceph_abort_msg("abort() called") ceph version 14.2.22 (ca74598065096e6fcbd8433c8779a2be0c889351) nautilus (stable) 1: (ceph::__ceph_abort(char const*, int, char const*, std::string const&)+0xdd) [0x7f30ea6c2a66] 2: (OSDMonitor::build_incremental(unsigned int, unsigned int, unsigned long)+0x89f) [0x563b0175382f] 3: (OSDMonitor::send_incremental(unsigned int, MonSession*, bool, boost::intrusive_ptr<MonOpRequest>)+0x11d) [0x563b01753e6d] 4: (OSDMonitor::check_osdmap_sub(Subscription*)+0xca) [0x563b0175a53a] 5: (Monitor::handle_subscribe(boost::intrusive_ptr<MonOpRequest>)+0xbbe) [0x563b0162c9ee] 6: (Monitor::dispatch_op(boost::intrusive_ptr<MonOpRequest>)+0x525) [0x563b0164b0a5] 7: (Monitor::_ms_dispatch(Message*)+0xcdb) [0x563b0164cd9b] 8: (Monitor::ms_dispatch(Message*)+0x26) [0x563b0167ad96] 9: (Dispatcher::ms_dispatch2(boost::intrusive_ptr<Message> const&)+0x26) [0x563b01677776] 10: (DispatchQueue::entry()+0x1699) [0x7f30ea8e7699] 11: (DispatchQueue::DispatchThread::entry()+0xd) [0x7f30ea99564d] 12: (()+0x7ea5) [0x7f30e74acea5] 13: (clone()+0x6d) [0x7f30e636fb0d] 0> 2024-10-09 09:52:55.260 7f30db4af700 -1 *** Caught signal (Aborted) ** in thread 7f30db4af700 thread_name:ms_dispatch ceph version 14.2.22 (ca74598065096e6fcbd8433c8779a2be0c889351) nautilus (stable) 1: (()+0xf630) [0x7f30e74b4630] 2: (gsignal()+0x37) [0x7f30e62a7387] 3: (abort()+0x148) [0x7f30e62a8a78] 4: (ceph::__ceph_abort(char const*, int, char const*, std::string const&)+0x1a5) [0x7f30ea6c2b2e] 5: (OSDMonitor::build_incremental(unsigned int, unsigned int, unsigned long)+0x89f) [0x563b0175382f] 6: (OSDMonitor::send_incremental(unsigned int, MonSession*, bool, boost::intrusive_ptr<MonOpRequest>)+0x11d) [0x563b01753e6d] 7: (OSDMonitor::check_osdmap_sub(Subscription*)+0xca) [0x563b0175a53a] 8: (Monitor::handle_subscribe(boost::intrusive_ptr<MonOpRequest>)+0xbbe) [0x563b0162c9ee] 9: (Monitor::dispatch_op(boost::intrusive_ptr<MonOpRequest>)+0x525) [0x563b0164b0a5] 10: (Monitor::_ms_dispatch(Message*)+0xcdb) [0x563b0164cd9b] 11: (Monitor::ms_dispatch(Message*)+0x26) [0x563b0167ad96] 12: (Dispatcher::ms_dispatch2(boost::intrusive_ptr<Message> const&)+0x26) [0x563b01677776] 13: (DispatchQueue::entry()+0x1699) [0x7f30ea8e7699] 14: (DispatchQueue::DispatchThread::entry()+0xd) [0x7f30ea99564d] 15: (()+0x7ea5) [0x7f30e74acea5] 16: (clone()+0x6d) [0x7f30e636fb0d] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this ceph -s cluster: id: 6e36443f-326b-4344-b013-c03a4b7bb8d7 health: HEALTH_WARN noout flag(s) set 314 pgs not deep-scrubbed in time 660 pgs not scrubbed in time 23 daemons have recently crashed services: mon: 3 daemons, quorum rbd02,rbd01,rbd03 (age 81m) mgr: rbd03(active, since 9d), standbys: rbd01, rbd02 osd: 360 osds: 360 up (since 9d), 360 in (since 12M) flags noout data: pools: 1 pools, 8192 pgs objects: 109.45M objects, 274 TiB usage: 488 TiB used, 3.4 PiB / 3.8 PiB avail pgs: 8116 active+clean 76 active+clean+scrubbing+deep io: client: 61 MiB/s rd, 34 MiB/s wr, 74.51k op/s rd, 2.06k op/s wr