Hi Martin,
Could you additionally do a minute worth of MDS perf dump captures at
5 second intervals during the observed slowness? This can be done
using
> ceph tell mds.<id> perf dump
and pipe the result to separate (time-stamped) files and share those.
On Fri, Feb 13, 2026 at 4:59 PM Martin Gerhard Loschwitz via
ceph-users <ceph-users(a)ceph.io> wrote:
>
> Folks,
>
> we are seeing very slow MDSes on a CephFS that has nearly nothing to do. Slow as in „200 metadata requests per second“ slow when putting a bit of load onto it. We managed to get a wallclock of the MDS:
>
> [root@node01 gdbpmp]# ./gdbpmp.py -i gdbpmp.data
>
> Thread: 7 (mds-rank-fin) - 17 samples
>
> + 100.00% clone
> + 100.00% start_thread
> + 100.00% Finisher::finisher_thread_entry()
> + 94.12% MDSIOContextBase::complete
> |+ 88.24% MDSRank::respawn
> ||+ 88.24% Context::complete
> || + 88.24% MDSDaemon::respawn
> || + 88.24% ceph::logging::Log::dump_recent()
> || + 52.94% ceph::logging::Log::_flush(std::vector<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >&, bool)
> || |+ 23.53% __snprintf_chk
> || ||+ 23.53% __vsnprintf_internal
> || || + 17.65% __vfprintf_internal
> || || + 5.88% __strchrnul_evex
> || || + 5.88% _IO_default_xsputn
> || |+ 17.65% ceph::logging::Log::_flush_logbuf()
> || ||+ 17.65% ceph::logging::Log::_log_safe_write(std::basic_string_view<char, std::char_traits<char> >)
> || || + 17.65% safe_write
> || || + 17.65% write
> || |+ 5.88% __tz_convert
> || | + 5.88% __tzfile_compute
> || | + 5.88% __strcmp_evex
> || + 5.88% __memmove_avx512_unaligned_erms
> |+ 5.88% std::lock_guard<ceph::fair_mutex>::lock_guard
> | + 5.88% ceph::fair_mutex::lock
> | + 5.88% std::condition_variable::wait<ceph::fair_mutex::lock()::{lambda()#1}>(std::unique_lock<std::mutex>&, ceph::fair_mutex::lock()::{lambda()#1})
> | + 5.88% std::condition_variable::wait(std::unique_lock<std::mutex>&)
> | + 5.88% pthread_cond_wait@@GLIBC_2.3.2
> | + 5.88% __futex_abstimed_wait_common
> + 5.88% std::condition_variable::wait(std::unique_lock<std::mutex>&)
> + 5.88% pthread_cond_wait@@GLIBC_2.3.2
> + 5.88% __futex_abstimed_wait_common
>
> Thread: 10 (mds-rank-progr) - 17 samples
>
> + 100.00% clone
> + 100.00% start_thread
> + 100.00% MDSRank::ProgressThread::entry
> + 100.00% std::_V2::condition_variable_any::wait<std::unique_lock<ceph::fair_mutex>, MDSRank::ProgressThread::entry()::<lambda()> >
> + 100.00% std::_V2::condition_variable_any::wait<std::unique_lock<ceph::fair_mutex> >
> + 88.24% std::condition_variable::wait(std::unique_lock<std::mutex>&)
> |+ 88.24% pthread_cond_wait@@GLIBC_2.3.2
> | + 88.24% __futex_abstimed_wait_common
> + 11.76% std::_V2::condition_variable_any::_Unlock<std::unique_lock<ceph::fair_mutex> >::~_Unlock
> + 11.76% std::unique_lock<ceph::fair_mutex>::lock
> + 11.76% ceph::fair_mutex::lock
> + 11.76% std::condition_variable::wait<ceph::fair_mutex::lock()::{lambda()#1}>(std::unique_lock<std::mutex>&, ceph::fair_mutex::lock()::{lambda()#1})
> + 11.76% std::condition_variable::wait(std::unique_lock<std::mutex>&)
> + 11.76% pthread_cond_wait@@GLIBC_2.3.2
> + 11.76% __futex_abstimed_wait_common
>
> Thread: 11 (mds-log-trim) - 17 samples
>
> + 100.00% clone
> + 100.00% start_thread
> + 100.00% execute_native_thread_routine
> + 100.00% MDLog::log_trim_upkeep
> + 100.00% std::_V2::condition_variable_any::wait_for<std::unique_lock<ceph::fair_mutex>, long, std::ratio<1l, 1000l> >
> + 100.00% std::_V2::condition_variable_any::wait_until<std::unique_lock<ceph::fair_mutex>, std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >
> + 82.35% std::_V2::condition_variable_any::_Unlock<std::unique_lock<ceph::fair_mutex> >::~_Unlock
> |+ 82.35% std::unique_lock<ceph::fair_mutex>::lock
> | + 82.35% ceph::fair_mutex::lock
> | + 82.35% std::condition_variable::wait<ceph::fair_mutex::lock()::{lambda()#1}>(std::unique_lock<std::mutex>&, ceph::fair_mutex::lock()::{lambda()#1})
> | + 82.35% std::condition_variable::wait(std::unique_lock<std::mutex>&)
> | + 82.35% pthread_cond_wait@@GLIBC_2.3.2
> | + 82.35% __futex_abstimed_wait_common
> + 17.65% std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > >
> + 17.65% std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > >
> + 17.65% std::__condvar::wait_until
> + 17.65% pthread_cond_clockwait(a)GLIBC_2.30
> + 17.65% __futex_abstimed_wait_common
>
> Thread: 12 (mds-cache-trim) - 17 samples
>
> + 100.00% clone
> + 100.00% start_thread
> + 100.00% execute_native_thread_routine
> + 100.00% MDCache::upkeep_main
> + 70.59% std::scoped_lock<ceph::fair_mutex>::scoped_lock
> |+ 70.59% ceph::fair_mutex::lock
> | + 70.59% std::condition_variable::wait<ceph::fair_mutex::lock()::{lambda()#1}>(std::unique_lock<std::mutex>&, ceph::fair_mutex::lock()::{lambda()#1})
> | + 70.59% std::condition_variable::wait(std::unique_lock<std::mutex>&)
> | + 70.59% pthread_cond_wait@@GLIBC_2.3.2
> | + 70.59% __futex_abstimed_wait_common
> + 23.53% std::condition_variable::wait_for<long, std::ratio<1l, 1000000000l> >
> |+ 23.53% std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > >
> | + 23.53% std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > >
> | + 23.53% std::__condvar::wait_until
> | + 23.53% pthread_cond_clockwait(a)GLIBC_2.30
> | + 23.53% __futex_abstimed_wait_common
> + 5.88% MDCache::check_memory_usage
> + 5.88% MemoryModel::sample
> + 5.88% MemoryModel::_sample
> + 5.88% std::getline<char, std::char_traits<char>, std::allocator<char> >
> + 5.88% std::basic_istream<char, std::char_traits<char> >& std::getline<char, std::char_traits<char>, std::allocator<char> >(std::basic_istream<char, std::char_traits<char> >&, std::__cxx11::basic_string<char
> , std::char_traits<char>, std::allocator<char> >&, char)
> + 5.88% std::basic_filebuf<char, std::char_traits<char> >::underflow()
> + 5.88% std::__basic_file<char>::xsgetn(char*, long)
> + 5.88% read
>
>
> Thread: 15 (safe_timer) - 17 samples
>
> + 100.00% clone
> + 100.00% start_thread
> + 100.00% CommonSafeTimerThread<std::mutex>::entry()
> + 100.00% CommonSafeTimer<std::mutex>::timer_thread()
> + 88.24% pthread_cond_clockwait(a)GLIBC_2.30
> |+ 88.24% __futex_abstimed_wait_common
> + 11.76% std::condition_variable::wait(std::unique_lock<std::mutex>&)
> + 11.76% pthread_cond_wait@@GLIBC_2.3.2
> + 11.76% __futex_abstimed_wait_common
>
> Thread: 17 (safe_timer) - 17 samples
>
> + 100.00% clone
> + 100.00% start_thread
> + 100.00% CommonSafeTimerThread<ceph::fair_mutex>::entry()
> + 100.00% CommonSafeTimer<ceph::fair_mutex>::timer_thread()
> + 70.59% std::_V2::condition_variable_any::_Unlock<std::unique_lock<ceph::fair_mutex> >::~_Unlock()
> |+ 70.59% std::condition_variable::wait(std::unique_lock<std::mutex>&)
> | + 70.59% pthread_cond_wait@@GLIBC_2.3.2
> | + 70.59% __futex_abstimed_wait_common
> + 29.41% pthread_cond_clockwait(a)GLIBC_2.30
> + 29.41% __futex_abstimed_wait_common
>
>
> Thread: 20 (ms_dispatch) - 17 samples
>
> + 100.00% clone
> + 100.00% start_thread
> + 100.00% DispatchQueue::DispatchThread::entry()
> + 100.00% DispatchQueue::entry()
> + 100.00% MDSDaemon::ms_dispatch2
> + 94.12% std::lock_guard<ceph::fair_mutex>::lock_guard
> |+ 94.12% ceph::fair_mutex::lock
> | + 94.12% std::condition_variable::wait<ceph::fair_mutex::lock()::{lambda()#1}>(std::unique_lock<std::mutex>&, ceph::fair_mutex::lock()::{lambda()#1})
> | + 94.12% std::condition_variable::wait(std::unique_lock<std::mutex>&)
> | + 94.12% pthread_cond_wait@@GLIBC_2.3.2
> | + 94.12% __futex_abstimed_wait_common
> + 5.88% MDSRankDispatcher::ms_dispatch
> + 5.88% MDSRank::_dispatch
> + 5.88% MDSRank::_advance_queues
> + 5.88% MDSContext::complete
> + 5.88% Context::complete
> + 5.88% Server::handle_client_readdir
> + 5.88% CInode::encode_inodestat
> + 5.88% encode<optmetadata_singleton<optmetadata_client_t<std::allocator>, std::allocator>, std::allocator>
> + 5.88% optmetadata_multiton<optmetadata_singleton<optmetadata_client_t<std::allocator>, std::allocator>, std::allocator>::encode
> + 5.88% ceph::buffer::v15_2_0::list::append_hole(unsigned int)
> + 5.88% ceph::buffer::v15_2_0::list::refill_append_space(unsigned int)
> + 5.88% mempool::pool_t::adjust_count(long, long)
>
> Thread: 21 (io_context_pool) - 17 samples
>
> + 100.00% clone
> + 100.00% start_thread
> + 100.00% execute_native_thread_routine
> + 100.00% std::thread::_State_impl<std::thread::_Invoker<std::tuple<make_named_thread<ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::basic_string_view<char, std::char_traits<char> >, ceph::async::i
> o_context_pool::start(short)::{lambda()#1}&&)::{lambda(auto:1&&, (auto:2&&)...)#1}, ceph::async::io_context_pool::start(short)::{lambda()#1}> > >::_M_run()
> + 100.00% std::thread::_Invoker<std::tuple<make_named_thread<ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::basic_string_view<char, std::char_traits<char> >, ceph::async::io_context_pool::start(sh
> ort)::{lambda()#1}&&)::{lambda(auto:1&&, (auto:2&&)...)#1}, ceph::async::io_context_pool::start(short)::{lambda()#1}> >::operator()()
> + 100.00% std::thread::_Invoker<std::tuple<make_named_thread<ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::basic_string_view<char, std::char_traits<char> >, ceph::async::io_context_pool::start(s
> hort)::{lambda()#1}&&)::{lambda(auto:1&&, (auto:2&&)...)#1}, ceph::async::io_context_pool::start(short)::{lambda()#1}> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>)
> + 100.00% std::__invoke<make_named_thread<ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::basic_string_view<char, std::char_traits<char> >, ceph::async::io_context_pool::start(short)::{lambda()#1
> }&&)::{lambda(auto:1&&, (auto:2&&)...)#1}, ceph::async::io_context_pool::start(short)::{lambda()#1}>(ceph::async::io_context_pool::start(short)::{lambda()#1}&&, ceph::async::io_context_pool::start(sho
> rt)::{lambda()#1}&&)
> + 100.00% std::__invoke_impl<void, make_named_thread<ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::basic_string_view<char, std::char_traits<char> >, ceph::async::io_context_pool::start(short):
> :{lambda()#1}&&)::{lambda(auto:1&&, (auto:2&&)...)#1}, ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::__invoke_other, make_named_thread<ceph::async::io_context_pool::start(short)::{la
> mbda()#1}>(std::basic_string_view<char, std::char_traits<char> >, ceph::async::io_context_pool::start(short)::{lambda()#1}&&)::{lambda(auto:1&&, (auto:2&&)...)#1}&&, ceph::async::io_context_pool::sta
> rt(short)::{lambda()#1}&&)
> + 100.00% make_named_thread<ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::basic_string_view<char, std::char_traits<char> >, ceph::async::io_context_pool::start(short)::{lambda()#1}&&)::{lambd
> a(auto:1&&, (auto:2&&)...)#1}::operator()<ceph::async::io_context_pool::start(short)::{lambda()#1}>(ceph::async::io_context_pool::start(short)::{lambda()#1}&&) const
> + 100.00% std::invoke<ceph::async::io_context_pool::start(short)::{lambda()#1}>(ceph::async::io_context_pool::start(short)::{lambda()#1}&&)
> + 100.00% std::__invoke<ceph::async::io_context_pool::start(short)::{lambda()#1}>(ceph::async::io_context_pool::start(short)::{lambda()#1}&&)
> + 100.00% std::__invoke_impl<void, ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::__invoke_other, ceph::async::io_context_pool::start(short)::{lambda()#1}&&)
> + 100.00% ceph::async::io_context_pool::start(short)::{lambda()#1}::operator()() const
> + 100.00% boost::asio::io_context::run
> + 100.00% boost::asio::detail::scheduler::run(boost::system::error_code&) [clone .constprop.0] [clone .isra.0]
> + 100.00% boost::asio::detail::scheduler::do_run_one
> + 94.12% boost::asio::detail::conditionally_enabled_event::wait
> |+ 94.12% boost::asio::detail::posix_event::wait<boost::asio::detail::conditionally_enabled_mutex::scoped_lock>
> | + 94.12% pthread_cond_wait@@GLIBC_2.3.2
> | + 94.12% __futex_abstimed_wait_common
> + 5.88% boost::asio::detail::scheduler_operation::complete
> + 5.88% boost::asio::detail::executor_op<boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul> const, void>, std::allocator<voi
> d>, boost::asio::detail::scheduler_operation>::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long)
> + 5.88% boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul> const, void>::operator()()
> + 5.88% boost::asio::detail::executor_op<boost::asio::detail::binder0<MonClient::handle_config(MConfig*)::{lambda()#2}>, std::allocator<void>, boost::asio::detail::scheduler_operation>::do_complete
> (void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long) [clone .lto_priv.0]
> + 5.88% ceph::common::ConfigProxy::set_mon_vals(ceph::common::CephContext*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<cha
> r, std::char_traits<char>, std::allocator<char> >, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__c
> xx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > co
> nst&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>) [clone .isra.0]
> + 5.88% md_config_t::set_mon_vals(ceph::common::CephContext*, ConfigValues&, ConfigTracker const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::_
> _cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocato
> r<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>)
> + 5.88% md_config_t::update_legacy_vals(ConfigValues&)
> + 5.88% md_config_t::update_legacy_val(ConfigValues&, Option const&, std::variant<long ConfigValues::*, unsigned long ConfigValues::*, std::__cxx11::basic_string<char, std::char_traits<char>, s
> td::allocator<char> > ConfigValues::*, double ConfigValues::*, bool ConfigValues::*, entity_addr_t ConfigValues::*, entity_addrvec_t ConfigValues::*, uuid_d ConfigValues::*>)
> + 5.88% md_config_t::_get_val(ConfigValues const&, Option const&, boost::container::small_vector<std::pair<Option const*, std::variant<std::monostate, std::__cxx11::basic_string<char, std::cha
> r_traits<char>, std::allocator<char> >, unsigned long, long, double, bool, entity_addr_t, entity_addrvec_t, std::chrono::duration<long, std::ratio<1l, 1l> >, std::chrono::duration<long
> , std::ratio<1l, 1000l> >, Option::size_t, uuid_d> const*>, 4ul, void, void>*, std::basic_ostream<char, std::char_traits<char> >*) const
> + 5.88% md_config_t::_get_val_nometa[abi:cxx11](ConfigValues const&, Option const&) const
>
> Thread: 22 (io_context_pool) - 17 samples
>
> + 100.00% clone
> + 100.00% start_thread
> + 100.00% execute_native_thread_routine
> + 100.00% std::thread::_State_impl<std::thread::_Invoker<std::tuple<make_named_thread<ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::basic_string_view<char, std::char_traits<char> >, ceph::async::i
> o_context_pool::start(short)::{lambda()#1}&&)::{lambda(auto:1&&, (auto:2&&)...)#1}, ceph::async::io_context_pool::start(short)::{lambda()#1}> > >::_M_run()
> + 100.00% std::thread::_Invoker<std::tuple<make_named_thread<ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::basic_string_view<char, std::char_traits<char> >, ceph::async::io_context_pool::start(sh
> ort)::{lambda()#1}&&)::{lambda(auto:1&&, (auto:2&&)...)#1}, ceph::async::io_context_pool::start(short)::{lambda()#1}> >::operator()()
> + 100.00% std::thread::_Invoker<std::tuple<make_named_thread<ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::basic_string_view<char, std::char_traits<char> >, ceph::async::io_context_pool::start(s
> hort)::{lambda()#1}&&)::{lambda(auto:1&&, (auto:2&&)...)#1}, ceph::async::io_context_pool::start(short)::{lambda()#1}> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>)
> + 100.00% std::__invoke<make_named_thread<ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::basic_string_view<char, std::char_traits<char> >, ceph::async::io_context_pool::start(short)::{lambda()#1
> }&&)::{lambda(auto:1&&, (auto:2&&)...)#1}, ceph::async::io_context_pool::start(short)::{lambda()#1}>(ceph::async::io_context_pool::start(short)::{lambda()#1}&&, ceph::async::io_context_pool::start(sho
> rt)::{lambda()#1}&&)
> + 100.00% std::__invoke_impl<void, make_named_thread<ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::basic_string_view<char, std::char_traits<char> >, ceph::async::io_context_pool::start(short):
> :{lambda()#1}&&)::{lambda(auto:1&&, (auto:2&&)...)#1}, ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::__invoke_other, make_named_thread<ceph::async::io_context_pool::start(short)::{la
> mbda()#1}>(std::basic_string_view<char, std::char_traits<char> >, ceph::async::io_context_pool::start(short)::{lambda()#1}&&)::{lambda(auto:1&&, (auto:2&&)...)#1}&&, ceph::async::io_context_pool::sta
> rt(short)::{lambda()#1}&&)
> + 100.00% make_named_thread<ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::basic_string_view<char, std::char_traits<char> >, ceph::async::io_context_pool::start(short)::{lambda()#1}&&)::{lambd
> a(auto:1&&, (auto:2&&)...)#1}::operator()<ceph::async::io_context_pool::start(short)::{lambda()#1}>(ceph::async::io_context_pool::start(short)::{lambda()#1}&&) const
> + 100.00% std::invoke<ceph::async::io_context_pool::start(short)::{lambda()#1}>(ceph::async::io_context_pool::start(short)::{lambda()#1}&&)
> + 100.00% std::__invoke<ceph::async::io_context_pool::start(short)::{lambda()#1}>(ceph::async::io_context_pool::start(short)::{lambda()#1}&&)
> + 100.00% std::__invoke_impl<void, ceph::async::io_context_pool::start(short)::{lambda()#1}>(std::__invoke_other, ceph::async::io_context_pool::start(short)::{lambda()#1}&&)
> + 100.00% ceph::async::io_context_pool::start(short)::{lambda()#1}::operator()() const
> + 100.00% boost::asio::io_context::run
> + 100.00% boost::asio::detail::scheduler::run(boost::system::error_code&) [clone .constprop.0] [clone .isra.0]
> + 100.00% boost::asio::detail::scheduler::do_run_one
> + 100.00% boost::asio::detail::conditionally_enabled_event::wait
> + 100.00% boost::asio::detail::posix_event::wait<boost::asio::detail::conditionally_enabled_mutex::scoped_lock>
> + 100.00% pthread_cond_wait@@GLIBC_2.3.2
> + 100.00% __futex_abstimed_wait_common
>
> Thread: 26 (msgr-worker-2) - 17 samples
>
> + 100.00% clone
> + 100.00% start_thread
> + 100.00% execute_native_thread_routine
> + 100.00% std::_Function_handler<void (), NetworkStack::add_thread(Worker*)::{lambda()#1}>::_M_invoke(std::_Any_data const&)
> + 100.00% EventCenter::process_events(unsigned int, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*)
> + 94.12% EpollDriver::event_wait(std::vector<FiredFileEvent, std::allocator<FiredFileEvent> >&, timeval*)
> |+ 94.12% epoll_wait
> + 5.88% AsyncConnection::process()
> + 5.88% ProtocolV2::run_continuation(Ct<ProtocolV2>&)
> + 5.88% ProtocolV2::handle_message()
> + 5.88% decode_message(ceph::common::CephContext*, int, ceph_msg_header&, ceph_msg_footer&, ceph::buffer::v15_2_0::list&, ceph::buffer::v15_2_0::list&, ceph::buffer::v15_2_0::list&, boost::intrusive_ptr<Conne
> ction>)
> + 5.88% MMgrMap::decode_payload()
> + 5.88% MgrMap::decode(ceph::buffer::v15_2_0::list::iterator_impl<true>&)
> + 5.88% MgrMap::ModuleInfo::decode(ceph::buffer::v15_2_0::list::iterator_impl<true>&)
> + 5.88% std::enable_if<denc_traits<std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocat
> or<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, void>::supported&&(!denc_traits<std::set<std::__cxx11::basic_string<char, std::cha
> r_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_trait
> s<char>, std::allocator<char> > > >, void>::need_contiguous), void>::type ceph::decode<std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cx
> x11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, denc_traits<std::set<s
> td::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__c
> xx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, void> >(std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx1
> 1::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, ceph::buffer::v15_2_0::
> list::iterator_impl<true>&)
> + 5.88% ceph::buffer::v15_2_0::ptr::release()
>
> Sidenote: while recording this, the MDS became so slow that the MDS cluster declared it dead … As can be seen, the MDS is waiting in futex to no end, but how can we find out what it is waiting for? FS has many small files but nearly no activity ….
>
> Thanks in advance
> Martin
>
> --
> Martin Gerhard Loschwitz
> Geschäftsführer / CEO, True West IT Services GmbH
> Phone: +49 2433 5253130
> Mobile: +49 176 61832178
> Address: Schmiedegasse 24a, 41836 Hückelhoven, Germany
> Legal: HRB 21985, Amtsgericht Mönchengladbach
> VAT: DE363893844
>
> True West IT Services GmbH is compliant with the GDPR regulation on data protection and privacy in the European Union and the European Economic Area. You can request the information on how we collect and process your private data according to the law by contacting the email sender.
>
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
--
Cheers,
Venky