looks to be related to this issue. https://tracker.ceph.com/issues/66717 I just double checked squid 19.2.2 and it doesn't seem to have this fix listed in the pr. Although it seems to be present in 19.2.3. I'm not quite sure if there is a workaround for this in the meantime. On Thu, 24 Jul 2025 at 11:03, Devender Singh <devender@netskrt.io> wrote:
Thanks for your reply…
Currently I disabled that option,, but below are the logs….
# zcat ceph-osd.*.log.1.gz |egrep -i "faile|error|down" 2025-07-11T17:57:27.303+0000 7f2aeacb4680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:57:27.303+0000 7f2aeacb4680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:57:27.303+0000 7f2aeacb4680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:57:27.303+0000 7f2aeacb4680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:57:27.303+0000 7f2aeacb4680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:57:27.303+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:57:27.303+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.307+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2025-07-11T17:57:27.307+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.311+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:57:27.311+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.311+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:57:27.311+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.315+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:57:27.315+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.319+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:57:27.319+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.323+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:57:27.323+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.327+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:57:27.327+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.331+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:57:27.331+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.335+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:57:27.335+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.343+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:57:27.347+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:57:27.355+0000 7f2aeacb4680 1 freelist shutdown 2025-07-11T17:57:27.355+0000 7f2aeacb4680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:57:27.355+0000 7f2aeacb4680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:57:27.995+0000 7f2aeacb4680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:57:27.995+0000 7f2aeacb4680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:57:27.995+0000 7f2aeacb4680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:57:27.995+0000 7f2aeacb4680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:57:27.995+0000 7f2aeacb4680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:57:27.995+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:57:27.995+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.995+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:57:27.995+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.995+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:57:27.995+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.995+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:57:27.995+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.995+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:57:27.995+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.995+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:57:27.995+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.995+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:57:27.995+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.995+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:57:27.995+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.995+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:57:27.995+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.995+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:57:27.995+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:27.995+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:57:27.995+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:57:28.083+0000 7f2aeacb4680 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2025-07-11T17:57:28.103+0000 7f2aeacb4680 0 bluestore(/var/lib/ceph/osd/ceph-4/) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2025-07-11T17:57:28.487+0000 7f2aeacb4680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:57:28.487+0000 7f2aeacb4680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:57:28.783+0000 7f2aeacb4680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:57:28.783+0000 7f2aeacb4680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:57:28.783+0000 7f2aeacb4680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:57:28.783+0000 7f2aeacb4680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:57:28.783+0000 7f2aeacb4680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:57:28.783+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:57:28.783+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:28.783+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:57:28.783+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:28.783+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:57:28.783+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:28.783+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:57:28.783+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:28.783+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:57:28.783+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:28.783+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:57:28.783+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:28.783+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:57:28.783+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:28.783+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:57:28.783+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:28.783+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:57:28.783+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:28.783+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:57:28.783+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:28.783+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:57:28.783+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:57:29.559+0000 7f2aeacb4680 1 bluestore(/var/lib/ceph/osd/ceph-4/) _fsck_on_open <<<FINISH>>> with 0 errors, 0 warnings, 0 repaired, 0 remaining in 0.670384 seconds 2025-07-11T17:57:29.563+0000 7f2aeacb4680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:57:29.563+0000 7f2aeacb4680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:57:29.735+0000 7f2aeacb4680 1 freelist shutdown 2025-07-11T17:57:30.155+0000 7f2aeacb4680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:57:30.155+0000 7f2aeacb4680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:57:30.155+0000 7f2aeacb4680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:57:30.155+0000 7f2aeacb4680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:57:30.155+0000 7f2aeacb4680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:57:30.155+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:57:30.155+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.155+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:57:30.155+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.155+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:57:30.155+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.155+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:57:30.155+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.155+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:57:30.155+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.155+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:57:30.155+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.155+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:57:30.155+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.155+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:57:30.155+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.155+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:57:30.155+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.155+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:57:30.155+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.155+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:57:30.155+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:57:30.227+0000 7f2aeacb4680 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2025-07-11T17:57:30.227+0000 7f2aeacb4680 0 bluestore(/var/lib/ceph/osd/ceph-4/) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2025-07-11T17:57:30.379+0000 7f2aeacb4680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:57:30.379+0000 7f2aeacb4680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:57:30.687+0000 7f2aeacb4680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:57:30.687+0000 7f2aeacb4680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:57:30.687+0000 7f2aeacb4680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:57:30.687+0000 7f2aeacb4680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:57:30.687+0000 7f2aeacb4680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:57:30.687+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:57:30.687+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.687+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:57:30.687+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.687+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:57:30.687+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.687+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:57:30.687+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.687+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:57:30.687+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.687+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:57:30.687+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.687+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:57:30.687+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.687+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:57:30.687+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.687+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:57:30.687+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.687+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:57:30.687+0000 7f2aeacb4680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:30.687+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:57:30.687+0000 7f2aeacb4680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:57:30.695+0000 7f2aeacb4680 3 rocksdb: [db/db_impl/db_impl_open.cc:1432] Failed to truncate log #4: IO error: No such file or directory: While open a file for appending: db.wal/000004.log: No such file or directory 2025-07-11T17:57:30.755+0000 7f2aeacb4680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:57:30.755+0000 7f2aeacb4680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:57:31.639+0000 7f2aeacb4680 1 freelist shutdown 2025-07-11T17:57:44.583+0000 7f214047d680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:57:44.583+0000 7f214047d680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:57:44.583+0000 7f214047d680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:57:44.583+0000 7f214047d680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:57:44.583+0000 7f214047d680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:57:44.583+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:57:44.583+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2025-07-11T17:57:44.587+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:57:44.587+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:44.591+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:57:44.591+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:44.591+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:57:44.591+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:44.595+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:57:44.595+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:44.599+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:57:44.599+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:44.607+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:57:44.607+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:44.611+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:57:44.611+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:44.615+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:57:44.615+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:44.623+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:57:44.623+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:44.627+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:57:44.635+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:57:44.647+0000 7f214047d680 1 freelist shutdown 2025-07-11T17:57:44.647+0000 7f214047d680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:57:44.647+0000 7f214047d680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:57:45.183+0000 7f214047d680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:57:45.183+0000 7f214047d680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:57:45.183+0000 7f214047d680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:57:45.183+0000 7f214047d680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:57:45.183+0000 7f214047d680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:57:45.183+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:57:45.183+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.183+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:57:45.183+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.183+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:57:45.183+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.183+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:57:45.183+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.183+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:57:45.183+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.183+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:57:45.183+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.183+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:57:45.183+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.183+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:57:45.183+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.183+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:57:45.183+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.183+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:57:45.183+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.187+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:57:45.187+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:57:45.267+0000 7f214047d680 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2025-07-11T17:57:45.291+0000 7f214047d680 0 bluestore(/var/lib/ceph/osd/ceph-5/) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2025-07-11T17:57:45.675+0000 7f214047d680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:57:45.675+0000 7f214047d680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:57:45.991+0000 7f214047d680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:57:45.991+0000 7f214047d680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:57:45.991+0000 7f214047d680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:57:45.991+0000 7f214047d680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:57:45.991+0000 7f214047d680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:57:45.991+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:57:45.991+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.991+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:57:45.991+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.991+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:57:45.991+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.991+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:57:45.991+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.991+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:57:45.991+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.991+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:57:45.991+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.991+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:57:45.991+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.991+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:57:45.991+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.991+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:57:45.991+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.991+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:57:45.991+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:45.991+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:57:45.991+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:57:46.703+0000 7f214047d680 1 bluestore(/var/lib/ceph/osd/ceph-5/) _fsck_on_open <<<FINISH>>> with 0 errors, 0 warnings, 0 repaired, 0 remaining in 0.604180 seconds 2025-07-11T17:57:46.703+0000 7f214047d680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:57:46.703+0000 7f214047d680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:57:46.911+0000 7f214047d680 1 freelist shutdown 2025-07-11T17:57:47.327+0000 7f214047d680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:57:47.327+0000 7f214047d680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:57:47.327+0000 7f214047d680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:57:47.327+0000 7f214047d680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:57:47.327+0000 7f214047d680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:57:47.327+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:57:47.327+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.327+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:57:47.327+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.327+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:57:47.327+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.327+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:57:47.327+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.327+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:57:47.327+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.327+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:57:47.327+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.327+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:57:47.327+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.327+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:57:47.327+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.327+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:57:47.327+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.327+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:57:47.327+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.327+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:57:47.327+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:57:47.399+0000 7f214047d680 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2025-07-11T17:57:47.399+0000 7f214047d680 0 bluestore(/var/lib/ceph/osd/ceph-5/) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2025-07-11T17:57:47.579+0000 7f214047d680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:57:47.579+0000 7f214047d680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:57:47.863+0000 7f214047d680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:57:47.863+0000 7f214047d680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:57:47.863+0000 7f214047d680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:57:47.863+0000 7f214047d680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:57:47.863+0000 7f214047d680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:57:47.863+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:57:47.863+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.863+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:57:47.863+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.863+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:57:47.863+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.863+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:57:47.863+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.863+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:57:47.863+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.863+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:57:47.863+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.863+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:57:47.863+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.867+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:57:47.867+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.867+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:57:47.867+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.867+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:57:47.867+0000 7f214047d680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:57:47.867+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:57:47.867+0000 7f214047d680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:57:47.871+0000 7f214047d680 3 rocksdb: [db/db_impl/db_impl_open.cc:1432] Failed to truncate log #4: IO error: No such file or directory: While open a file for appending: db.wal/000004.log: No such file or directory 2025-07-11T17:57:47.931+0000 7f214047d680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:57:47.931+0000 7f214047d680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:57:48.731+0000 7f214047d680 1 freelist shutdown 2025-07-11T17:58:03.399+0000 7f082a781680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:58:03.399+0000 7f082a781680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:58:03.399+0000 7f082a781680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:58:03.399+0000 7f082a781680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:58:03.399+0000 7f082a781680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:58:03.399+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:58:03.399+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:03.403+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:58:03.403+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2025-07-11T17:58:03.403+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:58:03.403+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:03.407+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:58:03.407+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:03.407+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:58:03.407+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:03.411+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:58:03.411+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:03.415+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:58:03.415+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:03.419+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:58:03.419+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:03.423+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:58:03.423+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:03.427+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:58:03.427+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:03.431+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:58:03.435+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:58:03.439+0000 7f082a781680 1 freelist shutdown 2025-07-11T17:58:03.439+0000 7f082a781680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:58:03.439+0000 7f082a781680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:58:04.055+0000 7f082a781680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:58:04.055+0000 7f082a781680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:58:04.055+0000 7f082a781680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:58:04.055+0000 7f082a781680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:58:04.055+0000 7f082a781680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:58:04.055+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:58:04.055+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.055+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:58:04.055+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.055+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:58:04.055+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.055+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:58:04.055+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.055+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:58:04.055+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.055+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:58:04.055+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.055+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:58:04.055+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.055+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:58:04.055+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.055+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:58:04.055+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.055+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:58:04.055+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.055+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:58:04.055+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:58:04.127+0000 7f082a781680 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2025-07-11T17:58:04.147+0000 7f082a781680 0 bluestore(/var/lib/ceph/osd/ceph-6/) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2025-07-11T17:58:04.459+0000 7f082a781680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:58:04.459+0000 7f082a781680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:58:04.583+0000 7f082a781680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:58:04.583+0000 7f082a781680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:58:04.583+0000 7f082a781680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:58:04.583+0000 7f082a781680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:58:04.583+0000 7f082a781680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:58:04.583+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:58:04.583+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.583+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:58:04.583+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.583+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:58:04.583+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.583+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:58:04.583+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.583+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:58:04.583+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.583+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:58:04.583+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.583+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:58:04.583+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.583+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:58:04.583+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.583+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:58:04.583+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.583+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:58:04.583+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:04.583+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:58:04.583+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:58:05.243+0000 7f082a781680 1 bluestore(/var/lib/ceph/osd/ceph-6/) _fsck_on_open <<<FINISH>>> with 0 errors, 0 warnings, 0 repaired, 0 remaining in 0.553290 seconds 2025-07-11T17:58:05.243+0000 7f082a781680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:58:05.243+0000 7f082a781680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:58:05.295+0000 7f082a781680 1 freelist shutdown 2025-07-11T17:58:05.711+0000 7f082a781680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:58:05.711+0000 7f082a781680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:58:05.711+0000 7f082a781680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:58:05.711+0000 7f082a781680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:58:05.711+0000 7f082a781680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:58:05.711+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:58:05.711+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:05.711+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:58:05.711+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:05.711+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:58:05.711+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:05.711+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:58:05.711+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:05.711+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:58:05.711+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:05.711+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:58:05.711+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:05.711+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:58:05.711+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:05.711+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:58:05.711+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:05.711+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:58:05.711+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:05.711+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:58:05.711+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:05.711+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:58:05.711+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:58:05.783+0000 7f082a781680 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2025-07-11T17:58:05.783+0000 7f082a781680 0 bluestore(/var/lib/ceph/osd/ceph-6/) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2025-07-11T17:58:05.935+0000 7f082a781680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:58:05.935+0000 7f082a781680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:58:06.247+0000 7f082a781680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:58:06.247+0000 7f082a781680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:58:06.247+0000 7f082a781680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:58:06.247+0000 7f082a781680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:58:06.247+0000 7f082a781680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:58:06.247+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:58:06.247+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:06.247+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:58:06.247+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:06.247+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:58:06.247+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:06.247+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:58:06.247+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:06.247+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:58:06.247+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:06.247+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:58:06.247+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:06.247+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:58:06.247+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:06.247+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:58:06.247+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:06.247+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:58:06.247+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:06.247+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:58:06.247+0000 7f082a781680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:06.247+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:58:06.247+0000 7f082a781680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:58:06.255+0000 7f082a781680 3 rocksdb: [db/db_impl/db_impl_open.cc:1432] Failed to truncate log #4: IO error: No such file or directory: While open a file for appending: db.wal/000004.log: No such file or directory 2025-07-11T17:58:06.315+0000 7f082a781680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:58:06.315+0000 7f082a781680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:58:07.251+0000 7f082a781680 1 freelist shutdown 2025-07-11T17:58:26.279+0000 7f058d726680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:58:26.279+0000 7f058d726680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:58:26.279+0000 7f058d726680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:58:26.279+0000 7f058d726680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:58:26.279+0000 7f058d726680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:58:26.279+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:58:26.279+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.283+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2025-07-11T17:58:26.283+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.283+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:58:26.283+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.287+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:58:26.287+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.287+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:58:26.287+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.287+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:58:26.291+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.291+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:58:26.291+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.295+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:58:26.295+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.295+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:58:26.299+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.299+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:58:26.299+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.303+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:58:26.307+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:58:26.311+0000 7f058d726680 1 freelist shutdown 2025-07-11T17:58:26.311+0000 7f058d726680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:58:26.311+0000 7f058d726680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:58:26.943+0000 7f058d726680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:58:26.943+0000 7f058d726680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:58:26.943+0000 7f058d726680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:58:26.943+0000 7f058d726680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:58:26.943+0000 7f058d726680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:58:26.943+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:58:26.943+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.943+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:58:26.943+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.943+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:58:26.943+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.943+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:58:26.943+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.943+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:58:26.943+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.943+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:58:26.943+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.943+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:58:26.943+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.943+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:58:26.943+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.943+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:58:26.943+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.943+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:58:26.943+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:26.943+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:58:26.943+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:58:27.023+0000 7f058d726680 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2025-07-11T17:58:27.039+0000 7f058d726680 0 bluestore(/var/lib/ceph/osd/ceph-7/) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2025-07-11T17:58:27.347+0000 7f058d726680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:58:27.347+0000 7f058d726680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:58:27.455+0000 7f058d726680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:58:27.455+0000 7f058d726680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:58:27.455+0000 7f058d726680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:58:27.455+0000 7f058d726680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:58:27.455+0000 7f058d726680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:58:27.455+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:58:27.455+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:27.455+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:58:27.455+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:27.455+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:58:27.455+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:27.455+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:58:27.455+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:27.455+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:58:27.455+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:27.455+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:58:27.455+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:27.455+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:58:27.455+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:27.455+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:58:27.455+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:27.455+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:58:27.455+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:27.455+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:58:27.455+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:27.455+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:58:27.455+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:58:28.127+0000 7f058d726680 1 bluestore(/var/lib/ceph/osd/ceph-7/) _fsck_on_open <<<FINISH>>> with 0 errors, 0 warnings, 0 repaired, 0 remaining in 0.565725 seconds 2025-07-11T17:58:28.127+0000 7f058d726680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:58:28.131+0000 7f058d726680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:58:28.179+0000 7f058d726680 1 freelist shutdown 2025-07-11T17:58:28.579+0000 7f058d726680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:58:28.579+0000 7f058d726680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:58:28.579+0000 7f058d726680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:58:28.579+0000 7f058d726680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:58:28.579+0000 7f058d726680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:58:28.583+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:58:28.583+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:28.583+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:58:28.583+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:28.583+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:58:28.583+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:28.583+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:58:28.583+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:28.583+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:58:28.583+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:28.583+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:58:28.583+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:28.583+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:58:28.583+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:28.583+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:58:28.583+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:28.583+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:58:28.583+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:28.583+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:58:28.583+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:28.583+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:58:28.583+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:58:28.655+0000 7f058d726680 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2025-07-11T17:58:28.655+0000 7f058d726680 0 bluestore(/var/lib/ceph/osd/ceph-7/) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2025-07-11T17:58:28.795+0000 7f058d726680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:58:28.795+0000 7f058d726680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:58:29.103+0000 7f058d726680 4 rocksdb: Options.error_if_exists: 0 2025-07-11T17:58:29.103+0000 7f058d726680 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2025-07-11T17:58:29.103+0000 7f058d726680 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2025-07-11T17:58:29.103+0000 7f058d726680 4 rocksdb: Options.allow_data_in_errors: 0 2025-07-11T17:58:29.103+0000 7f058d726680 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2025-07-11T17:58:29.103+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 0, name: default) 2025-07-11T17:58:29.103+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:29.103+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 1, name: m-0) 2025-07-11T17:58:29.103+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:29.103+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 2, name: m-1) 2025-07-11T17:58:29.103+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:29.103+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 3, name: m-2) 2025-07-11T17:58:29.107+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:29.107+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 4, name: p-0) 2025-07-11T17:58:29.107+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:29.107+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 5, name: p-1) 2025-07-11T17:58:29.107+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:29.107+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 6, name: p-2) 2025-07-11T17:58:29.107+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:29.107+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 7, name: O-0) 2025-07-11T17:58:29.107+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:29.107+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 8, name: O-1) 2025-07-11T17:58:29.107+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:29.107+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 9, name: O-2) 2025-07-11T17:58:29.107+0000 7f058d726680 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2025-07-11T17:58:29.107+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 10, name: L) 2025-07-11T17:58:29.107+0000 7f058d726680 2 rocksdb: [db/column_family.cc:578] Failed to register data paths of column family (id: 11, name: P) 2025-07-11T17:58:29.111+0000 7f058d726680 3 rocksdb: [db/db_impl/db_impl_open.cc:1432] Failed to truncate log #4: IO error: No such file or directory: While open a file for appending: db.wal/000004.log: No such file or directory 2025-07-11T17:58:29.171+0000 7f058d726680 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2025-07-11T17:58:29.171+0000 7f058d726680 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2025-07-11T17:58:30.015+0000 7f058d726680 1 freelist shutdown
Regards Dev
On Jul 24, 2025, at 10:57 AM, Joshua Blanch <joshua.blanch@clyso.com> wrote:
r with that config value but I’ve seen a bug in that version that causes osds t