And how about this? osd crush rm osd.14 Zitat von Manolis Daramas <mdaramas@motivian.com>:
Hi Eugen,
I think that it cannot even be removed with ceph osd purge command, see below:
root@node01:~# ceph osd purge 14 --force --yes-i-really-mean-it osd.14 does not exist
root@node01:~# ceph osd tree ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -1 2.05046 root default -3 0.68349 host node01 0 hdd 0.14650 osd.0 up 1.00000 1.00000 4 hdd 0.04880 osd.4 up 1.00000 1.00000 8 hdd 0.04880 osd.8 up 1.00000 1.00000 10 hdd 0.04880 osd.10 up 1.00000 1.00000 14 hdd 0.39059 osd.14 DNE 0 -5 0.68349 host node02 2 hdd 0.14650 osd.2 up 1.00000 1.00000 5 hdd 0.04880 osd.5 up 1.00000 1.00000 7 hdd 0.04880 osd.7 up 1.00000 1.00000 9 hdd 0.04880 osd.9 up 1.00000 1.00000 12 hdd 0.39059 osd.12 up 1.00000 1.00000 -7 0.68349 host node03 1 hdd 0.14650 osd.1 up 1.00000 1.00000 3 hdd 0.04880 osd.3 up 1.00000 1.00000 6 hdd 0.04880 osd.6 up 1.00000 1.00000 11 hdd 0.04880 osd.11 up 1.00000 1.00000 13 hdd 0.39059 osd.13 up 1.00000 1.00000
The output of journalctl with --no-pager option is below:
-- Logs begin at Thu 2023-11-23 06:16:17 EET, end at Tue 2023-12-05 16:01:49 EET. -- Dec 05 15:23:16 node01 systemd[1]: Started Ceph mon.node01 for be4304e4-b0d5-11ec-8c6a-2965d4229f37. Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.639+0000 7fab0e25bb80 0 set uid:gid to 167:167 (ceph:ceph) Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.639+0000 7fab0e25bb80 0 ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable), process ceph-mon, pid 7 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.639+0000 7fab0e25bb80 0 pidfile_write: ignore empty --pid-file Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.667+0000 7fab0e25bb80 0 load: jerasure load: lrc Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: RocksDB version: 6.15.5 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Compile date Oct 25 2023 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: DB SUMMARY Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: DB Session ID: 69YYUY1IJK4BA9QQDDK8 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: CURRENT file: CURRENT Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: IDENTITY file: IDENTITY Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: MANIFEST file: MANIFEST-000226 size: 308 Bytes Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-node01/store.db dir, Total Num: 3, files: 000091.sst 000094.sst 000100.sst Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-node01/store.db: 000227.log size: 0 ; Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.error_if_exists: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.create_if_missing: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.paranoid_checks: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.env: 0x55be1e3aea80 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.fs: Posix File System Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.info_log: 0x55be20918360 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_file_opening_threads: 16 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.statistics: (nil) Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.use_fsync: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_log_file_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_manifest_file_size: 1073741824 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.log_file_time_to_roll: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.keep_log_file_num: 1000 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.recycle_log_file_num: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_fallocate: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_mmap_reads: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_mmap_writes: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.use_direct_reads: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.create_missing_column_families: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.db_log_dir: Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-node01/store.db Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.table_cache_numshardbits: 6 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.WAL_ttl_seconds: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.WAL_size_limit_MB: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.manifest_preallocation_size: 4194304 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.is_fd_close_on_exec: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.advise_random_on_open: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.db_write_buffer_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.write_buffer_manager: 0x55be2168c990 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.access_hint_on_compaction_start: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.use_adaptive_mutex: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.rate_limiter: (nil) Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.wal_recovery_mode: 2 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.enable_thread_tracking: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.enable_pipelined_write: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.unordered_write: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.write_thread_max_yield_usec: 100 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.row_cache: None Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.wal_filter: None Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.avoid_flush_during_recovery: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_ingest_behind: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.preserve_deletes: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.two_write_queues: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.manual_wal_flush: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.atomic_flush: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.persist_stats_to_disk: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.write_dbid_to_manifest: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.log_readahead_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.file_checksum_gen_factory: Unknown Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.best_efforts_recovery: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_data_in_errors: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.db_host_id: __hostname__ Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_background_jobs: 2 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_background_compactions: -1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_subcompactions: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.delayed_write_rate : 16777216 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_total_wal_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.stats_dump_period_sec: 600 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.stats_persist_period_sec: 600 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.stats_history_buffer_size: 1048576 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_open_files: -1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bytes_per_sync: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.wal_bytes_per_sync: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.strict_bytes_per_sync: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_readahead_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_background_flushes: -1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Compression algorithms supported: Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kZSTDNotFinalCompression supported: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kZSTD supported: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kXpressCompression supported: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kLZ4HCCompression supported: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kZlibCompression supported: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kSnappyCompression supported: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kLZ4Compression supported: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kBZip2Compression supported: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Fast CRC32 supported: Supported on x86 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-node01/store.db/MANIFEST-000226 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.merge_operator: Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_filter: None Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_filter_factory: None Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.sst_partitioner_factory: None Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.memtable_factory: SkipListFactory Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.table_factory: BlockBasedTable Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55be2096afb0) Dec 05 15:23:17 node01 bash[1969]: cache_index_and_filter_blocks: 1 Dec 05 15:23:17 node01 bash[1969]: cache_index_and_filter_blocks_with_high_priority: 0 Dec 05 15:23:17 node01 bash[1969]: pin_l0_filter_and_index_blocks_in_cache: 0 Dec 05 15:23:17 node01 bash[1969]: pin_top_level_index_and_filter: 1 Dec 05 15:23:17 node01 bash[1969]: index_type: 0 Dec 05 15:23:17 node01 bash[1969]: data_block_index_type: 0 Dec 05 15:23:17 node01 bash[1969]: index_shortening: 1 Dec 05 15:23:17 node01 bash[1969]: data_block_hash_table_util_ratio: 0.750000 Dec 05 15:23:17 node01 bash[1969]: hash_index_allow_collision: 1 Dec 05 15:23:17 node01 bash[1969]: checksum: 1 Dec 05 15:23:17 node01 bash[1969]: no_block_cache: 0 Dec 05 15:23:17 node01 bash[1969]: block_cache: 0x55be2099d090 Dec 05 15:23:17 node01 bash[1969]: block_cache_name: BinnedLRUCache Dec 05 15:23:17 node01 bash[1969]: block_cache_options: Dec 05 15:23:17 node01 bash[1969]: capacity : 536870912 Dec 05 15:23:17 node01 bash[1969]: num_shard_bits : 4 Dec 05 15:23:17 node01 bash[1969]: strict_capacity_limit : 0 Dec 05 15:23:17 node01 bash[1969]: high_pri_pool_ratio: 0.000 Dec 05 15:23:17 node01 bash[1969]: block_cache_compressed: (nil) Dec 05 15:23:17 node01 bash[1969]: persistent_cache: (nil) Dec 05 15:23:17 node01 bash[1969]: block_size: 4096 Dec 05 15:23:17 node01 bash[1969]: block_size_deviation: 10 Dec 05 15:23:17 node01 bash[1969]: block_restart_interval: 16 Dec 05 15:23:17 node01 bash[1969]: index_block_restart_interval: 1 Dec 05 15:23:17 node01 bash[1969]: metadata_block_size: 4096 Dec 05 15:23:17 node01 bash[1969]: partition_filters: 0 Dec 05 15:23:17 node01 bash[1969]: use_delta_encoding: 1 Dec 05 15:23:17 node01 bash[1969]: filter_policy: rocksdb.BuiltinBloomFilter Dec 05 15:23:17 node01 bash[1969]: whole_key_filtering: 1 Dec 05 15:23:17 node01 bash[1969]: verify_compression: 0 Dec 05 15:23:17 node01 bash[1969]: read_amp_bytes_per_bit: 0 Dec 05 15:23:17 node01 bash[1969]: format_version: 4 Dec 05 15:23:17 node01 bash[1969]: enable_index_compression: 1 Dec 05 15:23:17 node01 bash[1969]: block_align: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.write_buffer_size: 33554432 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_write_buffer_number: 2 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression: NoCompression Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression: Disabled Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.prefix_extractor: nullptr Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.num_levels: 7 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.enabled: false Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.window_bits: -14 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.level: 32767 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.strategy: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.parallel_threads: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.enabled: false Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.level0_stop_writes_trigger: 36 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.target_file_size_base: 67108864 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.target_file_size_multiplier: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_compaction_bytes: 1677721600 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.arena_block_size: 4194304 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.disable_auto_compactions: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.table_properties_collectors: Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.inplace_update_support: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.inplace_update_num_locks: 10000 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.memtable_whole_key_filtering: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.memtable_huge_page_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bloom_locality: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_successive_merges: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.optimize_filters_for_hits: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.paranoid_file_checks: Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.force_consistency_checks: 1 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.report_bg_io_stats: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.ttl: 2592000 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.periodic_compaction_seconds: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.enable_blob_files: false Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.min_blob_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.blob_file_size: 268435456 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.blob_compression_type: NoCompression Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.enable_blob_garbage_collection: false Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.683+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:3457] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.683+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-node01/store.db/MANIFEST-000226 succeeded,manifest_file_number is 226, next_file_number is 228, last_sequence is 21201, log_number is 223,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.683+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 223 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:4082] Creating manifest 230 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:3457] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1701782597693048, "job": 1, "event": "recovery_started", "wal_files": [227]} Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #227 mode 2 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:4082] Creating manifest 231 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:3457] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1701782597693989, "job": 1, "event": "recovery_finished"} Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.691+0000 7fab0e25bb80 4 rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-node01/store.db/000227.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.691+0000 7fab0e25bb80 4 rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55be20931180 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.691+0000 7fab0e25bb80 4 rocksdb: DB pointer 0x55be20a06000 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.695+0000 7faafe4ad700 4 rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.695+0000 7faafe4ad700 4 rocksdb: [db/db_impl/db_impl.cc:903] Dec 05 15:23:17 node01 bash[1969]: ** DB Stats ** Dec 05 15:23:17 node01 bash[1969]: Uptime(secs): 0.0 total, 0.0 interval Dec 05 15:23:17 node01 bash[1969]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s Dec 05 15:23:17 node01 bash[1969]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Dec 05 15:23:17 node01 bash[1969]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Dec 05 15:23:17 node01 bash[1969]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s Dec 05 15:23:17 node01 bash[1969]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s Dec 05 15:23:17 node01 bash[1969]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent Dec 05 15:23:17 node01 bash[1969]: ** Compaction Stats [default] ** Dec 05 15:23:17 node01 bash[1969]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Dec 05 15:23:17 node01 bash[1969]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Dec 05 15:23:17 node01 bash[1969]: L0 2/0 7.21 MB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: L6 1/0 7.22 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: Sum 3/0 14.43 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: ** Compaction Stats [default] ** Dec 05 15:23:17 node01 bash[1969]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Dec 05 15:23:17 node01 bash[1969]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Dec 05 15:23:17 node01 bash[1969]: Uptime(secs): 0.0 total, 0.0 interval Dec 05 15:23:17 node01 bash[1969]: Flush(GB): cumulative 0.000, interval 0.000 Dec 05 15:23:17 node01 bash[1969]: AddFile(GB): cumulative 0.000, interval 0.000 Dec 05 15:23:17 node01 bash[1969]: AddFile(Total Files): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: AddFile(L0 Files): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: AddFile(Keys): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Dec 05 15:23:17 node01 bash[1969]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Dec 05 15:23:17 node01 bash[1969]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Dec 05 15:23:17 node01 bash[1969]: ** File Read Latency Histogram By Level [default] ** Dec 05 15:23:17 node01 bash[1969]: ** Compaction Stats [default] ** Dec 05 15:23:17 node01 bash[1969]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Dec 05 15:23:17 node01 bash[1969]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Dec 05 15:23:17 node01 bash[1969]: L0 2/0 7.21 MB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: L6 1/0 7.22 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: Sum 3/0 14.43 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: ** Compaction Stats [default] ** Dec 05 15:23:17 node01 bash[1969]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Dec 05 15:23:17 node01 bash[1969]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Dec 05 15:23:17 node01 bash[1969]: Uptime(secs): 0.0 total, 0.0 interval Dec 05 15:23:17 node01 bash[1969]: Flush(GB): cumulative 0.000, interval 0.000 Dec 05 15:23:17 node01 bash[1969]: AddFile(GB): cumulative 0.000, interval 0.000 Dec 05 15:23:17 node01 bash[1969]: AddFile(Total Files): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: AddFile(L0 Files): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: AddFile(Keys): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Dec 05 15:23:17 node01 bash[1969]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Dec 05 15:23:17 node01 bash[1969]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Dec 05 15:23:17 node01 bash[1969]: ** File Read Latency Histogram By Level [default] ** Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.719+0000 7fab0e25bb80 0 starting mon.node01 rank 0 at public addrs [v2:10.40.99.11:3300/0,v1:10.40.99.11:6789/0] at bind addrs [v2:10.40.99.11:3300/0,v1:10.40.99.11:6789/0] mon_data /var/lib/ceph/mon/ceph-node01 fsid be4304e4-b0d5-11ec-8c6a-2965d4229f37 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.723+0000 7fab0e25bb80 1 mon.node01@-1(???) e0 preinit fsid be4304e4-b0d5-11ec-8c6a-2965d4229f37 Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.731+0000 7fab0e25bb80 0 mon.node01@-1(???).osd e9898 crush map has features 3314933000854323200, adjusting msgr requires Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.731+0000 7fab0e25bb80 0 mon.node01@-1(???).osd e9898 crush map has features 432629239337189376, adjusting msgr requires Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.731+0000 7fab0e25bb80 0 mon.node01@-1(???).osd e9898 crush map has features 432629239337189376, adjusting msgr requires Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.731+0000 7fab0e25bb80 0 mon.node01@-1(???).osd e9898 crush map has features 432629239337189376, adjusting msgr requires Dec 05 15:23:17 node01 bash[1969]: terminate called after throwing an instance of 'std::invalid_argument' Dec 05 15:23:17 node01 bash[1969]: what(): stoull Dec 05 15:23:17 node01 bash[1969]: *** Caught signal (Aborted) ** Dec 05 15:23:17 node01 bash[1969]: in thread 7fab0e25bb80 thread_name:ceph-mon Dec 05 15:23:17 node01 bash[1969]: ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) Dec 05 15:23:17 node01 bash[1969]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fab0b6c8cf0] Dec 05 15:23:17 node01 bash[1969]: 2: gsignal() Dec 05 15:23:17 node01 bash[1969]: 3: abort() Dec 05 15:23:17 node01 bash[1969]: 4: /lib64/libstdc++.so.6(+0x9009b) [0x7fab0acda09b] Dec 05 15:23:17 node01 bash[1969]: 5: /lib64/libstdc++.so.6(+0x9654c) [0x7fab0ace054c] Dec 05 15:23:17 node01 bash[1969]: 6: /lib64/libstdc++.so.6(+0x965a7) [0x7fab0ace05a7] Dec 05 15:23:17 node01 bash[1969]: 7: /lib64/libstdc++.so.6(+0x96808) [0x7fab0ace0808] Dec 05 15:23:17 node01 bash[1969]: 8: /lib64/libstdc++.so.6(+0x91ff9) [0x7fab0acdbff9] Dec 05 15:23:17 node01 bash[1969]: 9: (LogMonitor::log_external_backlog()+0xe70) [0x55be1dc27190] Dec 05 15:23:17 node01 bash[1969]: 10: (LogMonitor::update_from_paxos(bool*)+0x54) [0x55be1dc27334] Dec 05 15:23:17 node01 bash[1969]: 11: (Monitor::refresh_from_paxos(bool*)+0x104) [0x55be1db9ae74] Dec 05 15:23:17 node01 bash[1969]: 12: (Monitor::preinit()+0x95d) [0x55be1dbc93ad] Dec 05 15:23:17 node01 bash[1969]: 13: main() Dec 05 15:23:17 node01 bash[1969]: 14: __libc_start_main() Dec 05 15:23:17 node01 bash[1969]: 15: _start() Dec 05 15:23:17 node01 bash[1969]: debug 2023-12-05T13:23:17.735+0000 7fab0e25bb80 -1 *** Caught signal (Aborted) ** Dec 05 15:23:17 node01 bash[1969]: in thread 7fab0e25bb80 thread_name:ceph-mon Dec 05 15:23:17 node01 bash[1969]: ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) Dec 05 15:23:17 node01 bash[1969]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fab0b6c8cf0] Dec 05 15:23:17 node01 bash[1969]: 2: gsignal() Dec 05 15:23:17 node01 bash[1969]: 3: abort() Dec 05 15:23:17 node01 bash[1969]: 4: /lib64/libstdc++.so.6(+0x9009b) [0x7fab0acda09b] Dec 05 15:23:17 node01 bash[1969]: 5: /lib64/libstdc++.so.6(+0x9654c) [0x7fab0ace054c] Dec 05 15:23:17 node01 bash[1969]: 6: /lib64/libstdc++.so.6(+0x965a7) [0x7fab0ace05a7] Dec 05 15:23:17 node01 bash[1969]: 7: /lib64/libstdc++.so.6(+0x96808) [0x7fab0ace0808] Dec 05 15:23:17 node01 bash[1969]: 8: /lib64/libstdc++.so.6(+0x91ff9) [0x7fab0acdbff9] Dec 05 15:23:17 node01 bash[1969]: 9: (LogMonitor::log_external_backlog()+0xe70) [0x55be1dc27190] Dec 05 15:23:17 node01 bash[1969]: 10: (LogMonitor::update_from_paxos(bool*)+0x54) [0x55be1dc27334] Dec 05 15:23:17 node01 bash[1969]: 11: (Monitor::refresh_from_paxos(bool*)+0x104) [0x55be1db9ae74] Dec 05 15:23:17 node01 bash[1969]: 12: (Monitor::preinit()+0x95d) [0x55be1dbc93ad] Dec 05 15:23:17 node01 bash[1969]: 13: main() Dec 05 15:23:17 node01 bash[1969]: 14: __libc_start_main() Dec 05 15:23:17 node01 bash[1969]: 15: _start() Dec 05 15:23:17 node01 bash[1969]: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. Dec 05 15:23:17 node01 bash[1969]: --- begin dump of recent events --- Dec 05 15:23:17 node01 bash[1969]: debug -290> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command assert hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -289> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command abort hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -288> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command leak_some_memory hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -287> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command perfcounters_dump hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -286> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command 1 hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -285> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command perf dump hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -284> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command perfcounters_schema hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -283> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command perf histogram dump hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -282> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command 2 hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -281> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command perf schema hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -280> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command perf histogram schema hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -279> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command perf reset hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -278> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command config show hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -277> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command config help hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -276> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command config set hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -275> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command config unset hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -274> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command config get hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -273> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command config diff hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -272> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command config diff get hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -271> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command injectargs hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -270> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command log flush hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -269> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command log dump hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -268> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command log reopen hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -267> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command dump_mempools hook 0x55be2168a068 Dec 05 15:23:17 node01 bash[1969]: debug -266> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 0 set uid:gid to 167:167 (ceph:ceph) Dec 05 15:23:17 node01 bash[1969]: debug -265> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 0 ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable), process ceph-mon, pid 7 Dec 05 15:23:17 node01 bash[1969]: debug -264> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 0 pidfile_write: ignore empty --pid-file Dec 05 15:23:17 node01 bash[1969]: debug -263> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 5 asok(0x55be20bc0000) init /var/run/ceph/ceph-mon.node01.asok Dec 05 15:23:17 node01 bash[1969]: debug -262> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 5 asok(0x55be20bc0000) bind_and_listen /var/run/ceph/ceph-mon.node01.asok Dec 05 15:23:17 node01 bash[1969]: debug -261> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command 0 hook 0x55be2096b030 Dec 05 15:23:17 node01 bash[1969]: debug -260> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command version hook 0x55be2096b030 Dec 05 15:23:17 node01 bash[1969]: debug -259> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command git_version hook 0x55be2096b030 Dec 05 15:23:17 node01 bash[1969]: debug -258> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command help hook 0x55be208de5b0 Dec 05 15:23:17 node01 bash[1969]: debug -257> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command get_command_descriptions hook 0x55be208de5a0 Dec 05 15:23:17 node01 bash[1969]: debug -256> 2023-12-05T13:23:17.639+0000 7fab076ba700 5 asok(0x55be20bc0000) entry start Dec 05 15:23:17 node01 bash[1969]: debug -255> 2023-12-05T13:23:17.667+0000 7fab0e25bb80 0 load: jerasure load: lrc Dec 05 15:23:17 node01 bash[1969]: debug -254> 2023-12-05T13:23:17.671+0000 7fab0e25bb80 1 set rocksdb option level_compaction_dynamic_level_bytes = true Dec 05 15:23:17 node01 bash[1969]: debug -253> 2023-12-05T13:23:17.671+0000 7fab0e25bb80 1 set rocksdb option write_buffer_size = 33554432 Dec 05 15:23:17 node01 bash[1969]: debug -252> 2023-12-05T13:23:17.671+0000 7fab0e25bb80 1 set rocksdb option compression = kNoCompression Dec 05 15:23:17 node01 bash[1969]: debug -251> 2023-12-05T13:23:17.671+0000 7fab0e25bb80 1 set rocksdb option level_compaction_dynamic_level_bytes = true Dec 05 15:23:17 node01 bash[1969]: debug -250> 2023-12-05T13:23:17.671+0000 7fab0e25bb80 1 set rocksdb option write_buffer_size = 33554432 Dec 05 15:23:17 node01 bash[1969]: debug -249> 2023-12-05T13:23:17.671+0000 7fab0e25bb80 1 set rocksdb option compression = kNoCompression Dec 05 15:23:17 node01 bash[1969]: debug -248> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 5 rocksdb: verify_sharding column families from rocksdb: [default] Dec 05 15:23:17 node01 bash[1969]: debug -247> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: RocksDB version: 6.15.5 Dec 05 15:23:17 node01 bash[1969]: debug -246> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ Dec 05 15:23:17 node01 bash[1969]: debug -245> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Compile date Oct 25 2023 Dec 05 15:23:17 node01 bash[1969]: debug -244> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: DB SUMMARY Dec 05 15:23:17 node01 bash[1969]: debug -243> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: DB Session ID: 69YYUY1IJK4BA9QQDDK8 Dec 05 15:23:17 node01 bash[1969]: debug -242> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: CURRENT file: CURRENT Dec 05 15:23:17 node01 bash[1969]: debug -241> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: IDENTITY file: IDENTITY Dec 05 15:23:17 node01 bash[1969]: debug -240> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: MANIFEST file: MANIFEST-000226 size: 308 Bytes Dec 05 15:23:17 node01 bash[1969]: debug -239> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-node01/store.db dir, Total Num: 3, files: 000091.sst 000094.sst 000100.sst Dec 05 15:23:17 node01 bash[1969]: debug -238> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-node01/store.db: 000227.log size: 0 ; Dec 05 15:23:17 node01 bash[1969]: debug -237> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.error_if_exists: 0 Dec 05 15:23:17 node01 bash[1969]: debug -236> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.create_if_missing: 0 Dec 05 15:23:17 node01 bash[1969]: debug -235> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.paranoid_checks: 1 Dec 05 15:23:17 node01 bash[1969]: debug -234> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 Dec 05 15:23:17 node01 bash[1969]: debug -233> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.env: 0x55be1e3aea80 Dec 05 15:23:17 node01 bash[1969]: debug -232> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.fs: Posix File System Dec 05 15:23:17 node01 bash[1969]: debug -231> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.info_log: 0x55be20918360 Dec 05 15:23:17 node01 bash[1969]: debug -230> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_file_opening_threads: 16 Dec 05 15:23:17 node01 bash[1969]: debug -229> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.statistics: (nil) Dec 05 15:23:17 node01 bash[1969]: debug -228> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.use_fsync: 0 Dec 05 15:23:17 node01 bash[1969]: debug -227> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_log_file_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug -226> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_manifest_file_size: 1073741824 Dec 05 15:23:17 node01 bash[1969]: debug -225> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.log_file_time_to_roll: 0 Dec 05 15:23:17 node01 bash[1969]: debug -224> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.keep_log_file_num: 1000 Dec 05 15:23:17 node01 bash[1969]: debug -223> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.recycle_log_file_num: 0 Dec 05 15:23:17 node01 bash[1969]: debug -222> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_fallocate: 1 Dec 05 15:23:17 node01 bash[1969]: debug -221> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_mmap_reads: 0 Dec 05 15:23:17 node01 bash[1969]: debug -220> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_mmap_writes: 0 Dec 05 15:23:17 node01 bash[1969]: debug -219> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.use_direct_reads: 0 Dec 05 15:23:17 node01 bash[1969]: debug -218> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 Dec 05 15:23:17 node01 bash[1969]: debug -217> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.create_missing_column_families: 0 Dec 05 15:23:17 node01 bash[1969]: debug -216> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.db_log_dir: Dec 05 15:23:17 node01 bash[1969]: debug -215> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-node01/store.db Dec 05 15:23:17 node01 bash[1969]: debug -214> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.table_cache_numshardbits: 6 Dec 05 15:23:17 node01 bash[1969]: debug -213> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.WAL_ttl_seconds: 0 Dec 05 15:23:17 node01 bash[1969]: debug -212> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.WAL_size_limit_MB: 0 Dec 05 15:23:17 node01 bash[1969]: debug -211> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 Dec 05 15:23:17 node01 bash[1969]: debug -210> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.manifest_preallocation_size: 4194304 Dec 05 15:23:17 node01 bash[1969]: debug -209> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.is_fd_close_on_exec: 1 Dec 05 15:23:17 node01 bash[1969]: debug -208> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.advise_random_on_open: 1 Dec 05 15:23:17 node01 bash[1969]: debug -207> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.db_write_buffer_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug -206> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.write_buffer_manager: 0x55be2168c990 Dec 05 15:23:17 node01 bash[1969]: debug -205> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.access_hint_on_compaction_start: 1 Dec 05 15:23:17 node01 bash[1969]: debug -204> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 Dec 05 15:23:17 node01 bash[1969]: debug -203> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 Dec 05 15:23:17 node01 bash[1969]: debug -202> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.use_adaptive_mutex: 0 Dec 05 15:23:17 node01 bash[1969]: debug -201> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.rate_limiter: (nil) Dec 05 15:23:17 node01 bash[1969]: debug -200> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 Dec 05 15:23:17 node01 bash[1969]: debug -199> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.wal_recovery_mode: 2 Dec 05 15:23:17 node01 bash[1969]: debug -198> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.enable_thread_tracking: 0 Dec 05 15:23:17 node01 bash[1969]: debug -197> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.enable_pipelined_write: 0 Dec 05 15:23:17 node01 bash[1969]: debug -196> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.unordered_write: 0 Dec 05 15:23:17 node01 bash[1969]: debug -195> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 Dec 05 15:23:17 node01 bash[1969]: debug -194> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 Dec 05 15:23:17 node01 bash[1969]: debug -193> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.write_thread_max_yield_usec: 100 Dec 05 15:23:17 node01 bash[1969]: debug -192> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 Dec 05 15:23:17 node01 bash[1969]: debug -191> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.row_cache: None Dec 05 15:23:17 node01 bash[1969]: debug -190> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.wal_filter: None Dec 05 15:23:17 node01 bash[1969]: debug -189> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.avoid_flush_during_recovery: 0 Dec 05 15:23:17 node01 bash[1969]: debug -188> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_ingest_behind: 0 Dec 05 15:23:17 node01 bash[1969]: debug -187> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.preserve_deletes: 0 Dec 05 15:23:17 node01 bash[1969]: debug -186> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.two_write_queues: 0 Dec 05 15:23:17 node01 bash[1969]: debug -185> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.manual_wal_flush: 0 Dec 05 15:23:17 node01 bash[1969]: debug -184> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.atomic_flush: 0 Dec 05 15:23:17 node01 bash[1969]: debug -183> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 Dec 05 15:23:17 node01 bash[1969]: debug -182> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.persist_stats_to_disk: 0 Dec 05 15:23:17 node01 bash[1969]: debug -181> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.write_dbid_to_manifest: 0 Dec 05 15:23:17 node01 bash[1969]: debug -180> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.log_readahead_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug -179> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.file_checksum_gen_factory: Unknown Dec 05 15:23:17 node01 bash[1969]: debug -178> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.best_efforts_recovery: 0 Dec 05 15:23:17 node01 bash[1969]: debug -177> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 Dec 05 15:23:17 node01 bash[1969]: debug -176> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 Dec 05 15:23:17 node01 bash[1969]: debug -175> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_data_in_errors: 0 Dec 05 15:23:17 node01 bash[1969]: debug -174> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.db_host_id: __hostname__ Dec 05 15:23:17 node01 bash[1969]: debug -173> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_background_jobs: 2 Dec 05 15:23:17 node01 bash[1969]: debug -172> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_background_compactions: -1 Dec 05 15:23:17 node01 bash[1969]: debug -171> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_subcompactions: 1 Dec 05 15:23:17 node01 bash[1969]: debug -170> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 Dec 05 15:23:17 node01 bash[1969]: debug -169> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 Dec 05 15:23:17 node01 bash[1969]: debug -168> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.delayed_write_rate : 16777216 Dec 05 15:23:17 node01 bash[1969]: debug -167> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_total_wal_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug -166> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 Dec 05 15:23:17 node01 bash[1969]: debug -165> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.stats_dump_period_sec: 600 Dec 05 15:23:17 node01 bash[1969]: debug -164> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.stats_persist_period_sec: 600 Dec 05 15:23:17 node01 bash[1969]: debug -163> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.stats_history_buffer_size: 1048576 Dec 05 15:23:17 node01 bash[1969]: debug -162> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_open_files: -1 Dec 05 15:23:17 node01 bash[1969]: debug -161> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bytes_per_sync: 0 Dec 05 15:23:17 node01 bash[1969]: debug -160> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.wal_bytes_per_sync: 0 Dec 05 15:23:17 node01 bash[1969]: debug -159> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.strict_bytes_per_sync: 0 Dec 05 15:23:17 node01 bash[1969]: debug -158> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_readahead_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug -157> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_background_flushes: -1 Dec 05 15:23:17 node01 bash[1969]: debug -156> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Compression algorithms supported: Dec 05 15:23:17 node01 bash[1969]: debug -155> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kZSTDNotFinalCompression supported: 0 Dec 05 15:23:17 node01 bash[1969]: debug -154> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kZSTD supported: 0 Dec 05 15:23:17 node01 bash[1969]: debug -153> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kXpressCompression supported: 0 Dec 05 15:23:17 node01 bash[1969]: debug -152> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kLZ4HCCompression supported: 1 Dec 05 15:23:17 node01 bash[1969]: debug -151> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kZlibCompression supported: 1 Dec 05 15:23:17 node01 bash[1969]: debug -150> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kSnappyCompression supported: 1 Dec 05 15:23:17 node01 bash[1969]: debug -149> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kLZ4Compression supported: 1 Dec 05 15:23:17 node01 bash[1969]: debug -148> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kBZip2Compression supported: 0 Dec 05 15:23:17 node01 bash[1969]: debug -147> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Fast CRC32 supported: Supported on x86 Dec 05 15:23:17 node01 bash[1969]: debug -146> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-node01/store.db/MANIFEST-000226 Dec 05 15:23:17 node01 bash[1969]: debug -145> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: Dec 05 15:23:17 node01 bash[1969]: debug -144> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator Dec 05 15:23:17 node01 bash[1969]: debug -143> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.merge_operator: Dec 05 15:23:17 node01 bash[1969]: debug -142> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_filter: None Dec 05 15:23:17 node01 bash[1969]: debug -141> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_filter_factory: None Dec 05 15:23:17 node01 bash[1969]: debug -140> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.sst_partitioner_factory: None Dec 05 15:23:17 node01 bash[1969]: debug -139> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.memtable_factory: SkipListFactory Dec 05 15:23:17 node01 bash[1969]: debug -138> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.table_factory: BlockBasedTable Dec 05 15:23:17 node01 bash[1969]: debug -137> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55be2096afb0) Dec 05 15:23:17 node01 bash[1969]: cache_index_and_filter_blocks: 1 Dec 05 15:23:17 node01 bash[1969]: cache_index_and_filter_blocks_with_high_priority: 0 Dec 05 15:23:17 node01 bash[1969]: pin_l0_filter_and_index_blocks_in_cache: 0 Dec 05 15:23:17 node01 bash[1969]: pin_top_level_index_and_filter: 1 Dec 05 15:23:17 node01 bash[1969]: index_type: 0 Dec 05 15:23:17 node01 bash[1969]: data_block_index_type: 0 Dec 05 15:23:17 node01 bash[1969]: index_shortening: 1 Dec 05 15:23:17 node01 bash[1969]: data_block_hash_table_util_ratio: 0.750000 Dec 05 15:23:17 node01 bash[1969]: hash_index_allow_collision: 1 Dec 05 15:23:17 node01 bash[1969]: checksum: 1 Dec 05 15:23:17 node01 bash[1969]: no_block_cache: 0 Dec 05 15:23:17 node01 bash[1969]: block_cache: 0x55be2099d090 Dec 05 15:23:17 node01 bash[1969]: block_cache_name: BinnedLRUCache Dec 05 15:23:17 node01 bash[1969]: block_cache_options: Dec 05 15:23:17 node01 bash[1969]: capacity : 536870912 Dec 05 15:23:17 node01 bash[1969]: num_shard_bits : 4 Dec 05 15:23:17 node01 bash[1969]: strict_capacity_limit : 0 Dec 05 15:23:17 node01 bash[1969]: high_pri_pool_ratio: 0.000 Dec 05 15:23:17 node01 bash[1969]: block_cache_compressed: (nil) Dec 05 15:23:17 node01 bash[1969]: persistent_cache: (nil) Dec 05 15:23:17 node01 bash[1969]: block_size: 4096 Dec 05 15:23:17 node01 bash[1969]: block_size_deviation: 10 Dec 05 15:23:17 node01 bash[1969]: block_restart_interval: 16 Dec 05 15:23:17 node01 bash[1969]: index_block_restart_interval: 1 Dec 05 15:23:17 node01 bash[1969]: metadata_block_size: 4096 Dec 05 15:23:17 node01 bash[1969]: partition_filters: 0 Dec 05 15:23:17 node01 bash[1969]: use_delta_encoding: 1 Dec 05 15:23:17 node01 bash[1969]: filter_policy: rocksdb.BuiltinBloomFilter Dec 05 15:23:17 node01 bash[1969]: whole_key_filtering: 1 Dec 05 15:23:17 node01 bash[1969]: verify_compression: 0 Dec 05 15:23:17 node01 bash[1969]: read_amp_bytes_per_bit: 0 Dec 05 15:23:17 node01 bash[1969]: format_version: 4 Dec 05 15:23:17 node01 bash[1969]: enable_index_compression: 1 Dec 05 15:23:17 node01 bash[1969]: block_align: 0 Dec 05 15:23:17 node01 bash[1969]: debug -136> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.write_buffer_size: 33554432 Dec 05 15:23:17 node01 bash[1969]: debug -135> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_write_buffer_number: 2 Dec 05 15:23:17 node01 bash[1969]: debug -134> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression: NoCompression Dec 05 15:23:17 node01 bash[1969]: debug -133> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression: Disabled Dec 05 15:23:17 node01 bash[1969]: debug -132> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.prefix_extractor: nullptr Dec 05 15:23:17 node01 bash[1969]: debug -131> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr Dec 05 15:23:17 node01 bash[1969]: debug -130> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.num_levels: 7 Dec 05 15:23:17 node01 bash[1969]: debug -129> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 Dec 05 15:23:17 node01 bash[1969]: debug -128> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 Dec 05 15:23:17 node01 bash[1969]: debug -127> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 Dec 05 15:23:17 node01 bash[1969]: debug -126> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 Dec 05 15:23:17 node01 bash[1969]: debug -125> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 Dec 05 15:23:17 node01 bash[1969]: debug -124> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 Dec 05 15:23:17 node01 bash[1969]: debug -123> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 Dec 05 15:23:17 node01 bash[1969]: debug -122> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 Dec 05 15:23:17 node01 bash[1969]: debug -121> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 Dec 05 15:23:17 node01 bash[1969]: debug -120> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.enabled: false Dec 05 15:23:17 node01 bash[1969]: debug -119> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.window_bits: -14 Dec 05 15:23:17 node01 bash[1969]: debug -118> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.level: 32767 Dec 05 15:23:17 node01 bash[1969]: debug -117> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.strategy: 0 Dec 05 15:23:17 node01 bash[1969]: debug -116> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 Dec 05 15:23:17 node01 bash[1969]: debug -115> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 Dec 05 15:23:17 node01 bash[1969]: debug -114> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.parallel_threads: 1 Dec 05 15:23:17 node01 bash[1969]: debug -113> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.enabled: false Dec 05 15:23:17 node01 bash[1969]: debug -112> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 Dec 05 15:23:17 node01 bash[1969]: debug -111> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 Dec 05 15:23:17 node01 bash[1969]: debug -110> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.level0_stop_writes_trigger: 36 Dec 05 15:23:17 node01 bash[1969]: debug -109> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.target_file_size_base: 67108864 Dec 05 15:23:17 node01 bash[1969]: debug -108> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.target_file_size_multiplier: 1 Dec 05 15:23:17 node01 bash[1969]: debug -107> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 Dec 05 15:23:17 node01 bash[1969]: debug -106> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 Dec 05 15:23:17 node01 bash[1969]: debug -105> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 Dec 05 15:23:17 node01 bash[1969]: debug -104> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 Dec 05 15:23:17 node01 bash[1969]: debug -103> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 Dec 05 15:23:17 node01 bash[1969]: debug -102> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 Dec 05 15:23:17 node01 bash[1969]: debug -101> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 Dec 05 15:23:17 node01 bash[1969]: debug -100> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 Dec 05 15:23:17 node01 bash[1969]: debug -99> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 Dec 05 15:23:17 node01 bash[1969]: debug -98> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 Dec 05 15:23:17 node01 bash[1969]: debug -97> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 Dec 05 15:23:17 node01 bash[1969]: debug -96> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_compaction_bytes: 1677721600 Dec 05 15:23:17 node01 bash[1969]: debug -95> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.arena_block_size: 4194304 Dec 05 15:23:17 node01 bash[1969]: debug -94> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 Dec 05 15:23:17 node01 bash[1969]: debug -93> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 Dec 05 15:23:17 node01 bash[1969]: debug -92> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 Dec 05 15:23:17 node01 bash[1969]: debug -91> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.disable_auto_compactions: 0 Dec 05 15:23:17 node01 bash[1969]: debug -90> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel Dec 05 15:23:17 node01 bash[1969]: debug -89> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio Dec 05 15:23:17 node01 bash[1969]: debug -88> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 Dec 05 15:23:17 node01 bash[1969]: debug -87> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 Dec 05 15:23:17 node01 bash[1969]: debug -86> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 Dec 05 15:23:17 node01 bash[1969]: debug -85> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 Dec 05 15:23:17 node01 bash[1969]: debug -84> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 Dec 05 15:23:17 node01 bash[1969]: debug -83> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize Dec 05 15:23:17 node01 bash[1969]: debug -82> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 Dec 05 15:23:17 node01 bash[1969]: debug -81> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 Dec 05 15:23:17 node01 bash[1969]: debug -80> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.table_properties_collectors: Dec 05 15:23:17 node01 bash[1969]: debug -79> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.inplace_update_support: 0 Dec 05 15:23:17 node01 bash[1969]: debug -78> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.inplace_update_num_locks: 10000 Dec 05 15:23:17 node01 bash[1969]: debug -77> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 Dec 05 15:23:17 node01 bash[1969]: debug -76> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.memtable_whole_key_filtering: 0 Dec 05 15:23:17 node01 bash[1969]: debug -75> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.memtable_huge_page_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug -74> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bloom_locality: 0 Dec 05 15:23:17 node01 bash[1969]: debug -73> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_successive_merges: 0 Dec 05 15:23:17 node01 bash[1969]: debug -72> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.optimize_filters_for_hits: 0 Dec 05 15:23:17 node01 bash[1969]: debug -71> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.paranoid_file_checks: 0 Dec 05 15:23:17 node01 bash[1969]: debug -70> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.force_consistency_checks: 1 Dec 05 15:23:17 node01 bash[1969]: debug -69> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.report_bg_io_stats: 0 Dec 05 15:23:17 node01 bash[1969]: debug -68> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.ttl: 2592000 Dec 05 15:23:17 node01 bash[1969]: debug -67> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.periodic_compaction_seconds: 0 Dec 05 15:23:17 node01 bash[1969]: debug -66> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.enable_blob_files: false Dec 05 15:23:17 node01 bash[1969]: debug -65> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.min_blob_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug -64> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.blob_file_size: 268435456 Dec 05 15:23:17 node01 bash[1969]: debug -63> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.blob_compression_type: NoCompression Dec 05 15:23:17 node01 bash[1969]: debug -62> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.enable_blob_garbage_collection: false Dec 05 15:23:17 node01 bash[1969]: debug -61> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 Dec 05 15:23:17 node01 bash[1969]: debug -60> 2023-12-05T13:23:17.683+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:3457] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. Dec 05 15:23:17 node01 bash[1969]: debug -59> 2023-12-05T13:23:17.683+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-node01/store.db/MANIFEST-000226 succeeded,manifest_file_number is 226, next_file_number is 228, last_sequence is 21201, log_number is 223,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 Dec 05 15:23:17 node01 bash[1969]: debug -58> 2023-12-05T13:23:17.683+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 223 Dec 05 15:23:17 node01 bash[1969]: debug -57> 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:4082] Creating manifest 230 Dec 05 15:23:17 node01 bash[1969]: debug -56> 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:3457] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. Dec 05 15:23:17 node01 bash[1969]: debug -55> 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1701782597693048, "job": 1, "event": "recovery_started", "wal_files": [227]} Dec 05 15:23:17 node01 bash[1969]: debug -54> 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #227 mode 2 Dec 05 15:23:17 node01 bash[1969]: debug -53> 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:4082] Creating manifest 231 Dec 05 15:23:17 node01 bash[1969]: debug -52> 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:3457] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. Dec 05 15:23:17 node01 bash[1969]: debug -51> 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1701782597693989, "job": 1, "event": "recovery_finished"} Dec 05 15:23:17 node01 bash[1969]: debug -50> 2023-12-05T13:23:17.691+0000 7fab0e25bb80 4 rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-node01/store.db/000227.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 Dec 05 15:23:17 node01 bash[1969]: debug -49> 2023-12-05T13:23:17.691+0000 7fab0e25bb80 4 rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55be20931180 Dec 05 15:23:17 node01 bash[1969]: debug -48> 2023-12-05T13:23:17.691+0000 7fab0e25bb80 4 rocksdb: DB pointer 0x55be20a06000 Dec 05 15:23:17 node01 bash[1969]: debug -47> 2023-12-05T13:23:17.695+0000 7faafe4ad700 4 rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- Dec 05 15:23:17 node01 bash[1969]: debug -46> 2023-12-05T13:23:17.695+0000 7faafe4ad700 4 rocksdb: [db/db_impl/db_impl.cc:903] Dec 05 15:23:17 node01 bash[1969]: ** DB Stats ** Dec 05 15:23:17 node01 bash[1969]: Uptime(secs): 0.0 total, 0.0 interval Dec 05 15:23:17 node01 bash[1969]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s Dec 05 15:23:17 node01 bash[1969]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Dec 05 15:23:17 node01 bash[1969]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Dec 05 15:23:17 node01 bash[1969]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s Dec 05 15:23:17 node01 bash[1969]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s Dec 05 15:23:17 node01 bash[1969]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent Dec 05 15:23:17 node01 bash[1969]: ** Compaction Stats [default] ** Dec 05 15:23:17 node01 bash[1969]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Dec 05 15:23:17 node01 bash[1969]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Dec 05 15:23:17 node01 bash[1969]: L0 2/0 7.21 MB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: L6 1/0 7.22 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: Sum 3/0 14.43 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: ** Compaction Stats [default] ** Dec 05 15:23:17 node01 bash[1969]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Dec 05 15:23:17 node01 bash[1969]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Dec 05 15:23:17 node01 bash[1969]: Uptime(secs): 0.0 total, 0.0 interval Dec 05 15:23:17 node01 bash[1969]: Flush(GB): cumulative 0.000, interval 0.000 Dec 05 15:23:17 node01 bash[1969]: AddFile(GB): cumulative 0.000, interval 0.000 Dec 05 15:23:17 node01 bash[1969]: AddFile(Total Files): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: AddFile(L0 Files): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: AddFile(Keys): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Dec 05 15:23:17 node01 bash[1969]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Dec 05 15:23:17 node01 bash[1969]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Dec 05 15:23:17 node01 bash[1969]: ** File Read Latency Histogram By Level [default] ** Dec 05 15:23:17 node01 bash[1969]: ** Compaction Stats [default] ** Dec 05 15:23:17 node01 bash[1969]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Dec 05 15:23:17 node01 bash[1969]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Dec 05 15:23:17 node01 bash[1969]: L0 2/0 7.21 MB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: L6 1/0 7.22 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: Sum 3/0 14.43 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: ** Compaction Stats [default] ** Dec 05 15:23:17 node01 bash[1969]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Dec 05 15:23:17 node01 bash[1969]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Dec 05 15:23:17 node01 bash[1969]: Uptime(secs): 0.0 total, 0.0 interval Dec 05 15:23:17 node01 bash[1969]: Flush(GB): cumulative 0.000, interval 0.000 Dec 05 15:23:17 node01 bash[1969]: AddFile(GB): cumulative 0.000, interval 0.000 Dec 05 15:23:17 node01 bash[1969]: AddFile(Total Files): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: AddFile(L0 Files): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: AddFile(Keys): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Dec 05 15:23:17 node01 bash[1969]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Dec 05 15:23:17 node01 bash[1969]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Dec 05 15:23:17 node01 bash[1969]: ** File Read Latency Histogram By Level [default] ** Dec 05 15:23:17 node01 bash[1969]: debug -45> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -44> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -43> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -42> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding auth protocol: none Dec 05 15:23:17 node01 bash[1969]: debug -41> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -40> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -39> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -38> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -37> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -36> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -35> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -34> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -33> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -32> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -31> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -30> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -29> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mon/ceph-node01/keyring Dec 05 15:23:17 node01 bash[1969]: debug -28> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 0 starting mon.node01 rank 0 at public addrs [v2:10.40.99.11:3300/0,v1:10.40.99.11:6789/0] at bind addrs [v2:10.40.99.11:3300/0,v1:10.40.99.11:6789/0] mon_data /var/lib/ceph/mon/ceph-node01 fsid be4304e4-b0d5-11ec-8c6a-2965d4229f37 Dec 05 15:23:17 node01 bash[1969]: debug -27> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -26> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -25> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -24> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding auth protocol: none Dec 05 15:23:17 node01 bash[1969]: debug -23> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -22> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -21> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -20> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -19> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -18> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -17> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -16> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -15> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -14> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -13> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -12> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -11> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mon/ceph-node01/keyring Dec 05 15:23:17 node01 bash[1969]: debug -10> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -9> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -8> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201) Dec 05 15:23:17 node01 bash[1969]: debug -7> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201) Dec 05 15:23:17 node01 bash[1969]: debug -6> 2023-12-05T13:23:17.723+0000 7fab0e25bb80 1 mon.node01@-1(???) e0 preinit fsid be4304e4-b0d5-11ec-8c6a-2965d4229f37 Dec 05 15:23:17 node01 bash[1969]: debug -5> 2023-12-05T13:23:17.723+0000 7fab0e25bb80 5 mon.node01@-1(???).mds e0 Unable to load 'last_metadata' Dec 05 15:23:17 node01 bash[1969]: debug -4> 2023-12-05T13:23:17.731+0000 7fab0e25bb80 0 mon.node01@-1(???).osd e9898 crush map has features 3314933000854323200, adjusting msgr requires Dec 05 15:23:17 node01 bash[1969]: debug -3> 2023-12-05T13:23:17.731+0000 7fab0e25bb80 0 mon.node01@-1(???).osd e9898 crush map has features 432629239337189376, adjusting msgr requires Dec 05 15:23:17 node01 bash[1969]: debug -2> 2023-12-05T13:23:17.731+0000 7fab0e25bb80 0 mon.node01@-1(???).osd e9898 crush map has features 432629239337189376, adjusting msgr requires Dec 05 15:23:17 node01 bash[1969]: debug -1> 2023-12-05T13:23:17.731+0000 7fab0e25bb80 0 mon.node01@-1(???).osd e9898 crush map has features 432629239337189376, adjusting msgr requires Dec 05 15:23:17 node01 bash[1969]: debug 0> 2023-12-05T13:23:17.735+0000 7fab0e25bb80 -1 *** Caught signal (Aborted) ** Dec 05 15:23:17 node01 bash[1969]: in thread 7fab0e25bb80 thread_name:ceph-mon Dec 05 15:23:17 node01 bash[1969]: ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) Dec 05 15:23:17 node01 bash[1969]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fab0b6c8cf0] Dec 05 15:23:17 node01 bash[1969]: 2: gsignal() Dec 05 15:23:17 node01 bash[1969]: 3: abort() Dec 05 15:23:17 node01 bash[1969]: 4: /lib64/libstdc++.so.6(+0x9009b) [0x7fab0acda09b] Dec 05 15:23:17 node01 bash[1969]: 5: /lib64/libstdc++.so.6(+0x9654c) [0x7fab0ace054c] Dec 05 15:23:17 node01 bash[1969]: 6: /lib64/libstdc++.so.6(+0x965a7) [0x7fab0ace05a7] Dec 05 15:23:17 node01 bash[1969]: 7: /lib64/libstdc++.so.6(+0x96808) [0x7fab0ace0808] Dec 05 15:23:17 node01 bash[1969]: 8: /lib64/libstdc++.so.6(+0x91ff9) [0x7fab0acdbff9] Dec 05 15:23:17 node01 bash[1969]: 9: (LogMonitor::log_external_backlog()+0xe70) [0x55be1dc27190] Dec 05 15:23:17 node01 bash[1969]: 10: (LogMonitor::update_from_paxos(bool*)+0x54) [0x55be1dc27334] Dec 05 15:23:17 node01 bash[1969]: 11: (Monitor::refresh_from_paxos(bool*)+0x104) [0x55be1db9ae74] Dec 05 15:23:17 node01 bash[1969]: 12: (Monitor::preinit()+0x95d) [0x55be1dbc93ad] Dec 05 15:23:17 node01 bash[1969]: 13: main() Dec 05 15:23:17 node01 bash[1969]: 14: __libc_start_main() Dec 05 15:23:17 node01 bash[1969]: 15: _start() Dec 05 15:23:17 node01 bash[1969]: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. Dec 05 15:23:17 node01 bash[1969]: --- logging levels --- Dec 05 15:23:17 node01 bash[1969]: 0/ 5 none Dec 05 15:23:17 node01 bash[1969]: 0/ 1 lockdep Dec 05 15:23:17 node01 bash[1969]: 0/ 1 context Dec 05 15:23:17 node01 bash[1969]: 1/ 1 crush Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mds Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mds_balancer Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mds_locker Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mds_log Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mds_log_expire Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mds_migrator Dec 05 15:23:17 node01 bash[1969]: 0/ 1 buffer Dec 05 15:23:17 node01 bash[1969]: 0/ 1 timer Dec 05 15:23:17 node01 bash[1969]: 0/ 1 filer Dec 05 15:23:17 node01 bash[1969]: 0/ 1 striper Dec 05 15:23:17 node01 bash[1969]: 0/ 1 objecter Dec 05 15:23:17 node01 bash[1969]: 0/ 5 rados Dec 05 15:23:17 node01 bash[1969]: 0/ 5 rbd Dec 05 15:23:17 node01 bash[1969]: 0/ 5 rbd_mirror Dec 05 15:23:17 node01 bash[1969]: 0/ 5 rbd_replay Dec 05 15:23:17 node01 bash[1969]: 0/ 5 rbd_pwl Dec 05 15:23:17 node01 bash[1969]: 0/ 5 journaler Dec 05 15:23:17 node01 bash[1969]: 0/ 5 objectcacher Dec 05 15:23:17 node01 bash[1969]: 0/ 5 immutable_obj_cache Dec 05 15:23:17 node01 bash[1969]: 0/ 5 client Dec 05 15:23:17 node01 bash[1969]: 1/ 5 osd Dec 05 15:23:17 node01 bash[1969]: 0/ 5 optracker Dec 05 15:23:17 node01 bash[1969]: 0/ 5 objclass Dec 05 15:23:17 node01 bash[1969]: 1/ 3 filestore Dec 05 15:23:17 node01 bash[1969]: 1/ 3 journal Dec 05 15:23:17 node01 bash[1969]: 0/ 0 ms Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mon Dec 05 15:23:17 node01 bash[1969]: 0/10 monc Dec 05 15:23:17 node01 bash[1969]: 1/ 5 paxos Dec 05 15:23:17 node01 bash[1969]: 0/ 5 tp Dec 05 15:23:17 node01 bash[1969]: 1/ 5 auth Dec 05 15:23:17 node01 bash[1969]: 1/ 5 crypto Dec 05 15:23:17 node01 bash[1969]: 1/ 1 finisher Dec 05 15:23:17 node01 bash[1969]: 1/ 1 reserver Dec 05 15:23:17 node01 bash[1969]: 1/ 5 heartbeatmap Dec 05 15:23:17 node01 bash[1969]: 1/ 5 perfcounter Dec 05 15:23:17 node01 bash[1969]: 1/ 5 rgw Dec 05 15:23:17 node01 bash[1969]: 1/ 5 rgw_sync Dec 05 15:23:17 node01 bash[1969]: 1/ 5 rgw_datacache Dec 05 15:23:17 node01 bash[1969]: 1/10 civetweb Dec 05 15:23:17 node01 bash[1969]: 1/ 5 javaclient Dec 05 15:23:17 node01 bash[1969]: 1/ 5 asok Dec 05 15:23:17 node01 bash[1969]: 1/ 1 throttle Dec 05 15:23:17 node01 bash[1969]: 0/ 0 refs Dec 05 15:23:17 node01 bash[1969]: 1/ 5 compressor Dec 05 15:23:17 node01 bash[1969]: 1/ 5 bluestore Dec 05 15:23:17 node01 bash[1969]: 1/ 5 bluefs Dec 05 15:23:17 node01 bash[1969]: 1/ 3 bdev Dec 05 15:23:17 node01 bash[1969]: 1/ 5 kstore Dec 05 15:23:17 node01 bash[1969]: 4/ 5 rocksdb Dec 05 15:23:17 node01 bash[1969]: 4/ 5 leveldb Dec 05 15:23:17 node01 bash[1969]: 4/ 5 memdb Dec 05 15:23:17 node01 bash[1969]: 1/ 5 fuse Dec 05 15:23:17 node01 bash[1969]: 2/ 5 mgr Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mgrc Dec 05 15:23:17 node01 bash[1969]: 1/ 5 dpdk Dec 05 15:23:17 node01 bash[1969]: 1/ 5 eventtrace Dec 05 15:23:17 node01 bash[1969]: 1/ 5 prioritycache Dec 05 15:23:17 node01 bash[1969]: 0/ 5 test Dec 05 15:23:17 node01 bash[1969]: 0/ 5 cephfs_mirror Dec 05 15:23:17 node01 bash[1969]: 0/ 5 cephsqlite Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_onode Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_odata Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_omap Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_tm Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_cleaner Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_lba Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_cache Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_journal Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_device Dec 05 15:23:17 node01 bash[1969]: 0/ 5 alienstore Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mclock Dec 05 15:23:17 node01 bash[1969]: 1/ 5 ceph_exporter Dec 05 15:23:17 node01 bash[1969]: -2/-2 (syslog threshold) Dec 05 15:23:17 node01 bash[1969]: 99/99 (stderr threshold) Dec 05 15:23:17 node01 bash[1969]: --- pthread ID / name mapping for recent threads --- Dec 05 15:23:17 node01 bash[1969]: 7faafe4ad700 / ceph-mon Dec 05 15:23:17 node01 bash[1969]: 7fab076ba700 / admin_socket Dec 05 15:23:17 node01 bash[1969]: 7fab0e25bb80 / ceph-mon Dec 05 15:23:17 node01 bash[1969]: max_recent 10000 Dec 05 15:23:17 node01 bash[1969]: max_new 10000 Dec 05 15:23:17 node01 bash[1969]: log_file Dec 05 15:23:17 node01 bash[1969]: --- end dump of recent events --- Dec 05 15:23:17 node01 bash[1969]: --- begin dump of recent events --- Dec 05 15:23:17 node01 bash[1969]: debug -290> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command assert hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -289> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command abort hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -288> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command leak_some_memory hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -287> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command perfcounters_dump hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -286> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command 1 hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -285> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command perf dump hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -284> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command perfcounters_schema hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -283> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command perf histogram dump hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -282> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command 2 hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -281> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command perf schema hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -280> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command perf histogram schema hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -279> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command perf reset hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -278> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command config show hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -277> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command config help hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -276> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command config set hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -275> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command config unset hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -274> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command config get hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -273> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command config diff hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -272> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command config diff get hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -271> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command injectargs hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -270> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command log flush hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -269> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command log dump hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -268> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command log reopen hook 0x55be208de8b0 Dec 05 15:23:17 node01 bash[1969]: debug -267> 2023-12-05T13:23:17.623+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command dump_mempools hook 0x55be2168a068 Dec 05 15:23:17 node01 bash[1969]: debug -266> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 0 set uid:gid to 167:167 (ceph:ceph) Dec 05 15:23:17 node01 bash[1969]: debug -265> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 0 ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable), process ceph-mon, pid 7 Dec 05 15:23:17 node01 bash[1969]: debug -264> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 0 pidfile_write: ignore empty --pid-file Dec 05 15:23:17 node01 bash[1969]: debug -263> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 5 asok(0x55be20bc0000) init /var/run/ceph/ceph-mon.node01.asok Dec 05 15:23:17 node01 bash[1969]: debug -262> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 5 asok(0x55be20bc0000) bind_and_listen /var/run/ceph/ceph-mon.node01.asok Dec 05 15:23:17 node01 bash[1969]: debug -261> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command 0 hook 0x55be2096b030 Dec 05 15:23:17 node01 bash[1969]: debug -260> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command version hook 0x55be2096b030 Dec 05 15:23:17 node01 bash[1969]: debug -259> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command git_version hook 0x55be2096b030 Dec 05 15:23:17 node01 bash[1969]: debug -258> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command help hook 0x55be208de5b0 Dec 05 15:23:17 node01 bash[1969]: debug -257> 2023-12-05T13:23:17.639+0000 7fab0e25bb80 5 asok(0x55be20bc0000) register_command get_command_descriptions hook 0x55be208de5a0 Dec 05 15:23:17 node01 bash[1969]: debug -256> 2023-12-05T13:23:17.639+0000 7fab076ba700 5 asok(0x55be20bc0000) entry start Dec 05 15:23:17 node01 bash[1969]: debug -255> 2023-12-05T13:23:17.667+0000 7fab0e25bb80 0 load: jerasure load: lrc Dec 05 15:23:17 node01 bash[1969]: debug -254> 2023-12-05T13:23:17.671+0000 7fab0e25bb80 1 set rocksdb option level_compaction_dynamic_level_bytes = true Dec 05 15:23:17 node01 bash[1969]: debug -253> 2023-12-05T13:23:17.671+0000 7fab0e25bb80 1 set rocksdb option write_buffer_size = 33554432 Dec 05 15:23:17 node01 bash[1969]: debug -252> 2023-12-05T13:23:17.671+0000 7fab0e25bb80 1 set rocksdb option compression = kNoCompression Dec 05 15:23:17 node01 bash[1969]: debug -251> 2023-12-05T13:23:17.671+0000 7fab0e25bb80 1 set rocksdb option level_compaction_dynamic_level_bytes = true Dec 05 15:23:17 node01 bash[1969]: debug -250> 2023-12-05T13:23:17.671+0000 7fab0e25bb80 1 set rocksdb option write_buffer_size = 33554432 Dec 05 15:23:17 node01 bash[1969]: debug -249> 2023-12-05T13:23:17.671+0000 7fab0e25bb80 1 set rocksdb option compression = kNoCompression Dec 05 15:23:17 node01 bash[1969]: debug -248> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 5 rocksdb: verify_sharding column families from rocksdb: [default] Dec 05 15:23:17 node01 bash[1969]: debug -247> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: RocksDB version: 6.15.5 Dec 05 15:23:17 node01 bash[1969]: debug -246> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ Dec 05 15:23:17 node01 bash[1969]: debug -245> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Compile date Oct 25 2023 Dec 05 15:23:17 node01 bash[1969]: debug -244> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: DB SUMMARY Dec 05 15:23:17 node01 bash[1969]: debug -243> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: DB Session ID: 69YYUY1IJK4BA9QQDDK8 Dec 05 15:23:17 node01 bash[1969]: debug -242> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: CURRENT file: CURRENT Dec 05 15:23:17 node01 bash[1969]: debug -241> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: IDENTITY file: IDENTITY Dec 05 15:23:17 node01 bash[1969]: debug -240> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: MANIFEST file: MANIFEST-000226 size: 308 Bytes Dec 05 15:23:17 node01 bash[1969]: debug -239> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-node01/store.db dir, Total Num: 3, files: 000091.sst 000094.sst 000100.sst Dec 05 15:23:17 node01 bash[1969]: debug -238> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-node01/store.db: 000227.log size: 0 ; Dec 05 15:23:17 node01 bash[1969]: debug -237> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.error_if_exists: 0 Dec 05 15:23:17 node01 bash[1969]: debug -236> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.create_if_missing: 0 Dec 05 15:23:17 node01 bash[1969]: debug -235> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.paranoid_checks: 1 Dec 05 15:23:17 node01 bash[1969]: debug -234> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 Dec 05 15:23:17 node01 bash[1969]: debug -233> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.env: 0x55be1e3aea80 Dec 05 15:23:17 node01 bash[1969]: debug -232> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.fs: Posix File System Dec 05 15:23:17 node01 bash[1969]: debug -231> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.info_log: 0x55be20918360 Dec 05 15:23:17 node01 bash[1969]: debug -230> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_file_opening_threads: 16 Dec 05 15:23:17 node01 bash[1969]: debug -229> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.statistics: (nil) Dec 05 15:23:17 node01 bash[1969]: debug -228> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.use_fsync: 0 Dec 05 15:23:17 node01 bash[1969]: debug -227> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_log_file_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug -226> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_manifest_file_size: 1073741824 Dec 05 15:23:17 node01 bash[1969]: debug -225> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.log_file_time_to_roll: 0 Dec 05 15:23:17 node01 bash[1969]: debug -224> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.keep_log_file_num: 1000 Dec 05 15:23:17 node01 bash[1969]: debug -223> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.recycle_log_file_num: 0 Dec 05 15:23:17 node01 bash[1969]: debug -222> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_fallocate: 1 Dec 05 15:23:17 node01 bash[1969]: debug -221> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_mmap_reads: 0 Dec 05 15:23:17 node01 bash[1969]: debug -220> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_mmap_writes: 0 Dec 05 15:23:17 node01 bash[1969]: debug -219> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.use_direct_reads: 0 Dec 05 15:23:17 node01 bash[1969]: debug -218> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 Dec 05 15:23:17 node01 bash[1969]: debug -217> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.create_missing_column_families: 0 Dec 05 15:23:17 node01 bash[1969]: debug -216> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.db_log_dir: Dec 05 15:23:17 node01 bash[1969]: debug -215> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-node01/store.db Dec 05 15:23:17 node01 bash[1969]: debug -214> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.table_cache_numshardbits: 6 Dec 05 15:23:17 node01 bash[1969]: debug -213> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.WAL_ttl_seconds: 0 Dec 05 15:23:17 node01 bash[1969]: debug -212> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.WAL_size_limit_MB: 0 Dec 05 15:23:17 node01 bash[1969]: debug -211> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 Dec 05 15:23:17 node01 bash[1969]: debug -210> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.manifest_preallocation_size: 4194304 Dec 05 15:23:17 node01 bash[1969]: debug -209> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.is_fd_close_on_exec: 1 Dec 05 15:23:17 node01 bash[1969]: debug -208> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.advise_random_on_open: 1 Dec 05 15:23:17 node01 bash[1969]: debug -207> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.db_write_buffer_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug -206> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.write_buffer_manager: 0x55be2168c990 Dec 05 15:23:17 node01 bash[1969]: debug -205> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.access_hint_on_compaction_start: 1 Dec 05 15:23:17 node01 bash[1969]: debug -204> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 Dec 05 15:23:17 node01 bash[1969]: debug -203> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 Dec 05 15:23:17 node01 bash[1969]: debug -202> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.use_adaptive_mutex: 0 Dec 05 15:23:17 node01 bash[1969]: debug -201> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.rate_limiter: (nil) Dec 05 15:23:17 node01 bash[1969]: debug -200> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 Dec 05 15:23:17 node01 bash[1969]: debug -199> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.wal_recovery_mode: 2 Dec 05 15:23:17 node01 bash[1969]: debug -198> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.enable_thread_tracking: 0 Dec 05 15:23:17 node01 bash[1969]: debug -197> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.enable_pipelined_write: 0 Dec 05 15:23:17 node01 bash[1969]: debug -196> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.unordered_write: 0 Dec 05 15:23:17 node01 bash[1969]: debug -195> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 Dec 05 15:23:17 node01 bash[1969]: debug -194> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 Dec 05 15:23:17 node01 bash[1969]: debug -193> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.write_thread_max_yield_usec: 100 Dec 05 15:23:17 node01 bash[1969]: debug -192> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 Dec 05 15:23:17 node01 bash[1969]: debug -191> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.row_cache: None Dec 05 15:23:17 node01 bash[1969]: debug -190> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.wal_filter: None Dec 05 15:23:17 node01 bash[1969]: debug -189> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.avoid_flush_during_recovery: 0 Dec 05 15:23:17 node01 bash[1969]: debug -188> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_ingest_behind: 0 Dec 05 15:23:17 node01 bash[1969]: debug -187> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.preserve_deletes: 0 Dec 05 15:23:17 node01 bash[1969]: debug -186> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.two_write_queues: 0 Dec 05 15:23:17 node01 bash[1969]: debug -185> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.manual_wal_flush: 0 Dec 05 15:23:17 node01 bash[1969]: debug -184> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.atomic_flush: 0 Dec 05 15:23:17 node01 bash[1969]: debug -183> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 Dec 05 15:23:17 node01 bash[1969]: debug -182> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.persist_stats_to_disk: 0 Dec 05 15:23:17 node01 bash[1969]: debug -181> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.write_dbid_to_manifest: 0 Dec 05 15:23:17 node01 bash[1969]: debug -180> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.log_readahead_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug -179> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.file_checksum_gen_factory: Unknown Dec 05 15:23:17 node01 bash[1969]: debug -178> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.best_efforts_recovery: 0 Dec 05 15:23:17 node01 bash[1969]: debug -177> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 Dec 05 15:23:17 node01 bash[1969]: debug -176> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 Dec 05 15:23:17 node01 bash[1969]: debug -175> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.allow_data_in_errors: 0 Dec 05 15:23:17 node01 bash[1969]: debug -174> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.db_host_id: __hostname__ Dec 05 15:23:17 node01 bash[1969]: debug -173> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_background_jobs: 2 Dec 05 15:23:17 node01 bash[1969]: debug -172> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_background_compactions: -1 Dec 05 15:23:17 node01 bash[1969]: debug -171> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_subcompactions: 1 Dec 05 15:23:17 node01 bash[1969]: debug -170> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 Dec 05 15:23:17 node01 bash[1969]: debug -169> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 Dec 05 15:23:17 node01 bash[1969]: debug -168> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.delayed_write_rate : 16777216 Dec 05 15:23:17 node01 bash[1969]: debug -167> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_total_wal_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug -166> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 Dec 05 15:23:17 node01 bash[1969]: debug -165> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.stats_dump_period_sec: 600 Dec 05 15:23:17 node01 bash[1969]: debug -164> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.stats_persist_period_sec: 600 Dec 05 15:23:17 node01 bash[1969]: debug -163> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.stats_history_buffer_size: 1048576 Dec 05 15:23:17 node01 bash[1969]: debug -162> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_open_files: -1 Dec 05 15:23:17 node01 bash[1969]: debug -161> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bytes_per_sync: 0 Dec 05 15:23:17 node01 bash[1969]: debug -160> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.wal_bytes_per_sync: 0 Dec 05 15:23:17 node01 bash[1969]: debug -159> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.strict_bytes_per_sync: 0 Dec 05 15:23:17 node01 bash[1969]: debug -158> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_readahead_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug -157> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_background_flushes: -1 Dec 05 15:23:17 node01 bash[1969]: debug -156> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Compression algorithms supported: Dec 05 15:23:17 node01 bash[1969]: debug -155> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kZSTDNotFinalCompression supported: 0 Dec 05 15:23:17 node01 bash[1969]: debug -154> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kZSTD supported: 0 Dec 05 15:23:17 node01 bash[1969]: debug -153> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kXpressCompression supported: 0 Dec 05 15:23:17 node01 bash[1969]: debug -152> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kLZ4HCCompression supported: 1 Dec 05 15:23:17 node01 bash[1969]: debug -151> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kZlibCompression supported: 1 Dec 05 15:23:17 node01 bash[1969]: debug -150> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kSnappyCompression supported: 1 Dec 05 15:23:17 node01 bash[1969]: debug -149> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kLZ4Compression supported: 1 Dec 05 15:23:17 node01 bash[1969]: debug -148> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: kBZip2Compression supported: 0 Dec 05 15:23:17 node01 bash[1969]: debug -147> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Fast CRC32 supported: Supported on x86 Dec 05 15:23:17 node01 bash[1969]: debug -146> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-node01/store.db/MANIFEST-000226 Dec 05 15:23:17 node01 bash[1969]: debug -145> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: Dec 05 15:23:17 node01 bash[1969]: debug -144> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator Dec 05 15:23:17 node01 bash[1969]: debug -143> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.merge_operator: Dec 05 15:23:17 node01 bash[1969]: debug -142> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_filter: None Dec 05 15:23:17 node01 bash[1969]: debug -141> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_filter_factory: None Dec 05 15:23:17 node01 bash[1969]: debug -140> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.sst_partitioner_factory: None Dec 05 15:23:17 node01 bash[1969]: debug -139> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.memtable_factory: SkipListFactory Dec 05 15:23:17 node01 bash[1969]: debug -138> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.table_factory: BlockBasedTable Dec 05 15:23:17 node01 bash[1969]: debug -137> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55be2096afb0) Dec 05 15:23:17 node01 bash[1969]: cache_index_and_filter_blocks: 1 Dec 05 15:23:17 node01 bash[1969]: cache_index_and_filter_blocks_with_high_priority: 0 Dec 05 15:23:17 node01 bash[1969]: pin_l0_filter_and_index_blocks_in_cache: 0 Dec 05 15:23:17 node01 bash[1969]: pin_top_level_index_and_filter: 1 Dec 05 15:23:17 node01 bash[1969]: index_type: 0 Dec 05 15:23:17 node01 bash[1969]: data_block_index_type: 0 Dec 05 15:23:17 node01 bash[1969]: index_shortening: 1 Dec 05 15:23:17 node01 bash[1969]: data_block_hash_table_util_ratio: 0.750000 Dec 05 15:23:17 node01 bash[1969]: hash_index_allow_collision: 1 Dec 05 15:23:17 node01 bash[1969]: checksum: 1 Dec 05 15:23:17 node01 bash[1969]: no_block_cache: 0 Dec 05 15:23:17 node01 bash[1969]: block_cache: 0x55be2099d090 Dec 05 15:23:17 node01 bash[1969]: block_cache_name: BinnedLRUCache Dec 05 15:23:17 node01 bash[1969]: block_cache_options: Dec 05 15:23:17 node01 bash[1969]: capacity : 536870912 Dec 05 15:23:17 node01 bash[1969]: num_shard_bits : 4 Dec 05 15:23:17 node01 bash[1969]: strict_capacity_limit : 0 Dec 05 15:23:17 node01 bash[1969]: high_pri_pool_ratio: 0.000 Dec 05 15:23:17 node01 bash[1969]: block_cache_compressed: (nil) Dec 05 15:23:17 node01 bash[1969]: persistent_cache: (nil) Dec 05 15:23:17 node01 bash[1969]: block_size: 4096 Dec 05 15:23:17 node01 bash[1969]: block_size_deviation: 10 Dec 05 15:23:17 node01 bash[1969]: block_restart_interval: 16 Dec 05 15:23:17 node01 bash[1969]: index_block_restart_interval: 1 Dec 05 15:23:17 node01 bash[1969]: metadata_block_size: 4096 Dec 05 15:23:17 node01 bash[1969]: partition_filters: 0 Dec 05 15:23:17 node01 bash[1969]: use_delta_encoding: 1 Dec 05 15:23:17 node01 bash[1969]: filter_policy: rocksdb.BuiltinBloomFilter Dec 05 15:23:17 node01 bash[1969]: whole_key_filtering: 1 Dec 05 15:23:17 node01 bash[1969]: verify_compression: 0 Dec 05 15:23:17 node01 bash[1969]: read_amp_bytes_per_bit: 0 Dec 05 15:23:17 node01 bash[1969]: format_version: 4 Dec 05 15:23:17 node01 bash[1969]: enable_index_compression: 1 Dec 05 15:23:17 node01 bash[1969]: block_align: 0 Dec 05 15:23:17 node01 bash[1969]: debug -136> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.write_buffer_size: 33554432 Dec 05 15:23:17 node01 bash[1969]: debug -135> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_write_buffer_number: 2 Dec 05 15:23:17 node01 bash[1969]: debug -134> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression: NoCompression Dec 05 15:23:17 node01 bash[1969]: debug -133> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression: Disabled Dec 05 15:23:17 node01 bash[1969]: debug -132> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.prefix_extractor: nullptr Dec 05 15:23:17 node01 bash[1969]: debug -131> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr Dec 05 15:23:17 node01 bash[1969]: debug -130> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.num_levels: 7 Dec 05 15:23:17 node01 bash[1969]: debug -129> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 Dec 05 15:23:17 node01 bash[1969]: debug -128> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 Dec 05 15:23:17 node01 bash[1969]: debug -127> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 Dec 05 15:23:17 node01 bash[1969]: debug -126> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 Dec 05 15:23:17 node01 bash[1969]: debug -125> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 Dec 05 15:23:17 node01 bash[1969]: debug -124> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 Dec 05 15:23:17 node01 bash[1969]: debug -123> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 Dec 05 15:23:17 node01 bash[1969]: debug -122> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 Dec 05 15:23:17 node01 bash[1969]: debug -121> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 Dec 05 15:23:17 node01 bash[1969]: debug -120> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bottommost_compression_opts.enabled: false Dec 05 15:23:17 node01 bash[1969]: debug -119> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.window_bits: -14 Dec 05 15:23:17 node01 bash[1969]: debug -118> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.level: 32767 Dec 05 15:23:17 node01 bash[1969]: debug -117> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.strategy: 0 Dec 05 15:23:17 node01 bash[1969]: debug -116> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 Dec 05 15:23:17 node01 bash[1969]: debug -115> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 Dec 05 15:23:17 node01 bash[1969]: debug -114> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.parallel_threads: 1 Dec 05 15:23:17 node01 bash[1969]: debug -113> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compression_opts.enabled: false Dec 05 15:23:17 node01 bash[1969]: debug -112> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 Dec 05 15:23:17 node01 bash[1969]: debug -111> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 Dec 05 15:23:17 node01 bash[1969]: debug -110> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.level0_stop_writes_trigger: 36 Dec 05 15:23:17 node01 bash[1969]: debug -109> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.target_file_size_base: 67108864 Dec 05 15:23:17 node01 bash[1969]: debug -108> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.target_file_size_multiplier: 1 Dec 05 15:23:17 node01 bash[1969]: debug -107> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 Dec 05 15:23:17 node01 bash[1969]: debug -106> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 Dec 05 15:23:17 node01 bash[1969]: debug -105> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 Dec 05 15:23:17 node01 bash[1969]: debug -104> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 Dec 05 15:23:17 node01 bash[1969]: debug -103> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 Dec 05 15:23:17 node01 bash[1969]: debug -102> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 Dec 05 15:23:17 node01 bash[1969]: debug -101> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 Dec 05 15:23:17 node01 bash[1969]: debug -100> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 Dec 05 15:23:17 node01 bash[1969]: debug -99> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 Dec 05 15:23:17 node01 bash[1969]: debug -98> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 Dec 05 15:23:17 node01 bash[1969]: debug -97> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 Dec 05 15:23:17 node01 bash[1969]: debug -96> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_compaction_bytes: 1677721600 Dec 05 15:23:17 node01 bash[1969]: debug -95> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.arena_block_size: 4194304 Dec 05 15:23:17 node01 bash[1969]: debug -94> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 Dec 05 15:23:17 node01 bash[1969]: debug -93> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 Dec 05 15:23:17 node01 bash[1969]: debug -92> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 Dec 05 15:23:17 node01 bash[1969]: debug -91> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.disable_auto_compactions: 0 Dec 05 15:23:17 node01 bash[1969]: debug -90> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel Dec 05 15:23:17 node01 bash[1969]: debug -89> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio Dec 05 15:23:17 node01 bash[1969]: debug -88> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 Dec 05 15:23:17 node01 bash[1969]: debug -87> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 Dec 05 15:23:17 node01 bash[1969]: debug -86> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 Dec 05 15:23:17 node01 bash[1969]: debug -85> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 Dec 05 15:23:17 node01 bash[1969]: debug -84> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 Dec 05 15:23:17 node01 bash[1969]: debug -83> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize Dec 05 15:23:17 node01 bash[1969]: debug -82> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 Dec 05 15:23:17 node01 bash[1969]: debug -81> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 Dec 05 15:23:17 node01 bash[1969]: debug -80> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.table_properties_collectors: Dec 05 15:23:17 node01 bash[1969]: debug -79> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.inplace_update_support: 0 Dec 05 15:23:17 node01 bash[1969]: debug -78> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.inplace_update_num_locks: 10000 Dec 05 15:23:17 node01 bash[1969]: debug -77> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 Dec 05 15:23:17 node01 bash[1969]: debug -76> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.memtable_whole_key_filtering: 0 Dec 05 15:23:17 node01 bash[1969]: debug -75> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.memtable_huge_page_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug -74> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.bloom_locality: 0 Dec 05 15:23:17 node01 bash[1969]: debug -73> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.max_successive_merges: 0 Dec 05 15:23:17 node01 bash[1969]: debug -72> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.optimize_filters_for_hits: 0 Dec 05 15:23:17 node01 bash[1969]: debug -71> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.paranoid_file_checks: 0 Dec 05 15:23:17 node01 bash[1969]: debug -70> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.force_consistency_checks: 1 Dec 05 15:23:17 node01 bash[1969]: debug -69> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.report_bg_io_stats: 0 Dec 05 15:23:17 node01 bash[1969]: debug -68> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.ttl: 2592000 Dec 05 15:23:17 node01 bash[1969]: debug -67> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.periodic_compaction_seconds: 0 Dec 05 15:23:17 node01 bash[1969]: debug -66> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.enable_blob_files: false Dec 05 15:23:17 node01 bash[1969]: debug -65> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.min_blob_size: 0 Dec 05 15:23:17 node01 bash[1969]: debug -64> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.blob_file_size: 268435456 Dec 05 15:23:17 node01 bash[1969]: debug -63> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.blob_compression_type: NoCompression Dec 05 15:23:17 node01 bash[1969]: debug -62> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.enable_blob_garbage_collection: false Dec 05 15:23:17 node01 bash[1969]: debug -61> 2023-12-05T13:23:17.679+0000 7fab0e25bb80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 Dec 05 15:23:17 node01 bash[1969]: debug -60> 2023-12-05T13:23:17.683+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:3457] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. Dec 05 15:23:17 node01 bash[1969]: debug -59> 2023-12-05T13:23:17.683+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-node01/store.db/MANIFEST-000226 succeeded,manifest_file_number is 226, next_file_number is 228, last_sequence is 21201, log_number is 223,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 Dec 05 15:23:17 node01 bash[1969]: debug -58> 2023-12-05T13:23:17.683+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 223 Dec 05 15:23:17 node01 bash[1969]: debug -57> 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:4082] Creating manifest 230 Dec 05 15:23:17 node01 bash[1969]: debug -56> 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:3457] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. Dec 05 15:23:17 node01 bash[1969]: debug -55> 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1701782597693048, "job": 1, "event": "recovery_started", "wal_files": [227]} Dec 05 15:23:17 node01 bash[1969]: debug -54> 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #227 mode 2 Dec 05 15:23:17 node01 bash[1969]: debug -53> 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:4082] Creating manifest 231 Dec 05 15:23:17 node01 bash[1969]: debug -52> 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: [db/version_set.cc:3457] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. Dec 05 15:23:17 node01 bash[1969]: debug -51> 2023-12-05T13:23:17.687+0000 7fab0e25bb80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1701782597693989, "job": 1, "event": "recovery_finished"} Dec 05 15:23:17 node01 bash[1969]: debug -50> 2023-12-05T13:23:17.691+0000 7fab0e25bb80 4 rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-node01/store.db/000227.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 Dec 05 15:23:17 node01 bash[1969]: debug -49> 2023-12-05T13:23:17.691+0000 7fab0e25bb80 4 rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55be20931180 Dec 05 15:23:17 node01 bash[1969]: debug -48> 2023-12-05T13:23:17.691+0000 7fab0e25bb80 4 rocksdb: DB pointer 0x55be20a06000 Dec 05 15:23:17 node01 bash[1969]: debug -47> 2023-12-05T13:23:17.695+0000 7faafe4ad700 4 rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- Dec 05 15:23:17 node01 bash[1969]: debug -46> 2023-12-05T13:23:17.695+0000 7faafe4ad700 4 rocksdb: [db/db_impl/db_impl.cc:903] Dec 05 15:23:17 node01 bash[1969]: ** DB Stats ** Dec 05 15:23:17 node01 bash[1969]: Uptime(secs): 0.0 total, 0.0 interval Dec 05 15:23:17 node01 bash[1969]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s Dec 05 15:23:17 node01 bash[1969]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Dec 05 15:23:17 node01 bash[1969]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Dec 05 15:23:17 node01 bash[1969]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s Dec 05 15:23:17 node01 bash[1969]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s Dec 05 15:23:17 node01 bash[1969]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent Dec 05 15:23:17 node01 bash[1969]: ** Compaction Stats [default] ** Dec 05 15:23:17 node01 bash[1969]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Dec 05 15:23:17 node01 bash[1969]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Dec 05 15:23:17 node01 bash[1969]: L0 2/0 7.21 MB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: L6 1/0 7.22 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: Sum 3/0 14.43 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: ** Compaction Stats [default] ** Dec 05 15:23:17 node01 bash[1969]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Dec 05 15:23:17 node01 bash[1969]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Dec 05 15:23:17 node01 bash[1969]: Uptime(secs): 0.0 total, 0.0 interval Dec 05 15:23:17 node01 bash[1969]: Flush(GB): cumulative 0.000, interval 0.000 Dec 05 15:23:17 node01 bash[1969]: AddFile(GB): cumulative 0.000, interval 0.000 Dec 05 15:23:17 node01 bash[1969]: AddFile(Total Files): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: AddFile(L0 Files): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: AddFile(Keys): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Dec 05 15:23:17 node01 bash[1969]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Dec 05 15:23:17 node01 bash[1969]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Dec 05 15:23:17 node01 bash[1969]: ** File Read Latency Histogram By Level [default] ** Dec 05 15:23:17 node01 bash[1969]: ** Compaction Stats [default] ** Dec 05 15:23:17 node01 bash[1969]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Dec 05 15:23:17 node01 bash[1969]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Dec 05 15:23:17 node01 bash[1969]: L0 2/0 7.21 MB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: L6 1/0 7.22 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: Sum 3/0 14.43 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Dec 05 15:23:17 node01 bash[1969]: ** Compaction Stats [default] ** Dec 05 15:23:17 node01 bash[1969]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Dec 05 15:23:17 node01 bash[1969]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Dec 05 15:23:17 node01 bash[1969]: Uptime(secs): 0.0 total, 0.0 interval Dec 05 15:23:17 node01 bash[1969]: Flush(GB): cumulative 0.000, interval 0.000 Dec 05 15:23:17 node01 bash[1969]: AddFile(GB): cumulative 0.000, interval 0.000 Dec 05 15:23:17 node01 bash[1969]: AddFile(Total Files): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: AddFile(L0 Files): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: AddFile(Keys): cumulative 0, interval 0 Dec 05 15:23:17 node01 bash[1969]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Dec 05 15:23:17 node01 bash[1969]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Dec 05 15:23:17 node01 bash[1969]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Dec 05 15:23:17 node01 bash[1969]: ** File Read Latency Histogram By Level [default] ** Dec 05 15:23:17 node01 bash[1969]: debug -45> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -44> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -43> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -42> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding auth protocol: none Dec 05 15:23:17 node01 bash[1969]: debug -41> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -40> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -39> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -38> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -37> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -36> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -35> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -34> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -33> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -32> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -31> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -30> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167a140) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -29> 2023-12-05T13:23:17.715+0000 7fab0e25bb80 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mon/ceph-node01/keyring Dec 05 15:23:17 node01 bash[1969]: debug -28> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 0 starting mon.node01 rank 0 at public addrs [v2:10.40.99.11:3300/0,v1:10.40.99.11:6789/0] at bind addrs [v2:10.40.99.11:3300/0,v1:10.40.99.11:6789/0] mon_data /var/lib/ceph/mon/ceph-node01 fsid be4304e4-b0d5-11ec-8c6a-2965d4229f37 Dec 05 15:23:17 node01 bash[1969]: debug -27> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -26> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -25> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -24> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding auth protocol: none Dec 05 15:23:17 node01 bash[1969]: debug -23> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -22> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -21> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -20> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -19> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -18> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -17> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -16> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -15> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -14> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -13> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: crc Dec 05 15:23:17 node01 bash[1969]: debug -12> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 AuthRegistry(0x55be2167aa40) adding con mode: secure Dec 05 15:23:17 node01 bash[1969]: debug -11> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mon/ceph-node01/keyring Dec 05 15:23:17 node01 bash[1969]: debug -10> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -9> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 5 adding auth protocol: cephx Dec 05 15:23:17 node01 bash[1969]: debug -8> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201) Dec 05 15:23:17 node01 bash[1969]: debug -7> 2023-12-05T13:23:17.719+0000 7fab0e25bb80 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201) Dec 05 15:23:17 node01 bash[1969]: debug -6> 2023-12-05T13:23:17.723+0000 7fab0e25bb80 1 mon.node01@-1(???) e0 preinit fsid be4304e4-b0d5-11ec-8c6a-2965d4229f37 Dec 05 15:23:17 node01 bash[1969]: debug -5> 2023-12-05T13:23:17.723+0000 7fab0e25bb80 5 mon.node01@-1(???).mds e0 Unable to load 'last_metadata' Dec 05 15:23:17 node01 bash[1969]: debug -4> 2023-12-05T13:23:17.731+0000 7fab0e25bb80 0 mon.node01@-1(???).osd e9898 crush map has features 3314933000854323200, adjusting msgr requires Dec 05 15:23:17 node01 bash[1969]: debug -3> 2023-12-05T13:23:17.731+0000 7fab0e25bb80 0 mon.node01@-1(???).osd e9898 crush map has features 432629239337189376, adjusting msgr requires Dec 05 15:23:17 node01 bash[1969]: debug -2> 2023-12-05T13:23:17.731+0000 7fab0e25bb80 0 mon.node01@-1(???).osd e9898 crush map has features 432629239337189376, adjusting msgr requires Dec 05 15:23:17 node01 bash[1969]: debug -1> 2023-12-05T13:23:17.731+0000 7fab0e25bb80 0 mon.node01@-1(???).osd e9898 crush map has features 432629239337189376, adjusting msgr requires Dec 05 15:23:17 node01 bash[1969]: debug 0> 2023-12-05T13:23:17.735+0000 7fab0e25bb80 -1 *** Caught signal (Aborted) ** Dec 05 15:23:17 node01 bash[1969]: in thread 7fab0e25bb80 thread_name:ceph-mon Dec 05 15:23:17 node01 bash[1969]: ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) Dec 05 15:23:17 node01 bash[1969]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fab0b6c8cf0] Dec 05 15:23:17 node01 bash[1969]: 2: gsignal() Dec 05 15:23:17 node01 bash[1969]: 3: abort() Dec 05 15:23:17 node01 bash[1969]: 4: /lib64/libstdc++.so.6(+0x9009b) [0x7fab0acda09b] Dec 05 15:23:17 node01 bash[1969]: 5: /lib64/libstdc++.so.6(+0x9654c) [0x7fab0ace054c] Dec 05 15:23:17 node01 bash[1969]: 6: /lib64/libstdc++.so.6(+0x965a7) [0x7fab0ace05a7] Dec 05 15:23:17 node01 bash[1969]: 7: /lib64/libstdc++.so.6(+0x96808) [0x7fab0ace0808] Dec 05 15:23:17 node01 bash[1969]: 8: /lib64/libstdc++.so.6(+0x91ff9) [0x7fab0acdbff9] Dec 05 15:23:17 node01 bash[1969]: 9: (LogMonitor::log_external_backlog()+0xe70) [0x55be1dc27190] Dec 05 15:23:17 node01 bash[1969]: 10: (LogMonitor::update_from_paxos(bool*)+0x54) [0x55be1dc27334] Dec 05 15:23:17 node01 bash[1969]: 11: (Monitor::refresh_from_paxos(bool*)+0x104) [0x55be1db9ae74] Dec 05 15:23:17 node01 bash[1969]: 12: (Monitor::preinit()+0x95d) [0x55be1dbc93ad] Dec 05 15:23:17 node01 bash[1969]: 13: main() Dec 05 15:23:17 node01 bash[1969]: 14: __libc_start_main() Dec 05 15:23:17 node01 bash[1969]: 15: _start() Dec 05 15:23:17 node01 bash[1969]: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. Dec 05 15:23:17 node01 bash[1969]: --- logging levels --- Dec 05 15:23:17 node01 bash[1969]: 0/ 5 none Dec 05 15:23:17 node01 bash[1969]: 0/ 1 lockdep Dec 05 15:23:17 node01 bash[1969]: 0/ 1 context Dec 05 15:23:17 node01 bash[1969]: 1/ 1 crush Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mds Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mds_balancer Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mds_locker Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mds_log Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mds_log_expire Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mds_migrator Dec 05 15:23:17 node01 bash[1969]: 0/ 1 buffer Dec 05 15:23:17 node01 bash[1969]: 0/ 1 timer Dec 05 15:23:17 node01 bash[1969]: 0/ 1 filer Dec 05 15:23:17 node01 bash[1969]: 0/ 1 striper Dec 05 15:23:17 node01 bash[1969]: 0/ 1 objecter Dec 05 15:23:17 node01 bash[1969]: 0/ 5 rados Dec 05 15:23:17 node01 bash[1969]: 0/ 5 rbd Dec 05 15:23:17 node01 bash[1969]: 0/ 5 rbd_mirror Dec 05 15:23:17 node01 bash[1969]: 0/ 5 rbd_replay Dec 05 15:23:17 node01 bash[1969]: 0/ 5 rbd_pwl Dec 05 15:23:17 node01 bash[1969]: 0/ 5 journaler Dec 05 15:23:17 node01 bash[1969]: 0/ 5 objectcacher Dec 05 15:23:17 node01 bash[1969]: 0/ 5 immutable_obj_cache Dec 05 15:23:17 node01 bash[1969]: 0/ 5 client Dec 05 15:23:17 node01 bash[1969]: 1/ 5 osd Dec 05 15:23:17 node01 bash[1969]: 0/ 5 optracker Dec 05 15:23:17 node01 bash[1969]: 0/ 5 objclass Dec 05 15:23:17 node01 bash[1969]: 1/ 3 filestore Dec 05 15:23:17 node01 bash[1969]: 1/ 3 journal Dec 05 15:23:17 node01 bash[1969]: 0/ 0 ms Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mon Dec 05 15:23:17 node01 bash[1969]: 0/10 monc Dec 05 15:23:17 node01 bash[1969]: 1/ 5 paxos Dec 05 15:23:17 node01 bash[1969]: 0/ 5 tp Dec 05 15:23:17 node01 bash[1969]: 1/ 5 auth Dec 05 15:23:17 node01 bash[1969]: 1/ 5 crypto Dec 05 15:23:17 node01 bash[1969]: 1/ 1 finisher Dec 05 15:23:17 node01 bash[1969]: 1/ 1 reserver Dec 05 15:23:17 node01 bash[1969]: 1/ 5 heartbeatmap Dec 05 15:23:17 node01 bash[1969]: 1/ 5 perfcounter Dec 05 15:23:17 node01 bash[1969]: 1/ 5 rgw Dec 05 15:23:17 node01 bash[1969]: 1/ 5 rgw_sync Dec 05 15:23:17 node01 bash[1969]: 1/ 5 rgw_datacache Dec 05 15:23:17 node01 bash[1969]: 1/10 civetweb Dec 05 15:23:17 node01 bash[1969]: 1/ 5 javaclient Dec 05 15:23:17 node01 bash[1969]: 1/ 5 asok Dec 05 15:23:17 node01 bash[1969]: 1/ 1 throttle Dec 05 15:23:17 node01 bash[1969]: 0/ 0 refs Dec 05 15:23:17 node01 bash[1969]: 1/ 5 compressor Dec 05 15:23:17 node01 bash[1969]: 1/ 5 bluestore Dec 05 15:23:17 node01 bash[1969]: 1/ 5 bluefs Dec 05 15:23:17 node01 bash[1969]: 1/ 3 bdev Dec 05 15:23:17 node01 bash[1969]: 1/ 5 kstore Dec 05 15:23:17 node01 bash[1969]: 4/ 5 rocksdb Dec 05 15:23:17 node01 bash[1969]: 4/ 5 leveldb Dec 05 15:23:17 node01 bash[1969]: 4/ 5 memdb Dec 05 15:23:17 node01 bash[1969]: 1/ 5 fuse Dec 05 15:23:17 node01 bash[1969]: 2/ 5 mgr Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mgrc Dec 05 15:23:17 node01 bash[1969]: 1/ 5 dpdk Dec 05 15:23:17 node01 bash[1969]: 1/ 5 eventtrace Dec 05 15:23:17 node01 bash[1969]: 1/ 5 prioritycache Dec 05 15:23:17 node01 bash[1969]: 0/ 5 test Dec 05 15:23:17 node01 bash[1969]: 0/ 5 cephfs_mirror Dec 05 15:23:17 node01 bash[1969]: 0/ 5 cephsqlite Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_onode Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_odata Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_omap Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_tm Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_cleaner Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_lba Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_cache Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_journal Dec 05 15:23:17 node01 bash[1969]: 0/ 5 seastore_device Dec 05 15:23:17 node01 bash[1969]: 0/ 5 alienstore Dec 05 15:23:17 node01 bash[1969]: 1/ 5 mclock Dec 05 15:23:17 node01 bash[1969]: 1/ 5 ceph_exporter Dec 05 15:23:17 node01 bash[1969]: -2/-2 (syslog threshold) Dec 05 15:23:17 node01 bash[1969]: 99/99 (stderr threshold) Dec 05 15:23:17 node01 bash[1969]: --- pthread ID / name mapping for recent threads --- Dec 05 15:23:17 node01 bash[1969]: 7faafe4ad700 / ceph-mon Dec 05 15:23:17 node01 bash[1969]: 7fab076ba700 / admin_socket Dec 05 15:23:17 node01 bash[1969]: 7fab0e25bb80 / ceph-mon Dec 05 15:23:17 node01 bash[1969]: max_recent 10000 Dec 05 15:23:17 node01 bash[1969]: max_new 10000 Dec 05 15:23:17 node01 bash[1969]: log_file /var/lib/ceph/crash/2023-12-05T13:23:17.739483Z_174a5e49-22df-4c2f-be14-4e88cc65b2f2/log Dec 05 15:23:17 node01 bash[1969]: --- end dump of recent events --- Dec 05 15:23:17 node01 systemd[1]: ceph-be4304e4-b0d5-11ec-8c6a-2965d4229f37@mon.node01.service: Main process exited, code=exited, status=134/n/a Dec 05 15:23:18 node01 systemd[1]: ceph-be4304e4-b0d5-11ec-8c6a-2965d4229f37@mon.node01.service: Failed with result 'exit-code'. Dec 05 15:23:29 node01 systemd[1]: ceph-be4304e4-b0d5-11ec-8c6a-2965d4229f37@mon.node01.service: Scheduled restart job, restart counter is at 1. Dec 05 15:23:29 node01 systemd[1]: Stopped Ceph mon.node01 for be4304e4-b0d5-11ec-8c6a-2965d4229f37. Dec 05 15:23:29 node01 systemd[1]: Started Ceph mon.node01 for be4304e4-b0d5-11ec-8c6a-2965d4229f37. Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.299+0000 7fd6a1f76b80 0 set uid:gid to 167:167 (ceph:ceph) Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.299+0000 7fd6a1f76b80 0 ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable), process ceph-mon, pid 7 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.299+0000 7fd6a1f76b80 0 pidfile_write: ignore empty --pid-file Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 0 load: jerasure load: lrc Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: RocksDB version: 6.15.5 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Compile date Oct 25 2023 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: DB SUMMARY Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: DB Session ID: OQS561SAY9M8P46I122B Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: CURRENT file: CURRENT Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: IDENTITY file: IDENTITY Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: MANIFEST file: MANIFEST-000231 size: 308 Bytes Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-node01/store.db dir, Total Num: 3, files: 000091.sst 000094.sst 000100.sst Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-node01/store.db: 000232.log size: 0 ; Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.error_if_exists: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.create_if_missing: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.paranoid_checks: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.env: 0x562241625a80 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.fs: Posix File System Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.info_log: 0x562243b9a360 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_file_opening_threads: 16 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.statistics: (nil) Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.use_fsync: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_log_file_size: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_manifest_file_size: 1073741824 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.log_file_time_to_roll: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.keep_log_file_num: 1000 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.recycle_log_file_num: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.allow_fallocate: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.allow_mmap_reads: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.allow_mmap_writes: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.use_direct_reads: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.create_missing_column_families: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.db_log_dir: Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-node01/store.db Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.table_cache_numshardbits: 6 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.WAL_ttl_seconds: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.WAL_size_limit_MB: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.manifest_preallocation_size: 4194304 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.is_fd_close_on_exec: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.advise_random_on_open: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.db_write_buffer_size: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.write_buffer_manager: 0x56224490e990 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.access_hint_on_compaction_start: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.use_adaptive_mutex: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.rate_limiter: (nil) Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.wal_recovery_mode: 2 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.enable_thread_tracking: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.enable_pipelined_write: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.unordered_write: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.write_thread_max_yield_usec: 100 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.row_cache: None Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.wal_filter: None Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.avoid_flush_during_recovery: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.allow_ingest_behind: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.preserve_deletes: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.two_write_queues: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.manual_wal_flush: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.atomic_flush: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.persist_stats_to_disk: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.write_dbid_to_manifest: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.log_readahead_size: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.file_checksum_gen_factory: Unknown Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.best_efforts_recovery: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.allow_data_in_errors: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.db_host_id: __hostname__ Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_background_jobs: 2 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_background_compactions: -1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_subcompactions: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.delayed_write_rate : 16777216 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_total_wal_size: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.stats_dump_period_sec: 600 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.stats_persist_period_sec: 600 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.stats_history_buffer_size: 1048576 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_open_files: -1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.bytes_per_sync: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.wal_bytes_per_sync: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.strict_bytes_per_sync: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compaction_readahead_size: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_background_flushes: -1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Compression algorithms supported: Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: kZSTDNotFinalCompression supported: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: kZSTD supported: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: kXpressCompression supported: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: kLZ4HCCompression supported: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: kZlibCompression supported: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: kSnappyCompression supported: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: kLZ4Compression supported: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: kBZip2Compression supported: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Fast CRC32 supported: Supported on x86 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-node01/store.db/MANIFEST-000231 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.merge_operator: Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compaction_filter: None Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compaction_filter_factory: None Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.sst_partitioner_factory: None Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.memtable_factory: SkipListFactory Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.table_factory: BlockBasedTable Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562243becfb0) Dec 05 15:23:29 node01 bash[6375]: cache_index_and_filter_blocks: 1 Dec 05 15:23:29 node01 bash[6375]: cache_index_and_filter_blocks_with_high_priority: 0 Dec 05 15:23:29 node01 bash[6375]: pin_l0_filter_and_index_blocks_in_cache: 0 Dec 05 15:23:29 node01 bash[6375]: pin_top_level_index_and_filter: 1 Dec 05 15:23:29 node01 bash[6375]: index_type: 0 Dec 05 15:23:29 node01 bash[6375]: data_block_index_type: 0 Dec 05 15:23:29 node01 bash[6375]: index_shortening: 1 Dec 05 15:23:29 node01 bash[6375]: data_block_hash_table_util_ratio: 0.750000 Dec 05 15:23:29 node01 bash[6375]: hash_index_allow_collision: 1 Dec 05 15:23:29 node01 bash[6375]: checksum: 1 Dec 05 15:23:29 node01 bash[6375]: no_block_cache: 0 Dec 05 15:23:29 node01 bash[6375]: block_cache: 0x562243c1f090 Dec 05 15:23:29 node01 bash[6375]: block_cache_name: BinnedLRUCache Dec 05 15:23:29 node01 bash[6375]: block_cache_options: Dec 05 15:23:29 node01 bash[6375]: capacity : 536870912 Dec 05 15:23:29 node01 bash[6375]: num_shard_bits : 4 Dec 05 15:23:29 node01 bash[6375]: strict_capacity_limit : 0 Dec 05 15:23:29 node01 bash[6375]: high_pri_pool_ratio: 0.000 Dec 05 15:23:29 node01 bash[6375]: block_cache_compressed: (nil) Dec 05 15:23:29 node01 bash[6375]: persistent_cache: (nil) Dec 05 15:23:29 node01 bash[6375]: block_size: 4096 Dec 05 15:23:29 node01 bash[6375]: block_size_deviation: 10 Dec 05 15:23:29 node01 bash[6375]: block_restart_interval: 16 Dec 05 15:23:29 node01 bash[6375]: index_block_restart_interval: 1 Dec 05 15:23:29 node01 bash[6375]: metadata_block_size: 4096 Dec 05 15:23:29 node01 bash[6375]: partition_filters: 0 Dec 05 15:23:29 node01 bash[6375]: use_delta_encoding: 1 Dec 05 15:23:29 node01 bash[6375]: filter_policy: rocksdb.BuiltinBloomFilter Dec 05 15:23:29 node01 bash[6375]: whole_key_filtering: 1 Dec 05 15:23:29 node01 bash[6375]: verify_compression: 0 Dec 05 15:23:29 node01 bash[6375]: read_amp_bytes_per_bit: 0 Dec 05 15:23:29 node01 bash[6375]: format_version: 4 Dec 05 15:23:29 node01 bash[6375]: enable_index_compression: 1 Dec 05 15:23:29 node01 bash[6375]: block_align: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.write_buffer_size: 33554432 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_write_buffer_number: 2 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compression: NoCompression Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.bottommost_compression: Disabled Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.prefix_extractor: nullptr Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.num_levels: 7 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.bottommost_compression_opts.enabled: false Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compression_opts.window_bits: -14 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compression_opts.level: 32767 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compression_opts.strategy: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compression_opts.parallel_threads: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compression_opts.enabled: false Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.level0_stop_writes_trigger: 36 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.target_file_size_base: 67108864 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.target_file_size_multiplier: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_compaction_bytes: 1677721600 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.arena_block_size: 4194304 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.disable_auto_compactions: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.table_properties_collectors: Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.inplace_update_support: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.inplace_update_num_locks: 10000 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.memtable_whole_key_filtering: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.memtable_huge_page_size: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.bloom_locality: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.max_successive_merges: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.optimize_filters_for_hits: 0 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.paranoid_file_checks: Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.force_consistency_checks: 1 Dec 05 15:23:29 node01 bash[6375]: debug 2023-12-05T13:23:29.303+0000 7fd6a1f76b80 4 rocksdb: Options.report_bg_io_stats: 0 Dec 05 15:23:29 [Text gekürzt]