Hi All We had a cluster (v13.2.4) with 32 osds in total. At first, an osd (osd.18) in cluster was down. So, we tried to remove the osd and added a new one (osd.32) with new ID. We unplugged the disk (osd.18) and plugged in a new disk in the same slot and add osd.32 into cluster. Then, osd.32 was booting, but, we found it takes much time (around 18 mins) for the osd to change to up state. Diving into osd.32 logs, we see that there is much rocksdb activity before osd.32 change to up state. Can anyone explain why this happened or give me any advice about how to prevent from this. Thanks. [osd.32 log] 2020-08-03 15:36:58.852 7f88021fa1c0 0 osd.32 0 done with init, starting boot process 2020-08-03 15:36:58.852 7f88021fa1c0 1 osd.32 0 start_boot 2020-08-03 15:36:58.854 7f87db02b700 -1 osd.32 0 waiting for initial osdmap 2020-08-03 15:36:58.855 7f87e4ba0700 -1 osd.32 0 failed to load OSD map for epoch 22010, got 0 bytes 2020-08-03 15:36:58.955 7f87e0836700 0 osd.32 22011 crush map has features 283675107524608, adjusting msgr requires for clients 2020-08-03 15:36:58.955 7f87e0836700 0 osd.32 22011 crush map has features 283675107524608 was 288232575208792577, adjusting msgr requires for mons *2020-08-03 15:36:58.955* 7f87e0836700 0 osd.32 22011 crush map has features 720859615486820352, adjusting msgr requires for osds 2020-08-03 15:37:31.182 7f87e1037700 4 rocksdb: [/home/gitlab/rpmbuild/BUILD/ceph-13.2.4/src/rocksdb/db/db_impl_write.cc:1346] [default] New memtable created with log file: #16. Immutable memtables: 0. 2020-08-03 15:37:31.285 7f87e8045700 4 rocksdb: (Original Log Time 2020/08/03-15:37:31.183995) [/home/gitlab/rpmbuild/BUILD/ceph-13.2.4/src/rocksdb/db/db_impl_compaction_flush.cc:1396] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 2020-08-03 15:37:31.285 7f87e8045700 4 rocksdb: [/home/gitlab/rpmbuild/BUILD/ceph-13.2.4/src/rocksdb/db/flush_job.cc:300] [default] [JOB 3] Flushing memtable with next log file: 16 -------- lots of rocksdb activity--------- 2020-08-03 15:54:21.704 7f87e8045700 4 rocksdb: (Original Log Time 2020/08/03-15:54:21.705680) [/home/gitlab/rpmbuild/BUILD/ceph-13.2.4/src/rocksdb/db/memtable_list.cc:397] [default] Level-0 commit table #112: memtable #1 done 2020-08-03 15:54:21.704 7f87e8045700 4 rocksdb: (Original Log Time 2020/08/03-15:54:21.705704) EVENT_LOG_v1 {"time_micros": 1596441261705697, "job": 51, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [1, 3, 0, 0, 0, 0, 0], "immutable_memtables": 0} 2020-08-03 15:54:21.704 7f87e8045700 4 rocksdb: (Original Log Time 2020/08/03-15:54:21.705721) [/home/gitlab/rpmbuild/BUILD/ceph-13.2.4/src/rocksdb/db/db_impl_compaction_flush.cc:172] [default] Level summary: base level 1 max bytes base 268435456 files[1 3 0 0 0 0 0] max score 0.75 *2020-08-03 15:54:38.567* 7f87e0836700 1 osd.32 502096 state: booting -> active 2020-08-03 15:54:38.567 7f87d5820700 1 osd.32 pg_epoch: 502096 pg[1.17e( empty local-lis/les=0/0 n=0 ec=11627/16 lis/c 501703/501703 les/c/f 501704/501704/0 502096/502096/502096) [32,26,28] r=0 lpr=502096 pi=[501703,502096)/1 crt=0'0 mlcod 0'0 unknown mbc={}] state<Start>: transitioning to Primary Best Jerry