Hi *, I'm investigating an interesting issue on two customer clusters (used for mirroring) I've not solved yet, but today we finally made some progress. Maybe someone has an idea where to look next, I'd appreciate any hints or comments. These are two (latest) Octopus clusters, main usage currently is RBD mirroring with snapshot mode (around 500 RBD images are synced every 30 minutes). They noticed very long startup times of MON daemons after reboot, times between 10 and 30 minutes (reboot time already subtracted). These delays are present on both sites. Today we got a maintenance window and started to check in more detail by just restarting the MON service (joins quorum within seconds), then stopping the MON service and wait a few minutes (still joins quorum within seconds). And then we stopped the service and waited for more than 5 minutes, simulating a reboot, and then we were able to reproduce it. The sync then takes around 15 minutes, we verified with other MONs as well. The MON store is around 2 GB of size (on HDD), I understand that the sync itself can take some time, but what is the threshold here? I tried to find a hint in the MON config, searching for timeouts with 300 seconds, there were only a few matches (mon_session_timeout is one of them), but I'm not sure if they can explain this behavior. Investigating the MON store (ceph-monstore-tool dump-keys) I noticed that there were more than 42 Million osd_snap keys, which is quite a lot and would explain the size of the MON store. But I'm also not sure if it's related to the long syncing process. Does that sound familiar to anyone? Thanks, Eugen