We run Ceph clusters with high PG counts (300 PGs/OSD, 32k total PGs and upwards). Something that I have noticed is that CRUSH changes (marking an OSD as out, or a new OSD joining the cluster) can cause many hundreds of PGs to remap and peer. During this time we get PG_AVAILABILITY health check warnings as the OSDs need to peer with their new PGs before they can serve IOs. RADOS tail latency suffers because IOs sent to a peering PG are blocked until peering completes. I believe that the more PGs we have, the more PGs need to peer, and the longer IOs are blocked for the pool. Does anyone have suggestions on how to deal with PG peering blocking IOs? I was thinking about gradually re-weighting OSDs instead of marking them out or in, but that can result in multiple redundant data moves. Elias