Hi Stefan I can't recall that that was the case and unfortunately we do not have enough history for our performance measurements to look back We are on nautilus. Please let me know your findings when you do your pg expansion on nautilus Grtz Marcel
OK, I'm really curious if you observed the following behaviour:
During, or shortly after the rebalance, did you see high CPU usage of the OSDs? In particular the ones that hosted the PGs before they were moved to the new nodes? As in ~ 300 % CPU per OSD (increasing from a few percent to 300% non stop)? RocksDB is doing housekeeping, And we observed before, and today again, on Mimic 13.2.8, that with a lot of OMAP/META data the OSDs that have to clean up consume a ridiculous amount of CPU (for hours on end). Triggering loads of slow ops and latency spikes in the somtimes (tens) of seconds.
Are you running nautilus? If you haven't seen this behaviour this might have been fixed in Nautlilus. Or you cluster is different from ours. We will do PG expansion after we have upgraded to Nautilus, so we'll definitely know by then.
Thanks,
Stefan