On Mon, 13 Oct 2025 12:27:27 +0200, Sven Grottke <sfs@math.tu-berlin.de> said:
[...] but one of our pools is erasure-coded on 16TB HDDs.
This is the beginning of many sad stories :-).
Two weeks ago, we upgraded from Reef to Squid (19.2.3) [...] an ever growing number of PGs that have not been deep-scrubbed or scrubbed in time. All these PGs are located on the HDD pool.
So far it is behaving as expected. Surprising that it did not happen before. Perhaps it started to become more fully used or to have more small files in that period of time.
I would like to increase the scrubbing and deep-scrubbing intervals for the HDDs, but not for the SSDs. Is there any way to do that other than setting it on a per-OSD base [...]
The expectation for a significant Ceph setup is that its servers are managed by something like Ansible, CfEngine, etc, where configuration updates are distributed to many OSDs by that kind of tool. To some extent "orchestration" and GUI frontends can supplement that, but for finer control some kind of CM engine is usually needed.