Hi Fabien, additional to what Anthony said you could do the following: - `ceph osd set nobackfill` to disable initial backfilling - `ceph config set osd osd_mclock_override_recovery_settings true` to override the mclock sheduler backfill settings - Let the orchestrator add one host each time. I would wait between each host until all the peering and stuff is done and only the backfilling is left over. From my experience adding a whole host is not a problem, unless you are hit by the pglog_dup bug (was fixed in pacific IIRC) - `ceph tell 'osd.*' injectargs '--osd-max-backfills 1'` to limit the backfilling as much as possible - `ceph osd unset nobackfill` to start the actuall backfill process - `ceph config set osd osd_mclock_override_recovery_settings false` after backfilling is done. I would restart all OSDs after that to make sure the OSDs got the correct backfilling values :) Make sure your mons have enough oomph to handle the workload. At least, that would be my approach when adding that amount of disks. Usually I Only add 36 disks at a time, when capacity get a little low :) Am Di., 6. Aug. 2024 um 17:10 Uhr schrieb Fabien Sirjean < fsirjean@eddie.fdn.fr>:
Hello everyone,
We need to add 180 20TB OSDs to our Ceph cluster, which currently consists of 540 OSDs of identical size (replicated size 3).
I'm not sure, though: is it a good idea to add all the OSDs at once? Or is it better to add them gradually?
The idea is to minimize the impact of rebalancing on the performance of CephFS, which is used in production.
Thanks in advance for your opinions and feedback 🙂
Wishing you a great summer,
Fabien _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im groüen Saal.