Re: Best "modern" approach to drain/fill a OSD server
Hello! I'm glad to see that this info is still proving useful to the community. Here's a pretty comprehensive document explaining the typical upmap-remapped workflow: https://docs.clyso.com/docs/kb/rados/adding-hosts-osds/ If it can be useful, I'm thinking of organizing an OSD balancing happy hour call in the next month or so. Maybe mid-May or so on a Friday afternoon (EU time) / morning (US time). Thanks! Dan -- Dan van der Ster Ceph Executive Council | CTO @ CLYSO Try our Ceph Analyzer -- https://analyzer.clyso.com/ https://clyso.com | dan.vanderster@clyso.com On Mon, Apr 27, 2026 at 7:11 PM Iztok Gregori via ceph-users <ceph-users@ceph.io> wrote:
Hi to all!
I'm searching the wisdom of the community on how to drain/fill an OSD server (like for removal, bulk disk replacing, filestore to bluestore conversion...) without impacting on the performance of the cluster.
We are using the 'ceph-gentle-reweight' script, which is running fine, but we are interested to know if there is a newer approach for this job.
Searching the web I found a pdf of the talk of Dan van der Ster about "Mastering Ceph Operations" (NYC Ceph Day 2024) in which he talks about using the 'balancer' in 'upmap' mode (and a "magic" script) to move the PGs from/to the OSDs in the background.
So the balancer upmap approach can be used to drain a node following this procedure: - set balancer tunables (like 'target_max_misplaced_ratio') - activate the balancer in upmap mode (if not already active) - 'ceph osd crush reweight-subtree' to 0.1 for the host we wish to drain - run the magic script 'upmap-remapped.py' - 'ceph osd crush reweight-subtree' to 0 for the host we wish to drain - do stuff on the node (remove, reinstall etc etc).
But what is the implication for the whole cluster, how this background job affects its performance?
I assume that 'target_max_misplaced_ratio' is a key parameter to avoid large data movement in a given time, there is more to look into?
And there are other approaches for the job?
Thanks a lot! Iztok
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi Dan! On 28/04/26 02:54, Dan van der Ster wrote:
Hello!
I'm glad to see that this info is still proving useful to the community. Here's a pretty comprehensive document explaining the typical upmap-remapped workflow: https://docs.clyso.com/docs/kb/rados/adding-hosts-osds/
Nice! I'm surprised that I didn't stumble on it after 3 days searching about the subject. Google is definitely getting worse. It's a perfect guide for the job.
If it can be useful, I'm thinking of organizing an OSD balancing happy hour call in the next month or so. Maybe mid-May or so on a Friday afternoon (EU time) / morning (US time).
I could also be interested! Iztok
participants (2)
-
Dan van der Ster
-
Iztok Gregori