Re: Best "modern" approach to drain/fill a OSD server

29 Apr 2026


      This does not use upmap. With the swap-bucket command, all the data of 
the source node (and some more) becomes "misplaced". I would expect it 
to work less direct in case lots of upmap remappings are in place.

On 4/29/26 04:44, Wannes Smet via ceph-users wrote:
...
Does swap-bucket use upmap? If so, wouldn't that mean that a lot of PGs no longer end up where CRUSH would have placed them?
________________________________
From: Eugen Block via ceph-users <ceph-users@ceph.io>
Sent: Tuesday, April 28, 2026 21:40
To: ceph-users@ceph.io <ceph-users@ceph.io>
Subject: [ceph-users] Re: Best "modern" approach to drain/fill a OSD server
+1 for swap-bucket, we've been using that as well successfully. What we also sometimes do, especially when time is a bit more critical, is to purge OSDs of an entire host (we also did this with two hosts at the same time due to high resiliency)
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
ZjQcmQRYFpfptBannerEnd
+1 for swap-bucket, we've been using that as well successfully.
What we also sometimes do, especially when time is a bit more
critical, is to purge OSDs of an entire host (we also did this with
two hosts at the same time due to high resiliency) and then let Ceph
recover. But I generally don't recommend that to unexperienced
operators.
Zitat von Harald Staub via ceph-users <ceph-users@ceph.io>:
...
Not recently, but a few years ago, I used the "Whole host
replacement" using "swap-bucket":
https://urldefense.com/v3/__https://docs.ceph.com/en/latest/rados/operations/bluestore-migration/__;!!FtrhtPsWDhZ6tw!CZ4Rn9HplN1eWCfW91sJ1DipsKeloy43URT2W50wu0MPuGwTsoWI9y2zDVxtWzd2zyrmgGlnQit2rgvd$[docs[.]ceph[.]com]
This worked really nicely. With source and destination about the
same host CRUSH weight, most PGs are moved directly from source to
destination. So also the copy speed was more or less the same most
of the time.
Cheers,
  Harry
On 4/28/26 11:08, Denis Polom via ceph-users wrote:
...
you can control max_backfills and some others if you  set
osd_mclock_override_recovery_settings to true
On 4/28/26 10:57, Janne Johansson via ceph-users wrote:
...
...
Another approach is to set the flags, destroy the OSDs on a
single host, redeploy them, set your backfill/recovery throttles
down to 1, and unset the flags.
This is harder with mclock right? With wpq you can still control
max_backfills, but not with mclock unless I am mistaken.
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-leave@ceph.io

ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-leave@ceph.io

Harald Staub

tags

participants (1)