Rebalancing after modifying CRUSH map
Hi all We are looking at implementing Ceph/CephFS for a project. Over time, we may wish to add additional replicas to our cluster. If we modify a CRUSH map, is there a way of then requesting Ceph to re-evaluate the placement of objects across the cluster according to the modified CRUSH map? Brett
This is done automatically. Every time the crush map changes, objects get moved around. Therefore, a typical procedure is - make sure ceph is HEALTH_OK - ceph osd set noout - ceph osd set norebalance - edit crush map - wait for peering to finish, all PGs must be active+clean - lots of PGs will also be re-mapped - ceph osd unset norebalance - ceph osd unset noout Before doing the last 2 steps, verify that no PGs are incomplete and no objects are degraded. Otherwise, fix first. Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Brett Randall <brett.randall@gmail.com> Sent: 09 June 2020 07:42:33 To: ceph-users@ceph.io Subject: [ceph-users] Rebalancing after modifying CRUSH map Hi all We are looking at implementing Ceph/CephFS for a project. Over time, we may wish to add additional replicas to our cluster. If we modify a CRUSH map, is there a way of then requesting Ceph to re-evaluate the placement of objects across the cluster according to the modified CRUSH map? Brett _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Den tis 9 juni 2020 kl 07:43 skrev Brett Randall <brett.randall@gmail.com>:
Hi all We are looking at implementing Ceph/CephFS for a project. Over time, we may wish to add additional replicas to our cluster. If we modify a CRUSH map, is there a way of then requesting Ceph to re-evaluate the placement of objects across the cluster according to the modified CRUSH map?
If you edit the crush map (including just adding a disk or a new host) then all the pools whose crushrules are affected by the change will more or less immediately try to move around data in order to fit the new crush map. If you ask the cluster to move in some impossible way, it will refuse and claim the PGs are misplaced/remapped but will still continue to serve data as usual, until you either make it possible to place after the new rules, or you revert the new rules. -- May the most significant bit of your life be positive.
Thanks Janne, this is fantastic to know. Brett --- original message --- On June 9, 2020, 4:21 PM GMT+10 icepic.dz@gmail.com wrote: Den tis 9 juni 2020 kl 07:43 skrev Brett Randall <brett.randall@gmail.com>:
Hi all
We are looking at implementing Ceph/CephFS for a project. Over time, we may wish to add additional replicas to our cluster. If we modify a CRUSH map, is there a way of then requesting Ceph to re-evaluate the placement of objects across the cluster according to the modified CRUSH map?
If you edit the crush map (including just adding a disk or a new host) then all the pools whose crushrules are affected by the change will more or less immediately try to move around data in order to fit the new crush map. If you ask the cluster to move in some impossible way, it will refuse and claim the PGs are misplaced/remapped but will still continue to serve data as usual, until you either make it possible to place after the new rules, or you revert the new rules. -- May the most significant bit of your life be positive. --- end of original message ---
participants (3)
-
Brett Randall
-
Frank Schilder
-
Janne Johansson