Simultaneous increasing numbers of PGs and OSDs
Hello, For one of our CephFS clusters, we need to increase the number of PGs in the main data pool from 8192 to 16384. We have also received new nodes to increase the capacity of this cluster, an almost 50% increase in capacity/number of nodes. We plan to use "upmap-remapped.py" as described in https://docs.clyso.com/docs/kb/rados/adding-hosts-osds/ and https://docs.clyso.com/docs/kb/rados/pg-splitting-with-upmap-remapped/ Is is safe/OK to combine the two operations ? That is: - disable rebalance/the balancer - "apply" the OSDs YAML, the new hosts are already added to the cluster - run upmap-remapped.py a few times until the number of misplaced objects is low - increase the number of PGs in the pool - run upmap-remapped.py until "pg_num" has reached the new value - re-enable rebalance/the balancer Or is it safer/recommended to do this in two distinct operations ? Loïc. -- | Loic Tortay <tortay@cc.in2p3.fr> - IN2P3 Computing Centre |
Den ons 14 okt. 2026 kl 11:37 skrev Loïc Tortay <ceph-users@ceph.io>:
Hello, For one of our CephFS clusters, we need to increase the number of PGs in the main data pool from 8192 to 16384. We have also received new nodes to increase the capacity of this cluster, an almost 50% increase in capacity/number of nodes.
We plan to use "upmap-remapped.py" as described in https://docs.clyso.com/docs/kb/rados/adding-hosts-osds/ and https://docs.clyso.com/docs/kb/rados/pg-splitting-with-upmap-remapped/
Is is safe/OK to combine the two operations ?
That is: - disable rebalance/the balancer - "apply" the OSDs YAML, the new hosts are already added to the cluster - run upmap-remapped.py a few times until the number of misplaced objects is low - increase the number of PGs in the pool - run upmap-remapped.py until "pg_num" has reached the new value - re-enable rebalance/the balancer
Or is it safer/recommended to do this in two distinct operations ?
I think it sounds safe enough. You probably don't want to move all the data twice, and the dangerous part (I guess) is the splitting, and that will happen regardless, so either it works or it fails (due to some bug) and adding new hosts shouldn't really change that, so I think it is safe to follow the listed procedure. -- May the most significant bit of your life be positive.
participants (2)
-
Janne Johansson
-
Loïc Tortay