Take a look at https://github.com/TheJJ/ceph-balancer We switched to it after lot of attempts to make internal balancer work as expected and now we have ~even OSD utilization across cluster: # ./placementoptimizer.py -v balance --ensure-optimal-moves --ensure-variance-decrease [2023-08-03 23:33:27,954] gathering cluster state via ceph api... [2023-08-03 23:33:36,081] running pg balancer [2023-08-03 23:33:36,088] current OSD fill rate per crushclasses: [2023-08-03 23:33:36,089] ssd: average=49.86%, median=50.27%, without_placement_constraints=53.01% [2023-08-03 23:33:36,090] cluster variance for crushclasses: [2023-08-03 23:33:36,090] ssd: 4.163 [2023-08-03 23:33:36,090] min osd.14 44.698% [2023-08-03 23:33:36,090] max osd.22 51.897% [2023-08-03 23:33:36,101] in descending full-order, couldn't empty osd.22, so we're done. if you want to try more often, set --max-full-move-attempts=$nr, this may unlock more balancing possibilities. [2023-08-03 23:33:36,101] -------------------------------------------------------------------------------- [2023-08-03 23:33:36,101] generated 0 remaps. [2023-08-03 23:33:36,101] total movement size: 0.0B. [2023-08-03 23:33:36,102] -------------------------------------------------------------------------------- [2023-08-03 23:33:36,102] old cluster variance per crushclass: [2023-08-03 23:33:36,102] ssd: 4.163 [2023-08-03 23:33:36,102] old min osd.14 44.698% [2023-08-03 23:33:36,102] old max osd.22 51.897% [2023-08-03 23:33:36,102] -------------------------------------------------------------------------------- [2023-08-03 23:33:36,103] new min osd.14 44.698% [2023-08-03 23:33:36,103] new max osd.22 51.897% [2023-08-03 23:33:36,103] new cluster variance: [2023-08-03 23:33:36,103] ssd: 4.163 [2023-08-03 23:33:36,103] -------------------------------------------------------------------------------- On 03.08.2023 16:38, Spiros Papageorgiou wrote:
On 03-Aug-23 12:11 PM, Eugen Block wrote:
ceph balancer status
I changed the PGs and it started rebalancing (and turned autoscaler off) , so now it will not report status:
It reports: "optimize_result": "Too many objects (0.088184 > 0.050000) are misplaced; try again later"
Lets wait a few hours to see what happens...
Thanx!
Sp
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io