Hello Andrei,
I have kinda the same problem but because it's production and i don't want to do
sudden moves that will have data redistribution and affect clients ( only with change
approval and stuff) but from what i played into other test clusters and according to
documentation... you need to also set the crush compact mode. (you can even configure a
plan if you want). After it finishes the balance then i believe you can turn off the
balancer (you don't have to have it on all the time)