Anthony, So it sounds like I need to make a new crush rule for replicated pools that specifies default-hdd and the device class? (Or should I go the other way around? I think I'd rather change the replicated pools even though there's more of them.) Then, after I create this new rule, I simply assign the pool to a new crush rule using a command similar to the one shown in your note in the link you referenced? Thanks. -Dave -- Dave Hall Binghamton University kdhall@binghamton.edu On Wed, Sep 18, 2024 at 2:10 PM Anthony D'Atri <anthony.datri@gmail.com> wrote:
Helllo,
I've reviewed some recent posts in this list and also searched Google for info about autoscale and overlapping roots. In what I have found I do not see anything that I can understand regarding how to fix the issue - probably because I don't deal with Crush on a regular basis.
Checkout the Note in this section: https://docs.ceph.com/en/reef/rados/operations/placement-groups/#viewing-pg-...
I added that last year I think it was as a result of how Rook was creating pools.
From what I read and looking at 'ceph osd crush rule dump', it looks like the 8 replicated pools have
"op": "take", "item": -1, "item_name": "default"
whereas the 2 EC pools have
"op": "take", "item": -2, "item_name": "default~hdd"
To be sure, all of my OSDs are identical - HDD with SSD WAL/DB.
Please advise on how to fix this.
The subtlety that's easy to miss is that when you specify a device class for only *some* pools, the pools/rules that specify a device class effectively act on a "shadow" CRUSH root. My terminology may be inexact there.
So I think if you adjust your CRUSH rules so that they all specify a device class -- in your case all the same device class -- your problem (and balancer performance perhaps) will improve.