The "reweight" field that is set from 1.0 to 0.0
Have you tried reweighting to a non-zero, near-zero crush weight, e.g., 0.0001? That should leave it in the crush map and simulate it being 'out'. - Joshua On Mon, 24 Aug 2026 at 14:12, Andras Pataki <ceph-users@ceph.io> wrote:
Hi friends of ceph,
I'd like to better understand how ceph moves PGs off of OSDs that are marked out for the purposes of simulating potential setup changes. When all OSDs are in (and no upmaps are present), I can use crushtool to find the PG to OSDs mappings of all PGs in the cluster given a crush map (for example: "crushtool -i crushmap.compiled --test --show-mappings --num-rep 3 --min-x 0 --max-x 511 --rule 3 --pool-id 3" would do this for one of our pools). If I change any weights in the crush map, I can recompile it, rerun crushtool to get an updated mapping, and I have found this to perfectly match what ceph does when the crush map is loaded into the cluster.
What I am a bit unsure about is how to get the PG to OSD mappings when an OSD is marked out. The "reweight" field that is set from 1.0 to 0.0 when an OSD is marked out is not in the crush map, it is in the osdmap only. So clearly some modification to the crush map is needed or some post processing of the crush mappings are in order to account for OSDs marked out. I originally thought that changing the OSD weight to 0.0 in the crush map without modifying the parent bucket weights would do the trick. While this approach does remap the correct PGs (the ones that originally contain the OSD that is marked out), the new mappings are not the right ones (i.e. the new mappings do not match what live ceph does).
So my question if someone can enlighten me or point me to the right direction: starting with a crush map, what do I need to do to simulate the mappings for an OSD that is marked out? I.e. what changes are needed beyond what crushtool --show-mappings would give and how to calculate those offline (without touching a live cluster)? I've tried looking at the code (OSDMap.cc) - but couldn't quite understand where the adjustments are made for OSDs marked out. Any pointers/hints/insights would be appreciated.
Thanks,
Andras _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io