[ceph-users] Num values for 3 DC 4+2 crush rule

15 Mar 2024

      I was just looking at our crush rules as we need to change them from 
failure domain host to failure domain datacenter. The replicated ones 
seem trivial but what about this one for EC 4+2?

rule rbd_ec_data {
         id 0
         type erasure
         step set_chooseleaf_tries 5
         step set_choose_tries 100
         step take default class hdd
         step chooseleaf indep 0 type host
         step emit
}

We already have this crush rule for EC 4+5:

"
rule cephfs.hdd.data {
         id 7
         type erasure
         step set_chooseleaf_tries 5
         step set_choose_tries 100
         step take default class hdd
         step choose indep 0 type datacenter
         step chooseleaf indep 3 type host
         step emit
}
"

I don't understand the "num" argument for the choose step. The 
documentation[1] says:

"
If {num} == 0, choose pool-num-replicas buckets (as many buckets as are 
available).

If pool-num-replicas > {num} > 0, choose that many buckets.

If {num} < 0, choose pool-num-replicas - {num} buckets.
"

Num is 0 but it's not replicated so how does this translate to picking 3 
of 3 datacenters?

I am thinking we should just change 3 to 2 for the chooseleaf line for 
the 4+2 rule since for 4+5 each DC needs 3 shards and for 4+2 each DC 
needs 2 shards. Comments?

Mvh.

Torkil

[1] https://docs.ceph.com/en/reef/rados/operations/crush-map-edits/

-- 
Torkil Svensgaard
Systems Administrator
Danish Research Centre for Magnetic Resonance DRCMR, Section 714
Copenhagen University Hospital Amager and Hvidovre
Kettegaard Allé 30, 2650 Hvidovre, Denmark