Question about the osdmaptool deviation calculations; For instance, ----- osdmaptool omap --upmap output.txt --upmap-pool cephfs_data-rep3 --upmap-max 1000 --upmap-deviation 5 osdmaptool: osdmap file 'omap' writing upmap command output to: output.txt checking for upmap cleanups upmap, max-count 1000, max deviation 5 limiting to pools cephfs_data-rep3 ([30]) pools cephfs_data-rep3 prepared 0/1000 changes Unable to find further optimization, or distribution is already perfect ----- The evaluated pool is all-on-hdd, and the pool was created with PGS > number of hdd OSDs in the cluster. So each hdd OSD is being used at least once by this pool. Is it correct to assume that the osdmaptool is relying on the equations set at ceph-17.2.5/src/osd/OSDMap.cc:5143 5143 // This function calculates the 2 maps osd_deviation and deviation_osd which 5144 // hold the deviation between the current number of PGs which map to an OSD 5145 // and the optimal number. ... # pgs_per_weight # ceph-17.2.5/src/osd/OSDMap.cc:4806 4806 float pgs_per_weight = total_pgs / osd_weight_total; # target # ceph-17.2.5/src/osd/OSDMap.cc:5156 5156 float target = osd_weight.at(oid) * pgs_per_weight; # deviation # ceph-17.2.5/src/osd/OSDMap.cc:5157 5157 float deviation = (float)opgs.size() - target; And so for pgs_per_weight I calculate ceph -f json osd df | jq '[ .nodes[] | select (.device_class == "hdd") .pgs ] | add' divided by ceph -f json osd df | jq '[ .nodes[] | select (.device_class == "hdd") .crush_weight ] | add' (each hdd OSD in this cluster has identical weight) target = osd_weight.at(oid) * pgs_per_weight I calculate deviation for each osd deviation = opgs.size - target where, opgs.size = the number of PGs at an OSD. i.e. The value of $19 for each $1, in `ceph osd df hdd | awk '{ print $1 " " $19 }'` The result is many many OSDs with a deviation well above the upmap_max_deviation which is at default: 5 So I am wondering if I am miscalculating something, or if I'm not aware of further things the osdmaptool is considering when formulating upmap suggestions? -Robert