Hello Everyone, We're working on a new cluster and seeing some oddities. The crush map viewer is not showing all hosts or OSDs. Cluster is NVMe w/4 hosts, each having 8 NVMe. Using 2 OSDs per NVMe and Encryption. Using Max size of 3, Min size of 2: [image: image.png] All OSDs appear to exist in: ceph osd tree root@prdhcistonode01:~# ceph osd tree ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -1 372.60156 root default -3 93.15039 host prdhcistonode01 0 ssd 5.82190 osd.0 up 1.00000 1.00000 1 ssd 5.82190 osd.1 up 1.00000 1.00000 2 ssd 5.82190 osd.2 up 1.00000 1.00000 3 ssd 5.82190 osd.3 up 1.00000 1.00000 4 ssd 5.82190 osd.4 up 1.00000 1.00000 5 ssd 5.82190 osd.5 up 1.00000 1.00000 6 ssd 5.82190 osd.6 up 1.00000 1.00000 7 ssd 5.82190 osd.7 up 1.00000 1.00000 8 ssd 5.82190 osd.8 up 1.00000 1.00000 9 ssd 5.82190 osd.9 up 1.00000 1.00000 10 ssd 5.82190 osd.10 up 1.00000 1.00000 11 ssd 5.82190 osd.11 up 1.00000 1.00000 12 ssd 5.82190 osd.12 up 1.00000 1.00000 13 ssd 5.82190 osd.13 up 1.00000 1.00000 14 ssd 5.82190 osd.14 up 1.00000 1.00000 15 ssd 5.82190 osd.15 up 1.00000 1.00000 -5 93.15039 host prdhcistonode02 17 ssd 5.82190 osd.17 up 1.00000 1.00000 18 ssd 5.82190 osd.18 up 1.00000 1.00000 19 ssd 5.82190 osd.19 up 1.00000 1.00000 20 ssd 5.82190 osd.20 up 1.00000 1.00000 21 ssd 5.82190 osd.21 up 1.00000 1.00000 22 ssd 5.82190 osd.22 up 1.00000 1.00000 23 ssd 5.82190 osd.23 up 1.00000 1.00000 24 ssd 5.82190 osd.24 up 1.00000 1.00000 25 ssd 5.82190 osd.25 up 1.00000 1.00000 26 ssd 5.82190 osd.26 up 1.00000 1.00000 27 ssd 5.82190 osd.27 up 1.00000 1.00000 28 ssd 5.82190 osd.28 up 1.00000 1.00000 29 ssd 5.82190 osd.29 up 1.00000 1.00000 30 ssd 5.82190 osd.30 up 1.00000 1.00000 48 ssd 5.82190 osd.48 up 1.00000 1.00000 49 ssd 5.82190 osd.49 up 1.00000 1.00000 -7 93.15039 host prdhcistonode03 16 ssd 5.82190 osd.16 up 1.00000 1.00000 31 ssd 5.82190 osd.31 up 1.00000 1.00000 32 ssd 5.82190 osd.32 up 1.00000 1.00000 33 ssd 5.82190 osd.33 up 1.00000 1.00000 34 ssd 5.82190 osd.34 up 1.00000 1.00000 35 ssd 5.82190 osd.35 up 1.00000 1.00000 36 ssd 5.82190 osd.36 up 1.00000 1.00000 37 ssd 5.82190 osd.37 up 1.00000 1.00000 38 ssd 5.82190 osd.38 up 1.00000 1.00000 39 ssd 5.82190 osd.39 up 1.00000 1.00000 40 ssd 5.82190 osd.40 up 1.00000 1.00000 41 ssd 5.82190 osd.41 up 1.00000 1.00000 42 ssd 5.82190 osd.42 up 1.00000 1.00000 43 ssd 5.82190 osd.43 up 1.00000 1.00000 44 ssd 5.82190 osd.44 up 1.00000 1.00000 45 ssd 5.82190 osd.45 up 1.00000 1.00000 -9 93.15039 host prdhcistonode04 46 ssd 5.82190 osd.46 up 1.00000 1.00000 47 ssd 5.82190 osd.47 up 1.00000 1.00000 50 ssd 5.82190 osd.50 up 1.00000 1.00000 51 ssd 5.82190 osd.51 up 1.00000 1.00000 52 ssd 5.82190 osd.52 up 1.00000 1.00000 53 ssd 5.82190 osd.53 up 1.00000 1.00000 54 ssd 5.82190 osd.54 up 1.00000 1.00000 55 ssd 5.82190 osd.55 up 1.00000 1.00000 56 ssd 5.82190 osd.56 up 1.00000 1.00000 57 ssd 5.82190 osd.57 up 1.00000 1.00000 58 ssd 5.82190 osd.58 up 1.00000 1.00000 59 ssd 5.82190 osd.59 up 1.00000 1.00000 60 ssd 5.82190 osd.60 up 1.00000 1.00000 61 ssd 5.82190 osd.61 up 1.00000 1.00000 62 ssd 5.82190 osd.62 up 1.00000 1.00000 63 ssd 5.82190 osd.63 up 1.00000 1.00000 Any suggestions would be greatly appreciated. Thanks, Marco
Hi Marco, looks like you have found a bug. It would be helpful to see the output of the /api/health/full REST API call that is used to generate that tree view. You can obtain this via the dashboard's built-in API browser (click Help icon -> API -> Health -> GET /health/full -> Try it out -> Execute -> save the response body) Also, do you see any errors in the browser's console when viewing that page? Thanks in advance! Lenz On 6/2/20 4:15 AM, Marco Pizzolo wrote:
We're working on a new cluster and seeing some oddities. The crush map viewer is not showing all hosts or OSDs. Cluster is NVMe w/4 hosts, each having 8 NVMe. Using 2 OSDs per NVMe and Encryption. Using Max size of 3, Min size of 2:
image.png
All OSDs appear to exist in: ceph osd tree
root@prdhcistonode01:~# ceph osd tree ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -1 372.60156 root default -3 93.15039 host prdhcistonode01 0 ssd 5.82190 osd.0 up 1.00000 1.00000 1 ssd 5.82190 osd.1 up 1.00000 1.00000 2 ssd 5.82190 osd.2 up 1.00000 1.00000 3 ssd 5.82190 osd.3 up 1.00000 1.00000 4 ssd 5.82190 osd.4 up 1.00000 1.00000 5 ssd 5.82190 osd.5 up 1.00000 1.00000 6 ssd 5.82190 osd.6 up 1.00000 1.00000 7 ssd 5.82190 osd.7 up 1.00000 1.00000 8 ssd 5.82190 osd.8 up 1.00000 1.00000 9 ssd 5.82190 osd.9 up 1.00000 1.00000 10 ssd 5.82190 osd.10 up 1.00000 1.00000 11 ssd 5.82190 osd.11 up 1.00000 1.00000 12 ssd 5.82190 osd.12 up 1.00000 1.00000 13 ssd 5.82190 osd.13 up 1.00000 1.00000 14 ssd 5.82190 osd.14 up 1.00000 1.00000 15 ssd 5.82190 osd.15 up 1.00000 1.00000 -5 93.15039 host prdhcistonode02 17 ssd 5.82190 osd.17 up 1.00000 1.00000 18 ssd 5.82190 osd.18 up 1.00000 1.00000 19 ssd 5.82190 osd.19 up 1.00000 1.00000 20 ssd 5.82190 osd.20 up 1.00000 1.00000 21 ssd 5.82190 osd.21 up 1.00000 1.00000 22 ssd 5.82190 osd.22 up 1.00000 1.00000 23 ssd 5.82190 osd.23 up 1.00000 1.00000 24 ssd 5.82190 osd.24 up 1.00000 1.00000 25 ssd 5.82190 osd.25 up 1.00000 1.00000 26 ssd 5.82190 osd.26 up 1.00000 1.00000 27 ssd 5.82190 osd.27 up 1.00000 1.00000 28 ssd 5.82190 osd.28 up 1.00000 1.00000 29 ssd 5.82190 osd.29 up 1.00000 1.00000 30 ssd 5.82190 osd.30 up 1.00000 1.00000 48 ssd 5.82190 osd.48 up 1.00000 1.00000 49 ssd 5.82190 osd.49 up 1.00000 1.00000 -7 93.15039 host prdhcistonode03 16 ssd 5.82190 osd.16 up 1.00000 1.00000 31 ssd 5.82190 osd.31 up 1.00000 1.00000 32 ssd 5.82190 osd.32 up 1.00000 1.00000 33 ssd 5.82190 osd.33 up 1.00000 1.00000 34 ssd 5.82190 osd.34 up 1.00000 1.00000 35 ssd 5.82190 osd.35 up 1.00000 1.00000 36 ssd 5.82190 osd.36 up 1.00000 1.00000 37 ssd 5.82190 osd.37 up 1.00000 1.00000 38 ssd 5.82190 osd.38 up 1.00000 1.00000 39 ssd 5.82190 osd.39 up 1.00000 1.00000 40 ssd 5.82190 osd.40 up 1.00000 1.00000 41 ssd 5.82190 osd.41 up 1.00000 1.00000 42 ssd 5.82190 osd.42 up 1.00000 1.00000 43 ssd 5.82190 osd.43 up 1.00000 1.00000 44 ssd 5.82190 osd.44 up 1.00000 1.00000 45 ssd 5.82190 osd.45 up 1.00000 1.00000 -9 93.15039 host prdhcistonode04 46 ssd 5.82190 osd.46 up 1.00000 1.00000 47 ssd 5.82190 osd.47 up 1.00000 1.00000 50 ssd 5.82190 osd.50 up 1.00000 1.00000 51 ssd 5.82190 osd.51 up 1.00000 1.00000 52 ssd 5.82190 osd.52 up 1.00000 1.00000 53 ssd 5.82190 osd.53 up 1.00000 1.00000 54 ssd 5.82190 osd.54 up 1.00000 1.00000 55 ssd 5.82190 osd.55 up 1.00000 1.00000 56 ssd 5.82190 osd.56 up 1.00000 1.00000 57 ssd 5.82190 osd.57 up 1.00000 1.00000 58 ssd 5.82190 osd.58 up 1.00000 1.00000 59 ssd 5.82190 osd.59 up 1.00000 1.00000 60 ssd 5.82190 osd.60 up 1.00000 1.00000 61 ssd 5.82190 osd.61 up 1.00000 1.00000 62 ssd 5.82190 osd.62 up 1.00000 1.00000 63 ssd 5.82190 osd.63 up 1.00000 1.00000
Any suggestions would be greatly appreciated.
Thanks, Marco
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- SUSE Software Solutions Germany GmbH - Maxfeldstr. 5 - 90409 Nuernberg GF: Felix Imendörffer, HRB 36809 (AG Nürnberg)
Hi Lenz, Hopefully this is the part that was required: "tree": { "nodes": [ { "id": -1, "name": "default", "type": "root", "type_id": 11, "children": [ -9, -7, -5, -3 ] }, { "id": -3, "name": "prdhcistonode01", "type": "host", "type_id": 1, "pool_weights": {}, "children": [ 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0 ] }, { "id": 0, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.0" }, { "id": 1, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.1" }, { "id": 2, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.2" }, { "id": 3, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.3" }, { "id": 4, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.4" }, { "id": 5, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.5" }, { "id": 6, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.6" }, { "id": 7, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.7" }, { "id": 8, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.8" }, { "id": 9, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.9" }, { "id": 10, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.10" }, { "id": 11, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.11" }, { "id": 12, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.12" }, { "id": 13, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.13" }, { "id": 14, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.14" }, { "id": 15, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.15" }, { "id": -5, "name": "prdhcistonode02", "type": "host", "type_id": 1, "pool_weights": {}, "children": [ 49, 48, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17 ] }, { "id": 17, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.17" }, { "id": 18, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.18" }, { "id": 19, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.19" }, { "id": 20, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.20" }, { "id": 21, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.21" }, { "id": 22, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.22" }, { "id": 23, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.23" }, { "id": 24, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.24" }, { "id": 25, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.25" }, { "id": 26, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.26" }, { "id": 27, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.27" }, { "id": 28, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.28" }, { "id": 29, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.29" }, { "id": 30, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.30" }, { "id": 48, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.48" }, { "id": 49, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.49" }, { "id": -7, "name": "prdhcistonode03", "type": "host", "type_id": 1, "pool_weights": {}, "children": [ 45, 44, 43, 42, 41, 40, 39, 38, 37, 36, 35, 34, 33, 32, 31, 16 ] }, { "id": 16, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.16" }, { "id": 31, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.31" }, { "id": 32, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.32" }, { "id": 33, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.33" }, { "id": 34, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.34" }, { "id": 35, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.35" }, { "id": 36, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.36" }, { "id": 37, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.37" }, { "id": 38, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.38" }, { "id": 39, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.39" }, { "id": 40, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.40" }, { "id": 41, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.41" }, { "id": 42, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.42" }, { "id": 43, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.43" }, { "id": 44, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.44" }, { "id": 45, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.45" }, { "id": -9, "name": "prdhcistonode04", "type": "host", "type_id": 1, "pool_weights": {}, "children": [ 63, 62, 61, 60, 59, 58, 57, 56, 55, 54, 53, 52, 51, 50, 47, 46 ] }, { "id": 46, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.46" }, { "id": 47, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.47" }, { "id": 50, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.50" }, { "id": 51, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.51" }, { "id": 52, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.52" }, { "id": 53, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.53" }, { "id": 54, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.54" }, { "id": 55, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.55" }, { "id": 56, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.56" }, { "id": 57, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.57" }, { "id": 58, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.58" }, { "id": 59, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.59" }, { "id": 60, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.60" }, { "id": 61, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.61" }, { "id": 62, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.62" }, { "id": 63, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.63" } ], "stray": [] }, "crush": { "devices": [ { "id": 0, "name": "osd.0", "class": "ssd" }, { "id": 1, "name": "osd.1", "class": "ssd" }, { "id": 2, "name": "osd.2", "class": "ssd" }, { "id": 3, "name": "osd.3", "class": "ssd" }, { "id": 4, "name": "osd.4", "class": "ssd" }, { "id": 5, "name": "osd.5", "class": "ssd" }, { "id": 6, "name": "osd.6", "class": "ssd" }, { "id": 7, "name": "osd.7", "class": "ssd" }, { "id": 8, "name": "osd.8", "class": "ssd" }, { "id": 9, "name": "osd.9", "class": "ssd" }, { "id": 10, "name": "osd.10", "class": "ssd" }, { "id": 11, "name": "osd.11", "class": "ssd" }, { "id": 12, "name": "osd.12", "class": "ssd" }, { "id": 13, "name": "osd.13", "class": "ssd" }, { "id": 14, "name": "osd.14", "class": "ssd" }, { "id": 15, "name": "osd.15", "class": "ssd" }, { "id": 16, "name": "osd.16", "class": "ssd" }, { "id": 17, "name": "osd.17", "class": "ssd" }, { "id": 18, "name": "osd.18", "class": "ssd" }, { "id": 19, "name": "osd.19", "class": "ssd" }, { "id": 20, "name": "osd.20", "class": "ssd" }, { "id": 21, "name": "osd.21", "class": "ssd" }, { "id": 22, "name": "osd.22", "class": "ssd" }, { "id": 23, "name": "osd.23", "class": "ssd" }, { "id": 24, "name": "osd.24", "class": "ssd" }, { "id": 25, "name": "osd.25", "class": "ssd" }, { "id": 26, "name": "osd.26", "class": "ssd" }, { "id": 27, "name": "osd.27", "class": "ssd" }, { "id": 28, "name": "osd.28", "class": "ssd" }, { "id": 29, "name": "osd.29", "class": "ssd" }, { "id": 30, "name": "osd.30", "class": "ssd" }, { "id": 31, "name": "osd.31", "class": "ssd" }, { "id": 32, "name": "osd.32", "class": "ssd" }, { "id": 33, "name": "osd.33", "class": "ssd" }, { "id": 34, "name": "osd.34", "class": "ssd" }, { "id": 35, "name": "osd.35", "class": "ssd" }, { "id": 36, "name": "osd.36", "class": "ssd" }, { "id": 37, "name": "osd.37", "class": "ssd" }, { "id": 38, "name": "osd.38", "class": "ssd" }, { "id": 39, "name": "osd.39", "class": "ssd" }, { "id": 40, "name": "osd.40", "class": "ssd" }, { "id": 41, "name": "osd.41", "class": "ssd" }, { "id": 42, "name": "osd.42", "class": "ssd" }, { "id": 43, "name": "osd.43", "class": "ssd" }, { "id": 44, "name": "osd.44", "class": "ssd" }, { "id": 45, "name": "osd.45", "class": "ssd" }, { "id": 46, "name": "osd.46", "class": "ssd" }, { "id": 47, "name": "osd.47", "class": "ssd" }, { "id": 48, "name": "osd.48", "class": "ssd" }, { "id": 49, "name": "osd.49", "class": "ssd" }, { "id": 50, "name": "osd.50", "class": "ssd" }, { "id": 51, "name": "osd.51", "class": "ssd" }, { "id": 52, "name": "osd.52", "class": "ssd" }, { "id": 53, "name": "osd.53", "class": "ssd" }, { "id": 54, "name": "osd.54", "class": "ssd" }, { "id": 55, "name": "osd.55", "class": "ssd" }, { "id": 56, "name": "osd.56", "class": "ssd" }, { "id": 57, "name": "osd.57", "class": "ssd" }, { "id": 58, "name": "osd.58", "class": "ssd" }, { "id": 59, "name": "osd.59", "class": "ssd" }, { "id": 60, "name": "osd.60", "class": "ssd" }, { "id": 61, "name": "osd.61", "class": "ssd" }, { "id": 62, "name": "osd.62", "class": "ssd" }, { "id": 63, "name": "osd.63", "class": "ssd" } ], "types": [ { "type_id": 0, "name": "osd" }, { "type_id": 1, "name": "host" }, { "type_id": 2, "name": "chassis" }, { "type_id": 3, "name": "rack" }, { "type_id": 4, "name": "row" }, { "type_id": 5, "name": "pdu" }, { "type_id": 6, "name": "pod" }, { "type_id": 7, "name": "room" }, { "type_id": 8, "name": "datacenter" }, { "type_id": 9, "name": "zone" }, { "type_id": 10, "name": "region" }, { "type_id": 11, "name": "root" } ], On Tue, Jun 2, 2020 at 6:48 AM Lenz Grimmer <lgrimmer@suse.com> wrote:
Hi Marco,
looks like you have found a bug. It would be helpful to see the output of the /api/health/full REST API call that is used to generate that tree view.
You can obtain this via the dashboard's built-in API browser (click Help icon -> API -> Health -> GET /health/full -> Try it out -> Execute -> save the response body)
Also, do you see any errors in the browser's console when viewing that page?
Thanks in advance!
Lenz
On 6/2/20 4:15 AM, Marco Pizzolo wrote:
We're working on a new cluster and seeing some oddities. The crush map viewer is not showing all hosts or OSDs. Cluster is NVMe w/4 hosts, each having 8 NVMe. Using 2 OSDs per NVMe and Encryption. Using Max size of 3, Min size of 2:
image.png
All OSDs appear to exist in: ceph osd tree
root@prdhcistonode01:~# ceph osd tree ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -1 372.60156 root default -3 93.15039 host prdhcistonode01 0 ssd 5.82190 osd.0 up 1.00000 1.00000 1 ssd 5.82190 osd.1 up 1.00000 1.00000 2 ssd 5.82190 osd.2 up 1.00000 1.00000 3 ssd 5.82190 osd.3 up 1.00000 1.00000 4 ssd 5.82190 osd.4 up 1.00000 1.00000 5 ssd 5.82190 osd.5 up 1.00000 1.00000 6 ssd 5.82190 osd.6 up 1.00000 1.00000 7 ssd 5.82190 osd.7 up 1.00000 1.00000 8 ssd 5.82190 osd.8 up 1.00000 1.00000 9 ssd 5.82190 osd.9 up 1.00000 1.00000 10 ssd 5.82190 osd.10 up 1.00000 1.00000 11 ssd 5.82190 osd.11 up 1.00000 1.00000 12 ssd 5.82190 osd.12 up 1.00000 1.00000 13 ssd 5.82190 osd.13 up 1.00000 1.00000 14 ssd 5.82190 osd.14 up 1.00000 1.00000 15 ssd 5.82190 osd.15 up 1.00000 1.00000 -5 93.15039 host prdhcistonode02 17 ssd 5.82190 osd.17 up 1.00000 1.00000 18 ssd 5.82190 osd.18 up 1.00000 1.00000 19 ssd 5.82190 osd.19 up 1.00000 1.00000 20 ssd 5.82190 osd.20 up 1.00000 1.00000 21 ssd 5.82190 osd.21 up 1.00000 1.00000 22 ssd 5.82190 osd.22 up 1.00000 1.00000 23 ssd 5.82190 osd.23 up 1.00000 1.00000 24 ssd 5.82190 osd.24 up 1.00000 1.00000 25 ssd 5.82190 osd.25 up 1.00000 1.00000 26 ssd 5.82190 osd.26 up 1.00000 1.00000 27 ssd 5.82190 osd.27 up 1.00000 1.00000 28 ssd 5.82190 osd.28 up 1.00000 1.00000 29 ssd 5.82190 osd.29 up 1.00000 1.00000 30 ssd 5.82190 osd.30 up 1.00000 1.00000 48 ssd 5.82190 osd.48 up 1.00000 1.00000 49 ssd 5.82190 osd.49 up 1.00000 1.00000 -7 93.15039 host prdhcistonode03 16 ssd 5.82190 osd.16 up 1.00000 1.00000 31 ssd 5.82190 osd.31 up 1.00000 1.00000 32 ssd 5.82190 osd.32 up 1.00000 1.00000 33 ssd 5.82190 osd.33 up 1.00000 1.00000 34 ssd 5.82190 osd.34 up 1.00000 1.00000 35 ssd 5.82190 osd.35 up 1.00000 1.00000 36 ssd 5.82190 osd.36 up 1.00000 1.00000 37 ssd 5.82190 osd.37 up 1.00000 1.00000 38 ssd 5.82190 osd.38 up 1.00000 1.00000 39 ssd 5.82190 osd.39 up 1.00000 1.00000 40 ssd 5.82190 osd.40 up 1.00000 1.00000 41 ssd 5.82190 osd.41 up 1.00000 1.00000 42 ssd 5.82190 osd.42 up 1.00000 1.00000 43 ssd 5.82190 osd.43 up 1.00000 1.00000 44 ssd 5.82190 osd.44 up 1.00000 1.00000 45 ssd 5.82190 osd.45 up 1.00000 1.00000 -9 93.15039 host prdhcistonode04 46 ssd 5.82190 osd.46 up 1.00000 1.00000 47 ssd 5.82190 osd.47 up 1.00000 1.00000 50 ssd 5.82190 osd.50 up 1.00000 1.00000 51 ssd 5.82190 osd.51 up 1.00000 1.00000 52 ssd 5.82190 osd.52 up 1.00000 1.00000 53 ssd 5.82190 osd.53 up 1.00000 1.00000 54 ssd 5.82190 osd.54 up 1.00000 1.00000 55 ssd 5.82190 osd.55 up 1.00000 1.00000 56 ssd 5.82190 osd.56 up 1.00000 1.00000 57 ssd 5.82190 osd.57 up 1.00000 1.00000 58 ssd 5.82190 osd.58 up 1.00000 1.00000 59 ssd 5.82190 osd.59 up 1.00000 1.00000 60 ssd 5.82190 osd.60 up 1.00000 1.00000 61 ssd 5.82190 osd.61 up 1.00000 1.00000 62 ssd 5.82190 osd.62 up 1.00000 1.00000 63 ssd 5.82190 osd.63 up 1.00000 1.00000
Any suggestions would be greatly appreciated.
Thanks, Marco
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- SUSE Software Solutions Germany GmbH - Maxfeldstr. 5 - 90409 Nuernberg GF: Felix Imendörffer, HRB 36809 (AG Nürnberg)
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi Marco, thank you. It seems as if the REST API output matches the output of "ceph osd tree", but the tree view in the dashboard somehow fails to display all nodes. We will investigate this. I've now submitted your report as a bug on our tracker: https://tracker.ceph.com/issues/45873 Please make sure to follow this issue in case we need additional information. Thank you! Lenz On 6/2/20 5:03 PM, Marco Pizzolo wrote:
Hopefully this is the part that was required:
"tree": { "nodes": [ { "id": -1, "name": "default", "type": "root", "type_id": 11, "children": [ -9, -7, -5, -3 ] }, { "id": -3, "name": "prdhcistonode01", "type": "host", "type_id": 1, "pool_weights": {}, "children": [ 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0 ] }, { "id": 0, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.0" }, { "id": 1, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.1" }, { "id": 2, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.2" }, { "id": 3, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.3" }, { "id": 4, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.4" }, { "id": 5, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.5" }, { "id": 6, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.6" }, { "id": 7, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.7" }, { "id": 8, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.8" }, { "id": 9, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.9" }, { "id": 10, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.10" }, { "id": 11, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.11" }, { "id": 12, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.12" }, { "id": 13, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.13" }, { "id": 14, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.14" }, { "id": 15, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.15" }, { "id": -5, "name": "prdhcistonode02", "type": "host", "type_id": 1, "pool_weights": {}, "children": [ 49, 48, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17 ] }, { "id": 17, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.17" }, { "id": 18, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.18" }, { "id": 19, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.19" }, { "id": 20, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.20" }, { "id": 21, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.21" }, { "id": 22, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.22" }, { "id": 23, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.23" }, { "id": 24, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.24" }, { "id": 25, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.25" }, { "id": 26, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.26" }, { "id": 27, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.27" }, { "id": 28, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.28" }, { "id": 29, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.29" }, { "id": 30, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.30" }, { "id": 48, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.48" }, { "id": 49, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.49" }, { "id": -7, "name": "prdhcistonode03", "type": "host", "type_id": 1, "pool_weights": {}, "children": [ 45, 44, 43, 42, 41, 40, 39, 38, 37, 36, 35, 34, 33, 32, 31, 16 ] }, { "id": 16, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.16" }, { "id": 31, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.31" }, { "id": 32, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.32" }, { "id": 33, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.33" }, { "id": 34, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.34" }, { "id": 35, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.35" }, { "id": 36, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.36" }, { "id": 37, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.37" }, { "id": 38, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.38" }, { "id": 39, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.39" }, { "id": 40, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.40" }, { "id": 41, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.41" }, { "id": 42, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.42" }, { "id": 43, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.43" }, { "id": 44, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.44" }, { "id": 45, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.45" }, { "id": -9, "name": "prdhcistonode04", "type": "host", "type_id": 1, "pool_weights": {}, "children": [ 63, 62, 61, 60, 59, 58, 57, 56, 55, 54, 53, 52, 51, 50, 47, 46 ] }, { "id": 46, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.46" }, { "id": 47, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.47" }, { "id": 50, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.50" }, { "id": 51, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.51" }, { "id": 52, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.52" }, { "id": 53, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.53" }, { "id": 54, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.54" }, { "id": 55, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.55" }, { "id": 56, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.56" }, { "id": 57, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.57" }, { "id": 58, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.58" }, { "id": 59, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.59" }, { "id": 60, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.60" }, { "id": 61, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.61" }, { "id": 62, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.62" }, { "id": 63, "device_class": "ssd", "type": "osd", "type_id": 0, "crush_weight": 5.8218994140625, "depth": 2, "pool_weights": {}, "exists": 1, "status": "up", "reweight": 1, "primary_affinity": 1, "name": "osd.63" } ], "stray": [] }, "crush": { "devices": [ { "id": 0, "name": "osd.0", "class": "ssd" }, { "id": 1, "name": "osd.1", "class": "ssd" }, { "id": 2, "name": "osd.2", "class": "ssd" }, { "id": 3, "name": "osd.3", "class": "ssd" }, { "id": 4, "name": "osd.4", "class": "ssd" }, { "id": 5, "name": "osd.5", "class": "ssd" }, { "id": 6, "name": "osd.6", "class": "ssd" }, { "id": 7, "name": "osd.7", "class": "ssd" }, { "id": 8, "name": "osd.8", "class": "ssd" }, { "id": 9, "name": "osd.9", "class": "ssd" }, { "id": 10, "name": "osd.10", "class": "ssd" }, { "id": 11, "name": "osd.11", "class": "ssd" }, { "id": 12, "name": "osd.12", "class": "ssd" }, { "id": 13, "name": "osd.13", "class": "ssd" }, { "id": 14, "name": "osd.14", "class": "ssd" }, { "id": 15, "name": "osd.15", "class": "ssd" }, { "id": 16, "name": "osd.16", "class": "ssd" }, { "id": 17, "name": "osd.17", "class": "ssd" }, { "id": 18, "name": "osd.18", "class": "ssd" }, { "id": 19, "name": "osd.19", "class": "ssd" }, { "id": 20, "name": "osd.20", "class": "ssd" }, { "id": 21, "name": "osd.21", "class": "ssd" }, { "id": 22, "name": "osd.22", "class": "ssd" }, { "id": 23, "name": "osd.23", "class": "ssd" }, { "id": 24, "name": "osd.24", "class": "ssd" }, { "id": 25, "name": "osd.25", "class": "ssd" }, { "id": 26, "name": "osd.26", "class": "ssd" }, { "id": 27, "name": "osd.27", "class": "ssd" }, { "id": 28, "name": "osd.28", "class": "ssd" }, { "id": 29, "name": "osd.29", "class": "ssd" }, { "id": 30, "name": "osd.30", "class": "ssd" }, { "id": 31, "name": "osd.31", "class": "ssd" }, { "id": 32, "name": "osd.32", "class": "ssd" }, { "id": 33, "name": "osd.33", "class": "ssd" }, { "id": 34, "name": "osd.34", "class": "ssd" }, { "id": 35, "name": "osd.35", "class": "ssd" }, { "id": 36, "name": "osd.36", "class": "ssd" }, { "id": 37, "name": "osd.37", "class": "ssd" }, { "id": 38, "name": "osd.38", "class": "ssd" }, { "id": 39, "name": "osd.39", "class": "ssd" }, { "id": 40, "name": "osd.40", "class": "ssd" }, { "id": 41, "name": "osd.41", "class": "ssd" }, { "id": 42, "name": "osd.42", "class": "ssd" }, { "id": 43, "name": "osd.43", "class": "ssd" }, { "id": 44, "name": "osd.44", "class": "ssd" }, { "id": 45, "name": "osd.45", "class": "ssd" }, { "id": 46, "name": "osd.46", "class": "ssd" }, { "id": 47, "name": "osd.47", "class": "ssd" }, { "id": 48, "name": "osd.48", "class": "ssd" }, { "id": 49, "name": "osd.49", "class": "ssd" }, { "id": 50, "name": "osd.50", "class": "ssd" }, { "id": 51, "name": "osd.51", "class": "ssd" }, { "id": 52, "name": "osd.52", "class": "ssd" }, { "id": 53, "name": "osd.53", "class": "ssd" }, { "id": 54, "name": "osd.54", "class": "ssd" }, { "id": 55, "name": "osd.55", "class": "ssd" }, { "id": 56, "name": "osd.56", "class": "ssd" }, { "id": 57, "name": "osd.57", "class": "ssd" }, { "id": 58, "name": "osd.58", "class": "ssd" }, { "id": 59, "name": "osd.59", "class": "ssd" }, { "id": 60, "name": "osd.60", "class": "ssd" }, { "id": 61, "name": "osd.61", "class": "ssd" }, { "id": 62, "name": "osd.62", "class": "ssd" }, { "id": 63, "name": "osd.63", "class": "ssd" } ], "types": [ { "type_id": 0, "name": "osd" }, { "type_id": 1, "name": "host" }, { "type_id": 2, "name": "chassis" }, { "type_id": 3, "name": "rack" }, { "type_id": 4, "name": "row" }, { "type_id": 5, "name": "pdu" }, { "type_id": 6, "name": "pod" }, { "type_id": 7, "name": "room" }, { "type_id": 8, "name": "datacenter" }, { "type_id": 9, "name": "zone" }, { "type_id": 10, "name": "region" }, { "type_id": 11, "name": "root" } ],
On Tue, Jun 2, 2020 at 6:48 AM Lenz Grimmer <lgrimmer@suse.com <mailto:lgrimmer@suse.com>> wrote:
Hi Marco,
looks like you have found a bug. It would be helpful to see the output of the /api/health/full REST API call that is used to generate that tree view.
You can obtain this via the dashboard's built-in API browser (click Help icon -> API -> Health -> GET /health/full -> Try it out -> Execute -> save the response body)
Also, do you see any errors in the browser's console when viewing that page?
Thanks in advance!
Lenz
On 6/2/20 4:15 AM, Marco Pizzolo wrote:
> We're working on a new cluster and seeing some oddities. The crush map > viewer is not showing all hosts or OSDs. Cluster is NVMe w/4 hosts, each > having 8 NVMe. Using 2 OSDs per NVMe and Encryption. Using Max size of > 3, Min size of 2: > > image.png > > All OSDs appear to exist in: ceph osd tree > > root@prdhcistonode01:~# ceph osd tree > ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF > -1 372.60156 root default > -3 93.15039 host prdhcistonode01 > 0 ssd 5.82190 osd.0 up 1.00000 1.00000 > 1 ssd 5.82190 osd.1 up 1.00000 1.00000 > 2 ssd 5.82190 osd.2 up 1.00000 1.00000 > 3 ssd 5.82190 osd.3 up 1.00000 1.00000 > 4 ssd 5.82190 osd.4 up 1.00000 1.00000 > 5 ssd 5.82190 osd.5 up 1.00000 1.00000 > 6 ssd 5.82190 osd.6 up 1.00000 1.00000 > 7 ssd 5.82190 osd.7 up 1.00000 1.00000 > 8 ssd 5.82190 osd.8 up 1.00000 1.00000 > 9 ssd 5.82190 osd.9 up 1.00000 1.00000 > 10 ssd 5.82190 osd.10 up 1.00000 1.00000 > 11 ssd 5.82190 osd.11 up 1.00000 1.00000 > 12 ssd 5.82190 osd.12 up 1.00000 1.00000 > 13 ssd 5.82190 osd.13 up 1.00000 1.00000 > 14 ssd 5.82190 osd.14 up 1.00000 1.00000 > 15 ssd 5.82190 osd.15 up 1.00000 1.00000 > -5 93.15039 host prdhcistonode02 > 17 ssd 5.82190 osd.17 up 1.00000 1.00000 > 18 ssd 5.82190 osd.18 up 1.00000 1.00000 > 19 ssd 5.82190 osd.19 up 1.00000 1.00000 > 20 ssd 5.82190 osd.20 up 1.00000 1.00000 > 21 ssd 5.82190 osd.21 up 1.00000 1.00000 > 22 ssd 5.82190 osd.22 up 1.00000 1.00000 > 23 ssd 5.82190 osd.23 up 1.00000 1.00000 > 24 ssd 5.82190 osd.24 up 1.00000 1.00000 > 25 ssd 5.82190 osd.25 up 1.00000 1.00000 > 26 ssd 5.82190 osd.26 up 1.00000 1.00000 > 27 ssd 5.82190 osd.27 up 1.00000 1.00000 > 28 ssd 5.82190 osd.28 up 1.00000 1.00000 > 29 ssd 5.82190 osd.29 up 1.00000 1.00000 > 30 ssd 5.82190 osd.30 up 1.00000 1.00000 > 48 ssd 5.82190 osd.48 up 1.00000 1.00000 > 49 ssd 5.82190 osd.49 up 1.00000 1.00000 > -7 93.15039 host prdhcistonode03 > 16 ssd 5.82190 osd.16 up 1.00000 1.00000 > 31 ssd 5.82190 osd.31 up 1.00000 1.00000 > 32 ssd 5.82190 osd.32 up 1.00000 1.00000 > 33 ssd 5.82190 osd.33 up 1.00000 1.00000 > 34 ssd 5.82190 osd.34 up 1.00000 1.00000 > 35 ssd 5.82190 osd.35 up 1.00000 1.00000 > 36 ssd 5.82190 osd.36 up 1.00000 1.00000 > 37 ssd 5.82190 osd.37 up 1.00000 1.00000 > 38 ssd 5.82190 osd.38 up 1.00000 1.00000 > 39 ssd 5.82190 osd.39 up 1.00000 1.00000 > 40 ssd 5.82190 osd.40 up 1.00000 1.00000 > 41 ssd 5.82190 osd.41 up 1.00000 1.00000 > 42 ssd 5.82190 osd.42 up 1.00000 1.00000 > 43 ssd 5.82190 osd.43 up 1.00000 1.00000 > 44 ssd 5.82190 osd.44 up 1.00000 1.00000 > 45 ssd 5.82190 osd.45 up 1.00000 1.00000 > -9 93.15039 host prdhcistonode04 > 46 ssd 5.82190 osd.46 up 1.00000 1.00000 > 47 ssd 5.82190 osd.47 up 1.00000 1.00000 > 50 ssd 5.82190 osd.50 up 1.00000 1.00000 > 51 ssd 5.82190 osd.51 up 1.00000 1.00000 > 52 ssd 5.82190 osd.52 up 1.00000 1.00000 > 53 ssd 5.82190 osd.53 up 1.00000 1.00000 > 54 ssd 5.82190 osd.54 up 1.00000 1.00000 > 55 ssd 5.82190 osd.55 up 1.00000 1.00000 > 56 ssd 5.82190 osd.56 up 1.00000 1.00000 > 57 ssd 5.82190 osd.57 up 1.00000 1.00000 > 58 ssd 5.82190 osd.58 up 1.00000 1.00000 > 59 ssd 5.82190 osd.59 up 1.00000 1.00000 > 60 ssd 5.82190 osd.60 up 1.00000 1.00000 > 61 ssd 5.82190 osd.61 up 1.00000 1.00000 > 62 ssd 5.82190 osd.62 up 1.00000 1.00000 > 63 ssd 5.82190 osd.63 up 1.00000 1.00000 > > Any suggestions would be greatly appreciated. > > Thanks, > Marco > > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io <mailto:ceph-users@ceph.io> > To unsubscribe send an email to ceph-users-leave@ceph.io <mailto:ceph-users-leave@ceph.io> >
-- SUSE Software Solutions Germany GmbH - Maxfeldstr. 5 - 90409 Nuernberg GF: Felix Imendörffer, HRB 36809 (AG Nürnberg)
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io <mailto:ceph-users@ceph.io> To unsubscribe send an email to ceph-users-leave@ceph.io <mailto:ceph-users-leave@ceph.io>
-- SUSE Software Solutions Germany GmbH - Maxfeldstr. 5 - 90409 Nuernberg GF: Felix Imendörffer, HRB 36809 (AG Nürnberg)
participants (2)
-
Lenz Grimmer
-
Marco Pizzolo