2 Jun
2020
2 Jun
'20
4:15 a.m.
Hello Everyone,
We're working on a new cluster and seeing some oddities. The crush map
viewer is not showing all hosts or OSDs. Cluster is NVMe w/4 hosts, each
having 8 NVMe. Using 2 OSDs per NVMe and Encryption. Using Max size of 3,
Min size of 2:
[image: image.png]
All OSDs appear to exist in: ceph osd tree
root@prdhcistonode01:~# ceph osd tree
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-1 372.60156 root default
-3 93.15039 host prdhcistonode01
0 ssd 5.82190 osd.0 up 1.00000 1.00000
1 ssd 5.82190 osd.1 up 1.00000 1.00000
2 ssd 5.82190 osd.2 up 1.00000 1.00000
3 ssd 5.82190 osd.3 up 1.00000 1.00000
4 ssd 5.82190 osd.4 up 1.00000 1.00000
5 ssd 5.82190 osd.5 up 1.00000 1.00000
6 ssd 5.82190 osd.6 up 1.00000 1.00000
7 ssd 5.82190 osd.7 up 1.00000 1.00000
8 ssd 5.82190 osd.8 up 1.00000 1.00000
9 ssd 5.82190 osd.9 up 1.00000 1.00000
10 ssd 5.82190 osd.10 up 1.00000 1.00000
11 ssd 5.82190 osd.11 up 1.00000 1.00000
12 ssd 5.82190 osd.12 up 1.00000 1.00000
13 ssd 5.82190 osd.13 up 1.00000 1.00000
14 ssd 5.82190 osd.14 up 1.00000 1.00000
15 ssd 5.82190 osd.15 up 1.00000 1.00000
-5 93.15039 host prdhcistonode02
17 ssd 5.82190 osd.17 up 1.00000 1.00000
18 ssd 5.82190 osd.18 up 1.00000 1.00000
19 ssd 5.82190 osd.19 up 1.00000 1.00000
20 ssd 5.82190 osd.20 up 1.00000 1.00000
21 ssd 5.82190 osd.21 up 1.00000 1.00000
22 ssd 5.82190 osd.22 up 1.00000 1.00000
23 ssd 5.82190 osd.23 up 1.00000 1.00000
24 ssd 5.82190 osd.24 up 1.00000 1.00000
25 ssd 5.82190 osd.25 up 1.00000 1.00000
26 ssd 5.82190 osd.26 up 1.00000 1.00000
27 ssd 5.82190 osd.27 up 1.00000 1.00000
28 ssd 5.82190 osd.28 up 1.00000 1.00000
29 ssd 5.82190 osd.29 up 1.00000 1.00000
30 ssd 5.82190 osd.30 up 1.00000 1.00000
48 ssd 5.82190 osd.48 up 1.00000 1.00000
49 ssd 5.82190 osd.49 up 1.00000 1.00000
-7 93.15039 host prdhcistonode03
16 ssd 5.82190 osd.16 up 1.00000 1.00000
31 ssd 5.82190 osd.31 up 1.00000 1.00000
32 ssd 5.82190 osd.32 up 1.00000 1.00000
33 ssd 5.82190 osd.33 up 1.00000 1.00000
34 ssd 5.82190 osd.34 up 1.00000 1.00000
35 ssd 5.82190 osd.35 up 1.00000 1.00000
36 ssd 5.82190 osd.36 up 1.00000 1.00000
37 ssd 5.82190 osd.37 up 1.00000 1.00000
38 ssd 5.82190 osd.38 up 1.00000 1.00000
39 ssd 5.82190 osd.39 up 1.00000 1.00000
40 ssd 5.82190 osd.40 up 1.00000 1.00000
41 ssd 5.82190 osd.41 up 1.00000 1.00000
42 ssd 5.82190 osd.42 up 1.00000 1.00000
43 ssd 5.82190 osd.43 up 1.00000 1.00000
44 ssd 5.82190 osd.44 up 1.00000 1.00000
45 ssd 5.82190 osd.45 up 1.00000 1.00000
-9 93.15039 host prdhcistonode04
46 ssd 5.82190 osd.46 up 1.00000 1.00000
47 ssd 5.82190 osd.47 up 1.00000 1.00000
50 ssd 5.82190 osd.50 up 1.00000 1.00000
51 ssd 5.82190 osd.51 up 1.00000 1.00000
52 ssd 5.82190 osd.52 up 1.00000 1.00000
53 ssd 5.82190 osd.53 up 1.00000 1.00000
54 ssd 5.82190 osd.54 up 1.00000 1.00000
55 ssd 5.82190 osd.55 up 1.00000 1.00000
56 ssd 5.82190 osd.56 up 1.00000 1.00000
57 ssd 5.82190 osd.57 up 1.00000 1.00000
58 ssd 5.82190 osd.58 up 1.00000 1.00000
59 ssd 5.82190 osd.59 up 1.00000 1.00000
60 ssd 5.82190 osd.60 up 1.00000 1.00000
61 ssd 5.82190 osd.61 up 1.00000 1.00000
62 ssd 5.82190 osd.62 up 1.00000 1.00000
63 ssd 5.82190 osd.63 up 1.00000 1.00000
Any suggestions would be greatly appreciated.
Thanks,
Marco
2 Jun
2 Jun
12:47 p.m.
Hi Marco,
looks like you have found a bug. It would be helpful to see the output
of the /api/health/full REST API call that is used to generate that tree
view.
You can obtain this via the dashboard's built-in API browser (click Help
icon -> API -> Health -> GET /health/full -> Try it out -> Execute ->
save the response body)
Also, do you see any errors in the browser's console when viewing that page?
Thanks in advance!
Lenz
On 6/2/20 4:15 AM, Marco Pizzolo wrote:
We're working on a new cluster and seeing some
oddities. The crush map
viewer is not showing all hosts or OSDs. Cluster is NVMe w/4 hosts, each
having 8 NVMe. Using 2 OSDs per NVMe and Encryption. Using Max size of
3, Min size of 2:
image.png
All OSDs appear to exist in: ceph osd tree
root@prdhcistonode01:~# ceph osd tree
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-1 372.60156 root default
-3 93.15039 host prdhcistonode01
0 ssd 5.82190 osd.0 up 1.00000 1.00000
1 ssd 5.82190 osd.1 up 1.00000 1.00000
2 ssd 5.82190 osd.2 up 1.00000 1.00000
3 ssd 5.82190 osd.3 up 1.00000 1.00000
4 ssd 5.82190 osd.4 up 1.00000 1.00000
5 ssd 5.82190 osd.5 up 1.00000 1.00000
6 ssd 5.82190 osd.6 up 1.00000 1.00000
7 ssd 5.82190 osd.7 up 1.00000 1.00000
8 ssd 5.82190 osd.8 up 1.00000 1.00000
9 ssd 5.82190 osd.9 up 1.00000 1.00000
10 ssd 5.82190 osd.10 up 1.00000 1.00000
11 ssd 5.82190 osd.11 up 1.00000 1.00000
12 ssd 5.82190 osd.12 up 1.00000 1.00000
13 ssd 5.82190 osd.13 up 1.00000 1.00000
14 ssd 5.82190 osd.14 up 1.00000 1.00000
15 ssd 5.82190 osd.15 up 1.00000 1.00000
-5 93.15039 host prdhcistonode02
17 ssd 5.82190 osd.17 up 1.00000 1.00000
18 ssd 5.82190 osd.18 up 1.00000 1.00000
19 ssd 5.82190 osd.19 up 1.00000 1.00000
20 ssd 5.82190 osd.20 up 1.00000 1.00000
21 ssd 5.82190 osd.21 up 1.00000 1.00000
22 ssd 5.82190 osd.22 up 1.00000 1.00000
23 ssd 5.82190 osd.23 up 1.00000 1.00000
24 ssd 5.82190 osd.24 up 1.00000 1.00000
25 ssd 5.82190 osd.25 up 1.00000 1.00000
26 ssd 5.82190 osd.26 up 1.00000 1.00000
27 ssd 5.82190 osd.27 up 1.00000 1.00000
28 ssd 5.82190 osd.28 up 1.00000 1.00000
29 ssd 5.82190 osd.29 up 1.00000 1.00000
30 ssd 5.82190 osd.30 up 1.00000 1.00000
48 ssd 5.82190 osd.48 up 1.00000 1.00000
49 ssd 5.82190 osd.49 up 1.00000 1.00000
-7 93.15039 host prdhcistonode03
16 ssd 5.82190 osd.16 up 1.00000 1.00000
31 ssd 5.82190 osd.31 up 1.00000 1.00000
32 ssd 5.82190 osd.32 up 1.00000 1.00000
33 ssd 5.82190 osd.33 up 1.00000 1.00000
34 ssd 5.82190 osd.34 up 1.00000 1.00000
35 ssd 5.82190 osd.35 up 1.00000 1.00000
36 ssd 5.82190 osd.36 up 1.00000 1.00000
37 ssd 5.82190 osd.37 up 1.00000 1.00000
38 ssd 5.82190 osd.38 up 1.00000 1.00000
39 ssd 5.82190 osd.39 up 1.00000 1.00000
40 ssd 5.82190 osd.40 up 1.00000 1.00000
41 ssd 5.82190 osd.41 up 1.00000 1.00000
42 ssd 5.82190 osd.42 up 1.00000 1.00000
43 ssd 5.82190 osd.43 up 1.00000 1.00000
44 ssd 5.82190 osd.44 up 1.00000 1.00000
45 ssd 5.82190 osd.45 up 1.00000 1.00000
-9 93.15039 host prdhcistonode04
46 ssd 5.82190 osd.46 up 1.00000 1.00000
47 ssd 5.82190 osd.47 up 1.00000 1.00000
50 ssd 5.82190 osd.50 up 1.00000 1.00000
51 ssd 5.82190 osd.51 up 1.00000 1.00000
52 ssd 5.82190 osd.52 up 1.00000 1.00000
53 ssd 5.82190 osd.53 up 1.00000 1.00000
54 ssd 5.82190 osd.54 up 1.00000 1.00000
55 ssd 5.82190 osd.55 up 1.00000 1.00000
56 ssd 5.82190 osd.56 up 1.00000 1.00000
57 ssd 5.82190 osd.57 up 1.00000 1.00000
58 ssd 5.82190 osd.58 up 1.00000 1.00000
59 ssd 5.82190 osd.59 up 1.00000 1.00000
60 ssd 5.82190 osd.60 up 1.00000 1.00000
61 ssd 5.82190 osd.61 up 1.00000 1.00000
62 ssd 5.82190 osd.62 up 1.00000 1.00000
63 ssd 5.82190 osd.63 up 1.00000 1.00000
Any suggestions would be greatly appreciated.
Thanks,
Marco
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
--
SUSE Software Solutions Germany GmbH - Maxfeldstr. 5 - 90409 Nuernberg
GF: Felix Imendörffer, HRB 36809 (AG Nürnberg)
5:03 p.m.
Hi Lenz,
Hopefully this is the part that was required:
"tree": {
"nodes": [
{
"id": -1,
"name": "default",
"type": "root",
"type_id": 11,
"children": [
-9,
-7,
-5,
-3
]
},
{
"id": -3,
"name": "prdhcistonode01",
"type": "host",
"type_id": 1,
"pool_weights": {},
"children": [
15,
14,
13,
12,
11,
10,
9,
8,
7,
6,
5,
4,
3,
2,
1,
0
]
},
{
"id": 0,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.0"
},
{
"id": 1,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.1"
},
{
"id": 2,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.2"
},
{
"id": 3,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.3"
},
{
"id": 4,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.4"
},
{
"id": 5,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.5"
},
{
"id": 6,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.6"
},
{
"id": 7,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.7"
},
{
"id": 8,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.8"
},
{
"id": 9,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.9"
},
{
"id": 10,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.10"
},
{
"id": 11,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.11"
},
{
"id": 12,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.12"
},
{
"id": 13,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.13"
},
{
"id": 14,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.14"
},
{
"id": 15,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.15"
},
{
"id": -5,
"name": "prdhcistonode02",
"type": "host",
"type_id": 1,
"pool_weights": {},
"children": [
49,
48,
30,
29,
28,
27,
26,
25,
24,
23,
22,
21,
20,
19,
18,
17
]
},
{
"id": 17,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.17"
},
{
"id": 18,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.18"
},
{
"id": 19,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.19"
},
{
"id": 20,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.20"
},
{
"id": 21,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.21"
},
{
"id": 22,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.22"
},
{
"id": 23,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.23"
},
{
"id": 24,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.24"
},
{
"id": 25,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.25"
},
{
"id": 26,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.26"
},
{
"id": 27,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.27"
},
{
"id": 28,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.28"
},
{
"id": 29,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.29"
},
{
"id": 30,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.30"
},
{
"id": 48,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.48"
},
{
"id": 49,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.49"
},
{
"id": -7,
"name": "prdhcistonode03",
"type": "host",
"type_id": 1,
"pool_weights": {},
"children": [
45,
44,
43,
42,
41,
40,
39,
38,
37,
36,
35,
34,
33,
32,
31,
16
]
},
{
"id": 16,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.16"
},
{
"id": 31,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.31"
},
{
"id": 32,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.32"
},
{
"id": 33,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.33"
},
{
"id": 34,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.34"
},
{
"id": 35,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.35"
},
{
"id": 36,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.36"
},
{
"id": 37,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.37"
},
{
"id": 38,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.38"
},
{
"id": 39,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.39"
},
{
"id": 40,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.40"
},
{
"id": 41,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.41"
},
{
"id": 42,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.42"
},
{
"id": 43,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.43"
},
{
"id": 44,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.44"
},
{
"id": 45,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.45"
},
{
"id": -9,
"name": "prdhcistonode04",
"type": "host",
"type_id": 1,
"pool_weights": {},
"children": [
63,
62,
61,
60,
59,
58,
57,
56,
55,
54,
53,
52,
51,
50,
47,
46
]
},
{
"id": 46,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.46"
},
{
"id": 47,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.47"
},
{
"id": 50,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.50"
},
{
"id": 51,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.51"
},
{
"id": 52,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.52"
},
{
"id": 53,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.53"
},
{
"id": 54,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.54"
},
{
"id": 55,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.55"
},
{
"id": 56,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.56"
},
{
"id": 57,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.57"
},
{
"id": 58,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.58"
},
{
"id": 59,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.59"
},
{
"id": 60,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.60"
},
{
"id": 61,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.61"
},
{
"id": 62,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.62"
},
{
"id": 63,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.63"
}
],
"stray": []
},
"crush": {
"devices": [
{
"id": 0,
"name": "osd.0",
"class": "ssd"
},
{
"id": 1,
"name": "osd.1",
"class": "ssd"
},
{
"id": 2,
"name": "osd.2",
"class": "ssd"
},
{
"id": 3,
"name": "osd.3",
"class": "ssd"
},
{
"id": 4,
"name": "osd.4",
"class": "ssd"
},
{
"id": 5,
"name": "osd.5",
"class": "ssd"
},
{
"id": 6,
"name": "osd.6",
"class": "ssd"
},
{
"id": 7,
"name": "osd.7",
"class": "ssd"
},
{
"id": 8,
"name": "osd.8",
"class": "ssd"
},
{
"id": 9,
"name": "osd.9",
"class": "ssd"
},
{
"id": 10,
"name": "osd.10",
"class": "ssd"
},
{
"id": 11,
"name": "osd.11",
"class": "ssd"
},
{
"id": 12,
"name": "osd.12",
"class": "ssd"
},
{
"id": 13,
"name": "osd.13",
"class": "ssd"
},
{
"id": 14,
"name": "osd.14",
"class": "ssd"
},
{
"id": 15,
"name": "osd.15",
"class": "ssd"
},
{
"id": 16,
"name": "osd.16",
"class": "ssd"
},
{
"id": 17,
"name": "osd.17",
"class": "ssd"
},
{
"id": 18,
"name": "osd.18",
"class": "ssd"
},
{
"id": 19,
"name": "osd.19",
"class": "ssd"
},
{
"id": 20,
"name": "osd.20",
"class": "ssd"
},
{
"id": 21,
"name": "osd.21",
"class": "ssd"
},
{
"id": 22,
"name": "osd.22",
"class": "ssd"
},
{
"id": 23,
"name": "osd.23",
"class": "ssd"
},
{
"id": 24,
"name": "osd.24",
"class": "ssd"
},
{
"id": 25,
"name": "osd.25",
"class": "ssd"
},
{
"id": 26,
"name": "osd.26",
"class": "ssd"
},
{
"id": 27,
"name": "osd.27",
"class": "ssd"
},
{
"id": 28,
"name": "osd.28",
"class": "ssd"
},
{
"id": 29,
"name": "osd.29",
"class": "ssd"
},
{
"id": 30,
"name": "osd.30",
"class": "ssd"
},
{
"id": 31,
"name": "osd.31",
"class": "ssd"
},
{
"id": 32,
"name": "osd.32",
"class": "ssd"
},
{
"id": 33,
"name": "osd.33",
"class": "ssd"
},
{
"id": 34,
"name": "osd.34",
"class": "ssd"
},
{
"id": 35,
"name": "osd.35",
"class": "ssd"
},
{
"id": 36,
"name": "osd.36",
"class": "ssd"
},
{
"id": 37,
"name": "osd.37",
"class": "ssd"
},
{
"id": 38,
"name": "osd.38",
"class": "ssd"
},
{
"id": 39,
"name": "osd.39",
"class": "ssd"
},
{
"id": 40,
"name": "osd.40",
"class": "ssd"
},
{
"id": 41,
"name": "osd.41",
"class": "ssd"
},
{
"id": 42,
"name": "osd.42",
"class": "ssd"
},
{
"id": 43,
"name": "osd.43",
"class": "ssd"
},
{
"id": 44,
"name": "osd.44",
"class": "ssd"
},
{
"id": 45,
"name": "osd.45",
"class": "ssd"
},
{
"id": 46,
"name": "osd.46",
"class": "ssd"
},
{
"id": 47,
"name": "osd.47",
"class": "ssd"
},
{
"id": 48,
"name": "osd.48",
"class": "ssd"
},
{
"id": 49,
"name": "osd.49",
"class": "ssd"
},
{
"id": 50,
"name": "osd.50",
"class": "ssd"
},
{
"id": 51,
"name": "osd.51",
"class": "ssd"
},
{
"id": 52,
"name": "osd.52",
"class": "ssd"
},
{
"id": 53,
"name": "osd.53",
"class": "ssd"
},
{
"id": 54,
"name": "osd.54",
"class": "ssd"
},
{
"id": 55,
"name": "osd.55",
"class": "ssd"
},
{
"id": 56,
"name": "osd.56",
"class": "ssd"
},
{
"id": 57,
"name": "osd.57",
"class": "ssd"
},
{
"id": 58,
"name": "osd.58",
"class": "ssd"
},
{
"id": 59,
"name": "osd.59",
"class": "ssd"
},
{
"id": 60,
"name": "osd.60",
"class": "ssd"
},
{
"id": 61,
"name": "osd.61",
"class": "ssd"
},
{
"id": 62,
"name": "osd.62",
"class": "ssd"
},
{
"id": 63,
"name": "osd.63",
"class": "ssd"
}
],
"types": [
{
"type_id": 0,
"name": "osd"
},
{
"type_id": 1,
"name": "host"
},
{
"type_id": 2,
"name": "chassis"
},
{
"type_id": 3,
"name": "rack"
},
{
"type_id": 4,
"name": "row"
},
{
"type_id": 5,
"name": "pdu"
},
{
"type_id": 6,
"name": "pod"
},
{
"type_id": 7,
"name": "room"
},
{
"type_id": 8,
"name": "datacenter"
},
{
"type_id": 9,
"name": "zone"
},
{
"type_id": 10,
"name": "region"
},
{
"type_id": 11,
"name": "root"
}
],
On Tue, Jun 2, 2020 at 6:48 AM Lenz Grimmer <lgrimmer(a)suse.com> wrote:
Hi Marco,
looks like you have found a bug. It would be helpful to see the output
of the /api/health/full REST API call that is used to generate that tree
view.
You can obtain this via the dashboard's built-in API browser (click Help
icon -> API -> Health -> GET /health/full -> Try it out -> Execute ->
save the response body)
Also, do you see any errors in the browser's console when viewing that
page?
Thanks in advance!
Lenz
On 6/2/20 4:15 AM, Marco Pizzolo wrote:
We're working on a new cluster and seeing
some oddities. The crush map
viewer is not showing all hosts or OSDs. Cluster is NVMe w/4 hosts, each
having 8 NVMe. Using 2 OSDs per NVMe and Encryption. Using Max size of
3, Min size of 2:
image.png
All OSDs appear to exist in: ceph osd tree
root@prdhcistonode01:~# ceph osd tree
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-1 372.60156 root default
-3 93.15039 host prdhcistonode01
0 ssd 5.82190 osd.0 up 1.00000 1.00000
1 ssd 5.82190 osd.1 up 1.00000 1.00000
2 ssd 5.82190 osd.2 up 1.00000 1.00000
3 ssd 5.82190 osd.3 up 1.00000 1.00000
4 ssd 5.82190 osd.4 up 1.00000 1.00000
5 ssd 5.82190 osd.5 up 1.00000 1.00000
6 ssd 5.82190 osd.6 up 1.00000 1.00000
7 ssd 5.82190 osd.7 up 1.00000 1.00000
8 ssd 5.82190 osd.8 up 1.00000 1.00000
9 ssd 5.82190 osd.9 up 1.00000 1.00000
10 ssd 5.82190 osd.10 up 1.00000 1.00000
11 ssd 5.82190 osd.11 up 1.00000 1.00000
12 ssd 5.82190 osd.12 up 1.00000 1.00000
13 ssd 5.82190 osd.13 up 1.00000 1.00000
14 ssd 5.82190 osd.14 up 1.00000 1.00000
15 ssd 5.82190 osd.15 up 1.00000 1.00000
-5 93.15039 host prdhcistonode02
17 ssd 5.82190 osd.17 up 1.00000 1.00000
18 ssd 5.82190 osd.18 up 1.00000 1.00000
19 ssd 5.82190 osd.19 up 1.00000 1.00000
20 ssd 5.82190 osd.20 up 1.00000 1.00000
21 ssd 5.82190 osd.21 up 1.00000 1.00000
22 ssd 5.82190 osd.22 up 1.00000 1.00000
23 ssd 5.82190 osd.23 up 1.00000 1.00000
24 ssd 5.82190 osd.24 up 1.00000 1.00000
25 ssd 5.82190 osd.25 up 1.00000 1.00000
26 ssd 5.82190 osd.26 up 1.00000 1.00000
27 ssd 5.82190 osd.27 up 1.00000 1.00000
28 ssd 5.82190 osd.28 up 1.00000 1.00000
29 ssd 5.82190 osd.29 up 1.00000 1.00000
30 ssd 5.82190 osd.30 up 1.00000 1.00000
48 ssd 5.82190 osd.48 up 1.00000 1.00000
49 ssd 5.82190 osd.49 up 1.00000 1.00000
-7 93.15039 host prdhcistonode03
16 ssd 5.82190 osd.16 up 1.00000 1.00000
31 ssd 5.82190 osd.31 up 1.00000 1.00000
32 ssd 5.82190 osd.32 up 1.00000 1.00000
33 ssd 5.82190 osd.33 up 1.00000 1.00000
34 ssd 5.82190 osd.34 up 1.00000 1.00000
35 ssd 5.82190 osd.35 up 1.00000 1.00000
36 ssd 5.82190 osd.36 up 1.00000 1.00000
37 ssd 5.82190 osd.37 up 1.00000 1.00000
38 ssd 5.82190 osd.38 up 1.00000 1.00000
39 ssd 5.82190 osd.39 up 1.00000 1.00000
40 ssd 5.82190 osd.40 up 1.00000 1.00000
41 ssd 5.82190 osd.41 up 1.00000 1.00000
42 ssd 5.82190 osd.42 up 1.00000 1.00000
43 ssd 5.82190 osd.43 up 1.00000 1.00000
44 ssd 5.82190 osd.44 up 1.00000 1.00000
45 ssd 5.82190 osd.45 up 1.00000 1.00000
-9 93.15039 host prdhcistonode04
46 ssd 5.82190 osd.46 up 1.00000 1.00000
47 ssd 5.82190 osd.47 up 1.00000 1.00000
50 ssd 5.82190 osd.50 up 1.00000 1.00000
51 ssd 5.82190 osd.51 up 1.00000 1.00000
52 ssd 5.82190 osd.52 up 1.00000 1.00000
53 ssd 5.82190 osd.53 up 1.00000 1.00000
54 ssd 5.82190 osd.54 up 1.00000 1.00000
55 ssd 5.82190 osd.55 up 1.00000 1.00000
56 ssd 5.82190 osd.56 up 1.00000 1.00000
57 ssd 5.82190 osd.57 up 1.00000 1.00000
58 ssd 5.82190 osd.58 up 1.00000 1.00000
59 ssd 5.82190 osd.59 up 1.00000 1.00000
60 ssd 5.82190 osd.60 up 1.00000 1.00000
61 ssd 5.82190 osd.61 up 1.00000 1.00000
62 ssd 5.82190 osd.62 up 1.00000 1.00000
63 ssd 5.82190 osd.63 up 1.00000 1.00000
Any suggestions would be greatly appreciated.
Thanks,
Marco
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
--
SUSE Software Solutions Germany GmbH - Maxfeldstr. 5 - 90409 Nuernberg
GF: Felix Imendörffer, HRB 36809 (AG Nürnberg)
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
4 Jun
4 Jun
9:59 a.m.
Hi Marco,
thank you. It seems as if the REST API output matches the output of
"ceph osd tree", but the tree view in the dashboard somehow fails to
display all nodes. We will investigate this.
I've now submitted your report as a bug on our tracker:
https://tracker.ceph.com/issues/45873
Please make sure to follow this issue in case we need additional
information. Thank you!
Lenz
On 6/2/20 5:03 PM, Marco Pizzolo wrote:
Hopefully this is the part that was required:
"tree": {
"nodes": [
{
"id": -1,
"name": "default",
"type": "root",
"type_id": 11,
"children": [
-9,
-7,
-5,
-3
]
},
{
"id": -3,
"name": "prdhcistonode01",
"type": "host",
"type_id": 1,
"pool_weights": {},
"children": [
15,
14,
13,
12,
11,
10,
9,
8,
7,
6,
5,
4,
3,
2,
1,
0
]
},
{
"id": 0,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.0"
},
{
"id": 1,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.1"
},
{
"id": 2,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.2"
},
{
"id": 3,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.3"
},
{
"id": 4,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.4"
},
{
"id": 5,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.5"
},
{
"id": 6,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.6"
},
{
"id": 7,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.7"
},
{
"id": 8,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.8"
},
{
"id": 9,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.9"
},
{
"id": 10,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.10"
},
{
"id": 11,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.11"
},
{
"id": 12,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.12"
},
{
"id": 13,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.13"
},
{
"id": 14,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.14"
},
{
"id": 15,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.15"
},
{
"id": -5,
"name": "prdhcistonode02",
"type": "host",
"type_id": 1,
"pool_weights": {},
"children": [
49,
48,
30,
29,
28,
27,
26,
25,
24,
23,
22,
21,
20,
19,
18,
17
]
},
{
"id": 17,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.17"
},
{
"id": 18,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.18"
},
{
"id": 19,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.19"
},
{
"id": 20,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.20"
},
{
"id": 21,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.21"
},
{
"id": 22,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.22"
},
{
"id": 23,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.23"
},
{
"id": 24,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.24"
},
{
"id": 25,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.25"
},
{
"id": 26,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.26"
},
{
"id": 27,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.27"
},
{
"id": 28,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.28"
},
{
"id": 29,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.29"
},
{
"id": 30,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.30"
},
{
"id": 48,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.48"
},
{
"id": 49,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.49"
},
{
"id": -7,
"name": "prdhcistonode03",
"type": "host",
"type_id": 1,
"pool_weights": {},
"children": [
45,
44,
43,
42,
41,
40,
39,
38,
37,
36,
35,
34,
33,
32,
31,
16
]
},
{
"id": 16,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.16"
},
{
"id": 31,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.31"
},
{
"id": 32,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.32"
},
{
"id": 33,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.33"
},
{
"id": 34,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.34"
},
{
"id": 35,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.35"
},
{
"id": 36,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.36"
},
{
"id": 37,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.37"
},
{
"id": 38,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.38"
},
{
"id": 39,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.39"
},
{
"id": 40,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.40"
},
{
"id": 41,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.41"
},
{
"id": 42,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.42"
},
{
"id": 43,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.43"
},
{
"id": 44,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.44"
},
{
"id": 45,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.45"
},
{
"id": -9,
"name": "prdhcistonode04",
"type": "host",
"type_id": 1,
"pool_weights": {},
"children": [
63,
62,
61,
60,
59,
58,
57,
56,
55,
54,
53,
52,
51,
50,
47,
46
]
},
{
"id": 46,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.46"
},
{
"id": 47,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.47"
},
{
"id": 50,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.50"
},
{
"id": 51,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.51"
},
{
"id": 52,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.52"
},
{
"id": 53,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.53"
},
{
"id": 54,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.54"
},
{
"id": 55,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.55"
},
{
"id": 56,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.56"
},
{
"id": 57,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.57"
},
{
"id": 58,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.58"
},
{
"id": 59,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.59"
},
{
"id": 60,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.60"
},
{
"id": 61,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.61"
},
{
"id": 62,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.62"
},
{
"id": 63,
"device_class": "ssd",
"type": "osd",
"type_id": 0,
"crush_weight": 5.8218994140625,
"depth": 2,
"pool_weights": {},
"exists": 1,
"status": "up",
"reweight": 1,
"primary_affinity": 1,
"name": "osd.63"
}
],
"stray": []
},
"crush": {
"devices": [
{
"id": 0,
"name": "osd.0",
"class": "ssd"
},
{
"id": 1,
"name": "osd.1",
"class": "ssd"
},
{
"id": 2,
"name": "osd.2",
"class": "ssd"
},
{
"id": 3,
"name": "osd.3",
"class": "ssd"
},
{
"id": 4,
"name": "osd.4",
"class": "ssd"
},
{
"id": 5,
"name": "osd.5",
"class": "ssd"
},
{
"id": 6,
"name": "osd.6",
"class": "ssd"
},
{
"id": 7,
"name": "osd.7",
"class": "ssd"
},
{
"id": 8,
"name": "osd.8",
"class": "ssd"
},
{
"id": 9,
"name": "osd.9",
"class": "ssd"
},
{
"id": 10,
"name": "osd.10",
"class": "ssd"
},
{
"id": 11,
"name": "osd.11",
"class": "ssd"
},
{
"id": 12,
"name": "osd.12",
"class": "ssd"
},
{
"id": 13,
"name": "osd.13",
"class": "ssd"
},
{
"id": 14,
"name": "osd.14",
"class": "ssd"
},
{
"id": 15,
"name": "osd.15",
"class": "ssd"
},
{
"id": 16,
"name": "osd.16",
"class": "ssd"
},
{
"id": 17,
"name": "osd.17",
"class": "ssd"
},
{
"id": 18,
"name": "osd.18",
"class": "ssd"
},
{
"id": 19,
"name": "osd.19",
"class": "ssd"
},
{
"id": 20,
"name": "osd.20",
"class": "ssd"
},
{
"id": 21,
"name": "osd.21",
"class": "ssd"
},
{
"id": 22,
"name": "osd.22",
"class": "ssd"
},
{
"id": 23,
"name": "osd.23",
"class": "ssd"
},
{
"id": 24,
"name": "osd.24",
"class": "ssd"
},
{
"id": 25,
"name": "osd.25",
"class": "ssd"
},
{
"id": 26,
"name": "osd.26",
"class": "ssd"
},
{
"id": 27,
"name": "osd.27",
"class": "ssd"
},
{
"id": 28,
"name": "osd.28",
"class": "ssd"
},
{
"id": 29,
"name": "osd.29",
"class": "ssd"
},
{
"id": 30,
"name": "osd.30",
"class": "ssd"
},
{
"id": 31,
"name": "osd.31",
"class": "ssd"
},
{
"id": 32,
"name": "osd.32",
"class": "ssd"
},
{
"id": 33,
"name": "osd.33",
"class": "ssd"
},
{
"id": 34,
"name": "osd.34",
"class": "ssd"
},
{
"id": 35,
"name": "osd.35",
"class": "ssd"
},
{
"id": 36,
"name": "osd.36",
"class": "ssd"
},
{
"id": 37,
"name": "osd.37",
"class": "ssd"
},
{
"id": 38,
"name": "osd.38",
"class": "ssd"
},
{
"id": 39,
"name": "osd.39",
"class": "ssd"
},
{
"id": 40,
"name": "osd.40",
"class": "ssd"
},
{
"id": 41,
"name": "osd.41",
"class": "ssd"
},
{
"id": 42,
"name": "osd.42",
"class": "ssd"
},
{
"id": 43,
"name": "osd.43",
"class": "ssd"
},
{
"id": 44,
"name": "osd.44",
"class": "ssd"
},
{
"id": 45,
"name": "osd.45",
"class": "ssd"
},
{
"id": 46,
"name": "osd.46",
"class": "ssd"
},
{
"id": 47,
"name": "osd.47",
"class": "ssd"
},
{
"id": 48,
"name": "osd.48",
"class": "ssd"
},
{
"id": 49,
"name": "osd.49",
"class": "ssd"
},
{
"id": 50,
"name": "osd.50",
"class": "ssd"
},
{
"id": 51,
"name": "osd.51",
"class": "ssd"
},
{
"id": 52,
"name": "osd.52",
"class": "ssd"
},
{
"id": 53,
"name": "osd.53",
"class": "ssd"
},
{
"id": 54,
"name": "osd.54",
"class": "ssd"
},
{
"id": 55,
"name": "osd.55",
"class": "ssd"
},
{
"id": 56,
"name": "osd.56",
"class": "ssd"
},
{
"id": 57,
"name": "osd.57",
"class": "ssd"
},
{
"id": 58,
"name": "osd.58",
"class": "ssd"
},
{
"id": 59,
"name": "osd.59",
"class": "ssd"
},
{
"id": 60,
"name": "osd.60",
"class": "ssd"
},
{
"id": 61,
"name": "osd.61",
"class": "ssd"
},
{
"id": 62,
"name": "osd.62",
"class": "ssd"
},
{
"id": 63,
"name": "osd.63",
"class": "ssd"
}
],
"types": [
{
"type_id": 0,
"name": "osd"
},
{
"type_id": 1,
"name": "host"
},
{
"type_id": 2,
"name": "chassis"
},
{
"type_id": 3,
"name": "rack"
},
{
"type_id": 4,
"name": "row"
},
{
"type_id": 5,
"name": "pdu"
},
{
"type_id": 6,
"name": "pod"
},
{
"type_id": 7,
"name": "room"
},
{
"type_id": 8,
"name": "datacenter"
},
{
"type_id": 9,
"name": "zone"
},
{
"type_id": 10,
"name": "region"
},
{
"type_id": 11,
"name": "root"
}
],
On Tue, Jun 2, 2020 at 6:48 AM Lenz Grimmer <lgrimmer(a)suse.com
<mailto:lgrimmer@suse.com>> wrote:
Hi Marco,
looks like you have found a bug. It would be helpful to see the output
of the /api/health/full REST API call that is used to generate that tree
view.
You can obtain this via the dashboard's built-in API browser (click Help
icon -> API -> Health -> GET /health/full -> Try it out -> Execute
->
save the response body)
Also, do you see any errors in the browser's console when viewing
that page?
Thanks in advance!
Lenz
On 6/2/20 4:15 AM, Marco Pizzolo wrote:
--
SUSE Software Solutions Germany GmbH - Maxfeldstr. 5 - 90409 Nuernberg
GF: Felix Imendörffer, HRB 36809 (AG Nürnberg)
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
<mailto:ceph-users@ceph.io>
To unsubscribe send an email to ceph-users-leave(a)ceph.io
<mailto:ceph-users-leave@ceph.io>
--
SUSE Software Solutions Germany GmbH - Maxfeldstr. 5 - 90409 Nuernberg
GF: Felix Imendörffer, HRB 36809 (AG Nürnberg)
We're working on a new cluster and seeing
some oddities. The
crush map
viewer is not showing all hosts or OSDs. Cluster
is NVMe w/4
hosts, each
having 8 NVMe. Using 2 OSDs per NVMe and
Encryption. Using Max
size of
3, Min size of 2:
image.png
All OSDs appear to exist in: ceph osd tree
root@prdhcistonode01:~# ceph osd tree
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT
PRI-AFF
-1 372.60156 root default
-3 93.15039 host prdhcistonode01
0 ssd 5.82190 osd.0 up 1.00000
1.00000
1 ssd 5.82190 osd.1
up 1.00000
1.00000
2 ssd 5.82190 osd.2
up 1.00000
1.00000
3 ssd 5.82190 osd.3
up 1.00000
1.00000
4 ssd 5.82190 osd.4
up 1.00000
1.00000
5 ssd 5.82190 osd.5
up 1.00000
1.00000
6 ssd 5.82190 osd.6
up 1.00000
1.00000
7 ssd 5.82190 osd.7
up 1.00000
1.00000
8 ssd 5.82190 osd.8
up 1.00000
1.00000
9 ssd 5.82190 osd.9
up 1.00000
1.00000
10 ssd 5.82190 osd.10
up 1.00000
1.00000
11 ssd 5.82190 osd.11
up 1.00000
1.00000
12 ssd 5.82190 osd.12
up 1.00000
1.00000
13 ssd 5.82190 osd.13
up 1.00000
1.00000
14 ssd 5.82190 osd.14
up 1.00000
1.00000
15 ssd 5.82190 osd.15
up 1.00000
1.00000
-5 93.15039 host prdhcistonode02
17 ssd 5.82190 osd.17 up 1.00000
1.00000
18 ssd 5.82190 osd.18
up 1.00000
1.00000
19 ssd 5.82190 osd.19
up 1.00000
1.00000
20 ssd 5.82190 osd.20
up 1.00000
1.00000
21 ssd 5.82190 osd.21
up 1.00000
1.00000
22 ssd 5.82190 osd.22
up 1.00000
1.00000
23 ssd 5.82190 osd.23
up 1.00000
1.00000
24 ssd 5.82190 osd.24
up 1.00000
1.00000
25 ssd 5.82190 osd.25
up 1.00000
1.00000
26 ssd 5.82190 osd.26
up 1.00000
1.00000
27 ssd 5.82190 osd.27
up 1.00000
1.00000
28 ssd 5.82190 osd.28
up 1.00000
1.00000
29 ssd 5.82190 osd.29
up 1.00000
1.00000
30 ssd 5.82190 osd.30
up 1.00000
1.00000
48 ssd 5.82190 osd.48
up 1.00000
1.00000
49 ssd 5.82190 osd.49
up 1.00000
1.00000
-7 93.15039 host prdhcistonode03
16 ssd 5.82190 osd.16 up 1.00000
1.00000
31 ssd 5.82190 osd.31
up 1.00000
1.00000
32 ssd 5.82190 osd.32
up 1.00000
1.00000
33 ssd 5.82190 osd.33
up 1.00000
1.00000
34 ssd 5.82190 osd.34
up 1.00000
1.00000
35 ssd 5.82190 osd.35
up 1.00000
1.00000
36 ssd 5.82190 osd.36
up 1.00000
1.00000
37 ssd 5.82190 osd.37
up 1.00000
1.00000
38 ssd 5.82190 osd.38
up 1.00000
1.00000
39 ssd 5.82190 osd.39
up 1.00000
1.00000
40 ssd 5.82190 osd.40
up 1.00000
1.00000
41 ssd 5.82190 osd.41
up 1.00000
1.00000
42 ssd 5.82190 osd.42
up 1.00000
1.00000
43 ssd 5.82190 osd.43
up 1.00000
1.00000
44 ssd 5.82190 osd.44
up 1.00000
1.00000
45 ssd 5.82190 osd.45
up 1.00000
1.00000
-9 93.15039 host prdhcistonode04
46 ssd 5.82190 osd.46 up 1.00000
1.00000
47 ssd 5.82190 osd.47
up 1.00000
1.00000
50 ssd 5.82190 osd.50
up 1.00000
1.00000
51 ssd 5.82190 osd.51
up 1.00000
1.00000
52 ssd 5.82190 osd.52
up 1.00000
1.00000
53 ssd 5.82190 osd.53
up 1.00000
1.00000
54 ssd 5.82190 osd.54
up 1.00000
1.00000
55 ssd 5.82190 osd.55
up 1.00000
1.00000
56 ssd 5.82190 osd.56
up 1.00000
1.00000
57 ssd 5.82190 osd.57
up 1.00000
1.00000
58 ssd 5.82190 osd.58
up 1.00000
1.00000
59 ssd 5.82190 osd.59
up 1.00000
1.00000
60 ssd 5.82190 osd.60
up 1.00000
1.00000
61 ssd 5.82190 osd.61
up 1.00000
1.00000
62 ssd 5.82190 osd.62
up 1.00000
1.00000
63 ssd 5.82190 osd.63
up 1.00000
1.00000
Any suggestions would be greatly appreciated.
Thanks,
Marco
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
<mailto:ceph-users@ceph.io>
To unsubscribe send an email to
ceph-users-leave(a)ceph.io
<mailto:ceph-users-leave@ceph.io>
1419
days inactive
1421
days old
3 comments
2 participants
participants (2)
-
Lenz Grimmer -
Marco Pizzolo