Hi, thanks to the advices received ( thank you Joachim, Anthony) i managed to solve this ... layer-8 problem. Not a ceph problem; I'm not proud, but if it can help within archives.. My ceph is fully routed and i had a typo on the routing table; therefore many hosts couldn't connect each other. The cluster is rebuilding and at this time, no dataloss. Ceph is rock solid, even against layer-8 issues.. Thanks for all Best regards Philippe 23 sept. 2024, 10:13 de joachim.kraftmayer@clyso.com:
Hi Phil,
is the ceph public and cluster network set in your ceph config dump or is there another ceph.conf on the local servers? Joachim
> joachim.kraftmayer@clyso.com
> www.clyso.com <http://www.clyso.com/>
Hohenzollernstr. 27, 80801 Munich
Utting a. A. | HR: Augsburg | HRB: 25866 | USt. ID-Nr.: DE2754306
Am Sa., 21. Sept. 2024 um 23:31 Uhr schrieb Phil <> infolist@schwarz-fr.net> >:
Digged further.
ceph osd dump --format json |jq '.osds[]| select (.osd==4)'
{ "osd": 4, "uuid": "OSD UID", "up": 0, "in": 0, "weight": 0, "primary_affinity": 1, "last_clean_begin": 0, "last_clean_end": 0, "up_from": 0, "up_thru": 0, "down_at": 0, "lost_at": 0, "public_addrs": { "addrvec": [] }, "cluster_addrs": { "addrvec": [] }, "heartbeat_back_addrs": { "addrvec": [] }, "heartbeat_front_addrs": { "addrvec": [] }, "public_addr": "(unrecognized address family 0)/0", "cluster_addr": "(unrecognized address family 0)/0", "heartbeat_back_addr": "(unrecognized address family 0)/0", "heartbeat_front_addr": "(unrecognized address family 0)/0", "state": [ "autoout", "exists", "new" ] }
Every other healthy osds fields are filled with proper values..
Clues match, but i keep on looking for the root cause.
Best regards
Philippe
21 sept. 2024, 21:29 de >> infolist@schwarz-fr.net>> :
Hi, on a healthy cluster, every osd creation made orphans osd.
# ceph osd tree ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -1 26.38051 root default -7 3.63869 host A 1 hdd 3.63869 osd.1 up 1.00000 1.00000 -11 1.81940 host F 2 hdd 1.81940 osd.2 up 1.00000 1.00000 -5 7.27737 host J 3 hdd 3.63869 osd.3 up 1.00000 1.00000 5 hdd 3.63869 osd.5 up 1.00000 1.00000 -3 7.27737 host K 0 hdd 3.63869 osd.0 up 1.00000 1.00000 8 hdd 3.63869 osd.8 up 1.00000 1.00000 -9 6.36768 host S 6 hdd 3.63869 osd.6 up 1.00000 1.00000 7 hdd 2.72899 osd.7 up 1.00000 1.00000 4 0 osd.4 down 0 1.00000 9 0 osd.9 down 0 1.00000
neither 4 nor 9 osd appear in the crushmap (decompiled) Destroying and recreating osd recreate orphan osd.
Any hint ? Thanks for all Best regards Philippe
_______________________________________________ ceph-users mailing list -- >> ceph-users@ceph.io To unsubscribe send an email to >> ceph-users-leave@ceph.io