Hi, thanks to the advices received ( thank you Joachim, Anthony) i managed to solve this ... layer-8 problem. Not a ceph problem; I'm not proud, but if it can help within archives.. My ceph is fully routed and i had a typo on the routing table; therefore many hosts couldn't connect each other. The cluster is rebuilding and at this time, no dataloss. Ceph is rock solid, even against layer-8 issues.. Thanks for all Best regards Philippe 23 sept. 2024, 10:13 de joachim.kraftmayer@clyso.com:

Hi Phil,

is the ceph public and cluster network set in your ceph config dump or is there another ceph.conf on the local servers? Joachim

  > joachim.kraftmayer@clyso.com

  > www.clyso.com <http://www.clyso.com/>

  Hohenzollernstr. 27, 80801 Munich

Utting a. A. | HR: Augsburg | HRB: 25866 | USt. ID-Nr.: DE2754306

Am Sa., 21. Sept. 2024 um 23:31 Uhr schrieb Phil <> infolist@schwarz-fr.net> >:

...

Digged further.

ceph osd dump --format json |jq '.osds[]| select (.osd==4)'

{   "osd": 4,   "uuid": "OSD UID",   "up": 0,   "in": 0,   "weight": 0,   "primary_affinity": 1,   "last_clean_begin": 0,   "last_clean_end": 0,   "up_from": 0,   "up_thru": 0,   "down_at": 0,   "lost_at": 0,   "public_addrs": {     "addrvec": []   },   "cluster_addrs": {     "addrvec": []   },   "heartbeat_back_addrs": {     "addrvec": []   },   "heartbeat_front_addrs": {     "addrvec": []   },   "public_addr": "(unrecognized address family 0)/0",   "cluster_addr": "(unrecognized address family 0)/0",   "heartbeat_back_addr": "(unrecognized address family 0)/0",   "heartbeat_front_addr": "(unrecognized address family 0)/0",   "state": [     "autoout",     "exists",     "new"   ] }

Every other healthy osds fields are filled with proper values..

Clues match, but i keep on looking for the root cause.

Best regards

Philippe

21 sept. 2024, 21:29 de >> infolist@schwarz-fr.net>> :

...

Hi, on a healthy cluster, every osd creation made orphans osd.

# ceph osd tree ID   CLASS  WEIGHT    TYPE NAME           STATUS  REWEIGHT  PRI-AFF -1         26.38051  root default                                  -7          3.63869      host A   1    hdd   3.63869          osd.1           up   1.00000  1.00000 -11          1.81940      host F   2    hdd   1.81940          osd.2           up   1.00000  1.00000 -5          7.27737      host J   3    hdd   3.63869          osd.3           up   1.00000  1.00000   5    hdd   3.63869          osd.5           up   1.00000  1.00000 -3          7.27737      host K   0    hdd   3.63869          osd.0           up   1.00000  1.00000   8    hdd   3.63869          osd.8           up   1.00000  1.00000 -9          6.36768      host S   6    hdd   3.63869          osd.6           up   1.00000  1.00000   7    hdd   2.72899          osd.7           up   1.00000  1.00000   4                0  osd.4                 down         0  1.00000   9                0  osd.9                 down         0  1.00000

neither 4 nor 9 osd appear in the crushmap (decompiled) Destroying and recreating osd recreate orphan osd.

Any hint ? Thanks for all Best regards Philippe 

_______________________________________________ ceph-users mailing list -- >> ceph-users@ceph.io To unsubscribe send an email to >> ceph-users-leave@ceph.io