5 Nov
2020
5 Nov
'20
2:28 p.m.
Hi,
we upgraded our ceph cluster from 14.2.9 to 15.2.5 but osds with 15.2.5
are not joining the cluster after restart. They hang with "1234 tick
checking mon for new map"
The Systems are Centos 7.8
I tried everything i could think of. But nothing helped. The mons and
mgrs are 15.2.5.
Anyone an Idea want could cause this?
2020-11-05T15:18:13.142+0100 7f02fce81700 1 osd.114 pg_epoch: 1234
pg[5.c9s0( v 839'39171 (839'37583,839'39171] local-lis/les=1129/1130
n=495 ec=813/813 lis/c=1129/1081 les/c/f=1130/1082/0 sis=1210)
[NONE,NONE,NONE,NONE,124,NONE,157,NONE,NONE,149,NONE]p124(4) r=-1
lpr=1212 pi=[1081,1210)/1 crt=839'39171 lcod 0'0 mlcod 0'0 unknown
mbc={} ps=[1~3]] state<Start>: transitioning to Stray
2020-11-05T15:18:13.143+0100 7f02fce81700 1 osd.114 pg_epoch: 1234
pg[1.736( v 1105'9685 (815'8100,1105'9685] local-lis/les=1131/1132 n=6
ec=719/719 lis/c=1131/1019 les/c/f=1132/1020/0 sis=1210
pruub=11.135073235s) [] r=-1 lpr=1212 pi=[1019,1210)/1 crt=1105'9685
lcod 0'0 mlcod 0'0 unknown mbc={}] state<Start>: transitioning to Stray
2020-11-05T15:18:13.143+0100 7f02fce81700 1 osd.114 pg_epoch: 1234
pg[4.3fd( v 1109'105689 (1109'102600,1109'105689]
local-lis/les=1131/1132 n=32 ec=740/740 lis/c=1131/1089
les/c/f=1132/1090/0 sis=1210 pruub=11.134586488s) [] r=-1 lpr=1212
pi=[1089,1210)/1 crt=1109'105689 lcod 0'0 mlcod 0'0 unknown mbc={}]
state<Start>: transitioning to Stray
2020-11-05T15:18:13.144+0100 7f02fce81700 1 osd.114 pg_epoch: 1234
pg[5.123s10( v 839'38976 (839'37410,839'38976] local-lis/les=1119/1120
n=498 ec=813/813 lis/c=1119/1064 les/c/f=1120/1065/0 sis=1210)
[NONE,NONE,NONE,NONE,NONE,NONE,NONE,NONE,NONE,NONE,NONE]p? r=-1 lpr=1212
pi=[1064,1210)/1 crt=839'38976 lcod 0'0 mlcod 0'0 unknown mbc={}
ps=[1~3]] state<Start>: transitioning to Stray
2020-11-05T15:18:13.145+0100 7f03140b5700 1 osd.114 1234
set_numa_affinity public network bond0 numa node 2
2020-11-05T15:18:13.145+0100 7f03140b5700 1 osd.114 1234
set_numa_affinity cluster network bond1 numa node 0
2020-11-05T15:18:13.145+0100 7f03140b5700 1 osd.114 1234
set_numa_affinity public and cluster network numa nodes do not match
2020-11-05T15:18:13.145+0100 7f03140b5700 1 osd.114 1234
set_numa_affinity not setting numa affinity
2020-11-05T15:18:14.072+0100 7f031713a700 1 osd.114 1234 tick checking
mon for new map
2020-11-05T15:18:44.397+0100 7f031713a700 1 osd.114 1234 tick checking
mon for new map
2020-11-05T15:19:15.370+0100 7f031713a700 1 osd.114 1234 tick checking
mon for new map
2020-11-05T15:19:46.206+0100 7f031713a700 1 osd.114 1234 tick checking
mon for new map
2020-11-05T15:20:16.400+0100 7f031713a700 1 osd.114 1234 tick checking
mon for new map
2020-11-05T15:20:46.483+0100 7f031713a700 1 osd.114 1234 tick checking
mon for new map
Regards
--
Ingo Ebel
Human knowledge belongs to the world.
RadioTux.de - Internet-Radio rund um Linux und Open Source
## https://twitter.com/ingoebel
## https://keybase.io/savar
## Jabber: ingo.ebel(a)ingoebel.de