Osds usually try to load the last clean map on boot, but if these osds finished the upgrade it must have hit an assert I think On Fri, Oct 24, 2025 at 8:06 AM Huseyin Cotuk <hcotuk@gmail.com> wrote:
Hi Can,
No, I did not add or recreate these OSDs. I tried cephadm to run failing OSDs with the help of following commands without success:
cephadm --image quay.io/ceph/ceph:v19 unit --name osd.$i --fsid 4e7e7d1c-22db-49c7-9f24-5a75cd3a3b9f restart
cephadm --image aade1b12b8e6 run --name osd.$i --fsid 4e7e7d1c-22db-49c7-9f24-5a75cd3a3b9f
BR, Huseyin hcotuk@gmail.com
On 24 Oct 2025, at 17:55, Can Özyurt <acozyurt@gmail.com> wrote:
Hi Huseyin,
Did you add or recreate these OSDs after the upgrade?
More debug logs from a failing OSD:
Oct 24 17:39:52 ank-backup01 ceph-4e7e7d1c-22db-49c7-9f24-5a75cd3a3b9f-osd-11[4100566]: 2025-10-24T14:39:52.271+0000 77f47e35c640 20 osd.11 70297 reports for 0 queries Oct 24 17:39:52 ank-backup01 ceph-4e7e7d1c-22db-49c7-9f24-5a75cd3a3b9f-osd-11[4100566]: 2025-10-24T14:39:52.271+0000 77f47e35c640 15 osd.11 70297 collect_pg_stats Oct 24 17:39:52 ank-backup01 ceph-osd[4100574]: osd.11 pg_epoch: 70297
Oct 24 17:39:52 ank-backup01 ceph-4e7e7d1c-22db-49c7-9f24-5a75cd3a3b9f-osd-11[4100566]: 2025-10-24T14:39:52.271+0000 77f47e35c640 20 osd.11 pg_epoch: 70297
Oct 24 17:39:52 ank-backup01 ceph-osd[4100574]: osd.11 pg_epoch: 70297
Oct 24 17:39:52 ank-backup01 ceph-4e7e7d1c-22db-49c7-9f24-5a75cd3a3b9f-osd-11[4100566]: 2025-10-24T14:39:52.271+0000 77f47e35c640 15 osd.11 pg_epoch: 70297
Oct 24 17:39:52 ank-backup01 ceph-osd[4100574]: osd.11 pg_epoch: 70297
On Fri, 24 Oct 2025 at 16:42, Huseyin Cotuk <hcotuk@gmail.com <mailto: hcotuk@gmail.com>> wrote: pg[23.b9s0( v 69083'8392506 (68272'8387884,69083'8392506] local-lis/les=69076/69077 n=8314181 ec=21807/21807 lis/c=69076/68833 les/c/f=69077/68834/39143 sis=70280) [11,NONE,NONE,12,31,21,7,NONE,NONE,26,0]p11(0) r=0 lpr=70297 pi=[68833,70280)/5 crt=69083'8392506 lcod 0'0 mlcod 0'0 unknown mbc={}] PeeringState::prepare_stats_for_publish reporting purged_snaps [] pg[23.b9s0( v 69083'8392506 (68272'8387884,69083'8392506] local-lis/les=69076/69077 n=8314181 ec=21807/21807 lis/c=69076/68833 les/c/f=69077/68834/39143 sis=70280) [11,NONE,NONE,12,31,21,7,NONE,NONE,26,0]p11(0) r=0 lpr=70297 pi=[68833,70280)/5 crt=69083'8392506 lcod 0'0 mlcod 0'0 unknown mbc={}] PeeringState::prepare_stats_for_publish reporting purged_snaps [] pg[23.b9s0( v 69083'8392506 (68272'8387884,69083'8392506] local-lis/les=69076/69077 n=8314181 ec=21807/21807 lis/c=69076/68833 les/c/f=69077/68834/39143 sis=70280) [11,NONE,NONE,12,31,21,7,NONE,NONE,26,0]p11(0) r=0 lpr=70297 pi=[68833,70280)/5 crt=69083'8392506 lcod 0'0 mlcod 0'0 unknown mbc={}] PeeringState::prepare_stats_for_publish publish_stats_to_osd 70297:52872808 pg[23.b9s0( v 69083'8392506 (68272'8387884,69083'8392506] local-lis/les=69076/69077 n=8314181 ec=21807/21807 lis/c=69076/68833 les/c/f=69077/68834/39143 sis=70280) [11,NONE,NONE,12,31,21,7,NONE,NONE,26,0]p11(0) r=0 lpr=70297 pi=[68833,70280)/5 crt=69083'8392506 lcod 0'0 mlcod 0'0 unknown mbc={}] PeeringState::prepare_stats_for_publish publish_stats_to_osd 70297:52872808 pg[23.5fs2( v 69081'8395094 (68276'8390628,69081'8395094] local-lis/les=69078/69079 n=8316950 ec=21807/21807 lis/c=69078/68833 les/c/f=69079/68834/39143 sis=70280) [NONE,NONE,11,8,14,16,0,20,17,18,NONE]p11(2) r=2 lpr=70297 pi=[68833,70280)/7 crt=69081'8395094 lcod 0'0 mlcod 0'0 unknown mbc={}] PeeringState::prepare_stats_for_publish reporting purged_snaps []
Selamlar, Huseyin Cotuk hcotuk@gmail.com <mailto:hcotuk@gmail.com>
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io <mailto:
ceph-users@ceph.io>
To unsubscribe send an email to ceph-users-leave@ceph.io <mailto: ceph-users-leave@ceph.io>
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io