You could try pausing the upgrade and manually "upgrading" the mds daemons
by redeploying them on the new image. Something like "ceph orch daemon
redeploy <mds-daemon-name> --image <17.2.6 image>" (daemon names should
match those in "ceph orch ps" output). If you do that for all of them and
then get them into an up state you should be able to resume the upgrade and
have it complete.
On Mon, Apr 10, 2023 at 3:25 PM Thomas Widhalm <widhalmt(a)widhalm.or.at>
wrote:
Hi,
If you remember, I hit bug
https://tracker.ceph.com/issues/58489 so I
was very relieved when 17.2.6 was released and started to update
immediately.
But now I'm stuck again with my broken MDS. MDS won't get into up:active
without the update but the update waits for them to get into up:active
state. Seems like a deadlock / chicken-egg problem to me.
Since I'm still relatively new to Ceph, could you help me?
What I see when watching the update status:
{
"target_image":
"
quay.io/ceph/ceph@sha256:1161e35e4e02cf377c93b913ce78773f8413f5a8d7c5eaee4b4773a4f9dd6635
",
"in_progress": true,
"which": "Upgrading all daemon types on all hosts",
"services_complete": [
"crash",
"mgr",
"mon",
"osd"
],
"progress": "18/40 daemons upgraded",
"message": "Error: UPGRADE_OFFLINE_HOST: Upgrade: Failed to connect
to host ceph01 at addr (192.168.23.61)",
"is_paused": false
}
(The offline host was one host that broke during the upgrade. I fixed
that in the meantime and the update went on.)
And in the log:
2023-04-10T19:23:48.750129+0000 mgr.ceph04.qaexpv [INF] Upgrade: Waiting
for mds.mds01.ceph04.hcmvae to be up:active (currently up:replay)
2023-04-10T19:23:58.758141+0000 mgr.ceph04.qaexpv [WRN] Upgrade: No mds
is up; continuing upgrade procedure to poke things in the right direction
Please give me a hint what I can do.
Cheers,
Thomas
--
http://www.widhalm.or.at
GnuPG : 6265BAE6 , A84CB603
Threema: H7AV7D33
Telegram, Signal: widhalmt(a)widhalm.or.at
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io