Re: ceph orch update fails - got new digests
On Fri, Apr 2, 2021 at 12:08 PM Alexander Sporleder <asporleder@age.mpg.de> wrote:
Hello Sage, thank you for your response!
I had some problems updating 15.2.8 -> 15.2.9 but after updating Podman to 3.0.1 and Ceph to 15.2.10 everything was fine again.
Then I started the update 15.2.10 -> 16.2.0 and in the beging everything worked well. But at some point the update got stucked and something broke the dashboard (port is in use). I stopped the update but it was not possible to start the update process again without the loop.
Now my mons, the mgr, a few OSDs and cephadm are V16.2.0.
Unfortunately the mgr is not logging to a file after I converted the cluster to cephadm.
ceph config set global log_to_file true You might also need to chown -R 167.167 /var/log/ceph/*/. if you're on debian/ubuntu (the packages installed on the host may have fiddled with the ownership; the uid for fedora/rhel/centos is different than the one for debian/ubuntu). s
I have a proposed fix for this here: https://github.com/ceph/ceph/pull/40577 Unfortunately, this won't help you until it is tested and merged and included in 16.2.1. If you'd like to finish your upgrade before then, you can upgrade to the pacific branch tip with ceph orch upgrade start quay.ceph.io/ceph-ci/ceph:pacific That will include a handful of patches beyond the initial 16.2.0 (so far, mostly small cephadm fixes), although be aware that that tag is rebuilt nightly and will include more as time goes on. There is probably some workaround to induce docker to consistently include (or not include) the "docker.io/" prefix on the digest but I'm not sure what that would be. You can also just leave the cluster as is (half-upgraded) until 16.2.1 is out. Thanks for your help debugging this! sage
participants (1)
-
Sage Weil