mon vanished after cephadm upgrade
I had a 3 mon CEPH cluster, after updating from 15.2.x to 16.2.x one of my mon's is showing as a stopped state in the Ceph Dashboard.And checking the cephadm logs on the server in question I can see "/usr/bin/docker: Error: No such object: ceph-30449cba-44e4-11eb-ba64-dda10beff041-mon.sn-m01"There is a few OSD services running on the same physical server and they all are starting/running fine via docker.I tried to do a cephadm apply mon to push a new mon to the same host, but it seems to not do anything, nothing shows in the same log file on sn-m01Also ceph -s shows full health and no errors and has no trace of the "failed" mon (not sure if this is expected), only in the ceph dashboard under services can I see the stopped not running mon. Sent via MXlogin
Hi Ashley, is sn-m01 listed in `ceph -s`? Which hosts are listed in `ceph orch ps --daemon-type mon ? Otherwise, there are a two helpful commands now: * `cpeh orch daemon rm mon.sn-m01` to remove the mon * `ceph orch daemon start mon.sn-m01` to start it again Am 14.05.21 um 14:14 schrieb Ashley Merrick:
I had a 3 mon CEPH cluster, after updating from 15.2.x to 16.2.x one of my mon's is showing as a stopped state in the Ceph Dashboard.And checking the cephadm logs on the server in question I can see "/usr/bin/docker: Error: No such object: ceph-30449cba-44e4-11eb-ba64-dda10beff041-mon.sn-m01"There is a few OSD services running on the same physical server and they all are starting/running fine via docker.I tried to do a cephadm apply mon to push a new mon to the same host, but it seems to not do anything, nothing shows in the same log file on sn-m01Also ceph -s shows full health and no errors and has no trace of the "failed" mon (not sure if this is expected), only in the ceph dashboard under services can I see the stopped not running mon.
Sent via MXlogin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hello,Is not listed under ceph -s, ceph-s reports no issues on the cluster.Is lised under orch ps and dashboard but reports "mon.sn-m01 sn-m01 stopped 114s ago 4M - <unknown> <unknown> <unknown>"Let me know if anything else useful you would like before I try remove and redeploy.Thanks
On Fri May 14 2021 21:44:11 GMT+0800 (Singapore Standard Time), Sebastian Wagner <sewagner@redhat.com> wrote: Hi Ashley,
is sn-m01 listed in `ceph -s`? Which hosts are listed in `ceph orch ps --daemon-type mon ?
Otherwise, there are a two helpful commands now:
* `cpeh orch daemon rm mon.sn-m01` to remove the mon * `ceph orch daemon start mon.sn-m01` to start it again
Am 14.05.21 um 14:14 schrieb Ashley Merrick:
I had a 3 mon CEPH cluster, after updating from 15.2.x to 16.2.x one of my mon's is showing as a stopped state in the Ceph Dashboard.And checking the cephadm logs on the server in question I can see "/usr/bin/docker: Error: No such object: ceph-30449cba-44e4-11eb-ba64-dda10beff041-mon.sn-m01"There is a few OSD services running on the same physical server and they all are starting/running fine via docker.I tried to do a cephadm apply mon to push a new mon to the same host, but it seems to not do anything, nothing shows in the same log file on sn-m01Also ceph -s shows full health and no errors and has no trace of the "failed" mon (not sure if this is expected), only in the ceph dashboard under services can I see the stopped not running mon.
Sent via MXlogin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Sent via MXlogin
participants (2)
-
Ashley Merrick
-
Sebastian Wagner