This happens (unfortunately) frequently to me. Look for the active mgr (ceph -s), and go restart the mgr service there (systemctl list-units |grep mgr then systemctl restart NAMEOFSERVICE). This normally resolves that error for me. You can look at the journalctl output and you'll likely see errors similar to this issue: https://tracker.ceph.com/issues/45032 I'm hopeful the PR in that issue will resolve the issue for us: https://github.com/ceph/ceph/pull/35022 See if that works - if not - you may want to look at the logs for the mgr (and potentially other) daemons using journalctl. On Tue, May 19, 2020 at 3:45 PM Gencer W. Genç <gencer@gencgiyen.com> wrote:
Hi,
I was browsing dashboard today. Then suddently it stopped working and i got 502 errors. I checked via root login and see thet ceph health is down to WARN.
I can access all rdb devices and CephFS. They work. All OSDs in server-1 is up.
health: HEALTH_WARN 1 hosts fail cephadm check failed to probe daemons or devices
I even restarted server-1. No luck.
I'm on server-1. cephadm complains it cannot access to server-1. In basic term, server-1 cannot access server-1 (192.168.0.1)
server-1: 192.168.0.1 server-2: 192.168.0.3
$ ssh -F =(ceph cephadm get-ssh-config) -i =(ceph config-key get mgr/cephadm/ssh_identity_key) root@server-1
Success.
cephadm says server-1 is not reachable. Why cephadm/ceph complain about this and what happend to my dashboard? I also redirected logs to file. I'm observing for any weird log input but unfortunately nothing helpful. Where should I look for? What am i missing?
So, when I ran this command:
$ ceph health detail
Output was:
ssh -F =(ceph cephadm get-ssh-config) -i =(ceph config-key get mgr/cephadm/ssh_identity_key) root@server-1 [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices host server-1 scrape failed: Failed to connect to server-1 (server-1). Check that the host is reachable and accepts connections using
HEALTH_WARN 1 hosts fail cephadm check; failed to probe daemons or devices [WRN] CEPHADM_HOST_CHECK_FAILED: 1 hosts fail cephadm check host server-1 failed check: Failed to connect to server-1 (server-1). Check that the host is reachable and accepts connections using the cephadm SSH key you may want to run: the cephadm SSH key you may want to run:
ssh -F =(ceph cephadm get-ssh-config) -i =(ceph config-key get mgr/cephadm/ssh_identity_key) root@server-1 host server-1 ceph-volume inventory failed: Failed to connect to server-1 (server-1). Check that the host is reachable and accepts connections using the cephadm SSH key you may want to run: ssh -F =(ceph cephadm get-ssh-config) -i =(ceph config-key get mgr/cephadm/ssh_identity_key) root@server-1
Thanks, Gencer. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io