It appears what happened: 1. This host was a MON host, and was the only listed MON in the /etc/ceph/ceph.conf file. 2. The MON went out of quorum with the other 4 monitors. 3. After adding all MON to the /etc/ceph/ceph.conf, then the `ceph -s` command worked again. 4. After restarting the host, the MON was able to get back in the quorum with the other monitors I think the issue was that when the MON was out of quorum, the ceph client could no longer connect when only having that MON as an option. Problem is solved - -Matt On Mon, Jun 14, 2021 at 2:07 PM Matt Larson <larsonmattr@gmail.com> wrote:

I was working through updates of CentOS and packages on each of the Ceph storage nodes of a cluster, and I hit an issue after I updated a package `ceph-osd` on the original Ceph node of the cluster.

1. Before the updates, the Ceph cluster was running 15.2.3 version 2. After the update of the package with `dnf install ceph-osd`, I was unable to check the status any more with `ceph -s`:

2021-06-14T14:01:53.216-0500 7efdef7fe700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]

[errno 13] RADOS permission denied (error connecting to the cluster)

3. The `ceph -v` command now shows as version 15.2.13.

Based on other reported issues this could be due recent changes with security fixes ( https://docs.ceph.com/en/latest/security/CVE-2021-20288/ ) that were introduced in 15.2.11.

This cluster had originally been built with `cephadm` tool and containerized daemons.

How can I restore the ability to connect with the command-line `ceph` client to check the status and all other interactions?

Thanks, Matt

-- Matt Larson, PhD Madison, WI 53705 U.S.A.

-- Matt Larson, PhD Madison, WI 53705 U.S.A.