Hi Eugen! On 23/05/26 23:54, Eugen Block via ceph-users wrote:

Hi,

I might be wrong, but I don't think it's a cert issue. If you look in the mgr log, do you see more information that just the SignatureDoesNotMatch error?

On further investigations I'm also convinced that is nothing to do with the certificates but more about the permissions of the 'dashboard' user. This is the exception that is thrown from the mgr logs:

[dashboard INFO rgw_client] Found RGW daemon with configuration: host=node-01.domain.com, port=4443, ssl=True [dashboard INFO rgw_client] Found RGW daemon with configuration: host=node-03.domain.com, port=4443, ssl=True [dashboard INFO rgw_client] Found RGW daemon with configuration: host=node-02.domain.com, port=4443, ssl=True [dashboard INFO rgw_client] Found RGW daemon with configuration: host=node-04.domain.com, port=4443, ssl=True [dashboard INFO rgw_client] Found RGW daemon with configuration: host=node-05.domain.com, port=4443, ssl=True [dashboard INFO request] [::ffff:140.105.2.28:41732] [GET] [200] [0.005s] [admin] [308.0B] /api/health/snapshot [dashboard INFO request] [::ffff:140.105.2.28:41732] [GET] [200] [0.007s] [admin] [22.0B] /api/prometheus/alertgroup [dashboard INFO request] [::ffff:140.105.2.28:41732] [GET] [200] [0.012s] [admin] [376.0B] /api/multi-cluster/get_config [dashboard INFO request] [::ffff:140.105.2.28:41760] [GET] [200] [0.028s] [admin] [1.2K] /api/summary [dashboard ERROR rest_client] RGW REST API failed GET req status: 403 [dashboard ERROR rgw_client] RGW REST API failed request with status code 403 (b'{"Code":"SignatureDoesNotMatch","Message":"","RequestId":"tx00000629cfb33d5e' b'da4c7-006a141bf6-308299-eros","HostId":"308299-zone-zonegroup"}') Traceback (most recent call last): File "/usr/share/ceph/mgr/dashboard/services/rgw_client.py", line 413, in __init__ self.userid = self._get_user_id(self.admin_path) if self.got_keys_from_config \ File "/usr/share/ceph/mgr/dashboard/rest_client.py", line 538, in func_wrapper return func( File "/usr/share/ceph/mgr/dashboard/services/rgw_client.py", line 448, in _get_user_id response = request() File "/usr/share/ceph/mgr/dashboard/rest_client.py", line 324, in __call__ resp = self.rest_client.do_request(method, self._gen_path(), params, File "/usr/share/ceph/mgr/dashboard/rest_client.py", line 422, in do_request raise RequestException( dashboard.rest_client.RequestException: RGW REST API failed request with status code 403 (b'{"Code":"SignatureDoesNotMatch","Message":"","RequestId":"tx00000629cfb33d5e' b'da4c7-006a141bf6-308299-zone","HostId":"308299-zone-zonegroup"}')

Could it be mismatching dashboard-rgw-api settings? Have you checked these settings?

ceph dashboard get-rgw-api-admin-resource

It's the default, 'admin'. I also reset the value with "reset-rgw-api-admin-resource", so technically now is 'unset'. Question what should be the value if not 'admin'?

ceph dashboard get-rgw-api-access-key

ceph dashboard get-rgw-api-secret-key

They should match with:

radosgw-admin user info --uid dashboard | jq -r '.keys'

The values are matching, I'see in both get-rgw-api-secret-key and get-rgw-api-access-key something like this: {'<REALM>': '<(ACCESS}SECRET)_KEY>'}

But as I wrote, it might something else, I would expect the mgr log to contain more details.

Indeed, I'll tried to backtrace the error in the code and I see that is failing in the rgw_client.py when is trying to get the user_id "of the user that is used to communicate with the RGW Admin Ops API". I suppose that is the 'admin' user from rgw-api-admin-resource, am I right? I didn't had time till now to investigate further. Thanks! Iztok