Hi! I can confirm that if I set "rgw_dns_name" to the FQDN of the virtual IP (and restart the rgw daemons) the Dashboard gets `SignatureDoesNotMatch`. Is this the expected behaviour? If I understand correctly the "rgw_dns_name" allow the virtual-hosted-style requests (<bucketname>.name.domain.com). Righ now we don't need it, but in the future we might. Cheers Iztok On 22/09/26 08:22, Iztok Gregori wrote:
Good morning to all!
After a couple of months I had the time to look again into the problem with 'SignatureDoesNotMatch' when accessing the "Object" section of the Ceph Dashboard.
I've upgraded my cluster to 20.2.4 but the dashboard was still getting the error. Trying to debug I notice that RGW_API_SSL_VERIFY was "false", I enabled it (switching it to True) (all the RGW are running with "legit" SSL certificates) and the error message changed in "Certificate did not match expected hostname". Ok, we are on something, I said. The cluster is running with short hostanmes and the certifcate has FQDN. I tinkered with some variables but to non avail...
Until today when I was gathering some logs to ask further help on this mailing-list, I put RGW_API_SSL_VERIFY to False and the dashboard worked again!!!
I don't think that switching RGW_API_SSL_VERIFY from False to True and then to False again, did the job. Yesterday I did a couple of "experiment" with different parameters, but once the tests were done I always revert them to the orginal values. Appart from "rgw_dns_name" which I deleted (and not re-created)...
Could be "rgw_dns_name" the culprit? It was the FQDN of the virtual ip used to access the cluster.
Have a great day! Cheers
Iztok
On 27/05/26 14:41, Eugen Block via ceph-users wrote:
I see what you mean, in our Squid cluster the curl request from within the mgr container fails with the same error. So I should probably expect that we're gonna hit the same dashboard issue once we upgrade to Tentacle, I guess.
Zitat von Iztok Gregori via ceph-users <ceph-users@ceph.io>:
On 27/05/26 12:47, Eugen Block via ceph-users wrote:
I'm not sure about the curl versions.
Curl was only a tool and it could be unrelated to the problem I'm facing, but it was interesting that when I try to query
# curl "https://rgw.domain.com/admin/metadata/user" --aws-sigv4 "aws:amz:ZONEGROUP:s3" --user "DASHBOARDKEY:DASHBOARDSECRET"
it worked fine with newer versions. But the older ones (like the one shipped with the MGR container) failed with the 'SignatureDoesNotMatch' error. When I tried with more verbosity I see that the oldest version is "signing" only 2 headers (SignedHeaders=host;x-amz-date) and the newer ones 3 (SignedHeaders=host;x-amz-content-sha256;x-amz-date), but this could be due to '--aws-sigv4' option in curl.
Did you notice the response from > "C U" the day before yesterday? He or she commented:
fqdn on the dashboard needs working certificate validation.
"Squid did cert work inline → bad cert raised crypto.Error → caught and rewrapped as ServerConfigException cleanly. Tentacle does cert work via a subprocess; failures can now come from (a) the fork itself, (b) PyO3 init crashing in the child (the documented subinterpreter
collision), or (c) genuine cert errors marshalled back as JSON. When (b) happens, the dashboard module's cert provisioning never completes, the module ends up half-initialized, and downstream dashboard→RGW calls fail with SignatureDoesNotMatch because the dashboard is operating without properly loaded credentials/sigv4 plumbing."
I did some tests using openssl s_client from the mgr container to verify the certificates and I had no problems. Clearly I wasn't using python, but unless is not using the system paths for certificates, it shouldn't matter.
Iztok
Zitat von Iztok Gregori via ceph-users <ceph-users@ceph.io>:
On 27/05/26 12:17, Eugen Block via ceph-users wrote:
Hi,
Zitat von Iztok Gregori via ceph-users <ceph-users@ceph.io>:
> On 27/05/26 09:36, Eugen Block via ceph-users wrote: >> The description in this tracker [0] looks identical, but it is >> supposed to be fixed. Do you have rgw_dns_name set? > > Yes, at certain point I configured client.rgw rgw_dns_name to the > FQDN belonging to the virtual IP of the ingress service.
does it mean that it works now?
No, still doesn't work. To try I've also remove it and then again set it, but without luck.
I also tried to call the API endpoint directly with curl using the dashboard credentials and found that from 'older' versions of curl (provided for example by Rocky 9) the command gives me the same error ('SignatureDoesNotMatch'), but when I was using newer one (8.5.0 on Ubuntu 22.04 and 8.9.1 on Rocky 10) it correctly answer my query.
Could it be the problem?
> BTW, how you 'unset' a configuration variable, you just put a > empty string or you use the rm option? > > ceph config set client.rgw rgw_dns_name '' > or > ceph config rm client.rgw rgw_dns_name > ?
That's the correct one (config rm).
Thank you! I sometimes get lost in the documentation :-)
Cheers Iztok _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io