Hi,
I just upgraded in 17.2.6 but in fact I had the same problem in 16.2.10.
I'm trying to configure the Ceph dashboard to monitor the RGWs (object
gateways used as S3 gw). Our cluster has 2 RGW realms (eros, fink) with
1 zonegroup per realm (p2io-eros and p2io-fink respectively) and 1 zone
per zonegroup (eros-z1 and fink-z1 respectively). RGW have been
configured without SSL due to the bug encountered in 16.2.10 (not sure
if it still applies, use of IP addresses that don't match the
certificate). There is a dashboard user in each realm and its
access/secret keys are matching what is in
mgr/dashboard/RGW_API_ACCESS_KEY and mgr/dashboard/RGW_API_SECRET_KEY (a
dict with one entry per realm).
Every attempt to access the RGW daemons section of the dashboard results
in :
-----
The Object Gateway Service is not configured
Error connecting to Object Gateway: RGW REST API failed request with
status code 403
(b'{"Code":"InvalidAccessKeyId","RequestId":"tx00000b4c2212597168bf0-00644116e3'
b'-edf7c3-eros-z1","HostId":"edf7c3-eros-z1-ijclab"}')
----
In the mgr logs, I have not been able to find more than this error. In
the RGW logs, for every dashboard attempt, I find:
------
Apr 20 12:41:39 valvd-rgw1.lal.in2p3.fr radosgw[1969455]: ======
starting new request req=0x7f2dcdb0b710 =====
Apr 20 12:41:39 valvd-rgw1.lal.in2p3.fr radosgw[1969455]: req
13025009517641239536 0.000000000s op->ERRORHANDLER: err_no=-2028
new_err_no=-2028
Apr 20 12:41:39 valvd-rgw1.lal.in2p3.fr radosgw[1969455]: ====== req
done req=0x7f2dcdb0b710 op status=0 http_status=403 latency=0.000000000s
======
Apr 20 12:41:39 valvd-rgw1.lal.in2p3.fr radosgw[1969455]: beast:
0x7f2dcdb0b710: 134.158.194.10 - - [20/Apr/2023:10:41:39.843 +0000] "GET
/admin/metadata/user?myself HTTP/1.1" 403 126 - "python-requests/2.20.0"
- latency=0.000000000s
------
In the logs above, I find a little bit strange the "HostID" value ending
with "eros-z1-ijclab". I'd expected it to be the zone name
"eros-z1".
The zone has been renamed a couple of times and it may be an old name,
despite the zone information in radosgw-admin shows nothing with this
name (in particular the API name has been renamed to match the zone
name). Anway the RGW works perfectly and the dashboard request seems to
reach the RGW so I don't have the feeling it is important...
Any suggestion on how to troubleshoot what happens would be appreciated.
Thanks in advance.
Best regards,
Michel
Show replies by date
Hi,
Still interested by some feedback... FYI, today I changed the
configuration of the RGW to https (for reasons unrelated to this
problem) and it seems the problem preventing the use of a RGW https with
the dashboard is fixed now. The problem described in my previous email
remains the same (same errors) whether http or https is enabled and I
really have no clue about the possible reason... I have not identified
also if there was a way to tell the dashboard to ignore the realm
causing the problem to see if it was working with the other realm...
Best regards,
Michel
Le 20/04/2023 à 12:55, Michel Jouvin a écrit :
> Hi,
>
> I just upgraded in 17.2.6 but in fact I had the same problem in
> 16.2.10. I'm trying to configure the Ceph dashboard to monitor the
> RGWs (object gateways used as S3 gw). Our cluster has 2 RGW realms
> (eros, fink) with 1 zonegroup per realm (p2io-eros and p2io-fink
> respectively) and 1 zone per zonegroup (eros-z1 and fink-z1
> respectively). RGW have been configured without SSL due to the bug
> encountered in 16.2.10 (not sure if it still applies, use of IP
> addresses that don't match the certificate). There is a dashboard user
> in each realm and its access/secret keys are matching what is in
> mgr/dashboard/RGW_API_ACCESS_KEY and mgr/dashboard/RGW_API_SECRET_KEY
> (a dict with one entry per realm).
>
> Every attempt to access the RGW daemons section of the dashboard
> results in :
>
> -----
>
> The Object Gateway Service is not configured
> Error connecting to Object Gateway: RGW REST API failed request with
> status code 403
>
(b'{"Code":"InvalidAccessKeyId","RequestId":"tx00000b4c2212597168bf0-00644116e3'
>
b'-edf7c3-eros-z1","HostId":"edf7c3-eros-z1-ijclab"}')
>
> ----
>
> In the mgr logs, I have not been able to find more than this error. In
> the RGW logs, for every dashboard attempt, I find:
>
> ------
>
> Apr 20 12:41:39 valvd-rgw1.lal.in2p3.fr radosgw[1969455]: ======
> starting new request req=0x7f2dcdb0b710 =====
> Apr 20 12:41:39 valvd-rgw1.lal.in2p3.fr radosgw[1969455]: req
> 13025009517641239536 0.000000000s op->ERRORHANDLER: err_no=-2028
> new_err_no=-2028
> Apr 20 12:41:39 valvd-rgw1.lal.in2p3.fr radosgw[1969455]: ====== req
> done req=0x7f2dcdb0b710 op status=0 http_status=403
> latency=0.000000000s ======
> Apr 20 12:41:39 valvd-rgw1.lal.in2p3.fr radosgw[1969455]: beast:
> 0x7f2dcdb0b710: 134.158.194.10 - - [20/Apr/2023:10:41:39.843 +0000]
> "GET /admin/metadata/user?myself HTTP/1.1" 403 126 -
> "python-requests/2.20.0" - latency=0.000000000s
> ------
>
> In the logs above, I find a little bit strange the "HostID" value
> ending with "eros-z1-ijclab". I'd expected it to be the zone name
> "eros-z1". The zone has been renamed a couple of times and it may be
> an old name, despite the zone information in radosgw-admin shows
> nothing with this name (in particular the API name has been renamed to
> match the zone name). Anway the RGW works perfectly and the dashboard
> request seems to reach the RGW so I don't have the feeling it is
> important...
>
> Any suggestion on how to troubleshoot what happens would be
> appreciated. Thanks in advance.
>
> Best regards,
>
> Michel
>