- ceph-users - lists.ceph.io

Re: Dashboard 'SignatureDoesNotMatch' error when connecting to RGW
by C U 25 May '26

25 May '26

fqdn on the dashboard needs working certificate validation. "Squid did cert work inline → bad cert raised crypto.Error → caught and rewrapped as ServerConfigException cleanly. Tentacle does cert work via a subprocess; failures can now come from (a) the fork itself, (b) PyO3 init crashing in the child (the documented subinterpreter collision), or (c) genuine cert errors marshalled back as JSON. When (b) happens, the dashboard module's cert provisioning never completes, the module ends up half-initialized, and downstream dashboard→RGW calls fail with SignatureDoesNotMatch because the dashboard is operating without properly loaded credentials/sigv4 plumbing." On 5/23/26 17:54, Eugen Block via ceph-users wrote: > Hi, > > I might be wrong, but I don't think it's a cert issue. If you look in > the mgr log, do you see more information that just the > SignatureDoesNotMatch error? > > Could it be mismatching dashboard-rgw-api settings? Have you checked > these settings? > > ceph dashboard get-rgw-api-admin-resource > > ceph dashboard get-rgw-api-access-key > > ceph dashboard get-rgw-api-secret-key > > They should match with: > > radosgw-admin user info --uid dashboard | jq -r '.keys' > > But as I wrote, it might something else, I would expect the mgr log to > contain more details. > > Regards > Eugen > > Zitat von Iztok Gregori via ceph-users <ceph-users(a)ceph.io>: > >> Hi to all! >> >> After upgrading my cluster from squid to tentacle (now on 20.2.1) I'm >> getting the following error when I try to access any 'page' in the >> Object section of the Ceph Dashboard: >> >> Error connecting to Object Gateway: RGW REST API failed request with >> status code 403 >> (b'{"Code":"SignatureDoesNotMatch","Message":"","RequestId":"tx0000000b5bf42356' >> b'85174-006a0d95b0-308299-eros","HostId":"308299-zone-default"}') >> >> I'm pretty sure that in Squid it was working (I don't usually access >> the Dashboard, but I did after the upgrade to check if everything is >> ok). >> >> The error 'SignatureDoesNotMatch' leads me to believe that there is a >> problem somewhere with the SSL certificates. But I put >> RGW_API_SSL_VERIFY to false... >> >> I have a SSL certificate (issued by harica.gr) which has different >> SANs including the hostname of all the nodes where RGW daemon is >> running (plus a cluster hostname and a wildcard hostname). I used >> this certificate for the ingress service and for the rgw daemons. >> Everything is done with a spec file using ceph orchestrator. I'm >> using the fullchain for the certificates and the the rgw/ingress >> services are running without any problem >> >> Or I'm completely wrong and the problem itself is with the access >> permissions (error code 403)? >> >> Any ideas? >> >> Thanks >> Iztok >> >> -- >> Iztok Gregori >> ICT Systems and Services >> Elettra - Sincrotrone Trieste S.C.p.A. >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io > > > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io

1 0

Re: Dashboard 'SignatureDoesNotMatch' error when connecting to RGW
by Eugen Block 23 May '26

23 May '26

Hi, I might be wrong, but I don't think it's a cert issue. If you look in the mgr log, do you see more information that just the SignatureDoesNotMatch error? Could it be mismatching dashboard-rgw-api settings? Have you checked these settings? ceph dashboard get-rgw-api-admin-resource ceph dashboard get-rgw-api-access-key ceph dashboard get-rgw-api-secret-key They should match with: radosgw-admin user info --uid dashboard | jq -r '.keys' But as I wrote, it might something else, I would expect the mgr log to contain more details. Regards Eugen Zitat von Iztok Gregori via ceph-users <ceph-users(a)ceph.io>: > Hi to all! > > After upgrading my cluster from squid to tentacle (now on 20.2.1) > I'm getting the following error when I try to access any 'page' in > the Object section of the Ceph Dashboard: > > Error connecting to Object Gateway: RGW REST API failed request with > status code 403 > (b'{"Code":"SignatureDoesNotMatch","Message":"","RequestId":"tx0000000b5bf42356' > b'85174-006a0d95b0-308299-eros","HostId":"308299-zone-default"}') > > I'm pretty sure that in Squid it was working (I don't usually access > the Dashboard, but I did after the upgrade to check if everything is > ok). > > The error 'SignatureDoesNotMatch' leads me to believe that there is > a problem somewhere with the SSL certificates. But I put > RGW_API_SSL_VERIFY to false... > > I have a SSL certificate (issued by harica.gr) which has different > SANs including the hostname of all the nodes where RGW daemon is > running (plus a cluster hostname and a wildcard hostname). I used > this certificate for the ingress service and for the rgw daemons. > Everything is done with a spec file using ceph orchestrator. I'm > using the fullchain for the certificates and the the rgw/ingress > services are running without any problem > > Or I'm completely wrong and the problem itself is with the access > permissions (error code 403)? > > Any ideas? > > Thanks > Iztok > > -- > Iztok Gregori > ICT Systems and Services > Elettra - Sincrotrone Trieste S.C.p.A. > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io

1 0

Ceph User + Dev Meetup: Help Us Find a Meetup Time – Quick Survey
by Anthony Middleton 22 May '26

22 May '26

Hey Ceph Friend! During this week's Ceph User + Developer Monthly Meetup, we discussed possibly moving our meeting time to a different time to serve participants across time zones. If you attend (or have wanted to attend but couldn't), we'd love your input! Please take a moment to fill out this short survey: https://forms.gle/g2PxYxC87PjKDJyu5. The survey also includes an option to express interest in leading a meetup in your time zone. If that's something you'd like to explore, let us know! For questions, feel free to reach out to me anytime. Best, ------- Anthony Middleton Community Manager, Ceph Foundation amiddleton(a)linuxfoundation.org LinkedIn <https://www.linkedin.com/company/ceph>| BlueSky <https://bsky.app/profile/ceph.io> | Twitter <https://x.com/Ceph>

1 0

May Ceph Science User Group Virtual Meeting
by Mattia 21 May '26

21 May '26

Dear all, We will be having a Ceph science/research/big cluster call on Tuesday May 26th as per ceph.io calendar (https://ceph.io/en/community/meetups/) We will start our meeting with Theofilos Mouratidis from switch.ch presenting "Switch Cloud Portal S3: a look into the future S3 service and it's current status" This is an informal open call of community members mostly from hpc/htc/research/big cluster environments, though anyone is welcome, where we discuss whatever is on our minds regarding Ceph. Updates, outages, features, maintenance, ... If you are interested in giving a short talk on one of the next meetings (it can be a status report, an introduction of your infra/ceph usecase or anything else of interest) you are welcome to reach out! If you would like to discuss something specific you can add it to the pad linked below. Virtual event details: Tue 26 May 2026 PDT 7:00 – 8:00 UTC 14:00 - 15:00 CEST 16:00 - 17:00 Main pad for discussions: https://pad.ceph.com/p/Ceph_Science_User_Group_20260526 Pad Index: https://pad.ceph.com/p/Ceph_Science_User_Group_Index Meetings will be recorded and posted to the Ceph Youtube channel. To join the meeting on a computer or mobile phone https://meet.jit.si/ceph-science-wg Ceph Community Calendar -- https://ceph.io/en/community/meetups/ See you there! Mattia

1 0

Dashboard 'SignatureDoesNotMatch' error when connecting to RGW
by Iztok Gregori 20 May '26

20 May '26

Hi to all! After upgrading my cluster from squid to tentacle (now on 20.2.1) I'm getting the following error when I try to access any 'page' in the Object section of the Ceph Dashboard: Error connecting to Object Gateway: RGW REST API failed request with status code 403 (b'{"Code":"SignatureDoesNotMatch","Message":"","RequestId":"tx0000000b5bf42356' b'85174-006a0d95b0-308299-eros","HostId":"308299-zone-default"}') I'm pretty sure that in Squid it was working (I don't usually access the Dashboard, but I did after the upgrade to check if everything is ok). The error 'SignatureDoesNotMatch' leads me to believe that there is a problem somewhere with the SSL certificates. But I put RGW_API_SSL_VERIFY to false... I have a SSL certificate (issued by harica.gr) which has different SANs including the hostname of all the nodes where RGW daemon is running (plus a cluster hostname and a wildcard hostname). I used this certificate for the ingress service and for the rgw daemons. Everything is done with a spec file using ceph orchestrator. I'm using the fullchain for the certificates and the the rgw/ingress services are running without any problem Or I'm completely wrong and the problem itself is with the access permissions (error code 403)? Any ideas? Thanks Iztok -- Iztok Gregori ICT Systems and Services Elettra - Sincrotrone Trieste S.C.p.A.

1 0

Experience with D3N RGW Data Cache
by Eugen Block 19 May '26

19 May '26

Hi, I can't recall having read about it on this list, searching the archives only shows rgw logs where d3n is NOT enabled. Has anyone some experience to share with D3N Cache [0]? I'd be interested if people use it in production and if it meets their expectation. After reading [0] I feel like it could help if you have a HDD pool for RGW data but need to access frequently used objects fast. Since the old "cache tier" has been deprecated this could be a replacement. Any comments are appreciated! Thanks! Eugen [0] https://docs.ceph.com/en/latest/radosgw/d3n_datacache/

1 0

Looking for recommendations on Ceph node specifications
by Adam Prycki 19 May '26

19 May '26

Hello, we are preparing to buy hardware for our new ceph clusters. Because of current economic conditions we are considering buying nodes consisting of JBODs + servers. We have 4 potential variants for OSD nodes. Variant A1 ~100 HDD JOBD + x86_64 server with block.db NVMe Variant A2 ~100 HDD JOBD + 2x x86_64 server with block.db NVMe JBOD split in half, each node gets 50 HDDs Variant B1 ~60 HDD JOBD + x86_64 server with block.db NVMe Variant B2 ~60 HDD JOBD + 2x x86_64 server with block.db NVMe JBOD split in half, each node gets 30 HDDs Splitting JBOD logically into 2 servers isn't an issue for use because we will replicate data on rack level and not host level. Common specifications for all variants 5-6GB of RAM per 1 HDD 2% of HDD capacity in NVMe devices for block.db (or none) 2x 50Gb or 2x 100Gb Ethernet per server (active-backup bonded interfaces) (CPU per OSD to be determined) Variant A1 is very unlikely to happen but we are curious what network interface speeds would you suggest for so many HDDs in one node. Variant A2 is the most likely the one we will choose for large deployment. Variant B1/B2 for smaller deployments. Does anyone of you run ceph on similar setups? Did you find any pitfall with it? What are your minimal recommendations for network speed per HDD, cpu per HDD, etc? In our experience most of our servers, even in large clusters, never max out the network interfaces or CPUs. We almost never rebuild or rebalance whole servers. 27 HDD nodes of our biggest CephFS cluster with EC usually have only 2-3Gbps of network traffic. Best regards Adam Prycki

1 0

Re: NVMe/TCP + VMware
by Alan Murrell 18 May '26

18 May '26

Hi Aviv, I have just upgraded my cluster to tentacle and will try it again and report back here. Regards, Alan On Sun, 2026-05-17 at 05:49 +0000, Aviv Caro via ceph-users wrote: > Hi, > > Ceph officially supports nvme/tcp only since tentacle release. In > squid some important code is missing. We can discuss in more in > #nvmeof channel in ceph workspace in slack. > > Aviv > > -----Original Message----- > From: Alan Murrell via ceph-users <ceph-users(a)ceph.io> > Sent: Saturday, 16 May 2026 3:18 > To: ceph-users(a)ceph.io > Subject: [EXTERNAL] [ceph-users] NVMe/TCP + VMware > > Hello, > > I am hoping someone might be able to help, or point me to a > "bulletproof" step-by-step howto of connecting my Ceph 19.2.3 cluster > to my VMware 8.0 cluster as I want to do some testing. > > I have spent the last day and half with Gemini trying to get this to > work, and have been able to get as far as VMware seemingly to see my > RBD images that I am exposing but just simply not seeing them as > disks to create a datastore on -- thye just simply do not show up at > all. > > I am a bit frustrated, but hoping that someone who has done this > (Gemini assures me that it is possible!) can provide some > guidance/pointiners/pont to a step-by-step howto. > > Thanks! :-) > > Regards, > > ALan > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an > email to ceph-users-leave(a)ceph.io > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io

1 0

Re: NVMe/TCP + VMware
by Aviv Caro 17 May '26

17 May '26

Hi, Ceph officially supports nvme/tcp only since tentacle release. In squid some important code is missing. We can discuss in more in #nvmeof channel in ceph workspace in slack. Aviv -----Original Message----- From: Alan Murrell via ceph-users <ceph-users(a)ceph.io> Sent: Saturday, 16 May 2026 3:18 To: ceph-users(a)ceph.io Subject: [EXTERNAL] [ceph-users] NVMe/TCP + VMware Hello, I am hoping someone might be able to help, or point me to a "bulletproof" step-by-step howto of connecting my Ceph 19.2.3 cluster to my VMware 8.0 cluster as I want to do some testing. I have spent the last day and half with Gemini trying to get this to work, and have been able to get as far as VMware seemingly to see my RBD images that I am exposing but just simply not seeing them as disks to create a datastore on -- thye just simply do not show up at all. I am a bit frustrated, but hoping that someone who has done this (Gemini assures me that it is possible!) can provide some guidance/pointiners/pont to a step-by-step howto. Thanks! :-) Regards, ALan _______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

1 0

NVMe/TCP + VMware
by Alan Murrell 16 May '26

16 May '26

Hello, I am hoping someone might be able to help, or point me to a "bulletproof" step-by-step howto of connecting my Ceph 19.2.3 cluster to my VMware 8.0 cluster as I want to do some testing. I have spent the last day and half with Gemini trying to get this to work, and have been able to get as far as VMware seemingly to see my RBD images that I am exposing but just simply not seeing them as disks to create a datastore on -- thye just simply do not show up at all. I am a bit frustrated, but hoping that someone who has done this (Gemini assures me that it is possible!) can provide some guidance/pointiners/pont to a step-by-step howto. Thanks! :-) Regards, ALan

1 0