Re: ACME for the mgmt-gateway
I see, the filter-by only exists on main at the moment. Another alternative is to use "ceph orch certmgr cert check" which should report expiring certificataes as well. Just FYI mgmt-gateway is actually an Nginx-based reverse proxy; that's why I didn't recommend putting an additional reverse proxy in front of it. Anyway good to know the Caddy-based workaround is working in your case. Best, Redouane. On Thu, Apr 9, 2026 at 10:33 AM Matthias Riße via ceph-users < ceph-users@ceph.io> wrote:
Hey Redouane,
thank you for the detailed response!
Am 06.04.26 um 18:10 schrieb Redouane Kachach via ceph-users:
Hi Matthias,
Currently, the mgmt-gateway (and the rest of cephadm services) don't support ACME integration. Users must manually provide/update the external certificates. So as I see it it's not a "lack of documentation", in this case ACME is not a supported option.
Even if there is nothing explicitly built-in there could be a documented "best-practice" approach outlining how to integrate with external tools like certbot, lego, or caddy. But I don't know if this is even common enough of a use-case, maybe most people are contend with using the self-signed certificates since the dashboard is internal-/admin-only anyway, and I am the outlier :)
That said, I think this can be achieved using the current implementation without any extra code changes in cephadm. The key insight is that cephadm's certmgr already has all the necessary components: it monitors cert expiry, and when it detects a certificate has changed in the store, it automatically
triggers a
reconfig of the mgmt-gateway. So the only missing piece for externally-managed certs is the renewal trigger itself. For that, "ceph orch certmgr cert ls --filter-by status=expiring" can be used to detect expiring certificates and kick off the renewal process.
This command doesn't seem to work, the --filter-by flag is unknown:
# ceph orch certmgr cert ls --filter-by status=expiring Invalid command: Unexpected argument '--filter-by' orch certmgr cert ls [--show-details] [--format {plain|json|json-pretty|yaml|xml-pretty|xml}] : Error EINVAL: invalid command # ceph --version ceph version 20.2.0 (69f84cc2651aa259a15bc192ddaabd3baba07489) tentacle (stable)
But certbot, lego, et al. on a cron timer should check themselves if the certs are close to expiry and only update them when they are about to expire, so this shouldn't be needed anyway.
The procedure can be:
1. Set certificate_source: reference in the mgmt-gateway spec. This tells certmgr the cert is externally owned and makes it accept pushes via "ceph orch certmgr cert set." 2. Run certbot (or your tool of choice) with DNS-01 (or HTTP)
challenge
which avoids any port-80 conflict with mgmt-gateway already owning
the
HTTPS endpoint. 3. A simple daily systemd timer runs a script that: - Queries ceph orch certmgr cert ls --filter-by "status=expiring"
to
check if renewal is needed - If yes, calls cerbot renew procedure - Pushes the new cert+key into certmgr via ceph "orch certmgr cert set / key set" - cephadm's serve loop detects the cert change on its next cycle
and
reconfigures mgmt-gateway automatically (no manual restart
needed).
4. For security, the script should use a dedicated client keyring
with
caps restricted to only the two certmgr write commands.
For now I've gone with the different approach of putting a caddy reverse proxy in front of the mgmt-gateway, but only because I had a config for that using our EAB setup lying around anyway. To do so I've reconfigured mgmt-gateway to only listen locally on a different port:
placement: label: mgmt networks: - 127.0.0.1 spec: port: 5000
and created a custom container service running caddy with the same placement rule:
placement: label: mgmt spec: args: - --net=host bind_mounts: - - type=bind - source=conf - destination=/etc/caddy - ro=true dirs: - conf files: conf/Caddyfile: - '{' - ' acme_ca <redacted>' - ' acme_eab {' - ' key_id <redacted>' - ' mac_key <redacted>' - ' }' - '}' - <redacted> { - ' reverse_proxy 127.0.0.1:5000' - '}' image: docker.io/library/caddy:2 ports: - 80 - 443
Seems to be working fine for now, but I still might replace this with setting the mgmt-gateway certs directly.
Best Matthias
Best, Redouane.
On Mon, Mar 30, 2026 at 2:27 PM Matthias Riße via ceph-users < ceph-users@ceph.io> wrote:
Hey all!
I've just managed to get the mgmt-gateway working on our Ceph Cluster with cephadm, which is quite cool to have as a single HA entrypoint. I am surprised about the lack of documentation to get ACME-signed certificates into / in front of the mgmt-gateway.
Is anyone doing this already? If yes, what's your setup? I am wondering if it is easier to put certbot, lego, or similar into a "custom container service" and update the mgmt-gateway service with renewed certs from there, or run an additional caddy instance (also as a custom container service?) as a reverse proxy in front of the mgmt-gateway and let it handle ACME.
Kind regards Matthias
--
---------------------------------------------------------------------------------------------
Forschungszentrum Jülich GmbH 52425 Jülich Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Stefan Müller Geschäftsführung: Prof. Dr. Astrid Lambrecht (Vorsitzende), Dr. Stephanie Bauer (stellvertretende Vorsitzende), Prof. Dr. Ir. Pieter Jansens, Prof. Dr. Laurens Kuipers
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
---------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------- Forschungszentrum Jülich GmbH 52425 Jülich Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Stefan Müller Geschäftsführung: Prof. Dr. Astrid Lambrecht (Vorsitzende), Dr. Stephanie Bauer (stellvertretende Vorsitzende), Prof. Dr. Ir. Pieter Jansens, Prof. Dr. Laurens Kuipers
---------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (1)
-
Redouane Kachach