- ceph-users - lists.ceph.io

Re: Questions about dm-cache
by Matthias Ferdinand 09 Apr '26

09 Apr '26

On Thu, Apr 09, 2026 at 03:22:00PM +0000, Alexis Bauvin via ceph-users wrote: > > Le 09/04/2026 16:44 CEST, Janne Johansson <icepic.dz(a)gmail.com> a écrit : > > > > > > Den tors 9 apr. 2026 kl 15:58 skrev Alexis Bauvin via ceph-users > > <ceph-users(a)ceph.io>: > > > Since a few versions, Ceph has deprecated caching tiers with no alternative mentioned. Thus I've been looking and somewhat settled on using dm-cache for two reasons: it's LVM and ceph-volume mentions it's supported and "transparent": > > > > > > > For ceph-volume, the use of dm-cache is transparent, there is no difference for the tool, and it treats dm-cache like a plain logical volume. > > > > > > However, what does this really mean? > > > > It means you do it disk-for-disk instead of a fast pool in front of a > > slow pool. In the end, the same results should be there, faster > > accesses for writes and recently accessed data. > > Oh, that makes sense. Thanks for the clarification, and it explains a lot. > > A subsequent question is: is it transparent to the point I can setup the cache on a live OSD, or do I need to recreate it and have ceph-volume have a go *with* the cache setup, so it knows about it? > The way I understand it is "ceph does not care about dm-cache" so I would need, at worst, to restart the OSD. disclaimer: never used dm-cache in production, last experimented with it several years ago. I seem to remember that the transparency of dm-cache comes with a risk: LVM device name stays the same whether the cache device is attached or not. Nothing keeps you from writing new data to the device with cache device detached even if said caching device still contains some unwritten data. So you might lose that in-cache data using LVM volume when cache device is not attached (e.g. after reboot, perhaps ceph grabbing the device before cache device attach). And when re-attaching a cache device later it might just kill the fs by replaying outdated cached data. I have used bcache in production with ceph. Bcache requires you to format the block device as a bcache device, and the backing device knows if cached data is missing and refuses to be mounted until the corresponding cache device is attached and such cached data is accessible. This provides better reliabilty, but with some upfront cost and some additional parameters to set (cache mode, watermarks, ...) Still, this might just be outdated knowledge of the past, YMMV. Matthias > > > > Is it that I can just slap an lvm cache on the LVs created by ceph-volume and call it a day? > > > Is this the canonical replacement to Ceph's native cache tiers? > > > > I guess it is one of the default alternatives now that tiering is off > > the map. We do a lot of "put WAL + DB on nvme partition, data on > > spin-disks" with something like 4 or 6 nvme partitions per OSD disk, > > so this mostly improves database access parts and less of the bulk > > data transfers, but we think that is ok. > > Yeah, HDD setups is what I tried to mimic, as I have the WAL+DB on actual write-intensive enterprise drives, but it can only do so much when the backend drive does GC pauses that last for 20 min and increase the IO latency to the 2s-20s range. I'm hoping caching can shield most workloads from it. > > > > Is the tuning of the cache settings somehow managed by Ceph or is it fully on the user? And, related, can the caching be declared in the orch OSD spec? > > > > If you stack devices in LVM, then I guess that is where any tuning > > would have to happen, though I don't think there is a large need to > > tune those settings a lot. The defaults should probably work fine for > > most people. > > Noted. I don't expect to have to tune a lot, but given the (horrible) behavior of my drives, being able to tune flushes and invalidations is important. > > > > As for the context: I have a cluster with *very* bad SSDs (consumer DRAM-less BX500) and I'm trying to get it somewhat OK until the market for flash improves. As the problems are due to GC pauses from the drive caused by the sustained IO, I'm hoping that putting a cache tier (with smaller faster enterprise SSDs) in front of the drives will change the IO behavior enough for the drives to perform their GC without impacting the cluster. I already have the good smaller drives that have WAL+DB for the slow drives, but they have enough free space to cache the block. > > > > I don't have any idea on which method will be best in the end, but I > > think dm-cache is stable enough for people to recommend it. > > That's what I gathered from poking around the internet. The fact that the ceph documentations mention it, while not an endorsement, is already meaningful for the viability of dm-cache. Thanks for the confirmation! > > Thank you for the reply and the pointers! > > > -- > > May the most significant bit of your life be positive. > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io

1 0

Re: Questions about dm-cache
by Janne Johansson 09 Apr '26

09 Apr '26

Den tors 9 apr. 2026 kl 15:58 skrev Alexis Bauvin via ceph-users <ceph-users(a)ceph.io>: > Since a few versions, Ceph has deprecated caching tiers with no alternative mentioned. Thus I've been looking and somewhat settled on using dm-cache for two reasons: it's LVM and ceph-volume mentions it's supported and "transparent": > > > For ceph-volume, the use of dm-cache is transparent, there is no difference for the tool, and it treats dm-cache like a plain logical volume. > > However, what does this really mean? It means you do it disk-for-disk instead of a fast pool in front of a slow pool. In the end, the same results should be there, faster accesses for writes and recently accessed data. > Is it that I can just slap an lvm cache on the LVs created by ceph-volume and call it a day? > Is this the canonical replacement to Ceph's native cache tiers? I guess it is one of the default alternatives now that tiering is off the map. We do a lot of "put WAL + DB on nvme partition, data on spin-disks" with something like 4 or 6 nvme partitions per OSD disk, so this mostly improves database access parts and less of the bulk data transfers, but we think that is ok. > Is the tuning of the cache settings somehow managed by Ceph or is it fully on the user? And, related, can the caching be declared in the orch OSD spec? If you stack devices in LVM, then I guess that is where any tuning would have to happen, though I don't think there is a large need to tune those settings a lot. The defaults should probably work fine for most people. > As for the context: I have a cluster with *very* bad SSDs (consumer DRAM-less BX500) and I'm trying to get it somewhat OK until the market for flash improves. As the problems are due to GC pauses from the drive caused by the sustained IO, I'm hoping that putting a cache tier (with smaller faster enterprise SSDs) in front of the drives will change the IO behavior enough for the drives to perform their GC without impacting the cluster. I already have the good smaller drives that have WAL+DB for the slow drives, but they have enough free space to cache the block. I don't have any idea on which method will be best in the end, but I think dm-cache is stable enough for people to recommend it. -- May the most significant bit of your life be positive.

2 1

Questions about dm-cache
by Alexis Bauvin 09 Apr '26

09 Apr '26

Hi, Since a few versions, Ceph has deprecated caching tiers with no alternative mentioned. Thus I've been looking and somewhat settled on using dm-cache for two reasons: it's LVM and ceph-volume mentions it's supported and "transparent": > For ceph-volume, the use of dm-cache is transparent, there is no difference for the tool, and it treats dm-cache like a plain logical volume. However, what does this really mean? Is it that I can just slap an lvm cache on the LVs created by ceph-volume and call it a day? Is this the canonical replacement to Ceph's native cache tiers? Is the tuning of the cache settings somehow managed by Ceph or is it fully on the user? And, related, can the caching be declared in the orch OSD spec? As for the context: I have a cluster with *very* bad SSDs (consumer DRAM-less BX500) and I'm trying to get it somewhat OK until the market for flash improves. As the problems are due to GC pauses from the drive caused by the sustained IO, I'm hoping that putting a cache tier (with smaller faster enterprise SSDs) in front of the drives will change the IO behavior enough for the drives to perform their GC without impacting the cluster. I already have the good smaller drives that have WAL+DB for the slow drives, but they have enough free space to cache the block. Thanks!

1 0

Re: ACME for the mgmt-gateway
by Redouane Kachach 09 Apr '26

09 Apr '26

I see, the filter-by only exists on main at the moment. Another alternative is to use "ceph orch certmgr cert check" which should report expiring certificataes as well. Just FYI mgmt-gateway is actually an Nginx-based reverse proxy; that's why I didn't recommend putting an additional reverse proxy in front of it. Anyway good to know the Caddy-based workaround is working in your case. Best, Redouane. On Thu, Apr 9, 2026 at 10:33 AM Matthias Riße via ceph-users < ceph-users(a)ceph.io> wrote: > Hey Redouane, > > thank you for the detailed response! > > Am 06.04.26 um 18:10 schrieb Redouane Kachach via ceph-users: > > Hi Matthias, > > > > Currently, the mgmt-gateway (and the rest of cephadm services) don't > > support ACME integration. Users must manually provide/update the > > external certificates. So as I see it it's not a "lack of documentation", > > in this case ACME is not a supported option. > > Even if there is nothing explicitly built-in there could be a documented > "best-practice" approach outlining how to integrate with external tools > like certbot, lego, or caddy. But I don't know if this is even common > enough of a use-case, maybe most people are contend with using the > self-signed certificates since the dashboard is internal-/admin-only > anyway, and I am the outlier :) > > > > > That said, I think this can be achieved using the current implementation > > without any extra code changes in cephadm. The key insight is that > > cephadm's certmgr already > > has all the necessary components: it monitors cert expiry, and when it > > detects a certificate has changed in the store, it automatically > triggers a > > reconfig of the mgmt-gateway. > > So the only missing piece for externally-managed certs is the renewal > > trigger itself. For that, "ceph orch certmgr cert ls --filter-by > > status=expiring" can be used to detect expiring > > certificates and kick off the renewal process. > > This command doesn't seem to work, the --filter-by flag is unknown: > > # ceph orch certmgr cert ls --filter-by status=expiring > Invalid command: Unexpected argument '--filter-by' > orch certmgr cert ls [--show-details] [--format > {plain|json|json-pretty|yaml|xml-pretty|xml}] : > Error EINVAL: invalid command > # ceph --version > ceph version 20.2.0 (69f84cc2651aa259a15bc192ddaabd3baba07489) tentacle > (stable) > > But certbot, lego, et al. on a cron timer should check themselves if the > certs are close to expiry and only update them when they are about to > expire, so this shouldn't be needed anyway. > > > > > The procedure can be: > > > > 1. Set certificate_source: reference in the mgmt-gateway spec. This > > tells certmgr the cert is externally owned and makes it accept > > pushes via "ceph > > orch certmgr cert set." > > 2. Run certbot (or your tool of choice) with DNS-01 (or HTTP) > challenge > > which avoids any port-80 conflict with mgmt-gateway already owning > the > > HTTPS endpoint. > > 3. A simple daily systemd timer runs a script that: > > - Queries ceph orch certmgr cert ls --filter-by "status=expiring" > to > > check if renewal is needed > > - If yes, calls cerbot renew procedure > > - Pushes the new cert+key into certmgr via ceph "orch certmgr cert > > set / key set" > > - cephadm's serve loop detects the cert change on its next cycle > and > > reconfigures mgmt-gateway automatically (no manual restart > needed). > > 4. For security, the script should use a dedicated client keyring > with > > caps restricted to only the two certmgr write commands. > > > > For now I've gone with the different approach of putting a caddy reverse > proxy in front of the mgmt-gateway, but only because I had a config for > that using our EAB setup lying around anyway. To do so I've reconfigured > mgmt-gateway to only listen locally on a different port: > > placement: > label: mgmt > networks: > - 127.0.0.1 > spec: > port: 5000 > > and created a custom container service running caddy with the same > placement rule: > > placement: > label: mgmt > spec: > args: > - --net=host > bind_mounts: > - - type=bind > - source=conf > - destination=/etc/caddy > - ro=true > dirs: > - conf > files: > conf/Caddyfile: > - '{' > - ' acme_ca <redacted>' > - ' acme_eab {' > - ' key_id <redacted>' > - ' mac_key <redacted>' > - ' }' > - '}' > - <redacted> { > - ' reverse_proxy 127.0.0.1:5000' > - '}' > image: docker.io/library/caddy:2 > ports: > - 80 > - 443 > > Seems to be working fine for now, but I still might replace this with > setting the mgmt-gateway certs directly. > > Best > Matthias > > > > > Best, > > Redouane. > > > > > > On Mon, Mar 30, 2026 at 2:27 PM Matthias Riße via ceph-users < > > ceph-users(a)ceph.io> wrote: > > > >> Hey all! > >> > >> I've just managed to get the mgmt-gateway working on our Ceph Cluster > >> with cephadm, which is quite cool to have as a single HA entrypoint. I > >> am surprised about the lack of documentation to get ACME-signed > >> certificates into / in front of the mgmt-gateway. > >> > >> Is anyone doing this already? If yes, what's your setup? I am wondering > >> if it is easier to put certbot, lego, or similar into a "custom > >> container service" and update the mgmt-gateway service with renewed > >> certs from there, or run an additional caddy instance (also as a custom > >> container service?) as a reverse proxy in front of the mgmt-gateway and > >> let it handle ACME. > >> > >> Kind regards > >> Matthias > >> > >> -- > >> > >> > --------------------------------------------------------------------------------------------- > >> > >> > --------------------------------------------------------------------------------------------- > >> Forschungszentrum Jülich GmbH > >> 52425 Jülich > >> Sitz der Gesellschaft: Jülich > >> Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 > >> Vorsitzender des Aufsichtsrats: MinDir Stefan Müller > >> Geschäftsführung: Prof. Dr. Astrid Lambrecht (Vorsitzende), > >> Dr. Stephanie Bauer (stellvertretende Vorsitzende), > >> Prof. Dr. Ir. Pieter Jansens, Prof. Dr. Laurens Kuipers > >> > >> > --------------------------------------------------------------------------------------------- > >> > >> > --------------------------------------------------------------------------------------------- > >> > >> _______________________________________________ > >> ceph-users mailing list -- ceph-users(a)ceph.io > >> To unsubscribe send an email to ceph-users-leave(a)ceph.io > >> > > _______________________________________________ > > ceph-users mailing list -- ceph-users(a)ceph.io > > To unsubscribe send an email to ceph-users-leave(a)ceph.io > > -- > > --------------------------------------------------------------------------------------------- > > --------------------------------------------------------------------------------------------- > Forschungszentrum Jülich GmbH > 52425 Jülich > Sitz der Gesellschaft: Jülich > Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 > Vorsitzender des Aufsichtsrats: MinDir Stefan Müller > Geschäftsführung: Prof. Dr. Astrid Lambrecht (Vorsitzende), > Dr. Stephanie Bauer (stellvertretende Vorsitzende), > Prof. Dr. Ir. Pieter Jansens, Prof. Dr. Laurens Kuipers > > --------------------------------------------------------------------------------------------- > > --------------------------------------------------------------------------------------------- > > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io >

1 0

Has anyone been able to use NFS in tentacle or it is really broken?
by Robert Sander 09 Apr '26

09 Apr '26

Hi, I just tried to deploy an NFS service on a freshly installed 20.2.1 cluster and it fails. ceph nfs cluster create nfs01 "label:nfs01" There is exactly one host with the label nfs01. There are some entries in the log showing errors but the last two are - Cannot acquire credentials for principal nfs - Cannot register NFS V3 on TCP /var/lib/ceph/2ac9b090-3363-11f1-bd6a-bc2411d682b7/nfs.nfs01.0.0.rgw01.hpfeuz/etc/ganesha/ganesha.conf contained "Protocols 3, 4;" in the NFS_CORE_PARAM section. After removing the 3 the container starts. NFSv3 was added in commit 1e69ebc6ceaf in February 2024. It was not contained in Ceph squid. Apr 09 14:23:18 HOST systemd[1]: Starting ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7(a)nfs.nfs01.0.0.HOST.hpfeuz.service - Ceph nfs.nfs01.0.0.HOST.hpfeuz for 2ac9b090-3363-11f1-bd6a-bc2411d682b7... Apr 09 14:23:18 HOST podman[31327]: 2026-04-09 14:23:18.942654365 +0200 CEST m=+0.048118777 container create c2b0e96637297b20d6bd39bf1eb68fc70b0fb5635dc72c9b887eeb82cd2bac77 (image=quay.io/ceph/ceph@sha256:0bae386bc859cd9a05b804d1ca16cca8853a64f90809044e2bf43095419dc337, name=ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team <ceph-maintainers(a)ceph.io>, CEPH_REF=tentacle, OSD_FLAVOR=default, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_SHA1=6a49aff47758778a5f5951e731d437c317f72fb2, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.build-date=20260331, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream…, io.buildah.version=1.43.0, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) Apr 09 14:23:19 HOST podman[31327]: 2026-04-09 14:23:19.000318083 +0200 CEST m=+0.105782538 container init c2b0e96637297b20d6bd39bf1eb68fc70b0fb5635dc72c9b887eeb82cd2bac77 (image=quay.io/ceph/ceph@sha256:0bae386bc859cd9a05b804d1ca16cca8853a64f90809044e2bf43095419dc337, name=ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream…, io.buildah.version=1.43.0, org.opencontainers.image.authors=Ceph Release Team <ceph-maintainers(a)ceph.io>, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_GIT_REPO=https://github.com/ceph/ceph.git, CEPH_SHA1=6a49aff47758778a5f5951e731d437c317f72fb2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_REF=tentacle, OSD_FLAVOR=default, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.build-date=20260331) Apr 09 14:23:19 HOST podman[31327]: 2026-04-09 14:23:19.004217805 +0200 CEST m=+0.109682235 container start c2b0e96637297b20d6bd39bf1eb68fc70b0fb5635dc72c9b887eeb82cd2bac77 (image=quay.io/ceph/ceph@sha256:0bae386bc859cd9a05b804d1ca16cca8853a64f90809044e2bf43095419dc337, name=ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.authors=Ceph Release Team <ceph-maintainers(a)ceph.io>, CEPH_REF=tentacle, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream…, org.label-schema.vendor=CentOS, ceph=True, CEPH_GIT_REPO=https://github.com/ceph/ceph.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20260331, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, CEPH_SHA1=6a49aff47758778a5f5951e731d437c317f72fb2, io.buildah.version=1.43.0, org.label-schema.license=GPLv2) Apr 09 14:23:19 HOST bash[31327]: c2b0e96637297b20d6bd39bf1eb68fc70b0fb5635dc72c9b887eeb82cd2bac77 Apr 09 14:23:19 HOST podman[31327]: 2026-04-09 14:23:18.922220554 +0200 CEST m=+0.027684991 image pull fb63cba66eea5516fbd9a1ec035c265abdbc1a1a8fc2dc0b51057557e49ca28c quay.io/ceph/ceph@sha256:0bae386bc859cd9a05b804d1ca16cca8853a64f90809044e2bf43095419dc337 Apr 09 14:23:19 HOST systemd[1]: Started ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7(a)nfs.nfs01.0.0.HOST.hpfeuz.service - Ceph nfs.nfs01.0.0.HOST.hpfeuz for 2ac9b090-3363-11f1-bd6a-bc2411d682b7. Apr 09 14:23:19 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:19 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] init_logging :LOG :NULL :LOG: Setting log level for all components to NIV_EVENT Apr 09 14:23:19 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:19 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] main :MAIN :EVENT :ganesha.nfsd Starting: Ganesha Version 5.9 Apr 09 14:23:19 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:19 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] nfs_set_param_from_conf :NFS STARTUP :EVENT :Configuration file successfully parsed Apr 09 14:23:19 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:19 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] monitoring_init :NFS STARTUP :EVENT :Init monitoring at 0.0.0.0:9587 Apr 09 14:23:19 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:19 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] fsal_init_fds_limit :MDCACHE LRU :EVENT :Setting the system-imposed limit on FDs to 1048576. Apr 09 14:23:19 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:19 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] init_server_pkgs :NFS STARTUP :EVENT :Initializing ID Mapper. Apr 09 14:23:19 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:19 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] init_server_pkgs :NFS STARTUP :EVENT :ID Mapper successfully initialized. Apr 09 14:23:19 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:19 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] nfs_start_grace :STATE :EVENT :NFS Server Now IN GRACE, duration 90 Apr 09 14:23:19 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:19 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] rados_kv_traverse :CLIENT ID :EVENT :Failed to lst kv ret=-2 Apr 09 14:23:19 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:19 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] rados_cluster_read_clids :CLIENT ID :EVENT :Failed to traverse recovery db: -2 Apr 09 14:23:19 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:19 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] nfs_start_grace :STATE :EVENT :grace reload client info completed from backend Apr 09 14:23:19 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:19 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] nfs_try_lift_grace :STATE :EVENT :check grace:reclaim complete(0) clid count(0) Apr 09 14:23:22 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:22 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] rados_cluster_end_grace :CLIENT ID :EVENT :Failed to remove rec-0000000000000001:0: -2 Apr 09 14:23:22 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:22 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] nfs_lift_grace_locked :STATE :EVENT :NFS Server Now NOT IN GRACE Apr 09 14:23:22 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:22 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] main :NFS STARTUP :WARN :No export entries found in configuration file !!! Apr 09 14:23:22 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:22 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] config_errs_to_log :CONFIG :WARN :Config File (/etc/ganesha/ganesha.conf:9): Unknown parameter (allow_set_io_flusher_fail) Apr 09 14:23:22 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:22 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] config_errs_to_log :CONFIG :WARN :Config File (/etc/ganesha/ganesha.conf:28): Unknown block (RADOS_URLS) Apr 09 14:23:22 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:22 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] config_errs_to_log :CONFIG :WARN :Config File (/etc/ganesha/ganesha.conf:33): Unknown block (RGW) Apr 09 14:23:22 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:22 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] lower_my_caps :NFS STARTUP :EVENT :CAP_SYS_RESOURCE was successfully removed for proper quota management in FSAL Apr 09 14:23:22 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:22 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] lower_my_caps :NFS STARTUP :EVENT :currently set capabilities are: cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_sys_chroot,cap_setfcap=ep Apr 09 14:23:22 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:22 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] gsh_dbus_pkginit :DBUS :CRIT :dbus_bus_get failed (Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory) Apr 09 14:23:22 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:22 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] gsh_dbus_register_path :DBUS :CRIT :dbus_connection_register_object_path called with no DBUS connection Apr 09 14:23:22 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:22 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] gsh_dbus_register_path :DBUS :CRIT :dbus_connection_register_object_path called with no DBUS connection Apr 09 14:23:22 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:22 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] gsh_dbus_register_path :DBUS :CRIT :dbus_connection_register_object_path called with no DBUS connection Apr 09 14:23:22 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:22 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] nfs_Init_svc :DISP :CRIT :Cannot acquire credentials for principal nfs Apr 09 14:23:22 HOST ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz[31339]: 09/04/2026 12:23:22 : epoch 69d79a37 : HOST : ganesha.nfsd-2[main] __Register_program :DISP :MAJ :Cannot register NFS V3 on TCP Apr 09 14:23:22 HOST podman[31583]: 2026-04-09 14:23:22.269193428 +0200 CEST m=+0.027823817 container died c2b0e96637297b20d6bd39bf1eb68fc70b0fb5635dc72c9b887eeb82cd2bac77 (image=quay.io/ceph/ceph@sha256:0bae386bc859cd9a05b804d1ca16cca8853a64f90809044e2bf43095419dc337, name=ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz, CEPH_SHA1=6a49aff47758778a5f5951e731d437c317f72fb2, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, io.buildah.version=1.43.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_REF=tentacle, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream…, org.label-schema.build-date=20260331, CEPH_GIT_REPO=https://github.com/ceph/ceph.git, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.license=GPLv2, org.opencontainers.image.authors=Ceph Release Team <ceph-maintainers(a)ceph.io>) Apr 09 14:23:22 HOST podman[31583]: 2026-04-09 14:23:22.325861901 +0200 CEST m=+0.084492274 container remove c2b0e96637297b20d6bd39bf1eb68fc70b0fb5635dc72c9b887eeb82cd2bac77 (image=quay.io/ceph/ceph@sha256:0bae386bc859cd9a05b804d1ca16cca8853a64f90809044e2bf43095419dc337, name=ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7-nfs-nfs01-0-0-HOST-hpfeuz, ceph=True, org.label-schema.vendor=CentOS, CEPH_REF=tentacle, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream…, OSD_FLAVOR=default, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_SHA1=6a49aff47758778a5f5951e731d437c317f72fb2, org.label-schema.build-date=20260331, io.buildah.version=1.43.0, org.opencontainers.image.authors=Ceph Release Team <ceph-maintainers(a)ceph.io>, CEPH_GIT_REPO=https://github.com/ceph/ceph.git, org.label-schema.name=CentOS Stream 9 Base Image, FROM_IMAGE=quay.io/centos/centos:stream9) Apr 09 14:23:22 HOST systemd[1]: ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7(a)nfs.nfs01.0.0.HOST.hpfeuz.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Apr 09 14:23:22 HOST systemd[1]: ceph-2ac9b090-3363-11f1-bd6a-bc2411d682b7(a)nfs.nfs01.0.0.HOST.hpfeuz.service: Failed with result 'exit-code'. Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 220009 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

1 0

Re: v20.2.2 merge window is now open!
by Janne Johansson 09 Apr '26

09 Apr '26

> > > It woulb be nice to include this: > https://tracker.ceph.com/issues/71310 > <https://qcflzmq6.r.us-east-1.awstrack.me/L0/https:%2F%2Ftracker.ceph.com%2F…> > Sorry for hijacking this thread, but could someone close to the list admins rework the mail list manager settings so that the above link does NOT point to https://qcflzmq6.r.us-east-1.awstrack.me/L0/https:%2F%2Ftracker.ceph.com%2F… and instead actually takes you to tracker.ceph.com directly? It's kind of a dark pattern to say "this link goes here" and then actually send people somewhere else. In my case, the ad-blockers and whatnot are preventing me from doing the double-jump, but it becomes tedious to have to cut-n-paste the text-of-the-link instead of just clicking the link directly. You have our emails already, don't have to grab our browser cookies or place trackers on us for these kinds of technical links. If anyone is REALLY interested in if people will go to the issue 71310, I'm sure the tracker site has access logs you can look at instead. -- May the most significant bit of your life be positive.

1 0

Re: v20.2.2 merge window is now open!
by Matthew H 09 Apr '26

09 Apr '26

Hi, I don't see a PR for this, nor do I see the tracker having a backport label to tentacle. It would be good to have you (or someone else) propose a PR, and it can be added to the tracker for review. ________________________________ From: Alfonso Escribano Merino via ceph-users <ceph-users(a)ceph.io> Sent: Thursday, April 9, 2026 1:07 AM To: Patrick Donnelly <pdonnell(a)redhat.com>; dev <dev(a)ceph.io> Cc: ceph-users <ceph-users(a)ceph.io> Subject: [ceph-users] Re: v20.2.2 merge window is now open! Hello, It woulb be nice to include this: https://tracker.ceph.com/issues/71310<https://qcflzmq6.r.us-east-1.awstrack.me/L0/https:%2F%2Ftracker.ceph.com%2F…> I have two classes of nvme disks, ones (TLC) with 7TB and others (QLC) with 30TB for different pools and it would be great to have the patch that is in #71310 merged into master to have it on 20.2.2 and also backport to older releases. Thanks, Alfonso Escribano Patrick Donnelly via ceph-users wrote: Hello everyone, v20.2.1 has finally been released! The tentacle merge window is open again. Please follow this tracker ticket for updates and processes for getting backporrts merged: https://tracker.ceph.com/issues/75878<https://qcflzmq6.r.us-east-1.awstrack.me/L0/https:%2F%2Ftracker.ceph.com%2F…> For discussion, please visit #ceph-upstream-releases-tentacle on Slack, respond to this mail, or come to the "Ceph Release Engineering" on Thursdays (see Ceph community calendar). Thanks, -- Alfonso Escribano alfonso.escribano(a)meteologica.com<mailto:alfonso.escribano@meteologica.com> [http://www.meteologica.com/assets/images/logo.png]<https://qcflzmq6.r.us-east-1.awstrack.me/L0/https:%2F%2Fwww.meteologica.com…> c/ Costa Brava 10 - 28034 Madrid Office: +34 91 456 10 01 This email and any files transmitted with it are confidential and may be privileged and are intended solely for the use of the individual or entity to whom they are addressed. If you are not an addressee, any unauthorized direct or indirect dissemination, distribution, publication, reading, or copying of this message and any attachments is strictly prohibited. If you have received this email in error please notify the sender immediately, and delete this email from your system. Email communication does not guarantee the confidentiality, integrity, or proper receipt of the messages sent. Meteológica, S.A. accepts no liability for such circumstances.

1 0

Re: ACME for the mgmt-gateway
by Matthias Riße 09 Apr '26

09 Apr '26

Hey Redouane, thank you for the detailed response! Am 06.04.26 um 18:10 schrieb Redouane Kachach via ceph-users: > Hi Matthias, > > Currently, the mgmt-gateway (and the rest of cephadm services) don't > support ACME integration. Users must manually provide/update the > external certificates. So as I see it it's not a "lack of documentation", > in this case ACME is not a supported option. Even if there is nothing explicitly built-in there could be a documented "best-practice" approach outlining how to integrate with external tools like certbot, lego, or caddy. But I don't know if this is even common enough of a use-case, maybe most people are contend with using the self-signed certificates since the dashboard is internal-/admin-only anyway, and I am the outlier :) > > That said, I think this can be achieved using the current implementation > without any extra code changes in cephadm. The key insight is that > cephadm's certmgr already > has all the necessary components: it monitors cert expiry, and when it > detects a certificate has changed in the store, it automatically triggers a > reconfig of the mgmt-gateway. > So the only missing piece for externally-managed certs is the renewal > trigger itself. For that, "ceph orch certmgr cert ls --filter-by > status=expiring" can be used to detect expiring > certificates and kick off the renewal process. This command doesn't seem to work, the --filter-by flag is unknown: # ceph orch certmgr cert ls --filter-by status=expiring Invalid command: Unexpected argument '--filter-by' orch certmgr cert ls [--show-details] [--format {plain|json|json-pretty|yaml|xml-pretty|xml}] : Error EINVAL: invalid command # ceph --version ceph version 20.2.0 (69f84cc2651aa259a15bc192ddaabd3baba07489) tentacle (stable) But certbot, lego, et al. on a cron timer should check themselves if the certs are close to expiry and only update them when they are about to expire, so this shouldn't be needed anyway. > > The procedure can be: > > 1. Set certificate_source: reference in the mgmt-gateway spec. This > tells certmgr the cert is externally owned and makes it accept > pushes via "ceph > orch certmgr cert set." > 2. Run certbot (or your tool of choice) with DNS-01 (or HTTP) challenge > which avoids any port-80 conflict with mgmt-gateway already owning the > HTTPS endpoint. > 3. A simple daily systemd timer runs a script that: > - Queries ceph orch certmgr cert ls --filter-by "status=expiring" to > check if renewal is needed > - If yes, calls cerbot renew procedure > - Pushes the new cert+key into certmgr via ceph "orch certmgr cert > set / key set" > - cephadm's serve loop detects the cert change on its next cycle and > reconfigures mgmt-gateway automatically (no manual restart needed). > 4. For security, the script should use a dedicated client keyring with > caps restricted to only the two certmgr write commands. > For now I've gone with the different approach of putting a caddy reverse proxy in front of the mgmt-gateway, but only because I had a config for that using our EAB setup lying around anyway. To do so I've reconfigured mgmt-gateway to only listen locally on a different port: placement: label: mgmt networks: - 127.0.0.1 spec: port: 5000 and created a custom container service running caddy with the same placement rule: placement: label: mgmt spec: args: - --net=host bind_mounts: - - type=bind - source=conf - destination=/etc/caddy - ro=true dirs: - conf files: conf/Caddyfile: - '{' - ' acme_ca <redacted>' - ' acme_eab {' - ' key_id <redacted>' - ' mac_key <redacted>' - ' }' - '}' - <redacted> { - ' reverse_proxy 127.0.0.1:5000' - '}' image: docker.io/library/caddy:2 ports: - 80 - 443 Seems to be working fine for now, but I still might replace this with setting the mgmt-gateway certs directly. Best Matthias > > Best, > Redouane. > > > On Mon, Mar 30, 2026 at 2:27 PM Matthias Riße via ceph-users < > ceph-users(a)ceph.io> wrote: > >> Hey all! >> >> I've just managed to get the mgmt-gateway working on our Ceph Cluster >> with cephadm, which is quite cool to have as a single HA entrypoint. I >> am surprised about the lack of documentation to get ACME-signed >> certificates into / in front of the mgmt-gateway. >> >> Is anyone doing this already? If yes, what's your setup? I am wondering >> if it is easier to put certbot, lego, or similar into a "custom >> container service" and update the mgmt-gateway service with renewed >> certs from there, or run an additional caddy instance (also as a custom >> container service?) as a reverse proxy in front of the mgmt-gateway and >> let it handle ACME. >> >> Kind regards >> Matthias >> >> -- >> >> --------------------------------------------------------------------------------------------- >> >> --------------------------------------------------------------------------------------------- >> Forschungszentrum Jülich GmbH >> 52425 Jülich >> Sitz der Gesellschaft: Jülich >> Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 >> Vorsitzender des Aufsichtsrats: MinDir Stefan Müller >> Geschäftsführung: Prof. Dr. Astrid Lambrecht (Vorsitzende), >> Dr. Stephanie Bauer (stellvertretende Vorsitzende), >> Prof. Dr. Ir. Pieter Jansens, Prof. Dr. Laurens Kuipers >> >> --------------------------------------------------------------------------------------------- >> >> --------------------------------------------------------------------------------------------- >> >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io >> > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io -- --------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------- Forschungszentrum Jülich GmbH 52425 Jülich Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Stefan Müller Geschäftsführung: Prof. Dr. Astrid Lambrecht (Vorsitzende), Dr. Stephanie Bauer (stellvertretende Vorsitzende), Prof. Dr. Ir. Pieter Jansens, Prof. Dr. Laurens Kuipers --------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------

1 0

Re: MGR stuck in starting state
by Iztok Gregori 09 Apr '26

09 Apr '26

Hi! A little update. We were able to make the ceph-mgr back working, but we are still don't know what was wrong. To made it work again we restarted (one at the time) the MONs, after restarting the 2 MON (out of 3) the MGR started working again. We are still investigation why we are unable to add new MONs to the cluster (they are stucked in probing/electing phase), but it is a topic for another thread. Cheers Iztok On 08/04/26 18:14, Iztok Gregori via ceph-users wrote: > Hi to all! > > First a little of background. We are in the process of a massive upgrade > from Octopus to Quincy (during which there is also the upgrade > (reinstall) from CentOS 7/Rocky 8 to Rocky 9). > > During the first phase we removed from the cluster 2 monitor/managers > and reinstalled them with Rocky 9 and Ceph 17.2.9). The 3 remaining > monitors/manager were: > - 2 x Rocky 8 with 15.2.17 > - 1 x CentOS 7 with 15.2.16 > > Then we upgraded Ceph from 15.2.17 to 17.9.6 on the 2 nodes with Rocky 8 > and restarted the services. So to summarize we have: > - 2 x MON Rocky 8 with 17.2.6 > - 1 x MON CentOS 7 with 15.2.16 > > and > - 1 x MGR CentOS 7 with 15.2.16 > > The next step was add the new (reinstalled) nodes as MON in the cluster > we encounter the first problem. Once started the new ceph-mon daemon > entered the cluster but all the commands ("ceph -s" for example) > remained stuck. If we stopped the new ceph-mon daemon the commands > restart to work (the status was in warning because the new ceph-mon was > down). To regain the quorum we deleted the new ceph-mon. > > And here I think we made a mistake, we started the ceph-mgr daemons on > the 2x Rocky 8 nodes. So we had: > - 2 x MGR Rocky 8 with 17.2.6 (standby) > - 1 x MGR CentOS 7 with 15.2.16 (active) > > We wrongly assume that the MGR was the culprit and we fail the active > MGR. From then on the status of the active MGR is in starting and all > the PGS are unknown: > > cluster: > id: 5bed1bf6-123f-4597-b8b1-d77931228548 > health: HEALTH_WARN > Reduced data availability: 13825 pgs inactive > > services: > mon: 3 daemons, quorum n-3-c7,n-2-r8,n-1-r8 (age 91m) > mgr: n-1-r8 (active, starting, since 13m), standbys: n-2-r8, n-3-c7 > osd: 852 osds: 849 up (since 8d), 849 in (since 7w) > > data: > pools: 11 pools, 13825 pgs > objects: 0 objects, 0 B > usage: 0 B used, 0 B / 0 B avail > pgs: 100.000% pgs unknown > 13825 unknown > > From the logs we see > >> 2026-04-08T16:59:26.244+0200 7fe0596a1700 0 ms_deliver_dispatch: >> unhandled message 0x5583db0d0c60 mon_map magic: 0 v1 from mon.2 >> v2:172.19.240.35:3300/0 >> 2026-04-08T16:59:26.807+0200 7fe0596a1700 1 mgr handle_mgr_map >> Activating! >> 2026-04-08T16:59:26.808+0200 7fe0596a1700 1 mgr handle_mgr_map I am >> now activating >> 2026-04-08T16:59:34.923+0200 7fdfd7e70700 1 mgr.server send_report >> Not sending PG status to monitor yet, waiting for OSDs >> 2026-04-08T16:59:36.924+0200 7fdfd7e70700 1 mgr.server send_report >> Not sending PG status to monitor yet, waiting for OSDs >> 2026-04-08T16:59:38.924+0200 7fdfd7e70700 1 mgr.server send_report >> Not sending PG status to monitor yet, waiting for OSDs >> 2026-04-08T16:59:40.924+0200 7fdfd7e70700 1 mgr.server send_report >> Not sending PG status to monitor yet, waiting for OSDs >> 2026-04-08T16:59:42.925+0200 7fdfd7e70700 1 mgr.server send_report >> Not sending PG status to monitor yet, waiting for OSDs >> 2026-04-08T16:59:44.925+0200 7fdfd7e70700 1 mgr.server send_report >> Not sending PG status to monitor yet, waiting for OSDs >> 2026-04-08T16:59:46.925+0200 7fdfd7e70700 1 mgr.server send_report >> Not sending PG status to monitor yet, waiting for OSDs >> 2026-04-08T16:59:48.926+0200 7fdfd7e70700 1 mgr.server send_report >> Not sending PG status to monitor yet, waiting for OSDs >> 2026-04-08T16:59:50.925+0200 7fdfd7e70700 1 mgr.server send_report >> Not sending PG status to monitor yet, waiting for OSDs >> 2026-04-08T16:59:52.926+0200 7fdfd7e70700 1 mgr.server send_report >> Giving up on OSDs that haven't reported yet, sending potentially >> incomplete PG state to mon > > The mgr dump looks like this: > >> { >> "epoch": 373, >> "active_gid": 16955846717, >> "active_name": "n-1-r8", >> "active_addrs": { >> "addrvec": [] >> }, >> "active_addr": "(unrecognized address family 0)/0", >> "active_change": "2026-04-08T16:59:26.282185+0200", >> "active_mgr_features": 4540138320759226367, >> "available": false, >> "standbys": [ >> { >> "gid": 16955784229, >> "name": "n-2-r8", >> "mgr_features": 4540138320759226367, >> "available_modules": [ > > We think that "unrecognized address family" could be the problem, > because we have a cluster/public network which is on a different > interface and could be that is not recognized/binded on mgr startup. But > we might be wrong. > > Somebody can give us some help? > > Thanks in advance > Iztok Gregori > > >

1 0

Re: v20.2.2 merge window is now open!
by Alfonso Escribano Merino 09 Apr '26

09 Apr '26

1 0