Hello! Here is the snippet from cephadm.log: cephadm ['--image', 'quay.io/ceph/ceph@sha256:1228c3d05e45fbc068a8c33614e4409b6dac688bcc77369b06009b5830fa8d86', '--timeout', '895', '_orch', 'deploy', '--fsid', '9a7485f3-9530-11f0-8bb9-6c92cff2f5d0'] 2025-12-10 09:45:26,706 7e80250e7080 DEBUG Loaded deploy configuration: {'fsid': '9a7485f3-9530-11f0-8bb9-6c92cff2f5d0', 'name': 'nfs.rootfs.0.0.atlas.luxjmr', 'image': '', 'deploy_arguments': [], 'params': {}, 'meta': {'service_n ame': 'nfs.rootfs', 'ports': [], 'ip': None, 'deployed_by': ['quay.io/ceph/ceph@sha256:1228c3d05e45fbc068a8c33614e4409b6dac688bcc77369b06009b5830fa8d86'], 'rank': 0, 'rank_generation': 0, 'extra_container_args': None, 'extra_entry point_args': None}, 'config_blobs': {'pool': '.nfs', 'namespace': 'rootfs', 'userid': 'nfs.rootfs.0.0.atlas.luxjmr', 'extra_args': ['-N', 'NIV_EVENT'], 'files': {'ganesha.conf': '# This file is generated by cephadm.\nNFS_CORE_PARAM {\n Enable_NLM = false;\n Enable_RQUOTA = false;\n Protocols = 3, 4;\n mount_path_pseudo = true;\n Enable_UDP = false;\n NFS_Port = 2049;\n allow_set_io_flusher_fail = true;\n Monitoring_Port = 9587;\n}\n\nNFSv4 {\n Delegations = false;\n RecoveryBackend = "rados_cluster";\n Minor_Versions = 1, 2;\n Server_Scope = "9a7485f3-9530-11f0-8bb9-6c92cff2f5d0-rootfs";\n IdmapConf = "/etc/ganesha/idmap.conf";\n}\n\nRADOS_KV {\n UserId = "nfs.rootfs.0.0.atlas.luxjmr";\n nodeid = 0;\n pool = ".nfs";\n namespace = "rootfs";\n}\n\nRADOS_URLS {\n UserId = "nfs.rootfs.0.0.atlas.luxjmr";\n watch_url = "rados://.nfs/rootfs/conf-nfs.rootfs";\n}\n\nRGW {\n cluster = "ceph";\n name = "client.nfs.rootfs.0.0.atlas.luxjmr-rgw";\n}\n\n%url rados://.nfs/rootfs/conf-nfs.rootfs', 'idmap.conf': ''}, 'config': '# minimal ceph.conf for 9a7485f3-9530-11f0-8bb9-6c92cff2f5d0\n[global]\n\tfsid = 9a7485f3-9530-11f0-8bb9-6c92cff2f5d0\n\tmon_host = [v2:134.169.37.21:3300/0,v1:134.169.37.21:6789/0] [v2:134.169.37.17:3300/0,v1:134.169.37.17:6789/0] [v2:134.169.37.12:3300/0,v1:134.169.37.12:6789/0] [v2:134.169.37.15:3300/0,v1:134.169.37.15:6789/0] [v2:134.169.37.14:3300/0,v1:134.169.37.14:6789/0]\n', 'keyring': '[client.nfs.rootfs.0.0.atlas.luxjmr]\nkey = AQAlMzlpAZ3EMBAAXYArqFVcLYC0ND5DjamvWw==\n', 'rgw': {'cluster': 'ceph', 'user': 'nfs.rootfs.0.0.atlas.luxjmr-rgw', 'keyring': '[client.nfs.rootfs.0.0.atlas.luxjmr-rgw]\nkey = AQAmMzlpyOP1BxAARjK7MCMV/u1CHCuCnCeqXg==\n'}}} 2025-12-10 09:45:26,706 7e80250e7080 DEBUG Determined image: 'quay.io/ceph/ceph@sha256:1228c3d05e45fbc068a8c33614e4409b6dac688bcc77369b06009b5830fa8d86' 2025-12-10 09:45:26,735 7e80250e7080 INFO Non-zero exit code 1 from /usr/bin/docker container inspect --format {{.State.Status}} ceph-9a7485f3-9530-11f0-8bb9-6c92cff2f5d0-nfs-rootfs-0-0-atlas-luxjmr 2025-12-10 09:45:26,735 7e80250e7080 INFO /usr/bin/docker: stdout 2025-12-10 09:45:26,735 7e80250e7080 INFO /usr/bin/docker: stderr Error response from daemon: No such container: ceph-9a7485f3-9530-11f0-8bb9-6c92cff2f5d0-nfs-rootfs-0-0-atlas-luxjmr 2025-12-10 09:45:26,748 7e80250e7080 INFO Non-zero exit code 1 from /usr/bin/docker container inspect --format {{.State.Status}} ceph-9a7485f3-9530-11f0-8bb9-6c92cff2f5d0-nfs.rootfs.0.0.atlas.luxjmr 2025-12-10 09:45:26,748 7e80250e7080 INFO /usr/bin/docker: stdout 2025-12-10 09:45:26,748 7e80250e7080 INFO /usr/bin/docker: stderr Error response from daemon: No such container: ceph-9a7485f3-9530-11f0-8bb9-6c92cff2f5d0-nfs.rootfs.0.0.atlas.luxjmr 2025-12-10 09:45:26,748 7e80250e7080 INFO Deploy daemon nfs.rootfs.0.0.atlas.luxjmr ... 2025-12-10 09:45:26,925 7e80250e7080 INFO Verifying port 0.0.0.0:2049 ... 2025-12-10 09:45:26,927 7e80250e7080 INFO Creating ganesha config... 2025-12-10 09:45:26,927 7e80250e7080 INFO Write file: /var/lib/ceph/9a7485f3-9530-11f0-8bb9-6c92cff2f5d0/nfs.rootfs.0.0.atlas.luxjmr/etc/ganesha/ganesha.conf 2025-12-10 09:45:26,927 7e80250e7080 INFO Write file: /var/lib/ceph/9a7485f3-9530-11f0-8bb9-6c92cff2f5d0/nfs.rootfs.0.0.atlas.luxjmr/etc/ganesha/idmap.conf 2025-12-10 09:45:27,242 7e80250e7080 DEBUG Non-zero exit code 1 from systemctl reset-failed ceph-9a7485f3-9530-11f0-8bb9-6c92cff2f5d0@nfs.rootfs.0.0.atlas.luxjmr 2025-12-10 09:45:27,242 7e80250e7080 DEBUG systemctl: stderr Failed to reset failed state of unit ceph-9a7485f3-9530-11f0-8bb9-6c92cff2f5d0@nfs.rootfs.0.0.atlas.luxjmr.service: Unit ceph-9a7485f3-9530-11f0-8bb9-6c92cff2f5d0@nfs.rootfs.0.0.atlas.luxjmr.service not loaded. 2025-12-10 09:45:27,541 7e80250e7080 DEBUG systemctl: stderr Created symlink /etc/systemd/system/ceph-9a7485f3-9530-11f0-8bb9-6c92cff2f5d0.target.wants/ceph-9a7485f3-9530-11f0-8bb9-6c92cff2f5d0@nfs.rootfs.0.0.atlas.luxjmr.service → /etc/systemd/system/ceph-9a7485f3-9530-11f0-8bb9-6c92cff2f5d0@.service. 2025-12-10 09:45:27,572 7e80250e7080 DEBUG firewalld does not appear to be present 2025-12-10 09:45:27,573 7e80250e7080 DEBUG Not possible to enable service <nfs>. firewalld.service is not available 2025-12-10 09:45:27,573 7e80250e7080 DEBUG firewalld does not appear to be present 2025-12-10 09:45:27,573 7e80250e7080 DEBUG Not possible to open ports <[2049]>. firewalld.service is not available With best regards Carsten Götze ------------------------------------------------------------------ Carsten Goetze Computer Graphics tel: +49 531 391-2109 TU Braunschweig fax: +49 531 391-2103 Muehlenpfordtstr. 23 eMail: c.goetze@cg.cs.tu-bs.de D-38106 Braunschweig http://www.cg.cs.tu-bs.de/people/goetze

Am 03.01.2026 um 18:52 schrieb Salatiel Filho <salatiel.filho@gmail.com>:

Does nfs deploy in ceph 20.2.0 requires firewalld? Would you mind sharing some logs that complain about that ?

Atenciosamente/Kind regards, Salatiel

On Tue, Dec 16, 2025 at 9:54 AM Carsten Götze via ceph-users <ceph-users@ceph.io> wrote:

...

Hi!

I'm running a ceph cluster using cephadm and recently upgraded from squid to tentacle 20.2.0. Until recently everything worked fine until I started the nfs module. The nfs daemons were reported to be running but after some 10 minutes or so all of them were reported to be dead except one. nfs service on port 2049 was never provided on any of the nodes even while the daemons were supposed to be running. As i found out later, the nfs daemons were never started at all, because the setup process required a systemd-firewalld to be installed on the system which of course wasn't. After some headaches with the newly installed firewalld I decided to roll back, delete the firewalld and postpone the nfs deployment. I then tried to stop the nfs daemons with 'ceph orch daemon stop', which did nothing, even after waiting some 10 minutes. I had to reissue the command several times to make the reportedly dead nfs daemons vanish from the 'ceph orch ps' list. The one daemon that was reported to be still running however would only die after 'ceph orch daemon stop --force' and was in an 'error' state thereafter and could not be removed from the 'ceph orch ps' list by no means. So I decided to delete the managing nfs service from the 'ceph orch ls' list, in hope that it would also tear down the remaining nfs daemon. This obviously was a bad idea since, the service is now in the state of deleting. However it cannot be deleted, because there is still the one daemon in error state, which cannot be deleted because it was never running at all. As a last measure I forcefully removed the docker container on the node with the cephadm command, but even though there are no traces left of that nfs daemon, it is still listed when running 'ceph orch ps'. I also noticed that the 'ceph orch device ls' is out of sync with reality and 'ceph orch ps' is still listing osds that I've already shutdown and deleted. I therefore suspect, that the orchestrator has stopped collecting state information from the nodes. Is there a way to force the orchestrator to sync its state information with the nodes? Where do I find meaningful logs for the orchestrator?

With best regards, Carsten Götze _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io