I deployed the nfs with ceph version 17.2.7 and then upgraded to 18.2.1 successfully, the ingress service is still present. Can you tell if it was there while you were on quincy? To fix it I would just apply the nfs.yaml again and see if the ingress service is deployed. To know what happened during (or after) the upgrade you'd probably have to look through the mgr logs... Zitat von Torkil Svensgaard <torkil(a)drcmr.dk>dk>:

Hi, you're probably right, when using your example command with multiple hosts it automatically sets "count:1" (don't mind the hostnames, it's an upgraded cluster currently running 18.2.1): # ceph orch ls nfs NAME PORTS RUNNING REFRESHED AGE PLACEMENT nfs.nfs-reef ?:2049 1/1 114s ago 3m nautilus;nautilus2;nautilus3;count:1 # ceph orch ls ingress NAME PORTS RUNNING REFRESHED AGE PLACEMENT ingress.nfs.nfs-reef 192.168.168.114:9049 1/1 2m ago 4m nautilus;nautilus2;nautilus3;count:1 So it's not really clear what happened to your ingress service. :-) But at least it works now, so that's good. Zitat von Torkil Svensgaard <torkil(a)drcmr.dk>dk>: > On 31/01/2024 09:36, Eugen Block wrote: >> Hi, >> >> if I understand this correctly, with the "keepalive-only" option >> only one ganesha instance is supposed to be deployed: >> >>> If a user additionally supplies --ingress-mode keepalive-only a >>> partial ingress service will be deployed that still provides a >>> virtual IP, but has nfs directly binding to that virtual IP and >>> leaves out any sort of load balancing or traffic redirection. This >>> setup will restrict users to deploying only 1 nfs daemon as >>> multiple cannot bind to the same port on the virtual IP. >> >> Maybe that's why it disappeared as you have 3 hosts in the >> placement parameter? Is the ingress service still present in 'ceph >> orch ls'? > > > The absence of the ingress service is puzzling me, as it worked just > fine prior to the upgrade and the upgrade shouldn't have touched the > service spec in any way? > > Mvh. > > Torkil > > [1] > https://docs.ceph.com/en/latest/cephadm/services/nfs/#nfs-with-virtual-ip-b… > > >> Regards, >> Eugen >> >> Zitat von Torkil Svensgaard <torkil(a)drcmr.dk>dk>: >> >>> Hi >>> >>> Last week we created an NFS service like this: >>> >>> " >>> ceph nfs cluster create jumbo >>> "ceph-flash1,ceph-flash2,ceph-flash3" --ingress --virtual_ip >>> 172.21.15.74/22 --ingress-mode keepalive-only >>> " >>> >>> Worked like a charm. Yesterday we upgraded from 17.2.7 to 18.20.0 >>> and the NFS virtual IP seems to have gone missing in the process: >>> >>> " >>> # ceph nfs cluster info jumbo >>> { >>>   "jumbo": { >>>     "backend": [ >>>       { >>>         "hostname": "ceph-flash1", >>>         "ip": "172.21.15.148", >>>         "port": 2049 >>>       } >>>     ], >>>     "virtual_ip": null >>>   } >>> } >>> " >>> >>> Service spec: >>> >>> " >>> service_type: nfs >>> service_id: jumbo >>> service_name: nfs.jumbo >>> placement: >>>   count: 1 >>>   hosts: >>>   - ceph-flash1 >>>   - ceph-flash2 >>>   - ceph-flash3 >>> spec: >>>   port: 2049 >>>   virtual_ip: 172.21.15.74 >>> " >>> >>> I've tried restarting the nfs.jumbo service which didn't help. Suggestions? >>> >>> Mvh. >>> >>> Torkil >>> >>> -- >>> Torkil Svensgaard >>> Sysadmin >>> MR-Forskningssektionen, afs. 714 >>> DRCMR, Danish Research Centre for Magnetic Resonance >>> Hvidovre Hospital >>> Kettegård Allé 30 >>> DK-2650 Hvidovre >>> Denmark >>> Tel: +45 386 22828 >>> E-mail: torkil(a)drcmr.dk >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users(a)ceph.io >>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >> >> >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io > > -- > Torkil Svensgaard > Sysadmin > MR-Forskningssektionen, afs. 714 > DRCMR, Danish Research Centre for Magnetic Resonance > Hvidovre Hospital > Kettegård Allé 30 > DK-2650 Hvidovre > Denmark > Tel: +45 386 22828 > E-mail: torkil(a)drcmr.dk > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io

I believe so, I just deployed it with the same command as you did, but on a single-node cluster (hence only one host): ceph:~ # ceph orch ls NAME PORTS RUNNING REFRESHED AGE PLACEMENT crash 1/1 3m ago 44m * ingress.nfs.test 192.168.124.187:9049 1/1 3m ago 36m ceph;count:1 mds.cephfs 2/2 3m ago 31m count:2 mgr 2/2 3m ago 44m count:2 mon 1/5 3m ago 44m count:5 nfs.test ?:2049 1/1 3m ago 36m ceph;count:1 osd.osd-hdd-ssd 3 3m ago 41m ceph The ingress.nfs.test has the virtual IP a provided. Zitat von Torkil Svensgaard <torkil(a)drcmr.dk>dk>: