[ceph-users] Re: NFS export of 2 disjoint sub-dir mounts

To answer my own question, assigning an explicit fsid to only one of the exports seems to overwrite the default for all file systems with the same default fsid. Hence, the export definitions /mnt/S1 -options NET /mnt/S2 -options IP and /mnt/S1 -options NET /mnt/S2 -options,fsid=100 IP are equivalent. My expectation was that /mnt/S1 would retain the default fsid. Using explicit fsids for each export as in: /mnt/S1 -options,fsid=100 NET /mnt/S2 -options,fsid=101 IP leads to the desired result. Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Frank Schilder <frans(a)dtu.dk> Sent: Monday, May 15, 2023 6:51 PM To: ceph-users(a)ceph.io Subject: [ceph-users] NFS export of 2 disjoint sub-dir mounts Hi all, I have a problem with exporting 2 different sub-folder ceph-fs kernel mounts via nfsd to the same IP address. The top-level structure on the ceph fs is something like /A/S1 and /A/S2. On a file server I mount /A/S1 and /A/S2 as two different file systems under /mnt/S1 and /mnt/S2 using the ceph fs kernel client. Then, these 2 mounts are exported with lines like these in /etc/exports: /mnt/S1 -options NET /mnt/S2 -options IP IP is an element of NET, meaning that the host at IP should be the only host being able to access /mnt/S1 and /mnt/S2. What we observe is that any attempt to mount the export /mnt/S1 on the host at IP results in /mnt/S2 being mounted instead. My first guess was that here we have a clash of fsids and the ceph fs is simply reporting the same fsid to nfsd and, hence, nfsd thinks both mountpoints contain the same. So I modified the second export line to /mnt/S2 -options,fsid=100 IP to no avail. The two folders are completely disjoint, neither symlinks nor hard-links between them. So it should be safe to export these as 2 different file systems. Exporting such constructs to non-overlapping networks/IPs works as expected - even when exporting subdirs of a dir (like exporting /A/B and /A/B/C from the same file server to strictly different IPs). It seems the same-IP config that breaks expectations. Am I missing here a magic -yes-i-really-know-what-i-am-doing hack? The file server is on AlmaLinux release 8.7 (Stone Smilodon) and all ceph packages match the ceph version octopus latest of our cluster. Thanks and best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 _______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io