Hi all,
I have a problem with exporting 2 different sub-folder ceph-fs kernel mounts via nfsd to
the same IP address. The top-level structure on the ceph fs is something like /A/S1 and
/A/S2. On a file server I mount /A/S1 and /A/S2 as two different file systems under
/mnt/S1 and /mnt/S2 using the ceph fs kernel client. Then, these 2 mounts are exported
with lines like these in /etc/exports:
/mnt/S1 -options NET
/mnt/S2 -options IP
IP is an element of NET, meaning that the host at IP should be the only host being able to
access /mnt/S1 and /mnt/S2. What we observe is that any attempt to mount the export
/mnt/S1 on the host at IP results in /mnt/S2 being mounted instead.
My first guess was that here we have a clash of fsids and the ceph fs is simply reporting
the same fsid to nfsd and, hence, nfsd thinks both mountpoints contain the same. So I
modified the second export line to
/mnt/S2 -options,fsid=100 IP
to no avail. The two folders are completely disjoint, neither symlinks nor hard-links
between them. So it should be safe to export these as 2 different file systems.
Exporting such constructs to non-overlapping networks/IPs works as expected - even when
exporting subdirs of a dir (like exporting /A/B and /A/B/C from the same file server to
strictly different IPs). It seems the same-IP config that breaks expectations.
Am I missing here a magic -yes-i-really-know-what-i-am-doing hack? The file server is on
AlmaLinux release 8.7 (Stone Smilodon) and all ceph packages match the ceph version
octopus latest of our cluster.
Thanks and best regards,
=================
Frank Schilder
AIT Risø Campus
Bygning 109, rum S14