On Wed, Jun 17, 2020 at 10:09 AM Jeff Layton <jlayton@redhat.com> wrote:
(I sent this to a smaller list of folks yesterday, but I think it
probably warrants wider discussion).

Recently Varsha added the necessary infrastructure to bring up
nfs-ganesha via vstart.sh. The current implementation requires that
ganesha already be installed on the box (usually via distro packaging),
but that poses a bit of a problem.

A distro ganesha package will have likely been built vs. a completely
different version of libcephfs and librados. Even if you build right off
of ceph master branch, you won't get the benefit of any recent client
bugfixes when you want to test ganesha. You'd have to build new ganesha
packages, install them, etc.

I think we ought to consider making a nfs-ganesha build an optional part
of a ceph build (maybe enable it with cmake -DNFS_GANESHA=ON or

It doesn't take very long to build it (typically only a minute or two on
my box), and we could disable the parts that ceph doesn't care about
(other FSALs primarily). We could also have vstart just error out when
you run it with NFS=X on a build that didn't have ganesha enabled.

OTOH, the potential downside here is that it'll likely add other build-
time dependencies, and would require some extra cmake or scripting
wizardry. Nothing insurmountable, but it might represent a maintenance
burden going forward, particularly for something that's basically only
going to be used for vstart.

I'm also not sure how we'd do this in practice. I don't think you can do
optional submodules, so we might have to look at other methods of
pulling in the ganesha tree, or just live with it as a submodule that
only gets used when ganesha is enabled.

libcephfs and librgw have semi-stable APIs by now. (And IIRC libcephfs has versioned symbols.)  So there's some risk that if a versioned symbol changes that an old build of ganesha would continue to use an old API. But one of the whole points of using versioned symbols — IMO — is to make people think really hard about changing the API.

How often does it happen in reality that someone runs a ganesha built against mimic/13 or nautilus/14 with octopus/15? I can't imagine it really happens that often. And if it does, I think there's a different problem; one that should be fixed.

I'm not really a git submodule and cmake expert, so I don't really know how much work it would be to add ganesha and the ntirpc submodule into the ceph build. And build them with all the right options for whatever the particular environment is. I'm sure it's doable though.

It doesn't really affect ganesha. Ganesha packages will still get built independent of whatever ceph is doing. I'd kinda hate to see there be YAPTDGF (yet another place to download ganesha from.)

Personally I'd rather try to figure out proper solutions to the concern you raise