On 13-10-2023 14:00, John Mulligan wrote:
On Friday, October 13, 2023 6:11:18 AM EDT Torkil Svensgaard wrote:
Hi
We have kerberos working with bare metal kernel NFS exporting RBDs. I can see in the ceph documentation[1] that nfs-ganesha should work with kerberos but I'm having little luck getting it to work.
Could you clarify how you are deploying the ganesha instances? I think you may be asking about cephadm deployed containers but it is not clear to me.
There are a couple of viable ways to deploy nfs-ganesha today: manually/"bare- metal", cephadm, and rook.
Of course, my bad. Ceph version 17.2.6-100.el9cp, nfs-ganesha deployed with cephadm as pr the documentation I linked: " ceph nfs export create cephfs --cluster-id cephfs_noHA --pseudo-path /testkrb5p_noHA --fsname cephfs_ssd --path=/test --sectype krb5p --sectype sys " " [ceph: root@lazy /]# ceph nfs export info cephfs_noHA /testkrb5p_noHA { "export_id": 1, "path": "/test", "cluster_id": "cephfs_noHA", "pseudo": "/testkrb5p_noHA", "access_type": "RW", "squash": "none", "security_label": true, "protocols": [ 4 ], "transports": [ "TCP" ], "fsal": { "name": "CEPH", "user_id": "nfs.cephfs_noHA.1", "fs_name": "cephfs_ssd" }, "clients": [], "sectype": [ "krb5p", "sys" ] } " Works with sys, not with krb5p. Thanks. Mvh. Torkil
This bit from the container log seems to suggest that some plumbing is missing?
" 13/10/2023 08:09:12 : epoch 6528fb25 : ceph-flash1 : ganesha.nfsd-2[main] nfs_rpc_cb_init_ccache :NFS STARTUP :EVENT
:Callback creds directory (/var/run/ganesha) already exists
13/10/2023 08:09:12 : epoch 6528fb25 : ceph-flash1 : ganesha.nfsd-2[main] find_keytab_entry :NFS CB :WARN :Configuration file does not specify default realm while getting default realm name 13/10/2023 08:09:12 : epoch 6528fb25 : ceph-flash1 : ganesha.nfsd-2[main] gssd_refresh_krb5_machine_credential :NFS CB :CRIT
:ERROR: gssd_refresh_krb5_machine_credential: no usable keytab entry
found in keytab /etc/krb5.keytab for connection with host localhost 13/10/2023 08:09:12 : epoch 6528fb25 : ceph-flash1 : ganesha.nfsd-2[main] nfs_rpc_cb_init_ccache :NFS STARTUP :WARN
:gssd_refresh_krb5_machine_credential failed (-1765328160:0)
"
Thoughts?
Mvh.
Torkil
[1] https://docs.ceph.com/en/quincy/mgr/nfs/#create-cephfs-export
Based on your mention of "container log" above I assume this is on either cephadm or rook. The rook team has been actively working on adding kerberized nfs support and we added the `sectype` option for that work. Currently, cephadm doesn't have support for kerberos because it lacks the server side components needed to connect to krb5/ldap.
I would like to see this support eventually come to cephadm but it's not there today IMO.
A manual deployment of nfs-ganesha ought to also be able to make use of this option. Ultimately, this generates ganesha config blocks and is mostly agnostic of the cluster/deployment method, but I have not tried it out myself.
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Torkil Svensgaard Systems Administrator Danish Research Centre for Magnetic Resonance DRCMR, Section 714 Copenhagen University Hospital Amager and Hvidovre Kettegaard Allé 30, 2650 Hvidovre, Denmark