I don't know much about the vfs plugin (nor cephfs for that matter) but I would suggest enabling client debug logging on the machine so you can see what the libcephfs code is doing since that's likely where the ENOENT is coming from. https://docs.ceph.com/en/latest/rados/troubleshooting/log-and-debug/ https://docs.ceph.com/en/latest/cephfs/client-config-ref/ On Fri, Nov 13, 2020 at 3:39 AM Frank Schilder <frans@dtu.dk> wrote:
You might need to give read permissions to the ceph config and key file for the user that runs the SAMBA service (samba?). Either add the SAMBA user to the group ceph, or change the group of the file.
The statement "/" file not found could just be an obfuscating message on an actual security/permission issue.
Other than that I don't really know what to look for. As I said, I gave up as well. Ceph kernel client does a good job for us with an ordinary SAMBA share defined on it.
Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14
________________________________________ From: Matt Larson <larsonmattr@gmail.com> Sent: 12 November 2020 18:18:32 To: Frank Schilder Cc: ceph-users Subject: Re: [ceph-users] Unable to clarify error using vfs_ceph (Samba gateway for CephFS)
Thank you Frank,
That was a good suggestion to make sure the mount wasn't the issue. I tried changing the `client.samba.upload` to have read access directly to '/' rather than '/upload' and to also change smb.conf to directly use 'path = /'. Still getting the same issue (log level 10 content below).
It appears that it is correctly reading `/etc/ceph/ceph.conf`. It does appear to be the ceph_mount where the failure occurs.
It would be great to have vfs_ceph working, but if I cannot I'll try to find other approaches.
[2020/11/12 10:47:39.360943, 10, pid=2723021, effective(0, 0), real(0, 0), class=vfs] ../../source3/smbd/vfs.c:65(vfs_find_backend_entry)
vfs_find_backend_entry called for ceph Successfully loaded vfs module [ceph] with the new modules system [2020/11/12 10:47:39.360966, 10, pid=2723021, effective(0, 0), real(0, 0), class=vfs] ../../source3/modules/vfs_ceph.c:103(cephwrap_connect) cephwrap_connect: [CEPH] calling: ceph_create [2020/11/12 10:47:39.365668, 10, pid=2723021, effective(0, 0), real(0, 0), class=vfs] ../../source3/modules/vfs_ceph.c:110(cephwrap_connect) cephwrap_connect: [CEPH] calling: ceph_conf_read_file with /etc/ceph/ceph.conf [2020/11/12 10:47:39.368842, 10, pid=2723021, effective(0, 0), real(0, 0), class=vfs] ../../source3/modules/vfs_ceph.c:116(cephwrap_connect) cephwrap_connect: [CEPH] calling: ceph_conf_get [2020/11/12 10:47:39.368895, 10, pid=2723021, effective(0, 0), real(0, 0), class=vfs] ../../source3/modules/vfs_ceph.c:133(cephwrap_connect) cephwrap_connect: [CEPH] calling: ceph_mount [2020/11/12 10:47:39.373319, 10, pid=2723021, effective(0, 0), real(0, 0), class=vfs] ../../source3/modules/vfs_ceph.c:160(cephwrap_connect) cephwrap_connect: [CEPH] Error return: No such file or directory [2020/11/12 10:47:39.373357, 1, pid=2723021, effective(0, 0), real(0, 0)] ../../source3/smbd/service.c:668(make_connection_snum) make_connection_snum: SMB_VFS_CONNECT for service 'cryofs_upload' at '/' failed: No such file or directory
On Thu, Nov 12, 2020 at 2:29 AM Frank Schilder <frans@dtu.dk> wrote:
You might face the same issue I had. vfs_ceph wants to have a key for the root of the cephfs, it is cutrently not possible to restrict access to a sub-directory mount. For this reason, I decided to go for a re-export of a kernel client mount.
I consider this a serious security issue in vfs_ceph and will not use it until it is possible to do sub-directory mounts.
I don't think its difficult to patch the vfs_ceph source code, if you need to use vfs_ceph and cannot afford to give access to "/" of the cephfs.
Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14
________________________________________ From: Matt Larson <larsonmattr@gmail.com> Sent: 12 November 2020 00:40:21 To: ceph-users Subject: [ceph-users] Unable to clarify error using vfs_ceph (Samba gateway for CephFS)
I am getting an error in the log.smbd from the Samba gateway that I don’t understand and looking for help from anyone who has gotten the vfs_ceph working.
Background:
I am trying to get a Samba gateway with CephFS working with the vfs_ceph module. I observed that the default Samba package on CentOS 7.7 did not come with the ceph.so vfs_ceph module, so I tried to compile a working Samba version with vfs_ceph.
Newer Samba versions have a requirement for GnuTLS >= 3.4.7, which is not an available package on CentOS 7.7 without a custom repository. I opted to build an earlier version of Samba.
On CentOS 7.7, I built Samba 4.11.16 with
[global] security = user map to guest = Bad User username map = /etc/samba/smbusers log level = 4 load printers = no printing = bsd printcap name = /dev/null disable spoolss = yes
[cryofs_upload] public = yes read only = yes guest ok = yes vfs objects = ceph path = /upload kernel share modes = no ceph:user_id = samba.upload ceph:config_file = /etc/ceph/ceph.conf
I have a file at /etc/ceph/ceph.conf including: fsid = redacted mon_host = redacted auth_cluster_required = cephx auth_service_required = cephx auth_client_required = cephx
I have an /etc/ceph/client.samba.upload.keyring /w key for the user `samba.upload`
However, connecting fails:
smbclient \\\\localhost\\cryofs_upload -U guest Enter guest's password: tree connect failed: NT_STATUS_UNSUCCESSFUL
The log.smbd gives these errors:
Initialising custom vfs hooks from [ceph] [2020/11/11 17:24:37.388460, 3] ../../lib/util/modules.c:167(load_module_absolute_path) load_module_absolute_path: Module '/usr/local/samba/lib/vfs/ceph.so' loaded [2020/11/11 17:24:37.402026, 1] ../../source3/smbd/service.c:668(make_connection_snum) make_connection_snum: SMB_VFS_CONNECT for service 'cryofs_upload' at '/upload' failed: No such file or directory
There is an /upload directory for which the samba.upload user has read access to in the CephFS.
What does this error mean: ‘no such file or directory’ ? Is it that vfs_ceph isn’t finding `/upload` or is some other file depended by vfs_ceph not been found? I have also tried to specify a local path rather than a CephFS path and will get the same error.
Is there any good guide that describes not just the Samba smb.conf, but also what should be in /etc/ceph/ceph.conf, and how to provide the key for the ceph:user_id ? I am really struggling to find good first-hand documentation for this.
Thanks, Matt
-- Matt Larson, PhD Madison, WI 53705 U.S.A. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Matt Larson, PhD Madison, WI 53705 U.S.A. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Cheers, Brad