Connecting A Client To 2 Different Ceph Clusters
Hi All, I haven't been able to find anything in the doco or on the web about this. I'm blessed with having available to me 2 different Ceph Clusters within our organisation (a "regular" Ceph Cluster and a Hyper-converged Proxmox Ceph Cluster). Both Ceph Clusters have a CephFS system on them (regular_cephfs and proxmox_cephfs). Obviously each Ceph Cluster has its own set of Manager, Monitor, and MDS Nodes, and each CephFS has its own user(s)/key(s). Also obviously, I've got the all the relevant key files install on my client in the /etc/ceph folder. How do I set up the ceph.conf file(s) on my clients so that I can use /etc/fstab to connect to each CephFS or each Ceph Cluster? My existing /etc/fstab file (extract of): ~~~ regular_cephfs_user@.regular_cephfs=/folder1 /mnt/folder1 ceph noatime,_netdev 0 0 proxmox_cephfs_user@.proxmox_cephfs=/folder2 /mnt/folder2 ceph noatime,_netdev 0 0 ~~~ Thanks in advance Dulux-Oz
Hi,
On 24 Aug 2024, at 10:57, duluxoz <duluxoz@gmail.com> wrote:
How do I set up the ceph.conf file(s) on my clients so that I can use /etc/fstab to connect to each CephFS or each Ceph Cluster?
You can try to setup your fstab like this: 10.0.0.1:3300,10.0.0.2:3300,10.0.0.3:3300:/folder1 /mnt ceph name=<client>,secret=<secret>,dirstat,ms_mode=prefer-crc,_netdev 0 0 And the same for the second client, change your mons addresses and cephx records accordingly k
Hi K, Thanks for getting back to me. So you wouldn't use multiple ceph.conf files? (Or a combined one, for that matter?) Dulux-Oz On 24/8/24 18:12, Konstantin Shalygin wrote:
Hi,
On 24 Aug 2024, at 10:57, duluxoz <duluxoz@gmail.com> wrote:
How do I set up the ceph.conf file(s) on my clients so that I can use /etc/fstab to connect to each CephFS or each Ceph Cluster?
You can try to setup your fstab like this:
10.0.0.1:3300,10.0.0.2:3300,10.0.0.3:3300:/folder1 /mnt ceph name=<client>,secret=<secret>,dirstat,ms_mode=prefer-crc,_netdev 0 0
And the same for the second client, change your mons addresses and cephx records accordingly
k
Hi, I don't think kernel read any files. Maybe only with some userland helper. In any case, it's up to you, but for the kernel ceph client to work, nothing but the kernel is needed, bindings in the form of files and user space utilities only complicate the system, like for me k Sent from my iPhone
On 24 Aug 2024, at 12:34, duluxoz <duluxoz@gmail.com> wrote:
So you wouldn't use multiple ceph.conf files? (Or a combined one, for that matter?)
Thanks Konstantin, that really helped a lot :-) For anyone else who stumbles across this issue here is the (extract) /etc/fstab file we use: ~~~ regular_cephfs_user@<Regular CephFS FSID>.regular_cephfs=/folder1 /mnt/folder1 ceph mon_addr=<Regular CephFS Monitor1 IP Address>/<Regular CephFS Monitor2 IP Address>/<Regular CephFS Monitor3 IP Address>,secretfile=/etc/ceph/regular_cephfs_user.secret,noatime,_netdev 0 0 proxmox_cephfs_user@<Proxmox CephFS FSID>.proxmox_cephfs=/folder2 /mnt/folder2 ceph mon_addr=<Proxmox CephFS Monitor1 IP Address>/<Proxmox CephFS Monitor2 IP Address>/<Proxmox CephFS Monitor3 IP Address>,secretfile=/etc/pve/priv/ceph/proxmox_cephfs_user.secret,noatime,_netdev 0 0 ~~~ Note that you can use the Clusters' Monitors' Hostnames instead of their IP Addresses - but you will need to have a good DNS system in place and/or have the relevant details in each client's /etc/hosts file. Cheers Dulux-Oz On 24/8/24 20:44, Konstantin Shalygin wrote:
Hi,
I don't think kernel read any files. Maybe only with some userland helper. In any case, it's up to you, but for the kernel ceph client to work, nothing but the kernel is needed, bindings in the form of files and user space utilities only complicate the system, like for me
k Sent from my iPhone
On 24 Aug 2024, at 12:34, duluxoz <duluxoz@gmail.com> wrote:
So you wouldn't use multiple ceph.conf files? (Or a combined one, for that matter?)
participants (2)
-
duluxoz
-
Konstantin Shalygin