Beginner's installation questions about network
Hello, I want to install Ceph Octopus on Ubuntu 20.04. The nodes for have 2 network interfaces: 192.168.1.0/24 for the cluster network, and a 10.10.0.0/16 is the public network. When I bootstrap with cephadm, which Network do I use? That means, do i use cephadm bootstrap --mon-ip 192.168.1.1 or do I have to use the other network? When adding the other host with ceph orch, which network I have to use: ceph orch host add ceph03i 192.168.20.2 … (or the 10.10 network) Thanks, Erich
On 2020-11-13 21:19, E Taka wrote:
Hello,
I want to install Ceph Octopus on Ubuntu 20.04. The nodes for have 2 network interfaces: 192.168.1.0/24 for the cluster network, and a 10.10.0.0/16 is the public network. When I bootstrap with cephadm, which Network do I use? That means, do i use cephadm bootstrap --mon-ip 192.168.1.1 or do I have to use the other network?
When adding the other host with ceph orch, which network I have to use: ceph orch host add ceph03i 192.168.20.2 … (or the 10.10 network)
I found the following info in the documentation: "You need to know which IP address to use for the cluster’s first monitor daemon. This is normally just the IP for the first host. If there are multiple networks and interfaces, be sure to choose one that will be accessible by any host accessing the Ceph cluster." So that would be the public network in your case. See [1]. Just curious: why do you want to use separate networks? You might as well use bonded interfaces on the public network (i.e. LACP) and have more redundancy there. I figure that you might even make more effective use of the bandwidth as well. Gr. Stefan [1]: https://docs.ceph.com/en/latest/cephadm/install/#bootstrap-a-new-cluster
Hi Stefan, the cluster network has its own switch and is faster than the public network. Thanks for pointing me to the documentation. I must have overlooked this sentence. But let me ask another question: do the OSD use the cluster network "magically"? I did not find this in the docs, but that may be my fault… Am Fr., 13. Nov. 2020 um 21:40 Uhr schrieb Stefan Kooman <stefan@bit.nl>:
On 2020-11-13 21:19, E Taka wrote:
Hello,
I want to install Ceph Octopus on Ubuntu 20.04. The nodes for have 2 network interfaces: 192.168.1.0/24 for the cluster network, and a 10.10.0.0/16 is the public network. When I bootstrap with cephadm, which Network do I use? That means, do i use cephadm bootstrap --mon-ip 192.168.1.1 or do I have to use the other network?
When adding the other host with ceph orch, which network I have to use: ceph orch host add ceph03i 192.168.20.2 … (or the 10.10 network)
I found the following info in the documentation:
"You need to know which IP address to use for the cluster’s first monitor daemon. This is normally just the IP for the first host. If there are multiple networks and interfaces, be sure to choose one that will be accessible by any host accessing the Ceph cluster."
So that would be the public network in your case. See [1].
Just curious: why do you want to use separate networks? You might as well use bonded interfaces on the public network (i.e. LACP) and have more redundancy there. I figure that you might even make more effective use of the bandwidth as well.
Gr. Stefan
[1]: https://docs.ceph.com/en/latest/cephadm/install/#bootstrap-a-new-cluster _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Den fre 13 nov. 2020 kl 21:50 skrev E Taka <0etaka0@gmail.com>:
Hi Stefan, the cluster network has its own switch and is faster than the public network. Thanks for pointing me to the documentation. I must have overlooked this sentence.
But let me ask another question: do the OSD use the cluster network "magically"? I did not find this in the docs, but that may be my fault…
Yes, for OSD<->OSD traffic, all the rest (OSD->MON, clients to OSD, rgw/mds -> OSD) go via the public interface. -- May the most significant bit of your life be positive.
The nodes for have 2 network interfaces: 192.168.1.0/24 for the cluster network, and a 10.10.0.0/16 is the public network. When I bootstrap with cephadm, which Network do I use?
Since clients will need to connect to the monitors, you’ll want to use the "public" network space for the setup. Then you’ll need to specify the cluster network via a configuration modification, which will tell the OSDs to use that network to talk to one another. ~ Sean On Nov 13, 2020, 2:21 PM -0600, E Taka <0etaka0@gmail.com>, wrote:
Hello,
I want to install Ceph Octopus on Ubuntu 20.04. The nodes for have 2 network interfaces: 192.168.1.0/24 for the cluster network, and a 10.10.0.0/16 is the public network. When I bootstrap with cephadm, which Network do I use? That means, do i use cephadm bootstrap --mon-ip 192.168.1.1 or do I have to use the other network?
When adding the other host with ceph orch, which network I have to use: ceph orch host add ceph03i 192.168.20.2 … (or the 10.10 network)
Thanks, Erich _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
I just recently installed my first ceph cluster using cephadm and used the following config file for the bootstrap: |cat /root/ceph.conf| |[global]| |||public| |network = ||192.168||.||30.0||/||24| |||cluster network = ||192.168||.||41.0||/||24| | | Then: |cephadm bootstrap -c /root/ceph.conf --mon-ip ||192.168||.||30.11| | | |>> 192.168.30.11 is my bootstrap node << | | | |I'm sure you can also change this on a running cluster, but I have no clue how. | || | | | | Am 15.11.2020 um 15:46 schrieb Sean Johnson:
The nodes for have 2 network interfaces: 192.168.1.0/24 for the cluster network, and a 10.10.0.0/16 is the public network. When I bootstrap with cephadm, which Network do I use? Since clients will need to connect to the monitors, you’ll want to use the "public" network space for the setup. Then you’ll need to specify the cluster network via a configuration modification, which will tell the OSDs to use that network to talk to one another.
~ Sean On Nov 13, 2020, 2:21 PM -0600, E Taka <0etaka0@gmail.com>, wrote:
Hello,
I want to install Ceph Octopus on Ubuntu 20.04. The nodes for have 2 network interfaces: 192.168.1.0/24 for the cluster network, and a 10.10.0.0/16 is the public network. When I bootstrap with cephadm, which Network do I use? That means, do i use cephadm bootstrap --mon-ip 192.168.1.1 or do I have to use the other network?
When adding the other host with ceph orch, which network I have to use: ceph orch host add ceph03i 192.168.20.2 … (or the 10.10 network)
Thanks, Erich _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (5)
-
E Taka
-
Janne Johansson
-
Oliver Weinmann
-
Sean Johnson
-
Stefan Kooman