Hi everyone. Status : Installing a ceph cluster Version : 17.2.7 Quincy OS : Debian 11. Each of my server got two ip address. One public and one private. When I'm trying to deploy my cluster with on a server server1 (the hostname) with cephadm bootstrap --mon-id hostname --mon-ip IP_PRIVATE --cluster-network PRIVATE_SUB I end up with private network for ceph config get mon public_network So I try to change it with ceph config set mon public_network PUBLIC_SUB still with lsof -i |grep -i listen I got ceph-mgr 31427 ceph 49u IPv4 119937 0t0 TCP server1-ceph.private.:7150 (LISTEN) node_expo 31572 nobody 3u IPv6 65495 0t0 TCP *:9100 (LISTEN) alertmana 31573 nobody 3u IPv6 21377 0t0 TCP *:9094 (LISTEN) alertmana 31573 nobody 8u IPv6 136298 0t0 TCP *:9093 (LISTEN) prometheu 31757 nobody 7u IPv6 109680 0t0 TCP *:9095 (LISTEN) grafana 31758 node-exporter 11u IPv6 100726 0t0 TCP *:3000 (LISTEN) ceph-mon 31850 ceph 27u IPv4 139664 0t0 TCP server1-ceph.private.:3300 (LISTEN) ceph-mon 31850 ceph 28u IPv4 139665 0t0 TCP server1-ceph.private.:6789 (LISTEN) So the ceph-mon listen on the private interface. Is this something normal ? Because according to https://access.redhat.com/documentation/fr-fr/red_hat_ceph_storage/5/html/co... only the OSD should listen on private network. Is they are anyway to configure booth public_network and private_network with cephadm bootstrap ? Regards. -- Albert SHIH 🦫 🐸 France Heure locale/Local time: jeu. 30 nov. 2023 18:27:08 CET
cluster_network is an optional add-on to handle some of the internal ceph traffic. Your mon address needs to be accessible/routable for anything outside your ceph cluster that wants to consume it. That should also be in your public_network range. I stumbled over this a few times in figuring out how to deploy ceph. Should the documentation be cleaned up? On Thu, Nov 30, 2023 at 12:37 PM Albert Shih <Albert.Shih@obspm.fr> wrote:
Hi everyone.
Status : Installing a ceph cluster Version : 17.2.7 Quincy OS : Debian 11.
Each of my server got two ip address. One public and one private.
When I'm trying to deploy my cluster with on a server
server1 (the hostname)
with
cephadm bootstrap --mon-id hostname --mon-ip IP_PRIVATE --cluster-network PRIVATE_SUB
I end up with private network for
ceph config get mon public_network
So I try to change it with
ceph config set mon public_network PUBLIC_SUB
still with lsof -i |grep -i listen I got
ceph-mgr 31427 ceph 49u IPv4 119937 0t0 TCP server1-ceph.private.:7150 (LISTEN) node_expo 31572 nobody 3u IPv6 65495 0t0 TCP *:9100 (LISTEN) alertmana 31573 nobody 3u IPv6 21377 0t0 TCP *:9094 (LISTEN) alertmana 31573 nobody 8u IPv6 136298 0t0 TCP *:9093 (LISTEN) prometheu 31757 nobody 7u IPv6 109680 0t0 TCP *:9095 (LISTEN) grafana 31758 node-exporter 11u IPv6 100726 0t0 TCP *:3000 (LISTEN) ceph-mon 31850 ceph 27u IPv4 139664 0t0 TCP server1-ceph.private.:3300 (LISTEN) ceph-mon 31850 ceph 28u IPv4 139665 0t0 TCP server1-ceph.private.:6789 (LISTEN)
So the ceph-mon listen on the private interface.
Is this something normal ? Because according to
https://access.redhat.com/documentation/fr-fr/red_hat_ceph_storage/5/html/co...
only the OSD should listen on private network.
Is they are anyway to configure booth public_network and private_network with cephadm bootstrap ?
Regards.
-- Albert SHIH 🦫 🐸 France Heure locale/Local time: jeu. 30 nov. 2023 18:27:08 CET _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (2)
-
Albert Shih
-
John Jasen