Change mon bind address / Change IPs with the orchestrator
Hello, I think I missunderstood the internal / public network concepts in the docs https://docs.ceph.com/docs/master/rados/configuration/network-config-ref/. Now there are two questions: - Is it somehow possible to bind the MON daemon to 0.0.0.0? I tried it with manually add the ip in /var/lib/ceph/{UUID}/mon.node01/config [mon.node01] public bind addr = 0.0.0.0 But that does not work, in netstat I can see, the mon still binds to it's internal ip. Is this an expected behaviour? If I set this value to the public ip, the other nodes cannot communicate with it, so this leads to the next question: - What's the Right way to correct the problem with the orchestrator? So the correct way to configure the ip's, would be to set every mon, mds and so on, to the public ip and just let the osd's stay on their internal ip. (described here https://docs.ceph.com/docs/master/rados/configuration/network-config-ref/) Do I have to remove every daemon and redeploy them with "ceph orch daemon rm" / "ceph orch apply"? Or do I have to go to every node and manually apply the settings in the daemon config file? Thanks in advance, Simon
On 6/3/20 4:49 PM, Simon Sutter wrote:
Hello,
I think I missunderstood the internal / public network concepts in the docs https://docs.ceph.com/docs/master/rados/configuration/network-config-ref/.
Now there are two questions:
- Is it somehow possible to bind the MON daemon to 0.0.0.0?
No
I tried it with manually add the ip in /var/lib/ceph/{UUID}/mon.node01/config
Won't work :-)
[mon.node01] public bind addr = 0.0.0.0
But that does not work, in netstat I can see, the mon still binds to it's internal ip. Is this an expected behaviour?
Yes. This is not an orchestrator thing, this is how the MONs work. They need to bind to a specific IP and that can't be 0.0.0.0 You then need to make sure proper routing is in place so all clients and OSDs can talk to the MONs. So don't attempt anything like NAT. Make sure everything works with proper IP-routing. Wido
If I set this value to the public ip, the other nodes cannot communicate with it, so this leads to the next question:
- What's the Right way to correct the problem with the orchestrator? So the correct way to configure the ip's, would be to set every mon, mds and so on, to the public ip and just let the osd's stay on their internal ip. (described here https://docs.ceph.com/docs/master/rados/configuration/network-config-ref/)
Do I have to remove every daemon and redeploy them with "ceph orch daemon rm" / "ceph orch apply"?
Or do I have to go to every node and manually apply the settings in the daemon config file?
Thanks in advance,
Simon
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hello, Ok, thanks Wido. I have now sorted out the correct network configuration and deployed new mon's with the new IP. Everything is now on the new IP and works so far. Simon ________________________________ Von: Wido den Hollander <wido@42on.com> Gesendet: Donnerstag, 4. Juni 2020 08:47:29 An: Simon Sutter; ceph-users@ceph.io Betreff: Re: [ceph-users] Change mon bind address / Change IPs with the orchestrator On 6/3/20 4:49 PM, Simon Sutter wrote:
Hello,
I think I missunderstood the internal / public network concepts in the docs https://docs.ceph.com/docs/master/rados/configuration/network-config-ref/.
Now there are two questions:
- Is it somehow possible to bind the MON daemon to 0.0.0.0?
No
I tried it with manually add the ip in /var/lib/ceph/{UUID}/mon.node01/config
Won't work :-)
[mon.node01] public bind addr = 0.0.0.0
But that does not work, in netstat I can see, the mon still binds to it's internal ip. Is this an expected behaviour?
Yes. This is not an orchestrator thing, this is how the MONs work. They need to bind to a specific IP and that can't be 0.0.0.0 You then need to make sure proper routing is in place so all clients and OSDs can talk to the MONs. So don't attempt anything like NAT. Make sure everything works with proper IP-routing. Wido
If I set this value to the public ip, the other nodes cannot communicate with it, so this leads to the next question:
- What's the Right way to correct the problem with the orchestrator? So the correct way to configure the ip's, would be to set every mon, mds and so on, to the public ip and just let the osd's stay on their internal ip. (described here https://docs.ceph.com/docs/master/rados/configuration/network-config-ref/)
Do I have to remove every daemon and redeploy them with "ceph orch daemon rm" / "ceph orch apply"?
Or do I have to go to every node and manually apply the settings in the daemon config file?
Thanks in advance,
Simon
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (2)
-
Simon Sutter
-
Wido den Hollander