Besides the recoverry scenario , in a write only scenario the cluster
network will use the almost the same bandwith as public network.
linyunfan
Anthony D'Atri <anthony.datri(a)gmail.com> 于2020年5月9日周六 下午4:32写道:
>
>
> > Hi,
> >
> > I deployed few clusters with two networks as well as only one network.
> > There has little impact between them for my experience.
> >
> > I did a performance test on nautilus cluster with two networks last week.
> > What I found is that the cluster network has low bandwidth usage
>
> During steady-state, sure. Heartbeats go over that, as do replication ops when
clients write data.
>
> During heavy recovery or backfill, including healing from failures, balancing,
adding/removing drives, much more will be used.
>
> Convention wisdom has been to not let that traffic DoS clients, or clients to DoS
heartbeats.
>
> But this I think dates to a time when 1Gb/s networks were common. If one’s using
modern multiple/bonded 25Gb/s or 40Gb/s links ….
>
> > while public network bandwidth is nearly full.
>
> If your public network is saturated, that actually is a problem, last thing you want
is to add recovery traffic, or to slow down heartbeats. For most people, it isn’t
saturated.
>
> How do you define “full” ? TOR uplinks? TORs to individual nodes? Switch
backplanes? Are you using bonding with the wrong hash policy?
>
> > As a result, I don't think the cluster network is necessary.
>
> For an increasing percentage of folks deploying production-quality clusters, agreed.
>
> >
> >
> > Willi Schiegel <willi.schiegel(a)technologit.de> 于2020年5月8日周五 下午6:14写道:
> >
> >> Hello Nghia,
> >>
> >> I once asked a similar question about network architecture and got the
> >> same answer as Martin wrote from Wido den Hollander:
> >>
> >> There is no need to have a public and cluster network with Ceph. Working
> >> as a Ceph consultant I've deployed multi-PB Ceph clusters with a single
> >> public network without any problems. Each node has a single IP-address,
> >> nothing more, nothing less.
> >>
> >> In the current Ceph manual you can read
> >>
> >> It is possible to run a Ceph Storage Cluster with two networks: a public
> >> (front-side) network and a cluster (back-side) network. However, this
> >> approach complicates network configuration (both hardware and software)
> >> and does not usually have a significant impact on overall performance.
> >> For this reason, we generally recommend that dual-NIC systems either be
> >> configured with two IPs on the same network, or bonded.
> >>
> >> I followed the advice from Wido "One system, one IP address" and
> >> everything works fine. So, you should be fine with one interface for
> >> MONs, MGRs, and OSDs.
> >>
> >> Best
> >> Willi
> >>
> >> On 5/8/20 11:57 AM, Nghia Viet Tran wrote:
> >>> Hi Martin,
> >>>
> >>> Thanks for your response. You mean one network interface for only MON
> >>> hosts or for the whole cluster including OSD hosts? I’m confusing now
> >>> because there are some projects that only useone public network for the
> >>> whole cluster. That means the rebalancing, replicating objects and
> >>> heartbeats from OSD hostswould affects the performance of Ceph client.
> >>>
> >>> *From: *Martin Verges <martin.verges(a)croit.io>
> >>> *Date: *Friday, May 8, 2020 at 16:20
> >>> *To: *Nghia Viet Tran <Nghia.Viet.Tran(a)mgm-tp.com>
> >>> *Cc: *"ceph-users(a)ceph.io" <ceph-users(a)ceph.io>
> >>> *Subject: *Re: [ceph-users] Cluster network and public network
> >>>
> >>> Hello Nghia,
> >>>
> >>> just use one network interface card and use frontend and backend
traffic
> >>> on the same. No problem with that.
> >>>
> >>> If you have a dual port card, use both ports as an LACP channel and
> >>> maybe separate it using VLANs if you want to, but not required as well.
> >>>
> >>>
> >>> --
> >>>
> >>> Martin Verges
> >>> Managing director
> >>>
> >>> Mobile: +49 174 9335695
> >>> E-Mail: martin.verges(a)croit.io <mailto:martin.verges@croit.io>
> >>> Chat:
https://t.me/MartinVerges
> >>>
> >>> croit GmbH, Freseniusstr. 31h, 81247 Munich
> >>> CEO: Martin Verges - VAT-ID: DE310638492
> >>> Com. register: Amtsgericht Munich HRB 231263
> >>>
> >>> Web:
https://croit.io
> >>> YouTube:
https://goo.gl/PGE1Bx
> >>>
> >>> Am Fr., 8. Mai 2020 um 09:29 Uhr schrieb Nghia Viet Tran
> >>> <Nghia.Viet.Tran(a)mgm-tp.com
<mailto:Nghia.Viet.Tran@mgm-tp.com>>:
> >>>
> >>> Hi everyone,
> >>>
> >>> I have a question about the network setup. From the document, It’s
> >>> recommended to have 2 NICs per hosts as described in below picture
> >>>
> >>> Diagram
> >>>
> >>> In the picture, OSD hosts will connect to the Cluster network for
> >>> replicate and heartbeat between OSDs, therefore, we definitely need
> >>> 2 NICs for it. But seems there are no connections between Ceph MON
> >>> and Cluster network. Can we install 1 NIC on Ceph MON then?
> >>>
> >>> I appreciated any comments!
> >>>
> >>> Thank you!
> >>>
> >>> --
> >>>
> >>> Nghia Viet Tran (Mr)
> >>>
> >>> _______________________________________________
> >>> ceph-users mailing list -- ceph-users(a)ceph.io
> >>> <mailto:ceph-users@ceph.io>
> >>> To unsubscribe send an email to ceph-users-leave(a)ceph.io
> >>> <mailto:ceph-users-leave@ceph.io>
> >>>
> >>>
> >>> _______________________________________________
> >>> ceph-users mailing list -- ceph-users(a)ceph.io
> >>> To unsubscribe send an email to ceph-users-leave(a)ceph.io
> >>>
> >> _______________________________________________
> >> ceph-users mailing list -- ceph-users(a)ceph.io
> >> To unsubscribe send an email to ceph-users-leave(a)ceph.io
> >>
> > _______________________________________________
> > ceph-users mailing list -- ceph-users(a)ceph.io
> > To unsubscribe send an email to ceph-users-leave(a)ceph.io
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io