25 Sep
2019
25 Sep
'19
7:55 p.m.
hi all:
I have a production cluster, and it had 24 hosts (528 osds,3mons) at a former.
Now we want to add 36 hosts so the osd increase to 1320 .
does the monitor need to increase?how many numbers of monitor node is recommended?
Another question is which monitor does monclient commnuicate with? And how it
decide?
Any suggestions are welcome!
25 Sep
25 Sep
10:11 p.m.
On Wed, Sep 25, 2019 at 10:26 AM 展荣臻（信泰） <zhanrzh_xt(a)teamsun.com.cn> wrote:
hi all:
I have a production cluster, and it had 24 hosts (528 osds,3mons) at a
former.
Now we want to add 36 hosts so the osd increase to 1320 .
We use 3 mons in this size cluster with no issues.
does the monitor need to increase?how many numbers
of monitor node is
recommended?
I do not believe there is any need or benefit to increase the number of
monitors.
Another question is which monitor does monclient
commnuicate with? And
how it decide?
Any suggestions are welcome!
don't know this one
--
Alex Gorbachev
Intelligent Systems Services Inc.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
10:19 p.m.
hello
as far as I know and according to the document the mon just share the
cluster map with the client. Not the data
"Storage cluster clients retrieve a copy of the cluster map from the
Ceph Monitor."
https://docs.ceph.com/docs/master/architecture/
Le mercredi 25 septembre 2019 à 22:25 +0800, 展荣臻（信泰） a écrit :
hi all:
I have a production cluster, and it had 24 hosts (528 osds,3mons)
at a former.
Now we want to add 36 hosts so the osd increase to 1320 .
does the monitor need to increase?how many numbers of monitor node
is recommended?
Another question is which monitor does monclient commnuicate with?
And how it decide?
Any suggestions are welcome!
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
10:22 p.m.
You don't need more mons to scale; but going to 5 mons would make the
cluster more robust, if it is cheap for you to do so.
If you assume that 1 mon rebooting for updates or maintenance is
routine, then 2/3 is vulnerable to one failure. 4/5 can survive an
unexpected additional failure while one is down for maintenance.
Considering your scale, this improvement in uptime might be worthwhile.
On Wed, Sep 25, 2019 at 10:26 AM 展荣臻（信泰） <zhanrzh_xt(a)teamsun.com.cn> wrote:
>
>
> hi all:
> I have a production cluster, and it had 24 hosts (528 osds,3mons) at a former.
> Now we want to add 36 hosts so the osd increase to 1320 .
> does the monitor need to increase?how many numbers of monitor node is recommended?
> Another question is which monitor does monclient commnuicate with? And how it
decide?
> Any suggestions are welcome!
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
11:36 p.m.
On 9/25/19 6:52 PM, Nathan Fish wrote:
You don't need more mons to scale; but going to 5
mons would make the
cluster more robust, if it is cheap for you to do so.
If you assume that 1 mon rebooting for updates or maintenance is
routine, then 2/3 is vulnerable to one failure. 4/5 can survive an
unexpected additional failure while one is down for maintenance.
Considering your scale, this improvement in uptime might be worthwhile.
I second that. Exactly my thoughts as you can survive longer with a
failing MON. I even recommend this for smaller clusters. 5 MONs can
tolerate a MON failure when one is already down for maintenance for example.
It's about getting the most nines behind 99,XXX% of uptime.
Clients 'randomly' connect to one of the Monitors. So more Monitors
allow you to scale to a larger number of clients, but I don't think
there is data on how many clients is too much for a single MON.
Wido
> On Wed, Sep 25, 2019 at 10:26 AM 展荣臻（信泰） <zhanrzh_xt(a)teamsun.com.cn> wrote:
>>
>>
>> hi all:
>> I have a production cluster, and it had 24 hosts (528 osds,3mons) at a
former.
>> Now we want to add 36 hosts so the osd increase to 1320 .
>> does the monitor need to increase?how many numbers of monitor node is
recommended?
>> Another question is which monitor does monclient commnuicate with? And how it
decide?
>> Any suggestions are welcome!
>> _______________________________________________
>> ceph-users mailing list -- ceph-users(a)ceph.io
>> To unsubscribe send an email to ceph-users-leave(a)ceph.io
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
>
26 Sep
26 Sep
8:35 a.m.
Thanks for your reply.
We don't maintain it frequently.
My confusion is whether the more monitor is more advantage for client(osd,rbdclient...) to
get clustermap.
Do All clients comunicate with one monitor of the cluster at the mean time ? If not
how client to decide to communicat with which monitor?
From: Nathan Fish
Date: 2019-09-26 00:52
To: 展荣臻（信泰）
CC: ceph-users
Subject: Re: [ceph-users] how many monitor should to deploy in a 1000+ osd cluster
You don't need more mons to scale; but going to 5 mons would make the
cluster more robust, if it is cheap for you to do so.
If you assume that 1 mon rebooting for updates or maintenance is
routine, then 2/3 is vulnerable to one failure. 4/5 can survive an
unexpected additional failure while one is down for maintenance.
Considering your scale, this improvement in uptime might be worthwhile.
On Wed, Sep 25, 2019 at 10:26 AM 展荣臻（信泰） <zhanrzh_xt(a)teamsun.com.cn> wrote:
>
>
> hi all:
> I have a production cluster, and it had 24 hosts (528 osds,3mons) at a former.
> Now we want to add 36 hosts so the osd increase to 1320 .
> does the monitor need to increase?how many numbers of monitor node is recommended?
> Another question is which monitor does monclient commnuicate with? And how it
decide?
> Any suggestions are welcome!
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
12:23 p.m.
On 9/26/19 5:05 AM, zhanrzh_xt(a)teamsun.com.cn wrote:
Thanks for your reply.
We don't maintain it frequently.
My confusion is whether the more monitor is more advantage for client(osd,rbdclient...)
to get clustermap.
Do All clients comunicate with one monitor of the cluster at the mean time ? If not
how client to decide to communicat with which monitor?
Clients talk with one monitor, but each client to a different Monitor.
They select the Monitor they talk to randomly.
Wido
> From: Nathan Fish
> Date: 2019-09-26 00:52
> To: 展荣臻（信泰）
> CC: ceph-users
> Subject: Re: [ceph-users] how many monitor should to deploy in a 1000+ osd cluster
> You don't need more mons to scale; but going to 5 mons would make the
> cluster more robust, if it is cheap for you to do so.
> If you assume that 1 mon rebooting for updates or maintenance is
> routine, then 2/3 is vulnerable to one failure. 4/5 can survive an
> unexpected additional failure while one is down for maintenance.
> Considering your scale, this improvement in uptime might be worthwhile.
>
> On Wed, Sep 25, 2019 at 10:26 AM 展荣臻（信泰） <zhanrzh_xt(a)teamsun.com.cn> wrote:
>>
>>
>> hi all:
>> I have a production cluster, and it had 24 hosts (528 osds,3mons) at a
former.
>> Now we want to add 36 hosts so the osd increase to 1320 .
>> does the monitor need to increase?how many numbers of monitor node is
recommended?
>> Another question is which monitor does monclient commnuicate with? And how it
decide?
>> Any suggestions are welcome!
>> _______________________________________________
>> ceph-users mailing list -- ceph-users(a)ceph.io
>> To unsubscribe send an email to ceph-users-leave(a)ceph.io
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
>
12:56 p.m.
I’d always been led to believe that when a client attaches it tries the mons in its conf
file in order until it contacts one, which may then hand it off to another. This was
given as an explanation for a situation I once had where 2/5 mons were down but clients
could run but couldn’t attach/boot. The idea was that since it was the *first* two mons
in the clients’ config file that were down, they would time out before they could try the
third. The immediate workaround was to blast out an interim config file with the first
two mons commented/removed.
This was in the Firefly or Hammer days, does that sound plausible and perhaps still how
things work?
I’ve at times bound a client (well, perhaps ‘ceph status’ isn’t a client as such?) to a
specific mon with the -m switch.
— aad
Thanks for
your reply.
We don't maintain it frequently.
My confusion is whether the more monitor is more advantage for client(osd,rbdclient...)
to get clustermap.
Do All clients comunicate with one monitor of the cluster at the mean time ? If not
how client to decide to communicat with which monitor?
Clients talk with one monitor, but each client to a different Monitor.
They select the Monitor they talk to randomly.
Wido
From: Nathan Fish
Date: 2019-09-26 00:52
To: 展荣臻（信泰）
CC: ceph-users
Subject: Re: [ceph-users] how many monitor should to deploy in a 1000+ osd cluster
You don't need more mons to scale; but going to 5 mons would make the
cluster more robust, if it is cheap for you to do so.
If you assume that 1 mon rebooting for updates or maintenance is
routine, then 2/3 is vulnerable to one failure. 4/5 can survive an
unexpected additional failure while one is down for maintenance.
Considering your scale, this improvement in uptime might be worthwhile.
On Wed, Sep 25, 2019 at 10:26 AM 展荣臻（信泰） <zhanrzh_xt(a)teamsun.com.cn> wrote:
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
hi all:
I have a production cluster, and it had 24 hosts (528 osds,3mons) at a former.
Now we want to add 36 hosts so the osd increase to 1320 .
does the monitor need to increase?how many numbers of monitor node is recommended?
Another question is which monitor does monclient commnuicate with? And how it
decide?
Any suggestions are welcome!
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
1674
days inactive
1675
days old
7 comments
7 participants
participants (7)
-
Alex Gorbachev -
Anthony D'Atri -
Nathan Fish -
Olivier AUDRY -
Wido den Hollander -
zhanrzh_xt＠teamsun.com.cn -
展荣臻（信泰）