30 Jan
2020
30 Jan
'20
3:34 p.m.
Iam testing failure scenarios for my cluster. I have 3 monitors. Lets say if mons 1 and 2
go down and so monitors can't form a quorum, how can I recover?
Are the instructions at followling link valid for deleting mons 1 and 2 from monmap,
https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/1.2.3/ht…
One more question - lets say I delete mons 1 and 2 from monmap. And the cluster has only
mon 3 remaining so mon 3 has quorum. Now what happens if mon 1 and 2 come up? Do they join
mon 3 and so there will again be 3 monitors in the cluster?
Thanks
30 Jan
30 Jan
3:37 p.m.
On 1/30/20 1:34 PM, vishal(a)denovogroup.org wrote:
Iam testing failure scenarios for my cluster. I have 3
monitors. Lets say if mons 1 and 2 go down and so monitors can't form a quorum, how
can I recover?
Are the instructions at followling link valid for deleting mons 1 and 2 from monmap,
https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/1.2.3/ht…
One more question - lets say I delete mons 1 and 2 from monmap. And the cluster has only
mon 3 remaining so mon 3 has quorum. Now what happens if mon 1 and 2 come up? Do they join
mon 3 and so there will again be 3 monitors in the cluster?
The epoch of the monmap has increased by removing mons 1 and 2 from the
map. Only mon 3 has this new map with the new epoch.
Therefor, if mon 1 and 2 boot they see the epoch of mon 3 is newer and
thus won't be able to join.
Wido
> Thanks
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
>
3:55 p.m.
On Thu, Jan 30, 2020 at 1:38 PM Wido den Hollander <wido(a)42on.com> wrote:
On 1/30/20 1:34 PM, vishal(a)denovogroup.org wrote:
If you delete the monitors from the map using ceph commands, so that
they KNOW they've been removed, this is fine. But you don't want to do
that to a cluster using offline tools: if monitor 3 dies before mons 1
and 2 turn on, they will find each other, not see another peer, and
say "hey, we are 2 of the 3 monitors in the map, let's form a quorum!"
-Greg
Iam testing failure scenarios for my cluster. I
have 3 monitors. Lets say if mons 1 and 2 go down and so monitors can't form a quorum,
how can I recover?
Are the instructions at followling link valid for deleting mons 1 and 2 from monmap,
https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/1.2.3/ht…
One more question - lets say I delete mons 1 and 2 from monmap. And the cluster has only
mon 3 remaining so mon 3 has quorum. Now what happens if mon 1 and 2 come up? Do they join
mon 3 and so there will again be 3 monitors in the cluster?
The epoch of the monmap has increased by removing mons 1 and 2 from the
map. Only mon 3 has this new map with the new epoch.
Therefor, if mon 1 and 2 boot they see the epoch of mon 3 is newer and
thus won't be able to join.
Wido
Thanks
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
4:24 p.m.
On 1/30/20 1:55 PM, Gregory Farnum wrote:
On Thu, Jan 30, 2020 at 1:38 PM Wido den Hollander
<wido(a)42on.com> wrote:
Aha, yes! That's a situation I didn't think of. Good addition to my answer.
Wido
> -Greg
>
>>
>> Wido
>>
>>> Thanks
>>> _______________________________________________
>>> ceph-users mailing list -- ceph-users(a)ceph.io
>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io
>>>
>> _______________________________________________
>> ceph-users mailing list -- ceph-users(a)ceph.io
>> To unsubscribe send an email to ceph-users-leave(a)ceph.io
>>
>
On 1/30/20 1:34 PM, vishal(a)denovogroup.org wrote:
If you delete the monitors from the map using ceph commands, so that
they KNOW they've been removed, this is fine. But you don't want to do
that to a cluster using offline tools: if monitor 3 dies before mons 1
and 2 turn on, they will find each other, not see another peer, and
say "hey, we are 2 of the 3 monitors in the map, let's form a quorum!"
Iam testing failure scenarios for my cluster. I
have 3 monitors. Lets say if mons 1 and 2 go down and so monitors can't form a quorum,
how can I recover?
Are the instructions at followling link valid for deleting mons 1 and 2 from monmap,
https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/1.2.3/ht…
One more question - lets say I delete mons 1 and 2 from monmap. And the cluster has only
mon 3 remaining so mon 3 has quorum. Now what happens if mon 1 and 2 come up? Do they join
mon 3 and so there will again be 3 monitors in the cluster?
The epoch of the monmap has increased by removing mons 1 and 2 from the
map. Only mon 3 has this new map with the new epoch.
Therefor, if mon 1 and 2 boot they see the epoch of mon 3 is newer and
thus won't be able to join.
31 Jan
31 Jan
7:27 a.m.
Thanks folks for the replies. Now I feel confident to test this out in my cluster.
Thanks
1551
days inactive
1552
days old
4 comments
3 participants
participants (3)
-
Gregory Farnum -
vishal＠denovogroup.org -
Wido den Hollander