Are replicas 4 or 6 safe during network partition? Will there be split-brain?
Hi, I'm evaluating if an even number of replicas is safe. Are 4 and 6 replicas still safe compared to replicas 5 and 7? With replicas 4 the min_size is 2. Does this mean in a split-brain situation there's a possibility that OSDs on both sides of the split could accept writes? Here's the scenario I'm thinking of. Please correct the assumptions I am making. So far I am unable to create this problem in my testing: There are two sites, A and B. There are 5 mons, 2 in A, 3 in B. Looking at just one PG and 4 replicas, we have 2 replicas in site A and 2 replicas in site B. Site A holds the primary OSD for this PG. When a network split happens, I/O would still be working in site A since there are still 2 OSDs, even without mon quorum. The primary OSD can still reach one of the other OSDs but not a quorum of mons. After a period of time, site B has quorum and changes the topology of the cluster to say it lost site A. Writes are also accepted in site A because min_size 2 is also satisfied. My question is - is this an actual scenario that can happen with an even number of replicas and an even network split? Do I need to go to 5 replicas with a min_size of 3 to prevent an even split from accepting writes on both sides? In that case one side would have 3 copies and the other would have 2. Thanks.
Hi, On 07.07.23 16:52, jcichra@cloudflare.com wrote:
There are two sites, A and B. There are 5 mons, 2 in A, 3 in B. Looking at just one PG and 4 replicas, we have 2 replicas in site A and 2 replicas in site B. Site A holds the primary OSD for this PG. When a network split happens, I/O would still be working in site A since there are still 2 OSDs, even without mon quorum.
The site without MON quorum will stop to work completely. Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 220009 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
participants (2)
-
jcichra@cloudflare.com
-
Robert Sander