[ceph-users] Re: Crush rule examples

21 Nov 2024

      Hi Frank,

thanks a lot for the hint, and I have read the documentation about this. 
What is not clear to me is this:

== snip
The first category of these failures that we will discuss involves 
inconsistent networks -- if there is a netsplit (a disconnection between 
two servers that splits the network into two pieces), Ceph might be 
unable to mark OSDs down and remove them from the acting PG sets.
== snip

Why is Ceph not able to mark OSDs down, and why is it unclear whether or 
not it is able to do so ("might")?

Cheers
Andre

Am 20.11.24 um 12:23 schrieb Frank Schilder:
...
Hi Andre,
I think what you really want to look at is stretch mode. There have been long discussions on this list why a crush rule with rep 4 and 2 copies per DC will not handle a DC failure as expected. Stretch mode will  make sure writes happen in a way that prevents split brain scenarios.
Hand-crafted crush rules for this purpose require 3 or more DCs.
Best regards,
=================
Frank Schilder
AIT Risø Campus
Bygning 109, rum S14
________________________________________
From: Janne Johansson <icepic.dz@gmail.com>
Sent: Wednesday, November 20, 2024 11:30 AM
To: Andre Tann
Cc: ceph-users@ceph.io
Subject: [ceph-users] Re: Crush rule examples
...
Sorry, sent too early. So here we go again:
   My setup looks like this:
DC1
            node01
            node02
            node03
            node04
            node05
        DC2
            node06
            node07
            node08
            node09
            node10
I want a replicated pool with size=4. Two copies should go in each DC,
and then no two copies on a single node.
How can I describe this in a crush rule?
This post seem to show that, except they have their root named "nvme"
and they split on rack and not dc, but that is not important.
https://unix.stackexchange.com/questions/781250/ceph-crush-rules-explanation...
with the answer at the bottom:
for example this should work as well, to have 4 replicas in total,
distributed across two racks:
step take default class nvme
step choose firstn 2 type rack
step chooseleaf firstn 2 type host
--
May the most significant bit of your life be positive.
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-leave@ceph.io
-- 
Andre Tann