[ceph-users] Re: Same data for two buildings

Denis, I’m doing something similar to you with 5 nodes, 4 with OSDs and a 5th just as a mon. I have pools set with 4 replicas, minimum 2, crush map configured so 2 replicas go to each DC and then to host level. 5th mon is in a third location, but could be a VM with higher latency somewhere else. It’s primary purpose is to provide quorum if I lose a DC. With this setup I can lose a full DC and still have a working cluster. The Pacific release has some optimisations around this ‘stretched cluster’ deployment. In Pacific, OSDs will only communicate with mon in same DC and In event of DC loss then pool min_size automatically reduces to 1. Mon election also has some optimisations which help in asymmetric network failures. There’s a good youtube video from early 2020 which talks about stretched cluster deployment and it’s actually in the development version admin documents on Ceph.io Regards A Sent from my iPhone On 17 Mar 2021, at 18:08, Denis Morejon Lopez <denis.morejon(a)etecsa.cu> wrote:  ⁣I have a ceph cluster with 5 nodes. 3 in one building, and 2 in the other one. I put this information in the CRUSH. So that ceph were able to put one copy of objects in the nodes of one building and the other copy to the nodes of the other building. I mean, I setup replicas=2 in order to put the same information in both locations. But I know ceph cluster needs half +1 up nodes to keep one part of cluster working. I need at least a manual procedure to recover one of the two buildings if the other get down, or even if the link between them get down. I do not need 100% up, just something to block and unblock some nodes, and starts the two nodes if the building down were the one with 3 nodes. _______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io