You can also prepare a crush rule to put only 2 chunks per host. That way you can still operate cluster even if a single host is down. Regards, On 3/25/21 7:53 PM, Martin Verges wrote: -- PS

Thank you for the answers. But I don't have problem with setting 8+2. The problem is the expansion. I need to move the 5 node with data in it and add 5 node later because they're in different city. The goal I'm trying to reach is 8+2 (host crush rule) So I want to cut the data 10 pieces and put them in 5 node. After adding the other 5 node I want to move 5 piece of the data to new nodes and have 8+2 in the end. Also the data is S3 and I shouldn't break RGW multisite to continue later... If I can not continue then the data will be garbage anyway. If I use internet for sync the data, it will take 1 month or more. Its easier to take 5 node in "B" and bring to "A" and create cluster, sync the data and move back to B :) Because of that I need to create 5 node cluster first with "8+2 EC", sync the data, move the 5 node to B datacenter, add the other 5 nodes later and rebalance all the data to reach 8+2 (host) But I really don't know it will work. I'm using Replication this is the first time dealing with EC setup. BTW: Every node has 20x14TB Sas and 4x900GB SSD for RGW index. ( SSD's are Replication 3) Dan van der Ster <dan(a)vanderster.com>om>, 25 Mar 2021 Per, 22:03 tarihinde şunu yazdı: > > Here's a crush ruleset for 8+2 that will choose 2 osds per host: > > > rule cephfs_data_82 { > id 4 > type erasure > min_size 3 > max_size 10 > step set_chooseleaf_tries 5 > step set_choose_tries 100 > step take default class hdd > step choose indep 5 type host > step choose indep 2 type osd > step emit > } > > > > This is kind of useful because if you set min_size to 8, you could even lose an entire host and stay online. > > Cheers, dan > > > > > > On Thu, Mar 25, 2021, 7:02 PM by morphin <morphinwithyou(a)gmail.com> wrote: >> >> Hello. >> >> I have 5 node Cluster in A datacenter. Also I have same 5 node in B datacenter. >> They're gonna be 10 node 8+2 EC cluster for backup but I need to add >> the 5 node later. >> I have to sync my S3 data with multisite on the 5 node cluster in A >> datacenter and move >> them to the B and add the other 5 node to the same cluster. >> >> The question is: Can I create 8+2 ec pool on 5 node cluster and add >> the 5 node later? How can I rebalance the data after that? >> Or is there any better solution in my case? what should I do? >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io