Hi everyone, I just like to know what's your opinion about the reliability of erasure coding. Of course I can understand if we want the «best of the best of the best» ;-) I can choose the replica method. I heard in many location “replica” are more reliable, “replica” are more efficient etc... Yes...well since 25 years I'm using raid (5, 6, lvm, raidz1, raidz2, etc.) I never loose data only once when a firmware bug in some xxxxx card crash the raid volume. Now 25 years later lot of people recommend to use replica so if I buy XTo I'm only going to have X/3 To (vs raidz2 where I loose 2 disks over 9-12 disks). So my question are : Anyone use in large scale erasure coding for critical (same level as raidz1/raid5 ou raidz2/raid6) ? Regards -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: jeu. 23 nov. 2023 14:51:28 CET
I use EC 4+2 on backup backup site, production site is running replica3, running 8 servers on backup side and 12 on production side number of OSDs per server is 16 on all of them Production has lacp bonded 25G networking for public and cluster network backup has just 10G networking with no redundancy On Thu, Nov 23, 2023 at 2:58 PM Albert Shih <Albert.Shih@obspm.fr> wrote:
Hi everyone,
I just like to know what's your opinion about the reliability of erasure coding.
Of course I can understand if we want the «best of the best of the best» ;-) I can choose the replica method.
I heard in many location “replica” are more reliable, “replica” are more efficient etc...
Yes...well since 25 years I'm using raid (5, 6, lvm, raidz1, raidz2, etc.) I never loose data only once when a firmware bug in some xxxxx card crash the raid volume.
Now 25 years later lot of people recommend to use replica so if I buy XTo I'm only going to have X/3 To (vs raidz2 where I loose 2 disks over 9-12 disks).
So my question are : Anyone use in large scale erasure coding for critical (same level as raidz1/raid5 ou raidz2/raid6) ?
Regards -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: jeu. 23 nov. 2023 14:51:28 CET _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Yes, lots of people are using EC. Which is more “reliable” depends on what you need. If you need to survive 4 failures, there are scenarios where RF=3 won’t do it for you. You could in such a case use an EC 4,4 profile, 8,4, etc. It’s a tradeoff between write speed and raw::usable ratio efficiency. Which do you need more? Depending on your data, EC may increase space amp significantly. With RF=2 I have seen overlapping failures in my career. The larger the deployment, the greater the chance of overlapping failures. In a distributed system especially, RF=2 or EC with m=1 are bad ideas if your data is valuable.
On Nov 23, 2023, at 8:58 AM, Albert Shih <Albert.Shih@obspm.fr> wrote:
Hi everyone,
I just like to know what's your opinion about the reliability of erasure coding.
Of course I can understand if we want the «best of the best of the best» ;-) I can choose the replica method.
I heard in many location “replica” are more reliable, “replica” are more efficient etc...
Yes...well since 25 years I'm using raid (5, 6, lvm, raidz1, raidz2, etc.) I never loose data only once when a firmware bug in some xxxxx card crash the raid volume.
Now 25 years later lot of people recommend to use replica so if I buy XTo I'm only going to have X/3 To (vs raidz2 where I loose 2 disks over 9-12 disks).
So my question are : Anyone use in large scale erasure coding for critical (same level as raidz1/raid5 ou raidz2/raid6) ?
Regards -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: jeu. 23 nov. 2023 14:51:28 CET _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Now 25 years later lot of people recommend to use replica so if I buy XTo I'm only going to have X/3 To (vs raidz2 where I loose 2 disks over 9-12 disks).
As seen from other answers, it changes which performance and space usage you want go have, but there are other factors too. Replica = 3 (or larger) will be as fast when one (or more) of the replicas is missing, but EC needs to start to do math on the data from one or more of the checksum shards before it can return data to the client, so they degrade differently also. It is never as easy as "one size fits all", the tradeoffs have multiple dimensions. -- May the most significant bit of your life be positive.
participants (4)
-
Albert Shih
-
Anthony D'Atri
-
Janne Johansson
-
Nino Kotur