It will likely never be as fast as replication, but it can be much faster than it is now. I put benchmarks in the original PR I resurrected for partial stripe reads: https://github.com/ceph/ceph/pull/52746 I'm trying to find some time this week to run some tests on Radek's PR so we can do some comparisons. Mark On 7/2/24 02:28, Sebastian Wagner wrote:
Hi Bill,
I noticed you mentioned the PR a few moments after I clicked on send 🙈. I really hope we end up with EC being on par with replicated pools without any meaningful performance impact.
Do you think we can put those design documents into the Ceph Github organization? IIRC, this is the third proposal on this mailing list that I saw just in a few months.
Best, Sebastian
Am 01.07.24 um 12:01 schrieb Bill Scales:
Hi,
Yes, most of the optimizations for partial reads are part of that pull request – it is referenced in the design document.
There is one further enhancement we would like to make which is to reduce the size of the reads to each shard – the pull request currently reads the whole chunk for any shard that needs to be read. This is not an issue with the current default chunk size which is 4K, but if you use a larger chunk size such as 256K then this means that short client reads will read excess data and then discard it.
Currently there are many reasons why using a small chunk size is a good idea, but once both partial reads and partial writes have been implemented performance can be improved further by using a larger chunk size because splitting I/Os into multiple chunks is expensive. There’s more discussion about the chunk size and how to accommodate small sized objects without large storage overheads later in the document.
Cheers,
Bill. bill_scales@uk.ibm.com <mailto:bill_scales@uk.ibm.com> IBM Distinguished Engineer, IBM Storage
*From: *Sebastian Wagner <sebastian.wagner@croit.io> *Date: *Monday, 1 July 2024 at 09:31 *To: *Bill Scales <bill_scales@uk.ibm.com>, dev@ceph.io <dev@ceph.io> *Subject: *[EXTERNAL] Re: Erasure coding enhancements - design for review
Hi Bill, Is the Partial Reads section the same as https: //github. com/ceph/ceph/pull/55196 ? Best, Sebastian Am 01. 07. 24 um 10: 08 schrieb Bill Scales: Hi, We are planning to enhance the performance of erasure coding, in particular for use with
Hi Bill,
Is the Partial Reads section the same as https://github.com/ceph/ceph/pull/55196 ?
Best, Sebastian
Am 01.07.24 um 10:08 schrieb Bill Scales:
Hi,
We are planning to enhance the performance of erasure coding, in particular for use with block and file. We've got a design document https://github.com/bill-scales/CephErasureCodingDesign that sets out what we are hoping to achieve. We welcome your feedback, either posting your comments in Slack on #ceph-devel , raising issues in github or getting in contact with myself
Cheers,
Bill. bill_scales@uk.ibm.com <mailto:bill_scales@uk.ibm.com> IBM Distinguished Engineer, IBM Storage
Unless otherwise stated above:
IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU
_______________________________________________
Dev mailing list --dev@ceph.io
To unsubscribe send an email todev-leave@ceph.io
-- Head of Software Development E-Mail: sebastian.wagner@croit.io
croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges, Andy Muthmann - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB 231263
Web <https://croit.io/> | LinkedIn <http://linkedin.com/company/croit> | Youtube <https://www.youtube.com/channel/UCIJJSKVdcSLGLBtwSFx_epw> | Twitter <https://twitter.com/croit_io>
TOP 100 Innovator Award Winner <https://croit.io/blog/croit-receives-top-100-seal> by compamedia Technology Fast50 Award <https://croit.io/blog/deloitte-technology-fast-50-award> Winner by Deloitte
Unless otherwise stated above:
IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU
-- Head of Software Development E-Mail: sebastian.wagner@croit.io
croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges, Andy Muthmann - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB 231263
Web <https://croit.io/> | LinkedIn <http://linkedin.com/company/croit> | Youtube <https://www.youtube.com/channel/UCIJJSKVdcSLGLBtwSFx_epw> | Twitter <https://twitter.com/croit_io>
TOP 100 Innovator Award Winner <https://croit.io/blog/croit-receives-top-100-seal> by compamedia Technology Fast50 Award <https://croit.io/blog/deloitte-technology-fast-50-award> Winner by Deloitte
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
-- Best Regards, Mark Nelson Head of Research and Development Clyso GmbH p: +49 89 21552391 12 | a: Minnesota, USA w: https://clyso.com | e: mark.nelson@clyso.com We are hiring: https://www.clyso.com/jobs/