Re: Two writes per shard in erasure coding
When EC performs a write to a shard, it must ensure that the write can be rolled back if required for recovery. It achieves this by creating a temporary clone of the old data, which must be deleted once that write is completed on all shards. If, once the write is completed, another write is queued for the same PG, then this clean up is triggered by the next write. If, however, there is no queued IO, EC must send a “dummy” op to trigger this clean up. For many (possibly most) workloads, the queue depth to each PG is not high, so the dummy ops are frequently used and do lead to IO amplification. For tentacle, we have a significant update to erasure coding to make it much more efficient and have significantly reduced the IO amplification for small writes. See my Ceph London talk about what we have achieved here: https://youtu.be/WH6dFrhllyo Tentacle does not attempt to improve the dummy op behaviour. There are potential improvements possible in this area and they would yield a small improvement to small-block work loads. Regards, Alex Ainscow Unless otherwise stated above: IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: Building C, IBM Hursley Office, Hursley Park Road, Winchester, Hampshire SO21 2JN
participants (1)
-
Alexander Ainscow