[ceph-users] A practical approach to efficiently store 100 billions small objects in Ceph

Bonjour, In the past weeks a few mailing list threads[0][1][2] explored the problem of storing billions of small objects in Ceph. There was great feedback (I learned at lot) and it turns out the solution is a rather simple aggregation of the ideas that were suggested during these discussions. It is described in detail here:    https://wiki.softwareheritage.org/wiki/A_practical_approach_to_efficiently_… The next step will be to write and run benchmarks[3]. Although I'm convinced it is a good solution, I've been wrong before and confirmation is required :-) Many thanks to all the participants in the discussions on behalf of the Software Heritage project[4]. Cheers [0] Storing 20 billions of immutable objects in Ceph, 75% <16KB https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/JSG2TXKNXPX… [1] Small RGW objects and RADOS 64KB minimun size https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/AEMW6O7WVJF… [2] Using RBD to pack billions of small files https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/RHQ5ZCHJISX… [3] https://forge.softwareheritage.org/T3054 [4] https://www.softwareheritage.org/ -- Loïc Dachary, Artisan Logiciel Libre