Hi Frederico,
On 04/02/2021 05:51, Federico Lucifredi wrote:
Hi Loïc,
I am intrigued, but am missing something: why not using RGW, and store the source code
files as objects? RGW has native compression and can take care of that behind the scenes.
Excellent question!
Is the desire to use RBD only due to minimum allocation sizes?
I *assume* that
since RGW does have specific strategies to take advantage of the fact that objects are
immutable and will never be removed:
* It will be slower to add artifacts in RGW than in an RBD image + index
* The metadata in RGW will be larger than an RBD image + index
However I have not verified this and if you have an opinion I'd love to hear it :-)
Cheers
Best -F
-- "'Problem' is a bleak word for challenge" - Richard Fish
_________________________________________
Federico Lucifredi
Product Management Director, Ceph Storage Platform
Red Hat
A273 4F57 58C0 7FE8 838D 4F87 AEEB EC18 4A73 88AC
redhat.com <http://redhat.com> TRIED. TESTED. TRUSTED.
On Sat, Jan 30, 2021 at 10:01 AM Loïc Dachary <loic(a)dachary.org
<mailto:loic@dachary.org>> wrote:
Bonjour,
In the context Software Heritage (a noble mission to preserve all source code)[0],
artifacts have an average size of ~3KB and there are billions of them. They never change
and are never deleted. To save space it would make sense to write them, one after the
other, in an every growing RBD volume (more than 100TB). An index, located somewhere else,
would record the offset and size of the artifacts in the volume.
I wonder if someone already implemented this idea with success? And if not... does
anyone see a reason why it would be a bad idea?
Cheers
[0]
https://docs.softwareheritage.org/ <https://docs.softwareheritage.org/>
--
Loïc Dachary, Artisan Logiciel Libre
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io <mailto:ceph-users@ceph.io>
To unsubscribe send an email to ceph-users-leave(a)ceph.io
<mailto:ceph-users-leave@ceph.io>
--
Loïc Dachary, Artisan Logiciel Libre