17 Apr
2024
17 Apr
'24
2:56 p.m.
Hi Johansson-san,
Thank you very much for your detailed explanation. I read some documents in
Ceph community. So I generally understand. Thank you very much for all the
useful advice. The characteristics of distributed storage seem to be quite
complex, so I will investigate various things when I have time.
Regards,
--
Mitsumasa KONDO
2024年4月16日(火) 15:35 Janne Johansson <icepic.dz(a)gmail.com>om>:
Den mån 15 apr. 2024 kl 13:09 skrev Mitsumasa KONDO
<
kondo.mitsumasa(a)gmail.com>gt;:
Hi Menguy-san,
Thank you for your reply. Users who use large IO with tiny volumes are a
nuisance to cloud providers.
I confirmed my ceph cluster with 40 SSDs. Each OSD on 1TB SSD has about
50
placement groups in my cluster. Therefore, each
PG has approximately 20GB
of space.
If we create a small 8GB volume, I had a feeling it wouldn't be
distributed
well, but it will be distributed well.
RBD images get split into 2 or 4M pieces when stored in ceph, so an 8G
RBD image will be split into 2048-or-4096 separate pieces that end up
"randomly" on the PGs the pool is based on, which means that if you
read or write the whole RBD image from start to end, you are going to
spread the load to all OSDs.
I think it works something like this, you ask librbd for an 8G image
named "myimage", and underneath it makes myimage.0, myimage.1, 2,3,4
and so on. The PG placement will depend on the object name, which of
course differs for all the pieces, and hence they end up on different
PGs, thereby spreading the load. If ceph did not do this, then you
could never make an RBD image that was larger than "smallest free
space on any of the pools OSDs" but also, it would mean that the RBD
client would be talking to the same single OSD for everything, and
that would not be a good way to use a clusters resources evenly.
--
May the most significant bit of your life be positive.