Hi Casey, thank you for the reply. I was wondering, just as the placement target is in the bucket metadata in the index, if it would not be possible to insert the storage-class information in the metadata of the object that is in the index as well. Or did I get it wrong and there is absolutely no type of object metadata in the index, just a listing of the objects? Thanks again, Marcelo. <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Livre de vírus. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>. <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> Em qua., 10 de fev. de 2021 às 11:43, Casey Bodley <cbodley@redhat.com> escreveu:
On Wed, Feb 10, 2021 at 8:31 AM Marcelo <raxidex@gmail.com> wrote:
Hello all!
We have a cluster where there are HDDs for data and NVMEs for journals
and
indexes. We recently added pure SSD hosts, and created a storage class SSD. To do this, we create a default.rgw.hot.data pool, associate a crush rule using SSD and create a HOT storage class in the placement-target. The problem is when we send an object to use a HOT storage class, it is in both the STANDARD storage class pool and the HOT pool.
STANDARD pool: # rados -p default.rgw.buckets.data ls d86dade5-d401-427b-870a-0670ec3ecb65.385198.4_LICENSE
# rados -p default.rgw.buckets.data stat d86dade5-d401-427b-870a-0670ec3ecb65.385198.4_LICENSE
default.rgw.buckets.data/d86dade5-d401-427b-870a-0670ec3ecb65.385198.4_LICENSE
mtime 2021-02-09 14: 54: 14.000000, size 0
HOT pool: # rados -p default.rgw.hot.data ls
d86dade5-d401-427b-870a-0670ec3ecb65.385198.4__shadow_.rmpla1NTgArcUQdSLpW4qEgTDlbhn9f_0
# rados -p default.rgw.hot.data stat
d86dade5-d401-427b-870a-0670ec3ecb65.385198.4__shadow_.rmpla1NTgArcUQdSLpW4qEgTDlbhn9f_0
default.rgw.hot.data/d86dade5-d401-427b-870a-0670ec3ecb65.385198.4__shadow_.rmpla1NTgArcUQdSLpW4qEgTDlbhn9f_0
mtime 2021-02-09 14: 54: 14.000000, size 15220
The object itself is in the HOT pool, however it creates this other object similar to an index in the STANDARD pool. Monitoring with iostat we noticed that this behavior generates an unnecessary IO on disks that do not need to be touched.
Why this behavior? Are there any ways around it?
this object in the STANDARD pool is called the 'head object', and it holds the s3 object's metadata - including an attribute that says which storage class the object's data is in
when an S3 client downloads the object with a 'GET /bucket/LICENSE' request, it doesn't specify the storage class. so radosgw has to find its head object in a known location (the bucket's default storage class pool) in order to figure out which pool holds the object's data
Thanks, Marcelo _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io