25 Nov
2023
25 Nov
'23
2:54 a.m.
Hi,
Other than get all objects of the pool and filter by image ID,
is there any easier way to get the number of allocated objects for
a RBD image?
What I really want to know is the actual usage of an image.
An allocated object could be used partially, but that's fine,
no need to be 100% accurate. To get the object count and
times object size, that should be sufficient.
"rbd export" exports actual used data, but to get the actual usage
by exporting the image seems too much. This brings up another
question, is there any way to know the export size before running it?
Thanks!
Tony
25 Nov
25 Nov
8:17 a.m.
Maybe I misunderstand, but isn’t ’rbd du‘ what you're looking for?
Zitat von Tony Liu <tonyliu0592(a)hotmail.com>om>:
Hi,
Other than get all objects of the pool and filter by image ID,
is there any easier way to get the number of allocated objects for
a RBD image?
What I really want to know is the actual usage of an image.
An allocated object could be used partially, but that's fine,
no need to be 100% accurate. To get the object count and
times object size, that should be sufficient.
"rbd export" exports actual used data, but to get the actual usage
by exporting the image seems too much. This brings up another
question, is there any way to know the export size before running it?
Thanks!
Tony
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
6 p.m.
Thank you Eugen! "rbd du" is it.
The used_size from "rbd du" is object count times object size.
That's the actual storage taken by the image in backend.
For export, it actually flattens and also sparsifies the image.
In case of many small data pieces, the export size is smaller than du size.
Thanks!
Tony
________________________________________
From: Eugen Block <eblock(a)nde.ag>
Sent: November 25, 2023 12:17 AM
To: ceph-users(a)ceph.io
Subject: [ceph-users] Re: easy way to find out the number of allocated objects for a RBD
image
Maybe I misunderstand, but isn’t ’rbd du‘ what you're looking for?
Zitat von Tony Liu <tonyliu0592(a)hotmail.com>om>:
Hi,
Other than get all objects of the pool and filter by image ID,
is there any easier way to get the number of allocated objects for
a RBD image?
What I really want to know is the actual usage of an image.
An allocated object could be used partially, but that's fine,
no need to be 100% accurate. To get the object count and
times object size, that should be sufficient.
"rbd export" exports actual used data, but to get the actual usage
by exporting the image seems too much. This brings up another
question, is there any way to know the export size before running it?
Thanks!
Tony
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
6:39 p.m.
If there’s a filesystem on the volume, running fstrim or mounting with the discard option
might significantly reduce usage and block count.
> On Nov 25, 2023, at 1:02 PM, Tony Liu <tonyliu0592(a)hotmail.com> wrote:
>
> Thank you Eugen! "rbd du" is it.
> The used_size from "rbd du" is object count times object size.
> That's the actual storage taken by the image in backend.
>
> For export, it actually flattens and also sparsifies the image.
> In case of many small data pieces, the export size is smaller than du size.
>
>
> Thanks!
> Tony
> ________________________________________
> From: Eugen Block <eblock(a)nde.ag>
> Sent: November 25, 2023 12:17 AM
> To: ceph-users(a)ceph.io
> Subject: [ceph-users] Re: easy way to find out the number of allocated objects for a
RBD image
>
> Maybe I misunderstand, but isn’t ’rbd du‘ what you're looking for?
>
> Zitat von Tony Liu <tonyliu0592(a)hotmail.com>om>:
>
>> Hi,
>>
>> Other than get all objects of the pool and filter by image ID,
>> is there any easier way to get the number of allocated objects for
>> a RBD image?
>>
>> What I really want to know is the actual usage of an image.
>> An allocated object could be used partially, but that's fine,
>> no need to be 100% accurate. To get the object count and
>> times object size, that should be sufficient.
>>
>> "rbd export" exports actual used data, but to get the actual usage
>> by exporting the image seems too much. This brings up another
>> question, is there any way to know the export size before running it?
>>
>>
>> Thanks!
>> Tony
>> _______________________________________________
>> ceph-users mailing list -- ceph-users(a)ceph.io
>> To unsubscribe send an email to ceph-users-leave(a)ceph.io
>
>
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
29 Jan
29 Jan
12:33 p.m.
On Sat, Nov 25, 2023 at 7:01 PM Tony Liu <tonyliu0592(a)hotmail.com> wrote:
Thank you Eugen! "rbd du" is it.
The used_size from "rbd du" is object count times object size.
That's the actual storage taken by the image in backend.
Somebody just quoted this sentence out of context, so I feel like
I need to elaborate.
Earlier in this thread, Tony said that
An allocated object could be used partially, but
that's fine,
no need to be 100% accurate. To get the object count and
times object size, that should be sufficient.
For this purpose, USED as reported by "rbd du" is fine. But it's not
the actual storage taken by the image on the OSDs. As a counterexample,
one could have an image with 100% USED (i.e. all objects allocated),
but with only 4096 bytes of actual storage taken per object, so "rbd
du" could be off by three orders of magnitude: 4k * number of allocated
objects vs 4M * number of allocated objects (assuming default object
size and ignoring replication factor).
"rbd du" really just gives an upper bound. Reporting a precise number
would be slow and currently there is no easy way to get it.
Thanks,
Ilya
>
> For export, it actually flattens and also sparsifies the image.
> In case of many small data pieces, the export size is smaller than du size.
>
>
> Thanks!
> Tony
> ________________________________________
> From: Eugen Block <eblock(a)nde.ag>
> Sent: November 25, 2023 12:17 AM
> To: ceph-users(a)ceph.io
> Subject: [ceph-users] Re: easy way to find out the number of allocated objects for a
RBD image
>
> Maybe I misunderstand, but isn’t ’rbd du‘ what you're looking for?
>
> Zitat von Tony Liu <tonyliu0592(a)hotmail.com>om>:
>
> > Hi,
> >
> > Other than get all objects of the pool and filter by image ID,
> > is there any easier way to get the number of allocated objects for
> > a RBD image?
> >
> > What I really want to know is the actual usage of an image.
> > An allocated object could be used partially, but that's fine,
> > no need to be 100% accurate. To get the object count and
> > times object size, that should be sufficient.
> >
> > "rbd export" exports actual used data, but to get the actual usage
> > by exporting the image seems too much. This brings up another
> > question, is there any way to know the export size before running it?
> >
> >
> > Thanks!
> > Tony
> > _______________________________________________
> > ceph-users mailing list -- ceph-users(a)ceph.io
> > To unsubscribe send an email to ceph-users-leave(a)ceph.io
>
>
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
101
days inactive
166
days old
4 comments
4 participants
participants (4)
-
Anthony D'Atri -
Eugen Block -
Ilya Dryomov -
Tony Liu