16 Aug
2023
16 Aug
'23
5:46 p.m.
Hi,
With Pacific and afterwards, which of the following two settings takes precedence?
bluestore_compression_min_blob_size (default 0)
bluestore_compression_min_blob_size_<type> (default 8192)
I'd assume the more specific setting takes precedence?
What's this "blob"? Is it write block?
Then, there is no compression for 4K write block?
I was not able to find it out from doc, hope someone could help to clarify.
Thanks!
Tony
18 Aug
18 Aug
6 a.m.
Hi,
First priority for compression blob is per-pool setting (taken if set)
# ceph osd pool get test compression_min_blob_size
Second, if set, bluestore_compression_min_blob_size config,
Third is selected from *_hdd *_ssd depending on the main block device in
the OSD being HDD or SSD.
This can be forced by setting bluestore_debug_enforce_settings to either
'hdd' or 'ssd' if your hardware improperly reports
# cat /sys/block/sda/queue/rotational
0
Blob is an organization unit in BlueStore
https://github.com/ceph/ceph/pull/51933
You can get compression for blocks smaller than 4K only if your main device
has native block size smaller.
# cat /sys/block/nvme1n1/queue/physical_block_size
512
In such a case you can set bluestore_min_alloc_size (in effect only during
mkfs) to some value smaller than 4K, but multiply of 512.
This path is not often tested, and you need to own unexpected side effects.
Best regards,
Adam Kupczyk
On Wed, 16 Aug 2023 at 19:46, Tony Liu <tonyliu0592(a)hotmail.com> wrote:
Hi,
With Pacific and afterwards, which of the following two settings takes
precedence?
bluestore_compression_min_blob_size (default 0)
bluestore_compression_min_blob_size_<type> (default 8192)
I'd assume the more specific setting takes precedence?
What's this "blob"? Is it write block?
Then, there is no compression for 4K write block?
I was not able to find it out from doc, hope someone could help to clarify.
Thanks!
Tony
_______________________________________________
Dev mailing list -- dev(a)ceph.io
To unsubscribe send an email to dev-leave(a)ceph.io
5:54 p.m.
Hi Adam,
This priority order is different from common sense, where specific setting has higher
priority than generic setting. With this order, compression_min_blob_size has higher
priority, then what's the point of having _hdd or _ssd settings, which will never get
applied?
Thanks!
Tony
________________________________________
From: Adam Kupczyk <akupczyk(a)redhat.com>
Sent: August 17, 2023 11:00 PM
To: Tony Liu
Cc: dev(a)ceph.io
Subject: Re: bluestore_compression_min_blob_size vs.
bluestore_compression_min_blob_size_<type>
Hi,
First priority for compression blob is per-pool setting (taken if set)
# ceph osd pool get test compression_min_blob_size
Second, if set, bluestore_compression_min_blob_size config,
Third is selected from *_hdd *_ssd depending on the main block device in the OSD being HDD
or SSD.
This can be forced by setting bluestore_debug_enforce_settings to either 'hdd' or
'ssd' if your hardware improperly reports
# cat /sys/block/sda/queue/rotational
0
Blob is an organization unit in BlueStore
https://github.com/ceph/ceph/pull/51933
You can get compression for blocks smaller than 4K only if your main device has native
block size smaller.
# cat /sys/block/nvme1n1/queue/physical_block_size
512
In such a case you can set bluestore_min_alloc_size (in effect only during mkfs) to some
value smaller than 4K, but multiply of 512.
This path is not often tested, and you need to own unexpected side effects.
Best regards,
Adam Kupczyk
On Wed, 16 Aug 2023 at 19:46, Tony Liu
<tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com>> wrote:
Hi,
With Pacific and afterwards, which of the following two settings takes precedence?
bluestore_compression_min_blob_size (default 0)
bluestore_compression_min_blob_size_<type> (default 8192)
I'd assume the more specific setting takes precedence?
What's this "blob"? Is it write block?
Then, there is no compression for 4K write block?
I was not able to find it out from doc, hope someone could help to clarify.
Thanks!
Tony
_______________________________________________
Dev mailing list -- dev@ceph.io<mailto:dev@ceph.io>
To unsubscribe send an email to dev-leave@ceph.io<mailto:dev-leave@ceph.io>
6:11 p.m.
Indeed, I would have thought that the unqualified setting would only ever be derived from
_hdd / _ssd. I believe we have other sets of options that behave that way.
On Aug 18, 2023, at 13:54, Tony Liu
<tonyliu0592(a)hotmail.com> wrote:
Hi Adam,
This priority order is different from common sense, where specific setting has higher
priority than generic setting. With this order, compression_min_blob_size has higher
priority, then what's the point of having _hdd or _ssd settings, which will never get
applied?
Thanks!
Tony
________________________________________
From: Adam Kupczyk <akupczyk(a)redhat.com>
Sent: August 17, 2023 11:00 PM
To: Tony Liu
Cc: dev(a)ceph.io
Subject: Re: bluestore_compression_min_blob_size vs.
bluestore_compression_min_blob_size_<type>
Hi,
First priority for compression blob is per-pool setting (taken if set)
# ceph osd pool get test compression_min_blob_size
Second, if set, bluestore_compression_min_blob_size config,
Third is selected from *_hdd *_ssd depending on the main block device in the OSD being
HDD or SSD.
This can be forced by setting bluestore_debug_enforce_settings to either 'hdd' or
'ssd' if your hardware improperly reports
# cat /sys/block/sda/queue/rotational
0
Blob is an organization unit in BlueStore
https://github.com/ceph/ceph/pull/51933
You can get compression for blocks smaller than 4K only if your main device has native
block size smaller.
# cat /sys/block/nvme1n1/queue/physical_block_size
512
In such a case you can set bluestore_min_alloc_size (in effect only during mkfs) to some
value smaller than 4K, but multiply of 512.
This path is not often tested, and you need to own unexpected side effects.
Best regards,
Adam Kupczyk
On Wed, 16 Aug 2023 at 19:46, Tony Liu
<tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com>> wrote:
Hi,
With Pacific and afterwards, which of the following two settings takes precedence?
bluestore_compression_min_blob_size (default 0)
bluestore_compression_min_blob_size_<type> (default 8192)
I'd assume the more specific setting takes precedence?
What's this "blob"? Is it write block?
Then, there is no compression for 4K write block?
I was not able to find it out from doc, hope someone could help to clarify.
Thanks!
Tony
_______________________________________________
Dev mailing list -- dev@ceph.io<mailto:dev@ceph.io>
To unsubscribe send an email to dev-leave@ceph.io<mailto:dev-leave@ceph.io>
_______________________________________________
Dev mailing list -- dev(a)ceph.io
To unsubscribe send an email to dev-leave(a)ceph.io
29 Aug
29 Aug
8:04 a.m.
Hi Tony,
Initially:
1) values for compression min and max blob size are taken from
compression_(min/max)_blob_size_(ssd/hdd)
but if value of compression_(min_max)_blob_size is set to non-0
2) that value is used as compression blob size in bluestore
but if a pool has set compression_(min/max)_blob_size setting
3) that value is used as compression blob size
I do not understand why you think of it as inverted priority.
The most generic case ceph tries to guess the best blob size by type of
media hw.
Once you know your hw you can override blob sizes using your knowledge.
And even further - you can have different expectations for blob sizes
depending on type of data you have.
For a pool with large unchanging objects (RGW?) you could choose a very
large blob size.
For a pool where you expect small accesses (RBD?) you probably want a
smaller blob.
Best regards,
Adam Kupczyk
On Fri, 18 Aug 2023 at 19:54, Tony Liu <tonyliu0592(a)hotmail.com> wrote:
Hi Adam,
This priority order is different from common sense, where specific setting
has higher
priority than generic setting. With this order, compression_min_blob_size
has higher
priority, then what's the point of having _hdd or _ssd settings, which
will never get applied?
Thanks!
Tony
________________________________________
From: Adam Kupczyk <akupczyk(a)redhat.com>
Sent: August 17, 2023 11:00 PM
To: Tony Liu
Cc: dev(a)ceph.io
Subject: Re: bluestore_compression_min_blob_size vs.
bluestore_compression_min_blob_size_<type>
Hi,
First priority for compression blob is per-pool setting (taken if set)
# ceph osd pool get test compression_min_blob_size
Second, if set, bluestore_compression_min_blob_size config,
Third is selected from *_hdd *_ssd depending on the main block device in
the OSD being HDD or SSD.
This can be forced by setting bluestore_debug_enforce_settings to either
'hdd' or 'ssd' if your hardware improperly reports
# cat /sys/block/sda/queue/rotational
0
Blob is an organization unit in BlueStore
https://github.com/ceph/ceph/pull/51933
You can get compression for blocks smaller than 4K only if your main
device has native block size smaller.
# cat /sys/block/nvme1n1/queue/physical_block_size
512
In such a case you can set bluestore_min_alloc_size (in effect only during
mkfs) to some value smaller than 4K, but multiply of 512.
This path is not often tested, and you need to own unexpected side effects.
Best regards,
Adam Kupczyk
On Wed, 16 Aug 2023 at 19:46, Tony Liu <tonyliu0592(a)hotmail.com<mailtomailto:
tonyliu0592(a)hotmail.com>> wrote:
Hi,
With Pacific and afterwards, which of the following two settings takes
precedence?
bluestore_compression_min_blob_size (default 0)
bluestore_compression_min_blob_size_<type> (default 8192)
I'd assume the more specific setting takes precedence?
What's this "blob"? Is it write block?
Then, there is no compression for 4K write block?
I was not able to find it out from doc, hope someone could help to clarify.
Thanks!
Tony
_______________________________________________
Dev mailing list -- dev@ceph.io<mailto:dev@ceph.io>
To unsubscribe send an email to dev-leave@ceph.io<mailto:dev-leave@ceph.io
4:28 p.m.
Hi Adam,
I've been focus on global compression, not per pool.
In case of global compression, only compression_(min/max)_blob_size_(ssd/hdd)
takes effect, compression_(min_max)_blob_size has no effect to global compression
at all. Is that correct?
If yes, it makes sense, but I'd name it compression_(min_max)_blob_size_per_pool
or something more clear.
Thanks!
Tony
________________________________________
From: Adam Kupczyk <akupczyk(a)redhat.com>
Sent: August 29, 2023 01:04 AM
To: Tony Liu
Cc: dev(a)ceph.io
Subject: Re: bluestore_compression_min_blob_size vs.
bluestore_compression_min_blob_size_<type>
Hi Tony,
Initially:
1) values for compression min and max blob size are taken from
compression_(min/max)_blob_size_(ssd/hdd)
but if value of compression_(min_max)_blob_size is set to non-0
2) that value is used as compression blob size in bluestore
but if a pool has set compression_(min/max)_blob_size setting
3) that value is used as compression blob size
I do not understand why you think of it as inverted priority.
The most generic case ceph tries to guess the best blob size by type of media hw.
Once you know your hw you can override blob sizes using your knowledge.
And even further - you can have different expectations for blob sizes depending on type of
data you have.
For a pool with large unchanging objects (RGW?) you could choose a very large blob size.
For a pool where you expect small accesses (RBD?) you probably want a smaller blob.
Best regards,
Adam Kupczyk
On Fri, 18 Aug 2023 at 19:54, Tony Liu
<tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com>> wrote:
Hi Adam,
This priority order is different from common sense, where specific setting has higher
priority than generic setting. With this order, compression_min_blob_size has higher
priority, then what's the point of having _hdd or _ssd settings, which will never get
applied?
Thanks!
Tony
________________________________________
From: Adam Kupczyk <akupczyk@redhat.com<mailto:akupczyk@redhat.com>>
Sent: August 17, 2023 11:00 PM
To: Tony Liu
Cc: dev@ceph.io<mailto:dev@ceph.io>
Subject: Re: bluestore_compression_min_blob_size vs.
bluestore_compression_min_blob_size_<type>
Hi,
First priority for compression blob is per-pool setting (taken if set)
# ceph osd pool get test compression_min_blob_size
Second, if set, bluestore_compression_min_blob_size config,
Third is selected from *_hdd *_ssd depending on the main block device in the OSD being HDD
or SSD.
This can be forced by setting bluestore_debug_enforce_settings to either 'hdd' or
'ssd' if your hardware improperly reports
# cat /sys/block/sda/queue/rotational
0
Blob is an organization unit in BlueStore
https://github.com/ceph/ceph/pull/51933
You can get compression for blocks smaller than 4K only if your main device has native
block size smaller.
# cat /sys/block/nvme1n1/queue/physical_block_size
512
In such a case you can set bluestore_min_alloc_size (in effect only during mkfs) to some
value smaller than 4K, but multiply of 512.
This path is not often tested, and you need to own unexpected side effects.
Best regards,
Adam Kupczyk
On Wed, 16 Aug 2023 at 19:46, Tony Liu
<tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com><mailto:tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com>>>
wrote:
Hi,
With Pacific and afterwards, which of the following two settings takes precedence?
bluestore_compression_min_blob_size (default 0)
bluestore_compression_min_blob_size_<type> (default 8192)
I'd assume the more specific setting takes precedence?
What's this "blob"? Is it write block?
Then, there is no compression for 4K write block?
I was not able to find it out from doc, hope someone could help to clarify.
Thanks!
Tony
_______________________________________________
Dev mailing list --
dev@ceph.io<mailto:dev@ceph.io><mailto:dev@ceph.io<mailto:dev@ceph.io>>
To unsubscribe send an email to
dev-leave@ceph.io<mailto:dev-leave@ceph.io><mailto:dev-leave@ceph.io<mailto:dev-leave@ceph.io>>
4:56 p.m.
Hi Tony,
Originally, there existed only compression_(min_max)_blob_size settings.
Later we realized that ssd and hdd fare better with distinct values.
In result compression_(min/max)_blob_size_(ssd/hdd) settings to populate
compression_(min/max)_blob_size were created.
Best regards,
Adam
On Tue, 29 Aug 2023 at 18:28, Tony Liu <tonyliu0592(a)hotmail.com> wrote:
Hi Adam,
I've been focus on global compression, not per pool.
In case of global compression, only
compression_(min/max)_blob_size_(ssd/hdd)
takes effect, compression_(min_max)_blob_size has no effect to global
compression
at all. Is that correct?
If yes, it makes sense, but I'd name it
compression_(min_max)_blob_size_per_pool
or something more clear.
Thanks!
Tony
________________________________________
From: Adam Kupczyk <akupczyk(a)redhat.com>
Sent: August 29, 2023 01:04 AM
To: Tony Liu
Cc: dev(a)ceph.io
Subject: Re: bluestore_compression_min_blob_size vs.
bluestore_compression_min_blob_size_<type>
Hi Tony,
Initially:
1) values for compression min and max blob size are taken from
compression_(min/max)_blob_size_(ssd/hdd)
but if value of compression_(min_max)_blob_size is set to non-0
2) that value is used as compression blob size in bluestore
but if a pool has set compression_(min/max)_blob_size setting
3) that value is used as compression blob size
I do not understand why you think of it as inverted priority.
The most generic case ceph tries to guess the best blob size by type of
media hw.
Once you know your hw you can override blob sizes using your knowledge.
And even further - you can have different expectations for blob sizes
depending on type of data you have.
For a pool with large unchanging objects (RGW?) you could choose a very
large blob size.
For a pool where you expect small accesses (RBD?) you probably want a
smaller blob.
Best regards,
Adam Kupczyk
On Fri, 18 Aug 2023 at 19:54, Tony Liu <tonyliu0592(a)hotmail.com<mailtomailto:
tonyliu0592(a)hotmail.com>> wrote:
Hi Adam,
This priority order is different from common sense, where specific setting
has higher
priority than generic setting. With this order, compression_min_blob_size
has higher
priority, then what's the point of having _hdd or _ssd settings, which
will never get applied?
Thanks!
Tony
________________________________________
From: Adam Kupczyk <akupczyk@redhat.com<mailto:akupczyk@redhat.com>>
Sent: August 17, 2023 11:00 PM
To: Tony Liu
Cc: dev@ceph.io<mailto:dev@ceph.io>
Subject: Re: bluestore_compression_min_blob_size vs.
bluestore_compression_min_blob_size_<type>
Hi,
First priority for compression blob is per-pool setting (taken if set)
# ceph osd pool get test compression_min_blob_size
Second, if set, bluestore_compression_min_blob_size config,
Third is selected from *_hdd *_ssd depending on the main block device in
the OSD being HDD or SSD.
This can be forced by setting bluestore_debug_enforce_settings to either
'hdd' or 'ssd' if your hardware improperly reports
# cat /sys/block/sda/queue/rotational
0
Blob is an organization unit in BlueStore
https://github.com/ceph/ceph/pull/51933
You can get compression for blocks smaller than 4K only if your main
device has native block size smaller.
# cat /sys/block/nvme1n1/queue/physical_block_size
512
In such a case you can set bluestore_min_alloc_size (in effect only during
mkfs) to some value smaller than 4K, but multiply of 512.
This path is not often tested, and you need to own unexpected side effects.
Best regards,
Adam Kupczyk
On Wed, 16 Aug 2023 at 19:46, Tony Liu <tonyliu0592(a)hotmail.com<mailtomailto:
tonyliu0592@hotmail.com><mailto:tonyliu0592@hotmail.com<mailto:
tonyliu0592(a)hotmail.com>>> wrote:
Hi,
With Pacific and afterwards, which of the following two settings takes
precedence?
bluestore_compression_min_blob_size (default 0)
bluestore_compression_min_blob_size_<type> (default 8192)
I'd assume the more specific setting takes precedence?
What's this "blob"? Is it write block?
Then, there is no compression for 4K write block?
I was not able to find it out from doc, hope someone could help to clarify.
Thanks!
Tony
_______________________________________________
Dev mailing list -- dev@ceph.io<mailto:dev@ceph.io><mailto:dev@ceph.io
<mailto:dev@ceph.io>>
To unsubscribe send an email to dev-leave@ceph.io<mailto:dev-leave@ceph.io
<mailto:dev-leave@ceph.io<mailto:dev-leave@ceph.io>>
6:27 p.m.
Hi Adam,
With the following configurations,
compression_min_blob_size 4096
compression_min_blob_size_ssd 8192
which is taken by Ceph for global compression on SSD device?
Thanks!
Tony
________________________________________
From: Adam Kupczyk <akupczyk(a)redhat.com>
Sent: August 29, 2023 09:56 AM
To: Tony Liu
Cc: dev(a)ceph.io
Subject: Re: bluestore_compression_min_blob_size vs.
bluestore_compression_min_blob_size_<type>
Hi Tony,
Originally, there existed only compression_(min_max)_blob_size settings.
Later we realized that ssd and hdd fare better with distinct values.
In result compression_(min/max)_blob_size_(ssd/hdd) settings to populate
compression_(min/max)_blob_size were created.
Best regards,
Adam
On Tue, 29 Aug 2023 at 18:28, Tony Liu
<tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com>> wrote:
Hi Adam,
I've been focus on global compression, not per pool.
In case of global compression, only compression_(min/max)_blob_size_(ssd/hdd)
takes effect, compression_(min_max)_blob_size has no effect to global compression
at all. Is that correct?
If yes, it makes sense, but I'd name it compression_(min_max)_blob_size_per_pool
or something more clear.
Thanks!
Tony
________________________________________
From: Adam Kupczyk <akupczyk@redhat.com<mailto:akupczyk@redhat.com>>
Sent: August 29, 2023 01:04 AM
To: Tony Liu
Cc: dev@ceph.io<mailto:dev@ceph.io>
Subject: Re: bluestore_compression_min_blob_size vs.
bluestore_compression_min_blob_size_<type>
Hi Tony,
Initially:
1) values for compression min and max blob size are taken from
compression_(min/max)_blob_size_(ssd/hdd)
but if value of compression_(min_max)_blob_size is set to non-0
2) that value is used as compression blob size in bluestore
but if a pool has set compression_(min/max)_blob_size setting
3) that value is used as compression blob size
I do not understand why you think of it as inverted priority.
The most generic case ceph tries to guess the best blob size by type of media hw.
Once you know your hw you can override blob sizes using your knowledge.
And even further - you can have different expectations for blob sizes depending on type of
data you have.
For a pool with large unchanging objects (RGW?) you could choose a very large blob size.
For a pool where you expect small accesses (RBD?) you probably want a smaller blob.
Best regards,
Adam Kupczyk
On Fri, 18 Aug 2023 at 19:54, Tony Liu
<tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com><mailto:tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com>>>
wrote:
Hi Adam,
This priority order is different from common sense, where specific setting has higher
priority than generic setting. With this order, compression_min_blob_size has higher
priority, then what's the point of having _hdd or _ssd settings, which will never get
applied?
Thanks!
Tony
________________________________________
From: Adam Kupczyk
<akupczyk@redhat.com<mailto:akupczyk@redhat.com><mailto:akupczyk@redhat.com<mailto:akupczyk@redhat.com>>>
Sent: August 17, 2023 11:00 PM
To: Tony Liu
Cc:
dev@ceph.io<mailto:dev@ceph.io><mailto:dev@ceph.io<mailto:dev@ceph.io>>
Subject: Re: bluestore_compression_min_blob_size vs.
bluestore_compression_min_blob_size_<type>
Hi,
First priority for compression blob is per-pool setting (taken if set)
# ceph osd pool get test compression_min_blob_size
Second, if set, bluestore_compression_min_blob_size config,
Third is selected from *_hdd *_ssd depending on the main block device in the OSD being HDD
or SSD.
This can be forced by setting bluestore_debug_enforce_settings to either 'hdd' or
'ssd' if your hardware improperly reports
# cat /sys/block/sda/queue/rotational
0
Blob is an organization unit in BlueStore
https://github.com/ceph/ceph/pull/51933
You can get compression for blocks smaller than 4K only if your main device has native
block size smaller.
# cat /sys/block/nvme1n1/queue/physical_block_size
512
In such a case you can set bluestore_min_alloc_size (in effect only during mkfs) to some
value smaller than 4K, but multiply of 512.
This path is not often tested, and you need to own unexpected side effects.
Best regards,
Adam Kupczyk
On Wed, 16 Aug 2023 at 19:46, Tony Liu
<tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com><mailto:tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com>><mailto:tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com><mailto:tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com>>>>
wrote:
Hi,
With Pacific and afterwards, which of the following two settings takes precedence?
bluestore_compression_min_blob_size (default 0)
bluestore_compression_min_blob_size_<type> (default 8192)
I'd assume the more specific setting takes precedence?
What's this "blob"? Is it write block?
Then, there is no compression for 4K write block?
I was not able to find it out from doc, hope someone could help to clarify.
Thanks!
Tony
_______________________________________________
Dev mailing list --
dev@ceph.io<mailto:dev@ceph.io><mailto:dev@ceph.io<mailto:dev@ceph.io>><mailto:dev@ceph.io<mailto:dev@ceph.io><mailto:dev@ceph.io<mailto:dev@ceph.io>>>
To unsubscribe send an email to
dev-leave@ceph.io<mailto:dev-leave@ceph.io><mailto:dev-leave@ceph.io<mailto:dev-leave@ceph.io>><mailto:dev-leave@ceph.io<mailto:dev-leave@ceph.io><mailto:dev-leave@ceph.io<mailto:dev-leave@ceph.io>>>
253
days inactive
266
days old
7 comments
3 participants
participants (3)
-
Adam Kupczyk -
Anthony D'Atri -
Tony Liu