Bluestore compression parameters in ceph.conf not used in mimic 13.2.8?
Dear all, I have two observations regarding bluestore compression config: 1) ceph.conf settings seem to be ignored. 2) The SSD default values seem not to save space using compression. To 1) We are running a mimic 13.2.8 cluster with OSDs deployed under mimic 13.2.2. Back then the interpretation of compression parameters was messed up, which has been fixed along the way from 13.2.2 to 13.2.8. To get it to work properly under 13.2.2 I needed to include these settings in ceph.conf: [osd] bluestore compression mode = aggressive bluestore compression min blob size hdd = 262144 and then also enable compression on all pools that should use compression. These settings are still present in ceph.conf, but they seem to be ignored when populating the config data base on mon startup or querying config parameters: # ceph config get osd.16 bluestore_compression_min_blob_size_hdd 131072 However: # ceph tell osd.16 config get bluestore_compression_min_blob_size_hdd 262144 and: # ceph config show osd.16 NAME VALUE SOURCE OVERRIDES IGNORES bluestore_compression_min_blob_size_hdd 262144 file This is really confusing. Is this intended? Which values will be used when deploying new OSDs? In general, it would really be helpful if one could query daemon/parameter groups as in " ceph config get osd bluestore_compression_min_blob_size_hdd" to get a list right away. To 2) In a long-long-ago discussion about how compression works, I was told that a blob of bluestore_compression_min_blob_size will be compressed and then distributed over a number of allocations of bluestore_min_alloc_size. The defaults for HDD and SSD are: bluestore_compression_min_blob_size_hdd 131072 bluestore_min_alloc_size_hdd 65536 bluestore_compression_min_blob_size_ssd 8192 bluestore_min_alloc_size_ssd 16384 If this explanation of the compression method is correct, these defaults allow up to 50% savings for HDD, but, erm, 0% for SSD as the uncompressed blob will use the same amount of space as the compressed one as both will require the same allocation size. Did something change here? Are compressed blobs now co-located in allocations? Thanks for your help, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14
Hi Frank, answering the second part. The following settings look senseless indeed: bluestore_compression_min_blob_size_ssd 8192 bluestore_min_alloc_size_ssd 16384 Presumably this was an incomplete backport from Nautilus which has proper numbers: 32K and 16K respectively. Fell free to create a ticket. Thanks, Igor On 4/1/2020 12:44 PM, Frank Schilder wrote:
Dear all,
I have two observations regarding bluestore compression config:
1) ceph.conf settings seem to be ignored. 2) The SSD default values seem not to save space using compression.
To 1) We are running a mimic 13.2.8 cluster with OSDs deployed under mimic 13.2.2. Back then the interpretation of compression parameters was messed up, which has been fixed along the way from 13.2.2 to 13.2.8. To get it to work properly under 13.2.2 I needed to include these settings in ceph.conf:
[osd] bluestore compression mode = aggressive bluestore compression min blob size hdd = 262144
and then also enable compression on all pools that should use compression. These settings are still present in ceph.conf, but they seem to be ignored when populating the config data base on mon startup or querying config parameters:
# ceph config get osd.16 bluestore_compression_min_blob_size_hdd 131072
However:
# ceph tell osd.16 config get bluestore_compression_min_blob_size_hdd 262144
and:
# ceph config show osd.16 NAME VALUE SOURCE OVERRIDES IGNORES bluestore_compression_min_blob_size_hdd 262144 file
This is really confusing. Is this intended? Which values will be used when deploying new OSDs?
In general, it would really be helpful if one could query daemon/parameter groups as in " ceph config get osd bluestore_compression_min_blob_size_hdd" to get a list right away.
To 2) In a long-long-ago discussion about how compression works, I was told that a blob of bluestore_compression_min_blob_size will be compressed and then distributed over a number of allocations of bluestore_min_alloc_size. The defaults for HDD and SSD are:
bluestore_compression_min_blob_size_hdd 131072 bluestore_min_alloc_size_hdd 65536
bluestore_compression_min_blob_size_ssd 8192 bluestore_min_alloc_size_ssd 16384
If this explanation of the compression method is correct, these defaults allow up to 50% savings for HDD, but, erm, 0% for SSD as the uncompressed blob will use the same amount of space as the compressed one as both will require the same allocation size.
Did something change here? Are compressed blobs now co-located in allocations?
Thanks for your help, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Dear Igor, thanks, done: https://tracker.ceph.com/issues/44878 . ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Igor Fedotov <ifedotov@suse.de> Sent: 01 April 2020 12:14:02 To: Frank Schilder; ceph-users Subject: Re: [ceph-users] Bluestore compression parameters in ceph.conf not used in mimic 13.2.8? Hi Frank, answering the second part. The following settings look senseless indeed: bluestore_compression_min_blob_size_ssd 8192 bluestore_min_alloc_size_ssd 16384 Presumably this was an incomplete backport from Nautilus which has proper numbers: 32K and 16K respectively. Fell free to create a ticket. Thanks, Igor On 4/1/2020 12:44 PM, Frank Schilder wrote:
Dear all,
I have two observations regarding bluestore compression config:
1) ceph.conf settings seem to be ignored. 2) The SSD default values seem not to save space using compression.
To 1) We are running a mimic 13.2.8 cluster with OSDs deployed under mimic 13.2.2. Back then the interpretation of compression parameters was messed up, which has been fixed along the way from 13.2.2 to 13.2.8. To get it to work properly under 13.2.2 I needed to include these settings in ceph.conf:
[osd] bluestore compression mode = aggressive bluestore compression min blob size hdd = 262144
and then also enable compression on all pools that should use compression. These settings are still present in ceph.conf, but they seem to be ignored when populating the config data base on mon startup or querying config parameters:
# ceph config get osd.16 bluestore_compression_min_blob_size_hdd 131072
However:
# ceph tell osd.16 config get bluestore_compression_min_blob_size_hdd 262144
and:
# ceph config show osd.16 NAME VALUE SOURCE OVERRIDES IGNORES bluestore_compression_min_blob_size_hdd 262144 file
This is really confusing. Is this intended? Which values will be used when deploying new OSDs?
In general, it would really be helpful if one could query daemon/parameter groups as in " ceph config get osd bluestore_compression_min_blob_size_hdd" to get a list right away.
To 2) In a long-long-ago discussion about how compression works, I was told that a blob of bluestore_compression_min_blob_size will be compressed and then distributed over a number of allocations of bluestore_min_alloc_size. The defaults for HDD and SSD are:
bluestore_compression_min_blob_size_hdd 131072 bluestore_min_alloc_size_hdd 65536
bluestore_compression_min_blob_size_ssd 8192 bluestore_min_alloc_size_ssd 16384
If this explanation of the compression method is correct, these defaults allow up to 50% savings for HDD, but, erm, 0% for SSD as the uncompressed blob will use the same amount of space as the compressed one as both will require the same allocation size.
Did something change here? Are compressed blobs now co-located in allocations?
Thanks for your help, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (2)
-
Frank Schilder
-
Igor Fedotov