Re: [ceph-users] out of memory bluestore osds

Hi Jaime, we only use the cache size parameters now if you've disabled autotuning.  With autotuning we adjust the cache size on the fly to try and keep the mapped process memory under the osd_memory_target.  You can set a lower memory target than default, though you will have far less cache for bluestore onodes and rocksdb.  You may notice that it's slower, especially if you have a big active data set you are processing.  I don't usually recommend setting the osd_memory_target below 2GB.  At some point it will have shrunk the caches as far as it can and the process memory may start exceeding the target.  (with our default rocksdb and pglog settings this usually happens somewhere between 1.3-1.7GB once the OSD has been sufficiently saturated with IO). Given memory prices right now, I'd still recommend upgrading RAM if you have the ability though.  You might be able to get away with setting each OSD to 2-2.5GB in your scenario but you'll be pushing it. I would not recommend lowering the osd_memory_cache_min.  You really want rocksdb indexes/filters fitting in cache, and as many bluestore onodes as you can get.  In any event, you'll still be bound by the (currently hardcoded) 64MB cache chunk allocation size in the autotuner which osd_memory_cache_min can't reduce (and that's per cache while osd_memory_cache_min is global for the kv,buffer, and rocksdb block caches).  IE each cache is going to get 64MB+growth room regardless of how low you set osd_memory_cache_min.  That's intentional as we don't want a single SST file in rocksdb to be able to completely blow everything else out of the block cache during compaction, only to quickly become invalid, removed from the cache, and make it look to the priority cache system like rocksdb doesn't actually need any more memory for cache. Mark On 8/7/19 7:44 AM, Jaime Ibar wrote: