On Wed, Oct 2, 2019 at 3:41 PM Paul Emmerich <paul.emmerich@croit.io> wrote:
On Wed, Oct 2, 2019 at 10:56 PM Robert LeBlanc <robert@leblancnet.us> wrote:
Is there a way to have leveldb compact more frequently or cause it to come up for air more frequently and respond to heartbeats and process some IO?
you can manually trigger a compaction via the admin socket (or was it via ceph tell?) with the compact command, but I don't think that this helps with your workload.
This would be really helpful, I've just been restarting OSD processes. After some digging, it looks like it was added to Luminous [1]. :(
I thought splitting PGs would help, but we are still seeing the problem (previously ~20 PGs per OSD to now ~150). I still have some space on the SSDs that I can double, almost triple the journal, but not sure if that will help in this situation.
no, a larger journal will not help for leveldb workloads.
Big difference between FileStore journals and BlueStore DB devices: BlueStore actually puts all the metadata onto the SSD permanently, a FileStore journal is just a journal (and 5 GB is large enough, it won't use that much space for small operations like deletions).
(The answer that you don't want to hear is probably the best way forward: upgrade to BlueStore)
We are planning to replace this cluster in ~6 months, so we don't want to upgrade it. I'm just looking for things that I may have missed that will keep the pager quieter until then. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1441693 ---------------- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1