Hi Seena, just to note  - this ticket might be relevant. https://tracker.ceph.com/issues/48276 Mind leaving a comment there? Thanks, Igor On 11/23/2020 2:51 AM, Seena Fallah wrote: > Now one of my OSDs gets segfault. > Here is the full trace: https://paste.ubuntu.com/p/4KHcCG9YQx/ > > On Mon, Nov 23, 2020 at 2:16 AM Seena Fallah <seenafallah(a)gmail.com> wrote: > >> Hi all, >> >> After I upgrade from 14.2.9 to 14.2.14 my OSDs are using less more memory >> than before! I give each OSD 6GB memory target and before the free memory >> was 20GB and now after 24h from the upgrade I have 104GB free memory of >> 128GB memory! Also, my OSD latency got increases! >> This happens in both SSD and HDD tier. >> >> Are there any notes from the upgrade I missed? Is it related to >> bluefs_buffered_io? >> If BlueFS do a direct IO shouldn't BlueFS/Bluestore use the targeted >> memory for its cache and does it mean before the upgrade the memory used >> was by a kernel that buffers the IO and wasn't for the ceph-osd? >> >> Thanks. >> > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io

This is what happens with my cluster (Screenshots attached). At 10:11 I turn on bluefs_buffered_io on all my OSDs and latency gets back but throughput decreases. I had these configs for all OSDs in recovery osd-max-backfills 1 osd-recovery-max-active 1 osd-recovery-op-priority 1 Do you have any idea why with these parameters latency got too much effect? On Tue, Nov 24, 2020 at 12:42 PM Seena Fallah <seenafallah(a)gmail.com> wrote: