Anthony D'Atri wrote:
"swap should be double your physical memory size <https://superuser.com/a/111510/98210>"
That smells like an idea from the days of BSD-style swap, where swap backed rather than extended physmem.
It is getting a little off-topic, but - I used to have Linux servers configured according to the above rule, and it worked. The difference was that even with 16 MB of RAM (yes, megabytes) and 32 MB of swap the system could be quite usable and handle memory usage spikes well. After all, it was possible to swap the whole RAM out and in under one second (with sequential access, of course). Today, some of my servers have 1.5 TB of RAM, and swapping the whole RAM out and in would take many minutes even on fast NVMe, not speaking about HDDs. So, having terabytes of swap space does not make sense. Swap should be used to enhance performance by paging out the idle pages and having a bit more avaliable to caching or whatever. Today's Zram serves more-or-less the same purpose, compressing unused pages instead of swapping them out.
For my workload, the bottleneck (if any) are the HDDs, so NVMes have lots of idle time, even with some WAL and OMAP traffic hitting them.
Note that instate %util has little meaning on any OSD. And there's more to the equation than "idle time".
Yes. Latency, for example. On the other hand, NVMes are quite parallel (and even HDDs use parallelization with TCQ/NCQ to some extent), so even having the storage busy 100 % of the time does not necessarily mean its _throughput_ is maxed out. -Yenya -- | Jan "Yenya" Kasprzak <kas at {fi.muni.cz - work | yenya.net - private}> | | https://www.fi.muni.cz/~kas/ GPG: 4096R/A45477D5 | We all agree on the necessity of compromise. We just can't agree on when it's necessary to compromise. --Larry Wall