[Ceph-users] Re: MDS failing under load with large cache sizes

However, now my client processes are basically in constant I/O wait state and the CephFS is slow for everybody. After I restarted the copy job, I got around 4k reqs/s and then it went down to 100 reqs/s with everybody waiting their turn. So yes, it does seem to help, but it increases latency by a magnitude.

Addition: I reduced the number to 256K and the cache size started inflating instantly (with about 140 reqs/s). So I reset it to 512K and the cache size started reducing slowly, though with fewer reqs/s. So I guess it is solving the problem, but only by trading it off against severe latency issues (order of magnitude as we saw).