Might bumping vm.min_free_kbytes mitigate this dynamic?  The default is kind of parsimonious; I routinely set to 4GB.  

> On Jul 29, 2026, at 3:14 AM, Dan van der Ster via ceph-users <ceph-users@ceph.io> wrote:
> 
> On Wed, Jul 29, 2026 at 4:01 AM Gregory Farnum via ceph-users
> <ceph-users@ceph.io> wrote:
>>> On Tue, Jul 28, 2026 at 3:05 AM Janne Johansson via ceph-users
>>> <ceph-users@ceph.io> wrote:
>>> I think the idea was that if you mount cephfs on a box running OSDs or a
>>> mon daemon, and it gets into some kind of trouble like running out of
>>> memory or so, then both the client side and the "server" side might be
>>> competing for the same resources that aren't there, making it worse.
>> This is a classic problem in network filesystems:
>> 1) The kernel runs out of memory and needs to flush out buffered
>> writes to free up space
>> 2) The client sends out a network message, which comes back in to the server
>> 3) The server needs to allocate memory to handle the incoming network message
>> 4) ...stuck
>> RBD and CephFS are susceptible to this, if you mount the kernels on
>> the same hosts as the OSDs. But the odds of hitting it are a lot lower
>> on modern systems with their RAM capacities than in the past.
> 
> It's been a couple of years, but I was still hitting this deadlock on
> el8 kernels with CephFS / ceph-osd co-located alongside some HPC-type
> MPI jobs.
> It's like Greg wrote -- a large client read/write that comes back to
> an OSD on the same host deadlocks the box.
> 
> Cheers, Dan
> 
> 
> 
>> -Greg
>>> If you have separate OSD hosts and they are having issues, then cephfs
>>> clients will be slow, but not much more than that, and if clients are
>>> having resource dips, a separate cluster doesn't mind at all. I think that
>>> is where the non-recommendation comes from, in case you do combine them, it
>>> will be hard to get out of this situation, any cpu/ram given to the client
>>> side would cause more IO to be requested, from the server side that doesn't
>>> have it. In any case it might make it lots harder to diagnose which end
>>> actually caused the issue if both suffer at the same time.
>>> Den tis 28 juli 2026 kl 14:45 skrev Marek Szuba via ceph-users <
>>> ceph-users@ceph.io>:
>>>> Dear fellow Cephies,
>>>> A few weeks ago I came across in Tentacle documentation on docs.ceph.org
>>>> (and OF COURSE now that I'm trying to quote it, I cannot find it any
>>>> more) a mention that CephFS should not be mounted on systems serving
>>>> Ceph because of there being a risk of harmful interference in the
>>>> kernel. The context suggested this pertained to one-node Ceph clusters,
>>>> that said I do not believe that was explicitly stated. Therefore, I very
>>>> much hope someone will be able to enlighten me whether:
>>>> * this is indeed still the case with Tentacle,
>>>> * it applies to one-node clusters only or all Ceph clusters,
>>>> * it only applies to CephFS mounts using the kernel driver, or ceph-fuse
>>>> as well.
>>>> Thank you in advance!
>>>> --
>>>> MS
>>>> _______________________________________________
>>>> ceph-users mailing list -- ceph-users@ceph.io
>>>> To unsubscribe send an email to ceph-users-leave@ceph.io
>>> --
>>> May the most significant bit of your life be positive.
>>> _______________________________________________
>>> ceph-users mailing list -- ceph-users@ceph.io
>>> To unsubscribe send an email to ceph-users-leave@ceph.io
>> _______________________________________________
>> ceph-users mailing list -- ceph-users@ceph.io
>> To unsubscribe send an email to ceph-users-leave@ceph.io
> _______________________________________________
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-leave@ceph.io