Le 11/04/2026 01:46 CEST, Anthony D'Atri <anthony.datri@gmail.com> a écrit :
,
Reads are always served from the slow device, and writes over a certain size go there directly.
My understanding from dm-cache docs was that no, in plain dm-cache, reads hit the cache drive. For the behavior you describe (only writes use the cache), you'd need dm-writecache, which is another mode for dm-cache.
Clarify: I was talking about conventional offloaded WAL+DB, not dm-cache.
Ah, indeed, my bad. We have the same understanding! I even tried to improve the situation by increasing the deferred size, which helped a tiny bit. Tiny.
As for the write size, I guess it's configurable, but I don't think it'll be an issue for my case: what I'm after is preventing the constant chatter or small random writes that hit the drive with a sustained workload. The occasional big write should not matter too much.
That’s what I was after. One potential strategy is fronting coarse-IU QLC SSDs, in which case one would reeeealy want the writes to the backing device to be an aligned, even multiple of the IU.
Which is why I have high hopes for dm-writecache in the long run. Hit high watermark, flush data in a as-much sequential and coalesced manner possible, then let the drive rest. And actually, to come back to my original question: I finally made the jump today, and added ~110GB of dm-writecache in front of the backing block devices, keeping WAL+DB dedicated and on the same drive (so that the cache can never be absent for an alive OSD, to be safe, treating "transparent" as "I'm ignoring it"). I'm happy to report that, indeed, Ceph did not bat an eye and keeps chugging along. Except that now, practically no IO hits the backing crappy drive, and the cluster has the performance of those good drives (well, as long as the working data set does not get flushed, but my workloads will make it true virtually always). Thanks to everybody that took part in the discussion, and answered the questions of both a Ceph newbie and LVM newbie!
I've seen use-cases where the hassle of WAL+DB offload doesn't actually make an appreciable improvement.
I guess it really depends on the drive you have. For HDDs it's a no brainer as far as I understand it,
I’ve worked with an HDD cluster that added colocated HDDs to existed offloaded HDDs. 40% of the OSDs were colocated and nobody noticed. There are no doubt workloads where it would make a bigger difference.
Rough haha, so much for the money in the dedicated drives.
but for SSDs you'd hit diminishing returns, even with NVMe WAL+DB.
Absolutely. And new spend on SAS / SATA SSDs these days is a bad idea. They are disappearing from the market.
Good thing I'm buying refurb then :)
It's a case where transparent caching of block with an NVMe would make an improvement where WAL+DB does not Yep
Alexis