Dear Mattias， Very glad to know that your setting with Bcache works well in production. How long have you been puting XFS on bcache on HDD in production? Which bcache version (i mean the kernel) do you use? or do you use a special version of bcache? thanks in advance, samuel huxiaoyu@horebdata.cn From: Matthias Ferdinand Date: 2021-04-19 17:35 To: huxiaoyu@horebdata.cn CC: ceph-users Subject: Re: [ceph-users] any experience on using Bcache on top of HDD OSD On Sun, Apr 18, 2021 at 10:31:30PM +0200, huxiaoyu@horebdata.cn wrote:
Dear Cephers,
Just curious about any one who has some experience on using Bcache on top of HDD OSD to accelerate IOPS performance?
If any, how about the stability and the performance improvement, and for how long the running time?
Hi, I have not used bcache with Bluestore, but I use bcache in a Jewel cluster with Filestore on XFS on bcache on HDD, and I haven't seen any bcache-related trouble with this setup so far. I don't have journal on bcache, journal is separated out to SSD. From time to time I drained and detached the caching device from an OSD just to see if the added complexity still has some value, but latency (as measured by iostat on the OSD) would go up by a factor of about 2 so I kept bcache active on HDD OSDs. For Bluestore I don't know how much (if at all) bcache would improve performance where WAL/DB is placed on SSD already. A word of warning: never use non-DC-class SSDs for bcache caching devices. bcache does some write amplification, and you will regret it if using consumer grade SSDs. Might even turn out slower than plain HDD with highly irregular latency spikes. Regards Matthias