On Tue, Mar 02, 2021 at 05:47:29PM +0800, Norman.Kern wrote: Hi, I used to tune writeback_percent as far down as 1. I guess rapid writeback helped keep complexity (CPU, additional I/O) of handling dirty blocks low. Hoarding dirty data for a better chance to eventually turn it into sequential I/O is an important gain on MD-RAID5/6, but not so much on a single disk. Perhaps at cache_available_percent < 30 bcache needs to do some garbage collection. This would at least give some CPU spike, and probably some additional I/O spike for the on-disk data structure. This is where you really want to have decent DC-grade caching devices that can keep up with this sort of write amplification spikes. Consumer-grade devices won't be able to, and even add their own very much noticeable internal garbage collection on top. Bypassing the caching SSD device on (non-sequential) I/O is usually a symptom of bcache detecting a saturated caching device, i.e. the SSDs are probably not DC-grade. At this point you get all the latency of the backing HDD, plus some more from metadata handling. You might even tune bcache to never bypass, but at this point, it would only further add to the latency. Regards Matthias

On 2021/3/2 上午5:09, Andreas John wrote: No, I created the OSD storage devices using bcache devices. > > > rgds, > > derjohn > > > On 01.03.21 05:37, Norman.Kern wrote: >> Hi, guys >> >> I am testing ceph on bcache devices,  I found the performance is not good as expected. Does anyone have any best practices for it?  Thanks. >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io

James, Can you tell me what's the hardware config of your bcache? I use the 400G SATA SSD as cache device and 10T HDD as the storage device.  Hardware relationed? On 2021/3/2 下午4:49, James Page wrote: > Hi Norman > > On Mon, Mar 1, 2021 at 4:38 AM Norman.Kern &lt;norman.kern(a)gmx.com&gt; wrote: > >> Hi, guys >> >> I am testing ceph on bcache devices, I found the performance is not good >> as expected. Does anyone have any best practices for it? Thanks. >> > I've used bcache quite a bit with Ceph with the following configuration > options tweaked > > a) use writeback mode rather than writethrough (which is the default) > > This ensures that the cache device is actually used for write caching > > b) turn off the sequential cutoff > > sequential_cutoff = 0 > > This means that sequential writes will also always go to the cache device > rather than the backing device > > c) disable the congestion read and write thresholds > > congested_read_threshold_us = congested_write_threshold_us = 0 > > The following repository: > > https://git.launchpad.net/charm-bcache-tuning/tree/src/files > > has a python script and systemd configuration todo b) and c) automatically > on all bcache devices on boot; a) we let the provisioning system take care > of. > > HTH > > >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io >> > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io

вт, 2 мар. 2021 г. в 13:52, James Page &lt;james.page(a)canonical.com&gt;om>: (Disclaimer: I have never tried to run Ceph on bcache in production, and the test cluster was destroyed before reaching its first deep scrub) Could you please explain the exact mechanics of the sequential cutoff? Does it only affect big sequential writes, or big sequential reads too? I am asking because of the potential of deep scrubs being cached instead of the "real" hot data. -- Alexander E. Patrakov CV: http://u.pc.cd/wT8otalK