6 Apr
2023
6 Apr
'23
9:26 p.m.
spinners are slow anyway, but on top of that SAS disks often default to
writecache=off. In use as a single disk with no risk of raid
write-holes, you can turn on writecache. On SAS, I would assume the
firmware does not lie about writes reaching stable storage (flushes).
# turn on temporarily:
sdparm --set=WCE /dev/sdX
# turn on persistently:
sdparm --set=WCE --save /dev/sdX
To check current state:
sdparm --get=WCE /dev/sdf
/dev/sdf: SEAGATE ST2000NM0045 DS03
WCE 0 [cha: y, def: 0, sav: 0]
"WCE 0" means: off
"sav: 0" means: off next time the disk is powered on
Matthias
On Thu, Apr 06, 2023 at 09:26:27AM -0400, Anthony D'Atri wrote:
> How bizarre, I haven’t dealt with this specific SKU before. Some Dell / LSI HBAs
call this passthrough mode, some “personality”, some “jbod mode”, dunno why they can’t be
consistent.
>
>
> > We are testing an experimental Ceph cluster with server and controller at
> > subject.
> >
> > The controller have not an HBA mode, but only a 'NonRAID' mode, come
sort of
> > 'auto RAID0' configuration.
>
> Dell’s CLI guide describes setting individual drives in Non-RAID, which *smells* like
passthrough, not the more-complex RAID0 workaround we had to do before passthrough.
>
>
https://www.dell.com/support/manuals/en-nz/perc-h750-sas/perc_cli_rg/set-dr…
> >
> > We are using SSD SATA disks (MICRON MTFDDAK480TDT) that perform very well,
> > and SAS HDD disks (SEAGATE ST8000NM014A) that instead perform very bad
> > (particulary, very low IOPS).
>
> Spinners are slow, this is news?
>
> That said, how slow is slow? Testing commands and results or it didn’t happen.
>
> Also, firmware matters. Run Dell’s DSU.
>
> > There's some hint for disk/controller configuration/optimization?
>
> Give us details, perccli /c0 show, test results etc.
>
> Use a different HBA if you have to use an HBA, one that doesn’t suffer an RoC.
Better yet, take an expansive look at TCO and don’t write off NVMe as infeasible. If your
cluster is experimental hopefully you aren’t stuck with a lot of these. Add up the cost
of an RoC HBA, optionally with cache RAM and BBU/supercap, add in the cost delta for SAS
HDDs over SATA. Add in the operational hassle of managing WAL+DB on those boot SSDs. Add
in the extra HDDs you’ll need to provision because of IOPS.
>
> >
> >
> > Thanks.
> >
> > --
> > Io credo nella chimica tanto quanto Giulio Cesare credeva nel caso...
> > mi va bene fino a quando non riguarda me :) (Emanuele Pucciarelli)
> >
> > _______________________________________________
> > ceph-users mailing list -- ceph-users(a)ceph.io
> > To unsubscribe send an email to ceph-users-leave(a)ceph.io
>
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io