One could use enterprise NVMe SSD (with PLP) as DB/WAL for those consumer SSD huxiaoyu(a)horebdata.cn From: mj Date: 2021-06-04 11:23 To: ceph-users Subject: [ceph-users] Re: SSD recommendations for RBD and VM's Hi, On 5/30/21 8:45 PM, mhnx wrote: But do these not lack power-loss protection..? We are running the Samsung PM883, as I was told that these would do much better as OSDs. MJ _______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

Hi, On 6/4/21 12:57 PM, mhnx wrote: I'm no expert (or power user) al all, but my reasoning is: if something power-related can take down one of my servers it can just as easily take down *all* my ceph servers at once. And that could just as easily render all three copies inacessible. Plus how I understand it also: using SSDs with PLP also reduces latency, as the SSDs don't need to flush after each write. (but again: this i my understanding, and I am no expert on the subject)

Or even two. I’ve been through a protracted outage (not power related) that involved widespread OSD flapping. Despite having not lost OSDs in the end, somehow a single RADOS object ended up lost, in an RBD head. Very much a corner case, but if we’d been using 2R it would have been gruesome. On another occasion I saw a power inductor / PSU failure take down power in an entire DC row. Fortunately we were using redundant PSUs on different circuits. One node went down nonetheless — the PSU on the surviving power feed had a previous issue that wasn’t caught because PSUs weren’t monitored. As with active/passive network bonds, this showed the importance of monitoring and addressing latent faults so you don’t find them at exactly the wrong time.

Thanks for the information. It is very useful when the latency is important. How about in my use case? It's read intensive, %90 data is cokie. It's coming from windows disk usage, PageFile is disabled. After Bootstorm I only see 5-10K I/O max. I really do not need that much I/O power. Also I'm thinking to put VM images in a REPx3 pool, and take their clone to REPx2 pool. Everymonth or Week we have to delete all these VM's and recreate them. Maybe I should use PLP disks for REPx3 Image pool and leave the rest 870evo. It makes sense. Thanks. Alexander E. Patrakov <patrakov(a)gmail.com>om>, 6 Haz 2021 Paz, 09:03 tarihinde şunu yazdı: > > пт, 4 июн. 2021 г. в 15:58, mhnx <morphinwithyou(a)gmail.com>om>: > > > > I wonder that when a osd came back from power-lost, all the data > > scrubbing and there are 2 other copies. > > PLP is important on mostly Block Storage, Ceph should easily recover > > from that situation. > > That's why I don't understand why I should pay more for PLP and other > > protections. > > > > In my use case %90 of the data is cookie %10 is coldish metadata and I > > don't wanna pay more features I don't need. That's it. > > Using 870evo's with Nvme WAL is a good idea but In this case the Price > > still goes up cause I'm using 2 osd per device. 60 osd Total across > > 21Host. > > > > Samsung PM883 is good. We're using them for different projects. It is > > at the top of the list. > > These are all TLC nands and I think lowest price in the market. Its > > better to have MLC but the price goes double. > > > > Samsung PM883 480GB SATA 6Gb/s V4 TLC 2.5" 7mm (1.3 DWPD) 1 $102,48 > > Micron 5300 PRO 480GB, SATA, 2.5", 3D TLC, 1.5DWPD 1 $110,54 > > Micron 5300 MAX 480GB, SATA, 2.5", 3D TLC, 5DWPD 1 $132,41 > > Intel D3-S4610 480GB SATA 6Gb/s 3D TLC 2.5" 7mm 3DWPD Rev.2 1 $131,26 > > > > > > Samsung 860 PRO SATA 2.5" SSD 512GB V-NAND 2bit MLC 600 TBW. > > 860PRO is MLC, that's good but TBW is half of the PM883 and no PLP. > > > > I think I'm going with the PM883 if there is no other advice from the community. > > > > mj <lists(a)merit.unu.edu>du>, 4 Haz 2021 Cum, 12:24 tarihinde şunu yazdı: > > > > > > Hi, > > > > > > On 5/30/21 8:45 PM, mhnx wrote: > > > > Hello Samuel. Thanks for the answer. > > > > > > > > Yes the Intel S4510 series is a good choice but it's expensive. > > > > I have 21 server and data distribution is quite well. > > > > At power loss I don't think I'll lose data. All the VM's using same > > > > image and the rest is cookie. > > > > In this case I'm not sure I should spend extra money on PLP. > > > > > > > > Actually I like Samsung 870 EVO. It's cheap and I think 300TBW will be > > > > enough for 5-10years. > > > > Do you know any better ssd with the same price range as 870 EVO? > > > > > > > > Samsung 870 evo (500GB) = 5 Years or 300 TBW - $64.99 > > > > Samsung 860 pro (512GB) = 5 Years or 600 TBW - $99 > > > > > > > > > > But do these not lack power-loss protection..? > > > > > > We are running the Samsung PM883, as I was told that these would do much > > > better as OSDs. > > > > > > MJ > > With Ceph, power loss protection is important not because it protects > the data after power loss, but because it allows the drive not to > waste time on fsyncs (which Ceph issues for every write). I.e. better > performance. > > -- > Alexander E. Patrakov > CV: http://u.pc.cd/wT8otalK