Re: SSD recommendations for RBD and VM's
Hello Samuel. Thanks for the answer. Yes the Intel S4510 series is a good choice but it's expensive. I have 21 server and data distribution is quite well. At power loss I don't think I'll lose data. All the VM's using same image and the rest is cookie. In this case I'm not sure I should spend extra money on PLP. Actually I like Samsung 870 EVO. It's cheap and I think 300TBW will be enough for 5-10years. Do you know any better ssd with the same price range as 870 EVO? Samsung 870 evo (500GB) = 5 Years or 300 TBW - $64.99 Samsung 860 pro (512GB) = 5 Years or 600 TBW - $99
I would recommend Intel S4510 series, which has power loss protection (PLP).
If you do not care about PLP, lower-cost Samsung 870EVO and Crucial MX500 should also be OK (with separate DB/WAL on enterprise SSD with PLP)
Samuel
________________________________ huxiaoyu@horebdata.cn
From: by morphin Date: 2021-05-30 02:48 To: Anthony D'Atri CC: Ceph Users Subject: [ceph-users] Re: SSD recommendations for RBD and VM's Hello Anthony.
I use Qemu and I don't need size. I've 1000 vm and usually they're clones from the same rbd image. The image is 30GB. Right now I've 7TB Stored data. rep x3 = 20TB data. It's mostly read intensive. Usage is stable and does not grow. So I need I/O more than capacity. That's why I'm looking for 256-512GB SSD's. I think right now 480-512GB is sweet spot for $ / GB. So 60PCS 512GB will be enough. Actually 120PCS 256GB will be better but the price goes up. I have Dell R720-740 and I use SATA Intel DCS3700 for journal. I've 40PCS 100GB. I'm gonna make them OSD as well. 7 years and DC S3700 still rocks. Not even one of them is dead. The SSD must be Low price & High TBW life span. Rest is not important.
Anthony D'Atri <anthony.datri@gmail.com>, 30 May 2021 Paz, 02:26 tarihinde şunu yazdı:
The choice depends on scale, your choice of chassis / form factor, budget, workload and needs.
The sizes you list seem awfully small. Tell us more about your use-case. OpenStack? Proxmox? QEMU? VMware? Converged? Dedicated ? —aad
On May 29, 2021, at 2:10 PM, by morphin <morphinwithyou@gmail.com> wrote:
Hello.
I have virtualization env and I'm looking new SSD for HDD replacement. What are the best Performance / Price SSDs in the market right now? I'm looking 1TB, 512GB, 480GB, 256GB, 240GB.
Is there a SSD recommendation list for ceph? _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Pls check Crucial MX500 2TB drive, i think it is a bit cheaper than Samsung 970 EVO, and it is reliable as well. samuel From: mhnx Date: 2021-05-30 20:45 To: huxiaoyu@horebdata.cn CC: Anthony D'Atri; ceph-users Subject: Re: [ceph-users] Re: SSD recommendations for RBD and VM's Hello Samuel. Thanks for the answer. Yes the Intel S4510 series is a good choice but it's expensive. I have 21 server and data distribution is quite well. At power loss I don't think I'll lose data. All the VM's using same image and the rest is cookie. In this case I'm not sure I should spend extra money on PLP. Actually I like Samsung 870 EVO. It's cheap and I think 300TBW will be enough for 5-10years. Do you know any better ssd with the same price range as 870 EVO? Samsung 870 evo (500GB) = 5 Years or 300 TBW - $64.99 Samsung 860 pro (512GB) = 5 Years or 600 TBW - $99
I would recommend Intel S4510 series, which has power loss protection (PLP).
If you do not care about PLP, lower-cost Samsung 870EVO and Crucial MX500 should also be OK (with separate DB/WAL on enterprise SSD with PLP)
Samuel
________________________________ huxiaoyu@horebdata.cn
From: by morphin Date: 2021-05-30 02:48 To: Anthony D'Atri CC: Ceph Users Subject: [ceph-users] Re: SSD recommendations for RBD and VM's Hello Anthony.
I use Qemu and I don't need size. I've 1000 vm and usually they're clones from the same rbd image. The image is 30GB. Right now I've 7TB Stored data. rep x3 = 20TB data. It's mostly read intensive. Usage is stable and does not grow. So I need I/O more than capacity. That's why I'm looking for 256-512GB SSD's. I think right now 480-512GB is sweet spot for $ / GB. So 60PCS 512GB will be enough. Actually 120PCS 256GB will be better but the price goes up. I have Dell R720-740 and I use SATA Intel DCS3700 for journal. I've 40PCS 100GB. I'm gonna make them OSD as well. 7 years and DC S3700 still rocks. Not even one of them is dead. The SSD must be Low price & High TBW life span. Rest is not important.
Anthony D'Atri <anthony.datri@gmail.com>, 30 May 2021 Paz, 02:26 tarihinde şunu yazdı:
The choice depends on scale, your choice of chassis / form factor, budget, workload and needs.
The sizes you list seem awfully small. Tell us more about your use-case. OpenStack? Proxmox? QEMU? VMware? Converged? Dedicated ? —aad
On May 29, 2021, at 2:10 PM, by morphin <morphinwithyou@gmail.com> wrote:
Hello.
I have virtualization env and I'm looking new SSD for HDD replacement. What are the best Performance / Price SSDs in the market right now? I'm looking 1TB, 512GB, 480GB, 256GB, 240GB.
Is there a SSD recommendation list for ceph? _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi, On 5/30/21 8:45 PM, mhnx wrote:
Hello Samuel. Thanks for the answer.
Yes the Intel S4510 series is a good choice but it's expensive. I have 21 server and data distribution is quite well. At power loss I don't think I'll lose data. All the VM's using same image and the rest is cookie. In this case I'm not sure I should spend extra money on PLP.
Actually I like Samsung 870 EVO. It's cheap and I think 300TBW will be enough for 5-10years. Do you know any better ssd with the same price range as 870 EVO?
Samsung 870 evo (500GB) = 5 Years or 300 TBW - $64.99 Samsung 860 pro (512GB) = 5 Years or 600 TBW - $99
But do these not lack power-loss protection..? We are running the Samsung PM883, as I was told that these would do much better as OSDs. MJ
One could use enterprise NVMe SSD (with PLP) as DB/WAL for those consumer SSD huxiaoyu@horebdata.cn From: mj Date: 2021-06-04 11:23 To: ceph-users Subject: [ceph-users] Re: SSD recommendations for RBD and VM's Hi, On 5/30/21 8:45 PM, mhnx wrote:
Hello Samuel. Thanks for the answer.
Yes the Intel S4510 series is a good choice but it's expensive. I have 21 server and data distribution is quite well. At power loss I don't think I'll lose data. All the VM's using same image and the rest is cookie. In this case I'm not sure I should spend extra money on PLP.
Actually I like Samsung 870 EVO. It's cheap and I think 300TBW will be enough for 5-10years. Do you know any better ssd with the same price range as 870 EVO?
Samsung 870 evo (500GB) = 5 Years or 300 TBW - $64.99 Samsung 860 pro (512GB) = 5 Years or 600 TBW - $99
But do these not lack power-loss protection..? We are running the Samsung PM883, as I was told that these would do much better as OSDs. MJ _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
I wonder that when a osd came back from power-lost, all the data scrubbing and there are 2 other copies. PLP is important on mostly Block Storage, Ceph should easily recover from that situation. That's why I don't understand why I should pay more for PLP and other protections. In my use case %90 of the data is cookie %10 is coldish metadata and I don't wanna pay more features I don't need. That's it. Using 870evo's with Nvme WAL is a good idea but In this case the Price still goes up cause I'm using 2 osd per device. 60 osd Total across 21Host. Samsung PM883 is good. We're using them for different projects. It is at the top of the list. These are all TLC nands and I think lowest price in the market. Its better to have MLC but the price goes double. Samsung PM883 480GB SATA 6Gb/s V4 TLC 2.5" 7mm (1.3 DWPD) 1 $102,48 Micron 5300 PRO 480GB, SATA, 2.5", 3D TLC, 1.5DWPD 1 $110,54 Micron 5300 MAX 480GB, SATA, 2.5", 3D TLC, 5DWPD 1 $132,41 Intel D3-S4610 480GB SATA 6Gb/s 3D TLC 2.5" 7mm 3DWPD Rev.2 1 $131,26 Samsung 860 PRO SATA 2.5" SSD 512GB V-NAND 2bit MLC 600 TBW. 860PRO is MLC, that's good but TBW is half of the PM883 and no PLP. I think I'm going with the PM883 if there is no other advice from the community. mj <lists@merit.unu.edu>, 4 Haz 2021 Cum, 12:24 tarihinde şunu yazdı:
Hi,
On 5/30/21 8:45 PM, mhnx wrote:
Hello Samuel. Thanks for the answer.
Yes the Intel S4510 series is a good choice but it's expensive. I have 21 server and data distribution is quite well. At power loss I don't think I'll lose data. All the VM's using same image and the rest is cookie. In this case I'm not sure I should spend extra money on PLP.
Actually I like Samsung 870 EVO. It's cheap and I think 300TBW will be enough for 5-10years. Do you know any better ssd with the same price range as 870 EVO?
Samsung 870 evo (500GB) = 5 Years or 300 TBW - $64.99 Samsung 860 pro (512GB) = 5 Years or 600 TBW - $99
But do these not lack power-loss protection..?
We are running the Samsung PM883, as I was told that these would do much better as OSDs.
MJ _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi, On 6/4/21 12:57 PM, mhnx wrote:
I wonder that when a osd came back from power-lost, all the data scrubbing and there are 2 other copies. PLP is important on mostly Block Storage, Ceph should easily recover from that situation. That's why I don't understand why I should pay more for PLP and other protections.
I'm no expert (or power user) al all, but my reasoning is: if something power-related can take down one of my servers it can just as easily take down *all* my ceph servers at once. And that could just as easily render all three copies inacessible. Plus how I understand it also: using SSDs with PLP also reduces latency, as the SSDs don't need to flush after each write. (but again: this i my understanding, and I am no expert on the subject)
"Plus how I understand it also: using SSDs with PLP also reduces latency, as the SSDs don't need to flush after each write." I didn't know that but it makes sense. I should dig into this. Thanks. mj <lists@merit.unu.edu>, 4 Haz 2021 Cum, 14:24 tarihinde şunu yazdı:
Hi,
On 6/4/21 12:57 PM, mhnx wrote:
I wonder that when a osd came back from power-lost, all the data scrubbing and there are 2 other copies. PLP is important on mostly Block Storage, Ceph should easily recover from that situation. That's why I don't understand why I should pay more for PLP and other protections.
I'm no expert (or power user) al all, but my reasoning is: if something power-related can take down one of my servers it can just as easily take down *all* my ceph servers at once.
And that could just as easily render all three copies inacessible.
Plus how I understand it also: using SSDs with PLP also reduces latency, as the SSDs don't need to flush after each write.
(but again: this i my understanding, and I am no expert on the subject)
I wonder that when a osd came back from power-lost, all the data scrubbing and there are 2 other copies. PLP is important on mostly Block Storage, Ceph should easily recover from that situation. That's why I don't understand why I should pay more for PLP and other protections.
I'm no expert (or power user) al all, but my reasoning is: if something power-related can take down one of my servers it can just as easily take down *all* my ceph servers at once.
And that could just as easily render all three copies inacessible.
Or even two. I’ve been through a protracted outage (not power related) that involved widespread OSD flapping. Despite having not lost OSDs in the end, somehow a single RADOS object ended up lost, in an RBD head. Very much a corner case, but if we’d been using 2R it would have been gruesome. On another occasion I saw a power inductor / PSU failure take down power in an entire DC row. Fortunately we were using redundant PSUs on different circuits. One node went down nonetheless — the PSU on the surviving power feed had a previous issue that wasn’t caught because PSUs weren’t monitored. As with active/passive network bonds, this showed the importance of monitoring and addressing latent faults so you don’t find them at exactly the wrong time.
пт, 4 июн. 2021 г. в 15:58, mhnx <morphinwithyou@gmail.com>:
I wonder that when a osd came back from power-lost, all the data scrubbing and there are 2 other copies. PLP is important on mostly Block Storage, Ceph should easily recover from that situation. That's why I don't understand why I should pay more for PLP and other protections.
In my use case %90 of the data is cookie %10 is coldish metadata and I don't wanna pay more features I don't need. That's it. Using 870evo's with Nvme WAL is a good idea but In this case the Price still goes up cause I'm using 2 osd per device. 60 osd Total across 21Host.
Samsung PM883 is good. We're using them for different projects. It is at the top of the list. These are all TLC nands and I think lowest price in the market. Its better to have MLC but the price goes double.
Samsung PM883 480GB SATA 6Gb/s V4 TLC 2.5" 7mm (1.3 DWPD) 1 $102,48 Micron 5300 PRO 480GB, SATA, 2.5", 3D TLC, 1.5DWPD 1 $110,54 Micron 5300 MAX 480GB, SATA, 2.5", 3D TLC, 5DWPD 1 $132,41 Intel D3-S4610 480GB SATA 6Gb/s 3D TLC 2.5" 7mm 3DWPD Rev.2 1 $131,26
Samsung 860 PRO SATA 2.5" SSD 512GB V-NAND 2bit MLC 600 TBW. 860PRO is MLC, that's good but TBW is half of the PM883 and no PLP.
I think I'm going with the PM883 if there is no other advice from the community.
mj <lists@merit.unu.edu>, 4 Haz 2021 Cum, 12:24 tarihinde şunu yazdı:
Hi,
On 5/30/21 8:45 PM, mhnx wrote:
Hello Samuel. Thanks for the answer.
Yes the Intel S4510 series is a good choice but it's expensive. I have 21 server and data distribution is quite well. At power loss I don't think I'll lose data. All the VM's using same image and the rest is cookie. In this case I'm not sure I should spend extra money on PLP.
Actually I like Samsung 870 EVO. It's cheap and I think 300TBW will be enough for 5-10years. Do you know any better ssd with the same price range as 870 EVO?
Samsung 870 evo (500GB) = 5 Years or 300 TBW - $64.99 Samsung 860 pro (512GB) = 5 Years or 600 TBW - $99
But do these not lack power-loss protection..?
We are running the Samsung PM883, as I was told that these would do much better as OSDs.
MJ
With Ceph, power loss protection is important not because it protects the data after power loss, but because it allows the drive not to waste time on fsyncs (which Ceph issues for every write). I.e. better performance. -- Alexander E. Patrakov CV: http://u.pc.cd/wT8otalK
Thanks for the information. It is very useful when the latency is important. How about in my use case? It's read intensive, %90 data is cokie. It's coming from windows disk usage, PageFile is disabled. After Bootstorm I only see 5-10K I/O max. I really do not need that much I/O power. Also I'm thinking to put VM images in a REPx3 pool, and take their clone to REPx2 pool. Everymonth or Week we have to delete all these VM's and recreate them. Maybe I should use PLP disks for REPx3 Image pool and leave the rest 870evo. It makes sense. Thanks. Alexander E. Patrakov <patrakov@gmail.com>, 6 Haz 2021 Paz, 09:03 tarihinde şunu yazdı:
пт, 4 июн. 2021 г. в 15:58, mhnx <morphinwithyou@gmail.com>:
I wonder that when a osd came back from power-lost, all the data scrubbing and there are 2 other copies. PLP is important on mostly Block Storage, Ceph should easily recover from that situation. That's why I don't understand why I should pay more for PLP and other protections.
In my use case %90 of the data is cookie %10 is coldish metadata and I don't wanna pay more features I don't need. That's it. Using 870evo's with Nvme WAL is a good idea but In this case the Price still goes up cause I'm using 2 osd per device. 60 osd Total across 21Host.
Samsung PM883 is good. We're using them for different projects. It is at the top of the list. These are all TLC nands and I think lowest price in the market. Its better to have MLC but the price goes double.
Samsung PM883 480GB SATA 6Gb/s V4 TLC 2.5" 7mm (1.3 DWPD) 1 $102,48 Micron 5300 PRO 480GB, SATA, 2.5", 3D TLC, 1.5DWPD 1 $110,54 Micron 5300 MAX 480GB, SATA, 2.5", 3D TLC, 5DWPD 1 $132,41 Intel D3-S4610 480GB SATA 6Gb/s 3D TLC 2.5" 7mm 3DWPD Rev.2 1 $131,26
Samsung 860 PRO SATA 2.5" SSD 512GB V-NAND 2bit MLC 600 TBW. 860PRO is MLC, that's good but TBW is half of the PM883 and no PLP.
I think I'm going with the PM883 if there is no other advice from the community.
mj <lists@merit.unu.edu>, 4 Haz 2021 Cum, 12:24 tarihinde şunu yazdı:
Hi,
On 5/30/21 8:45 PM, mhnx wrote:
Hello Samuel. Thanks for the answer.
Yes the Intel S4510 series is a good choice but it's expensive. I have 21 server and data distribution is quite well. At power loss I don't think I'll lose data. All the VM's using same image and the rest is cookie. In this case I'm not sure I should spend extra money on PLP.
Actually I like Samsung 870 EVO. It's cheap and I think 300TBW will be enough for 5-10years. Do you know any better ssd with the same price range as 870 EVO?
Samsung 870 evo (500GB) = 5 Years or 300 TBW - $64.99 Samsung 860 pro (512GB) = 5 Years or 600 TBW - $99
But do these not lack power-loss protection..?
We are running the Samsung PM883, as I was told that these would do much better as OSDs.
MJ
With Ceph, power loss protection is important not because it protects the data after power loss, but because it allows the drive not to waste time on fsyncs (which Ceph issues for every write). I.e. better performance.
-- Alexander E. Patrakov CV: http://u.pc.cd/wT8otalK
participants (5)
-
Alexander E. Patrakov
-
Anthony D'Atri
-
huxiaoyu＠horebdata.cn
-
mhnx
-
mj