9 Mar
2020
9 Mar
'20
7:30 a.m.
Hi team, I'm planning to invest in hardware for a PoC and I would like your
feedback before the purchase:
The goal is to deploy a *16TB* storage cluster, with *3 replicas* thus *3
nodes*.
System configuration: https://pcpartpicker.com/list/cfDpDx ($400 USD per
node)
Some notes about the configuration:
- 4-core processor for 4 OSD daemons
- 8GB RAM for the first 4TB of storage, that will increase to 16GB of
RAM when 16TB of storage.
- Motherboard:
- 4 x SATA 6 Gb/s (one per each OSD disk)
- 2 x PCI-E x1 Slots (1 will be used for an additional Gigabit
Ethernet)
- 1 x M.2 Slots for the host OS
- Ram can increase up-to 32 GB, and another SATA 6b/s controller can
be added on PCI-E x1 for growth up to *32TB*
As noted, the plan is to deploy nodes with *4TB* and gradually add *12TB* as
needed, memory also should be increased to *16GB* after *8TB* threshold.
Edit
Questions to validate before the purchase
1. Does the hardware components make sense for the *16TB* growth projection?
2. Is it easy to gradually add more capacity to each node (*4TB* each time
per node)?
Thanks for your support!
--
Ignacio Ocampo
9 Mar
9 Mar
4:41 p.m.
Hola Ignacio,
El 9/3/20 a las 3:00, Ignacio Ocampo escribió:
Hi team, I'm planning to invest in hardware for a
PoC and I would like your
feedback before the purchase:
The goal is to deploy a *16TB* storage cluster, with *3 replicas* thus *3
nodes*.
System configuration: https://pcpartpicker.com/list/cfDpDx ($400 USD per
node)
Some notes about the configuration:
- 4-core processor for 4 OSD daemons
- 8GB RAM for the first 4TB of storage, that will increase to 16GB of
RAM when 16TB of storage.
- Motherboard:
- 4 x SATA 6 Gb/s (one per each OSD disk)
- 2 x PCI-E x1 Slots (1 will be used for an additional Gigabit
Ethernet)
- 1 x M.2 Slots for the host OS
- Ram can increase up-to 32 GB, and another SATA 6b/s controller can
be added on PCI-E x1 for growth up to *32TB*
As noted, the plan is to deploy nodes with *4TB* and gradually add *12TB* as
needed, memory also should be increased to *16GB* after *8TB* threshold.
Edit
Questions to validate before the purchase
1. Does the hardware components make sense for the *16TB* growth projection?
2. Is it easy to gradually add more capacity to each node (*4TB* each time
per node)?
Thanks for your support!
You may find that having only one disk per node, will make the storage
quite slow.
I think you're low on RAM to use bluestore, specially when adding new
disks, please check:
https://docs.ceph.com/docs/master/start/hardware-recommendations/
It is easy to add additional disks, yes.
You may also want to try some solution for easier deployment for such a
small cluster, I can recommend Proxmox VE (https://www.proxmox.com) but
there are others.
Cheers and good luck
Eneko
--
Zuzendari Teknikoa / Director Técnico
Binovo IT Human Project, S.L.
Telf. 943569206
Astigarragako bidea 2, 2º izq. oficina 11; 20180 Oiartzun (Gipuzkoa)
www.binovo.es
7:49 p.m.
Hi Eneko,
I was considering 1 GB per TB, but I will switch that to 4 GB per TB to consider blue
storage.
Regarding the number of devices, Will 2 disks per node help with the cluster speed?
My ultimate goal is to provide support to a second project which will be an OpenStack
deployment.
Oliver Audrey, I prefer to invest in hardware rather than renting it, since I want to
create a private cloud.
Thanks!
Ignacio Ocampo
On Mar 9,
2020, at 4:12 AM, Eneko Lacunza <elacunza(a)binovo.es> wrote:
Hola Ignacio,
El 9/3/20 a las 3:00, Ignacio Ocampo escribió:
Hi team, I'm planning to invest in hardware for a PoC and I would like your
feedback before the purchase:
The goal is to deploy a *16TB* storage cluster, with *3 replicas* thus *3
nodes*.
System configuration: https://pcpartpicker.com/list/cfDpDx ($400 USD per
node)
Some notes about the configuration:
- 4-core processor for 4 OSD daemons
- 8GB RAM for the first 4TB of storage, that will increase to 16GB of
RAM when 16TB of storage.
- Motherboard:
- 4 x SATA 6 Gb/s (one per each OSD disk)
- 2 x PCI-E x1 Slots (1 will be used for an additional Gigabit
Ethernet)
- 1 x M.2 Slots for the host OS
- Ram can increase up-to 32 GB, and another SATA 6b/s controller can
be added on PCI-E x1 for growth up to *32TB*
As noted, the plan is to deploy nodes with *4TB* and gradually add *12TB* as
needed, memory also should be increased to *16GB* after *8TB* threshold.
Edit
Questions to validate before the purchase
1. Does the hardware components make sense for the *16TB* growth projection?
2. Is it easy to gradually add more capacity to each node (*4TB* each time
per node)?
Thanks for your support!
You may find that having only one disk per node, will make the storage quite slow.
I think you're low on RAM to use bluestore, specially when adding new disks, please
check:
https://docs.ceph.com/docs/master/start/hardware-recommendations/
It is easy to add additional disks, yes.
You may also want to try some solution for easier deployment for such a small cluster, I
can recommend Proxmox VE (https://www.proxmox.com) but there are others.
Cheers and good luck
Eneko
--
Zuzendari Teknikoa / Director Técnico
Binovo IT Human Project, S.L.
Telf. 943569206
Astigarragako bidea 2, 2º izq. oficina 11; 20180 Oiartzun (Gipuzkoa)
www.binovo.es
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
8:15 p.m.
Hi Ignacio,
El 9/3/20 a las 15:19, Ignacio Ocampo escribió:
I was considering 1 GB per TB, but I will switch that
to 4 GB per TB
to consider blue storage.
Much better. :)
Regarding the number of devices, Will 2 disks per node help with the
cluster speed?
Yes, it will help. Generally speaking, more disks=better
performance.
You may consider using the server's OS m.2 slot disk for Bluestore's db
to improve performance; if you do so, make sure to use enterprise SSDs
(like Intel D3-S4510 for example). You may need to get a better AM4 CPU
too, specially if using SSD for Bluestore:
https://cpu.userbenchmark.com/Compare/AMD-Ryzen-5-2600-vs-AMD-A8-9600-APU-2…
https://cpu.userbenchmark.com/Compare/AMD-Ryzen-5-3600-vs-AMD-A8-9600-APU-2…
My ultimate goal is to provide support to a second project which will
be an OpenStack deployment.
It will be a small deployment right? :-) Because the
config you
referenced is quite underpowered.
How many VMs? What level of IO? (IO per second?)
You'll be able to perform about 100-150 IOPS with one HDD per server...
Cheers
Eneko
Oliver Audrey, I prefer to invest in hardware rather than renting it,
since I want to create a private cloud.
Thanks!
*Ignacio Ocampo*
> On Mar 9, 2020, at 4:12 AM, Eneko Lacunza <elacunza(a)binovo.es> wrote:
>
> Hola Ignacio,
>
> El 9/3/20 a las 3:00, Ignacio Ocampo escribió:
>> Hi team, I'm planning to invest in hardware for a PoC and I would
>> like your
>> feedback before the purchase:
>>
>> The goal is to deploy a *16TB* storage cluster, with *3 replicas*
>> thus *3
>> nodes*.
>>
>> System configuration: https://pcpartpicker.com/list/cfDpDx ($400 USD per
>> node)
>>
>> Some notes about the configuration:
>>
>> - 4-core processor for 4 OSD daemons
>> - 8GB RAM for the first 4TB of storage, that will increase to 16GB of
>> RAM when 16TB of storage.
>> - Motherboard:
>> - 4 x SATA 6 Gb/s (one per each OSD disk)
>> - 2 x PCI-E x1 Slots (1 will be used for an additional Gigabit
>> Ethernet)
>> - 1 x M.2 Slots for the host OS
>> - Ram can increase up-to 32 GB, and another SATA 6b/s
>> controller can
>> be added on PCI-E x1 for growth up to *32TB*
>>
>> As noted, the plan is to deploy nodes with *4TB* and gradually add
>> *12TB* as
>> needed, memory also should be increased to *16GB* after *8TB* threshold.
>> Edit
>> Questions to validate before the purchase
>>
>> 1. Does the hardware components make sense for the *16TB* growth
>> projection?
>>
>> 2. Is it easy to gradually add more capacity to each node (*4TB*
>> each time
>> per node)?
>> Thanks for your support!
>
> You may find that having only one disk per node, will make the
> storage quite slow.
>
> I think you're low on RAM to use bluestore, specially when adding new
> disks, please check:
> https://docs.ceph.com/docs/master/start/hardware-recommendations/
>
> It is easy to add additional disks, yes.
>
> You may also want to try some solution for easier deployment for such
> a small cluster, I can recommend Proxmox VE (https://www.proxmox.com)
> but there are others.
>
> Cheers and good luck
> Eneko
>
> --
> Zuzendari Teknikoa / Director Técnico
> Binovo IT Human Project, S.L.
> Telf. 943569206
> Astigarragako bidea 2, 2º izq. oficina 11; 20180 Oiartzun (Gipuzkoa)
> www.binovo.es
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
--
Zuzendari Teknikoa / Director Técnico
Binovo IT Human Project, S.L.
Telf. 943569206
Astigarragako bidea 2, 2º izq. oficina 11; 20180 Oiartzun (Gipuzkoa)
www.binovo.es
4:57 p.m.
hello
if it's only for a poc you can try to rent devices for some month.
for less that 100€ per month you can get this kind of device at ovh.com
- 32go RAM
- Intel Xeon-E 2274G - 4 c/8 t - 4 GHz/4.9 GHz
- 3× 4 To HDD SATA Soft RAID or 2× 960 Go SSD NVMe Soft RAID
- 2 x 2Gbps network
Personnaly I use bunch of them for my kubernetes prod (the one with 2×
960 Go SSD NVMe) and it's working pretty well.
You can tweak to get more or different disk setup, more ram, more
network speed.
Everything is here :
https://www.ovh.com/fr/serveurs_dedies/infra/tarifs/
The device I'm talking about is INFRA-1
oau
Le dimanche 08 mars 2020 à 19:00 -0700, Ignacio Ocampo a écrit :
Hi team, I'm planning to invest in hardware for a
PoC and I would
like your
feedback before the purchase:
The goal is to deploy a *16TB* storage cluster, with *3 replicas*
thus *3
nodes*.
System configuration: https://pcpartpicker.com/list/cfDpDx ($400 USD
per
node)
Some notes about the configuration:
- 4-core processor for 4 OSD daemons
- 8GB RAM for the first 4TB of storage, that will increase to 16GB
of
RAM when 16TB of storage.
- Motherboard:
- 4 x SATA 6 Gb/s (one per each OSD disk)
- 2 x PCI-E x1 Slots (1 will be used for an additional Gigabit
Ethernet)
- 1 x M.2 Slots for the host OS
- Ram can increase up-to 32 GB, and another SATA 6b/s
controller can
be added on PCI-E x1 for growth up to *32TB*
As noted, the plan is to deploy nodes with *4TB* and gradually add
*12TB* as
needed, memory also should be increased to *16GB* after *8TB*
threshold.
Edit
Questions to validate before the purchase
1. Does the hardware components make sense for the *16TB* growth
projection?
2. Is it easy to gradually add more capacity to each node (*4TB* each
time
per node)?
Thanks for your support!
9:31 p.m.
Ignacio;
Personally, I like to use hardware for a proof of concept that I can roll over into the
final system, or repurpose if the project is denied.
As such, I would recommend these:
Supermicro 5019A-12TN4
Barebones<https://www.supermicro.com/products/system/1U/5019/SYS-5019A-1…
I built our PoC around three of them (2 x 12TB Seagate Ironwolf drives, plus 1x 256 GB
Intel M.2 SSDs), then they got turned into MONs for the production cluster. The
production cluster ended up with larger M.2s, and smaller spinners, as I was concerned
about recovery time for 24 - 36TB per node, with only 4 x 1Gb network.
Just my 2 cents.
Thank you,
Dominic L. Hilsbos, MBA
Director - Information Technology
Perform Air International Inc.
DHilsbos(a)PerformAir.com
www.PerformAir.com
-----Original Message-----
From: Ignacio Ocampo [mailto:nafiux@gmail.com]
Sent: Sunday, March 08, 2020 7:00 PM
To: ceph-users(a)ceph.io
Subject: [ceph-users] Hardware feedback before purchasing for a PoC
Hi team, I'm planning to invest in hardware for a PoC and I would like your
feedback before the purchase:
The goal is to deploy a *16TB* storage cluster, with *3 replicas* thus *3
nodes*.
System configuration: https://pcpartpicker.com/list/cfDpDx ($400 USD per
node)
Some notes about the configuration:
- 4-core processor for 4 OSD daemons
- 8GB RAM for the first 4TB of storage, that will increase to 16GB of
RAM when 16TB of storage.
- Motherboard:
- 4 x SATA 6 Gb/s (one per each OSD disk)
- 2 x PCI-E x1 Slots (1 will be used for an additional Gigabit
Ethernet)
- 1 x M.2 Slots for the host OS
- Ram can increase up-to 32 GB, and another SATA 6b/s controller can
be added on PCI-E x1 for growth up to *32TB*
As noted, the plan is to deploy nodes with *4TB* and gradually add *12TB* as
needed, memory also should be increased to *16GB* after *8TB* threshold.
Edit
Questions to validate before the purchase
1. Does the hardware components make sense for the *16TB* growth projection?
2. Is it easy to gradually add more capacity to each node (*4TB* each time
per node)?
Thanks for your support!
--
Ignacio Ocampo
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
1529
days inactive
1529
days old
5 comments
4 participants
participants (4)
-
DHilsbos＠performair.com -
Eneko Lacunza -
Ignacio Ocampo -
Olivier AUDRY