Hi guys, I was looking at some Huawei ARM-based servers and the datasheets are very interesting. The high CPU core numbers and the SoC architecture should be ideal for a distributed storage like Ceph, at least in theory. I'm planning to build a new Ceph cluster in the future and my best case scenario right now is to buy 7 servers with 2 x Intel Silver 12-core or 2 x Gold 20-core CPUs, 32 SATA drives each. And of course - SSDs/NVMEs for wal/db and the metadata pools , 256GB RAM and so on. I'm curious however if the ARM servers are better or not for this use case (object-storage only). For example, instead of using 2xSilver/Gold server, I can use a Taishan 5280 server with 2x Kungpen 920 ARM CPUs with up to 128 cores in total . So I can have twice as many CPU cores (or even more) per server comparing with x86. Probably the price is lower for the ARM servers as well. Has anyone tested Ceph in such scenario ? Is the Ceph software really optimised for the ARM architecture ? What do you think about this ? Thanks !
At least in the past, there have been a couple of things you really want to focus on regarding ARM and performance (beyond the obvious core count/clockspeed/ipc/etc): 1) HW acceleration for things like CRC32, MD5, etc 2) context switching overhead 3) memory throughput 4) SATA controller and PCIe throughput Unfortunately these are the kinds of things that you can't easily generalize between ARM vs x86. Some ARM processors are going to do wildly better than others. The good news is that driving 32 SATA drives shouldn't be particularly CPU intensive (128 modern ARM cores is probably sufficient) so the real question is probably what the SATA controller setup looks like. I think you're best bet if you are serious about pursuing the ARM route is try to get access to one of the Taishan 5280 servers and do some tests. You could try renting out an EC2 ARM systems and play around there but I doubt it will be a very apples->apples comparison. Mark On 11/24/20 6:12 AM, Adrian Nicolae wrote:
Hi guys,
I was looking at some Huawei ARM-based servers and the datasheets are very interesting. The high CPU core numbers and the SoC architecture should be ideal for a distributed storage like Ceph, at least in theory.
I'm planning to build a new Ceph cluster in the future and my best case scenario right now is to buy 7 servers with 2 x Intel Silver 12-core or 2 x Gold 20-core CPUs, 32 SATA drives each. And of course - SSDs/NVMEs for wal/db and the metadata pools , 256GB RAM and so on.
I'm curious however if the ARM servers are better or not for this use case (object-storage only). For example, instead of using 2xSilver/Gold server, I can use a Taishan 5280 server with 2x Kungpen 920 ARM CPUs with up to 128 cores in total . So I can have twice as many CPU cores (or even more) per server comparing with x86. Probably the price is lower for the ARM servers as well.
Has anyone tested Ceph in such scenario ? Is the Ceph software really optimised for the ARM architecture ? What do you think about this ?
Thanks ! _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
I had hoped to stay out of this, but here I go.
4) SATA controller and PCIe throughput
SoftIron claims “wire speed” with their custom hardware FWIW.
Unfortunately these are the kinds of things that you can't easily generalize between ARM vs x86. Some ARM processors are going to do wildly better than others.
Absolutely, just like an Intel Atom will deliver a different experience than a high-end Xeon. Those of us in the US, though, can see markedly different pricing and product availability than other countries.
Hi guys,
I was looking at some Huawei ARM-based servers and the datasheets are very interesting. The high CPU core numbers and the SoC architecture should be ideal for a distributed storage like Ceph, at least in theory.
I'm planning to build a new Ceph cluster in the future and my best case scenario right now is to buy 7 servers with 2 x Intel Silver 12-core or 2 x Gold 20-core CPUs, 32 SATA drives each. And of course - SSDs/NVMEs for wal/db and the metadata pools , 256GB RAM and so on.
Consider if a single-socket Epyc would let you upgrade to QLC drives, save all the external WAL/DB hassles and potentially the need for an HBA at all.
I'm curious however if the ARM servers are better or not for this use case (object-storage only).
Everyone’s needs are unique, but my sense is that object storage can be a reasonable fit for the right ARM gear if your use-case cares more about throughput than latency or IOPS.
For example, instead of using 2xSilver/Gold server, I can use a Taishan 5280 server with 2x Kungpen 920 ARM CPUs with up to 128 cores in total . So I can have twice as many CPU cores (or even more) per server comparing with x86.
More but slower cores work well in some cases, and are miserable in others. cf. Sun’s CMT systems from last decade. Conventional wisdom is that for CephFS MDS, for example, you want fewer but faster cores, so that’s probably an application for which such a CPU isn’t a good fit.
Probably the price is lower for the ARM servers as well.
That’s a function in part of your haggling skills. Oh and power (and thus cooling reduction) is not imaginary. Time and again I’ve seen DC racks only partially populated because they ran out of amps before RUs. Racks, RUs, TORs, switch ports, these things all cost money. Like when people think cheap 1TB drives are a bargain, you have to look at TCO, including power, heat, and how many TB you can fit in a chassis and a rack.
Has anyone tested Ceph in such scenario ? Is the Ceph software really optimised for the ARM architecture ? What do you think about this ?
SoftIron gear has been running production Ceph for a few years. They tell me the code is unmodified.
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
Has anyone tested Ceph in such scenario ? Is the Ceph software really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling such setups: https://softiron.com/ https://www.ambedded.com.tw/ Regards -- Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Hello,
I'm curious however if the ARM servers are better or not for this use case (object-storage only). For example, instead of using 2xSilver/Gold server, I can use a Taishan 5280 server with 2x Kungpen 920 ARM CPUs with up to 128 cores in total . So I can have twice as many CPU cores (or even more) per server comparing with x86. Probably the price is lower for the ARM servers as well.
Even if they would be cheaper, which I strongly doubt, you will get less performance out of them. More cores won't give you any benefit in Ceph, but having much faster cores is somewhat of a game changer. Just use a good AMD Epyc for best price/performance/power ratio.
Has anyone tested Ceph in such scenario? Is the Ceph software really optimised for the ARM architecture ? What do you think about this ?
If you choose ARM for your Ceph, you are one of very very few people and will most properly hit some crazy bugs that will cause trouble. A high price to pay in my opinion just for an "imaginary" performance or power reduction benefit. Storage has to run 24*7 all year long without a single incident. Everything else in my world is inacceptable. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.verges@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB 231263 Web: https://croit.io YouTube: https://goo.gl/PGE1Bx Am Di., 24. Nov. 2020 um 13:57 Uhr schrieb Robert Sander <r.sander@heinlein-support.de>:
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
Has anyone tested Ceph in such scenario ? Is the Ceph software really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling such setups:
https://softiron.com/ https://www.ambedded.com.tw/
Regards -- Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-43 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
So yes you can get the servers for a considerably lower price than Intel. Its not just about the CPU cost but many arm servers are based on a SOC that includes networking so the overall cost of the motherboard/processor/networking is a lot lower. It doesn't reduce the price of the storage or memory though which are a large part of the cost. Power consumption on ARM processors is considerably lower than on Intel. Think in the order of 100W per server which adds up over the course of a year. ARM have done work to extend the ISA-L so they have hardware accelerated things like erasure coding in the same way that intel have. Having done a lot of testing on ARM including IO500 work then I can certainly say it works. We didn't hit any problems that where arm specific and certainly pushed things performance wise. Yes you have to consider clock speeds of the processors as quite a few things in CEPH are single threaded but having all the extra cores means you can run many processes for things like MDS and OSD’s. Darren From: Martin Verges <martin.verges@croit.io> Date: Tuesday, 24 November 2020 at 13:09 To: Robert Sander <r.sander@heinlein-support.de> Cc: ceph-users <ceph-users@ceph.io> Subject: [ceph-users] Re: Ceph on ARM ? Hello,
I'm curious however if the ARM servers are better or not for this use case (object-storage only). For example, instead of using 2xSilver/Gold server, I can use a Taishan 5280 server with 2x Kungpen 920 ARM CPUs with up to 128 cores in total . So I can have twice as many CPU cores (or even more) per server comparing with x86. Probably the price is lower for the ARM servers as well.
Even if they would be cheaper, which I strongly doubt, you will get less performance out of them. More cores won't give you any benefit in Ceph, but having much faster cores is somewhat of a game changer. Just use a good AMD Epyc for best price/performance/power ratio.
Has anyone tested Ceph in such scenario? Is the Ceph software really optimised for the ARM architecture ? What do you think about this ?
If you choose ARM for your Ceph, you are one of very very few people and will most properly hit some crazy bugs that will cause trouble. A high price to pay in my opinion just for an "imaginary" performance or power reduction benefit. Storage has to run 24*7 all year long without a single incident. Everything else in my world is inacceptable. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.verges@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB 231263 Web: https://croit.io YouTube: https://goo.gl/PGE1Bx Am Di., 24. Nov. 2020 um 13:57 Uhr schrieb Robert Sander <r.sander@heinlein-support.de>:
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
Has anyone tested Ceph in such scenario ? Is the Ceph software really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling such setups:
https://softiron.com/ https://www.ambedded.com.tw/
Regards -- Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-43 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Just to add our perspective here, I’d like to second Darren’s message here that Ceph is great on ARM. We’re intensely familiar with this as we’ve been running it smoothly for years. We have some very credible customer references that have been running our product at scale for a while. I know there are others on this list that also run ARM at scale as well. There’s a strong argument to be made that it’s preferable to do things this way - power efficiency (which can have an impact on rack density as well as cost) is one of those arguments but it doesn’t tell the whole story - especially when you start asking how running storage nodes hot impacts disk MTBF as just one example. While we’d fix them if they existed, we’ve never hit the “crazy bugs” on ARM mentioned above - I believe that Ceph’s CI builds against ARM, and we do this in our lab as well. I can’t speak to your specific configuration and silicon, but if you’d like to hear more about some of our customer stories I’m sure we can facilitate :) Danny
On 24 Nov 2020, at 13:58, Darren Soothill <darren.soothill@suse.com> wrote:
So yes you can get the servers for a considerably lower price than Intel.
Its not just about the CPU cost but many arm servers are based on a SOC that includes networking so the overall cost of the motherboard/processor/networking is a lot lower.
It doesn't reduce the price of the storage or memory though which are a large part of the cost.
Power consumption on ARM processors is considerably lower than on Intel. Think in the order of 100W per server which adds up over the course of a year.
ARM have done work to extend the ISA-L so they have hardware accelerated things like erasure coding in the same way that intel have.
Having done a lot of testing on ARM including IO500 work then I can certainly say it works. We didn't hit any problems that where arm specific and certainly pushed things performance wise.
Yes you have to consider clock speeds of the processors as quite a few things in CEPH are single threaded but having all the extra cores means you can run many processes for things like MDS and OSD’s.
Darren
From: Martin Verges <martin.verges@croit.io> Date: Tuesday, 24 November 2020 at 13:09 To: Robert Sander <r.sander@heinlein-support.de> Cc: ceph-users <ceph-users@ceph.io> Subject: [ceph-users] Re: Ceph on ARM ? Hello,
I'm curious however if the ARM servers are better or not for this use case (object-storage only). For example, instead of using 2xSilver/Gold server, I can use a Taishan 5280 server with 2x Kungpen 920 ARM CPUs with up to 128 cores in total . So I can have twice as many CPU cores (or even more) per server comparing with x86. Probably the price is lower for the ARM servers as well.
Even if they would be cheaper, which I strongly doubt, you will get less performance out of them. More cores won't give you any benefit in Ceph, but having much faster cores is somewhat of a game changer. Just use a good AMD Epyc for best price/performance/power ratio.
Has anyone tested Ceph in such scenario? Is the Ceph software really optimised for the ARM architecture ? What do you think about this ?
If you choose ARM for your Ceph, you are one of very very few people and will most properly hit some crazy bugs that will cause trouble. A high price to pay in my opinion just for an "imaginary" performance or power reduction benefit. Storage has to run 24*7 all year long without a single incident. Everything else in my world is inacceptable.
-- Martin Verges Managing director
Mobile: +49 174 9335695 E-Mail: martin.verges@croit.io Chat: https://t.me/MartinVerges
croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB 231263
Web: https://croit.io YouTube: https://goo.gl/PGE1Bx
Am Di., 24. Nov. 2020 um 13:57 Uhr schrieb Robert Sander <r.sander@heinlein-support.de>:
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
Has anyone tested Ceph in such scenario ? Is the Ceph software really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling such setups:
https://softiron.com/ https://www.ambedded.com.tw/
Regards -- Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-43 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Adrian; I've always considered the advantage of ARM to be the reduction in the failure domain. Instead of one server with 2 processors, and 2 power supplies, in 1 case, running 48 disks, you can do 4 cases containing 8 power supplies, and 32 processors running 32 (or 64...) disks. The architecture is different with ARM; you pair an ARM SoC up with just one or 2 disks, and you only run the OSD software. Thank you, Dominic L. Hilsbos, MBA Director – Information Technology Perform Air International Inc. DHilsbos@PerformAir.com www.PerformAir.com -----Original Message----- From: Robert Sander [mailto:r.sander@heinlein-support.de] Sent: Tuesday, November 24, 2020 5:56 AM To: ceph-users@ceph.io Subject: [ceph-users] Re: Ceph on ARM ? Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
Has anyone tested Ceph in such scenario ? Is the Ceph software really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling such setups: https://softiron.com/ https://www.ambedded.com.tw/ Regards -- Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
I've been running ceph on a heterogeneous mix of rock64 and rpi4 SBCs. i've had to do my own builds, as the upstream ones started off with thunked-out checksumming due to (afaict) different arm feature sets between upstream's build targets and my SBCs, but other than that one, haven't run into any arm-specific issues. i've had to clamp down cache size to avoid memory exhaustion on the smaller boards, and ran into some corruption due to seemingly a bad interaction between o_direct and zram swap on a particular kernel version, but those aren't problems unique to ceph on arm. i should also mention that i'm really very tolerant of latency with this cluster, as this is just some homelab garbage running on the cheapest slowest spinning disk available, so there's some bounds on what i'm asserting. On Tue, Nov 24, 2020 at 11:24 AM <DHilsbos@performair.com> wrote:
Adrian;
I've always considered the advantage of ARM to be the reduction in the failure domain. Instead of one server with 2 processors, and 2 power supplies, in 1 case, running 48 disks, you can do 4 cases containing 8 power supplies, and 32 processors running 32 (or 64...) disks.
The architecture is different with ARM; you pair an ARM SoC up with just one or 2 disks, and you only run the OSD software.
Thank you,
Dominic L. Hilsbos, MBA Director – Information Technology Perform Air International Inc. DHilsbos@PerformAir.com www.PerformAir.com
-----Original Message----- From: Robert Sander [mailto:r.sander@heinlein-support.de] Sent: Tuesday, November 24, 2020 5:56 AM To: ceph-users@ceph.io Subject: [ceph-users] Re: Ceph on ARM ?
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
Has anyone tested Ceph in such scenario ? Is the Ceph software really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling such setups:
https://softiron.com/ https://www.ambedded.com.tw/
Regards -- Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-43 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi guys, Thank you all for your input. I will try to get my hands on a few Huawei servers for testing before deciding. The specs are really interesting, it looks like they have some power in them besides the high core numbers (2.6 Ghz) : https://en.wikichip.org/wiki/hisilicon/kunpeng/920-6426 https://e.huawei.com/en/products/servers/taishan-server/taishan-5280-v2 I'll let you know how it goes. On 11/24/2020 6:36 PM, Peter Woodman wrote:
I've been running ceph on a heterogeneous mix of rock64 and rpi4 SBCs. i've had to do my own builds, as the upstream ones started off with thunked-out checksumming due to (afaict) different arm feature sets between upstream's build targets and my SBCs, but other than that one, haven't run into any arm-specific issues. i've had to clamp down cache size to avoid memory exhaustion on the smaller boards, and ran into some corruption due to seemingly a bad interaction between o_direct and zram swap on a particular kernel version, but those aren't problems unique to ceph on arm.
i should also mention that i'm really very tolerant of latency with this cluster, as this is just some homelab garbage running on the cheapest slowest spinning disk available, so there's some bounds on what i'm asserting.
On Tue, Nov 24, 2020 at 11:24 AM <DHilsbos@performair.com <mailto:DHilsbos@performair.com>> wrote:
Adrian;
I've always considered the advantage of ARM to be the reduction in the failure domain. Instead of one server with 2 processors, and 2 power supplies, in 1 case, running 48 disks, you can do 4 cases containing 8 power supplies, and 32 processors running 32 (or 64...) disks.
The architecture is different with ARM; you pair an ARM SoC up with just one or 2 disks, and you only run the OSD software.
Thank you,
Dominic L. Hilsbos, MBA Director – Information Technology Perform Air International Inc. DHilsbos@PerformAir.com www.PerformAir.com <http://www.PerformAir.com>
-----Original Message----- From: Robert Sander [mailto:r.sander@heinlein-support.de <mailto:r.sander@heinlein-support.de>] Sent: Tuesday, November 24, 2020 5:56 AM To: ceph-users@ceph.io <mailto:ceph-users@ceph.io> Subject: [ceph-users] Re: Ceph on ARM ?
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
> Has anyone tested Ceph in such scenario ? Is the Ceph software > really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling such setups:
https://softiron.com/ <https://softiron.com/> https://www.ambedded.com.tw/ <https://www.ambedded.com.tw/>
Regards -- Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de <http://www.heinlein-support.de>
Tel: 030 / 405051-43 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io <mailto:ceph-users@ceph.io> To unsubscribe send an email to ceph-users-leave@ceph.io <mailto:ceph-users-leave@ceph.io>
Indeed it does run very happily on ARM. We have three of the Mars 400 appliances from Ambedded and they work exceedingly well. 8 micro servers per chassis. 1 is a MON server and the rest are OSD (or other services). Each micro server is running CentOS 7 and ceph so real easy to maintain. And each chassis is only 105W IIRC. -- *Kevin Thorpe* VP of Enterprise Platform *W* *|* www.predictx.com *P * *|* +44 (0)20 3005 6750 <+44%2020%203005%206750> | +44 (0)808 204 0344 <+44%20808%20204%200344> *A* *|* 7th Floor, 52 Grosvenor Gardens, London SW1W 0AU <https://maps.google.com/?q=7th+Floor,+52+Grosvenor+Gardens,+London+SW1W+0AU&entry=gmail&source=g> <http://twitter.com/predictxai> <https://www.linkedin.com/company/1339603/> _________________________________________________________ This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. On Tue, 24 Nov 2020 at 16:23, <DHilsbos@performair.com> wrote:
Adrian;
I've always considered the advantage of ARM to be the reduction in the failure domain. Instead of one server with 2 processors, and 2 power supplies, in 1 case, running 48 disks, you can do 4 cases containing 8 power supplies, and 32 processors running 32 (or 64...) disks.
The architecture is different with ARM; you pair an ARM SoC up with just one or 2 disks, and you only run the OSD software.
Thank you,
Dominic L. Hilsbos, MBA Director – Information Technology Perform Air International Inc. DHilsbos@PerformAir.com www.PerformAir.com
-----Original Message----- From: Robert Sander [mailto:r.sander@heinlein-support.de] Sent: Tuesday, November 24, 2020 5:56 AM To: ceph-users@ceph.io Subject: [ceph-users] Re: Ceph on ARM ?
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
Has anyone tested Ceph in such scenario ? Is the Ceph software really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling such setups:
https://softiron.com/ https://www.ambedded.com.tw/
Regards -- Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-43 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
How does ARM compare to Xeon in latency and cluster utilization? -----Original Message-----; ceph-users Subject: [ceph-users] Re: Ceph on ARM ? Indeed it does run very happily on ARM. We have three of the Mars 400 appliances from Ambedded and they work exceedingly well. 8 micro servers per chassis. 1 is a MON server and the rest are OSD (or other services). Each micro server is running CentOS 7 and ceph so real easy to maintain. And each chassis is only 105W IIRC. -- *Kevin Thorpe* VP of Enterprise Platform *W* *|* www.predictx.com *P * *|* +44 (0)20 3005 6750 <+44%2020%203005%206750> | +44 (0)808 204 0344 <+44%20808%20204%200344> *A* *|* 7th Floor, 52 Grosvenor Gardens, London SW1W 0AU <https://maps.google.com/?q=7th+Floor,+52+Grosvenor+Gardens,+London+SW1W+0AU&entry=gmail&source=g> <http://twitter.com/predictxai> <https://www.linkedin.com/company/1339603/> _________________________________________________________ This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. On Tue, 24 Nov 2020 at 16:23, <DHilsbos@performair.com> wrote:
Adrian;
I've always considered the advantage of ARM to be the reduction in the
failure domain. Instead of one server with 2 processors, and 2 power supplies, in 1 case, running 48 disks, you can do 4 cases containing 8 power supplies, and 32 processors running 32 (or 64...) disks.
The architecture is different with ARM; you pair an ARM SoC up with just one or 2 disks, and you only run the OSD software.
Thank you,
Dominic L. Hilsbos, MBA Director – Information Technology Perform Air International Inc. DHilsbos@PerformAir.com www.PerformAir.com
-----Original Message----- From: Robert Sander [mailto:r.sander@heinlein-support.de] Sent: Tuesday, November 24, 2020 5:56 AM To: ceph-users@ceph.io Subject: [ceph-users] Re: Ceph on ARM ?
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
Has anyone tested Ceph in such scenario ? Is the Ceph software really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling
such setups:
https://softiron.com/ https://www.ambedded.com.tw/
Regards -- Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-43 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (11)
-
Adrian Nicolae
-
Anthony D'Atri
-
Danny Abukalam
-
Darren Soothill
-
DHilsbos＠performair.com
-
Kevin Thorpe
-
Marc Roos
-
Mark Nelson
-
Martin Verges
-
Peter Woodman
-
Robert Sander