24 Nov
2020
24 Nov
'20
1:12 p.m.
Hi guys,
I was looking at some Huawei ARM-based servers and the datasheets are
very interesting. The high CPU core numbers and the SoC architecture
should be ideal for a distributed storage like Ceph, at least in theory.
I'm planning to build a new Ceph cluster in the future and my best
case scenario right now is to buy 7 servers with 2 x Intel Silver
12-core or 2 x Gold 20-core CPUs, 32 SATA drives each. And of course -
SSDs/NVMEs for wal/db and the metadata pools , 256GB RAM and so on.
I'm curious however if the ARM servers are better or not for this use
case (object-storage only). For example, instead of using 2xSilver/Gold
server, I can use a Taishan 5280 server with 2x Kungpen 920 ARM CPUs
with up to 128 cores in total . So I can have twice as many CPU cores
(or even more) per server comparing with x86. Probably the price is
lower for the ARM servers as well.
Has anyone tested Ceph in such scenario ? Is the Ceph software
really optimised for the ARM architecture ? What do you think about this ?
Thanks !
24 Nov
24 Nov
1:48 p.m.
At least in the past, there have been a couple of things you really want
to focus on regarding ARM and performance (beyond the obvious core
count/clockspeed/ipc/etc):
1) HW acceleration for things like CRC32, MD5, etc
2) context switching overhead
3) memory throughput
4) SATA controller and PCIe throughput
Unfortunately these are the kinds of things that you can't easily
generalize between ARM vs x86. Some ARM processors are going to do
wildly better than others. The good news is that driving 32 SATA drives
shouldn't be particularly CPU intensive (128 modern ARM cores is
probably sufficient) so the real question is probably what the SATA
controller setup looks like. I think you're best bet if you are serious
about pursuing the ARM route is try to get access to one of the Taishan
5280 servers and do some tests. You could try renting out an EC2 ARM
systems and play around there but I doubt it will be a very
apples->apples comparison.
Mark
On 11/24/20 6:12 AM, Adrian Nicolae wrote:
Hi guys,
I was looking at some Huawei ARM-based servers and the datasheets are
very interesting. The high CPU core numbers and the SoC architecture
should be ideal for a distributed storage like Ceph, at least in theory.
I'm planning to build a new Ceph cluster in the future and my best
case scenario right now is to buy 7 servers with 2 x Intel Silver
12-core or 2 x Gold 20-core CPUs, 32 SATA drives each. And of course -
SSDs/NVMEs for wal/db and the metadata pools , 256GB RAM and so on.
I'm curious however if the ARM servers are better or not for this use
case (object-storage only). For example, instead of using
2xSilver/Gold server, I can use a Taishan 5280 server with 2x Kungpen
920 ARM CPUs with up to 128 cores in total . So I can have twice as
many CPU cores (or even more) per server comparing with x86. Probably
the price is lower for the ARM servers as well.
Has anyone tested Ceph in such scenario ? Is the Ceph software
really optimised for the ARM architecture ? What do you think about
this ?
Thanks !
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
8:29 p.m.
I had hoped to stay out of this, but here I go.
4) SATA controller and PCIe throughput
SoftIron claims “wire speed” with their custom hardware FWIW.
Unfortunately these are the kinds of things that you
can't easily generalize between ARM vs x86. Some ARM processors are going to do
wildly better than others.
Absolutely, just like an Intel Atom will deliver a different experience than a high-end
Xeon. Those of us in the US, though, can see markedly different pricing and product
availability than other countries.
> Hi guys,
>
> I was looking at some Huawei ARM-based servers and the datasheets are very
interesting. The high CPU core numbers and the SoC architecture should be ideal for a
distributed storage like Ceph, at least in theory.
>
> I'm planning to build a new Ceph cluster in the future and my best case scenario
right now is to buy 7 servers with 2 x Intel Silver 12-core or 2 x Gold 20-core CPUs, 32
SATA drives each. And of course - SSDs/NVMEs for wal/db and the metadata pools , 256GB RAM
and so on.
Consider if a single-socket Epyc would let you upgrade to QLC drives, save all the
external WAL/DB hassles and potentially the need for an HBA at all.
> I'm curious however if the ARM servers are
better or not for this use case (object-storage only).
Everyone’s needs are unique, but my sense is that object storage can be a reasonable fit
for the right ARM gear if your use-case cares more about throughput than latency or IOPS.
> For example, instead of using 2xSilver/Gold
server, I can use a Taishan 5280 server with 2x Kungpen 920 ARM CPUs with up to 128 cores
in total . So I can have twice as many CPU cores (or even more) per server comparing with
x86.
More but slower cores work well in some cases, and are miserable in others. cf. Sun’s CMT
systems from last decade. Conventional wisdom is that for CephFS MDS, for example, you
want fewer but faster cores, so that’s probably an application for which such a CPU isn’t
a good fit.
> Probably the price is lower for the ARM servers
as well.
That’s a function in part of your haggling skills. Oh and power (and thus cooling
reduction) is not imaginary. Time and again I’ve seen DC racks only partially populated
because they ran out of amps before RUs. Racks, RUs, TORs, switch ports, these things all
cost money. Like when people think cheap 1TB drives are a bargain, you have to look at
TCO, including power, heat, and how many TB you can fit in a chassis and a rack.
> Has anyone tested Ceph in such scenario ? Is
the Ceph software really optimised for the ARM architecture ? What do you think about
this ?
SoftIron gear has been running production Ceph for a few years. They tell me the code is
unmodified.
1:56 p.m.
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
Has anyone tested Ceph in such scenario ? Is the
Ceph software
really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling
such setups:
https://softiron.com/
https://www.ambedded.com.tw/
Regards
--
Robert Sander
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-43
Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin
2:08 p.m.
Hello,
I'm curious however if the ARM servers are better
or not for this use case (object-storage only). For example, instead of using
2xSilver/Gold server, I can use a Taishan 5280 server with 2x Kungpen 920 ARM CPUs with up
to 128 cores in total . So I can have twice as many CPU cores (or even more) per server
comparing with x86. Probably the price is lower for the ARM servers as well.
Even if they would be cheaper, which I strongly doubt, you will get
less performance out of them. More cores won't give you any benefit in
Ceph, but having much faster cores is somewhat of a game changer. Just
use a good AMD Epyc for best price/performance/power ratio.
Has anyone tested Ceph in such scenario? Is the Ceph
software really optimised for the ARM architecture ? What do you think about this ?
If you choose ARM for your Ceph, you are one of very very few people
and will most properly hit some crazy bugs that will cause trouble. A
high price to pay in my opinion just for an "imaginary" performance or
power reduction benefit. Storage has to run 24*7 all year long without
a single incident. Everything else in my world is inacceptable.
--
Martin Verges
Managing director
Mobile: +49 174 9335695
E-Mail: martin.verges(a)croit.io
Chat: https://t.me/MartinVerges
croit GmbH, Freseniusstr. 31h, 81247 Munich
CEO: Martin Verges - VAT-ID: DE310638492
Com. register: Amtsgericht Munich HRB 231263
Web: https://croit.io
YouTube: https://goo.gl/PGE1Bx
Am Di., 24. Nov. 2020 um 13:57 Uhr schrieb Robert Sander
<r.sander(a)heinlein-support.de>de>:
>
> Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
>
> > Has anyone tested Ceph in such scenario ? Is the Ceph software
> > really optimised for the ARM architecture ?
>
> Personally I have not run Ceph on ARM, but there are companies selling
> such setups:
>
> https://softiron.com/
> https://www.ambedded.com.tw/
>
> Regards
> --
> Robert Sander
> Heinlein Support GmbH
> Schwedter Str. 8/9b, 10119 Berlin
>
> http://www.heinlein-support.de
>
> Tel: 030 / 405051-43
> Fax: 030 / 405051-19
>
> Zwangsangaben lt. §35a GmbHG:
> HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
> Geschäftsführer: Peer Heinlein -- Sitz: Berlin
>
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
2:58 p.m.
So yes you can get the servers for a considerably lower price than Intel.
Its not just about the CPU cost but many arm servers are based on a SOC that includes
networking so the overall cost of the motherboard/processor/networking is a lot lower.
It doesn't reduce the price of the storage or memory though which are a large part of
the cost.
Power consumption on ARM processors is considerably lower than on Intel. Think in the
order of 100W per server which adds up over the course of a year.
ARM have done work to extend the ISA-L so they have hardware accelerated things like
erasure coding in the same way that intel have.
Having done a lot of testing on ARM including IO500 work then I can certainly say it
works. We didn't hit any problems that where arm specific and certainly pushed things
performance wise.
Yes you have to consider clock speeds of the processors as quite a few things in CEPH are
single threaded but having all the extra cores means you can run many processes for things
like MDS and OSD’s.
Darren
From: Martin Verges <martin.verges(a)croit.io>
Date: Tuesday, 24 November 2020 at 13:09
To: Robert Sander <r.sander(a)heinlein-support.de>
Cc: ceph-users <ceph-users(a)ceph.io>
Subject: [ceph-users] Re: Ceph on ARM ?
Hello,
I'm curious however if the ARM servers are better
or not for this use case (object-storage only). For example, instead of using
2xSilver/Gold server, I can use a Taishan 5280 server with 2x Kungpen 920 ARM CPUs with up
to 128 cores in total . So I can have twice as many CPU cores (or even more) per server
comparing with x86. Probably the price is lower for the ARM servers as well.
Even if they would be cheaper, which I strongly doubt, you will get
less performance out of them. More cores won't give you any benefit in
Ceph, but having much faster cores is somewhat of a game changer. Just
use a good AMD Epyc for best price/performance/power ratio.
Has anyone tested Ceph in such scenario? Is the Ceph
software really optimised for the ARM architecture ? What do you think about this ?
If you choose ARM for your Ceph, you are one of very very few people
and will most properly hit some crazy bugs that will cause trouble. A
high price to pay in my opinion just for an "imaginary" performance or
power reduction benefit. Storage has to run 24*7 all year long without
a single incident. Everything else in my world is inacceptable.
--
Martin Verges
Managing director
Mobile: +49 174 9335695
E-Mail: martin.verges(a)croit.io
Chat: https://t.me/MartinVerges
croit GmbH, Freseniusstr. 31h, 81247 Munich
CEO: Martin Verges - VAT-ID: DE310638492
Com. register: Amtsgericht Munich HRB 231263
Web: https://croit.io
YouTube: https://goo.gl/PGE1Bx
Am Di., 24. Nov. 2020 um 13:57 Uhr schrieb Robert Sander
<r.sander(a)heinlein-support.de>de>:
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
Has anyone tested Ceph in such scenario ? Is
the Ceph software
really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling
such setups:
https://softiron.com/
https://www.ambedded.com.tw/
Regards
--
Robert Sander
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-43
Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
7:52 p.m.
Just to add our perspective here, I’d like to second Darren’s message here that Ceph is
great on ARM. We’re intensely familiar with this as we’ve been running it smoothly for
years. We have some very credible customer references that have been running our product
at scale for a while. I know there are others on this list that also run ARM at scale as
well.
There’s a strong argument to be made that it’s preferable to do things this way - power
efficiency (which can have an impact on rack density as well as cost) is one of those
arguments but it doesn’t tell the whole story - especially when you start asking how
running storage nodes hot impacts disk MTBF as just one example.
While we’d fix them if they existed, we’ve never hit the “crazy bugs” on ARM mentioned
above - I believe that Ceph’s CI builds against ARM, and we do this in our lab as well.
I can’t speak to your specific configuration and silicon, but if you’d like to hear more
about some of our customer stories I’m sure we can facilitate :)
Danny
On 24 Nov 2020, at 13:58, Darren Soothill
<darren.soothill(a)suse.com> wrote:
So yes you can get the servers for a considerably lower price than Intel.
Its not just about the CPU cost but many arm servers are based on a SOC that includes
networking so the overall cost of the motherboard/processor/networking is a lot lower.
It doesn't reduce the price of the storage or memory though which are a large part of
the cost.
Power consumption on ARM processors is considerably lower than on Intel. Think in the
order of 100W per server which adds up over the course of a year.
ARM have done work to extend the ISA-L so they have hardware accelerated things like
erasure coding in the same way that intel have.
Having done a lot of testing on ARM including IO500 work then I can certainly say it
works. We didn't hit any problems that where arm specific and certainly pushed things
performance wise.
Yes you have to consider clock speeds of the processors as quite a few things in CEPH are
single threaded but having all the extra cores means you can run many processes for things
like MDS and OSD’s.
Darren
From: Martin Verges <martin.verges(a)croit.io>
Date: Tuesday, 24 November 2020 at 13:09
To: Robert Sander <r.sander(a)heinlein-support.de>
Cc: ceph-users <ceph-users(a)ceph.io>
Subject: [ceph-users] Re: Ceph on ARM ?
Hello,
I'm curious however if the ARM servers are
better or not for this use case (object-storage only). For example, instead of using
2xSilver/Gold server, I can use a Taishan 5280 server with 2x Kungpen 920 ARM CPUs with up
to 128 cores in total . So I can have twice as many CPU cores (or even more) per server
comparing with x86. Probably the price is lower for the ARM servers as well.
Even if they would be cheaper, which I strongly doubt, you will get
less performance out of them. More cores won't give you any benefit in
Ceph, but having much faster cores is somewhat of a game changer. Just
use a good AMD Epyc for best price/performance/power ratio.
Has anyone tested Ceph in such scenario? Is the
Ceph software really optimised for the ARM architecture ? What do you think about this ?
If you choose ARM for your Ceph, you are one of very very few people
and will most properly hit some crazy bugs that will cause trouble. A
high price to pay in my opinion just for an "imaginary" performance or
power reduction benefit. Storage has to run 24*7 all year long without
a single incident. Everything else in my world is inacceptable.
--
Martin Verges
Managing director
Mobile: +49 174 9335695
E-Mail: martin.verges(a)croit.io
Chat: https://t.me/MartinVerges
croit GmbH, Freseniusstr. 31h, 81247 Munich
CEO: Martin Verges - VAT-ID: DE310638492
Com. register: Amtsgericht Munich HRB 231263
Web: https://croit.io
YouTube: https://goo.gl/PGE1Bx
Am Di., 24. Nov. 2020 um 13:57 Uhr schrieb Robert Sander
<r.sander(a)heinlein-support.de>de>:
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io Has anyone tested Ceph in such scenario ? Is
the Ceph software
really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling
such setups:
https://softiron.com/
https://www.ambedded.com.tw/
Regards
--
Robert Sander
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-43
Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io 
5:23 p.m.
Adrian;
I've always considered the advantage of ARM to be the reduction in the failure domain.
Instead of one server with 2 processors, and 2 power supplies, in 1 case, running 48
disks, you can do 4 cases containing 8 power supplies, and 32 processors running 32 (or
64...) disks.
The architecture is different with ARM; you pair an ARM SoC up with just one or 2 disks,
and you only run the OSD software.
Thank you,
Dominic L. Hilsbos, MBA
Director – Information Technology
Perform Air International Inc.
DHilsbos(a)PerformAir.com
www.PerformAir.com
-----Original Message-----
From: Robert Sander [mailto:r.sander@heinlein-support.de]
Sent: Tuesday, November 24, 2020 5:56 AM
To: ceph-users(a)ceph.io
Subject: [ceph-users] Re: Ceph on ARM ?
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
Has anyone tested Ceph in such scenario ? Is the
Ceph software
really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling such setups:
https://softiron.com/
https://www.ambedded.com.tw/
Regards
--
Robert Sander
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-43
Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin
5:36 p.m.
I've been running ceph on a heterogeneous mix of rock64 and rpi4 SBCs. i've
had to do my own builds, as the upstream ones started off with thunked-out
checksumming due to (afaict) different arm feature sets between upstream's
build targets and my SBCs, but other than that one, haven't run into any
arm-specific issues. i've had to clamp down cache size to avoid memory
exhaustion on the smaller boards, and ran into some corruption due to
seemingly a bad interaction between o_direct and zram swap on a particular
kernel version, but those aren't problems unique to ceph on arm.
i should also mention that i'm really very tolerant of latency with this
cluster, as this is just some homelab garbage running on the cheapest
slowest spinning disk available, so there's some bounds on what i'm
asserting.
On Tue, Nov 24, 2020 at 11:24 AM <DHilsbos(a)performair.com> wrote:
Adrian;
I've always considered the advantage of ARM to be the reduction in the
failure domain. Instead of one server with 2 processors, and 2 power
supplies, in 1 case, running 48 disks, you can do 4 cases containing 8
power supplies, and 32 processors running 32 (or 64...) disks.
The architecture is different with ARM; you pair an ARM SoC up with just
one or 2 disks, and you only run the OSD software.
Thank you,
Dominic L. Hilsbos, MBA
Director – Information Technology
Perform Air International Inc.
DHilsbos(a)PerformAir.com
www.PerformAir.com
-----Original Message-----
From: Robert Sander [mailto:r.sander@heinlein-support.de]
Sent: Tuesday, November 24, 2020 5:56 AM
To: ceph-users(a)ceph.io
Subject: [ceph-users] Re: Ceph on ARM ?
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
Has anyone tested Ceph in such scenario ? Is
the Ceph software
really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling
such setups:
https://softiron.com/
https://www.ambedded.com.tw/
Regards
--
Robert Sander
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-43
Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
25 Nov
25 Nov
9:30 a.m.
Hi guys,
Thank you all for your input. I will try to get my hands on a few Huawei
servers for testing before deciding.
The specs are really interesting, it looks like they have some power in
them besides the high core numbers (2.6 Ghz) :
https://en.wikichip.org/wiki/hisilicon/kunpeng/920-6426
https://e.huawei.com/en/products/servers/taishan-server/taishan-5280-v2
I'll let you know how it goes.
On 11/24/2020 6:36 PM, Peter Woodman wrote:
I've been running ceph on a heterogeneous mix of
rock64 and rpi4 SBCs.
i've had to do my own builds, as the upstream ones started off with
thunked-out checksumming due to (afaict) different arm feature sets
between upstream's build targets and my SBCs, but other than that one,
haven't run into any arm-specific issues. i've had to clamp down cache
size to avoid memory exhaustion on the smaller boards, and ran into
some corruption due to seemingly a bad interaction between o_direct
and zram swap on a particular kernel version, but those aren't
problems unique to ceph on arm.
i should also mention that i'm really very tolerant of latency with
this cluster, as this is just some homelab garbage running on the
cheapest slowest spinning disk available, so there's some bounds on
what i'm asserting.
On Tue, Nov 24, 2020 at 11:24 AM <DHilsbos(a)performair.com
<mailto:DHilsbos@performair.com>> wrote:
Adrian;
I've always considered the advantage of ARM to be the reduction in
the failure domain. Instead of one server with 2 processors, and
2 power supplies, in 1 case, running 48 disks, you can do 4 cases
containing 8 power supplies, and 32 processors running 32 (or
64...) disks.
The architecture is different with ARM; you pair an ARM SoC up
with just one or 2 disks, and you only run the OSD software.
Thank you,
Dominic L. Hilsbos, MBA
Director – Information Technology
Perform Air International Inc.
DHilsbos(a)PerformAir.com
www.PerformAir.com <http://www.PerformAir.com>
-----Original Message-----
From: Robert Sander [mailto:r.sander@heinlein-support.de
<mailto:r.sander@heinlein-support.de>]
Sent: Tuesday, November 24, 2020 5:56 AM
To: ceph-users(a)ceph.io <mailto:ceph-users@ceph.io>
Subject: [ceph-users] Re: Ceph on ARM ?
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
Has anyone tested Ceph in such scenario ? Is
the Ceph software
really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies
selling such setups:
https://softiron.com/ <https://softiron.com/>
https://www.ambedded.com.tw/ <https://www.ambedded.com.tw/>
Regards
--
Robert Sander
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de <http://www.heinlein-support.de>
Tel: 030 / 405051-43
Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
<mailto:ceph-users@ceph.io>
To unsubscribe send an email to ceph-users-leave(a)ceph.io
<mailto:ceph-users-leave@ceph.io>
10:19 a.m.
Indeed it does run very happily on ARM. We have three of the Mars 400
appliances from Ambedded and they work exceedingly well. 8 micro servers
per chassis. 1 is a MON server and the rest are OSD (or other services).
Each micro server is running CentOS 7 and ceph so real easy to maintain.
And each chassis is only 105W IIRC.
--
*Kevin Thorpe*
VP of Enterprise Platform
*W* *|* www.predictx.com
*P * *|* +44 (0)20 3005 6750 <+44%2020%203005%206750> | +44 (0)808 204 0344
<+44%20808%20204%200344>
*A* *|* 7th Floor, 52 Grosvenor Gardens, London SW1W 0AU
<https://maps.google.com/?q=7th+Floor,+52+Grosvenor+Gardens,+London+SW1W+0AU&entry=gmail&source=g>
<http://twitter.com/predictxai>
<https://www.linkedin.com/company/1339603/>
_________________________________________________________
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system manager.
This message contains confidential information and is intended only for the
individual named. If you are not the named addressee you should not
disseminate, distribute or copy this e-mail. Please notify the sender
immediately by e-mail if you have received this e-mail by mistake and
delete this e-mail from your system. If you are not the intended recipient
you are notified that disclosing, copying, distributing or taking any
action in reliance on the contents of this information is strictly
prohibited.
On Tue, 24 Nov 2020 at 16:23, <DHilsbos(a)performair.com> wrote:
Adrian;
I've always considered the advantage of ARM to be the reduction in the
failure domain. Instead of one server with 2 processors, and 2 power
supplies, in 1 case, running 48 disks, you can do 4 cases containing 8
power supplies, and 32 processors running 32 (or 64...) disks.
The architecture is different with ARM; you pair an ARM SoC up with just
one or 2 disks, and you only run the OSD software.
Thank you,
Dominic L. Hilsbos, MBA
Director – Information Technology
Perform Air International Inc.
DHilsbos(a)PerformAir.com
www.PerformAir.com
-----Original Message-----
From: Robert Sander [mailto:r.sander@heinlein-support.de]
Sent: Tuesday, November 24, 2020 5:56 AM
To: ceph-users(a)ceph.io
Subject: [ceph-users] Re: Ceph on ARM ?
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
Has anyone tested Ceph in such scenario ? Is
the Ceph software
really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling
such setups:
https://softiron.com/
https://www.ambedded.com.tw/
Regards
--
Robert Sander
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-43
Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
10:35 a.m.
How does ARM compare to Xeon in latency and cluster utilization?
-----Original Message-----; ceph-users
Subject: [ceph-users] Re: Ceph on ARM ?
Indeed it does run very happily on ARM. We have three of the Mars 400
appliances from Ambedded and they work exceedingly well. 8 micro servers
per chassis. 1 is a MON server and the rest are OSD (or other services).
Each micro server is running CentOS 7 and ceph so real easy to maintain.
And each chassis is only 105W IIRC.
--
*Kevin Thorpe*
VP of Enterprise Platform
*W* *|* www.predictx.com
*P * *|* +44 (0)20 3005 6750 <+44%2020%203005%206750> | +44 (0)808 204
0344 <+44%20808%20204%200344>
*A* *|* 7th Floor, 52 Grosvenor Gardens, London SW1W 0AU
<https://maps.google.com/?q=7th+Floor,+52+Grosvenor+Gardens,+London+SW1W+0AU&entry=gmail&source=g>
<http://twitter.com/predictxai>
<https://www.linkedin.com/company/1339603/>
_________________________________________________________
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they are
addressed.
If you have received this email in error please notify the system
manager.
This message contains confidential information and is intended only for
the individual named. If you are not the named addressee you should not
disseminate, distribute or copy this e-mail. Please notify the sender
immediately by e-mail if you have received this e-mail by mistake and
delete this e-mail from your system. If you are not the intended
recipient you are notified that disclosing, copying, distributing or
taking any action in reliance on the contents of this information is
strictly prohibited.
On Tue, 24 Nov 2020 at 16:23, <DHilsbos(a)performair.com> wrote:
Adrian;
I've always considered the advantage of ARM to be the reduction in the
failure domain. Instead of one server with 2
processors, and 2 power
supplies, in 1 case, running 48 disks, you can do 4 cases containing
8 power supplies, and 32 processors running 32 (or 64...) disks.
The architecture is different with ARM; you pair an ARM SoC up with
just one or 2 disks, and you only run the OSD software.
Thank you,
Dominic L. Hilsbos, MBA
Director – Information Technology
Perform Air International Inc.
DHilsbos(a)PerformAir.com
www.PerformAir.com
-----Original Message-----
From: Robert Sander [mailto:r.sander@heinlein-support.de]
Sent: Tuesday, November 24, 2020 5:56 AM
To: ceph-users(a)ceph.io
Subject: [ceph-users] Re: Ceph on ARM ?
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
Has anyone tested Ceph in such scenario ? Is
the Ceph software
really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling such setups:
https://softiron.com/
https://www.ambedded.com.tw/
Regards
--
Robert Sander
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-43
Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an
email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an
email to ceph-users-leave(a)ceph.io
1258
days inactive
1258
days old
11 comments
11 participants
participants (11)
-
Adrian Nicolae -
Anthony D'Atri -
Danny Abukalam -
Darren Soothill -
DHilsbos＠performair.com -
Kevin Thorpe -
Marc Roos -
Mark Nelson -
Martin Verges -
Peter Woodman -
Robert Sander