Just to add our perspective here, I’d like to second Darren’s message here that Ceph is
great on ARM. We’re intensely familiar with this as we’ve been running it smoothly for
years. We have some very credible customer references that have been running our product
at scale for a while. I know there are others on this list that also run ARM at scale as
well.
There’s a strong argument to be made that it’s preferable to do things this way - power
efficiency (which can have an impact on rack density as well as cost) is one of those
arguments but it doesn’t tell the whole story - especially when you start asking how
running storage nodes hot impacts disk MTBF as just one example.
While we’d fix them if they existed, we’ve never hit the “crazy bugs” on ARM mentioned
above - I believe that Ceph’s CI builds against ARM, and we do this in our lab as well.
I can’t speak to your specific configuration and silicon, but if you’d like to hear more
about some of our customer stories I’m sure we can facilitate :)
Danny
On 24 Nov 2020, at 13:58, Darren Soothill
<darren.soothill(a)suse.com> wrote:
So yes you can get the servers for a considerably lower price than Intel.
Its not just about the CPU cost but many arm servers are based on a SOC that includes
networking so the overall cost of the motherboard/processor/networking is a lot lower.
It doesn't reduce the price of the storage or memory though which are a large part of
the cost.
Power consumption on ARM processors is considerably lower than on Intel. Think in the
order of 100W per server which adds up over the course of a year.
ARM have done work to extend the ISA-L so they have hardware accelerated things like
erasure coding in the same way that intel have.
Having done a lot of testing on ARM including IO500 work then I can certainly say it
works. We didn't hit any problems that where arm specific and certainly pushed things
performance wise.
Yes you have to consider clock speeds of the processors as quite a few things in CEPH are
single threaded but having all the extra cores means you can run many processes for things
like MDS and OSD’s.
Darren
From: Martin Verges <martin.verges(a)croit.io>
Date: Tuesday, 24 November 2020 at 13:09
To: Robert Sander <r.sander(a)heinlein-support.de>
Cc: ceph-users <ceph-users(a)ceph.io>
Subject: [ceph-users] Re: Ceph on ARM ?
Hello,
I'm curious however if the ARM servers are
better or not for this use case (object-storage only). For example, instead of using
2xSilver/Gold server, I can use a Taishan 5280 server with 2x Kungpen 920 ARM CPUs with up
to 128 cores in total . So I can have twice as many CPU cores (or even more) per server
comparing with x86. Probably the price is lower for the ARM servers as well.
Even if they would be cheaper, which I strongly doubt, you will get
less performance out of them. More cores won't give you any benefit in
Ceph, but having much faster cores is somewhat of a game changer. Just
use a good AMD Epyc for best price/performance/power ratio.
Has anyone tested Ceph in such scenario? Is the
Ceph software really optimised for the ARM architecture ? What do you think about this ?
If you choose ARM for your Ceph, you are one of very very few people
and will most properly hit some crazy bugs that will cause trouble. A
high price to pay in my opinion just for an "imaginary" performance or
power reduction benefit. Storage has to run 24*7 all year long without
a single incident. Everything else in my world is inacceptable.
--
Martin Verges
Managing director
Mobile: +49 174 9335695
E-Mail: martin.verges(a)croit.io
Chat:
https://t.me/MartinVerges
croit GmbH, Freseniusstr. 31h, 81247 Munich
CEO: Martin Verges - VAT-ID: DE310638492
Com. register: Amtsgericht Munich HRB 231263
Web:
https://croit.io
YouTube:
https://goo.gl/PGE1Bx
Am Di., 24. Nov. 2020 um 13:57 Uhr schrieb Robert Sander
<r.sander(a)heinlein-support.de>de>:
Am 24.11.20 um 13:12 schrieb Adrian Nicolae:
Has anyone tested Ceph in such scenario ? Is
the Ceph software
really optimised for the ARM architecture ?
Personally I have not run Ceph on ARM, but there are companies selling
such setups:
https://softiron.com/
https://www.ambedded.com.tw/
Regards
--
Robert Sander
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-43
Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io