I think this depends on the type of backing disk. We use the following CPUs:
Intel(R) Xeon(R) CPU E5-2660 v4 @ 2.00GHz
Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz
My experience is, that a HDD OSD hardly gets to 100% of 1 hyper thread load even under
heavy recovery/rebalance operations on 8+2 and 6+2 EC pools with compression set to
aggressive. The CPU is mostly doing wait-IO, that is, the disk is the real bottle neck,
not the processor power. With SSDs I have seen 2HT at 100% and 2 more at 50% each. I guess
NVMe might be more demanding.
A server with 12 HDD and 1 SSD should be fine with a modern CPU with 8 cores. 16 threads
sounds like an 8 core CPU. The 2nd generation Intel® Xeon® Silver 4209T with 8 cores
should easily handle that (single socket system). We have the 16-core Intel silver in a
dual socket system currently connected to 5HDD and 7SSD and I did a rebalance operation
yesterday. The CPU user load did not exceed 2%, it can handle OSD processes easily. The
server is dimensioned to run up to 12HDD and 14SSD OSDs (Dell R740xd2). As far as I can
tell, the CPU configuration is overpowered for that.
Just for info, we use ganglia to record node utilisation. I use 1-year records and pick
peak loads I observed for dimensioning the CPUs. These records include some very heavy
recovery periods.
Best regards,
=================
Frank Schilder
AIT Risø Campus
Bygning 109, rum S14
________________________________________
From: Tony Liu <tonyliu0592(a)hotmail.com>
Sent: 13 November 2020 04:57:53
To: Nathan Fish
Cc: ceph-users(a)ceph.io
Subject: [ceph-users] Re: which of cpu frequency and number of threads servers osd
better?
Thanks Nathan!
Tony
-----Original Message-----
From: Nathan Fish <lordcirth(a)gmail.com>
Sent: Thursday, November 12, 2020 7:43 PM
To: Tony Liu <tonyliu0592(a)hotmail.com>
Cc: ceph-users(a)ceph.io
Subject: Re: [ceph-users] which of cpu frequency and number of threads
servers osd better?
From what I've seen, OSD daemons tend to bottleneck on the first 2
threads, while getting some use out of another 2. So 32 threads at 3.0
would be a lot better. Note that you may get better performance
splitting off some of that SSD for block.db partitions or at least
block.wal for the HDDs.
On Thu, Nov 12, 2020 at 9:57 PM Tony Liu <tonyliu0592(a)hotmail.com> wrote:
>
> Hi,
>
> For example, 16 threads with 3.2GHz and 32 threads with 3.0GHz, which
> makes 11 OSDs (10x12TB HDD and 1x960GB SSD) with better performance?
>
>
> Thanks!
> Tony
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an
> email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io