I think this depends on the type of backing disk. We use the following CPUs: Intel(R) Xeon(R) CPU E5-2660 v4 @ 2.00GHz Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz My experience is, that a HDD OSD hardly gets to 100% of 1 hyper thread load even under heavy recovery/rebalance operations on 8+2 and 6+2 EC pools with compression set to aggressive. The CPU is mostly doing wait-IO, that is, the disk is the real bottle neck, not the processor power. With SSDs I have seen 2HT at 100% and 2 more at 50% each. I guess NVMe might be more demanding. A server with 12 HDD and 1 SSD should be fine with a modern CPU with 8 cores. 16 threads sounds like an 8 core CPU. The 2nd generation Intel® Xeon® Silver 4209T with 8 cores should easily handle that (single socket system). We have the 16-core Intel silver in a dual socket system currently connected to 5HDD and 7SSD and I did a rebalance operation yesterday. The CPU user load did not exceed 2%, it can handle OSD processes easily. The server is dimensioned to run up to 12HDD and 14SSD OSDs (Dell R740xd2). As far as I can tell, the CPU configuration is overpowered for that. Just for info, we use ganglia to record node utilisation. I use 1-year records and pick peak loads I observed for dimensioning the CPUs. These records include some very heavy recovery periods. Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Tony Liu <tonyliu0592@hotmail.com> Sent: 13 November 2020 04:57:53 To: Nathan Fish Cc: ceph-users@ceph.io Subject: [ceph-users] Re: which of cpu frequency and number of threads servers osd better? Thanks Nathan! Tony
-----Original Message----- From: Nathan Fish <lordcirth@gmail.com> Sent: Thursday, November 12, 2020 7:43 PM To: Tony Liu <tonyliu0592@hotmail.com> Cc: ceph-users@ceph.io Subject: Re: [ceph-users] which of cpu frequency and number of threads servers osd better?
From what I've seen, OSD daemons tend to bottleneck on the first 2 threads, while getting some use out of another 2. So 32 threads at 3.0 would be a lot better. Note that you may get better performance splitting off some of that SSD for block.db partitions or at least block.wal for the HDDs.
On Thu, Nov 12, 2020 at 9:57 PM Tony Liu <tonyliu0592@hotmail.com> wrote:
Hi,
For example, 16 threads with 3.2GHz and 32 threads with 3.0GHz, which makes 11 OSDs (10x12TB HDD and 1x960GB SSD) with better performance?
Thanks! Tony _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io