AFAIK the standing recommendation for all flash setups is to prefer fewer but faster cores, so something like a 75F3 might be yielding better latency. Plus you probably want to experiment with partitioning the NVMEs and running multiple OSDs per drive - either 2 or 4. On Sat, 7 Oct 2023 at 08:23, Gustavo Fahnle <gfahnle@nubi2go.com> wrote:
Hi,
Currently, I have an OpenStack installation with a Ceph cluster consisting of 4 servers for OSD, each with 16TB SATA HDDs. My intention is to add a second, independent Ceph cluster to provide faster disks for OpenStack VMs. The idea for this second cluster is to exclusively provide RBD services to OpenStack. I plan to start with a cluster composed of 3 mon/mgr nodes similar to what we currently have (3 virtualized servers with VMware) with 4 cores, 8GB of memory, 80GB disk and 10GB network each server. In the current cluster, these nodes have low resource consumption, less than 10% CPU usage, 40% memory usage, and less than 100Mb/s of network usage.
For the OSDs, I'm thinking of starting with 3 or 4 servers, specifically Supermicro AS-1114S-WN10RT, each with:
1 AMD EPYC 7713P Gen 3 processor (64 Core, 128 Threads, 2.0GHz) 256GB of RAM 2 x NVME 1TB for the operating system 10 x NVME Kingston DC1500M U.2 7.68TB for the OSDs Two Intel NIC E810-XXVDA2 25GbE Dual Port (2 x SFP28) PCIe 4.0 x8 cards Connected to 2 MikroTik CRS518-16XS-2XQ-RM switches at 100GbE per server Connection to OpenStack would be via 4 x 10GB to our core switch.
I would like to hear opinions about this configuration, recommendations, criticisms, etc.
If any of you have references or experience with any of the components in this initial configuration, they would be very welcome.
Thank you very much in advance.
Gustavo Fahnle
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io