[ceph-users] Adding a HDD "Tier" for low-perf storage

Hello! Since NVME drives become quite expensive, we are thinking about adding a "HDD" storage tier, for backups and low-performance storage (RBD only. Not RGW nor CephFS) Currently, we're running Squid (19.2.3) on 15 servers with Intel Xeon 6238 x 112 CPUs, 754 GiB RAM, 4 x 25 Gbps NIC interfaces (yes, they're not enough...). Ubuntu 22.04 Total OSDs: 423 x 3.5 TB ~ (1.9 PB) We want to add 5 servers, each with: - 12 x 20 TB HDD, - 4 NVME 2TB for WAL+DB, - 2 SSD for operative system, - 256 GB RAM, - 24 CPU - 4 x 25 Gbps NIC interfaces. One pool with replica 3 and another with EC 3+2 Please feel free to share any comments or experiences you may have on this topic—they are more than welcome. Thanks in advance! Javier.-