Hi Anthony,
They do for sure. Note that when using these for WAL+DB offload, conventional wisdom is to not exceed 5:1 for SATA SSDs. 36:1 as you propose would likely yield *worse* performance than leaving WAL+DB colocated with the main OSD data
Thanks for the info, that's helpful to know.
I might consider constructing your CephFS with the metadata and first data pools on 3x replicated pools constrained to the SSD OSDs and a second data pool added for the HDDs, replicated or EC as you see fit. Then use the layout
I will explore this path.
Remember that Ceph does replication for you. This strategy is effectively RAID on top of RAID. It would overload your SATA SSDs, resulting in slow RADOS and MDS ops. Your Prom /Grafana would show false alarms. My Cephalocon presentation from 2024 demonstrates this dynamic, and I’ve seen it on community systems.
If mirrored, each of those SSDs would see the writes from 36 OSDs. Which would really be ugly.
The main motivation for RAID1 is having more resilience for the OS, so in case one SSD fails, the system stays online.
* Download SST from Solidigm and ensure the SSDs have up to date firmware. Or if Dell, DSU, etc.
Noted, thank you!
what I would do is mirror ~500 GiB for the OS, with minimal partioning and no swap, and use the balance on *each* SSD for an unmirrored OSD.
I tried that with my HBA, but it didn't work, it seems that MD is a better idea anyway. I will go ahead and try that. One thing that popped up in my mind, if I do this, the reamining unused space, e.g., /dev/sdX2 in both disks, will be *partitions*, and not *disks*. Does Ceph have a problem with using partitions for OSDs?
I think you’d not have a good experience, see my compromise suggestion above. Remind me how many chassis total? As a certain chassis count, the importance of mirrored boot (and network bonding) diminishes.
We have 13x chassis. I am already convinced to not have OSDs on VDs :-)
Does it show that the BBU is a supercap vs a battery?
It looks like supercap: Cachevault_Info : =============== -------------------- Property Value -------------------- Type CVPM05 Temperature 24 C State Optimal -------------------- ... GasGaugeStatus : ============== ------------------------------ Property Value ------------------------------ Pack Energy 136 J Capacitance 100 % Remaining Reserve Space 0 ------------------------------
When I see RAID HBAs I either just set passthrough and ignore the RoC
I think I'll do just that and just use MD for the small RAID volume for the OS. Thank you, Gustavo