Also, only one Ethernet port.  

Worse yet they have *zero* HIPPI ports! Can you imagine!?

Never used HIPPI

Almost nobody has ;)

A 48-port gigabit managed switch is reasonably accessible to the home
gamer, both in terms of availability and cost.

USD249 from Netgear, interesting.

Second-hand 10GbE switches can be found for reasonable prices, but a
new one is pricey!  Too expensive for my liking.

FWIW the MikroTik units are surprisingly affordable, eg. 4x SFP+ for USD139.f

Why the fixation on multiple network interfaces?

… because Ceph needs one interface for the "public" network and one for
the "private" network?

A private / back-end / replication network is optional.  Ceph hums right along if one isn’t defined.  If you want to bond because GE is slow and you have a compatible switch, that’s a different thing.

Truth be told, I'd like to ditch the BMCs, but most BIOSes have a
fixation of needing a monitor and keyboard to configure them.  

Ask me sometime about my adventures with chassis manufacturers and serial consoles ;)

I haven't lost my mind…
...it's backed up on a tape somewhere.

Make sure you read those tapes, I had this DDS4 changer where the head got misaligned ….