That is an interesting point. We are using 12 on 1
nvme journal for our Filestore nodes (which seems to work ok). The workload for wal + db
is different so that could be a factor. However when I've looked at the IO metrics for
the nvme it seems to be only lightly loaded, so does not appear to be the issue (at 1st
sight anyway).
How are you determining “lightly loaded”. Not iostat %util I hope.
Also the particular nvme model could be a factor (in
the 6 vs 12 area) - what type are you using?
I suspect that SSD OSDs would favor the 6 end of the scale; 12 doesn’t seem wholly
unreasonable for HDD OSDs. In the end though, add the hassle factor and the cost of the
NVMe device — when provisioned with enough capacity — and maybe just using SATA SSDs
makese sease.