Hello, Ceph users, how can I create a CRUSH rule for one replica on NVMe and the rest on HDDs, with all replicas on different hosts? One of my clusters has been upgraded to two brand new ~6TB NVMe disks together with previously used two 12TB HDDs per host. Right now I have two HDD-based OSDs per host, each with ~200GB partition on NVMe for metadata. The rest of NVMe is free, so I can now create two additional ~6TB NVMe OSDs per host. My question is how to create a CRUSH rule which would have three replicas on three different hosts, with one of these replicas on NVMe, and the remaining two on HDD. In the docs here: https://docs.ceph.com/en/latest/rados/operations/crush-map/#custom-crush-rul... there is an example of a SSD-first rule, which is close to what I would like to do, but it seems the SSD replica in that rule can end up on the same host as one of the HDD replicas. How can I avoid that? And a noob question -- how can I create a crush rule from the "rule something { ... }" text in that example? Thanks! -Yenya -- | Jan "Yenya" Kasprzak <kas at {fi.muni.cz - work | yenya.net - private}> | | https://www.fi.muni.cz/~kas/ GPG: 4096R/A45477D5 | I don't like Python; its lack of inline, anonymous, multi-statement functions makes me sad. --Eric Wastl