Hi, You will get slow performance: EC is slow, HDD are slow too With 400 iops per device, you get 89600 iops for the whole cluster, raw With 8+3EC, each logical write is mapped to 11 physical writes You get only 8145 write IOPS (is my math correct ?), which I find very low for a PB storage So, use flash-based storage, or at least not EC (which is for cold storage)
* We're used to run mons and mgrs daemons on a few of our OSD nodes, without any issue so far : is this a bad idea for a big cluster ? Nothing troublesome here
* We thought using cache tiering on an SSD pool, but a large part of the PB is used on a daily basis, so we expect the cache to be not so effective and really expensive ? Cache tiering is not recommended You will not be able to manage this cache (managing which content needs to be cached), thus degrading performances
* Could a 2x10G network be enough ? If you buy hardware, get 25Gbps
Thanks for reading, and sharing your experiences !
F.
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io