What are the deleterious effects when using lower shard number on ceph osd?
The document of ceph osd config( http://docs.ceph.com/docs/master/rados/configuration/osd-config-ref/#caveats) said that using lower shard number may have deleterious effects. I want to know what the deleterious effects are. I test the performance of ceph cluster using different osd_op_num_shards and different osd_op_num_threads_per_shard configuration. I found that Ceph cluster will get bad performance if using lower shard number. And I also found that using many thread number with lower shard number can get the same improvement of performance with multi-shard. So I think I can use lower shard number with many thread number per shard to replace many shard number. But I don't know if there are other bad effects besides bad-performance. I want to use dmclock feature in my ceph cluster but it can get good effect only when using lower shard number. The environment of my ceph cluster: - 2 nodes - 3 osds per node - nvme ssd as osd - CPU:Intel(R) Xeon(R) Gold 6130 - Mem:187GB The main configuration of my ceph cluster: auth cluster required = cephx auth service required = cephx auth client required = cephx osd journal size = 1024 filestore xattr use omap = true mon_allow_pool_delete=true objecter_inflight_ops = 10240 objecter_inflight_op_bytes = 104857600 osd_pool_default_size=2 osd_pool_default_min_size=1 osd_op_queue_cut_off=high osd_op_num_shards_ssd=1 osd_op_num_threads_per_shard_ssd=1 osd_op_queue=wpq Thanks for assistance! Liu Chaoyang
participants (1)
-
刘朝洋