The document of ceph osd config(
http://docs.ceph.com/docs/master/rados/configuration/osd-config-ref/#caveats)
said that using lower shard number may have deleterious effects. I want to
know what the deleterious effects are.
I test the performance of ceph cluster using different osd_op_num_shards
and different osd_op_num_threads_per_shard configuration. I found that Ceph
cluster will get bad performance if using lower shard number. And I also
found that using many thread number with lower shard number can get the
same improvement of performance with multi-shard. So I think I can use
lower shard number with many thread number per shard to replace many shard
number. But I don't know if there are other bad effects besides
bad-performance. I want to use dmclock feature in my ceph cluster but it
can get good effect only when using lower shard number.
The environment of my ceph cluster:
- 2 nodes
- 3 osds per node
- nvme ssd as osd
- CPU:Intel(R) Xeon(R) Gold 6130
- Mem:187GB
The main configuration of my ceph cluster:
auth cluster required = cephx
auth service required = cephx
auth client required = cephx
osd journal size = 1024
filestore xattr use omap = true
mon_allow_pool_delete=true
objecter_inflight_ops = 10240
objecter_inflight_op_bytes = 104857600
osd_pool_default_size=2
osd_pool_default_min_size=1
osd_op_queue_cut_off=high
osd_op_num_shards_ssd=1
osd_op_num_threads_per_shard_ssd=1
osd_op_queue=wpq
Thanks for assistance!
Liu Chaoyang