What is the maximum number of Rados gateway objects in one cluster using the bucket index and in one bucket?
16 Nov
2023
16 Nov
'23
10:08 a.m.
Hello. We are using Ceph storage to test whether we can run the service by uploading and saving more than 40 billion files. So I'd like to check the contents below. 1) Maximum number of Rados gateway objects that can be stored in one cluster using the bucket index 2) Maximum number of Rados gateway objects that can be stored in one bucket Although we have referred to the limitations on the number of Rados gateway objects mentioned in existing documents, it seems theoretically unlimited If you have operated the number of files at the level we think in actual services or products, we would appreciate it if you could share them. Below are related documents and related settings values. > Related documents - https://documentation.suse.com/ses/5.5/html/ses-all/cha-ceph-gw.html - https://www.ibm.com/docs/en/storage-ceph/6?topic=resharding-limitations-bucket-index - https://docs.ceph.com/en/latest/dev/radosgw/bucket_index/ > Related config - rgw_dynamic_resharding: true - rgw_max_objs_per_shard: 100000 - rgw_max_dynamic_shards : 65521
2 Jan
2 Jan
2:20 a.m.
hi steveļ¼ I saw your configuration, and through calculation, I feel it cannot handle 40 billion files. rgw_max_objs_per_shard * rgw_max_dynamic_shards = 6552100000 6552100000/10000/10000 = 6.5521 billion
985
Age (days ago)
1032
Last active (days ago)
1 comments
2 participants
participants (2)
-
changzhi tan
-
steve jung