The one side effect you might see if you actually create a million or more bucklets for a single user, is a large OMAP warning, as the mapping of buckets to owners is not sharded. Matt On Mon, May 24, 2021 at 1:51 AM Michal Strnad <michal.strnad@cesnet.cz> wrote:
Thank you. So we can create millions of buckets associated to only one S3 account without any limitation or side effect? Does anyone use it this way?
Many thanks in advance. Michal
On 5/23/21 9:42 PM, Janne Johansson wrote:
Many buckets.
Den sön 23 maj 2021 kl 20:53 skrev Michal Strnad <michal.strnad@cesnet.cz>:
Hi all,
We need to store millions of files using S3 protocol in Ceph (version Nautilus), but have projects where isn't appropriate or possible to create a lot of S3 accounts. Is it better to have multiple S3 buckets or one bucket with sub folders?
For example AWS service from Amazon allows you to create up to 100 buckets in each of your AWS cloud accounts. You can request more buckets, up to a maximum quota of 1,000, by submitting a service limit increase. There is no limit on the number of objects you can store in a bucket, but in the Ceph we run into a problem with listing and resharding with a millions of files in one bucket.
Thank you
Michal
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://www.redhat.com/en/technologies/storage tel. 734-821-5101 fax. 734-769-8938 cel. 734-216-5309