RGW listing millions of objects takes too much time
9 Dec
2019
9 Dec
'19
10:46 a.m.
Dear All, I have almost 30 million objects and I want to list them and index them somewhere else, Im using boto3 with continuation Marker but it takes almost 9 hours can I run it in multiple threads to make it faster? what solution do you suggest to speedup this process, Thanks
2497
Age (days ago)
2497
Last active (days ago)
0 comments
1 participants
participants (1)
-
Arash Shams