RGW listing millions of objects takes too much time
9 Dec
2019
9 Dec
'19
3:46 p.m.
Dear All, I have almost 30 million objects and I want to list them and index them somewhere else, Im using boto3 with continuation Marker but it takes almost 9 hours can I run it in multiple threads to make it faster? what solution do you suggest to speedup this process, Thanks
2469
Age (days ago)
2469
Last active (days ago)
0 comments
1 participants
participants (1)
-
Arash Shams