Thank you again Casey for putting us on the right track regarding the changes in multisite resharding support. When going through the various changelogs we didn't pay too much attention to those changes as this cluster doesn't use any multisite features. We have now upgraded to Reef and all buckets except one are usable again. The rados objects seem to still exist, so we might have luck with something like rgw-restore-bucket-index. On 03/04/2024 21:43, Casey Bodley wrote:
On Wed, Apr 3, 2024 at 3:09 PM Lorenz Bausch <info@lorenzbausch.de> wrote:
Hi Casey,
thank you so much for analysis! We tested the upgraded intensively, but the buckets in our test environment were probably too small to get dynamically resharded.
after upgrading to the Quincy release, rgw would look at the wrong object names when trying to list those buckets. As we're currently running Quincy, do you think objects/bucket indexes might already be altered in a way which makes them also unusable for Reef?
for multisite resharding support, the bucket instance metadata now stores an additional 'layout' structure which contains all of the information necessary to locate its bucket index objects. on reshard, the Red Hat pacific release would have stored that information with the bucket. the upstream Reef release should be able to interpret that layout data correctly
however, if the Quincy release overwrites that bucket instance metadata (via an operation like PutBucketAcl, PutBucketPolicy, etc), the corresponding layout information would be erased such that an upgrade to Reef would not be able to find the real bucket index shard objects
Kind regards, Lorenz _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io