Hi,

We’re at 14.2.2

We just found a broken bucket index, trying to repair with the common commands ]# radosgw-admin bucket check --check-objects –fix finish instantly, but bucket should have near 60-70TB info.

[root@CEPH-MON01 home]# radosgw-admin bucket check --check-objects --bucket BUCKETNAME --debug_rgw=10

2019-09-05 20:08:13.908 7fdc8db37580 10  cannot find current period zonegroup using local zonegroup

2019-09-05 20:08:13.910 7fdc8db37580 10 Cannot find current period zone using local zone

2019-09-05 20:08:13.933 7fdc8db37580  2 all 8 watchers are set, enabling cache

2019-09-05 20:08:13.942 7fdc56ffd700  2 RGWDataChangesLog::ChangesRenewThread: start

2019-09-05 20:08:13.948 7fdc8db37580 10 cache get: name=default.rgw.data.root++BUCKETNAME : miss

2019-09-05 20:08:13.950 7fdc8db37580 10 cache put: name=default.rgw.data.root++BUCKETNAME info.flags=0x16

2019-09-05 20:08:13.950 7fdc8db37580 10 adding default.rgw.data.root++BUCKETNAME to cache LRU end

2019-09-05 20:08:13.950 7fdc8db37580 10 updating xattr: name=ceph.objclass.version bl.length()=42

2019-09-05 20:08:13.950 7fdc8db37580 10 cache get: name=default.rgw.data.root++BUCKETNAME : type miss (requested=0x11, cached=0x16)

2019-09-05 20:08:13.950 7fdc8db37580 10 cache put: name=default.rgw.data.root++BUCKETNAME info.flags=0x11

2019-09-05 20:08:13.950 7fdc8db37580 10 moving default.rgw.data.root++BUCKETNAME to cache LRU end

2019-09-05 20:08:13.950 7fdc8db37580 10 cache get: name=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4-fdc16f590a82.16313306.1 : miss

2019-09-05 20:08:13.951 7fdc8db37580 10 cache put: name=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4-fdc16f590a82.16313306.1 info.flags=0x16

2019-09-05 20:08:13.951 7fdc8db37580 10 adding default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4-fdc16f590a82.16313306.1 to cache LRU end

2019-09-05 20:08:13.951 7fdc8db37580 10 updating xattr: name=ceph.objclass.version bl.length()=42

2019-09-05 20:08:13.951 7fdc8db37580 10 updating xattr: name=user.rgw.acl bl.length()=145

2019-09-05 20:08:13.951 7fdc8db37580 10 updating xattr: name=user.rgw.lc bl.length()=467

2019-09-05 20:08:13.951 7fdc8db37580 10 cache get: name=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4-fdc16f590a82.16313306.1 : type miss (requested=0x13, cached=0x16)

2019-09-05 20:08:13.951 7fdc8db37580 10 cache put: name=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4-fdc16f590a82.16313306.1 info.flags=0x13

2019-09-05 20:08:13.951 7fdc8db37580 10 moving default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4-fdc16f590a82.16313306.1 to cache LRU end

2019-09-05 20:08:13.951 7fdc8db37580 10 updating xattr: name=ceph.objclass.version bl.length()=42

2019-09-05 20:08:13.951 7fdc8db37580 10 updating xattr: name=user.rgw.acl bl.length()=145

2019-09-05 20:08:13.951 7fdc8db37580 10 updating xattr: name=user.rgw.lc bl.length()=467

2019-09-05 20:08:13.951 7fdc8db37580 10 chain_cache_entry: cache_locator=default.rgw.data.root++BUCKETNAME

2019-09-05 20:08:13.951 7fdc8db37580 10 chain_cache_entry: cache_locator=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4-fdc16f590a82.16313306.1

2019-09-05 20:08:13.951 7fdc8db37580 10 cache get: name=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4-fdc16f590a82.16313306.1 : type miss (requested=0x16, cached=0x13)

2019-09-05 20:08:13.952 7fdc8db37580 10 cache put: name=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4-fdc16f590a82.16313306.1 info.flags=0x16

2019-09-05 20:08:13.952 7fdc8db37580 10 moving default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4-fdc16f590a82.16313306.1 to cache LRU end

2019-09-05 20:08:13.952 7fdc8db37580 10 updating xattr: name=ceph.objclass.version bl.length()=42

2019-09-05 20:08:13.952 7fdc8db37580 10 updating xattr: name=user.rgw.acl bl.length()=145

2019-09-05 20:08:13.952 7fdc8db37580 10 updating xattr: name=user.rgw.lc bl.length()=467

2019-09-05 20:08:13.952 7fdc8db37580 10 cache get: name=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4-fdc16f590a82.16313306.1 : hit (requested=0x11, cached=0x17)

2019-09-05 20:08:13.952 7fdc8db37580 10 cls_bucket_list_ordered BUCKETNAME[48efb8c3-693c-4fe0-bbe4-fdc16f590a82.3856921.7] start _multipart_[] num_entries 1001

2019-09-05 20:08:13.960 7fdc8db37580  2 removed watcher, disabling cache

Any recomendations for forcé recreate bucket index?

Regards

Manuel