[ceph-users] RGW bucket check --check-objects -fix failed

Hi, We’re at 14.2.2 We just found a broken bucket index, trying to repair with the common commands ]# radosgw-admin bucket check --check-objects –fix finish instantly, but bucket should have near 60-70TB info. [root@CEPH-MON01 home]# radosgw-admin bucket check --check-objects --bucket BUCKETNAME --debug_rgw=10 2019-09-05 20:08:13.908 7fdc8db37580 10 cannot find current period zonegroup using local zonegroup 2019-09-05 20:08:13.910 7fdc8db37580 10 Cannot find current period zone using local zone 2019-09-05 20:08:13.933 7fdc8db37580 2 all 8 watchers are set, enabling cache 2019-09-05 20:08:13.942 7fdc56ffd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-09-05 20:08:13.948 7fdc8db37580 10 cache get: name=default.rgw.data.root++BUCKETNAME : miss 2019-09-05 20:08:13.950 7fdc8db37580 10 cache put: name=default.rgw.data.root++BUCKETNAME info.flags=0x16 2019-09-05 20:08:13.950 7fdc8db37580 10 adding default.rgw.data.root++BUCKETNAME to cache LRU end 2019-09-05 20:08:13.950 7fdc8db37580 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-09-05 20:08:13.950 7fdc8db37580 10 cache get: name=default.rgw.data.root++BUCKETNAME : type miss (requested=0x11, cached=0x16) 2019-09-05 20:08:13.950 7fdc8db37580 10 cache put: name=default.rgw.data.root++BUCKETNAME info.flags=0x11 2019-09-05 20:08:13.950 7fdc8db37580 10 moving default.rgw.data.root++BUCKETNAME to cache LRU end 2019-09-05 20:08:13.950 7fdc8db37580 10 cache get: name=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4- fdc16f590a82.16313306.1 : miss 2019-09-05 20:08:13.951 7fdc8db37580 10 cache put: name=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4- fdc16f590a82.16313306.1 info.flags=0x16 2019-09-05 20:08:13.951 7fdc8db37580 10 adding default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4-fdc16 f590a82.16313306.1 to cache LRU end 2019-09-05 20:08:13.951 7fdc8db37580 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-09-05 20:08:13.951 7fdc8db37580 10 updating xattr: name=user.rgw.acl bl.length()=145 2019-09-05 20:08:13.951 7fdc8db37580 10 updating xattr: name=user.rgw.lc bl.length()=467 2019-09-05 20:08:13.951 7fdc8db37580 10 cache get: name=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4- fdc16f590a82.16313306.1 : type miss (requested=0x13, cached=0x16) 2019-09-05 20:08:13.951 7fdc8db37580 10 cache put: name=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4- fdc16f590a82.16313306.1 info.flags=0x13 2019-09-05 20:08:13.951 7fdc8db37580 10 moving default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4-fdc16 f590a82.16313306.1 to cache LRU end 2019-09-05 20:08:13.951 7fdc8db37580 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-09-05 20:08:13.951 7fdc8db37580 10 updating xattr: name=user.rgw.acl bl.length()=145 2019-09-05 20:08:13.951 7fdc8db37580 10 updating xattr: name=user.rgw.lc bl.length()=467 2019-09-05 20:08:13.951 7fdc8db37580 10 chain_cache_entry: cache_locator=default.rgw.data.root++BUCKETNAME 2019-09-05 20:08:13.951 7fdc8db37580 10 chain_cache_entry: cache_locator=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4 fe0-bbe4-fdc16f590a82.16313306.1 2019-09-05 20:08:13.951 7fdc8db37580 10 cache get: name=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4- fdc16f590a82.16313306.1 : type miss (requested=0x16, cached=0x13) 2019-09-05 20:08:13.952 7fdc8db37580 10 cache put: name=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4- fdc16f590a82.16313306.1 info.flags=0x16 2019-09-05 20:08:13.952 7fdc8db37580 10 moving default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4-fdc16 f590a82.16313306.1 to cache LRU end 2019-09-05 20:08:13.952 7fdc8db37580 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-09-05 20:08:13.952 7fdc8db37580 10 updating xattr: name=user.rgw.acl bl.length()=145 2019-09-05 20:08:13.952 7fdc8db37580 10 updating xattr: name=user.rgw.lc bl.length()=467 2019-09-05 20:08:13.952 7fdc8db37580 10 cache get: name=default.rgw.data.root++.bucket.meta.BUCKETNAME:48efb8c3-693c-4fe0-bbe4- fdc16f590a82.16313306.1 : hit (requested=0x11, cached=0x17) 2019-09-05 20:08:13.952 7fdc8db37580 10 cls_bucket_list_ordered BUCKETNAME[48efb8c3-693c-4fe0-bbe4-fdc16f590a82.3856921.7] start _multipart_[] num_entries 1001 2019-09-05 20:08:13.960 7fdc8db37580 2 removed watcher, disabling cache Any recomendations for forcé recreate bucket index? Regards Manuel