Hi Drew,
On Friday, March 5th, 2021 at 20:22, Drew Weaver <drew.weaver(a)thenap.com> wrote:
Sorry for multi-reply, I got that command to run:
for obj in $(rados -p default.rgw.buckets.index ls | grep
2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637); do printf "%-60s %7d\n" $obj
$(rados -p default.rgw.buckets.index listomapkeys $obj | wc -l); done;
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.0 10423
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.15 10445
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.3 10542
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.6 10511
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.13 10414
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.12 10479
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.2 10486
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.5 10448
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.4 10470
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.8 10474
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.1 10470
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.10 10411
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.7 10445
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.14 10413
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.9 10356
.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.11 10410
So this looks fine, it means that this bucket is sharded properly and the index keys are
evenly distributed.
However, the problematic object is
`.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.213.0`; you said there's no bucket
with that ID, which means it's probably a stale instance. Does it still exist in
`default.rgw.buckets.index`? If so, you can try running `radosgw-admin reshard
stale-instances rm`, it should get rid of it (you can verify afterwards with `rados -p
default.rgw.buckets.index ls`).
--
Ben