Unable to reshard bucket
Hello all, I'm having an issue with a bucket that refuses to be resharded..for the record, the cluster was recently upgraded from 13.2.4 to 13.2.10. # radosgw-admin reshard add --bucket foo --num-shards 3300 ERROR: the bucket is currently undergoing resharding and cannot be added to the reshard list at this time # radosgw-admin reshard list [] # radosgw-admin reshard status --bucket=foo [ { "reshard_status": "not-resharding", "new_bucket_instance_id": "", "num_shards": -1 }, <snip> # radosgw-admin reshard cancel --bucket foo ERROR: failed to remove entry from reshard log, oid=reshard.0000000009 tenant= bucket=foo # radosgw-admin reshard stale-instances list [] Is there anything else I should check to troubleshoot this? I was able to reshard another bucket since the upgrade, so I suspect there's something lingering that's blocking this.
Perhaps the next step is to examine the generated logs from: radosgw-admin reshard status --bucket=foo --debug-rgw=20 --debug-ms=1 radosgw-admin reshard cancel --bucket foo --debug-rgw=20 --debug-ms=1 Eric -- J. Eric Ivancich he / him / his Red Hat Storage Ann Arbor, Michigan, USA
On May 11, 2020, at 12:25 PM, Timothy Geier <tgeier@accertify.com> wrote:
Hello all,
I'm having an issue with a bucket that refuses to be resharded..for the record, the cluster was recently upgraded from 13.2.4 to 13.2.10.
# radosgw-admin reshard add --bucket foo --num-shards 3300 ERROR: the bucket is currently undergoing resharding and cannot be added to the reshard list at this time
# radosgw-admin reshard list []
# radosgw-admin reshard status --bucket=foo [ { "reshard_status": "not-resharding", "new_bucket_instance_id": "", "num_shards": -1 }, <snip>
# radosgw-admin reshard cancel --bucket foo ERROR: failed to remove entry from reshard log, oid=reshard.0000000009 tenant= bucket=foo
# radosgw-admin reshard stale-instances list []
Is there anything else I should check to troubleshoot this? I was able to reshard another bucket since the upgrade, so I suspect there's something lingering that's blocking this. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Thank you..I looked through both logs and noticed this in the cancel one: osd_op(unknown.0.0:4164 41.2 41:55b0279d:reshard::reshard.0000000009:head [call rgw.reshard_remove] snapc 0=[] ondisk+write+known_if_redirected e24984) v8 -- 0x7fe9b3625710 con 0 osd_op_reply(4164 reshard.0000000009 [call] v24984'105796943 uv105796922 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (203651653 0 0) 0x7fe9880044a0 con 0x7fe9b3625b70 ERROR: failed to remove entry from reshard log, oid=reshard.0000000009 tenant= bucket=foo Is there anything else that I should look for? It looks like the cancel process thinks that reshard.0000000009 is present (and probably blocking my attempts at resharding) but it's not actually there and thus can't be removed.
participants (2)
-
Eric Ivancich
-
Timothy Geier