Re: rgw recovering shards
On 10/27/19 6:01 AM, Frank R wrote:
I hate to be a pain but I have one more question.
After I run
radosgw-admin reshard stale-instances rm
if I run
radosgw-admin reshard stale-instances list
some new entries appear for a bucket that no longer exists. Is there a way to cancel the operation on the old bucket?
`radosgw-admin reshard stale-instances rm` should fix all your issues, if this not you should debug this. k
Hi Konstantin, Thanks very much for your help. Things seem to be running smoothly now. One remaining issue on the secondary side is that I see: ----- oldest incremental change not applied: 2019-10-22 00:24:09.0.720448s ----- Replication appears to be working fine when I upload files or create buckets. The sync error log doesn't report anything recent. Do you know of a way to determine what this error is? thanks Frank On Mon, Oct 28, 2019 at 4:38 AM Konstantin Shalygin <k0ste@k0ste.ru> wrote:
On 10/27/19 6:01 AM, Frank R wrote:
I hate to be a pain but I have one more question.
After I run
radosgw-admin reshard stale-instances rm
if I run
radosgw-admin reshard stale-instances list
some new entries appear for a bucket that no longer exists. Is there a way to cancel the operation on the old bucket?
`radosgw-admin reshard stale-instances rm` should fix all your issues, if this not you should debug this.
k
So, I ended up checking all datalog shards with: radosgw-admin data sync status --shard-id=XY --source-zone=us-east-1 and found one with a few hundred references to a bucket that had been deleted. I ended up shutting down HAProxy on both ends and running radosgw-admin data sync init This seemed to clear out the shards and things are running normally now. radosgw-admin stale instances list didn't find these for some reason. thx Frank On Wed, Oct 30, 2019 at 2:55 AM Konstantin Shalygin <k0ste@k0ste.ru> wrote:
On 10/29/19 10:56 PM, Frank R wrote:
oldest incremental change not applied: 2019-10-22 00:24:09.0.720448s
May be zone period is not the same on both sides?
k
participants (2)
-
Frank R
-
Konstantin Shalygin