4 Aug
2020
4 Aug
'20
4:16 p.m.
Hi
I am trying to delete a bucket using the following command:
# radosgw-admin bucket rm --bucket=<bucket-name> --purge-objects
However, in console I get the following messages. About 100+ of those messages per second.
2020-08-04T17:11:06.411+0100 7fe64cacf080 1 RGWRados::Bucket::List::list_objects_ordered
INFO ordered bucket listing requires read #1
The command has been running for about 35 days days and it still hasn't finished. The
size of the bucket is under 1TB for sure. Probably around 500GB.
I have recently removed about a dozen of old buckets without any issues. It's this
particular bucket that is being very stubborn.
Anything I can do to remove it, including it's objects and any orphans it might have?
Thanks
Andrei
6 Aug
6 Aug
11:55 a.m.
BUMP...
----- Original Message -----
From: "Andrei Mikhailovsky"
<andrei(a)arhont.com>
To: "ceph-users" <ceph-users(a)ceph.io>
Sent: Tuesday, 4 August, 2020 17:16:28
Subject: [ceph-users] RGW unable to delete a bucket
> Hi
>
> I am trying to delete a bucket using the following command:
>
> # radosgw-admin bucket rm --bucket=<bucket-name> --purge-objects
>
> However, in console I get the following messages. About 100+ of those messages
> per second.
>
> 2020-08-04T17:11:06.411+0100 7fe64cacf080 1
> RGWRados::Bucket::List::list_objects_ordered INFO ordered bucket listing
> requires read #1
>
>
> The command has been running for about 35 days days and it still hasn't
> finished. The size of the bucket is under 1TB for sure. Probably around 500GB.
>
> I have recently removed about a dozen of old buckets without any issues. It's
> this particular bucket that is being very stubborn.
>
> Anything I can do to remove it, including it's objects and any orphans it might
> have?
>
>
> Thanks
>
> Andrei
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
6:40 p.m.
You'r not the only one affected by this issue
As far as i know several huge companies hitted this bug too, but private patches or tools
are not public released.
This is caused for the a resharding process during upload in previous versions.
Workarround for us.:
- Delete objects of the bucket at rados level.
- Delete the index file of the bucket.
Pray to god to not happen again.
Still pending backporting to Nautilus of the new experimental tool to find orphans in RGW
Maybe @Matt Benjamin can give us and ETA for get ready that tool backported...
Regards
-----Mensaje original-----
De: Andrei Mikhailovsky <andrei(a)arhont.com>
Enviado el: jueves, 6 de agosto de 2020 13:55
Para: ceph-users <ceph-users(a)ceph.io>
Asunto: [ceph-users] Re: RGW unable to delete a bucket
BUMP...
----- Original Message -----
From: "Andrei Mikhailovsky"
<andrei(a)arhont.com>
To: "ceph-users" <ceph-users(a)ceph.io>
Sent: Tuesday, 4 August, 2020 17:16:28
Subject: [ceph-users] RGW unable to delete a bucket
Hi
I am trying to delete a bucket using the following command:
# radosgw-admin bucket rm --bucket=<bucket-name> --purge-objects
However, in console I get the following messages. About 100+ of those
messages per second.
2020-08-04T17:11:06.411+0100 7fe64cacf080 1
RGWRados::Bucket::List::list_objects_ordered INFO ordered bucket
listing requires read #1
The command has been running for about 35 days days and it still
hasn't finished. The size of the bucket is under 1TB for sure. Probably around
500GB.
I have recently removed about a dozen of old buckets without any
issues. It's this particular bucket that is being very stubborn.
Anything I can do to remove it, including it's objects and any orphans
it might have?
Thanks
Andrei
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an
email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to
ceph-users-leave(a)ceph.io
12 Aug
12 Aug
4:57 p.m.
Hi, What version are you using? There seems to exist a bug
(https://tracker.ceph.com/issues/42358), we had the same problem using redhat 12.2.12-84.
Te only way to stop the rgw logs to crash the machine was to restart the service.
To remove the bucket we had to upgrade to 12.2.12.-115
----- Mensagem original -----
De: "Andrei Mikhailovsky" <andrei(a)arhont.com>
Para: "ceph-users" <ceph-users(a)ceph.io>
Enviadas: Terça-feira, 4 de agosto de 2020 13:16:28
Assunto: [ceph-users] RGW unable to delete a bucket
Hi
I am trying to delete a bucket using the following command:
# radosgw-admin bucket rm --bucket=<bucket-name> --purge-objects
However, in console I get the following messages. About 100+ of those messages per second.
2020-08-04T17:11:06.411+0100 7fe64cacf080 1 RGWRados::Bucket::List::list_objects_ordered
INFO ordered bucket listing requires read #1
The command has been running for about 35 days days and it still hasn't finished. The
size of the bucket is under 1TB for sure. Probably around 500GB.
I have recently removed about a dozen of old buckets without any issues. It's this
particular bucket that is being very stubborn.
Anything I can do to remove it, including it's objects and any orphans it might have?
Thanks
Andrei
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
1360
days inactive
1368
days old
3 comments
3 participants
participants (3)
-
Andrei Mikhailovsky -
EDH - Manuel Rios -
Marcelo Mariano Miziara