Deletion of objects and garbage collector
Hi, I read at many places that when an object is deleted from ceph it is queued for deletion with the garbage collector (GC). However, when I delete objects of various sizes (both less than 4MB and large sized greater than 4MB i.e. MPU) I always find gc list as empty. I tried disabling GC also to make sure that it does not run and delete it, yet I did not find any object when I run the command. Here's the output: *radosgw-admin gc list* [ { "tag": "2~KsaJkJwSGeuVzeKpkHAe_5vJ3JqZmKc", "time": "2020-04-10 18:25:23.0.769037s", "objs": [] } ] NOTE: I tried to delete an object as on 14th April and 15th April. I issued "s3cmd del" command. Ceph Version I am using in Luminous. Please let me know how object deletes work. -- Regards, Priya
Hi Priya, Did you try to list the objects using radosgw-admin gc list immediately after deleting them? Try using the command: radosgw-admin gc list --include-all, this lists all the expired and non expired entries. radosgw-admin gc list, only lists entries that have expired. Objects that have been gc'ed expire after a duration that is determined by rgw_gc_obj_min_wait the default value of which is 2 hrs. Thanks, Pritha On Wed, Apr 15, 2020 at 3:25 PM Priya Sehgal <priya.sehgal@gmail.com> wrote:
Hi, I read at many places that when an object is deleted from ceph it is queued for deletion with the garbage collector (GC). However, when I delete objects of various sizes (both less than 4MB and large sized greater than 4MB i.e. MPU) I always find gc list as empty. I tried disabling GC also to make sure that it does not run and delete it, yet I did not find any object when I run the command. Here's the output: *radosgw-admin gc list*
[ { "tag": "2~KsaJkJwSGeuVzeKpkHAe_5vJ3JqZmKc", "time": "2020-04-10 18:25:23.0.769037s", "objs": [] } ]
NOTE: I tried to delete an object as on 14th April and 15th April.
I issued "s3cmd del" command. Ceph Version I am using in Luminous.
Please let me know how object deletes work.
-- Regards, Priya _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi Pritha, I tried running radosgw-admin gc list --include-all and it shows the shadow objects. One question - the above list only has list of shadow and multi-part objects. So, objects that are less than 4MB - will they be not queued - since they are part of the HEAD and the HEAD gets deleted when we issue s3cmd del command? Thanks, Priya On Wed, Apr 15, 2020 at 3:33 PM Pritha Srivastava <prsrivas@redhat.com> wrote:
Hi Priya,
Did you try to list the objects using radosgw-admin gc list immediately after deleting them?
Try using the command: radosgw-admin gc list --include-all, this lists all the expired and non expired entries. radosgw-admin gc list, only lists entries that have expired. Objects that have been gc'ed expire after a duration that is determined by rgw_gc_obj_min_wait the default value of which is 2 hrs.
Thanks, Pritha
On Wed, Apr 15, 2020 at 3:25 PM Priya Sehgal <priya.sehgal@gmail.com> wrote:
Hi, I read at many places that when an object is deleted from ceph it is queued for deletion with the garbage collector (GC). However, when I delete objects of various sizes (both less than 4MB and large sized greater than 4MB i.e. MPU) I always find gc list as empty. I tried disabling GC also to make sure that it does not run and delete it, yet I did not find any object when I run the command. Here's the output: *radosgw-admin gc list*
[ { "tag": "2~KsaJkJwSGeuVzeKpkHAe_5vJ3JqZmKc", "time": "2020-04-10 18:25:23.0.769037s", "objs": [] } ]
NOTE: I tried to delete an object as on 14th April and 15th April.
I issued "s3cmd del" command. Ceph Version I am using in Luminous.
Please let me know how object deletes work.
-- Regards, Priya _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Regards, Priya
Yes, those head objects are deleted inline and only the tail objects are added to gc to be deleted once they expire. Thanks, Pritha On Wed, Apr 15, 2020 at 4:25 PM Priya Sehgal <priya.sehgal@gmail.com> wrote:
Hi Pritha, I tried running radosgw-admin gc list --include-all and it shows the shadow objects.
One question - the above list only has list of shadow and multi-part objects. So, objects that are less than 4MB - will they be not queued - since they are part of the HEAD and the HEAD gets deleted when we issue s3cmd del command?
Thanks, Priya
On Wed, Apr 15, 2020 at 3:33 PM Pritha Srivastava <prsrivas@redhat.com> wrote:
Hi Priya,
Did you try to list the objects using radosgw-admin gc list immediately after deleting them?
Try using the command: radosgw-admin gc list --include-all, this lists all the expired and non expired entries. radosgw-admin gc list, only lists entries that have expired. Objects that have been gc'ed expire after a duration that is determined by rgw_gc_obj_min_wait the default value of which is 2 hrs.
Thanks, Pritha
On Wed, Apr 15, 2020 at 3:25 PM Priya Sehgal <priya.sehgal@gmail.com> wrote:
Hi, I read at many places that when an object is deleted from ceph it is queued for deletion with the garbage collector (GC). However, when I delete objects of various sizes (both less than 4MB and large sized greater than 4MB i.e. MPU) I always find gc list as empty. I tried disabling GC also to make sure that it does not run and delete it, yet I did not find any object when I run the command. Here's the output: *radosgw-admin gc list*
[ { "tag": "2~KsaJkJwSGeuVzeKpkHAe_5vJ3JqZmKc", "time": "2020-04-10 18:25:23.0.769037s", "objs": [] } ]
NOTE: I tried to delete an object as on 14th April and 15th April.
I issued "s3cmd del" command. Ceph Version I am using in Luminous.
Please let me know how object deletes work.
-- Regards, Priya _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Regards, Priya
Thanks. This is helpful. On Wed 15 Apr, 2020, 9:39 PM Pritha Srivastava, <prsrivas@redhat.com> wrote:
Yes, those head objects are deleted inline and only the tail objects are added to gc to be deleted once they expire.
Thanks, Pritha
On Wed, Apr 15, 2020 at 4:25 PM Priya Sehgal <priya.sehgal@gmail.com> wrote:
Hi Pritha, I tried running radosgw-admin gc list --include-all and it shows the shadow objects.
One question - the above list only has list of shadow and multi-part objects. So, objects that are less than 4MB - will they be not queued - since they are part of the HEAD and the HEAD gets deleted when we issue s3cmd del command?
Thanks, Priya
On Wed, Apr 15, 2020 at 3:33 PM Pritha Srivastava <prsrivas@redhat.com> wrote:
Hi Priya,
Did you try to list the objects using radosgw-admin gc list immediately after deleting them?
Try using the command: radosgw-admin gc list --include-all, this lists all the expired and non expired entries. radosgw-admin gc list, only lists entries that have expired. Objects that have been gc'ed expire after a duration that is determined by rgw_gc_obj_min_wait the default value of which is 2 hrs.
Thanks, Pritha
On Wed, Apr 15, 2020 at 3:25 PM Priya Sehgal <priya.sehgal@gmail.com> wrote:
Hi, I read at many places that when an object is deleted from ceph it is queued for deletion with the garbage collector (GC). However, when I delete objects of various sizes (both less than 4MB and large sized greater than 4MB i.e. MPU) I always find gc list as empty. I tried disabling GC also to make sure that it does not run and delete it, yet I did not find any object when I run the command. Here's the output: *radosgw-admin gc list*
[ { "tag": "2~KsaJkJwSGeuVzeKpkHAe_5vJ3JqZmKc", "time": "2020-04-10 18:25:23.0.769037s", "objs": [] } ]
NOTE: I tried to delete an object as on 14th April and 15th April.
I issued "s3cmd del" command. Ceph Version I am using in Luminous.
Please let me know how object deletes work.
-- Regards, Priya _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Regards, Priya
participants (2)
-
Pritha Srivastava
-
Priya Sehgal