Can you provide the output from "rados -p volumes listomapvals rbd_trash"? On Wed, Mar 10, 2021 at 8:03 AM Enrico Bocchi <enrico.bocchi@cern.ch> wrote:
Hello everyone,
We have an unpurgeable image living in the trash of one of our clusters: # rbd --pool volumes trash ls 5afa5e5a07b8bc volume-02d959fe-a693-4acb-95e2-ca04b965389b
If we try to purge the whole trash it says the image is being restored but we have never tried to do that: # rbd --pool volumes trash purge Removing images: 0% complete...failed. 2021-03-10 13:58:42.849 7f78b3fc9c80 -1 librbd::api::Trash: remove: error: image is pending restoration.
When trying to delete manually, it says there are some watchers, but this is actually not the case:
# rbd --pool volumes trash remove 5afa5e5a07b8bc rbd: error: image still has watchers2021-03-10 14:00:21.262 7f93ee8f8c80 -1 librbd::api::Trash: remove: error: image is pending restoration. This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. Removing image: 0% complete...failed.
# rados listwatchers -p volumes rbd_header.5afa5e5a07b8bc #
We have tried to stat the first 10 rbd_data objects and they were all deleted. We know we can manually delete the omapkey from rbd_trash but we though it would be better to understand how an image might get in this state. Has anyone seen this before?
Many thanks! Cheers, Enrico
-- Enrico Bocchi CERN European Laboratory for Particle Physics IT - Storage Group - General Storage Services Mailbox: G20500 - Office: 31-2-010 1211 Genève 23 Switzerland _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Jason