Ilya Dryomov wrote: : On Thu, Jul 1, 2021 at 8:37 AM Jan Kasprzak <kas@fi.muni.cz> wrote: : > : > Hello, Ceph users, : > : > How can I figure out why it is not possible to unprotect a snapshot : > in a RBD image? I use this RBD pool for OpenNebula, and somehow there : > is a snapshot in one image, which OpenNebula does not see. So I wanted : > to delete the snapshot: : > : > # rbd info one/one-1312 : > rbd image 'one-1312': : > size 8 GiB in 2048 objects : > order 22 (4 MiB objects) : > snapshot_count: 1 : > id: 6732dccd50fa75 : > block_name_prefix: rbd_data.6732dccd50fa75 : > format: 2 : > features: layering, exclusive-lock, object-map, fast-diff, deep-flatten : > op_features: : > flags: : > create_timestamp: Wed Jun 30 10:41:59 2021 : > access_timestamp: Wed Jun 30 16:48:30 2021 : > modify_timestamp: Wed Jun 30 15:52:18 2021 : > : > # rbd snap ls one/one-1312 : > SNAPID NAME SIZE PROTECTED TIMESTAMP : > 1727 snap 8 GiB yes Wed Jun 30 16:11:39 2021 : > : > # rbd snap rm one/one-1312@snap : > Removing snap: 0% complete...failed. : > rbd: snapshot 'snap' is protected from removal. : > 2021-07-01 08:33:41.489 7f79c6ffd700 -1 librbd::Operations: snapshot is protected : > : > # rbd snap unprotect one/one-1312@snap : > 2021-07-01 08:28:40.747 7f3cb6ffd700 -1 librbd::SnapshotUnprotectRequest: cannot unprotect: at least 1 child(ren) [68ba8e7bace188] in pool 'one' : > 2021-07-01 08:28:40.749 7f3cb6ffd700 -1 librbd::SnapshotUnprotectRequest: encountered error: (16) Device or resource busy : > 2021-07-01 08:28:40.749 7f3cb6ffd700 -1 librbd::SnapshotUnprotectRequest: 0x56522f10e830 should_complete_error: ret_val=-16 : > rbd: unprotecting snap failed: 2021-07-01 08:28:40.751 7f3cb6ffd700 -1 librbd::SnapshotUnprotectRequest: 0x56522f10e830 should_complete_error: ret_val=-16 : > : > As far as I can see neither the snapshot nor the RBD image itself is : > used by a running qemu in my cluster. How can I delete the snapshot : > or debug the problem further? : : Hi Jan, : : There seems to be a clone image that is based on that snapshot. : "rbd children one/one-1312@snap" should give you its name. Hi Ilya, thanks for the tip. But apparently there seem to be no children of the snap or the base image: # rbd children one/one-1312@snap # rbd children one/one-1312 # -Yenya -- | Jan "Yenya" Kasprzak <kas at {fi.muni.cz - work | yenya.net - private}> | | http://www.fi.muni.cz/~kas/ GPG: 4096R/A45477D5 | We all agree on the necessity of compromise. We just can't agree on when it's necessary to compromise. --Larry Wall

Ilya Dryomov wrote: : On Thu, Jul 1, 2021 at 8:37 AM Jan Kasprzak <kas@fi.muni.cz> wrote: : > : > # rbd snap unprotect one/one-1312@snap : > 2021-07-01 08:28:40.747 7f3cb6ffd700 -1 librbd::SnapshotUnprotectRequest: cannot unprotect: at least 1 child(ren) [68ba8e7bace188] in pool 'one' : > 2021-07-01 08:28:40.749 7f3cb6ffd700 -1 librbd::SnapshotUnprotectRequest: encountered error: (16) Device or resource busy : > 2021-07-01 08:28:40.749 7f3cb6ffd700 -1 librbd::SnapshotUnprotectRequest: 0x56522f10e830 should_complete_error: ret_val=-16 : > rbd: unprotecting snap failed: 2021-07-01 08:28:40.751 7f3cb6ffd700 -1 librbd::SnapshotUnprotectRequest: 0x56522f10e830 should_complete_error: ret_val=-16 : > : > As far as I can see neither the snapshot nor the RBD image itself is : > used by a running qemu in my cluster. How can I delete the snapshot : > or debug the problem further? : : There seems to be a clone image that is based on that snapshot. : "rbd children one/one-1312@snap" should give you its name. OK, there was a child which did not show up in "rbd children" - the previously deleted clone. We use the trash feature, so it did not get deleted altogether, but moved to trash instead. I guess "rbd children" should be modified to show also images fro trash. after rbd restore --pool one one-1312-4742-0 I was able to delete the image using "rbd rm one/one-1312-4742-0", unprotect the snapshot with "rbd snap unprotect one/one-1312@snap", and finally delete it with "rbd snap rm one/one-1312@snap". So my immediate problem is fixed, but it would be nice if "rbd children" can also display images from trash. -Yenya -- | Jan "Yenya" Kasprzak <kas at {fi.muni.cz - work | yenya.net - private}> | | http://www.fi.muni.cz/~kas/ GPG: 4096R/A45477D5 | We all agree on the necessity of compromise. We just can't agree on when it's necessary to compromise. --Larry Wall

Ilya Dryomov wrote: : On Thu, Jul 1, 2021 at 10:50 AM Jan Kasprzak <kas@fi.muni.cz> wrote: : > : > Ilya Dryomov wrote: : > : On Thu, Jul 1, 2021 at 8:37 AM Jan Kasprzak <kas@fi.muni.cz> wrote: : > : > : > : > # rbd snap unprotect one/one-1312@snap : > : > 2021-07-01 08:28:40.747 7f3cb6ffd700 -1 librbd::SnapshotUnprotectRequest: cannot unprotect: at least 1 child(ren) [68ba8e7bace188] in pool 'one' : > : > 2021-07-01 08:28:40.749 7f3cb6ffd700 -1 librbd::SnapshotUnprotectRequest: encountered error: (16) Device or resource busy : > : > 2021-07-01 08:28:40.749 7f3cb6ffd700 -1 librbd::SnapshotUnprotectRequest: 0x56522f10e830 should_complete_error: ret_val=-16 : > : > rbd: unprotecting snap failed: 2021-07-01 08:28:40.751 7f3cb6ffd700 -1 librbd::SnapshotUnprotectRequest: 0x56522f10e830 should_complete_error: ret_val=-16 : > : > : > : > As far as I can see neither the snapshot nor the RBD image itself is : > : > used by a running qemu in my cluster. How can I delete the snapshot : > : > or debug the problem further? : > : : > : There seems to be a clone image that is based on that snapshot. : > : "rbd children one/one-1312@snap" should give you its name. : > : > OK, there was a child which did not show up in "rbd children" - the : > previously deleted clone. We use the trash feature, so it did not : > get deleted altogether, but moved to trash instead. I guess "rbd children" : > should be modified to show also images fro trash. : > : > after rbd restore --pool one one-1312-4742-0 I was able to delete : > the image using "rbd rm one/one-1312-4742-0", unprotect the snapshot : > with "rbd snap unprotect one/one-1312@snap", and finally delete it : > with "rbd snap rm one/one-1312@snap". : > : > So my immediate problem is fixed, but it would be nice if "rbd children" : > can also display images from trash. : : "rbd children -a" does that as noted in my previous reply. OK, thanks. Our replies were probably sent near the same time. I think OpenNebula should be modified to use this. I will fill up an issue in their github repo. Anyway, thanks for your help. -Yenya -- | Jan "Yenya" Kasprzak <kas at {fi.muni.cz - work | yenya.net - private}> | | http://www.fi.muni.cz/~kas/ GPG: 4096R/A45477D5 | We all agree on the necessity of compromise. We just can't agree on when it's necessary to compromise. --Larry Wall