Am 09.06.21 um 13:28 schrieb Ilya Dryomov:
On Wed, Jun 9, 2021 at 11:24 AM Peter Lieven <pl@kamp.de> wrote:
Hi,
we currently run into an issue where a rbd ls for a namespace returns ENOENT for some of the images in that namespace.
/usr/bin/rbd --conf=XXX --id XXX ls 'mypool/28ef9470-76eb-4f77-bc1b-99077764ff7c' -l --format=json 2021-06-09 11:03:34.916 7f2225ffb700 -1 librbd::io::AioCompletion: 0x55cacccc2390 fail: (2) No such file or directory 2021-06-09 11:03:34.916 7f2225ffb700 -1 librbd::io::AioCompletion: 0x55caccd2b920 fail: (2) No such file or directory 2021-06-09 11:03:34.920 7f2225ffb700 -1 librbd::io::AioCompletion: 0x55caccd9b4e0 fail: (2) No such file or directory rbd: error opening 34810ac2-3112-4fef-938c-b76338b0eeaf.raw: (2) No such file or directory rbd: error opening c9882583-6dd5-4eca-bb82-3e81f7d63fa9.raw: (2) No such file or directory rbd: error opening 5d5251d1-f017-4382-845c-65e504683742.raw: (2) No such file or directory 2021-06-09 11:03:34.924 7f2225ffb700 -1 librbd::io::AioCompletion: 0x55cacce07b00 fail: (2) No such file or directory rbd: error opening c625b898-ec34-4446-9455-d2b70d9e378f.raw: (2) No such file or directory 2021-06-09 11:03:34.924 7f2225ffb700 -1 librbd::io::AioCompletion: 0x55caccd7cce0 fail: (2) No such file or directory rbd: error opening 990c4bbe-6a7b-4adf-aab8-432e18d79e58.raw: (2) No such file or directory 2021-06-09 11:03:34.924 7f2225ffb700 -1 librbd::io::AioCompletion: 0x55cacce336f0 fail: (2) No such file or directory rbd: error opening 7382eb5b-a3eb-41e2-89b6-512f7b1d86c0.raw: (2) No such file or directory [{"image":"108600c6-2312-4d61-9f5b-35b351112512.raw","size":31457280000,"format":2,"lock_type":"exclusive"},{"image":"1292ef0c-2333-44f1-be30-39105f7d176e.raw","size":262149242880,"format":2,"lock_type":"exclusive"},{"image":"8cda5c3f-cdbd-42f4-918f-1480354e7965.raw","size":262149242880,"format":2,"lock_type":"exclusive"}] rbd: listing images failed: (2) No such file or directory
The way to trigger this state was that the images which show "No such file or directory" were deleted with rbd rm, but the operation was interrupted (rbd process was killed) due to a timeout.
What is the best way to recover from this and how to properly clean up?
Release is nautilus 14.2.20 Hi Peter,
Does "rbd ls" without "-l" succeed?
Yes, it does: /usr/bin/rbd --conf=XXX --id XXX ls 'mypool/28ef9470-76eb-4f77-bc1b-99077764ff7c' --format=json ["108600c6-2312-4d61-9f5b-35b351112512.raw","1292ef0c-2333-44f1-be30-39105f7d176e.raw","8cda5c3f-cdbd-42f4-918f-1480354e7965.raw","34810ac2-3112-4fef-938c-b76338b0eeaf.raw","c9882583-6dd5-4eca-bb82-3e81f7d63fa9.raw","5d5251d1-f017-4382-845c-65e504683742.raw","c625b898-ec34-4446-9455-d2b70d9e378f.raw","990c4bbe-6a7b-4adf-aab8-432e18d79e58.raw","7382eb5b-a3eb-41e2-89b6-512f7b1d86c0.raw"] Peter