rbd snap create now working and just hangs forever
Hi, I have a customer VM that is running fine, but I can not make snapshots anymore. rbd snap create rbd/IMAGE@test-bb-1 just hangs forever. When I checked the status with rbd status rbd/IMAGE it shows one watcher, the cpu node where the VM is running. What can I do to investigate further, without restarting the VM. This is the only affected VM and it stopped working three days ago.
On Thu, Apr 22, 2021 at 4:20 PM Boris Behrens <bb@kervyn.de> wrote:
Hi,
I have a customer VM that is running fine, but I can not make snapshots anymore. rbd snap create rbd/IMAGE@test-bb-1 just hangs forever.
Hi Boris, Run $ rbd snap create rbd/IMAGE@test-bb-1 --debug-ms=1 --debug-rbd=20 let it hang for a few minutes and attach the output.
When I checked the status with rbd status rbd/IMAGE it shows one watcher, the cpu node where the VM is running.
What can I do to investigate further, without restarting the VM. This is the only affected VM and it stopped working three days ago.
Can you think of any event related to the cluster, that VM or the VM fleet in general that occurred three days ago? Thanks, Ilya
Am Do., 22. Apr. 2021 um 16:43 Uhr schrieb Ilya Dryomov <idryomov@gmail.com
:
On Thu, Apr 22, 2021 at 4:20 PM Boris Behrens <bb@kervyn.de> wrote:
Hi,
I have a customer VM that is running fine, but I can not make snapshots anymore. rbd snap create rbd/IMAGE@test-bb-1 just hangs forever.
Hi Boris,
Run
$ rbd snap create rbd/IMAGE@test-bb-1 --debug-ms=1 --debug-rbd=20
let it hang for a few minutes and attach the output.
I just pasted a short snip here: https://pastebin.com/B3Xgpbzd If you need more I can give it to you, but the output is very large.
When I checked the status with rbd status rbd/IMAGE it shows one watcher, the cpu node where the VM is running.
What can I do to investigate further, without restarting the VM. This is the only affected VM and it stopped working three days ago.
Can you think of any event related to the cluster, that VM or the VM fleet in general that occurred three days ago?
We had an incident where the cpu nodes connected to the wrong cluster, but
this VM was not affected IIRC. Cheers Boris -- Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im groüen Saal.
On Thu, Apr 22, 2021 at 5:08 PM Boris Behrens <bb@kervyn.de> wrote:
Am Do., 22. Apr. 2021 um 16:43 Uhr schrieb Ilya Dryomov <idryomov@gmail.com>:
On Thu, Apr 22, 2021 at 4:20 PM Boris Behrens <bb@kervyn.de> wrote:
Hi,
I have a customer VM that is running fine, but I can not make snapshots anymore. rbd snap create rbd/IMAGE@test-bb-1 just hangs forever.
Hi Boris,
Run
$ rbd snap create rbd/IMAGE@test-bb-1 --debug-ms=1 --debug-rbd=20
let it hang for a few minutes and attach the output.
I just pasted a short snip here: https://pastebin.com/B3Xgpbzd If you need more I can give it to you, but the output is very large.
Paste the first couple thousand lines (i.e. from the very beginning), that should be enough. Thanks, Ilya
Am Do., 22. Apr. 2021 um 17:27 Uhr schrieb Ilya Dryomov <idryomov@gmail.com
:
On Thu, Apr 22, 2021 at 5:08 PM Boris Behrens <bb@kervyn.de> wrote:
Am Do., 22. Apr. 2021 um 16:43 Uhr schrieb Ilya Dryomov <
idryomov@gmail.com>:
On Thu, Apr 22, 2021 at 4:20 PM Boris Behrens <bb@kervyn.de> wrote:
Hi,
I have a customer VM that is running fine, but I can not make
snapshots
anymore. rbd snap create rbd/IMAGE@test-bb-1 just hangs forever.
Hi Boris,
Run
$ rbd snap create rbd/IMAGE@test-bb-1 --debug-ms=1 --debug-rbd=20
let it hang for a few minutes and attach the output.
I just pasted a short snip here: https://pastebin.com/B3Xgpbzd If you need more I can give it to you, but the output is very large.
Paste the first couple thousand lines (i.e. from the very beginning), that should be enough.
good luck :) -- Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im groüen Saal.
On Thu, Apr 22, 2021 at 6:00 PM Boris Behrens <bb@kervyn.de> wrote:
Am Do., 22. Apr. 2021 um 17:27 Uhr schrieb Ilya Dryomov <idryomov@gmail.com>:
On Thu, Apr 22, 2021 at 5:08 PM Boris Behrens <bb@kervyn.de> wrote:
Am Do., 22. Apr. 2021 um 16:43 Uhr schrieb Ilya Dryomov <idryomov@gmail.com>:
On Thu, Apr 22, 2021 at 4:20 PM Boris Behrens <bb@kervyn.de> wrote:
Hi,
I have a customer VM that is running fine, but I can not make snapshots anymore. rbd snap create rbd/IMAGE@test-bb-1 just hangs forever.
Hi Boris,
Run
$ rbd snap create rbd/IMAGE@test-bb-1 --debug-ms=1 --debug-rbd=20
let it hang for a few minutes and attach the output.
I just pasted a short snip here: https://pastebin.com/B3Xgpbzd If you need more I can give it to you, but the output is very large.
Paste the first couple thousand lines (i.e. from the very beginning), that should be enough.
sure: https://pastebin.com/GsKpLbqG
good luck :)
What is the output of "rbd status"? I know you said it shows one watcher, but I need to see it. Thanks, Ilya
participants (2)
-
Boris Behrens
-
Ilya Dryomov