6 Sep
2019
6 Sep
'19
3:34 p.m.
I am suspecting that there is a bug in 12.2.8-0ubuntu0.18.04.1. We have too ceph client
n01 and n03 with following software versions.
# n01 ceph-common:
Installed: 12.2.8-0ubuntu0.18.04.1
Candidate: 12.2.12-0ubuntu0.18.04.2
# n03 ceph-common:
Installed: 12.2.11-0ubuntu0.18.04.2
Candidate: 12.2.12-0ubuntu0.18.04.2
# n01 uname -a
Linux n01 4.15.0-43-generic #46-Ubuntu SMP Thu Dec 6 14:45:28 UTC 2018 x86_64 x86_64
x86_64 GNU/Linux
# n03 uname -a
Linux n03 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64
x86_64 GNU/Linux
Problem:
N01 can mount its RBD after reboot whereas N03 can’t. To verify that it is something on
client side, I upgrade n01 to have the same package version to n03 including ceph-common.
And n01 does indeed fail to mount its CEPH RBD partition. Syslog shows messages like the
following:
Sep 6 12:52:33 n01 kernel: [ 875.160275] rbd: rbd0: no lock owners detected
Sep 6 12:52:33 n01 kernel: [ 875.161069] rbd: rbd0: client2508785 seems dead, breaking
lock
Sep 6 12:52:33 n01 kernel: [ 875.161723] rbd: rbd0: blacklist of client2508785 failed:
-13
Sep 6 12:52:33 n01 kernel: [ 875.161724] rbd: rbd0: failed to acquire lock: -13
Sep 6 12:52:33 n01 kernel: [ 875.161956] rbd: rbd0: no lock owners detected
Sep 6 12:52:33 n01 kernel: [ 875.162755] rbd: rbd0: client2508785 seems dead, breaking
lock
Sep 6 12:52:33 n01 kernel: [ 875.163415] rbd: rbd0: blacklist of client2508785 failed:
-13
Sep 6 12:52:33 n01 kernel: [ 875.163415] rbd: rbd0: failed to acquire lock: -13
Sep 6 12:52:33 n01 kernel: [ 875.163645] rbd: rbd0: no lock owners detected
Sep 6 12:52:33 n01 kernel: [ 875.164431] rbd: rbd0: client2508785 seems dead, breaking
lock
Sep 6 12:52:33 n01 kernel: [ 875.165082] rbd: rbd0: blacklist of client2508785 failed:
-13
Sep 6 12:52:33 n01 kernel: [ 875.165082] rbd: rbd0: failed to acquire lock: -13
Sep 6 12:52:33 n01 kernel: [ 875.165355] rbd: rbd0: no lock owners detected
[signature_1482564062]
Vang Quy Le
Special Consultant in Data Science and Infrastructure
T: (+45) 9940 7710 | Email: vle@its.aau.dk<mailto:vle@its.aau.dk>
Kontor 0-1-91 | Selma Lagerløfs Vej 300 | DK-9220 Aalborg Ø |
