15 Aug
2019
15 Aug
'19
9:25 a.m.
On 15/08/2019 11.42, Peter Sarossy wrote:
hey folks,
I spent the past 2 hours digging through the forums and similar sources with no luck..
I use ceph storage for docker stacks, and this issue has taken the whole thing down as I
cannot mount their volumes back...
Starting yesterday, some of my nodes cannot mount the filesystem and it just hangs on the
mount command, while the logs are full of the messages below. It doesn't matter which
MDS node is active for some clients it just works, and for others it doesn't.
Any hints?
2019-08-15 02:20:55.230 7f6e80e47700 0 --1-
[v2:10.0.0.8:6800/859279545,v1:10.0.0.8:6801/859279545] >> v1:10.0.0.1:0/844039356
conn(0x564928535000 0x56492851b800 :6801 s=READ_FOOTER_AND_DISPATCH pgs=13406 cs=3203
l=0).handle_message_footer Signature check failed
2019-08-15 02:20:56.254 7f6e7fe45700 0 SIGN: MSG 1 Message signature does not match
contents.
2019-08-15 02:20:56.254 7f6e7fe45700 0 SIGN: MSG 1Signature on message:
2019-08-15 02:20:56.254 7f6e7fe45700 0 SIGN: MSG 1 sig: 1045888080092928376
2019-08-15 02:20:56.254 7f6e7fe45700 0 SIGN: MSG 1Locally calculated signature:
2019-08-15 02:20:56.254 7f6e7fe45700 0 SIGN: MSG 1 sig_check:13982737427498638198
2019-08-15 02:20:56.254 7f6e7fe45700 0 Signature failed.
2019-08-15 02:20:56.254 7f6e7fe45700 0 --1-
[v2:10.0.0.8:6800/859279545,v1:10.0.0.8:6801/859279545] >> v1:10.0.0.1:0/844039356
conn(0x564928535000 0x56492851b800 :6801 s=READ_FOOTER_AND_DISPATCH pgs=13410 cs=3205
l=0).handle_message_footer Signature check failed
2019-08-15 02:20:57.246 7f6e80646700 0 SIGN: MSG 1 Message signature does not match
contents.
2019-08-15 02:20:57.246 7f6e80646700 0 SIGN: MSG 1Signature on message:
2019-08-15 02:20:57.246 7f6e80646700 0 SIGN: MSG 1 sig: 1045888080092928376
2019-08-15 02:20:57.246 7f6e80646700 0 SIGN: MSG 1Locally calculated signature:
2019-08-15 02:20:57.246 7f6e80646700 0 SIGN: MSG 1 sig_check:13982737427498638198
2019-08-15 02:20:57.246 7f6e80646700 0 Signature failed.
2019-08-15 02:20:57.246 7f6e80646700 0 --1-
[v2:10.0.0.8:6800/859279545,v1:10.0.0.8:6801/859279545] >> v1:10.0.0.1:0/844039356
conn(0x564928535000 0x56492851b800 :6801 s=READ_FOOTER_AND_DISPATCH pgs=13414 cs=3207
l=0).handle_message_footer Signature check failed
Are you using Canonical Livepatch? Looks like they didn't test a recent
patch and broke CephFS in the process. You'll probably have to reboot
the affected clients.
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2019-August/036513.html
--
Hector Martin (hector(a)marcansoft.com)
Public Key: https://mrcn.st/pub