Hi, Yeah the negative pid is interesting. AFAICT we use a negative pid to indicate that the lock was taken on another host: https://github.com/torvalds/linux/blob/master/fs/ceph/locks.c#L119 https://github.com/torvalds/linux/commit/9d5b86ac13c573795525ecac6ed2db39ab2... "Finally, we convert remote filesystems to present remote pids using negative numbers. Have lustre, 9p, ceph, cifs, and dlm negate the remote pid returned for F_GETLK lock requests." The good news is that my colleagues managed to clear this filelock by restarting dovecot on a couple nodes. But I'm still curious if others have a nice way to debug such things. Cheers, Dan On Mon, Nov 9, 2020 at 8:11 PM Anthony D'Atri <anthony.datri@gmail.com> wrote:
Looks like a - in front of the 9605 — signed/unsigned int flern?
On Nov 9, 2020, at 4:59 AM, Dan van der Ster <dan@vanderster.com> wrote:
Hi all,
MDS version v14.2.11 Client kernel 3.10.0-1127.19.1.el7.x86_64
We are seeing a strange issue with a dovecot use-case on cephfs. Occasionally we have dovecot reporting a file locked, such as:
Nov 09 13:55:00 dovecot-backend-00.cern.ch dovecot[27710]: imap(reguero)<23945><fRA6B6yznq68uE28>: Error: Mailbox Deleted Items: Timeout (180s) while waiting for lock for transaction log file /mail/users/r/reguero//mdbox/mailboxes/Deleted Items/dbox-Mails/dovecot.index.log (WRITE lock held by pid -9605)
We checked all hosts that have mounted the cephfs -- there is no pid 9605.
Is there any way to see who exactly created the lock? ceph_filelock has a client id, but I didn't find a way to inspect the cephfs_metadata to see the ceph_filelock directly.
Otherwise, are other Dovecot/CephFS users seeing this? Did you switch to flock or lockfile instead of fnctlk locks?
Thanks!
Dan
P.S. here is the output from print locks tool from the kernel client:
Read lock: Type: 1 (0: Read, 1: Write, 2: Unlocked) Whence: 0 (0: start, 1: current, 2: end) Offset: 0 Len: 1 Pid: -9605 Write lock: Type: 1 (0: Read, 1: Write, 2: Unlocked) Whence: 0 (0: start, 1: current, 2: end) Offset: 0 Len: 1 Pid: -9605
and same file from a 15.2.5 fuse client :
Read lock: Type: 1 (0: Read, 1: Write, 2: Unlocked) Whence: 0 (0: start, 1: current, 2: end) Offset: 0 Len: 0 Pid: 0 Write lock: Type: 1 (0: Read, 1: Write, 2: Unlocked) Whence: 0 (0: start, 1: current, 2: end) Offset: 0 Len: 0 Pid: 0 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io