I'm less familiar with the fuse client than the kernel one, but this
sounds wrong.
In step 5, Client A should just do a synchronous read from the OSD since
it no longer has Fc caps. Why is it seeing old data? Has Client B just
not yet sent issued the write to the OSD? If so, was Client B issued Fb
caps?
-- Jeff
On Thu, 2020-09-24 at 15:34 +0800, Xiaoxi Chen wrote:
Could you explain why client can add page cache later?
Please
correct where it is wrong.
1. Client A has page cache of file X
2. Client B open X for write, it will take write lock and MDS
will revoke Fc of Client A, which will result in Client A drop its
cache.
3. Client A try to read X, Client A can go ahead to read from
OSD, which gets the old data. (will clinet A issue a getattr to MDS?
will the getattr been blocked? I see some discussion pointing to
https://github.com/ukernel/ceph/commit/7db1563416b5559310dbbc834795b83a4ccd…)
4. Client B writes data
5. Client A still get old data.
-Xiaoxi
Yan, Zheng <ukernel(a)gmail.com> 于2020年9月24日周四 上午11:50写道:
On Thu, Sep 24, 2020 at 11:07 AM Xiaoxi Chen
<superdebuger(a)gmail.com> wrote:
Hi zheng,
We are seeing inconsistent among clients ones one client update a file(by scp),
some of the node see new contents but some of the nodes don't. The inconsistent can
last 30mins to a few hours and fix by its own. I think it should because some of the node
not dropping the page cache properly.
Looking into the code I see when Fc cap revoke, fuse client drop objectcache ,
queue a task to finisher thread to do fuse_lowlevel_notify_inval_inode, then ack the cap
revoke. So seems there is a window between the cap-revoke-ack , and the final
fuse_lowlevel_notify_inval_inode finished, in this window page cache still valid and user
can read stale data. Though it is strange that the window can be that large(no pg issue
during the window).
Could you please confirm if this is the real problems and why it is implemented in
this way?
yes, it's real problem. fuse_lowlevel_notify_inval_inode() does not
prevent client add page cache later. If there are multiple fuse
clients read/modify same file, you'd better to set
fuse_disable_pagecache option to true.
> -xiaoxi
_______________________________________________
Dev mailing list -- dev(a)ceph.io
To unsubscribe send an email to dev-leave(a)ceph.io