cephfs inode size handling and inode_drop field in struct MetaRequest
I'm working on a F_SETLEASE implementation for kcephfs, and am hitting a deadlock of sorts, due to a truncate triggering a cap revoke at an inopportune time. The issue is that truncates to a smaller size are always done via synchronous call to the MDS, whereas a truncate larger does not if Fx caps are held. That synchronous call causes the MDS to issue the client a cap revoke for caps that the lease holds references on (Frw, in particular). The client code has been this way since the inception and I haven't been able to locate any rationale for it. Some questions about this: 1) Why doesn't the client ever buffer a truncate to smaller size? It seems like that is something that could be done without a synchronous MDS call if we hold Fx caps. 2) The client setattr implementations set inode_drop values in the MetaRequest, but as far as I can tell, those values end up being ignored by the MDS. What purpose does inode_drop actually serve? Is this field vestigial? Thanks, -- Jeff Layton <jlayton@redhat.com>
On Thu, 2020-10-29 at 11:19 -0400, Jeff Layton wrote:
I'm working on a F_SETLEASE implementation for kcephfs, and am hitting a deadlock of sorts, due to a truncate triggering a cap revoke at an inopportune time.
The issue is that truncates to a smaller size are always done via synchronous call to the MDS, whereas a truncate larger does not if Fx caps are held. That synchronous call causes the MDS to issue the client a cap revoke for caps that the lease holds references on (Frw, in particular).
The client code has been this way since the inception and I haven't been able to locate any rationale for it. Some questions about this:
1) Why doesn't the client ever buffer a truncate to smaller size? It seems like that is something that could be done without a synchronous MDS call if we hold Fx caps.
2) The client setattr implementations set inode_drop values in the MetaRequest, but as far as I can tell, those values end up being ignored by the MDS. What purpose does inode_drop actually serve? Is this field vestigial?
I think I answered the second question myself. It _is_ potentially used to encoded a cap release into the call. That's not happening here because of the extra references held by the lease. I think I see a couple of potential fixes for that problem. I think the first question is still valid though. -- Jeff Layton <jlayton@redhat.com>
On Fri, Oct 30, 2020 at 12:53 AM Jeff Layton <jlayton@redhat.com> wrote:
On Thu, 2020-10-29 at 11:19 -0400, Jeff Layton wrote:
I'm working on a F_SETLEASE implementation for kcephfs, and am hitting a deadlock of sorts, due to a truncate triggering a cap revoke at an inopportune time.
The issue is that truncates to a smaller size are always done via synchronous call to the MDS, whereas a truncate larger does not if Fx caps are held. That synchronous call causes the MDS to issue the client a cap revoke for caps that the lease holds references on (Frw, in particular).
The client code has been this way since the inception and I haven't been able to locate any rationale for it. Some questions about this:
1) Why doesn't the client ever buffer a truncate to smaller size? It seems like that is something that could be done without a synchronous MDS call if we hold Fx caps.
because we need to increate truncate_seq. truncate_seq makes OSD drop/adjust write operation that were sent before the truncate.
2) The client setattr implementations set inode_drop values in the MetaRequest, but as far as I can tell, those values end up being ignored by the MDS. What purpose does inode_drop actually serve? Is this field vestigial?
I think I answered the second question myself. It _is_ potentially used to encoded a cap release into the call. That's not happening here because of the extra references held by the lease. I think I see a couple of potential fixes for that problem.
I think the first question is still valid though. -- Jeff Layton <jlayton@redhat.com>
participants (2)
-
Jeff Layton
-
Yan, Zheng