| Message ID | 1728406678-16857-1-git-send-email-dai.ngo@oracle.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | [1/1] NFS: remove revoked delegation from server's delegation list | expand |
On Tue, 2024-10-08 at 09:57 -0700, Dai Ngo wrote: > After the delegation is returned to the NFS server remove it > from the server's delegations list to reduce the time it takes > to scan this list. > > Piggyback the state manager's run to return marked delegations > for this job. > > Network trace captured while running the below script shows the > time taken to service the CB_RECALL increases gradually due to > the overhead of traversing the delegation list in > nfs_delegation_find_inode_server. > > The NFS server in this test is a Solaris server which issues > CB_RECALL when receiving the all-zero stateid in the SETATTR. > > #!/bin/sh > > mount=/mnt/data > for i in $(seq 1 20) > do > echo $i > mkdir $mount/testtarfile$i > time tar -C $mount/testtarfile$i -xf 5000_files.tar > done > > Signed-off-by: Dai Ngo <dai.ngo@oracle.com> > --- > fs/nfs/delegation.c | 11 +++++++++++ > 1 file changed, 11 insertions(+) > > diff --git a/fs/nfs/delegation.c b/fs/nfs/delegation.c > index 20cb2008f9e4..65b10f7ea232 100644 > --- a/fs/nfs/delegation.c > +++ b/fs/nfs/delegation.c > @@ -642,6 +642,17 @@ static int > nfs_server_return_marked_delegations(struct nfs_server *server, > > if (test_bit(NFS_DELEGATION_INODE_FREEING, > &delegation->flags)) > continue; > + if (test_bit(NFS_DELEGATION_REVOKED, &delegation- > >flags)) { > + inode = > nfs_delegation_grab_inode(delegation); > + if (inode) { > + rcu_read_unlock(); > + if > (nfs_detach_delegation(NFS_I(inode), delegation, server)) > + nfs_put_delegation(delegatio > n); > + iput(inode); > + cond_resched(); > + goto restart; > + } > + } > if (!nfs_delegation_need_return(delegation)) { > if (nfs4_is_valid_delegation(delegation, 0)) > prev = delegation; This is not the right place to garbage collect revoked delegations. We can do something like the above in nfs_revoke_delegation() and in nfs_delegation_mark_returned() instead.
On 10/8/24 10:50 AM, Trond Myklebust wrote: > On Tue, 2024-10-08 at 09:57 -0700, Dai Ngo wrote: >> After the delegation is returned to the NFS server remove it >> from the server's delegations list to reduce the time it takes >> to scan this list. >> >> Piggyback the state manager's run to return marked delegations >> for this job. >> >> Network trace captured while running the below script shows the >> time taken to service the CB_RECALL increases gradually due to >> the overhead of traversing the delegation list in >> nfs_delegation_find_inode_server. >> >> The NFS server in this test is a Solaris server which issues >> CB_RECALL when receiving the all-zero stateid in the SETATTR. >> >> #!/bin/sh >> >> mount=/mnt/data >> for i in $(seq 1 20) >> do >> echo $i >> mkdir $mount/testtarfile$i >> time tar -C $mount/testtarfile$i -xf 5000_files.tar >> done >> >> Signed-off-by: Dai Ngo <dai.ngo@oracle.com> >> --- >> fs/nfs/delegation.c | 11 +++++++++++ >> 1 file changed, 11 insertions(+) >> >> diff --git a/fs/nfs/delegation.c b/fs/nfs/delegation.c >> index 20cb2008f9e4..65b10f7ea232 100644 >> --- a/fs/nfs/delegation.c >> +++ b/fs/nfs/delegation.c >> @@ -642,6 +642,17 @@ static int >> nfs_server_return_marked_delegations(struct nfs_server *server, >> >> if (test_bit(NFS_DELEGATION_INODE_FREEING, >> &delegation->flags)) >> continue; >> + if (test_bit(NFS_DELEGATION_REVOKED, &delegation- >>> flags)) { >> + inode = >> nfs_delegation_grab_inode(delegation); >> + if (inode) { >> + rcu_read_unlock(); >> + if >> (nfs_detach_delegation(NFS_I(inode), delegation, server)) >> + nfs_put_delegation(delegatio >> n); >> + iput(inode); >> + cond_resched(); >> + goto restart; >> + } >> + } >> if (!nfs_delegation_need_return(delegation)) { >> if (nfs4_is_valid_delegation(delegation, 0)) >> prev = delegation; > This is not the right place to garbage collect revoked delegations. > > We can do something like the above in nfs_revoke_delegation() and in > nfs_delegation_mark_returned() instead. The code path of nfs_revoke_delegation is only called when the client gets SEQ4_STATUS_RECALLABLE_STATE_REVOKED error on SEQUENCE op and when the client tries to reclaim the open state. So this does not help for this problem. I'll move this to nfs_delegation_mark_returned(). Thanks, -Dai >
diff --git a/fs/nfs/delegation.c b/fs/nfs/delegation.c index 20cb2008f9e4..65b10f7ea232 100644 --- a/fs/nfs/delegation.c +++ b/fs/nfs/delegation.c @@ -642,6 +642,17 @@ static int nfs_server_return_marked_delegations(struct nfs_server *server, if (test_bit(NFS_DELEGATION_INODE_FREEING, &delegation->flags)) continue; + if (test_bit(NFS_DELEGATION_REVOKED, &delegation->flags)) { + inode = nfs_delegation_grab_inode(delegation); + if (inode) { + rcu_read_unlock(); + if (nfs_detach_delegation(NFS_I(inode), delegation, server)) + nfs_put_delegation(delegation); + iput(inode); + cond_resched(); + goto restart; + } + } if (!nfs_delegation_need_return(delegation)) { if (nfs4_is_valid_delegation(delegation, 0)) prev = delegation;
After the delegation is returned to the NFS server remove it from the server's delegations list to reduce the time it takes to scan this list. Piggyback the state manager's run to return marked delegations for this job. Network trace captured while running the below script shows the time taken to service the CB_RECALL increases gradually due to the overhead of traversing the delegation list in nfs_delegation_find_inode_server. The NFS server in this test is a Solaris server which issues CB_RECALL when receiving the all-zero stateid in the SETATTR. #!/bin/sh mount=/mnt/data for i in $(seq 1 20) do echo $i mkdir $mount/testtarfile$i time tar -C $mount/testtarfile$i -xf 5000_files.tar done Signed-off-by: Dai Ngo <dai.ngo@oracle.com> --- fs/nfs/delegation.c | 11 +++++++++++ 1 file changed, 11 insertions(+)