[for-6.13,v2,00/15] nfs/nfsd: fixes and improvements for LOCALIO

Message

Mike Snitzer Nov. 14, 2024, 3:59 a.m. UTC

Hi,

This v2 is rebased ontop of Chuck's nfsd-next and Trond's nfs-next,
see: https://git.kernel.org/pub/scm/linux/kernel/git/snitzer/linux.git/log/?h=nfs-localio-for-next

(Neil, ~3 patches from v1 that you Reviewed-by were already staged by
Trond, without your Reviewed-by, because they were in the nfs patch
queue carried over from just after the 6.12 merge window closed).

Changes since v1:
- added Reviewed-by tags and Fixes where provided/suggested
- removed unnecessary use of RCU read-side critical section in
  nfs_server_uuid_is_local 
- switched to using RCU_INIT_POINTER to set nfs_uuid->net to NULL
- updated Documentation/filesystems/nfs/localio.rst to reflect the
  percpu_ref change from nfsd_serv to nfsd_net. Also discuss O_DIRECT
  relative to LOCALIO and document the nfs module param (Chuck, I do
  think we need it, otherwise O_DIRECT regressions are possible).
- removed redundant code and simplify LOCALIO enablement, the
  nfs_client struct is still used in nfs_common/nfslocalio.c but it is
  very limited (clp->cl_uuid and the established tracepoints).
- eliminated repeat INIT_WORK in nfs_local_probe_async
- kept the NFS v3 LOCALIO reconnect approach but eliminated the use of
  a mutex and also increased the throttling from every 256 to 512
  IOs. Updated patch header to expalin that more work is needed.

I think the final NFSv3 reconnect patch works well for now, but I am
completely on board with making it work well for the scenarios that
were discussed. I just didn't have time to perfect the approach. So
I'll defer to Trond, Anna and others on whether or not its best to
just drop patch 15 for now -- so continue to let LOCALIO get disabled,
if the server is restarted, and require manual LOCALIO recovery (via
remount).

Thanks,
Mike

Mike Snitzer (15):
  nfs_common: must not hold RCU while calling nfsd_file_put_local
  nfs/localio: add direct IO enablement with sync and async IO support
  nfsd: add nfsd_file_{get,put} to 'nfs_to' nfsd_localio_operations
  nfs_common: rename functions that invalidate LOCALIO nfs_clients
  nfs_common: move localio_lock to new lock member of nfs_uuid_t
  nfs: cache all open LOCALIO nfsd_file(s) in client
  nfsd: update percpu_ref to manage references on nfsd_net
  nfsd: rename nfsd_serv_ prefixed methods and variables with nfsd_net_
  nfsd: nfsd_file_acquire_local no longer returns GC'd nfsd_file
  nfs_common: rename nfslocalio nfs_uuid_lock to nfs_uuids_lock
  nfs_common: track all open nfsd_files per LOCALIO nfs_client
  nfs_common: add nfs_localio trace events
  nfs/localio: remove redundant code and simplify LOCALIO enablement
  nfs: probe for LOCALIO when v4 client reconnects to server
  nfs: probe for LOCALIO when v3 client reconnects to server

 Documentation/filesystems/nfs/localio.rst |  78 +++----
 fs/nfs/client.c                           |   4 +-
 fs/nfs/direct.c                           |   1 +
 fs/nfs/flexfilelayout/flexfilelayout.c    |  29 ++-
 fs/nfs/flexfilelayout/flexfilelayout.h    |   1 +
 fs/nfs/inode.c                            |   3 +
 fs/nfs/internal.h                         |  14 +-
 fs/nfs/localio.c                          | 250 ++++++++++++++++-----
 fs/nfs/nfs3proc.c                         |  32 ++-
 fs/nfs/nfs4state.c                        |   1 +
 fs/nfs/nfstrace.h                         |  32 ---
 fs/nfs/pagelist.c                         |   5 +-
 fs/nfs/write.c                            |   3 +-
 fs/nfs_common/Makefile                    |   3 +-
 fs/nfs_common/localio_trace.c             |  10 +
 fs/nfs_common/localio_trace.h             |  56 +++++
 fs/nfs_common/nfslocalio.c                | 258 +++++++++++++++++-----
 fs/nfsd/filecache.c                       |  32 ++-
 fs/nfsd/filecache.h                       |   2 +-
 fs/nfsd/localio.c                         |   9 +-
 fs/nfsd/netns.h                           |  12 +-
 fs/nfsd/nfsctl.c                          |   6 +-
 fs/nfsd/nfssvc.c                          |  40 ++--
 include/linux/nfs_fs.h                    |  22 +-
 include/linux/nfs_fs_sb.h                 |   2 +-
 include/linux/nfs_xdr.h                   |   1 +
 include/linux/nfslocalio.h                |  64 ++++--
 27 files changed, 688 insertions(+), 282 deletions(-)
 create mode 100644 fs/nfs_common/localio_trace.c
 create mode 100644 fs/nfs_common/localio_trace.h

Comments

Chuck Lever Nov. 14, 2024, 4:14 p.m. UTC | #1

From: Chuck Lever <[e-mail]>

On Wed, 13 Nov 2024 22:59:37 -0500, Mike Snitzer wrote:                                              
> Move holding the RCU from nfs_to_nfsd_file_put_local to
> nfs_to_nfsd_net_put.  It is the call to nfs_to->nfsd_serv_put that
> requires the RCU anyway (the puts for nfsd_file and netns were
> combined to avoid an extra indirect reference but that
> micro-optimization isn't possible now).
> 
> [...]                                                                        

Applied to nfsd-next for v6.13, thanks!                                                                

[01/15] nfs_common: must not hold RCU while calling nfsd_file_put_local
        commit: f055a6cafaea151c2df2a75f31c3ecfaa8b90b9e                                                                      

--                                                                              
Chuck Lever