mbox series

[v8,00/18] nfs/nfsd: add support for localio

Message ID 20240626182438.69539-1-snitzer@kernel.org (mailing list archive)
Headers show
Series nfs/nfsd: add support for localio | expand

Message

Mike Snitzer June 26, 2024, 6:24 p.m. UTC
Hi,

Changes since v7:
- Switched from using SRCU to percpu_ref to interlock
  nfsd_destroy_serv() and nfsd_open_local_fh().
- Dropped the "nfs/localio: use dedicated workqueues for filesystem
  read and write" patch, will revisit if/when needed based on evidence
- Changed NFSD_MAY_LOCALIO from 0x800000 to 0x2000.
- Various renames in fs/nfsd/localio.c XDR code suggested by Chuck. 
- Fixed localio_procedures1 and ARRAY_SIZE  suggested by Neil.
- Fixed nfsd_uuid_is_local() to dereference nfsd_uuid within rcu
- Removed a few dprintk in fs/{nfs,nfsd}/localio.c
- Documentation improvements suggested by Jeff.

TODO:
- Must fix xfstests generic/355 (clear suid bit on write)
- Must fix localio's nfs_get_vfs_attr() to support NFS v4 same as is
  done with nfsd4_change_attribute(). But first attempt to do so was
  met with a crash due to the extra STATX_BTIME | STATX_CHANGE_COOKIE
  being included in the request_mask passed to vfs_getattr().

All review and comments are welcome!

Thanks,
Mike

My git tree is here:
https://git.kernel.org/pub/scm/linux/kernel/git/snitzer/linux.git/

This v8 is both branch nfs-localio-for-6.11 (always tracks latest)
and nfs-localio-for-6.11.v8

Mike Snitzer (10):
  nfs_common: add NFS LOCALIO auxiliary protocol enablement
  nfsd: add "localio" support
  nfsd/localio: manage netns reference in nfsd_open_local_fh
  nfsd: use percpu_ref to interlock nfsd_destroy_serv and nfsd_open_local_fh
  nfs/nfsd: add Kconfig options to allow localio to be enabled
  nfs: factor out {encode,decode}_opaque_fixed to nfs_xdr.h
  SUNRPC: remove call_allocate() BUG_ON if p_arglen=0 to allow RPC with void arg
  nfs: implement client support for NFS_LOCALIO_PROGRAM
  nfsd: implement server support for NFS_LOCALIO_PROGRAM
  nfs: add Documentation/filesystems/nfs/localio.rst

NeilBrown (1):
  SUNRPC: replace program list with program array

Trond Myklebust (2):
  NFS: Enable localio for non-pNFS I/O
  pnfs/flexfiles: Enable localio for flexfiles I/O

Weston Andros Adamson (5):
  nfs: pass nfs_client to nfs_initiate_pgio
  nfs: pass descriptor thru nfs_initiate_pgio path
  nfs: pass struct file to nfs_init_pgio and nfs_init_commit
  sunrpc: add rpcauth_map_to_svc_cred_local
  nfs: add "localio" support

 Documentation/filesystems/nfs/localio.rst | 135 ++++
 fs/Kconfig                                |   3 +
 fs/nfs/Kconfig                            |  14 +
 fs/nfs/Makefile                           |   1 +
 fs/nfs/blocklayout/blocklayout.c          |   6 +-
 fs/nfs/client.c                           |  15 +-
 fs/nfs/filelayout/filelayout.c            |  16 +-
 fs/nfs/flexfilelayout/flexfilelayout.c    | 131 +++-
 fs/nfs/flexfilelayout/flexfilelayout.h    |   2 +
 fs/nfs/flexfilelayout/flexfilelayoutdev.c |   6 +
 fs/nfs/inode.c                            |   4 +
 fs/nfs/internal.h                         |  60 +-
 fs/nfs/localio.c                          | 793 ++++++++++++++++++++++
 fs/nfs/nfs4xdr.c                          |  13 -
 fs/nfs/nfstrace.h                         |  61 ++
 fs/nfs/pagelist.c                         |  32 +-
 fs/nfs/pnfs.c                             |  24 +-
 fs/nfs/pnfs.h                             |   6 +-
 fs/nfs/pnfs_nfs.c                         |   2 +-
 fs/nfs/write.c                            |  13 +-
 fs/nfs_common/Makefile                    |   3 +
 fs/nfs_common/nfslocalio.c                |  74 ++
 fs/nfsd/Kconfig                           |  14 +
 fs/nfsd/Makefile                          |   1 +
 fs/nfsd/filecache.c                       |   2 +-
 fs/nfsd/localio.c                         | 329 +++++++++
 fs/nfsd/netns.h                           |  12 +-
 fs/nfsd/nfsctl.c                          |   2 +-
 fs/nfsd/nfsd.h                            |   2 +-
 fs/nfsd/nfssvc.c                          | 116 +++-
 fs/nfsd/trace.h                           |   3 +-
 fs/nfsd/vfs.h                             |   9 +
 include/linux/nfs.h                       |   9 +
 include/linux/nfs_fs.h                    |   2 +
 include/linux/nfs_fs_sb.h                 |  10 +
 include/linux/nfs_xdr.h                   |  20 +-
 include/linux/nfslocalio.h                |  41 ++
 include/linux/sunrpc/auth.h               |   4 +
 include/linux/sunrpc/svc.h                |   7 +-
 net/sunrpc/auth.c                         |  15 +
 net/sunrpc/clnt.c                         |   1 -
 net/sunrpc/svc.c                          |  68 +-
 net/sunrpc/svc_xprt.c                     |   2 +-
 net/sunrpc/svcauth_unix.c                 |   3 +-
 44 files changed, 1951 insertions(+), 135 deletions(-)
 create mode 100644 Documentation/filesystems/nfs/localio.rst
 create mode 100644 fs/nfs/localio.c
 create mode 100644 fs/nfs_common/nfslocalio.c
 create mode 100644 fs/nfsd/localio.c
 create mode 100644 include/linux/nfslocalio.h

Comments

Chuck Lever III June 26, 2024, 6:49 p.m. UTC | #1
On Wed, Jun 26, 2024 at 02:24:20PM -0400, Mike Snitzer wrote:
> Hi,
> 
> Changes since v7:
> - Switched from using SRCU to percpu_ref to interlock
>   nfsd_destroy_serv() and nfsd_open_local_fh().
> - Dropped the "nfs/localio: use dedicated workqueues for filesystem
>   read and write" patch, will revisit if/when needed based on evidence
> - Changed NFSD_MAY_LOCALIO from 0x800000 to 0x2000.
> - Various renames in fs/nfsd/localio.c XDR code suggested by Chuck. 
> - Fixed localio_procedures1 and ARRAY_SIZE  suggested by Neil.
> - Fixed nfsd_uuid_is_local() to dereference nfsd_uuid within rcu
> - Removed a few dprintk in fs/{nfs,nfsd}/localio.c
> - Documentation improvements suggested by Jeff.
> 
> TODO:
> - Must fix xfstests generic/355 (clear suid bit on write)
> - Must fix localio's nfs_get_vfs_attr() to support NFS v4 same as is
>   done with nfsd4_change_attribute(). But first attempt to do so was
>   met with a crash due to the extra STATX_BTIME | STATX_CHANGE_COOKIE
>   being included in the request_mask passed to vfs_getattr().
> 
> All review and comments are welcome!
> 
> Thanks,
> Mike
> 
> My git tree is here:
> https://git.kernel.org/pub/scm/linux/kernel/git/snitzer/linux.git/
> 
> This v8 is both branch nfs-localio-for-6.11 (always tracks latest)
> and nfs-localio-for-6.11.v8
> 
> Mike Snitzer (10):
>   nfs_common: add NFS LOCALIO auxiliary protocol enablement
>   nfsd: add "localio" support
>   nfsd/localio: manage netns reference in nfsd_open_local_fh
>   nfsd: use percpu_ref to interlock nfsd_destroy_serv and nfsd_open_local_fh
>   nfs/nfsd: add Kconfig options to allow localio to be enabled
>   nfs: factor out {encode,decode}_opaque_fixed to nfs_xdr.h
>   SUNRPC: remove call_allocate() BUG_ON if p_arglen=0 to allow RPC with void arg
>   nfs: implement client support for NFS_LOCALIO_PROGRAM
>   nfsd: implement server support for NFS_LOCALIO_PROGRAM
>   nfs: add Documentation/filesystems/nfs/localio.rst
> 
> NeilBrown (1):
>   SUNRPC: replace program list with program array
> 
> Trond Myklebust (2):
>   NFS: Enable localio for non-pNFS I/O
>   pnfs/flexfiles: Enable localio for flexfiles I/O
> 
> Weston Andros Adamson (5):
>   nfs: pass nfs_client to nfs_initiate_pgio
>   nfs: pass descriptor thru nfs_initiate_pgio path
>   nfs: pass struct file to nfs_init_pgio and nfs_init_commit
>   sunrpc: add rpcauth_map_to_svc_cred_local
>   nfs: add "localio" support
> 
>  Documentation/filesystems/nfs/localio.rst | 135 ++++
>  fs/Kconfig                                |   3 +
>  fs/nfs/Kconfig                            |  14 +
>  fs/nfs/Makefile                           |   1 +
>  fs/nfs/blocklayout/blocklayout.c          |   6 +-
>  fs/nfs/client.c                           |  15 +-
>  fs/nfs/filelayout/filelayout.c            |  16 +-
>  fs/nfs/flexfilelayout/flexfilelayout.c    | 131 +++-
>  fs/nfs/flexfilelayout/flexfilelayout.h    |   2 +
>  fs/nfs/flexfilelayout/flexfilelayoutdev.c |   6 +
>  fs/nfs/inode.c                            |   4 +
>  fs/nfs/internal.h                         |  60 +-
>  fs/nfs/localio.c                          | 793 ++++++++++++++++++++++
>  fs/nfs/nfs4xdr.c                          |  13 -
>  fs/nfs/nfstrace.h                         |  61 ++
>  fs/nfs/pagelist.c                         |  32 +-
>  fs/nfs/pnfs.c                             |  24 +-
>  fs/nfs/pnfs.h                             |   6 +-
>  fs/nfs/pnfs_nfs.c                         |   2 +-
>  fs/nfs/write.c                            |  13 +-
>  fs/nfs_common/Makefile                    |   3 +
>  fs/nfs_common/nfslocalio.c                |  74 ++
>  fs/nfsd/Kconfig                           |  14 +
>  fs/nfsd/Makefile                          |   1 +
>  fs/nfsd/filecache.c                       |   2 +-
>  fs/nfsd/localio.c                         | 329 +++++++++
>  fs/nfsd/netns.h                           |  12 +-
>  fs/nfsd/nfsctl.c                          |   2 +-
>  fs/nfsd/nfsd.h                            |   2 +-
>  fs/nfsd/nfssvc.c                          | 116 +++-
>  fs/nfsd/trace.h                           |   3 +-
>  fs/nfsd/vfs.h                             |   9 +
>  include/linux/nfs.h                       |   9 +
>  include/linux/nfs_fs.h                    |   2 +
>  include/linux/nfs_fs_sb.h                 |  10 +
>  include/linux/nfs_xdr.h                   |  20 +-
>  include/linux/nfslocalio.h                |  41 ++
>  include/linux/sunrpc/auth.h               |   4 +
>  include/linux/sunrpc/svc.h                |   7 +-
>  net/sunrpc/auth.c                         |  15 +
>  net/sunrpc/clnt.c                         |   1 -
>  net/sunrpc/svc.c                          |  68 +-
>  net/sunrpc/svc_xprt.c                     |   2 +-
>  net/sunrpc/svcauth_unix.c                 |   3 +-
>  44 files changed, 1951 insertions(+), 135 deletions(-)
>  create mode 100644 Documentation/filesystems/nfs/localio.rst
>  create mode 100644 fs/nfs/localio.c
>  create mode 100644 fs/nfs_common/nfslocalio.c
>  create mode 100644 fs/nfsd/localio.c
>  create mode 100644 include/linux/nfslocalio.h
> 
> -- 
> 2.44.0
> 

Shall we start to think about how to merge this?

Should all of it go through one tree, or can the NFSD pieces be
taken via the NFSD tree and the NFS pieces via the NFS client tree?

Trond, Anna, opinions?
Anna Schumaker June 26, 2024, 8:45 p.m. UTC | #2
On Wed, Jun 26, 2024 at 2:49 PM Chuck Lever <chuck.lever@oracle.com> wrote:
>
> On Wed, Jun 26, 2024 at 02:24:20PM -0400, Mike Snitzer wrote:
> > Hi,
> >
> > Changes since v7:
> > - Switched from using SRCU to percpu_ref to interlock
> >   nfsd_destroy_serv() and nfsd_open_local_fh().
> > - Dropped the "nfs/localio: use dedicated workqueues for filesystem
> >   read and write" patch, will revisit if/when needed based on evidence
> > - Changed NFSD_MAY_LOCALIO from 0x800000 to 0x2000.
> > - Various renames in fs/nfsd/localio.c XDR code suggested by Chuck.
> > - Fixed localio_procedures1 and ARRAY_SIZE  suggested by Neil.
> > - Fixed nfsd_uuid_is_local() to dereference nfsd_uuid within rcu
> > - Removed a few dprintk in fs/{nfs,nfsd}/localio.c
> > - Documentation improvements suggested by Jeff.
> >
> > TODO:
> > - Must fix xfstests generic/355 (clear suid bit on write)
> > - Must fix localio's nfs_get_vfs_attr() to support NFS v4 same as is
> >   done with nfsd4_change_attribute(). But first attempt to do so was
> >   met with a crash due to the extra STATX_BTIME | STATX_CHANGE_COOKIE
> >   being included in the request_mask passed to vfs_getattr().
> >
> > All review and comments are welcome!
> >
> > Thanks,
> > Mike
> >
> > My git tree is here:
> > https://git.kernel.org/pub/scm/linux/kernel/git/snitzer/linux.git/
> >
> > This v8 is both branch nfs-localio-for-6.11 (always tracks latest)
> > and nfs-localio-for-6.11.v8
> >
> > Mike Snitzer (10):
> >   nfs_common: add NFS LOCALIO auxiliary protocol enablement
> >   nfsd: add "localio" support
> >   nfsd/localio: manage netns reference in nfsd_open_local_fh
> >   nfsd: use percpu_ref to interlock nfsd_destroy_serv and nfsd_open_local_fh
> >   nfs/nfsd: add Kconfig options to allow localio to be enabled
> >   nfs: factor out {encode,decode}_opaque_fixed to nfs_xdr.h
> >   SUNRPC: remove call_allocate() BUG_ON if p_arglen=0 to allow RPC with void arg
> >   nfs: implement client support for NFS_LOCALIO_PROGRAM
> >   nfsd: implement server support for NFS_LOCALIO_PROGRAM
> >   nfs: add Documentation/filesystems/nfs/localio.rst
> >
> > NeilBrown (1):
> >   SUNRPC: replace program list with program array
> >
> > Trond Myklebust (2):
> >   NFS: Enable localio for non-pNFS I/O
> >   pnfs/flexfiles: Enable localio for flexfiles I/O
> >
> > Weston Andros Adamson (5):
> >   nfs: pass nfs_client to nfs_initiate_pgio
> >   nfs: pass descriptor thru nfs_initiate_pgio path
> >   nfs: pass struct file to nfs_init_pgio and nfs_init_commit
> >   sunrpc: add rpcauth_map_to_svc_cred_local
> >   nfs: add "localio" support
> >
> >  Documentation/filesystems/nfs/localio.rst | 135 ++++
> >  fs/Kconfig                                |   3 +
> >  fs/nfs/Kconfig                            |  14 +
> >  fs/nfs/Makefile                           |   1 +
> >  fs/nfs/blocklayout/blocklayout.c          |   6 +-
> >  fs/nfs/client.c                           |  15 +-
> >  fs/nfs/filelayout/filelayout.c            |  16 +-
> >  fs/nfs/flexfilelayout/flexfilelayout.c    | 131 +++-
> >  fs/nfs/flexfilelayout/flexfilelayout.h    |   2 +
> >  fs/nfs/flexfilelayout/flexfilelayoutdev.c |   6 +
> >  fs/nfs/inode.c                            |   4 +
> >  fs/nfs/internal.h                         |  60 +-
> >  fs/nfs/localio.c                          | 793 ++++++++++++++++++++++
> >  fs/nfs/nfs4xdr.c                          |  13 -
> >  fs/nfs/nfstrace.h                         |  61 ++
> >  fs/nfs/pagelist.c                         |  32 +-
> >  fs/nfs/pnfs.c                             |  24 +-
> >  fs/nfs/pnfs.h                             |   6 +-
> >  fs/nfs/pnfs_nfs.c                         |   2 +-
> >  fs/nfs/write.c                            |  13 +-
> >  fs/nfs_common/Makefile                    |   3 +
> >  fs/nfs_common/nfslocalio.c                |  74 ++
> >  fs/nfsd/Kconfig                           |  14 +
> >  fs/nfsd/Makefile                          |   1 +
> >  fs/nfsd/filecache.c                       |   2 +-
> >  fs/nfsd/localio.c                         | 329 +++++++++
> >  fs/nfsd/netns.h                           |  12 +-
> >  fs/nfsd/nfsctl.c                          |   2 +-
> >  fs/nfsd/nfsd.h                            |   2 +-
> >  fs/nfsd/nfssvc.c                          | 116 +++-
> >  fs/nfsd/trace.h                           |   3 +-
> >  fs/nfsd/vfs.h                             |   9 +
> >  include/linux/nfs.h                       |   9 +
> >  include/linux/nfs_fs.h                    |   2 +
> >  include/linux/nfs_fs_sb.h                 |  10 +
> >  include/linux/nfs_xdr.h                   |  20 +-
> >  include/linux/nfslocalio.h                |  41 ++
> >  include/linux/sunrpc/auth.h               |   4 +
> >  include/linux/sunrpc/svc.h                |   7 +-
> >  net/sunrpc/auth.c                         |  15 +
> >  net/sunrpc/clnt.c                         |   1 -
> >  net/sunrpc/svc.c                          |  68 +-
> >  net/sunrpc/svc_xprt.c                     |   2 +-
> >  net/sunrpc/svcauth_unix.c                 |   3 +-
> >  44 files changed, 1951 insertions(+), 135 deletions(-)
> >  create mode 100644 Documentation/filesystems/nfs/localio.rst
> >  create mode 100644 fs/nfs/localio.c
> >  create mode 100644 fs/nfs_common/nfslocalio.c
> >  create mode 100644 fs/nfsd/localio.c
> >  create mode 100644 include/linux/nfslocalio.h
> >
> > --
> > 2.44.0
> >
>
> Shall we start to think about how to merge this?
>
> Should all of it go through one tree, or can the NFSD pieces be
> taken via the NFSD tree and the NFS pieces via the NFS client tree?
>
> Trond, Anna, opinions?

I don't have a strong preference either way. If it can be divided up
easily then it could go in through separate trees, but if not I'm okay
with one tree. I'm assuming aim for 6.12 so it has some soak time in
-next?

Anna

>
> --
> Chuck Lever