[RFC,00/11] Linux NFS server support for multiple Write chunks

Message

Chuck Lever III March 19, 2020, 3:20 p.m. UTC

The RPC/RDMA version 1 protocol allows clients to provision more
than one Write chunk in each RPC/RDMA message. The Linux NFS client
never has need to construct a message with multiple Write chunks, so
the Linux NFS server has never implemented support for them.

At testing events, we discovered that the Solaris NFS client can
emit such requests on occasion, but only when an application invokes
readv(2) on a "forcedirectio" mount -- rare, indeed.

Even so, it's been on my "to-do" list for quite some time to get the
Linux NFS server to handle multiple Write chunks. While addressing
the recent NFSD/RDMA bug with Linux filesystems that do not have a 
.read_splice method [1], I realized that it was time to get this one
off my plate.

So here is an attempt to support NFS/RDMA clients that send multiple
Write chunks. To do this generically requires more xdr_buf slicing
and dicing than the simple "zero or one" implementation.

At the same time, the ability to send RDMA Write requests _outside_
the .xpo_sendto path is introduced. Extensive testing has not
revealed any functional or performance regression with this change.

Thoughts and comments are welcome.


[1] - https://bugzilla.kernel.org/show_bug.cgi?id=198053

---

Chuck Lever (11):

      SUNRPC: Adjust synopsis of xdr_buf_subsegment()
      svcrdma: Clean up RDMA Write path
      NFSD: Invoke svc_encode_read_payload in "read" NFSD encoders
      svcrdma: Post RDMA Writes while XDR encoding replies
      svcrdma: Clean up svc_rdma_encode_reply_chunk()
      svcrdma: Cache number of Write chunks
      svcrdma: Add a data structure to track READ payloads
      svcrdma: Add svc_rdma_skip_payloads()
      svcrdma: Support multiple READ payloads when pulling up
      svcrdma: Support multiple READ payloads in svc_rdma_map_reply_msg()
      svcrdma: Support multiple Write chunks in svc_rdma_send_reply_chunk


 fs/nfsd/nfs3xdr.c                       |   4 +
 fs/nfsd/nfs4xdr.c                       |   3 +
 fs/nfsd/nfsxdr.c                        |   4 +
 include/linux/sunrpc/svc_rdma.h         |  24 +-
 include/trace/events/rpcrdma.h          |  16 +-
 net/sunrpc/xprtrdma/svc_rdma_recvfrom.c |  32 +-
 net/sunrpc/xprtrdma/svc_rdma_rw.c       | 134 +++++---
 net/sunrpc/xprtrdma/svc_rdma_sendto.c   | 556 ++++++++++++++++++++------------
 8 files changed, 486 insertions(+), 287 deletions(-)

--
Chuck Lever