| Message ID | 160641091184.350621.18059939865682566764.stgit@klimt.1015granger.net (mailing list archive) |
|---|---|
| State | Not Applicable |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | [v2] SUNRPC: Use zero-copy to perform socket send operations | expand |
| Context | Check | Description |
|---|---|---|
| netdev/cover_letter | success | Link |
| netdev/fixes_present | success | Link |
| netdev/patch_count | success | Link |
| netdev/tree_selection | success | Guessed tree name to be net-next |
| netdev/subject_prefix | warning | Target tree name not specified in the subject |
| netdev/source_inline | success | Was 0 now: 0 |
| netdev/verify_signedoff | success | Link |
| netdev/module_param | success | Was 0 now: 0 |
| netdev/build_32bit | success | Errors and warnings before: 0 this patch: 0 |
| netdev/kdoc | success | Errors and warnings before: 0 this patch: 0 |
| netdev/verify_fixes | success | Link |
| netdev/checkpatch | success | total: 0 errors, 0 warnings, 0 checks, 20 lines checked |
| netdev/build_allmodconfig_warn | success | Errors and warnings before: 0 this patch: 0 |
| netdev/header_inline | success | Link |
| netdev/stable | success | Stable not CCed |
diff --git a/net/sunrpc/socklib.c b/net/sunrpc/socklib.c index d52313af82bc..af47596a7bdd 100644 --- a/net/sunrpc/socklib.c +++ b/net/sunrpc/socklib.c @@ -226,9 +226,12 @@ static int xprt_send_pagedata(struct socket *sock, struct msghdr *msg, if (err < 0) return err; + msg->msg_flags |= MSG_ZEROCOPY; iov_iter_bvec(&msg->msg_iter, WRITE, xdr->bvec, xdr_buf_pagecount(xdr), xdr->page_len + xdr->page_base); - return xprt_sendmsg(sock, msg, base + xdr->page_base); + err = xprt_sendmsg(sock, msg, base + xdr->page_base); + msg->msg_flags &= ~MSG_ZEROCOPY; + return err; } /* Common case: diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index b248f2349437..6f199cbb055d 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -1176,6 +1176,7 @@ static void svc_tcp_init(struct svc_sock *svsk, struct svc_serv *serv) svsk->sk_datalen = 0; memset(&svsk->sk_pages[0], 0, sizeof(svsk->sk_pages)); + sock_set_flag(sk, SOCK_ZEROCOPY); tcp_sk(sk)->nonagle |= TCP_NAGLE_OFF; set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags);
Daire Byrne reports a ~50% aggregrate throughput regression on his Linux NFS server after commit da1661b93bf4 ("SUNRPC: Teach server to use xprt_sock_sendmsg for socket sends"), which replaced kernel_send_page() calls in NFSD's socket send path with calls to sock_sendmsg() using iov_iter. Investigation showed that tcp_sendmsg() was not using zero-copy to send the xdr_buf's bvec pages, but instead was relying on memcpy. Set up the socket and each msghdr that bears bvec pages to use the zero-copy mechanism in tcp_sendmsg. The server side enables full zero-copy, but the client side does not, because pages passed to the the RPC layer are not guaranteed to be stable. They include unlocked page cache pages as well as O_DIRECT pages. Reported-by: Daire Byrne <daire@dneg.com> BugLink: https://bugzilla.kernel.org/show_bug.cgi?id=209439 Fixes: da1661b93bf4 ("SUNRPC: Teach server to use xprt_sock_sendmsg for socket sends") Signed-off-by: Chuck Lever <chuck.lever@oracle.com> --- net/sunrpc/socklib.c | 5 ++++- net/sunrpc/svcsock.c | 1 + 2 files changed, 5 insertions(+), 1 deletion(-) Hi- Because I got a 0-day regression report this morning based on the original version of this patch, I thought I should repost with a revised version that is corrected to disable zero-copy on the client-side, as we previously discussed. Changes since v1: - Disable zero-copy optimization on the client-side