diff mbox series

[v3,4/5] SUNRPC: Revert e0a912e8ddba

Message ID 168979148257.1905271.8311839188162164611.stgit@morisot.1015granger.net (mailing list archive)
State New, archived
Headers show
Series Send RPC-on-TCP with one sock_sendmsg() call | expand

Commit Message

Chuck Lever July 19, 2023, 6:31 p.m. UTC 
From: Chuck Lever <chuck.lever@oracle.com>

Flamegraph analysis showed that the cork/uncork calls consume
nearly a third of the CPU time spent in svc_tcp_sendto(). The
other two consumers are mutex lock/unlock and svc_tcp_sendmsg().

Now that svc_tcp_sendto() coalesces RPC messages properly, there
is no need to introduce artificial delays to prevent sending
partial messages.

After applying this change, I measured a 1.2K read IOPS increase
for 8KB random I/O (several percent) on 56Gb IP over IB.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 include/linux/sunrpc/svcsock.h |    2 --
 net/sunrpc/svcsock.c           |    6 ------
 2 files changed, 8 deletions(-)

Comments

David Howells July 26, 2023, 12:14 p.m. UTC | #1 
Chuck Lever <cel@kernel.org> wrote:

> Flamegraph analysis showed that the cork/uncork calls consume
> nearly a third of the CPU time spent in svc_tcp_sendto(). The
> other two consumers are mutex lock/unlock and svc_tcp_sendmsg().
> 
> Now that svc_tcp_sendto() coalesces RPC messages properly, there
> is no need to introduce artificial delays to prevent sending
> partial messages.
> 
> After applying this change, I measured a 1.2K read IOPS increase
> for 8KB random I/O (several percent) on 56Gb IP over IB.
> 
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>

Reviewed-by: David Howells <dhowells@redhat.com>
diff mbox series

Patch

diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h
index caf3308f1f07..a7ea54460b1a 100644
--- a/include/linux/sunrpc/svcsock.h
+++ b/include/linux/sunrpc/svcsock.h
@@ -35,8 +35,6 @@  struct svc_sock {
 	/* Total length of the data (not including fragment headers)
 	 * received so far in the fragments making up this rpc: */
 	u32			sk_datalen;
-	/* Number of queued send requests */
-	atomic_t		sk_sendqlen;
 
 	struct page_frag_cache  sk_frag_cache;
 
diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c
index f28790f282c2..7b7358908a21 100644
--- a/net/sunrpc/svcsock.c
+++ b/net/sunrpc/svcsock.c
@@ -1267,22 +1267,17 @@  static int svc_tcp_sendto(struct svc_rqst *rqstp)
 	svc_tcp_release_ctxt(xprt, rqstp->rq_xprt_ctxt);
 	rqstp->rq_xprt_ctxt = NULL;
 
-	atomic_inc(&svsk->sk_sendqlen);
 	mutex_lock(&xprt->xpt_mutex);
 	if (svc_xprt_is_dead(xprt))
 		goto out_notconn;
-	tcp_sock_set_cork(svsk->sk_sk, true);
 	err = svc_tcp_sendmsg(svsk, rqstp, marker, &sent);
 	trace_svcsock_tcp_send(xprt, err < 0 ? (long)err : sent);
 	if (err < 0 || sent != (xdr->len + sizeof(marker)))
 		goto out_close;
-	if (atomic_dec_and_test(&svsk->sk_sendqlen))
-		tcp_sock_set_cork(svsk->sk_sk, false);
 	mutex_unlock(&xprt->xpt_mutex);
 	return sent;
 
 out_notconn:
-	atomic_dec(&svsk->sk_sendqlen);
 	mutex_unlock(&xprt->xpt_mutex);
 	return -ENOTCONN;
 out_close:
@@ -1291,7 +1286,6 @@  static int svc_tcp_sendto(struct svc_rqst *rqstp)
 		  (err < 0) ? "got error" : "sent",
 		  (err < 0) ? err : sent, xdr->len);
 	svc_xprt_deferred_close(xprt);
-	atomic_dec(&svsk->sk_sendqlen);
 	mutex_unlock(&xprt->xpt_mutex);
 	return -EAGAIN;
 }