diff mbox series

[v1,4/7] svcrdma: Switch Receive CQ to soft IRQ

Message ID 169988300485.6417.15944939065289556526.stgit@bazille.1015granger.net (mailing list archive)
State Not Applicable
Headers show
Series Switch NFS server CQs to use soft IRQ | expand

Commit Message

Chuck Lever Nov. 13, 2023, 1:43 p.m. UTC 
From: Chuck Lever <chuck.lever@oracle.com>

The original rationale for handling Receive completions in process
context was to eliminate the use of a bottom-half-disabled spin
lock. This was intended to simplify assumptions made in the Receive
code paths and reduce lock contention.

However, a completion handled during soft IRQ is considerably lower
in average latency than taking a spin lock that disables bottom
halves, since with soft IRQ, the completion interrupt no longer has
to get scheduled onto a workqueue.

Now that Receive contexts are pre-allocated and the RPC service
thread scheduler is constant time, moving Receive completion
processing to soft IRQ is safe and simple.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 net/sunrpc/xprtrdma/svc_rdma_recvfrom.c  |    4 ++--
 net/sunrpc/xprtrdma/svc_rdma_transport.c |    4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
diff mbox series

Patch

diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
index 6191ce20f89e..4ee219924433 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
@@ -810,14 +810,14 @@  int svc_rdma_recvfrom(struct svc_rqst *rqstp)
 	rqstp->rq_xprt_ctxt = NULL;
 
 	ctxt = NULL;
-	spin_lock(&rdma_xprt->sc_rq_dto_lock);
+	spin_lock_bh(&rdma_xprt->sc_rq_dto_lock);
 	ctxt = svc_rdma_next_recv_ctxt(&rdma_xprt->sc_rq_dto_q);
 	if (ctxt)
 		list_del(&ctxt->rc_list);
 	else
 		/* No new incoming requests, terminate the loop */
 		clear_bit(XPT_DATA, &xprt->xpt_flags);
-	spin_unlock(&rdma_xprt->sc_rq_dto_lock);
+	spin_unlock_bh(&rdma_xprt->sc_rq_dto_lock);
 
 	/* Unblock the transport for the next receive */
 	svc_xprt_received(xprt);
diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c
index 2abd895046ee..7bd50efeeb4e 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_transport.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c
@@ -433,8 +433,8 @@  static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt)
 					    IB_POLL_WORKQUEUE);
 	if (IS_ERR(newxprt->sc_sq_cq))
 		goto errout;
-	newxprt->sc_rq_cq =
-		ib_alloc_cq_any(dev, newxprt, rq_depth, IB_POLL_WORKQUEUE);
+	newxprt->sc_rq_cq = ib_alloc_cq_any(dev, newxprt, rq_depth,
+					    IB_POLL_SOFTIRQ);
 	if (IS_ERR(newxprt->sc_rq_cq))
 		goto errout;