From patchwork Tue Jun 6 17:33:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13269503 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCCA2C7EE31 for ; Tue, 6 Jun 2023 17:33:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233419AbjFFRdt (ORCPT ); Tue, 6 Jun 2023 13:33:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238712AbjFFRd1 (ORCPT ); Tue, 6 Jun 2023 13:33:27 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 317381707; Tue, 6 Jun 2023 10:33:26 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id AABBD6305C; Tue, 6 Jun 2023 17:33:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C336BC433EF; Tue, 6 Jun 2023 17:33:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686072805; bh=yXFi2/2FH/2B0iC3mHMSuEHuBGVTQgXcjPnxZDOEOI4=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=o3t/keUmgg9/Le8Csud8E9LxyZljh0FqYRYc2wXPAuC4ejS5u/HeG47Q+EiZzqWvN H8CFHi75H0eSSojaNPZGSCO/fh2bwmaqbVIysvr0zUwnlACwoQ8e+Jl+Te/y11Z6/V vBC27MsOUgsj6FFKYsu+Z1ljI96MTJzloNFMW7QUB199UgThGl4gkJn0xLSzqt7qQs nGNDZbL0KQ6pwIQI8rAgYhovP8/bKyVenWBiY1jWIVgBzNnC1quFiTqHNR0K0ihfy3 uCRf9sw1JZUDdR5LMe//y/xJiCYxkhEZAmVRTFbytaaBu9ogYAE183HbTxPCa2O7DA 3+q6r2dQbmihA== Subject: [PATCH v1 1/4] SUNRPC: Revert cc93ce9529a6 From: Chuck Lever To: linux-nfs@vger.kernel.org Cc: Chuck Lever , linux-rdma@vger.kernel.org, tom@talpey.com Date: Tue, 06 Jun 2023 13:33:23 -0400 Message-ID: <168607280379.2076.16904085572327572955.stgit@manet.1015granger.net> In-Reply-To: <168607259937.2076.15447551371235387735.stgit@manet.1015granger.net> References: <168607259937.2076.15447551371235387735.stgit@manet.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chuck Lever Pre-requisite for releasing pages in the send completion handler. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index a35d1e055b1a..8e7ccef74207 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -975,11 +975,6 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) ret = svc_rdma_send_reply_msg(rdma, sctxt, rctxt, rqstp); if (ret < 0) goto put_ctxt; - - /* Prevent svc_xprt_release() from releasing the page backing - * rq_res.head[0].iov_base. It's no longer being accessed by - * the I/O device. */ - rqstp->rq_respages++; return 0; reply_chunk: From patchwork Tue Jun 6 17:33:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13269504 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71B42C7EE24 for ; Tue, 6 Jun 2023 17:33:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236116AbjFFRdu (ORCPT ); Tue, 6 Jun 2023 13:33:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238806AbjFFRdd (ORCPT ); Tue, 6 Jun 2023 13:33:33 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B488C1705; Tue, 6 Jun 2023 10:33:32 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 231966305C; Tue, 6 Jun 2023 17:33:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3D2ACC433EF; Tue, 6 Jun 2023 17:33:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686072811; bh=UpXOOvsn5wQY9HQgBgMPhKKi5IeBhkq0JbPhinW+NfM=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=GRNXooSYH+UZil6RkLMYymYNx5NWL6tWm758xHx2goibO+mKX7N0tcmiKPaBfnvp2 0lxPVdThRFsMKMO1ePMCOIMMvHpjsLmQ3kjbv2cmmMcsJD9kbTlTXy9cyaWuKwITjR QbSkIK4wUa2uD/Ja1GXrLRujy50loiS1/4MN6bBQfOKHba4MeWbXRIMKCraLPJZDqU Xr4XbD3t/806ADtD2k0Dq4JPZxRxuWSuFeLNJRhrFXrzLWXjIuC6DcyNnA78GzxgwG ripsAVJuETsQ3n3gQhbygfWPAx79ia4hVQJ1Yvrt3J0Xtr1l8x5VR14xX63YjduXdm WGqJwNzLBEWHg== Subject: [PATCH v1 2/4] SUNRPC: Revert 579900670ac7 From: Chuck Lever To: linux-nfs@vger.kernel.org Cc: Chuck Lever , linux-rdma@vger.kernel.org, tom@talpey.com Date: Tue, 06 Jun 2023 13:33:30 -0400 Message-ID: <168607281028.2076.11971808021420433856.stgit@manet.1015granger.net> In-Reply-To: <168607259937.2076.15447551371235387735.stgit@manet.1015granger.net> References: <168607259937.2076.15447551371235387735.stgit@manet.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chuck Lever Pre-requisite for releasing pages in the send completion handler. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 3 ++- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 25 +++++++++++++++++++++++++ 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index a0f3ea357977..8e654da55170 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -158,8 +158,9 @@ struct svc_rdma_send_ctxt { struct xdr_buf sc_hdrbuf; struct xdr_stream sc_stream; void *sc_xprt_buf; + int sc_page_count; int sc_cur_sge_no; - + struct page *sc_pages[RPCSVC_MAXPAGES]; struct ib_sge sc_sges[]; }; diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 8e7ccef74207..4c62bc41ea40 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -213,6 +213,7 @@ struct svc_rdma_send_ctxt *svc_rdma_send_ctxt_get(struct svcxprt_rdma *rdma) ctxt->sc_send_wr.num_sge = 0; ctxt->sc_cur_sge_no = 0; + ctxt->sc_page_count = 0; return ctxt; out_empty: @@ -227,6 +228,8 @@ struct svc_rdma_send_ctxt *svc_rdma_send_ctxt_get(struct svcxprt_rdma *rdma) * svc_rdma_send_ctxt_put - Return send_ctxt to free list * @rdma: controlling svcxprt_rdma * @ctxt: object to return to the free list + * + * Pages left in sc_pages are DMA unmapped and released. */ void svc_rdma_send_ctxt_put(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *ctxt) @@ -234,6 +237,9 @@ void svc_rdma_send_ctxt_put(struct svcxprt_rdma *rdma, struct ib_device *device = rdma->sc_cm_id->device; unsigned int i; + for (i = 0; i < ctxt->sc_page_count; ++i) + put_page(ctxt->sc_pages[i]); + /* The first SGE contains the transport header, which * remains mapped until @ctxt is destroyed. */ @@ -798,6 +804,25 @@ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, svc_rdma_xb_dma_map, &args); } +/* The svc_rqst and all resources it owns are released as soon as + * svc_rdma_sendto returns. Transfer pages under I/O to the ctxt + * so they are released by the Send completion handler. + */ +static inline void svc_rdma_save_io_pages(struct svc_rqst *rqstp, + struct svc_rdma_send_ctxt *ctxt) +{ + int i, pages = rqstp->rq_next_page - rqstp->rq_respages; + + ctxt->sc_page_count += pages; + for (i = 0; i < pages; i++) { + ctxt->sc_pages[i] = rqstp->rq_respages[i]; + rqstp->rq_respages[i] = NULL; + } + + /* Prevent svc_xprt_release from releasing pages in rq_pages */ + rqstp->rq_next_page = rqstp->rq_respages; +} + /* Prepare the portion of the RPC Reply that will be transmitted * via RDMA Send. The RPC-over-RDMA transport header is prepared * in sc_sges[0], and the RPC xdr_buf is prepared in following sges. From patchwork Tue Jun 6 17:33:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13269505 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0867AC7EE33 for ; Tue, 6 Jun 2023 17:33:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237748AbjFFRdu (ORCPT ); Tue, 6 Jun 2023 13:33:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40908 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238829AbjFFRdk (ORCPT ); Tue, 6 Jun 2023 13:33:40 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1BC1D10F8; Tue, 6 Jun 2023 10:33:39 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8D1A963041; Tue, 6 Jun 2023 17:33:38 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A4C4EC433EF; Tue, 6 Jun 2023 17:33:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686072818; bh=jKBxrwkgIAG9zDtXjb4u4ycgi1ynitcvqXX5JqNiDUU=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=LwF1gxFlnUmTvghk2DchgLMj1QXGke23SgwHmoC8Duv7VMQN13HKwnl5D3mCQtr0Y xuKT9m9/b86ZgLeF4UiKae/pNJcSNTMRRRoBscRqvuHs7y7icU835gVpwlnlTrektW DhxWEzVOYlge/oSxAXq0rFvAXVnwT6+vBwcFxUyvkZYexlHjp9vCVRkFTdCADG9Hfj pcTd3/CMbC7lTA4GzHUPUAd3vAjbCT7h8dSywqIPpGT5ayjc1kD1+8LzmGIyRRtP3j 9rM/VojZLGWOiQfoTo72BGPyumNMnQcf2iMAR4tYPjFoH47cwl2cF+FsPvTSx6punN AQwTp4abGn8ZQ== Subject: [PATCH v1 3/4] SUNRPC: Revert 2a1e4f21d841 From: Chuck Lever To: linux-nfs@vger.kernel.org Cc: Chuck Lever , linux-rdma@vger.kernel.org, tom@talpey.com Date: Tue, 06 Jun 2023 13:33:36 -0400 Message-ID: <168607281673.2076.15123945076794227890.stgit@manet.1015granger.net> In-Reply-To: <168607259937.2076.15447551371235387735.stgit@manet.1015granger.net> References: <168607259937.2076.15447551371235387735.stgit@manet.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chuck Lever Pre-requisite for releasing pages in the send completion handler. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 1 - net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 8 +------- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 27 ++++++++++++--------------- 3 files changed, 13 insertions(+), 23 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 8e654da55170..a5ee0af2a310 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -154,7 +154,6 @@ struct svc_rdma_send_ctxt { struct ib_send_wr sc_send_wr; struct ib_cqe sc_cqe; - struct completion sc_done; struct xdr_buf sc_hdrbuf; struct xdr_stream sc_stream; void *sc_xprt_buf; diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index aa2227a7e552..7420a2c990c7 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -93,13 +93,7 @@ static int svc_rdma_bc_sendto(struct svcxprt_rdma *rdma, */ get_page(virt_to_page(rqst->rq_buffer)); sctxt->sc_send_wr.opcode = IB_WR_SEND; - ret = svc_rdma_send(rdma, sctxt); - if (ret < 0) - return ret; - - ret = wait_for_completion_killable(&sctxt->sc_done); - svc_rdma_send_ctxt_put(rdma, sctxt); - return ret; + return svc_rdma_send(rdma, sctxt); } /* Server-side transport endpoint wants a whole page for its send diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 4c62bc41ea40..1ae4236d04a3 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -147,7 +147,6 @@ svc_rdma_send_ctxt_alloc(struct svcxprt_rdma *rdma) ctxt->sc_send_wr.wr_cqe = &ctxt->sc_cqe; ctxt->sc_send_wr.sg_list = ctxt->sc_sges; ctxt->sc_send_wr.send_flags = IB_SEND_SIGNALED; - init_completion(&ctxt->sc_done); ctxt->sc_cqe.done = svc_rdma_wc_send; ctxt->sc_xprt_buf = buffer; xdr_buf_init(&ctxt->sc_hdrbuf, ctxt->sc_xprt_buf, @@ -286,12 +285,12 @@ static void svc_rdma_wc_send(struct ib_cq *cq, struct ib_wc *wc) container_of(cqe, struct svc_rdma_send_ctxt, sc_cqe); svc_rdma_wake_send_waiters(rdma, 1); - complete(&ctxt->sc_done); if (unlikely(wc->status != IB_WC_SUCCESS)) goto flushed; trace_svcrdma_wc_send(wc, &ctxt->sc_cid); + svc_rdma_send_ctxt_put(rdma, ctxt); return; flushed: @@ -299,6 +298,7 @@ static void svc_rdma_wc_send(struct ib_cq *cq, struct ib_wc *wc) trace_svcrdma_wc_send_err(wc, &ctxt->sc_cid); else trace_svcrdma_wc_send_flush(wc, &ctxt->sc_cid); + svc_rdma_send_ctxt_put(rdma, ctxt); svc_xprt_deferred_close(&rdma->sc_xprt); } @@ -315,8 +315,6 @@ int svc_rdma_send(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *ctxt) struct ib_send_wr *wr = &ctxt->sc_send_wr; int ret; - reinit_completion(&ctxt->sc_done); - /* Sync the transport header buffer */ ib_dma_sync_single_for_device(rdma->sc_pd->device, wr->sg_list[0].addr, @@ -808,8 +806,8 @@ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, * svc_rdma_sendto returns. Transfer pages under I/O to the ctxt * so they are released by the Send completion handler. */ -static inline void svc_rdma_save_io_pages(struct svc_rqst *rqstp, - struct svc_rdma_send_ctxt *ctxt) +static void svc_rdma_save_io_pages(struct svc_rqst *rqstp, + struct svc_rdma_send_ctxt *ctxt) { int i, pages = rqstp->rq_next_page - rqstp->rq_respages; @@ -852,6 +850,8 @@ static int svc_rdma_send_reply_msg(struct svcxprt_rdma *rdma, if (ret < 0) return ret; + svc_rdma_save_io_pages(rqstp, sctxt); + if (rctxt->rc_inv_rkey) { sctxt->sc_send_wr.opcode = IB_WR_SEND_WITH_INV; sctxt->sc_send_wr.ex.invalidate_rkey = rctxt->rc_inv_rkey; @@ -859,13 +859,7 @@ static int svc_rdma_send_reply_msg(struct svcxprt_rdma *rdma, sctxt->sc_send_wr.opcode = IB_WR_SEND; } - ret = svc_rdma_send(rdma, sctxt); - if (ret < 0) - return ret; - - ret = wait_for_completion_killable(&sctxt->sc_done); - svc_rdma_send_ctxt_put(rdma, sctxt); - return ret; + return svc_rdma_send(rdma, sctxt); } /** @@ -931,8 +925,7 @@ void svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, sctxt->sc_sges[0].length = sctxt->sc_hdrbuf.len; if (svc_rdma_send(rdma, sctxt)) goto put_ctxt; - - wait_for_completion_killable(&sctxt->sc_done); + return; put_ctxt: svc_rdma_send_ctxt_put(rdma, sctxt); @@ -1006,6 +999,10 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) if (ret != -E2BIG && ret != -EINVAL) goto put_ctxt; + /* Send completion releases payload pages that were part + * of previously posted RDMA Writes. + */ + svc_rdma_save_io_pages(rqstp, sctxt); svc_rdma_send_error_msg(rdma, sctxt, rctxt, ret); return 0; From patchwork Tue Jun 6 17:33:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13269506 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54860C7EE29 for ; Tue, 6 Jun 2023 17:34:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237890AbjFFRdv (ORCPT ); Tue, 6 Jun 2023 13:33:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238832AbjFFRdq (ORCPT ); Tue, 6 Jun 2023 13:33:46 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 71C9112D; Tue, 6 Jun 2023 10:33:45 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0D27D63041; Tue, 6 Jun 2023 17:33:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 24C8BC433EF; Tue, 6 Jun 2023 17:33:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686072824; bh=EKTSa+v5GbZS0K05Nc2fWCEetJaTwG5m3sEiNCmIIiY=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=Px71UESCyD75jMRhSX3GGlUIgeRDpdeGowk3hwCAOp9uGzRj7EhMEuFSeue5uYNHH yqny4Y4REbo89G6XUuGQNnj8qKH1bR6n1ylBF4CS2YT8mVscPc6WhpzpOhWnO4ydJt a4qHDO4pnrod0F4T+wXIGFvpIfSDT4cscMygBZ0L6cnsMVE/jMOOld6Slzad0B3SGU wdrhVuJfOMT4930ZufnsKYJz5H2kCOoddCUkyGPtCw46TN+HECmsc7yTGOGLpyiTco JLZ0vs4I5ATBYyrnjJf0ZQibgEQUutHOo0OTS9aE0zhllWOxZn1VXlCHLk4lEEt3s7 BjQ2DWYHaV7Yw== Subject: [PATCH v1 4/4] SUNRPC: Optimize page release in svc_rdma_sendto() From: Chuck Lever To: linux-nfs@vger.kernel.org Cc: Chuck Lever , linux-rdma@vger.kernel.org, tom@talpey.com Date: Tue, 06 Jun 2023 13:33:43 -0400 Message-ID: <168607282316.2076.15999420482159168604.stgit@manet.1015granger.net> In-Reply-To: <168607259937.2076.15447551371235387735.stgit@manet.1015granger.net> References: <168607259937.2076.15447551371235387735.stgit@manet.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chuck Lever Now that we have bulk page allocation and release APIs, it's more efficient to use those than it is for nfsd threads to wait for send completions. Previous patches have eliminated the calls to wait_for_completion() and complete(), in order to avoid scheduler overhead. Now release pages-under-I/O in the send completion handler using the efficient bulk release API. I've measured a 7% reduction in cumulative CPU utilization in svc_rdma_sendto(), svc_rdma_wc_send(), and svc_xprt_release(). In particular, using release_pages() instead of complete() cuts the time per svc_rdma_wc_send() call by two-thirds. This helps improve scalability because svc_rdma_wc_send() is single-threaded per connection. Signed-off-by: Chuck Lever Reviewed-by: Tom Talpey --- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 1ae4236d04a3..24228f3611e8 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -236,8 +236,8 @@ void svc_rdma_send_ctxt_put(struct svcxprt_rdma *rdma, struct ib_device *device = rdma->sc_cm_id->device; unsigned int i; - for (i = 0; i < ctxt->sc_page_count; ++i) - put_page(ctxt->sc_pages[i]); + if (ctxt->sc_page_count) + release_pages(ctxt->sc_pages, ctxt->sc_page_count); /* The first SGE contains the transport header, which * remains mapped until @ctxt is destroyed.