From patchwork Mon Jun 22 21:01:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11619119 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ED08F912 for ; Mon, 22 Jun 2020 21:01:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CE8772073E for ; Mon, 22 Jun 2020 21:01:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ExLEv35M" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728683AbgFVVBI (ORCPT ); Mon, 22 Jun 2020 17:01:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33498 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728545AbgFVVBH (ORCPT ); Mon, 22 Jun 2020 17:01:07 -0400 Received: from mail-qv1-xf42.google.com (mail-qv1-xf42.google.com [IPv6:2607:f8b0:4864:20::f42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9299DC061573; Mon, 22 Jun 2020 14:01:07 -0700 (PDT) Received: by mail-qv1-xf42.google.com with SMTP id u8so1599025qvj.12; Mon, 22 Jun 2020 14:01:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=Xqol0YYHpEvswBoUVlP6v8tsfvVQp+w5nGY/v7Hvy2s=; b=ExLEv35MNMsw8UbV3X40MXkpBk0iE1XQEej76A7Jh2C0die6PCERb5maLCyePmxLbM fCMUXpFBvNoLUn5s/b71vbulfopgLek+1FbahMPpTkKaQaFT7dPeDCooltdK/dD2bOnG fJp2DhiQiZ+8rIkOrn2eo07mrxVsmcl/kUUCezK9W3vYUHil3mF6j3OT9fyI+FUV4SJ0 w/eze16uqMm6kQFf8V4gW45eslKaiQZ3JrsirTLswI50fOAtOmg8YAcQfyLC3a26+zC6 08SlnnR28ylGxDNhQGNckNNJrHggaXXIFqaupN0mGXxSM3XcNuYe/sNPdr3MA5Pz0bdd IPnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=Xqol0YYHpEvswBoUVlP6v8tsfvVQp+w5nGY/v7Hvy2s=; b=fiJeilv/7OibkNVSim9NE5aWBUKixGl+AIjsEYC7AtmyFjkLGRBwBy2M/f8mwhMNOS OMqmO/HhWaRwptXVZ90kyVLvqeUAa2xAoke0TEfapYZDSJBtDOfWMxWhdU0L9MTEvBnF ad2+tnp80kEPVGrQQO9pEbj/jACCm4uEaMiXEo/V+1u+aH+b/w7G7VdE6KOjsRTFGXEq foGFLgiVjbZJAHV/VvP59PGuSljSl80L+mZinMygbOstPWSHqbgMThksVTNNunfoitOC 9vwNLgS9gHZ8DpYZpgoU0mOFlp30IhkfLok63FGmWtgnW6Tsf90c1hdjGzyMY9Zi/yPY kyog== X-Gm-Message-State: AOAM5339NT2DTceHodzYz5fLKfSaq7luhb2Uq1HijpmwonBJJXnoSm7J 93ioCIJZG9nQgCZQhn37I1mXeZhI X-Google-Smtp-Source: ABdhPJzVbg9n2ESXM1FuwWOYYmeCUjiy7C54nmCXusxXpTVhdHo9LJVbwgPpcZIdf4HMPYMzD9C1TQ== X-Received: by 2002:a0c:dc8c:: with SMTP id n12mr8081862qvk.221.1592859666523; Mon, 22 Jun 2020 14:01:06 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id w204sm14980202qka.41.2020.06.22.14.01.05 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Jun 2020 14:01:06 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 05ML15cQ018822; Mon, 22 Jun 2020 21:01:05 GMT Subject: [PATCH v1 1/6] svcrdma: Fix page leak in svc_rdma_recv_read_chunk() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 22 Jun 2020 17:01:05 -0400 Message-ID: <20200622210105.2144.36726.stgit@klimt.1015granger.net> In-Reply-To: <20200622205906.2144.43930.stgit@klimt.1015granger.net> References: <20200622205906.2144.43930.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Commit 07d0ff3b0cd2 ("svcrdma: Clean up Read chunk path") moved the page saver logic so that it gets executed event when an error occurs. In that case, the I/O is never posted, and those pages are then leaked. Errors in this path, however, are quite rare. Fixes: 07d0ff3b0cd2 ("svcrdma: Clean up Read chunk path") Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_rw.c | 28 +++++++++++++++++++++------- 1 file changed, 21 insertions(+), 7 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 5eb35309ecef..83806fa94def 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -684,7 +684,6 @@ static int svc_rdma_build_read_chunk(struct svc_rqst *rqstp, struct svc_rdma_read_info *info, __be32 *p) { - unsigned int i; int ret; ret = -EINVAL; @@ -707,12 +706,6 @@ static int svc_rdma_build_read_chunk(struct svc_rqst *rqstp, info->ri_chunklen += rs_length; } - /* Pages under I/O have been copied to head->rc_pages. - * Prevent their premature release by svc_xprt_release() . - */ - for (i = 0; i < info->ri_readctxt->rc_page_count; i++) - rqstp->rq_pages[i] = NULL; - return ret; } @@ -807,6 +800,26 @@ static int svc_rdma_build_pz_read_chunk(struct svc_rqst *rqstp, return ret; } +/* Pages under I/O have been copied to head->rc_pages. Ensure they + * are not released by svc_xprt_release() until the I/O is complete. + * + * This has to be done after all Read WRs are constructed to properly + * handle a page that is part of I/O on behalf of two different RDMA + * segments. + * + * Do this only if I/O has been posted. Otherwise, we do indeed want + * svc_xprt_release() to clean things up properly. + */ +static void svc_rdma_save_io_pages(struct svc_rqst *rqstp, + const unsigned int start, + const unsigned int num_pages) +{ + unsigned int i; + + for (i = start; i < num_pages + start; i++) + rqstp->rq_pages[i] = NULL; +} + /** * svc_rdma_recv_read_chunk - Pull a Read chunk from the client * @rdma: controlling RDMA transport @@ -860,6 +873,7 @@ int svc_rdma_recv_read_chunk(struct svcxprt_rdma *rdma, struct svc_rqst *rqstp, ret = svc_rdma_post_chunk_ctxt(&info->ri_cc); if (ret < 0) goto out_err; + svc_rdma_save_io_pages(rqstp, 0, head->rc_page_count); return 0; out_err: From patchwork Mon Jun 22 21:01:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11619123 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 22D3B161F for ; Mon, 22 Jun 2020 21:01:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 07DDE20767 for ; Mon, 22 Jun 2020 21:01:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="qcII2JTr" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730224AbgFVVBN (ORCPT ); Mon, 22 Jun 2020 17:01:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33512 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728545AbgFVVBM (ORCPT ); Mon, 22 Jun 2020 17:01:12 -0400 Received: from mail-qv1-xf44.google.com (mail-qv1-xf44.google.com [IPv6:2607:f8b0:4864:20::f44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC65CC061573; Mon, 22 Jun 2020 14:01:12 -0700 (PDT) Received: by mail-qv1-xf44.google.com with SMTP id g7so8650132qvx.11; Mon, 22 Jun 2020 14:01:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=qdqoEbktn4y6PV/kskrC+LFY1GfPSLmprda8TiArdPQ=; b=qcII2JTrTJMeqBN1qnNetl8aBPED4Juz6u04LAR86snYYRfRP7iG0bF+XuuFwRdGS+ e/PneF2BhSmN5c1Tfqjc4V1v+ccDLr3aSOzWzEk2m/GezVOA3vlDJZv3qzapAZ04dbQQ nD/SbytLoVh8GBGNjc/Gns/d4sn3Xic4EBjvjwZSQLcljSEbyMiBbJkfI9HfUv8mJtYu WSs53wUWUpjXUDYaHlJb0lDtaOVgcKkBXF/TQTnM8VNA6fkpQcZffmEN0wW+1MfLejaE XqAQ+w7kah6v1j8fX51Mkl6C+KgRPCz3Y/Iw/Ch006XZgPslw+2qZRVHwvw3vwEJukBx z5Ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=qdqoEbktn4y6PV/kskrC+LFY1GfPSLmprda8TiArdPQ=; b=HYyNpPaWfp3wUGzRuoDHo/CUb9H5I92HjYVOLD186ouW9F772J/fTL+uzx8BYuuIz3 6rtUTEPsdVmPDI/PwESV28f0W99PzOBEXgbFJuVsMiTHq81QbuPu4a+7mbG+Hq9SxrT2 Un93265qqpXM+7/1/s0ddozdX+gNvU0PtnmwuP7JVPp1tZDPFHzn8azjfC7cCPUHD7YW FqS/zLtdqKGxrkLOfpg8S+r0G7yftd5gejPWV8Oa3Tmg7M0V9ZTBST6ZUTsm+eYyw//w a/PGojX9T0v2D5x8MdNjubVjAlaXrf4tqho7Jlr8nGkAuQeTxOYpBDb6yYjARmtM6xrD Zxww== X-Gm-Message-State: AOAM530c70J3HbOL1tONEzPRcU7RnLtQ8rdC9JCPcuxePVjd8oR4JlMj Dzu+BUBBcn1mdHPBe5X8soWl8f/H X-Google-Smtp-Source: ABdhPJz0HDBzCtmKbXuOD0U/Sl8f4TI0FjKpDCRkvOq7tyIQjmcoPs3jhMYutikOKmGsfB3oluKzkw== X-Received: by 2002:ad4:476a:: with SMTP id d10mr11319225qvx.13.1592859671759; Mon, 22 Jun 2020 14:01:11 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id e25sm1161316qtc.93.2020.06.22.14.01.11 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Jun 2020 14:01:11 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 05ML1AZk018825; Mon, 22 Jun 2020 21:01:10 GMT Subject: [PATCH v1 2/6] svcrdma: Remove save_io_pages() call from send_error_msg() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 22 Jun 2020 17:01:10 -0400 Message-ID: <20200622210110.2144.55065.stgit@klimt.1015granger.net> In-Reply-To: <20200622205906.2144.43930.stgit@klimt.1015granger.net> References: <20200622205906.2144.43930.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Commit 4757d90b15d8 ("svcrdma: Report Write/Reply chunk overruns") made an effort to preserve I/O pages until RDMA Write completion. In a subsequent patch, I intend to de-duplicate the two functions that send ERR_CHUNK responses. Pull the save_io_pages() call out of svc_rdma_send_error_msg() to make it more like svc_rdma_send_error(). Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 38e7c3c8c4a9..2f88d01e8d27 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -806,8 +806,7 @@ static int svc_rdma_send_reply_msg(struct svcxprt_rdma *rdma, /* Given the client-provided Write and Reply chunks, the server was not * able to form a complete reply. Return an RDMA_ERROR message so the - * client can retire this RPC transaction. As above, the Send completion - * routine releases payload pages that were part of a previous RDMA Write. + * client can retire this RPC transaction. * * Remote Invalidation is skipped for simplicity. */ @@ -834,8 +833,6 @@ static int svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, *p = err_chunk; trace_svcrdma_err_chunk(*rdma_argp); - svc_rdma_save_io_pages(rqstp, ctxt); - ctxt->sc_send_wr.num_sge = 1; ctxt->sc_send_wr.opcode = IB_WR_SEND; ctxt->sc_sges[0].length = ctxt->sc_hdrbuf.len; @@ -930,6 +927,10 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) if (ret != -E2BIG && ret != -EINVAL) goto err1; + /* Send completion releases payload pages that were part + * of previously posted RDMA Writes. + */ + svc_rdma_save_io_pages(rqstp, sctxt); ret = svc_rdma_send_error_msg(rdma, sctxt, rqstp); if (ret < 0) goto err1; From patchwork Mon Jun 22 21:01:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11619127 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1F78C161F for ; Mon, 22 Jun 2020 21:01:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 07E042076A for ; Mon, 22 Jun 2020 21:01:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BUOGsxRa" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730291AbgFVVBU (ORCPT ); Mon, 22 Jun 2020 17:01:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33532 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728545AbgFVVBU (ORCPT ); Mon, 22 Jun 2020 17:01:20 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5A68C061573; Mon, 22 Jun 2020 14:01:19 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id l6so13273329qkc.6; Mon, 22 Jun 2020 14:01:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=m3s/UkyYnkq3YGDNG0dmiQUvsV/sibLQqZw3O2Q0D4M=; b=BUOGsxRaB9qHlUIiMx4499bcIAZkGJo/OoNNgyEjeaq60KXonah1ehvPY79hAPeEZ8 LoZIr4ipBdzxgHw0w8tNaBE9fK+lVGOittK2KamW2hTle+3mbY20Pl4Kb8WsTKCdoHoA xDkcgWD6YTCy+mQ2ZNAvmIM8LtrlLzYJVgyVGI086eWNG0jJTKI3kYrZ8weUPvo7zgeM iDQtDosPuJBTbSulREsJKWi+QzTObNi0tVe6+qKmss61Xu2OYYXCi45HEhTLqaswTAVc N27KbRiSJpYMLebOtB9XzHkqJoOq2yJu/45MWFQSfQYLG3KdJ0g/UBuAijt5OQpED+lU c4WQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=m3s/UkyYnkq3YGDNG0dmiQUvsV/sibLQqZw3O2Q0D4M=; b=UNJJr9C4UkS+pwgXYFAh2SQ9j+aQS3c4sbyoT+bvtbveiJk3xddrhNDBLhT5Lkr6tS 00WdBOlpqCFGSoyLsidh5C5VAxCrWWT6D6+bh6ZXGanJR574EOzWEDvg0mA+gFLMDItN x5oyPUzHrSmT2epkFF+wiucbdVRYuPrrlBGG1+FZUZFrJvLktIS3E/ss3rEvWkZAUteV ubx+L8kkr4UWsxpAPzltULXZHHapKF7VBIIqKg1hEezAjCr9B0MPgOvufJNQesTEOCAj RZXQ5qvV9LmC7QAoPZc2f3nhT+CjnJQTbAl2Go939velTtRDQo9Fm/OV+PZD+lGdyBza qvtA== X-Gm-Message-State: AOAM530AWy7ybMA+QrhmEjuBQpFLikHnYOrtExCrcFIvHY655QaEhZvR fJqMIz0bdUdIaGjLmVP3NUyveky/ X-Google-Smtp-Source: ABdhPJyS9bzKGidfjVzYtCk5BsA8p28xW0LL/PuFhUdgRSZFph3f0xcauUbysszFEEdfrcrGJMYnTQ== X-Received: by 2002:a37:9dc1:: with SMTP id g184mr3751846qke.135.1592859677050; Mon, 22 Jun 2020 14:01:17 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id m14sm15490111qke.99.2020.06.22.14.01.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Jun 2020 14:01:16 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 05ML1Fkv018828; Mon, 22 Jun 2020 21:01:15 GMT Subject: [PATCH v1 3/6] svcrdma: Add @rctxt parameter to svc_rdma_send_error() functions From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 22 Jun 2020 17:01:15 -0400 Message-ID: <20200622210115.2144.66473.stgit@klimt.1015granger.net> In-Reply-To: <20200622205906.2144.43930.stgit@klimt.1015granger.net> References: <20200622205906.2144.43930.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Another step towards making svc_rdma_send_error_msg() and svc_rdma_send_error() similar enough to eliminate one of them. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 9 +++++---- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 23 +++++++++++------------ 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index e426fedb9524..60d855116ae7 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -715,10 +715,11 @@ static void rdma_read_complete(struct svc_rqst *rqstp, } static void svc_rdma_send_error(struct svcxprt_rdma *xprt, - __be32 *rdma_argp, int status) + struct svc_rdma_recv_ctxt *rctxt, + int status) { + __be32 *p, *rdma_argp = rctxt->rc_recv_buf; struct svc_rdma_send_ctxt *ctxt; - __be32 *p; int ret; ctxt = svc_rdma_send_ctxt_get(xprt); @@ -900,13 +901,13 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) return 0; out_err: - svc_rdma_send_error(rdma_xprt, p, ret); + svc_rdma_send_error(rdma_xprt, ctxt, ret); svc_rdma_recv_ctxt_put(rdma_xprt, ctxt); return 0; out_postfail: if (ret == -EINVAL) - svc_rdma_send_error(rdma_xprt, p, ret); + svc_rdma_send_error(rdma_xprt, ctxt, ret); svc_rdma_recv_ctxt_put(rdma_xprt, ctxt); return ret; diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 2f88d01e8d27..47ada61411c3 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -811,18 +811,17 @@ static int svc_rdma_send_reply_msg(struct svcxprt_rdma *rdma, * Remote Invalidation is skipped for simplicity. */ static int svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, - struct svc_rdma_send_ctxt *ctxt, - struct svc_rqst *rqstp) + struct svc_rdma_send_ctxt *sctxt, + struct svc_rdma_recv_ctxt *rctxt) { - struct svc_rdma_recv_ctxt *rctxt = rqstp->rq_xprt_ctxt; __be32 *rdma_argp = rctxt->rc_recv_buf; __be32 *p; - rpcrdma_set_xdrlen(&ctxt->sc_hdrbuf, 0); - xdr_init_encode(&ctxt->sc_stream, &ctxt->sc_hdrbuf, ctxt->sc_xprt_buf, - NULL); + rpcrdma_set_xdrlen(&sctxt->sc_hdrbuf, 0); + xdr_init_encode(&sctxt->sc_stream, &sctxt->sc_hdrbuf, + sctxt->sc_xprt_buf, NULL); - p = xdr_reserve_space(&ctxt->sc_stream, RPCRDMA_HDRLEN_ERR); + p = xdr_reserve_space(&sctxt->sc_stream, RPCRDMA_HDRLEN_ERR); if (!p) return -ENOMSG; @@ -833,10 +832,10 @@ static int svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, *p = err_chunk; trace_svcrdma_err_chunk(*rdma_argp); - ctxt->sc_send_wr.num_sge = 1; - ctxt->sc_send_wr.opcode = IB_WR_SEND; - ctxt->sc_sges[0].length = ctxt->sc_hdrbuf.len; - return svc_rdma_send(rdma, &ctxt->sc_send_wr); + sctxt->sc_send_wr.num_sge = 1; + sctxt->sc_send_wr.opcode = IB_WR_SEND; + sctxt->sc_sges[0].length = sctxt->sc_hdrbuf.len; + return svc_rdma_send(rdma, &sctxt->sc_send_wr); } /** @@ -931,7 +930,7 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) * of previously posted RDMA Writes. */ svc_rdma_save_io_pages(rqstp, sctxt); - ret = svc_rdma_send_error_msg(rdma, sctxt, rqstp); + ret = svc_rdma_send_error_msg(rdma, sctxt, rctxt); if (ret < 0) goto err1; return 0; From patchwork Mon Jun 22 21:01:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11619131 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5D1D6161F for ; Mon, 22 Jun 2020 21:01:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 457B12076A for ; Mon, 22 Jun 2020 21:01:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PNTz1S2H" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730327AbgFVVBX (ORCPT ); Mon, 22 Jun 2020 17:01:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728545AbgFVVBX (ORCPT ); Mon, 22 Jun 2020 17:01:23 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 139E7C061573; Mon, 22 Jun 2020 14:01:23 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id w1so16913919qkw.5; Mon, 22 Jun 2020 14:01:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=O0yZJkvLrK6LFmPqzu5suqBMbRhXZtym0DCUxXeWW9I=; b=PNTz1S2H3JQktJ2KRCeXr9K5fB3rCUjsbhx4eQ7huiD9lc0XwkD8Okx14VINqigAFM 6pNrLS+AIEn1DYvQblEd2kRV10zylD6Hwj2yjaogrqsHjx+LE7k9+Z1U/dQkj/eE1qE3 xPYh9pwUGVPjDLXDGNeE/pD7fUqB4lTcLmw10WxarvLA4Fevjzv1PGC/mQXIw/BHlQbc Cc3kIgnl1Wzgf7Ec41DHzoA/XpNDl7CWQrt55YIcWyAIR4Ep5aBSVdlLZDvivyt/oWXv X2CQOdtRXi6uAbuI049Y3twLJiE92j+BAIfLW3g7fT3JHXNCCeG1Fab+I7hZdZWKkulg /6fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=O0yZJkvLrK6LFmPqzu5suqBMbRhXZtym0DCUxXeWW9I=; b=qgW3/LfXQ5jvPuteReHpeBykSJ3dWGQ35tHzMbf1sYVLtC+9bpfX8+b0tOXZX5nC8P dNzNGKgKIpgGh+r1SCEjBjXn5606ynSRDg9Vb3z9oPMbscOYHCESi33kdlHDgWpWOykK c1fH5zs6f/Z98FjTwESYRnJzGb5UEqsDX2NSZ9ktwivJQbTlqXcEtMM9K3001aec9Wat DF39Pf4lhfLkM8eLbYZJpMKZ3DKn5YnhYz2YxCziLOQK2f6WZskA0x9telXaoQFg2FqE KvkVsgJdpUw1Cceg7NYJsoUWYznYTSx/8MptAnIDOH9fE7g3ZzqdRdlvPjhaBYKniWH2 Mf2A== X-Gm-Message-State: AOAM530G98247fuQEhOIa/ozJRug0lNQjJ5aCnEUpY7g+486a0VwlVz6 yN+uVyP+4g+vjq1OTE3Tvv5noLRF X-Google-Smtp-Source: ABdhPJyn+8qESwT72eiJX/81svftMEA2Qzg55itQs5Kqf3aJQL4JhSmxsPJiyPBw4qw9cuf3oWkuMQ== X-Received: by 2002:a37:a89:: with SMTP id 131mr17228696qkk.92.1592859682150; Mon, 22 Jun 2020 14:01:22 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id p7sm9430593qki.61.2020.06.22.14.01.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Jun 2020 14:01:21 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 05ML1Lqg018832; Mon, 22 Jun 2020 21:01:21 GMT Subject: [PATCH v1 4/6] svcrdma: Add a @status parameter to svc_rdma_send_error_msg() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 22 Jun 2020 17:01:21 -0400 Message-ID: <20200622210121.2144.12399.stgit@klimt.1015granger.net> In-Reply-To: <20200622205906.2144.43930.stgit@klimt.1015granger.net> References: <20200622205906.2144.43930.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The common "send RDMA_ERR" function should be in svc_rdma_sendto.c, since that is where the other Send-related functions are located. So from here, I will beef up svc_rdma_send_error_msg() and deprecate svc_rdma_send_error(). A generic svc_rdma_send_error_msg() will need to handle both ERR_CHUNK and ERR_VERS. Copy that logic from svc_rdma_send_error() to svc_rdma_send_error_msg(). Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 47ada61411c3..73fe7a213169 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -812,7 +812,8 @@ static int svc_rdma_send_reply_msg(struct svcxprt_rdma *rdma, */ static int svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *sctxt, - struct svc_rdma_recv_ctxt *rctxt) + struct svc_rdma_recv_ctxt *rctxt, + int status) { __be32 *rdma_argp = rctxt->rc_recv_buf; __be32 *p; @@ -821,16 +822,35 @@ static int svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, xdr_init_encode(&sctxt->sc_stream, &sctxt->sc_hdrbuf, sctxt->sc_xprt_buf, NULL); - p = xdr_reserve_space(&sctxt->sc_stream, RPCRDMA_HDRLEN_ERR); + p = xdr_reserve_space(&sctxt->sc_stream, + rpcrdma_fixed_maxsz * sizeof(*p)); if (!p) return -ENOMSG; *p++ = *rdma_argp; *p++ = *(rdma_argp + 1); *p++ = rdma->sc_fc_credits; - *p++ = rdma_error; - *p = err_chunk; - trace_svcrdma_err_chunk(*rdma_argp); + *p = rdma_error; + + switch (status) { + case -EPROTONOSUPPORT: + p = xdr_reserve_space(&sctxt->sc_stream, 3 * sizeof(*p)); + if (!p) + return -ENOMSG; + + *p++ = err_vers; + *p++ = rpcrdma_version; + *p = rpcrdma_version; + trace_svcrdma_err_vers(*rdma_argp); + break; + default: + p = xdr_reserve_space(&sctxt->sc_stream, sizeof(*p)); + if (!p) + return -ENOMSG; + + *p = err_chunk; + trace_svcrdma_err_chunk(*rdma_argp); + } sctxt->sc_send_wr.num_sge = 1; sctxt->sc_send_wr.opcode = IB_WR_SEND; @@ -930,7 +950,7 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) * of previously posted RDMA Writes. */ svc_rdma_save_io_pages(rqstp, sctxt); - ret = svc_rdma_send_error_msg(rdma, sctxt, rctxt); + ret = svc_rdma_send_error_msg(rdma, sctxt, rctxt, ret); if (ret < 0) goto err1; return 0; From patchwork Mon Jun 22 21:01:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11619135 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0A2B392A for ; Mon, 22 Jun 2020 21:01:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E16992076A for ; Mon, 22 Jun 2020 21:01:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MCXPXJgB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730421AbgFVVB3 (ORCPT ); Mon, 22 Jun 2020 17:01:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728545AbgFVVB2 (ORCPT ); Mon, 22 Jun 2020 17:01:28 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90970C061573; Mon, 22 Jun 2020 14:01:28 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id q198so9075566qka.2; Mon, 22 Jun 2020 14:01:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=zQeMNl3XW1m0/k9561YNZvuwp+BUiHHSMSMoXCctjAY=; b=MCXPXJgByh5lkP9bP+Sl7X9c5bKxw8ZCGyIBeZuZVMQ1u+H8yC1gLqBrlAIuFPAJoF fmQwZ+6+ghNY6zpNKBTx4fC1SMe/kCsOu6ypE2D61Vt5buo4T4wzxjeVu0ciww03dDL6 eDAjboZfcqntXGwkpU4Is1AKa2CLoIT9LEp4JDhTwCgS624NcE6VR7bJHoWt6EwR2PMe vFYSM5hx6TT+MCe0CxzMlZHQol62VcC3/g5/fHvYSSgekYIwwgYyTYskNAJHwkqnqnzJ tz2vld3J+hPYvGVbFKkEmANuEI2Evk5deTtP7oSgyTPR3qqHaljuDBoNEO2EuMwkKql2 39Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=zQeMNl3XW1m0/k9561YNZvuwp+BUiHHSMSMoXCctjAY=; b=mBAz8iwEohfqG2FqwzqVq4wllqhAUch0lvciN00yg4PmecEJR2X0Y2ul3Sl7PTkDEj teKT+f6SNTRylLZZFqrakI2wDTFDBfy6JcFQjXFC0vKDWMGbhQjWv/BQxbEGEAxjFWLn OoxhmrdPMyGvKdzfrhypsRh3bmPsckVNDpCYd/YvGd3IVzIXQPIOTKe+KSpYxyZhalR+ lFqQpcxd4rMLa4yIwqpz099artSJoAE4CE5xeai4sETlFU+XGoqAeYMTJIfE8ySTUN4u peqrusnMXF47nzoOFkCXJ6oeIiRIfVW3QXpC+VldWO1z95u8CP+0qes4VQ6tYeFt5ebh LxaA== X-Gm-Message-State: AOAM532TwOPeZNbS2gFHF5eH05mOnUFireyrfnCDkiYAih5qXEstRXWx RJTMwIEJMan8clGkuo+SWk/gpvHl X-Google-Smtp-Source: ABdhPJx4yfZrv5VQDty7ZoG+0nUHLyjXmeJzt9LXPBySlmpY+7IKRwdTto9kSQcwrSziEulV+rWIUw== X-Received: by 2002:a37:9e92:: with SMTP id h140mr2665304qke.46.1592859687509; Mon, 22 Jun 2020 14:01:27 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id q24sm15361413qkj.103.2020.06.22.14.01.26 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Jun 2020 14:01:26 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 05ML1Q4w018835; Mon, 22 Jun 2020 21:01:26 GMT Subject: [PATCH v1 5/6] svcrdma: Eliminate return value for svc_rdma_send_error_msg() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 22 Jun 2020 17:01:26 -0400 Message-ID: <20200622210126.2144.96792.stgit@klimt.1015granger.net> In-Reply-To: <20200622205906.2144.43930.stgit@klimt.1015granger.net> References: <20200622205906.2144.43930.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Like svc_rdma_send_error(), have svc_rdma_send_error_msg() handle any error conditions internally, rather than duplicating that recovery logic at every call site. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 73fe7a213169..fb548b548c4b 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -810,10 +810,10 @@ static int svc_rdma_send_reply_msg(struct svcxprt_rdma *rdma, * * Remote Invalidation is skipped for simplicity. */ -static int svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, - struct svc_rdma_send_ctxt *sctxt, - struct svc_rdma_recv_ctxt *rctxt, - int status) +static void svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, + struct svc_rdma_send_ctxt *sctxt, + struct svc_rdma_recv_ctxt *rctxt, + int status) { __be32 *rdma_argp = rctxt->rc_recv_buf; __be32 *p; @@ -825,7 +825,7 @@ static int svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, p = xdr_reserve_space(&sctxt->sc_stream, rpcrdma_fixed_maxsz * sizeof(*p)); if (!p) - return -ENOMSG; + goto put_ctxt; *p++ = *rdma_argp; *p++ = *(rdma_argp + 1); @@ -836,7 +836,7 @@ static int svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, case -EPROTONOSUPPORT: p = xdr_reserve_space(&sctxt->sc_stream, 3 * sizeof(*p)); if (!p) - return -ENOMSG; + goto put_ctxt; *p++ = err_vers; *p++ = rpcrdma_version; @@ -846,7 +846,7 @@ static int svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, default: p = xdr_reserve_space(&sctxt->sc_stream, sizeof(*p)); if (!p) - return -ENOMSG; + goto put_ctxt; *p = err_chunk; trace_svcrdma_err_chunk(*rdma_argp); @@ -855,7 +855,12 @@ static int svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, sctxt->sc_send_wr.num_sge = 1; sctxt->sc_send_wr.opcode = IB_WR_SEND; sctxt->sc_sges[0].length = sctxt->sc_hdrbuf.len; - return svc_rdma_send(rdma, &sctxt->sc_send_wr); + if (svc_rdma_send(rdma, &sctxt->sc_send_wr)) + goto put_ctxt; + return; + +put_ctxt: + svc_rdma_send_ctxt_put(rdma, sctxt); } /** @@ -950,9 +955,7 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) * of previously posted RDMA Writes. */ svc_rdma_save_io_pages(rqstp, sctxt); - ret = svc_rdma_send_error_msg(rdma, sctxt, rctxt, ret); - if (ret < 0) - goto err1; + svc_rdma_send_error_msg(rdma, sctxt, rctxt, ret); return 0; err1: From patchwork Mon Jun 22 21:01:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11619139 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BDD76161F for ; Mon, 22 Jun 2020 21:01:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A505A20767 for ; Mon, 22 Jun 2020 21:01:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Y6k3e9KP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730431AbgFVVBe (ORCPT ); Mon, 22 Jun 2020 17:01:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33574 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728545AbgFVVBd (ORCPT ); Mon, 22 Jun 2020 17:01:33 -0400 Received: from mail-qv1-xf43.google.com (mail-qv1-xf43.google.com [IPv6:2607:f8b0:4864:20::f43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 91C74C061573; Mon, 22 Jun 2020 14:01:33 -0700 (PDT) Received: by mail-qv1-xf43.google.com with SMTP id g7so8650664qvx.11; Mon, 22 Jun 2020 14:01:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=qI6XUHEnEM+k66AQNlTZ1Xi4+elDUxoiA9HIMNVsMuE=; b=Y6k3e9KPmg24akmIW5czCumCB1ISgNNvHloxu+C9Usau3FBRM++pFnPmpePlaV/MlV bWiVbBqtGAr1WJ6KtaPjEarXK5HVllDAsizMFQgAOacUzMvncmphNPavuJX7FWsl14jl xDZ++HPm6sAftP/rhhIctrN80iWSwOqjnAyfaPHXTiDnpg6j74FfGwOEsZHyXDH75rSw 4ellPxgamro8vfSSSBNmu2MdjBhnEH0HRV2P1qbF2ooRpV1Y8SgUn01k8mgmkquENxLk /LTV6e51wtFmeK0PiobOcfUv2C0anV2u7z3KzRK8mSk4Wbqfy6c8K0Gpl99NQWawtpUZ 5T7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=qI6XUHEnEM+k66AQNlTZ1Xi4+elDUxoiA9HIMNVsMuE=; b=AB2gXuD65eFUSpJlo2M5xLPbFlT+kuoh1KdnEX4GjRnEXon17YONKnMPBdgy0dw/Ox 9I2niBZuWbtTUwmvPF9OuVdYHdtotCZ31FMITw0aYZnPyR1THcRbeMQBKFK2M7Cdzmbg gkKGCnkPvXWjq42NzrY7SkNWsxKexo9UpSOI5Yu48EURsWLPPXkrd9yxNaw9n8g8Jf6y o765AiRfpDGfwndAt7XEZwwsXWqUJCnYehErH5zEgC0YS9ZAILwk5EcebMVIIKL4N4/B KpUwChxboJVomt0Y6JFcNDxUOKxDsbofj/i+s3z2XGyUWbC15Qt6uWAb+6SK1IX/HZlK Z/IA== X-Gm-Message-State: AOAM531n4XMa3xef18W2UXtqKgKZjA0uuxF5iolixUjioIx8scMoWumM 3OQ2aymVYWLRgGVW7X5dCBEvtEIl X-Google-Smtp-Source: ABdhPJx05Az7wh6bBZlQuhcBT676A1sw10IaOFOfgreD62rdabOf4K5dsRKfI7pM8UWOEcEfL4iw2Q== X-Received: by 2002:a0c:b50a:: with SMTP id d10mr13287196qve.192.1592859692488; Mon, 22 Jun 2020 14:01:32 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id c27sm4854623qka.23.2020.06.22.14.01.31 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Jun 2020 14:01:32 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 05ML1V1L018838; Mon, 22 Jun 2020 21:01:31 GMT Subject: [PATCH v1 6/6] svcrdma: Make svc_rdma_send_error_msg() a global function From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 22 Jun 2020 17:01:31 -0400 Message-ID: <20200622210131.2144.914.stgit@klimt.1015granger.net> In-Reply-To: <20200622205906.2144.43930.stgit@klimt.1015granger.net> References: <20200622205906.2144.43930.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Prepare for svc_rdma_send_error_msg() to be invoked from another source file. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 4 ++++ net/sunrpc/xprtrdma/svc_rdma_sendto.c | 28 +++++++++++++++++++--------- 2 files changed, 23 insertions(+), 9 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 7ed82625dc0b..1579f7a14ab4 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -195,6 +195,10 @@ extern int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *sctxt, const struct svc_rdma_recv_ctxt *rctxt, struct xdr_buf *xdr); +extern void svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, + struct svc_rdma_send_ctxt *sctxt, + struct svc_rdma_recv_ctxt *rctxt, + int status); extern int svc_rdma_sendto(struct svc_rqst *); extern int svc_rdma_read_payload(struct svc_rqst *rqstp, unsigned int offset, unsigned int length); diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index fb548b548c4b..57041298fe4f 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -804,16 +804,25 @@ static int svc_rdma_send_reply_msg(struct svcxprt_rdma *rdma, return svc_rdma_send(rdma, &sctxt->sc_send_wr); } -/* Given the client-provided Write and Reply chunks, the server was not - * able to form a complete reply. Return an RDMA_ERROR message so the - * client can retire this RPC transaction. - * - * Remote Invalidation is skipped for simplicity. +/** + * svc_rdma_send_error_msg - Send an RPC/RDMA v1 error response + * @rdma: controlling transport context + * @sctxt: Send context for the response + * @rctxt: Receive context for incoming bad message + * @status: negative errno indicating error that occurred + * + * Given the client-provided Read, Write, and Reply chunks, the + * server was not able to parse the Call or form a complete Reply. + * Return an RDMA_ERROR message so the client can retire the RPC + * transaction. + * + * The caller does not have to release @sctxt. It is released by + * Send completion, or by this function on error. */ -static void svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, - struct svc_rdma_send_ctxt *sctxt, - struct svc_rdma_recv_ctxt *rctxt, - int status) +void svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, + struct svc_rdma_send_ctxt *sctxt, + struct svc_rdma_recv_ctxt *rctxt, + int status) { __be32 *rdma_argp = rctxt->rc_recv_buf; __be32 *p; @@ -852,6 +861,7 @@ static void svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, trace_svcrdma_err_chunk(*rdma_argp); } + /* Remote Invalidation is skipped for simplicity. */ sctxt->sc_send_wr.num_sge = 1; sctxt->sc_send_wr.opcode = IB_WR_SEND; sctxt->sc_sges[0].length = sctxt->sc_hdrbuf.len;