From patchwork Fri Jul 14 15:13:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13313771 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05FF5C001DC for ; Fri, 14 Jul 2023 15:14:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235764AbjGNPOB (ORCPT ); Fri, 14 Jul 2023 11:14:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235611AbjGNPOA (ORCPT ); Fri, 14 Jul 2023 11:14:00 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C7F72708; Fri, 14 Jul 2023 08:13:59 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1197661D45; Fri, 14 Jul 2023 15:13:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0E576C433C8; Fri, 14 Jul 2023 15:13:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689347638; bh=AjVTiZGxGBSrlBE9CQQLTcrSS2ZLHDUVxtNcNgrghTs=; h=Subject:From:To:Cc:Date:From; b=Sk3NHpATkvI6p0L4o0QETvAL6ovz9R9/463BjESUzCTQVjm7cL/IQ7Qxkxh1jTKy/ NDqifVByRjcxwewBtJN8BxHF9AX21s99lyCZ/Ys8D0ArFR2VehhvFR1o0piMH3Dk30 KdCZUEI2r6lwx/6MyFJHHQBR5dQWbG7XarnrxAZ4eJlWiTjRssK5k5Yyg7/WQmfKY0 0h4wl8FKfW/7U7MXlUkEVMNJYWZNDW7LWmbVYthJKE6+hquDz96kpuQzNWJIyR8q4x KcYPUw6p/XAW9C7dLWuahxp+w6hc/81wBtEouuldDBOFPAQIHt6TW02ke5O0X0m3mh IJ+UN0ZC3xgeQ== Subject: [PATCH] xprtrdma: Remap Receive buffers after a reconnect From: Chuck Lever To: anna.schumaker@netapp.com Cc: Chuck Lever , linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org, tom@talpey.com, kolga@netapp.com Date: Fri, 14 Jul 2023 11:13:56 -0400 Message-ID: <168934757954.2781502.4228890662497418497.stgit@morisot.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chuck Lever On server-initiated disconnect, rpcrdma_xprt_disconnect() DMA-unmaps the transport's Receive buffers, but rpcrdma_post_recvs() neglected to remap them after a new connection had been established. The result is immediate failure of the new connection with the Receives flushing with LOCAL_PROT_ERR. Fixes: 671c450b6fe0 ("xprtrdma: Fix oops in Receive handler after device removal") Signed-off-by: Chuck Lever Reviewed-by: Tom Talpey Reported-by: Olga Kornievskaia --- net/sunrpc/xprtrdma/verbs.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) Hi Anna- Can you apply this for 6.5-rc ? diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index b098fde373ab..28c0771c4e8c 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -935,9 +935,6 @@ struct rpcrdma_rep *rpcrdma_rep_create(struct rpcrdma_xprt *r_xprt, if (!rep->rr_rdmabuf) goto out_free; - if (!rpcrdma_regbuf_dma_map(r_xprt, rep->rr_rdmabuf)) - goto out_free_regbuf; - rep->rr_cid.ci_completion_id = atomic_inc_return(&r_xprt->rx_ep->re_completion_ids); @@ -956,8 +953,6 @@ struct rpcrdma_rep *rpcrdma_rep_create(struct rpcrdma_xprt *r_xprt, spin_unlock(&buf->rb_lock); return rep; -out_free_regbuf: - rpcrdma_regbuf_free(rep->rr_rdmabuf); out_free: kfree(rep); out: @@ -1363,6 +1358,10 @@ void rpcrdma_post_recvs(struct rpcrdma_xprt *r_xprt, int needed, bool temp) rep = rpcrdma_rep_create(r_xprt, temp); if (!rep) break; + if (!rpcrdma_regbuf_dma_map(r_xprt, rep->rr_rdmabuf)) { + rpcrdma_rep_put(buf, rep); + break; + } rep->rr_cid.ci_queue_id = ep->re_attr.recv_cq->res.id; trace_xprtrdma_post_recv(rep);