From patchwork Mon Mar 19 18:23:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 10294011 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 72CEF602C2 for ; Mon, 19 Mar 2018 18:23:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5F16C29381 for ; Mon, 19 Mar 2018 18:23:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5333929461; Mon, 19 Mar 2018 18:23:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 93F232943A for ; Mon, 19 Mar 2018 18:23:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031411AbeCSSXZ (ORCPT ); Mon, 19 Mar 2018 14:23:25 -0400 Received: from mail-it0-f66.google.com ([209.85.214.66]:53624 "EHLO mail-it0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1031408AbeCSSXU (ORCPT ); Mon, 19 Mar 2018 14:23:20 -0400 Received: by mail-it0-f66.google.com with SMTP id b136-v6so5350555iti.3; Mon, 19 Mar 2018 11:23:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:user-agent:mime-version :content-transfer-encoding; bh=0YO15Chwr1X9eRevrxp7fORz+k+EypEPQUC83Nb14L8=; b=Xd96DZWbdDeqLqpdNI8vUJz1vGcn5/xa3DSv1zWcO0Oh2hdrHT4pwLDuVQa7mXDxKj UTl8f+oPBOrCD/9O3V5pRHU5Tv7cv10WJDp7y9fkXCUe4kSHm7Ky1/nSdwweGWw5BGny CYQNBf95U89OT221xNAFHEOeZYBxkvmXNV/CRwKSNVi8CEeB3/U2lrJYE5aJgKUKGSJS lABdY1vQ9cfnNd3i41f7p9xAGKB5hMcj31aS5gasiij6lacYvY1pbkJv0W7QfaFueB57 OrWgEx5Ik9pss/cCpFDZWV2j3wBnx/DzL46ZTLThknQWl4sZfw74wbnKacTYlxsKyFJd SPJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :user-agent:mime-version:content-transfer-encoding; bh=0YO15Chwr1X9eRevrxp7fORz+k+EypEPQUC83Nb14L8=; b=UbEuaPpETAbcmCP3e5RCskKflHgAAC5cki7DultIsE9jTeg2MnBStSaQrXOjHRuac+ zhtKphBpU/wj3OFmerJnHUu6+vCWik01b53HBQKiJafVGPIFqdR3VSMO6nRlTC12zhH4 fhUBBd1yzV3w5zs5n0PJY0XDpuDxWDPu4SQzPI1xZoq2CVY2D8sfUmPO6f4YU7FF+4fh gah1zpoVi75qtGdEnuCIXdFqC7yjXpGvrm5kD92X1l2Vfn9UlyOBr4gQXFhCbFD6KvHn 6xURNrw3USSxWlPghkdWJQb+1JxDQH5bYk4v0LRsYyCy1EdFpEYunNDD1/k5GFyBPPh6 K3Hw== X-Gm-Message-State: AElRT7Gqi2BkUATZ/r2oPaxKym2rPaXX+xvWSj6FBWcbWvFJYqr2/xA2 HK4j0uNfruIRDHiadX0XkZrNzQ== X-Google-Smtp-Source: AG47ELswEgTH6HsJgTIqWX8E2vh0THq1tpVpNPpKi5thcS1+zjNrw/6Tnjlv/1fUcATPu3AjcZwTnQ== X-Received: by 2002:a24:46cd:: with SMTP id j196-v6mr852775itb.8.1521483799623; Mon, 19 Mar 2018 11:23:19 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id e3-v6sm13500ite.21.2018.03.19.11.23.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Mar 2018 11:23:19 -0700 (PDT) Received: from manet.1015granger.net (manet.1015granger.net [192.168.1.51]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id w2JINGPV011315; Mon, 19 Mar 2018 18:23:17 GMT Subject: [PATCH] xprtrdma: Fix corner cases when handling device removal From: Chuck Lever To: anna.schumaker@netapp.com Cc: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org, stable@vger.kernel.org Date: Mon, 19 Mar 2018 14:23:16 -0400 Message-ID: <20180319182316.3060.7675.stgit@manet.1015granger.net> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Michal Kalderon has found some corner cases around device unload with active NFS mounts that I didn't have the imagination to test when xprtrdma device removal was added last year. - The ULP device removal handler is responsible for deallocating the PD. That wasn't clear to me initially, and my own testing suggested it was not necessary, but that is incorrect. - The transport destruction path can no longer assume that there is a valid ID. - When destroying a transport, ensure that ib_free_cq() is not invoked on a CQ that was already released. Reported-by: Michal Kalderon Fixes: bebd031866ca ("xprtrdma: Support unplugging an HCA from ...") Signed-off-by: Chuck Lever Cc: stable@vger.kernel.org --- net/sunrpc/xprtrdma/verbs.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index fe518b3..5e3afed 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -250,7 +250,6 @@ wait_for_completion(&ia->ri_remove_done); ia->ri_id = NULL; - ia->ri_pd = NULL; ia->ri_device = NULL; /* Return 1 to ensure the core destroys the id. */ return 1; @@ -448,7 +447,9 @@ ia->ri_id->qp = NULL; } ib_free_cq(ep->rep_attr.recv_cq); + ep->rep_attr.recv_cq = NULL; ib_free_cq(ep->rep_attr.send_cq); + ep->rep_attr.send_cq = NULL; /* The ULP is responsible for ensuring all DMA * mappings and MRs are gone. @@ -461,6 +462,8 @@ rpcrdma_dma_unmap_regbuf(req->rl_recvbuf); } rpcrdma_mrs_destroy(buf); + ib_dealloc_pd(ia->ri_pd); + ia->ri_pd = NULL; /* Allow waiters to continue */ complete(&ia->ri_remove_done); @@ -628,14 +631,16 @@ { cancel_delayed_work_sync(&ep->rep_connect_worker); - if (ia->ri_id->qp) { + if (ia->ri_id && ia->ri_id->qp) { rpcrdma_ep_disconnect(ep, ia); rdma_destroy_qp(ia->ri_id); ia->ri_id->qp = NULL; } - ib_free_cq(ep->rep_attr.recv_cq); - ib_free_cq(ep->rep_attr.send_cq); + if (ep->rep_attr.recv_cq) + ib_free_cq(ep->rep_attr.recv_cq); + if (ep->rep_attr.send_cq) + ib_free_cq(ep->rep_attr.send_cq); } /* Re-establish a connection after a device removal event.