From patchwork Tue Apr 11 17:23:18 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 9675801 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D3B1D600CB for ; Tue, 11 Apr 2017 17:23:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C214628579 for ; Tue, 11 Apr 2017 17:23:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B57E02858F; Tue, 11 Apr 2017 17:23:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2055C28579 for ; Tue, 11 Apr 2017 17:23:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752489AbdDKRX2 (ORCPT ); Tue, 11 Apr 2017 13:23:28 -0400 Received: from mail-io0-f196.google.com ([209.85.223.196]:35644 "EHLO mail-io0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751824AbdDKRXZ (ORCPT ); Tue, 11 Apr 2017 13:23:25 -0400 Received: by mail-io0-f196.google.com with SMTP id t68so1645247iof.2; Tue, 11 Apr 2017 10:23:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=SlJLwyZzyOaezF73lAe8sc9nHPEnjKRuhVSkgti67AM=; b=DBVXZb6dECCdfr3FIo4juP4FQ01CsgO8Dzmi0aui1WyMQ41rs5dhiSpb1z95kkk09z P475B0DBVpN54KA/P8B8JdqFQWE2baRMxa1OzW8FTCQIrZnAhbKD/mz4sVOzn5aPT2u7 hWcbvI5tvcHJpE59TScJiI8vC3htizNTnF3+jy8emnLBEDiSyF8rgHOvF344EYfDEpwZ E1rquOekfyyCNJmqzCiKBp6dDE+XmvIaiP8lipK6wxZCkxZojsgtrKNvGpC2mgVwiqYe LbDq4Vmice8e80r612TLfCZSSMr3fKBxS4B9SK2iNPSD8FcJy27kbOAmEjAWvi9gUWcS zQMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=SlJLwyZzyOaezF73lAe8sc9nHPEnjKRuhVSkgti67AM=; b=WIhOczF/azqaXvS8KE5Vc+AlBxmYUQY+qPypqcviRZq431SQ0odse2xOxqoVKCGUHf khmdJArLyz/To7CjRRow6TMmDBkKylo3ZAdCoYnPqXkwyPyxvSNunr9k3Em/zk7Iy+9a iQDcdUEFPEo6WE5M1s5Xhz7A9QSAqnJqNCQhWl3cIGk8N0jH75VkPat9vENms9YNPqNe BRfi/uZsQkkCs+C94uEKNwQ1/Ap3TEHrLwqIBBhS6dACJZTlQ8pSfchMh+2S9kc5X4Ia cVrx8laHPOYb4E+LrIY7cJx5zXBwUO/vP5BtRMzRzroCEzZS1JgGFywZgNyWymwjjqES QSXQ== X-Gm-Message-State: AN3rC/4s/6mKXkjSMzYjxoYX7WYLHZXheHfhJU5xIusI6e7jv7xif91+Q0rZEgaTTTB/Cg== X-Received: by 10.36.83.67 with SMTP id n64mr17917676itb.81.1491931399748; Tue, 11 Apr 2017 10:23:19 -0700 (PDT) Received: from manet.1015granger.net (c-68-46-169-226.hsd1.mi.comcast.net. [68.46.169.226]) by smtp.gmail.com with ESMTPSA id w133sm1172276itf.2.2017.04.11.10.23.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Apr 2017 10:23:19 -0700 (PDT) Subject: [PATCH v3 07/13] xprtrdma: Refactor rpcrdma_ep_connect From: Chuck Lever To: anna.schumaker@netapp.com Cc: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Tue, 11 Apr 2017 13:23:18 -0400 Message-ID: <20170411172318.11300.69958.stgit@manet.1015granger.net> In-Reply-To: <20170411171820.11300.55637.stgit@manet.1015granger.net> References: <20170411171820.11300.55637.stgit@manet.1015granger.net> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP I'm about to add another arm to if (ep->rep_connected != 0) It will be cleaner to use a switch statement here. We'll be looking for a couple of specific errnos, or "anything else," basically to sort out the difference between a normal reconnect and recovery from device removal. This is a refactoring change only. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/verbs.c | 109 +++++++++++++++++++++++++------------------ 1 file changed, 63 insertions(+), 46 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 938fd9e..6479ad3 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -711,6 +711,57 @@ static void rpcrdma_destroy_id(struct rdma_cm_id *id) ib_free_cq(ep->rep_attr.send_cq); } +static int +rpcrdma_ep_reconnect(struct rpcrdma_xprt *r_xprt, struct rpcrdma_ep *ep, + struct rpcrdma_ia *ia) +{ + struct sockaddr *sap = (struct sockaddr *)&r_xprt->rx_data.addr; + struct rdma_cm_id *id, *old; + int err, rc; + + dprintk("RPC: %s: reconnecting...\n", __func__); + + rpcrdma_ep_disconnect(ep, ia); + + rc = -EHOSTUNREACH; + id = rpcrdma_create_id(r_xprt, ia, sap); + if (IS_ERR(id)) + goto out; + + /* As long as the new ID points to the same device as the + * old ID, we can reuse the transport's existing PD and all + * previously allocated MRs. Also, the same device means + * the transport's previous DMA mappings are still valid. + * + * This is a sanity check only. There should be no way these + * point to two different devices here. + */ + old = id; + rc = -ENETUNREACH; + if (ia->ri_device != id->device) { + pr_err("rpcrdma: can't reconnect on different device!\n"); + goto out_destroy; + } + + err = rdma_create_qp(id, ia->ri_pd, &ep->rep_attr); + if (err) { + dprintk("RPC: %s: rdma_create_qp returned %d\n", + __func__, err); + goto out_destroy; + } + + /* Atomically replace the transport's ID and QP. */ + rc = 0; + old = ia->ri_id; + ia->ri_id = id; + rdma_destroy_qp(old); + +out_destroy: + rpcrdma_destroy_id(old); +out: + return rc; +} + /* * Connect unconnected endpoint. */ @@ -719,61 +770,25 @@ static void rpcrdma_destroy_id(struct rdma_cm_id *id) { struct rpcrdma_xprt *r_xprt = container_of(ia, struct rpcrdma_xprt, rx_ia); - struct rdma_cm_id *id, *old; - struct sockaddr *sap; unsigned int extras; - int rc = 0; + int rc; - if (ep->rep_connected != 0) { retry: - dprintk("RPC: %s: reconnecting...\n", __func__); - - rpcrdma_ep_disconnect(ep, ia); - - sap = (struct sockaddr *)&r_xprt->rx_data.addr; - id = rpcrdma_create_id(r_xprt, ia, sap); - if (IS_ERR(id)) { - rc = -EHOSTUNREACH; - goto out; - } - /* TEMP TEMP TEMP - fail if new device: - * Deregister/remarshal *all* requests! - * Close and recreate adapter, pd, etc! - * Re-determine all attributes still sane! - * More stuff I haven't thought of! - * Rrrgh! - */ - if (ia->ri_device != id->device) { - printk("RPC: %s: can't reconnect on " - "different device!\n", __func__); - rpcrdma_destroy_id(id); - rc = -ENETUNREACH; - goto out; - } - /* END TEMP */ - rc = rdma_create_qp(id, ia->ri_pd, &ep->rep_attr); - if (rc) { - dprintk("RPC: %s: rdma_create_qp failed %i\n", - __func__, rc); - rpcrdma_destroy_id(id); - rc = -ENETUNREACH; - goto out; - } - - old = ia->ri_id; - ia->ri_id = id; - - rdma_destroy_qp(old); - rpcrdma_destroy_id(old); - } else { + switch (ep->rep_connected) { + case 0: dprintk("RPC: %s: connecting...\n", __func__); rc = rdma_create_qp(ia->ri_id, ia->ri_pd, &ep->rep_attr); if (rc) { dprintk("RPC: %s: rdma_create_qp failed %i\n", __func__, rc); - /* do not update ep->rep_connected */ - return -ENETUNREACH; + rc = -ENETUNREACH; + goto out_noupdate; } + break; + default: + rc = rpcrdma_ep_reconnect(r_xprt, ep, ia); + if (rc) + goto out; } ep->rep_connected = 0; @@ -801,6 +816,8 @@ static void rpcrdma_destroy_id(struct rdma_cm_id *id) out: if (rc) ep->rep_connected = rc; + +out_noupdate: return rc; }