From patchwork Mon Jun 20 16:10:38 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 9188085 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5021D601C0 for ; Mon, 20 Jun 2016 16:14:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3EE0920410 for ; Mon, 20 Jun 2016 16:14:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3395E27A98; Mon, 20 Jun 2016 16:14:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D199220410 for ; Mon, 20 Jun 2016 16:14:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754100AbcFTQNd (ORCPT ); Mon, 20 Jun 2016 12:13:33 -0400 Received: from mail-it0-f65.google.com ([209.85.214.65]:36332 "EHLO mail-it0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754050AbcFTQNN (ORCPT ); Mon, 20 Jun 2016 12:13:13 -0400 Received: by mail-it0-f65.google.com with SMTP id h190so7373961ith.3; Mon, 20 Jun 2016 09:10:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=YNF5p0cpDKGyldPYTq1IayLgOKMNXbpsODt9QsshoIo=; b=ZzZ+nqoDxEhVnLnWEM7Whr+vU750VOBZh5NGNuX9iC+Sx3am5rXaupRLjAzIqnQgzM zzopF5WECz5O0GomPcSUTUiwMKWVwjgSIeiC/78ROccOy9sE+PJFoyxJ93Hk24yNqmEv G7rakrC6aGQe5RT/rTLaeJMqaLLuE4zuQX98X459o3m54Bqg+hJ+GNyYkM2I50JqYDsn HrTg8K702n7rM5DAU0VHIthvkpL7rPMVYU4FPGF7B8d+CONFmlIydaGkVgTJUWZl58j6 7sAiRVZTxh/6rCvSYxZkx2ealVRdbHZH12bEia7i9B0rXXLu/gIaBprY7+rr3gZV/ERQ L0Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=YNF5p0cpDKGyldPYTq1IayLgOKMNXbpsODt9QsshoIo=; b=m9j2F783cbufPt7/JO9w1i/g9K7kuYB+R/TWmrB8wxs2bFxNUcmX3F99Dr1pR9a3Ju GlNkQVKzIypri0sq0KAEDAeA2+oTdhwkjsLHJPtLcbOTvhH7HtIMlo7aaZguEZpXdBW1 Hx6SMSAUAgCBQkBlRowQULtX6jlbG7pHlqxp2JdY+KUuqQnBo5S6g0CM+Hl51e5KeZKa k5hkd5EKZN9fy0faDDz/shN3uuGn6zzMfpgxyljZeNV7CqBjiD6s/2r+2SZViwlg/JDU 9qsdR8eKTzh7qF23IwI8HNNW9yKAR4yS2ttPlYErZCwoB6LaRiRQkbl53LRtcXjZYvJG MEsA== X-Gm-Message-State: ALyK8tL1fRV1OXQpzpQ4RCGQP9IFgUqa84vCG3v7xqqVV0P0StKsVo0X+Z3bNMuGJqtmsg== X-Received: by 10.36.253.66 with SMTP id m63mr17958870ith.24.1466439039346; Mon, 20 Jun 2016 09:10:39 -0700 (PDT) Received: from manet.1015granger.net ([2604:8800:100:81fc:ec4:7aff:fe6c:1dce]) by smtp.gmail.com with ESMTPSA id f69sm27161225ioj.39.2016.06.20.09.10.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 Jun 2016 09:10:38 -0700 (PDT) Subject: [PATCH v3 15/25] xprtrdma: Release orphaned MRs immediately From: Chuck Lever To: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Mon, 20 Jun 2016 12:10:38 -0400 Message-ID: <20160620161038.10809.47684.stgit@manet.1015granger.net> In-Reply-To: <20160620155751.10809.22262.stgit@manet.1015granger.net> References: <20160620155751.10809.22262.stgit@manet.1015granger.net> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Instead of leaving orphaned MRs to be released when the transport is destroyed, release them immediately. The MR free list can now be replenished if it becomes exhausted. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/fmr_ops.c | 19 +++++++++++++------ net/sunrpc/xprtrdma/frwr_ops.c | 19 +++++++++++++------ 2 files changed, 26 insertions(+), 12 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/net/sunrpc/xprtrdma/fmr_ops.c b/net/sunrpc/xprtrdma/fmr_ops.c index 6802b8f..828ba46 100644 --- a/net/sunrpc/xprtrdma/fmr_ops.c +++ b/net/sunrpc/xprtrdma/fmr_ops.c @@ -135,15 +135,22 @@ fmr_op_recover_mr(struct rpcrdma_mw *mw) /* ORDER: then DMA unmap */ ib_dma_unmap_sg(r_xprt->rx_ia.ri_device, mw->mw_sg, mw->mw_nents, mw->mw_dir); - if (rc) { - pr_err("rpcrdma: FMR reset status %d, %p orphaned\n", - rc, mw); - r_xprt->rx_stats.mrs_orphaned++; - return; - } + if (rc) + goto out_release; rpcrdma_put_mw(r_xprt, mw); r_xprt->rx_stats.mrs_recovered++; + return; + +out_release: + pr_err("rpcrdma: FMR reset failed (%d), %p released\n", rc, mw); + r_xprt->rx_stats.mrs_orphaned++; + + spin_lock(&r_xprt->rx_buf.rb_mwlock); + list_del(&mw->mw_all); + spin_unlock(&r_xprt->rx_buf.rb_mwlock); + + fmr_op_release_mr(mw); } static int diff --git a/net/sunrpc/xprtrdma/frwr_ops.c b/net/sunrpc/xprtrdma/frwr_ops.c index e77776b..f4c06c8 100644 --- a/net/sunrpc/xprtrdma/frwr_ops.c +++ b/net/sunrpc/xprtrdma/frwr_ops.c @@ -177,15 +177,22 @@ frwr_op_recover_mr(struct rpcrdma_mw *mw) rc = __frwr_reset_mr(ia, mw); ib_dma_unmap_sg(ia->ri_device, mw->mw_sg, mw->mw_nents, mw->mw_dir); - if (rc) { - pr_err("rpcrdma: FRMR reset status %d, %p orphaned\n", - rc, mw); - r_xprt->rx_stats.mrs_orphaned++; - return; - } + if (rc) + goto out_release; rpcrdma_put_mw(r_xprt, mw); r_xprt->rx_stats.mrs_recovered++; + return; + +out_release: + pr_err("rpcrdma: FRMR reset failed %d, %p release\n", rc, mw); + r_xprt->rx_stats.mrs_orphaned++; + + spin_lock(&r_xprt->rx_buf.rb_mwlock); + list_del(&mw->mw_all); + spin_unlock(&r_xprt->rx_buf.rb_mwlock); + + frwr_op_release_mr(mw); } static int