From patchwork Fri Jan 3 16:56:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11317083 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A82DF930 for ; Fri, 3 Jan 2020 16:56:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8734521734 for ; Fri, 3 Jan 2020 16:56:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lZUVS9SH" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728021AbgACQ4a (ORCPT ); Fri, 3 Jan 2020 11:56:30 -0500 Received: from mail-yw1-f67.google.com ([209.85.161.67]:37061 "EHLO mail-yw1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727912AbgACQ43 (ORCPT ); Fri, 3 Jan 2020 11:56:29 -0500 Received: by mail-yw1-f67.google.com with SMTP id z7so18752752ywd.4; Fri, 03 Jan 2020 08:56:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=BnRDLTtdlUh8mqBrpT4Rt9IaTfUaFir1pql1uwAgVUw=; b=lZUVS9SHDR73AWg2s1Ne7zXRaAlgMiXaH46nB2d8IMC2s5B7UwummjEm7ISk3Y94Ae a8l5XsRuYp5bH3aJPj78Gh/EaDHaJ3Nzm3C7BHfPwWJ0DNDYPLZS8Zqk6hDff+M8UYRO LMC/0z5Na2KY0to27TnnTG8mtkHdd7ByOwf0iVPVZFp6YkZAa7CrF1qiWHBiKAetu132 u6tHnBWctRtWzw+9LP2lALNVSLsf9Z8yLb7ziiqgwI3Ks7ZKqRaZU3XvCh0jd+4AuGU+ KXPRTSrnwvH4ME2nEz0SQbg0ZxdFynS47Hsn/pVebckpeDdhWs8H+1ullOi7P8fqsN+f TmNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=BnRDLTtdlUh8mqBrpT4Rt9IaTfUaFir1pql1uwAgVUw=; b=iChRT7tPrLZ/1JpaXRiWoM6lED75dvllA73Ht4Q1F512dWK/r4kafVELmSC2MITQuZ CKn4ttyE77cJ7n91cYIA900ofGl8dGwzxi3pRSz4H/lOLlEiri0fO3m+KCK8vF1aZN4B dSuQCArOGbuM7Mr/gYXAYSZeyMS55R3jX1hVy9Gvn/kjCmep/LGfAORYvbp9I/H8aryk L74Z/kJkUTTD9bUTNucF+mU/twQIktSBIU9sZNFu1oGj1rx2SZhDG0VIf+9++ychPRHC In/Z/ZujFQXc5GCIgNfaIXX14Ss3jmGiDsnfS96sMIS3mUdJ7SAMM1ph3anc7eVpna4W wC7w== X-Gm-Message-State: APjAAAX9+BL7I7hHXaJVOu59mdfdRQOeI9kIpcGNVXAQcsWgXu/65ONM bIzSAI9i2ObPLeBGUEJD2RR0ivAG X-Google-Smtp-Source: APXvYqzXr9NrpPBA7CmCZD2Mv2mb1ZoxjVo8hmcQvOtYLyYRvixiXRGJuzUEm5wO5zItgi2UfmkvTg== X-Received: by 2002:a0d:fb46:: with SMTP id l67mr34564580ywf.38.1578070588429; Fri, 03 Jan 2020 08:56:28 -0800 (PST) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id f138sm24511068ywb.99.2020.01.03.08.56.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Jan 2020 08:56:28 -0800 (PST) Received: from morisot.1015granger.net (morisot.1015granger.net [192.168.1.67]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 003GuR4m016383; Fri, 3 Jan 2020 16:56:27 GMT Subject: [PATCH v1 1/9] xprtrdma: Eliminate ri_max_send_sges From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 03 Jan 2020 11:56:27 -0500 Message-ID: <157807058713.4606.5667131330030980818.stgit@morisot.1015granger.net> In-Reply-To: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> References: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> User-Agent: StGit/0.19 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean-up. The max_send_sge value also happens to be stored in ep->rep_attr. Let's keep just a single copy. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/frwr_ops.c | 10 ++++++++++ net/sunrpc/xprtrdma/rpc_rdma.c | 2 +- net/sunrpc/xprtrdma/verbs.c | 17 +++-------------- net/sunrpc/xprtrdma/xprt_rdma.h | 1 - 4 files changed, 14 insertions(+), 16 deletions(-) diff --git a/net/sunrpc/xprtrdma/frwr_ops.c b/net/sunrpc/xprtrdma/frwr_ops.c index 50025408dc2a..24b8b75d0d49 100644 --- a/net/sunrpc/xprtrdma/frwr_ops.c +++ b/net/sunrpc/xprtrdma/frwr_ops.c @@ -183,6 +183,16 @@ int frwr_open(struct rpcrdma_ia *ia, struct rpcrdma_ep *ep) { struct ib_device_attr *attrs = &ia->ri_id->device->attrs; int max_qp_wr, depth, delta; + unsigned int max_sge; + + max_sge = min_t(unsigned int, attrs->max_send_sge, + RPCRDMA_MAX_SEND_SGES); + if (max_sge < RPCRDMA_MIN_SEND_SGES) { + pr_err("rpcrdma: HCA provides only %u send SGEs\n", max_sge); + return -ENOMEM; + } + ep->rep_attr.cap.max_send_sge = max_sge; + ep->rep_attr.cap.max_recv_sge = 1; ia->ri_mrtype = IB_MR_TYPE_MEM_REG; if (attrs->device_cap_flags & IB_DEVICE_SG_GAPS_REG) diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c index aec3beb93b25..af917228d245 100644 --- a/net/sunrpc/xprtrdma/rpc_rdma.c +++ b/net/sunrpc/xprtrdma/rpc_rdma.c @@ -145,7 +145,7 @@ static bool rpcrdma_args_inline(struct rpcrdma_xprt *r_xprt, remaining -= min_t(unsigned int, PAGE_SIZE - offset, remaining); offset = 0; - if (++count > r_xprt->rx_ia.ri_max_send_sges) + if (++count > r_xprt->rx_ep.rep_attr.cap.max_send_sge) return false; } } diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 4d7d7555569c..0ad81d3c735e 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -469,21 +469,12 @@ int rpcrdma_ep_create(struct rpcrdma_xprt *r_xprt) struct rpcrdma_ia *ia = &r_xprt->rx_ia; struct rpcrdma_connect_private *pmsg = &ep->rep_cm_private; struct ib_cq *sendcq, *recvcq; - unsigned int max_sge; int rc; ep->rep_max_requests = xprt_rdma_slot_table_entries; ep->rep_inline_send = xprt_rdma_max_inline_write; ep->rep_inline_recv = xprt_rdma_max_inline_read; - max_sge = min_t(unsigned int, ia->ri_id->device->attrs.max_send_sge, - RPCRDMA_MAX_SEND_SGES); - if (max_sge < RPCRDMA_MIN_SEND_SGES) { - pr_warn("rpcrdma: HCA provides only %d send SGEs\n", max_sge); - return -ENOMEM; - } - ia->ri_max_send_sges = max_sge; - rc = frwr_open(ia, ep); if (rc) return rc; @@ -491,8 +482,6 @@ int rpcrdma_ep_create(struct rpcrdma_xprt *r_xprt) ep->rep_attr.event_handler = rpcrdma_qp_event_handler; ep->rep_attr.qp_context = ep; ep->rep_attr.srq = NULL; - ep->rep_attr.cap.max_send_sge = max_sge; - ep->rep_attr.cap.max_recv_sge = 1; ep->rep_attr.cap.max_inline_data = 0; ep->rep_attr.sq_sig_type = IB_SIGNAL_REQ_WR; ep->rep_attr.qp_type = IB_QPT_RC; @@ -795,11 +784,11 @@ static void rpcrdma_sendctxs_destroy(struct rpcrdma_buffer *buf) kfree(buf->rb_sc_ctxs); } -static struct rpcrdma_sendctx *rpcrdma_sendctx_create(struct rpcrdma_ia *ia) +static struct rpcrdma_sendctx *rpcrdma_sendctx_create(struct rpcrdma_ep *ep) { struct rpcrdma_sendctx *sc; - sc = kzalloc(struct_size(sc, sc_sges, ia->ri_max_send_sges), + sc = kzalloc(struct_size(sc, sc_sges, ep->rep_attr.cap.max_send_sge), GFP_KERNEL); if (!sc) return NULL; @@ -827,7 +816,7 @@ static int rpcrdma_sendctxs_create(struct rpcrdma_xprt *r_xprt) buf->rb_sc_last = i - 1; for (i = 0; i <= buf->rb_sc_last; i++) { - sc = rpcrdma_sendctx_create(&r_xprt->rx_ia); + sc = rpcrdma_sendctx_create(&r_xprt->rx_ep); if (!sc) return -ENOMEM; diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h index d796d68609ed..7655a99fd559 100644 --- a/net/sunrpc/xprtrdma/xprt_rdma.h +++ b/net/sunrpc/xprtrdma/xprt_rdma.h @@ -73,7 +73,6 @@ struct rpcrdma_ia { int ri_async_rc; unsigned int ri_max_segs; unsigned int ri_max_frwr_depth; - unsigned int ri_max_send_sges; bool ri_implicit_roundup; enum ib_mr_type ri_mrtype; unsigned long ri_flags; From patchwork Fri Jan 3 16:56:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11317087 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 12E2A109A for ; Fri, 3 Jan 2020 16:56:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DC8C321734 for ; Fri, 3 Jan 2020 16:56:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jtb1Df4J" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728041AbgACQ4f (ORCPT ); Fri, 3 Jan 2020 11:56:35 -0500 Received: from mail-yb1-f195.google.com ([209.85.219.195]:33247 "EHLO mail-yb1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727912AbgACQ4f (ORCPT ); Fri, 3 Jan 2020 11:56:35 -0500 Received: by mail-yb1-f195.google.com with SMTP id n66so18855508ybg.0; Fri, 03 Jan 2020 08:56:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=zwhon8II32DWrYSmdSH74tObIQ31P4JpVuINWcxt7H8=; b=jtb1Df4JQrV4EJZ3K3gElpQ3Oukw9PwmhSE6nu8toBWy8F3UJx1JdT5Hzb9PXDq1iB u51V5+KE9zT9tjUnYi+Q/4aelMwPzGQNyCXLfnvNvu/qFzIw+npZzs1vIfaVGZ/wUf+D q2RRj2SDDvIzRNraEs0A1Ov+sNE5qvlCeuz2mbfmQsQPIu0p+7am+7XIB1noMrKgP0Ol +UF/7Z75EEf+LV5LRNbAvh1kaSo6pKrwikVYZ9n1g7o42ulcQXCMAGNdYkqF0DyRHiP5 cDJkIdJg8kk7jiM5xU2+Yvf04yePO5FwWsJtdYRtHql8kuSyaQlWjOv82o3f7aGl1WvA 6xNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=zwhon8II32DWrYSmdSH74tObIQ31P4JpVuINWcxt7H8=; b=NHE8ZvxtgAJl2de7q1phOcl6/zchnKKGvWoWDSO6M7uKk1tCfYMVvMUvu++guM0K97 mPBVFuf5mdgwD7wUhD1xiMmoay4ZmmhRKjAZ0HQ7aRBQSSiLx7P0y9LvWl3wu13+r4oc b0yLrHXv7r27QBK0XEMt8VsKZiAiDGE99CpQLtuXuY5IcMSYNClDo9j65SqQEfEnrCIj ib2GoUoEWM2c2gjMrSl+GgxDrCFFwMwFP6to6HDbVOH73cyfem/27TMx2MZrMxi4jyYl ydqGGaEBqCa2e4osCtYE6zCUjbYU83qctp80sIrRr/XSGq/oZNkIrFc5aBXUf1ovta6V Qtmg== X-Gm-Message-State: APjAAAXtQuMcho9FdrqTr3QjS0mstFL+8PoJhhoHqrxPefF1hW6zTFe5 ZXjyDSxUfP5yukaj6M4ABcBT1Ku1 X-Google-Smtp-Source: APXvYqzTR8fw6RM84HUegSojixZGghJ0We77Djifqlc54DfCCp/c3s8kBl19mzHlrBGNQyNmUKw8XA== X-Received: by 2002:a25:99c2:: with SMTP id q2mr51083887ybo.365.1578070593771; Fri, 03 Jan 2020 08:56:33 -0800 (PST) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id g29sm23955007ywk.31.2020.01.03.08.56.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Jan 2020 08:56:33 -0800 (PST) Received: from morisot.1015granger.net (morisot.1015granger.net [192.168.1.67]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 003GuW5w016386; Fri, 3 Jan 2020 16:56:32 GMT Subject: [PATCH v1 2/9] xprtrdma: Make sendctx queue lifetime the same as connection lifetime From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 03 Jan 2020 11:56:32 -0500 Message-ID: <157807059245.4606.283126960136817326.stgit@morisot.1015granger.net> In-Reply-To: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> References: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> User-Agent: StGit/0.19 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org The size of the sendctx queue depends on the value stored in ia->ri_max_send_sges. This value is determined by querying the underlying device. Eventually, rpcrdma_ia_open() and rpcrdma_ep_create() will be called in the connect worker rather than at transport set-up time. The underlying device will not have been chosen device set-up time. The sendctx queue will thus have to be created after the underlying device has been chosen via address and route resolution; in other words, in the connect worker. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 12 ++++++++---- net/sunrpc/xprtrdma/verbs.c | 22 +++++++++++++++------- 2 files changed, 23 insertions(+), 11 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 28dd498e683e..6d9075507bd6 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -698,6 +698,7 @@ TRACE_EVENT(xprtrdma_post_send, TP_STRUCT__entry( __field(const void *, req) + __field(const void *, sc) __field(unsigned int, task_id) __field(unsigned int, client_id) __field(int, num_sge) @@ -712,14 +713,15 @@ TRACE_EVENT(xprtrdma_post_send, __entry->client_id = rqst->rq_task->tk_client ? rqst->rq_task->tk_client->cl_clid : -1; __entry->req = req; + __entry->sc = req->rl_sendctx; __entry->num_sge = req->rl_wr.num_sge; __entry->signaled = req->rl_wr.send_flags & IB_SEND_SIGNALED; __entry->status = status; ), - TP_printk("task:%u@%u req=%p (%d SGE%s) %sstatus=%d", + TP_printk("task:%u@%u req=%p sc=%p (%d SGE%s) %sstatus=%d", __entry->task_id, __entry->client_id, - __entry->req, __entry->num_sge, + __entry->req, __entry->sc, __entry->num_sge, (__entry->num_sge == 1 ? "" : "s"), (__entry->signaled ? "signaled " : ""), __entry->status @@ -818,6 +820,7 @@ TRACE_EVENT(xprtrdma_wc_send, TP_STRUCT__entry( __field(const void *, req) + __field(const void *, sc) __field(unsigned int, unmap_count) __field(unsigned int, status) __field(unsigned int, vendor_err) @@ -825,13 +828,14 @@ TRACE_EVENT(xprtrdma_wc_send, TP_fast_assign( __entry->req = sc->sc_req; + __entry->sc = sc; __entry->unmap_count = sc->sc_unmap_count; __entry->status = wc->status; __entry->vendor_err = __entry->status ? wc->vendor_err : 0; ), - TP_printk("req=%p, unmapped %u pages: %s (%u/0x%x)", - __entry->req, __entry->unmap_count, + TP_printk("req=%p sc=%p unmapped=%u: %s (%u/0x%x)", + __entry->req, __entry->sc, __entry->unmap_count, rdma_show_wc_status(__entry->status), __entry->status, __entry->vendor_err ) diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 0ad81d3c735e..3cc6d19f7f3a 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -74,6 +74,8 @@ /* * internal functions */ +static int rpcrdma_sendctxs_create(struct rpcrdma_xprt *r_xprt); +static void rpcrdma_sendctxs_destroy(struct rpcrdma_xprt *r_xprt); static void rpcrdma_sendctx_put_locked(struct rpcrdma_xprt *r_xprt, struct rpcrdma_sendctx *sc); static void rpcrdma_reqs_reset(struct rpcrdma_xprt *r_xprt); @@ -427,6 +429,7 @@ rpcrdma_ia_remove(struct rpcrdma_ia *ia) rpcrdma_regbuf_dma_unmap(req->rl_recvbuf); } rpcrdma_mrs_destroy(r_xprt); + rpcrdma_sendctxs_destroy(r_xprt); ib_dealloc_pd(ia->ri_pd); ia->ri_pd = NULL; @@ -704,6 +707,10 @@ rpcrdma_ep_connect(struct rpcrdma_ep *ep, struct rpcrdma_ia *ia) rpcrdma_reset_cwnd(r_xprt); rpcrdma_post_recvs(r_xprt, true); + rc = rpcrdma_sendctxs_create(r_xprt); + if (rc) + goto out; + rc = rdma_connect(ia->ri_id, &ep->rep_remote_cma); if (rc) goto out; @@ -756,6 +763,7 @@ rpcrdma_ep_disconnect(struct rpcrdma_ep *ep, struct rpcrdma_ia *ia) rpcrdma_xprt_drain(r_xprt); rpcrdma_reqs_reset(r_xprt); rpcrdma_mrs_destroy(r_xprt); + rpcrdma_sendctxs_destroy(r_xprt); } /* Fixed-size circular FIFO queue. This implementation is wait-free and @@ -775,13 +783,17 @@ rpcrdma_ep_disconnect(struct rpcrdma_ep *ep, struct rpcrdma_ia *ia) * queue activity, and rpcrdma_xprt_drain has flushed all remaining * Send requests. */ -static void rpcrdma_sendctxs_destroy(struct rpcrdma_buffer *buf) +static void rpcrdma_sendctxs_destroy(struct rpcrdma_xprt *r_xprt) { + struct rpcrdma_buffer *buf = &r_xprt->rx_buf; unsigned long i; + if (!buf->rb_sc_ctxs) + return; for (i = 0; i <= buf->rb_sc_last; i++) kfree(buf->rb_sc_ctxs[i]); kfree(buf->rb_sc_ctxs); + buf->rb_sc_ctxs = NULL; } static struct rpcrdma_sendctx *rpcrdma_sendctx_create(struct rpcrdma_ep *ep) @@ -809,7 +821,6 @@ static int rpcrdma_sendctxs_create(struct rpcrdma_xprt *r_xprt) * Sends are posted. */ i = buf->rb_max_requests + RPCRDMA_MAX_BC_REQUESTS; - dprintk("RPC: %s: allocating %lu send_ctxs\n", __func__, i); buf->rb_sc_ctxs = kcalloc(i, sizeof(sc), GFP_KERNEL); if (!buf->rb_sc_ctxs) return -ENOMEM; @@ -823,6 +834,8 @@ static int rpcrdma_sendctxs_create(struct rpcrdma_xprt *r_xprt) buf->rb_sc_ctxs[i] = sc; } + buf->rb_sc_head = 0; + buf->rb_sc_tail = 0; return 0; } @@ -1165,10 +1178,6 @@ int rpcrdma_buffer_create(struct rpcrdma_xprt *r_xprt) init_llist_head(&buf->rb_free_reps); - rc = rpcrdma_sendctxs_create(r_xprt); - if (rc) - goto out; - return 0; out: rpcrdma_buffer_destroy(buf); @@ -1244,7 +1253,6 @@ static void rpcrdma_mrs_destroy(struct rpcrdma_xprt *r_xprt) void rpcrdma_buffer_destroy(struct rpcrdma_buffer *buf) { - rpcrdma_sendctxs_destroy(buf); rpcrdma_reps_destroy(buf); while (!list_empty(&buf->rb_send_bufs)) { From patchwork Fri Jan 3 16:56:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11317091 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4DCBE1580 for ; Fri, 3 Jan 2020 16:56:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1FE9A21734 for ; Fri, 3 Jan 2020 16:56:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="t7DaGFDv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728055AbgACQ4k (ORCPT ); Fri, 3 Jan 2020 11:56:40 -0500 Received: from mail-yw1-f67.google.com ([209.85.161.67]:46370 "EHLO mail-yw1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727912AbgACQ4k (ORCPT ); Fri, 3 Jan 2020 11:56:40 -0500 Received: by mail-yw1-f67.google.com with SMTP id u139so18725351ywf.13; Fri, 03 Jan 2020 08:56:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=VJgu2/3umFtGBIl1WJIjq2UBGgp/XxZOmw1CIFWppGc=; b=t7DaGFDvxdBjTGvPDBoUC4ENy4RIDdxepx/SkbWMRBvE7obWBV1VupAg5SBmTKBSHM BnPMMI23qQDk6N1hlbokg3iUwIWsdP97SeVp+Suyzy9K3EyTwTQV4Mvc09mMYK7JHCmd cfOplPSES1lfP/VxYDzYAlmGx17OJx/S3uUZSdlpfcfZmJTm2WkITC9LYPcwQ4RYhj+C B7o8Q9AgTLM6zx4T78/rqWZfBKqvAGc5RNf6jQI/eXjnQdUuj7Skh7Oc0/MiW6k5LIHg UMLwDg6VgJMhBMJYotHdUr3mWb/hwlMMHKDRYogEQ2p2ONqkxN7OqDkA0I5XyFeeNBcT zwlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=VJgu2/3umFtGBIl1WJIjq2UBGgp/XxZOmw1CIFWppGc=; b=q+SCt/5+8oCayGGjF1X0ZVDLCmzM7XmJBLo6rafDo45WUikvoHtirmOAm1E2+kf5Ww DdDNBNJAVEetDtXca1DWwgK0+INKCOiG6+gp+11rvKXylqDuOFjVCVScTziJc+EQQdsb u/O9zUpBlTy3htXli004Sqp6P0xkJfaSatC9a+J7JXII8eBr6K4+9qiBmdfwo01OCOVP 9JWOMBQecU0PcuXUUpCRiMUmaDMduSWUrxD7SGQ2SrGbqV+B6nVUxb8RYyPqAwPbiep7 AP0RKSvAB+lxtP0WwQkQUuPaF37ZFpiVvpvzjWIVcxSnzfWckBHuCFQkvXrtuN0DEGYk Smrw== X-Gm-Message-State: APjAAAV5CKcwo6g8jqj+kiXF7iAjhEnpPbhh+5WYWotZZZyNui5CM5pH Wz/+J6K859ac/7xpovkwqKNdkI8w X-Google-Smtp-Source: APXvYqwagpEIm/aQ+x6AlnDIdaOcH3OgB+WmcW8S+8EkAbpACNYo8BndqcAGJYAql26IalwGYu4ncQ== X-Received: by 2002:a81:3b45:: with SMTP id i66mr68380387ywa.289.1578070599018; Fri, 03 Jan 2020 08:56:39 -0800 (PST) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id j11sm23415484ywg.37.2020.01.03.08.56.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Jan 2020 08:56:38 -0800 (PST) Received: from morisot.1015granger.net (morisot.1015granger.net [192.168.1.67]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 003GubWk016389; Fri, 3 Jan 2020 16:56:37 GMT Subject: [PATCH v1 3/9] xprtrdma: Refactor initialization of ep->rep_max_requests From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 03 Jan 2020 11:56:37 -0500 Message-ID: <157807059776.4606.16811783083488098904.stgit@morisot.1015granger.net> In-Reply-To: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> References: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> User-Agent: StGit/0.19 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: there is no need to keep two copies of the same value. Also, in subsequent patches, rpcrdma_ep_create() will be called in the connect worker rather than at set-up time. Minor fix: Initialize the transport's sendctx to the value based on the capabilities of the underlying device, not the maximum setting. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/rpc_rdma.c | 6 +++--- net/sunrpc/xprtrdma/transport.c | 3 ++- net/sunrpc/xprtrdma/verbs.c | 8 ++++---- net/sunrpc/xprtrdma/xprt_rdma.h | 5 ++--- 4 files changed, 11 insertions(+), 11 deletions(-) diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c index af917228d245..520323ddc930 100644 --- a/net/sunrpc/xprtrdma/rpc_rdma.c +++ b/net/sunrpc/xprtrdma/rpc_rdma.c @@ -909,7 +909,7 @@ rpcrdma_marshal_req(struct rpcrdma_xprt *r_xprt, struct rpc_rqst *rqst) goto out_err; *p++ = rqst->rq_xid; *p++ = rpcrdma_version; - *p++ = cpu_to_be32(r_xprt->rx_buf.rb_max_requests); + *p++ = r_xprt->rx_buf.rb_max_requests; /* When the ULP employs a GSS flavor that guarantees integrity * or privacy, direct data placement of individual data items @@ -1480,8 +1480,8 @@ void rpcrdma_reply_handler(struct rpcrdma_rep *rep) if (credits == 0) credits = 1; /* don't deadlock */ - else if (credits > buf->rb_max_requests) - credits = buf->rb_max_requests; + else if (credits > r_xprt->rx_ep.rep_max_requests) + credits = r_xprt->rx_ep.rep_max_requests; if (buf->rb_credits != credits) rpcrdma_update_cwnd(r_xprt, credits); rpcrdma_post_recvs(r_xprt, false); diff --git a/net/sunrpc/xprtrdma/transport.c b/net/sunrpc/xprtrdma/transport.c index 7395eb2cfdeb..f868a75057ad 100644 --- a/net/sunrpc/xprtrdma/transport.c +++ b/net/sunrpc/xprtrdma/transport.c @@ -316,7 +316,8 @@ xprt_setup_rdma(struct xprt_create *args) if (args->addrlen > sizeof(xprt->addr)) return ERR_PTR(-EBADF); - xprt = xprt_alloc(args->net, sizeof(struct rpcrdma_xprt), 0, 0); + xprt = xprt_alloc(args->net, sizeof(struct rpcrdma_xprt), 0, + xprt_rdma_slot_table_entries); if (!xprt) return ERR_PTR(-ENOMEM); diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 3cc6d19f7f3a..4f9595b72888 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -474,13 +474,14 @@ int rpcrdma_ep_create(struct rpcrdma_xprt *r_xprt) struct ib_cq *sendcq, *recvcq; int rc; - ep->rep_max_requests = xprt_rdma_slot_table_entries; + ep->rep_max_requests = r_xprt->rx_xprt.max_reqs; ep->rep_inline_send = xprt_rdma_max_inline_write; ep->rep_inline_recv = xprt_rdma_max_inline_read; rc = frwr_open(ia, ep); if (rc) return rc; + r_xprt->rx_buf.rb_max_requests = cpu_to_be32(ep->rep_max_requests); ep->rep_attr.event_handler = rpcrdma_qp_event_handler; ep->rep_attr.qp_context = ep; @@ -820,7 +821,7 @@ static int rpcrdma_sendctxs_create(struct rpcrdma_xprt *r_xprt) * the ->send_request call to fail temporarily before too many * Sends are posted. */ - i = buf->rb_max_requests + RPCRDMA_MAX_BC_REQUESTS; + i = r_xprt->rx_ep.rep_max_requests + RPCRDMA_MAX_BC_REQUESTS; buf->rb_sc_ctxs = kcalloc(i, sizeof(sc), GFP_KERNEL); if (!buf->rb_sc_ctxs) return -ENOMEM; @@ -1154,7 +1155,6 @@ int rpcrdma_buffer_create(struct rpcrdma_xprt *r_xprt) struct rpcrdma_buffer *buf = &r_xprt->rx_buf; int i, rc; - buf->rb_max_requests = r_xprt->rx_ep.rep_max_requests; buf->rb_bc_srv_max_requests = 0; spin_lock_init(&buf->rb_lock); INIT_LIST_HEAD(&buf->rb_mrs); @@ -1166,7 +1166,7 @@ int rpcrdma_buffer_create(struct rpcrdma_xprt *r_xprt) INIT_LIST_HEAD(&buf->rb_all_reps); rc = -ENOMEM; - for (i = 0; i < buf->rb_max_requests; i++) { + for (i = 0; i < r_xprt->rx_xprt.max_reqs; i++) { struct rpcrdma_req *req; req = rpcrdma_req_create(r_xprt, RPCRDMA_V1_DEF_INLINE_SIZE * 2, diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h index 7655a99fd559..0fde694144f5 100644 --- a/net/sunrpc/xprtrdma/xprt_rdma.h +++ b/net/sunrpc/xprtrdma/xprt_rdma.h @@ -98,7 +98,7 @@ struct rpcrdma_ep { wait_queue_head_t rep_connect_wait; struct rpcrdma_connect_private rep_cm_private; struct rdma_conn_param rep_remote_cma; - unsigned int rep_max_requests; /* set by /proc */ + unsigned int rep_max_requests; /* depends on device */ unsigned int rep_inline_send; /* negotiated */ unsigned int rep_inline_recv; /* negotiated */ int rep_receive_count; @@ -372,7 +372,7 @@ struct rpcrdma_buffer { struct llist_head rb_free_reps; - u32 rb_max_requests; + __be32 rb_max_requests; u32 rb_credits; /* most recent credit grant */ u32 rb_bc_srv_max_requests; @@ -582,7 +582,6 @@ static inline void rpcrdma_set_xdrlen(struct xdr_buf *xdr, size_t len) /* RPC/RDMA module init - xprtrdma/transport.c */ -extern unsigned int xprt_rdma_slot_table_entries; extern unsigned int xprt_rdma_max_inline_read; extern unsigned int xprt_rdma_max_inline_write; void xprt_rdma_format_addresses(struct rpc_xprt *xprt, struct sockaddr *sap); From patchwork Fri Jan 3 16:56:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11317095 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 525C717F0 for ; Fri, 3 Jan 2020 16:56:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 23C6E21734 for ; Fri, 3 Jan 2020 16:56:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="upmqYuB4" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728079AbgACQ4q (ORCPT ); Fri, 3 Jan 2020 11:56:46 -0500 Received: from mail-yw1-f65.google.com ([209.85.161.65]:41110 "EHLO mail-yw1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727912AbgACQ4q (ORCPT ); Fri, 3 Jan 2020 11:56:46 -0500 Received: by mail-yw1-f65.google.com with SMTP id l22so18749088ywc.8; Fri, 03 Jan 2020 08:56:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=cRJRoQcIJ5xui/uZWNETubw9otNoq/moRTgYVs7gtO8=; b=upmqYuB4wOOjD8PxywTnCZKWCpLmHJnw/3Ht0iF/sspncxazSITgbg8eJPBr//eiiv rFqHauvmnnlfvk9dZsrcK4v+TwLe8eOoKVj2MzaFPAYQvVkKNCjqY6cJ4uyW53TwyK46 WVkIiH84zp0yEka9DHRYWmdqmrwJpBw46fFM0SJOzwX65yrgjguF4/rXi70quW8sCZkC QS68dfIqEKPTae2rjpBmX2QygeahC/j/7Us762C7av3E/gZV6zlWrMHWxsCelqZozpCW l/zB0ap4JmnxbUBrz2s6cmJbzeeslyKixkJhcS8w+AD8tMApA8DM5w8Bl1KnqwDL3Dg8 RPoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=cRJRoQcIJ5xui/uZWNETubw9otNoq/moRTgYVs7gtO8=; b=aybRq7GiZKoSZR06t1lBi9eN7hEIkuxQlbeUFYMa78zMZAm4Q0iwZXXDq1OwC7p2Dl 1xZSSAb8BX+KoXuKfNn+8OUFZ67NzavE02hK2GsqW4n3fScSrmnoT6Bs7i3gCxBRZPJp d8e2Ry+CBqr5BsMdpBoq1spZMTHh9w/MVuydCI4XSWrlNGhKLWNSi+vp9Ds1HHCzvnSl vKOOk3jPyyh0RM2qFzNjqEYKtQKFxCoAx4UMuC17Xwq+DqhxRR4XENF/9rhnl62sugpB erPizhkjd3ggl7PDoYCG2R1jmeCrvf3V5TL0e4fVJoPk7pib0IWZJf0JL1Lv2Vl/GwPY xvfA== X-Gm-Message-State: APjAAAUN0+n0MHqI+uA9WTeg6rRTvYrf2D72lS/MdN6Eupz1O7PkRQhL IoLQn1E6mxWxx5e843xI3IuBHXmV X-Google-Smtp-Source: APXvYqy5u48wmQ0t4muthCLxzMLBVL2DpyZZ6/FB2IecIY/GukRXhGssgAQOGHZAYf+J3auUnBDpeA== X-Received: by 2002:a81:de53:: with SMTP id o19mr68676539ywl.62.1578070604348; Fri, 03 Jan 2020 08:56:44 -0800 (PST) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id g5sm23627694ywk.46.2020.01.03.08.56.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Jan 2020 08:56:44 -0800 (PST) Received: from morisot.1015granger.net (morisot.1015granger.net [192.168.1.67]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 003GuhfZ016392; Fri, 3 Jan 2020 16:56:43 GMT Subject: [PATCH v1 4/9] xprtrdma: Eliminate per-transport "max pages" From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 03 Jan 2020 11:56:43 -0500 Message-ID: <157807060307.4606.13026227190289178598.stgit@morisot.1015granger.net> In-Reply-To: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> References: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> User-Agent: StGit/0.19 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org To support device hotplug and migrating a connection between devices of different capabilities, we have to guarantee that all in-kernel devices can support the same max NFS payload size (1 megabyte). This means that possibly one or two in-tree devices are no longer supported for NFS/RDMA because they cannot support 1MB rsize/wsize. The only one I confirmed was cxgb3, but it has already been removed from the kernel. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/frwr_ops.c | 40 +++++++++++++++------------------------ net/sunrpc/xprtrdma/rpc_rdma.c | 2 +- net/sunrpc/xprtrdma/transport.c | 14 ++++---------- net/sunrpc/xprtrdma/verbs.c | 4 ++-- net/sunrpc/xprtrdma/xprt_rdma.h | 3 +-- 5 files changed, 23 insertions(+), 40 deletions(-) diff --git a/net/sunrpc/xprtrdma/frwr_ops.c b/net/sunrpc/xprtrdma/frwr_ops.c index 24b8b75d0d49..f31ff54bb266 100644 --- a/net/sunrpc/xprtrdma/frwr_ops.c +++ b/net/sunrpc/xprtrdma/frwr_ops.c @@ -171,7 +171,7 @@ int frwr_init_mr(struct rpcrdma_ia *ia, struct rpcrdma_mr *mr) * ep->rep_attr.cap.max_send_wr * ep->rep_attr.cap.max_recv_wr * ep->rep_max_requests - * ia->ri_max_segs + * ia->ri_max_rdma_segs * * And these FRWR-related fields: * ia->ri_max_frwr_depth @@ -202,14 +202,12 @@ int frwr_open(struct rpcrdma_ia *ia, struct rpcrdma_ep *ep) * capability, but perform optimally when the MRs are not larger * than a page. */ - if (attrs->max_sge_rd > 1) + if (attrs->max_sge_rd > RPCRDMA_MAX_HDR_SEGS) ia->ri_max_frwr_depth = attrs->max_sge_rd; else ia->ri_max_frwr_depth = attrs->max_fast_reg_page_list_len; if (ia->ri_max_frwr_depth > RPCRDMA_MAX_DATA_SEGS) ia->ri_max_frwr_depth = RPCRDMA_MAX_DATA_SEGS; - dprintk("RPC: %s: max FR page list depth = %u\n", - __func__, ia->ri_max_frwr_depth); /* Add room for frwr register and invalidate WRs. * 1. FRWR reg WR for head @@ -253,30 +251,22 @@ int frwr_open(struct rpcrdma_ia *ia, struct rpcrdma_ep *ep) ep->rep_attr.cap.max_recv_wr += RPCRDMA_BACKWARD_WRS; ep->rep_attr.cap.max_recv_wr += 1; /* for ib_drain_rq */ - ia->ri_max_segs = + ia->ri_max_rdma_segs = DIV_ROUND_UP(RPCRDMA_MAX_DATA_SEGS, ia->ri_max_frwr_depth); /* Reply chunks require segments for head and tail buffers */ - ia->ri_max_segs += 2; - if (ia->ri_max_segs > RPCRDMA_MAX_HDR_SEGS) - ia->ri_max_segs = RPCRDMA_MAX_HDR_SEGS; - return 0; -} - -/** - * frwr_maxpages - Compute size of largest payload - * @r_xprt: transport - * - * Returns maximum size of an RPC message, in pages. - * - * FRWR mode conveys a list of pages per chunk segment. The - * maximum length of that list is the FRWR page list depth. - */ -size_t frwr_maxpages(struct rpcrdma_xprt *r_xprt) -{ - struct rpcrdma_ia *ia = &r_xprt->rx_ia; + ia->ri_max_rdma_segs += 2; + if (ia->ri_max_rdma_segs > RPCRDMA_MAX_HDR_SEGS) + ia->ri_max_rdma_segs = RPCRDMA_MAX_HDR_SEGS; + + /* Ensure the underlying device is capable of conveying the + * largest r/wsize NFS will ask for. This guarantees that + * failing over from one RDMA device to another will not + * break NFS I/O. + */ + if ((ia->ri_max_rdma_segs * ia->ri_max_frwr_depth) < RPCRDMA_MAX_SEGS) + return -ENOMEM; - return min_t(unsigned int, RPCRDMA_MAX_DATA_SEGS, - (ia->ri_max_segs - 2) * ia->ri_max_frwr_depth); + return 0; } /** diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c index 520323ddc930..c6dcea06c754 100644 --- a/net/sunrpc/xprtrdma/rpc_rdma.c +++ b/net/sunrpc/xprtrdma/rpc_rdma.c @@ -111,7 +111,7 @@ static unsigned int rpcrdma_max_reply_header_size(unsigned int maxsegs) */ void rpcrdma_set_max_header_sizes(struct rpcrdma_xprt *r_xprt) { - unsigned int maxsegs = r_xprt->rx_ia.ri_max_segs; + unsigned int maxsegs = r_xprt->rx_ia.ri_max_rdma_segs; struct rpcrdma_ep *ep = &r_xprt->rx_ep; ep->rep_max_inline_send = diff --git a/net/sunrpc/xprtrdma/transport.c b/net/sunrpc/xprtrdma/transport.c index f868a75057ad..3cfeba68ee9a 100644 --- a/net/sunrpc/xprtrdma/transport.c +++ b/net/sunrpc/xprtrdma/transport.c @@ -359,19 +359,13 @@ xprt_setup_rdma(struct xprt_create *args) if (rc) goto out3; - INIT_DELAYED_WORK(&new_xprt->rx_connect_worker, - xprt_rdma_connect_worker); - - xprt->max_payload = frwr_maxpages(new_xprt); - if (xprt->max_payload == 0) - goto out4; - xprt->max_payload <<= PAGE_SHIFT; - dprintk("RPC: %s: transport data payload maximum: %zu bytes\n", - __func__, xprt->max_payload); - if (!try_module_get(THIS_MODULE)) goto out4; + INIT_DELAYED_WORK(&new_xprt->rx_connect_worker, + xprt_rdma_connect_worker); + xprt->max_payload = RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT; + dprintk("RPC: %s: %s:%s\n", __func__, xprt->address_strings[RPC_DISPLAY_ADDR], xprt->address_strings[RPC_DISPLAY_PORT]); diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 4f9595b72888..601a1bd56260 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -935,7 +935,7 @@ rpcrdma_mrs_create(struct rpcrdma_xprt *r_xprt) struct rpcrdma_ia *ia = &r_xprt->rx_ia; unsigned int count; - for (count = 0; count < ia->ri_max_segs; count++) { + for (count = 0; count < ia->ri_max_rdma_segs; count++) { struct rpcrdma_mr *mr; int rc; @@ -1017,7 +1017,7 @@ struct rpcrdma_req *rpcrdma_req_create(struct rpcrdma_xprt *r_xprt, size_t size, /* Compute maximum header buffer size in bytes */ maxhdrsize = rpcrdma_fixed_maxsz + 3 + - r_xprt->rx_ia.ri_max_segs * rpcrdma_readchunk_maxsz; + r_xprt->rx_ia.ri_max_rdma_segs * rpcrdma_readchunk_maxsz; maxhdrsize *= sizeof(__be32); rb = rpcrdma_regbuf_alloc(__roundup_pow_of_two(maxhdrsize), DMA_TO_DEVICE, flags); diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h index 0fde694144f5..aac4cf959c3a 100644 --- a/net/sunrpc/xprtrdma/xprt_rdma.h +++ b/net/sunrpc/xprtrdma/xprt_rdma.h @@ -71,7 +71,7 @@ struct rpcrdma_ia { struct rdma_cm_id *ri_id; struct ib_pd *ri_pd; int ri_async_rc; - unsigned int ri_max_segs; + unsigned int ri_max_rdma_segs; unsigned int ri_max_frwr_depth; bool ri_implicit_roundup; enum ib_mr_type ri_mrtype; @@ -539,7 +539,6 @@ void frwr_reset(struct rpcrdma_req *req); int frwr_open(struct rpcrdma_ia *ia, struct rpcrdma_ep *ep); int frwr_init_mr(struct rpcrdma_ia *ia, struct rpcrdma_mr *mr); void frwr_release_mr(struct rpcrdma_mr *mr); -size_t frwr_maxpages(struct rpcrdma_xprt *r_xprt); struct rpcrdma_mr_seg *frwr_map(struct rpcrdma_xprt *r_xprt, struct rpcrdma_mr_seg *seg, int nsegs, bool writing, __be32 xid, From patchwork Fri Jan 3 16:56:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11317099 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD1BE109A for ; Fri, 3 Jan 2020 16:56:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BC0E621734 for ; Fri, 3 Jan 2020 16:56:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Q1HIICC4" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728088AbgACQ4v (ORCPT ); Fri, 3 Jan 2020 11:56:51 -0500 Received: from mail-yw1-f68.google.com ([209.85.161.68]:41117 "EHLO mail-yw1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727912AbgACQ4v (ORCPT ); Fri, 3 Jan 2020 11:56:51 -0500 Received: by mail-yw1-f68.google.com with SMTP id l22so18749225ywc.8; Fri, 03 Jan 2020 08:56:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=tCZVxSAnaTTodNkIrs6d0LpkpyWPH5ZzAla3zGsgHkU=; b=Q1HIICC46YeH3jzrPdeseIemA4f5HPZ/M8Hhmi/KkuaQdQPD23YGPM9AVHA5Ito+oE spJEXqgcHR183nu9j96DA20VPjkzYSJ1mI8rPenhvmnia+C5MMhxT+ebytB9hEP/4SoI b5d7+hMbyEmX1rEVMPiQ5ofKuFRZlTxQgd24RoLh8mhAMXQY04jpKOOcTZSx6nOCVFOg BibQqPkrsiBXEgzUoSrzQk2ZTCEjNThpMnwbkCbkK7KMJVMawCzG2e6YaAhyM3UnR4+0 RcJlSCWWTswmDuyXo9c00SBG5+ZFaVll6L4eM/0M24sF9i/ImU618GBnWxnTPkN6sYBn 18aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=tCZVxSAnaTTodNkIrs6d0LpkpyWPH5ZzAla3zGsgHkU=; b=oZdl/kOhNo/vKn02uFFJkBfwBGgTxLNv2MpjvXpNAUOIa9NRJ8WbgTdj1xl/LsU3Rm gWqIahkB3rlTtD64kFsCBjau43fFB7d/rEsH3iXZsVslklhnxTJo67ZOs0z0rcGfF1U8 GDX8oNgLDdMtsemQZH7bCqgK5LhByG+U81EBYpS7dg2mJZe30Jzb6xkLfhmVcHzk4tA0 Y3osRcdpwJfavN+fBpX+3PxGC2BYETEXfBe4ztrPyHG7J8MTFhqD5/tf+zYIcJk5vEak qdJGNu1oLBQlWrpckTLR5GvGr+NHLtEOa8vQ3a6/dlU0B7idWQCTWY2EXurDuIojp8FK HtDQ== X-Gm-Message-State: APjAAAVp1Wob8OLcvLaqy52XeBnX1txvZv30j4LCRl0sdcQfSUhbKug6 v7Kv6RHT+LQ9tyjGtRYVX0FGW47R X-Google-Smtp-Source: APXvYqxdBkM0PZOHm01LJvEUjEuLei6FG97nU5xn8UC5bQTvPG0dUDaDo8oTBr38ta1An0MRMSbsbA== X-Received: by 2002:a81:39d7:: with SMTP id g206mr66333288ywa.484.1578070609757; Fri, 03 Jan 2020 08:56:49 -0800 (PST) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id g29sm23955290ywk.31.2020.01.03.08.56.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Jan 2020 08:56:49 -0800 (PST) Received: from morisot.1015granger.net (morisot.1015granger.net [192.168.1.67]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 003GumKD016395; Fri, 3 Jan 2020 16:56:48 GMT Subject: [PATCH v1 5/9] xprtrdma: Refactor frwr_is_supported From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 03 Jan 2020 11:56:48 -0500 Message-ID: <157807060839.4606.7268843859672908333.stgit@morisot.1015granger.net> In-Reply-To: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> References: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> User-Agent: StGit/0.19 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Refactor: Perform the "is supported" check in rpcrdma_ep_create() instead of in rpcrdma_ia_open(). frwr_open() is where most of the logic to query device attributes is already located. The current code displays a redundant error message when the device does not support FRWR. As an additional clean-up, this patch removes the extra message. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/frwr_ops.c | 54 ++++++++++++++++----------------------- net/sunrpc/xprtrdma/verbs.c | 14 +--------- net/sunrpc/xprtrdma/xprt_rdma.h | 4 +-- 3 files changed, 25 insertions(+), 47 deletions(-) diff --git a/net/sunrpc/xprtrdma/frwr_ops.c b/net/sunrpc/xprtrdma/frwr_ops.c index f31ff54bb266..b22a9d7f9c8d 100644 --- a/net/sunrpc/xprtrdma/frwr_ops.c +++ b/net/sunrpc/xprtrdma/frwr_ops.c @@ -50,28 +50,6 @@ # define RPCDBG_FACILITY RPCDBG_TRANS #endif -/** - * frwr_is_supported - Check if device supports FRWR - * @device: interface adapter to check - * - * Returns true if device supports FRWR, otherwise false - */ -bool frwr_is_supported(struct ib_device *device) -{ - struct ib_device_attr *attrs = &device->attrs; - - if (!(attrs->device_cap_flags & IB_DEVICE_MEM_MGT_EXTENSIONS)) - goto out_not_supported; - if (attrs->max_fast_reg_page_list_len == 0) - goto out_not_supported; - return true; - -out_not_supported: - pr_info("rpcrdma: 'frwr' mode is not supported by device %s\n", - device->name); - return false; -} - /** * frwr_release_mr - Destroy one MR * @mr: MR allocated by frwr_init_mr @@ -163,13 +141,12 @@ int frwr_init_mr(struct rpcrdma_ia *ia, struct rpcrdma_mr *mr) } /** - * frwr_open - Prepare an endpoint for use with FRWR - * @ia: interface adapter this endpoint will use - * @ep: endpoint to prepare + * frwr_query_device - Prepare a transport for use with FRWR + * @r_xprt: controlling transport instance + * @device: RDMA device to query * * On success, sets: - * ep->rep_attr.cap.max_send_wr - * ep->rep_attr.cap.max_recv_wr + * ep->rep_attr * ep->rep_max_requests * ia->ri_max_rdma_segs * @@ -177,14 +154,27 @@ int frwr_init_mr(struct rpcrdma_ia *ia, struct rpcrdma_mr *mr) * ia->ri_max_frwr_depth * ia->ri_mrtype * - * On failure, a negative errno is returned. + * Return values: + * On success, returns zero. + * %-EINVAL - the device does not support FRWR memory registration + * %-ENOMEM - the device is not sufficiently capable for NFS/RDMA */ -int frwr_open(struct rpcrdma_ia *ia, struct rpcrdma_ep *ep) +int frwr_query_device(struct rpcrdma_xprt *r_xprt, + const struct ib_device *device) { - struct ib_device_attr *attrs = &ia->ri_id->device->attrs; + const struct ib_device_attr *attrs = &device->attrs; + struct rpcrdma_ia *ia = &r_xprt->rx_ia; + struct rpcrdma_ep *ep = &r_xprt->rx_ep; int max_qp_wr, depth, delta; unsigned int max_sge; + if (!(attrs->device_cap_flags & IB_DEVICE_MEM_MGT_EXTENSIONS) || + attrs->max_fast_reg_page_list_len == 0) { + pr_err("rpcrdma: 'frwr' mode is not supported by device %s\n", + device->name); + return -EINVAL; + } + max_sge = min_t(unsigned int, attrs->max_send_sge, RPCRDMA_MAX_SEND_SGES); if (max_sge < RPCRDMA_MIN_SEND_SGES) { @@ -231,7 +221,7 @@ int frwr_open(struct rpcrdma_ia *ia, struct rpcrdma_ep *ep) } while (delta > 0); } - max_qp_wr = ia->ri_id->device->attrs.max_qp_wr; + max_qp_wr = attrs->max_qp_wr; max_qp_wr -= RPCRDMA_BACKWARD_WRS; max_qp_wr -= 1; if (max_qp_wr < RPCRDMA_MIN_SLOT_TABLE) @@ -242,7 +232,7 @@ int frwr_open(struct rpcrdma_ia *ia, struct rpcrdma_ep *ep) if (ep->rep_attr.cap.max_send_wr > max_qp_wr) { ep->rep_max_requests = max_qp_wr / depth; if (!ep->rep_max_requests) - return -EINVAL; + return -ENOMEM; ep->rep_attr.cap.max_send_wr = ep->rep_max_requests * depth; } ep->rep_attr.cap.max_send_wr += RPCRDMA_BACKWARD_WRS; diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 601a1bd56260..92a52934b622 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -367,18 +367,6 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt) goto out_err; } - switch (xprt_rdma_memreg_strategy) { - case RPCRDMA_FRWR: - if (frwr_is_supported(ia->ri_id->device)) - break; - /*FALLTHROUGH*/ - default: - pr_err("rpcrdma: Device %s does not support memreg mode %d\n", - ia->ri_id->device->name, xprt_rdma_memreg_strategy); - rc = -EINVAL; - goto out_err; - } - return 0; out_err: @@ -478,7 +466,7 @@ int rpcrdma_ep_create(struct rpcrdma_xprt *r_xprt) ep->rep_inline_send = xprt_rdma_max_inline_write; ep->rep_inline_recv = xprt_rdma_max_inline_read; - rc = frwr_open(ia, ep); + rc = frwr_query_device(r_xprt, ia->ri_id->device); if (rc) return rc; r_xprt->rx_buf.rb_max_requests = cpu_to_be32(ep->rep_max_requests); diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h index aac4cf959c3a..0aed1e98f2bf 100644 --- a/net/sunrpc/xprtrdma/xprt_rdma.h +++ b/net/sunrpc/xprtrdma/xprt_rdma.h @@ -534,9 +534,9 @@ rpcrdma_data_dir(bool writing) /* Memory registration calls xprtrdma/frwr_ops.c */ -bool frwr_is_supported(struct ib_device *device); void frwr_reset(struct rpcrdma_req *req); -int frwr_open(struct rpcrdma_ia *ia, struct rpcrdma_ep *ep); +int frwr_query_device(struct rpcrdma_xprt *r_xprt, + const struct ib_device *device); int frwr_init_mr(struct rpcrdma_ia *ia, struct rpcrdma_mr *mr); void frwr_release_mr(struct rpcrdma_mr *mr); struct rpcrdma_mr_seg *frwr_map(struct rpcrdma_xprt *r_xprt, From patchwork Fri Jan 3 16:56:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11317103 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DABEC139A for ; Fri, 3 Jan 2020 16:56:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B000B21734 for ; Fri, 3 Jan 2020 16:56:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XHEG8rgI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728095AbgACQ46 (ORCPT ); Fri, 3 Jan 2020 11:56:58 -0500 Received: from mail-yb1-f196.google.com ([209.85.219.196]:43636 "EHLO mail-yb1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727912AbgACQ46 (ORCPT ); Fri, 3 Jan 2020 11:56:58 -0500 Received: by mail-yb1-f196.google.com with SMTP id v24so18838477ybd.10; Fri, 03 Jan 2020 08:56:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=xs/OIZCaFg0QyQKRLKX5a5ofmuLVOt9cgDTAZQ1kDVU=; b=XHEG8rgI2fn+QGylKblgWuazG75l+JycMXKhLohCeCva15mY/k2YcMnC0YIK/o88he Mti1wnvbBuZA6sDee2SmFPMgbcEghQsUNSRFbn+rKG67sm7gBGTOH3/GVdF2YlNFAUtX Fctv1XcVxsV1EaQ901Vppt0cS/0stDbHNQIuPF8XDPhwyfnG3VjdzmG3IyPDNRAbjZSo UWD/RNtVL8ILC65XXDUE3Twgw86HMXrN2IlFK7TfLcMXV7vlWPLtLmsM5K8tYvPiTCeM m0ixcs8D6H1Y9NOXbtRfE46VaZWbl4VPWy1qWPeqLvvnLOZ5oEVVXOsI1PsOmjYGDxbn uw6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=xs/OIZCaFg0QyQKRLKX5a5ofmuLVOt9cgDTAZQ1kDVU=; b=SGLDEHs+daRJzPhZViAemcW0ywuuoivhM2ObgejUAua0NaHHkfpiVbpseDbjhVLtfn sn9htkdk30fEe28d1UTepfsMdij7U40SsQzi1qIamUwo8AsWrDCanwFzxdjmkl2coXGc JqxSQCQAW4fWTUfCwp0QoF2LWpV3iuOr1idqrSG5R4KJ5nWTV63Nj6WdaRL1K3fJpii/ nZqjvD1c7DNphVuNXZrNE351GNMRbcVReiRglTwdrp78DNQ3OGBBUwMKdJTvBj1Ib7rR RcyqOCLqi/0O86y2ZZKXj5jt5YYNI9cdGXMylSiLO205RwbvpIXKWJ9xPXzQVZ7ZmjlU 7gQA== X-Gm-Message-State: APjAAAWWHRqK1GKikInNjwT4qaKiyQYXwq5dIdAZS9R3gU/YWJle6+kQ zVh92ug/5uqdW7vg/BEd0zQvvmYW X-Google-Smtp-Source: APXvYqwr6nviD2ppveahVUt6RI2VuWk2jlDz0Z/LMsBGnb28f6gOPCkhYXoE2ogyfvSEgr3WyDThPg== X-Received: by 2002:a25:846:: with SMTP id 67mr24636369ybi.460.1578070614859; Fri, 03 Jan 2020 08:56:54 -0800 (PST) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id d10sm23973286ywd.107.2020.01.03.08.56.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Jan 2020 08:56:54 -0800 (PST) Received: from morisot.1015granger.net (morisot.1015granger.net [192.168.1.67]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 003Gur0Q016398; Fri, 3 Jan 2020 16:56:53 GMT Subject: [PATCH v1 6/9] xprtrdma: Allocate and map transport header buffers at connect time From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 03 Jan 2020 11:56:53 -0500 Message-ID: <157807061371.4606.808542074244029453.stgit@morisot.1015granger.net> In-Reply-To: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> References: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> User-Agent: StGit/0.19 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Currently the underlying RDMA device is chosen at transport set-up time. But it will soon be at connect time instead. The maximum size of a transport header is based on device capabilities. Thus transport header buffers have to be allocated _after_ the underlying device has been chosen (via address and route resolution); ie, in the connect worker. Thus, move the allocation of transport header buffers to the connect worker, after the point at which the underlying RDMA device has been chosen. This also means the RDMA device is available to do a DMA mapping of these buffers at connect time, instead of in the hot I/O path. Make that optimization as well. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/backchannel.c | 4 + net/sunrpc/xprtrdma/rpc_rdma.c | 10 +-- net/sunrpc/xprtrdma/verbs.c | 106 +++++++++++++++++++++++++++---------- net/sunrpc/xprtrdma/xprt_rdma.h | 1 4 files changed, 84 insertions(+), 37 deletions(-) diff --git a/net/sunrpc/xprtrdma/backchannel.c b/net/sunrpc/xprtrdma/backchannel.c index 9d02eae353c6..1a0ae0c61353 100644 --- a/net/sunrpc/xprtrdma/backchannel.c +++ b/net/sunrpc/xprtrdma/backchannel.c @@ -194,6 +194,10 @@ static struct rpc_rqst *rpcrdma_bc_rqst_get(struct rpcrdma_xprt *r_xprt) req = rpcrdma_req_create(r_xprt, size, GFP_KERNEL); if (!req) return NULL; + if (rpcrdma_req_setup(r_xprt, req)) { + rpcrdma_req_destroy(req); + return NULL; + } xprt->bc_alloc_count++; rqst = &req->rl_slot; diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c index c6dcea06c754..28020ec104d4 100644 --- a/net/sunrpc/xprtrdma/rpc_rdma.c +++ b/net/sunrpc/xprtrdma/rpc_rdma.c @@ -580,22 +580,19 @@ void rpcrdma_sendctx_unmap(struct rpcrdma_sendctx *sc) /* Prepare an SGE for the RPC-over-RDMA transport header. */ -static bool rpcrdma_prepare_hdr_sge(struct rpcrdma_xprt *r_xprt, +static void rpcrdma_prepare_hdr_sge(struct rpcrdma_xprt *r_xprt, struct rpcrdma_req *req, u32 len) { struct rpcrdma_sendctx *sc = req->rl_sendctx; struct rpcrdma_regbuf *rb = req->rl_rdmabuf; struct ib_sge *sge = &sc->sc_sges[req->rl_wr.num_sge++]; - if (!rpcrdma_regbuf_dma_map(r_xprt, rb)) - return false; sge->addr = rdmab_addr(rb); sge->length = len; sge->lkey = rdmab_lkey(rb); ib_dma_sync_single_for_device(rdmab_device(rb), sge->addr, sge->length, DMA_TO_DEVICE); - return true; } /* The head iovec is straightforward, as it is usually already @@ -836,10 +833,9 @@ inline int rpcrdma_prepare_send_sges(struct rpcrdma_xprt *r_xprt, req->rl_wr.num_sge = 0; req->rl_wr.opcode = IB_WR_SEND; - ret = -EIO; - if (!rpcrdma_prepare_hdr_sge(r_xprt, req, hdrlen)) - goto out_unmap; + rpcrdma_prepare_hdr_sge(r_xprt, req, hdrlen); + ret = -EIO; switch (rtype) { case rpcrdma_noch_pullup: if (!rpcrdma_prepare_noch_pullup(r_xprt, req, xdr)) diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 92a52934b622..7d06c6cd3d26 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -78,6 +78,7 @@ static int rpcrdma_sendctxs_create(struct rpcrdma_xprt *r_xprt); static void rpcrdma_sendctxs_destroy(struct rpcrdma_xprt *r_xprt); static void rpcrdma_sendctx_put_locked(struct rpcrdma_xprt *r_xprt, struct rpcrdma_sendctx *sc); +static int rpcrdma_reqs_setup(struct rpcrdma_xprt *r_xprt); static void rpcrdma_reqs_reset(struct rpcrdma_xprt *r_xprt); static void rpcrdma_reps_unmap(struct rpcrdma_xprt *r_xprt); static void rpcrdma_mrs_create(struct rpcrdma_xprt *r_xprt); @@ -380,6 +381,8 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt) * * Divest transport H/W resources associated with this adapter, * but allow it to be restored later. + * + * Caller must hold the transport send lock. */ void rpcrdma_ia_remove(struct rpcrdma_ia *ia) @@ -387,8 +390,6 @@ rpcrdma_ia_remove(struct rpcrdma_ia *ia) struct rpcrdma_xprt *r_xprt = container_of(ia, struct rpcrdma_xprt, rx_ia); struct rpcrdma_ep *ep = &r_xprt->rx_ep; - struct rpcrdma_buffer *buf = &r_xprt->rx_buf; - struct rpcrdma_req *req; /* This is similar to rpcrdma_ep_destroy, but: * - Don't cancel the connect worker. @@ -411,11 +412,7 @@ rpcrdma_ia_remove(struct rpcrdma_ia *ia) * mappings and MRs are gone. */ rpcrdma_reps_unmap(r_xprt); - list_for_each_entry(req, &buf->rb_allreqs, rl_all) { - rpcrdma_regbuf_dma_unmap(req->rl_rdmabuf); - rpcrdma_regbuf_dma_unmap(req->rl_sendbuf); - rpcrdma_regbuf_dma_unmap(req->rl_recvbuf); - } + rpcrdma_reqs_reset(r_xprt); rpcrdma_mrs_destroy(r_xprt); rpcrdma_sendctxs_destroy(r_xprt); ib_dealloc_pd(ia->ri_pd); @@ -714,6 +711,11 @@ rpcrdma_ep_connect(struct rpcrdma_ep *ep, struct rpcrdma_ia *ia) goto out; } + rc = rpcrdma_reqs_setup(r_xprt); + if (rc) { + rpcrdma_ep_disconnect(ep, ia); + goto out; + } rpcrdma_mrs_create(r_xprt); out: @@ -995,32 +997,19 @@ struct rpcrdma_req *rpcrdma_req_create(struct rpcrdma_xprt *r_xprt, size_t size, gfp_t flags) { struct rpcrdma_buffer *buffer = &r_xprt->rx_buf; - struct rpcrdma_regbuf *rb; struct rpcrdma_req *req; - size_t maxhdrsize; req = kzalloc(sizeof(*req), flags); if (req == NULL) goto out1; - /* Compute maximum header buffer size in bytes */ - maxhdrsize = rpcrdma_fixed_maxsz + 3 + - r_xprt->rx_ia.ri_max_rdma_segs * rpcrdma_readchunk_maxsz; - maxhdrsize *= sizeof(__be32); - rb = rpcrdma_regbuf_alloc(__roundup_pow_of_two(maxhdrsize), - DMA_TO_DEVICE, flags); - if (!rb) - goto out2; - req->rl_rdmabuf = rb; - xdr_buf_init(&req->rl_hdrbuf, rdmab_data(rb), rdmab_length(rb)); - req->rl_sendbuf = rpcrdma_regbuf_alloc(size, DMA_TO_DEVICE, flags); if (!req->rl_sendbuf) - goto out3; + goto out2; req->rl_recvbuf = rpcrdma_regbuf_alloc(size, DMA_NONE, flags); if (!req->rl_recvbuf) - goto out4; + goto out3; INIT_LIST_HEAD(&req->rl_free_mrs); INIT_LIST_HEAD(&req->rl_registered); @@ -1029,10 +1018,8 @@ struct rpcrdma_req *rpcrdma_req_create(struct rpcrdma_xprt *r_xprt, size_t size, spin_unlock(&buffer->rb_lock); return req; -out4: - kfree(req->rl_sendbuf); out3: - kfree(req->rl_rdmabuf); + kfree(req->rl_sendbuf); out2: kfree(req); out1: @@ -1040,23 +1027,82 @@ struct rpcrdma_req *rpcrdma_req_create(struct rpcrdma_xprt *r_xprt, size_t size, } /** - * rpcrdma_reqs_reset - Reset all reqs owned by a transport + * rpcrdma_req_setup - Per-connection instance setup of an rpcrdma_req object * @r_xprt: controlling transport instance + * @req: rpcrdma_req object to set up * - * ASSUMPTION: the rb_allreqs list is stable for the duration, + * Returns zero on success, and a negative errno on failure. + */ +int rpcrdma_req_setup(struct rpcrdma_xprt *r_xprt, struct rpcrdma_req *req) +{ + struct rpcrdma_regbuf *rb; + size_t maxhdrsize; + + /* Compute maximum header buffer size in bytes */ + maxhdrsize = rpcrdma_fixed_maxsz + 3 + + r_xprt->rx_ia.ri_max_rdma_segs * rpcrdma_readchunk_maxsz; + maxhdrsize *= sizeof(__be32); + rb = rpcrdma_regbuf_alloc(__roundup_pow_of_two(maxhdrsize), + DMA_TO_DEVICE, GFP_KERNEL); + if (!rb) + goto out; + + if (!__rpcrdma_regbuf_dma_map(r_xprt, rb)) + goto out_free; + + req->rl_rdmabuf = rb; + xdr_buf_init(&req->rl_hdrbuf, rdmab_data(rb), rdmab_length(rb)); + return 0; + +out_free: + rpcrdma_regbuf_free(rb); +out: + return -ENOMEM; +} + +/* ASSUMPTION: the rb_allreqs list is stable for the duration, * and thus can be walked without holding rb_lock. Eg. the * caller is holding the transport send lock to exclude * device removal or disconnection. */ -static void rpcrdma_reqs_reset(struct rpcrdma_xprt *r_xprt) +static int rpcrdma_reqs_setup(struct rpcrdma_xprt *r_xprt) { struct rpcrdma_buffer *buf = &r_xprt->rx_buf; struct rpcrdma_req *req; + int rc; list_for_each_entry(req, &buf->rb_allreqs, rl_all) { - /* Credits are valid only for one connection */ - req->rl_slot.rq_cong = 0; + rc = rpcrdma_req_setup(r_xprt, req); + if (rc) + return rc; } + return 0; +} + +static void rpcrdma_req_reset(struct rpcrdma_req *req) +{ + /* Credits are valid for only one connection */ + req->rl_slot.rq_cong = 0; + + rpcrdma_regbuf_free(req->rl_rdmabuf); + req->rl_rdmabuf = NULL; + + rpcrdma_regbuf_dma_unmap(req->rl_sendbuf); + rpcrdma_regbuf_dma_unmap(req->rl_recvbuf); +} + +/* ASSUMPTION: the rb_allreqs list is stable for the duration, + * and thus can be walked without holding rb_lock. Eg. the + * caller is holding the transport send lock to exclude + * device removal or disconnection. + */ +static void rpcrdma_reqs_reset(struct rpcrdma_xprt *r_xprt) +{ + struct rpcrdma_buffer *buf = &r_xprt->rx_buf; + struct rpcrdma_req *req; + + list_for_each_entry(req, &buf->rb_allreqs, rl_all) + rpcrdma_req_reset(req); } static struct rpcrdma_rep *rpcrdma_rep_create(struct rpcrdma_xprt *r_xprt, diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h index 0aed1e98f2bf..37d5080c250b 100644 --- a/net/sunrpc/xprtrdma/xprt_rdma.h +++ b/net/sunrpc/xprtrdma/xprt_rdma.h @@ -478,6 +478,7 @@ void rpcrdma_post_recvs(struct rpcrdma_xprt *r_xprt, bool temp); */ struct rpcrdma_req *rpcrdma_req_create(struct rpcrdma_xprt *r_xprt, size_t size, gfp_t flags); +int rpcrdma_req_setup(struct rpcrdma_xprt *r_xprt, struct rpcrdma_req *req); void rpcrdma_req_destroy(struct rpcrdma_req *req); int rpcrdma_buffer_create(struct rpcrdma_xprt *); void rpcrdma_buffer_destroy(struct rpcrdma_buffer *); From patchwork Fri Jan 3 16:56:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11317107 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4A74D1580 for ; Fri, 3 Jan 2020 16:57:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 29867206DB for ; Fri, 3 Jan 2020 16:57:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="D8MXXIN5" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727912AbgACQ5B (ORCPT ); Fri, 3 Jan 2020 11:57:01 -0500 Received: from mail-yb1-f195.google.com ([209.85.219.195]:39577 "EHLO mail-yb1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728097AbgACQ5B (ORCPT ); Fri, 3 Jan 2020 11:57:01 -0500 Received: by mail-yb1-f195.google.com with SMTP id b12so18840297ybg.6; Fri, 03 Jan 2020 08:57:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=CF/aJ8+vnP2vj/nR80InQ8WZ6kBg+hhJmHh5njAIuJE=; b=D8MXXIN5rAV7ZM50WQXHH/88Doo998v83o0oC4YKhtDxqqWkdOxpjibLd0ryQHDRwY W4qRdAk0qlAR2z4IbimMc3rMV35u6Y6gtDg8w4M/KRPYS1jMb2SYDW9kohSe3qkx+M4S eVWoJ07nSMnwVT6B7j+9BqUklyxgyP32Cs5BW1NA+JQ6WwyJaXAU72qxW2aQ5qLcSlB4 CrQ7nnwpfdNTeVOdSsBzraIzgs+xHoKgymtguswNUmwLXwMAxLMkVWV9NqkLbBXtMOaQ QHKu0tCcPY+Fwc4lJ/thU5UwZqzWuwBsUQLRbUd4LVeRX+9iPEOs3UwmI819ZcsiG18M wnZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=CF/aJ8+vnP2vj/nR80InQ8WZ6kBg+hhJmHh5njAIuJE=; b=UkTqxyH8/Nzy0IyfKYi5saoNXgUxDBSZCzhLYSC19p9WBytjFsaANbROcfB2+81s1R 4jzsBXJa86EOnRr0HFGurvNoAPyS/1zMrN7kMhR4HPQ88zeGrmamELEjyT5q2gHxYv2Y Kim30/nhBQXS+rXhFjum/j36XZSI8qorgAdRjteJ5uNflu929r7MKxzRH51Bz91ROtEn 6ZtwAS+ZZAjbPS9vlCow10I0wTH9K5G3jlU4Ab5Kjhm4qME5XDUS1PU+nIUmnnUtoqZX UpYgTkX/3ZEFER9vr28czuF7i/70B91WPV+60A3LDS8ZdII0cXBEXkh6QS0cPLjK40/B V/OA== X-Gm-Message-State: APjAAAX/1aP1RcOhL3F6v2bGuD46q0enm5SzIdef73GpdWsKM3XeNAJ2 zTlfGgz7oGm442kdQl8bJMO/IOX2 X-Google-Smtp-Source: APXvYqzRrxSD8JfT4N1ugYaDCEf9rBprZIuypH69HdcXQidYNmVKhjBpxhTPLuiefS47EdIa33QL4Q== X-Received: by 2002:a25:aab4:: with SMTP id t49mr62122149ybi.325.1578070620053; Fri, 03 Jan 2020 08:57:00 -0800 (PST) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id w185sm23947286ywe.75.2020.01.03.08.56.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Jan 2020 08:56:59 -0800 (PST) Received: from morisot.1015granger.net (morisot.1015granger.net [192.168.1.67]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 003GuwWL016401; Fri, 3 Jan 2020 16:56:58 GMT Subject: [PATCH v1 7/9] xprtrdma: Destroy rpcrdma_rep when Receive is flushed From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 03 Jan 2020 11:56:58 -0500 Message-ID: <157807061881.4606.3320048417922431045.stgit@morisot.1015granger.net> In-Reply-To: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> References: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> User-Agent: StGit/0.19 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org This reduces the hardware and memory footprint of an unconnected transport. At some point in the future, transport reconnect will allow resolving the destination IP address through a different device. The current change enables reps for the new connection to be allocated on whichever NUMA node the new device affines to after a reconnect. Note that this does not destroy _all_ the transport's reps... there will be a few that are still part of a running RPC completion. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/verbs.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 7d06c6cd3d26..269df615a024 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -80,6 +80,7 @@ static void rpcrdma_sendctx_put_locked(struct rpcrdma_xprt *r_xprt, struct rpcrdma_sendctx *sc); static int rpcrdma_reqs_setup(struct rpcrdma_xprt *r_xprt); static void rpcrdma_reqs_reset(struct rpcrdma_xprt *r_xprt); +static void rpcrdma_rep_destroy(struct rpcrdma_rep *rep); static void rpcrdma_reps_unmap(struct rpcrdma_xprt *r_xprt); static void rpcrdma_mrs_create(struct rpcrdma_xprt *r_xprt); static void rpcrdma_mrs_destroy(struct rpcrdma_xprt *r_xprt); @@ -177,7 +178,7 @@ rpcrdma_wc_receive(struct ib_cq *cq, struct ib_wc *wc) return; out_flushed: - rpcrdma_recv_buffer_put(rep); + rpcrdma_rep_destroy(rep); } static void rpcrdma_update_cm_private(struct rpcrdma_xprt *r_xprt, @@ -1105,6 +1106,9 @@ static void rpcrdma_reqs_reset(struct rpcrdma_xprt *r_xprt) rpcrdma_req_reset(req); } +/* No locking needed here. This function is called only by the + * Receive completion handler. + */ static struct rpcrdma_rep *rpcrdma_rep_create(struct rpcrdma_xprt *r_xprt, bool temp) { @@ -1137,6 +1141,9 @@ static struct rpcrdma_rep *rpcrdma_rep_create(struct rpcrdma_xprt *r_xprt, return NULL; } +/* No locking needed here. This function is invoked only by the + * Receive completion handler, or during transport shutdown. + */ static void rpcrdma_rep_destroy(struct rpcrdma_rep *rep) { list_del(&rep->rr_all); From patchwork Fri Jan 3 16:57:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11317113 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 50AE51580 for ; Fri, 3 Jan 2020 16:57:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3060B21734 for ; Fri, 3 Jan 2020 16:57:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="aL416D7q" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728097AbgACQ5H (ORCPT ); Fri, 3 Jan 2020 11:57:07 -0500 Received: from mail-yw1-f66.google.com ([209.85.161.66]:41128 "EHLO mail-yw1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727969AbgACQ5G (ORCPT ); Fri, 3 Jan 2020 11:57:06 -0500 Received: by mail-yw1-f66.google.com with SMTP id l22so18749566ywc.8; Fri, 03 Jan 2020 08:57:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=do4XYB3cetSwSCpHsQwhYK9wBCxtcun36fD+roGwm0o=; b=aL416D7qAa6kasaUjy8jxOnm14XtSXL76EzyRBeMq66iggapMNyEELX0vZS/eHYTq/ PpdsZzu4v63cGMhvETY/UKsT2YaFg7Qt64plUUGq1PQCmAdc4WEmaa2AUrY6wZ6ilH7e dwMWXpzpzxSTdNTWbMUwwRKwh3+Jfu2cxCNRpw6H9ffLww62iJRT8NkALLR6aXkh5M25 KMSiCXLHkcE8AjqWzbhOyPSYgPi14BTKPeKPpr53lAAhJJ+yKgdoiAd0WPUGe0qJipIo 3KcoKftjhus2FtmZ0ub45+KeGh/EdTiCZl8a1mwsg6gFo5xaTA5OAYRAFamXGshNFdVF rN1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=do4XYB3cetSwSCpHsQwhYK9wBCxtcun36fD+roGwm0o=; b=acB53jTBV9yBYHP0eafq9iSNUAzHJVHszn3vpo3hA4UQMMrK2yMZse2ZjkO0uNoQVN 0NJlRYYH1U3mNMolvpOTfFsfusY0gbH5ueU7NFbMGnMyuDyCAN/p+yvWrs5Cw1CsSGpe PKesVDTxVsJV8q5YrXEhYpP27/6XHLvQMi/WNoXBmOZ5D8NOFjvtE7+ri84t6r7HRKGx dDSeM/SWr0RU5fcEznDve5br/kiEPwijEk0MzD9HtBbA8m/NPpmutJ7kanxOw9cOGjtL mxf3DMROtUucKvqu/GOu2S0TBcjgSEQPnVA3UDCHFblnXb6mJHhcCeu4oF+pypRyIfa4 JNiw== X-Gm-Message-State: APjAAAWJEhRGkf0kFLunF3SH2HcxCmBgBuA/dArkDqzxn8Wpo+++7Q18 1C+mUQO4JOzbsNnfJHZkun+hYZCu X-Google-Smtp-Source: APXvYqzwEz1aBNgEg7xNWVfAj1TCRciP0DA7KomCVZA3/o6qi+8iGS80/nbcb7FxpfA8/6NZzU4yXA== X-Received: by 2002:a81:9c14:: with SMTP id m20mr64492596ywa.143.1578070625398; Fri, 03 Jan 2020 08:57:05 -0800 (PST) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id a23sm23502387ywa.32.2020.01.03.08.57.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Jan 2020 08:57:05 -0800 (PST) Received: from morisot.1015granger.net (morisot.1015granger.net [192.168.1.67]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 003Gv4Hi016404; Fri, 3 Jan 2020 16:57:04 GMT Subject: [PATCH v1 8/9] xprtrdma: Destroy reps from previous connection instance From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 03 Jan 2020 11:57:04 -0500 Message-ID: <157807062412.4606.7516445302332824137.stgit@morisot.1015granger.net> In-Reply-To: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> References: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> User-Agent: StGit/0.19 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org To safely get rid of all rpcrdma_reps from a particular connection instance, xprtrdma has to wait until each of those reps is finished being used. A rep may be backing the rq_rcv_buf of an RPC that has just completed, for example. Since it is safe to invoke rpcrdma_rep_destroy() only in the Receive completion handler, simply mark reps remaining in the rb_all_reps list after the transport is drained. These will then be deleted as rpcrdma_post_recvs pulls them off the rep free list. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/verbs.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 269df615a024..3169887f8547 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -1173,8 +1173,10 @@ static void rpcrdma_reps_unmap(struct rpcrdma_xprt *r_xprt) struct rpcrdma_buffer *buf = &r_xprt->rx_buf; struct rpcrdma_rep *rep; - list_for_each_entry(rep, &buf->rb_all_reps, rr_all) + list_for_each_entry(rep, &buf->rb_all_reps, rr_all) { rpcrdma_regbuf_dma_unmap(rep->rr_rdmabuf); + rep->rr_temp = true; + } } static void rpcrdma_reps_destroy(struct rpcrdma_buffer *buf) From patchwork Fri Jan 3 16:57:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11317115 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D37301580 for ; Fri, 3 Jan 2020 16:57:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B24B521734 for ; Fri, 3 Jan 2020 16:57:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IeMU7q1G" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728128AbgACQ5M (ORCPT ); Fri, 3 Jan 2020 11:57:12 -0500 Received: from mail-yw1-f68.google.com ([209.85.161.68]:44422 "EHLO mail-yw1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727969AbgACQ5M (ORCPT ); Fri, 3 Jan 2020 11:57:12 -0500 Received: by mail-yw1-f68.google.com with SMTP id t141so18745569ywc.11; Fri, 03 Jan 2020 08:57:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=2Q2YPq+rSLrKcR+aFtKjbLTauVxcEyEoCr3Nslf/jTc=; b=IeMU7q1G1vhhLg/HXZ27mTcV/Oq3BvsOgOUQ2trLI+7HLpSZklrNgTl/oTfd9mRaLC rhdsxaSV7TW2boIfupIY1Hzs9MlIl4og16H2JOTZLWIkNWEu81jSb+EUegFYV3h5EyP5 5q7XoQZlFAAfe5GR4f348v/c19JYJLBvnpXTTSgaedezR1ZGwTI+7w0Fcv5r3viXkM3s HfKGOWHHxmVFy2J4jVVXWVtExC0uSZbLCC+Uy0d/hxxKH6jfkJCu6JkUN4C0U1tP0aIV YlGcjWK7wmVGmE0qyLIYkwf++e1gyu1jZy2uL6Ig5GwofDLOT5GDUgABfAukZE1LTt+3 fyyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=2Q2YPq+rSLrKcR+aFtKjbLTauVxcEyEoCr3Nslf/jTc=; b=IZsMdgX2yfEH+Xu372ePRRZuV7i/xXspMn8NWVYppGnVWQpkWl9uAmZHRVn8MeFWTh hrgx744SH+jcIEu3tkE8MdhptT0hhcAKheXclgXm4y4GNTT7uRnIL+WhMf5UYMqJP7l2 cp+pZk1s0s0L8YSJoiNP2/mseKmPA43rrB87xWbD1myGSrlr7BdHWt93jfHrP/MnV5WC 0tMnVQQ0sJF1uCJfIqLQjMJDmzZ/kKIozE4+lhlo78XmEK7+SONNCm1khvJG5p6zJBCI hWEkSpPNrg/S/PEys1o/k48ppefTDh8G5f3QY5nQay1uYCQXUrJpf/tl8VaRwXzz05st bSUQ== X-Gm-Message-State: APjAAAWX8l9d+WiiVg+3vT0ubRSIp8OXALpCOmmnrdCjZs47nx6RtLn6 7M+UdOPccvg5S1N1g9lgEeukZeJj X-Google-Smtp-Source: APXvYqxHDnZoKxLEQC1+lhiHx0wNMD2on4pKiOk0U9z2qcCrZHQF84vjLgr14rk6/lG4QiS89KZBNA== X-Received: by 2002:a81:a906:: with SMTP id g6mr56576219ywh.186.1578070630736; Fri, 03 Jan 2020 08:57:10 -0800 (PST) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id 189sm24351008ywc.16.2020.01.03.08.57.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Jan 2020 08:57:10 -0800 (PST) Received: from morisot.1015granger.net (morisot.1015granger.net [192.168.1.67]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 003Gv9F1016407; Fri, 3 Jan 2020 16:57:09 GMT Subject: [PATCH v1 9/9] xprtrdma: DMA map rr_rdma_buf as each rpcrdma_rep is created From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 03 Jan 2020 11:57:09 -0500 Message-ID: <157807062944.4606.8470156130970554891.stgit@morisot.1015granger.net> In-Reply-To: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> References: <157807044515.4606.732915438702066797.stgit@morisot.1015granger.net> User-Agent: StGit/0.19 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: This simplifies the logic in rpcrdma_post_recvs. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/verbs.c | 31 +++++++++++-------------------- 1 file changed, 11 insertions(+), 20 deletions(-) diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 3169887f8547..c3e92b5607b5 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -1109,8 +1109,9 @@ static void rpcrdma_reqs_reset(struct rpcrdma_xprt *r_xprt) /* No locking needed here. This function is called only by the * Receive completion handler. */ -static struct rpcrdma_rep *rpcrdma_rep_create(struct rpcrdma_xprt *r_xprt, - bool temp) +static noinline +struct rpcrdma_rep *rpcrdma_rep_create(struct rpcrdma_xprt *r_xprt, + bool temp) { struct rpcrdma_rep *rep; @@ -1123,6 +1124,9 @@ static struct rpcrdma_rep *rpcrdma_rep_create(struct rpcrdma_xprt *r_xprt, if (!rep->rr_rdmabuf) goto out_free; + if (!rpcrdma_regbuf_dma_map(r_xprt, rep->rr_rdmabuf)) + goto out_free_regbuf; + xdr_buf_init(&rep->rr_hdrbuf, rdmab_data(rep->rr_rdmabuf), rdmab_length(rep->rr_rdmabuf)); rep->rr_cqe.done = rpcrdma_wc_receive; @@ -1135,6 +1139,8 @@ static struct rpcrdma_rep *rpcrdma_rep_create(struct rpcrdma_xprt *r_xprt, list_add(&rep->rr_all, &r_xprt->rx_buf.rb_all_reps); return rep; +out_free_regbuf: + rpcrdma_regbuf_free(rep->rr_rdmabuf); out_free: kfree(rep); out: @@ -1536,7 +1542,7 @@ void rpcrdma_post_recvs(struct rpcrdma_xprt *r_xprt, bool temp) { struct rpcrdma_buffer *buf = &r_xprt->rx_buf; struct rpcrdma_ep *ep = &r_xprt->rx_ep; - struct ib_recv_wr *i, *wr, *bad_wr; + struct ib_recv_wr *wr, *bad_wr; struct rpcrdma_rep *rep; int needed, count, rc; @@ -1563,23 +1569,15 @@ void rpcrdma_post_recvs(struct rpcrdma_xprt *r_xprt, bool temp) if (!rep) break; + trace_xprtrdma_post_recv(rep); rep->rr_recv_wr.next = wr; wr = &rep->rr_recv_wr; --needed; + ++count; } if (!wr) goto out; - for (i = wr; i; i = i->next) { - rep = container_of(i, struct rpcrdma_rep, rr_recv_wr); - - if (!rpcrdma_regbuf_dma_map(r_xprt, rep->rr_rdmabuf)) - goto release_wrs; - - trace_xprtrdma_post_recv(rep); - ++count; - } - rc = ib_post_recv(r_xprt->rx_ia.ri_id->qp, wr, (const struct ib_recv_wr **)&bad_wr); out: @@ -1596,11 +1594,4 @@ void rpcrdma_post_recvs(struct rpcrdma_xprt *r_xprt, bool temp) } ep->rep_receive_count += count; return; - -release_wrs: - for (i = wr; i;) { - rep = container_of(i, struct rpcrdma_rep, rr_recv_wr); - i = i->next; - rpcrdma_recv_buffer_put(rep); - } }