From patchwork Thu Mar 19 15:20:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11447505 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E09A792A for ; Thu, 19 Mar 2020 15:20:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C06A620BED for ; Thu, 19 Mar 2020 15:20:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="pKqpvb9Q" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727464AbgCSPUe (ORCPT ); Thu, 19 Mar 2020 11:20:34 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:43687 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727001AbgCSPUe (ORCPT ); Thu, 19 Mar 2020 11:20:34 -0400 Received: by mail-qt1-f193.google.com with SMTP id l13so2077085qtv.10; Thu, 19 Mar 2020 08:20:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=CIxixJZwpGLo0sVVb9Fs9GENGxOabLmSEhF/beS+CUs=; b=pKqpvb9Q0p1Pbf+ZrXo4eb6DnCbdVOlbkmfP7SXeHojXC+CMF7XugwNYQwno6zZN7B 6fEgAyKuT3QwWL5gx1DywyyzhNr2O1IHu0u5REPv1k7deq/E1uABfB695D6w8udU7uX2 7iNto/liDpkJtdLHzVhPIlyoxGqQxsehUyRADT3UJjk0yZpo1vQOcV80Z8mREDvDmq7n VpGL5NwwNpDF8GblXw4hDEqTQmJr8GFuHNxre6YZURJjYAHDozxd7U75TzOwh+GXc6AC cTsvzk3miiWE1OrpyDrTdXwlKd0D2Bifudm4v3rztcd39yMeuBnEbVz2qzM+lUBdf47G k9Xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=CIxixJZwpGLo0sVVb9Fs9GENGxOabLmSEhF/beS+CUs=; b=mJ7lSaSB6C7zK2LDkb5j4rrtudZRMN8TJaBgMOHUWzttJzMgYBuq6Zx3D/CvnOpd08 1JJTG/N/r2w9BR7Til1zufRwVPqcIln1Sg72zbTpxaV3c/FHacK1FvF+gCkto/9UyH66 cKgdcdjRM8DlnfUrXlhZFpNiQwdwKwXPvB8Vkcrl8VzK1BrnkzrzonVAEE1pAXJQUSQT cOCRmJa536Q9R7uZh2Lkc9Y190nXn9wTOO/sVbbcDT+SYkAAJsyxAUzF9SU8+nSMrBoP EzRcJeiUmFEVv73adLfe6MmZU18jvwiGHs4T8vsd0rqLDkdWybZB3je27PJvELxLivNt KzhQ== X-Gm-Message-State: ANhLgQ2MJBXXndpK9z0EKofOhDEgha14UbHvnzwPfnd/w5smArrq4pao 8klFEfRV+29eYxDLDTO/Ef2ZXqFk5pY= X-Google-Smtp-Source: ADFU+vuGslyW59Tsmb+QpDMJ17l2ulj/3fKIj/OktAout4Z7lHNr22T3gY6H4296Y2Icn4lHb6VkEw== X-Received: by 2002:aed:38ea:: with SMTP id k97mr3419602qte.327.1584631230596; Thu, 19 Mar 2020 08:20:30 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id v1sm1749730qtc.30.2020.03.19.08.20.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Mar 2020 08:20:29 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 02JFKSrs011148; Thu, 19 Mar 2020 15:20:28 GMT Subject: [PATCH RFC 01/11] SUNRPC: Adjust synopsis of xdr_buf_subsegment() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 19 Mar 2020 11:20:28 -0400 Message-ID: <20200319152028.16298.49075.stgit@klimt.1015granger.net> In-Reply-To: <20200319150136.16298.68813.stgit@klimt.1015granger.net> References: <20200319150136.16298.68813.stgit@klimt.1015granger.net> User-Agent: StGit/0.22 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org This makes it easier for callers to keep that buffer const as well. Signed-off-by: Chuck Lever --- include/linux/sunrpc/xdr.h | 3 ++- net/sunrpc/xdr.c | 5 ++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/include/linux/sunrpc/xdr.h b/include/linux/sunrpc/xdr.h index 8529d6e33137..3caa1fe75408 100644 --- a/include/linux/sunrpc/xdr.h +++ b/include/linux/sunrpc/xdr.h @@ -183,7 +183,8 @@ xdr_adjust_iovec(struct kvec *iov, __be32 *p) */ extern void xdr_shift_buf(struct xdr_buf *, size_t); extern void xdr_buf_from_iov(struct kvec *, struct xdr_buf *); -extern int xdr_buf_subsegment(struct xdr_buf *, struct xdr_buf *, unsigned int, unsigned int); +extern int xdr_buf_subsegment(const struct xdr_buf *buf, struct xdr_buf *subbuf, + unsigned int base, unsigned int len); extern int xdr_buf_read_mic(struct xdr_buf *, struct xdr_netobj *, unsigned int); extern int read_bytes_from_xdr_buf(struct xdr_buf *, unsigned int, void *, unsigned int); extern int write_bytes_to_xdr_buf(struct xdr_buf *, unsigned int, void *, unsigned int); diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c index e5497dc2475b..0f7f5d9058e2 100644 --- a/net/sunrpc/xdr.c +++ b/net/sunrpc/xdr.c @@ -1105,9 +1105,8 @@ EXPORT_SYMBOL_GPL(xdr_buf_from_iov); * * Returns -1 if base of length are out of bounds. */ -int -xdr_buf_subsegment(struct xdr_buf *buf, struct xdr_buf *subbuf, - unsigned int base, unsigned int len) +int xdr_buf_subsegment(const struct xdr_buf *buf, struct xdr_buf *subbuf, + unsigned int base, unsigned int len) { subbuf->buflen = subbuf->len = len; if (base < buf->head[0].iov_len) { From patchwork Thu Mar 19 15:20:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11447509 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9FBEF1864 for ; Thu, 19 Mar 2020 15:20:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7F6F120B1F for ; Thu, 19 Mar 2020 15:20:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="a+BMqUzQ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727473AbgCSPUh (ORCPT ); Thu, 19 Mar 2020 11:20:37 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:43359 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727286AbgCSPUh (ORCPT ); Thu, 19 Mar 2020 11:20:37 -0400 Received: by mail-qk1-f193.google.com with SMTP id x18so3378922qki.10; Thu, 19 Mar 2020 08:20:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=HMqB6B32gcebmHDDCkX4+KrgHIuxRBFp34c6lzvAzzU=; b=a+BMqUzQu4/bLLGeDQ+zSX6D8aRrZpujiAFBFAjrY9CXgfYt86/IfsTwHzLtRm7wcW eXcr8S2ZG2oWK9Jqlp1Yw19Uue/PtZNri7XI7h0XV0DUMLkocma6KyUqnttSn2zGP1hN ESuhmgzKxqu2eLtBFt8XQnSfLO0r2vabKqPc2IesZvUhow3qdKiq+WB9GjDRFHovdx/G q9aI7eImJB7yVSIP/TX7AHFGsdr9l8XOODPhurfEJd0lC4b+fJ2eysq8r+TPFpP0X/x8 i+BkdOLDK68Vl388TgRkabLcON9mpcKpest/WokqWtFOFf61UoJWWQLX3HZtPLy553bf 1eLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=HMqB6B32gcebmHDDCkX4+KrgHIuxRBFp34c6lzvAzzU=; b=KZc7taaNSkaWIJ4P3zBndiO9KE46RoI8gsRhptUa84K/MaUuIhbe2KOSp4dyrUDvyc /3USse8OrXgfu8VQfmdRtr+zHC/cNApsITuJ0fV5aDULS0JPNzMZktbR4YFpVJQOqIw5 KEsYp3G0P647383tXdl+s+LUmweMExdsOobnlLoVyZDnQbxrziVpFttTi7E3LAICmZmq vuOOHy9N01dB81YAVJArwYIyPLr6zCLrJXdwW1tyN3S+LFmuAwaWnm1sZG02m1UEYi2M eUJD1SoUHPpLuYvCkHHISUJRBhLTTBW8fIgWJWRp8l37wMG3cIWkFl87PO4Amlg8hLew eIBw== X-Gm-Message-State: ANhLgQ1TrlCZGSd59XIbl9E1DStS9etHkBr/6tqM0T37Q0CV0RaAkpU4 QWebzu3/FJTCg1GUo0M60Bt5LWveBCA= X-Google-Smtp-Source: ADFU+vtuiNfmEuAwKn19+gN8wSQ6P3/HVrmLKh/WrUqx5ka/9tast/eA7sVKZ8xLX3ppcTW4C1p3Ag== X-Received: by 2002:a05:620a:715:: with SMTP id 21mr2909485qkc.81.1584631235430; Thu, 19 Mar 2020 08:20:35 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id o14sm1694479qtq.12.2020.03.19.08.20.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Mar 2020 08:20:34 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 02JFKY6Y011151; Thu, 19 Mar 2020 15:20:34 GMT Subject: [PATCH RFC 02/11] svcrdma: Clean up RDMA Write path From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 19 Mar 2020 11:20:34 -0400 Message-ID: <20200319152034.16298.54031.stgit@klimt.1015granger.net> In-Reply-To: <20200319150136.16298.68813.stgit@klimt.1015granger.net> References: <20200319150136.16298.68813.stgit@klimt.1015granger.net> User-Agent: StGit/0.22 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Constify the xdr_buf argument to ensure the code here does not modify it, to enable callers to also use "const struct xdr_buf *". Also, rename the helper functions, which emit RDMA Writes, not RDMA Sends. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_rw.c | 36 +++++++++++++++--------------------- 1 file changed, 15 insertions(+), 21 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index bd7c195d872e..529f00a6896c 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -161,7 +161,7 @@ struct svc_rdma_write_info { __be32 *wi_segs; /* SGL constructor arguments */ - struct xdr_buf *wi_xdr; + const struct xdr_buf *wi_xdr; unsigned char *wi_base; unsigned int wi_next_off; @@ -369,7 +369,7 @@ static void svc_rdma_pagelist_to_sg(struct svc_rdma_write_info *info, struct svc_rdma_rw_ctxt *ctxt) { unsigned int sge_no, sge_bytes, page_off, page_no; - struct xdr_buf *xdr = info->wi_xdr; + const struct xdr_buf *xdr = info->wi_xdr; struct scatterlist *sg; struct page **page; @@ -470,27 +470,22 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, return -EIO; } -/* Send one of an xdr_buf's kvecs by itself. To send a Reply - * chunk, the whole RPC Reply is written back to the client. - * This function writes either the head or tail of the xdr_buf - * containing the Reply. +/* RDMA Write an iov. */ -static int svc_rdma_send_xdr_kvec(struct svc_rdma_write_info *info, - struct kvec *vec) +static int svc_rdma_write_vec(struct svc_rdma_write_info *info, + const struct kvec *vec) { info->wi_base = vec->iov_base; return svc_rdma_build_writes(info, svc_rdma_vec_to_sg, vec->iov_len); } -/* Send an xdr_buf's page list by itself. A Write chunk is just - * the page list. A Reply chunk is @xdr's head, page list, and - * tail. This function is shared between the two types of chunk. +/* RDMA Write an xdr_buf's page list by itself. */ -static int svc_rdma_send_xdr_pagelist(struct svc_rdma_write_info *info, - struct xdr_buf *xdr, - unsigned int offset, - unsigned long length) +static int svc_rdma_write_pages(struct svc_rdma_write_info *info, + const struct xdr_buf *xdr, + unsigned int offset, + unsigned long length) { info->wi_xdr = xdr; info->wi_next_off = offset - xdr->head[0].iov_len; @@ -527,7 +522,7 @@ int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, __be32 *wr_ch, if (!info) return -ENOMEM; - ret = svc_rdma_send_xdr_pagelist(info, xdr, offset, length); + ret = svc_rdma_write_pages(info, xdr, offset, length); if (ret < 0) goto out_err; @@ -567,7 +562,7 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, if (!info) return -ENOMEM; - ret = svc_rdma_send_xdr_kvec(info, &xdr->head[0]); + ret = svc_rdma_write_vec(info, &xdr->head[0]); if (ret < 0) goto out_err; consumed = xdr->head[0].iov_len; @@ -576,16 +571,15 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, * client did not provide Write chunks. */ if (!rctxt->rc_write_list && xdr->page_len) { - ret = svc_rdma_send_xdr_pagelist(info, xdr, - xdr->head[0].iov_len, - xdr->page_len); + ret = svc_rdma_write_pages(info, xdr, xdr->head[0].iov_len, + xdr->page_len); if (ret < 0) goto out_err; consumed += xdr->page_len; } if (xdr->tail[0].iov_len) { - ret = svc_rdma_send_xdr_kvec(info, &xdr->tail[0]); + ret = svc_rdma_write_vec(info, &xdr->tail[0]); if (ret < 0) goto out_err; consumed += xdr->tail[0].iov_len; From patchwork Thu Mar 19 15:20:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11447513 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7E26F1864 for ; Thu, 19 Mar 2020 15:20:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5D37620658 for ; Thu, 19 Mar 2020 15:20:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GGgAJxVr" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727521AbgCSPUm (ORCPT ); Thu, 19 Mar 2020 11:20:42 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:43716 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727001AbgCSPUm (ORCPT ); Thu, 19 Mar 2020 11:20:42 -0400 Received: by mail-qt1-f196.google.com with SMTP id l13so2077713qtv.10; Thu, 19 Mar 2020 08:20:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=OkrrehG9xo9jvtaQ6siMXgY5KHjfnEpiFk4fD6yvyD8=; b=GGgAJxVrfLoQCNtOptRaBbgsKkS0kgKSsfEgGev3l3U25wVzW5Rz1eP4w6ZBCBGKyz x7RupbF3KETUPYUUFer7xM1P4sO0ZqfZJivXXCitR5cMKZLbKtgFdfIwVbfu8DdJy6/6 PbmYck4uhHIJ6ywBAWXkbcnmxZpUnIZHqvNhe//3A4D5HP8Dc2E6D+rsPiQl4kmJd9kV RlUqX7ILmE3TdTvgyHKrsCL2C+wicYURIAafcd6Agd6svNMFOUPEWGN34RzqVHKkzN6K TsA8fg215LReRLIeSAx2dfIPlx1SFI8MFKET92LsaUiNf754Uc2UMNLCD8XvL8ff56jZ 2Vgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=OkrrehG9xo9jvtaQ6siMXgY5KHjfnEpiFk4fD6yvyD8=; b=LkGu6rJBeZM10SOTiVdvlbiZR6x5niAxkuEQ/LEvZWn0kuOMh5iPmyCyfGC4cjeJuC Cq0Vm17cS0+uGdaJdOJSIZJi9JaPRUlN8XujPQseEcpeX8Vbcfn3TNGHqxczNF1bNDtF phq3yOyPn2YHGNGZkHGCODByOBNxTT6JjlgQDacuGYgfJAbBQU93WqvrgEY5vkjwKMRk WDrOBp4SVh4hVYtKUPH5KZdsz191ElN6Re+s9uOBzTQtt+UXUtTfu2mKJch7C6nZoaIk SF52yBjs1DhEtn3JPsfp81vMuDHxX+HBosG22C28fTOqc5uLgQae0QA4OvXvDeNvmWxp d58A== X-Gm-Message-State: ANhLgQ1r5kCCm/bH1zD4Zj0PSs+KsDeO/bVal3oLGwVnhahZiNimo3W4 VtGxDlJtTy9aC1whLs9pW4g5tG+NE5M= X-Google-Smtp-Source: ADFU+vtyS+IrHxPmV2hKwQWswW8oGhgAsSPHufmaQBvnuL3VwV9sBdNT2+kTivPlNVf2BPnkcj5McA== X-Received: by 2002:ac8:3141:: with SMTP id h1mr3509780qtb.108.1584631240889; Thu, 19 Mar 2020 08:20:40 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id g6sm1668807qtd.85.2020.03.19.08.20.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Mar 2020 08:20:40 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 02JFKdvl011154; Thu, 19 Mar 2020 15:20:39 GMT Subject: [PATCH RFC 03/11] NFSD: Invoke svc_encode_read_payload in "read" NFSD encoders From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 19 Mar 2020 11:20:39 -0400 Message-ID: <20200319152039.16298.94469.stgit@klimt.1015granger.net> In-Reply-To: <20200319150136.16298.68813.stgit@klimt.1015granger.net> References: <20200319150136.16298.68813.stgit@klimt.1015granger.net> User-Agent: StGit/0.22 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Have the NFSD encoders annotate the boundaries of every direct-data-placement eligible READ data payload. Then change svcrdma to use that annotation instead of the xdr->page_len when handling Write chunks. For NFSv4 on RDMA, that enables the ability to recognize multiple READ payloads per compound. Next step is to support multiple Write chunks. Signed-off-by: Chuck Lever --- fs/nfsd/nfs3xdr.c | 4 ++++ fs/nfsd/nfs4xdr.c | 3 +++ fs/nfsd/nfsxdr.c | 4 ++++ net/sunrpc/xprtrdma/svc_rdma_sendto.c | 24 +++++++----------------- 4 files changed, 18 insertions(+), 17 deletions(-) diff --git a/fs/nfsd/nfs3xdr.c b/fs/nfsd/nfs3xdr.c index aae514d40b64..8c272efbc94e 100644 --- a/fs/nfsd/nfs3xdr.c +++ b/fs/nfsd/nfs3xdr.c @@ -712,6 +712,8 @@ nfs3svc_encode_readlinkres(struct svc_rqst *rqstp, __be32 *p) *p = 0; rqstp->rq_res.tail[0].iov_len = 4 - (resp->len&3); } + svc_encode_read_payload(rqstp, rqstp->rq_res.head[0].iov_len, + resp->len); return 1; } else return xdr_ressize_check(rqstp, p); @@ -737,6 +739,8 @@ nfs3svc_encode_readres(struct svc_rqst *rqstp, __be32 *p) *p = 0; rqstp->rq_res.tail[0].iov_len = 4 - (resp->count & 3); } + svc_encode_read_payload(rqstp, rqstp->rq_res.head[0].iov_len, + resp->count); return 1; } else return xdr_ressize_check(rqstp, p); diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c index 996ac01ee977..9c2cfddcc29c 100644 --- a/fs/nfsd/nfs4xdr.c +++ b/fs/nfsd/nfs4xdr.c @@ -3547,6 +3547,8 @@ static __be32 nfsd4_encode_splice_read( buf->page_len = 0; return nfserr; } + svc_encode_read_payload(read->rd_rqstp, buf->head[0].iov_len, + maxcount); *(p++) = htonl(eof); *(p++) = htonl(maxcount); @@ -3713,6 +3715,7 @@ nfsd4_encode_readlink(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd xdr_truncate_encode(xdr, length_offset); return nfserr; } + svc_encode_read_payload(readlink->rl_rqstp, length_offset, maxcount); wire_count = htonl(maxcount); write_bytes_to_xdr_buf(xdr->buf, length_offset, &wire_count, 4); diff --git a/fs/nfsd/nfsxdr.c b/fs/nfsd/nfsxdr.c index b51fe515f06f..98ea417042a6 100644 --- a/fs/nfsd/nfsxdr.c +++ b/fs/nfsd/nfsxdr.c @@ -462,6 +462,8 @@ nfssvc_encode_readlinkres(struct svc_rqst *rqstp, __be32 *p) *p = 0; rqstp->rq_res.tail[0].iov_len = 4 - (resp->len&3); } + svc_encode_read_payload(rqstp, rqstp->rq_res.head[0].iov_len, + resp->len); return 1; } @@ -482,6 +484,8 @@ nfssvc_encode_readres(struct svc_rqst *rqstp, __be32 *p) *p = 0; rqstp->rq_res.tail[0].iov_len = 4 - (resp->count&3); } + svc_encode_read_payload(rqstp, rqstp->rq_res.head[0].iov_len, + resp->count); return 1; } diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 90cba3058f04..0e6372a13018 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -445,7 +445,6 @@ static ssize_t svc_rdma_encode_write_chunk(__be32 *src, * svc_rdma_encode_write_list - Encode RPC Reply's Write chunk list * @rctxt: Reply context with information about the RPC Call * @sctxt: Send context for the RPC Reply - * @length: size in bytes of the payload in the first Write chunk * * The client provides a Write chunk list in the Call message. Fill * in the segments in the first Write chunk in the Reply's transport @@ -462,12 +461,12 @@ static ssize_t svc_rdma_encode_write_chunk(__be32 *src, */ static ssize_t svc_rdma_encode_write_list(const struct svc_rdma_recv_ctxt *rctxt, - struct svc_rdma_send_ctxt *sctxt, - unsigned int length) + struct svc_rdma_send_ctxt *sctxt) { ssize_t len, ret; - ret = svc_rdma_encode_write_chunk(rctxt->rc_write_list, sctxt, length); + ret = svc_rdma_encode_write_chunk(rctxt->rc_write_list, sctxt, + rctxt->rc_read_payload_length); if (ret < 0) return ret; len = ret; @@ -890,21 +889,12 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) goto err0; if (wr_lst) { /* XXX: Presume the client sent only one Write chunk */ - unsigned long offset; - unsigned int length; - - if (rctxt->rc_read_payload_length) { - offset = rctxt->rc_read_payload_offset; - length = rctxt->rc_read_payload_length; - } else { - offset = xdr->head[0].iov_len; - length = xdr->page_len; - } - ret = svc_rdma_send_write_chunk(rdma, wr_lst, xdr, offset, - length); + ret = svc_rdma_send_write_chunk(rdma, wr_lst, xdr, + rctxt->rc_read_payload_offset, + rctxt->rc_read_payload_length); if (ret < 0) goto err2; - if (svc_rdma_encode_write_list(rctxt, sctxt, length) < 0) + if (svc_rdma_encode_write_list(rctxt, sctxt) < 0) goto err0; } else { if (xdr_stream_encode_item_absent(&sctxt->sc_stream) < 0) From patchwork Thu Mar 19 15:20:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11447517 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 06ECB1864 for ; Thu, 19 Mar 2020 15:20:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DADB020B1F for ; Thu, 19 Mar 2020 15:20:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gg4FsZzB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727550AbgCSPUt (ORCPT ); Thu, 19 Mar 2020 11:20:49 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:34405 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727001AbgCSPUt (ORCPT ); Thu, 19 Mar 2020 11:20:49 -0400 Received: by mail-qk1-f194.google.com with SMTP id f3so3465044qkh.1; Thu, 19 Mar 2020 08:20:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=vN2rA0AcWEDMwd9QdhqOqb4NZAXp+ArWLgjdCEOp1tc=; b=gg4FsZzBRQZbimoD6T0dQwLD6gJodg4OUqUwzKyM3dX3LhaiaTUQzOxGd2ROLRO0BM BW/OcA4AXk7XYebfaaA7elk1yaup/kNDpx8mDOANgZErEc0i4BzwXiAhVSOHCMyzylMC +eK8A7S10nuK6m/ArYbku5dM6QKfYNQLzUHIqpGSUOc9DVuQZQdjGQz7NhDKLz1USvDY c4yeyJ3CprYPImyryS3wNJu3+x9lZFbjwyPIYuMAFGHe8AT2x+vAIJz4VHoqX7aYiCcL XshSoDq6hw54LIwG6XU6DyCg4nhCeMZw/91gMwVsksoHE8g7TThjiIPzkhEZJJ+T52pP OsoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=vN2rA0AcWEDMwd9QdhqOqb4NZAXp+ArWLgjdCEOp1tc=; b=Fq4ZQbSNQu2qQzMSDNXNcLdEIqAtiulWeAathTLrevcleV5jf5rP2Pmfyzt9mJq4EO iYWpp47gxyZf3wnMbF7jPRGsOPwSHxjYTjLXVvQrHUKHlKntGBliThHGTFTC6emihSTC tYJGQsnIvyYsAIUDaEMJ0oODe0WNDL/jeNuNc8EKbLXcpOAx/L4siQYUkdmO3tKp1DNT 96940bcpPSRCwDvdeDPTjIVdw9ezjSPdfNyU5Q/vja/IJ+gSiAudshBEOENusN2u88qz qJ0WB4uzMyuvI/ORgPwslH2OiRGzmD0Mhb3cRCuMPLtG0KGzVbKB4WFXJO0TCB0SPuHA 8Vxw== X-Gm-Message-State: ANhLgQ3+bRgfCZKUEb7D0kKxIIvYOMARDLEHTuJOOOaJ0JE+9hSm8fzH Qepc+TlpPRL956MqWrUW76fzYicNy/k= X-Google-Smtp-Source: ADFU+vt+zZDGE3+yVq5SVz6fUzEXc3I63oFWmqdoIFrHKBRkJpJtirfHDnJJms+KN9dnULf96oeMXg== X-Received: by 2002:a37:391:: with SMTP id 139mr3475752qkd.6.1584631246160; Thu, 19 Mar 2020 08:20:46 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id n46sm1827607qtb.48.2020.03.19.08.20.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Mar 2020 08:20:45 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 02JFKirW011157; Thu, 19 Mar 2020 15:20:44 GMT Subject: [PATCH RFC 04/11] svcrdma: Post RDMA Writes while XDR encoding replies From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 19 Mar 2020 11:20:44 -0400 Message-ID: <20200319152044.16298.90383.stgit@klimt.1015granger.net> In-Reply-To: <20200319150136.16298.68813.stgit@klimt.1015granger.net> References: <20200319150136.16298.68813.stgit@klimt.1015granger.net> User-Agent: StGit/0.22 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The only RPC/RDMA ordering requirement between RDMA Writes and RDMA Sends is that the responder must post the Writes on the Send queue before posting the Send that conveys the RPC Reply for that Write payload. The Linux NFS server implementation now has a transport method that can post READ Payload Writes earlier than svc_rdma_sendto: ->xpo_read_payload. Goals: - Get RDMA Writes going earlier so they are more likely to be complete at the remote end before the Send completes. - Allow more parallelism when dispatching RDMA operations by posting RDMA Writes before taking xpt_mutex. Some care must be taken with pulled-up Replies. We don't want to push the Write chunk and then send the same payload data via Send. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 14 +++-- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 6 +- net/sunrpc/xprtrdma/svc_rdma_rw.c | 94 ++++++++++++++++++++++++------- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 84 +++++++++++++++------------- 4 files changed, 128 insertions(+), 70 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 78fe2ac6dc6c..49864264a9a5 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -125,6 +125,12 @@ enum { #define RPCSVC_MAXPAYLOAD_RDMA RPCSVC_MAXPAYLOAD +struct svc_rdma_payload { + __be32 *rp_chunk; + unsigned int rp_offset; + unsigned int rp_length; +}; + struct svc_rdma_recv_ctxt { struct llist_node rc_node; struct list_head rc_list; @@ -139,10 +145,8 @@ struct svc_rdma_recv_ctxt { unsigned int rc_page_count; unsigned int rc_hdr_count; u32 rc_inv_rkey; - __be32 *rc_write_list; + struct svc_rdma_payload rc_read_payload; __be32 *rc_reply_chunk; - unsigned int rc_read_payload_offset; - unsigned int rc_read_payload_length; struct page *rc_pages[RPCSVC_MAXPAGES]; }; @@ -178,9 +182,7 @@ extern int svc_rdma_recv_read_chunk(struct svcxprt_rdma *rdma, struct svc_rqst *rqstp, struct svc_rdma_recv_ctxt *head, __be32 *p); extern int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, - __be32 *wr_ch, struct xdr_buf *xdr, - unsigned int offset, - unsigned long length); + __be32 *wr_ch, const struct xdr_buf *xdr); extern int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, const struct svc_rdma_recv_ctxt *rctxt, struct xdr_buf *xdr); diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index 54469b72b25f..3e02dfa351ff 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -193,7 +193,7 @@ svc_rdma_recv_ctxt_get(struct svcxprt_rdma *rdma) out: ctxt->rc_page_count = 0; - ctxt->rc_read_payload_length = 0; + ctxt->rc_read_payload.rp_length = 0; return ctxt; out_empty: @@ -479,7 +479,7 @@ static bool xdr_check_write_list(struct svc_rdma_recv_ctxt *rctxt) p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); if (!p) return false; - rctxt->rc_write_list = p; + rctxt->rc_read_payload.rp_chunk = p; while (*p != xdr_zero) { if (!xdr_check_write_chunk(rctxt, MAX_BYTES_WRITE_CHUNK)) return false; @@ -489,7 +489,7 @@ static bool xdr_check_write_list(struct svc_rdma_recv_ctxt *rctxt) return false; } if (!chcount) - rctxt->rc_write_list = NULL; + rctxt->rc_read_payload.rp_chunk = NULL; return chcount < 2; } diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 529f00a6896c..065dfd8b5b22 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -470,19 +470,39 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, return -EIO; } -/* RDMA Write an iov. +/** + * svc_rdma_iov_write - Construct RDMA Writes from an iov + * @info: pointer to write arguments + * @iov: kvec to write + * + * Returns: + * On succes, returns zero + * %-E2BIG if the client-provided Write chunk is too small + * %-ENOMEM if a resource has been exhausted + * %-EIO if an rdma-rw error occurred */ -static int svc_rdma_write_vec(struct svc_rdma_write_info *info, - const struct kvec *vec) +static int svc_rdma_iov_write(struct svc_rdma_write_info *info, + const struct kvec *iov) { - info->wi_base = vec->iov_base; + info->wi_base = iov->iov_base; return svc_rdma_build_writes(info, svc_rdma_vec_to_sg, - vec->iov_len); + iov->iov_len); } -/* RDMA Write an xdr_buf's page list by itself. +/** + * svc_rdma_pages_write - Construct RDMA Writes from pages + * @info: pointer to write arguments + * @xdr: xdr_buf with pages to write + * @offset: offset into the content of @xdr + * @length: number of bytes to write + * + * Returns: + * On succes, returns zero + * %-E2BIG if the client-provided Write chunk is too small + * %-ENOMEM if a resource has been exhausted + * %-EIO if an rdma-rw error occurred */ -static int svc_rdma_write_pages(struct svc_rdma_write_info *info, +static int svc_rdma_pages_write(struct svc_rdma_write_info *info, const struct xdr_buf *xdr, unsigned int offset, unsigned long length) @@ -493,13 +513,49 @@ static int svc_rdma_write_pages(struct svc_rdma_write_info *info, length); } +/** + * svc_rdma_xb_write - Construct RDMA Writes to write an xdr_buf + * @xdr: xdr_buf to write + * @info: pointer to write arguments + * + * Returns: + * On succes, returns zero + * %-E2BIG if the client-provided Write chunk is too small + * %-ENOMEM if a resource has been exhausted + * %-EIO if an rdma-rw error occurred + */ +static int svc_rdma_xb_write(const struct xdr_buf *xdr, + struct svc_rdma_write_info *info) +{ + int ret; + + if (xdr->head[0].iov_len) { + ret = svc_rdma_iov_write(info, &xdr->head[0]); + if (ret < 0) + return ret; + } + + if (xdr->page_len) { + ret = svc_rdma_pages_write(info, xdr, xdr->head[0].iov_len, + xdr->page_len); + if (ret < 0) + return ret; + } + + if (xdr->tail[0].iov_len) { + ret = svc_rdma_iov_write(info, &xdr->tail[0]); + if (ret < 0) + return ret; + } + + return xdr->len; +} + /** * svc_rdma_send_write_chunk - Write all segments in a Write chunk * @rdma: controlling RDMA transport * @wr_ch: Write chunk provided by client * @xdr: xdr_buf containing the data payload - * @offset: payload's byte offset in @xdr - * @length: size of payload, in bytes * * Returns a non-negative number of bytes the chunk consumed, or * %-E2BIG if the payload was larger than the Write chunk, @@ -509,21 +565,17 @@ static int svc_rdma_write_pages(struct svc_rdma_write_info *info, * %-EIO if rdma_rw initialization failed (DMA mapping, etc). */ int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, __be32 *wr_ch, - struct xdr_buf *xdr, - unsigned int offset, unsigned long length) + const struct xdr_buf *xdr) { struct svc_rdma_write_info *info; int ret; - if (!length) - return 0; - info = svc_rdma_write_info_alloc(rdma, wr_ch); if (!info) return -ENOMEM; - ret = svc_rdma_write_pages(info, xdr, offset, length); - if (ret < 0) + ret = svc_rdma_xb_write(xdr, info); + if (ret != xdr->len) goto out_err; ret = svc_rdma_post_chunk_ctxt(&info->wi_cc); @@ -531,7 +583,7 @@ int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, __be32 *wr_ch, goto out_err; trace_svcrdma_send_write_chunk(xdr->page_len); - return length; + return xdr->len; out_err: svc_rdma_write_info_free(info); @@ -562,7 +614,7 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, if (!info) return -ENOMEM; - ret = svc_rdma_write_vec(info, &xdr->head[0]); + ret = svc_rdma_iov_write(info, &xdr->head[0]); if (ret < 0) goto out_err; consumed = xdr->head[0].iov_len; @@ -570,8 +622,8 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, /* Send the page list in the Reply chunk only if the * client did not provide Write chunks. */ - if (!rctxt->rc_write_list && xdr->page_len) { - ret = svc_rdma_write_pages(info, xdr, xdr->head[0].iov_len, + if (!rctxt->rc_read_payload.rp_chunk && xdr->page_len) { + ret = svc_rdma_pages_write(info, xdr, xdr->head[0].iov_len, xdr->page_len); if (ret < 0) goto out_err; @@ -579,7 +631,7 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, } if (xdr->tail[0].iov_len) { - ret = svc_rdma_write_vec(info, &xdr->tail[0]); + ret = svc_rdma_iov_write(info, &xdr->tail[0]); if (ret < 0) goto out_err; consumed += xdr->tail[0].iov_len; diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 0e6372a13018..94adeb2a43cf 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -395,9 +395,8 @@ static ssize_t svc_rdma_encode_write_segment(__be32 *src, /** * svc_rdma_encode_write_chunk - Encode one Write chunk - * @src: matching Write chunk in the RPC Call header * @sctxt: Send context for the RPC Reply - * @remaining: size in bytes of the payload in the Write chunk + * @payload: Write chunk information to encode * * Copy a Write chunk from the Call transport header to the * Reply transport header. Update each segment's length field @@ -408,14 +407,16 @@ static ssize_t svc_rdma_encode_write_segment(__be32 *src, * that was consumed by the Write chunk * %-EMSGSIZE on XDR buffer overflow */ -static ssize_t svc_rdma_encode_write_chunk(__be32 *src, - struct svc_rdma_send_ctxt *sctxt, - unsigned int remaining) +static ssize_t svc_rdma_encode_write_chunk(struct svc_rdma_send_ctxt *sctxt, + const struct svc_rdma_payload *payload) { - unsigned int i, nsegs; + unsigned int i, nsegs, remaining; ssize_t len, ret; + __be32 *src; len = 0; + src = payload->rp_chunk; + remaining = payload->rp_length; trace_svcrdma_encode_write_chunk(remaining); src++; @@ -465,11 +466,14 @@ svc_rdma_encode_write_list(const struct svc_rdma_recv_ctxt *rctxt, { ssize_t len, ret; - ret = svc_rdma_encode_write_chunk(rctxt->rc_write_list, sctxt, - rctxt->rc_read_payload_length); - if (ret < 0) - return ret; - len = ret; + len = 0; + if (rctxt->rc_read_payload.rp_chunk) { + ret = svc_rdma_encode_write_chunk(sctxt, + &rctxt->rc_read_payload); + if (ret < 0) + return ret; + len += ret; + } /* Terminate the Write list */ ret = xdr_stream_encode_item_absent(&sctxt->sc_stream); @@ -498,8 +502,12 @@ svc_rdma_encode_reply_chunk(const struct svc_rdma_recv_ctxt *rctxt, struct svc_rdma_send_ctxt *sctxt, unsigned int length) { - return svc_rdma_encode_write_chunk(rctxt->rc_reply_chunk, sctxt, - length); + struct svc_rdma_payload payload = { + .rp_chunk = rctxt->rc_reply_chunk, + .rp_length = length, + }; + + return svc_rdma_encode_write_chunk(sctxt, &payload); } static int svc_rdma_dma_map_page(struct svcxprt_rdma *rdma, @@ -553,11 +561,13 @@ static bool svc_rdma_pull_up_needed(struct svcxprt_rdma *rdma, const struct svc_rdma_recv_ctxt *rctxt, struct xdr_buf *xdr) { + bool read_payload_present = rctxt && rctxt->rc_read_payload.rp_chunk; int elements; /* For small messages, copying bytes is cheaper than DMA mapping. */ - if (sctxt->sc_hdrbuf.len + xdr->len < RPCRDMA_PULLUP_THRESH) + if (!read_payload_present && + sctxt->sc_hdrbuf.len + xdr->len < RPCRDMA_PULLUP_THRESH) return true; /* Check whether the xdr_buf has more elements than can @@ -567,7 +577,7 @@ static bool svc_rdma_pull_up_needed(struct svcxprt_rdma *rdma, elements = 1; /* xdr->pages */ - if (!rctxt || !rctxt->rc_write_list) { + if (!read_payload_present) { unsigned int remaining; unsigned long pageoff; @@ -615,7 +625,7 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, tailbase = xdr->tail[0].iov_base; taillen = xdr->tail[0].iov_len; - if (rctxt && rctxt->rc_write_list) { + if (rctxt && rctxt->rc_read_payload.rp_chunk) { u32 xdrpad; xdrpad = xdr_pad_size(xdr->page_len); @@ -700,7 +710,7 @@ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, * have added XDR padding in the tail buffer, and that * should not be included inline. */ - if (rctxt && rctxt->rc_write_list) { + if (rctxt && rctxt->rc_read_payload.rp_chunk) { base = xdr->tail[0].iov_base; len = xdr->tail[0].iov_len; xdr_pad = xdr_pad_size(xdr->page_len); @@ -858,9 +868,7 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) container_of(xprt, struct svcxprt_rdma, sc_xprt); struct svc_rdma_recv_ctxt *rctxt = rqstp->rq_xprt_ctxt; __be32 *rdma_argp = rctxt->rc_recv_buf; - __be32 *wr_lst = rctxt->rc_write_list; __be32 *rp_ch = rctxt->rc_reply_chunk; - struct xdr_buf *xdr = &rqstp->rq_res; struct svc_rdma_send_ctxt *sctxt; __be32 *p; int ret; @@ -887,19 +895,8 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) if (svc_rdma_encode_read_list(sctxt) < 0) goto err0; - if (wr_lst) { - /* XXX: Presume the client sent only one Write chunk */ - ret = svc_rdma_send_write_chunk(rdma, wr_lst, xdr, - rctxt->rc_read_payload_offset, - rctxt->rc_read_payload_length); - if (ret < 0) - goto err2; - if (svc_rdma_encode_write_list(rctxt, sctxt) < 0) - goto err0; - } else { - if (xdr_stream_encode_item_absent(&sctxt->sc_stream) < 0) - goto err0; - } + if (svc_rdma_encode_write_list(rctxt, sctxt) < 0) + goto err0; if (rp_ch) { ret = svc_rdma_send_reply_chunk(rdma, rctxt, &rqstp->rq_res); if (ret < 0) @@ -946,23 +943,30 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) * @offset: payload's byte offset in @xdr * @length: size of payload, in bytes * - * Returns zero on success. - * - * For the moment, just record the xdr_buf location of the READ - * payload. svc_rdma_sendto will use that location later when - * we actually send the payload. + * Returns: + * %-EMSGSIZE on XDR buffer overflow */ int svc_rdma_read_payload(struct svc_rqst *rqstp, unsigned int offset, unsigned int length) { struct svc_rdma_recv_ctxt *rctxt = rqstp->rq_xprt_ctxt; + struct xdr_buf uninitialized_var(subbuf); + struct svcxprt_rdma *rdma; + + if (!rctxt->rc_read_payload.rp_chunk || !length) + return 0; /* XXX: Just one READ payload slot for now, since our * transport implementation currently supports only one * Write chunk. */ - rctxt->rc_read_payload_offset = offset; - rctxt->rc_read_payload_length = length; + rctxt->rc_read_payload.rp_offset = offset; + rctxt->rc_read_payload.rp_length = length; - return 0; + if (xdr_buf_subsegment(&rqstp->rq_res, &subbuf, offset, length)) + return -EMSGSIZE; + + rdma = container_of(rqstp->rq_xprt, struct svcxprt_rdma, sc_xprt); + return svc_rdma_send_write_chunk(rdma, rctxt->rc_read_payload.rp_chunk, + &subbuf); } From patchwork Thu Mar 19 15:20:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11447521 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4A44B1864 for ; Thu, 19 Mar 2020 15:20:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2A3A920B1F for ; Thu, 19 Mar 2020 15:20:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OcMozvFB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727217AbgCSPUy (ORCPT ); Thu, 19 Mar 2020 11:20:54 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:33694 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727001AbgCSPUy (ORCPT ); Thu, 19 Mar 2020 11:20:54 -0400 Received: by mail-qk1-f196.google.com with SMTP id p6so3464555qkm.0; Thu, 19 Mar 2020 08:20:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=pS9rupJUqhMQcoM6W8MhtxI26MZJvmGSodgflnz/efU=; b=OcMozvFBQ2gs9DPc9WYC6ozueLLbwrPFU1wM+ZQGMwEKilLUM5dX7BujB4euZKTd2U sfvI6//KnmwYMAkunCVlUOMnpRvYhtYkty65CUufHRsTQduL4rIuOSL7nw04mNL8UW0a YelRWBceseYOx33VVMeGsc36paq2Murs4G5Mf42rQHtoyiA6G06ydLK5YRMTW7rWEtcG QZ+YoxHqK6uxFAunMzyS5BFimJU3g33fghIEEh5bOezQ+cLsguRwQHHCczfHfRKCDcBV burERjqROVlauWVgF2/KgyzZXKisz9ZXSZ5kPpFJ9Cgp/xb/irs0+8V42Vncc014xyMV FLQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=pS9rupJUqhMQcoM6W8MhtxI26MZJvmGSodgflnz/efU=; b=OfFmAHY6hoinNaPCQZBZnKHVDTD48ubjQIQ/NoCirVoe0jatptaejVP4VBlyDdS2RT I0blVeHFvCWCbSZxx97lk9gqomVjOHwHAj3rDo9/IJ4gL4eJTWNgj1N1PeQ46jdLqtjB C/rT1PiaJUUKEvET2UP4g6rPtvojKFw5reX1LjZudM69ni7MLIubPbZmNJVJPA5QqpeZ Pbmmi4ZySHut4HUx56astdxyzK1BG1WTZt/Q5NHeB0Xjk0tI7/ziI9S7vbN03PW5jSbU uOAs9T0RUEtk99Lomk6ghETis46VeBNahgui2mdqVdnO0G8ckZyfwQuAqQfjgIpwHVS4 3y5Q== X-Gm-Message-State: ANhLgQ3FAAW2nXucGYMs9bueMHAgrUUaWWeCt/3f1tJcDUbG31qp71+Z AqLQt/dPqO7h8jOfb/03uDgwZ2ARUJI= X-Google-Smtp-Source: ADFU+vuKonM8drxTAruxlk09RD+QqebwCpC7ChQFPnUbzpD9m+X4Pis1Jf5+7ALEa+xBcij7l/iPgg== X-Received: by 2002:ae9:efd1:: with SMTP id d200mr3090642qkg.79.1584631251314; Thu, 19 Mar 2020 08:20:51 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id 31sm1747520qta.56.2020.03.19.08.20.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Mar 2020 08:20:50 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 02JFKnx1011160; Thu, 19 Mar 2020 15:20:49 GMT Subject: [PATCH RFC 05/11] svcrdma: Clean up svc_rdma_encode_reply_chunk() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 19 Mar 2020 11:20:49 -0400 Message-ID: <20200319152049.16298.55010.stgit@klimt.1015granger.net> In-Reply-To: <20200319150136.16298.68813.stgit@klimt.1015granger.net> References: <20200319150136.16298.68813.stgit@klimt.1015granger.net> User-Agent: StGit/0.22 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Clean up: Match the control flow of svc_rdma_encode_write_list(). Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_rw.c | 3 +++ net/sunrpc/xprtrdma/svc_rdma_sendto.c | 25 ++++++++++++------------- 2 files changed, 15 insertions(+), 13 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 065dfd8b5b22..080a55456911 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -610,6 +610,9 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, struct svc_rdma_write_info *info; int consumed, ret; + if (!rctxt->rc_reply_chunk) + return 0; + info = svc_rdma_write_info_alloc(rdma, rctxt->rc_reply_chunk); if (!info) return -ENOMEM; diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 94adeb2a43cf..89bc8db0289e 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -507,7 +507,10 @@ svc_rdma_encode_reply_chunk(const struct svc_rdma_recv_ctxt *rctxt, .rp_length = length, }; - return svc_rdma_encode_write_chunk(sctxt, &payload); + if (!rctxt->rc_reply_chunk) + return xdr_stream_encode_item_absent(&sctxt->sc_stream); + else + return svc_rdma_encode_write_chunk(sctxt, &payload); } static int svc_rdma_dma_map_page(struct svcxprt_rdma *rdma, @@ -868,7 +871,6 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) container_of(xprt, struct svcxprt_rdma, sc_xprt); struct svc_rdma_recv_ctxt *rctxt = rqstp->rq_xprt_ctxt; __be32 *rdma_argp = rctxt->rc_recv_buf; - __be32 *rp_ch = rctxt->rc_reply_chunk; struct svc_rdma_send_ctxt *sctxt; __be32 *p; int ret; @@ -888,25 +890,22 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) rpcrdma_fixed_maxsz * sizeof(*p)); if (!p) goto err0; + + ret = svc_rdma_send_reply_chunk(rdma, rctxt, &rqstp->rq_res); + if (ret < 0) + goto err2; + *p++ = *rdma_argp; *p++ = *(rdma_argp + 1); *p++ = rdma->sc_fc_credits; - *p = rp_ch ? rdma_nomsg : rdma_msg; + *p = rctxt->rc_reply_chunk ? rdma_nomsg : rdma_msg; if (svc_rdma_encode_read_list(sctxt) < 0) goto err0; if (svc_rdma_encode_write_list(rctxt, sctxt) < 0) goto err0; - if (rp_ch) { - ret = svc_rdma_send_reply_chunk(rdma, rctxt, &rqstp->rq_res); - if (ret < 0) - goto err2; - if (svc_rdma_encode_reply_chunk(rctxt, sctxt, ret) < 0) - goto err0; - } else { - if (xdr_stream_encode_item_absent(&sctxt->sc_stream) < 0) - goto err0; - } + if (svc_rdma_encode_reply_chunk(rctxt, sctxt, ret) < 0) + goto err0; ret = svc_rdma_send_reply_msg(rdma, sctxt, rctxt, rqstp); if (ret < 0) From patchwork Thu Mar 19 15:20:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11447525 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 967B71864 for ; Thu, 19 Mar 2020 15:21:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7664720B1F for ; Thu, 19 Mar 2020 15:21:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jkopDnI8" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727001AbgCSPU7 (ORCPT ); Thu, 19 Mar 2020 11:20:59 -0400 Received: from mail-qv1-f66.google.com ([209.85.219.66]:44814 "EHLO mail-qv1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727555AbgCSPU7 (ORCPT ); Thu, 19 Mar 2020 11:20:59 -0400 Received: by mail-qv1-f66.google.com with SMTP id w5so1171040qvp.11; Thu, 19 Mar 2020 08:20:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=Gp44ocBSq1NN/nreEQ4cJNlnHwu6GLp98VU98gHXy44=; b=jkopDnI8MWukOdIxSpNzZzZRP89skf3IqE739QmiQA2HC2+Pf1N+GT/uuB5Yvuq0pX trKQxR3isUBopQC8Z6PXVMgZxGUUACKGxdPTzZGA+oSWH6pNKJQL8aO/GdDMoC8Qe5MC 4Laa33i9osWfEtuCb/R1tx3ejI54pMSUHGEh9R7rDmPADA+KiX3V3JC+vxh1dK0jMxBp 9AfMSK4w7gHTwQC5SvqZQCsdObnwMI4VsE+mnEPkjuM/ZdRqSAzjE/RNQvgy+46Tveb1 J0wC0+N6V/KBgPxHMtufASpzgdVjLthPRD1Xrs9KFnq4KyLNheKUOSQ9EuBdGW4BfXZW kBIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=Gp44ocBSq1NN/nreEQ4cJNlnHwu6GLp98VU98gHXy44=; b=nhIzYCr66s0v074vWF6lwh3zFeZMtAuOqnYxyQ+hDAek9aaAd5f/k3gqLURV2RfWWw aj+rE1n5zziUQbLdSjJEROJXJNMLUs7kh32LJsPH3Vi4+W+v+iTtBUoYoZwJwLfsDmaR piJLgw/ih6lwjiYNlnAfFf3/3B7BxILcK87AqAIadYqZcgCLSiAp9Iz1jE2enL2Spj+K s9K6IctPbV553eRGp8RR7oMfS7hdzclUGKbog1Ekkee4iVPiLbid9DaGPU8MSllxrQD3 vJqDoIGl3uMhhXmZ39ydEebbi4N1SQnaX9p6yPWNUJt7RyS/5Vvsrck6KhC5rEqbjTdZ ZUQA== X-Gm-Message-State: ANhLgQ1iYkjdPse/aixRe4Izvb8JZdF+H1pupAcyg+fz0ciManspyJkJ mB16ZMNvK1iwS0ImR25RNPx4IUvO6Jo= X-Google-Smtp-Source: ADFU+vuliig09J3014NpS0KWbBuVggf/a7WDeo/uL1oqImCRC5F0B0onja1jkJowtlgRJ4Izv39DJg== X-Received: by 2002:ad4:4c81:: with SMTP id bs1mr3542525qvb.2.1584631256687; Thu, 19 Mar 2020 08:20:56 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id q15sm1783277qtj.83.2020.03.19.08.20.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Mar 2020 08:20:56 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 02JFKtMq011163; Thu, 19 Mar 2020 15:20:55 GMT Subject: [PATCH RFC 06/11] svcrdma: Cache number of Write chunks From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 19 Mar 2020 11:20:55 -0400 Message-ID: <20200319152055.16298.10403.stgit@klimt.1015granger.net> In-Reply-To: <20200319150136.16298.68813.stgit@klimt.1015granger.net> References: <20200319150136.16298.68813.stgit@klimt.1015granger.net> User-Agent: StGit/0.22 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Have the Call header decoder count the number of Write chunks it finds and cache that count for use in the Send path. Currently, the Linux NFS server implementation accepts only zero or one, but a subsequent patch will allow it to handle more than one. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 1 + net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 2 ++ net/sunrpc/xprtrdma/svc_rdma_rw.c | 2 +- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 10 +++++----- 4 files changed, 9 insertions(+), 6 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 49864264a9a5..9af9d4dff330 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -147,6 +147,7 @@ struct svc_rdma_recv_ctxt { u32 rc_inv_rkey; struct svc_rdma_payload rc_read_payload; __be32 *rc_reply_chunk; + unsigned int rc_num_write_chunks; struct page *rc_pages[RPCSVC_MAXPAGES]; }; diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index 3e02dfa351ff..95b88f68f8ca 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -194,6 +194,7 @@ svc_rdma_recv_ctxt_get(struct svcxprt_rdma *rdma) out: ctxt->rc_page_count = 0; ctxt->rc_read_payload.rp_length = 0; + ctxt->rc_num_write_chunks = 0; return ctxt; out_empty: @@ -488,6 +489,7 @@ static bool xdr_check_write_list(struct svc_rdma_recv_ctxt *rctxt) if (!p) return false; } + rctxt->rc_num_write_chunks = chcount; if (!chcount) rctxt->rc_read_payload.rp_chunk = NULL; return chcount < 2; diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 080a55456911..8ad137c7e6a0 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -625,7 +625,7 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, /* Send the page list in the Reply chunk only if the * client did not provide Write chunks. */ - if (!rctxt->rc_read_payload.rp_chunk && xdr->page_len) { + if (!rctxt->rc_num_write_chunks && xdr->page_len) { ret = svc_rdma_pages_write(info, xdr, xdr->head[0].iov_len, xdr->page_len); if (ret < 0) diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 89bc8db0289e..b6dd5ae2ad76 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -467,7 +467,7 @@ svc_rdma_encode_write_list(const struct svc_rdma_recv_ctxt *rctxt, ssize_t len, ret; len = 0; - if (rctxt->rc_read_payload.rp_chunk) { + if (rctxt->rc_num_write_chunks) { ret = svc_rdma_encode_write_chunk(sctxt, &rctxt->rc_read_payload); if (ret < 0) @@ -564,7 +564,7 @@ static bool svc_rdma_pull_up_needed(struct svcxprt_rdma *rdma, const struct svc_rdma_recv_ctxt *rctxt, struct xdr_buf *xdr) { - bool read_payload_present = rctxt && rctxt->rc_read_payload.rp_chunk; + bool read_payload_present = rctxt && rctxt->rc_num_write_chunks; int elements; /* For small messages, copying bytes is cheaper than DMA mapping. @@ -628,7 +628,7 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, tailbase = xdr->tail[0].iov_base; taillen = xdr->tail[0].iov_len; - if (rctxt && rctxt->rc_read_payload.rp_chunk) { + if (rctxt && rctxt->rc_num_write_chunks) { u32 xdrpad; xdrpad = xdr_pad_size(xdr->page_len); @@ -713,7 +713,7 @@ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, * have added XDR padding in the tail buffer, and that * should not be included inline. */ - if (rctxt && rctxt->rc_read_payload.rp_chunk) { + if (rctxt && rctxt->rc_num_write_chunks) { base = xdr->tail[0].iov_base; len = xdr->tail[0].iov_len; xdr_pad = xdr_pad_size(xdr->page_len); @@ -952,7 +952,7 @@ int svc_rdma_read_payload(struct svc_rqst *rqstp, unsigned int offset, struct xdr_buf uninitialized_var(subbuf); struct svcxprt_rdma *rdma; - if (!rctxt->rc_read_payload.rp_chunk || !length) + if (!rctxt->rc_num_write_chunks || !length) return 0; /* XXX: Just one READ payload slot for now, since our From patchwork Thu Mar 19 15:21:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11447529 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9546D1864 for ; Thu, 19 Mar 2020 15:21:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 736F920658 for ; Thu, 19 Mar 2020 15:21:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eoTb5E0s" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727555AbgCSPVF (ORCPT ); Thu, 19 Mar 2020 11:21:05 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:36562 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727602AbgCSPVF (ORCPT ); Thu, 19 Mar 2020 11:21:05 -0400 Received: by mail-qt1-f193.google.com with SMTP id m33so2115165qtb.3; Thu, 19 Mar 2020 08:21:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=W9QWgCM9scAQEaigd59+7hUo52ALMqQBWZHV5srqb3k=; b=eoTb5E0seQ3eah8dggxawhozbIdszjT66gVpKwMY6zbcovZvDqwdtidoiFpsa35LUx /nw5zTuBoydg6246v+wd9HUbRI3yzAwU+3qIqhXhZ64vEt6G7xkOkmE7fUtpOTwg7tFc PVlKoUg51+XN0lacfDdjMSkhtmHn3xu6q7qZwSNi5jSE8XxPTB8PcWK8fZSaR+YPGfGv BsWOv+vV6/iXSa5vmj14jk5CvKVfuX6JWa6yV1b6xOwJhzNnOQgXhnrYbmZW/ut8A5OM ILm6aTUexnDpL20aBpUNUa36kh+yOPLEcKeUPNYxYzeE6cJ0tG3WDm6ejCYVrq2QQFWY IsLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=W9QWgCM9scAQEaigd59+7hUo52ALMqQBWZHV5srqb3k=; b=dxmdtXBJUbJYto0HK8bxqZ9J1NY/zNOpqRRw2rYy95jyO7Jo5dx6Bt1fiBVVl1mSBY Mt5ZQNvHryfhYDVKKgciU7a7Y1kySra1wlVNPy7b9KK3nxiBTHfa5b3YZ61Tw0rGWBh+ +3v4TrgjfaaKLbkv7shMw8IvsOn4MfuKNhAZxn5Rha62p9rV/yZ3ak/KDHbBicL2R5QQ +JIDTXtpC0WVeltPUB08/8QrG6s+q7L2762IT/Xfucks4HFY6jgdJBIzp1YLoiK7UVMJ h5XrFK15H6mQQoVJl96Bs19SLV42ge2/DuKqnfGpCa6Fo3K5xX2Ix1uXycTrCzvjnc/O pTOA== X-Gm-Message-State: ANhLgQ12Cn+7PGl7mUn3Iyt7QLS4qegQYlQ3agBNvfohmLLM+6lsBstl BaonnCEqausJuvzwqIWnIUxWJOYIXQo= X-Google-Smtp-Source: ADFU+vu5T+YGFnVdse+uvroAiLF0fN2UCEA8uhPhTmtv593N0OAGullZYl8nCqebUQy/1SMdJKvwAA== X-Received: by 2002:ac8:6c6:: with SMTP id j6mr3439654qth.231.1584631262002; Thu, 19 Mar 2020 08:21:02 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id k13sm1720470qtm.11.2020.03.19.08.21.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Mar 2020 08:21:01 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 02JFL0sU011166; Thu, 19 Mar 2020 15:21:00 GMT Subject: [PATCH RFC 07/11] svcrdma: Add a data structure to track READ payloads From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 19 Mar 2020 11:21:00 -0400 Message-ID: <20200319152100.16298.44517.stgit@klimt.1015granger.net> In-Reply-To: <20200319150136.16298.68813.stgit@klimt.1015granger.net> References: <20200319150136.16298.68813.stgit@klimt.1015granger.net> User-Agent: StGit/0.22 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The Linux NFS/RDMA server implementation currently supports only a single Write chunk per RPC/RDMA request. Requests with more than one are so rare there has never been a strong need to support more. However we are aware of at least one existing NFS client implementation that can generate such requests, so let's dig in. Allocate a data structure at Receive time to keep track of the set of READ payloads and the Write chunks. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 3 ++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 30 +++++++++++++++++++----- net/sunrpc/xprtrdma/svc_rdma_rw.c | 2 +- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 38 +++++++++++++++++-------------- 4 files changed, 47 insertions(+), 26 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 9af9d4dff330..37e4c597dc71 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -145,9 +145,10 @@ struct svc_rdma_recv_ctxt { unsigned int rc_page_count; unsigned int rc_hdr_count; u32 rc_inv_rkey; - struct svc_rdma_payload rc_read_payload; + struct svc_rdma_payload *rc_read_payloads; __be32 *rc_reply_chunk; unsigned int rc_num_write_chunks; + unsigned int rc_cur_payload; struct page *rc_pages[RPCSVC_MAXPAGES]; }; diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index 95b88f68f8ca..2c3ab554c6ec 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -193,8 +193,9 @@ svc_rdma_recv_ctxt_get(struct svcxprt_rdma *rdma) out: ctxt->rc_page_count = 0; - ctxt->rc_read_payload.rp_length = 0; ctxt->rc_num_write_chunks = 0; + ctxt->rc_cur_payload = 0; + ctxt->rc_read_payloads = NULL; return ctxt; out_empty: @@ -217,7 +218,8 @@ void svc_rdma_recv_ctxt_put(struct svcxprt_rdma *rdma, for (i = 0; i < ctxt->rc_page_count; i++) put_page(ctxt->rc_pages[i]); - + kfree(ctxt->rc_read_payloads); + ctxt->rc_read_payloads = NULL; if (!ctxt->rc_temp) llist_add(&ctxt->rc_node, &rdma->sc_recv_ctxts); else @@ -474,13 +476,13 @@ static bool xdr_check_write_chunk(struct svc_rdma_recv_ctxt *rctxt, u32 maxlen) */ static bool xdr_check_write_list(struct svc_rdma_recv_ctxt *rctxt) { - u32 chcount = 0; - __be32 *p; + u32 i, segcount, chcount = 0; + __be32 *p, *saved; p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); if (!p) return false; - rctxt->rc_read_payload.rp_chunk = p; + saved = p; while (*p != xdr_zero) { if (!xdr_check_write_chunk(rctxt, MAX_BYTES_WRITE_CHUNK)) return false; @@ -491,8 +493,22 @@ static bool xdr_check_write_list(struct svc_rdma_recv_ctxt *rctxt) } rctxt->rc_num_write_chunks = chcount; if (!chcount) - rctxt->rc_read_payload.rp_chunk = NULL; - return chcount < 2; + return true; + + rctxt->rc_read_payloads = kcalloc(chcount, + sizeof(struct svc_rdma_payload), + GFP_KERNEL); + if (!rctxt->rc_read_payloads) + return false; + + i = 0; + p = saved; + while (*p != xdr_zero) { + rctxt->rc_read_payloads[i++].rp_chunk = p++; + segcount = be32_to_cpup(p++); + p += segcount * rpcrdma_segment_maxsz; + } + return true; } /* Sanity check the Reply chunk. diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 8ad137c7e6a0..5f326c18b47c 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -625,7 +625,7 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, /* Send the page list in the Reply chunk only if the * client did not provide Write chunks. */ - if (!rctxt->rc_num_write_chunks && xdr->page_len) { + if (!rctxt->rc_cur_payload && xdr->page_len) { ret = svc_rdma_pages_write(info, xdr, xdr->head[0].iov_len, xdr->page_len); if (ret < 0) diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index b6dd5ae2ad76..9fe7b0d1e335 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -447,10 +447,11 @@ static ssize_t svc_rdma_encode_write_chunk(struct svc_rdma_send_ctxt *sctxt, * @rctxt: Reply context with information about the RPC Call * @sctxt: Send context for the RPC Reply * - * The client provides a Write chunk list in the Call message. Fill - * in the segments in the first Write chunk in the Reply's transport - * header with the number of bytes consumed in each segment. - * Remaining chunks are returned unused. + * The client provided a Write list in the Call message. For each + * READ payload, fill in the segments in the Write chunks in the + * Reply's transport header with the number of bytes consumed + * in each segment. Any remaining Write chunks are returned to + * the client unused. * * Assumptions: * - Client has provided only one Write chunk @@ -465,11 +466,12 @@ svc_rdma_encode_write_list(const struct svc_rdma_recv_ctxt *rctxt, struct svc_rdma_send_ctxt *sctxt) { ssize_t len, ret; + unsigned int i; len = 0; - if (rctxt->rc_num_write_chunks) { + for (i = 0; i < rctxt->rc_num_write_chunks; i++) { ret = svc_rdma_encode_write_chunk(sctxt, - &rctxt->rc_read_payload); + &rctxt->rc_read_payloads[i]); if (ret < 0) return ret; len += ret; @@ -564,7 +566,7 @@ static bool svc_rdma_pull_up_needed(struct svcxprt_rdma *rdma, const struct svc_rdma_recv_ctxt *rctxt, struct xdr_buf *xdr) { - bool read_payload_present = rctxt && rctxt->rc_num_write_chunks; + bool read_payload_present = rctxt && rctxt->rc_cur_payload; int elements; /* For small messages, copying bytes is cheaper than DMA mapping. @@ -628,7 +630,7 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, tailbase = xdr->tail[0].iov_base; taillen = xdr->tail[0].iov_len; - if (rctxt && rctxt->rc_num_write_chunks) { + if (rctxt && rctxt->rc_cur_payload) { u32 xdrpad; xdrpad = xdr_pad_size(xdr->page_len); @@ -708,12 +710,12 @@ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, if (ret < 0) return ret; - /* If a Write chunk is present, the xdr_buf's page list + /* If Write chunks are present, the xdr_buf's page list * is not included inline. However the Upper Layer may * have added XDR padding in the tail buffer, and that * should not be included inline. */ - if (rctxt && rctxt->rc_num_write_chunks) { + if (rctxt && rctxt->rc_cur_payload) { base = xdr->tail[0].iov_base; len = xdr->tail[0].iov_len; xdr_pad = xdr_pad_size(xdr->page_len); @@ -951,21 +953,23 @@ int svc_rdma_read_payload(struct svc_rqst *rqstp, unsigned int offset, struct svc_rdma_recv_ctxt *rctxt = rqstp->rq_xprt_ctxt; struct xdr_buf uninitialized_var(subbuf); struct svcxprt_rdma *rdma; + unsigned int i; if (!rctxt->rc_num_write_chunks || !length) return 0; - /* XXX: Just one READ payload slot for now, since our - * transport implementation currently supports only one - * Write chunk. - */ - rctxt->rc_read_payload.rp_offset = offset; - rctxt->rc_read_payload.rp_length = length; + if (rctxt->rc_cur_payload > rctxt->rc_num_write_chunks) + return -ENOENT; + i = rctxt->rc_cur_payload++; + + rctxt->rc_read_payloads[i].rp_offset = offset; + rctxt->rc_read_payloads[i].rp_length = length; if (xdr_buf_subsegment(&rqstp->rq_res, &subbuf, offset, length)) return -EMSGSIZE; rdma = container_of(rqstp->rq_xprt, struct svcxprt_rdma, sc_xprt); - return svc_rdma_send_write_chunk(rdma, rctxt->rc_read_payload.rp_chunk, + return svc_rdma_send_write_chunk(rdma, + rctxt->rc_read_payloads[i].rp_chunk, &subbuf); } From patchwork Thu Mar 19 15:21:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11447533 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D77FE1864 for ; Thu, 19 Mar 2020 15:21:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B5D3520658 for ; Thu, 19 Mar 2020 15:21:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lN2h8Rkm" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727682AbgCSPVK (ORCPT ); Thu, 19 Mar 2020 11:21:10 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:46453 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727670AbgCSPVJ (ORCPT ); Thu, 19 Mar 2020 11:21:09 -0400 Received: by mail-qt1-f195.google.com with SMTP id t13so2067494qtn.13; Thu, 19 Mar 2020 08:21:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=ONYS8BfGMnS9X+7ErK9RmzTVB4qRV04mWZjkKLMBGTE=; b=lN2h8Rkm61tfMYfCo/cfhWV0TZt/hC7GAXz1Wz6qftjFHpIgATb26Dz2HG2r9MBZum IawkUhos6mMQO7yvqmWayYDXkb9ACbehgH3/Ay+JlMdM65H2tcJ+j37bDp0Pxv2T/KMo B21wD9PuLu1VN436ajsC4PLyk/4uBhZwPbUEMQdEurjaLcVI9KcdFj5mXNynx3nLFU+O 4wmdKoewFlH0FhStxlWcb6wMrp4t1hDvSzasqC/Ikxd818z0kUoYcGD6DPq9xvIaj+9V DDdWCpdu55brQHcCV4EdOoBBaMur+YXUDL5YqcdsE6ooEz8ka9/ZO7k920Al1aaPp5HO qIrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=ONYS8BfGMnS9X+7ErK9RmzTVB4qRV04mWZjkKLMBGTE=; b=JH01ihAqcj0TX7z11+b7pRoF3o+hxS41VkaZN5UUJDyaJer7bcNBjgYuLAd/9HlJVL pjVN0CL7A9AXJOlaV0F7F1kC63aWTfzIcTxtJr8a8GRMbsSzoYyXW3bZSqnFTV8AH6By +xNOpYeWlaqnkerpqA9pNPX3om5J2uR3KwRkQVfrTiVh9JBCkX7sy9lbWCMVhpUbkRvt 2S1el1BU3HFQhnDqIx1yGZGBq4wQgBDM5SpwHa/jeTg9/XQgg+SrykDSn57bTy6lfkwU K4OiDHuEOAPhijJQ7/3OYxYKKcImky0j3g7Y53q9GXyny8fHmBX1pHKuYHg7VXi/SrNI uPFw== X-Gm-Message-State: ANhLgQ31KUWGiUDlOD0Bdo5mS0JmK/hMXAMioc7tbOaPNeN61tLX6Rda sjvVkI6ACvsKL2f6PRzg0r79Zc3yygM= X-Google-Smtp-Source: ADFU+vvhcLZ+qVBYypHZp7aJNo8OVCbR2ABowPmO3d1Df2apAD87CqyJPYU097KrhSnL1sPizw8iHg== X-Received: by 2002:ac8:3141:: with SMTP id h1mr3512107qtb.108.1584631267414; Thu, 19 Mar 2020 08:21:07 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id p35sm1773177qtk.2.2020.03.19.08.21.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Mar 2020 08:21:06 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 02JFL5jU011169; Thu, 19 Mar 2020 15:21:05 GMT Subject: [PATCH RFC 08/11] svcrdma: Add svc_rdma_skip_payloads() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 19 Mar 2020 11:21:05 -0400 Message-ID: <20200319152105.16298.14148.stgit@klimt.1015granger.net> In-Reply-To: <20200319150136.16298.68813.stgit@klimt.1015granger.net> References: <20200319150136.16298.68813.stgit@klimt.1015granger.net> User-Agent: StGit/0.22 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org We'll need a generic mechanism for processing only the parts of an egress RPC message that are _not_ a READ payload. This will be used in subsequent patches. This is a separate patch to reduce the complexity of subsequent patches, so that the logic of this new mechanism can be separately reviewed. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 4 ++ net/sunrpc/xprtrdma/svc_rdma_sendto.c | 72 +++++++++++++++++++++++++++++++++ 2 files changed, 76 insertions(+) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 37e4c597dc71..93642a889535 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -195,6 +195,10 @@ extern struct svc_rdma_send_ctxt * svc_rdma_send_ctxt_get(struct svcxprt_rdma *rdma); extern void svc_rdma_send_ctxt_put(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *ctxt); +extern int svc_rdma_skip_payloads(const struct xdr_buf *xdr, + const struct svc_rdma_recv_ctxt *rctxt, + int (*actor)(const struct xdr_buf *, void *), + void *data); extern int svc_rdma_send(struct svcxprt_rdma *rdma, struct ib_send_wr *wr); extern int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *sctxt, diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 9fe7b0d1e335..85c91d0debb4 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -515,6 +515,78 @@ svc_rdma_encode_reply_chunk(const struct svc_rdma_recv_ctxt *rctxt, return svc_rdma_encode_write_chunk(sctxt, &payload); } +static inline int +xdr_buf_process_region(const struct xdr_buf *xdr, + unsigned int offset, unsigned int length, + int (*actor)(const struct xdr_buf *, void *), + void *data) +{ + struct xdr_buf subbuf; + + if (!length) + return 0; + if (xdr_buf_subsegment(xdr, &subbuf, offset, length)) + return -EMSGSIZE; + return actor(&subbuf, data); +} + +/** + * svc_rdma_skip_payloads - Call an actor for non-payload regions of @xdr + * @xdr: xdr_buf to process + * @rctxt: Write and Reply chunks provided by client + * @actor: function to invoke on that region + * @data: pointer to arguments for @actor + * + * This mechanism must ignore not only READ payloads that were already + * sent via RDMA Write, but also XDR padding for those payloads that + * the upper layer has added. + * + * Assumptions: + * The xdr->len and rp_ fields are aligned to 4-byte multiples. + * + * Returns: + * On success, zero, + * %-EMSGSIZE on XDR buffer overflow, or + * The return value of @actor + */ +int svc_rdma_skip_payloads(const struct xdr_buf *xdr, + const struct svc_rdma_recv_ctxt *rctxt, + int (*actor)(const struct xdr_buf *, void *), + void *data) +{ + const unsigned int num_payloads = rctxt ? rctxt->rc_cur_payload : 0; + unsigned int offset, length; + int i, ret; + + if (likely(!num_payloads)) + return actor(xdr, data); + + /* Before the first READ payload */ + offset = 0; + length = rctxt->rc_read_payloads[0].rp_offset; + ret = xdr_buf_process_region(xdr, offset, length, actor, data); + if (ret < 0) + return ret; + + /* Any middle READ payloads */ + for (i = 0; i + 1 < num_payloads; i++) { + offset = xdr_align_size(length + rctxt->rc_read_payloads[i].rp_length); + length = rctxt->rc_read_payloads[i + 1].rp_offset - offset; + ret = xdr_buf_process_region(xdr, offset, length, actor, data); + if (ret < 0) + return ret; + } + + /* After the last READ payload */ + offset = xdr_align_size(length + rctxt->rc_read_payloads[i].rp_length); + length = xdr->len - offset; + ret = xdr_buf_process_region(xdr, offset, length, actor, data); + if (ret < 0) + return ret; + + return 0; +} + static int svc_rdma_dma_map_page(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *ctxt, struct page *page, From patchwork Thu Mar 19 15:21:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11447537 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B0E2D14B4 for ; Thu, 19 Mar 2020 15:21:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9047C20B1F for ; Thu, 19 Mar 2020 15:21:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ULJgzB2k" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727690AbgCSPVR (ORCPT ); Thu, 19 Mar 2020 11:21:17 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:42954 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727685AbgCSPVQ (ORCPT ); Thu, 19 Mar 2020 11:21:16 -0400 Received: by mail-qt1-f195.google.com with SMTP id g16so2088028qtp.9; Thu, 19 Mar 2020 08:21:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=+dY/ASI5FAtYHOJqm4kl9v3500pCo4QwYF3AcBfoFAM=; b=ULJgzB2k4lSrNNmm4f725ILiCv/cmifwT8nbIh0s4viC/at8sgDmeLgWZPncUmoGHE X3DtztJFk4H4AKC7HCD9y4OHjNnPa9tQJYk5bwHrTgCaJh2nVafSGRnyd5hj0mjSZO9t uh0GRCRteHpgeiLzMplAyEM/8kQjpD5I8WeXPcrTCOI6hTae+D0KAj29LBygMFQYcxCY 2ZvexHpp5qGXg9XGlLSV9LtaiTksU2eg1MU+DqBnWwmoTfIxqDpbR4XNgtXRF8xT/Opo Ku2wcYI6OcQyh4rsTXG6nvw+NqnrpGaHJ/PssIJZWfYFA9zOkyD+LYEYkUOdyfqkXT41 Vscw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=+dY/ASI5FAtYHOJqm4kl9v3500pCo4QwYF3AcBfoFAM=; b=PvcRi8VftHZoGmEIqFhkHx6meViD4ulrUcreMgpGC4hXwSHCKHKdQLzRMjMOUMK+Ly rfyp6QDwf3mkSMkS4ikDYVTvFSyvp3z8XaO5t7NhZ4k4/2JL0tLqv1fKFVQIkIP8aH/M Fkuf4oB7TkOZcIa20Fb7RQjpg/t+4cDj/lMW9TAeOStwOsV+WgIu8YCEiQSeeWKZxbB+ mpFJWS75NSvPkbYcxr7ashIRNZNbJEaZcumkA2cI5x6vPeCWRNysJp7jZTnCAfnXlaon oyzeaJK18oWN/ZEAVCHU4RZD30sKf7x3JdJdceMsqPiqZ1w/Jkd/3eNx1+sUnmz1sHuo Hk9g== X-Gm-Message-State: ANhLgQ1tlUE0PrFMgNVXvOIywijpZzaPx63bDpbMzbVr4Cz0xY2ZFuM4 beMHYwlo8MdElHXzR7eHLDQBIMK0DRo= X-Google-Smtp-Source: ADFU+vuK+axvGwramevkSPh8FDtz5Glnuj4/Fc5CM0poA69a+ox7/MmM39cl8jTvQ18bzMh8RLheqg== X-Received: by 2002:ac8:4785:: with SMTP id k5mr3373960qtq.256.1584631272949; Thu, 19 Mar 2020 08:21:12 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id e66sm1714343qkd.129.2020.03.19.08.21.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Mar 2020 08:21:12 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 02JFLAEH011172; Thu, 19 Mar 2020 15:21:10 GMT Subject: [PATCH RFC 09/11] svcrdma: Support multiple READ payloads when pulling up From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 19 Mar 2020 11:21:10 -0400 Message-ID: <20200319152110.16298.9209.stgit@klimt.1015granger.net> In-Reply-To: <20200319150136.16298.68813.stgit@klimt.1015granger.net> References: <20200319150136.16298.68813.stgit@klimt.1015granger.net> User-Agent: StGit/0.22 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org When counting the number of SGEs needed to construct a Send request, do not count READ payloads. And, when copying the Reply message into the pull-up buffer, READ payloads are not to be copied to the Send buffer. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 15 ++- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 186 ++++++++++++++++++++------------- 2 files changed, 121 insertions(+), 80 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 9238d233f8cf..ff2d943d1540 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1641,20 +1641,25 @@ TRACE_EVENT(svcrdma_dma_map_rwctx, TRACE_EVENT(svcrdma_send_pullup, TP_PROTO( - unsigned int len + unsigned int hdrlen, + unsigned int msglen ), - TP_ARGS(len), + TP_ARGS(hdrlen, msglen), TP_STRUCT__entry( - __field(unsigned int, len) + __field(unsigned int, hdrlen) + __field(unsigned int, msglen) ), TP_fast_assign( - __entry->len = len; + __entry->hdrlen = hdrlen; + __entry->msglen = msglen; ), - TP_printk("len=%u", __entry->len) + TP_printk("hdr=%u msg=%u (total %u)", + __entry->hdrlen, __entry->msglen, + __entry->hdrlen + __entry->msglen) ); TRACE_EVENT(svcrdma_send_failed, diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 85c91d0debb4..037be0bdb557 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -622,6 +622,45 @@ static int svc_rdma_dma_map_buf(struct svcxprt_rdma *rdma, offset_in_page(base), len); } +struct svc_rdma_pullup_data { + u8 *pd_dest; + unsigned int pd_length; + unsigned int pd_num_sges; +}; + +/** + * svc_rdma_xb_count_sges - Count how many SGEs will be needed + * @xdr: xdr_buf containing portion of an RPC message to transmit + * @data: pointer to arguments + * + * Returns: + * Number of SGEs needed to Send the contents of @xdr inline + */ +static int svc_rdma_xb_count_sges(const struct xdr_buf *xdr, + void *data) +{ + struct svc_rdma_pullup_data *args = data; + unsigned int remaining; + unsigned long offset; + + if (xdr->head[0].iov_len) + ++args->pd_num_sges; + + offset = offset_in_page(xdr->page_base); + remaining = xdr->page_len; + while (remaining) { + ++args->pd_num_sges; + remaining -= min_t(u32, PAGE_SIZE - offset, remaining); + offset = 0; + } + + if (xdr->tail[0].iov_len) + ++args->pd_num_sges; + + args->pd_length += xdr->len; + return 0; +} + /** * svc_rdma_pull_up_needed - Determine whether to use pull-up * @rdma: controlling transport @@ -630,50 +669,70 @@ static int svc_rdma_dma_map_buf(struct svcxprt_rdma *rdma, * @xdr: xdr_buf containing RPC message to transmit * * Returns: - * %true if pull-up must be used - * %false otherwise + * %true if pull-up must be used + * %false otherwise */ -static bool svc_rdma_pull_up_needed(struct svcxprt_rdma *rdma, - struct svc_rdma_send_ctxt *sctxt, +static bool svc_rdma_pull_up_needed(const struct svcxprt_rdma *rdma, + const struct svc_rdma_send_ctxt *sctxt, const struct svc_rdma_recv_ctxt *rctxt, - struct xdr_buf *xdr) + const struct xdr_buf *xdr) { - bool read_payload_present = rctxt && rctxt->rc_cur_payload; - int elements; + /* Resources needed for the transport header */ + struct svc_rdma_pullup_data args = { + .pd_length = sctxt->sc_hdrbuf.len, + .pd_num_sges = 1, + }; + int ret; - /* For small messages, copying bytes is cheaper than DMA mapping. - */ - if (!read_payload_present && - sctxt->sc_hdrbuf.len + xdr->len < RPCRDMA_PULLUP_THRESH) + ret = svc_rdma_skip_payloads(xdr, rctxt, svc_rdma_xb_count_sges, + &args); + if (ret < 0) + return false; + + if (args.pd_length < RPCRDMA_PULLUP_THRESH) return true; + return args.pd_num_sges >= rdma->sc_max_send_sges; +} - /* Check whether the xdr_buf has more elements than can - * fit in a single RDMA Send. - */ - /* xdr->head */ - elements = 1; - - /* xdr->pages */ - if (!read_payload_present) { - unsigned int remaining; - unsigned long pageoff; - - pageoff = xdr->page_base & ~PAGE_MASK; - remaining = xdr->page_len; - while (remaining) { - ++elements; - remaining -= min_t(u32, PAGE_SIZE - pageoff, - remaining); - pageoff = 0; - } +/** + * svc_rdma_xb_linearize - Copy region of xdr_buf to flat buffer + * @xdr: xdr_buf containing portion of an RPC message to copy + * @data: pointer to arguments + * + * Returns: + * Always zero. + */ +static int svc_rdma_xb_linearize(const struct xdr_buf *xdr, + void *data) +{ + struct svc_rdma_pullup_data *args = data; + unsigned int len, remaining; + unsigned long pageoff; + struct page **ppages; + + if (xdr->head[0].iov_len) { + memcpy(args->pd_dest, xdr->head[0].iov_base, xdr->head[0].iov_len); + args->pd_dest += xdr->head[0].iov_len; } - /* xdr->tail */ - if (xdr->tail[0].iov_len) - ++elements; + ppages = xdr->pages + (xdr->page_base >> PAGE_SHIFT); + pageoff = offset_in_page(xdr->page_base); + remaining = xdr->page_len; + while (remaining) { + len = min_t(u32, PAGE_SIZE - pageoff, remaining); + memcpy(args->pd_dest, page_address(*ppages), len); + remaining -= len; + args->pd_dest += len; + pageoff = 0; + } + + if (xdr->tail[0].iov_len) { + memcpy(args->pd_dest, xdr->tail[0].iov_base, xdr->tail[0].iov_len); + args->pd_dest += xdr->tail[0].iov_len; + } - /* assume 1 SGE is needed for the transport header */ - return elements >= rdma->sc_max_send_sges; + args->pd_length += xdr->len; + return 0; } /** @@ -686,53 +745,30 @@ static bool svc_rdma_pull_up_needed(struct svcxprt_rdma *rdma, * The device is not capable of sending the reply directly. * Assemble the elements of @xdr into the transport header buffer. * - * Returns zero on success, or a negative errno on failure. + * Assumptions: + * pull_up_needed has determined that @xdr will fit in the buffer. + * + * Returns: + * %0 if pull-up was successful + * %-EMSGSIZE if a buffer manipulation problem occurred */ -static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, +static int svc_rdma_pull_up_reply_msg(const struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *sctxt, const struct svc_rdma_recv_ctxt *rctxt, const struct xdr_buf *xdr) { - unsigned char *dst, *tailbase; - unsigned int taillen; - - dst = sctxt->sc_xprt_buf + sctxt->sc_hdrbuf.len; - memcpy(dst, xdr->head[0].iov_base, xdr->head[0].iov_len); - dst += xdr->head[0].iov_len; - - tailbase = xdr->tail[0].iov_base; - taillen = xdr->tail[0].iov_len; - if (rctxt && rctxt->rc_cur_payload) { - u32 xdrpad; - - xdrpad = xdr_pad_size(xdr->page_len); - if (taillen && xdrpad) { - tailbase += xdrpad; - taillen -= xdrpad; - } - } else { - unsigned int len, remaining; - unsigned long pageoff; - struct page **ppages; - - ppages = xdr->pages + (xdr->page_base >> PAGE_SHIFT); - pageoff = xdr->page_base & ~PAGE_MASK; - remaining = xdr->page_len; - while (remaining) { - len = min_t(u32, PAGE_SIZE - pageoff, remaining); - - memcpy(dst, page_address(*ppages), len); - remaining -= len; - dst += len; - pageoff = 0; - } - } + struct svc_rdma_pullup_data args = { + .pd_dest = sctxt->sc_xprt_buf + sctxt->sc_hdrbuf.len, + }; + int ret; - if (taillen) - memcpy(dst, tailbase, taillen); + ret = svc_rdma_skip_payloads(xdr, rctxt, svc_rdma_xb_linearize, + &args); + if (ret < 0) + return ret; - sctxt->sc_sges[0].length += xdr->len; - trace_svcrdma_send_pullup(sctxt->sc_sges[0].length); + sctxt->sc_sges[0].length = sctxt->sc_hdrbuf.len + args.pd_length; + trace_svcrdma_send_pullup(sctxt->sc_hdrbuf.len, args.pd_length); return 0; } From patchwork Thu Mar 19 15:21:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11447541 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 064281864 for ; Thu, 19 Mar 2020 15:21:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D896820658 for ; Thu, 19 Mar 2020 15:21:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ePLs7D97" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727476AbgCSPVU (ORCPT ); Thu, 19 Mar 2020 11:21:20 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:34422 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727540AbgCSPVU (ORCPT ); Thu, 19 Mar 2020 11:21:20 -0400 Received: by mail-qt1-f195.google.com with SMTP id 10so2124211qtp.1; Thu, 19 Mar 2020 08:21:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=q1Li6OEH78mDk0zb58KdbS5Wq0GTDNIY6pmtCFwMNIE=; b=ePLs7D978F4rjdaRtx0HB0o1EoAz/5pmyqWirfHBu21Qb7ZINvdaV0K0sdlIfQYraB rC5zIHdXtF4RYuBgX6Hria/mnndL/2qrzx/B8/pQiZrIG67tJKGdR7eImmkQO6IsgqFr Py+NOMA+GMUbroLQTpJd9mn0A85qZ9tZ9qW1eXAty+9k6W19jH+Bs/cwLFiuUxdqamrx koppG3u9lkdvYnDrRIQm5aaxp4U70mguF37cZ+cLvP0K5H9VPS59vF+FX4qMvFRABFN0 oEZu6mmTGF6ln0es1YpP5b3RNG6YtHsZHD71GwrzYibDvFmUCaP3uattBdIcNM0VlkE5 RhjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=q1Li6OEH78mDk0zb58KdbS5Wq0GTDNIY6pmtCFwMNIE=; b=HCDSM2drzfNUCeBE/Jvxhn7CMJrq4tYuSraBA3tDkwnI4CypRWOJ9gXB+9JKOh4H1U EueFqMvF3DUSTq0o9vvAU8XI8LGR4pw1lTTls2col5uCfU8TLxhF2QItEcy+jUi4gS57 YHxpsZ0FpX2VMw+BQLZFDcKcqe5rdOq8q7vp0LQu7m+pXDDu36QSpp6yOrlNVAQJ3CBS eD+8ro8oWcATXTzOfRzdWdh6IrkKmlu8J25Z/c+jbCJsDeJMo7UMnFMGjDKHE+JkzYzK AdsJyD29e2BniSgApqJXMZxRZww1fOesVqsv9KS1FtXuFdKtGIXZ2k689bByu09xPYYL TqkQ== X-Gm-Message-State: ANhLgQ3ilb2i3e+QQSwUqZaNZQe7UdjxQNAcUABSbVZfitmoyUbHC3mb kdcYyn1LzL8J9H+w4wHO6I2HK4zH2kc= X-Google-Smtp-Source: ADFU+vtjAmehoL6Yu+AdUJNcWUd1NtFomwQdAQ1uWPENR/Q/qYWbkbuVBS7RyRDfLqbMEumVqb4flg== X-Received: by 2002:ac8:c4f:: with SMTP id l15mr3455718qti.177.1584631277998; Thu, 19 Mar 2020 08:21:17 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id f203sm1721777qke.100.2020.03.19.08.21.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Mar 2020 08:21:17 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 02JFLG7A011175; Thu, 19 Mar 2020 15:21:16 GMT Subject: [PATCH RFC 10/11] svcrdma: Support multiple READ payloads in svc_rdma_map_reply_msg() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 19 Mar 2020 11:21:16 -0400 Message-ID: <20200319152116.16298.94729.stgit@klimt.1015granger.net> In-Reply-To: <20200319150136.16298.68813.stgit@klimt.1015granger.net> References: <20200319150136.16298.68813.stgit@klimt.1015granger.net> User-Agent: StGit/0.22 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The function is restructured to DMA map only the parts of rq_res that do not contain a READ payload. This change has been tested to confirm that it does not cause a regression in the no Write chunk and single Write chunk cases. Multiple Write chunks have not been tested. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 2 include/trace/events/rpcrdma.h | 1 net/sunrpc/xprtrdma/svc_rdma_sendto.c | 173 +++++++++++++++++++-------------- 3 files changed, 99 insertions(+), 77 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 93642a889535..6f235d66e6fc 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -203,7 +203,7 @@ extern int svc_rdma_send(struct svcxprt_rdma *rdma, struct ib_send_wr *wr); extern int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *sctxt, const struct svc_rdma_recv_ctxt *rctxt, - struct xdr_buf *xdr); + const struct xdr_buf *xdr); extern int svc_rdma_sendto(struct svc_rqst *); extern int svc_rdma_read_payload(struct svc_rqst *rqstp, unsigned int offset, unsigned int length); diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index ff2d943d1540..b270069c90a0 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1612,6 +1612,7 @@ DECLARE_EVENT_CLASS(svcrdma_dma_map_class, TP_ARGS(rdma, dma_addr, length)) DEFINE_SVC_DMA_EVENT(dma_map_page); +DEFINE_SVC_DMA_EVENT(dma_map_failed); DEFINE_SVC_DMA_EVENT(dma_unmap_page); TRACE_EVENT(svcrdma_dma_map_rwctx, diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 037be0bdb557..435b3c0f3b6e 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -587,39 +587,111 @@ int svc_rdma_skip_payloads(const struct xdr_buf *xdr, return 0; } -static int svc_rdma_dma_map_page(struct svcxprt_rdma *rdma, - struct svc_rdma_send_ctxt *ctxt, - struct page *page, - unsigned long offset, - unsigned int len) +struct svc_rdma_map_data { + struct svcxprt_rdma *md_rdma; + struct svc_rdma_send_ctxt *md_ctxt; +}; + +/** + * svc_rdma_page_dma_map - DMA map one page + * @data: pointer to arguments + * @page: struct page to DMA map + * @offset: offset into the page + * @len: number of bytes to map + * + * Returns: + * %0 if DMA mapping was successful + * %-EIO if the page cannot be DMA mapped + */ +static int svc_rdma_page_dma_map(void *data, struct page *page, + unsigned long offset, unsigned int len) { + struct svc_rdma_map_data *args = data; + struct svcxprt_rdma *rdma = args->md_rdma; + struct svc_rdma_send_ctxt *ctxt = args->md_ctxt; struct ib_device *dev = rdma->sc_cm_id->device; dma_addr_t dma_addr; + ++ctxt->sc_cur_sge_no; + dma_addr = ib_dma_map_page(dev, page, offset, len, DMA_TO_DEVICE); - trace_svcrdma_dma_map_page(rdma, dma_addr, len); if (ib_dma_mapping_error(dev, dma_addr)) goto out_maperr; + trace_svcrdma_dma_map_page(rdma, dma_addr, len); ctxt->sc_sges[ctxt->sc_cur_sge_no].addr = dma_addr; ctxt->sc_sges[ctxt->sc_cur_sge_no].length = len; ctxt->sc_send_wr.num_sge++; return 0; out_maperr: + trace_svcrdma_dma_map_failed(rdma, dma_addr, len); return -EIO; } -/* ib_dma_map_page() is used here because svc_rdma_dma_unmap() +/** + * svc_rdma_iov_dma_map - DMA map an iovec + * @data: pointer to arguments + * @iov: kvec to DMA map + * + * ib_dma_map_page() is used here because svc_rdma_dma_unmap() * handles DMA-unmap and it uses ib_dma_unmap_page() exclusively. + * + * Returns: + * %0 if DMA mapping was successful + * %-EIO if the iovec cannot be DMA mapped */ -static int svc_rdma_dma_map_buf(struct svcxprt_rdma *rdma, - struct svc_rdma_send_ctxt *ctxt, - unsigned char *base, - unsigned int len) +static int svc_rdma_iov_dma_map(void *data, const struct kvec *iov) { - return svc_rdma_dma_map_page(rdma, ctxt, virt_to_page(base), - offset_in_page(base), len); + if (!iov->iov_len) + return 0; + return svc_rdma_page_dma_map(data, virt_to_page(iov->iov_base), + offset_in_page(iov->iov_base), + iov->iov_len); +} + +/** + * svc_rdma_xb_dma_map - DMA map all segments of an xdr_buf + * @xdr: xdr_buf containing portion of an RPC message to transmit + * @data: pointer to arguments + * + * Returns: + * %0 if DMA mapping was successful + * %-EIO if DMA mapping failed + * + * On failure, any DMA mappings that have been already done must be + * unmapped by the caller. + */ +static int svc_rdma_xb_dma_map(const struct xdr_buf *xdr, void *data) +{ + unsigned int len, remaining; + unsigned long pageoff; + struct page **ppages; + int ret; + + ret = svc_rdma_iov_dma_map(data, &xdr->head[0]); + if (ret < 0) + return ret; + + ppages = xdr->pages + (xdr->page_base >> PAGE_SHIFT); + pageoff = offset_in_page(xdr->page_base); + remaining = xdr->page_len; + while (remaining) { + len = min_t(u32, PAGE_SIZE - pageoff, remaining); + + ret = svc_rdma_page_dma_map(data, *ppages++, pageoff, len); + if (ret < 0) + return ret; + + remaining -= len; + pageoff = 0; + } + + ret = svc_rdma_iov_dma_map(data, &xdr->tail[0]); + if (ret < 0) + return ret; + + return xdr->len; } struct svc_rdma_pullup_data { @@ -720,7 +792,7 @@ static int svc_rdma_xb_linearize(const struct xdr_buf *xdr, remaining = xdr->page_len; while (remaining) { len = min_t(u32, PAGE_SIZE - pageoff, remaining); - memcpy(args->pd_dest, page_address(*ppages), len); + memcpy(args->pd_dest, page_address(*ppages) + pageoff, len); remaining -= len; args->pd_dest += len; pageoff = 0; @@ -778,22 +850,22 @@ static int svc_rdma_pull_up_reply_msg(const struct svcxprt_rdma *rdma, * @rctxt: Write and Reply chunks provided by client * @xdr: prepared xdr_buf containing RPC message * - * Load the xdr_buf into the ctxt's sge array, and DMA map each - * element as it is added. The Send WR's num_sge field is set. + * Returns: + * %0 if DMA mapping was successful. + * %-EMSGSIZE if a buffer manipulation problem occurred + * %-EIO if DMA mapping failed * - * Returns zero on success, or a negative errno on failure. + * The Send WR's num_sge field is set in all cases. */ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *sctxt, const struct svc_rdma_recv_ctxt *rctxt, - struct xdr_buf *xdr) + const struct xdr_buf *xdr) { - unsigned int len, remaining; - unsigned long page_off; - struct page **ppages; - unsigned char *base; - u32 xdr_pad; - int ret; + struct svc_rdma_map_data args = { + .md_rdma = rdma, + .md_ctxt = sctxt, + }; /* Set up the (persistently-mapped) transport header SGE. */ sctxt->sc_send_wr.num_sge = 1; @@ -811,58 +883,7 @@ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, if (svc_rdma_pull_up_needed(rdma, sctxt, rctxt, xdr)) return svc_rdma_pull_up_reply_msg(rdma, sctxt, rctxt, xdr); - ++sctxt->sc_cur_sge_no; - ret = svc_rdma_dma_map_buf(rdma, sctxt, - xdr->head[0].iov_base, - xdr->head[0].iov_len); - if (ret < 0) - return ret; - - /* If Write chunks are present, the xdr_buf's page list - * is not included inline. However the Upper Layer may - * have added XDR padding in the tail buffer, and that - * should not be included inline. - */ - if (rctxt && rctxt->rc_cur_payload) { - base = xdr->tail[0].iov_base; - len = xdr->tail[0].iov_len; - xdr_pad = xdr_pad_size(xdr->page_len); - - if (len && xdr_pad) { - base += xdr_pad; - len -= xdr_pad; - } - - goto tail; - } - - ppages = xdr->pages + (xdr->page_base >> PAGE_SHIFT); - page_off = xdr->page_base & ~PAGE_MASK; - remaining = xdr->page_len; - while (remaining) { - len = min_t(u32, PAGE_SIZE - page_off, remaining); - - ++sctxt->sc_cur_sge_no; - ret = svc_rdma_dma_map_page(rdma, sctxt, *ppages++, - page_off, len); - if (ret < 0) - return ret; - - remaining -= len; - page_off = 0; - } - - base = xdr->tail[0].iov_base; - len = xdr->tail[0].iov_len; -tail: - if (len) { - ++sctxt->sc_cur_sge_no; - ret = svc_rdma_dma_map_buf(rdma, sctxt, base, len); - if (ret < 0) - return ret; - } - - return 0; + return svc_rdma_skip_payloads(xdr, rctxt, svc_rdma_xb_dma_map, &args); } /* The svc_rqst and all resources it owns are released as soon as From patchwork Thu Mar 19 15:21:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11447545 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 358761864 for ; Thu, 19 Mar 2020 15:21:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 126422072D for ; Thu, 19 Mar 2020 15:21:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="M7Ikn4fD" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727709AbgCSPV0 (ORCPT ); Thu, 19 Mar 2020 11:21:26 -0400 Received: from mail-qv1-f67.google.com ([209.85.219.67]:42728 "EHLO mail-qv1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727540AbgCSPV0 (ORCPT ); Thu, 19 Mar 2020 11:21:26 -0400 Received: by mail-qv1-f67.google.com with SMTP id ca9so1178685qvb.9; Thu, 19 Mar 2020 08:21:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=gpO5ar6l5mf3CxyUoyQziF/JWwmHy57fKI7SFtAvMFI=; b=M7Ikn4fDyjNwqvpXEOzphdMtJIO9vT3y34Ig6HfNR+5f0iDZ3RpNV2ahnlHvJcKrWe b8LXMkFgTzoCtjumosSy9OjzMw+G355kHYSGyiu7lz9jMzH5mQ3urJjpIptRI09o9pOq vcrlvBF5Jj979h3cLq61sfB/ckyqEINO29ASzSHCWUspQ8f3TEagUQrVJfOrmVl2qyAA k+1ntjOH0BOom9Ph035I7zEUEO/JcbW3YvugzuU9GCfoqK3N3/CmhO9/nnc03FIUlfdM sn09qLSG6LZKiPPWqZJaQHBsMvAOxOJUpCzJfpQ53PXTfNpIcR4djTyMCHzPzBF7y9ic Q2Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=gpO5ar6l5mf3CxyUoyQziF/JWwmHy57fKI7SFtAvMFI=; b=CMPrdfJSyXxZ2EJ0yVrQ4zBKjsXTVC7UAQggFnknlf6Mqj5O/ZVCqG51HgKH/9jAd6 Wm6XOVrspsoHpa+TY7z5nBzkMP/WurWD9T9/ji+2KjhYWtRn0ivRKxw4bJ9Rm3EM4/Ps FjXT3lAtb5gVxMVxpmnwKQyDbG5/ind3JKDbSXHGeAWTL3WWFecxjBU4W323IZTvmaJo SCjRa7h7rCpAUEmICeAUyqfHphXd0dhjlWH3MqUNs72cq85SA5wfTEVSbLnykO+noYrE YVYwrw0BYWZex06nq5IURpxGesLfKnmr4FM+hAUBHXEq2fDNe3lwR++YEiRcXBpLfNQ5 hxJw== X-Gm-Message-State: ANhLgQ37Xg+A6y7JHSMK9cILzmahAFN9GT1L5bFRMdm8FRxkODjkc7iV D2SOfH9kHxGSUzagZ5zspytjVRrWVTk= X-Google-Smtp-Source: ADFU+vsiEheXBND/xC1S5G06JovVh4Bt3dw35QXC6KwrMbHp59VSwGd2hLSJxqAR2ZDuVLQ8rjqkJw== X-Received: by 2002:ad4:42ce:: with SMTP id f14mr3394240qvr.115.1584631283231; Thu, 19 Mar 2020 08:21:23 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id t123sm1695152qkc.81.2020.03.19.08.21.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Mar 2020 08:21:22 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 02JFLL12011178; Thu, 19 Mar 2020 15:21:21 GMT Subject: [PATCH RFC 11/11] svcrdma: Support multiple Write chunks in svc_rdma_send_reply_chunk From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 19 Mar 2020 11:21:21 -0400 Message-ID: <20200319152121.16298.31324.stgit@klimt.1015granger.net> In-Reply-To: <20200319150136.16298.68813.stgit@klimt.1015granger.net> References: <20200319150136.16298.68813.stgit@klimt.1015granger.net> User-Agent: StGit/0.22 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org This should be the last stop. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 2 +- net/sunrpc/xprtrdma/svc_rdma_rw.c | 35 ++++++++--------------------------- 2 files changed, 9 insertions(+), 28 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 6f235d66e6fc..2ea119edae8f 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -187,7 +187,7 @@ extern int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, __be32 *wr_ch, const struct xdr_buf *xdr); extern int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, const struct svc_rdma_recv_ctxt *rctxt, - struct xdr_buf *xdr); + const struct xdr_buf *xdr); /* svc_rdma_sendto.c */ extern void svc_rdma_send_ctxts_destroy(struct svcxprt_rdma *rdma); diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 5f326c18b47c..88dbcf9e580f 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -516,7 +516,7 @@ static int svc_rdma_pages_write(struct svc_rdma_write_info *info, /** * svc_rdma_xb_write - Construct RDMA Writes to write an xdr_buf * @xdr: xdr_buf to write - * @info: pointer to write arguments + * @data: pointer to write arguments * * Returns: * On succes, returns zero @@ -524,9 +524,9 @@ static int svc_rdma_pages_write(struct svc_rdma_write_info *info, * %-ENOMEM if a resource has been exhausted * %-EIO if an rdma-rw error occurred */ -static int svc_rdma_xb_write(const struct xdr_buf *xdr, - struct svc_rdma_write_info *info) +static int svc_rdma_xb_write(const struct xdr_buf *xdr, void *data) { + struct svc_rdma_write_info *info = data; int ret; if (xdr->head[0].iov_len) { @@ -605,10 +605,10 @@ int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, __be32 *wr_ch, */ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, const struct svc_rdma_recv_ctxt *rctxt, - struct xdr_buf *xdr) + const struct xdr_buf *xdr) { struct svc_rdma_write_info *info; - int consumed, ret; + int ret; if (!rctxt->rc_reply_chunk) return 0; @@ -617,35 +617,16 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, if (!info) return -ENOMEM; - ret = svc_rdma_iov_write(info, &xdr->head[0]); + ret = svc_rdma_skip_payloads(xdr, rctxt, svc_rdma_xb_write, info); if (ret < 0) goto out_err; - consumed = xdr->head[0].iov_len; - - /* Send the page list in the Reply chunk only if the - * client did not provide Write chunks. - */ - if (!rctxt->rc_cur_payload && xdr->page_len) { - ret = svc_rdma_pages_write(info, xdr, xdr->head[0].iov_len, - xdr->page_len); - if (ret < 0) - goto out_err; - consumed += xdr->page_len; - } - - if (xdr->tail[0].iov_len) { - ret = svc_rdma_iov_write(info, &xdr->tail[0]); - if (ret < 0) - goto out_err; - consumed += xdr->tail[0].iov_len; - } ret = svc_rdma_post_chunk_ctxt(&info->wi_cc); if (ret < 0) goto out_err; - trace_svcrdma_send_reply_chunk(consumed); - return consumed; + trace_svcrdma_send_reply_chunk(xdr->len); + return xdr->len; out_err: svc_rdma_write_info_free(info);