From patchwork Mon Oct 26 18:53:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858269 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A1E7697 for ; Mon, 26 Oct 2020 18:54:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1036F207E8 for ; Mon, 26 Oct 2020 18:54:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dspodzqn" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1790970AbgJZSyC (ORCPT ); Mon, 26 Oct 2020 14:54:02 -0400 Received: from mail-qv1-f65.google.com ([209.85.219.65]:35592 "EHLO mail-qv1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1790967AbgJZSyB (ORCPT ); Mon, 26 Oct 2020 14:54:01 -0400 Received: by mail-qv1-f65.google.com with SMTP id cv1so4826103qvb.2; Mon, 26 Oct 2020 11:54:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=Kpfr4F0on43i4V/zu9BuZKzeiR4ktVja9X2eWer5VgQ=; b=dspodzqnm43fSgfcnYiLK5ikoj9NXgdHbh9/QAWwPyCYBTlVRu+faHRPotrYLRbEZE yWJqNcolEvsLNJK048Mv8M18keR7k6G9fxUvYQqiYeteUePqZ2nCDVYNysKvFFVMlP/f I3w5U8z4JD52vjfIaSpVrsWtH4UW5JA1eNIPgbHsgTKKNqGaHirRyuBDA2gPr7HFELfS ieDkovoGcJH1R46u5IAZG7IodgvwXniTjfvOPcwN5Rl6Td/dNVVWgRl2MTB/RSzLHViB q1ELOOpRuONWl7frwjO2wFla7QnBe5G7YRJbs/AC9GEPIcBVZb+iqkIX4sRn/l33ho47 8o+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=Kpfr4F0on43i4V/zu9BuZKzeiR4ktVja9X2eWer5VgQ=; b=Itdzjhq4PcLVPF7n0sAJnKEnA/8DFEN3MuvEQ7MMjZvw6tGHDlnXjnJ+oTutRRwnde SiNQZ9eCIXd/iW04qpAZBTOGqYZ5NapRrwhU/uyCdqtK5Pmc8MrUzT2KNcbE4UhUuaei HIahwEFhqyvGmgL6Aqn40FCeHxOfZ4d9dnB5kbn3/4qCLCbuYT+cHb/dykgYwqWvlfy3 veeANu88htPpEMKN/KE2zM/ecjhJSAT4TEguLG4TRsTl5hjL612dI0PVC0zYPaLlFTMv BpiVV/JWWP3uJL7smdfqSN7ODYXCTAOmfBmnA5XzQoqbrMf1yg6FgjmE/djiwrmDGE6g GQvQ== X-Gm-Message-State: AOAM533/0B5cnJJTgo0SVNH0c55gm83yh4UgY5Xqb92hgolKikayQdZ+ dv22AY0gKVQh0eBiAlGiV9UrZ8HSXi8= X-Google-Smtp-Source: ABdhPJxITajcEXTKryhppiIXCPihrfwQ5i5j1wDextPKAPbWwXUUNTGPUrfrGluFMLAVr8TqWNwk5g== X-Received: by 2002:ad4:55ea:: with SMTP id bu10mr19049524qvb.28.1603738439973; Mon, 26 Oct 2020 11:53:59 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id 128sm6924148qkm.76.2020.10.26.11.53.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:53:59 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QIrwIm013616; Mon, 26 Oct 2020 18:53:58 GMT Subject: [PATCH 01/20] SUNRPC: Adjust synopsis of xdr_buf_subsegment() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:53:58 -0400 Message-ID: <160373843825.1886.3133196140333045174.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: This enables xdr_buf_subsegment()'s callers to pass in a const pointer to that buffer. Signed-off-by: Chuck Lever --- include/linux/sunrpc/xdr.h | 3 ++- net/sunrpc/xdr.c | 5 ++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/include/linux/sunrpc/xdr.h b/include/linux/sunrpc/xdr.h index 9548d075e06d..ec2a22ccdc2a 100644 --- a/include/linux/sunrpc/xdr.h +++ b/include/linux/sunrpc/xdr.h @@ -183,7 +183,8 @@ xdr_adjust_iovec(struct kvec *iov, __be32 *p) */ extern void xdr_shift_buf(struct xdr_buf *, size_t); extern void xdr_buf_from_iov(struct kvec *, struct xdr_buf *); -extern int xdr_buf_subsegment(struct xdr_buf *, struct xdr_buf *, unsigned int, unsigned int); +extern int xdr_buf_subsegment(const struct xdr_buf *buf, struct xdr_buf *subbuf, + unsigned int base, unsigned int len); extern void xdr_buf_trim(struct xdr_buf *, unsigned int); extern int read_bytes_from_xdr_buf(struct xdr_buf *, unsigned int, void *, unsigned int); extern int write_bytes_to_xdr_buf(struct xdr_buf *, unsigned int, void *, unsigned int); diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c index 71e03b930b70..28f81769a27c 100644 --- a/net/sunrpc/xdr.c +++ b/net/sunrpc/xdr.c @@ -1379,9 +1379,8 @@ EXPORT_SYMBOL_GPL(xdr_buf_from_iov); * * Returns -1 if base of length are out of bounds. */ -int -xdr_buf_subsegment(struct xdr_buf *buf, struct xdr_buf *subbuf, - unsigned int base, unsigned int len) +int xdr_buf_subsegment(const struct xdr_buf *buf, struct xdr_buf *subbuf, + unsigned int base, unsigned int len) { subbuf->buflen = subbuf->len = len; if (base < buf->head[0].iov_len) { From patchwork Mon Oct 26 18:54:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858281 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 135FD1592 for ; Mon, 26 Oct 2020 18:54:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E598421D7B for ; Mon, 26 Oct 2020 18:54:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DTIoIjAF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1790978AbgJZSyJ (ORCPT ); Mon, 26 Oct 2020 14:54:09 -0400 Received: from mail-qv1-f68.google.com ([209.85.219.68]:36390 "EHLO mail-qv1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1790972AbgJZSyG (ORCPT ); Mon, 26 Oct 2020 14:54:06 -0400 Received: by mail-qv1-f68.google.com with SMTP id ev17so4819997qvb.3; Mon, 26 Oct 2020 11:54:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=sEDE7+ACL+xJFfG4qz46x0tNWG9y9S9DlLiy1LgVxXw=; b=DTIoIjAFwkf5REDTKO4hGxNtQX2IqCp28VxAIQa8ATyCOdg06T7pmKA6GyygUBchvr Xcl5BRHyaAIsj//r6vt4c2WRrOVl2U5zy+Eqozmcr6uqVTokoH1zBh9R0Gwr+sV7RMP7 +j7nJUesCoUcVudL/1kORVbB1uQLmI5MV4y+X8OBej/8k4BiTehx0m+3DpriPHCCteGC j/trTUt94bu3ma6UwPLpwL1zBG+ncSlsG2S+eE8gUJIz8OiKOklNx2aaS/xdvBT0TjFQ 5xGzwKK8ZBjzJk77azqUR7114vQP2udPaAhXKjuaVzSr30xzGko1286CQJXJoyg/hLAr wufw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=sEDE7+ACL+xJFfG4qz46x0tNWG9y9S9DlLiy1LgVxXw=; b=e6HMhjCz1Yo96f/R+OVLLE/OQIdzYUzNlAS0V7KGk6K88cCHbHWOMlzG/5yxqR+WYr XQdCb29ulw8zJA5D7Kl3VLUOuwyvb5SWOlVH26GTrM1BytbX3ppz2eHK4fHyvyIT2sjq LD/Yu2dj6O+grJYkYsJIGgVMmOJLU/JgGY8dPFPC3jBwZWH0rNBBXHv3K/1MmDYjX1dW M2znNrcgaX1hEomB1mDSqO9xtsD6g8Y0r6PmOoYne2TSDuJl57H7N7LONeIhKuXwsX42 dvMVJ3Urax5K8BSTriou+pVwJh5wMtj1/vma2WUEA3QYtA2++rSKgSM/oTl4IysKRqhK uZYA== X-Gm-Message-State: AOAM530wHLN4iRmUrH811xokdfrBjKPHC/eDU6j9kH/hMNJG9qsPKEPg c1TX+DSc2qsmt/b2gE72oyN+QQasYmM= X-Google-Smtp-Source: ABdhPJwS0xdtI+/pG1erCjV8nwW7lK7gR8pk7s54k8P1HLMb077j+eEGW+xaRZcyta1OldWs2r6G3g== X-Received: by 2002:ad4:45ca:: with SMTP id v10mr15245143qvt.48.1603738445317; Mon, 26 Oct 2020 11:54:05 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id v14sm6996695qta.44.2020.10.26.11.54.04 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:54:04 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QIs3Cu013619; Mon, 26 Oct 2020 18:54:03 GMT Subject: [PATCH 02/20] svcrdma: Const-ify the xdr_buf arguments From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:54:03 -0400 Message-ID: <160373844359.1886.5240330169912563769.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: Ensure the code in rw.c does not modify the argument, and enable callers to also use "const struct xdr_buf *". Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_rw.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 80a0c0e87590..d8b2e22c56c1 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -197,7 +197,7 @@ struct svc_rdma_write_info { __be32 *wi_segs; /* SGL constructor arguments */ - struct xdr_buf *wi_xdr; + const struct xdr_buf *wi_xdr; unsigned char *wi_base; unsigned int wi_next_off; @@ -405,7 +405,7 @@ static void svc_rdma_pagelist_to_sg(struct svc_rdma_write_info *info, struct svc_rdma_rw_ctxt *ctxt) { unsigned int sge_no, sge_bytes, page_off, page_no; - struct xdr_buf *xdr = info->wi_xdr; + const struct xdr_buf *xdr = info->wi_xdr; struct scatterlist *sg; struct page **page; From patchwork Mon Oct 26 18:54:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858275 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4CB92697 for ; Mon, 26 Oct 2020 18:54:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 20DD72168B for ; Mon, 26 Oct 2020 18:54:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QflcznoL" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1790490AbgJZSyN (ORCPT ); Mon, 26 Oct 2020 14:54:13 -0400 Received: from mail-qv1-f67.google.com ([209.85.219.67]:41532 "EHLO mail-qv1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1790983AbgJZSyL (ORCPT ); Mon, 26 Oct 2020 14:54:11 -0400 Received: by mail-qv1-f67.google.com with SMTP id t20so4804891qvv.8; Mon, 26 Oct 2020 11:54:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=5rXx/16uib6np7VcLhT7mPqPQTU8gPTikD6nti0nPkM=; b=QflcznoL1pqRsgK0u3urRKjkBYQ5IjcBl1uxYSOHHRnG8x8ZUfXGzaH8fCzFmvspgf BBWlCbAy0sI2pMmmz6wfxPbUmZZYaId07DsO0chlgp2eh3AN+lkG+j/2GHdTBhODL5lX D08OtYsewGWz8Eh+c3zinzQfaMOgzoT4qiXJ17BH/OIPblj6FMEZU/z5hgzjxcp8s+F5 XneV9g045RBpHyglXSISOB0rlfMBJsQuu1REhoNf0Bx2J/I+7rSvmlEIj+G8NHU6cREF q2AphsNUd26IzXPOkTE61g/UdzCzWpzTOx7ce7cxFjBXeFaWyOaa8nj9GuAqxbSxBwMI /qFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=5rXx/16uib6np7VcLhT7mPqPQTU8gPTikD6nti0nPkM=; b=I+l3TIKkVlyVYMYxWLqS/B4xJck3ownCdsCNHgqhpE8zWC/ZcD8JMnPwE8NU0ECmeS kRVdtF+wpccgI8ZLleXolcGKGrHV7MSOzcRMbMlxmf4wgKH0mIZTpQilriG4sSXzTlHD RXcI4HI5nWu2DQvkWfACq9Hf34TalZ6Gwl8AuoxGinK65/RsKYOWpvn7FSh92voW8nFP DXl4XlV1KWo5/PI4Hlhuo8Uxk2TFKCR0t3yAj9HVhH8ov0Y7EsXULn24DtYCqHtXe3Jy 1Da+AhYt5xzJDXBq5D6Od/3zIHDEsZUL3FLmtIMu4jTNNLfF0F7Qj3a1awSNZDXRxgfK /cQA== X-Gm-Message-State: AOAM531wn9zPNJHXAiialnuv/FA3X/OAkuIlznWJbkL2thuwskyXJn8S 0joTyYCCNDGicNSUPNnPe6pL0N/taGE= X-Google-Smtp-Source: ABdhPJxFUrGyOB/SSZNPD2te7SpWiQvih7piu3T+wIMlFnZuOhFlgTK+v44UHmKuKbx8nl00eeFH7g== X-Received: by 2002:a0c:8d05:: with SMTP id r5mr15704324qvb.31.1603738450538; Mon, 26 Oct 2020 11:54:10 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id 185sm7035475qke.16.2020.10.26.11.54.09 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:54:09 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QIs821013622; Mon, 26 Oct 2020 18:54:08 GMT Subject: [PATCH 03/20] svcrdma: Refactor the RDMA Write path From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:54:08 -0400 Message-ID: <160373844889.1886.5773551832943365156.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Refactor for subsequent changes. Constify the xdr_buf argument to ensure the code here does not modify it, and to enable callers to pass in a "const struct xdr_buf *". At the same time, rename the helper functions, which emit RDMA Writes, not RDMA Sends, and add documenting comments. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_rw.c | 56 +++++++++++++++++++++++-------------- 1 file changed, 35 insertions(+), 21 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index d8b2e22c56c1..03c32b441d32 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -493,27 +493,42 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, return -E2BIG; } -/* Send one of an xdr_buf's kvecs by itself. To send a Reply - * chunk, the whole RPC Reply is written back to the client. - * This function writes either the head or tail of the xdr_buf - * containing the Reply. +/** + * svc_rdma_iov_write - Construct RDMA Writes from an iov + * @info: pointer to write arguments + * @iov: kvec to write + * + * Returns: + * On succes, returns zero + * %-E2BIG if the client-provided Write chunk is too small + * %-ENOMEM if a resource has been exhausted + * %-EIO if an rdma-rw error occurred */ -static int svc_rdma_send_xdr_kvec(struct svc_rdma_write_info *info, - struct kvec *vec) +static int svc_rdma_iov_write(struct svc_rdma_write_info *info, + const struct kvec *iov) { - info->wi_base = vec->iov_base; + info->wi_base = iov->iov_base; return svc_rdma_build_writes(info, svc_rdma_vec_to_sg, - vec->iov_len); + iov->iov_len); } -/* Send an xdr_buf's page list by itself. A Write chunk is just - * the page list. A Reply chunk is @xdr's head, page list, and - * tail. This function is shared between the two types of chunk. +/** + * svc_rdma_pages_write - Construct RDMA Writes from pages + * @info: pointer to write arguments + * @xdr: xdr_buf with pages to write + * @offset: offset into the content of @xdr + * @length: number of bytes to write + * + * Returns: + * On succes, returns zero + * %-E2BIG if the client-provided Write chunk is too small + * %-ENOMEM if a resource has been exhausted + * %-EIO if an rdma-rw error occurred */ -static int svc_rdma_send_xdr_pagelist(struct svc_rdma_write_info *info, - struct xdr_buf *xdr, - unsigned int offset, - unsigned long length) +static int svc_rdma_pages_write(struct svc_rdma_write_info *info, + const struct xdr_buf *xdr, + unsigned int offset, + unsigned long length) { info->wi_xdr = xdr; info->wi_next_off = offset - xdr->head[0].iov_len; @@ -550,7 +565,7 @@ int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, __be32 *wr_ch, if (!info) return -ENOMEM; - ret = svc_rdma_send_xdr_pagelist(info, xdr, offset, length); + ret = svc_rdma_pages_write(info, xdr, offset, length); if (ret < 0) goto out_err; @@ -590,7 +605,7 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, if (!info) return -ENOMEM; - ret = svc_rdma_send_xdr_kvec(info, &xdr->head[0]); + ret = svc_rdma_iov_write(info, &xdr->head[0]); if (ret < 0) goto out_err; consumed = xdr->head[0].iov_len; @@ -599,16 +614,15 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, * client did not provide Write chunks. */ if (!rctxt->rc_write_list && xdr->page_len) { - ret = svc_rdma_send_xdr_pagelist(info, xdr, - xdr->head[0].iov_len, - xdr->page_len); + ret = svc_rdma_pages_write(info, xdr, xdr->head[0].iov_len, + xdr->page_len); if (ret < 0) goto out_err; consumed += xdr->page_len; } if (xdr->tail[0].iov_len) { - ret = svc_rdma_send_xdr_kvec(info, &xdr->tail[0]); + ret = svc_rdma_iov_write(info, &xdr->tail[0]); if (ret < 0) goto out_err; consumed += xdr->tail[0].iov_len; From patchwork Mon Oct 26 18:54:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858283 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CC02414B2 for ; Mon, 26 Oct 2020 18:54:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A604B2168B for ; Mon, 26 Oct 2020 18:54:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="thxQtmIE" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1781517AbgJZSyS (ORCPT ); Mon, 26 Oct 2020 14:54:18 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:35216 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1790986AbgJZSyS (ORCPT ); Mon, 26 Oct 2020 14:54:18 -0400 Received: by mail-qt1-f194.google.com with SMTP id c15so7527335qtc.2; Mon, 26 Oct 2020 11:54:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=/cB5PNF3Rd/l7UIrWtwLYYjC84a31izIF+s/sz3PIio=; b=thxQtmIEx0zML83QeU4wVAs/gV+ZXecp9oPGchXDKBCPhDXwwXRcYWVDJPAHSRAfHo 3625LbWiMM1HKASvtdhViLWARVUf5dDTNr138QBJyjzAv6WQI0jDLN0u1HZ8C1/K9b40 83oL1tXPokKzPgY5hn0rLleUomlIGb3ozehnC7khbahex6/Yu8LNucM0S9X07Zc+TCvP D3OaO6Ks5NTuYuS/EOcLT++i9Lm71YM31/ZlY/YpB+fhbvFu8/GDpeiXHDOgBEdnKF4J aP0BuCgON9dPVMMxChomwvFFOJYXS1GIQHlzXJ+e6+bOosfyHBjqzNaQT792vVgbAMQy CHZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=/cB5PNF3Rd/l7UIrWtwLYYjC84a31izIF+s/sz3PIio=; b=E854h7zlcHUHq7pg9S2jJxecYbZkIGfUMy3XiaIKeQYfrR79ycqo6SRuAVywDFovb8 TVO+DcYhSndvx13LW3RwrbZS3/ZTAjQpIRsLcY8W9UvaF47VnFN8gQfB8RXu9Lv/AlJo LbXub1uPLfEy5b1csIzlxv4MY2BTtA3SrhnmodrymLdsjED/HeYAOjKdexTHp9t125v+ A6oDlCU20wvDQfDQB0F0yZyPL4jZGdc2Uol1Ij+9RyCSu/cOFlknjcPQTqSIE0mm0++1 /v6g2Utcy8ZBWKGd55HJNtLIIkdL3vxV1KO2Up3ro1aEv+XjLEJPeEwnOLiBVYX3EoOC DNWw== X-Gm-Message-State: AOAM530MFi8RCNwTh2avZu1cQUw9bHprAmFIsg3a5QLOXK/t7qXq+rt0 yN4bJDfVpsBJhB/jbav1pG/yOTVWiJE= X-Google-Smtp-Source: ABdhPJxxjyExSPqPNaErf/P/aGnK0dbeMogABnZMYsmYb4QHRuINfxk02L/YdgiD0F7Ku4s0d4fRXg== X-Received: by 2002:ac8:397a:: with SMTP id t55mr17510990qtb.105.1603738455934; Mon, 26 Oct 2020 11:54:15 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id g11sm6943682qkl.30.2020.10.26.11.54.15 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:54:15 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QIsELs013625; Mon, 26 Oct 2020 18:54:14 GMT Subject: [PATCH 04/20] SUNRPC: Rename svc_encode_read_payload() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:54:14 -0400 Message-ID: <160373845420.1886.3075276814923041440.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: "result payload" is a less confusing name for these payloads. "READ payload" reflects only the NFS usage. Signed-off-by: Chuck Lever --- fs/nfsd/nfs4xdr.c | 2 +- include/linux/sunrpc/svc.h | 6 +++--- include/linux/sunrpc/svc_rdma.h | 4 ++-- include/linux/sunrpc/svc_xprt.h | 4 ++-- net/sunrpc/svc.c | 11 ++++++----- net/sunrpc/svcsock.c | 8 ++++---- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 8 ++++---- net/sunrpc/xprtrdma/svc_rdma_transport.c | 2 +- 8 files changed, 23 insertions(+), 22 deletions(-) diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c index 833a2c64dfe8..7e24fb3ca36e 100644 --- a/fs/nfsd/nfs4xdr.c +++ b/fs/nfsd/nfs4xdr.c @@ -3829,7 +3829,7 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp, read->rd_length = maxcount; if (nfserr) return nfserr; - if (svc_encode_read_payload(resp->rqstp, starting_len + 8, maxcount)) + if (svc_encode_result_payload(resp->rqstp, starting_len + 8, maxcount)) return nfserr_io; xdr_truncate_encode(xdr, starting_len + 8 + xdr_align_size(maxcount)); diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h index 386628b36bc7..c220b734fa69 100644 --- a/include/linux/sunrpc/svc.h +++ b/include/linux/sunrpc/svc.h @@ -519,9 +519,9 @@ void svc_wake_up(struct svc_serv *); void svc_reserve(struct svc_rqst *rqstp, int space); struct svc_pool * svc_pool_for_cpu(struct svc_serv *serv, int cpu); char * svc_print_addr(struct svc_rqst *, char *, size_t); -int svc_encode_read_payload(struct svc_rqst *rqstp, - unsigned int offset, - unsigned int length); +int svc_encode_result_payload(struct svc_rqst *rqstp, + unsigned int offset, + unsigned int length); unsigned int svc_fill_write_vector(struct svc_rqst *rqstp, struct page **pages, struct kvec *first, size_t total); diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 9dc3a3b88391..2b870a3f391b 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -207,8 +207,8 @@ extern void svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, struct svc_rdma_recv_ctxt *rctxt, int status); extern int svc_rdma_sendto(struct svc_rqst *); -extern int svc_rdma_read_payload(struct svc_rqst *rqstp, unsigned int offset, - unsigned int length); +extern int svc_rdma_result_payload(struct svc_rqst *rqstp, unsigned int offset, + unsigned int length); /* svc_rdma_transport.c */ extern struct svc_xprt_class svc_rdma_class; diff --git a/include/linux/sunrpc/svc_xprt.h b/include/linux/sunrpc/svc_xprt.h index aca35ab5cff2..92455e0d5244 100644 --- a/include/linux/sunrpc/svc_xprt.h +++ b/include/linux/sunrpc/svc_xprt.h @@ -21,8 +21,8 @@ struct svc_xprt_ops { int (*xpo_has_wspace)(struct svc_xprt *); int (*xpo_recvfrom)(struct svc_rqst *); int (*xpo_sendto)(struct svc_rqst *); - int (*xpo_read_payload)(struct svc_rqst *, unsigned int, - unsigned int); + int (*xpo_result_payload)(struct svc_rqst *, unsigned int, + unsigned int); void (*xpo_release_rqst)(struct svc_rqst *); void (*xpo_detach)(struct svc_xprt *); void (*xpo_free)(struct svc_xprt *); diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index c211b607239e..b41500645c3f 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -1622,7 +1622,7 @@ u32 svc_max_payload(const struct svc_rqst *rqstp) EXPORT_SYMBOL_GPL(svc_max_payload); /** - * svc_encode_read_payload - mark a range of bytes as a READ payload + * svc_encode_result_payload - mark a range of bytes as a result payload * @rqstp: svc_rqst to operate on * @offset: payload's byte offset in rqstp->rq_res * @length: size of payload, in bytes @@ -1630,12 +1630,13 @@ EXPORT_SYMBOL_GPL(svc_max_payload); * Returns zero on success, or a negative errno if a permanent * error occurred. */ -int svc_encode_read_payload(struct svc_rqst *rqstp, unsigned int offset, - unsigned int length) +int svc_encode_result_payload(struct svc_rqst *rqstp, unsigned int offset, + unsigned int length) { - return rqstp->rq_xprt->xpt_ops->xpo_read_payload(rqstp, offset, length); + return rqstp->rq_xprt->xpt_ops->xpo_result_payload(rqstp, offset, + length); } -EXPORT_SYMBOL_GPL(svc_encode_read_payload); +EXPORT_SYMBOL_GPL(svc_encode_result_payload); /** * svc_fill_write_vector - Construct data argument for VFS write call diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index c2752e2b9ce3..b248f2349437 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -181,8 +181,8 @@ static void svc_set_cmsg_data(struct svc_rqst *rqstp, struct cmsghdr *cmh) } } -static int svc_sock_read_payload(struct svc_rqst *rqstp, unsigned int offset, - unsigned int length) +static int svc_sock_result_payload(struct svc_rqst *rqstp, unsigned int offset, + unsigned int length) { return 0; } @@ -635,7 +635,7 @@ static const struct svc_xprt_ops svc_udp_ops = { .xpo_create = svc_udp_create, .xpo_recvfrom = svc_udp_recvfrom, .xpo_sendto = svc_udp_sendto, - .xpo_read_payload = svc_sock_read_payload, + .xpo_result_payload = svc_sock_result_payload, .xpo_release_rqst = svc_udp_release_rqst, .xpo_detach = svc_sock_detach, .xpo_free = svc_sock_free, @@ -1123,7 +1123,7 @@ static const struct svc_xprt_ops svc_tcp_ops = { .xpo_create = svc_tcp_create, .xpo_recvfrom = svc_tcp_recvfrom, .xpo_sendto = svc_tcp_sendto, - .xpo_read_payload = svc_sock_read_payload, + .xpo_result_payload = svc_sock_result_payload, .xpo_release_rqst = svc_tcp_release_rqst, .xpo_detach = svc_tcp_sock_detach, .xpo_free = svc_sock_free, diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index c3d588b149aa..c8411b4f3492 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -979,19 +979,19 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) } /** - * svc_rdma_read_payload - special processing for a READ payload + * svc_rdma_result_payload - special processing for a result payload * @rqstp: svc_rqst to operate on * @offset: payload's byte offset in @xdr * @length: size of payload, in bytes * * Returns zero on success. * - * For the moment, just record the xdr_buf location of the READ + * For the moment, just record the xdr_buf location of the result * payload. svc_rdma_sendto will use that location later when * we actually send the payload. */ -int svc_rdma_read_payload(struct svc_rqst *rqstp, unsigned int offset, - unsigned int length) +int svc_rdma_result_payload(struct svc_rqst *rqstp, unsigned int offset, + unsigned int length) { struct svc_rdma_recv_ctxt *rctxt = rqstp->rq_xprt_ctxt; diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c index fb044792b571..afba4e9d5425 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c @@ -80,7 +80,7 @@ static const struct svc_xprt_ops svc_rdma_ops = { .xpo_create = svc_rdma_create, .xpo_recvfrom = svc_rdma_recvfrom, .xpo_sendto = svc_rdma_sendto, - .xpo_read_payload = svc_rdma_read_payload, + .xpo_result_payload = svc_rdma_result_payload, .xpo_release_rqst = svc_rdma_release_rqst, .xpo_detach = svc_rdma_detach, .xpo_free = svc_rdma_free, From patchwork Mon Oct 26 18:54:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858287 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4C33A14B2 for ; Mon, 26 Oct 2020 18:54:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2AD37207E8 for ; Mon, 26 Oct 2020 18:54:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YYzVnud0" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1781641AbgJZSyY (ORCPT ); Mon, 26 Oct 2020 14:54:24 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:35065 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1781485AbgJZSyX (ORCPT ); Mon, 26 Oct 2020 14:54:23 -0400 Received: by mail-qk1-f193.google.com with SMTP id 140so9394934qko.2; Mon, 26 Oct 2020 11:54:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=uqD6g+Yc2Ltv6zvOspRgs9zml+qI/nm02oprbdY0Ahs=; b=YYzVnud07KmBzl7A8HFjxK1cQbpXYowMAnaehNISnnma+3xgncncvzKcfc9xrh5Uza Xf6JNgCn+j8Rg75qEhho25Q7xCIS9Ppwcp/uwhk5HdGAw4UKwb7pxvQpIFkPOCeNR681 EY6F8OKhHTyoOzQqmmhj6/ejPSFP6qrQuLbj7xRvQsgpgYeVxj2Gf+qRQeeUKrSxRzB3 6+MIn0HhNSMbpQzyhbs0uRNDyNO/fv5Ea/CkequJuRggLgOXmpaR0mJZ/yd+MJ+3ur3w t+zzqbwW5MaRm41sc1YKi70g7rDV4Zg9++fh0yN0N2jM+sExCgQcKHO7ISdKZblehNry fobA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=uqD6g+Yc2Ltv6zvOspRgs9zml+qI/nm02oprbdY0Ahs=; b=ttAYUzLpJbNWjFmIqvNvLIzYy048jLX4U8kyvLno+wnUV5oD9tJmO8ttWSZl6TjXuK 57zlTeQJ05axwotCIJaPj3+5AuMLRPcX6WCuDQo52UvYlbXvRkUat7NtVHipqgPvTT5M nuJX56wXFzLpP85dVUs4sjcONRXxbx51NFpsa+IqupjEzkIOF4h1c2Gka6DySKpNb8AC 8GMxs5SA82vPiddRCcqWCecKeGiUJ77XEPU4oH9ql9a0wkBcVS+7JmB9R97egkQQXVhs AmYZpYZkcdWiRbHXf3q9PPcn6yKwZU/7BMqFkEmUtalGIeOY0qH6R0hE9jxBqUiCA9EE izbg== X-Gm-Message-State: AOAM532uQu8dv5Py8mPCgXTOivemHbSMLthQIuIdOVMmmvNI/bNiASij 2EDiDo57t/bG+4OdsGPVtGi36yQpdb0= X-Google-Smtp-Source: ABdhPJzvOSxlUtRv3TFnjf6VQ7qciMGpdcnyO97JRNI+CGGUDgh9jb2vi7QY2KHRyx41zY5EH+jYDA== X-Received: by 2002:a37:74b:: with SMTP id 72mr19405719qkh.429.1603738461294; Mon, 26 Oct 2020 11:54:21 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id r62sm7245249qkd.80.2020.10.26.11.54.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:54:20 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QIsJRb013628; Mon, 26 Oct 2020 18:54:19 GMT Subject: [PATCH 05/20] NFSD: Invoke svc_encode_result_payload() in "read" NFSD encoders From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:54:19 -0400 Message-ID: <160373845951.1886.5782161983478765177.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Have the NFSD encoders annotate the boundaries of every direct-data-placement eligible result data payload. Then change svcrdma to use that annotation instead of the xdr->page_len when handling Write chunks. For NFSv4 on RDMA, that enables the ability to recognize multiple result payloads per compound. This is a pre-requisite for supporting multiple Write chunks per RPC transaction. Signed-off-by: Chuck Lever --- fs/nfsd/nfs3xdr.c | 4 ++++ fs/nfsd/nfs4xdr.c | 3 +++ fs/nfsd/nfsxdr.c | 4 ++++ net/sunrpc/xprtrdma/svc_rdma_sendto.c | 24 +++++++----------------- 4 files changed, 18 insertions(+), 17 deletions(-) diff --git a/fs/nfsd/nfs3xdr.c b/fs/nfsd/nfs3xdr.c index 9c23b6acf234..f38cd31dbbec 100644 --- a/fs/nfsd/nfs3xdr.c +++ b/fs/nfsd/nfs3xdr.c @@ -720,6 +720,8 @@ nfs3svc_encode_readlinkres(struct svc_rqst *rqstp, __be32 *p) *p = 0; rqstp->rq_res.tail[0].iov_len = 4 - (resp->len&3); } + svc_encode_result_payload(rqstp, rqstp->rq_res.head[0].iov_len, + resp->len); return 1; } else return xdr_ressize_check(rqstp, p); @@ -746,6 +748,8 @@ nfs3svc_encode_readres(struct svc_rqst *rqstp, __be32 *p) *p = 0; rqstp->rq_res.tail[0].iov_len = 4 - (resp->count & 3); } + svc_encode_result_payload(rqstp, rqstp->rq_res.head[0].iov_len, + resp->count); return 1; } else return xdr_ressize_check(rqstp, p); diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c index 7e24fb3ca36e..e56f0385022c 100644 --- a/fs/nfsd/nfs4xdr.c +++ b/fs/nfsd/nfs4xdr.c @@ -3777,6 +3777,8 @@ static __be32 nfsd4_encode_splice_read( buf->page_len = 0; return nfserr; } + svc_encode_result_payload(read->rd_rqstp, buf->head[0].iov_len, + maxcount); *(p++) = htonl(eof); *(p++) = htonl(maxcount); @@ -3921,6 +3923,7 @@ nfsd4_encode_readlink(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd xdr_truncate_encode(xdr, length_offset); return nfserr; } + svc_encode_result_payload(readlink->rl_rqstp, length_offset, maxcount); wire_count = htonl(maxcount); write_bytes_to_xdr_buf(xdr->buf, length_offset, &wire_count, 4); diff --git a/fs/nfsd/nfsxdr.c b/fs/nfsd/nfsxdr.c index 8a288c8fcd57..a23ea58a098e 100644 --- a/fs/nfsd/nfsxdr.c +++ b/fs/nfsd/nfsxdr.c @@ -483,6 +483,8 @@ nfssvc_encode_readlinkres(struct svc_rqst *rqstp, __be32 *p) *p = 0; rqstp->rq_res.tail[0].iov_len = 4 - (resp->len&3); } + svc_encode_result_payload(rqstp, rqstp->rq_res.head[0].iov_len, + resp->len); return 1; } @@ -507,6 +509,8 @@ nfssvc_encode_readres(struct svc_rqst *rqstp, __be32 *p) *p = 0; rqstp->rq_res.tail[0].iov_len = 4 - (resp->count&3); } + svc_encode_result_payload(rqstp, rqstp->rq_res.head[0].iov_len, + resp->count); return 1; } diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index c8411b4f3492..d6436c13d5c4 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -448,7 +448,6 @@ static ssize_t svc_rdma_encode_write_chunk(__be32 *src, * svc_rdma_encode_write_list - Encode RPC Reply's Write chunk list * @rctxt: Reply context with information about the RPC Call * @sctxt: Send context for the RPC Reply - * @length: size in bytes of the payload in the first Write chunk * * The client provides a Write chunk list in the Call message. Fill * in the segments in the first Write chunk in the Reply's transport @@ -465,12 +464,12 @@ static ssize_t svc_rdma_encode_write_chunk(__be32 *src, */ static ssize_t svc_rdma_encode_write_list(const struct svc_rdma_recv_ctxt *rctxt, - struct svc_rdma_send_ctxt *sctxt, - unsigned int length) + struct svc_rdma_send_ctxt *sctxt) { ssize_t len, ret; - ret = svc_rdma_encode_write_chunk(rctxt->rc_write_list, sctxt, length); + ret = svc_rdma_encode_write_chunk(rctxt->rc_write_list, sctxt, + rctxt->rc_read_payload_length); if (ret < 0) return ret; len = ret; @@ -923,21 +922,12 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) goto err0; if (wr_lst) { /* XXX: Presume the client sent only one Write chunk */ - unsigned long offset; - unsigned int length; - - if (rctxt->rc_read_payload_length) { - offset = rctxt->rc_read_payload_offset; - length = rctxt->rc_read_payload_length; - } else { - offset = xdr->head[0].iov_len; - length = xdr->page_len; - } - ret = svc_rdma_send_write_chunk(rdma, wr_lst, xdr, offset, - length); + ret = svc_rdma_send_write_chunk(rdma, wr_lst, xdr, + rctxt->rc_read_payload_offset, + rctxt->rc_read_payload_length); if (ret < 0) goto err2; - if (svc_rdma_encode_write_list(rctxt, sctxt, length) < 0) + if (svc_rdma_encode_write_list(rctxt, sctxt) < 0) goto err0; } else { if (xdr_stream_encode_item_absent(&sctxt->sc_stream) < 0) From patchwork Mon Oct 26 18:54:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858291 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CB43414B2 for ; Mon, 26 Oct 2020 18:54:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A4846207E8 for ; Mon, 26 Oct 2020 18:54:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Gs7X3+70" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1781671AbgJZSy2 (ORCPT ); Mon, 26 Oct 2020 14:54:28 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:41274 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1781485AbgJZSy2 (ORCPT ); Mon, 26 Oct 2020 14:54:28 -0400 Received: by mail-qt1-f193.google.com with SMTP id z33so7511343qth.8; Mon, 26 Oct 2020 11:54:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=QupHDxG9OyvXv5W0PToGomC+N8agkcUWuF2MeaqeYdo=; b=Gs7X3+70Gf30pYKl5L5yOWIECGijXesF87gSaYtNSwxT9hQZ5+ft9pEtpo+XTRq8cB frFkD8tzUIDvHzQmtpIDTR5txsl2qGyC7HWMbjO7Lu++Eyen8KUK7hMIkHYmQP4cuvlj Y1bXvKZJnl1lpcAt1vBB9st2j68DJre//MxUEOoGQasWs3mNLnI1B0Bk3jpvCNVok2Sr zuA3hakXiv6D1Wb1OHAxfIfHsXaX8qmqqjb3vcK8rOPVsvezniKOsnuj7jy8XzGu/MXK bhzy/T8ZNwDaFPp7Pg2xMpIRAMhPkL2qVY/E0Vdpc/R26h7jRjyHYRA77zixD0efAikH Pz+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=QupHDxG9OyvXv5W0PToGomC+N8agkcUWuF2MeaqeYdo=; b=ZPtDFfi6j3FGDAaKlxK4vl0iF5WCH8CTJYIWbaFukX2zDR25XRdI4MIeYR/UZFX7td ML9/bnKCvQK9i9Ji8GSLDkiDYA1EK4RfXUNcdIgJBCydzwGGx5Lhn9YeYvpobAletcNx drTlEjKml6I/11+nI7XSOeCYDJ7lqLh1k2WqZfGk1toc2r3zQy5NEJWnqoJdNsDS1Gbh d7aPb4/C1pWO4LoNjSv50Bi1xyfat0COBVV5K+Cw3Z97zbqbve20KbszMTZRChe3CXAv TlXRRDbw4UnizF4t5DmtHCyZ1XWg2gxtzksi85mx2CJAH8WSgmiyuwtbpR8Z8hQWsGN9 1/Ug== X-Gm-Message-State: AOAM532UmkkN5nVrIsu53f6i4G+qcYyy5PW+ZFynfOHDC7yW0HnMzSp7 A5NwP4X2rP8DBaxxDhdyuclpre8tbK4= X-Google-Smtp-Source: ABdhPJxBFoIiV+mXsoSLb5WjXaZxhb/wH/7C4v2Ygi9bS6Fvwzi8XSh3wFeYacVFPBUEBM8zTYYMDw== X-Received: by 2002:ac8:4808:: with SMTP id g8mr17621071qtq.18.1603738466678; Mon, 26 Oct 2020 11:54:26 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id q38sm7687156qtc.56.2020.10.26.11.54.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:54:25 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QIsOs0013631; Mon, 26 Oct 2020 18:54:24 GMT Subject: [PATCH 06/20] svcrdma: Post RDMA Writes while XDR encoding replies From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:54:24 -0400 Message-ID: <160373846482.1886.10871991261377476320.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org The only RPC/RDMA ordering requirement between RDMA Writes and RDMA Sends is that the responder must post the Writes on the Send queue before posting the Send that conveys the RPC Reply for that Write payload. The Linux NFS server implementation now has a transport method that can post result Payload Writes earlier than svc_rdma_sendto: ->xpo_result_payload() This gets RDMA Writes going earlier so they are more likely to be complete at the remote end before the Send completes. Some care must be taken with pulled-up Replies. We don't want to push the Write chunk and then send the same payload data via Send. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 4 +- net/sunrpc/xprtrdma/svc_rdma_rw.c | 52 +++++++++++++++++++++++------ net/sunrpc/xprtrdma/svc_rdma_sendto.c | 59 ++++++++++++++++++--------------- 3 files changed, 75 insertions(+), 40 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 2b870a3f391b..f5a3c852bb90 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -183,9 +183,7 @@ extern int svc_rdma_recv_read_chunk(struct svcxprt_rdma *rdma, struct svc_rqst *rqstp, struct svc_rdma_recv_ctxt *head, __be32 *p); extern int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, - __be32 *wr_ch, struct xdr_buf *xdr, - unsigned int offset, - unsigned long length); + __be32 *wr_ch, const struct xdr_buf *xdr); extern int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, const struct svc_rdma_recv_ctxt *rctxt, struct xdr_buf *xdr); diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 03c32b441d32..d732785d0380 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -536,13 +536,49 @@ static int svc_rdma_pages_write(struct svc_rdma_write_info *info, length); } +/** + * svc_rdma_xb_write - Construct RDMA Writes to write an xdr_buf + * @xdr: xdr_buf to write + * @info: pointer to write arguments + * + * Returns: + * On succes, returns zero + * %-E2BIG if the client-provided Write chunk is too small + * %-ENOMEM if a resource has been exhausted + * %-EIO if an rdma-rw error occurred + */ +static int svc_rdma_xb_write(const struct xdr_buf *xdr, + struct svc_rdma_write_info *info) +{ + int ret; + + if (xdr->head[0].iov_len) { + ret = svc_rdma_iov_write(info, &xdr->head[0]); + if (ret < 0) + return ret; + } + + if (xdr->page_len) { + ret = svc_rdma_pages_write(info, xdr, xdr->head[0].iov_len, + xdr->page_len); + if (ret < 0) + return ret; + } + + if (xdr->tail[0].iov_len) { + ret = svc_rdma_iov_write(info, &xdr->tail[0]); + if (ret < 0) + return ret; + } + + return xdr->len; +} + /** * svc_rdma_send_write_chunk - Write all segments in a Write chunk * @rdma: controlling RDMA transport * @wr_ch: Write chunk provided by client * @xdr: xdr_buf containing the data payload - * @offset: payload's byte offset in @xdr - * @length: size of payload, in bytes * * Returns a non-negative number of bytes the chunk consumed, or * %-E2BIG if the payload was larger than the Write chunk, @@ -552,21 +588,17 @@ static int svc_rdma_pages_write(struct svc_rdma_write_info *info, * %-EIO if rdma_rw initialization failed (DMA mapping, etc). */ int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, __be32 *wr_ch, - struct xdr_buf *xdr, - unsigned int offset, unsigned long length) + const struct xdr_buf *xdr) { struct svc_rdma_write_info *info; int ret; - if (!length) - return 0; - info = svc_rdma_write_info_alloc(rdma, wr_ch); if (!info) return -ENOMEM; - ret = svc_rdma_pages_write(info, xdr, offset, length); - if (ret < 0) + ret = svc_rdma_xb_write(xdr, info); + if (ret != xdr->len) goto out_err; ret = svc_rdma_post_chunk_ctxt(&info->wi_cc); @@ -574,7 +606,7 @@ int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, __be32 *wr_ch, goto out_err; trace_svcrdma_send_write_chunk(xdr->page_len); - return length; + return xdr->len; out_err: svc_rdma_write_info_free(info); diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index d6436c13d5c4..fb6ba1177fd8 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -468,11 +468,14 @@ svc_rdma_encode_write_list(const struct svc_rdma_recv_ctxt *rctxt, { ssize_t len, ret; - ret = svc_rdma_encode_write_chunk(rctxt->rc_write_list, sctxt, - rctxt->rc_read_payload_length); - if (ret < 0) - return ret; - len = ret; + len = 0; + if (rctxt->rc_write_list) { + ret = svc_rdma_encode_write_chunk(rctxt->rc_write_list, sctxt, + rctxt->rc_read_payload_length); + if (ret < 0) + return ret; + len = ret; + } /* Terminate the Write list */ ret = xdr_stream_encode_item_absent(&sctxt->sc_stream); @@ -556,11 +559,13 @@ static bool svc_rdma_pull_up_needed(struct svcxprt_rdma *rdma, const struct svc_rdma_recv_ctxt *rctxt, struct xdr_buf *xdr) { + bool write_chunk_present = rctxt && rctxt->rc_write_list; int elements; /* For small messages, copying bytes is cheaper than DMA mapping. */ - if (sctxt->sc_hdrbuf.len + xdr->len < RPCRDMA_PULLUP_THRESH) + if (!write_chunk_present && + sctxt->sc_hdrbuf.len + xdr->len < RPCRDMA_PULLUP_THRESH) return true; /* Check whether the xdr_buf has more elements than can @@ -893,9 +898,7 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) container_of(xprt, struct svcxprt_rdma, sc_xprt); struct svc_rdma_recv_ctxt *rctxt = rqstp->rq_xprt_ctxt; __be32 *rdma_argp = rctxt->rc_recv_buf; - __be32 *wr_lst = rctxt->rc_write_list; __be32 *rp_ch = rctxt->rc_reply_chunk; - struct xdr_buf *xdr = &rqstp->rq_res; struct svc_rdma_send_ctxt *sctxt; __be32 *p; int ret; @@ -920,19 +923,8 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) if (svc_rdma_encode_read_list(sctxt) < 0) goto err0; - if (wr_lst) { - /* XXX: Presume the client sent only one Write chunk */ - ret = svc_rdma_send_write_chunk(rdma, wr_lst, xdr, - rctxt->rc_read_payload_offset, - rctxt->rc_read_payload_length); - if (ret < 0) - goto err2; - if (svc_rdma_encode_write_list(rctxt, sctxt) < 0) - goto err0; - } else { - if (xdr_stream_encode_item_absent(&sctxt->sc_stream) < 0) - goto err0; - } + if (svc_rdma_encode_write_list(rctxt, sctxt) < 0) + goto err0; if (rp_ch) { ret = svc_rdma_send_reply_chunk(rdma, rctxt, &rqstp->rq_res); if (ret < 0) @@ -974,16 +966,25 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) * @offset: payload's byte offset in @xdr * @length: size of payload, in bytes * - * Returns zero on success. - * - * For the moment, just record the xdr_buf location of the result - * payload. svc_rdma_sendto will use that location later when - * we actually send the payload. + * Return values: + * On success, returns the number of bytes in the payload + * If nothing needs to be done, returns zero + * %-EMSGSIZE on XDR buffer overflow + * %-E2BIG if the payload was larger than the Write chunk + * %-EINVAL if client provided too many segments + * %-ENOMEM if rdma_rw context pool was exhausted + * %-ENOTCONN if posting failed (connection is lost) + * %-EIO if rdma_rw initialization failed (DMA mapping, etc) */ int svc_rdma_result_payload(struct svc_rqst *rqstp, unsigned int offset, unsigned int length) { struct svc_rdma_recv_ctxt *rctxt = rqstp->rq_xprt_ctxt; + struct svcxprt_rdma *rdma; + struct xdr_buf subbuf; + + if (!rctxt->rc_write_list || !length) + return 0; /* XXX: Just one READ payload slot for now, since our * transport implementation currently supports only one @@ -992,5 +993,9 @@ int svc_rdma_result_payload(struct svc_rqst *rqstp, unsigned int offset, rctxt->rc_read_payload_offset = offset; rctxt->rc_read_payload_length = length; - return 0; + if (xdr_buf_subsegment(&rqstp->rq_res, &subbuf, offset, length)) + return -EMSGSIZE; + + rdma = container_of(rqstp->rq_xprt, struct svcxprt_rdma, sc_xprt); + return svc_rdma_send_write_chunk(rdma, rctxt->rc_write_list, &subbuf); } From patchwork Mon Oct 26 18:54:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858295 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DCA77697 for ; Mon, 26 Oct 2020 18:54:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B8CB7207E8 for ; Mon, 26 Oct 2020 18:54:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="aaPDuk+L" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1781686AbgJZSyd (ORCPT ); Mon, 26 Oct 2020 14:54:33 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:37781 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1781485AbgJZSyd (ORCPT ); Mon, 26 Oct 2020 14:54:33 -0400 Received: by mail-qk1-f196.google.com with SMTP id z6so9410902qkz.4; Mon, 26 Oct 2020 11:54:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=G02z9jH3D5SvIJXqLArGLnDX9Qn0gnV9J50q6rxowBc=; b=aaPDuk+LpseSAFMloJAolR3bP1M/hgsE1pmR/9VF55/SkdNBwwI5liT+1KMwAXK0Rt eBDJkoFmYe8keV1GJa6cMGuL3IUGQP9f0LWTCbYUd0zQ8OwMtslxRyPrXX3XmWdSDz5E aIw5t2rIKfbRbLYLOWNAeYujgsHO2z0rHkYVyOn4kUhvPaelIPHnsoqjkvTVijY0R81Y CFztISFIpUfI49yElAp+JtYn/YnxEMk9GQ0802VCICvu20t9LnVFUbsXrmw/wHZjeG1V kHtz7jLtS+RmPst8OkvQdsv9tT3cpEHoPxxiyLeGPHO81yW2JhD8V4Pl254GfOSHV4Rd vv6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=G02z9jH3D5SvIJXqLArGLnDX9Qn0gnV9J50q6rxowBc=; b=ej4dxhyUFhqjt4BJNQqErdv/9smTel2/H1YtMKsZdW5lCRHlhxb7y+HcO98YSKj7ii 8FrFgkJoHhhXlIxSkUxV/80mfTZGt+aHwzoZmXub+auX9hBfF9ZQuKq/P/1C+rObgn7e ZKU0PghzrtJAtuqCSD06ft1n2toe4yE269l75fvbKRTcr7jd5J0loPfVMDGag4++N+69 YnqkQPJQeHqMSX6Y6UsIxVWeaEBZ5IVafhvek4HaiCTOPDQ4iaAkTqs9ix9IjCuaG8gS Go1FvIWyQuM8ANvmqNBt1Fq7v3OhyTJe2KubYbWxWoaUyxtTj56t//gfehsC/mSGdQlc QdBA== X-Gm-Message-State: AOAM530GikGG5snHl3kpOJvTgFO08bRfOHE7mATVIl+IH44XRKt85uFZ Je6nOBDFobZOSvBTeSIJ5+6pEBC7fTI= X-Google-Smtp-Source: ABdhPJxsJoq21tFAHN8g5yTPkffCClNYI9uRJbiNDh2GFkBBK/eaj67b894Abjs8Zl4lcsTrD74IdA== X-Received: by 2002:a37:a9cf:: with SMTP id s198mr19012166qke.216.1603738471789; Mon, 26 Oct 2020 11:54:31 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id t65sm7409334qkc.52.2020.10.26.11.54.30 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:54:31 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QIsUVD013634; Mon, 26 Oct 2020 18:54:30 GMT Subject: [PATCH 07/20] svcrdma: Clean up svc_rdma_encode_reply_chunk() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:54:30 -0400 Message-ID: <160373847013.1886.15364356647550565452.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Refactor: Match the control flow of svc_rdma_encode_write_list(). Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_rw.c | 3 +++ net/sunrpc/xprtrdma/svc_rdma_sendto.c | 23 +++++++++++------------ 2 files changed, 14 insertions(+), 12 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index d732785d0380..5f667d964cd6 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -633,6 +633,9 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, struct svc_rdma_write_info *info; int consumed, ret; + if (!rctxt->rc_reply_chunk) + return 0; + info = svc_rdma_write_info_alloc(rdma, rctxt->rc_reply_chunk); if (!info) return -ENOMEM; diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index fb6ba1177fd8..3e7ba06788b0 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -504,6 +504,9 @@ svc_rdma_encode_reply_chunk(const struct svc_rdma_recv_ctxt *rctxt, struct svc_rdma_send_ctxt *sctxt, unsigned int length) { + if (!rctxt->rc_reply_chunk) + return xdr_stream_encode_item_absent(&sctxt->sc_stream); + return svc_rdma_encode_write_chunk(rctxt->rc_reply_chunk, sctxt, length); } @@ -898,7 +901,6 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) container_of(xprt, struct svcxprt_rdma, sc_xprt); struct svc_rdma_recv_ctxt *rctxt = rqstp->rq_xprt_ctxt; __be32 *rdma_argp = rctxt->rc_recv_buf; - __be32 *rp_ch = rctxt->rc_reply_chunk; struct svc_rdma_send_ctxt *sctxt; __be32 *p; int ret; @@ -916,25 +918,22 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) rpcrdma_fixed_maxsz * sizeof(*p)); if (!p) goto err0; + + ret = svc_rdma_send_reply_chunk(rdma, rctxt, &rqstp->rq_res); + if (ret < 0) + goto err2; + *p++ = *rdma_argp; *p++ = *(rdma_argp + 1); *p++ = rdma->sc_fc_credits; - *p = rp_ch ? rdma_nomsg : rdma_msg; + *p = rctxt->rc_reply_chunk ? rdma_nomsg : rdma_msg; if (svc_rdma_encode_read_list(sctxt) < 0) goto err0; if (svc_rdma_encode_write_list(rctxt, sctxt) < 0) goto err0; - if (rp_ch) { - ret = svc_rdma_send_reply_chunk(rdma, rctxt, &rqstp->rq_res); - if (ret < 0) - goto err2; - if (svc_rdma_encode_reply_chunk(rctxt, sctxt, ret) < 0) - goto err0; - } else { - if (xdr_stream_encode_item_absent(&sctxt->sc_stream) < 0) - goto err0; - } + if (svc_rdma_encode_reply_chunk(rctxt, sctxt, ret) < 0) + goto err0; ret = svc_rdma_send_reply_msg(rdma, sctxt, rctxt, rqstp); if (ret < 0) From patchwork Mon Oct 26 18:54:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858303 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AF6FB14C0 for ; Mon, 26 Oct 2020 18:54:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5F5C921D7B for ; Mon, 26 Oct 2020 18:54:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ntS/AAZd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1781690AbgJZSyl (ORCPT ); Mon, 26 Oct 2020 14:54:41 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:35100 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1781703AbgJZSyk (ORCPT ); Mon, 26 Oct 2020 14:54:40 -0400 Received: by mail-qk1-f196.google.com with SMTP id 140so9395811qko.2; Mon, 26 Oct 2020 11:54:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=wu3XOdGbkHibd96A6V6XV5F8it0NUJhYHwRFYpHKKcY=; b=ntS/AAZdDqY5VuKVb0j7GpI9CQ7JFYri6JobA2wEMtznMVUTCvdLUhhtsjEMuc0OKX voRO2XZ+BjM8BvC7BKX6ZSVry/meAv5NWPuYUwj3AVDkNgb0501m5CKB7pLycoAa+yXe iFNxRfjeJr5P05vxOA8I7Ob+KvREX1MMMitCnCjFIqCiVk7Fc08WVBlpDiEH5rJuipot yZS9d56L5L2EwgR438kP4v2Gbxylqmy670LwK5WT9tSp0VHbSyMe9M0y72AEF7XTSOIp UXN3MA1dvsdANwGmuX5zdhVxggweobfq+1Ccfwlpj8ER2uNdaU97ZdP4iZztKWJ6He8K DaaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=wu3XOdGbkHibd96A6V6XV5F8it0NUJhYHwRFYpHKKcY=; b=aJkhHO35IXlqZmwqWzqULKsFLjMSBVKfaLLMAA8mnkg6kHJ3o32D+DfllhsKd1g2k6 NCYhJYnX7s11p84zUq+BZMjMbGNqZw12v/xPCRzqaZ0+ylpGCEq4rnhSCGRrKTy4Hf1E f7YKXUzSXX/VTHvAAy8cqCEYQMJuq2gd+4Sel4mhOCgUF7qB4e9Q66u2KgY7tfI5jwH1 P0J3+5GWrkgLjP06MEf8kEqLJ/x7ciUbQdYUbTV1hnUU5gdpOQlQIA0YzyoTWeFcVEMi xSxyy4kpuMjobmmicttrC4aDC4LWuAuHjCpVmVa1LlLtLdEl1epjDmmwcvMLhp6F8dq6 jljQ== X-Gm-Message-State: AOAM530oVodMeNfRaJ29s3ru0lamy+7eqGjlTTz6hdiSNXhSsXOlx9M8 mc5O9qQcPCLKIGr5NOwIpnUFOYtvqLs= X-Google-Smtp-Source: ABdhPJwkfvRrd+8Wr8AHVBsJ0RbuzCakg82wrzURNaeBby8GlziJ5pNqiDmC1WZWl18YuGXbotGUHA== X-Received: by 2002:a05:620a:1221:: with SMTP id v1mr18531416qkj.98.1603738477360; Mon, 26 Oct 2020 11:54:37 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id f4sm7419742qtd.35.2020.10.26.11.54.36 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:54:36 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QIsZRH013637; Mon, 26 Oct 2020 18:54:35 GMT Subject: [PATCH 08/20] svcrdma: Add a "parsed chunk list" data structure From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:54:35 -0400 Message-ID: <160373847543.1886.8846090291470374221.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org This simple data structure binds the location of each data payload inside of an RPC message to the chunk that will be used to push it to or pull it from the client. There are several benefits to this small additional overhead: * It enables support for more than one chunk in incoming Read and Write lists. * It translates the version-specific on-the-wire format into a generic in-memory structure, enabling support for multiple versions of the RPC/RDMA transport protocol. * It enables the server to re-organize a chunk list if it needs to adjust where Read chunk data lands in server memory without altering the contents of the XDR-encoded Receive buffer. Construction of these lists is done while sanity checking each incoming RPC/RDMA header. Subsequent patches will make use of the generated data structures. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 12 + include/linux/sunrpc/svc_rdma_pcl.h | 128 +++++++++++++ include/trace/events/rpcrdma.h | 75 +++++++- net/sunrpc/xprtrdma/Makefile | 2 net/sunrpc/xprtrdma/svc_rdma_pcl.c | 306 +++++++++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 196 ++++++++++++-------- 6 files changed, 635 insertions(+), 84 deletions(-) create mode 100644 include/linux/sunrpc/svc_rdma_pcl.h create mode 100644 net/sunrpc/xprtrdma/svc_rdma_pcl.c diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index f5a3c852bb90..a89d4209fe2a 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -47,6 +47,8 @@ #include #include #include +#include + #include #include @@ -142,8 +144,18 @@ struct svc_rdma_recv_ctxt { unsigned int rc_page_count; unsigned int rc_hdr_count; u32 rc_inv_rkey; + + struct svc_rdma_pcl rc_call_pcl; + + struct svc_rdma_pcl rc_read_pcl; + __be32 *rc_write_list; + struct svc_rdma_chunk *rc_cur_result_payload; + struct svc_rdma_pcl rc_write_pcl; + __be32 *rc_reply_chunk; + struct svc_rdma_pcl rc_reply_pcl; + unsigned int rc_read_payload_offset; unsigned int rc_read_payload_length; struct page *rc_pages[RPCSVC_MAXPAGES]; diff --git a/include/linux/sunrpc/svc_rdma_pcl.h b/include/linux/sunrpc/svc_rdma_pcl.h new file mode 100644 index 000000000000..7516ad0fae80 --- /dev/null +++ b/include/linux/sunrpc/svc_rdma_pcl.h @@ -0,0 +1,128 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2020, Oracle and/or its affiliates + */ + +#ifndef SVC_RDMA_PCL_H +#define SVC_RDMA_PCL_H + +#include + +struct svc_rdma_segment { + u32 rs_handle; + u32 rs_length; + u64 rs_offset; +}; + +struct svc_rdma_chunk { + struct list_head ch_list; + + u32 ch_position; + u32 ch_length; + u32 ch_payload_length; + + u32 ch_segcount; + struct svc_rdma_segment ch_segments[]; +}; + +struct svc_rdma_pcl { + unsigned int cl_count; + struct list_head cl_chunks; +}; + +/** + * pcl_init - Initialize a parsed chunk list + * @pcl: parsed chunk list to initialize + * + */ +static inline void pcl_init(struct svc_rdma_pcl *pcl) +{ + INIT_LIST_HEAD(&pcl->cl_chunks); +} + +/** + * pcl_is_empty - Return true if parsed chunk list is empty + * @pcl: parsed chunk list + * + */ +static inline bool pcl_is_empty(const struct svc_rdma_pcl *pcl) +{ + return list_empty(&pcl->cl_chunks); +} + +/** + * pcl_first_chunk - Return first chunk in a parsed chunk list + * @pcl: parsed chunk list + * + * Returns the first chunk in the list, or NULL if the list is empty. + */ +static inline struct svc_rdma_chunk * +pcl_first_chunk(const struct svc_rdma_pcl *pcl) +{ + if (pcl_is_empty(pcl)) + return NULL; + return list_first_entry(&pcl->cl_chunks, struct svc_rdma_chunk, + ch_list); +} + +/** + * pcl_next_chunk - Return next chunk in a parsed chunk list + * @pcl: a parsed chunk list + * @chunk: chunk in @pcl + * + * Returns the next chunk in the list, or NULL if @chunk is already last. + */ +static inline struct svc_rdma_chunk * +pcl_next_chunk(const struct svc_rdma_pcl *pcl, struct svc_rdma_chunk *chunk) +{ + if (list_is_last(&chunk->ch_list, &pcl->cl_chunks)) + return NULL; + return list_next_entry(chunk, ch_list); +} + +/** + * pcl_for_each_chunk - Iterate over chunks in a parsed chunk list + * @pos: the loop cursor + * @pcl: a parsed chunk list + */ +#define pcl_for_each_chunk(pos, pcl) \ + for (pos = list_first_entry(&(pcl)->cl_chunks, struct svc_rdma_chunk, ch_list); \ + &pos->ch_list != &(pcl)->cl_chunks; \ + pos = list_next_entry(pos, ch_list)) + +/** + * pcl_for_each_segment - Iterate over segments in a parsed chunk + * @pos: the loop cursor + * @chunk: a parsed chunk + */ +#define pcl_for_each_segment(pos, chunk) \ + for (pos = &(chunk)->ch_segments[0]; \ + pos <= &(chunk)->ch_segments[(chunk)->ch_segcount - 1]; \ + pos++) + +/** + * pcl_chunk_end_offset - Return offset of byte range following @chunk + * @chunk: chunk in @pcl + * + * Returns starting offset of the region just after @chunk + */ +static inline unsigned int +pcl_chunk_end_offset(const struct svc_rdma_chunk *chunk) +{ + return xdr_align_size(chunk->ch_position + chunk->ch_payload_length); +} + +struct svc_rdma_recv_ctxt; + +extern void pcl_free(struct svc_rdma_pcl *pcl); +extern bool pcl_alloc_call(struct svc_rdma_recv_ctxt *rctxt, __be32 *p); +extern bool pcl_alloc_read(struct svc_rdma_recv_ctxt *rctxt, __be32 *p); +extern bool pcl_alloc_write(struct svc_rdma_recv_ctxt *rctxt, + struct svc_rdma_pcl *pcl, __be32 *p); +extern int pcl_process_nonpayloads(const struct svc_rdma_pcl *pcl, + const struct xdr_buf *xdr, + int (*actor)(const struct xdr_buf *, + void *), + void *data); + +#endif /* SVC_RDMA_PCL_H */ diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index bf1065772228..72b941aef43b 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1446,12 +1446,83 @@ DECLARE_EVENT_CLASS(svcrdma_segment_event, ), \ TP_ARGS(handle, length, offset)) -DEFINE_SEGMENT_EVENT(decode_wseg); -DEFINE_SEGMENT_EVENT(encode_rseg); DEFINE_SEGMENT_EVENT(send_rseg); DEFINE_SEGMENT_EVENT(encode_wseg); DEFINE_SEGMENT_EVENT(send_wseg); +TRACE_EVENT(svcrdma_decode_rseg, + TP_PROTO( + const struct rpc_rdma_cid *cid, + const struct svc_rdma_chunk *chunk, + const struct svc_rdma_segment *segment + ), + + TP_ARGS(cid, chunk, segment), + + TP_STRUCT__entry( + __field(u32, cq_id) + __field(int, completion_id) + __field(u32, segno) + __field(u32, position) + __field(u32, handle) + __field(u32, length) + __field(u64, offset) + ), + + TP_fast_assign( + __entry->cq_id = cid->ci_queue_id; + __entry->completion_id = cid->ci_completion_id; + __entry->segno = chunk->ch_segcount; + __entry->position = chunk->ch_position; + __entry->handle = segment->rs_handle; + __entry->length = segment->rs_length; + __entry->offset = segment->rs_offset; + ), + + TP_printk("cq_id=%u cid=%d segno=%u position=%u %u@0x%016llx:0x%08x", + __entry->cq_id, __entry->completion_id, + __entry->segno, __entry->position, __entry->length, + (unsigned long long)__entry->offset, __entry->handle + ) +); + +TRACE_EVENT(svcrdma_decode_wseg, + TP_PROTO( + const struct rpc_rdma_cid *cid, + const struct svc_rdma_chunk *chunk, + u32 segno + ), + + TP_ARGS(cid, chunk, segno), + + TP_STRUCT__entry( + __field(u32, cq_id) + __field(int, completion_id) + __field(u32, segno) + __field(u32, handle) + __field(u32, length) + __field(u64, offset) + ), + + TP_fast_assign( + const struct svc_rdma_segment *segment = + &chunk->ch_segments[segno]; + + __entry->cq_id = cid->ci_queue_id; + __entry->completion_id = cid->ci_completion_id; + __entry->segno = segno; + __entry->handle = segment->rs_handle; + __entry->length = segment->rs_length; + __entry->offset = segment->rs_offset; + ), + + TP_printk("cq_id=%u cid=%d segno=%u %u@0x%016llx:0x%08x", + __entry->cq_id, __entry->completion_id, + __entry->segno, __entry->length, + (unsigned long long)__entry->offset, __entry->handle + ) +); + DECLARE_EVENT_CLASS(svcrdma_chunk_event, TP_PROTO( u32 length diff --git a/net/sunrpc/xprtrdma/Makefile b/net/sunrpc/xprtrdma/Makefile index 8ed0377d7a18..55b21bae866d 100644 --- a/net/sunrpc/xprtrdma/Makefile +++ b/net/sunrpc/xprtrdma/Makefile @@ -4,5 +4,5 @@ obj-$(CONFIG_SUNRPC_XPRT_RDMA) += rpcrdma.o rpcrdma-y := transport.o rpc_rdma.o verbs.o frwr_ops.o \ svc_rdma.o svc_rdma_backchannel.o svc_rdma_transport.o \ svc_rdma_sendto.o svc_rdma_recvfrom.o svc_rdma_rw.o \ - module.o + svc_rdma_pcl.o module.o rpcrdma-$(CONFIG_SUNRPC_BACKCHANNEL) += backchannel.o diff --git a/net/sunrpc/xprtrdma/svc_rdma_pcl.c b/net/sunrpc/xprtrdma/svc_rdma_pcl.c new file mode 100644 index 000000000000..b63cfeaa2923 --- /dev/null +++ b/net/sunrpc/xprtrdma/svc_rdma_pcl.c @@ -0,0 +1,306 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2020 Oracle. All rights reserved. + */ + +#include +#include + +#include "xprt_rdma.h" +#include + +/** + * pcl_free - Release all memory associated with a parsed chunk list + * @pcl: parsed chunk list + * + */ +void pcl_free(struct svc_rdma_pcl *pcl) +{ + while (!list_empty(&pcl->cl_chunks)) { + struct svc_rdma_chunk *chunk; + + chunk = pcl_first_chunk(pcl); + list_del(&chunk->ch_list); + kfree(chunk); + } +} + +static struct svc_rdma_chunk *pcl_alloc_chunk(u32 segcount, u32 position) +{ + struct svc_rdma_chunk *chunk; + + chunk = kmalloc(struct_size(chunk, ch_segments, segcount), GFP_KERNEL); + if (!chunk) + return NULL; + + chunk->ch_position = position; + chunk->ch_length = 0; + chunk->ch_payload_length = 0; + chunk->ch_segcount = 0; + return chunk; +} + +static struct svc_rdma_chunk * +pcl_lookup_position(struct svc_rdma_pcl *pcl, u32 position) +{ + struct svc_rdma_chunk *pos; + + pcl_for_each_chunk(pos, pcl) { + if (pos->ch_position == position) + return pos; + } + return NULL; +} + +static void pcl_insert_position(struct svc_rdma_pcl *pcl, + struct svc_rdma_chunk *chunk) +{ + struct svc_rdma_chunk *pos; + + pcl_for_each_chunk(pos, pcl) { + if (pos->ch_position > chunk->ch_position) + break; + } + __list_add(&chunk->ch_list, pos->ch_list.prev, &pos->ch_list); + pcl->cl_count++; +} + +static void pcl_set_read_segment(const struct svc_rdma_recv_ctxt *rctxt, + struct svc_rdma_chunk *chunk, + u32 handle, u32 length, u64 offset) +{ + struct svc_rdma_segment *segment; + + segment = &chunk->ch_segments[chunk->ch_segcount]; + segment->rs_handle = handle; + segment->rs_length = length; + segment->rs_offset = offset; + + trace_svcrdma_decode_rseg(&rctxt->rc_cid, chunk, segment); + + chunk->ch_length += length; + chunk->ch_segcount++; +} + +/** + * pcl_alloc_call - Construct a parsed chunk list for the Call body + * @rctxt: Ingress receive context + * @p: Start of an un-decoded Read list + * + * Assumptions: + * - The incoming Read list has already been sanity checked. + * - cl_count is already set to the number of segments in + * the un-decoded list. + * - The list might not be in order by position. + * + * Return values: + * %true: Parsed chunk list was successfully constructed, and + * cl_count is updated to be the number of chunks (ie. + * unique positions) in the Read list. + * %false: Memory allocation failed. + */ +bool pcl_alloc_call(struct svc_rdma_recv_ctxt *rctxt, __be32 *p) +{ + struct svc_rdma_pcl *pcl = &rctxt->rc_call_pcl; + unsigned int i, segcount = pcl->cl_count; + + pcl->cl_count = 0; + for (i = 0; i < segcount; i++) { + struct svc_rdma_chunk *chunk; + u32 position, handle, length; + u64 offset; + + p++; /* skip the list discriminator */ + p = xdr_decode_read_segment(p, &position, &handle, + &length, &offset); + if (position != 0) + continue; + + if (pcl_is_empty(pcl)) { + chunk = pcl_alloc_chunk(segcount, position); + if (!chunk) + return false; + pcl_insert_position(pcl, chunk); + } else { + chunk = list_first_entry(&pcl->cl_chunks, + struct svc_rdma_chunk, + ch_list); + } + + pcl_set_read_segment(rctxt, chunk, handle, length, offset); + } + + return true; +} + +/** + * pcl_alloc_read - Construct a parsed chunk list for normal Read chunks + * @rctxt: Ingress receive context + * @p: Start of an un-decoded Read list + * + * Assumptions: + * - The incoming Read list has already been sanity checked. + * - cl_count is already set to the number of segments in + * the un-decoded list. + * - The list might not be in order by position. + * + * Return values: + * %true: Parsed chunk list was successfully constructed, and + * cl_count is updated to be the number of chunks (ie. + * unique position values) in the Read list. + * %false: Memory allocation failed. + * + * TODO: + * - Check for chunk range overlaps + */ +bool pcl_alloc_read(struct svc_rdma_recv_ctxt *rctxt, __be32 *p) +{ + struct svc_rdma_pcl *pcl = &rctxt->rc_read_pcl; + unsigned int i, segcount = pcl->cl_count; + + pcl->cl_count = 0; + for (i = 0; i < segcount; i++) { + struct svc_rdma_chunk *chunk; + u32 position, handle, length; + u64 offset; + + p++; /* skip the list discriminator */ + p = xdr_decode_read_segment(p, &position, &handle, + &length, &offset); + if (position == 0) + continue; + + chunk = pcl_lookup_position(pcl, position); + if (!chunk) { + chunk = pcl_alloc_chunk(segcount, position); + if (!chunk) + return false; + pcl_insert_position(pcl, chunk); + } + + pcl_set_read_segment(rctxt, chunk, handle, length, offset); + } + + return true; +} + +/** + * pcl_alloc_write - Construct a parsed chunk list from a Write list + * @rctxt: Ingress receive context + * @pcl: Parsed chunk list to populate + * @p: Start of an un-decoded Write list + * + * Assumptions: + * - The incoming Write list has already been sanity checked, and + * - cl_count is set to the number of chunks in the un-decoded list. + * + * Return values: + * %true: Parsed chunk list was successfully constructed. + * %false: Memory allocation failed. + */ +bool pcl_alloc_write(struct svc_rdma_recv_ctxt *rctxt, + struct svc_rdma_pcl *pcl, __be32 *p) +{ + struct svc_rdma_segment *segment; + struct svc_rdma_chunk *chunk; + unsigned int i, j; + u32 segcount; + + for (i = 0; i < pcl->cl_count; i++) { + p++; /* skip the list discriminator */ + segcount = be32_to_cpup(p++); + + chunk = pcl_alloc_chunk(segcount, 0); + if (!chunk) + return false; + list_add_tail(&chunk->ch_list, &pcl->cl_chunks); + + for (j = 0; j < segcount; j++) { + segment = &chunk->ch_segments[j]; + p = xdr_decode_rdma_segment(p, &segment->rs_handle, + &segment->rs_length, + &segment->rs_offset); + trace_svcrdma_decode_wseg(&rctxt->rc_cid, chunk, j); + + chunk->ch_length += segment->rs_length; + chunk->ch_segcount++; + } + } + return true; +} + +static int pcl_process_region(const struct xdr_buf *xdr, + unsigned int offset, unsigned int length, + int (*actor)(const struct xdr_buf *, void *), + void *data) +{ + struct xdr_buf subbuf; + + if (!length) + return 0; + if (xdr_buf_subsegment(xdr, &subbuf, offset, length)) + return -EMSGSIZE; + return actor(&subbuf, data); +} + +/** + * pcl_process_nonpayloads - Process non-payload regions inside @xdr + * @pcl: Chunk list to process + * @xdr: xdr_buf to process + * @actor: Function to invoke on each non-payload region + * @data: Arguments for @actor + * + * This mechanism must ignore not only result payloads that were already + * sent via RDMA Write, but also XDR padding for those payloads that + * the upper layer has added. + * + * Assumptions: + * The xdr->len and ch_position fields are aligned to 4-byte multiples. + * + * Returns: + * On success, zero, + * %-EMSGSIZE on XDR buffer overflow, or + * The return value of @actor + */ +int pcl_process_nonpayloads(const struct svc_rdma_pcl *pcl, + const struct xdr_buf *xdr, + int (*actor)(const struct xdr_buf *, void *), + void *data) +{ + struct svc_rdma_chunk *chunk, *next; + unsigned int start; + int ret; + + chunk = pcl_first_chunk(pcl); + + /* No result payloads were generated */ + if (!chunk || !chunk->ch_payload_length) + return actor(xdr, data); + + /* Process the region before the first result payload */ + ret = pcl_process_region(xdr, 0, chunk->ch_position, actor, data); + if (ret < 0) + return ret; + + /* Process the regions between each middle result payload */ + while ((next = pcl_next_chunk(pcl, chunk))) { + if (!next->ch_payload_length) + break; + + start = pcl_chunk_end_offset(chunk); + ret = pcl_process_region(xdr, start, next->ch_position - start, + actor, data); + if (ret < 0) + return ret; + + chunk = next; + } + + /* Process the region after the last result payload */ + start = pcl_chunk_end_offset(chunk); + ret = pcl_process_region(xdr, start, xdr->len - start, actor, data); + if (ret < 0) + return ret; + + return 0; +} diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index c6ea2903c21a..ec9d259b149c 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -93,6 +93,7 @@ * (see rdma_read_complete() below). */ +#include #include #include #include @@ -143,6 +144,10 @@ svc_rdma_recv_ctxt_alloc(struct svcxprt_rdma *rdma) goto fail2; svc_rdma_recv_cid_init(rdma, &ctxt->rc_cid); + pcl_init(&ctxt->rc_call_pcl); + pcl_init(&ctxt->rc_read_pcl); + pcl_init(&ctxt->rc_write_pcl); + pcl_init(&ctxt->rc_reply_pcl); ctxt->rc_recv_wr.next = NULL; ctxt->rc_recv_wr.wr_cqe = &ctxt->rc_cqe; @@ -226,6 +231,11 @@ void svc_rdma_recv_ctxt_put(struct svcxprt_rdma *rdma, for (i = 0; i < ctxt->rc_page_count; i++) put_page(ctxt->rc_pages[i]); + pcl_free(&ctxt->rc_call_pcl); + pcl_free(&ctxt->rc_read_pcl); + pcl_free(&ctxt->rc_write_pcl); + pcl_free(&ctxt->rc_reply_pcl); + if (!ctxt->rc_temp) llist_add(&ctxt->rc_node, &rdma->sc_recv_ctxts); else @@ -385,100 +395,123 @@ static void svc_rdma_build_arg_xdr(struct svc_rqst *rqstp, arg->len = ctxt->rc_byte_len; } -/* This accommodates the largest possible Write chunk. - */ -#define MAX_BYTES_WRITE_CHUNK ((u32)(RPCSVC_MAXPAGES << PAGE_SHIFT)) - -/* This accommodates the largest possible Position-Zero - * Read chunk or Reply chunk. - */ -#define MAX_BYTES_SPECIAL_CHUNK ((u32)((RPCSVC_MAXPAGES + 2) << PAGE_SHIFT)) - -/* Sanity check the Read list. +/** + * xdr_count_read_segments - Count number of Read segments in Read list + * @rctxt: Ingress receive context + * @p: Start of an un-decoded Read list * - * Implementation limits: - * - This implementation supports only one Read chunk. + * Before allocating anything, ensure the ingress Read list is safe + * to use. * - * Sanity checks: - * - Read list does not overflow Receive buffer. - * - Segment size limited by largest NFS data payload. - * - * The segment count is limited to how many segments can - * fit in the transport header without overflowing the - * buffer. That's about 40 Read segments for a 1KB inline - * threshold. + * The segment count is limited to how many segments can fit in the + * transport header without overflowing the buffer. That's about 40 + * Read segments for a 1KB inline threshold. * * Return values: - * %true: Read list is valid. @rctxt's xdr_stream is updated - * to point to the first byte past the Read list. - * %false: Read list is corrupt. @rctxt's xdr_stream is left - * in an unknown state. + * %true: Read list is valid. @rctxt's xdr_stream is updated to point + * to the first byte past the Read list. rc_read_pcl and + * rc_call_pcl cl_count fields are set to the number of + * Read segments in the list. + * %false: Read list is corrupt. @rctxt's xdr_stream is left in an + * unknown state. */ -static bool xdr_check_read_list(struct svc_rdma_recv_ctxt *rctxt) +static bool xdr_count_read_segments(struct svc_rdma_recv_ctxt *rctxt, __be32 *p) { - u32 position, len; - bool first; - __be32 *p; - - p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); - if (!p) - return false; - - len = 0; - first = true; + rctxt->rc_call_pcl.cl_count = 0; + rctxt->rc_read_pcl.cl_count = 0; while (xdr_item_is_present(p)) { + u32 position, handle, length; + u64 offset; + p = xdr_inline_decode(&rctxt->rc_stream, rpcrdma_readseg_maxsz * sizeof(*p)); if (!p) return false; - if (first) { - position = be32_to_cpup(p); - first = false; - } else if (be32_to_cpup(p) != position) { - return false; + xdr_decode_read_segment(p, &position, &handle, + &length, &offset); + if (position) { + if (position & 3) + return false; + ++rctxt->rc_read_pcl.cl_count; + } else { + ++rctxt->rc_call_pcl.cl_count; } - p += 2; - len += be32_to_cpup(p); p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); if (!p) return false; } - return len <= MAX_BYTES_SPECIAL_CHUNK; + return true; } -/* The segment count is limited to how many segments can - * fit in the transport header without overflowing the - * buffer. That's about 60 Write segments for a 1KB inline - * threshold. +/* Sanity check the Read list. + * + * Sanity checks: + * - Read list does not overflow Receive buffer. + * - Chunk size limited by largest NFS data payload. + * + * Return values: + * %true: Read list is valid. @rctxt's xdr_stream is updated + * to point to the first byte past the Read list. + * %false: Read list is corrupt. @rctxt's xdr_stream is left + * in an unknown state. */ -static bool xdr_check_write_chunk(struct svc_rdma_recv_ctxt *rctxt, u32 maxlen) +static bool xdr_check_read_list(struct svc_rdma_recv_ctxt *rctxt) { - u32 i, segcount, total; __be32 *p; p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); if (!p) return false; - segcount = be32_to_cpup(p); + if (!xdr_count_read_segments(rctxt, p)) + return false; + if (!pcl_alloc_call(rctxt, p)) + return false; + return pcl_alloc_read(rctxt, p); +} - total = 0; - for (i = 0; i < segcount; i++) { - u32 handle, length; - u64 offset; +static bool xdr_check_write_chunk(struct svc_rdma_recv_ctxt *rctxt) +{ + u32 segcount; + __be32 *p; - p = xdr_inline_decode(&rctxt->rc_stream, - rpcrdma_segment_maxsz * sizeof(*p)); - if (!p) - return false; + if (xdr_stream_decode_u32(&rctxt->rc_stream, &segcount)) + return false; - xdr_decode_rdma_segment(p, &handle, &length, &offset); - trace_svcrdma_decode_wseg(handle, length, offset); + /* A bogus segcount causes this buffer overflow check to fail. */ + p = xdr_inline_decode(&rctxt->rc_stream, + segcount * rpcrdma_segment_maxsz * sizeof(*p)); + return p != NULL; +} - total += length; +/** + * xdr_count_write_chunks - Count number of Write chunks in Write list + * @rctxt: Received header and decoding state + * @p: start of an un-decoded Write list + * + * Before allocating anything, ensure the ingress Write list is + * safe to use. + * + * Return values: + * %true: Write list is valid. @rctxt's xdr_stream is updated + * to point to the first byte past the Write list, and + * the number of Write chunks is in rc_write_pcl.cl_count. + * %false: Write list is corrupt. @rctxt's xdr_stream is left + * in an indeterminate state. + */ +static bool xdr_count_write_chunks(struct svc_rdma_recv_ctxt *rctxt, __be32 *p) +{ + rctxt->rc_write_pcl.cl_count = 0; + while (xdr_item_is_present(p)) { + if (!xdr_check_write_chunk(rctxt)) + return false; + ++rctxt->rc_write_pcl.cl_count; + p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); + if (!p) + return false; } - return total <= maxlen; + return true; } /* Sanity check the Write list. @@ -498,24 +531,22 @@ static bool xdr_check_write_chunk(struct svc_rdma_recv_ctxt *rctxt, u32 maxlen) */ static bool xdr_check_write_list(struct svc_rdma_recv_ctxt *rctxt) { - u32 chcount = 0; __be32 *p; p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); if (!p) return false; - rctxt->rc_write_list = p; - while (xdr_item_is_present(p)) { - if (!xdr_check_write_chunk(rctxt, MAX_BYTES_WRITE_CHUNK)) - return false; - ++chcount; - p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); - if (!p) - return false; - } - if (!chcount) - rctxt->rc_write_list = NULL; - return chcount < 2; + + rctxt->rc_write_list = NULL; + if (!xdr_count_write_chunks(rctxt, p)) + return false; + if (!pcl_alloc_write(rctxt, &rctxt->rc_write_pcl, p)) + return false; + + if (!pcl_is_empty(&rctxt->rc_write_pcl)) + rctxt->rc_write_list = p; + rctxt->rc_cur_result_payload = pcl_first_chunk(&rctxt->rc_write_pcl); + return rctxt->rc_write_pcl.cl_count < 2; } /* Sanity check the Reply chunk. @@ -537,13 +568,16 @@ static bool xdr_check_reply_chunk(struct svc_rdma_recv_ctxt *rctxt) p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); if (!p) return false; + rctxt->rc_reply_chunk = NULL; - if (xdr_item_is_present(p)) { - if (!xdr_check_write_chunk(rctxt, MAX_BYTES_SPECIAL_CHUNK)) - return false; - rctxt->rc_reply_chunk = p; - } - return true; + if (!xdr_item_is_present(p)) + return true; + if (!xdr_check_write_chunk(rctxt)) + return false; + + rctxt->rc_reply_chunk = p; + rctxt->rc_reply_pcl.cl_count = 1; + return pcl_alloc_write(rctxt, &rctxt->rc_reply_pcl, p); } /* RPC-over-RDMA Version One private extension: Remote Invalidation. From patchwork Mon Oct 26 18:54:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858307 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BF12814C0 for ; Mon, 26 Oct 2020 18:54:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 989E82168B for ; Mon, 26 Oct 2020 18:54:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GBpz6QiH" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1781434AbgJZSyq (ORCPT ); Mon, 26 Oct 2020 14:54:46 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:35106 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1781703AbgJZSyo (ORCPT ); Mon, 26 Oct 2020 14:54:44 -0400 Received: by mail-qk1-f196.google.com with SMTP id 140so9396038qko.2; Mon, 26 Oct 2020 11:54:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=/UwfH7nwiffhdnewCAuG5LcH0M5Ze2QoaFOiJ4vkAGQ=; b=GBpz6QiHNuUHhtu9jal0pfqfn0x5ftum7RuLlwsbDmSzypODAfqU4eAEjgT8TbqSoF k2jLiGFyish0Si6OhYZ0t1+OKHbG9CIO5PT998fS5e8BVhFTmsUA/2fw908KWFHwHnNg CLmSdF9AXijla0EraAwGuI5eqbh+VO8Ij51UPdh3Uuk6+xGNKRaM2WrzPaG8NDpoCs84 gDYsoh0GCJef7snAJ6oryXrw2fuDTt5cnJQ42WBtX7QhsWOf8et3M0qRVjy1b/ZeAMIN IIw4R8NSgb8T4N9CTvZAwBQlybEPbor4O5m//H0awsl/Zwyt2n1AWCDuHEy6QwEYK7SJ y3EQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=/UwfH7nwiffhdnewCAuG5LcH0M5Ze2QoaFOiJ4vkAGQ=; b=n0eq2OO8JS3we+OkkRaoX2DWhVgGanfCfF1Pyr8Em7/gSGDD+lLAwhbsixx8vujd7f F2GuuwfnYmwSDRHBhxfWCF0dmCF4fvQVYJxo6PJ7upwRmn9qow0ztrK5EyYlNOFCtqAT FkRGl4VYOZvdFD7QXiZDyexpFo/L93sNlDRC6LMIX4ODGQIwroR5wBfQs4+cd6YYL9AG ClXzJTav6QWc5RAULFVCTixV2p0xC9aqtr3tbcCNhDny8zjCo0LMF7GEN+166OiZRZtS 7T17SkAm5PF2E7R0hkZPL3FuwMkncWfcRauY6oVX71uOnqxvzxZ5wAzRoa9rdIbwFnLk hXWQ== X-Gm-Message-State: AOAM532UlCyoU7QcwsK3xgWLYK1ifv3jI9c2yxui6IlyoX037Oky1CjL tR0A1keoGidPcSwejz4KK2MhzKBD2IM= X-Google-Smtp-Source: ABdhPJwr3WKwafdpycABxgXEQlAv5YzgpY1TY1SVUGV4QAh1PCpg1Ui+0dtC/jSe9MCogZCkc5zoJg== X-Received: by 2002:a05:620a:12ea:: with SMTP id f10mr17113719qkl.480.1603738482597; Mon, 26 Oct 2020 11:54:42 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id s3sm7103631qkj.27.2020.10.26.11.54.41 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:54:42 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QIseIH013640; Mon, 26 Oct 2020 18:54:40 GMT Subject: [PATCH 09/20] svcrdma: Use parsed chunk lists to derive the inv_rkey From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:54:40 -0400 Message-ID: <160373848080.1886.5015616032132594298.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Refactor: Don't duplicate header decoding smarts here. Instead, use the new parsed chunk lists. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 67 ++++++++++++++----------------- 1 file changed, 30 insertions(+), 37 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index ec9d259b149c..2755ca178b09 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -586,60 +586,53 @@ static bool xdr_check_reply_chunk(struct svc_rdma_recv_ctxt *rctxt) * * If there is exactly one distinct R_key in the received transport * header, set rc_inv_rkey to that R_key. Otherwise, set it to zero. - * - * Perform this operation while the received transport header is - * still in the CPU cache. */ static void svc_rdma_get_inv_rkey(struct svcxprt_rdma *rdma, struct svc_rdma_recv_ctxt *ctxt) { - __be32 inv_rkey, *p; - u32 i, segcount; + struct svc_rdma_segment *segment; + struct svc_rdma_chunk *chunk; + u32 inv_rkey; ctxt->rc_inv_rkey = 0; if (!rdma->sc_snd_w_inv) return; - inv_rkey = xdr_zero; - p = ctxt->rc_recv_buf; - p += rpcrdma_fixed_maxsz; - - /* Read list */ - while (xdr_item_is_present(p++)) { - p++; /* position */ - if (inv_rkey == xdr_zero) - inv_rkey = *p; - else if (inv_rkey != *p) - return; - p += 4; + inv_rkey = 0; + pcl_for_each_chunk(chunk, &ctxt->rc_call_pcl) { + pcl_for_each_segment(segment, chunk) { + if (inv_rkey == 0) + inv_rkey = segment->rs_handle; + else if (inv_rkey != segment->rs_handle) + return; + } } - - /* Write list */ - while (xdr_item_is_present(p++)) { - segcount = be32_to_cpup(p++); - for (i = 0; i < segcount; i++) { - if (inv_rkey == xdr_zero) - inv_rkey = *p; - else if (inv_rkey != *p) + pcl_for_each_chunk(chunk, &ctxt->rc_read_pcl) { + pcl_for_each_segment(segment, chunk) { + if (inv_rkey == 0) + inv_rkey = segment->rs_handle; + else if (inv_rkey != segment->rs_handle) return; - p += 4; } } - - /* Reply chunk */ - if (xdr_item_is_present(p++)) { - segcount = be32_to_cpup(p++); - for (i = 0; i < segcount; i++) { - if (inv_rkey == xdr_zero) - inv_rkey = *p; - else if (inv_rkey != *p) + pcl_for_each_chunk(chunk, &ctxt->rc_write_pcl) { + pcl_for_each_segment(segment, chunk) { + if (inv_rkey == 0) + inv_rkey = segment->rs_handle; + else if (inv_rkey != segment->rs_handle) return; - p += 4; } } - - ctxt->rc_inv_rkey = be32_to_cpu(inv_rkey); + pcl_for_each_chunk(chunk, &ctxt->rc_reply_pcl) { + pcl_for_each_segment(segment, chunk) { + if (inv_rkey == 0) + inv_rkey = segment->rs_handle; + else if (inv_rkey != segment->rs_handle) + return; + } + } + ctxt->rc_inv_rkey = inv_rkey; } /** From patchwork Mon Oct 26 18:54:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858313 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EE3A614B2 for ; Mon, 26 Oct 2020 18:54:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C97F12168B for ; Mon, 26 Oct 2020 18:54:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="J6qk6gJU" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1781570AbgJZSyu (ORCPT ); Mon, 26 Oct 2020 14:54:50 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:44483 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1781411AbgJZSyt (ORCPT ); Mon, 26 Oct 2020 14:54:49 -0400 Received: by mail-qk1-f195.google.com with SMTP id s14so9383199qkg.11; Mon, 26 Oct 2020 11:54:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=YmDGVD1h9UeAesVa6tAnE/M/BMcF/UUcrSwGwabb9zU=; b=J6qk6gJUOV7/DNdkZOqLA8DEfZXO1/LEr81Q1OSeBXKYdFOuQ9fKn8TGg3Y3XiN+SH eGJL8NC/x6IwLwZmNhw8gAoFXmXEx9lkg2oCQ/3LLiQ+ZQr+KU/Y4rwqXIGTBnB5UmOu ZMb0jzSevmRfULnTT3sjdsoKTk/hOuP5/FD8usfNPmf5Su8vumKc8Qnts9CpVpijG9vt ozz/FnXaIrGHZI8deNoct3N/yQKX5Yep7p1pliNUoc96epsQVwlVLnHK+RN2Z5dKFQ82 fTGOhYjZhMlyDGsMQfrnY4DGg2B6sDp2TYKPD7IkB8G1pO/GvsRtKCYPBWw3fbKG2Iin b3Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=YmDGVD1h9UeAesVa6tAnE/M/BMcF/UUcrSwGwabb9zU=; b=dQrxtO935HFt2+WjSMJKm8Bm5+6Gd5yOqARFddoeEIiZtILx5xny49HJA+mEb+Hq59 MJ+Xah2xbtjZBsJfyPDG/CLAA+45GflBwafUwE3p4L5qCQa9XGMZwiyMw6Jz+fkl+8mI iRDUSeeZi4Pq9/K2xmgvthrPlJpIwjnJ03bJqXNB2e1qEeiv63N1PLSYPUga1Fz2w+gO uVj7c8cPNRcsh4/+oLgGMyljOTPlf2pvfxD3BGi1VgKKiZPEKRI8VYuZywV1s4+VlWWk k0Bdr1FVsH1tsm4Rs+GPT4lHANmUkfTwgDtMGTpB0Kt0qi/cbq6+uOxbr+r4D+J8tkaK mDLA== X-Gm-Message-State: AOAM530F+buctYpi8+B+dxogeaDuzEN9gyZNzqLwYkHIh8++3s2G9PB6 L1sNbWY3ZCOwh4F3xZsnuf7hslZdXwQ= X-Google-Smtp-Source: ABdhPJw/qToBnwVZQ8rzZ6jvs0OOFqcNEfFiqHS+CZApYR3d/yeP7/EVXRKyOv55suGHXi7Zp3CMIg== X-Received: by 2002:a05:620a:1322:: with SMTP id p2mr17818897qkj.211.1603738488169; Mon, 26 Oct 2020 11:54:48 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id p38sm7462909qtb.20.2020.10.26.11.54.46 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:54:47 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QIsk6F013643; Mon, 26 Oct 2020 18:54:46 GMT Subject: [PATCH 10/20] svcrdma: Use parsed chunk lists to detect reverse direction replies From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:54:46 -0400 Message-ID: <160373848616.1886.3476213491131167397.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Refactor: Don't duplicate header decoding smarts here. Instead, use the new parsed chunk lists. Note that the XID sanity test is also removed. The XID is already looked up by the cb handler, and is rejected if it's not recognized. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 1 + net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 29 ++++++++++++++--------------- 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index a89d4209fe2a..74247a33b6c6 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -144,6 +144,7 @@ struct svc_rdma_recv_ctxt { unsigned int rc_page_count; unsigned int rc_hdr_count; u32 rc_inv_rkey; + __be32 rc_msgtype; struct svc_rdma_pcl rc_call_pcl; diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index 2755ca178b09..72b07e8aa3c9 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -668,7 +668,8 @@ static int svc_rdma_xdr_decode_req(struct xdr_buf *rq_arg, if (*p != rpcrdma_version) goto out_version; p += 2; - switch (*p) { + rctxt->rc_msgtype = *p; + switch (rctxt->rc_msgtype) { case rdma_msg: break; case rdma_nomsg: @@ -762,30 +763,28 @@ static void svc_rdma_send_error(struct svcxprt_rdma *rdma, * the RPC/RDMA header small and fixed in size, so it is * straightforward to check the RPC header's direction field. */ -static bool svc_rdma_is_backchannel_reply(struct svc_xprt *xprt, - __be32 *rdma_resp) +static bool svc_rdma_is_reverse_direction_reply(struct svc_xprt *xprt, + struct svc_rdma_recv_ctxt *rctxt) { - __be32 *p; + __be32 *p = rctxt->rc_recv_buf; if (!xprt->xpt_bc_xprt) return false; - p = rdma_resp + 3; - if (*p++ != rdma_msg) + if (rctxt->rc_msgtype != rdma_msg) return false; - if (*p++ != xdr_zero) + if (!pcl_is_empty(&rctxt->rc_call_pcl)) + return false; + if (!pcl_is_empty(&rctxt->rc_read_pcl)) return false; - if (*p++ != xdr_zero) + if (!pcl_is_empty(&rctxt->rc_write_pcl)) return false; - if (*p++ != xdr_zero) + if (!pcl_is_empty(&rctxt->rc_reply_pcl)) return false; - /* XID sanity */ - if (*p++ != *rdma_resp) - return false; - /* call direction */ - if (*p == cpu_to_be32(RPC_CALL)) + /* RPC call direction */ + if (*(p + 8) == cpu_to_be32(RPC_CALL)) return false; return true; @@ -868,7 +867,7 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) goto out_drop; rqstp->rq_xprt_hlen = ret; - if (svc_rdma_is_backchannel_reply(xprt, p)) + if (svc_rdma_is_reverse_direction_reply(xprt, ctxt)) goto out_backchannel; svc_rdma_get_inv_rkey(rdma_xprt, ctxt); From patchwork Mon Oct 26 18:54:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858319 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A295614B2 for ; Mon, 26 Oct 2020 18:54:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7FFB421D7B for ; Mon, 26 Oct 2020 18:54:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CmBqgRXj" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1781894AbgJZSy4 (ORCPT ); Mon, 26 Oct 2020 14:54:56 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:45801 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1781766AbgJZSy4 (ORCPT ); Mon, 26 Oct 2020 14:54:56 -0400 Received: by mail-qk1-f195.google.com with SMTP id 188so9382846qkk.12; Mon, 26 Oct 2020 11:54:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=AzHrdzOL6+tZZI5Fvarus5ZZbYzgkG0ucE2JWb53Ag0=; b=CmBqgRXj+Gjwwgmk81TsNFStGzqgjEEdjj+IJAH1Jr8WjqUCwIi8QaWmO38+8T28jY sBOjqwIYJaliLXLo+6fAST5dUt3A0058IedyHFf8MezOrkO1cOomgHpH6DN9W4aaAtvE zEsdcdYwRHQDfqylPfO0P4h47rWQeCE6oV9/JaBu6FCdiKZ9hEGsAW6KBX6F2wAwuObL 1NOZuBRdbH4nEbjN01SWnS/1vMMTjlCt0S3GWNnF3dNjWPSrW4KrKBwV8lWZRZatZhBw wEG1rLpXPSjGHr/NGcNCwsWkDl3Y+ol4pq1rzbxZTlliIP7Q6K6sgv6KmtloEWbaDGf/ e+sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=AzHrdzOL6+tZZI5Fvarus5ZZbYzgkG0ucE2JWb53Ag0=; b=ATO1N3k0rHPpZK8ou9bUrxWkDOd/7zlhhLZsStZklyHZ2kJdvJPJfZMYSKrWFaOj+C Xj7lqzaeDa8dJuK512ycJBAbjeyC2JiNJCbfNofYHhd+tR9ExeoC0zp5sSDJjocGBDGM tLS54OgURdZg7GRsZEmCmLvxSCHxpCPiNapra/tMT/MknW0QbGHl5mj5eJJcg9cl6xBp nrBiaNbqitp3miMMGrDHj0GZsPBuVXhjk46Wo3HwJH/0f2RauqQt0JoRXWFWljeVYxUK 1WA0tHn+GQj4byfJMNsaD3B3c+d6rpCG3Hrsz5RigsXOWAb2uw1yvN26qlDfv+IlL1Wd HBTA== X-Gm-Message-State: AOAM5300ihHh8fNwZzgvxYzcyXVigPtA0yeelEPzXYI+r9owoYGJYxiB zZmt7vrLVW9ER+QQ6cpX7/pJwqdaZ8Y= X-Google-Smtp-Source: ABdhPJyRSDJhPM7eU17hC1cxNfcTA/TKnL8IR0pvv7InAx0gsAPpMM3GFw1GS7BnZ95ekC97RLpAog== X-Received: by 2002:a05:620a:150b:: with SMTP id i11mr2319980qkk.46.1603738493184; Mon, 26 Oct 2020 11:54:53 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id m12sm4419666qtu.52.2020.10.26.11.54.52 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:54:52 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QIspaN013646; Mon, 26 Oct 2020 18:54:51 GMT Subject: [PATCH 11/20] svcrdma: Use parsed chunk lists to construct RDMA Writes From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:54:51 -0400 Message-ID: <160373849146.1886.1814463689751473091.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Refactor: Instead of re-parsing the ingress RPC Call transport header when constructing RDMA Writes, use the new parsed chunk lists for the Write list and Reply chunk, which are version-agnostic and already XDR-decoded. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 5 +-- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 1 - net/sunrpc/xprtrdma/svc_rdma_rw.c | 47 +++++++++++++++---------------- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 22 +++++++++------ 4 files changed, 38 insertions(+), 37 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 74247a33b6c6..d9148787efff 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -157,8 +157,6 @@ struct svc_rdma_recv_ctxt { __be32 *rc_reply_chunk; struct svc_rdma_pcl rc_reply_pcl; - unsigned int rc_read_payload_offset; - unsigned int rc_read_payload_length; struct page *rc_pages[RPCSVC_MAXPAGES]; }; @@ -196,7 +194,8 @@ extern int svc_rdma_recv_read_chunk(struct svcxprt_rdma *rdma, struct svc_rqst *rqstp, struct svc_rdma_recv_ctxt *head, __be32 *p); extern int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, - __be32 *wr_ch, const struct xdr_buf *xdr); + const struct svc_rdma_chunk *chunk, + const struct xdr_buf *xdr); extern int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, const struct svc_rdma_recv_ctxt *rctxt, struct xdr_buf *xdr); diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index 72b07e8aa3c9..7d44e9d2e7a3 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -207,7 +207,6 @@ svc_rdma_recv_ctxt_get(struct svcxprt_rdma *rdma) out: ctxt->rc_page_count = 0; - ctxt->rc_read_payload_length = 0; return ctxt; out_empty: diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 5f667d964cd6..05dd0896860f 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -190,11 +190,11 @@ static void svc_rdma_cc_release(struct svc_rdma_chunk_ctxt *cc, * - Stores arguments for the SGL constructor functions */ struct svc_rdma_write_info { + const struct svc_rdma_chunk *wi_chunk; + /* write state of this chunk */ unsigned int wi_seg_off; unsigned int wi_seg_no; - unsigned int wi_nsegs; - __be32 *wi_segs; /* SGL constructor arguments */ const struct xdr_buf *wi_xdr; @@ -205,7 +205,8 @@ struct svc_rdma_write_info { }; static struct svc_rdma_write_info * -svc_rdma_write_info_alloc(struct svcxprt_rdma *rdma, __be32 *chunk) +svc_rdma_write_info_alloc(struct svcxprt_rdma *rdma, + const struct svc_rdma_chunk *chunk) { struct svc_rdma_write_info *info; @@ -213,10 +214,9 @@ svc_rdma_write_info_alloc(struct svcxprt_rdma *rdma, __be32 *chunk) if (!info) return info; + info->wi_chunk = chunk; info->wi_seg_off = 0; info->wi_seg_no = 0; - info->wi_nsegs = be32_to_cpup(++chunk); - info->wi_segs = ++chunk; svc_rdma_cc_init(rdma, &info->wi_cc); info->wi_cc.cc_cqe.done = svc_rdma_write_done; return info; @@ -443,40 +443,36 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, { struct svc_rdma_chunk_ctxt *cc = &info->wi_cc; struct svcxprt_rdma *rdma = cc->cc_rdma; + const struct svc_rdma_segment *seg; struct svc_rdma_rw_ctxt *ctxt; - __be32 *seg; int ret; - seg = info->wi_segs + info->wi_seg_no * rpcrdma_segment_maxsz; do { unsigned int write_len; - u32 handle, length; u64 offset; - if (info->wi_seg_no >= info->wi_nsegs) + seg = &info->wi_chunk->ch_segments[info->wi_seg_no]; + if (!seg) goto out_overflow; - xdr_decode_rdma_segment(seg, &handle, &length, &offset); - offset += info->wi_seg_off; - - write_len = min(remaining, length - info->wi_seg_off); + write_len = min(remaining, seg->rs_length - info->wi_seg_off); ctxt = svc_rdma_get_rw_ctxt(rdma, (write_len >> PAGE_SHIFT) + 2); if (!ctxt) return -ENOMEM; constructor(info, write_len, ctxt); - ret = svc_rdma_rw_ctx_init(rdma, ctxt, offset, handle, + offset = seg->rs_offset + info->wi_seg_off; + ret = svc_rdma_rw_ctx_init(rdma, ctxt, offset, seg->rs_handle, DMA_TO_DEVICE); if (ret < 0) return -EIO; - trace_svcrdma_send_wseg(handle, write_len, offset); + trace_svcrdma_send_wseg(seg->rs_handle, write_len, offset); list_add(&ctxt->rw_list, &cc->cc_rwctxts); cc->cc_sqecount += ret; - if (write_len == length - info->wi_seg_off) { - seg += 4; + if (write_len == seg->rs_length - info->wi_seg_off) { info->wi_seg_no++; info->wi_seg_off = 0; } else { @@ -489,7 +485,7 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, out_overflow: trace_svcrdma_small_wrch_err(rdma, remaining, info->wi_seg_no, - info->wi_nsegs); + info->wi_chunk->ch_segcount); return -E2BIG; } @@ -577,7 +573,7 @@ static int svc_rdma_xb_write(const struct xdr_buf *xdr, /** * svc_rdma_send_write_chunk - Write all segments in a Write chunk * @rdma: controlling RDMA transport - * @wr_ch: Write chunk provided by client + * @chunk: Write chunk provided by the client * @xdr: xdr_buf containing the data payload * * Returns a non-negative number of bytes the chunk consumed, or @@ -587,13 +583,14 @@ static int svc_rdma_xb_write(const struct xdr_buf *xdr, * %-ENOTCONN if posting failed (connection is lost), * %-EIO if rdma_rw initialization failed (DMA mapping, etc). */ -int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, __be32 *wr_ch, +int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, + const struct svc_rdma_chunk *chunk, const struct xdr_buf *xdr) { struct svc_rdma_write_info *info; int ret; - info = svc_rdma_write_info_alloc(rdma, wr_ch); + info = svc_rdma_write_info_alloc(rdma, chunk); if (!info) return -ENOMEM; @@ -631,12 +628,14 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, struct xdr_buf *xdr) { struct svc_rdma_write_info *info; + struct svc_rdma_chunk *chunk; int consumed, ret; - if (!rctxt->rc_reply_chunk) + if (pcl_is_empty(&rctxt->rc_reply_pcl)) return 0; - info = svc_rdma_write_info_alloc(rdma, rctxt->rc_reply_chunk); + chunk = pcl_first_chunk(&rctxt->rc_reply_pcl); + info = svc_rdma_write_info_alloc(rdma, chunk); if (!info) return -ENOMEM; @@ -648,7 +647,7 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, /* Send the page list in the Reply chunk only if the * client did not provide Write chunks. */ - if (!rctxt->rc_write_list && xdr->page_len) { + if (pcl_is_empty(&rctxt->rc_write_pcl) && xdr->page_len) { ret = svc_rdma_pages_write(info, xdr, xdr->head[0].iov_len, xdr->page_len); if (ret < 0) diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 3e7ba06788b0..f697e79757a6 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -466,12 +466,14 @@ static ssize_t svc_rdma_encode_write_list(const struct svc_rdma_recv_ctxt *rctxt, struct svc_rdma_send_ctxt *sctxt) { + struct svc_rdma_chunk *chunk; ssize_t len, ret; len = 0; if (rctxt->rc_write_list) { + chunk = pcl_first_chunk(&rctxt->rc_write_pcl); ret = svc_rdma_encode_write_chunk(rctxt->rc_write_list, sctxt, - rctxt->rc_read_payload_length); + chunk->ch_payload_length); if (ret < 0) return ret; len = ret; @@ -979,22 +981,24 @@ int svc_rdma_result_payload(struct svc_rqst *rqstp, unsigned int offset, unsigned int length) { struct svc_rdma_recv_ctxt *rctxt = rqstp->rq_xprt_ctxt; + struct svc_rdma_chunk *chunk; struct svcxprt_rdma *rdma; struct xdr_buf subbuf; - if (!rctxt->rc_write_list || !length) + chunk = rctxt->rc_cur_result_payload; + if (!length || !chunk) return 0; + rctxt->rc_cur_result_payload = + pcl_next_chunk(&rctxt->rc_write_pcl, chunk); + if (length > chunk->ch_length) + return -E2BIG; - /* XXX: Just one READ payload slot for now, since our - * transport implementation currently supports only one - * Write chunk. - */ - rctxt->rc_read_payload_offset = offset; - rctxt->rc_read_payload_length = length; + chunk->ch_position = offset; + chunk->ch_payload_length = length; if (xdr_buf_subsegment(&rqstp->rq_res, &subbuf, offset, length)) return -EMSGSIZE; rdma = container_of(rqstp->rq_xprt, struct svcxprt_rdma, sc_xprt); - return svc_rdma_send_write_chunk(rdma, rctxt->rc_write_list, &subbuf); + return svc_rdma_send_write_chunk(rdma, chunk, &subbuf); } From patchwork Mon Oct 26 18:54:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858323 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 02197697 for ; Mon, 26 Oct 2020 18:55:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D109A21D7B for ; Mon, 26 Oct 2020 18:55:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GJB40w/9" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1782364AbgJZSzA (ORCPT ); Mon, 26 Oct 2020 14:55:00 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:33102 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1781766AbgJZSzA (ORCPT ); Mon, 26 Oct 2020 14:55:00 -0400 Received: by mail-qk1-f193.google.com with SMTP id t128so4355208qke.0; Mon, 26 Oct 2020 11:54:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=N39Q7UiVa1pF0nSVJzC+DPeVrvgMGrHN67iQxvGi/EM=; b=GJB40w/9yLIMIo89ENelN6PU/F3eKgo5eZHLIS2AxtZtYHX4ZT0fy7EhD7q4RosLOo +n9NSxh+J8zGiAMX99fpp25/e07LByo4swpa1MTg6w92rZ6uG85BtiUuZHwFtC53aWzX +Y9EY1a0BTogkOfcuo//GS44+y7BWFB35v1IVOuqeOSoIFzBfGtjrSHRuTQRPyttXaPG 4OZbflC/Ut6FgfrBhcydsiWZI/5jhmPEyaX0WASlNDJWcvtbmTglFwqG1+3SIHQ+SmLE kK3V3eu+SwDZbYW0yp4lM293wXlsroyImFgTg6jdUvJeCoFtujjdj11zqPHqPnmMVK60 BltA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=N39Q7UiVa1pF0nSVJzC+DPeVrvgMGrHN67iQxvGi/EM=; b=l6ALtHiVffJD9ShngAcuPnVVytphAE/c0ZfXhmklaz9OODg0YFHVGpX7cFYBOGkF0N UoOjS2UV7VNYu/Sp9yzh7ljEDIGAFO4yddHIra7k0cc3/girivOBwgfMS79TbcYvlXRB EInpiCEyJX1x/n/ijzC9Tq9jVeufP5vQuniAxkZkFEUCXLErfUTnmGkM9I9HqsWNPxhm dkj2A+5+V7NBkFUpncK4/emcWpBwZPhogPtfuq4ZzQIXKrF0ko2z/8TbeARPbO6eN3Gg PBG5EiIgQ9z4LzvHyX9jIsw88FOzpah76HLRPab0QRmz+huK18gTPJbBzukqnEDXajqx /Dcg== X-Gm-Message-State: AOAM530uRqRQ+ON62leA1hxyYnRAcnpp+yo0c0wKGhvV3stApTP9oEl4 TuxdhFaY13GLa+SHpJwIHqYIjGu/OtA= X-Google-Smtp-Source: ABdhPJwqhEmMKrdvJHKEUFCFncTiSwZvIUQxThf9IAMXPK0z6e15cTQo1hDQZaz7eXgdGhkk1b/xOg== X-Received: by 2002:a37:aa91:: with SMTP id t139mr16372072qke.124.1603738498474; Mon, 26 Oct 2020 11:54:58 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id z30sm1527895qtc.15.2020.10.26.11.54.57 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:54:57 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QIsutI013649; Mon, 26 Oct 2020 18:54:56 GMT Subject: [PATCH 12/20] svcrdma: Use parsed chunk lists to encode Reply transport headers From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:54:56 -0400 Message-ID: <160373849677.1886.2727168886556909829.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Refactor: Instead of re-parsing the ingress RPC Call transport header when constructing the egress RPC Reply transport header, use the new parsed Write list and Reply chunk, which are version- agnostic and already XDR decoded. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 37 +++++++++++- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 105 ++++++++++++++------------------- 2 files changed, 80 insertions(+), 62 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 72b941aef43b..5218e0f9596a 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1447,9 +1447,44 @@ DECLARE_EVENT_CLASS(svcrdma_segment_event, TP_ARGS(handle, length, offset)) DEFINE_SEGMENT_EVENT(send_rseg); -DEFINE_SEGMENT_EVENT(encode_wseg); DEFINE_SEGMENT_EVENT(send_wseg); +TRACE_EVENT(svcrdma_encode_wseg, + TP_PROTO( + const struct svc_rdma_send_ctxt *ctxt, + u32 segno, + u32 handle, + u32 length, + u64 offset + ), + + TP_ARGS(ctxt, segno, handle, length, offset), + + TP_STRUCT__entry( + __field(u32, cq_id) + __field(int, completion_id) + __field(u32, segno) + __field(u32, handle) + __field(u32, length) + __field(u64, offset) + ), + + TP_fast_assign( + __entry->cq_id = ctxt->sc_cid.ci_queue_id; + __entry->completion_id = ctxt->sc_cid.ci_completion_id; + __entry->segno = segno; + __entry->handle = handle; + __entry->length = length; + __entry->offset = offset; + ), + + TP_printk("cq_id=%u cid=%d segno=%u %u@0x%016llx:0x%08x", + __entry->cq_id, __entry->completion_id, + __entry->segno, __entry->length, + (unsigned long long)__entry->offset, __entry->handle + ) +); + TRACE_EVENT(svcrdma_decode_rseg, TP_PROTO( const struct rpc_rdma_cid *cid, diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index f697e79757a6..fd8d62b1e640 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -358,49 +358,42 @@ static ssize_t svc_rdma_encode_read_list(struct svc_rdma_send_ctxt *sctxt) /** * svc_rdma_encode_write_segment - Encode one Write segment - * @src: matching Write chunk in the RPC Call header * @sctxt: Send context for the RPC Reply + * @chunk: Write chunk to push * @remaining: remaining bytes of the payload left in the Write chunk + * @segno: which segment in the chunk * * Return values: * On success, returns length in bytes of the Reply XDR buffer - * that was consumed by the Write segment + * that was consumed by the Write segment, and updates @remaining * %-EMSGSIZE on XDR buffer overflow */ -static ssize_t svc_rdma_encode_write_segment(__be32 *src, - struct svc_rdma_send_ctxt *sctxt, - unsigned int *remaining) +static ssize_t svc_rdma_encode_write_segment(struct svc_rdma_send_ctxt *sctxt, + const struct svc_rdma_chunk *chunk, + u32 *remaining, unsigned int segno) { + const struct svc_rdma_segment *segment = &chunk->ch_segments[segno]; + const size_t len = rpcrdma_segment_maxsz * sizeof(__be32); + u32 length; __be32 *p; - const size_t len = rpcrdma_segment_maxsz * sizeof(*p); - u32 handle, length; - u64 offset; p = xdr_reserve_space(&sctxt->sc_stream, len); if (!p) return -EMSGSIZE; - xdr_decode_rdma_segment(src, &handle, &length, &offset); - - if (*remaining < length) { - /* segment only partly filled */ - length = *remaining; - *remaining = 0; - } else { - /* entire segment was consumed */ - *remaining -= length; - } - xdr_encode_rdma_segment(p, handle, length, offset); - - trace_svcrdma_encode_wseg(handle, length, offset); + length = min_t(u32, *remaining, segment->rs_length); + *remaining -= length; + xdr_encode_rdma_segment(p, segment->rs_handle, length, + segment->rs_offset); + trace_svcrdma_encode_wseg(sctxt, segno, segment->rs_handle, length, + segment->rs_offset); return len; } /** * svc_rdma_encode_write_chunk - Encode one Write chunk - * @src: matching Write chunk in the RPC Call header * @sctxt: Send context for the RPC Reply - * @remaining: size in bytes of the payload in the Write chunk + * @chunk: Write chunk to push * * Copy a Write chunk from the Call transport header to the * Reply transport header. Update each segment's length field @@ -411,33 +404,30 @@ static ssize_t svc_rdma_encode_write_segment(__be32 *src, * that was consumed by the Write chunk * %-EMSGSIZE on XDR buffer overflow */ -static ssize_t svc_rdma_encode_write_chunk(__be32 *src, - struct svc_rdma_send_ctxt *sctxt, - unsigned int remaining) +static ssize_t svc_rdma_encode_write_chunk(struct svc_rdma_send_ctxt *sctxt, + const struct svc_rdma_chunk *chunk) { - unsigned int i, nsegs; + u32 remaining = chunk->ch_payload_length; + unsigned int segno; ssize_t len, ret; - len = 0; trace_svcrdma_encode_write_chunk(remaining); - src++; + len = 0; ret = xdr_stream_encode_item_present(&sctxt->sc_stream); if (ret < 0) - return -EMSGSIZE; + return ret; len += ret; - nsegs = be32_to_cpup(src++); - ret = xdr_stream_encode_u32(&sctxt->sc_stream, nsegs); + ret = xdr_stream_encode_u32(&sctxt->sc_stream, chunk->ch_segcount); if (ret < 0) - return -EMSGSIZE; + return ret; len += ret; - for (i = nsegs; i; i--) { - ret = svc_rdma_encode_write_segment(src, sctxt, &remaining); + for (segno = 0; segno < chunk->ch_segcount; segno++) { + ret = svc_rdma_encode_write_segment(sctxt, chunk, &remaining, segno); if (ret < 0) - return -EMSGSIZE; - src += rpcrdma_segment_maxsz; + return ret; len += ret; } @@ -449,34 +439,23 @@ static ssize_t svc_rdma_encode_write_chunk(__be32 *src, * @rctxt: Reply context with information about the RPC Call * @sctxt: Send context for the RPC Reply * - * The client provides a Write chunk list in the Call message. Fill - * in the segments in the first Write chunk in the Reply's transport - * header with the number of bytes consumed in each segment. - * Remaining chunks are returned unused. - * - * Assumptions: - * - Client has provided only one Write chunk - * * Return values: * On success, returns length in bytes of the Reply XDR buffer * that was consumed by the Reply's Write list * %-EMSGSIZE on XDR buffer overflow */ -static ssize_t -svc_rdma_encode_write_list(const struct svc_rdma_recv_ctxt *rctxt, - struct svc_rdma_send_ctxt *sctxt) +static ssize_t svc_rdma_encode_write_list(struct svc_rdma_recv_ctxt *rctxt, + struct svc_rdma_send_ctxt *sctxt) { struct svc_rdma_chunk *chunk; ssize_t len, ret; len = 0; - if (rctxt->rc_write_list) { - chunk = pcl_first_chunk(&rctxt->rc_write_pcl); - ret = svc_rdma_encode_write_chunk(rctxt->rc_write_list, sctxt, - chunk->ch_payload_length); + pcl_for_each_chunk(chunk, &rctxt->rc_write_pcl) { + ret = svc_rdma_encode_write_chunk(sctxt, chunk); if (ret < 0) return ret; - len = ret; + len += ret; } /* Terminate the Write list */ @@ -493,24 +472,28 @@ svc_rdma_encode_write_list(const struct svc_rdma_recv_ctxt *rctxt, * @sctxt: Send context for the RPC Reply * @length: size in bytes of the payload in the Reply chunk * - * Assumptions: - * - Reply can always fit in the client-provided Reply chunk - * * Return values: * On success, returns length in bytes of the Reply XDR buffer * that was consumed by the Reply's Reply chunk * %-EMSGSIZE on XDR buffer overflow + * %-E2BIG if the RPC message is larger than the Reply chunk */ static ssize_t -svc_rdma_encode_reply_chunk(const struct svc_rdma_recv_ctxt *rctxt, +svc_rdma_encode_reply_chunk(struct svc_rdma_recv_ctxt *rctxt, struct svc_rdma_send_ctxt *sctxt, unsigned int length) { - if (!rctxt->rc_reply_chunk) + struct svc_rdma_chunk *chunk; + + if (pcl_is_empty(&rctxt->rc_reply_pcl)) return xdr_stream_encode_item_absent(&sctxt->sc_stream); - return svc_rdma_encode_write_chunk(rctxt->rc_reply_chunk, sctxt, - length); + chunk = pcl_first_chunk(&rctxt->rc_reply_pcl); + if (length > chunk->ch_length) + return -E2BIG; + + chunk->ch_payload_length = length; + return svc_rdma_encode_write_chunk(sctxt, chunk); } static int svc_rdma_dma_map_page(struct svcxprt_rdma *rdma, @@ -928,7 +911,7 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) *p++ = *rdma_argp; *p++ = *(rdma_argp + 1); *p++ = rdma->sc_fc_credits; - *p = rctxt->rc_reply_chunk ? rdma_nomsg : rdma_msg; + *p = pcl_is_empty(&rctxt->rc_reply_pcl) ? rdma_msg : rdma_nomsg; if (svc_rdma_encode_read_list(sctxt) < 0) goto err0; From patchwork Mon Oct 26 18:55:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858329 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7094014C0 for ; Mon, 26 Oct 2020 18:55:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4BF9221D42 for ; Mon, 26 Oct 2020 18:55:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gY2e3wgJ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1781766AbgJZSzG (ORCPT ); Mon, 26 Oct 2020 14:55:06 -0400 Received: from mail-qv1-f67.google.com ([209.85.219.67]:40478 "EHLO mail-qv1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1781712AbgJZSzG (ORCPT ); Mon, 26 Oct 2020 14:55:06 -0400 Received: by mail-qv1-f67.google.com with SMTP id 63so2947090qva.7; Mon, 26 Oct 2020 11:55:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=vWL+cDt2Y6NkYEnEYS+ArN4PXs78b+qtMN6zVKDF8iA=; b=gY2e3wgJQP703ZlbsgliHpxkhLZ2q/WmGvwJMhOlub0gCZyWeeYlf9U84ECHLb7udK wnsQjRuhff9H+sZ7ok6UGN5NScKZF3SHFTFT4e9OlXo87UP3ZiVd6HRZ3ZS11ccINmki REG6zDsSdwgOTVvICW+KBcZSn8ypC1txOEmE3vVGM3XOw98UdBo/a3hkuiEJcO06IHIu bzO3lKM40UsctgMbiiehJVVMp2ieaUhNa1JTQ3DBolmONnvhx+TW4omPylKe4L9ZK/Jr Qec3pidANl2sal0ZxnN1cSTfeX8Orj7tvsNXxGlugM5rWgyxiKHZjpVuEr5ruaEa7J5H DtMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=vWL+cDt2Y6NkYEnEYS+ArN4PXs78b+qtMN6zVKDF8iA=; b=ng/umUVxucd7d54XcCx1Zj/1IAeHzZSG/v3W8Jc3Xxyy1sPhjZtPh0diZtY9TOE+PG SAgkdvO88j6Oy0zEMKKaIAvHwEBCAsVdXnqllmGbXFk1excCE/fqhO5+1pgvBahXuni5 gCxKxwM9jbKtvYEBSMzcXCtn1tAqocLfIHZRO6Y+ibMblE+rDf7PNXVYD5sit+Y9/7V4 cAiqW8OmI3+aXQ++hml0JuNL9liwXl6BZAX9atco9tHwTCz5tLL2Ucygc/Pc8Ol5Gq/b GvmHZxBHELKLDoVdAMQG6OH7j5erOPOOqO5cvINOB3Xa4ik5T0QjymFzbyoDvMfgn+YW 5CLg== X-Gm-Message-State: AOAM533a+AyUKrneA2j2WbpcErw+q4FsvP01MmJS70VK3IPo7tPlGdzN U9iMw7cNfRiSDtxLiBfySO28ApIcBHc= X-Google-Smtp-Source: ABdhPJw0iIE6SRk66jTUS6IHSkT1b4m4ekqSX9jrl/Fu2jci4L5ovrBvbNk/nAK/9coczgN/NWUliQ== X-Received: by 2002:a05:6214:4af:: with SMTP id w15mr18231885qvz.51.1603738503654; Mon, 26 Oct 2020 11:55:03 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id s11sm6961635qks.64.2020.10.26.11.55.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:55:02 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QIt212013652; Mon, 26 Oct 2020 18:55:02 GMT Subject: [PATCH 13/20] svcrdma: Support multiple write chunks when pulling up From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:55:02 -0400 Message-ID: <160373850208.1886.1587096744698083631.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org When counting the number of SGEs needed to construct a Send request, do not count result payloads. And, when copying the Reply message into the pull-up buffer, result payloads are not to be copied to the Send buffer. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 2 include/trace/events/rpcrdma.h | 20 ++- net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 14 +- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 9 + net/sunrpc/xprtrdma/svc_rdma_sendto.c | 188 +++++++++++++++++----------- 5 files changed, 146 insertions(+), 87 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index d9148787efff..7090af1a9791 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -182,6 +182,8 @@ extern void svc_rdma_handle_bc_reply(struct svc_rqst *rqstp, /* svc_rdma_recvfrom.c */ extern void svc_rdma_recv_ctxts_destroy(struct svcxprt_rdma *rdma); extern bool svc_rdma_post_recvs(struct svcxprt_rdma *rdma); +extern struct svc_rdma_recv_ctxt * + svc_rdma_recv_ctxt_get(struct svcxprt_rdma *rdma); extern void svc_rdma_recv_ctxt_put(struct svcxprt_rdma *rdma, struct svc_rdma_recv_ctxt *ctxt); extern void svc_rdma_flush_recv_queues(struct svcxprt_rdma *rdma); diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 5218e0f9596a..afc58accb9cf 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1805,20 +1805,30 @@ TRACE_EVENT(svcrdma_small_wrch_err, TRACE_EVENT(svcrdma_send_pullup, TP_PROTO( - unsigned int len + const struct svc_rdma_send_ctxt *ctxt, + unsigned int msglen ), - TP_ARGS(len), + TP_ARGS(ctxt, msglen), TP_STRUCT__entry( - __field(unsigned int, len) + __field(u32, cq_id) + __field(int, completion_id) + __field(unsigned int, hdrlen) + __field(unsigned int, msglen) ), TP_fast_assign( - __entry->len = len; + __entry->cq_id = ctxt->sc_cid.ci_queue_id; + __entry->completion_id = ctxt->sc_cid.ci_completion_id; + __entry->hdrlen = ctxt->sc_hdrbuf.len, + __entry->msglen = msglen; ), - TP_printk("len=%u", __entry->len) + TP_printk("cq_id=%u cid=%d hdr=%u msg=%u (total %u)", + __entry->cq_id, __entry->completion_id, + __entry->hdrlen, __entry->msglen, + __entry->hdrlen + __entry->msglen) ); TRACE_EVENT(svcrdma_send_err, diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index 5e7c4ba9e147..63f8be974df2 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -74,11 +74,17 @@ void svc_rdma_handle_bc_reply(struct svc_rqst *rqstp, */ static int svc_rdma_bc_sendto(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst, - struct svc_rdma_send_ctxt *ctxt) + struct svc_rdma_send_ctxt *sctxt) { + struct svc_rdma_recv_ctxt *rctxt; int ret; - ret = svc_rdma_map_reply_msg(rdma, ctxt, NULL, &rqst->rq_snd_buf); + rctxt = svc_rdma_recv_ctxt_get(rdma); + if (!rctxt) + return -EIO; + + ret = svc_rdma_map_reply_msg(rdma, sctxt, rctxt, &rqst->rq_snd_buf); + svc_rdma_recv_ctxt_put(rdma, rctxt); if (ret < 0) return -EIO; @@ -86,8 +92,8 @@ static int svc_rdma_bc_sendto(struct svcxprt_rdma *rdma, * the rq_buffer before all retransmits are complete. */ get_page(virt_to_page(rqst->rq_buffer)); - ctxt->sc_send_wr.opcode = IB_WR_SEND; - return svc_rdma_send(rdma, ctxt); + sctxt->sc_send_wr.opcode = IB_WR_SEND; + return svc_rdma_send(rdma, sctxt); } /* Server-side transport endpoint wants a whole page for its send diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index 7d44e9d2e7a3..af32c3ad45a6 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -194,8 +194,13 @@ void svc_rdma_recv_ctxts_destroy(struct svcxprt_rdma *rdma) } } -static struct svc_rdma_recv_ctxt * -svc_rdma_recv_ctxt_get(struct svcxprt_rdma *rdma) +/** + * svc_rdma_recv_ctxt_get - Allocate a recv_ctxt + * @rdma: controlling svcxprt_rdma + * + * Returns a recv_ctxt or (rarely) NULL if none are available. + */ +struct svc_rdma_recv_ctxt *svc_rdma_recv_ctxt_get(struct svcxprt_rdma *rdma) { struct svc_rdma_recv_ctxt *ctxt; struct llist_node *node; diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index fd8d62b1e640..b21beaa0114e 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -531,6 +531,45 @@ static int svc_rdma_dma_map_buf(struct svcxprt_rdma *rdma, offset_in_page(base), len); } +struct svc_rdma_pullup_data { + u8 *pd_dest; + unsigned int pd_length; + unsigned int pd_num_sges; +}; + +/** + * svc_rdma_xb_count_sges - Count how many SGEs will be needed + * @xdr: xdr_buf containing portion of an RPC message to transmit + * @data: pointer to arguments + * + * Returns: + * Number of SGEs needed to Send the contents of @xdr inline + */ +static int svc_rdma_xb_count_sges(const struct xdr_buf *xdr, + void *data) +{ + struct svc_rdma_pullup_data *args = data; + unsigned int remaining; + unsigned long offset; + + if (xdr->head[0].iov_len) + ++args->pd_num_sges; + + offset = offset_in_page(xdr->page_base); + remaining = xdr->page_len; + while (remaining) { + ++args->pd_num_sges; + remaining -= min_t(u32, PAGE_SIZE - offset, remaining); + offset = 0; + } + + if (xdr->tail[0].iov_len) + ++args->pd_num_sges; + + args->pd_length += xdr->len; + return 0; +} + /** * svc_rdma_pull_up_needed - Determine whether to use pull-up * @rdma: controlling transport @@ -539,50 +578,71 @@ static int svc_rdma_dma_map_buf(struct svcxprt_rdma *rdma, * @xdr: xdr_buf containing RPC message to transmit * * Returns: - * %true if pull-up must be used - * %false otherwise + * %true if pull-up must be used + * %false otherwise */ -static bool svc_rdma_pull_up_needed(struct svcxprt_rdma *rdma, - struct svc_rdma_send_ctxt *sctxt, +static bool svc_rdma_pull_up_needed(const struct svcxprt_rdma *rdma, + const struct svc_rdma_send_ctxt *sctxt, const struct svc_rdma_recv_ctxt *rctxt, - struct xdr_buf *xdr) + const struct xdr_buf *xdr) { - bool write_chunk_present = rctxt && rctxt->rc_write_list; - int elements; + /* Resources needed for the transport header */ + struct svc_rdma_pullup_data args = { + .pd_length = sctxt->sc_hdrbuf.len, + .pd_num_sges = 1, + }; + int ret; - /* For small messages, copying bytes is cheaper than DMA mapping. - */ - if (!write_chunk_present && - sctxt->sc_hdrbuf.len + xdr->len < RPCRDMA_PULLUP_THRESH) + ret = pcl_process_nonpayloads(&rctxt->rc_write_pcl, xdr, + svc_rdma_xb_count_sges, &args); + if (ret < 0) + return false; + + if (args.pd_length < RPCRDMA_PULLUP_THRESH) return true; + return args.pd_num_sges >= rdma->sc_max_send_sges; +} - /* Check whether the xdr_buf has more elements than can - * fit in a single RDMA Send. - */ - /* xdr->head */ - elements = 1; - - /* xdr->pages */ - if (!rctxt || !rctxt->rc_write_list) { - unsigned int remaining; - unsigned long pageoff; - - pageoff = xdr->page_base & ~PAGE_MASK; - remaining = xdr->page_len; - while (remaining) { - ++elements; - remaining -= min_t(u32, PAGE_SIZE - pageoff, - remaining); - pageoff = 0; - } +/** + * svc_rdma_xb_linearize - Copy region of xdr_buf to flat buffer + * @xdr: xdr_buf containing portion of an RPC message to copy + * @data: pointer to arguments + * + * Returns: + * Always zero. + */ +static int svc_rdma_xb_linearize(const struct xdr_buf *xdr, + void *data) +{ + struct svc_rdma_pullup_data *args = data; + unsigned int len, remaining; + unsigned long pageoff; + struct page **ppages; + + if (xdr->head[0].iov_len) { + memcpy(args->pd_dest, xdr->head[0].iov_base, xdr->head[0].iov_len); + args->pd_dest += xdr->head[0].iov_len; } - /* xdr->tail */ - if (xdr->tail[0].iov_len) - ++elements; + ppages = xdr->pages + (xdr->page_base >> PAGE_SHIFT); + pageoff = offset_in_page(xdr->page_base); + remaining = xdr->page_len; + while (remaining) { + len = min_t(u32, PAGE_SIZE - pageoff, remaining); + memcpy(args->pd_dest, page_address(*ppages) + pageoff, len); + remaining -= len; + args->pd_dest += len; + pageoff = 0; + ppages++; + } - /* assume 1 SGE is needed for the transport header */ - return elements >= rdma->sc_max_send_sges; + if (xdr->tail[0].iov_len) { + memcpy(args->pd_dest, xdr->tail[0].iov_base, xdr->tail[0].iov_len); + args->pd_dest += xdr->tail[0].iov_len; + } + + args->pd_length += xdr->len; + return 0; } /** @@ -595,54 +655,30 @@ static bool svc_rdma_pull_up_needed(struct svcxprt_rdma *rdma, * The device is not capable of sending the reply directly. * Assemble the elements of @xdr into the transport header buffer. * - * Returns zero on success, or a negative errno on failure. + * Assumptions: + * pull_up_needed has determined that @xdr will fit in the buffer. + * + * Returns: + * %0 if pull-up was successful + * %-EMSGSIZE if a buffer manipulation problem occurred */ -static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, +static int svc_rdma_pull_up_reply_msg(const struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *sctxt, const struct svc_rdma_recv_ctxt *rctxt, const struct xdr_buf *xdr) { - unsigned char *dst, *tailbase; - unsigned int taillen; - - dst = sctxt->sc_xprt_buf + sctxt->sc_hdrbuf.len; - memcpy(dst, xdr->head[0].iov_base, xdr->head[0].iov_len); - dst += xdr->head[0].iov_len; - - tailbase = xdr->tail[0].iov_base; - taillen = xdr->tail[0].iov_len; - if (rctxt && rctxt->rc_write_list) { - u32 xdrpad; - - xdrpad = xdr_pad_size(xdr->page_len); - if (taillen && xdrpad) { - tailbase += xdrpad; - taillen -= xdrpad; - } - } else { - unsigned int len, remaining; - unsigned long pageoff; - struct page **ppages; - - ppages = xdr->pages + (xdr->page_base >> PAGE_SHIFT); - pageoff = xdr->page_base & ~PAGE_MASK; - remaining = xdr->page_len; - while (remaining) { - len = min_t(u32, PAGE_SIZE - pageoff, remaining); - - memcpy(dst, page_address(*ppages) + pageoff, len); - remaining -= len; - dst += len; - pageoff = 0; - ppages++; - } - } + struct svc_rdma_pullup_data args = { + .pd_dest = sctxt->sc_xprt_buf + sctxt->sc_hdrbuf.len, + }; + int ret; - if (taillen) - memcpy(dst, tailbase, taillen); + ret = pcl_process_nonpayloads(&rctxt->rc_write_pcl, xdr, + svc_rdma_xb_linearize, &args); + if (ret < 0) + return ret; - sctxt->sc_sges[0].length += xdr->len; - trace_svcrdma_send_pullup(sctxt->sc_sges[0].length); + sctxt->sc_sges[0].length = sctxt->sc_hdrbuf.len + args.pd_length; + trace_svcrdma_send_pullup(sctxt, args.pd_length); return 0; } From patchwork Mon Oct 26 18:55:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858331 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4AF40697 for ; Mon, 26 Oct 2020 18:55:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2AC2121D7B for ; Mon, 26 Oct 2020 18:55:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NNdNmzL7" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1782250AbgJZSzL (ORCPT ); Mon, 26 Oct 2020 14:55:11 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:42940 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1781712AbgJZSzL (ORCPT ); Mon, 26 Oct 2020 14:55:11 -0400 Received: by mail-qt1-f196.google.com with SMTP id h12so2461137qtc.9; Mon, 26 Oct 2020 11:55:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=d+l+CZoRjizN0kcs9YlZbmHpHnFcD+aEO5hzRa2NF5k=; b=NNdNmzL758T/bcUOfKSiggNKjF9VkYkI5w7jNCOhZXWG8oryLYCBQbfmYawR0g65+H 2pMlfJDS3DTFNhPa5uQYKFfun9jmslAYVgQh9sN/Zw176JbKXKz3sXCmqILVcUEsXgnM f9xQ82k0BDEBqP63C5Y/ehV392/RvV8yZTyC0a2zMTr0UUgIMaZqQ1bqjRNlUVn3gpz5 ZxOCawoCvMMtvNJq7V15f6XeofHtYXrig+gurjcJEfhitvJp3bdlL9nlIaLa5mJlrzJ4 fjjawSFOY2jgDxyms29pZBI4TJqaMou8iMO0wYCc4AsAkNbw0K+Woa0Plg7cDK/2UkuO /mFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=d+l+CZoRjizN0kcs9YlZbmHpHnFcD+aEO5hzRa2NF5k=; b=mrPX656vaRLNltba/dbO46UtXdkLjuh3HnVg4r+V+pnHSkgn2ZqkqjnA4PKOxRwBb9 K28zJB4tWuJOg9T+3Wxc9605qrqYSVKVdRtE6otlq5L39xPvZZw6aWJQsfHxm6ayOiYw bP2IbUFKHSK4B4qkl6W4hO4bFd3LlvIaULWqkm5umYys9v5Xu7fHJl5gNX960J+RU8vd QpK/ZTQFBCYfW3nfWfIv6T1w9SqsPnNTxY4W2qkAt2VfTRva8hj9OvDZQcDOelJKvHT1 0lQs7evNNkLILRMiHjtJ2e0wDGpC44EucW00HtibPSNC764Kzwr+5J5t7eMQYE2o2i1U Tcgw== X-Gm-Message-State: AOAM530wutR19yKaiEESF3aKsgWWc7qOfn1MJROGukrRLahF0LhaDjXk 8D3kzH4gMjpIGQHylwXFWxqq7WKayzk= X-Google-Smtp-Source: ABdhPJwmBcRQUD4547itHdhlOYL0OQRcWBhrikvXNC06f9OumN9/efIrQ9j5g6QzcbrosvNEIEBXew== X-Received: by 2002:ac8:6f6f:: with SMTP id u15mr18983244qtv.115.1603738508885; Mon, 26 Oct 2020 11:55:08 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id c12sm7348333qtp.14.2020.10.26.11.55.07 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:55:08 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QIt7t4013655; Mon, 26 Oct 2020 18:55:07 GMT Subject: [PATCH 14/20] svcrdma: Support multiple Write chunks in svc_rdma_map_reply_msg() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:55:07 -0400 Message-ID: <160373850716.1886.4046562444210479073.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Refactor: svc_rdma_map_reply_msg() is restructured to DMA map only the parts of rq_res that do not contain a result payload. This change has been tested to confirm that it does not cause a regression in the no Write chunk and single Write chunk cases. Multiple Write chunks have not been tested. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 2 include/trace/events/rpcrdma.h | 1 net/sunrpc/xprtrdma/svc_rdma_sendto.c | 174 +++++++++++++++++++-------------- 3 files changed, 100 insertions(+), 77 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 7090af1a9791..e09fafba00d7 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -213,7 +213,7 @@ extern int svc_rdma_send(struct svcxprt_rdma *rdma, extern int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *sctxt, const struct svc_rdma_recv_ctxt *rctxt, - struct xdr_buf *xdr); + const struct xdr_buf *xdr); extern void svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *sctxt, struct svc_rdma_recv_ctxt *rctxt, diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index afc58accb9cf..054dedd0280c 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1687,6 +1687,7 @@ DECLARE_EVENT_CLASS(svcrdma_dma_map_class, TP_ARGS(rdma, dma_addr, length)) DEFINE_SVC_DMA_EVENT(dma_map_page); +DEFINE_SVC_DMA_EVENT(dma_map_err); DEFINE_SVC_DMA_EVENT(dma_unmap_page); TRACE_EVENT(svcrdma_dma_map_rw_err, diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index b21beaa0114e..7d35bd6224ea 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -496,39 +496,111 @@ svc_rdma_encode_reply_chunk(struct svc_rdma_recv_ctxt *rctxt, return svc_rdma_encode_write_chunk(sctxt, chunk); } -static int svc_rdma_dma_map_page(struct svcxprt_rdma *rdma, - struct svc_rdma_send_ctxt *ctxt, - struct page *page, - unsigned long offset, - unsigned int len) +struct svc_rdma_map_data { + struct svcxprt_rdma *md_rdma; + struct svc_rdma_send_ctxt *md_ctxt; +}; + +/** + * svc_rdma_page_dma_map - DMA map one page + * @data: pointer to arguments + * @page: struct page to DMA map + * @offset: offset into the page + * @len: number of bytes to map + * + * Returns: + * %0 if DMA mapping was successful + * %-EIO if the page cannot be DMA mapped + */ +static int svc_rdma_page_dma_map(void *data, struct page *page, + unsigned long offset, unsigned int len) { + struct svc_rdma_map_data *args = data; + struct svcxprt_rdma *rdma = args->md_rdma; + struct svc_rdma_send_ctxt *ctxt = args->md_ctxt; struct ib_device *dev = rdma->sc_cm_id->device; dma_addr_t dma_addr; + ++ctxt->sc_cur_sge_no; + dma_addr = ib_dma_map_page(dev, page, offset, len, DMA_TO_DEVICE); - trace_svcrdma_dma_map_page(rdma, dma_addr, len); if (ib_dma_mapping_error(dev, dma_addr)) goto out_maperr; + trace_svcrdma_dma_map_page(rdma, dma_addr, len); ctxt->sc_sges[ctxt->sc_cur_sge_no].addr = dma_addr; ctxt->sc_sges[ctxt->sc_cur_sge_no].length = len; ctxt->sc_send_wr.num_sge++; return 0; out_maperr: + trace_svcrdma_dma_map_err(rdma, dma_addr, len); return -EIO; } -/* ib_dma_map_page() is used here because svc_rdma_dma_unmap() +/** + * svc_rdma_iov_dma_map - DMA map an iovec + * @data: pointer to arguments + * @iov: kvec to DMA map + * + * ib_dma_map_page() is used here because svc_rdma_dma_unmap() * handles DMA-unmap and it uses ib_dma_unmap_page() exclusively. + * + * Returns: + * %0 if DMA mapping was successful + * %-EIO if the iovec cannot be DMA mapped */ -static int svc_rdma_dma_map_buf(struct svcxprt_rdma *rdma, - struct svc_rdma_send_ctxt *ctxt, - unsigned char *base, - unsigned int len) +static int svc_rdma_iov_dma_map(void *data, const struct kvec *iov) { - return svc_rdma_dma_map_page(rdma, ctxt, virt_to_page(base), - offset_in_page(base), len); + if (!iov->iov_len) + return 0; + return svc_rdma_page_dma_map(data, virt_to_page(iov->iov_base), + offset_in_page(iov->iov_base), + iov->iov_len); +} + +/** + * svc_rdma_xb_dma_map - DMA map all segments of an xdr_buf + * @xdr: xdr_buf containing portion of an RPC message to transmit + * @data: pointer to arguments + * + * Returns: + * %0 if DMA mapping was successful + * %-EIO if DMA mapping failed + * + * On failure, any DMA mappings that have been already done must be + * unmapped by the caller. + */ +static int svc_rdma_xb_dma_map(const struct xdr_buf *xdr, void *data) +{ + unsigned int len, remaining; + unsigned long pageoff; + struct page **ppages; + int ret; + + ret = svc_rdma_iov_dma_map(data, &xdr->head[0]); + if (ret < 0) + return ret; + + ppages = xdr->pages + (xdr->page_base >> PAGE_SHIFT); + pageoff = offset_in_page(xdr->page_base); + remaining = xdr->page_len; + while (remaining) { + len = min_t(u32, PAGE_SIZE - pageoff, remaining); + + ret = svc_rdma_page_dma_map(data, *ppages++, pageoff, len); + if (ret < 0) + return ret; + + remaining -= len; + pageoff = 0; + } + + ret = svc_rdma_iov_dma_map(data, &xdr->tail[0]); + if (ret < 0) + return ret; + + return xdr->len; } struct svc_rdma_pullup_data { @@ -688,22 +760,22 @@ static int svc_rdma_pull_up_reply_msg(const struct svcxprt_rdma *rdma, * @rctxt: Write and Reply chunks provided by client * @xdr: prepared xdr_buf containing RPC message * - * Load the xdr_buf into the ctxt's sge array, and DMA map each - * element as it is added. The Send WR's num_sge field is set. + * Returns: + * %0 if DMA mapping was successful. + * %-EMSGSIZE if a buffer manipulation problem occurred + * %-EIO if DMA mapping failed * - * Returns zero on success, or a negative errno on failure. + * The Send WR's num_sge field is set in all cases. */ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *sctxt, const struct svc_rdma_recv_ctxt *rctxt, - struct xdr_buf *xdr) + const struct xdr_buf *xdr) { - unsigned int len, remaining; - unsigned long page_off; - struct page **ppages; - unsigned char *base; - u32 xdr_pad; - int ret; + struct svc_rdma_map_data args = { + .md_rdma = rdma, + .md_ctxt = sctxt, + }; /* Set up the (persistently-mapped) transport header SGE. */ sctxt->sc_send_wr.num_sge = 1; @@ -712,7 +784,7 @@ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, /* If there is a Reply chunk, nothing follows the transport * header, and we're done here. */ - if (rctxt && rctxt->rc_reply_chunk) + if (!pcl_is_empty(&rctxt->rc_reply_pcl)) return 0; /* For pull-up, svc_rdma_send() will sync the transport header. @@ -721,58 +793,8 @@ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, if (svc_rdma_pull_up_needed(rdma, sctxt, rctxt, xdr)) return svc_rdma_pull_up_reply_msg(rdma, sctxt, rctxt, xdr); - ++sctxt->sc_cur_sge_no; - ret = svc_rdma_dma_map_buf(rdma, sctxt, - xdr->head[0].iov_base, - xdr->head[0].iov_len); - if (ret < 0) - return ret; - - /* If a Write chunk is present, the xdr_buf's page list - * is not included inline. However the Upper Layer may - * have added XDR padding in the tail buffer, and that - * should not be included inline. - */ - if (rctxt && rctxt->rc_write_list) { - base = xdr->tail[0].iov_base; - len = xdr->tail[0].iov_len; - xdr_pad = xdr_pad_size(xdr->page_len); - - if (len && xdr_pad) { - base += xdr_pad; - len -= xdr_pad; - } - - goto tail; - } - - ppages = xdr->pages + (xdr->page_base >> PAGE_SHIFT); - page_off = xdr->page_base & ~PAGE_MASK; - remaining = xdr->page_len; - while (remaining) { - len = min_t(u32, PAGE_SIZE - page_off, remaining); - - ++sctxt->sc_cur_sge_no; - ret = svc_rdma_dma_map_page(rdma, sctxt, *ppages++, - page_off, len); - if (ret < 0) - return ret; - - remaining -= len; - page_off = 0; - } - - base = xdr->tail[0].iov_base; - len = xdr->tail[0].iov_len; -tail: - if (len) { - ++sctxt->sc_cur_sge_no; - ret = svc_rdma_dma_map_buf(rdma, sctxt, base, len); - if (ret < 0) - return ret; - } - - return 0; + return pcl_process_nonpayloads(&rctxt->rc_write_pcl, xdr, + svc_rdma_xb_dma_map, &args); } /* The svc_rqst and all resources it owns are released as soon as From patchwork Mon Oct 26 18:55:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858347 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7FBE814B2 for ; Mon, 26 Oct 2020 18:55:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 604B221D42 for ; Mon, 26 Oct 2020 18:55:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="obdQsOKv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1782952AbgJZSzQ (ORCPT ); Mon, 26 Oct 2020 14:55:16 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:38777 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1781817AbgJZSzQ (ORCPT ); Mon, 26 Oct 2020 14:55:16 -0400 Received: by mail-qt1-f194.google.com with SMTP id q26so7518658qtb.5; Mon, 26 Oct 2020 11:55:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=4KJm145UWBWPfJj4byRZtYjTtFKvdCbaxgLINuLuBEM=; b=obdQsOKvuaWgGgX7Jx/8jSfKJTYezF6B74jSK5bokLN+DohTX1g539hYvmuyRwF6fV wIkDZoIJGXfYE0SuFJa0LlGEIEDPqfpHjciiiNAzMr3ux3nHM2gposqi4JvSn/Ef89WJ 8ol9uKhjENDSqGKaxnq0M0n4gBD8Fp76nmAFojhTSM31VSgad91OHd1zPe9WOtBk/wbq OfdjGUrv2GG6ZXiHO13m6zQeoSEfSlm3X/z7/a/gIv0G0C1r1jXzrEib098teTBTQE2S LY5deTgrdnayuuLb1CVmBqD/JmKBJyFUiBk+JNOjXed3pqkoXZGSnTvBym5GApFZnxw6 hy0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=4KJm145UWBWPfJj4byRZtYjTtFKvdCbaxgLINuLuBEM=; b=DX0mV844NE3hTwcQbec0QpR3gBmpmuAcysHcn+kirWkzW0Dhv6c/SBmFlFoxnLb+vK O9B1U3C53BQ7L6WGzwk9W9L12uNZJPGdJuwmLA+RQZ1edLwSCo3RFSI+rd2lzamLxbBd jI4rzTlhEAm+A1Wkr+wJB4Q5ca8ndRvXp1pNUhyi9l8O+rYs5E/LecdADkVzwAkE+0ft zI8EbpaAHwUNtLhibTTpJYB51K4PsiYD/NNMyNTgfaD98CpZ53KNsewOa6VOJgPJm+aF +Y1XMWhaY74i+en0XirfFm4YrrjBLGKO3Pt+zLVUsdlb6JJw1Sc0oLZI8vMJp3Hrukyo kthA== X-Gm-Message-State: AOAM532JYSMWRm60dJXr9ykWsnyYon/aSmkMMcTaJSXL7QUZ7aGSWd5v DFep/dw7SLEDQoiiVjuk28prVlEMTCM= X-Google-Smtp-Source: ABdhPJwVYvqKAF+LlcfpjC970dojFcr7R/hIMLiiw5cVSmr3HR1UI0qjvqWXmZ5EvH6TKWVG9fihWA== X-Received: by 2002:ac8:4c8e:: with SMTP id j14mr16556868qtv.223.1603738514105; Mon, 26 Oct 2020 11:55:14 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id g27sm7171004qkk.135.2020.10.26.11.55.13 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:55:13 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QItCrh013658; Mon, 26 Oct 2020 18:55:12 GMT Subject: [PATCH 15/20] svcrdma: Support multiple Write chunks in svc_rdma_send_reply_chunk From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:55:12 -0400 Message-ID: <160373851247.1886.687426546558863258.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Refactor svc_rdma_send_reply_chunk() so that it Sends only the parts of rq_res that do not contain a result payload. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 2 +- net/sunrpc/xprtrdma/svc_rdma_rw.c | 36 +++++++++--------------------------- 2 files changed, 10 insertions(+), 28 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index e09fafba00d7..85fbec47d4b5 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -200,7 +200,7 @@ extern int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, const struct xdr_buf *xdr); extern int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, const struct svc_rdma_recv_ctxt *rctxt, - struct xdr_buf *xdr); + const struct xdr_buf *xdr); /* svc_rdma_sendto.c */ extern void svc_rdma_send_ctxts_destroy(struct svcxprt_rdma *rdma); diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 05dd0896860f..4efa1fa3f6fb 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -535,7 +535,7 @@ static int svc_rdma_pages_write(struct svc_rdma_write_info *info, /** * svc_rdma_xb_write - Construct RDMA Writes to write an xdr_buf * @xdr: xdr_buf to write - * @info: pointer to write arguments + * @data: pointer to write arguments * * Returns: * On succes, returns zero @@ -543,9 +543,9 @@ static int svc_rdma_pages_write(struct svc_rdma_write_info *info, * %-ENOMEM if a resource has been exhausted * %-EIO if an rdma-rw error occurred */ -static int svc_rdma_xb_write(const struct xdr_buf *xdr, - struct svc_rdma_write_info *info) +static int svc_rdma_xb_write(const struct xdr_buf *xdr, void *data) { + struct svc_rdma_write_info *info = data; int ret; if (xdr->head[0].iov_len) { @@ -625,11 +625,11 @@ int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, */ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, const struct svc_rdma_recv_ctxt *rctxt, - struct xdr_buf *xdr) + const struct xdr_buf *xdr) { struct svc_rdma_write_info *info; struct svc_rdma_chunk *chunk; - int consumed, ret; + int ret; if (pcl_is_empty(&rctxt->rc_reply_pcl)) return 0; @@ -639,35 +639,17 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, if (!info) return -ENOMEM; - ret = svc_rdma_iov_write(info, &xdr->head[0]); + ret = pcl_process_nonpayloads(&rctxt->rc_write_pcl, xdr, + svc_rdma_xb_write, info); if (ret < 0) goto out_err; - consumed = xdr->head[0].iov_len; - - /* Send the page list in the Reply chunk only if the - * client did not provide Write chunks. - */ - if (pcl_is_empty(&rctxt->rc_write_pcl) && xdr->page_len) { - ret = svc_rdma_pages_write(info, xdr, xdr->head[0].iov_len, - xdr->page_len); - if (ret < 0) - goto out_err; - consumed += xdr->page_len; - } - - if (xdr->tail[0].iov_len) { - ret = svc_rdma_iov_write(info, &xdr->tail[0]); - if (ret < 0) - goto out_err; - consumed += xdr->tail[0].iov_len; - } ret = svc_rdma_post_chunk_ctxt(&info->wi_cc); if (ret < 0) goto out_err; - trace_svcrdma_send_reply_chunk(consumed); - return consumed; + trace_svcrdma_send_reply_chunk(xdr->len); + return xdr->len; out_err: svc_rdma_write_info_free(info); From patchwork Mon Oct 26 18:55:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858353 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 119AB697 for ; Mon, 26 Oct 2020 18:55:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E4FA0207E8 for ; Mon, 26 Oct 2020 18:55:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="d8MJ69Hd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1783054AbgJZSzV (ORCPT ); Mon, 26 Oct 2020 14:55:21 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:46404 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1782994AbgJZSzV (ORCPT ); Mon, 26 Oct 2020 14:55:21 -0400 Received: by mail-qk1-f196.google.com with SMTP id a23so9385756qkg.13; Mon, 26 Oct 2020 11:55:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=rXmy/SzKRHLADNcq+tGVZ5hWUR6cy6tHo3UYwHOKd5M=; b=d8MJ69HdcKc9XsQZBFxZEBDIBICKk94UtDCpam5vxfwDM58WJN4n+WyEsNenQvLHoy A+O7Ss8dWMQ9ILcs3kSEY8hmuaLKkOUzH+Y6mCZPXoj81KtvLGcVF80D1gPi0yyIKQ8S 74vOUX6DQcgVbOA7s9hB2pgxD2HTDEOpROWDF7i0aogAlNY26iE+q8lL1FlqJBRwsGeF taKLZ6QLm8TAiObVZaIwwfB8jRxeiYSFGriUZVEx5ZHNt2rM71/Gl85aVhKa+lbeApF1 CCwf+luc1PlKhqjV0rp+ddfXgNbAMLVDffQVmzNXtcqzjV7IOMoEqK1XN9zTVHzCP8Fw nstA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=rXmy/SzKRHLADNcq+tGVZ5hWUR6cy6tHo3UYwHOKd5M=; b=WelJGD0Be5rFduUDmNTQrEDIrtJN9wTEcWYImtfh6YhCC/g160dWTSpAmlUdKxCIqJ uGK12x/NVVpKihZY0s6YlFCmpyHdyZOkUOm9/6UbfkRfsyaKx+WfL0uBq7RcbeJ2bIkH Bzte6CA2WjG7Kdn752hi1mJdaSJi7jaRCC6fpyWywLYPp0Ofm39smdkFQizvcHq7WrpZ +bSNj8Q6TcclDHM612JYL3NCB6HEsVhphRzXFL92P9rSFERU6HgFDeZOqp/2Hp6AGn36 3ZDps0wjiJhm5GzzNG58UKcL5nUEAaMr7fNXXcnwfjqXfZtMs0Es2zUeK5TLudvVLfOL XWaw== X-Gm-Message-State: AOAM532cm6CV7R3VYR+d0RoHqw4TPzwKUkMPmcpQPI/fOJQI/CulgJIb Gwg8hhZpzFz9WKJYhDcpJxSd9ZDL1K8= X-Google-Smtp-Source: ABdhPJwypUtansVDE3Dc00839n7uL26V2I8zX7H3IyhfqCf33mGHZi+qe+vrX5++C6E4652rMjK22Q== X-Received: by 2002:a37:4984:: with SMTP id w126mr17935036qka.104.1603738519629; Mon, 26 Oct 2020 11:55:19 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id z26sm7008430qki.40.2020.10.26.11.55.18 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:55:18 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QItHkp013661; Mon, 26 Oct 2020 18:55:17 GMT Subject: [PATCH 16/20] svcrdma: Remove chunk list pointers From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:55:17 -0400 Message-ID: <160373851777.1886.8052540379115036293.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: These pointers are no longer used. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 4 ---- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 8 +------- 2 files changed, 1 insertion(+), 11 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 85fbec47d4b5..6f247d043731 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -149,12 +149,8 @@ struct svc_rdma_recv_ctxt { struct svc_rdma_pcl rc_call_pcl; struct svc_rdma_pcl rc_read_pcl; - - __be32 *rc_write_list; struct svc_rdma_chunk *rc_cur_result_payload; struct svc_rdma_pcl rc_write_pcl; - - __be32 *rc_reply_chunk; struct svc_rdma_pcl rc_reply_pcl; struct page *rc_pages[RPCSVC_MAXPAGES]; diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index af32c3ad45a6..dd10b1de227d 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -540,17 +540,13 @@ static bool xdr_check_write_list(struct svc_rdma_recv_ctxt *rctxt) p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); if (!p) return false; - - rctxt->rc_write_list = NULL; if (!xdr_count_write_chunks(rctxt, p)) return false; if (!pcl_alloc_write(rctxt, &rctxt->rc_write_pcl, p)) return false; - if (!pcl_is_empty(&rctxt->rc_write_pcl)) - rctxt->rc_write_list = p; rctxt->rc_cur_result_payload = pcl_first_chunk(&rctxt->rc_write_pcl); - return rctxt->rc_write_pcl.cl_count < 2; + return true; } /* Sanity check the Reply chunk. @@ -573,13 +569,11 @@ static bool xdr_check_reply_chunk(struct svc_rdma_recv_ctxt *rctxt) if (!p) return false; - rctxt->rc_reply_chunk = NULL; if (!xdr_item_is_present(p)) return true; if (!xdr_check_write_chunk(rctxt)) return false; - rctxt->rc_reply_chunk = p; rctxt->rc_reply_pcl.cl_count = 1; return pcl_alloc_write(rctxt, &rctxt->rc_reply_pcl, p); } From patchwork Mon Oct 26 18:55:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858355 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 97EC4697 for ; Mon, 26 Oct 2020 18:55:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 742AC21D42 for ; Mon, 26 Oct 2020 18:55:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="u4SqqDZZ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1781840AbgJZSz1 (ORCPT ); Mon, 26 Oct 2020 14:55:27 -0400 Received: from mail-qv1-f66.google.com ([209.85.219.66]:33228 "EHLO mail-qv1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1780843AbgJZSz0 (ORCPT ); Mon, 26 Oct 2020 14:55:26 -0400 Received: by mail-qv1-f66.google.com with SMTP id w9so4836145qvj.0; Mon, 26 Oct 2020 11:55:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=pwlWqdUZIEkYkK9f30HVQeJD00T2Q9dGcArjB32xi2c=; b=u4SqqDZZyKu350YQ+yuUhztJK27jPmFRDoS9lL846Kz2yam4ePDCwsxWbu0gB40H0d Sb4cfDFFG0TMUMHxugShcNBgmWQZAKnIZXLNmCEn+dzllAsKXK9wqydCrQndlgWAGDCJ 2RzWfTg5jDInBQsAEGcB82hX1OSCD6F1mHloye35RrPTzBNgW4LiRiiAdJ1WshRU9JO0 B/ut2HY2cR/yqlUMAWz08OS35ajqzNiwfnuitYvQQ2WPgWIcy9jA6SGsy2MpDgAC9PvK 00I148TbyLgUOYNsKAY4qZA0mw+ud9BdMdvqFfiUXVLdi6ahaWcnxX8+ZJKza/HorWhH ugpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=pwlWqdUZIEkYkK9f30HVQeJD00T2Q9dGcArjB32xi2c=; b=FSqrG+8AhmXu9ZBhXaeTxvGwH3n35zwKDcws3Q2tyP//1hCombNm0CVTq1NcvMsS7/ FboW5CqK3szubrt0DfzztIcrIaO8FQ71Hopb7mNsnpJjrvisWhCAHBMIl2pEDVcrZoBW 8qh3lKGlZFHXo/nA6Gj8M0IsPBoF8W24j+h4wEYYbvvdIkbMRlISA9ogOBkBIw1QV5IM 2ynpH5h1d/TsYUbvw37q3hUvxlA8Q/UJ4UUjKLXPVNGxbGYmMz+iH8vkyXSiXNLpnV/J NMAoxOi3xy0vPRC4ZyJ6s1kUkXMLq1BOIrVzbV5YiWrD/gd/oz5fOfkqfK03gBcdLENs yt/Q== X-Gm-Message-State: AOAM531xkbZKGqCKXuxPamyeTw28K+dgsZOSyXHtW4b62ci84iISCmh/ YFupmDumngBJ47okgXiJpp4DFFYpvAE= X-Google-Smtp-Source: ABdhPJxrsDAh9YWoosbMJ9p/Jn8dUoJM80/8M4OijANZs29Si5JLg/2XU5Hh+kfPyISvyPyrLzukLQ== X-Received: by 2002:a0c:a482:: with SMTP id x2mr19173200qvx.47.1603738524787; Mon, 26 Oct 2020 11:55:24 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id 9sm7333699qkv.110.2020.10.26.11.55.23 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:55:24 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QItNxO013664; Mon, 26 Oct 2020 18:55:23 GMT Subject: [PATCH 17/20] svcrdma: Clean up chunk tracepoints From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:55:23 -0400 Message-ID: <160373852307.1886.2787183721434561772.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org We already have trace_svcrdma_decode_rseg(), which records each ingress Read segment. Instead of reporting those again when they are about to be posted as RDMA Reads, let's fire one tracepoint before posting each type of chunk. So we'll get: nfsd-1998 [002] 321.666615: svcrdma_decode_rseg: cq.id=4 cid=42 segno=0 position=0 192@0x013ca9ebfae14000:0xb0010b05 nfsd-1998 [002] 321.666615: svcrdma_decode_rseg: cq.id=4 cid=42 segno=1 position=0 7688@0x013ca9ebf914e000:0xb0010a05 nfsd-1998 [002] 321.666615: svcrdma_decode_rseg: cq.id=4 cid=42 segno=2 position=0 28@0x013ca9ebfae15000:0xb0010905 nfsd-1998 [002] 321.666622: svcrdma_decode_rqst: cq.id=4 cid=42 xid=0x013ca9eb vers=1 credits=128 proc=RDMA_NOMSG hdrlen=100 nfsd-1998 [002] 321.666642: svcrdma_post_read_chunk: cq.id=3 cid=112 sqecount=3 kworker/2:1H-221 [002] 321.673949: svcrdma_wc_read: cq.id=3 cid=112 status=SUCCESS (0/0x0) Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 110 ++++----------------------------- net/sunrpc/xprtrdma/svc_rdma_rw.c | 27 ++++---- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 2 - 3 files changed, 26 insertions(+), 113 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 054dedd0280c..896aafc37b09 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1410,45 +1410,6 @@ DEFINE_BADREQ_EVENT(drop); DEFINE_BADREQ_EVENT(badproc); DEFINE_BADREQ_EVENT(parse); -DECLARE_EVENT_CLASS(svcrdma_segment_event, - TP_PROTO( - u32 handle, - u32 length, - u64 offset - ), - - TP_ARGS(handle, length, offset), - - TP_STRUCT__entry( - __field(u32, handle) - __field(u32, length) - __field(u64, offset) - ), - - TP_fast_assign( - __entry->handle = handle; - __entry->length = length; - __entry->offset = offset; - ), - - TP_printk("%u@0x%016llx:0x%08x", - __entry->length, (unsigned long long)__entry->offset, - __entry->handle - ) -); - -#define DEFINE_SEGMENT_EVENT(name) \ - DEFINE_EVENT(svcrdma_segment_event, svcrdma_##name,\ - TP_PROTO( \ - u32 handle, \ - u32 length, \ - u64 offset \ - ), \ - TP_ARGS(handle, length, offset)) - -DEFINE_SEGMENT_EVENT(send_rseg); -DEFINE_SEGMENT_EVENT(send_wseg); - TRACE_EVENT(svcrdma_encode_wseg, TP_PROTO( const struct svc_rdma_send_ctxt *ctxt, @@ -1558,62 +1519,6 @@ TRACE_EVENT(svcrdma_decode_wseg, ) ); -DECLARE_EVENT_CLASS(svcrdma_chunk_event, - TP_PROTO( - u32 length - ), - - TP_ARGS(length), - - TP_STRUCT__entry( - __field(u32, length) - ), - - TP_fast_assign( - __entry->length = length; - ), - - TP_printk("length=%u", - __entry->length - ) -); - -#define DEFINE_CHUNK_EVENT(name) \ - DEFINE_EVENT(svcrdma_chunk_event, svcrdma_##name, \ - TP_PROTO( \ - u32 length \ - ), \ - TP_ARGS(length)) - -DEFINE_CHUNK_EVENT(send_pzr); -DEFINE_CHUNK_EVENT(encode_write_chunk); -DEFINE_CHUNK_EVENT(send_write_chunk); -DEFINE_CHUNK_EVENT(encode_read_chunk); -DEFINE_CHUNK_EVENT(send_reply_chunk); - -TRACE_EVENT(svcrdma_send_read_chunk, - TP_PROTO( - u32 length, - u32 position - ), - - TP_ARGS(length, position), - - TP_STRUCT__entry( - __field(u32, length) - __field(u32, position) - ), - - TP_fast_assign( - __entry->length = length; - __entry->position = position; - ), - - TP_printk("length=%u position=%u", - __entry->length, __entry->position - ) -); - DECLARE_EVENT_CLASS(svcrdma_error_event, TP_PROTO( __be32 xid @@ -1936,7 +1841,7 @@ TRACE_EVENT(svcrdma_rq_post_err, ) ); -TRACE_EVENT(svcrdma_post_chunk, +DECLARE_EVENT_CLASS(svcrdma_post_chunk_class, TP_PROTO( const struct rpc_rdma_cid *cid, int sqecount @@ -1962,6 +1867,19 @@ TRACE_EVENT(svcrdma_post_chunk, ) ); +#define DEFINE_POST_CHUNK_EVENT(name) \ + DEFINE_EVENT(svcrdma_post_chunk_class, \ + svcrdma_post_##name##_chunk, \ + TP_PROTO( \ + const struct rpc_rdma_cid *cid, \ + int sqecount \ + ), \ + TP_ARGS(cid, sqecount)) + +DEFINE_POST_CHUNK_EVENT(read); +DEFINE_POST_CHUNK_EVENT(write); +DEFINE_POST_CHUNK_EVENT(reply); + DEFINE_COMPLETION_EVENT(svcrdma_wc_read); DEFINE_COMPLETION_EVENT(svcrdma_wc_write); diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 4efa1fa3f6fb..0de95207eaf1 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -358,7 +358,6 @@ static int svc_rdma_post_chunk_ctxt(struct svc_rdma_chunk_ctxt *cc) do { if (atomic_sub_return(cc->cc_sqecount, &rdma->sc_sq_avail) > 0) { - trace_svcrdma_post_chunk(&cc->cc_cid, cc->cc_sqecount); ret = ib_post_send(rdma->sc_qp, first_wr, &bad_wr); if (ret) break; @@ -468,8 +467,6 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, if (ret < 0) return -EIO; - trace_svcrdma_send_wseg(seg->rs_handle, write_len, offset); - list_add(&ctxt->rw_list, &cc->cc_rwctxts); cc->cc_sqecount += ret; if (write_len == seg->rs_length - info->wi_seg_off) { @@ -588,21 +585,22 @@ int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, const struct xdr_buf *xdr) { struct svc_rdma_write_info *info; + struct svc_rdma_chunk_ctxt *cc; int ret; info = svc_rdma_write_info_alloc(rdma, chunk); if (!info) return -ENOMEM; + cc = &info->wi_cc; ret = svc_rdma_xb_write(xdr, info); if (ret != xdr->len) goto out_err; - ret = svc_rdma_post_chunk_ctxt(&info->wi_cc); + trace_svcrdma_post_write_chunk(&cc->cc_cid, cc->cc_sqecount); + ret = svc_rdma_post_chunk_ctxt(cc); if (ret < 0) goto out_err; - - trace_svcrdma_send_write_chunk(xdr->page_len); return xdr->len; out_err: @@ -628,6 +626,7 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, const struct xdr_buf *xdr) { struct svc_rdma_write_info *info; + struct svc_rdma_chunk_ctxt *cc; struct svc_rdma_chunk *chunk; int ret; @@ -638,17 +637,18 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, info = svc_rdma_write_info_alloc(rdma, chunk); if (!info) return -ENOMEM; + cc = &info->wi_cc; ret = pcl_process_nonpayloads(&rctxt->rc_write_pcl, xdr, svc_rdma_xb_write, info); if (ret < 0) goto out_err; - ret = svc_rdma_post_chunk_ctxt(&info->wi_cc); + trace_svcrdma_post_reply_chunk(&cc->cc_cid, cc->cc_sqecount); + ret = svc_rdma_post_chunk_ctxt(cc); if (ret < 0) goto out_err; - trace_svcrdma_send_reply_chunk(xdr->len); return xdr->len; out_err: @@ -735,10 +735,8 @@ static int svc_rdma_build_read_chunk(struct svc_rqst *rqstp, if (ret < 0) break; - trace_svcrdma_send_rseg(handle, length, offset); info->ri_chunklen += length; } - return ret; } @@ -760,8 +758,6 @@ static int svc_rdma_build_normal_read_chunk(struct svc_rqst *rqstp, if (ret < 0) goto out; - trace_svcrdma_send_read_chunk(info->ri_chunklen, info->ri_position); - head->rc_hdr_count = 0; /* Split the Receive buffer between the head and tail @@ -816,8 +812,6 @@ static int svc_rdma_build_pz_read_chunk(struct svc_rqst *rqstp, if (ret < 0) goto out; - trace_svcrdma_send_pzr(info->ri_chunklen); - head->rc_arg.len += info->ri_chunklen; head->rc_arg.buflen += info->ri_chunklen; @@ -874,6 +868,7 @@ int svc_rdma_recv_read_chunk(struct svcxprt_rdma *rdma, struct svc_rqst *rqstp, struct svc_rdma_recv_ctxt *head, __be32 *p) { struct svc_rdma_read_info *info; + struct svc_rdma_chunk_ctxt *cc; int ret; /* The request (with page list) is constructed in @@ -891,6 +886,7 @@ int svc_rdma_recv_read_chunk(struct svcxprt_rdma *rdma, struct svc_rqst *rqstp, info = svc_rdma_read_info_alloc(rdma); if (!info) return -ENOMEM; + cc = &info->ri_cc; info->ri_readctxt = head; info->ri_pageno = 0; info->ri_pageoff = 0; @@ -903,7 +899,8 @@ int svc_rdma_recv_read_chunk(struct svcxprt_rdma *rdma, struct svc_rqst *rqstp, if (ret < 0) goto out_err; - ret = svc_rdma_post_chunk_ctxt(&info->ri_cc); + trace_svcrdma_post_read_chunk(&cc->cc_cid, cc->cc_sqecount); + ret = svc_rdma_post_chunk_ctxt(cc); if (ret < 0) goto out_err; svc_rdma_save_io_pages(rqstp, 0, head->rc_page_count); diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 7d35bd6224ea..035eb99b8ede 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -411,8 +411,6 @@ static ssize_t svc_rdma_encode_write_chunk(struct svc_rdma_send_ctxt *sctxt, unsigned int segno; ssize_t len, ret; - trace_svcrdma_encode_write_chunk(remaining); - len = 0; ret = xdr_stream_encode_item_present(&sctxt->sc_stream); if (ret < 0) From patchwork Mon Oct 26 18:55:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858359 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0BC00697 for ; Mon, 26 Oct 2020 18:55:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DD4E82085B for ; Mon, 26 Oct 2020 18:55:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="adpT11Gw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1780843AbgJZSzc (ORCPT ); Mon, 26 Oct 2020 14:55:32 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:36046 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404083AbgJZSzb (ORCPT ); Mon, 26 Oct 2020 14:55:31 -0400 Received: by mail-qk1-f196.google.com with SMTP id r7so9408095qkf.3; Mon, 26 Oct 2020 11:55:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=anT1sLFW8H6h/m1zuKewPaTkoaFSjz3HZ6lIFFJwDfc=; b=adpT11GwF3BLVPG/LE4t7wyVIVN/tpja386EHA3uAZgCUle7oSFuKWI2aM9Z8Zwqqw rRkYPuncvVKGlJMJKC86LIpQQLwGrhGB7h3vO1Zh88zC/oR2v7aQhta2QZOuzXNQu/jp ivQ5GEwlBSlSSk3WQbnw43vjOnJ6mO/5A616Gn1s/4+odl06SWDedvpr7Xb0S3F80p7k GS2loazU9VDIVLWoVVu6+xNSHduji92G2iHuwAUuJwrd7Bo3JBS76T4lACtCEBys+EOl 4KrPAEpe2JLx/bZ1xioS0+/XKzTkFM5YVB60/EbEW7RP39z36kQmcNVv7V/Kawzat2S9 s8aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=anT1sLFW8H6h/m1zuKewPaTkoaFSjz3HZ6lIFFJwDfc=; b=Kl0jlO8PhN2gXPLV6zHDJylnrp8ZOIZEb3a4/96MqT0jUQ6LSjXRJ8QxwoNJykNgcd b5LuaN8jpVsgOHcRkyFIZ1RU37+E7ufs4TKf8vry0nR7PSBd7iWerfdNzFBiSfvrLKW6 +xjTMccPGk+U1g8ayTZtmaStc63fPKJEjWiYw9gKM8DGPpFPQV298OHd2BsRsgAaizVs kE2i3W1Z98okRI2ydyyGOv4RwdzgnBvPU1SrfWYAldUl27Dy6UJMr66vSH2zrMw+ybHF S/3lArtTNu5YVM95xwouC/kVAIQ0IH3zifn6FcryMD9RtomBBKdDk7uMEyK9j6cQmHwl S+5w== X-Gm-Message-State: AOAM531owiLVheGH7bydITstQG6LOS6Hp7EjEE0VeL/HY9QWnQAEdnm5 V1CnXI13xBJrw6W8wsizSva48mtFgGY= X-Google-Smtp-Source: ABdhPJyfjvNg9jHLBZEnKT3YK9KJGHpr1CzSWYtonHE3WUvywosmX1xrlrLnct1+2fIshSFRbnAwfg== X-Received: by 2002:a37:c49:: with SMTP id 70mr16000744qkm.345.1603738530020; Mon, 26 Oct 2020 11:55:30 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id s3sm7105274qkj.27.2020.10.26.11.55.29 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:55:29 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QItSHR013667; Mon, 26 Oct 2020 18:55:28 GMT Subject: [PATCH 18/20] svcrdma: Rename info::ri_chunklen From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:55:28 -0400 Message-ID: <160373852838.1886.14180778700427228388.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org I'm about to change the purpose of ri_chunklen: Instead of tracking the number of bytes in one Read chunk, it will track the total number of bytes in the Read list. Rename it for clarity. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_rw.c | 31 +++++++++++++++---------------- 1 file changed, 15 insertions(+), 16 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 0de95207eaf1..104b1d5a2203 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -262,7 +262,7 @@ struct svc_rdma_read_info { unsigned int ri_position; unsigned int ri_pageno; unsigned int ri_pageoff; - unsigned int ri_chunklen; + unsigned int ri_totalbytes; struct svc_rdma_chunk_ctxt ri_cc; }; @@ -724,7 +724,6 @@ static int svc_rdma_build_read_chunk(struct svc_rqst *rqstp, int ret; ret = -EINVAL; - info->ri_chunklen = 0; while (*p++ != xdr_zero && be32_to_cpup(p++) == info->ri_position) { u32 handle, length; u64 offset; @@ -735,7 +734,7 @@ static int svc_rdma_build_read_chunk(struct svc_rqst *rqstp, if (ret < 0) break; - info->ri_chunklen += length; + info->ri_totalbytes += length; } return ret; } @@ -752,6 +751,8 @@ static int svc_rdma_build_normal_read_chunk(struct svc_rqst *rqstp, __be32 *p) { struct svc_rdma_recv_ctxt *head = info->ri_readctxt; + struct xdr_buf *buf = &head->rc_arg; + unsigned int length; int ret; ret = svc_rdma_build_read_chunk(rqstp, info, p); @@ -780,11 +781,10 @@ static int svc_rdma_build_normal_read_chunk(struct svc_rqst *rqstp, * Currently these chunks always start at page offset 0, * thus the rounded-up length never crosses a page boundary. */ - info->ri_chunklen = XDR_QUADLEN(info->ri_chunklen) << 2; - - head->rc_arg.page_len = info->ri_chunklen; - head->rc_arg.len += info->ri_chunklen; - head->rc_arg.buflen += info->ri_chunklen; + length = XDR_QUADLEN(info->ri_totalbytes) << 2; + buf->page_len = length; + buf->len += length; + buf->buflen += length; out: return ret; @@ -806,22 +806,20 @@ static int svc_rdma_build_pz_read_chunk(struct svc_rqst *rqstp, __be32 *p) { struct svc_rdma_recv_ctxt *head = info->ri_readctxt; + struct xdr_buf *buf = &head->rc_arg; int ret; ret = svc_rdma_build_read_chunk(rqstp, info, p); if (ret < 0) goto out; - head->rc_arg.len += info->ri_chunklen; - head->rc_arg.buflen += info->ri_chunklen; + buf->len += info->ri_totalbytes; + buf->buflen += info->ri_totalbytes; head->rc_hdr_count = 1; - head->rc_arg.head[0].iov_base = page_address(head->rc_pages[0]); - head->rc_arg.head[0].iov_len = min_t(size_t, PAGE_SIZE, - info->ri_chunklen); - - head->rc_arg.page_len = info->ri_chunklen - - head->rc_arg.head[0].iov_len; + buf->head[0].iov_base = page_address(head->rc_pages[0]); + buf->head[0].iov_len = min_t(size_t, PAGE_SIZE, info->ri_totalbytes); + buf->page_len = info->ri_totalbytes - buf->head[0].iov_len; out: return ret; @@ -890,6 +888,7 @@ int svc_rdma_recv_read_chunk(struct svcxprt_rdma *rdma, struct svc_rqst *rqstp, info->ri_readctxt = head; info->ri_pageno = 0; info->ri_pageoff = 0; + info->ri_totalbytes = 0; info->ri_position = be32_to_cpup(p + 1); if (info->ri_position) From patchwork Mon Oct 26 18:55:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858363 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7D94914B2 for ; Mon, 26 Oct 2020 18:55:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5976421D41 for ; Mon, 26 Oct 2020 18:55:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="C6cAJ68f" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1782281AbgJZSzi (ORCPT ); Mon, 26 Oct 2020 14:55:38 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:39451 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404083AbgJZSzi (ORCPT ); Mon, 26 Oct 2020 14:55:38 -0400 Received: by mail-qt1-f193.google.com with SMTP id i7so6810722qti.6; Mon, 26 Oct 2020 11:55:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=YOLkHLFkULxlBpNrbT9Jrbppk5noxw7p9O3VpLnTLPg=; b=C6cAJ68fAv0+VGD9HBI6j+3t1t1Y8jhJ3FB5RPjPPM3ra+zQ9fiD+ArcIGuDtSlVCx ai3yOzbT7Vzu5mXYtPIEEHZj33/jYE3DYWjICx0fr1Y3LA0QMmd4RehA8NZJvWDsvsEW JsTC0WHpXWZzhgg/6RvsLlgLF6yfUhEkHHeVKE+RGrzB33eQH37yux8R2Azvno891SDv CkmbrwrpGucwQLTPQd2SbqRiCcqifXCC8bdSErTvIrv7hw4zmoB7zXHKY4H5rRAgxWta CgcIu3OOIwqD3cVtnK9Jn4+onUAnMAw3zaKas9J+Lg6zA/nYPHoZMFy0Zw/fyETexOCR C8OA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=YOLkHLFkULxlBpNrbT9Jrbppk5noxw7p9O3VpLnTLPg=; b=mq73XQ0jaE4K3F6xY8kzifNGtiQMQUW5lWAw8G0KFMVj64Vn0/6kamdNtuhQEZxi42 Cvz63NX+ru4LnKvf6i93TjgOltnq6mP+7Xz9oGiKyF174eFFMjMgL65m9KEOA14GOSEp v0tLCGm1jYe1utc8tqE8med+zWLThqmzNWDpFdjoSPM4Dni6F/FGJ0r7od4Va2Qj5iZy j4Kg7CXr83Uhq9oe5yX/1sRo2wl4nv/R0uGwkVVfHN3/r2soCYfhoKyUeTj24nJnw9wA 3riyP7nuPpr8VfAsRyt8LlJCerap9ddnH2OIpHiA8v5WNjtVCwv3c+xEUeKKqvB8Uaoj +RPQ== X-Gm-Message-State: AOAM533Xew0VIy2e0iWzc+KuResvb9LLaOMBVzNAfYQ5VYXn08Xsp6yv HlP95m4kIGdLSpxn9bQJLbRuCKYi/qk= X-Google-Smtp-Source: ABdhPJw7MnXKiV2jjskT7BF2DUKItyxxMimYeDaPhMPeVOt1Rzt345ycUWtaDBYMJPTFe+EcI7lItg== X-Received: by 2002:ac8:734c:: with SMTP id q12mr8645521qtp.384.1603738535249; Mon, 26 Oct 2020 11:55:35 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id u26sm7129705qkj.17.2020.10.26.11.55.34 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:55:34 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QItXCF013670; Mon, 26 Oct 2020 18:55:33 GMT Subject: [PATCH 19/20] svcrdma: Use the new parsed chunk list when pulling Read chunks From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:55:33 -0400 Message-ID: <160373853368.1886.15667445822588290097.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org As a pre-requisite for handling multiple Read chunks in each Read list, convert svc_rdma_recv_read_chunk() to use the new parsed Read chunk list. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 6 + net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 16 +-- net/sunrpc/xprtrdma/svc_rdma_rw.c | 160 ++++++++++++++++++++----------- 3 files changed, 111 insertions(+), 71 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 6f247d043731..294b56e61522 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -188,15 +188,15 @@ extern int svc_rdma_recvfrom(struct svc_rqst *); /* svc_rdma_rw.c */ extern void svc_rdma_destroy_rw_ctxts(struct svcxprt_rdma *rdma); -extern int svc_rdma_recv_read_chunk(struct svcxprt_rdma *rdma, - struct svc_rqst *rqstp, - struct svc_rdma_recv_ctxt *head, __be32 *p); extern int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, const struct svc_rdma_chunk *chunk, const struct xdr_buf *xdr); extern int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, const struct svc_rdma_recv_ctxt *rctxt, const struct xdr_buf *xdr); +extern int svc_rdma_process_read_list(struct svcxprt_rdma *rdma, + struct svc_rqst *rqstp, + struct svc_rdma_recv_ctxt *head); /* svc_rdma_sendto.c */ extern void svc_rdma_send_ctxts_destroy(struct svcxprt_rdma *rdma); diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index dd10b1de227d..cbdb71247755 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -824,7 +824,6 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) struct svcxprt_rdma *rdma_xprt = container_of(xprt, struct svcxprt_rdma, sc_xprt); struct svc_rdma_recv_ctxt *ctxt; - __be32 *p; int ret; rqstp->rq_xprt_ctxt = NULL; @@ -857,7 +856,6 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) rqstp->rq_respages = rqstp->rq_pages; rqstp->rq_next_page = rqstp->rq_respages; - p = (__be32 *)rqstp->rq_arg.head[0].iov_base; ret = svc_rdma_xdr_decode_req(&rqstp->rq_arg, ctxt); if (ret < 0) goto out_err; @@ -870,9 +868,9 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) svc_rdma_get_inv_rkey(rdma_xprt, ctxt); - p += rpcrdma_fixed_maxsz; - if (*p != xdr_zero) - goto out_readchunk; + if (!pcl_is_empty(&ctxt->rc_read_pcl) || + !pcl_is_empty(&ctxt->rc_call_pcl)) + goto out_readlist; complete: rqstp->rq_xprt_ctxt = ctxt; @@ -880,10 +878,10 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) svc_xprt_copy_addrs(rqstp, xprt); return rqstp->rq_arg.len; -out_readchunk: - ret = svc_rdma_recv_read_chunk(rdma_xprt, rqstp, ctxt, p); +out_readlist: + ret = svc_rdma_process_read_list(rdma_xprt, rqstp, ctxt); if (ret < 0) - goto out_postfail; + goto out_readfail; return 0; out_err: @@ -891,7 +889,7 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) svc_rdma_recv_ctxt_put(rdma_xprt, ctxt); return 0; -out_postfail: +out_readfail: if (ret == -EINVAL) svc_rdma_send_error(rdma_xprt, ctxt, ret); svc_rdma_recv_ctxt_put(rdma_xprt, ctxt); diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 104b1d5a2203..6ec7bdc7b4d3 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -258,8 +258,8 @@ static void svc_rdma_write_done(struct ib_cq *cq, struct ib_wc *wc) /* State for pulling a Read chunk. */ struct svc_rdma_read_info { + struct svc_rqst *ri_rqst; struct svc_rdma_recv_ctxt *ri_readctxt; - unsigned int ri_position; unsigned int ri_pageno; unsigned int ri_pageoff; unsigned int ri_totalbytes; @@ -656,17 +656,29 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, return ret; } +/** + * svc_rdma_build_read_segment - Build RDMA Read WQEs to pull one RDMA segment + * @info: context for ongoing I/O + * @segment: co-ordinates of remote memory to be read + * + * Returns: + * %0: the Read WR chain was constructed successfully + * %-EINVAL: there were not enough rq_pages to finish + * %-ENOMEM: allocating a local resources failed + * %-EIO: a DMA mapping error occurred + */ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info, - struct svc_rqst *rqstp, - u32 rkey, u32 len, u64 offset) + const struct svc_rdma_segment *segment) { struct svc_rdma_recv_ctxt *head = info->ri_readctxt; struct svc_rdma_chunk_ctxt *cc = &info->ri_cc; + struct svc_rqst *rqstp = info->ri_rqst; struct svc_rdma_rw_ctxt *ctxt; - unsigned int sge_no, seg_len; + unsigned int sge_no, seg_len, len; struct scatterlist *sg; int ret; + len = segment->rs_length; sge_no = PAGE_ALIGN(info->ri_pageoff + len) >> PAGE_SHIFT; ctxt = svc_rdma_get_rw_ctxt(cc->cc_rdma, sge_no); if (!ctxt) @@ -700,8 +712,8 @@ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info, goto out_overrun; } - ret = svc_rdma_rw_ctx_init(cc->cc_rdma, ctxt, offset, rkey, - DMA_FROM_DEVICE); + ret = svc_rdma_rw_ctx_init(cc->cc_rdma, ctxt, segment->rs_offset, + segment->rs_handle, DMA_FROM_DEVICE); if (ret < 0) return -EIO; @@ -714,48 +726,63 @@ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info, return -EINVAL; } -/* Walk the segments in the Read chunk starting at @p and construct - * RDMA Read operations to pull the chunk to the server. +/** + * svc_rdma_build_read_chunk - Build RDMA Read WQEs to pull one RDMA chunk + * @info: context for ongoing I/O + * @chunk: Read chunk to pull + * + * Return values: + * %0: the Read WR chain was constructed successfully + * %-EINVAL: there were not enough resources to finish + * %-ENOMEM: allocating a local resources failed + * %-EIO: a DMA mapping error occurred */ -static int svc_rdma_build_read_chunk(struct svc_rqst *rqstp, - struct svc_rdma_read_info *info, - __be32 *p) +static int svc_rdma_build_read_chunk(struct svc_rdma_read_info *info, + const struct svc_rdma_chunk *chunk) { + const struct svc_rdma_segment *segment; int ret; ret = -EINVAL; - while (*p++ != xdr_zero && be32_to_cpup(p++) == info->ri_position) { - u32 handle, length; - u64 offset; - - p = xdr_decode_rdma_segment(p, &handle, &length, &offset); - ret = svc_rdma_build_read_segment(info, rqstp, handle, length, - offset); + pcl_for_each_segment(segment, chunk) { + ret = svc_rdma_build_read_segment(info, segment); if (ret < 0) break; - - info->ri_totalbytes += length; + info->ri_totalbytes += segment->rs_length; } return ret; } -/* Construct RDMA Reads to pull over a normal Read chunk. The chunk - * data lands in the page list of head->rc_arg.pages. +/** + * svc_rdma_read_data_items - Construct RDMA Reads to pull data item Read chunks + * @info: context for RDMA Reads + * + * The chunk data lands in the page list of head->rc_arg.pages. * * Currently NFSD does not look at the head->rc_arg.tail[0] iovec. * Therefore, XDR round-up of the Read chunk and trailing * inline content must both be added at the end of the pagelist. + * + * Return values: + * %0: RDMA Read WQEs were successfully built + * %-EINVAL: client provided too many chunks or segments, + * %-ENOMEM: rdma_rw context pool was exhausted, + * %-ENOTCONN: posting failed (connection is lost), + * %-EIO: rdma_rw initialization failed (DMA mapping, etc). */ -static int svc_rdma_build_normal_read_chunk(struct svc_rqst *rqstp, - struct svc_rdma_read_info *info, - __be32 *p) +static int svc_rdma_read_data_items(struct svc_rdma_read_info *info) { struct svc_rdma_recv_ctxt *head = info->ri_readctxt; struct xdr_buf *buf = &head->rc_arg; + struct svc_rdma_chunk *chunk; unsigned int length; int ret; - ret = svc_rdma_build_read_chunk(rqstp, info, p); + if (head->rc_read_pcl.cl_count > 1) + return -EINVAL; + + chunk = pcl_first_chunk(&head->rc_read_pcl); + ret = svc_rdma_build_read_chunk(info, chunk); if (ret < 0) goto out; @@ -766,11 +793,9 @@ static int svc_rdma_build_normal_read_chunk(struct svc_rqst *rqstp, * chunk is not included in either the pagelist or in * the tail. */ - head->rc_arg.tail[0].iov_base = - head->rc_arg.head[0].iov_base + info->ri_position; - head->rc_arg.tail[0].iov_len = - head->rc_arg.head[0].iov_len - info->ri_position; - head->rc_arg.head[0].iov_len = info->ri_position; + buf->tail[0].iov_base = buf->head[0].iov_base + chunk->ch_position; + buf->tail[0].iov_len = buf->head[0].iov_len - chunk->ch_position; + buf->head[0].iov_len = chunk->ch_position; /* Read chunk may need XDR roundup (see RFC 8166, s. 3.4.5.2). * @@ -790,26 +815,36 @@ static int svc_rdma_build_normal_read_chunk(struct svc_rqst *rqstp, return ret; } -/* Construct RDMA Reads to pull over a Position Zero Read chunk. - * The start of the data lands in the first page just after - * the Transport header, and the rest lands in the page list of +/** + * svc_rdma_read_special - Build RDMA Read WQEs to pull a Long Message + * @info: context for RDMA Reads + * + * The start of the data lands in the first page just after the + * Transport header, and the rest lands in the page list of * head->rc_arg.pages. * * Assumptions: - * - A PZRC has an XDR-aligned length (no implicit round-up). - * - There can be no trailing inline content (IOW, we assume - * a PZRC is never sent in an RDMA_MSG message, though it's - * allowed by spec). + * - A PZRC is never sent in an RDMA_MSG message, though it's + * allowed by spec. + * + * Return values: + * %0: RDMA Read WQEs were successfully built + * %-EINVAL: client provided too many chunks or segments, + * %-ENOMEM: rdma_rw context pool was exhausted, + * %-ENOTCONN: posting failed (connection is lost), + * %-EIO: rdma_rw initialization failed (DMA mapping, etc). */ -static int svc_rdma_build_pz_read_chunk(struct svc_rqst *rqstp, - struct svc_rdma_read_info *info, - __be32 *p) +static int svc_rdma_read_special(struct svc_rdma_read_info *info) { struct svc_rdma_recv_ctxt *head = info->ri_readctxt; struct xdr_buf *buf = &head->rc_arg; int ret; - ret = svc_rdma_build_read_chunk(rqstp, info, p); + if (head->rc_call_pcl.cl_count > 1) + return -EINVAL; + + ret = svc_rdma_build_read_chunk(info, + pcl_first_chunk(&head->rc_call_pcl)); if (ret < 0) goto out; @@ -846,24 +881,31 @@ static void svc_rdma_save_io_pages(struct svc_rqst *rqstp, } /** - * svc_rdma_recv_read_chunk - Pull a Read chunk from the client + * svc_rdma_process_read_list - Pull list of Read chunks from the client * @rdma: controlling RDMA transport * @rqstp: set of pages to use as Read sink buffers * @head: pages under I/O collect here - * @p: pointer to start of Read chunk * - * Returns: - * %0 if all needed RDMA Reads were posted successfully, - * %-EINVAL if client provided too many segments, - * %-ENOMEM if rdma_rw context pool was exhausted, - * %-ENOTCONN if posting failed (connection is lost), - * %-EIO if rdma_rw initialization failed (DMA mapping, etc). + * The RPC/RDMA protocol assumes that the upper layer's XDR decoders + * pull each Read chunk as they decode an incoming RPC message. * - * Assumptions: - * - All Read segments in @p have the same Position value. + * On Linux, however, the server needs to have a fully-constructed RPC + * message in rqstp->rq_arg when there is a positive return code from + * ->xpo_recvfrom. So the Read list is safety-checked immediately when + * it is received, then here the whole Read list is pulled all at once. + * The ingress RPC message is fully reconstructed once all associated + * RDMA Reads have completed. + * + * Return values: + * %1: all needed RDMA Reads were posted successfully, + * %-EINVAL: client provided too many chunks or segments, + * %-ENOMEM: rdma_rw context pool was exhausted, + * %-ENOTCONN: posting failed (connection is lost), + * %-EIO: rdma_rw initialization failed (DMA mapping, etc). */ -int svc_rdma_recv_read_chunk(struct svcxprt_rdma *rdma, struct svc_rqst *rqstp, - struct svc_rdma_recv_ctxt *head, __be32 *p) +int svc_rdma_process_read_list(struct svcxprt_rdma *rdma, + struct svc_rqst *rqstp, + struct svc_rdma_recv_ctxt *head) { struct svc_rdma_read_info *info; struct svc_rdma_chunk_ctxt *cc; @@ -885,16 +927,16 @@ int svc_rdma_recv_read_chunk(struct svcxprt_rdma *rdma, struct svc_rqst *rqstp, if (!info) return -ENOMEM; cc = &info->ri_cc; + info->ri_rqst = rqstp; info->ri_readctxt = head; info->ri_pageno = 0; info->ri_pageoff = 0; info->ri_totalbytes = 0; - info->ri_position = be32_to_cpup(p + 1); - if (info->ri_position) - ret = svc_rdma_build_normal_read_chunk(rqstp, info, p); + if (pcl_is_empty(&head->rc_call_pcl)) + ret = svc_rdma_read_data_items(info); else - ret = svc_rdma_build_pz_read_chunk(rqstp, info, p); + ret = svc_rdma_read_special(info); if (ret < 0) goto out_err; @@ -903,7 +945,7 @@ int svc_rdma_recv_read_chunk(struct svcxprt_rdma *rdma, struct svc_rqst *rqstp, if (ret < 0) goto out_err; svc_rdma_save_io_pages(rqstp, 0, head->rc_page_count); - return 0; + return 1; out_err: svc_rdma_read_info_free(info); From patchwork Mon Oct 26 18:55:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11858367 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 81B3A14B2 for ; Mon, 26 Oct 2020 18:55:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 574B721D7B for ; Mon, 26 Oct 2020 18:55:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="k9S6M3wl" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1782650AbgJZSzn (ORCPT ); Mon, 26 Oct 2020 14:55:43 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:33183 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404083AbgJZSzn (ORCPT ); Mon, 26 Oct 2020 14:55:43 -0400 Received: by mail-qk1-f195.google.com with SMTP id t128so4357351qke.0; Mon, 26 Oct 2020 11:55:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=Vymb2yCzvYitWm6SQObXkOBnQGFIWDdTY+l3B7ejUEA=; b=k9S6M3wl/gIevM0n/0x3VaxjmEbcbwQ2AeA8qaZx7myIv4aIgIPF8KRixz4WaWiJ04 J4q8XlXEfYsx/rWU2c243HkI6ugThhf6KF1FnQYcTINKoBiZKlCDETtpLuCoVR9v2Xct Qt+6ekfEQnYiCPqi4ssplufuXuSK3sLTWkgsnyp4MqXSZDwGd1f0Jd4rp4r/EEunGYC+ n0wMig7YJzgPD8DyHqAkiaeoy4s/tXLnTLK2x50nARR0lDDKzuQKkmsMr6ZsrrxWfHNw y56OzLXoT+SBWsyHpOZMvwsnaP4M0A6Lv7pwWvS4dKM8KiP8vEa1qqFaRwil5HbGKjR4 Q2Lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=Vymb2yCzvYitWm6SQObXkOBnQGFIWDdTY+l3B7ejUEA=; b=oqzo/ALZXoI/dm67H5kdgJWwoNqw9JjA/jlg35blmDXyuntDWLh67Tfkv1F386RW3F qDTp198+MOmb00dcErWtxDv/de+qHbWVJ0wzAwH4eSNFM1hDVtjGYsev6ltIJwvXKsWt aY0nE3R/Z0u5grVx+eU61A1EqB18MhHD58P5pMJ5QOYwkxuTWyB+0Hx/fmjSzZGvfKj6 UYuOQ/Owjs05xNaNRwmIWqzZRaj97336dOC4P8sPQnM2ltL5fxeNACrRaF5ymUiljJlF CKawI2bw8WDwiAQLJtyAQC/IaT4ROyyTrsw20+tBN37K+8jNoO2XDRBV7ShD4G3kTEMB oPfw== X-Gm-Message-State: AOAM532DC0O2tdvGPPrZlKbv0g9CJDLOHW15v+mu9FAetnouXI081yk2 ty5ep002fIfqd64AlHrJwTYfeGN9O1w= X-Google-Smtp-Source: ABdhPJy/u9F53qWlo1VUKHWtiJ1o7msQVgZP6QQkFTR4SkkjhY/U2HCx8/UMrrzpzLbP/a9XO+SlCg== X-Received: by 2002:a37:e202:: with SMTP id g2mr19684270qki.450.1603738540496; Mon, 26 Oct 2020 11:55:40 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id h9sm6959269qth.78.2020.10.26.11.55.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 11:55:39 -0700 (PDT) Sender: Chuck Lever Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 09QItcOa013673; Mon, 26 Oct 2020 18:55:38 GMT Subject: [PATCH 20/20] svcrdma: support multiple Read chunks per RPC From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Mon, 26 Oct 2020 14:55:38 -0400 Message-ID: <160373853877.1886.1700569114249029475.stgit@klimt.1015granger.net> In-Reply-To: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net> User-Agent: StGit/0.23-29-ga622f1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org An efficient way to handle multiple Read chunks is to post them all together and then take a single completion. This is also how the code is already structured: when the Read completion fires, all portions of the incoming RPC message are available to be assembled. The difficult problem is setting up the Read sink buffers so that the server pulls the client's data into place, making subsequent pull-up unnecessary. There are several cases: * No Read chunks. No-op. * One data item Read chunk. This is the fast case, where the inline part of the RPC-over-RDMA message becomes the head and tail, and the data item chunk is placed in buf->pages. * A Position-zero Read chunk. Treated like TCP: the Read chunk is pulled into contiguous pages. + A Position-zero Read chunk with data item chunks. Treated like TCP: all of the Read chunks are pulled into contiguous pages. + Multiple data item chunks. Treated like TCP: the inline part is copied and the data item chunks are pulled into contiguous pages. The "*" cases are already supported. This patch adds support for the "+" cases. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_rw.c | 236 +++++++++++++++++++++++++++++++++++-- 1 file changed, 222 insertions(+), 14 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 6ec7bdc7b4d3..12aa4c53b48f 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -754,7 +754,121 @@ static int svc_rdma_build_read_chunk(struct svc_rdma_read_info *info, } /** - * svc_rdma_read_data_items - Construct RDMA Reads to pull data item Read chunks + * svc_rdma_copy_inline_range - Copy part of the inline content into pages + * @info: context for RDMA Reads + * @offset: offset into the Receive buffer of region to copy + * @remaining: length of region to copy + * + * Take a page at a time from rqstp->rq_pages and copy the inline + * content from the Receive buffer into that page. Update + * info->ri_pageno and info->ri_pageoff so that the next RDMA Read + * result will land contiguously with the copied content. + * + * Return values: + * %0: Inline content was successfully copied + * %-EINVAL: offset or length was incorrect + */ +static int svc_rdma_copy_inline_range(struct svc_rdma_read_info *info, + unsigned int offset, + unsigned int remaining) +{ + struct svc_rdma_recv_ctxt *head = info->ri_readctxt; + unsigned char *dst, *src = head->rc_recv_buf; + struct svc_rqst *rqstp = info->ri_rqst; + unsigned int page_no, numpages; + + numpages = PAGE_ALIGN(info->ri_pageoff + remaining) >> PAGE_SHIFT; + for (page_no = 0; page_no < numpages; page_no++) { + unsigned int page_len; + + page_len = min_t(unsigned int, remaining, + PAGE_SIZE - info->ri_pageoff); + + head->rc_arg.pages[info->ri_pageno] = + rqstp->rq_pages[info->ri_pageno]; + if (!info->ri_pageoff) + head->rc_page_count++; + + dst = page_address(head->rc_arg.pages[info->ri_pageno]); + memcpy(dst + info->ri_pageno, src + offset, page_len); + + info->ri_totalbytes += page_len; + info->ri_pageoff += page_len; + if (info->ri_pageoff == PAGE_SIZE) { + info->ri_pageno++; + info->ri_pageoff = 0; + } + remaining -= page_len; + offset += page_len; + } + + return -EINVAL; +} + +/** + * svc_rdma_read_multiple_chunks - Construct RDMA Reads to pull data item Read chunks + * @info: context for RDMA Reads + * + * The chunk data lands in head->rc_arg as a series of contiguous pages, + * like an incoming TCP call. + * + * Return values: + * %0: RDMA Read WQEs were successfully built + * %-EINVAL: client provided too many chunks or segments, + * %-ENOMEM: rdma_rw context pool was exhausted, + * %-ENOTCONN: posting failed (connection is lost), + * %-EIO: rdma_rw initialization failed (DMA mapping, etc). + */ +static noinline int svc_rdma_read_multiple_chunks(struct svc_rdma_read_info *info) +{ + struct svc_rdma_recv_ctxt *head = info->ri_readctxt; + const struct svc_rdma_pcl *pcl = &head->rc_read_pcl; + struct svc_rdma_chunk *chunk, *next; + struct xdr_buf *buf = &head->rc_arg; + unsigned int start, length; + int ret; + + start = 0; + chunk = pcl_first_chunk(pcl); + length = chunk->ch_position; + ret = svc_rdma_copy_inline_range(info, start, length); + if (ret < 0) + return ret; + + pcl_for_each_chunk(chunk, pcl) { + ret = svc_rdma_build_read_chunk(info, chunk); + if (ret < 0) + return ret; + + next = pcl_next_chunk(pcl, chunk); + if (!next) + break; + + start += length; + length = next->ch_position - info->ri_totalbytes; + ret = svc_rdma_copy_inline_range(info, start, length); + if (ret < 0) + return ret; + } + + start += length; + length = head->rc_byte_len - start; + ret = svc_rdma_copy_inline_range(info, start, length); + if (ret < 0) + return ret; + + buf->len += info->ri_totalbytes; + buf->buflen += info->ri_totalbytes; + + head->rc_hdr_count = 1; + buf->head[0].iov_base = page_address(head->rc_pages[0]); + buf->head[0].iov_len = min_t(size_t, PAGE_SIZE, info->ri_totalbytes); + buf->page_len = info->ri_totalbytes - buf->head[0].iov_len; + return 0; +} + +/** + * svc_rdma_read_data_item - Construct RDMA Reads to pull data item Read chunks * @info: context for RDMA Reads * * The chunk data lands in the page list of head->rc_arg.pages. @@ -770,7 +884,7 @@ static int svc_rdma_build_read_chunk(struct svc_rdma_read_info *info, * %-ENOTCONN: posting failed (connection is lost), * %-EIO: rdma_rw initialization failed (DMA mapping, etc). */ -static int svc_rdma_read_data_items(struct svc_rdma_read_info *info) +static int svc_rdma_read_data_item(struct svc_rdma_read_info *info) { struct svc_rdma_recv_ctxt *head = info->ri_readctxt; struct xdr_buf *buf = &head->rc_arg; @@ -778,9 +892,6 @@ static int svc_rdma_read_data_items(struct svc_rdma_read_info *info) unsigned int length; int ret; - if (head->rc_read_pcl.cl_count > 1) - return -EINVAL; - chunk = pcl_first_chunk(&head->rc_read_pcl); ret = svc_rdma_build_read_chunk(info, chunk); if (ret < 0) @@ -815,6 +926,104 @@ static int svc_rdma_read_data_items(struct svc_rdma_read_info *info) return ret; } +/** + * svc_rdma_read_chunk_range - Build RDMA Read WQEs for portion of a chunk + * @info: context for RDMA Reads + * @chunk: parsed Call chunk to pull + * @offset: offset of region to pull + * @length: length of region to pull + * + * Return values: + * %0: RDMA Read WQEs were successfully built + * %-EINVAL: there were not enough resources to finish + * %-ENOMEM: rdma_rw context pool was exhausted, + * %-ENOTCONN: posting failed (connection is lost), + * %-EIO: rdma_rw initialization failed (DMA mapping, etc). + */ +static int svc_rdma_read_chunk_range(struct svc_rdma_read_info *info, + const struct svc_rdma_chunk *chunk, + unsigned int offset, unsigned int length) +{ + const struct svc_rdma_segment *segment; + int ret; + + ret = -EINVAL; + pcl_for_each_segment(segment, chunk) { + struct svc_rdma_segment dummy; + + if (offset > segment->rs_length) { + offset -= segment->rs_length; + continue; + } + + dummy.rs_handle = segment->rs_handle; + dummy.rs_length = min_t(u32, length, segment->rs_length) - offset; + dummy.rs_offset = segment->rs_offset + offset; + + ret = svc_rdma_build_read_segment(info, &dummy); + if (ret < 0) + break; + + info->ri_totalbytes += dummy.rs_length; + length -= dummy.rs_length; + offset = 0; + } + return ret; +} + +/** + * svc_rdma_read_call_chunk - Build RDMA Read WQEs to pull a Long Message + * @info: context for RDMA Reads + * + * Return values: + * %0: RDMA Read WQEs were successfully built + * %-EINVAL: there were not enough resources to finish + * %-ENOMEM: rdma_rw context pool was exhausted, + * %-ENOTCONN: posting failed (connection is lost), + * %-EIO: rdma_rw initialization failed (DMA mapping, etc). + */ +static int svc_rdma_read_call_chunk(struct svc_rdma_read_info *info) +{ + struct svc_rdma_recv_ctxt *head = info->ri_readctxt; + const struct svc_rdma_chunk *call_chunk = + pcl_first_chunk(&head->rc_call_pcl); + const struct svc_rdma_pcl *pcl = &head->rc_read_pcl; + struct svc_rdma_chunk *chunk, *next; + unsigned int start, length; + int ret; + + if (pcl_is_empty(pcl)) + return svc_rdma_build_read_chunk(info, call_chunk); + + start = 0; + chunk = pcl_first_chunk(pcl); + length = chunk->ch_position; + ret = svc_rdma_read_chunk_range(info, call_chunk, start, length); + if (ret < 0) + return ret; + + pcl_for_each_chunk(chunk, pcl) { + ret = svc_rdma_build_read_chunk(info, chunk); + if (ret < 0) + return ret; + + next = pcl_next_chunk(pcl, chunk); + if (!next) + break; + + start += length; + length = next->ch_position - info->ri_totalbytes; + ret = svc_rdma_read_chunk_range(info, call_chunk, + start, length); + if (ret < 0) + return ret; + } + + start += length; + length = call_chunk->ch_length - start; + return svc_rdma_read_chunk_range(info, call_chunk, start, length); +} + /** * svc_rdma_read_special - Build RDMA Read WQEs to pull a Long Message * @info: context for RDMA Reads @@ -834,17 +1043,13 @@ static int svc_rdma_read_data_items(struct svc_rdma_read_info *info) * %-ENOTCONN: posting failed (connection is lost), * %-EIO: rdma_rw initialization failed (DMA mapping, etc). */ -static int svc_rdma_read_special(struct svc_rdma_read_info *info) +static noinline int svc_rdma_read_special(struct svc_rdma_read_info *info) { struct svc_rdma_recv_ctxt *head = info->ri_readctxt; struct xdr_buf *buf = &head->rc_arg; int ret; - if (head->rc_call_pcl.cl_count > 1) - return -EINVAL; - - ret = svc_rdma_build_read_chunk(info, - pcl_first_chunk(&head->rc_call_pcl)); + ret = svc_rdma_read_call_chunk(info); if (ret < 0) goto out; @@ -933,9 +1138,12 @@ int svc_rdma_process_read_list(struct svcxprt_rdma *rdma, info->ri_pageoff = 0; info->ri_totalbytes = 0; - if (pcl_is_empty(&head->rc_call_pcl)) - ret = svc_rdma_read_data_items(info); - else + if (pcl_is_empty(&head->rc_call_pcl)) { + if (head->rc_read_pcl.cl_count == 1) + ret = svc_rdma_read_data_item(info); + else + ret = svc_rdma_read_multiple_chunks(info); + } else ret = svc_rdma_read_special(info); if (ret < 0) goto out_err;