From patchwork Mon Sep 3 15:29:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 10586067 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 47E1814BD for ; Mon, 3 Sep 2018 15:30:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 37AF129631 for ; Mon, 3 Sep 2018 15:30:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2C2F72963A; Mon, 3 Sep 2018 15:30:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AEB3529631 for ; Mon, 3 Sep 2018 15:30:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727286AbeICTur (ORCPT ); Mon, 3 Sep 2018 15:50:47 -0400 Received: from mail-io0-f196.google.com ([209.85.223.196]:38759 "EHLO mail-io0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726314AbeICTuq (ORCPT ); Mon, 3 Sep 2018 15:50:46 -0400 Received: by mail-io0-f196.google.com with SMTP id y3-v6so691397ioc.5 for ; Mon, 03 Sep 2018 08:30:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=NJ/HjQ1DwXt+fg8GNQ9/k4/SwMjbFWs84pss80ROCg8=; b=tzbylxRzWEK84QaHZRyFxbwXGqG9U2fY1s7K9uPeETaN/mCYFv57J3pZ7NFzqPSRw4 8xgEASg8A4j29iayx+Vszn1Yu9l2/Hni5h6hcfjOPoqNNtxmlX6liZLJ2WSaxmBbxzsV eDk4RZjQK9RkRXuOo//C9eTXY7aGpIFIliOAG/9WT7QKeO/hjnmbZ7YsDNYW6KPMOryW peNO1F00QBaurb3dgI1l9CSD9+bLRreIVcVMy2jlOzRl2oxc3Dy3eT2V0FEK4CJGZ0Q7 0ZNW+jxx2akbpLXtbe7+eNfLsYsGLFiZhwEOWUjQbFj3AbiACM2XSXNzvGBD9jiSujQ5 WVNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NJ/HjQ1DwXt+fg8GNQ9/k4/SwMjbFWs84pss80ROCg8=; b=jiIQZFw6Pl1C7uNDR1Nc2s3qZMq21SfDD3KVhUM93bUB3Q7Xcy1tQzHAxuRwfOL3+U EywxTvMkyO+IBjXHcNITg08VTkkPJHhlII6j4hkm+br2Fx8NOlM2GxcKDf1TSfbWaBbE m+OpwNGrI0kR3NB3NXSQghr8xBzyDiTMftWDn/+u9ZSy6uWxFnU1TTfcor2iWQDjAmm9 2LqTcxNK2seLE/f8N0kwgKfOttRPgp5HQsaB1mQF/2VyZ9fJJXbrXASrZpYH3PUX/8Fw BV9FxCF6m1hbX2fj+RCY9Ldc3Y2LH1M8VczotJbm7hqg8YmrkOaXlMYwv4oHLDOYiZ5z sp9g== X-Gm-Message-State: APzg51ATe9Mg7dN4z9ISGR1qvwHHvsCHzaJAj3W96dZLrWk6i//TESNi QP6uYUColJxLQJDYFsoHrlqPcR4= X-Google-Smtp-Source: ANB0Vdbm3QEzNhfatxGXw2EPRwXMjOdDQLs4gneO2DRQjEdssXJJTSrTFW8M9rYUzHCKJ6nz6+QaPQ== X-Received: by 2002:a6b:5315:: with SMTP id h21-v6mr20367690iob.51.1535988606190; Mon, 03 Sep 2018 08:30:06 -0700 (PDT) Received: from leira.trondhjem.org.localdomain (c-68-40-195-73.hsd1.mi.comcast.net. [68.40.195.73]) by smtp.gmail.com with ESMTPSA id c25-v6sm7040027iob.30.2018.09.03.08.30.05 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 03 Sep 2018 08:30:05 -0700 (PDT) From: Trond Myklebust X-Google-Original-From: Trond Myklebust To: linux-nfs@vger.kernel.org Subject: [PATCH 08/27] SUNRPC: Add socket transmit queue offset tracking Date: Mon, 3 Sep 2018 11:29:17 -0400 Message-Id: <20180903152936.24325-9-trond.myklebust@hammerspace.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180903152936.24325-8-trond.myklebust@hammerspace.com> References: <20180903152936.24325-1-trond.myklebust@hammerspace.com> <20180903152936.24325-2-trond.myklebust@hammerspace.com> <20180903152936.24325-3-trond.myklebust@hammerspace.com> <20180903152936.24325-4-trond.myklebust@hammerspace.com> <20180903152936.24325-5-trond.myklebust@hammerspace.com> <20180903152936.24325-6-trond.myklebust@hammerspace.com> <20180903152936.24325-7-trond.myklebust@hammerspace.com> <20180903152936.24325-8-trond.myklebust@hammerspace.com> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Trond Myklebust --- include/linux/sunrpc/xprtsock.h | 7 ++++++ net/sunrpc/xprtsock.c | 40 ++++++++++++++++++--------------- 2 files changed, 29 insertions(+), 18 deletions(-) diff --git a/include/linux/sunrpc/xprtsock.h b/include/linux/sunrpc/xprtsock.h index 90d5ca8e65f4..005cfb6e7238 100644 --- a/include/linux/sunrpc/xprtsock.h +++ b/include/linux/sunrpc/xprtsock.h @@ -42,6 +42,13 @@ struct sock_xprt { flags; } recv; + /* + * State of TCP transmit queue + */ + struct { + u32 offset; + } xmit; + /* * Connection of transports */ diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index ec1e3f93e707..629cc45e1e6c 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -461,7 +461,7 @@ static int xs_nospace(struct rpc_task *task) int ret = -EAGAIN; dprintk("RPC: %5u xmit incomplete (%u left of %u)\n", - task->tk_pid, req->rq_slen - req->rq_bytes_sent, + task->tk_pid, req->rq_slen - transport->xmit.offset, req->rq_slen); /* Protect against races with write_space */ @@ -528,19 +528,22 @@ static int xs_local_send_request(struct rpc_task *task) req->rq_svec->iov_base, req->rq_svec->iov_len); req->rq_xtime = ktime_get(); - status = xs_sendpages(transport->sock, NULL, 0, xdr, req->rq_bytes_sent, + status = xs_sendpages(transport->sock, NULL, 0, xdr, + transport->xmit.offset, true, &sent); dprintk("RPC: %s(%u) = %d\n", - __func__, xdr->len - req->rq_bytes_sent, status); + __func__, xdr->len - transport->xmit.offset, status); if (status == -EAGAIN && sock_writeable(transport->inet)) status = -ENOBUFS; if (likely(sent > 0) || status == 0) { - req->rq_bytes_sent += sent; - req->rq_xmit_bytes_sent += sent; + transport->xmit.offset += sent; + req->rq_bytes_sent = transport->xmit.offset; if (likely(req->rq_bytes_sent >= req->rq_slen)) { + req->rq_xmit_bytes_sent += transport->xmit.offset; req->rq_bytes_sent = 0; + transport->xmit.offset = 0; return 0; } status = -EAGAIN; @@ -592,10 +595,10 @@ static int xs_udp_send_request(struct rpc_task *task) return -ENOTCONN; req->rq_xtime = ktime_get(); status = xs_sendpages(transport->sock, xs_addr(xprt), xprt->addrlen, - xdr, req->rq_bytes_sent, true, &sent); + xdr, 0, true, &sent); dprintk("RPC: xs_udp_send_request(%u) = %d\n", - xdr->len - req->rq_bytes_sent, status); + xdr->len, status); /* firewall is blocking us, don't return -EAGAIN or we end up looping */ if (status == -EPERM) @@ -684,17 +687,20 @@ static int xs_tcp_send_request(struct rpc_task *task) while (1) { sent = 0; status = xs_sendpages(transport->sock, NULL, 0, xdr, - req->rq_bytes_sent, zerocopy, &sent); + transport->xmit.offset, + zerocopy, &sent); dprintk("RPC: xs_tcp_send_request(%u) = %d\n", - xdr->len - req->rq_bytes_sent, status); + xdr->len - transport->xmit.offset, status); /* If we've sent the entire packet, immediately * reset the count of bytes sent. */ - req->rq_bytes_sent += sent; - req->rq_xmit_bytes_sent += sent; + transport->xmit.offset += sent; + req->rq_bytes_sent = transport->xmit.offset; if (likely(req->rq_bytes_sent >= req->rq_slen)) { + req->rq_xmit_bytes_sent += transport->xmit.offset; req->rq_bytes_sent = 0; + transport->xmit.offset = 0; return 0; } @@ -760,18 +766,13 @@ static int xs_tcp_send_request(struct rpc_task *task) */ static void xs_tcp_release_xprt(struct rpc_xprt *xprt, struct rpc_task *task) { - struct rpc_rqst *req; + struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt); if (task != xprt->snd_task) return; if (task == NULL) goto out_release; - req = task->tk_rqstp; - if (req == NULL) - goto out_release; - if (req->rq_bytes_sent == 0) - goto out_release; - if (req->rq_bytes_sent == req->rq_snd_buf.len) + if (transport->xmit.offset == 0 || !xprt_connected(xprt)) goto out_release; set_bit(XPRT_CLOSE_WAIT, &xprt->state); out_release: @@ -2021,6 +2022,8 @@ static int xs_local_finish_connecting(struct rpc_xprt *xprt, write_unlock_bh(&sk->sk_callback_lock); } + transport->xmit.offset = 0; + /* Tell the socket layer to start connecting... */ xprt->stat.connect_count++; xprt->stat.connect_start = jiffies; @@ -2384,6 +2387,7 @@ static int xs_tcp_finish_connecting(struct rpc_xprt *xprt, struct socket *sock) transport->recv.len = 0; transport->recv.copied = 0; transport->recv.flags = TCP_RCV_COPY_FRAGHDR | TCP_RCV_COPY_XID; + transport->xmit.offset = 0; /* Tell the socket layer to start connecting... */ xprt->stat.connect_count++;