From patchwork Fri Jun 23 21:18:08 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 9807363 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0B5156086C for ; Fri, 23 Jun 2017 21:18:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EAFE628584 for ; Fri, 23 Jun 2017 21:18:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DFAA92864B; Fri, 23 Jun 2017 21:18:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E64FF28584 for ; Fri, 23 Jun 2017 21:18:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754527AbdFWVSM (ORCPT ); Fri, 23 Jun 2017 17:18:12 -0400 Received: from mail-it0-f67.google.com ([209.85.214.67]:36647 "EHLO mail-it0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753598AbdFWVSK (ORCPT ); Fri, 23 Jun 2017 17:18:10 -0400 Received: by mail-it0-f67.google.com with SMTP id 185so8364331itv.3; Fri, 23 Jun 2017 14:18:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=PCbTUEYkwjet65sOy6H0H0d1XFu4KE2HnQF/PLGea3g=; b=ew39gEgfVroHoP1DhPiGiQSz8tZo5o7mkVN3jGK96Nx5Mr34gUEDzUgQ/o0dfHseFp F2mmFuWBist1DeUjqOk6IH2Rweu/tkjhuxeyzQKq9Aa7hQV7UEjWzGuPTGsINrCVsynt MrBpgUa2753QQbQ4zGiYbaiCKUH7x8pWV//47dvQcBdknS7DnEZdizCCZPqK5NQCjFsR fzoSf58YoBrZV8nxX0jQm/tFyHFTbMkGxuZomS3Zr6jeIrFvtA3eVL91fOHBZzjcxLPH 7s80TH8gkKGs4q6ybSkbpyRvHT/HBUKZhNOsYDJ8v461EgutGHVWC2HB5u8RzGRPEsJA P2tg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=PCbTUEYkwjet65sOy6H0H0d1XFu4KE2HnQF/PLGea3g=; b=of5NYpDINWcdvgZeRSxO60+riJCT3313Aewuo2zJXqgXsw/YrLejvEjwfKZBOA11Sj Qwwy3qDaYhWiT3/2fEsGm9RjI4aemfDEcRI47357KLIvkMY2EgvJ2RmRleZP70ECSkf+ oXoHzBhndRUiMDheB+SgA3saNSKI5GkAMbcv/+ZJX63h85e1seKhhBRyB8IkdtPHYrIf K11OcabkAAqoi05PyXfH5o0P4xcdMpDdSX1rVIjqmgyqNjSNiRbHVkrHyutIp2xZN8nD Yuectqe5Z2KqW3L1aYkskG0o7O6HhvE6hqy1oYEjdabdvpNiDJ5XNoD0w0Ky3sopF43G CtSw== X-Gm-Message-State: AKS2vOwoG9sxJgQJDC0fMpIFyJkOgr73W9DqXTSSjYSZK4AgEzYULT5R num2gaA8MhpzeA== X-Received: by 10.36.203.135 with SMTP id u129mr9387749itg.3.1498252689250; Fri, 23 Jun 2017 14:18:09 -0700 (PDT) Received: from klimt.1015granger.net (c-68-46-169-226.hsd1.mi.comcast.net. [68.46.169.226]) by smtp.gmail.com with ESMTPSA id 128sm3667946ioe.6.2017.06.23.14.18.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Jun 2017 14:18:08 -0700 (PDT) Subject: [PATCH v3 08/18] svcrdma: Don't account for Receive queue "starvation" From: Chuck Lever To: bfields@fieldses.org Cc: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Fri, 23 Jun 2017 17:18:08 -0400 Message-ID: <20170623211808.5162.68255.stgit@klimt.1015granger.net> In-Reply-To: <20170623211150.5162.59075.stgit@klimt.1015granger.net> References: <20170623211150.5162.59075.stgit@klimt.1015granger.net> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP >From what I can tell, calling ->recvfrom when there is no work to do is a normal part of operation. This is the only way svc_recv can tell when there is no more data ready to receive on the transport. Neither the TCP nor the UDP transport implementations have a "starve" metric. The cost of receive starvation accounting is bumping an atomic, which results in extra (IMO unnecessary) bus traffic between CPU sockets, while holding a spin lock. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 21 ++++++--------------- 1 file changed, 6 insertions(+), 15 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index b480893..1452bd0 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -844,9 +844,9 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) struct svc_xprt *xprt = rqstp->rq_xprt; struct svcxprt_rdma *rdma_xprt = container_of(xprt, struct svcxprt_rdma, sc_xprt); - struct svc_rdma_op_ctxt *ctxt = NULL; + struct svc_rdma_op_ctxt *ctxt; struct rpcrdma_msg *rmsgp; - int ret = 0; + int ret; dprintk("svcrdma: rqstp=%p\n", rqstp); @@ -863,21 +863,13 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) struct svc_rdma_op_ctxt, list); list_del(&ctxt->list); } else { - atomic_inc(&rdma_stat_rq_starve); + /* No new incoming requests, terminate the loop */ clear_bit(XPT_DATA, &xprt->xpt_flags); - ctxt = NULL; + spin_unlock(&rdma_xprt->sc_rq_dto_lock); + return 0; } spin_unlock(&rdma_xprt->sc_rq_dto_lock); - if (!ctxt) { - /* This is the EAGAIN path. The svc_recv routine will - * return -EAGAIN, the nfsd thread will go to call into - * svc_recv again and we shouldn't be on the active - * transport list - */ - if (test_bit(XPT_CLOSE, &xprt->xpt_flags)) - goto defer; - goto out; - } + dprintk("svcrdma: processing ctxt=%p on xprt=%p, rqstp=%p\n", ctxt, rdma_xprt, rqstp); atomic_inc(&rdma_stat_recv); @@ -920,7 +912,6 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) + rqstp->rq_arg.page_len + rqstp->rq_arg.tail[0].iov_len; svc_rdma_put_context(ctxt, 0); - out: dprintk("svcrdma: ret=%d, rq_arg.len=%u, " "rq_arg.head[0].iov_base=%p, rq_arg.head[0].iov_len=%zd\n", ret, rqstp->rq_arg.len,