From patchwork Tue May 12 21:22:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544069 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DB119139A for ; Tue, 12 May 2020 21:22:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BF6A620659 for ; Tue, 12 May 2020 21:22:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="I5w1k7bw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728882AbgELVWN (ORCPT ); Tue, 12 May 2020 17:22:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1728220AbgELVWM (ORCPT ); Tue, 12 May 2020 17:22:12 -0400 Received: from mail-qv1-xf43.google.com (mail-qv1-xf43.google.com [IPv6:2607:f8b0:4864:20::f43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B0B81C061A0C; Tue, 12 May 2020 14:22:12 -0700 (PDT) Received: by mail-qv1-xf43.google.com with SMTP id a4so3948086qvj.3; Tue, 12 May 2020 14:22:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=9JR3+o/T8DQ0blKl8KSpczb99YGs1HuXoGZVrCIoAfE=; b=I5w1k7bwqGaJhXS7Gp5qDSA29nKiT33kh1CUxAKICAXBR6iC69rJlr2brv4UYJnGwg mUlUdzIArIq4zRYPwh4ETv8ajj/rHHJMbWQ6WLIEetHMGyyvc/bRAd7eTgd0S0QtlVpH UYIsinDZORIza0SfaXv5G+gYPb5tTf6Lx7tprmkeF1bmQhrScjh134UHsV8NGJPiU1cO F3BlSHBe/R+JdGMocAQxp+HWL4S4ITrPJ7+gOGKDENbKyDwyiBK5vc1wqR0Q6WhD3dzY rrfu+xhNMGlqxQBO0fVikii05PmLJWukGyH/mYvctZAV/iLXTH+rCGZ/yPyNnpaUq/qE yiFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=9JR3+o/T8DQ0blKl8KSpczb99YGs1HuXoGZVrCIoAfE=; b=tc50VOalCy54e3KgVkDFW/WEhrGtSbSk/uV1QZvk+/6cRRHXRiqSHxX282a1tyu659 hxh8xgNf/QeIEk9ud+wfxJleoUPaa4s8q36yAC/Ejy6K+oDd4Uo3aB1+RPPlb4MSDYwC /sAtDryRJSWkSVwMItUMJ6ppD/n8uuAffBhgjChMHRe+PsqD+BGZlxa5PZs2UuFFxqHP nBbjJNHXmfB/I5zSs7uqKZ9UtokgnTAFefVfyAV9tXCleAYbCpEUHDBYBnrrCQ0UuIT2 29SDqpvl0gy3vDXkzOPsz7T9b0ZISFDG6tu10VVoEaENAeQ2xQF74l3Ltn2E4ShfvR9O lvnQ== X-Gm-Message-State: AOAM532OJm4CKoFYHNmP1nkWOPGO2+eOhVbzw6BEvya+DIuliC3MokwY Y+wS73nZTKkcj41nO+v1w968l3uW X-Google-Smtp-Source: ABdhPJwQjI0ZtEt6nneV9KG4zVp0a59eJEssbn+Z1FQAwYKs6br5HjFY4L0S3vNv2/reYp8vDm0BBw== X-Received: by 2002:a0c:f485:: with SMTP id i5mr3125939qvm.144.1589318531549; Tue, 12 May 2020 14:22:11 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id e26sm11973844qka.85.2020.05.12.14.22.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:22:11 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLMAVg009875; Tue, 12 May 2020 21:22:10 GMT Subject: [PATCH v2 01/29] SUNRPC: Move xpt_mutex into socket xpo_sendto methods From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:22:10 -0400 Message-ID: <20200512212210.5826.49628.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org It appears that the RPC/RDMA transport does not need serialization of calls to its xpo_sendto method. Move the mutex into the socket methods that still need that serialization. Tail latencies are unambiguously better with this patch applied. fio randrw 8KB 70/30 on NFSv3, smaller numbers are better: clat percentiles (usec): With xpt_mutex: r | 99.99th=[ 8848] w | 99.99th=[ 9634] Without xpt_mutex: r | 99.99th=[ 8586] w | 99.99th=[ 8979] Serializing the construction of RPC/RDMA transport headers is not really necessary at this point, because the Linux NFS server implementation never changes its credit grant on a connection. If that should change, then svc_rdma_sendto will need to serialize access to the transport's credit grant fields. Reported-by: kbuild test robot [ cel: fix uninitialized variable warning ] Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_xprt.h | 6 +++++ net/sunrpc/svc_xprt.c | 12 ++-------- net/sunrpc/svcsock.c | 25 ++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 35 +++++++++++++--------------- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 10 +++----- net/sunrpc/xprtsock.c | 12 ++++++++-- 6 files changed, 64 insertions(+), 36 deletions(-) diff --git a/include/linux/sunrpc/svc_xprt.h b/include/linux/sunrpc/svc_xprt.h index 9e1e046de176..aca35ab5cff2 100644 --- a/include/linux/sunrpc/svc_xprt.h +++ b/include/linux/sunrpc/svc_xprt.h @@ -117,6 +117,12 @@ static inline int register_xpt_user(struct svc_xprt *xpt, struct svc_xpt_user *u return 0; } +static inline bool svc_xprt_is_dead(const struct svc_xprt *xprt) +{ + return (test_bit(XPT_DEAD, &xprt->xpt_flags) != 0) || + (test_bit(XPT_CLOSE, &xprt->xpt_flags) != 0); +} + int svc_reg_xprt_class(struct svc_xprt_class *); void svc_unreg_xprt_class(struct svc_xprt_class *); void svc_xprt_init(struct net *, struct svc_xprt_class *, struct svc_xprt *, diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c index 2284ff038dad..07cdbf7d5764 100644 --- a/net/sunrpc/svc_xprt.c +++ b/net/sunrpc/svc_xprt.c @@ -914,16 +914,10 @@ int svc_send(struct svc_rqst *rqstp) xb->page_len + xb->tail[0].iov_len; trace_svc_sendto(xb); - - /* Grab mutex to serialize outgoing data. */ - mutex_lock(&xprt->xpt_mutex); trace_svc_stats_latency(rqstp); - if (test_bit(XPT_DEAD, &xprt->xpt_flags) - || test_bit(XPT_CLOSE, &xprt->xpt_flags)) - len = -ENOTCONN; - else - len = xprt->xpt_ops->xpo_sendto(rqstp); - mutex_unlock(&xprt->xpt_mutex); + + len = xprt->xpt_ops->xpo_sendto(rqstp); + trace_svc_send(rqstp, len); svc_xprt_release(rqstp); diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 023514e392b3..3e7b6445e317 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -506,6 +506,9 @@ static int svc_udp_recvfrom(struct svc_rqst *rqstp) * svc_udp_sendto - Send out a reply on a UDP socket * @rqstp: completed svc_rqst * + * xpt_mutex ensures @rqstp's whole message is written to the socket + * without interruption. + * * Returns the number of bytes sent, or a negative errno. */ static int svc_udp_sendto(struct svc_rqst *rqstp) @@ -531,6 +534,11 @@ static int svc_udp_sendto(struct svc_rqst *rqstp) svc_set_cmsg_data(rqstp, cmh); + mutex_lock(&xprt->xpt_mutex); + + if (svc_xprt_is_dead(xprt)) + goto out_notconn; + err = xprt_sock_sendmsg(svsk->sk_sock, &msg, xdr, 0, 0, &sent); xdr_free_bvec(xdr); if (err == -ECONNREFUSED) { @@ -538,9 +546,15 @@ static int svc_udp_sendto(struct svc_rqst *rqstp) err = xprt_sock_sendmsg(svsk->sk_sock, &msg, xdr, 0, 0, &sent); xdr_free_bvec(xdr); } + + mutex_unlock(&xprt->xpt_mutex); if (err < 0) return err; return sent; + +out_notconn: + mutex_unlock(&xprt->xpt_mutex); + return -ENOTCONN; } static int svc_udp_has_wspace(struct svc_xprt *xprt) @@ -1063,6 +1077,9 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) * svc_tcp_sendto - Send out a reply on a TCP socket * @rqstp: completed svc_rqst * + * xpt_mutex ensures @rqstp's whole message is written to the socket + * without interruption. + * * Returns the number of bytes sent, or a negative errno. */ static int svc_tcp_sendto(struct svc_rqst *rqstp) @@ -1080,12 +1097,19 @@ static int svc_tcp_sendto(struct svc_rqst *rqstp) svc_release_skb(rqstp); + mutex_lock(&xprt->xpt_mutex); + if (svc_xprt_is_dead(xprt)) + goto out_notconn; err = xprt_sock_sendmsg(svsk->sk_sock, &msg, xdr, 0, marker, &sent); xdr_free_bvec(xdr); if (err < 0 || sent != (xdr->len + sizeof(marker))) goto out_close; + mutex_unlock(&xprt->xpt_mutex); return sent; +out_notconn: + mutex_unlock(&xprt->xpt_mutex); + return -ENOTCONN; out_close: pr_notice("rpc-srv/tcp: %s: %s %d when sending %d bytes - shutting down socket\n", xprt->xpt_server->sv_name, @@ -1093,6 +1117,7 @@ static int svc_tcp_sendto(struct svc_rqst *rqstp) (err < 0) ? err : sent, xdr->len); set_bit(XPT_CLOSE, &xprt->xpt_flags); svc_xprt_enqueue(xprt); + mutex_unlock(&xprt->xpt_mutex); return -EAGAIN; } diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index af7eb8d202ae..d9aab4504f2c 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -210,34 +210,31 @@ rpcrdma_bc_send_request(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst) return -ENOTCONN; } -/* Send an RPC call on the passive end of a transport - * connection. +/** + * xprt_rdma_bc_send_request - Send a reverse-direction Call + * @rqst: rpc_rqst containing Call message to be sent + * + * Return values: + * %0 if the message was sent successfully + * %ENOTCONN if the message was not sent */ -static int -xprt_rdma_bc_send_request(struct rpc_rqst *rqst) +static int xprt_rdma_bc_send_request(struct rpc_rqst *rqst) { struct svc_xprt *sxprt = rqst->rq_xprt->bc_xprt; - struct svcxprt_rdma *rdma; + struct svcxprt_rdma *rdma = + container_of(sxprt, struct svcxprt_rdma, sc_xprt); int ret; dprintk("svcrdma: sending bc call with xid: %08x\n", be32_to_cpu(rqst->rq_xid)); - mutex_lock(&sxprt->xpt_mutex); - - ret = -ENOTCONN; - rdma = container_of(sxprt, struct svcxprt_rdma, sc_xprt); - if (!test_bit(XPT_DEAD, &sxprt->xpt_flags)) { - ret = rpcrdma_bc_send_request(rdma, rqst); - if (ret == -ENOTCONN) - svc_close_xprt(sxprt); - } + if (test_bit(XPT_DEAD, &sxprt->xpt_flags)) + return -ENOTCONN; - mutex_unlock(&sxprt->xpt_mutex); - - if (ret < 0) - return ret; - return 0; + ret = rpcrdma_bc_send_request(rdma, rqst); + if (ret == -ENOTCONN) + svc_close_xprt(sxprt); + return ret; } static void diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index b6c8643867f2..38e7c3c8c4a9 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -868,12 +868,10 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) __be32 *p; int ret; - /* Create the RDMA response header. xprt->xpt_mutex, - * acquired in svc_send(), serializes RPC replies. The - * code path below that inserts the credit grant value - * into each transport header runs only inside this - * critical section. - */ + ret = -ENOTCONN; + if (svc_xprt_is_dead(xprt)) + goto err0; + ret = -ENOMEM; sctxt = svc_rdma_send_ctxt_get(rdma); if (!sctxt) diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index 845d0be805ec..839c49330785 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -2548,8 +2548,16 @@ static int bc_sendto(struct rpc_rqst *req) return sent; } -/* - * The send routine. Borrows from svc_send +/** + * bc_send_request - Send a backchannel Call on a TCP socket + * @req: rpc_rqst containing Call message to be sent + * + * xpt_mutex ensures @rqstp's whole message is written to the socket + * without interruption. + * + * Return values: + * %0 if the message was sent successfully + * %ENOTCONN if the message was not sent */ static int bc_send_request(struct rpc_rqst *req) { From patchwork Tue May 12 21:22:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544073 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4BBFC139A for ; Tue, 12 May 2020 21:22:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 27F0220659 for ; Tue, 12 May 2020 21:22:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="uenUzDQe" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731048AbgELVWT (ORCPT ); Tue, 12 May 2020 17:22:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1730200AbgELVWT (ORCPT ); Tue, 12 May 2020 17:22:19 -0400 Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA191C061A0C; Tue, 12 May 2020 14:22:17 -0700 (PDT) Received: by mail-qt1-x843.google.com with SMTP id c24so6012280qtw.7; Tue, 12 May 2020 14:22:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=yfG48olU7TS5yzA2N599YfkoG5dmTwMqyAOMPXu150M=; b=uenUzDQe/1auRbiENxah6O0wk2RDIh94ojB99SiaF9ELj0NUnveRRqrN7xHn5Y2zc6 iHJIpIoHCMb7QEOHYJEnz0Jw/x9UIZlPaTZM2AgVHLB0E7V+lvxCe69HJ6hggUP5HI8l 1HGhcl3u0UZ2VaiEfLsqVV6syj3D6t8EvN7ZNLX2T0Atrfo1MwmP0/XFY8Qs9OEIEhwT w1XdGjv3iO1FkL9jP+NGZ7dHzJGIK73gfeYDqi5nb3699kNNTn1PY5MxSfBiuYBiR1q8 8Zv3uRKvLXULF97Bh0c6zEBMJf7aJGXDRBdhA8I35q8I8svxypl7MFwkbHrq3Vy7Yy8r xCFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=yfG48olU7TS5yzA2N599YfkoG5dmTwMqyAOMPXu150M=; b=C3+Ovv+QoWVWCio2Q27E1SFSlPCz+8WlkN58OiwMyV3U1oyGfwY2AiwpeUFm39spjP J/NkLPK9m5YvmVnXDH5G6jw0iZQm4DOIYcxqw6VoPNxtdyol9fOcHR6FioZLBG4e+/Ji 5pz4pKhASzx9aK1chihW91sIuIAtE9dRKh9YNOIN7b5SwmS9m8HbdTcLXAIMo8pgv4cw V3N07nBW+l5YPeU+Pk3aqYnOCX2KIvjRwlyuqg+rHc/cc8bJwQVLETu3Akou/VqFMa4f ATwXOlbPo9bxvYMcOqpe1YNg4mUFJqd1X5mNHTqJSNe6AkD4HcGY9EQAwCUhSWfn6lbL HhIQ== X-Gm-Message-State: AOAM531GWrL39UTyj3sh92LxVRg+mwDkERiig0TDiSr2kQUT4mNLT5Qm Ks2plR3eTbRDpchF6Peeep9WeD4m X-Google-Smtp-Source: ABdhPJzkKfANumAkcXKDJj1xj6WE61BjeF1OrtW+ZcnqhTiaVuej+CAb1Eu3mEfiLaMnLzDn8jZqdw== X-Received: by 2002:ac8:5241:: with SMTP id y1mr9852274qtn.165.1589318536754; Tue, 12 May 2020 14:22:16 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id b129sm10407238qkc.58.2020.05.12.14.22.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:22:16 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLMFc4009878; Tue, 12 May 2020 21:22:15 GMT Subject: [PATCH v2 02/29] svcrdma: Clean up the tracing for rw_ctx_init errors From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:22:15 -0400 Message-ID: <20200512212215.5826.47419.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org - De-duplicate code - Rename the tracepoint with "_err" to allow enabling via glob - Report the sg_cnt for the failing rw_ctx - Fix a dumb signage issue Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 12 +++++--- net/sunrpc/xprtrdma/svc_rdma_rw.c | 56 +++++++++++++++++++++++-------------- 2 files changed, 43 insertions(+), 25 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 132c3c778a43..f231975064cb 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1583,28 +1583,32 @@ DECLARE_EVENT_CLASS(svcrdma_dma_map_class, DEFINE_SVC_DMA_EVENT(dma_map_page); DEFINE_SVC_DMA_EVENT(dma_unmap_page); -TRACE_EVENT(svcrdma_dma_map_rwctx, +TRACE_EVENT(svcrdma_dma_map_rw_err, TP_PROTO( const struct svcxprt_rdma *rdma, + unsigned int nents, int status ), - TP_ARGS(rdma, status), + TP_ARGS(rdma, nents, status), TP_STRUCT__entry( __field(int, status) + __field(unsigned int, nents) __string(device, rdma->sc_cm_id->device->name) __string(addr, rdma->sc_xprt.xpt_remotebuf) ), TP_fast_assign( __entry->status = status; + __entry->nents = nents; __assign_str(device, rdma->sc_cm_id->device->name); __assign_str(addr, rdma->sc_xprt.xpt_remotebuf); ), - TP_printk("addr=%s device=%s status=%d", - __get_str(addr), __get_str(device), __entry->status + TP_printk("addr=%s device=%s nents=%u status=%d", + __get_str(addr), __get_str(device), __entry->nents, + __entry->status ) ); diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 23c2d3ce0dc9..db70709e165a 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -39,7 +39,7 @@ static void svc_rdma_wc_read_done(struct ib_cq *cq, struct ib_wc *wc); struct svc_rdma_rw_ctxt { struct list_head rw_list; struct rdma_rw_ctx rw_ctx; - int rw_nents; + unsigned int rw_nents; struct sg_table rw_sg_table; struct scatterlist rw_first_sgl[]; }; @@ -107,6 +107,34 @@ void svc_rdma_destroy_rw_ctxts(struct svcxprt_rdma *rdma) } } +/** + * svc_rdma_rw_ctx_init - Prepare a R/W context for I/O + * @rdma: controlling transport instance + * @ctxt: R/W context to prepare + * @offset: RDMA offset + * @handle: RDMA tag/handle + * @direction: I/O direction + * + * Returns on success, the number of WQEs that will be needed + * on the workqueue, or a negative errno. + */ +static int svc_rdma_rw_ctx_init(struct svcxprt_rdma *rdma, + struct svc_rdma_rw_ctxt *ctxt, + u64 offset, u32 handle, + enum dma_data_direction direction) +{ + int ret; + + ret = rdma_rw_ctx_init(&ctxt->rw_ctx, rdma->sc_qp, rdma->sc_port_num, + ctxt->rw_sg_table.sgl, ctxt->rw_nents, + 0, offset, handle, direction); + if (unlikely(ret < 0)) { + svc_rdma_put_rw_ctxt(rdma, ctxt); + trace_svcrdma_dma_map_rw_err(rdma, ctxt->rw_nents, ret); + } + return ret; +} + /* A chunk context tracks all I/O for moving one Read or Write * chunk. This is a a set of rdma_rw's that handle data movement * for all segments of one chunk. @@ -431,12 +459,10 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, goto out_noctx; constructor(info, write_len, ctxt); - ret = rdma_rw_ctx_init(&ctxt->rw_ctx, rdma->sc_qp, - rdma->sc_port_num, ctxt->rw_sg_table.sgl, - ctxt->rw_nents, 0, seg_offset, - seg_handle, DMA_TO_DEVICE); + ret = svc_rdma_rw_ctx_init(rdma, ctxt, seg_offset, seg_handle, + DMA_TO_DEVICE); if (ret < 0) - goto out_initerr; + return -EIO; trace_svcrdma_send_wseg(seg_handle, write_len, seg_offset); @@ -462,11 +488,6 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, out_noctx: dprintk("svcrdma: no R/W ctxs available\n"); return -ENOMEM; - -out_initerr: - svc_rdma_put_rw_ctxt(rdma, ctxt); - trace_svcrdma_dma_map_rwctx(rdma, ret); - return -EIO; } /* Send one of an xdr_buf's kvecs by itself. To send a Reply @@ -646,12 +667,10 @@ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info, goto out_overrun; } - ret = rdma_rw_ctx_init(&ctxt->rw_ctx, cc->cc_rdma->sc_qp, - cc->cc_rdma->sc_port_num, - ctxt->rw_sg_table.sgl, ctxt->rw_nents, - 0, offset, rkey, DMA_FROM_DEVICE); + ret = svc_rdma_rw_ctx_init(cc->cc_rdma, ctxt, offset, rkey, + DMA_FROM_DEVICE); if (ret < 0) - goto out_initerr; + return -EIO; list_add(&ctxt->rw_list, &cc->cc_rwctxts); cc->cc_sqecount += ret; @@ -664,11 +683,6 @@ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info, out_overrun: dprintk("svcrdma: request overruns rq_pages\n"); return -EINVAL; - -out_initerr: - trace_svcrdma_dma_map_rwctx(cc->cc_rdma, ret); - svc_rdma_put_rw_ctxt(cc->cc_rdma, ctxt); - return -EIO; } /* Walk the segments in the Read chunk starting at @p and construct From patchwork Tue May 12 21:22:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544077 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1948615E6 for ; Tue, 12 May 2020 21:22:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0218A20659 for ; Tue, 12 May 2020 21:22:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="K6feggtg" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730200AbgELVWX (ORCPT ); Tue, 12 May 2020 17:22:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1728220AbgELVWX (ORCPT ); Tue, 12 May 2020 17:22:23 -0400 Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02511C061A0C; Tue, 12 May 2020 14:22:23 -0700 (PDT) Received: by mail-qt1-x841.google.com with SMTP id p12so12410680qtn.13; Tue, 12 May 2020 14:22:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=cc9RbaP1PbGzDZ6es5NE10b7zglpoGibXUKC8RXcruk=; b=K6feggtgkeIPWE2/KT5FV5O1Cqz25lEddS8v3A8wV9CgR4KlOX2PUYlLrbQCKBw5wj DiosAIJpkKF1uOkHxZHcnygTuSZwtzn/3dQyww3qWWCHzccrBEXol00XpfojmfkjG00e 4t9iberWW1D4wSPbionjja6jVdErmUOPl5IbDTE/e5fi+AFnzaTjVX2tJsqnmXoHpUhN udKLFcCbsCmuqQDIsYiI4SQ0oafvemDUUKCx5hqf8kXY0ZDccUkRZgdia290d9gRQWqj a9p1qXSq/AxmuGqUUyiIpF8NSp9eyktDavtVocZxEPAaAI7Wpky1+xboypmn/9SftPfx bnlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=cc9RbaP1PbGzDZ6es5NE10b7zglpoGibXUKC8RXcruk=; b=lRHn9FY0yiZ99vCqe7yoUmm/hCHUfpP2Vsu85C7ZFBBDceMXxHT+G3JzJtb1xnB50F 4zN95hlYkYHixL8da55oJjr/+44GQfIvHjogwDX8kQNCTIe1vYjRskoaRnUwp4HQ5EQJ Cy+1pb2oM5QLkhrKNBH31b2teTVHXFrSfI1/Gp4Wy60p0Gl16n30GdiMSOChurEOS+ma C1IEAYHtH3UIi7m/K/duM9CLd4IQ0tIIKOWOJJu3Y/uWkSsIrsBV4iSJBqzmL2zyt3Ae oWmj2kBWAjK7LhRFXCXcM8fG0bM4QHYWakGkBKHQMrZr0Dxz/Mop/w39gQiqis1b/3v5 MSGA== X-Gm-Message-State: AGi0PuadXccyA/s+mxElW4bu4d0YAvgwow/GMM0jFJf3H0mA0VZ6cgpw IDPKkxcH0QdbhmVbn3N6xix0G3NB X-Google-Smtp-Source: APiQypK2jnXVdOofv6UsEdoj4FFNYhMrtSBwNvjoNb81Ns4R5Echsdquq2slB1LkyddVYMKDnIF4wA== X-Received: by 2002:ac8:105:: with SMTP id e5mr11983950qtg.15.1589318542053; Tue, 12 May 2020 14:22:22 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id w43sm4922425qtw.68.2020.05.12.14.22.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:22:21 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLMK4c009881; Tue, 12 May 2020 21:22:20 GMT Subject: [PATCH v2 03/29] svcrdma: Clean up handling of get_rw_ctx errors From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:22:20 -0400 Message-ID: <20200512212220.5826.25025.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: Replace two dprintk call sites with a tracepoint. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 25 +++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_rw.c | 27 +++++++++++---------------- 2 files changed, 36 insertions(+), 16 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index f231975064cb..aca9d0f3d769 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1612,6 +1612,31 @@ TRACE_EVENT(svcrdma_dma_map_rw_err, ) ); +TRACE_EVENT(svcrdma_no_rwctx_err, + TP_PROTO( + const struct svcxprt_rdma *rdma, + unsigned int num_sges + ), + + TP_ARGS(rdma, num_sges), + + TP_STRUCT__entry( + __field(unsigned int, num_sges) + __string(device, rdma->sc_cm_id->device->name) + __string(addr, rdma->sc_xprt.xpt_remotebuf) + ), + + TP_fast_assign( + __entry->num_sges = num_sges; + __assign_str(device, rdma->sc_cm_id->device->name); + __assign_str(addr, rdma->sc_xprt.xpt_remotebuf); + ), + + TP_printk("addr=%s device=%s num_sges=%d", + __get_str(addr), __get_str(device), __entry->num_sges + ) +); + TRACE_EVENT(svcrdma_send_pullup, TP_PROTO( unsigned int len diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index db70709e165a..c2d49f607cfe 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -67,19 +67,22 @@ svc_rdma_get_rw_ctxt(struct svcxprt_rdma *rdma, unsigned int sges) ctxt = kmalloc(struct_size(ctxt, rw_first_sgl, SG_CHUNK_SIZE), GFP_KERNEL); if (!ctxt) - goto out; + goto out_noctx; INIT_LIST_HEAD(&ctxt->rw_list); } ctxt->rw_sg_table.sgl = ctxt->rw_first_sgl; if (sg_alloc_table_chained(&ctxt->rw_sg_table, sges, ctxt->rw_sg_table.sgl, - SG_CHUNK_SIZE)) { - kfree(ctxt); - ctxt = NULL; - } -out: + SG_CHUNK_SIZE)) + goto out_free; return ctxt; + +out_free: + kfree(ctxt); +out_noctx: + trace_svcrdma_no_rwctx_err(rdma, sges); + return NULL; } static void svc_rdma_put_rw_ctxt(struct svcxprt_rdma *rdma, @@ -456,7 +459,7 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, ctxt = svc_rdma_get_rw_ctxt(rdma, (write_len >> PAGE_SHIFT) + 2); if (!ctxt) - goto out_noctx; + return -ENOMEM; constructor(info, write_len, ctxt); ret = svc_rdma_rw_ctx_init(rdma, ctxt, seg_offset, seg_handle, @@ -484,10 +487,6 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, dprintk("svcrdma: inadequate space in Write chunk (%u)\n", info->wi_nsegs); return -E2BIG; - -out_noctx: - dprintk("svcrdma: no R/W ctxs available\n"); - return -ENOMEM; } /* Send one of an xdr_buf's kvecs by itself. To send a Reply @@ -637,7 +636,7 @@ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info, sge_no = PAGE_ALIGN(info->ri_pageoff + len) >> PAGE_SHIFT; ctxt = svc_rdma_get_rw_ctxt(cc->cc_rdma, sge_no); if (!ctxt) - goto out_noctx; + return -ENOMEM; ctxt->rw_nents = sge_no; sg = ctxt->rw_sg_table.sgl; @@ -676,10 +675,6 @@ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info, cc->cc_sqecount += ret; return 0; -out_noctx: - dprintk("svcrdma: no R/W ctxs available\n"); - return -ENOMEM; - out_overrun: dprintk("svcrdma: request overruns rq_pages\n"); return -EINVAL; From patchwork Tue May 12 21:22:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544081 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D091A139A for ; Tue, 12 May 2020 21:22:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B64CE205C9 for ; Tue, 12 May 2020 21:22:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nOZYK+I4" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731156AbgELVW3 (ORCPT ); Tue, 12 May 2020 17:22:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46020 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1728220AbgELVW3 (ORCPT ); Tue, 12 May 2020 17:22:29 -0400 Received: from mail-qt1-x82a.google.com (mail-qt1-x82a.google.com [IPv6:2607:f8b0:4864:20::82a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C0ACCC061A0C; Tue, 12 May 2020 14:22:28 -0700 (PDT) Received: by mail-qt1-x82a.google.com with SMTP id c24so6012670qtw.7; Tue, 12 May 2020 14:22:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=SK3QJDZhRtOH7x7kVsqSB1LJX7VwQFBUigHKiDxq+fw=; b=nOZYK+I4NrtI1BR6QI1o3anunsbskNf4PBFuRvBJbyKAriC80fy2WWrvtGD8TaCUXB PYh/6rVYL1rzaOixMrKA9XY2FZibbS3LsqjUG5A86XU88bHr87Qai3oPB+mh3/Ina9c4 wxv9+2Ogj1ZLd1foNdm1r4vG7Ti/u+RP4mz8rdrYT5avta3FULI/Li7rLLE2ZyBVc9zj DtvkxVfNaN4/ae+fXuwZ6tABqAn0NFM5MNF25eKeeTKD3f5Pdvhuxv9wuYjQAIBoKVJB S+1ej4jC43/BWDE0AMU/EzKVFXL/6I1I0D5xNguBX48euCtRs9EQc6bOJoUYUqwf+PPU Hrrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=SK3QJDZhRtOH7x7kVsqSB1LJX7VwQFBUigHKiDxq+fw=; b=UHpe2oqosfsC/tlCBk1YiyBZXTQwbR/AKjWfU/DqrPYRb+IQlJeBBpmPEdfGhKC+me leO2up7xwNbD4yak/9uEtUYHCfPghq6YQ9O6DsJSsURdWdF5tKuc+DGPkJ8YwhIoKlC4 RzvSs4m1EM7VAsjI0+zVeEHbFWdx1yJ2Mj5lQLDdKIApBSwOj+6ZG0fWBcMIrR/pxiqa /nLj67tzcgIBJISvQQZOZW1VGkgPNatCCL+oFAz+llOqW+8AHkoaf5HrWRCV6n1TBQO8 kXBIW3qoLLBgQNEpjO1+k19QVGQaWYeQIg70imkBQHbpCSpI/oXCNuU86hVeTpQyYMLz o/sQ== X-Gm-Message-State: AGi0PuahvJNk7x/5byRVIw3coygwh37uJB8sNMdE3brjhiFwHjp7Mpip Sjsve9G8yxzSXbuUzrEoVyFRicr1 X-Google-Smtp-Source: APiQypJ1PNDNEUb89RPfrUndFk3/m3FbgvzkTjOidoVhWmeoS2U5wktXWNMMsKexPiseZz4iNvSW3w== X-Received: by 2002:ac8:699a:: with SMTP id o26mr23258177qtq.92.1589318547857; Tue, 12 May 2020 14:22:27 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id j83sm11610207qke.7.2020.05.12.14.22.26 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:22:26 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLMPTY009885; Tue, 12 May 2020 21:22:25 GMT Subject: [PATCH v2 04/29] svcrdma: Trace page overruns when constructing RDMA Reads From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:22:25 -0400 Message-ID: <20200512212225.5826.17442.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: Replace a dprintk call site with a tracepoint. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 28 ++++++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_rw.c | 2 +- 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index aca9d0f3d769..d6da6b8d521d 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1637,6 +1637,34 @@ TRACE_EVENT(svcrdma_no_rwctx_err, ) ); +TRACE_EVENT(svcrdma_page_overrun_err, + TP_PROTO( + const struct svcxprt_rdma *rdma, + const struct svc_rqst *rqst, + unsigned int pageno + ), + + TP_ARGS(rdma, rqst, pageno), + + TP_STRUCT__entry( + __field(unsigned int, pageno) + __field(u32, xid) + __string(device, rdma->sc_cm_id->device->name) + __string(addr, rdma->sc_xprt.xpt_remotebuf) + ), + + TP_fast_assign( + __entry->pageno = pageno; + __entry->xid = __be32_to_cpu(rqst->rq_xid); + __assign_str(device, rdma->sc_cm_id->device->name); + __assign_str(addr, rdma->sc_xprt.xpt_remotebuf); + ), + + TP_printk("addr=%s device=%s xid=0x%08x pageno=%u", __get_str(addr), + __get_str(device), __entry->xid, __entry->pageno + ) +); + TRACE_EVENT(svcrdma_send_pullup, TP_PROTO( unsigned int len diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index c2d49f607cfe..17098a11d2ad 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -676,7 +676,7 @@ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info, return 0; out_overrun: - dprintk("svcrdma: request overruns rq_pages\n"); + trace_svcrdma_page_overrun_err(cc->cc_rdma, rqstp, info->ri_pageno); return -EINVAL; } From patchwork Tue May 12 21:22:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544085 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 81B7815E6 for ; Tue, 12 May 2020 21:22:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6B86720659 for ; Tue, 12 May 2020 21:22:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="enuIa/CA" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731237AbgELVWf (ORCPT ); Tue, 12 May 2020 17:22:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46038 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1728220AbgELVWe (ORCPT ); Tue, 12 May 2020 17:22:34 -0400 Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A34DC061A0C; Tue, 12 May 2020 14:22:34 -0700 (PDT) Received: by mail-qk1-x742.google.com with SMTP id i14so14127830qka.10; Tue, 12 May 2020 14:22:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=TV42Sou8aMygUuJA03fyTdblg9o3Ei8GWJ1p5HoMi9k=; b=enuIa/CAxDLGyEpBWEgOIVDJdEAUWX4V+0vJorOkAn/t2md+bXZ1CxQ7cP6jDhixbY 4KiA4zvK3+XQX2NtcVAxEJu+CB4CIQ0oCDMwa6bwGlG7bCQSqW/8ed/sT10gVDNOxdKP XbeRGp7yFRwaLGNUC4E3V1sy3l0DLG9pS8XE2j4QkGEuXb5G5+7VZl15BDL/LJ44VNVE zu7D+7HEtRja7reIZ/12ZHi2CXORXEgJSj2XjVNtKH4at7jDjmjvG+i8ZGH6fQCHb86B Rrp/Z2gXx9nmPUFVgoWDU6EmwBZvEnL1IMPAv1QuOkLikeAL6Yo8B4093TD5ZJPKtE8a nd5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=TV42Sou8aMygUuJA03fyTdblg9o3Ei8GWJ1p5HoMi9k=; b=S+qroTk+GMIutd7YwocEOcVtEofVXENciGu6ABF+aOfcbDZxlCOt7xW6v3yChuWrIt DWbVb+HZqqMUdNhHg6A5rTO/Iz16qYnPiJZmfT4MaNkNeEMtQNVLvNE2OiCyyGUGuQnp VAzJIt+EhtObgFhJ/HyqKsObc1spHZaHkcLP6IXaPPYrafJT1hv6FrGLZAOy0xjo9IiI EHZKW6ctlO7YsEDi2rAvtE/AvTmubK8qSRkruM6V7gJ0oQTu1TFOS+8oxYPTsB9Tsi5J Pq32AERdh8Xe8Jyk3KNt8SJjuH5FfCEfHWu4keB8l1qRxbwwt2/Ml7jNKiih6Ys+k2DR 6ZPA== X-Gm-Message-State: AGi0PubviATXVH3lBe5yuEAh1BCjlZpR3VuyFo6hxk1EA24Q2Pr89Am7 x3ZFvWBADCk86Tr/B4wxJwPzz6ja X-Google-Smtp-Source: APiQypL/K8rWQ2yd1hLYJMU2l3ZJs94bTyXFWtra+z3X5xnMfkeYRzJDnIpZjRvStNEi90bP4XVGoQ== X-Received: by 2002:a37:a0d5:: with SMTP id j204mr23179612qke.128.1589318553142; Tue, 12 May 2020 14:22:33 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id h3sm12329782qkf.15.2020.05.12.14.22.32 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:22:32 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLMVxY009888; Tue, 12 May 2020 21:22:31 GMT Subject: [PATCH v2 05/29] svcrdma: trace undersized Write chunks From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:22:31 -0400 Message-ID: <20200512212231.5826.45494.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: Replace a dprintk call site. This is the last remaining dprintk call site in svc_rdma_rw.c, so remove dprintk infrastructure as well. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 32 ++++++++++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_rw.c | 7 ++----- 2 files changed, 34 insertions(+), 5 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index d6da6b8d521d..c046b198072a 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1665,6 +1665,38 @@ TRACE_EVENT(svcrdma_page_overrun_err, ) ); +TRACE_EVENT(svcrdma_small_wrch_err, + TP_PROTO( + const struct svcxprt_rdma *rdma, + unsigned int remaining, + unsigned int seg_no, + unsigned int num_segs + ), + + TP_ARGS(rdma, remaining, seg_no, num_segs), + + TP_STRUCT__entry( + __field(unsigned int, remaining) + __field(unsigned int, seg_no) + __field(unsigned int, num_segs) + __string(device, rdma->sc_cm_id->device->name) + __string(addr, rdma->sc_xprt.xpt_remotebuf) + ), + + TP_fast_assign( + __entry->remaining = remaining; + __entry->seg_no = seg_no; + __entry->num_segs = num_segs; + __assign_str(device, rdma->sc_cm_id->device->name); + __assign_str(addr, rdma->sc_xprt.xpt_remotebuf); + ), + + TP_printk("addr=%s device=%s remaining=%u seg_no=%u num_segs=%u", + __get_str(addr), __get_str(device), __entry->remaining, + __entry->seg_no, __entry->num_segs + ) +); + TRACE_EVENT(svcrdma_send_pullup, TP_PROTO( unsigned int len diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 17098a11d2ad..5eb35309ecef 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -9,13 +9,10 @@ #include #include -#include #include "xprt_rdma.h" #include -#define RPCDBG_FACILITY RPCDBG_SVCXPRT - static void svc_rdma_write_done(struct ib_cq *cq, struct ib_wc *wc); static void svc_rdma_wc_read_done(struct ib_cq *cq, struct ib_wc *wc); @@ -484,8 +481,8 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, return 0; out_overflow: - dprintk("svcrdma: inadequate space in Write chunk (%u)\n", - info->wi_nsegs); + trace_svcrdma_small_wrch_err(rdma, remaining, info->wi_seg_no, + info->wi_nsegs); return -E2BIG; } From patchwork Tue May 12 21:22:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544089 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F01D0139A for ; Tue, 12 May 2020 21:22:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D422420769 for ; Tue, 12 May 2020 21:22:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iPprTsnF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731250AbgELVWj (ORCPT ); Tue, 12 May 2020 17:22:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46050 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1728220AbgELVWj (ORCPT ); Tue, 12 May 2020 17:22:39 -0400 Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3B9EC061A0C; Tue, 12 May 2020 14:22:38 -0700 (PDT) Received: by mail-qt1-x841.google.com with SMTP id h26so12437012qtu.8; Tue, 12 May 2020 14:22:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=lCCExvyODMO0v1Z3g8fcgjDrSeXwtuhz60/Y3Fx4pvQ=; b=iPprTsnF9sbTDlAzZPE0YHJf+19f+qbmllWolcWFA1IYd2vYuNJB3UmzwX4BXWF8t+ n9IiNa/u5bzNhYw33D80J8crw5pdjyQ6SyM1Y407ddoTatyi0XaQv5+wUTfuO1MVozS9 nNA85AU3HfURS7XvWAKd0P5K7C8aM/xogaMmFqQzTBNknGUMSnqKt+7Zx1nftSympLhk 51szN9LqFaiJ9syTa/1EbQ4p7ieybMj6zp8FoB5Gj2I44f0oJhcPuNswuBi042DoXKfs tZ9h2XCIZ46CCkZpJZ/ICmB77RMzah8FIWgWO/SpIDZry3Nzua4VMl+/aqCrhgEJlB6o gEzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=lCCExvyODMO0v1Z3g8fcgjDrSeXwtuhz60/Y3Fx4pvQ=; b=nEWpmi/6gZkOv4SEADq0qFmCgyq00urJQYC6yJGZ2PM6MfH6iohl3nDJzmn5tHQDLt IerAm8S+KrIXMmeCQzgOQE/OOhoYGj1SMl6nCb74zqlSk47TscDt9W8ly3VaogePBiEj 9MlNZPOykX7ohJRLRpC5mgBpXdCzR4ldX3FfsD3m3S4OqjVe/69RbF8xXCp/vqgmp7Oz 75Te/7qbHWOR9hTSIMH/PZsKWqaMxCEecDJBYjvLHNq5+wn6yYFeQf3I8VyIir4NlI/s kqgSBEMP7Wr0zgB2t0w3XP/jr4bvgc5OctXZ7eVH60Brp8k0a4dM0W9UiTz9hmJwAOhT XoYw== X-Gm-Message-State: AGi0PubZxM8DQhRsTiPOzzYmZdJ59Pnq9sz3oiLBriEYTIAPfHTC5rsh 7s9QgLjLkMF/vSDhJ9/jM5nj/8dg X-Google-Smtp-Source: APiQypLZx+mC7OqbYGTulD4S88AVXUNDVqKBe7fdmM2z6ZJLJr0jbgC3E6sRMyvWoNQKTjGi6x5aCA== X-Received: by 2002:ac8:7942:: with SMTP id r2mr24816671qtt.288.1589318557918; Tue, 12 May 2020 14:22:37 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id l24sm12968117qtp.8.2020.05.12.14.22.37 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:22:37 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLMaY7009892; Tue, 12 May 2020 21:22:36 GMT Subject: [PATCH v2 06/29] svcrdma: Fix backchannel return code From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:22:36 -0400 Message-ID: <20200512212236.5826.94193.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Way back when I was writing the RPC/RDMA server-side backchannel code, I misread the TCP backchannel reply handler logic. When svc_tcp_recvfrom() successfully receives a backchannel reply, it does not return -EAGAIN. It sets XPT_DATA and returns zero. Update svc_rdma_recvfrom() to return zero. Here, XPT_DATA doesn't need to be set again: it is set whenever a new message is received, behind a spin lock in a single threaded context. Also, if handling the cb reply is not successful, the message is simply dropped. There's no special message framing to deal with as there is in the TCP case. Now that the handle_bc_reply() return value is ignored, I've removed the dprintk call sites in the error exit of handle_bc_reply() in favor of trace points in other areas that already report the error cases. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 5 +--- net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 38 +++++++--------------------- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 11 ++++---- 3 files changed, 17 insertions(+), 37 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index cbcfbd0521e3..8518c3f37e56 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -160,9 +160,8 @@ struct svc_rdma_send_ctxt { }; /* svc_rdma_backchannel.c */ -extern int svc_rdma_handle_bc_reply(struct rpc_xprt *xprt, - __be32 *rdma_resp, - struct xdr_buf *rcvbuf); +extern void svc_rdma_handle_bc_reply(struct svc_rqst *rqstp, + struct svc_rdma_recv_ctxt *rctxt); /* svc_rdma_recvfrom.c */ extern void svc_rdma_recv_ctxts_destroy(struct svcxprt_rdma *rdma); diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index d9aab4504f2c..b174f3b109a5 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -15,26 +15,25 @@ #undef SVCRDMA_BACKCHANNEL_DEBUG /** - * svc_rdma_handle_bc_reply - Process incoming backchannel reply - * @xprt: controlling backchannel transport - * @rdma_resp: pointer to incoming transport header - * @rcvbuf: XDR buffer into which to decode the reply + * svc_rdma_handle_bc_reply - Process incoming backchannel Reply + * @rqstp: resources for handling the Reply + * @rctxt: Received message * - * Returns: - * %0 if @rcvbuf is filled in, xprt_complete_rqst called, - * %-EAGAIN if server should call ->recvfrom again. */ -int svc_rdma_handle_bc_reply(struct rpc_xprt *xprt, __be32 *rdma_resp, - struct xdr_buf *rcvbuf) +void svc_rdma_handle_bc_reply(struct svc_rqst *rqstp, + struct svc_rdma_recv_ctxt *rctxt) { + struct svc_xprt *sxprt = rqstp->rq_xprt; + struct rpc_xprt *xprt = sxprt->xpt_bc_xprt; struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(xprt); + struct xdr_buf *rcvbuf = &rqstp->rq_arg; struct kvec *dst, *src = &rcvbuf->head[0]; + __be32 *rdma_resp = rctxt->rc_recv_buf; struct rpc_rqst *req; u32 credits; size_t len; __be32 xid; __be32 *p; - int ret; p = (__be32 *)src->iov_base; len = src->iov_len; @@ -49,14 +48,10 @@ int svc_rdma_handle_bc_reply(struct rpc_xprt *xprt, __be32 *rdma_resp, __func__, (int)len, p); #endif - ret = -EAGAIN; - if (src->iov_len < 24) - goto out_shortreply; - spin_lock(&xprt->queue_lock); req = xprt_lookup_rqst(xprt, xid); if (!req) - goto out_notfound; + goto out_unlock; dst = &req->rq_private_buf.head[0]; memcpy(&req->rq_private_buf, &req->rq_rcv_buf, sizeof(struct xdr_buf)); @@ -77,25 +72,12 @@ int svc_rdma_handle_bc_reply(struct rpc_xprt *xprt, __be32 *rdma_resp, spin_unlock(&xprt->transport_lock); spin_lock(&xprt->queue_lock); - ret = 0; xprt_complete_rqst(req->rq_task, rcvbuf->len); xprt_unpin_rqst(req); rcvbuf->len = 0; out_unlock: spin_unlock(&xprt->queue_lock); -out: - return ret; - -out_shortreply: - dprintk("svcrdma: short bc reply: xprt=%p, len=%zu\n", - xprt, src->iov_len); - goto out; - -out_notfound: - dprintk("svcrdma: unrecognized bc reply: xprt=%p, xid=%08x\n", - xprt, be32_to_cpu(xid)); - goto out_unlock; } /* Send a backwards direction RPC call. diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index efa5fcb5793f..eee7c6478b30 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -878,12 +878,9 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) goto out_drop; rqstp->rq_xprt_hlen = ret; - if (svc_rdma_is_backchannel_reply(xprt, p)) { - ret = svc_rdma_handle_bc_reply(xprt->xpt_bc_xprt, p, - &rqstp->rq_arg); - svc_rdma_recv_ctxt_put(rdma_xprt, ctxt); - return ret; - } + if (svc_rdma_is_backchannel_reply(xprt, p)) + goto out_backchannel; + svc_rdma_get_inv_rkey(rdma_xprt, ctxt); p += rpcrdma_fixed_maxsz; @@ -913,6 +910,8 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) svc_rdma_recv_ctxt_put(rdma_xprt, ctxt); return ret; +out_backchannel: + svc_rdma_handle_bc_reply(rqstp, ctxt); out_drop: svc_rdma_recv_ctxt_put(rdma_xprt, ctxt); return 0; From patchwork Tue May 12 21:22:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544093 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4C3B3139A for ; Tue, 12 May 2020 21:22:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 323B820659 for ; Tue, 12 May 2020 21:22:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RLjiezBf" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731269AbgELVWq (ORCPT ); Tue, 12 May 2020 17:22:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46066 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1728220AbgELVWq (ORCPT ); Tue, 12 May 2020 17:22:46 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ECB7FC061A0C; Tue, 12 May 2020 14:22:44 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id f83so15230998qke.13; Tue, 12 May 2020 14:22:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=RdAX8caYJ0iXy3SCEZHENL/3iYUYiev43YHvBxvtV8I=; b=RLjiezBfVXZPxNiHqaJ/tW4QQ3tLqsZdtPW3VZi3H3VZuRtqJlcJOkHCKA+3h/CRA6 sFrJ13NzFmSrNj0uk92vcteQFXMMqPpwM8X+Jcdm8EdxGSZA1NbYZP37uiwxfIAMjuXD +2uehUwyjPoNoWcAQu0gwl39ppHiWSqGU0RICJvU1m2G58BYw06bYdPZkOmV7TZBBmse +zeUOjVA9TmXS4bIX9hGkWRiSTQ5IGy9X2y8bFLQ0kldjEBzwdTnS3yOdX5llgXJ52IS czGsleUUQK/lQnsoZ+t5m3MHphpOKzVDtukeVYYEXZQ4DaWHBt/q0GZD2RLFufOgic2z Y2Qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=RdAX8caYJ0iXy3SCEZHENL/3iYUYiev43YHvBxvtV8I=; b=MyvDug5A8OxlY4PotJk55x6L/T/xhHxGEQ54Ba2j8ssa/oiCnZdWoS50Fvz76Ymr1u QrTgHR9Yp1bkL4SbjR8rN8ae2hi5V7NVYbqU+ieO6hXJI79LfitbLBhiLJvLwD1t16pR YmdJ/YxfvnNTWGoCPEKEudrziCi0xfNfShqi4xx+BG3TWSH3fRad+5t/2g73e74Gl8Ii W0BiSF/pfsUF/EwRWM3OFYVeTyce+pQKeboF9uHtZVAQUconRZ6HI6zm7IJd6IJQTSC8 nIA9dm16l1YDzHi1is5cFMLxnxWtPv87tHnjVoM9D+nLLppdO6GK+2KuJG2CXGum+Z80 zvlQ== X-Gm-Message-State: AGi0PuZHF60kPlMRqauyso/h0s8bECWowGi/nbEP6CZMjTtiX26zyzEd ZkMrEFplL3KdNdlBhgAA3N8glco8 X-Google-Smtp-Source: APiQypIFu6Tcw40LibUT81AOfYOPwzjViyc8W0cFBWmESMOu2E/1oPIiTFyKxRFNGU6et2mNZoPwgQ== X-Received: by 2002:ae9:e886:: with SMTP id a128mr18591562qkg.204.1589318563163; Tue, 12 May 2020 14:22:43 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id x5sm14254163qtx.35.2020.05.12.14.22.42 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:22:42 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLMfuw009895; Tue, 12 May 2020 21:22:41 GMT Subject: [PATCH v2 07/29] svcrdma: Remove backchannel dprintk call sites From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:22:41 -0400 Message-ID: <20200512212241.5826.53572.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 48 +++------------------------- 1 file changed, 5 insertions(+), 43 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index b174f3b109a5..1ee73f7cf931 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -10,10 +10,6 @@ #include "xprt_rdma.h" #include -#define RPCDBG_FACILITY RPCDBG_SVCXPRT - -#undef SVCRDMA_BACKCHANNEL_DEBUG - /** * svc_rdma_handle_bc_reply - Process incoming backchannel Reply * @rqstp: resources for handling the Reply @@ -31,33 +27,17 @@ void svc_rdma_handle_bc_reply(struct svc_rqst *rqstp, __be32 *rdma_resp = rctxt->rc_recv_buf; struct rpc_rqst *req; u32 credits; - size_t len; - __be32 xid; - __be32 *p; - - p = (__be32 *)src->iov_base; - len = src->iov_len; - xid = *rdma_resp; - -#ifdef SVCRDMA_BACKCHANNEL_DEBUG - pr_info("%s: xid=%08x, length=%zu\n", - __func__, be32_to_cpu(xid), len); - pr_info("%s: RPC/RDMA: %*ph\n", - __func__, (int)RPCRDMA_HDRLEN_MIN, rdma_resp); - pr_info("%s: RPC: %*ph\n", - __func__, (int)len, p); -#endif spin_lock(&xprt->queue_lock); - req = xprt_lookup_rqst(xprt, xid); + req = xprt_lookup_rqst(xprt, *rdma_resp); if (!req) goto out_unlock; dst = &req->rq_private_buf.head[0]; memcpy(&req->rq_private_buf, &req->rq_rcv_buf, sizeof(struct xdr_buf)); - if (dst->iov_len < len) + if (dst->iov_len < src->iov_len) goto out_unlock; - memcpy(dst->iov_base, p, len); + memcpy(dst->iov_base, src->iov_base, src->iov_len); xprt_pin_rqst(req); spin_unlock(&xprt->queue_lock); @@ -66,7 +46,6 @@ void svc_rdma_handle_bc_reply(struct svc_rqst *rqstp, credits = 1; /* don't deadlock */ else if (credits > r_xprt->rx_buf.rb_bc_max_requests) credits = r_xprt->rx_buf.rb_bc_max_requests; - spin_lock(&xprt->transport_lock); xprt->cwnd = credits << RPC_CWNDSHIFT; spin_unlock(&xprt->transport_lock); @@ -174,10 +153,6 @@ rpcrdma_bc_send_request(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst) *p++ = xdr_zero; *p = xdr_zero; -#ifdef SVCRDMA_BACKCHANNEL_DEBUG - pr_info("%s: %*ph\n", __func__, 64, rqst->rq_buffer); -#endif - rqst->rq_xtime = ktime_get(); rc = svc_rdma_bc_sendto(rdma, rqst, ctxt); if (rc) @@ -188,7 +163,6 @@ rpcrdma_bc_send_request(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst) svc_rdma_send_ctxt_put(rdma, ctxt); drop_connection: - dprintk("svcrdma: failed to send bc call\n"); return -ENOTCONN; } @@ -207,9 +181,6 @@ static int xprt_rdma_bc_send_request(struct rpc_rqst *rqst) container_of(sxprt, struct svcxprt_rdma, sc_xprt); int ret; - dprintk("svcrdma: sending bc call with xid: %08x\n", - be32_to_cpu(rqst->rq_xid)); - if (test_bit(XPT_DEAD, &sxprt->xpt_flags)) return -ENOTCONN; @@ -222,8 +193,6 @@ static int xprt_rdma_bc_send_request(struct rpc_rqst *rqst) static void xprt_rdma_bc_close(struct rpc_xprt *xprt) { - dprintk("svcrdma: %s: xprt %p\n", __func__, xprt); - xprt_disconnect_done(xprt); xprt->cwnd = RPC_CWNDSHIFT; } @@ -231,8 +200,6 @@ xprt_rdma_bc_close(struct rpc_xprt *xprt) static void xprt_rdma_bc_put(struct rpc_xprt *xprt) { - dprintk("svcrdma: %s: xprt %p\n", __func__, xprt); - xprt_rdma_free_addresses(xprt); xprt_free(xprt); } @@ -267,19 +234,14 @@ xprt_setup_rdma_bc(struct xprt_create *args) struct rpc_xprt *xprt; struct rpcrdma_xprt *new_xprt; - if (args->addrlen > sizeof(xprt->addr)) { - dprintk("RPC: %s: address too large\n", __func__); + if (args->addrlen > sizeof(xprt->addr)) return ERR_PTR(-EBADF); - } xprt = xprt_alloc(args->net, sizeof(*new_xprt), RPCRDMA_MAX_BC_REQUESTS, RPCRDMA_MAX_BC_REQUESTS); - if (!xprt) { - dprintk("RPC: %s: couldn't allocate rpc_xprt\n", - __func__); + if (!xprt) return ERR_PTR(-ENOMEM); - } xprt->timeout = &xprt_rdma_bc_timeout; xprt_set_bound(xprt); From patchwork Tue May 12 21:22:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544097 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C5D9A15E6 for ; Tue, 12 May 2020 21:22:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AE56F20659 for ; Tue, 12 May 2020 21:22:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="oQL1vrBm" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731271AbgELVWv (ORCPT ); Tue, 12 May 2020 17:22:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731270AbgELVWu (ORCPT ); Tue, 12 May 2020 17:22:50 -0400 Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C995C061A0C; Tue, 12 May 2020 14:22:49 -0700 (PDT) Received: by mail-qk1-x742.google.com with SMTP id f13so14669802qkh.2; Tue, 12 May 2020 14:22:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=ETA1f6UFmjBQn0duK0wp3dkMLZQ+ZPmkQmCOTXOeD+c=; b=oQL1vrBmHwazTeMCsCOGOPwvlF5CXJmHGpwcrq2+8cTjISrtJeSO8DrKaHiZSoLC0S KBCMWPTb+NCP0Wn9eobqhW8RxUI+KwcP8bIPTHu5Cb2OgSwaWSudxLvQ4lMy5fYVwZdL ZMX962DD/9OMCdtJL4DcnxTLOzHr7GnGo6GKo+DU1RDYwXtEkTGyndyLF0QuaEVpqtI5 APuacGHcLOxmS6MUV35T5UU4WjFAm8Xzx71I+6jyxdm20j3sxlNYPcN0IZ3UrY/gTtd4 Ad3a8nzP+2UgCXoGG12P4lNnB0ja0+TsPgdRoe3aFPo7vXZ4a/D528d15nWD9rhUp6nX 0slg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=ETA1f6UFmjBQn0duK0wp3dkMLZQ+ZPmkQmCOTXOeD+c=; b=C/sTliLhFMh66mLeppLFmMbm+phPntRHTgtuWq25WQlt5int6WIA5bgq55YHcdm4EK EJXp0pmzQO42TVFN5KxU8H0Hec8PNUjAGS5DLWk5UsmXmimWzXUP6mn+fHriEzwufm3B wvgB4w+ky1qM75OVD0+ckuXtINb1q/XrEULDkr7tUlEPPuxwBcQ/E7uQ8dfWXLiIU6DN PcKdF4XattbwM3uQFrYRjEl89mlOg1/VFVyQqTw10RLMxvWNfzJCqGSzHeGSrsj0UFwF 0r6kMoi9g9uz2GRW6tGXxNch3Z9DIJFTGgixWBmoe7W9y1ZFC4QzGNg7sFXrO/MyhQ+1 LrYg== X-Gm-Message-State: AGi0PuYQqwCQaUfEr+WUErdc1P7Mk6AN/uv4PfdGf4Lw36BhXw4KhNnx U8kUXdBP9PfWCLTBu2g+J9rWJL2l X-Google-Smtp-Source: APiQypI8+lOgbwZm4zSRMvOTfPABkzuXYTWeM9ewF2vv/ceIY3ALSSplf0cHJWDcv1IIifa/2f2jDQ== X-Received: by 2002:a37:c96:: with SMTP id 144mr21363791qkm.138.1589318568349; Tue, 12 May 2020 14:22:48 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id u56sm5257206qtb.91.2020.05.12.14.22.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:22:47 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLMl2n009898; Tue, 12 May 2020 21:22:47 GMT Subject: [PATCH v2 08/29] svcrdma: Rename tracepoints that record header decoding errors From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:22:47 -0400 Message-ID: <20200512212247.5826.42055.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: Use a consistent naming convention so that these trace points can be enabled quickly via a glob. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 5 +++-- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 10 +++++----- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index c046b198072a..53b24c8c7860 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1355,7 +1355,7 @@ TRACE_EVENT(svcrdma_decode_rqst, show_rpcrdma_proc(__entry->proc), __entry->hdrlen) ); -TRACE_EVENT(svcrdma_decode_short, +TRACE_EVENT(svcrdma_decode_short_err, TP_PROTO( unsigned int hdrlen ), @@ -1399,7 +1399,8 @@ DECLARE_EVENT_CLASS(svcrdma_badreq_event, ); #define DEFINE_BADREQ_EVENT(name) \ - DEFINE_EVENT(svcrdma_badreq_event, svcrdma_decode_##name,\ + DEFINE_EVENT(svcrdma_badreq_event, \ + svcrdma_decode_##name##_err, \ TP_PROTO( \ __be32 *p \ ), \ diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index eee7c6478b30..e426fedb9524 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -665,23 +665,23 @@ static int svc_rdma_xdr_decode_req(struct xdr_buf *rq_arg, return hdr_len; out_short: - trace_svcrdma_decode_short(rq_arg->len); + trace_svcrdma_decode_short_err(rq_arg->len); return -EINVAL; out_version: - trace_svcrdma_decode_badvers(rdma_argp); + trace_svcrdma_decode_badvers_err(rdma_argp); return -EPROTONOSUPPORT; out_drop: - trace_svcrdma_decode_drop(rdma_argp); + trace_svcrdma_decode_drop_err(rdma_argp); return 0; out_proc: - trace_svcrdma_decode_badproc(rdma_argp); + trace_svcrdma_decode_badproc_err(rdma_argp); return -EINVAL; out_inval: - trace_svcrdma_decode_parse(rdma_argp); + trace_svcrdma_decode_parse_err(rdma_argp); return -EINVAL; } From patchwork Tue May 12 21:22:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544101 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 251F2139A for ; Tue, 12 May 2020 21:22:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0CBBC205C9 for ; Tue, 12 May 2020 21:22:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="VfQiSHcC" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731281AbgELVWz (ORCPT ); Tue, 12 May 2020 17:22:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46096 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731270AbgELVWz (ORCPT ); Tue, 12 May 2020 17:22:55 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 08618C061A0C; Tue, 12 May 2020 14:22:55 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id y22so2627185qki.3; Tue, 12 May 2020 14:22:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=WhwomO42XDAEJPW2/cr+MUt5RrB/mfN0Bet2spmrCJc=; b=VfQiSHcCV1VsVyv8uNIgf9kLYQGVQlvA1DkXDXOUj0rl4EGXOYiu24m9bRKc6d9f16 C/iqqIqg1/3nmSeaiU3opAJyMoIQNqYjAV+AVzoV+qkcQ3rdnDs2fGVI1/ukJ48DGmlX hvzlQRoMP9nq9ar4ljo8ErmRZUcPQEjb6AG0fO7rNZgcLViTSgwaBueTa/2EkNm2eyF+ FMWmlSzEMkparbHwEMmKCtU5LfzK+MUkmHuy2YyTI1pY2Ec63mPfYWdCG0UkW/Yvf4P2 O/dZj8FfC2mEumMAfiJccyOAUMM6FnD/13iCPTcZt0kW8qgiRLJHusBottUDxsNGNN+c vrsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=WhwomO42XDAEJPW2/cr+MUt5RrB/mfN0Bet2spmrCJc=; b=fRkh6rl+8lfUVd6dlmQbzWqOUzGDN4K78/rcfcE7xX+ozRiRC72wqvPF8w18tQcxQj jl+S7w+AMIt/TPK1O+t4uHQ9A1Ky0EQeflIRTguTGBTtaV2M+koyMe+bhg/1FL81DN31 0WRKiEa13EthExxm/xd2yrSHQHtS0l1I+mXr0KTZPZRX7wGKTkbCCjvD4h+3IzJEQgXT ZSkMvdljtvRkkzNjTLUbjwTzkpyhXUol9xIbdUATnzuDxfoA36M84KjFNxftyG25Ikes aOfZ0iUg6Gr/uUMcgoy4xVn8Cxg02KB03v18KtlwO5XOoboOuuUIbvK+XZIV3wEPkV73 h5tg== X-Gm-Message-State: AGi0PuauzhjMyLeurUotqr/kGYXqBKsAjQhLZS/veOF8nALk73BYo6Mx AI2O7lpt8bYxhF/QdUIXN/j3+Kop X-Google-Smtp-Source: APiQypLqHDqJwzbgIgamZ05xjhcVawPKDfZSdyK/JSy+EAPuOgrHmUbAJTslTXiG//fUTJDY9k9Xig== X-Received: by 2002:a37:5bc4:: with SMTP id p187mr9646830qkb.97.1589318573688; Tue, 12 May 2020 14:22:53 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id l8sm13738519qtl.86.2020.05.12.14.22.53 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:22:53 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLMq3d009901; Tue, 12 May 2020 21:22:52 GMT Subject: [PATCH v2 09/29] svcrdma: Remove the SVCRDMA_DEBUG macro From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:22:52 -0400 Message-ID: <20200512212252.5826.81631.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: Commit d21b05f101ae ("rdma: SVCRMDA Header File") introduced the SVCRDMA_DEBUG macro, but it doesn't seem to have been used. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 1 - 1 file changed, 1 deletion(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 8518c3f37e56..7ed82625dc0b 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -48,7 +48,6 @@ #include #include #include -#define SVCRDMA_DEBUG /* Default and maximum inline threshold sizes */ enum { From patchwork Tue May 12 21:22:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544105 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AF52515E6 for ; Tue, 12 May 2020 21:23:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9752E20659 for ; Tue, 12 May 2020 21:23:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IiHVmuBP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725938AbgELVXC (ORCPT ); Tue, 12 May 2020 17:23:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46112 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1728220AbgELVXB (ORCPT ); Tue, 12 May 2020 17:23:01 -0400 Received: from mail-qv1-xf41.google.com (mail-qv1-xf41.google.com [IPv6:2607:f8b0:4864:20::f41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2E97CC061A0C; Tue, 12 May 2020 14:23:00 -0700 (PDT) Received: by mail-qv1-xf41.google.com with SMTP id ep1so7228197qvb.0; Tue, 12 May 2020 14:23:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=9mmPVJ7AXhJKSRjZ7YCmZe+7ZE3UVCX5QvDqCSfsE+Y=; b=IiHVmuBPFJNurHvTyLACuWwBuplaEqopBuK4HzwxEo0qaz6tWfjcjyurTY5RF6CDgu nZHwhKzyfl/RfkPlSalnhf/6pzRJ0bh1nzprfpK7obWZ8u65dg5hmuqVQKU/hTdMMJZp sLhQAoLAfyzj5hB2g6wSCEK0AvyGFMtxh58lOUGQyEVKVL7mJyafEC/Az6gOSQm/Zy7R 2JTmyKlo0c2i5iMCTbBgjvcUjsH0TPOIabF5krgq4LQnRzBjCnrDEd8s/CzK+vioImrV 6Fq2jrwSvegXwZCGTT7FT8XbHPvDQM7T3ThIfCi4TXl0dqiFxaGa4MEIRPbOXBIWNTc8 OShg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=9mmPVJ7AXhJKSRjZ7YCmZe+7ZE3UVCX5QvDqCSfsE+Y=; b=cZ5nl97xU2DimDh0XgBXt14SC+Rk6fQ/cVXS4g/jclUgXmW/5WoFtFVa8dUJWrZctR w60m9sGbyibEs+7hl0+3Zh4bLW5RJZaVParWzGlwvD1JsoEXPZKsZVDl6qkWFALXoLXT ZYjhDdfoa54MN2gtGaw5hdYATfBY6Qvt+/stZKbJ+JKysPJBN4WkMjZ3jkO0eCKiQWtL 5WkoXQI6TpHBP3hGZ58HK7h1SieOfoC4BvZTA3K1XRunUQhhI/PvioP/8S2LiLzC6DIK 6ki6KrjTeDte4mFeySqxx6kdpxDrS1i8DEWPni6lG2z1Rnx10YTonnbKKfNpNm9xwfYs VoDw== X-Gm-Message-State: AGi0PuZJfibKORTrux627E9M3TZ/ta+XgqVmTGVtdB5jqEcmopnXLd/D dR7N2lWESL53qz35tj5Ce8rN4DXy X-Google-Smtp-Source: APiQypK2i3sjMg6fCIhrGeSATLxzRvbjS7TMvawi+2aDKJQJ1q8i6Esh+bj3xbtPySH3NKAYBuZIUA== X-Received: by 2002:ad4:4e4d:: with SMTP id eb13mr22377090qvb.169.1589318579123; Tue, 12 May 2020 14:22:59 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id i24sm7622994qtm.85.2020.05.12.14.22.58 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:22:58 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLMv90009904; Tue, 12 May 2020 21:22:57 GMT Subject: [PATCH v2 10/29] svcrdma: Displayed remote IP address should match stored address From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:22:57 -0400 Message-ID: <20200512212257.5826.13180.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: After commit 1e091c3bbf51 ("svcrdma: Ignore source port when computing DRC hash"), the IP address stored in xpt_remote always has a port number of zero. Thus, there's no need to display the port number when displaying the IP address of a remote NFS/RDMA client. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_transport.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c index ea54785db4f8..0a1125277a48 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c @@ -211,7 +211,12 @@ static void handle_connect_req(struct rdma_cm_id *new_cma_id, newxprt->sc_ord = param->initiator_depth; sa = (struct sockaddr *)&newxprt->sc_cm_id->route.addr.dst_addr; - svc_xprt_set_remote(&newxprt->sc_xprt, sa, svc_addr_len(sa)); + newxprt->sc_xprt.xpt_remotelen = svc_addr_len(sa); + memcpy(&newxprt->sc_xprt.xpt_remote, sa, + newxprt->sc_xprt.xpt_remotelen); + snprintf(newxprt->sc_xprt.xpt_remotebuf, + sizeof(newxprt->sc_xprt.xpt_remotebuf) - 1, "%pISc", sa); + /* The remote port is arbitrary and not under the control of the * client ULP. Set it to a fixed value so that the DRC continues * to be effective after a reconnect. From patchwork Tue May 12 21:23:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544109 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7037D139A for ; Tue, 12 May 2020 21:23:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5467520659 for ; Tue, 12 May 2020 21:23:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eK/5zme6" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731291AbgELVXG (ORCPT ); Tue, 12 May 2020 17:23:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1728220AbgELVXG (ORCPT ); Tue, 12 May 2020 17:23:06 -0400 Received: from mail-qv1-xf41.google.com (mail-qv1-xf41.google.com [IPv6:2607:f8b0:4864:20::f41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0ED78C061A0C; Tue, 12 May 2020 14:23:06 -0700 (PDT) Received: by mail-qv1-xf41.google.com with SMTP id di6so7113630qvb.10; Tue, 12 May 2020 14:23:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=MV0EoO+8yVK6ghDt30lbfGE+CAEo1PY+DasnmJYde9U=; b=eK/5zme6rq0/ZJFMlWRfeEScDx6QmsTVBPpVpiEe/tF2gIUn4MzXAswpXGcXuxwdQJ FrrLdEPvRyENXJQbOJ44daT3XSg5jBqJB9jE7BR3RKQtF2k5gmDZEXl78+37v6lDyobL An8wKB9JnJkF9EsuaR+Ddl2hIFs6SY1op/nDSD6UwfjUl8vnPG/SQAv87Z3Prkt1QYg0 bCnbHyY7Cm7+aT0l0NDqPPq/fq3PJA22QflSu+pewr2l5Rza8Zv6Oo8rOpZUF3OHSlKT OrWcTNdRzOH/3obmLjLYPucjnXzHXbE+/neiKY8jK3QtjPzJp9FHkxioyF5k0MYGfjKw CW6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=MV0EoO+8yVK6ghDt30lbfGE+CAEo1PY+DasnmJYde9U=; b=uHqDwPgZGFYfBWOhD4HHsuCgj7CfHtB3uwLO1qBlaowS3xc71+Joc4TJNODNOXNjrC NkVWEiJ2UssBrW0CMHL9OmLdEEeDl86LJpYHy2zWcXhi8HrAeT0lXSewNVB6TCuHDLic SJearBGFFLHuvGU/PfH/ov54Qvu0X/MSdMB/eOiR+cHpG2DcV+r9RVfYlsPaFHIqHUSf uEw9c5hhSb9laMdyOwpoX8cyr9xggeH8qZWMez3nmbSsBY3MXHjf8cWdtJgI0/SlvIQ1 bf0gg5v57ImfE7TBFDE9a+htpI2hkfzLTXx1KVdLso+kvJxm53CA9EoRMABzZouSQfkZ HY/A== X-Gm-Message-State: AGi0PuYzLOp+SgnqYNwjpiskZxrJDeHsTiJkzavDSWoXusG8ufYntBLT uM4w7+lR8d5EKQiRDvcVh9+9dkci X-Google-Smtp-Source: APiQypJMEmshn0/6wSYHgikdyxvUeRFyNabGRlq6SmtksGpaZdwOAbM6DniwjiWKoO7P76qmm1F6bg== X-Received: by 2002:a0c:8324:: with SMTP id j33mr23313003qva.23.1589318584890; Tue, 12 May 2020 14:23:04 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id e23sm11433766qkm.63.2020.05.12.14.23.04 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:23:04 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLN272009907; Tue, 12 May 2020 21:23:02 GMT Subject: [PATCH v2 11/29] svcrdma: Add tracepoints to report ->xpo_accept failures From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:23:02 -0400 Message-ID: <20200512212302.5826.42765.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Failure to accept a connection is typically due to a problem specific to a transport type. Also, ->xpo_accept returns NULL on error rather than reporting a specific problem. So, add failure-specific tracepoints in svc_rdma_accept(). Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 38 +++++++++++++++++++++++++++++- net/sunrpc/xprtrdma/svc_rdma_transport.c | 27 +++++++++------------ 2 files changed, 48 insertions(+), 17 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 53b24c8c7860..79ef2ab7743c 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1309,9 +1309,45 @@ DECLARE_EVENT_CLASS(svcrdma_xprt_event, TP_ARGS(xprt)) DEFINE_XPRT_EVENT(accept); -DEFINE_XPRT_EVENT(fail); DEFINE_XPRT_EVENT(free); +DECLARE_EVENT_CLASS(svcrdma_accept_class, + TP_PROTO( + const struct svcxprt_rdma *rdma, + long status + ), + + TP_ARGS(rdma, status), + + TP_STRUCT__entry( + __field(long, status) + __string(addr, rdma->sc_xprt.xpt_remotebuf) + ), + + TP_fast_assign( + __entry->status = status; + __assign_str(addr, rdma->sc_xprt.xpt_remotebuf); + ), + + TP_printk("addr=%s status=%ld", + __get_str(addr), __entry->status + ) +); + +#define DEFINE_ACCEPT_EVENT(name) \ + DEFINE_EVENT(svcrdma_accept_class, svcrdma_##name##_err, \ + TP_PROTO( \ + const struct svcxprt_rdma *rdma, \ + long status \ + ), \ + TP_ARGS(rdma, status)) + +DEFINE_ACCEPT_EVENT(pd); +DEFINE_ACCEPT_EVENT(qp); +DEFINE_ACCEPT_EVENT(fabric); +DEFINE_ACCEPT_EVENT(initdepth); +DEFINE_ACCEPT_EVENT(accept); + TRACE_DEFINE_ENUM(RDMA_MSG); TRACE_DEFINE_ENUM(RDMA_NOMSG); TRACE_DEFINE_ENUM(RDMA_MSGP); diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c index 0a1125277a48..f3b5ad2bec2f 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c @@ -410,9 +410,6 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) if (!newxprt) return NULL; - dprintk("svcrdma: newxprt from accept queue = %p, cm_id=%p\n", - newxprt, newxprt->sc_cm_id); - dev = newxprt->sc_cm_id->device; newxprt->sc_port_num = newxprt->sc_cm_id->port_num; @@ -448,21 +445,17 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) newxprt->sc_pd = ib_alloc_pd(dev, 0); if (IS_ERR(newxprt->sc_pd)) { - dprintk("svcrdma: error creating PD for connect request\n"); + trace_svcrdma_pd_err(newxprt, PTR_ERR(newxprt->sc_pd)); goto errout; } newxprt->sc_sq_cq = ib_alloc_cq_any(dev, newxprt, newxprt->sc_sq_depth, IB_POLL_WORKQUEUE); - if (IS_ERR(newxprt->sc_sq_cq)) { - dprintk("svcrdma: error creating SQ CQ for connect request\n"); + if (IS_ERR(newxprt->sc_sq_cq)) goto errout; - } newxprt->sc_rq_cq = ib_alloc_cq_any(dev, newxprt, rq_depth, IB_POLL_WORKQUEUE); - if (IS_ERR(newxprt->sc_rq_cq)) { - dprintk("svcrdma: error creating RQ CQ for connect request\n"); + if (IS_ERR(newxprt->sc_rq_cq)) goto errout; - } memset(&qp_attr, 0, sizeof qp_attr); qp_attr.event_handler = qp_event_handler; @@ -486,7 +479,7 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) ret = rdma_create_qp(newxprt->sc_cm_id, newxprt->sc_pd, &qp_attr); if (ret) { - dprintk("svcrdma: failed to create QP, ret=%d\n", ret); + trace_svcrdma_qp_err(newxprt, ret); goto errout; } newxprt->sc_qp = newxprt->sc_cm_id->qp; @@ -494,8 +487,10 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) if (!(dev->attrs.device_cap_flags & IB_DEVICE_MEM_MGT_EXTENSIONS)) newxprt->sc_snd_w_inv = false; if (!rdma_protocol_iwarp(dev, newxprt->sc_port_num) && - !rdma_ib_or_roce(dev, newxprt->sc_port_num)) + !rdma_ib_or_roce(dev, newxprt->sc_port_num)) { + trace_svcrdma_fabric_err(newxprt, -EINVAL); goto errout; + } if (!svc_rdma_post_recvs(newxprt)) goto errout; @@ -517,15 +512,17 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) conn_param.initiator_depth = min_t(int, newxprt->sc_ord, dev->attrs.max_qp_init_rd_atom); if (!conn_param.initiator_depth) { - dprintk("svcrdma: invalid ORD setting\n"); ret = -EINVAL; + trace_svcrdma_initdepth_err(newxprt, ret); goto errout; } conn_param.private_data = &pmsg; conn_param.private_data_len = sizeof(pmsg); ret = rdma_accept(newxprt->sc_cm_id, &conn_param); - if (ret) + if (ret) { + trace_svcrdma_accept_err(newxprt, ret); goto errout; + } #if IS_ENABLED(CONFIG_SUNRPC_DEBUG) dprintk("svcrdma: new connection %p accepted:\n", newxprt); @@ -544,8 +541,6 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) return &newxprt->sc_xprt; errout: - dprintk("svcrdma: failure accepting new connection rc=%d.\n", ret); - trace_svcrdma_xprt_fail(&newxprt->sc_xprt); /* Take a reference in case the DTO handler runs */ svc_xprt_get(&newxprt->sc_xprt); if (newxprt->sc_qp && !IS_ERR(newxprt->sc_qp)) From patchwork Tue May 12 21:23:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544113 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 48FCB1668 for ; Tue, 12 May 2020 21:23:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 30F1520659 for ; Tue, 12 May 2020 21:23:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WOFIHh2A" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731293AbgELVXM (ORCPT ); Tue, 12 May 2020 17:23:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46142 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1728220AbgELVXM (ORCPT ); Tue, 12 May 2020 17:23:12 -0400 Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 92ACAC061A0C; Tue, 12 May 2020 14:23:10 -0700 (PDT) Received: by mail-qk1-x742.google.com with SMTP id g185so15249518qke.7; Tue, 12 May 2020 14:23:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=hra1t6ZMjxlnkAjll6l3Ni1WveftdJxGwRCSmO0AxDM=; b=WOFIHh2AlKs7JLT9QnyycKbX36KQlvHRIENVADLL6wX6GQWQ2vV5/4NGOjz8VUI27i bM090+eSF1JmfLNK/YhCBQ24dLc89w1BwR7QBo4THcW7PLhqrQ7uzzehpnZ8nK9WoiPI slo3txMxT0OGK8JlEm5XcdPPCkbjxSLv0e5/u7QYQ+NXVmU/e6keWxqtIAhQNNelqWvD Aac/Tm7ApfAlEte3H/7Bccw5Aoyvi0LBeWdFkZ3SPgEBHQIpTfS0X7K2XWelFVFLs6rs YrdIJwYbLyshPZlKaInSEHX7Bz94JKsA8sp0Ixm/z3olXyTgZuGL4Rvy+zbXmZwel9CW N+2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=hra1t6ZMjxlnkAjll6l3Ni1WveftdJxGwRCSmO0AxDM=; b=kKgxOBJvPr+PBFVkLw8sEBiZht1LFoDNyhIvxs/Fl6vE/UZO2roj9Kigex1ONTNShO T/zpChyhQzxFGwjrwerlnaVqOP8GdYgug12HzEHTOvNPw2IX54KN2gv1Nby66Q/RZxsV 2ncTp15eipsExsMlfvef5nAyVQg/Khn8A/925vaF0s4rJdrLPJKkUCDl/yfYvQC7g53u fCvHUI0Up5dB2yCYf4PXHJ25e6/oi4YkWrd/Z9w9TUMkjhUjNMMSHZW7e45P5u7CLOkg rmTBod0x0pyg8sb33iJzudUEDuLB/IDZGWoiftednMRcsZ5z2bG9GdTW4G0TYUccLxTe YJ0w== X-Gm-Message-State: AGi0PuYeSKGTx1/AEfbWWMVItaNKfvp+EwSzjh+8m+bOpCYo+yrOAcbO FEO75qe3xJbQlmK7JbThWE1Nr+qT X-Google-Smtp-Source: APiQypJkj6Q0H8xWjg8PC+C0XnFaJX2+KXWjWeXspC2AqlmixqvmzL8huMhxN7ZzfLHEw5QZ5TEsBg== X-Received: by 2002:a05:620a:55b:: with SMTP id o27mr24819214qko.161.1589318589657; Tue, 12 May 2020 14:23:09 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id c68sm12191243qke.129.2020.05.12.14.23.08 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:23:09 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLN83s009910; Tue, 12 May 2020 21:23:08 GMT Subject: [PATCH v2 12/29] SUNRPC: Remove kernel memory address from svc_xprt tracepoints From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:23:08 -0400 Message-ID: <20200512212308.5826.1735.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: The xprt=%p was meant to distinguish events from different transports, but the addr=%s does that just as well and does not expose kernel memory addresses. Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 27 +++++++-------------------- 1 file changed, 7 insertions(+), 20 deletions(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index ffd2215950dc..53f2461cf552 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1173,22 +1173,19 @@ TRACE_EVENT(svc_xprt_do_enqueue, TP_ARGS(xprt, rqst), TP_STRUCT__entry( - __field(struct svc_xprt *, xprt) __field(int, pid) __field(unsigned long, flags) __string(addr, xprt->xpt_remotebuf) ), TP_fast_assign( - __entry->xprt = xprt; __entry->pid = rqst? rqst->rq_task->pid : 0; __entry->flags = xprt->xpt_flags; __assign_str(addr, xprt->xpt_remotebuf); ), - TP_printk("xprt=%p addr=%s pid=%d flags=%s", - __entry->xprt, __get_str(addr), - __entry->pid, show_svc_xprt_flags(__entry->flags)) + TP_printk("addr=%s pid=%d flags=%s", __get_str(addr), + __entry->pid, show_svc_xprt_flags(__entry->flags)) ); DECLARE_EVENT_CLASS(svc_xprt_event, @@ -1197,20 +1194,17 @@ DECLARE_EVENT_CLASS(svc_xprt_event, TP_ARGS(xprt), TP_STRUCT__entry( - __field(struct svc_xprt *, xprt) __field(unsigned long, flags) __string(addr, xprt->xpt_remotebuf) ), TP_fast_assign( - __entry->xprt = xprt; __entry->flags = xprt->xpt_flags; __assign_str(addr, xprt->xpt_remotebuf); ), - TP_printk("xprt=%p addr=%s flags=%s", - __entry->xprt, __get_str(addr), - show_svc_xprt_flags(__entry->flags)) + TP_printk("addr=%s flags=%s", __get_str(addr), + show_svc_xprt_flags(__entry->flags)) ); DEFINE_EVENT(svc_xprt_event, svc_xprt_no_write_space, @@ -1223,24 +1217,20 @@ TRACE_EVENT(svc_xprt_dequeue, TP_ARGS(rqst), TP_STRUCT__entry( - __field(struct svc_xprt *, xprt) __field(unsigned long, flags) __field(unsigned long, wakeup) __string(addr, rqst->rq_xprt->xpt_remotebuf) ), TP_fast_assign( - __entry->xprt = rqst->rq_xprt; __entry->flags = rqst->rq_xprt->xpt_flags; __entry->wakeup = ktime_to_us(ktime_sub(ktime_get(), rqst->rq_qtime)); __assign_str(addr, rqst->rq_xprt->xpt_remotebuf); ), - TP_printk("xprt=%p addr=%s flags=%s wakeup-us=%lu", - __entry->xprt, __get_str(addr), - show_svc_xprt_flags(__entry->flags), - __entry->wakeup) + TP_printk("addr=%s flags=%s wakeup-us=%lu", __get_str(addr), + show_svc_xprt_flags(__entry->flags), __entry->wakeup) ); TRACE_EVENT(svc_wake_up, @@ -1265,21 +1255,18 @@ TRACE_EVENT(svc_handle_xprt, TP_ARGS(xprt, len), TP_STRUCT__entry( - __field(struct svc_xprt *, xprt) __field(int, len) __field(unsigned long, flags) __string(addr, xprt->xpt_remotebuf) ), TP_fast_assign( - __entry->xprt = xprt; __entry->len = len; __entry->flags = xprt->xpt_flags; __assign_str(addr, xprt->xpt_remotebuf); ), - TP_printk("xprt=%p addr=%s len=%d flags=%s", - __entry->xprt, __get_str(addr), + TP_printk("addr=%s len=%d flags=%s", __get_str(addr), __entry->len, show_svc_xprt_flags(__entry->flags)) ); From patchwork Tue May 12 21:23:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544117 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CBB37139A for ; Tue, 12 May 2020 21:23:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B452B20659 for ; Tue, 12 May 2020 21:23:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CZ/QjaF+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731295AbgELVXQ (ORCPT ); Tue, 12 May 2020 17:23:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46154 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1728220AbgELVXP (ORCPT ); Tue, 12 May 2020 17:23:15 -0400 Received: from mail-qt1-x844.google.com (mail-qt1-x844.google.com [IPv6:2607:f8b0:4864:20::844]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A928EC061A0C; Tue, 12 May 2020 14:23:15 -0700 (PDT) Received: by mail-qt1-x844.google.com with SMTP id g16so11678948qtp.11; Tue, 12 May 2020 14:23:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=x9bL/Iqx8u9/y33FYLTD6gAy1N032CHzoVQnLll7ihc=; b=CZ/QjaF+YHrKTPQP/W2jR2bRx6g9OaKgdWXXz3pXFziGH7njDVNg/TWQsQbYj5Suof jivb6x2rMk9TW74FBoWkqkqQF+1N1OLto9Fp+HhBQ1iA/IHIhWevOvriu60DkswV4MNb xzpfNsh4cei/D8tNsNhcGhb1Djldzs60VoRwII0BGjAkd95T8OPXAuCcqxbk5H3Jxtp3 NAbnTTvt4BcB7zypc85JcNsIl2uh7tLpxQTZWo3d8iAAtsfx50/RoGInNhFFDEbv/fFP jKfcOwX6C/7EbGePWCBWWtwr0mKc5/JoMPCKJYXyk8t0WcvTNKPUVKxq7lOIPaaJSVwN MK1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=x9bL/Iqx8u9/y33FYLTD6gAy1N032CHzoVQnLll7ihc=; b=UfkkqZ5EfO4Aoo+VjZwtOwJg8SNfBr4pAhl92LI5Nx673Qbd8nQnULBmXBWy7VmNb2 +I2PN2MNYgYE27HpMU1VFaKmnzz1j//UD0GKdSb6pC79qfyfUW2Ft14iB8Yg1CT2lJMb 6AXqXMw0TT9CtWL7aGhRaCk+3fd0qtxPgfI4XzImXEnPMqImkcsj0aMD1BORqccnbWqo Xp8ha67Hx/JErBUQ40gtZ7FTPMNWOuhtCAF2DB811BBVbEFFg9Y2gRCpa0dMIqPubL0V IySPMhu8gNmMrrxHGX2vmb6S/0P6n1MWF9i1uUTwbP7H2+CE9c2AmSwY2vAq9q4rzJ2w 07Tg== X-Gm-Message-State: AGi0PubZiaRDZ5Wmkfof7rUPb+Az9gmCpZ6RSS0pAjqAhFlWfVBYasLu DwnEuBXw5izYnC0rsoRDsEYy0/gR X-Google-Smtp-Source: APiQypK7uRinI6wYPYDf7RqXnls0oPpUD9pDIqayrowTEW5Li57VyNQudHN9Oj+DJF6SbfIFF1SAJw== X-Received: by 2002:ac8:342a:: with SMTP id u39mr14702761qtb.80.1589318594727; Tue, 12 May 2020 14:23:14 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id v46sm10401700qtj.74.2020.05.12.14.23.14 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:23:14 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLNDmX009913; Tue, 12 May 2020 21:23:13 GMT Subject: [PATCH v2 13/29] SUNRPC: Tracepoint to record errors in svc_xpo_create() From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:23:13 -0400 Message-ID: <20200512212313.5826.1908.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Capture transport creation failures. Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 29 +++++++++++++++++++++++++++++ net/sunrpc/svc_xprt.c | 7 ++++++- 2 files changed, 35 insertions(+), 1 deletion(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index 53f2461cf552..f3296ed2b753 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1167,6 +1167,35 @@ DEFINE_EVENT(svc_rqst_status, svc_send, { (1UL << XPT_KILL_TEMP), "XPT_KILL_TEMP"}, \ { (1UL << XPT_CONG_CTRL), "XPT_CONG_CTRL"}) +TRACE_EVENT(svc_xprt_create_err, + TP_PROTO( + const char *program, + const char *protocol, + struct sockaddr *sap, + const struct svc_xprt *xprt + ), + + TP_ARGS(program, protocol, sap, xprt), + + TP_STRUCT__entry( + __field(long, error) + __string(program, program) + __string(protocol, protocol) + __array(unsigned char, addr, sizeof(struct sockaddr_in6)) + ), + + TP_fast_assign( + __entry->error = PTR_ERR(xprt); + __assign_str(program, program); + __assign_str(protocol, protocol); + memcpy(__entry->addr, sap, sizeof(__entry->addr)); + ), + + TP_printk("addr=%pISpc program=%s protocol=%s error=%ld", + __entry->addr, __get_str(program), __get_str(protocol), + __entry->error) +); + TRACE_EVENT(svc_xprt_do_enqueue, TP_PROTO(struct svc_xprt *xprt, struct svc_rqst *rqst), diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c index 07cdbf7d5764..f89e04210a48 100644 --- a/net/sunrpc/svc_xprt.c +++ b/net/sunrpc/svc_xprt.c @@ -206,6 +206,7 @@ static struct svc_xprt *__svc_xpo_create(struct svc_xprt_class *xcl, .sin6_port = htons(port), }; #endif + struct svc_xprt *xprt; struct sockaddr *sap; size_t len; @@ -224,7 +225,11 @@ static struct svc_xprt *__svc_xpo_create(struct svc_xprt_class *xcl, return ERR_PTR(-EAFNOSUPPORT); } - return xcl->xcl_ops->xpo_create(serv, net, sap, len, flags); + xprt = xcl->xcl_ops->xpo_create(serv, net, sap, len, flags); + if (IS_ERR(xprt)) + trace_svc_xprt_create_err(serv->sv_program->pg_name, + xcl->xcl_name, sap, xprt); + return xprt; } /* From patchwork Tue May 12 21:23:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544121 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD7E5139A for ; Tue, 12 May 2020 21:23:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8F14820659 for ; Tue, 12 May 2020 21:23:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nYrVrQok" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731298AbgELVXW (ORCPT ); Tue, 12 May 2020 17:23:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1728220AbgELVXV (ORCPT ); Tue, 12 May 2020 17:23:21 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D0C4C061A0C; Tue, 12 May 2020 14:23:21 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id b188so15234882qkd.9; Tue, 12 May 2020 14:23:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=naFN0o6bDHNF8CmTkqitirTzkLhYyHiZdARmOi27zX8=; b=nYrVrQoklM+rSPEL2XT1bIHeKY983+EcNRStkyaInohfLTsPvaR2G02jyqVuHpShpF 82Lg3suMNlSjmWZDd1cu2JefnHO3PBN5GpseuCulgbOOzTpE6JgeOS3TD/G9vNH+Lrha OPoG7GnnpqPnC2WtgRLrWHGo0RgyCqV6Md4kCrqLO9Dw/vbBXqkQoCb9gkUIMJX0rkqR CfJO8nEIrBWGBQkHldkNfLhlLgHJivH1fgj1ne1XJhLxLTypR9dG2VG6S05yb4T/tEze 4qn6NatROLwe8PMESSTkFMrYnRGiFAssfJJ4yP/EjXQlnW/tv+XWM+lFzkoOWbma+3BG vE3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=naFN0o6bDHNF8CmTkqitirTzkLhYyHiZdARmOi27zX8=; b=AQISUkhcVyglk/35KTG5TnUtGykYu6AG8Bum2/5vFC9ApI5W5pI3OPExDNp3rlCpLp GudES4J8KdYZWlAfCId0f3yjSql0rTCllA8/pTKkgJFhjc5dutINGkb+b4X+07CERc+C CRtn7YfG7+SZB1fEuNqm/4xQoblJAsb1KQcnK824GqpHv3hIb8sSeHzzuLFqRd/x4CWI UwnL9BMZp70idI7OLIDmx4940X+9gpCKDLRfiUiTS5enT4YgU/08SdDeQEFm/J/5wOYV 3A3rPp8ti/iA7+vS3daZXHw2OvzJzKMBBRBOk4qfv1SvTmO7GhNjdUKF7lFY2Eeg8xff vU0w== X-Gm-Message-State: AGi0PuZMhfiYgKCaz794v8Zf+Hym5ojBDgfV3HbRYJE4rgz1kg+K1znp xtm3QVcPBYYjNB5+naZguvS/5nLo X-Google-Smtp-Source: APiQypK7tTGq2wC29WwAKzAw/ozv8s4rlweYsPK7qc91og1dNGEugOFz3meBYhBYwMszbyXRQGds+Q== X-Received: by 2002:a37:9586:: with SMTP id x128mr22373505qkd.90.1589318599946; Tue, 12 May 2020 14:23:19 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id m7sm13355592qti.6.2020.05.12.14.23.19 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:23:19 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLNIaU009916; Tue, 12 May 2020 21:23:18 GMT Subject: [PATCH v2 14/29] SUNRPC: Trace a few more generic svc_xprt events From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:23:18 -0400 Message-ID: <20200512212318.5826.93587.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org In lieu of dprintks or tracepoints in each individual transport implementation, introduce tracepoints in the generic part of the RPC layer. These typically fire for connection lifetime events, so shouldn't contribute a lot of noise. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 32 ------------------------- include/trace/events/sunrpc.h | 39 ++++++++++++++++++++++++++++-- net/sunrpc/svc_xprt.c | 22 ++++------------- net/sunrpc/svcsock.c | 12 --------- net/sunrpc/xprtrdma/svc_rdma_transport.c | 21 +++------------- 5 files changed, 45 insertions(+), 81 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 79ef2ab7743c..bdcde7d33f14 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1279,38 +1279,6 @@ TRACE_EVENT(xprtrdma_leaked_rep, ** Server-side RPC/RDMA events **/ -DECLARE_EVENT_CLASS(svcrdma_xprt_event, - TP_PROTO( - const struct svc_xprt *xprt - ), - - TP_ARGS(xprt), - - TP_STRUCT__entry( - __field(const void *, xprt) - __string(addr, xprt->xpt_remotebuf) - ), - - TP_fast_assign( - __entry->xprt = xprt; - __assign_str(addr, xprt->xpt_remotebuf); - ), - - TP_printk("xprt=%p addr=%s", - __entry->xprt, __get_str(addr) - ) -); - -#define DEFINE_XPRT_EVENT(name) \ - DEFINE_EVENT(svcrdma_xprt_event, svcrdma_xprt_##name, \ - TP_PROTO( \ - const struct svc_xprt *xprt \ - ), \ - TP_ARGS(xprt)) - -DEFINE_XPRT_EVENT(accept); -DEFINE_XPRT_EVENT(free); - DECLARE_EVENT_CLASS(svcrdma_accept_class, TP_PROTO( const struct svcxprt_rdma *rdma, diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index f3296ed2b753..d40ec8f5c220 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1236,9 +1236,42 @@ DECLARE_EVENT_CLASS(svc_xprt_event, show_svc_xprt_flags(__entry->flags)) ); -DEFINE_EVENT(svc_xprt_event, svc_xprt_no_write_space, - TP_PROTO(struct svc_xprt *xprt), - TP_ARGS(xprt)); +#define DEFINE_SVC_XPRT_EVENT(name) \ + DEFINE_EVENT(svc_xprt_event, svc_xprt_##name, \ + TP_PROTO( \ + struct svc_xprt *xprt \ + ), \ + TP_ARGS(xprt)) + +DEFINE_SVC_XPRT_EVENT(no_write_space); +DEFINE_SVC_XPRT_EVENT(close); +DEFINE_SVC_XPRT_EVENT(detach); +DEFINE_SVC_XPRT_EVENT(free); + +TRACE_EVENT(svc_xprt_accept, + TP_PROTO( + const struct svc_xprt *xprt, + const char *service + ), + + TP_ARGS(xprt, service), + + TP_STRUCT__entry( + __string(addr, xprt->xpt_remotebuf) + __string(protocol, xprt->xpt_class->xcl_name) + __string(service, service) + ), + + TP_fast_assign( + __assign_str(addr, xprt->xpt_remotebuf); + __assign_str(protocol, xprt->xpt_class->xcl_name) + __assign_str(service, service); + ), + + TP_printk("addr=%s protocol=%s service=%s", + __get_str(addr), __get_str(protocol), __get_str(service) + ) +); TRACE_EVENT(svc_xprt_dequeue, TP_PROTO(struct svc_rqst *rqst), diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c index f89e04210a48..0a546ef02dde 100644 --- a/net/sunrpc/svc_xprt.c +++ b/net/sunrpc/svc_xprt.c @@ -153,6 +153,7 @@ static void svc_xprt_free(struct kref *kref) xprt_put(xprt->xpt_bc_xprt); if (xprt->xpt_bc_xps) xprt_switch_put(xprt->xpt_bc_xps); + trace_svc_xprt_free(xprt); xprt->xpt_ops->xpo_free(xprt); module_put(owner); } @@ -309,15 +310,11 @@ int svc_create_xprt(struct svc_serv *serv, const char *xprt_name, { int err; - dprintk("svc: creating transport %s[%d]\n", xprt_name, port); err = _svc_create_xprt(serv, xprt_name, net, family, port, flags, cred); if (err == -EPROTONOSUPPORT) { request_module("svc%s", xprt_name); err = _svc_create_xprt(serv, xprt_name, net, family, port, flags, cred); } - if (err < 0) - dprintk("svc: transport %s not found, err %d\n", - xprt_name, -err); return err; } EXPORT_SYMBOL_GPL(svc_create_xprt); @@ -785,7 +782,6 @@ static int svc_handle_xprt(struct svc_rqst *rqstp, struct svc_xprt *xprt) int len = 0; if (test_bit(XPT_CLOSE, &xprt->xpt_flags)) { - dprintk("svc_recv: found XPT_CLOSE\n"); if (test_and_clear_bit(XPT_KILL_TEMP, &xprt->xpt_flags)) xprt->xpt_ops->xpo_kill_temp_xprt(xprt); svc_delete_xprt(xprt); @@ -804,6 +800,7 @@ static int svc_handle_xprt(struct svc_rqst *rqstp, struct svc_xprt *xprt) if (newxpt) { newxpt->xpt_cred = get_cred(xprt->xpt_cred); svc_add_new_temp_xprt(serv, newxpt); + trace_svc_xprt_accept(newxpt, serv->sv_name); } else module_put(xprt->xpt_class->xcl_owner); } else if (svc_xprt_reserve_slot(rqstp, xprt)) { @@ -840,14 +837,6 @@ int svc_recv(struct svc_rqst *rqstp, long timeout) struct svc_serv *serv = rqstp->rq_server; int len, err; - dprintk("svc: server %p waiting for data (to = %ld)\n", - rqstp, timeout); - - if (rqstp->rq_xprt) - printk(KERN_ERR - "svc_recv: service %p, transport not NULL!\n", - rqstp); - err = svc_alloc_arg(rqstp); if (err) goto out; @@ -895,7 +884,6 @@ EXPORT_SYMBOL_GPL(svc_recv); void svc_drop(struct svc_rqst *rqstp) { trace_svc_drop(rqstp); - dprintk("svc: xprt %p dropped request\n", rqstp->rq_xprt); svc_xprt_release(rqstp); } EXPORT_SYMBOL_GPL(svc_drop); @@ -1030,11 +1018,10 @@ static void svc_delete_xprt(struct svc_xprt *xprt) struct svc_serv *serv = xprt->xpt_server; struct svc_deferred_req *dr; - /* Only do this once */ if (test_and_set_bit(XPT_DEAD, &xprt->xpt_flags)) - BUG(); + return; - dprintk("svc: svc_delete_xprt(%p)\n", xprt); + trace_svc_xprt_detach(xprt); xprt->xpt_ops->xpo_detach(xprt); if (xprt->xpt_bc_xprt) xprt->xpt_bc_xprt->ops->close(xprt->xpt_bc_xprt); @@ -1055,6 +1042,7 @@ static void svc_delete_xprt(struct svc_xprt *xprt) void svc_close_xprt(struct svc_xprt *xprt) { + trace_svc_xprt_close(xprt); set_bit(XPT_CLOSE, &xprt->xpt_flags); if (test_and_set_bit(XPT_BUSY, &xprt->xpt_flags)) /* someone else will have to effect the close */ diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 3e7b6445e317..cf4bd198c19d 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -727,7 +727,6 @@ static struct svc_xprt *svc_tcp_accept(struct svc_xprt *xprt) int err, slen; RPC_IFDEBUG(char buf[RPC_MAX_ADDRBUFLEN]); - dprintk("svc: tcp_accept %p sock %p\n", svsk, sock); if (!sock) return NULL; @@ -1363,11 +1362,6 @@ static struct svc_xprt *svc_create_socket(struct svc_serv *serv, int newlen; int family; int val; - RPC_IFDEBUG(char buf[RPC_MAX_ADDRBUFLEN]); - - dprintk("svc: svc_create_socket(%s, %d, %s)\n", - serv->sv_program->pg_name, protocol, - __svc_print_addr(sin, buf, sizeof(buf))); if (protocol != IPPROTO_UDP && protocol != IPPROTO_TCP) { printk(KERN_WARNING "svc: only UDP and TCP " @@ -1427,7 +1421,6 @@ static struct svc_xprt *svc_create_socket(struct svc_serv *serv, svc_xprt_set_local(&svsk->sk_xprt, newsin, newlen); return (struct svc_xprt *)svsk; bummer: - dprintk("svc: svc_create_socket error = %d\n", -error); sock_release(sock); return ERR_PTR(error); } @@ -1441,8 +1434,6 @@ static void svc_sock_detach(struct svc_xprt *xprt) struct svc_sock *svsk = container_of(xprt, struct svc_sock, sk_xprt); struct sock *sk = svsk->sk_sk; - dprintk("svc: svc_sock_detach(%p)\n", svsk); - /* put back the old socket callbacks */ lock_sock(sk); sk->sk_state_change = svsk->sk_ostate; @@ -1459,8 +1450,6 @@ static void svc_tcp_sock_detach(struct svc_xprt *xprt) { struct svc_sock *svsk = container_of(xprt, struct svc_sock, sk_xprt); - dprintk("svc: svc_tcp_sock_detach(%p)\n", svsk); - svc_sock_detach(xprt); if (!test_bit(XPT_LISTENER, &xprt->xpt_flags)) { @@ -1475,7 +1464,6 @@ static void svc_tcp_sock_detach(struct svc_xprt *xprt) static void svc_sock_free(struct svc_xprt *xprt) { struct svc_sock *svsk = container_of(xprt, struct svc_sock, sk_xprt); - dprintk("svc: svc_sock_free(%p)\n", svsk); if (svsk->sk_sock->file) sockfd_put(svsk->sk_sock); diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c index f3b5ad2bec2f..d38be57b00ed 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c @@ -314,11 +314,8 @@ static struct svc_xprt *svc_rdma_create(struct svc_serv *serv, struct svcxprt_rdma *cma_xprt; int ret; - dprintk("svcrdma: Creating RDMA listener\n"); - if ((sa->sa_family != AF_INET) && (sa->sa_family != AF_INET6)) { - dprintk("svcrdma: Address family %d is not supported.\n", sa->sa_family); + if (sa->sa_family != AF_INET && sa->sa_family != AF_INET6) return ERR_PTR(-EAFNOSUPPORT); - } cma_xprt = svc_rdma_create_xprt(serv, net); if (!cma_xprt) return ERR_PTR(-ENOMEM); @@ -329,7 +326,6 @@ static struct svc_xprt *svc_rdma_create(struct svc_serv *serv, RDMA_PS_TCP, IB_QPT_RC); if (IS_ERR(listen_id)) { ret = PTR_ERR(listen_id); - dprintk("svcrdma: rdma_create_id failed = %d\n", ret); goto err0; } @@ -338,23 +334,17 @@ static struct svc_xprt *svc_rdma_create(struct svc_serv *serv, */ #if IS_ENABLED(CONFIG_IPV6) ret = rdma_set_afonly(listen_id, 1); - if (ret) { - dprintk("svcrdma: rdma_set_afonly failed = %d\n", ret); + if (ret) goto err1; - } #endif ret = rdma_bind_addr(listen_id, sa); - if (ret) { - dprintk("svcrdma: rdma_bind_addr failed = %d\n", ret); + if (ret) goto err1; - } cma_xprt->sc_cm_id = listen_id; ret = rdma_listen(listen_id, RPCRDMA_LISTEN_BACKLOG); - if (ret) { - dprintk("svcrdma: rdma_listen failed = %d\n", ret); + if (ret) goto err1; - } /* * We need to use the address from the cm_id in case the @@ -537,7 +527,6 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) dprintk(" ord : %d\n", conn_param.initiator_depth); #endif - trace_svcrdma_xprt_accept(&newxprt->sc_xprt); return &newxprt->sc_xprt; errout: @@ -578,8 +567,6 @@ static void __svc_rdma_free(struct work_struct *work) container_of(work, struct svcxprt_rdma, sc_work); struct svc_xprt *xprt = &rdma->sc_xprt; - trace_svcrdma_xprt_free(xprt); - if (rdma->sc_qp && !IS_ERR(rdma->sc_qp)) ib_drain_qp(rdma->sc_qp); From patchwork Tue May 12 21:23:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544125 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5103F15E6 for ; Tue, 12 May 2020 21:23:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 37270205C9 for ; Tue, 12 May 2020 21:23:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WrQo85PM" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728220AbgELVX0 (ORCPT ); Tue, 12 May 2020 17:23:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46188 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726885AbgELVX0 (ORCPT ); Tue, 12 May 2020 17:23:26 -0400 Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F8DDC061A0C; Tue, 12 May 2020 14:23:26 -0700 (PDT) Received: by mail-qk1-x741.google.com with SMTP id z80so9736621qka.0; Tue, 12 May 2020 14:23:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=X4fXdAWhojbxrpWwGIfQWlHhfLTmVqHl5gL0tZf73us=; b=WrQo85PM41OMEET8F7B3uRk+ivjJIkQIH5Me0WKlLok+ZkVJapIVGaoLsCWkxL/lsT uTJQoaMgunZ/Hrl8+BTg7V6J/kXWz8BUKpDR+NleQXE/KxnpS+uLgHynW5QSGSzFtqtP 9IZpxaAtXRNNFh9k8PqrMF5POwzh5KkdrOWzpBDFOMc5RVUDLcbYP7B+DQxkroP+9K02 9PpkUqjz82bcKzvGsRPSDRNAlPjh4a/T09GcASDqQpswH7d2qLhym9bs2g4HghRGO9Nc 9aZ/UVRAa86GRJxd/WQ6dAkX5pW2F2rqtO7yVZa/JOpDsrY6nY3UEB4tl4/Jcgd+pnbh aXZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=X4fXdAWhojbxrpWwGIfQWlHhfLTmVqHl5gL0tZf73us=; b=DFtr2yYQ3FckqCsVTFm2Ai0SKHdS6a0RivN3I9tRnSuZN8lsz8Urz+P/WKwKa54CMd XS/DfArZI6zRdHqTr2CGrL4hniIHIZQ9FqPt/MLcRqW9tfbVIg3zc14Rz2HtRvNlmFPJ A1r7g5Av8g6ZFCd7mUupjpcZGicZfnIhIAvgbRK+LJ8CLnvmpELvQmoWyqLOuuCv7ui0 fF079Gar0UvwMqtPH9qxZuAJ7x0pP2otam/LPLTW2KaTzjNpLi6HzR2NY52dE0bkukGQ F9TXVcu38X/ayHIaTt6DjD5wqrgx2Jxb2MzoHEX4ydrXEAG7J/zcbeAbRA/nAk2T/ND3 xc8w== X-Gm-Message-State: AGi0Pub1lPcfEWrCnPi1MbqbgmZIubcS/byaIY+z+/SeD2LcyCieqSY1 YTbCCEQdqkYXZ6MnY42c4XcCEgZi X-Google-Smtp-Source: APiQypL/OEJwLx81cTpHqL6Ymc2CwyMB2/ogoCWlFg6mWGDov4thES/Jes+ytxAmSxVp06/5QgcQAw== X-Received: by 2002:a05:620a:490:: with SMTP id 16mr22533198qkr.203.1589318605141; Tue, 12 May 2020 14:23:25 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id h13sm13473015qti.32.2020.05.12.14.23.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:23:24 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLNNk4009919; Tue, 12 May 2020 21:23:23 GMT Subject: [PATCH v2 15/29] SUNRPC: Remove "#include " From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:23:23 -0400 Message-ID: <20200512212323.5826.47423.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: Commit 850cbaddb52d ("udp: use it's own memory accounting schema") removed the last skb-related tracepoint from svcsock.c, so it is no longer necessary to include trace/events/skb.h. Signed-off-by: Chuck Lever --- net/sunrpc/svcsock.c | 1 - 1 file changed, 1 deletion(-) diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index cf4bd198c19d..1c4d0aacc531 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -45,7 +45,6 @@ #include #include #include -#include #include #include From patchwork Tue May 12 21:23:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544129 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7139C139A for ; Tue, 12 May 2020 21:23:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4D8B12054F for ; Tue, 12 May 2020 21:23:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="haZZqZRQ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731307AbgELVXc (ORCPT ); Tue, 12 May 2020 17:23:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726885AbgELVXc (ORCPT ); Tue, 12 May 2020 17:23:32 -0400 Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 68EEFC061A0C; Tue, 12 May 2020 14:23:32 -0700 (PDT) Received: by mail-qt1-x841.google.com with SMTP id o19so886095qtr.10; Tue, 12 May 2020 14:23:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=r877wUtRvZ6Zp5A+x8HMM1onHY30LaZ1pvUz3eI0Qhc=; b=haZZqZRQ9hybKouewJehtVrJO2lVPFTovUu0Ch2tsXOybboGUStW1TdKUoJqB5azSB dZnxoNRmg2ePTBZRw19/8Pwp7Bkvea+xs4BVZQK9tJchNqtP+u0yT8LA0XHtZSDJPFvK O90X72xrKwN0vuOLzNrDvrfiYwy2puGGktsTY4lLCZ6PeKNYz0/4rZ5ufFI97k49bbtS m8ADrfYIOmlrLKPnW4VVdz2PU2d4Dy1JEG4w46/hzy++xsyagW4OBzwkVO1pgLQBadmN xWYErsC8U26OLCx0z+uQos0dqIjAByZwTdCidkX3larryH4LT0LGOG5cbfwpmax9Zjnj /w2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=r877wUtRvZ6Zp5A+x8HMM1onHY30LaZ1pvUz3eI0Qhc=; b=pqaDfg1OF5xwyRGxFch9EUX2833Xz7BcpqsDSrmx1528WCurkcGsXFcDDhMILamU+5 25h37QfLtRMAIfexzq0i6o6zmWArCfqH/ennQG+UsscbNQLvYu0cLUVxQScae8JB4vdQ rj7T6nN/UKIBk5M58fZvtNAYft6MCxh1PA59mxYgKBtDS3fogtCmOagXK4ZjiYplK2vD 0aBZ/9gjeDk+YGbG2/2vFs9UXvjzU1K0KzfQ57kpI4lxw29RT5whUA6nViohbwwR0G95 PHEDa6hNbuZNpW2fY0lF+tIotf0CyeSI2U6PhhU5Fwir7VKM1uLpNDr8Pdmf1+qqhIRm adSg== X-Gm-Message-State: AGi0PuahyfK4L7vUNVojDGpVdsJivWq34v4O0t8QMTDZtlPoi+1sEopl ej4fS2yeN0lXIX6GSwyzynew0kiU X-Google-Smtp-Source: APiQypLwUKK3n5RxLT1MbfucHEgGPDndK/O7X0gQBG95ZL7RQXi+OzLj9T9qIig4W48moXSw/HK3FQ== X-Received: by 2002:ac8:1194:: with SMTP id d20mr25113057qtj.359.1589318611278; Tue, 12 May 2020 14:23:31 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id b129sm10409732qkc.58.2020.05.12.14.23.29 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:23:30 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLNTre009922; Tue, 12 May 2020 21:23:29 GMT Subject: [PATCH v2 16/29] SUNRPC: Add more svcsock tracepoints From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:23:29 -0400 Message-ID: <20200512212329.5826.70743.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org In addition to tracing recently-updated socket sendto events, this commit adds a trace event class that can be used for additional svcsock-related tracepoints in subsequent patches. Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 97 +++++++++++++++++++++++++++++++++++++++++ net/sunrpc/svcsock.c | 30 ++++--------- 2 files changed, 107 insertions(+), 20 deletions(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index d40ec8f5c220..bf086640b14a 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -14,6 +14,39 @@ #include #include +TRACE_DEFINE_ENUM(SOCK_STREAM); +TRACE_DEFINE_ENUM(SOCK_DGRAM); +TRACE_DEFINE_ENUM(SOCK_RAW); +TRACE_DEFINE_ENUM(SOCK_RDM); +TRACE_DEFINE_ENUM(SOCK_SEQPACKET); +TRACE_DEFINE_ENUM(SOCK_DCCP); +TRACE_DEFINE_ENUM(SOCK_PACKET); + +#define show_socket_type(type) \ + __print_symbolic(type, \ + { SOCK_STREAM, "STREAM" }, \ + { SOCK_DGRAM, "DGRAM" }, \ + { SOCK_RAW, "RAW" }, \ + { SOCK_RDM, "RDM" }, \ + { SOCK_SEQPACKET, "SEQPACKET" }, \ + { SOCK_DCCP, "DCCP" }, \ + { SOCK_PACKET, "PACKET" }) + +/* This list is known to be incomplete, add new enums as needed. */ +TRACE_DEFINE_ENUM(AF_UNSPEC); +TRACE_DEFINE_ENUM(AF_UNIX); +TRACE_DEFINE_ENUM(AF_LOCAL); +TRACE_DEFINE_ENUM(AF_INET); +TRACE_DEFINE_ENUM(AF_INET6); + +#define rpc_show_address_family(family) \ + __print_symbolic(family, \ + { AF_UNSPEC, "AF_UNSPEC" }, \ + { AF_UNIX, "AF_UNIX" }, \ + { AF_LOCAL, "AF_LOCAL" }, \ + { AF_INET, "AF_INET" }, \ + { AF_INET6, "AF_INET6" }) + DECLARE_EVENT_CLASS(xdr_buf_class, TP_PROTO( const struct xdr_buf *xdr @@ -1384,6 +1417,70 @@ DECLARE_EVENT_CLASS(svc_deferred_event, DEFINE_SVC_DEFERRED_EVENT(drop); DEFINE_SVC_DEFERRED_EVENT(revisit); +TRACE_EVENT(svcsock_new_socket, + TP_PROTO( + const struct socket *socket + ), + + TP_ARGS(socket), + + TP_STRUCT__entry( + __field(unsigned long, type) + __field(unsigned long, family) + __field(bool, listener) + ), + + TP_fast_assign( + __entry->type = socket->type; + __entry->family = socket->sk->sk_family; + __entry->listener = (socket->sk->sk_state == TCP_LISTEN); + ), + + TP_printk("type=%s family=%s%s", + show_socket_type(__entry->type), + rpc_show_address_family(__entry->family), + __entry->listener ? " (listener)" : "" + ) +); + +DECLARE_EVENT_CLASS(svcsock_class, + TP_PROTO( + const struct svc_xprt *xprt, + ssize_t result + ), + + TP_ARGS(xprt, result), + + TP_STRUCT__entry( + __field(ssize_t, result) + __field(unsigned long, flags) + __string(addr, xprt->xpt_remotebuf) + ), + + TP_fast_assign( + __entry->result = result; + __entry->flags = xprt->xpt_flags; + __assign_str(addr, xprt->xpt_remotebuf); + ), + + TP_printk("addr=%s result=%zd flags=%s", __get_str(addr), + __entry->result, show_svc_xprt_flags(__entry->flags) + ) +); + +#define DEFINE_SVCSOCK_EVENT(name) \ + DEFINE_EVENT(svcsock_class, svcsock_##name, \ + TP_PROTO( \ + const struct svc_xprt *xprt, \ + ssize_t result \ + ), \ + TP_ARGS(xprt, result)) + +DEFINE_SVCSOCK_EVENT(udp_send); +DEFINE_SVCSOCK_EVENT(tcp_send); +DEFINE_SVCSOCK_EVENT(data_ready); +DEFINE_SVCSOCK_EVENT(write_space); + DECLARE_EVENT_CLASS(cache_event, TP_PROTO( const struct cache_detail *cd, diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 1c4d0aacc531..758b835ad4ce 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -54,6 +54,8 @@ #include #include +#include + #include "socklib.h" #include "sunrpc.h" @@ -281,13 +283,10 @@ static void svc_data_ready(struct sock *sk) struct svc_sock *svsk = (struct svc_sock *)sk->sk_user_data; if (svsk) { - dprintk("svc: socket %p(inet %p), busy=%d\n", - svsk, sk, - test_bit(XPT_BUSY, &svsk->sk_xprt.xpt_flags)); - /* Refer to svc_setup_socket() for details. */ rmb(); svsk->sk_odata(sk); + trace_svcsock_data_ready(&svsk->sk_xprt, 0); if (!test_and_set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags)) svc_xprt_enqueue(&svsk->sk_xprt); } @@ -301,11 +300,9 @@ static void svc_write_space(struct sock *sk) struct svc_sock *svsk = (struct svc_sock *)(sk->sk_user_data); if (svsk) { - dprintk("svc: socket %p(inet %p), write_space busy=%d\n", - svsk, sk, test_bit(XPT_BUSY, &svsk->sk_xprt.xpt_flags)); - /* Refer to svc_setup_socket() for details. */ rmb(); + trace_svcsock_write_space(&svsk->sk_xprt, 0); svsk->sk_owspace(sk); svc_xprt_enqueue(&svsk->sk_xprt); } @@ -545,6 +542,7 @@ static int svc_udp_sendto(struct svc_rqst *rqstp) err = xprt_sock_sendmsg(svsk->sk_sock, &msg, xdr, 0, 0, &sent); xdr_free_bvec(xdr); } + trace_svcsock_udp_send(xprt, err); mutex_unlock(&xprt->xpt_mutex); if (err < 0) @@ -616,7 +614,7 @@ static struct svc_xprt_class svc_udp_class = { static void svc_udp_init(struct svc_sock *svsk, struct svc_serv *serv) { - int err, level, optname, one = 1; + int level, optname, one = 1; svc_xprt_init(sock_net(svsk->sk_sock->sk), &svc_udp_class, &svsk->sk_xprt, serv); @@ -647,9 +645,8 @@ static void svc_udp_init(struct svc_sock *svsk, struct svc_serv *serv) default: BUG(); } - err = kernel_setsockopt(svsk->sk_sock, level, optname, - (char *)&one, sizeof(one)); - dprintk("svc: kernel_setsockopt returned %d\n", err); + kernel_setsockopt(svsk->sk_sock, level, optname, (char *)&one, + sizeof(one)); } /* @@ -1100,6 +1097,7 @@ static int svc_tcp_sendto(struct svc_rqst *rqstp) goto out_notconn; err = xprt_sock_sendmsg(svsk->sk_sock, &msg, xdr, 0, marker, &sent); xdr_free_bvec(xdr); + trace_svcsock_tcp_send(xprt, err < 0 ? err : sent); if (err < 0 || sent != (xdr->len + sizeof(marker))) goto out_close; mutex_unlock(&xprt->xpt_mutex); @@ -1170,13 +1168,11 @@ static void svc_tcp_init(struct svc_sock *svsk, struct svc_serv *serv) set_bit(XPT_CACHE_AUTH, &svsk->sk_xprt.xpt_flags); set_bit(XPT_CONG_CTRL, &svsk->sk_xprt.xpt_flags); if (sk->sk_state == TCP_LISTEN) { - dprintk("setting up TCP socket for listening\n"); strcpy(svsk->sk_xprt.xpt_remotebuf, "listener"); set_bit(XPT_LISTENER, &svsk->sk_xprt.xpt_flags); sk->sk_data_ready = svc_tcp_listen_data_ready; set_bit(XPT_CONN, &svsk->sk_xprt.xpt_flags); } else { - dprintk("setting up TCP socket for reading\n"); sk->sk_state_change = svc_tcp_state_change; sk->sk_data_ready = svc_data_ready; sk->sk_write_space = svc_write_space; @@ -1226,7 +1222,6 @@ static struct svc_sock *svc_setup_socket(struct svc_serv *serv, int pmap_register = !(flags & SVC_SOCK_ANONYMOUS); int err = 0; - dprintk("svc: svc_setup_socket %p\n", sock); svsk = kzalloc(sizeof(*svsk), GFP_KERNEL); if (!svsk) return ERR_PTR(-ENOMEM); @@ -1263,12 +1258,7 @@ static struct svc_sock *svc_setup_socket(struct svc_serv *serv, else svc_tcp_init(svsk, serv); - dprintk("svc: svc_setup_socket created %p (inet %p), " - "listen %d close %d\n", - svsk, svsk->sk_sk, - test_bit(XPT_LISTENER, &svsk->sk_xprt.xpt_flags), - test_bit(XPT_CLOSE, &svsk->sk_xprt.xpt_flags)); - + trace_svcsock_new_socket(sock); return svsk; } From patchwork Tue May 12 21:23:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544133 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 02E5515E6 for ; Tue, 12 May 2020 21:23:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DAA4120659 for ; Tue, 12 May 2020 21:23:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="tcBzrUf6" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731304AbgELVXh (ORCPT ); Tue, 12 May 2020 17:23:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46216 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726382AbgELVXh (ORCPT ); Tue, 12 May 2020 17:23:37 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6177C061A0C; Tue, 12 May 2020 14:23:36 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id 4so12455912qtb.4; Tue, 12 May 2020 14:23:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=stmkbiHWzCCZRNsPWiFjJVSrWBUV/AE/XfhmyB/uKqQ=; b=tcBzrUf68dDEXvfNulylowUskLACU5ts6ggIL7WhZThrm7hg2UKRdpLbFKIZHaOQvd xtV6RTya8/AXQnWXdPSCSxRYFg3oZeMLYvwaDsGCaifFbcER90FY8TMtZmvCU9dT7cNa ydljUHkRhosIDldSwXIajAB88O/yK4pcM1+ppLgGDmOKy85VbVNc3Kjn/a0iRHhubxm2 XvP/oeaYtrX8vn14w9ejSwrq9xjPbLIx757B9J/nTupIgp1yvSGFAHPKSb7SUBdHavPd wqgdSbetEp+iHR5VKODXI17GLAcFznPD6tOwCAfkRAIKD+WexPOPAVObkl7/VugZs9pf 5bYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=stmkbiHWzCCZRNsPWiFjJVSrWBUV/AE/XfhmyB/uKqQ=; b=jzl2YtzIsPUzizNJmZ39zDaX6/AGK2vJtlP7E5fd9e9e+SZt0iTuyZmEO9BAoM3hrm gWFHzJV7ySiotlKWmy9js9Qk4I3jBMjpWyqZP6Mg0nUCOYIjv+RA5GdNV9vvgYVTjeaD ulEBW0hHSfdvNJ7IQ1kZihrIhmteDWiHP2mhZe5bYz4IIsWaaCDFj9Vz06SVSb1a5fcX bQ+/JswLL5n7jZYOZi6NrqoCVWcYw3z09HZ9Y39/bBO0gZd5nbrt466omVq5xDs/N+rM tnhlhiYGfNwWw2SAtBaR/m1gyvd0Tbxxu8q0qUDQ++XcWWvTt42GcTD5Y7A6ZvDYAyo2 25mg== X-Gm-Message-State: AGi0Pub3FVLYEc7shCyeUX5KAgeJYi/UIJPOBDOP1JUTCvDv7UrYj8d8 1QgAZiX5K8sk/iErVsesbE5j+88S X-Google-Smtp-Source: APiQypIJo3fdq6cmYIGHUo8p3vbrjFI5NnnZnJXA17cEzBIfd4olN2zy25m4qe7tWhWRkazc97e9WQ== X-Received: by 2002:ac8:1b58:: with SMTP id p24mr24750941qtk.29.1589318615762; Tue, 12 May 2020 14:23:35 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id h18sm6082014qkh.3.2020.05.12.14.23.35 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:23:35 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLNYNn009925; Tue, 12 May 2020 21:23:34 GMT Subject: [PATCH v2 17/29] SUNRPC: Replace dprintk call sites in TCP state change callouts From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:23:34 -0400 Message-ID: <20200512212334.5826.55386.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Report TCP socket state changes and accept failures via tracepoints, replacing dprintk() call sites. No tracepoint is added in svc_tcp_listen_data_ready. There's no information available there that isn't also reported by the svcsock_new_socket and the accept failure tracepoints. Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 67 +++++++++++++++++++++++++++++++++++++++++ net/sunrpc/svcsock.c | 35 ++++----------------- 2 files changed, 73 insertions(+), 29 deletions(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index bf086640b14a..ed8c991d4f04 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1481,6 +1481,73 @@ DEFINE_SVCSOCK_EVENT(tcp_send); DEFINE_SVCSOCK_EVENT(data_ready); DEFINE_SVCSOCK_EVENT(write_space); +TRACE_EVENT(svcsock_tcp_state, + TP_PROTO( + const struct svc_xprt *xprt, + const struct socket *socket + ), + + TP_ARGS(xprt, socket), + + TP_STRUCT__entry( + __field(unsigned long, socket_state) + __field(unsigned long, sock_state) + __field(unsigned long, flags) + __string(addr, xprt->xpt_remotebuf) + ), + + TP_fast_assign( + __entry->socket_state = socket->state; + __entry->sock_state = socket->sk->sk_state; + __entry->flags = xprt->xpt_flags; + __assign_str(addr, xprt->xpt_remotebuf); + ), + + TP_printk("addr=%s state=%s sk_state=%s flags=%s", __get_str(addr), + rpc_show_socket_state(__entry->socket_state), + rpc_show_sock_state(__entry->sock_state), + show_svc_xprt_flags(__entry->flags) + ) +); + +DECLARE_EVENT_CLASS(svcsock_accept_class, + TP_PROTO( + const struct svc_xprt *xprt, + const char *service, + long status + ), + + TP_ARGS(xprt, service, status), + + TP_STRUCT__entry( + __field(long, status) + __string(service, service) + __array(unsigned char, addr, sizeof(struct sockaddr_in6)) + ), + + TP_fast_assign( + __entry->status = status; + __assign_str(service, service); + memcpy(__entry->addr, &xprt->xpt_local, sizeof(__entry->addr)); + ), + + TP_printk("listener=%pISpc service=%s status=%ld", + __entry->addr, __get_str(service), __entry->status + ) +); + +#define DEFINE_ACCEPT_EVENT(name) \ + DEFINE_EVENT(svcsock_accept_class, svcsock_##name##_err, \ + TP_PROTO( \ + const struct svc_xprt *xprt, \ + const char *service, \ + long status \ + ), \ + TP_ARGS(xprt, service, status)) + +DEFINE_ACCEPT_EVENT(accept); +DEFINE_ACCEPT_EVENT(getpeername); + DECLARE_EVENT_CLASS(cache_event, TP_PROTO( const struct cache_detail *cd, diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 758b835ad4ce..4ac1180c6306 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -657,9 +657,6 @@ static void svc_tcp_listen_data_ready(struct sock *sk) { struct svc_sock *svsk = (struct svc_sock *)sk->sk_user_data; - dprintk("svc: socket %p TCP (listen) state change %d\n", - sk, sk->sk_state); - if (svsk) { /* Refer to svc_setup_socket() for details. */ rmb(); @@ -680,8 +677,7 @@ static void svc_tcp_listen_data_ready(struct sock *sk) if (svsk) { set_bit(XPT_CONN, &svsk->sk_xprt.xpt_flags); svc_xprt_enqueue(&svsk->sk_xprt); - } else - printk("svc: socket %p: no user data\n", sk); + } } } @@ -692,15 +688,11 @@ static void svc_tcp_state_change(struct sock *sk) { struct svc_sock *svsk = (struct svc_sock *)sk->sk_user_data; - dprintk("svc: socket %p TCP (connected) state change %d (svsk %p)\n", - sk, sk->sk_state, sk->sk_user_data); - - if (!svsk) - printk("svc: socket %p: no user data\n", sk); - else { + if (svsk) { /* Refer to svc_setup_socket() for details. */ rmb(); svsk->sk_ostate(sk); + trace_svcsock_tcp_state(&svsk->sk_xprt, svsk->sk_sock); if (sk->sk_state != TCP_ESTABLISHED) { set_bit(XPT_CLOSE, &svsk->sk_xprt.xpt_flags); svc_xprt_enqueue(&svsk->sk_xprt); @@ -721,7 +713,6 @@ static struct svc_xprt *svc_tcp_accept(struct svc_xprt *xprt) struct socket *newsock; struct svc_sock *newsvsk; int err, slen; - RPC_IFDEBUG(char buf[RPC_MAX_ADDRBUFLEN]); if (!sock) return NULL; @@ -735,30 +726,18 @@ static struct svc_xprt *svc_tcp_accept(struct svc_xprt *xprt) else if (err != -EAGAIN) net_warn_ratelimited("%s: accept failed (err %d)!\n", serv->sv_name, -err); + trace_svcsock_accept_err(xprt, serv->sv_name, err); return NULL; } set_bit(XPT_CONN, &svsk->sk_xprt.xpt_flags); err = kernel_getpeername(newsock, sin); if (err < 0) { - net_warn_ratelimited("%s: peername failed (err %d)!\n", - serv->sv_name, -err); + trace_svcsock_getpeername_err(xprt, serv->sv_name, err); goto failed; /* aborted connection or whatever */ } slen = err; - /* Ideally, we would want to reject connections from unauthorized - * hosts here, but when we get encryption, the IP of the host won't - * tell us anything. For now just warn about unpriv connections. - */ - if (!svc_port_is_privileged(sin)) { - dprintk("%s: connect from unprivileged port: %s\n", - serv->sv_name, - __svc_print_addr(sin, buf, sizeof(buf))); - } - dprintk("%s: connect from %s\n", serv->sv_name, - __svc_print_addr(sin, buf, sizeof(buf))); - /* Reset the inherited callbacks before calling svc_setup_socket */ newsock->sk->sk_state_change = svsk->sk_ostate; newsock->sk->sk_data_ready = svsk->sk_odata; @@ -776,10 +755,8 @@ static struct svc_xprt *svc_tcp_accept(struct svc_xprt *xprt) svc_xprt_set_remote(&newsvsk->sk_xprt, sin, slen); err = kernel_getsockname(newsock, sin); slen = err; - if (unlikely(err < 0)) { - dprintk("svc_tcp_accept: kernel_getsockname error %d\n", -err); + if (unlikely(err < 0)) slen = offsetof(struct sockaddr, sa_data); - } svc_xprt_set_local(&newsvsk->sk_xprt, sin, slen); if (sock_is_loopback(newsock->sk)) From patchwork Tue May 12 21:23:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544137 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3C008139A for ; Tue, 12 May 2020 21:23:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2487620659 for ; Tue, 12 May 2020 21:23:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QtLn5W1c" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731309AbgELVXm (ORCPT ); Tue, 12 May 2020 17:23:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731308AbgELVXm (ORCPT ); Tue, 12 May 2020 17:23:42 -0400 Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com [IPv6:2607:f8b0:4864:20::72c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2586FC061A0C; Tue, 12 May 2020 14:23:42 -0700 (PDT) Received: by mail-qk1-x72c.google.com with SMTP id f189so9588737qkd.5; Tue, 12 May 2020 14:23:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=5BzQNrQOxWDpb7CbWBXAzsOoSrfajvJCRTfn9EK/+x0=; b=QtLn5W1ctHOtlRosAujCCtTvp1Y1S+9wNK+FM4cOpNDtQVwPbxGWmdEgjBubHC9KE/ RF2I9KwS4xufDY+oEaWtFDNFJ5vbVoD7eQNMyRRjBAwap01eAEspbnVWIMbXx2BjiGa2 qM9L3b2ekD/WRO30IgjQ45nn3vbK5oE/mGw8FHlRGZ0IAl9Vv0Zc8km5EshQOG04vj+2 IXRCSxmYKq6JJQidTOj+r4WQ6rGOgjwNeb1Om22M7DHZWLH1mMM/NxsXPp8x5m+jbzWV EdlTb8aidC+7sUp0WutYoFvMLrSdpeY+NaozbMlwoJZDPXEagTod/lEob6CUTCeaXQMN MXfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=5BzQNrQOxWDpb7CbWBXAzsOoSrfajvJCRTfn9EK/+x0=; b=ZtlBxiDMsoU/e/O4ZimlhbyFIJty3pMxkSC2kh/BT1yhpaliuzGfLWXslBtYC79OZQ ybMpAxPAoDbQ1Ha/ndwVA6JqK/6ubizVNFVoqFf5v4Kv2LPbrahQnTxntxRXM3AWY/Eb e7OqrDzrrVTfzMi8l6IySBN78PnAaAHhB+E0w83UwQSvBqH+C6t4jS77WYOktFo8WgR/ OhawSJCS41Hb4HTGHNa7Co+c2pf6bqFck5l3+lZV1qQqK0R7HAjqvT7Vr+2XCXvj1MPj csgZPQoYlLNx6fjNs4q2ZLLmFW34p4Z68hPQtjz+SxJks6Yik3d4ItDXe4WEIPthN6KR 6heg== X-Gm-Message-State: AGi0PuZ6kHe+Nau10I0f6yf1VOF2xHBUNrg3H6dH82DB42q9lR+LHc2N 88p3LZZ6okeO7759wczk/JdWrvic X-Google-Smtp-Source: APiQypLqTDrGM3l1HzLidfUJ/epKZYNAFDWokaAViJ6PkvwjpucQ6BswmN3VY07Up4EI/ojx9HishA== X-Received: by 2002:a37:668b:: with SMTP id a133mr21769615qkc.488.1589318620981; Tue, 12 May 2020 14:23:40 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id o20sm700023qtl.75.2020.05.12.14.23.40 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:23:40 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLNdCn009930; Tue, 12 May 2020 21:23:39 GMT Subject: [PATCH v2 18/29] SUNRPC: Trace server-side rpcbind registration events From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:23:39 -0400 Message-ID: <20200512212339.5826.17675.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 80 +++++++++++++++++++++++++++++++++++++++++ net/sunrpc/svc.c | 19 ++-------- 2 files changed, 84 insertions(+), 15 deletions(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index ed8c991d4f04..6d85bbb7b8b1 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1581,6 +1581,86 @@ DEFINE_CACHE_EVENT(cache_entry_update); DEFINE_CACHE_EVENT(cache_entry_make_negative); DEFINE_CACHE_EVENT(cache_entry_no_listener); +DECLARE_EVENT_CLASS(register_class, + TP_PROTO( + const char *program, + const u32 version, + const int family, + const unsigned short protocol, + const unsigned short port, + int error + ), + + TP_ARGS(program, version, family, protocol, port, error), + + TP_STRUCT__entry( + __field(u32, version) + __field(unsigned long, family) + __field(unsigned short, protocol) + __field(unsigned short, port) + __field(int, error) + __string(program, program) + ), + + TP_fast_assign( + __entry->version = version; + __entry->family = family; + __entry->protocol = protocol; + __entry->port = port; + __entry->error = error; + __assign_str(program, program); + ), + + TP_printk("program=%sv%u proto=%s port=%u family=%s error=%d", + __get_str(program), __entry->version, + __entry->protocol == IPPROTO_UDP ? "udp" : "tcp", + __entry->port, rpc_show_address_family(__entry->family), + __entry->error + ) +); + +#define DEFINE_REGISTER_EVENT(name) \ + DEFINE_EVENT(register_class, svc_##name, \ + TP_PROTO( \ + const char *program, \ + const u32 version, \ + const int family, \ + const unsigned short protocol, \ + const unsigned short port, \ + int error \ + ), \ + TP_ARGS(program, version, family, protocol, \ + port, error)) + +DEFINE_REGISTER_EVENT(register); +DEFINE_REGISTER_EVENT(noregister); + +TRACE_EVENT(svc_unregister, + TP_PROTO( + const char *program, + const u32 version, + int error + ), + + TP_ARGS(program, version, error), + + TP_STRUCT__entry( + __field(u32, version) + __field(int, error) + __string(program, program) + ), + + TP_fast_assign( + __entry->version = version; + __entry->error = error; + __assign_str(program, program); + ), + + TP_printk("program=%sv%u error=%d", + __get_str(program), __entry->version, __entry->error + ) +); + #endif /* _TRACE_SUNRPC_H */ #include diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index 9ed3126600ce..3e74d61ca7da 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -991,6 +991,7 @@ static int __svc_register(struct net *net, const char *progname, #endif } + trace_svc_register(progname, version, protocol, port, family, error); return error; } @@ -1000,11 +1001,6 @@ int svc_rpcbind_set_version(struct net *net, unsigned short proto, unsigned short port) { - dprintk("svc: svc_register(%sv%d, %s, %u, %u)\n", - progp->pg_name, version, - proto == IPPROTO_UDP? "udp" : "tcp", - port, family); - return __svc_register(net, progp->pg_name, progp->pg_prog, version, family, proto, port); @@ -1024,11 +1020,8 @@ int svc_generic_rpcbind_set(struct net *net, return 0; if (vers->vs_hidden) { - dprintk("svc: svc_register(%sv%d, %s, %u, %u)" - " (but not telling portmap)\n", - progp->pg_name, version, - proto == IPPROTO_UDP? "udp" : "tcp", - port, family); + trace_svc_noregister(progp->pg_name, version, proto, + port, family, 0); return 0; } @@ -1106,8 +1099,7 @@ static void __svc_unregister(struct net *net, const u32 program, const u32 versi if (error == -EPROTONOSUPPORT) error = rpcb_register(net, program, version, 0, 0); - dprintk("svc: %s(%sv%u), error %d\n", - __func__, progname, version, error); + trace_svc_unregister(progname, version, error); } /* @@ -1132,9 +1124,6 @@ static void svc_unregister(const struct svc_serv *serv, struct net *net) continue; if (progp->pg_vers[i]->vs_hidden) continue; - - dprintk("svc: attempting to unregister %sv%u\n", - progp->pg_name, i); __svc_unregister(net, progp->pg_prog, i, progp->pg_name); } } From patchwork Tue May 12 21:23:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544141 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A1A12139A for ; Tue, 12 May 2020 21:23:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 898C520659 for ; Tue, 12 May 2020 21:23:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LbKrd8DK" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731312AbgELVXs (ORCPT ); Tue, 12 May 2020 17:23:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46246 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731308AbgELVXr (ORCPT ); Tue, 12 May 2020 17:23:47 -0400 Received: from mail-qt1-x844.google.com (mail-qt1-x844.google.com [IPv6:2607:f8b0:4864:20::844]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F669C061A0C; Tue, 12 May 2020 14:23:47 -0700 (PDT) Received: by mail-qt1-x844.google.com with SMTP id l1so8929930qtp.6; Tue, 12 May 2020 14:23:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=51O4yhSr65SN0UuqjkFOH+vH4pbqbD6haQ3RcBeQzKM=; b=LbKrd8DKmBZbQPF/0KDzkIDMUSlHy+ugiGMqhzOSIScDgEGhPU9TIvXvti0OlPe7iI nT47Y/9i9DdbQiQ4s7BYWaAyyYdkKQTajFoXZi4pk/eZ4YmqZ4FFWbnMrOv1AxdiKjrY JvwIdQlXuZ7hyLEws4s+Xa6t8IXbmoN/ddPp0Nir8g0jyrBhkS5UvNLbRX3CeX1EMbls PgNOEQQo93niG08NMXc6RUnwSmjYf4At+tAW35/AK0gLllXqy3ybFZpMwJz5KILjN7He zRKm7E8ZZZMll8x2rjgXvhndLdoE+hQw09YkCoCClwlXSrF2lgr/PYurmk7pWVYA/h9K k4NA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=51O4yhSr65SN0UuqjkFOH+vH4pbqbD6haQ3RcBeQzKM=; b=NqdC6CEujIWQFqZaeRU9E9Q0vB6el8gWcilSNVr0Ligg6YNLUJTStFR/nMCChSaYq8 oACDUZcN2JaYPqBFICTjKvtMEU3FGq1iyP/0ExhkWc2RHLKuHHnUQyQe09C0e4SpC41q xtrVnKD9jF2+tqwULOSSIlarIVVDb+hGQccyuFOSnodbFfWNhstU9cHubxtJfueUKMKl 6MvOPUcCtHMkU1Ktd4R8oATNRC+4ZmdBSK3ojxgj0T/wPu8WUts06vcouake+R1S5VgR +uGNeMyVADrU3iwx3jiS4UqhzH1rE6gGMktY+nEznnJGiuc4QmAdCAJwe6PHrtH7gXpV MAMw== X-Gm-Message-State: AGi0PuaDZZYFqDzgueHSh5/CfPsVclgx9VrMY48Uw/zG6BjqSaiMNeId /zLfxrYUefx+7jjlLTR7gDkioDcY X-Google-Smtp-Source: APiQypJxPYGzg3+++yCtLqi+34sPvGIovQbTOLbh/M/bmXnmF+0XVcn0Y1FO80zW3JtEAfTYmpF2SQ== X-Received: by 2002:ac8:4b45:: with SMTP id e5mr23695041qts.86.1589318626253; Tue, 12 May 2020 14:23:46 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id r30sm7633073qtj.95.2020.05.12.14.23.45 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:23:45 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLNiqO009933; Tue, 12 May 2020 21:23:44 GMT Subject: [PATCH v2 19/29] SUNRPC: Rename svc_sock::sk_reclen From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:23:44 -0400 Message-ID: <20200512212344.5826.34952.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up. I find the name of the svc_sock::sk_reclen field confusing, so I've changed it to better reflect its function. This field is not read directly to get the record length. Rather, it is a buffer containing a record marker that needs to be decoded. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svcsock.h | 6 +++--- net/sunrpc/svcsock.c | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index 771baadaee9d..b7ac7fe68306 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -28,7 +28,7 @@ struct svc_sock { /* private TCP part */ /* On-the-wire fragment header: */ - __be32 sk_reclen; + __be32 sk_marker; /* As we receive a record, this includes the length received so * far (including the fragment header): */ u32 sk_tcplen; @@ -41,12 +41,12 @@ struct svc_sock { static inline u32 svc_sock_reclen(struct svc_sock *svsk) { - return ntohl(svsk->sk_reclen) & RPC_FRAGMENT_SIZE_MASK; + return be32_to_cpu(svsk->sk_marker) & RPC_FRAGMENT_SIZE_MASK; } static inline u32 svc_sock_final_rec(struct svc_sock *svsk) { - return ntohl(svsk->sk_reclen) & RPC_LAST_STREAM_FRAGMENT; + return be32_to_cpu(svsk->sk_marker) & RPC_LAST_STREAM_FRAGMENT; } /* diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 4ac1180c6306..d63b21f3f207 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -841,7 +841,7 @@ static int svc_tcp_recv_record(struct svc_sock *svsk, struct svc_rqst *rqstp) struct kvec iov; want = sizeof(rpc_fraghdr) - svsk->sk_tcplen; - iov.iov_base = ((char *) &svsk->sk_reclen) + svsk->sk_tcplen; + iov.iov_base = ((char *)&svsk->sk_marker) + svsk->sk_tcplen; iov.iov_len = want; len = svc_recvfrom(rqstp, &iov, 1, want, 0); if (len < 0) @@ -938,7 +938,7 @@ static void svc_tcp_fragment_received(struct svc_sock *svsk) svc_sock_final_rec(svsk) ? "final" : "nonfinal", svc_sock_reclen(svsk)); svsk->sk_tcplen = 0; - svsk->sk_reclen = 0; + svsk->sk_marker = xdr_zero; } /* @@ -1154,7 +1154,7 @@ static void svc_tcp_init(struct svc_sock *svsk, struct svc_serv *serv) sk->sk_data_ready = svc_data_ready; sk->sk_write_space = svc_write_space; - svsk->sk_reclen = 0; + svsk->sk_marker = xdr_zero; svsk->sk_tcplen = 0; svsk->sk_datalen = 0; memset(&svsk->sk_pages[0], 0, sizeof(svsk->sk_pages)); From patchwork Tue May 12 21:23:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544145 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6DCF915E6 for ; Tue, 12 May 2020 21:23:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 561AE20659 for ; Tue, 12 May 2020 21:23:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ghCZPXLS" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731317AbgELVXw (ORCPT ); Tue, 12 May 2020 17:23:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46262 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731308AbgELVXw (ORCPT ); Tue, 12 May 2020 17:23:52 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4EACCC061A0C; Tue, 12 May 2020 14:23:52 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id o19so886881qtr.10; Tue, 12 May 2020 14:23:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=mEbGapYI4L5VEY7chiiFIUuJ8+Vvzo9WpaWDp1wo7qY=; b=ghCZPXLSnNu26vGYXtbBM7/0VSJgDYAFxNyr41r9Fqi5Y+CmDdIQ5e26kCdlx8lxmF 20Pea9yFZSlpw/n8bsmxf5YqqEWFFyy21N+K64PwD3fTA4WEWPiuBru1wWY4uAsauuWF MyWk6jInU1ekPmjxH5VW5JmIkCLKx1sL5ijjZbJW1V2gGUW9v6ZUewWbx8+uQyRLfndr l5v0h52YIm26u09zM3CVWkg69B8fSyIqqhs3zCzOt3krgVOph1UaYRMnnR/3Tqj+cGpx zUHMlfmej3yyP0MKg5/Ft9J4/XXo/9O6dIbQQe0a+IvNODo7D1P/BZk5VZHVbeWOvWH6 kCYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=mEbGapYI4L5VEY7chiiFIUuJ8+Vvzo9WpaWDp1wo7qY=; b=tKp8pzZVW2tcVt6yh7jUbomA+5ubzYKBnmV4agK5bc036LnLdUdoef4NdIdTOwbZDR x1gdoe9uIVjJ3nwl2DrSswNgu+Dfmq8cC/w1zlrZbow/HqFqc9R3shHgLmCIiQI12976 5aS3M32jXxG54fiv89zAHnElwvYNmwVFnHU04k2p0HwAiKCj1d65s+6qf5pvZXU6Di1p se/DULk7qb/zer5MovU7y2Y7RZG0eHrGJIZz+16PoMRQJevGyowZtJ0P68JUBrTS02eX k5J7PhcqWdpA3ZdA8/Sn1K9UKVGd9WQtzQAhpMV2gx6aVpgry7QGUdthW7/z2GW23IPv 1+ZA== X-Gm-Message-State: AGi0PuYmXPtVrQqwNd5czosB4iodu7kd7xwZ/0/W0gL+mtr1U7mNNRW7 J51mF498ba2r0LVWqWXU5yUWVplW X-Google-Smtp-Source: APiQypKVm/rJTzdSQSeUs6sg84W6RxcbdRhGCDZZgEt+Bin+/PsPrpBGL2XuIyY0igv5XIHji6Wilw== X-Received: by 2002:ac8:380d:: with SMTP id q13mr12091446qtb.200.1589318631330; Tue, 12 May 2020 14:23:51 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id c24sm10593361qtd.26.2020.05.12.14.23.50 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:23:50 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLNosE009936; Tue, 12 May 2020 21:23:50 GMT Subject: [PATCH v2 20/29] SUNRPC: Restructure svc_tcp_recv_record() From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:23:50 -0400 Message-ID: <20200512212350.5826.96262.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Refactor: svc_recvfrom() is going to be converted to read into bio_vecs in a moment. Unhook the only other caller, svc_tcp_recv_record(), which just wants to read the 4-byte stream record marker into a kvec. While we're in the area, streamline this helper by straight-lining the hot path, replace dprintk call sites with tracepoints, and reduce the number of atomic bit operations in this path. Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 24 +++++++++++++++++++ net/sunrpc/svcsock.c | 51 ++++++++++++++++++++--------------------- 2 files changed, 49 insertions(+), 26 deletions(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index 6d85bbb7b8b1..ec4ae34a1f84 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1443,6 +1443,30 @@ TRACE_EVENT(svcsock_new_socket, ) ); +TRACE_EVENT(svcsock_marker, + TP_PROTO( + const struct svc_xprt *xprt, + __be32 marker + ), + + TP_ARGS(xprt, marker), + + TP_STRUCT__entry( + __field(unsigned int, length) + __field(bool, last) + __string(addr, xprt->xpt_remotebuf) + ), + + TP_fast_assign( + __entry->length = be32_to_cpu(marker) & RPC_FRAGMENT_SIZE_MASK; + __entry->last = be32_to_cpu(marker) & RPC_LAST_STREAM_FRAGMENT; + __assign_str(addr, xprt->xpt_remotebuf); + ), + + TP_printk("addr=%s length=%u%s", __get_str(addr), + __entry->length, __entry->last ? " (last)" : "") +); + DECLARE_EVENT_CLASS(svcsock_class, TP_PROTO( const struct svc_xprt *xprt, diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index d63b21f3f207..9c1eb13aa9b8 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -828,47 +828,45 @@ static void svc_tcp_clear_pages(struct svc_sock *svsk) } /* - * Receive fragment record header. - * If we haven't gotten the record length yet, get the next four bytes. + * Receive fragment record header into sk_marker. */ -static int svc_tcp_recv_record(struct svc_sock *svsk, struct svc_rqst *rqstp) +static ssize_t svc_tcp_read_marker(struct svc_sock *svsk, + struct svc_rqst *rqstp) { - struct svc_serv *serv = svsk->sk_xprt.xpt_server; - unsigned int want; - int len; + ssize_t want, len; + /* If we haven't gotten the record length yet, + * get the next four bytes. + */ if (svsk->sk_tcplen < sizeof(rpc_fraghdr)) { + struct msghdr msg = { NULL }; struct kvec iov; want = sizeof(rpc_fraghdr) - svsk->sk_tcplen; iov.iov_base = ((char *)&svsk->sk_marker) + svsk->sk_tcplen; iov.iov_len = want; - len = svc_recvfrom(rqstp, &iov, 1, want, 0); + iov_iter_kvec(&msg.msg_iter, READ, &iov, 1, want); + len = sock_recvmsg(svsk->sk_sock, &msg, MSG_DONTWAIT); if (len < 0) - goto error; + return len; svsk->sk_tcplen += len; - if (len < want) { - dprintk("svc: short recvfrom while reading record " - "length (%d of %d)\n", len, want); - return -EAGAIN; + /* call again to read the remaining bytes */ + goto err_short; } - - dprintk("svc: TCP record, %d bytes\n", svc_sock_reclen(svsk)); + trace_svcsock_marker(&svsk->sk_xprt, svsk->sk_marker); if (svc_sock_reclen(svsk) + svsk->sk_datalen > - serv->sv_max_mesg) { - net_notice_ratelimited("RPC: fragment too large: %d\n", - svc_sock_reclen(svsk)); - goto err_delete; - } + svsk->sk_xprt.xpt_server->sv_max_mesg) + goto err_too_large; } - return svc_sock_reclen(svsk); -error: - dprintk("RPC: TCP recv_record got %d\n", len); - return len; -err_delete: + +err_too_large: + net_notice_ratelimited("svc: %s %s RPC fragment too large: %d\n", + __func__, svsk->sk_xprt.xpt_server->sv_name, + svc_sock_reclen(svsk)); set_bit(XPT_CLOSE, &svsk->sk_xprt.xpt_flags); +err_short: return -EAGAIN; } @@ -961,12 +959,13 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) test_bit(XPT_CONN, &svsk->sk_xprt.xpt_flags), test_bit(XPT_CLOSE, &svsk->sk_xprt.xpt_flags)); - len = svc_tcp_recv_record(svsk, rqstp); + clear_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); + len = svc_tcp_read_marker(svsk, rqstp); if (len < 0) goto error; base = svc_tcp_restore_pages(svsk, rqstp); - want = svc_sock_reclen(svsk) - (svsk->sk_tcplen - sizeof(rpc_fraghdr)); + want = len - (svsk->sk_tcplen - sizeof(rpc_fraghdr)); vec = rqstp->rq_vec; From patchwork Tue May 12 21:23:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544149 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 33885139A for ; Tue, 12 May 2020 21:23:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 14E7420659 for ; Tue, 12 May 2020 21:23:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WJMBDgP/" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731320AbgELVX6 (ORCPT ); Tue, 12 May 2020 17:23:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46280 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731308AbgELVX6 (ORCPT ); Tue, 12 May 2020 17:23:58 -0400 Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1EAF8C061A0C; Tue, 12 May 2020 14:23:58 -0700 (PDT) Received: by mail-qt1-x843.google.com with SMTP id l1so8930291qtp.6; Tue, 12 May 2020 14:23:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=SckrKGeb70N9IZUWgmauTBxdU9UMVK6bfHfwmMpZ4q8=; b=WJMBDgP/z8519tzkcS5iO2TRmkNNij1a6ZVqEVv/FqcRCQkIAOcHPWH9kKaiCvdE17 BJY7kXzQAbT75fUy3nBQoxZngjrxdyzYeK/5GPLH0rYSozwUIxM2p6b0Yh5/ml4aA6zK dt4BW5sbsqCJ+/2j71Aq78v0KIIX6SzLne/b29FVJITdaWwuaAMjf9BtQt2crnTbbFB/ 6mUqPRgxFNdx7eYWjqXMjoNIdlyAHkmgFYOwpCkOBimx89aJC3nPkpqHFGMKyFAlRyOf n4Kc5nGxtvlx5OrhiD6tnFtpXyh+PgzZaonYIa+5xYj/5tbrf63ZNTBCiVrsXElTSPl9 /8IQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=SckrKGeb70N9IZUWgmauTBxdU9UMVK6bfHfwmMpZ4q8=; b=U7nWhn7zO0t5cac4BVX0hjZpSRqqyhDKoLQFXqpINR7ZAT9rtV7zZw5oy8Un+cweb1 /GcCQAFNTcq4VBNoF18OUUPNaLbwLdfcTsh0t75lUEShybCOBcNoW9ToMnSS7F/xHHs4 KnixrhqhZ3d2xGD3NZhP+/32oy+Barwq6gYjOwdWDQauhFfnDdLEuK0s5BSPiJEjvW39 kOf6o5WyV25VtoM4khLVg/ua6mGmJGWdukfARRbbwNOEH3rOlhIEmsTetiax7LgpCLQt Mrbz2zaStXSIgWY27S+MJv16tvtMmhEpGtBBh06YxecKTJkDrzQuAPfJhOzvrcJMAXTa fx5Q== X-Gm-Message-State: AOAM5302NDTPNnPS2nrqjLiFy3LmRuUjrQsX1dCtHtqe7J03SdQ4LW8o qUTERHM6KI359zi/WOfr/BAV9y5w X-Google-Smtp-Source: ABdhPJzkHZzShywYEf5YzOahvwdY6Eb37QwPF14tW7RP+Dkbp3og1Lkb6mAaiL9nFEn8KRajkRt7jA== X-Received: by 2002:ac8:7591:: with SMTP id s17mr884997qtq.17.1589318636492; Tue, 12 May 2020 14:23:56 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id d142sm12080372qkg.64.2020.05.12.14.23.55 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:23:56 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLNt93009939; Tue, 12 May 2020 21:23:55 GMT Subject: [PATCH v2 21/29] SUNRPC: Refactor svc_recvfrom() From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:23:55 -0400 Message-ID: <20200512212355.5826.95185.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org This function is not currently "generic" so remove the documenting comment and rename it appropriately. Its internals are converted to use bio_vecs for reading from the transport socket. In existing typical sunrpc uses of bio_vecs, the bio_vec array is allocated dynamically. Here, instead, an array of bio_vecs is added to svc_rqst. The lifetime of this array can be greater than one call to xpo_recvfrom(): - Multiple calls to xpo_recvfrom() might be needed to read an RPC message completely. - At some later point, rq_arg.bvecs will point to this array and it will carry the received message into svc_process(). I also expect that a future optimization will remove either the rq_vec or rq_pages array in favor of rq_bvec, thus conserving the size of struct svc_rqst. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc.h | 1 include/trace/events/sunrpc.h | 1 net/sunrpc/svcsock.c | 190 +++++++++++++++++++++-------------------- 3 files changed, 99 insertions(+), 93 deletions(-) diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h index fd390894a584..05da19a0516d 100644 --- a/include/linux/sunrpc/svc.h +++ b/include/linux/sunrpc/svc.h @@ -254,6 +254,7 @@ struct svc_rqst { struct page * *rq_page_end; /* one past the last page */ struct kvec rq_vec[RPCSVC_MAXPAGES]; /* generally useful.. */ + struct bio_vec rq_bvec[RPCSVC_MAXPAGES]; __be32 rq_xid; /* transmission id */ u32 rq_prog; /* program number */ diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index ec4ae34a1f84..bfea554bd91f 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1502,6 +1502,7 @@ DECLARE_EVENT_CLASS(svcsock_class, DEFINE_SVCSOCK_EVENT(udp_send); DEFINE_SVCSOCK_EVENT(tcp_send); +DEFINE_SVCSOCK_EVENT(tcp_recv); DEFINE_SVCSOCK_EVENT(data_ready); DEFINE_SVCSOCK_EVENT(write_space); diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 9c1eb13aa9b8..f482cfd0d49d 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -109,19 +109,15 @@ static void svc_reclassify_socket(struct socket *sock) } #endif -/* - * Release an skbuff after use - */ -static void svc_release_skb(struct svc_rqst *rqstp) +static void svc_tcp_release_rqst(struct svc_rqst *rqstp) { struct sk_buff *skb = rqstp->rq_xprt_ctxt; if (skb) { struct svc_sock *svsk = container_of(rqstp->rq_xprt, struct svc_sock, sk_xprt); - rqstp->rq_xprt_ctxt = NULL; - dprintk("svc: service %p, releasing skb %p\n", rqstp, skb); + rqstp->rq_xprt_ctxt = NULL; skb_free_datagram_locked(svsk->sk_sk, skb); } } @@ -219,34 +215,60 @@ static int svc_one_sock_name(struct svc_sock *svsk, char *buf, int remaining) return len; } +#if ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE +static void svc_flush_bvec(const struct bio_vec *bvec, size_t size, size_t seek) +{ + struct bvec_iter bi = { + .bi_size = size, + }; + struct bio_vec bv; + + bvec_iter_advance(bvec, &bi, seek & PAGE_MASK); + for_each_bvec(bv, bvec, bi, bi) + flush_dcache_page(bv.bv_page); +} +#else +static inline void svc_flush_bvec(const struct bio_vec *bvec, size_t size, + size_t seek) +{ +} +#endif + /* - * Generic recvfrom routine. + * Read from @rqstp's transport socket. The incoming message fills whole + * pages in @rqstp's rq_pages array until the last page of the message + * has been received into a partial page. */ -static ssize_t svc_recvfrom(struct svc_rqst *rqstp, struct kvec *iov, - unsigned int nr, size_t buflen, unsigned int base) +static ssize_t svc_tcp_read_msg(struct svc_rqst *rqstp, size_t buflen, + size_t seek) { struct svc_sock *svsk = container_of(rqstp->rq_xprt, struct svc_sock, sk_xprt); + struct bio_vec *bvec = rqstp->rq_bvec; struct msghdr msg = { NULL }; + unsigned int i; ssize_t len; + size_t t; - rqstp->rq_xprt_hlen = 0; + for (i = 0, t = 0; t < buflen; i++, t += PAGE_SIZE) { + bvec[i].bv_page = rqstp->rq_pages[i]; + bvec[i].bv_len = PAGE_SIZE; + bvec[i].bv_offset = 0; + } + rqstp->rq_respages = &rqstp->rq_pages[i]; + rqstp->rq_next_page = rqstp->rq_respages + 1; - clear_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); - iov_iter_kvec(&msg.msg_iter, READ, iov, nr, buflen); - if (base != 0) { - iov_iter_advance(&msg.msg_iter, base); - buflen -= base; + iov_iter_bvec(&msg.msg_iter, READ, bvec, i, buflen); + if (seek) { + iov_iter_advance(&msg.msg_iter, seek); + buflen -= seek; } len = sock_recvmsg(svsk->sk_sock, &msg, MSG_DONTWAIT); - /* If we read a full record, then assume there may be more - * data to read (stream based sockets only!) - */ - if (len == buflen) - set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); + if (len < 0) + return len; - dprintk("svc: socket %p recvfrom(%p, %zu) = %zd\n", - svsk, iov[0].iov_base, iov[0].iov_len, len); + svc_flush_bvec(bvec, len, seek); + trace_svcsock_tcp_recv(&svsk->sk_xprt, len); return len; } @@ -773,18 +795,18 @@ static struct svc_xprt *svc_tcp_accept(struct svc_xprt *xprt) return NULL; } -static unsigned int svc_tcp_restore_pages(struct svc_sock *svsk, struct svc_rqst *rqstp) +static size_t svc_tcp_restore_pages(struct svc_sock *svsk, + struct svc_rqst *rqstp) { - unsigned int i, len, npages; + size_t len = svsk->sk_datalen; + unsigned int i, npages; - if (svsk->sk_datalen == 0) - return 0; - len = svsk->sk_datalen; + if (!len) + return len; npages = (len + PAGE_SIZE - 1) >> PAGE_SHIFT; for (i = 0; i < npages; i++) { if (rqstp->rq_pages[i] != NULL) put_page(rqstp->rq_pages[i]); - BUG_ON(svsk->sk_pages[i] == NULL); rqstp->rq_pages[i] = svsk->sk_pages[i]; svsk->sk_pages[i] = NULL; } @@ -915,49 +937,40 @@ static int receive_cb_reply(struct svc_sock *svsk, struct svc_rqst *rqstp) return -EAGAIN; } -static int copy_pages_to_kvecs(struct kvec *vec, struct page **pages, int len) -{ - int i = 0; - int t = 0; - - while (t < len) { - vec[i].iov_base = page_address(pages[i]); - vec[i].iov_len = PAGE_SIZE; - i++; - t += PAGE_SIZE; - } - return i; -} - static void svc_tcp_fragment_received(struct svc_sock *svsk) { /* If we have more data, signal svc_xprt_enqueue() to try again */ - dprintk("svc: TCP %s record (%d bytes)\n", - svc_sock_final_rec(svsk) ? "final" : "nonfinal", - svc_sock_reclen(svsk)); svsk->sk_tcplen = 0; svsk->sk_marker = xdr_zero; } -/* - * Receive data from a TCP socket. +/** + * svc_tcp_recvfrom - Receive data from a TCP socket + * @rqstp: request structure into which to receive an RPC Call + * + * Called in a loop when XPT_DATA has been set. + * + * Read the 4-byte stream record marker, then use the record length + * in that marker to set up exactly the resources needed to receive + * the next RPC message into @rqstp. + * + * Returns: + * On success, the number of bytes in a received RPC Call, or + * %0 if a complete RPC Call message was not ready to return + * + * The zero return case handles partial receives and callback Replies. + * The state of a partial receive is preserved in the svc_sock for + * the next call to svc_tcp_recvfrom. */ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) { struct svc_sock *svsk = container_of(rqstp->rq_xprt, struct svc_sock, sk_xprt); struct svc_serv *serv = svsk->sk_xprt.xpt_server; - int len; - struct kvec *vec; - unsigned int want, base; + size_t want, base; + ssize_t len; __be32 *p; __be32 calldir; - int pnum; - - dprintk("svc: tcp_recv %p data %d conn %d close %d\n", - svsk, test_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags), - test_bit(XPT_CONN, &svsk->sk_xprt.xpt_flags), - test_bit(XPT_CLOSE, &svsk->sk_xprt.xpt_flags)); clear_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); len = svc_tcp_read_marker(svsk, rqstp); @@ -966,38 +979,19 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) base = svc_tcp_restore_pages(svsk, rqstp); want = len - (svsk->sk_tcplen - sizeof(rpc_fraghdr)); - - vec = rqstp->rq_vec; - - pnum = copy_pages_to_kvecs(&vec[0], &rqstp->rq_pages[0], base + want); - - rqstp->rq_respages = &rqstp->rq_pages[pnum]; - rqstp->rq_next_page = rqstp->rq_respages + 1; - - /* Now receive data */ - len = svc_recvfrom(rqstp, vec, pnum, base + want, base); + len = svc_tcp_read_msg(rqstp, base + want, base); if (len >= 0) { svsk->sk_tcplen += len; svsk->sk_datalen += len; } - if (len != want || !svc_sock_final_rec(svsk)) { - svc_tcp_save_pages(svsk, rqstp); - if (len < 0 && len != -EAGAIN) - goto err_delete; - if (len == want) - svc_tcp_fragment_received(svsk); - else - dprintk("svc: incomplete TCP record (%d of %d)\n", - (int)(svsk->sk_tcplen - sizeof(rpc_fraghdr)), - svc_sock_reclen(svsk)); - goto err_noclose; - } - - if (svsk->sk_datalen < 8) { - svsk->sk_datalen = 0; - goto err_delete; /* client is nuts. */ - } + if (len != want || !svc_sock_final_rec(svsk)) + goto err_incomplete; + if (svsk->sk_datalen < 8) + goto err_nuts; + /* + * At this point, a full stream RPC record has been read. + */ rqstp->rq_arg.len = svsk->sk_datalen; rqstp->rq_arg.page_base = 0; if (rqstp->rq_arg.len <= rqstp->rq_arg.head[0].iov_len) { @@ -1005,7 +999,6 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) rqstp->rq_arg.page_len = 0; } else rqstp->rq_arg.page_len = rqstp->rq_arg.len - rqstp->rq_arg.head[0].iov_len; - rqstp->rq_xprt_ctxt = NULL; rqstp->rq_prot = IPPROTO_TCP; if (test_bit(XPT_LOCAL, &svsk->sk_xprt.xpt_flags)) @@ -1018,7 +1011,13 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) if (calldir) len = receive_cb_reply(svsk, rqstp); - /* Reset TCP read info */ + /* + * Force another call to svc_tcp_recvfrom to check for + * more data waiting on the socket. + */ + set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); + + /* Reset TCP read info to prepare for the next record */ svsk->sk_datalen = 0; svc_tcp_fragment_received(svsk); @@ -1028,20 +1027,25 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) svc_xprt_copy_addrs(rqstp, &svsk->sk_xprt); if (serv->sv_stats) serv->sv_stats->nettcpcnt++; - return rqstp->rq_arg.len; +err_incomplete: + svc_tcp_save_pages(svsk, rqstp); + if (len < 0 && len != -EAGAIN) + goto err_delete; + if (len == want) + svc_tcp_fragment_received(svsk); + return 0; error: if (len != -EAGAIN) goto err_delete; - dprintk("RPC: TCP recvfrom got EAGAIN\n"); return 0; +err_nuts: + svsk->sk_datalen = 0; err_delete: - printk(KERN_NOTICE "%s: recvfrom returned errno %d\n", - svsk->sk_xprt.xpt_server->sv_name, -len); + pr_notice("%s: recvfrom returns %zd\n", serv->sv_name, len); set_bit(XPT_CLOSE, &svsk->sk_xprt.xpt_flags); -err_noclose: - return 0; /* record not complete */ + return 0; } /** @@ -1066,7 +1070,7 @@ static int svc_tcp_sendto(struct svc_rqst *rqstp) unsigned int uninitialized_var(sent); int err; - svc_release_skb(rqstp); + svc_tcp_release_rqst(rqstp); mutex_lock(&xprt->xpt_mutex); if (svc_xprt_is_dead(xprt)) @@ -1106,7 +1110,7 @@ static const struct svc_xprt_ops svc_tcp_ops = { .xpo_recvfrom = svc_tcp_recvfrom, .xpo_sendto = svc_tcp_sendto, .xpo_read_payload = svc_sock_read_payload, - .xpo_release_rqst = svc_release_skb, + .xpo_release_rqst = svc_tcp_release_rqst, .xpo_detach = svc_tcp_sock_detach, .xpo_free = svc_sock_free, .xpo_has_wspace = svc_tcp_has_wspace, From patchwork Tue May 12 21:24:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544153 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CF1FB15E6 for ; Tue, 12 May 2020 21:24:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AF05420731 for ; Tue, 12 May 2020 21:24:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="qGA/L72P" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731324AbgELVYG (ORCPT ); Tue, 12 May 2020 17:24:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731308AbgELVYE (ORCPT ); Tue, 12 May 2020 17:24:04 -0400 Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AE60DC061A0C; Tue, 12 May 2020 14:24:02 -0700 (PDT) Received: by mail-qt1-x841.google.com with SMTP id c24so6016437qtw.7; Tue, 12 May 2020 14:24:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=x/iQsa+RorRGYCjZvcK6WkMCCdDeHvIHsOUUPTkhWdM=; b=qGA/L72PrRXtFheXpDJ8jEdyV0AEJxZBhIiyn4DkzOrA2wytnQ3VjszODCgDzHGQIF F3ezBbpXp7C6LlesHC2ka6SgMtDkkSE85kJCoWh/hpe2wCGV0sM96qZ1HWWYTu8pnTDA kSi/aGsXYMcPLRY7rsi7dwQQCp+Lq6imdU72iyS5ZLuBcGw6YeOkSrVYvfex8R1gYiMW hf58BJdX6WW6SPK9B/uNqQetP7EYySktiofk+2Lobi6vUeYf1Mk5MD6Lzoj2EwdRThDg SqWhYxJ0KD/Lt2XUtu+lES3wmMUBJt0p/7WL1358JrLCIvTsZOkqqjkfA/7EEWx2phdD VwFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=x/iQsa+RorRGYCjZvcK6WkMCCdDeHvIHsOUUPTkhWdM=; b=SoFqH7IqJ6keJrUMDWJ3Hp/GrH+/CUlx4ovBVnOlXAbIMmjMTL7NlwWikPE2JHvmGY DjUx3gPfRfN+5IBgNLmP4cEg79x7v+lgdX0WLV7tW4f+LvP5rDBv9acdzn88tAOmvDSe s+l8P2k2O2OkqpCtHL25P9jSJkU+EXqKyXZ9Alu08zsxPmVVUQsOm07/8njkRyr2nCoJ B+pSq9lbO3XyhvADQsGlMNB369ZwrFlRrSPI4ZAr7C/2fyNud1cGZ7mwWCXLd4EWVwio JyeZ4xIHg+jT5vZl2CjE2xmDcncy5DQd22ccvBbpFkAi3xYU4ZAI6pGBFE1J5xh+A6Hl 8nkw== X-Gm-Message-State: AGi0PuYxj6DDBeJSTour9MM3/AR9Ba8kpkXleFTSHbSXsErUASE9i/Fk scn8FFJrZaVAUl5p+cF8EHKs/x0y X-Google-Smtp-Source: APiQypKk1McrchTakwlf0BIZlmyt1Xwa8htv5HZ1HmYTmIBr+ohNjG44rOynHs4zKET3Y/O9Z5FUeQ== X-Received: by 2002:ac8:2db0:: with SMTP id p45mr21619605qta.354.1589318641756; Tue, 12 May 2020 14:24:01 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id a1sm13317115qtj.65.2020.05.12.14.24.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:24:01 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLO0Oe009942; Tue, 12 May 2020 21:24:00 GMT Subject: [PATCH v2 22/29] SUNRPC: Restructure svc_udp_recvfrom() From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:24:00 -0400 Message-ID: <20200512212400.5826.94699.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up. At this point, we are not ready yet to support bio_vecs in the UDP transport implementation. However, we can clean up svc_udp_recvfrom() to match the tracing and straight-lining recently changes made in svc_tcp_recvfrom(). Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 1 + net/sunrpc/svcsock.c | 68 +++++++++++++++++++++++------------------ 2 files changed, 39 insertions(+), 30 deletions(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index bfea554bd91f..6e32c4e16f7d 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1501,6 +1501,7 @@ DECLARE_EVENT_CLASS(svcsock_class, TP_ARGS(xprt, result)) DEFINE_SVCSOCK_EVENT(udp_send); +DEFINE_SVCSOCK_EVENT(udp_recv); DEFINE_SVCSOCK_EVENT(tcp_send); DEFINE_SVCSOCK_EVENT(tcp_recv); DEFINE_SVCSOCK_EVENT(data_ready); diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index f482cfd0d49d..d31a807c2c39 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -122,14 +122,12 @@ static void svc_tcp_release_rqst(struct svc_rqst *rqstp) } } -static void svc_release_udp_skb(struct svc_rqst *rqstp) +static void svc_udp_release_rqst(struct svc_rqst *rqstp) { struct sk_buff *skb = rqstp->rq_xprt_ctxt; if (skb) { rqstp->rq_xprt_ctxt = NULL; - - dprintk("svc: service %p, releasing skb %p\n", rqstp, skb); consume_skb(skb); } } @@ -409,8 +407,15 @@ static int svc_udp_get_dest_address(struct svc_rqst *rqstp, return 0; } -/* - * Receive a datagram from a UDP socket. +/** + * svc_udp_recvfrom - Receive a datagram from a UDP socket. + * @rqstp: request structure into which to receive an RPC Call + * + * Called in a loop when XPT_DATA has been set. + * + * Returns: + * On success, the number of bytes in a received RPC Call, or + * %0 if a complete RPC Call message was not ready to return */ static int svc_udp_recvfrom(struct svc_rqst *rqstp) { @@ -444,20 +449,14 @@ static int svc_udp_recvfrom(struct svc_rqst *rqstp) svc_sock_setbufsize(svsk, serv->sv_nrthreads + 3); clear_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); - skb = NULL; err = kernel_recvmsg(svsk->sk_sock, &msg, NULL, 0, 0, MSG_PEEK | MSG_DONTWAIT); - if (err >= 0) - skb = skb_recv_udp(svsk->sk_sk, 0, 1, &err); - - if (skb == NULL) { - if (err != -EAGAIN) { - /* possibly an icmp error */ - dprintk("svc: recvfrom returned error %d\n", -err); - set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); - } - return 0; - } + if (err < 0) + goto out_recv_err; + skb = skb_recv_udp(svsk->sk_sk, 0, 1, &err); + if (!skb) + goto out_recv_err; + len = svc_addr_len(svc_addr(rqstp)); rqstp->rq_addrlen = len; if (skb->tstamp == 0) { @@ -468,26 +467,21 @@ static int svc_udp_recvfrom(struct svc_rqst *rqstp) sock_write_timestamp(svsk->sk_sk, skb->tstamp); set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); /* there may be more data... */ - len = skb->len; + len = skb->len; rqstp->rq_arg.len = len; + trace_svcsock_udp_recv(&svsk->sk_xprt, len); rqstp->rq_prot = IPPROTO_UDP; - if (!svc_udp_get_dest_address(rqstp, cmh)) { - net_warn_ratelimited("svc: received unknown control message %d/%d; dropping RPC reply datagram\n", - cmh->cmsg_level, cmh->cmsg_type); - goto out_free; - } + if (!svc_udp_get_dest_address(rqstp, cmh)) + goto out_cmsg_err; rqstp->rq_daddrlen = svc_addr_len(svc_daddr(rqstp)); if (skb_is_nonlinear(skb)) { /* we have to copy */ local_bh_disable(); - if (csum_partial_copy_to_xdr(&rqstp->rq_arg, skb)) { - local_bh_enable(); - /* checksum error */ - goto out_free; - } + if (csum_partial_copy_to_xdr(&rqstp->rq_arg, skb)) + goto out_bh_enable; local_bh_enable(); consume_skb(skb); } else { @@ -515,6 +509,20 @@ static int svc_udp_recvfrom(struct svc_rqst *rqstp) serv->sv_stats->netudpcnt++; return len; + +out_recv_err: + if (err != -EAGAIN) { + /* possibly an icmp error */ + set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); + } + trace_svcsock_udp_recv(&svsk->sk_xprt, err); + return 0; +out_cmsg_err: + net_warn_ratelimited("svc: received unknown control message %d/%d; dropping RPC reply datagram\n", + cmh->cmsg_level, cmh->cmsg_type); + goto out_free; +out_bh_enable: + local_bh_enable(); out_free: kfree_skb(skb); return 0; @@ -548,7 +556,7 @@ static int svc_udp_sendto(struct svc_rqst *rqstp) unsigned int uninitialized_var(sent); int err; - svc_release_udp_skb(rqstp); + svc_udp_release_rqst(rqstp); svc_set_cmsg_data(rqstp, cmh); @@ -617,7 +625,7 @@ static const struct svc_xprt_ops svc_udp_ops = { .xpo_recvfrom = svc_udp_recvfrom, .xpo_sendto = svc_udp_sendto, .xpo_read_payload = svc_sock_read_payload, - .xpo_release_rqst = svc_release_udp_skb, + .xpo_release_rqst = svc_udp_release_rqst, .xpo_detach = svc_sock_detach, .xpo_free = svc_sock_free, .xpo_has_wspace = svc_udp_has_wspace, From patchwork Tue May 12 21:24:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544157 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D199C139A for ; Tue, 12 May 2020 21:24:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B9B0F20731 for ; Tue, 12 May 2020 21:24:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="F6Bsl1lX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731331AbgELVYJ (ORCPT ); Tue, 12 May 2020 17:24:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731308AbgELVYI (ORCPT ); Tue, 12 May 2020 17:24:08 -0400 Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BEF27C061A0C; Tue, 12 May 2020 14:24:08 -0700 (PDT) Received: by mail-qt1-x841.google.com with SMTP id h26so12440630qtu.8; Tue, 12 May 2020 14:24:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=ygVfFXoXHsJFejgiuBymCQ4HlLT1pPdMAMnaNdaa1JU=; b=F6Bsl1lXcfP89qEgyoL4CCzNxA2aXC7ac73lZD6C1VD0GpIRO2gYlBGp+u5nHclbjk q7gz5Jj0+H3vZKQHcgMKt8qvH7y6i8qAsNbOWscVEKXpelsvmR1zLra5na9mUG+R8Ljy 29QhSMa4XOjfV3A362IRo5TkDT4/30+Rq3doGuaHi/5JN6URgxKVVc9Y1TWNjC+fh63u AGAWiUa55JkHuhJofmWIQvy7MRiQEcYs0egz3wARLtz+D+HdYo+f2hMHM3OfCttqe9KQ R6GLNuSX4cqA6WaI530fNp4Uo+AXDg2XSaxYFGlWvIilZbNY1j0YM2IPFTOmV3IN89BL exew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=ygVfFXoXHsJFejgiuBymCQ4HlLT1pPdMAMnaNdaa1JU=; b=emBek9x5zU7iUNQBzMMTZtu5cjs4fpyjTiZB6Qz2ecAaUFlkPWPoXpmBEeY1neeb2j /QBKRo+4D/14UbB7gj6345ICSfr55fFFeKtGWtIyt0S0/gCRoks2jPGpD6+HVdt7TJMm tiS1cDatCAJSVVszzkuh6/QziwZfrR0/f6J/KeLUZ5KTH4zxDAismrwCwQCdYoiKMVwk dlO+y2WVVf5bDOg6X2w8vAhJaEjL5snD5Fzz8XJKOgrJphiQLRaGlrpReq5EiCH8SVKe +v8el9hNLx6z6/AWHGdE0TkEqbqfNXZnNHeHCRLRBycRJqBOynqjRITvUyxu4uVEvALy SA8A== X-Gm-Message-State: AGi0PuYKWN7+QPsT2ulrK9h13Zzdm9L7gJeyIwgwI+fe6KTkEVsD2Kai xMhSnr1TXadxTZUuJ9Q3hmbFrM2I X-Google-Smtp-Source: APiQypL0nb+3Aexi6rST0nBEty1rjjr13v51Tya82/1TmN4SJxI0pHOtaV7fN7VveJX2UJn6NPoMBQ== X-Received: by 2002:ac8:6b44:: with SMTP id x4mr13273225qts.353.1589318647918; Tue, 12 May 2020 14:24:07 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id y18sm13924352qty.41.2020.05.12.14.24.07 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:24:07 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLO5Cv009945; Tue, 12 May 2020 21:24:05 GMT Subject: [PATCH v2 23/29] SUNRPC: svc_show_status() macro should have enum definitions From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:24:05 -0400 Message-ID: <20200512212405.5826.89968.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: Add missing TRACE_DEFINE_ENUMs in include/trace/events/sunrpc.h Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index 6e32c4e16f7d..6fa08c36c6ca 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1057,6 +1057,17 @@ TRACE_EVENT(svc_recv, show_rqstp_flags(__entry->flags)) ); +TRACE_DEFINE_ENUM(SVC_GARBAGE); +TRACE_DEFINE_ENUM(SVC_SYSERR); +TRACE_DEFINE_ENUM(SVC_VALID); +TRACE_DEFINE_ENUM(SVC_NEGATIVE); +TRACE_DEFINE_ENUM(SVC_OK); +TRACE_DEFINE_ENUM(SVC_DROP); +TRACE_DEFINE_ENUM(SVC_CLOSE); +TRACE_DEFINE_ENUM(SVC_DENIED); +TRACE_DEFINE_ENUM(SVC_PENDING); +TRACE_DEFINE_ENUM(SVC_COMPLETE); + #define svc_show_status(status) \ __print_symbolic(status, \ { SVC_GARBAGE, "SVC_GARBAGE" }, \ From patchwork Tue May 12 21:24:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544161 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 88C9F139A for ; Tue, 12 May 2020 21:24:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6BA3120673 for ; Tue, 12 May 2020 21:24:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="tKVBeBR2" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731333AbgELVYO (ORCPT ); Tue, 12 May 2020 17:24:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46320 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731308AbgELVYN (ORCPT ); Tue, 12 May 2020 17:24:13 -0400 Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5D78FC061A0C; Tue, 12 May 2020 14:24:13 -0700 (PDT) Received: by mail-qt1-x841.google.com with SMTP id c24so6016930qtw.7; Tue, 12 May 2020 14:24:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=lJyCmn+wrw9fOiuFD6p/GmVDOI8LWar9S6pYuPj9wAU=; b=tKVBeBR2OS9cNgqUcofkku1Db6+FrGUWwXRvQmy7kffHfn3DbnI5Pp1IQceC094oVM oHNuGPww8T2zbwO93t94DKop2+Z9+MDSJAKHjG7TFGcS87Gro0pzZQLRYvIVh5qo6u3c R04LtJ+I2reEG0q3xD3PqzIM28EMMp/GPEumAKoV+qcIl5JFb5fAjk9JKS2As0gT/D0z F1bVKaWaZ4DeQTqh5yGNcyMVVOtYbkTpzalOJZOa0XGifE+A3mKcriKPTQCMYu3Q2AIs nX59QZqfAQ9UV3l9zBqDmHSaJjEdFgn7THFVd4ljGC+YusGDCcBrn6HWtktHbNrpUZby 4G5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=lJyCmn+wrw9fOiuFD6p/GmVDOI8LWar9S6pYuPj9wAU=; b=PYSs8qlq4mE2k7/wWaAKcfjfioO0Jd4lhR3oRMs++h1wj2JzmmM86W3M9keclmPl42 znXfZbcMPvpmrKYHFJTzhxnhVNgEYc8wA7V2QoKpN1PjTcN1JMtYj9O3rhBzPNVkcJek JuPvTuHOgxDmqxeWj0C/wOQKRCjFckjnK9iCXBTWIhh+DZsJ9FntjdWi0zsFjCU6So1G 6EHS7G/cb7rdJha9qa083OaB0VM9uE1zOTwUKIC3MK/lTDfqiUS5QOkbJ4cN0zrjpq5r wUGlBFFG6sxeZVPivF2nos1QNlSfNG/wxhdo3dXD77DLf2CVdPAWYE60RD2lSX4Dn6+j nWQw== X-Gm-Message-State: AGi0PualdyJAuviTzZnBkuATNOt1hVjtkgmC7lIo9SXbDpQthaHRBal7 OvMjVp5E/lODTogGOsU37FWxSLNm X-Google-Smtp-Source: APiQypLR0RmnSR3/DuGlINlacyDQkEFa7Qlybnld98gMiWO18WLSXS1hh0PfTfXREbG2DB2R2jceiw== X-Received: by 2002:ac8:6c23:: with SMTP id k3mr24343231qtu.107.1589318652110; Tue, 12 May 2020 14:24:12 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id v4sm12055938qkv.43.2020.05.12.14.24.11 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:24:11 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLOALD009948; Tue, 12 May 2020 21:24:10 GMT Subject: [PATCH v2 24/29] NFSD: Add tracepoints to NFSD's duplicate reply cache From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:24:10 -0400 Message-ID: <20200512212410.5826.28992.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Try to capture DRC failures. Two additional clean-ups: - Introduce Doxygen-style comments for the main entry points - Remove a dprintk that fires for an allocation failure. This was the only dprintk in the REPCACHE class. Reported-by: kbuild test robot [ cel: force typecast for display of checksum values ] Signed-off-by: Chuck Lever --- fs/nfsd/nfscache.c | 57 +++++++++++++++++++++++++++++++------------------- fs/nfsd/trace.h | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 94 insertions(+), 22 deletions(-) diff --git a/fs/nfsd/nfscache.c b/fs/nfsd/nfscache.c index 96352ab7bd81..945d2f5e760e 100644 --- a/fs/nfsd/nfscache.c +++ b/fs/nfsd/nfscache.c @@ -20,8 +20,7 @@ #include "nfsd.h" #include "cache.h" - -#define NFSDDBG_FACILITY NFSDDBG_REPCACHE +#include "trace.h" /* * We use this value to determine the number of hash buckets from the max @@ -323,8 +322,10 @@ nfsd_cache_key_cmp(const struct svc_cacherep *key, const struct svc_cacherep *rp, struct nfsd_net *nn) { if (key->c_key.k_xid == rp->c_key.k_xid && - key->c_key.k_csum != rp->c_key.k_csum) + key->c_key.k_csum != rp->c_key.k_csum) { ++nn->payload_misses; + trace_nfsd_drc_mismatch(nn, key, rp); + } return memcmp(&key->c_key, &rp->c_key, sizeof(key->c_key)); } @@ -377,15 +378,22 @@ nfsd_cache_insert(struct nfsd_drc_bucket *b, struct svc_cacherep *key, return ret; } -/* +/** + * nfsd_cache_lookup - Find an entry in the duplicate reply cache + * @rqstp: Incoming Call to find + * * Try to find an entry matching the current call in the cache. When none * is found, we try to grab the oldest expired entry off the LRU list. If * a suitable one isn't there, then drop the cache_lock and allocate a * new one, then search again in case one got inserted while this thread * didn't hold the lock. + * + * Return values: + * %RC_DOIT: Process the request normally + * %RC_REPLY: Reply from cache + * %RC_DROPIT: Do not process the request further */ -int -nfsd_cache_lookup(struct svc_rqst *rqstp) +int nfsd_cache_lookup(struct svc_rqst *rqstp) { struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id); struct svc_cacherep *rp, *found; @@ -399,7 +407,7 @@ nfsd_cache_lookup(struct svc_rqst *rqstp) rqstp->rq_cacherep = NULL; if (type == RC_NOCACHE) { nfsdstats.rcnocache++; - return rtn; + goto out; } csum = nfsd_cache_csum(rqstp); @@ -409,10 +417,8 @@ nfsd_cache_lookup(struct svc_rqst *rqstp) * preallocate an entry. */ rp = nfsd_reply_cache_alloc(rqstp, csum, nn); - if (!rp) { - dprintk("nfsd: unable to allocate DRC entry!\n"); - return rtn; - } + if (!rp) + goto out; spin_lock(&b->cache_lock); found = nfsd_cache_insert(b, rp, nn); @@ -431,8 +437,10 @@ nfsd_cache_lookup(struct svc_rqst *rqstp) /* go ahead and prune the cache */ prune_bucket(b, nn); - out: + +out_unlock: spin_unlock(&b->cache_lock); +out: return rtn; found_entry: @@ -442,13 +450,13 @@ nfsd_cache_lookup(struct svc_rqst *rqstp) /* Request being processed */ if (rp->c_state == RC_INPROG) - goto out; + goto out_trace; /* From the hall of fame of impractical attacks: * Is this a user who tries to snoop on the cache? */ rtn = RC_DOIT; if (!test_bit(RQ_SECURE, &rqstp->rq_flags) && rp->c_secure) - goto out; + goto out_trace; /* Compose RPC reply header */ switch (rp->c_type) { @@ -460,7 +468,7 @@ nfsd_cache_lookup(struct svc_rqst *rqstp) break; case RC_REPLBUFF: if (!nfsd_cache_append(rqstp, &rp->c_replvec)) - goto out; /* should not happen */ + goto out_unlock; /* should not happen */ rtn = RC_REPLY; break; default: @@ -468,13 +476,19 @@ nfsd_cache_lookup(struct svc_rqst *rqstp) nfsd_reply_cache_free_locked(b, rp, nn); } - goto out; +out_trace: + trace_nfsd_drc_found(nn, rqstp, rtn); + goto out_unlock; } -/* - * Update a cache entry. This is called from nfsd_dispatch when - * the procedure has been executed and the complete reply is in - * rqstp->rq_res. +/** + * nfsd_cache_update - Update an entry in the duplicate reply cache. + * @rqstp: svc_rqst with a finished Reply + * @cachetype: which cache to update + * @statp: Reply's status code + * + * This is called from nfsd_dispatch when the procedure has been + * executed and the complete reply is in rqstp->rq_res. * * We're copying around data here rather than swapping buffers because * the toplevel loop requires max-sized buffers, which would be a waste @@ -487,8 +501,7 @@ nfsd_cache_lookup(struct svc_rqst *rqstp) * nfsd failed to encode a reply that otherwise would have been cached. * In this case, nfsd_cache_update is called with statp == NULL. */ -void -nfsd_cache_update(struct svc_rqst *rqstp, int cachetype, __be32 *statp) +void nfsd_cache_update(struct svc_rqst *rqstp, int cachetype, __be32 *statp) { struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id); struct svc_cacherep *rp = rqstp->rq_cacherep; diff --git a/fs/nfsd/trace.h b/fs/nfsd/trace.h index 78c574251c60..371dc198d28e 100644 --- a/fs/nfsd/trace.h +++ b/fs/nfsd/trace.h @@ -432,6 +432,65 @@ TRACE_EVENT(nfsd_file_fsnotify_handle_event, __entry->nlink, __entry->mode, __entry->mask) ); +#include "cache.h" + +TRACE_DEFINE_ENUM(RC_DROPIT); +TRACE_DEFINE_ENUM(RC_REPLY); +TRACE_DEFINE_ENUM(RC_DOIT); + +#define show_drc_retval(x) \ + __print_symbolic(x, \ + { RC_DROPIT, "DROPIT" }, \ + { RC_REPLY, "REPLY" }, \ + { RC_DOIT, "DOIT" }) + +TRACE_EVENT(nfsd_drc_found, + TP_PROTO( + const struct nfsd_net *nn, + const struct svc_rqst *rqstp, + int result + ), + TP_ARGS(nn, rqstp, result), + TP_STRUCT__entry( + __field(unsigned long long, boot_time) + __field(unsigned long, result) + __field(u32, xid) + ), + TP_fast_assign( + __entry->boot_time = nn->boot_time; + __entry->result = result; + __entry->xid = be32_to_cpu(rqstp->rq_xid); + ), + TP_printk("boot_time=%16llx xid=0x%08x result=%s", + __entry->boot_time, __entry->xid, + show_drc_retval(__entry->result)) + +); + +TRACE_EVENT(nfsd_drc_mismatch, + TP_PROTO( + const struct nfsd_net *nn, + const struct svc_cacherep *key, + const struct svc_cacherep *rp + ), + TP_ARGS(nn, key, rp), + TP_STRUCT__entry( + __field(unsigned long long, boot_time) + __field(u32, xid) + __field(u32, cached) + __field(u32, ingress) + ), + TP_fast_assign( + __entry->boot_time = nn->boot_time; + __entry->xid = be32_to_cpu(key->c_key.k_xid); + __entry->cached = (__force u32)key->c_key.k_csum; + __entry->ingress = (__force u32)rp->c_key.k_csum; + ), + TP_printk("boot_time=%16llx xid=0x%08x cached-csum=0x%08x ingress-csum=0x%08x", + __entry->boot_time, __entry->xid, __entry->cached, + __entry->ingress) +); + #endif /* _NFSD_TRACE_H */ #undef TRACE_INCLUDE_PATH From patchwork Tue May 12 21:24:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544165 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 83C7315E6 for ; Tue, 12 May 2020 21:24:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 640A620731 for ; Tue, 12 May 2020 21:24:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YUkSULYU" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731308AbgELVYU (ORCPT ); Tue, 12 May 2020 17:24:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46338 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731334AbgELVYT (ORCPT ); Tue, 12 May 2020 17:24:19 -0400 Received: from mail-qt1-x844.google.com (mail-qt1-x844.google.com [IPv6:2607:f8b0:4864:20::844]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66BBDC061A0C; Tue, 12 May 2020 14:24:18 -0700 (PDT) Received: by mail-qt1-x844.google.com with SMTP id b1so11674991qtt.1; Tue, 12 May 2020 14:24:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=2K2KRK3RypI38A4j3fLeziuEON8iXBtWzncIU09Ocs0=; b=YUkSULYU11zfI/BLfKqaX6uuciN0gadMrIr5n1faiRB9uuiE03vYzJNejehdM16UcH DMTpGVtydfkhdeoXIc7fQQOGB6uqCp9eVXYAYNB7VerBWM1MD9hGM1mI+LTDCkvqpFso 5UXckU4Is30ljqy1m9Z1xj2GBih+n6Mx91LHOcWu6QxGylTpAyrrf24+tQ3tq6l6Tbvu vO62noo7fs0D63myDLJJ6kS1wbDWs5JH/WbaOyhoVhS+bK5uErw9PVzB/22Yp3jzHj5W Pc2zU8oWQmJV7hnHmMEgGXVnErj1YgXGPG0wZyMYPeLUgfkjqII5J0UQk2WrO7mAByS1 3Y6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=2K2KRK3RypI38A4j3fLeziuEON8iXBtWzncIU09Ocs0=; b=qE/XUUZfSCycXQ4S6sJCOpsb93W+D37oSwZkSZCRR0Tj68HMtvXm9BZUzARZZxR52H szH/swdF3Vm0QRSL30anjlRKGO3iAhM1yxw81VWnW1F4q4OJaTcViqVyi03FbqZl9bSr fjb0i0tnVQ4253aBQQwi/KLcKOxBInYPqyq7X4HjpwfpUv6qUf1GZ++33TDrrWhYPZg1 pv/83vh459KC43jgWH4VChwQy0v2i8scir4q8vkZ4lXN5nfJXz7FhIrpCvvEmhQPT++M JUG645HilPZH5W2v2BuxNazuBplAVFxCpiP8xT9HEIcEVHG5jlAXrPgwNNNsykP75NcF eXRQ== X-Gm-Message-State: AGi0PuYOAoxJMGPsLBEYrD38vSozQXERkC1tdeui81+dzxDQbFq/RAk/ U2DnSpCJXYXtdw3l3cALspwx5wWp X-Google-Smtp-Source: APiQypJp1boEEm2K1Kyfy9SCoaCOObESSuDvF3KzztAftP9plgFR+zXAA6MuJdZN0F+czTsnfc9Adw== X-Received: by 2002:ac8:6b55:: with SMTP id x21mr15087462qts.75.1589318657155; Tue, 12 May 2020 14:24:17 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id e3sm686732qtj.25.2020.05.12.14.24.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:24:16 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLOFds009951; Tue, 12 May 2020 21:24:15 GMT Subject: [PATCH v2 25/29] NFSD: Add tracepoints to the NFSD state management code From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:24:15 -0400 Message-ID: <20200512212415.5826.20435.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Capture obvious events and replace dprintk() call sites. Introduce infrastructure so that adding more tracepoints in this code later is simplified. Signed-off-by: Chuck Lever --- fs/nfsd/nfs4state.c | 52 +++++++------------- fs/nfsd/state.h | 7 --- fs/nfsd/trace.h | 133 +++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 152 insertions(+), 40 deletions(-) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index c107caa56525..04d80f9e2f91 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -51,6 +51,7 @@ #include "netns.h" #include "pnfs.h" #include "filecache.h" +#include "trace.h" #define NFSDDBG_FACILITY NFSDDBG_PROC @@ -167,9 +168,6 @@ renew_client_locked(struct nfs4_client *clp) return; } - dprintk("renewing client (clientid %08x/%08x)\n", - clp->cl_clientid.cl_boot, - clp->cl_clientid.cl_id); list_move_tail(&clp->cl_lru, &nn->client_lru); clp->cl_time = ktime_get_boottime_seconds(); } @@ -1922,8 +1920,7 @@ STALE_CLIENTID(clientid_t *clid, struct nfsd_net *nn) */ if (clid->cl_boot == (u32)nn->boot_time) return 0; - dprintk("NFSD stale clientid (%08x/%08x) boot_time %08llx\n", - clid->cl_boot, clid->cl_id, nn->boot_time); + trace_nfsd_clid_stale(clid); return 1; } @@ -3879,11 +3876,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, if (clp_used_exchangeid(conf)) goto out; if (!same_creds(&conf->cl_cred, &rqstp->rq_cred)) { - char addr_str[INET6_ADDRSTRLEN]; - rpc_ntop((struct sockaddr *) &conf->cl_addr, addr_str, - sizeof(addr_str)); - dprintk("NFSD: setclientid: string in use by client " - "at %s\n", addr_str); + trace_nfsd_clid_inuse_err(conf); goto out; } } @@ -4076,7 +4069,6 @@ nfsd4_init_slabs(void) out_free_client_slab: kmem_cache_destroy(client_slab); out: - dprintk("nfsd4: out of memory while initializing nfsv4\n"); return -ENOMEM; } @@ -4508,6 +4500,8 @@ nfsd_break_deleg_cb(struct file_lock *fl) struct nfs4_delegation *dp = (struct nfs4_delegation *)fl->fl_owner; struct nfs4_file *fp = dp->dl_stid.sc_file; + trace_nfsd_deleg_break(&dp->dl_stid.sc_stateid); + /* * We don't want the locks code to timeout the lease for us; * we'll remove it ourself if a delegation isn't returned @@ -5018,8 +5012,7 @@ nfs4_open_delegation(struct svc_fh *fh, struct nfsd4_open *open, memcpy(&open->op_delegate_stateid, &dp->dl_stid.sc_stateid, sizeof(dp->dl_stid.sc_stateid)); - dprintk("NFSD: delegation stateid=" STATEID_FMT "\n", - STATEID_VAL(&dp->dl_stid.sc_stateid)); + trace_nfsd_deleg_open(&dp->dl_stid.sc_stateid); open->op_delegate_type = NFS4_OPEN_DELEGATE_READ; nfs4_put_stid(&dp->dl_stid); return; @@ -5136,9 +5129,7 @@ nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nf nfs4_open_delegation(current_fh, open, stp); nodeleg: status = nfs_ok; - - dprintk("%s: stateid=" STATEID_FMT "\n", __func__, - STATEID_VAL(&stp->st_stid.sc_stateid)); + trace_nfsd_deleg_none(&stp->st_stid.sc_stateid); out: /* 4.1 client trying to upgrade/downgrade delegation? */ if (open->op_delegate_type == NFS4_OPEN_DELEGATE_NONE && dp && @@ -5192,8 +5183,7 @@ nfsd4_renew(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, __be32 status; struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id); - dprintk("process_renew(%08x/%08x): starting\n", - clid->cl_boot, clid->cl_id); + trace_nfsd_clid_renew(clid); status = lookup_clientid(clid, cstate, nn, false); if (status) goto out; @@ -5214,6 +5204,7 @@ nfsd4_end_grace(struct nfsd_net *nn) if (nn->grace_ended) return; + trace_nfsd_grace_complete(nn); nn->grace_ended = true; /* * If the server goes down again right now, an NFSv4 @@ -5279,13 +5270,10 @@ nfs4_laundromat(struct nfsd_net *nn) copy_stateid_t *cps_t; int i; - dprintk("NFSD: laundromat service - starting\n"); - if (clients_still_reclaiming(nn)) { new_timeo = 0; goto out; } - dprintk("NFSD: end of grace period\n"); nfsd4_end_grace(nn); INIT_LIST_HEAD(&reaplist); @@ -5307,8 +5295,7 @@ nfs4_laundromat(struct nfsd_net *nn) break; } if (mark_client_expired_locked(clp)) { - dprintk("NFSD: client in use (clientid %08x)\n", - clp->cl_clientid.cl_id); + trace_nfsd_clid_expired(&clp->cl_clientid); continue; } list_add(&clp->cl_lru, &reaplist); @@ -5316,8 +5303,7 @@ nfs4_laundromat(struct nfsd_net *nn) spin_unlock(&nn->client_lock); list_for_each_safe(pos, next, &reaplist) { clp = list_entry(pos, struct nfs4_client, cl_lru); - dprintk("NFSD: purging unused client (clientid %08x)\n", - clp->cl_clientid.cl_id); + trace_nfsd_clid_purged(&clp->cl_clientid); list_del_init(&clp->cl_lru); expire_client(clp); } @@ -5407,7 +5393,6 @@ laundromat_main(struct work_struct *laundry) laundromat_work); t = nfs4_laundromat(nn); - dprintk("NFSD: laundromat_main - sleeping for %lld seconds\n", t); queue_delayed_work(laundry_wq, &nn->laundromat_work, t*HZ); } @@ -5948,8 +5933,7 @@ nfs4_preprocess_seqid_op(struct nfsd4_compound_state *cstate, u32 seqid, struct nfs4_stid *s; struct nfs4_ol_stateid *stp = NULL; - dprintk("NFSD: %s: seqid=%d stateid = " STATEID_FMT "\n", __func__, - seqid, STATEID_VAL(stateid)); + trace_nfsd_preprocess(seqid, stateid); *stpp = NULL; status = nfsd4_lookup_stateid(cstate, stateid, typemask, &s, nn); @@ -6018,9 +6002,7 @@ nfsd4_open_confirm(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, oo->oo_flags |= NFS4_OO_CONFIRMED; nfs4_inc_and_copy_stateid(&oc->oc_resp_stateid, &stp->st_stid); mutex_unlock(&stp->st_mutex); - dprintk("NFSD: %s: success, seqid=%d stateid=" STATEID_FMT "\n", - __func__, oc->oc_seqid, STATEID_VAL(&stp->st_stid.sc_stateid)); - + trace_nfsd_open_confirm(oc->oc_seqid, &stp->st_stid.sc_stateid); nfsd4_client_record_create(oo->oo_owner.so_client); status = nfs_ok; put_stateid: @@ -7072,7 +7054,7 @@ nfs4_client_to_reclaim(struct xdr_netobj name, struct xdr_netobj princhash, unsigned int strhashval; struct nfs4_client_reclaim *crp; - dprintk("NFSD nfs4_client_to_reclaim NAME: %.*s\n", name.len, name.data); + trace_nfsd_clid_reclaim(nn, name.len, name.data); crp = alloc_reclaim(); if (crp) { strhashval = clientstr_hashval(name); @@ -7122,7 +7104,7 @@ nfsd4_find_reclaim_client(struct xdr_netobj name, struct nfsd_net *nn) unsigned int strhashval; struct nfs4_client_reclaim *crp = NULL; - dprintk("NFSD: nfs4_find_reclaim_client for name %.*s\n", name.len, name.data); + trace_nfsd_clid_find(nn, name.len, name.data); strhashval = clientstr_hashval(name); list_for_each_entry(crp, &nn->reclaim_str_hashtbl[strhashval], cr_strhash) { @@ -7686,6 +7668,9 @@ nfsd_recall_delegations(struct list_head *reaplist) list_for_each_entry_safe(dp, next, reaplist, dl_recall_lru) { list_del_init(&dp->dl_recall_lru); clp = dp->dl_stid.sc_client; + + trace_nfsd_deleg_recall(&dp->dl_stid.sc_stateid); + /* * We skipped all entries that had a zero dl_time before, * so we can now reset the dl_time back to 0. If a delegation @@ -7868,6 +7853,7 @@ nfs4_state_start_net(struct net *net) goto skip_grace; printk(KERN_INFO "NFSD: starting %lld-second grace period (net %x)\n", nn->nfsd4_grace, net->ns.inum); + trace_nfsd_grace_start(nn); queue_delayed_work(laundry_wq, &nn->laundromat_work, nn->nfsd4_grace * HZ); return 0; diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h index 68d3f30ee760..3b408532a5dc 100644 --- a/fs/nfsd/state.h +++ b/fs/nfsd/state.h @@ -64,13 +64,6 @@ typedef struct { refcount_t sc_count; } copy_stateid_t; -#define STATEID_FMT "(%08x/%08x/%08x/%08x)" -#define STATEID_VAL(s) \ - (s)->si_opaque.so_clid.cl_boot, \ - (s)->si_opaque.so_clid.cl_id, \ - (s)->si_opaque.so_id, \ - (s)->si_generation - struct nfsd4_callback { struct nfs4_client *cb_clp; struct rpc_message cb_msg; diff --git a/fs/nfsd/trace.h b/fs/nfsd/trace.h index 371dc198d28e..7237fe2f3de9 100644 --- a/fs/nfsd/trace.h +++ b/fs/nfsd/trace.h @@ -277,6 +277,7 @@ DECLARE_EVENT_CLASS(nfsd_stateid_class, DEFINE_EVENT(nfsd_stateid_class, nfsd_##name, \ TP_PROTO(stateid_t *stp), \ TP_ARGS(stp)) + DEFINE_STATEID_EVENT(layoutstate_alloc); DEFINE_STATEID_EVENT(layoutstate_unhash); DEFINE_STATEID_EVENT(layoutstate_free); @@ -288,6 +289,138 @@ DEFINE_STATEID_EVENT(layout_recall_done); DEFINE_STATEID_EVENT(layout_recall_fail); DEFINE_STATEID_EVENT(layout_recall_release); +DEFINE_STATEID_EVENT(deleg_open); +DEFINE_STATEID_EVENT(deleg_none); +DEFINE_STATEID_EVENT(deleg_break); +DEFINE_STATEID_EVENT(deleg_recall); + +DECLARE_EVENT_CLASS(nfsd_stateseqid_class, + TP_PROTO(u32 seqid, const stateid_t *stp), + TP_ARGS(seqid, stp), + TP_STRUCT__entry( + __field(u32, seqid) + __field(u32, cl_boot) + __field(u32, cl_id) + __field(u32, si_id) + __field(u32, si_generation) + ), + TP_fast_assign( + __entry->seqid = seqid; + __entry->cl_boot = stp->si_opaque.so_clid.cl_boot; + __entry->cl_id = stp->si_opaque.so_clid.cl_id; + __entry->si_id = stp->si_opaque.so_id; + __entry->si_generation = stp->si_generation; + ), + TP_printk("seqid=%u client %08x:%08x stateid %08x:%08x", + __entry->seqid, __entry->cl_boot, __entry->cl_id, + __entry->si_id, __entry->si_generation) +) + +#define DEFINE_STATESEQID_EVENT(name) \ +DEFINE_EVENT(nfsd_stateseqid_class, nfsd_##name, \ + TP_PROTO(u32 seqid, const stateid_t *stp), \ + TP_ARGS(seqid, stp)) + +DEFINE_STATESEQID_EVENT(preprocess); +DEFINE_STATESEQID_EVENT(open_confirm); + +DECLARE_EVENT_CLASS(nfsd_clientid_class, + TP_PROTO(const clientid_t *clid), + TP_ARGS(clid), + TP_STRUCT__entry( + __field(u32, cl_boot) + __field(u32, cl_id) + ), + TP_fast_assign( + __entry->cl_boot = clid->cl_boot; + __entry->cl_id = clid->cl_id; + ), + TP_printk("client %08x:%08x", __entry->cl_boot, __entry->cl_id) +) + +#define DEFINE_CLIENTID_EVENT(name) \ +DEFINE_EVENT(nfsd_clientid_class, nfsd_clid_##name, \ + TP_PROTO(const clientid_t *clid), \ + TP_ARGS(clid)) + +DEFINE_CLIENTID_EVENT(expired); +DEFINE_CLIENTID_EVENT(purged); +DEFINE_CLIENTID_EVENT(renew); +DEFINE_CLIENTID_EVENT(stale); + +DECLARE_EVENT_CLASS(nfsd_net_class, + TP_PROTO(const struct nfsd_net *nn), + TP_ARGS(nn), + TP_STRUCT__entry( + __field(unsigned long long, boot_time) + ), + TP_fast_assign( + __entry->boot_time = nn->boot_time; + ), + TP_printk("boot_time=%16llx", __entry->boot_time) +) + +#define DEFINE_NET_EVENT(name) \ +DEFINE_EVENT(nfsd_net_class, nfsd_##name, \ + TP_PROTO(const struct nfsd_net *nn), \ + TP_ARGS(nn)) + +DEFINE_NET_EVENT(grace_start); +DEFINE_NET_EVENT(grace_complete); + +DECLARE_EVENT_CLASS(nfsd_clid_class, + TP_PROTO(const struct nfsd_net *nn, + unsigned int namelen, + const unsigned char *namedata), + TP_ARGS(nn, namelen, namedata), + TP_STRUCT__entry( + __field(unsigned long long, boot_time) + __field(unsigned int, namelen) + __dynamic_array(unsigned char, name, namelen) + ), + TP_fast_assign( + __entry->boot_time = nn->boot_time; + __entry->namelen = namelen; + memcpy(__get_dynamic_array(name), namedata, namelen); + ), + TP_printk("boot_time=%16llx nfs4_clientid=%.*s", + __entry->boot_time, __entry->namelen, __get_str(name)) +) + +#define DEFINE_CLID_EVENT(name) \ +DEFINE_EVENT(nfsd_clid_class, nfsd_clid_##name, \ + TP_PROTO(const struct nfsd_net *nn, \ + unsigned int namelen, \ + const unsigned char *namedata), \ + TP_ARGS(nn, namelen, namedata)) + +DEFINE_CLID_EVENT(find); +DEFINE_CLID_EVENT(reclaim); + +TRACE_EVENT(nfsd_clid_inuse_err, + TP_PROTO(const struct nfs4_client *clp), + TP_ARGS(clp), + TP_STRUCT__entry( + __field(u32, cl_boot) + __field(u32, cl_id) + __array(unsigned char, addr, sizeof(struct sockaddr_in6)) + __field(unsigned int, namelen) + __dynamic_array(unsigned char, name, clp->cl_name.len) + ), + TP_fast_assign( + __entry->cl_boot = clp->cl_clientid.cl_boot; + __entry->cl_id = clp->cl_clientid.cl_id; + memcpy(__entry->addr, &clp->cl_addr, + sizeof(struct sockaddr_in6)); + __entry->namelen = clp->cl_name.len; + memcpy(__get_dynamic_array(name), clp->cl_name.data, + clp->cl_name.len); + ), + TP_printk("nfs4_clientid %.*s already in use by %pISpc, client %08x:%08x", + __entry->namelen, __get_str(name), __entry->addr, + __entry->cl_boot, __entry->cl_id) +) + TRACE_DEFINE_ENUM(NFSD_FILE_HASHED); TRACE_DEFINE_ENUM(NFSD_FILE_PENDING); TRACE_DEFINE_ENUM(NFSD_FILE_BREAK_READ); From patchwork Tue May 12 21:24:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544173 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7244A15E6 for ; Tue, 12 May 2020 21:24:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 52E1620673 for ; Tue, 12 May 2020 21:24:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hf58BU91" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731344AbgELVYb (ORCPT ); Tue, 12 May 2020 17:24:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46372 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731334AbgELVYa (ORCPT ); Tue, 12 May 2020 17:24:30 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B157DC061A0C; Tue, 12 May 2020 14:24:30 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id j2so12165397qtr.12; Tue, 12 May 2020 14:24:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=istpkKnWmDhahhy972nAvnBVvi17hvssJ2/nNPPO9Lk=; b=hf58BU91JLnMy8ygB0wBQkMOXbT+oOqKUnCu1NHyQHR+27PUEKITWKk4565hWr4rpi HufOgUbCiJhqq/pgeyreNuHFmD806h5B9WCfrHYX7SgI2ZQw0YgjTElTMrtfSjQUUCRr d6wrp21r0edgByxQGDE24RudcoaL2eVnLIRmIrREWERPULZ3OTvZGPiA+b8baXdP63RV xUN0gAbyIVW2lLSm7ySpBe/YiFRzkSI4x+r6ZWU2w9es7idICBzbwpERbQmrP78ClAeU TdUEyVr1z1UNsZQgYONhvC/2rdrpWvYMp1K5ipgeJb3JYYVfMh03a6BNY1kAzcVJhDzy PFnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=istpkKnWmDhahhy972nAvnBVvi17hvssJ2/nNPPO9Lk=; b=QoW6e9uJTaYGDZqiWfZN2OHLF2ZTQTR1T8PcAtBKyo/iU3evPS4N6Gn1/Fit9fxNmo se11K2TH7erjcVSzdyujviYbrZqTXm1x/E3tI5fDNf+vt5nocl9x4j5ZH5MFnWcVGMZu uq720N7sPagDH7gxtmJJxeOqltqhLvagWQ1vOFVfCG2iJyoYZMEQ70bj3+KJppIa5AMy TnQ0zk4AJTBw9yRWnqfho0b8wR7HoyqeXa3qP3dKvRPiGUGzpQZ4hbDt2U/MNu1TghTb UgP1xMUZPRv++Yu3swPceDhNVcTbb5QJxQiefHTXmatRUarhXoimPzrSF1GoGUsN4tAO An5A== X-Gm-Message-State: AOAM530JxsYOMkOSTwtsQ/q7qkDC49hICaJI4WuwXg33KQsOEhir/Spn XYjTuWe1Z8jlb0CqFhLAekqdRyuQ X-Google-Smtp-Source: ABdhPJx4la4wFSuzUaGAwdb6H/fJkFCKTh0V5yl/EQkV/F2iePCHaXOyou0umS+4tBohSTwJrN5wHw== X-Received: by 2002:aed:2626:: with SMTP id z35mr5914747qtc.55.1589318669479; Tue, 12 May 2020 14:24:29 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id p81sm12093881qke.67.2020.05.12.14.24.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:24:22 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLOLe1009954; Tue, 12 May 2020 21:24:21 GMT Subject: [PATCH v2 26/29] NFSD: Add tracepoints for monitoring NFSD callbacks From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:24:21 -0400 Message-ID: <20200512212421.5826.49609.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Signed-off-by: Chuck Lever --- fs/nfsd/nfs4callback.c | 37 +++++++----- fs/nfsd/nfs4state.c | 6 +- fs/nfsd/trace.h | 153 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 177 insertions(+), 19 deletions(-) diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c index 5cf91322de0f..966ca75418c8 100644 --- a/fs/nfsd/nfs4callback.c +++ b/fs/nfsd/nfs4callback.c @@ -38,6 +38,7 @@ #include "nfsd.h" #include "state.h" #include "netns.h" +#include "trace.h" #include "xdr4cb.h" #include "xdr4.h" @@ -904,16 +905,20 @@ static int setup_callback_client(struct nfs4_client *clp, struct nfs4_cb_conn *c if (clp->cl_minorversion == 0) { if (!clp->cl_cred.cr_principal && - (clp->cl_cred.cr_flavor >= RPC_AUTH_GSS_KRB5)) + (clp->cl_cred.cr_flavor >= RPC_AUTH_GSS_KRB5)) { + trace_nfsd_cb_setup_err(clp, -EINVAL); return -EINVAL; + } args.client_name = clp->cl_cred.cr_principal; args.prognumber = conn->cb_prog; args.protocol = XPRT_TRANSPORT_TCP; args.authflavor = clp->cl_cred.cr_flavor; clp->cl_cb_ident = conn->cb_ident; } else { - if (!conn->cb_xprt) + if (!conn->cb_xprt) { + trace_nfsd_cb_setup_err(clp, -EINVAL); return -EINVAL; + } clp->cl_cb_conn.cb_xprt = conn->cb_xprt; clp->cl_cb_session = ses; args.bc_xprt = conn->cb_xprt; @@ -925,32 +930,27 @@ static int setup_callback_client(struct nfs4_client *clp, struct nfs4_cb_conn *c /* Create RPC client */ client = rpc_create(&args); if (IS_ERR(client)) { - dprintk("NFSD: couldn't create callback client: %ld\n", - PTR_ERR(client)); + trace_nfsd_cb_setup_err(clp, PTR_ERR(client)); return PTR_ERR(client); } cred = get_backchannel_cred(clp, client, ses); if (!cred) { + trace_nfsd_cb_setup_err(clp, -ENOMEM); rpc_shutdown_client(client); return -ENOMEM; } clp->cl_cb_client = client; clp->cl_cb_cred = cred; + trace_nfsd_cb_setup(clp); return 0; } -static void warn_no_callback_path(struct nfs4_client *clp, int reason) -{ - dprintk("NFSD: warning: no callback path to client %.*s: error %d\n", - (int)clp->cl_name.len, clp->cl_name.data, reason); -} - static void nfsd4_mark_cb_down(struct nfs4_client *clp, int reason) { if (test_bit(NFSD4_CLIENT_CB_UPDATE, &clp->cl_flags)) return; clp->cl_cb_state = NFSD4_CB_DOWN; - warn_no_callback_path(clp, reason); + trace_nfsd_cb_state(clp); } static void nfsd4_mark_cb_fault(struct nfs4_client *clp, int reason) @@ -958,17 +958,20 @@ static void nfsd4_mark_cb_fault(struct nfs4_client *clp, int reason) if (test_bit(NFSD4_CLIENT_CB_UPDATE, &clp->cl_flags)) return; clp->cl_cb_state = NFSD4_CB_FAULT; - warn_no_callback_path(clp, reason); + trace_nfsd_cb_state(clp); } static void nfsd4_cb_probe_done(struct rpc_task *task, void *calldata) { struct nfs4_client *clp = container_of(calldata, struct nfs4_client, cl_cb_null); + trace_nfsd_cb_done(clp, task->tk_status); if (task->tk_status) nfsd4_mark_cb_down(clp, task->tk_status); - else + else { clp->cl_cb_state = NFSD4_CB_UP; + trace_nfsd_cb_state(clp); + } } static void nfsd4_cb_probe_release(void *calldata) @@ -993,6 +996,7 @@ static const struct rpc_call_ops nfsd4_cb_probe_ops = { void nfsd4_probe_callback(struct nfs4_client *clp) { clp->cl_cb_state = NFSD4_CB_UNKNOWN; + trace_nfsd_cb_state(clp); set_bit(NFSD4_CLIENT_CB_UPDATE, &clp->cl_flags); nfsd4_run_cb(&clp->cl_cb_null); } @@ -1009,6 +1013,7 @@ void nfsd4_change_callback(struct nfs4_client *clp, struct nfs4_cb_conn *conn) spin_lock(&clp->cl_lock); memcpy(&clp->cl_cb_conn, conn, sizeof(struct nfs4_cb_conn)); spin_unlock(&clp->cl_lock); + trace_nfsd_cb_state(clp); } /* @@ -1165,8 +1170,7 @@ static void nfsd4_cb_done(struct rpc_task *task, void *calldata) struct nfsd4_callback *cb = calldata; struct nfs4_client *clp = cb->cb_clp; - dprintk("%s: minorversion=%d\n", __func__, - clp->cl_minorversion); + trace_nfsd_cb_done(clp, task->tk_status); if (!nfsd4_cb_sequence_done(task, cb)) return; @@ -1271,6 +1275,7 @@ static void nfsd4_process_cb_update(struct nfsd4_callback *cb) * kill the old client: */ if (clp->cl_cb_client) { + trace_nfsd_cb_shutdown(clp); rpc_shutdown_client(clp->cl_cb_client); clp->cl_cb_client = NULL; put_cred(clp->cl_cb_cred); @@ -1314,6 +1319,8 @@ nfsd4_run_cb_work(struct work_struct *work) struct rpc_clnt *clnt; int flags; + trace_nfsd_cb_work(clp, cb->cb_msg.rpc_proc->p_name); + if (cb->cb_need_restart) { cb->cb_need_restart = false; } else { diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 04d80f9e2f91..88433be551b7 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -2842,14 +2842,12 @@ gen_callback(struct nfs4_client *clp, struct nfsd4_setclientid *se, struct svc_r conn->cb_prog = se->se_callback_prog; conn->cb_ident = se->se_callback_ident; memcpy(&conn->cb_saddr, &rqstp->rq_daddr, rqstp->rq_daddrlen); + trace_nfsd_cb_args(clp, conn); return; out_err: conn->cb_addr.ss_family = AF_UNSPEC; conn->cb_addrlen = 0; - dprintk("NFSD: this client (clientid %08x/%08x) " - "will not receive delegations\n", - clp->cl_clientid.cl_boot, clp->cl_clientid.cl_id); - + trace_nfsd_cb_nodelegs(clp); return; } diff --git a/fs/nfsd/trace.h b/fs/nfsd/trace.h index 7237fe2f3de9..1861db1bdc67 100644 --- a/fs/nfsd/trace.h +++ b/fs/nfsd/trace.h @@ -624,6 +624,159 @@ TRACE_EVENT(nfsd_drc_mismatch, __entry->ingress) ); +TRACE_EVENT(nfsd_cb_args, + TP_PROTO( + const struct nfs4_client *clp, + const struct nfs4_cb_conn *conn + ), + TP_ARGS(clp, conn), + TP_STRUCT__entry( + __field(u32, cl_boot) + __field(u32, cl_id) + __field(u32, prog) + __field(u32, ident) + __array(unsigned char, addr, sizeof(struct sockaddr_in6)) + ), + TP_fast_assign( + __entry->cl_boot = clp->cl_clientid.cl_boot; + __entry->cl_id = clp->cl_clientid.cl_id; + __entry->prog = conn->cb_prog; + __entry->ident = conn->cb_ident; + memcpy(__entry->addr, &conn->cb_addr, + sizeof(struct sockaddr_in6)); + ), + TP_printk("client %08x:%08x callback addr=%pISpc prog=%u ident=%u", + __entry->cl_boot, __entry->cl_id, + __entry->addr, __entry->prog, __entry->ident) +); + +TRACE_EVENT(nfsd_cb_nodelegs, + TP_PROTO(const struct nfs4_client *clp), + TP_ARGS(clp), + TP_STRUCT__entry( + __field(u32, cl_boot) + __field(u32, cl_id) + ), + TP_fast_assign( + __entry->cl_boot = clp->cl_clientid.cl_boot; + __entry->cl_id = clp->cl_clientid.cl_id; + ), + TP_printk("client %08x:%08x", __entry->cl_boot, __entry->cl_id) +) + +TRACE_DEFINE_ENUM(NFSD4_CB_UP); +TRACE_DEFINE_ENUM(NFSD4_CB_UNKNOWN); +TRACE_DEFINE_ENUM(NFSD4_CB_DOWN); +TRACE_DEFINE_ENUM(NFSD4_CB_FAULT); + +#define show_cb_state(val) \ + __print_symbolic(val, \ + { NFSD4_CB_UP, "UP" }, \ + { NFSD4_CB_UNKNOWN, "UNKNOWN" }, \ + { NFSD4_CB_DOWN, "DOWN" }, \ + { NFSD4_CB_FAULT, "FAULT"}) + +DECLARE_EVENT_CLASS(nfsd_cb_class, + TP_PROTO(const struct nfs4_client *clp), + TP_ARGS(clp), + TP_STRUCT__entry( + __field(unsigned long, state) + __field(u32, cl_boot) + __field(u32, cl_id) + __array(unsigned char, addr, sizeof(struct sockaddr_in6)) + ), + TP_fast_assign( + __entry->state = clp->cl_cb_state; + __entry->cl_boot = clp->cl_clientid.cl_boot; + __entry->cl_id = clp->cl_clientid.cl_id; + memcpy(__entry->addr, &clp->cl_cb_conn.cb_addr, + sizeof(struct sockaddr_in6)); + ), + TP_printk("addr=%pISpc client %08x:%08x state=%s", + __entry->addr, __entry->cl_boot, __entry->cl_id, + show_cb_state(__entry->state)) +); + +#define DEFINE_NFSD_CB_EVENT(name) \ +DEFINE_EVENT(nfsd_cb_class, nfsd_cb_##name, \ + TP_PROTO(const struct nfs4_client *clp), \ + TP_ARGS(clp)) + +DEFINE_NFSD_CB_EVENT(setup); +DEFINE_NFSD_CB_EVENT(state); +DEFINE_NFSD_CB_EVENT(shutdown); + +TRACE_EVENT(nfsd_cb_setup_err, + TP_PROTO( + const struct nfs4_client *clp, + long error + ), + TP_ARGS(clp, error), + TP_STRUCT__entry( + __field(long, error) + __field(u32, cl_boot) + __field(u32, cl_id) + __array(unsigned char, addr, sizeof(struct sockaddr_in6)) + ), + TP_fast_assign( + __entry->error = error; + __entry->cl_boot = clp->cl_clientid.cl_boot; + __entry->cl_id = clp->cl_clientid.cl_id; + memcpy(__entry->addr, &clp->cl_cb_conn.cb_addr, + sizeof(struct sockaddr_in6)); + ), + TP_printk("addr=%pISpc client %08x:%08x error=%ld", + __entry->addr, __entry->cl_boot, __entry->cl_id, __entry->error) +); + +TRACE_EVENT(nfsd_cb_work, + TP_PROTO( + const struct nfs4_client *clp, + const char *procedure + ), + TP_ARGS(clp, procedure), + TP_STRUCT__entry( + __field(u32, cl_boot) + __field(u32, cl_id) + __string(procedure, procedure) + __array(unsigned char, addr, sizeof(struct sockaddr_in6)) + ), + TP_fast_assign( + __entry->cl_boot = clp->cl_clientid.cl_boot; + __entry->cl_id = clp->cl_clientid.cl_id; + __assign_str(procedure, procedure) + memcpy(__entry->addr, &clp->cl_cb_conn.cb_addr, + sizeof(struct sockaddr_in6)); + ), + TP_printk("addr=%pISpc client %08x:%08x procedure=%s", + __entry->addr, __entry->cl_boot, __entry->cl_id, + __get_str(procedure)) +); + +TRACE_EVENT(nfsd_cb_done, + TP_PROTO( + const struct nfs4_client *clp, + int status + ), + TP_ARGS(clp, status), + TP_STRUCT__entry( + __field(u32, cl_boot) + __field(u32, cl_id) + __field(int, status) + __array(unsigned char, addr, sizeof(struct sockaddr_in6)) + ), + TP_fast_assign( + __entry->cl_boot = clp->cl_clientid.cl_boot; + __entry->cl_id = clp->cl_clientid.cl_id; + __entry->status = status; + memcpy(__entry->addr, &clp->cl_cb_conn.cb_addr, + sizeof(struct sockaddr_in6)); + ), + TP_printk("addr=%pISpc client %08x:%08x status=%d", + __entry->addr, __entry->cl_boot, __entry->cl_id, + __entry->status) +); + #endif /* _NFSD_TRACE_H */ #undef TRACE_INCLUDE_PATH From patchwork Tue May 12 21:24:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544169 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 16A89139A for ; Tue, 12 May 2020 21:24:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F1FAA20731 for ; Tue, 12 May 2020 21:24:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RgCeRcXv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731340AbgELVYa (ORCPT ); Tue, 12 May 2020 17:24:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46364 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731334AbgELVYa (ORCPT ); Tue, 12 May 2020 17:24:30 -0400 Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B2D7AC061A0C; Tue, 12 May 2020 14:24:28 -0700 (PDT) Received: by mail-qk1-x742.google.com with SMTP id a136so6307715qkg.6; Tue, 12 May 2020 14:24:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=OIkLNkuUBokCAU8V9zFcyjFw2DAuyf2mNKo6UP//+hw=; b=RgCeRcXv+Y9Gy+Boznxpc2n4Hx3p5qz/huIrwl4H++W03d3DrdH3ZKQV1FA/NVsio9 ZaAZy7agryLxzj1TfkDtAWFHAozUtXsiPwj1ynzcjPFBOu6uMvaVSVE5+xgSJV90wzE4 3/o2NUctBnhBqv/+Q74DJWoTAfJKr94wFKkkT01gwLSTKrtWGZyY/Z72N9TxqALIsyKG fKzarAb6cSdTL+k1GBM8jLhKP5nLrYMdx82xncTFvzG9iLoBn+bEXtCEeroJaS7fs/4s lzcACjcqxnSBExRMgdcJtWcvL5J9Nhcx02y14E9FjsjjJ5EYAdPrlIZlv4N5BvTaheR7 YG0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=OIkLNkuUBokCAU8V9zFcyjFw2DAuyf2mNKo6UP//+hw=; b=jyuw8wzBAXVzlEgoGDWgIBTyYB+yR2CsEk7LkwQ7XVbC30njBhem8bKcFgs0lMlHY7 l2BwgW4/aNLtRyovQ9xRr/5rp+APVomuptOAv7cQ3lwBjd7kxpiwtyxnVDPZZLbQYLYp aDzrNHA+HzqjZOL4QsYdAJ3xJw6yOIcUM3qvUNcOkT11btMUpCLIpED28hHt44RoaxYn YCzhf3A5qyej2Xm6PjPKxpycW8Vfp/UqQBb+MPfQ9orlPbuk5jCCCtp0A+Iw+v0VP1dM m9MntiX6piATs4/zBuu6TVIsp3vHn6xJNmXVFVP3uJZmRpQ4zQi4y17vRBvCGHq9QTCn JhFw== X-Gm-Message-State: AGi0PuZ8OmJ6C/0+DxEoz4lOdSLzukYJYOod+0nszKFx3XqBgVFMxXQ1 W1KHd6t5if8C4d28uT5OKspDgWzT X-Google-Smtp-Source: APiQypJCwZiPgwjH5hCWSnuRipVuXtyFCkaXcU+veZmlIfqL35uJqMPsb3AUIXyeBiZvZ7r0lBxyfg== X-Received: by 2002:a05:620a:1379:: with SMTP id d25mr23165394qkl.173.1589318667680; Tue, 12 May 2020 14:24:27 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id c19sm12936229qtn.94.2020.05.12.14.24.27 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:24:27 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLOQe3009957; Tue, 12 May 2020 21:24:26 GMT Subject: [PATCH v2 27/29] SUNRPC: Clean up request deferral tracepoints From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:24:26 -0400 Message-ID: <20200512212426.5826.9093.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org - Rename these so they are easy to enable and search for as a set - Move the tracepoints to get a more accurate sense of control flow - Tracepoints should not fire on xprt shutdown - Display memory address in case data structure had been corrupted - Abandon dprintk in these paths I haven't ever gotten one of these tracepoints to trigger. I wonder if we should simply remove them. Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 11 ++++++++--- net/sunrpc/svc_xprt.c | 12 ++++++------ 2 files changed, 14 insertions(+), 9 deletions(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index 6fa08c36c6ca..b15b5c5dad16 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1406,27 +1406,32 @@ DECLARE_EVENT_CLASS(svc_deferred_event, TP_ARGS(dr), TP_STRUCT__entry( + __field(const void *, dr) __field(u32, xid) __string(addr, dr->xprt->xpt_remotebuf) ), TP_fast_assign( + __entry->dr = dr; __entry->xid = be32_to_cpu(*(__be32 *)(dr->args + (dr->xprt_hlen>>2))); __assign_str(addr, dr->xprt->xpt_remotebuf); ), - TP_printk("addr=%s xid=0x%08x", __get_str(addr), __entry->xid) + TP_printk("addr=%s dr=%p xid=0x%08x", __get_str(addr), __entry->dr, + __entry->xid) ); + #define DEFINE_SVC_DEFERRED_EVENT(name) \ - DEFINE_EVENT(svc_deferred_event, svc_##name##_deferred, \ + DEFINE_EVENT(svc_deferred_event, svc_defer_##name, \ TP_PROTO( \ const struct svc_deferred_req *dr \ ), \ TP_ARGS(dr)) DEFINE_SVC_DEFERRED_EVENT(drop); -DEFINE_SVC_DEFERRED_EVENT(revisit); +DEFINE_SVC_DEFERRED_EVENT(queue); +DEFINE_SVC_DEFERRED_EVENT(recv); TRACE_EVENT(svcsock_new_socket, TP_PROTO( diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c index 0a546ef02dde..c1ff8cdb5b2b 100644 --- a/net/sunrpc/svc_xprt.c +++ b/net/sunrpc/svc_xprt.c @@ -1145,16 +1145,15 @@ static void svc_revisit(struct cache_deferred_req *dreq, int too_many) set_bit(XPT_DEFERRED, &xprt->xpt_flags); if (too_many || test_bit(XPT_DEAD, &xprt->xpt_flags)) { spin_unlock(&xprt->xpt_lock); - dprintk("revisit canceled\n"); + trace_svc_defer_drop(dr); svc_xprt_put(xprt); - trace_svc_drop_deferred(dr); kfree(dr); return; } - dprintk("revisit queued\n"); dr->xprt = NULL; list_add(&dr->handle.recent, &xprt->xpt_deferred); spin_unlock(&xprt->xpt_lock); + trace_svc_defer_queue(dr); svc_xprt_enqueue(xprt); svc_xprt_put(xprt); } @@ -1200,22 +1199,24 @@ static struct cache_deferred_req *svc_defer(struct cache_req *req) memcpy(dr->args, rqstp->rq_arg.head[0].iov_base - skip, dr->argslen << 2); } + trace_svc_defer(rqstp); svc_xprt_get(rqstp->rq_xprt); dr->xprt = rqstp->rq_xprt; set_bit(RQ_DROPME, &rqstp->rq_flags); dr->handle.revisit = svc_revisit; - trace_svc_defer(rqstp); return &dr->handle; } /* * recv data from a deferred request into an active one */ -static int svc_deferred_recv(struct svc_rqst *rqstp) +static noinline int svc_deferred_recv(struct svc_rqst *rqstp) { struct svc_deferred_req *dr = rqstp->rq_deferred; + trace_svc_defer_recv(dr); + /* setup iov_base past transport header */ rqstp->rq_arg.head[0].iov_base = dr->args + (dr->xprt_hlen>>2); /* The iov_len does not include the transport header bytes */ @@ -1246,7 +1247,6 @@ static struct svc_deferred_req *svc_deferred_dequeue(struct svc_xprt *xprt) struct svc_deferred_req, handle.recent); list_del_init(&dr->handle.recent); - trace_svc_revisit_deferred(dr); } else clear_bit(XPT_DEFERRED, &xprt->xpt_flags); spin_unlock(&xprt->xpt_lock); From patchwork Tue May 12 21:24:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544177 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5B48E139A for ; Tue, 12 May 2020 21:24:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 44B9520731 for ; Tue, 12 May 2020 21:24:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="S3EtqSkT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731346AbgELVYf (ORCPT ); Tue, 12 May 2020 17:24:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731334AbgELVYf (ORCPT ); Tue, 12 May 2020 17:24:35 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 27634C061A0C; Tue, 12 May 2020 14:24:34 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id x12so12448667qts.9; Tue, 12 May 2020 14:24:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=eqiSLYH4rONyA5taIkTlAN6bgSJZ9blCTCjeQaIdYU0=; b=S3EtqSkTBfvGqy3tMIEJwg+iT336gbBxx4CTFb+MbL3f/0sh9Wy0yAuOif7nVxy5OY 17Jkg3UOUlxZQ0sHApoQUmFMZiCZ4IUp3j73Fk7kRg9dYA81v8AzEXByOTaYlwD9rnY1 2nRtFnpcJeXyidTQ63cFHU/01PbKAYtgG749Oau0yJlWrZw+I1ttDEa4R/XhgbyYQKcg 9zdiP2V2SyqpIMwWvgnKefSQvXMC7QQoaN55iaNVn6X4T6jsRii3kbNYgXCPTULgIB6H XQErwnQqUVdvQ3s6U5F4oh2yBvf5DN11WUSTHNebk/X05CqyBOZn4I/w/JMR/mF8RPQy XeWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=eqiSLYH4rONyA5taIkTlAN6bgSJZ9blCTCjeQaIdYU0=; b=qJvzWc6VnGe8XptSL7JnVuLb/02fikyi/S8j5ghvuvJrOrBgbxQ10VJ21AQgaUrwNE AvNNYC6ZWXa+tLsCSyDT/eHlMyDTkn4nULtnzWQQuz0zz03AMN6J03p28DpOjSGXA5Wd KSjSk4ym2WOU/FxFMaL2AyPnMUsfyFQgM5Cx802PIXbsulL+eHVpG60Uf4MpB0DvPsu9 mKsktMv4a0Cg/f8RbGN8jP0ePG2dRB/wDSl6/d7Rczi4XD4EkPB6oeKWM43IT2fqEczq cMuLd1ssuczXIXDTrgVVNlYu2Ak0d4Bl6s6QO+MgNCI5g3akSKZU32x/tCJfM85x5yrU QsAg== X-Gm-Message-State: AGi0PuZMaRAF3WF7jdgDucY/jEaabMr+iRm9e5sSVtnvz+CVc5ZBGWd2 farC6M8rSCRCFhGLikD/Lb1GjiKC X-Google-Smtp-Source: APiQypK5Y75ctGptTBbtCYEnnRw5CsbuXaOYHxbVvdbeHsbIxE7iQMEru8OyEH1DL8RJ19w/0sVlFw== X-Received: by 2002:aed:3ac3:: with SMTP id o61mr24403502qte.132.1589318672723; Tue, 12 May 2020 14:24:32 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id a188sm11521281qkb.68.2020.05.12.14.24.32 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:24:32 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLOVR3009960; Tue, 12 May 2020 21:24:31 GMT Subject: [PATCH v2 28/29] NFSD: Squash an annoying compiler warning From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:24:31 -0400 Message-ID: <20200512212431.5826.89537.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: Fix gcc empty-body warning when -Wextra is used. ../fs/nfsd/nfs4state.c:3898:3: warning: suggest braces around empty body in an ‘else’ statement [-Wempty-body] Signed-off-by: Chuck Lever --- fs/nfsd/nfs4state.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 88433be551b7..8b83341cae6b 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -3881,12 +3881,11 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, unconf = find_unconfirmed_client_by_name(&clname, nn); if (unconf) unhash_client_locked(unconf); + /* We need to handle only case 1: probable callback update */ if (conf && same_verf(&conf->cl_verifier, &clverifier)) { - /* case 1: probable callback update */ copy_clid(new, conf); gen_confirm(new, nn); - } else /* case 4 (new client) or cases 2, 3 (client reboot): */ - ; + } new->cl_minorversion = 0; gen_callback(new, setclid, rqstp); add_to_unconfirmed(new); From patchwork Tue May 12 21:24:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11544181 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 486381668 for ; Tue, 12 May 2020 21:24:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2B54220673 for ; Tue, 12 May 2020 21:24:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="M+jVFcTc" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731351AbgELVYj (ORCPT ); Tue, 12 May 2020 17:24:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46396 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731334AbgELVYj (ORCPT ); Tue, 12 May 2020 17:24:39 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 19854C061A0C; Tue, 12 May 2020 14:24:39 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id a136so6308189qkg.6; Tue, 12 May 2020 14:24:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=k4aFKZu7Ir4BTmsKFh4XOyBP0EdUtSdQg2tscndZkEA=; b=M+jVFcTcJSeCBnmcmEiK2kPUr7D/qnL8f2t9aTTtMIlMxdlSu7L9v4Pv61jB65Vzem dEHHAc8E9SDeuJ0MW6wI1xVIwMtuyF4SXdRPH1sTzQ1NjrkH36bbyxyT+uaivj5B1BBR xZJs3yug7nKmIPJ2p54fJQAU7zzBzuLDjoIuxQxhubxtAW0lgz5wRjaD9h5pHD+zI+Ba YnwtVWXi+D6JYU04Z7q+58bHYvdVvjanyly9Ej7iIF87p7js/5kC777TWFBZ/T2SqUwv ydaafYsM6d8u2ifOTFoX9ArOCGgloSgPj5FayY92rXvOlyiNUF/5NtXpMsTCv5TPbxLt t3+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=k4aFKZu7Ir4BTmsKFh4XOyBP0EdUtSdQg2tscndZkEA=; b=dTDYNYh8azxqG0q/OOaeyWgVlchGVQayOj2KZBMFamezPrHhWJptGwZeHXMOtZbt6H VVHBE358KLP+fjgrzEgnQSB1H39B678UJ65VoKd3NMcjNWyGSbb0tI++8HIAKU3wGq4B 205yd4LQ/cKUb8G+aN9PiGPkQlZk62gpUSUqyNrbhvDXbohrCcFH04rpQ/7zumnE2dO1 jp6fZQ5Vru8o7d1HwpfTFWzrcZUf5eVXS2YRL9oTW5wZPE9y/tnp4M31UUdW/prT0qQV UG7EWRBFvwiY62kt6EeYL8O1RQmsmS0IjbVZKkgsaxDWYo5P2T/CiKpZDeY8NL/EWkuq o6IA== X-Gm-Message-State: AGi0PuZLmKGKnbVQ6y6ZYydu51TxUF+13illxV6w4EphOQGlgiWPd6j6 HKjBB/PzhKgeqyZ3VGI012pkj6nn X-Google-Smtp-Source: APiQypI6Dv6llDwJwnfJOABbUlW+jI5qhq5LP/2DiJmSUuAsbm0PAEAruooW9loMHX/H9QfK/eVfjw== X-Received: by 2002:a05:620a:484:: with SMTP id 4mr12021090qkr.1.1589318677925; Tue, 12 May 2020 14:24:37 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id u56sm5261914qtb.91.2020.05.12.14.24.37 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2020 14:24:37 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 04CLOarv009963; Tue, 12 May 2020 21:24:36 GMT Subject: [PATCH v2 29/29] NFSD: Fix improperly-formatted Doxygen comments From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Tue, 12 May 2020 17:24:36 -0400 Message-ID: <20200512212436.5826.37327.stgit@klimt.1015granger.net> In-Reply-To: <20200512211640.5826.77139.stgit@klimt.1015granger.net> References: <20200512211640.5826.77139.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-31-g4b47 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org fs/nfsd/nfsctl.c:256: warning: Function parameter or member 'file' not described in 'write_unlock_ip' fs/nfsd/nfsctl.c:256: warning: Function parameter or member 'buf' not described in 'write_unlock_ip' fs/nfsd/nfsctl.c:256: warning: Function parameter or member 'size' not described in 'write_unlock_ip' fs/nfsd/nfsctl.c:295: warning: Function parameter or member 'file' not described in 'write_unlock_fs' fs/nfsd/nfsctl.c:295: warning: Function parameter or member 'buf' not described in 'write_unlock_fs' fs/nfsd/nfsctl.c:295: warning: Function parameter or member 'size' not described in 'write_unlock_fs' fs/nfsd/nfsctl.c:352: warning: Function parameter or member 'file' not described in 'write_filehandle' fs/nfsd/nfsctl.c:352: warning: Function parameter or member 'buf' not described in 'write_filehandle' fs/nfsd/nfsctl.c:352: warning: Function parameter or member 'size' not described in 'write_filehandle' fs/nfsd/nfsctl.c:434: warning: Function parameter or member 'file' not described in 'write_threads' fs/nfsd/nfsctl.c:434: warning: Function parameter or member 'buf' not described in 'write_threads' fs/nfsd/nfsctl.c:434: warning: Function parameter or member 'size' not described in 'write_threads' fs/nfsd/nfsctl.c:478: warning: Function parameter or member 'file' not described in 'write_pool_threads' fs/nfsd/nfsctl.c:478: warning: Function parameter or member 'buf' not described in 'write_pool_threads' fs/nfsd/nfsctl.c:478: warning: Function parameter or member 'size' not described in 'write_pool_threads' fs/nfsd/nfsctl.c:697: warning: Function parameter or member 'file' not described in 'write_versions' fs/nfsd/nfsctl.c:697: warning: Function parameter or member 'buf' not described in 'write_versions' fs/nfsd/nfsctl.c:697: warning: Function parameter or member 'size' not described in 'write_versions' fs/nfsd/nfsctl.c:858: warning: Function parameter or member 'file' not described in 'write_ports' fs/nfsd/nfsctl.c:858: warning: Function parameter or member 'buf' not described in 'write_ports' fs/nfsd/nfsctl.c:858: warning: Function parameter or member 'size' not described in 'write_ports' fs/nfsd/nfsctl.c:892: warning: Function parameter or member 'file' not described in 'write_maxblksize' fs/nfsd/nfsctl.c:892: warning: Function parameter or member 'buf' not described in 'write_maxblksize' fs/nfsd/nfsctl.c:892: warning: Function parameter or member 'size' not described in 'write_maxblksize' fs/nfsd/nfsctl.c:941: warning: Function parameter or member 'file' not described in 'write_maxconn' fs/nfsd/nfsctl.c:941: warning: Function parameter or member 'buf' not described in 'write_maxconn' fs/nfsd/nfsctl.c:941: warning: Function parameter or member 'size' not described in 'write_maxconn' fs/nfsd/nfsctl.c:1023: warning: Function parameter or member 'file' not described in 'write_leasetime' fs/nfsd/nfsctl.c:1023: warning: Function parameter or member 'buf' not described in 'write_leasetime' fs/nfsd/nfsctl.c:1023: warning: Function parameter or member 'size' not described in 'write_leasetime' fs/nfsd/nfsctl.c:1039: warning: Function parameter or member 'file' not described in 'write_gracetime' fs/nfsd/nfsctl.c:1039: warning: Function parameter or member 'buf' not described in 'write_gracetime' fs/nfsd/nfsctl.c:1039: warning: Function parameter or member 'size' not described in 'write_gracetime' fs/nfsd/nfsctl.c:1094: warning: Function parameter or member 'file' not described in 'write_recoverydir' fs/nfsd/nfsctl.c:1094: warning: Function parameter or member 'buf' not described in 'write_recoverydir' fs/nfsd/nfsctl.c:1094: warning: Function parameter or member 'size' not described in 'write_recoverydir' fs/nfsd/nfsctl.c:1125: warning: Function parameter or member 'file' not described in 'write_v4_end_grace' fs/nfsd/nfsctl.c:1125: warning: Function parameter or member 'buf' not described in 'write_v4_end_grace' fs/nfsd/nfsctl.c:1125: warning: Function parameter or member 'size' not described in 'write_v4_end_grace' fs/nfsd/nfs4proc.c:1164: warning: Function parameter or member 'nss' not described in 'nfsd4_interssc_connect' fs/nfsd/nfs4proc.c:1164: warning: Function parameter or member 'rqstp' not described in 'nfsd4_interssc_connect' fs/nfsd/nfs4proc.c:1164: warning: Function parameter or member 'mount' not described in 'nfsd4_interssc_connect' fs/nfsd/nfs4proc.c:1262: warning: Function parameter or member 'rqstp' not described in 'nfsd4_setup_inter_ssc' fs/nfsd/nfs4proc.c:1262: warning: Function parameter or member 'cstate' not described in 'nfsd4_setup_inter_ssc' fs/nfsd/nfs4proc.c:1262: warning: Function parameter or member 'copy' not described in 'nfsd4_setup_inter_ssc' fs/nfsd/nfs4proc.c:1262: warning: Function parameter or member 'mount' not described in 'nfsd4_setup_inter_ssc' Signed-off-by: Chuck Lever --- fs/nfsd/nfs4proc.c | 7 +++---- fs/nfsd/nfsctl.c | 26 +++++++++++++------------- 2 files changed, 16 insertions(+), 17 deletions(-) diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c index 0e75f7fb5fec..ec1b28d08c1c 100644 --- a/fs/nfsd/nfs4proc.c +++ b/fs/nfsd/nfs4proc.c @@ -1155,7 +1155,7 @@ extern void nfs_sb_deactive(struct super_block *sb); #define NFSD42_INTERSSC_MOUNTOPS "vers=4.2,addr=%s,sec=sys" -/** +/* * Support one copy source server for now. */ static __be32 @@ -1245,10 +1245,9 @@ nfsd4_interssc_disconnect(struct vfsmount *ss_mnt) mntput(ss_mnt); } -/** - * nfsd4_setup_inter_ssc - * +/* * Verify COPY destination stateid. + * * Connect to the source server with NFSv4.1. * Create the source struct file for nfsd_copy_range. * Called with COPY cstate: diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c index 3bb2db947d29..b48eac3bb72b 100644 --- a/fs/nfsd/nfsctl.c +++ b/fs/nfsd/nfsctl.c @@ -238,7 +238,7 @@ static inline struct net *netns(struct file *file) return file_inode(file)->i_sb->s_fs_info; } -/** +/* * write_unlock_ip - Release all locks used by a client * * Experimental. @@ -277,7 +277,7 @@ static ssize_t write_unlock_ip(struct file *file, char *buf, size_t size) return nlmsvc_unlock_all_by_ip(sap); } -/** +/* * write_unlock_fs - Release all locks on a local file system * * Experimental. @@ -327,7 +327,7 @@ static ssize_t write_unlock_fs(struct file *file, char *buf, size_t size) return error; } -/** +/* * write_filehandle - Get a variable-length NFS file handle by path * * On input, the buffer contains a '\n'-terminated C string comprised of @@ -402,7 +402,7 @@ static ssize_t write_filehandle(struct file *file, char *buf, size_t size) return mesg - buf; } -/** +/* * write_threads - Start NFSD, or report the current number of running threads * * Input: @@ -452,7 +452,7 @@ static ssize_t write_threads(struct file *file, char *buf, size_t size) return scnprintf(buf, SIMPLE_TRANSACTION_LIMIT, "%d\n", rv); } -/** +/* * write_pool_threads - Set or report the current number of threads per pool * * Input: @@ -661,7 +661,7 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size) return tlen + len; } -/** +/* * write_versions - Set or report the available NFS protocol versions * * Input: @@ -811,7 +811,7 @@ static ssize_t __write_ports(struct file *file, char *buf, size_t size, return -EINVAL; } -/** +/* * write_ports - Pass a socket file descriptor or transport name to listen on * * Input: @@ -867,7 +867,7 @@ static ssize_t write_ports(struct file *file, char *buf, size_t size) int nfsd_max_blksize; -/** +/* * write_maxblksize - Set or report the current NFS blksize * * Input: @@ -917,7 +917,7 @@ static ssize_t write_maxblksize(struct file *file, char *buf, size_t size) nfsd_max_blksize); } -/** +/* * write_maxconn - Set or report the current max number of connections * * Input: @@ -998,7 +998,7 @@ static ssize_t nfsd4_write_time(struct file *file, char *buf, size_t size, return rv; } -/** +/* * write_leasetime - Set or report the current NFSv4 lease time * * Input: @@ -1025,7 +1025,7 @@ static ssize_t write_leasetime(struct file *file, char *buf, size_t size) return nfsd4_write_time(file, buf, size, &nn->nfsd4_lease, nn); } -/** +/* * write_gracetime - Set or report current NFSv4 grace period time * * As above, but sets the time of the NFSv4 grace period. @@ -1069,7 +1069,7 @@ static ssize_t __write_recoverydir(struct file *file, char *buf, size_t size, nfs4_recoverydir()); } -/** +/* * write_recoverydir - Set or report the pathname of the recovery directory * * Input: @@ -1101,7 +1101,7 @@ static ssize_t write_recoverydir(struct file *file, char *buf, size_t size) return rv; } -/** +/* * write_v4_end_grace - release grace period for nfsd's v4.x lock manager * * Input: