From patchwork Fri May 1 17:33:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11523007 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8A9B481 for ; Fri, 1 May 2020 17:33:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 739402173E for ; Fri, 1 May 2020 17:33:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eaO0Q6Us" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729260AbgEARd3 (ORCPT ); Fri, 1 May 2020 13:33:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1728495AbgEARd3 (ORCPT ); Fri, 1 May 2020 13:33:29 -0400 Received: from mail-qt1-x844.google.com (mail-qt1-x844.google.com [IPv6:2607:f8b0:4864:20::844]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 62D5EC061A0C; Fri, 1 May 2020 10:33:29 -0700 (PDT) Received: by mail-qt1-x844.google.com with SMTP id k12so8498435qtm.4; Fri, 01 May 2020 10:33:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=VL2gfxbWc+cf0Wxk2VwYcXWvICIUvv49PplPw5QhzN8=; b=eaO0Q6UsHyLztCL8VBzl4dUrVxibimgf/VQdpzkA8ufZyszO9v63dp6j+BNuL6kLHM DRgk4OIS2QcymWEKqRtQF40Rm6wx0ocNmd32RAJIME/WxgD2pWLM0uVDxmXdZYWKFt/d JfsBW5QwtPIxJ9tCs3VMF3lwFdTz1DgS6hJs1UIktWadrHhpZKTeFIn0qr/EJ//nxZIC tJS48B/BORUpwp+kYEr+I5s/gwpVIWpYB8Sk3LxvvDWeH38VAaAGEgkhEDdlpsRy9YdP bEZ7DmMIB8ZzomRq/qVqrSgsplL2Sbp35Hha+nyKfnCQPixqP047bNp9TRwLea67T9Ii mqBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=VL2gfxbWc+cf0Wxk2VwYcXWvICIUvv49PplPw5QhzN8=; b=aAoTsN2Jlsay8mvLxO+RurYx1cQtzpe3vYiytZCOU+JEgTOn3hkvqW0k0kRAjfgkUz iV7Md3LLjtgKhAie8f8UEsEwyytQEs0iapJdZaykusN03rR88joNIKrsO4fomumIDWaX 1MmHMPHbWn0nxhJgAuSvloQYLGImiykyr7pr0qqf6knnDFdOjqMIPSkyPcFO77zq65MF MjYVJkJhPs0l6bbsRiDtz29xv7+/QOhlPIu6LPeXIgGQ44+n4aAKwsEo6y9u8zCogZPn HkLou59OqJxU8Ro4or+HLm0aVqGnGNWhRCe/tDNjRraqQX80rAipa8ntphCwzyX0FTxo rGoQ== X-Gm-Message-State: AGi0PuZAmNVuHihSWOCZdnaSyyHOQU768Ka0IdTJ2xf/vJOUqVVlSP3T uQcTyNbKl0K1Lo849TKke0hqghHr X-Google-Smtp-Source: APiQypIeGPJwXNo17mkFTzUSqFjLmlkZDxyGyvzvxs+UihAMvXA2SulNMPV+yFxCNQeQhAh1FUT0qA== X-Received: by 2002:ac8:3665:: with SMTP id n34mr4896322qtb.227.1588354408545; Fri, 01 May 2020 10:33:28 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id w42sm3078261qtj.63.2020.05.01.10.33.27 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:33:28 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HXQ9F026703; Fri, 1 May 2020 17:33:26 GMT Subject: [PATCH v1 1/7] SUNRPC: svc_show_status() macro needs enum definitions From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 01 May 2020 13:33:26 -0400 Message-ID: <20200501173326.3798.84253.stgit@klimt.1015granger.net> In-Reply-To: <20200501172849.3798.75190.stgit@klimt.1015granger.net> References: <20200501172849.3798.75190.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: Add missing TRACE_DEFINE_ENUMs in include/trace/events/sunrpc.h Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index 3158b3f7e01e..287011041e92 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1024,6 +1024,17 @@ TRACE_EVENT(svc_recv, show_rqstp_flags(__entry->flags)) ); +TRACE_DEFINE_ENUM(SVC_GARBAGE); +TRACE_DEFINE_ENUM(SVC_SYSERR); +TRACE_DEFINE_ENUM(SVC_VALID); +TRACE_DEFINE_ENUM(SVC_NEGATIVE); +TRACE_DEFINE_ENUM(SVC_OK); +TRACE_DEFINE_ENUM(SVC_DROP); +TRACE_DEFINE_ENUM(SVC_CLOSE); +TRACE_DEFINE_ENUM(SVC_DENIED); +TRACE_DEFINE_ENUM(SVC_PENDING); +TRACE_DEFINE_ENUM(SVC_COMPLETE); + #define svc_show_status(status) \ __print_symbolic(status, \ { SVC_GARBAGE, "SVC_GARBAGE" }, \ From patchwork Fri May 1 17:33:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11523011 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3679581 for ; Fri, 1 May 2020 17:33:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 17B7F2173E for ; Fri, 1 May 2020 17:33:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="h68Tenzy" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729466AbgEARdf (ORCPT ); Fri, 1 May 2020 13:33:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1728495AbgEARdf (ORCPT ); Fri, 1 May 2020 13:33:35 -0400 Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 189A4C061A0C; Fri, 1 May 2020 10:33:35 -0700 (PDT) Received: by mail-qk1-x741.google.com with SMTP id k81so7221316qke.5; Fri, 01 May 2020 10:33:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=17ev/R51/wNJXkN78+GXjfpKzuivdLOD6aPmxFqiJ0k=; b=h68Tenzyi6dvNUpZ6pB0LlpMNzb+Ewx5ckTFFBA0bRANBpGn0ns+xS4JuimLkSx8dx ddPSllKHg6Nuq8h+bPXLZR2NS8ab/YfZl1p09AQPwRKvSYbPBK2sOfoWGeszZNx0Ox5m Hi4SMpELvR681hsZwTLQJ4Oa8050zxMXmywQV6AGKaTyLeIKWAkYvbhuhdQK37hs3qEp pHfiOFr8Wai+2eI4/ZoMNrCQjTRSJN06JFhqXTLr3sn38vviZsb/b6yhsJTPTaioOP1E Y3I1reKBPkCn4OhX5Vt5X+kcHn97YEHoh4JUFxopqyG3YpOTaqMIk/KHEFt3uSwkWHzu D/YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=17ev/R51/wNJXkN78+GXjfpKzuivdLOD6aPmxFqiJ0k=; b=Vh/0afU9BogR7pQ/LIRQXa35v1bsHAcAuEXu9d9TARgjMa0xzmbwY2B0HRAR7BWndz U19Q8a4IkLOM6JF8qp3wYqQVfV+QMM93BlmpRs292vrsWNYRyANUATDjKQko5sa5dUSH RD2PypKZFPbuEH1x16taiTVLacy1iB8zFsmqq8jWdeToFUrKjtbmegRYILg2VvuccjZJ TJak+h9amp6KG4xXy0VWqysvn7RnjCps4hS9VnjHiIjNQBkmLzMVdmUmZkIYJ2lhmTQ0 8upFYjLPartmmC6HN7lUP0LFUETN/YksdUX2yM0PwM30WBsuFHUqJgPVYt5OeQoDjE2m +aYw== X-Gm-Message-State: AGi0PubHcQvxYOz7Jk604ld6SuhTQ9P3wA0riXj3rIbqo7jryMLHQCIk 34QdDJ9Ke+snBO3YgAPrYH5qB9lb X-Google-Smtp-Source: APiQypJdxhclOlkl52CZ68ga+ccw+yI2WDavkUvFj7Xw/6wNkojc1S3LX2cOvziY6k2zVyZPNSZ5EQ== X-Received: by 2002:a05:620a:228b:: with SMTP id o11mr4816692qkh.189.1588354413897; Fri, 01 May 2020 10:33:33 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id z26sm3226656qkg.39.2020.05.01.10.33.33 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:33:33 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HXWIp026706; Fri, 1 May 2020 17:33:32 GMT Subject: [PATCH v1 2/7] SUNRPC: Move xpt_mutex into socket xpo_sendto methods From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 01 May 2020 13:33:32 -0400 Message-ID: <20200501173332.3798.28627.stgit@klimt.1015granger.net> In-Reply-To: <20200501172849.3798.75190.stgit@klimt.1015granger.net> References: <20200501172849.3798.75190.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org It appears that the RPC/RDMA transport does not need serialization of calls to its xpo_sendto method. Move the mutex into the socket methods that still need that serialization. Tail latencies are unambiguously better with this patch applied. fio randrw 8KB 70/30 on NFSv3, smaller numbers are better: clat percentiles (usec): With xpt_mutex: r | 99.99th=[ 8848] w | 99.99th=[ 9634] Without xpt_mutex: r | 99.99th=[ 8586] w | 99.99th=[ 8979] Serializing the construction of RPC/RDMA transport headers is not really necessary at this point, because the Linux NFS server implementation never changes its credit grant on a connection. If that should change, then svc_rdma_sendto will need to serialize access to the transport's credit grant fields. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_xprt.h | 6 +++++ net/sunrpc/svc_xprt.c | 12 ++-------- net/sunrpc/svcsock.c | 25 ++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 35 +++++++++++++--------------- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 9 ++----- net/sunrpc/xprtsock.c | 12 ++++++++-- 6 files changed, 63 insertions(+), 36 deletions(-) diff --git a/include/linux/sunrpc/svc_xprt.h b/include/linux/sunrpc/svc_xprt.h index 9e1e046de176..aca35ab5cff2 100644 --- a/include/linux/sunrpc/svc_xprt.h +++ b/include/linux/sunrpc/svc_xprt.h @@ -117,6 +117,12 @@ static inline int register_xpt_user(struct svc_xprt *xpt, struct svc_xpt_user *u return 0; } +static inline bool svc_xprt_is_dead(const struct svc_xprt *xprt) +{ + return (test_bit(XPT_DEAD, &xprt->xpt_flags) != 0) || + (test_bit(XPT_CLOSE, &xprt->xpt_flags) != 0); +} + int svc_reg_xprt_class(struct svc_xprt_class *); void svc_unreg_xprt_class(struct svc_xprt_class *); void svc_xprt_init(struct net *, struct svc_xprt_class *, struct svc_xprt *, diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c index e12ec68cd0ff..e8092225e784 100644 --- a/net/sunrpc/svc_xprt.c +++ b/net/sunrpc/svc_xprt.c @@ -914,16 +914,10 @@ int svc_send(struct svc_rqst *rqstp) xb->page_len + xb->tail[0].iov_len; trace_svc_sendto(xb); - - /* Grab mutex to serialize outgoing data. */ - mutex_lock(&xprt->xpt_mutex); trace_svc_stats_latency(rqstp); - if (test_bit(XPT_DEAD, &xprt->xpt_flags) - || test_bit(XPT_CLOSE, &xprt->xpt_flags)) - len = -ENOTCONN; - else - len = xprt->xpt_ops->xpo_sendto(rqstp); - mutex_unlock(&xprt->xpt_mutex); + + len = xprt->xpt_ops->xpo_sendto(rqstp); + trace_svc_send(rqstp, len); svc_xprt_release(rqstp); diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 023514e392b3..3e7b6445e317 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -506,6 +506,9 @@ static int svc_udp_recvfrom(struct svc_rqst *rqstp) * svc_udp_sendto - Send out a reply on a UDP socket * @rqstp: completed svc_rqst * + * xpt_mutex ensures @rqstp's whole message is written to the socket + * without interruption. + * * Returns the number of bytes sent, or a negative errno. */ static int svc_udp_sendto(struct svc_rqst *rqstp) @@ -531,6 +534,11 @@ static int svc_udp_sendto(struct svc_rqst *rqstp) svc_set_cmsg_data(rqstp, cmh); + mutex_lock(&xprt->xpt_mutex); + + if (svc_xprt_is_dead(xprt)) + goto out_notconn; + err = xprt_sock_sendmsg(svsk->sk_sock, &msg, xdr, 0, 0, &sent); xdr_free_bvec(xdr); if (err == -ECONNREFUSED) { @@ -538,9 +546,15 @@ static int svc_udp_sendto(struct svc_rqst *rqstp) err = xprt_sock_sendmsg(svsk->sk_sock, &msg, xdr, 0, 0, &sent); xdr_free_bvec(xdr); } + + mutex_unlock(&xprt->xpt_mutex); if (err < 0) return err; return sent; + +out_notconn: + mutex_unlock(&xprt->xpt_mutex); + return -ENOTCONN; } static int svc_udp_has_wspace(struct svc_xprt *xprt) @@ -1063,6 +1077,9 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) * svc_tcp_sendto - Send out a reply on a TCP socket * @rqstp: completed svc_rqst * + * xpt_mutex ensures @rqstp's whole message is written to the socket + * without interruption. + * * Returns the number of bytes sent, or a negative errno. */ static int svc_tcp_sendto(struct svc_rqst *rqstp) @@ -1080,12 +1097,19 @@ static int svc_tcp_sendto(struct svc_rqst *rqstp) svc_release_skb(rqstp); + mutex_lock(&xprt->xpt_mutex); + if (svc_xprt_is_dead(xprt)) + goto out_notconn; err = xprt_sock_sendmsg(svsk->sk_sock, &msg, xdr, 0, marker, &sent); xdr_free_bvec(xdr); if (err < 0 || sent != (xdr->len + sizeof(marker))) goto out_close; + mutex_unlock(&xprt->xpt_mutex); return sent; +out_notconn: + mutex_unlock(&xprt->xpt_mutex); + return -ENOTCONN; out_close: pr_notice("rpc-srv/tcp: %s: %s %d when sending %d bytes - shutting down socket\n", xprt->xpt_server->sv_name, @@ -1093,6 +1117,7 @@ static int svc_tcp_sendto(struct svc_rqst *rqstp) (err < 0) ? err : sent, xdr->len); set_bit(XPT_CLOSE, &xprt->xpt_flags); svc_xprt_enqueue(xprt); + mutex_unlock(&xprt->xpt_mutex); return -EAGAIN; } diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index af7eb8d202ae..d9aab4504f2c 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -210,34 +210,31 @@ rpcrdma_bc_send_request(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst) return -ENOTCONN; } -/* Send an RPC call on the passive end of a transport - * connection. +/** + * xprt_rdma_bc_send_request - Send a reverse-direction Call + * @rqst: rpc_rqst containing Call message to be sent + * + * Return values: + * %0 if the message was sent successfully + * %ENOTCONN if the message was not sent */ -static int -xprt_rdma_bc_send_request(struct rpc_rqst *rqst) +static int xprt_rdma_bc_send_request(struct rpc_rqst *rqst) { struct svc_xprt *sxprt = rqst->rq_xprt->bc_xprt; - struct svcxprt_rdma *rdma; + struct svcxprt_rdma *rdma = + container_of(sxprt, struct svcxprt_rdma, sc_xprt); int ret; dprintk("svcrdma: sending bc call with xid: %08x\n", be32_to_cpu(rqst->rq_xid)); - mutex_lock(&sxprt->xpt_mutex); - - ret = -ENOTCONN; - rdma = container_of(sxprt, struct svcxprt_rdma, sc_xprt); - if (!test_bit(XPT_DEAD, &sxprt->xpt_flags)) { - ret = rpcrdma_bc_send_request(rdma, rqst); - if (ret == -ENOTCONN) - svc_close_xprt(sxprt); - } + if (test_bit(XPT_DEAD, &sxprt->xpt_flags)) + return -ENOTCONN; - mutex_unlock(&sxprt->xpt_mutex); - - if (ret < 0) - return ret; - return 0; + ret = rpcrdma_bc_send_request(rdma, rqst); + if (ret == -ENOTCONN) + svc_close_xprt(sxprt); + return ret; } static void diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index b6c8643867f2..b08152009a72 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -868,12 +868,9 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) __be32 *p; int ret; - /* Create the RDMA response header. xprt->xpt_mutex, - * acquired in svc_send(), serializes RPC replies. The - * code path below that inserts the credit grant value - * into each transport header runs only inside this - * critical section. - */ + if (svc_xprt_is_dead(xprt)) + goto err0; + ret = -ENOMEM; sctxt = svc_rdma_send_ctxt_get(rdma); if (!sctxt) diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index 845d0be805ec..839c49330785 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -2548,8 +2548,16 @@ static int bc_sendto(struct rpc_rqst *req) return sent; } -/* - * The send routine. Borrows from svc_send +/** + * bc_send_request - Send a backchannel Call on a TCP socket + * @req: rpc_rqst containing Call message to be sent + * + * xpt_mutex ensures @rqstp's whole message is written to the socket + * without interruption. + * + * Return values: + * %0 if the message was sent successfully + * %ENOTCONN if the message was not sent */ static int bc_send_request(struct rpc_rqst *req) { From patchwork Fri May 1 17:33:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11523033 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D00E214B4 for ; Fri, 1 May 2020 17:34:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B700024956 for ; Fri, 1 May 2020 17:34:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YvGTyLYH" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729693AbgEARdl (ORCPT ); Fri, 1 May 2020 13:33:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60818 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1728495AbgEARdk (ORCPT ); Fri, 1 May 2020 13:33:40 -0400 Received: from mail-qv1-xf42.google.com (mail-qv1-xf42.google.com [IPv6:2607:f8b0:4864:20::f42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 00210C061A0E; Fri, 1 May 2020 10:33:39 -0700 (PDT) Received: by mail-qv1-xf42.google.com with SMTP id ck5so5039291qvb.11; Fri, 01 May 2020 10:33:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=DBCaK7pbCwMuwP6HQRx0Jo8MIPjemXKq5oiMN0wKyEc=; b=YvGTyLYHhAJD4VrOP3i08HZMyTysOSxIbiKY7KXwB8OqEPJMA5vZAyuQf5+09QEtKG A/rKWiKq7Qts/Pk/qktKda6OVcLsoulSDUOHxEoHcTgvkvbuSwnYj6MBspKpwLRmOWR7 +oeM9mpOomnVpDJmdzGEqSjIn8OeLp7flHCsxPjh8GP58zNietveM2YrWlOXv19a6e1M p3K+8OazJMCiiZY8MMwl0wwpToB1sqveRYwq+BNgDiCrdPSi1/tTbXsGkwdGrSuw0fNl FWjGdGgy5FmD6lLTM75Ir/G7i7PQrXOdfgvOGyp5//C5OapSoCs/5Ex/m8O+7ReH+frS gfGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=DBCaK7pbCwMuwP6HQRx0Jo8MIPjemXKq5oiMN0wKyEc=; b=unM7svCe6XoXvaNyh1cTCJhHa8uUhcnd0hYGS8npFN0xNIP2KwKqGV/UR3GmqGmQGO hc4VXDuxwGcVjr67PVUceaOP4rkTZ4Y6RKHEf6OpB2XSe7HlvwDfGUSRPaSJdqsot8lB P+Db3Hc13SaSO4ClmWirIIywa8bLsIls37Pifw7x/3dxga+uL9t/qDElzFEVbWq4fHzO ktltG+70kINIM05rOP6JaAt1STfPWvQPPmIoNiOgKGKdF3wgfWkFvzXtn6OutnqBUUXR KwueT4/JimdhgmltjuT/1TGoPEwvO4NyFuSEINlkYHgr93aLt83JedvMpnlczDmpc0o0 0yeg== X-Gm-Message-State: AGi0Pua9ME4IAERiV7xPtOvH3a645igwmXWEYVo5zKHN+bsUh85qjVWr tq1sijxHehoc5D4G0QoSSskwGY1P X-Google-Smtp-Source: APiQypKTHdxR3b23BoTUslj+SlqCsA/irFvUJbdGu9w1jsizPCK62ezHEaJsvT+J0HDMhWQHWYCuVQ== X-Received: by 2002:ad4:45ac:: with SMTP id y12mr4950493qvu.227.1588354419080; Fri, 01 May 2020 10:33:39 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id e8sm3013948qkl.57.2020.05.01.10.33.38 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:33:38 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HXbfh026709; Fri, 1 May 2020 17:33:37 GMT Subject: [PATCH v1 3/7] svcrdma: Displayed remote IP address should match stored address From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 01 May 2020 13:33:37 -0400 Message-ID: <20200501173337.3798.58834.stgit@klimt.1015granger.net> In-Reply-To: <20200501172849.3798.75190.stgit@klimt.1015granger.net> References: <20200501172849.3798.75190.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_transport.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c index ea54785db4f8..0a1125277a48 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c @@ -211,7 +211,12 @@ static void handle_connect_req(struct rdma_cm_id *new_cma_id, newxprt->sc_ord = param->initiator_depth; sa = (struct sockaddr *)&newxprt->sc_cm_id->route.addr.dst_addr; - svc_xprt_set_remote(&newxprt->sc_xprt, sa, svc_addr_len(sa)); + newxprt->sc_xprt.xpt_remotelen = svc_addr_len(sa); + memcpy(&newxprt->sc_xprt.xpt_remote, sa, + newxprt->sc_xprt.xpt_remotelen); + snprintf(newxprt->sc_xprt.xpt_remotebuf, + sizeof(newxprt->sc_xprt.xpt_remotebuf) - 1, "%pISc", sa); + /* The remote port is arbitrary and not under the control of the * client ULP. Set it to a fixed value so that the DRC continues * to be effective after a reconnect. From patchwork Fri May 1 17:33:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11523017 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6052114B4 for ; Fri, 1 May 2020 17:33:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4977A2173E for ; Fri, 1 May 2020 17:33:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="r+hcyevn" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730130AbgEARdq (ORCPT ); Fri, 1 May 2020 13:33:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60848 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729744AbgEARdp (ORCPT ); Fri, 1 May 2020 13:33:45 -0400 Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A47CC08E859; Fri, 1 May 2020 10:33:45 -0700 (PDT) Received: by mail-qk1-x742.google.com with SMTP id t3so9949456qkg.1; Fri, 01 May 2020 10:33:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=ZIrrhhCkTF8/0jA7qBhHVLSGfrWSKZiFnqX6JC1kv/k=; b=r+hcyevnr2KJ/MCTEhpA5q4b/zsLQvou4u9nzR1ay8qFbI5meskRV+wAWsY4m6YJ/U N2gJT77YCAUF5dwPKg9f/JQzOJ1ruOMw8UhDz00L23UOueNZyfswdo/HUmfxSFfC6wPC Bg61UgqtxhuoiCjS1pw0c9nASsUnATQNP4X4ua067ie3RJwEIhLXqDOF2uDWYf0xstuT VwX5Img2V7lh5hvD8UdH5xKa4uTCgQ+ypEW7irjGnmyhkSRC1iTvrunOFpRdrk6qi1Tr gGY6+2MvLLy/MCwjLEpVQi+GiFQZd8bNMLb2hZcjx1eR0Og16E9ZJm0V8cH5ubz5hsnx Hm+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=ZIrrhhCkTF8/0jA7qBhHVLSGfrWSKZiFnqX6JC1kv/k=; b=R/SQnOwI5u62/t+6qTh9SJEOXxBc+tqOWbyThScDRPMn+yBNpzRT4D25XQZyDebCvt jEMRiuEEu/VCPyKOhVaM9L4OiRuozsGzBHi+x4coc1gMdyVDcGgNUmiHa3/3DOvO3s40 jVN3GWbUKD7unLSv1pvpJsZdRtoreureMzpupGYOKg3j9SIqIlqPTiP7pGT3IJJKCxDA 4L2XnGJOfO4asvXIOTjxkG3fFQ6i8EzQOQBpcyPptXAwV5bpl7rgYuBBrG1yfJVCaeqx 8uTeXdKgE07TsvvuL6pN2ywPXN/UQfL8GcD95LHmFhkeK9HdFGeLA6mmHf7iK+SkNsnJ fzOw== X-Gm-Message-State: AGi0Pua4Kd+W4um7jCz35rAPQ2AP0JxnVMcQtOupNsq//Bw99J5r3sF3 119BtTUVvZ86UDqDd6MP25vJFLSI X-Google-Smtp-Source: APiQypLI5RpFp09nQROihB2kgcUmYVJU5ssRqNDPbdefdL8tfIdhjoQfWOHezGTe0ymC1MXG4Ab0NQ== X-Received: by 2002:a37:b105:: with SMTP id a5mr4735183qkf.308.1588354424453; Fri, 01 May 2020 10:33:44 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id q57sm3112805qtj.55.2020.05.01.10.33.43 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:33:44 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HXgZI026712; Fri, 1 May 2020 17:33:42 GMT Subject: [PATCH v1 4/7] SUNRPC: Remove kernel memory address from svc_xprt tracepoints From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 01 May 2020 13:33:42 -0400 Message-ID: <20200501173342.3798.60366.stgit@klimt.1015granger.net> In-Reply-To: <20200501172849.3798.75190.stgit@klimt.1015granger.net> References: <20200501172849.3798.75190.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org The xprt=%p was meant to distinguish events from different transports, but the addr=%s does that just as well and does not expose kernel memory addresses. Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 27 +++++++-------------------- 1 file changed, 7 insertions(+), 20 deletions(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index 287011041e92..282f703b3976 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1184,22 +1184,19 @@ TRACE_EVENT(svc_xprt_do_enqueue, TP_ARGS(xprt, rqst), TP_STRUCT__entry( - __field(struct svc_xprt *, xprt) __field(int, pid) __field(unsigned long, flags) __string(addr, xprt->xpt_remotebuf) ), TP_fast_assign( - __entry->xprt = xprt; __entry->pid = rqst? rqst->rq_task->pid : 0; __entry->flags = xprt->xpt_flags; __assign_str(addr, xprt->xpt_remotebuf); ), - TP_printk("xprt=%p addr=%s pid=%d flags=%s", - __entry->xprt, __get_str(addr), - __entry->pid, show_svc_xprt_flags(__entry->flags)) + TP_printk("addr=%s pid=%d flags=%s", __get_str(addr), + __entry->pid, show_svc_xprt_flags(__entry->flags)) ); DECLARE_EVENT_CLASS(svc_xprt_event, @@ -1208,20 +1205,17 @@ DECLARE_EVENT_CLASS(svc_xprt_event, TP_ARGS(xprt), TP_STRUCT__entry( - __field(struct svc_xprt *, xprt) __field(unsigned long, flags) __string(addr, xprt->xpt_remotebuf) ), TP_fast_assign( - __entry->xprt = xprt; __entry->flags = xprt->xpt_flags; __assign_str(addr, xprt->xpt_remotebuf); ), - TP_printk("xprt=%p addr=%s flags=%s", - __entry->xprt, __get_str(addr), - show_svc_xprt_flags(__entry->flags)) + TP_printk("addr=%s flags=%s", __get_str(addr), + show_svc_xprt_flags(__entry->flags)) ); DEFINE_EVENT(svc_xprt_event, svc_xprt_no_write_space, @@ -1234,24 +1228,20 @@ TRACE_EVENT(svc_xprt_dequeue, TP_ARGS(rqst), TP_STRUCT__entry( - __field(struct svc_xprt *, xprt) __field(unsigned long, flags) __field(unsigned long, wakeup) __string(addr, rqst->rq_xprt->xpt_remotebuf) ), TP_fast_assign( - __entry->xprt = rqst->rq_xprt; __entry->flags = rqst->rq_xprt->xpt_flags; __entry->wakeup = ktime_to_us(ktime_sub(ktime_get(), rqst->rq_qtime)); __assign_str(addr, rqst->rq_xprt->xpt_remotebuf); ), - TP_printk("xprt=%p addr=%s flags=%s wakeup-us=%lu", - __entry->xprt, __get_str(addr), - show_svc_xprt_flags(__entry->flags), - __entry->wakeup) + TP_printk("addr=%s flags=%s wakeup-us=%lu", __get_str(addr), + show_svc_xprt_flags(__entry->flags), __entry->wakeup) ); TRACE_EVENT(svc_wake_up, @@ -1276,21 +1266,18 @@ TRACE_EVENT(svc_handle_xprt, TP_ARGS(xprt, len), TP_STRUCT__entry( - __field(struct svc_xprt *, xprt) __field(int, len) __field(unsigned long, flags) __string(addr, xprt->xpt_remotebuf) ), TP_fast_assign( - __entry->xprt = xprt; __entry->len = len; __entry->flags = xprt->xpt_flags; __assign_str(addr, xprt->xpt_remotebuf); ), - TP_printk("xprt=%p addr=%s len=%d flags=%s", - __entry->xprt, __get_str(addr), + TP_printk("addr=%s len=%d flags=%s", __get_str(addr), __entry->len, show_svc_xprt_flags(__entry->flags)) ); From patchwork Fri May 1 17:33:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11523029 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 060AF14B4 for ; Fri, 1 May 2020 17:34:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E361024956 for ; Fri, 1 May 2020 17:34:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PsTcrLSP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730287AbgEAReR (ORCPT ); Fri, 1 May 2020 13:34:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60876 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1730229AbgEARdu (ORCPT ); Fri, 1 May 2020 13:33:50 -0400 Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B258FC08E859; Fri, 1 May 2020 10:33:50 -0700 (PDT) Received: by mail-qt1-x841.google.com with SMTP id e17so8451154qtp.7; Fri, 01 May 2020 10:33:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=mN34Y6mlei8qRJsOn3ab5qgvrc/s/iI5ogq9M2gjqQs=; b=PsTcrLSPxEpJld6ThNTd9ShWJRA1il58CyKyjhMZPrs4U1yJgd7Z4h58y9JkT4tPzf rDhiHUexgCbUAJSzgca316cqm+WvqlW0NgzXBLZO52s7mjigCbfCHj5ey8Rs5z6v6bbV 0SEOw05Cggj6UiHyamaCYFSPS6F2AUqcvQcK7czMpIVNFjmuj2a3TG4qR2GAw1DIlTf/ iC33J9oVqhhWfAn7vgE9Stj56m4cG+zj6hT+aDTsp9SNcXbGULPYUNTlggRuMpHXbYhh 10O0YXJ2LkF33GF1RA7vCxN3dlhqZAuHkNWv7OD6NCudhROb+yZ89XhGx1FeSia/1vFf zMkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=mN34Y6mlei8qRJsOn3ab5qgvrc/s/iI5ogq9M2gjqQs=; b=XPlQRIlVN1/OLAI2j5RtbkJuPMDhs78lpUuD/ZjR++UpERvdkY/vuZmi8M7JYNXDU2 kObKLl30B6eNfWk8oqnO1fSFVU+OlUAFcCSq7Ue5+CWhTy/ahlk+4M1KAle/dIQqHTuK /SI2TowVsbb3PmW3cNjp+EOpPhe9oX8ekcbk+ej+/+lbnFG0FPVcYCPDkt1NvxqZfJ0T Ob6rBwbBbIbj9r0KmK2oQb3PVxT+CMq0VjuQ1rPGiBYfowxiKF/BQcQ0Km7+Jwq25BQu jV7tXNDveaszQNH1Csq7B5SkoWTrMWo0Ipv7/MF7b2JMxaEHreNiPwyZQb7HTAOcsKPF gd0A== X-Gm-Message-State: AGi0Pubes6S9A/ZC7Erdk5vLLGPJWj7tvC1W2Kw16Xv38sh2XJwct3H/ VNeDXEgtLu2sR9tJ6C3veuLcmpLv X-Google-Smtp-Source: APiQypLqWoRwjWaTllW3gPA3dHrhF3NLbjKoMl4GJ9rGMgf3ZOjS6rbpoGwSo7HhVH8iCVpoGsSX4g== X-Received: by 2002:ac8:7581:: with SMTP id s1mr4865751qtq.260.1588354429722; Fri, 01 May 2020 10:33:49 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id i5sm2920366qtw.97.2020.05.01.10.33.48 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:33:49 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HXmXB026715; Fri, 1 May 2020 17:33:48 GMT Subject: [PATCH v1 5/7] SUNRPC: Tracepoint to record errors in svc_xpo_create() From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 01 May 2020 13:33:48 -0400 Message-ID: <20200501173348.3798.89598.stgit@klimt.1015granger.net> In-Reply-To: <20200501172849.3798.75190.stgit@klimt.1015granger.net> References: <20200501172849.3798.75190.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org This enables capture of failures for an audit log. Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 29 +++++++++++++++++++++++++++++ net/sunrpc/svc_xprt.c | 7 ++++++- 2 files changed, 35 insertions(+), 1 deletion(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index 282f703b3976..5bcc0089e9a5 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1178,6 +1178,35 @@ DEFINE_EVENT(svc_rqst_status, svc_send, { (1UL << XPT_KILL_TEMP), "XPT_KILL_TEMP"}, \ { (1UL << XPT_CONG_CTRL), "XPT_CONG_CTRL"}) +TRACE_EVENT(svc_xprt_create_err, + TP_PROTO( + const char *program, + const char *protocol, + struct sockaddr *sap, + const struct svc_xprt *xprt + ), + + TP_ARGS(program, protocol, sap, xprt), + + TP_STRUCT__entry( + __field(long, error) + __string(program, program) + __string(protocol, protocol) + __array(unsigned char, addr, sizeof(struct sockaddr_in6)) + ), + + TP_fast_assign( + __entry->error = PTR_ERR(xprt); + __assign_str(program, program); + __assign_str(protocol, protocol); + memcpy(__entry->addr, sap, sizeof(__entry->addr)); + ), + + TP_printk("addr=%pISpc program=%s protocol=%s error=%ld", + __entry->addr, __get_str(program), __get_str(protocol), + __entry->error) +); + TRACE_EVENT(svc_xprt_do_enqueue, TP_PROTO(struct svc_xprt *xprt, struct svc_rqst *rqst), diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c index e8092225e784..c444c01c6b6b 100644 --- a/net/sunrpc/svc_xprt.c +++ b/net/sunrpc/svc_xprt.c @@ -206,6 +206,7 @@ static struct svc_xprt *__svc_xpo_create(struct svc_xprt_class *xcl, .sin6_port = htons(port), }; #endif + struct svc_xprt *xprt; struct sockaddr *sap; size_t len; @@ -224,7 +225,11 @@ static struct svc_xprt *__svc_xpo_create(struct svc_xprt_class *xcl, return ERR_PTR(-EAFNOSUPPORT); } - return xcl->xcl_ops->xpo_create(serv, net, sap, len, flags); + xprt = xcl->xcl_ops->xpo_create(serv, net, sap, len, flags); + if (IS_ERR(xprt)) + trace_svc_xprt_create_err(serv->sv_program->pg_name, + xcl->xcl_name, sap, xprt); + return xprt; } /* From patchwork Fri May 1 17:33:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11523023 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 84B59913 for ; Fri, 1 May 2020 17:34:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 63D2824958 for ; Fri, 1 May 2020 17:34:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gtj1KGcv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730379AbgEAReK (ORCPT ); Fri, 1 May 2020 13:34:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60896 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1730285AbgEARd4 (ORCPT ); Fri, 1 May 2020 13:33:56 -0400 Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0DBF5C061A0C; Fri, 1 May 2020 10:33:56 -0700 (PDT) Received: by mail-qt1-x843.google.com with SMTP id s30so8503373qth.2; Fri, 01 May 2020 10:33:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=er1o6KEwg352K3HTM8AaL4GVOU2fAsBzAdx2UuvjTg0=; b=gtj1KGcvz6z43p9h23CuEFbegrT8NHowwvNElrJV69FSHjdbHrvDpDnb6aMlexFD9y dbwBcSE+3Z/eQuQuHGVbMFPWu0Zy4V5tr15PhY7dWmSIQQz/k2F2mseQ+kIPfoXk1S4r sSoEV3ZyTTHTgClEZMBrheZgNExE/OHHm2+BS4xDs9q5kNfH79KbPnXSakvUePVg9+IV sq9XaZRJia/XNYU+peK0XwaXt2QEIE5BdK3ibwV/Mok32XcOgVoTrIA0L9VaZc0aN4Y9 y9YA//61my6Mx2BQo2k4+iSDodpICE7HF9Nku1XDVlrzi08ktKpIxsqgXtcDLo658ItC Pfng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=er1o6KEwg352K3HTM8AaL4GVOU2fAsBzAdx2UuvjTg0=; b=sb1bFNrdCwsf4IK0nWMbNmpr9w/LtBw93cJdN2SuSoXWq4xvpsrYW6eNecPxgpxPLW F+E6+i/6icltQQinqcvePsAGp+vALiNmxhjyclcG1KmFbjFPnJrQMo7VQk9UaY+JPCzI 40H6yA4QFnJE82ryewvot+CZkiOx8qkeHl5Y5QRX0FV1OdOrTiNgDZnC7rJfhM+Y6DWB JnMBSESpCNFGwli9Uik3xZ9/ANNTPTjrUxlt1Aaex+dlcWBEHYPigjuCQFVZPfdmn1d4 DAIG//mJi+EbpGfpIoJxJmi42oSLKTh5ChX5U5lyMBvm/W09XWSuxtkPXNet8qByOc/F dH0g== X-Gm-Message-State: AGi0PubY1BR9VgAb8oJ6re/Kr6dT2IbwERTAg4Oo0rreHOm4Jv1MTgMX K+UbCWPcD6RmPfr6X0zKntyIlxje X-Google-Smtp-Source: APiQypLKZimWQux2p4AEtmqukWfURZgqplKtSVh5q10PQ9Ql9BrQLsjfQVsOhDFOoA/wks9exSZr3A== X-Received: by 2002:ac8:4c8d:: with SMTP id j13mr4877333qtv.142.1588354434870; Fri, 01 May 2020 10:33:54 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id e4sm3210774qkn.11.2020.05.01.10.33.54 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:33:54 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HXrDF026718; Fri, 1 May 2020 17:33:53 GMT Subject: [PATCH v1 6/7] SUNRPC: Trace a few more generic svc_xprt events From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 01 May 2020 13:33:53 -0400 Message-ID: <20200501173353.3798.97235.stgit@klimt.1015granger.net> In-Reply-To: <20200501172849.3798.75190.stgit@klimt.1015granger.net> References: <20200501172849.3798.75190.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org In lieu of dprintks or tracepoints in each individual transport implementation, introduce tracepoints in the generic part of the RPC layer. These typically fire for connection lifetime events, so shouldn't contribute a lot of noise. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 33 ------------------------- include/trace/events/sunrpc.h | 39 ++++++++++++++++++++++++++++-- net/sunrpc/svc_xprt.c | 22 ++++------------- net/sunrpc/svcsock.c | 12 --------- net/sunrpc/xprtrdma/svc_rdma_transport.c | 23 +++--------------- 5 files changed, 45 insertions(+), 84 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 596e0a803477..6de7a9202a68 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1283,39 +1283,6 @@ TRACE_EVENT(xprtrdma_leaked_rep, ** Server-side RPC/RDMA events **/ -DECLARE_EVENT_CLASS(svcrdma_xprt_event, - TP_PROTO( - const struct svc_xprt *xprt - ), - - TP_ARGS(xprt), - - TP_STRUCT__entry( - __field(const void *, xprt) - __string(addr, xprt->xpt_remotebuf) - ), - - TP_fast_assign( - __entry->xprt = xprt; - __assign_str(addr, xprt->xpt_remotebuf); - ), - - TP_printk("xprt=%p addr=%s", - __entry->xprt, __get_str(addr) - ) -); - -#define DEFINE_XPRT_EVENT(name) \ - DEFINE_EVENT(svcrdma_xprt_event, svcrdma_xprt_##name, \ - TP_PROTO( \ - const struct svc_xprt *xprt \ - ), \ - TP_ARGS(xprt)) - -DEFINE_XPRT_EVENT(accept); -DEFINE_XPRT_EVENT(fail); -DEFINE_XPRT_EVENT(free); - TRACE_DEFINE_ENUM(RDMA_MSG); TRACE_DEFINE_ENUM(RDMA_NOMSG); TRACE_DEFINE_ENUM(RDMA_MSGP); diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index 5bcc0089e9a5..561a438bcc66 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1247,9 +1247,42 @@ DECLARE_EVENT_CLASS(svc_xprt_event, show_svc_xprt_flags(__entry->flags)) ); -DEFINE_EVENT(svc_xprt_event, svc_xprt_no_write_space, - TP_PROTO(struct svc_xprt *xprt), - TP_ARGS(xprt)); +#define DEFINE_SVC_XPRT_EVENT(name) \ + DEFINE_EVENT(svc_xprt_event, svc_xprt_##name, \ + TP_PROTO( \ + struct svc_xprt *xprt \ + ), \ + TP_ARGS(xprt)) + +DEFINE_SVC_XPRT_EVENT(no_write_space); +DEFINE_SVC_XPRT_EVENT(close); +DEFINE_SVC_XPRT_EVENT(detach); +DEFINE_SVC_XPRT_EVENT(free); + +TRACE_EVENT(svc_xprt_accept, + TP_PROTO( + const struct svc_xprt *xprt, + const char *service + ), + + TP_ARGS(xprt, service), + + TP_STRUCT__entry( + __string(addr, xprt->xpt_remotebuf) + __string(protocol, xprt->xpt_class->xcl_name) + __string(service, service) + ), + + TP_fast_assign( + __assign_str(addr, xprt->xpt_remotebuf); + __assign_str(protocol, xprt->xpt_class->xcl_name) + __assign_str(service, service); + ), + + TP_printk("addr=%s protocol=%s service=%s", + __get_str(addr), __get_str(protocol), __get_str(service) + ) +); TRACE_EVENT(svc_xprt_dequeue, TP_PROTO(struct svc_rqst *rqst), diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c index c444c01c6b6b..c1ff8cdb5b2b 100644 --- a/net/sunrpc/svc_xprt.c +++ b/net/sunrpc/svc_xprt.c @@ -153,6 +153,7 @@ static void svc_xprt_free(struct kref *kref) xprt_put(xprt->xpt_bc_xprt); if (xprt->xpt_bc_xps) xprt_switch_put(xprt->xpt_bc_xps); + trace_svc_xprt_free(xprt); xprt->xpt_ops->xpo_free(xprt); module_put(owner); } @@ -309,15 +310,11 @@ int svc_create_xprt(struct svc_serv *serv, const char *xprt_name, { int err; - dprintk("svc: creating transport %s[%d]\n", xprt_name, port); err = _svc_create_xprt(serv, xprt_name, net, family, port, flags, cred); if (err == -EPROTONOSUPPORT) { request_module("svc%s", xprt_name); err = _svc_create_xprt(serv, xprt_name, net, family, port, flags, cred); } - if (err < 0) - dprintk("svc: transport %s not found, err %d\n", - xprt_name, -err); return err; } EXPORT_SYMBOL_GPL(svc_create_xprt); @@ -785,7 +782,6 @@ static int svc_handle_xprt(struct svc_rqst *rqstp, struct svc_xprt *xprt) int len = 0; if (test_bit(XPT_CLOSE, &xprt->xpt_flags)) { - dprintk("svc_recv: found XPT_CLOSE\n"); if (test_and_clear_bit(XPT_KILL_TEMP, &xprt->xpt_flags)) xprt->xpt_ops->xpo_kill_temp_xprt(xprt); svc_delete_xprt(xprt); @@ -804,6 +800,7 @@ static int svc_handle_xprt(struct svc_rqst *rqstp, struct svc_xprt *xprt) if (newxpt) { newxpt->xpt_cred = get_cred(xprt->xpt_cred); svc_add_new_temp_xprt(serv, newxpt); + trace_svc_xprt_accept(newxpt, serv->sv_name); } else module_put(xprt->xpt_class->xcl_owner); } else if (svc_xprt_reserve_slot(rqstp, xprt)) { @@ -840,14 +837,6 @@ int svc_recv(struct svc_rqst *rqstp, long timeout) struct svc_serv *serv = rqstp->rq_server; int len, err; - dprintk("svc: server %p waiting for data (to = %ld)\n", - rqstp, timeout); - - if (rqstp->rq_xprt) - printk(KERN_ERR - "svc_recv: service %p, transport not NULL!\n", - rqstp); - err = svc_alloc_arg(rqstp); if (err) goto out; @@ -895,7 +884,6 @@ EXPORT_SYMBOL_GPL(svc_recv); void svc_drop(struct svc_rqst *rqstp) { trace_svc_drop(rqstp); - dprintk("svc: xprt %p dropped request\n", rqstp->rq_xprt); svc_xprt_release(rqstp); } EXPORT_SYMBOL_GPL(svc_drop); @@ -1030,11 +1018,10 @@ static void svc_delete_xprt(struct svc_xprt *xprt) struct svc_serv *serv = xprt->xpt_server; struct svc_deferred_req *dr; - /* Only do this once */ if (test_and_set_bit(XPT_DEAD, &xprt->xpt_flags)) - BUG(); + return; - dprintk("svc: svc_delete_xprt(%p)\n", xprt); + trace_svc_xprt_detach(xprt); xprt->xpt_ops->xpo_detach(xprt); if (xprt->xpt_bc_xprt) xprt->xpt_bc_xprt->ops->close(xprt->xpt_bc_xprt); @@ -1055,6 +1042,7 @@ static void svc_delete_xprt(struct svc_xprt *xprt) void svc_close_xprt(struct svc_xprt *xprt) { + trace_svc_xprt_close(xprt); set_bit(XPT_CLOSE, &xprt->xpt_flags); if (test_and_set_bit(XPT_BUSY, &xprt->xpt_flags)) /* someone else will have to effect the close */ diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 3e7b6445e317..cf4bd198c19d 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -727,7 +727,6 @@ static struct svc_xprt *svc_tcp_accept(struct svc_xprt *xprt) int err, slen; RPC_IFDEBUG(char buf[RPC_MAX_ADDRBUFLEN]); - dprintk("svc: tcp_accept %p sock %p\n", svsk, sock); if (!sock) return NULL; @@ -1363,11 +1362,6 @@ static struct svc_xprt *svc_create_socket(struct svc_serv *serv, int newlen; int family; int val; - RPC_IFDEBUG(char buf[RPC_MAX_ADDRBUFLEN]); - - dprintk("svc: svc_create_socket(%s, %d, %s)\n", - serv->sv_program->pg_name, protocol, - __svc_print_addr(sin, buf, sizeof(buf))); if (protocol != IPPROTO_UDP && protocol != IPPROTO_TCP) { printk(KERN_WARNING "svc: only UDP and TCP " @@ -1427,7 +1421,6 @@ static struct svc_xprt *svc_create_socket(struct svc_serv *serv, svc_xprt_set_local(&svsk->sk_xprt, newsin, newlen); return (struct svc_xprt *)svsk; bummer: - dprintk("svc: svc_create_socket error = %d\n", -error); sock_release(sock); return ERR_PTR(error); } @@ -1441,8 +1434,6 @@ static void svc_sock_detach(struct svc_xprt *xprt) struct svc_sock *svsk = container_of(xprt, struct svc_sock, sk_xprt); struct sock *sk = svsk->sk_sk; - dprintk("svc: svc_sock_detach(%p)\n", svsk); - /* put back the old socket callbacks */ lock_sock(sk); sk->sk_state_change = svsk->sk_ostate; @@ -1459,8 +1450,6 @@ static void svc_tcp_sock_detach(struct svc_xprt *xprt) { struct svc_sock *svsk = container_of(xprt, struct svc_sock, sk_xprt); - dprintk("svc: svc_tcp_sock_detach(%p)\n", svsk); - svc_sock_detach(xprt); if (!test_bit(XPT_LISTENER, &xprt->xpt_flags)) { @@ -1475,7 +1464,6 @@ static void svc_tcp_sock_detach(struct svc_xprt *xprt) static void svc_sock_free(struct svc_xprt *xprt) { struct svc_sock *svsk = container_of(xprt, struct svc_sock, sk_xprt); - dprintk("svc: svc_sock_free(%p)\n", svsk); if (svsk->sk_sock->file) sockfd_put(svsk->sk_sock); diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c index 0a1125277a48..aa68dc706006 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c @@ -314,11 +314,8 @@ static struct svc_xprt *svc_rdma_create(struct svc_serv *serv, struct svcxprt_rdma *cma_xprt; int ret; - dprintk("svcrdma: Creating RDMA listener\n"); - if ((sa->sa_family != AF_INET) && (sa->sa_family != AF_INET6)) { - dprintk("svcrdma: Address family %d is not supported.\n", sa->sa_family); + if (sa->sa_family != AF_INET && sa->sa_family != AF_INET6) return ERR_PTR(-EAFNOSUPPORT); - } cma_xprt = svc_rdma_create_xprt(serv, net); if (!cma_xprt) return ERR_PTR(-ENOMEM); @@ -329,7 +326,6 @@ static struct svc_xprt *svc_rdma_create(struct svc_serv *serv, RDMA_PS_TCP, IB_QPT_RC); if (IS_ERR(listen_id)) { ret = PTR_ERR(listen_id); - dprintk("svcrdma: rdma_create_id failed = %d\n", ret); goto err0; } @@ -338,23 +334,17 @@ static struct svc_xprt *svc_rdma_create(struct svc_serv *serv, */ #if IS_ENABLED(CONFIG_IPV6) ret = rdma_set_afonly(listen_id, 1); - if (ret) { - dprintk("svcrdma: rdma_set_afonly failed = %d\n", ret); + if (ret) goto err1; - } #endif ret = rdma_bind_addr(listen_id, sa); - if (ret) { - dprintk("svcrdma: rdma_bind_addr failed = %d\n", ret); + if (ret) goto err1; - } cma_xprt->sc_cm_id = listen_id; ret = rdma_listen(listen_id, RPCRDMA_LISTEN_BACKLOG); - if (ret) { - dprintk("svcrdma: rdma_listen failed = %d\n", ret); + if (ret) goto err1; - } /* * We need to use the address from the cm_id in case the @@ -540,12 +530,9 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) dprintk(" ord : %d\n", conn_param.initiator_depth); #endif - trace_svcrdma_xprt_accept(&newxprt->sc_xprt); return &newxprt->sc_xprt; errout: - dprintk("svcrdma: failure accepting new connection rc=%d.\n", ret); - trace_svcrdma_xprt_fail(&newxprt->sc_xprt); /* Take a reference in case the DTO handler runs */ svc_xprt_get(&newxprt->sc_xprt); if (newxprt->sc_qp && !IS_ERR(newxprt->sc_qp)) @@ -583,8 +570,6 @@ static void __svc_rdma_free(struct work_struct *work) container_of(work, struct svcxprt_rdma, sc_work); struct svc_xprt *xprt = &rdma->sc_xprt; - trace_svcrdma_xprt_free(xprt); - if (rdma->sc_qp && !IS_ERR(rdma->sc_qp)) ib_drain_qp(rdma->sc_qp); From patchwork Fri May 1 17:33:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11523019 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9D9F581 for ; Fri, 1 May 2020 17:34:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 80A2B24958 for ; Fri, 1 May 2020 17:34:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="sFxoynyD" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730340AbgEAReH (ORCPT ); Fri, 1 May 2020 13:34:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1730341AbgEAReC (ORCPT ); Fri, 1 May 2020 13:34:02 -0400 Received: from mail-qv1-xf43.google.com (mail-qv1-xf43.google.com [IPv6:2607:f8b0:4864:20::f43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 784D1C061A0C; Fri, 1 May 2020 10:34:01 -0700 (PDT) Received: by mail-qv1-xf43.google.com with SMTP id h6so5065603qvz.8; Fri, 01 May 2020 10:34:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=93ayo/94AvL0bcM6DZvZdFqTzlt58jM0zJHfBTCx9/c=; b=sFxoynyDgqZiuXieYju7OuDXSouWl8WGb5L2zgCwOh7hdBiCNlM8XyUVn7qV0yMQyY dd1MAyXlDcXtzu6z/+9InpArek0ZBxOAT3XbvrIIDm5rjbCjn1aPfdMJL7c2fLLglaXV uZowzqS2VAjeuqPJGbpl81VrnMarSXKCwvVQVAW5Su+/yNx044fKslyAcEWbBWWTO54Q GSLE58XApHOK2JcHZ7QKi9IdFC4WzvJRIfT53fjjAJS9WfW7w6geLwjjc6v2cNGB1Emc 9HkeyVK7ix9OVeA+XXgAcQtHUkMUK698ktvawGbBhAGcjJ6Xi3kpOhy5e8/z34CfOT4X NBcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=93ayo/94AvL0bcM6DZvZdFqTzlt58jM0zJHfBTCx9/c=; b=gy+wwx3OsKLfAF9gagImAB0cDkI7mrhzfH9U8zgxv3EgbpZ+vHQiFCRXkrlCCE/gJr FjKlqiSeWPSEb68SiUBrzF2i2nJSU2rsNkMNvLnNnSNzkaYeaHeOs5dtv5WJdx50suca ZWVDCTwcXYbUAUd8u4fPfpvd7LlzsgDxM5JcH9wFfkHGYTpI1wdymIr7PKNxVW2m+1gJ MUgrgsTZTfq/TCKbN8pNNYUmIK9l0H9j6Kv/36ZvJmx6p+WSGQesWrjXOVYcq4gfV2Gk 3uzDYlN3V4jXq8F1mgCbKC0EYHGX8trTxiqRNOoGu1cIKE7dRLwNLZv5wR/D6klFugum Wb+w== X-Gm-Message-State: AGi0Pua3SPBmXCWQ9C0cGl78N+Pu7HL1CPpzjVnTuGTokJPZWGWRpv06 jtcY+6Z0EMm8oOwfD5sEG+yw4i97 X-Google-Smtp-Source: APiQypKaNqUY5uWRvUL7ZENxWB17e8yl44eoOS4CIwldjJQBjKJoYIYa1earOez6A8RuW845BKPs3A== X-Received: by 2002:ad4:562e:: with SMTP id cb14mr4876712qvb.249.1588354440446; Fri, 01 May 2020 10:34:00 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id q62sm2024519qke.22.2020.05.01.10.33.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:33:59 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HXwlo026721; Fri, 1 May 2020 17:33:58 GMT Subject: [PATCH v1 7/7] svcrdma: Add tracepoints to report ->xpo_accept failures From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Fri, 01 May 2020 13:33:58 -0400 Message-ID: <20200501173358.3798.73582.stgit@klimt.1015granger.net> In-Reply-To: <20200501172849.3798.75190.stgit@klimt.1015granger.net> References: <20200501172849.3798.75190.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Failure to accept a connection is typically due to a problem specific to a transport type. Also, ->xpo_accept returns NULL on error rather than reporting a specific problem. So, add failure-specific tracepoints in svc_rdma_accept(). Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 37 ++++++++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_transport.c | 25 +++++++++----------- 2 files changed, 48 insertions(+), 14 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 6de7a9202a68..c25e11564598 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1283,6 +1283,43 @@ TRACE_EVENT(xprtrdma_leaked_rep, ** Server-side RPC/RDMA events **/ +DECLARE_EVENT_CLASS(svcrdma_accept_class, + TP_PROTO( + const struct svcxprt_rdma *rdma, + long status + ), + + TP_ARGS(rdma, status), + + TP_STRUCT__entry( + __field(long, status) + __string(addr, rdma->sc_xprt.xpt_remotebuf) + ), + + TP_fast_assign( + __entry->status = status; + __assign_str(addr, rdma->sc_xprt.xpt_remotebuf); + ), + + TP_printk("addr=%s status=%ld", + __get_str(addr), __entry->status + ) +); + +#define DEFINE_ACCEPT_EVENT(name) \ + DEFINE_EVENT(svcrdma_accept_class, svcrdma_##name##_err, \ + TP_PROTO( \ + const struct svcxprt_rdma *rdma, \ + long status \ + ), \ + TP_ARGS(rdma, status)) + +DEFINE_ACCEPT_EVENT(pd); +DEFINE_ACCEPT_EVENT(qp); +DEFINE_ACCEPT_EVENT(fabric); +DEFINE_ACCEPT_EVENT(initdepth); +DEFINE_ACCEPT_EVENT(accept); + TRACE_DEFINE_ENUM(RDMA_MSG); TRACE_DEFINE_ENUM(RDMA_NOMSG); TRACE_DEFINE_ENUM(RDMA_MSGP); diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c index aa68dc706006..d38be57b00ed 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c @@ -400,9 +400,6 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) if (!newxprt) return NULL; - dprintk("svcrdma: newxprt from accept queue = %p, cm_id=%p\n", - newxprt, newxprt->sc_cm_id); - dev = newxprt->sc_cm_id->device; newxprt->sc_port_num = newxprt->sc_cm_id->port_num; @@ -438,21 +435,17 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) newxprt->sc_pd = ib_alloc_pd(dev, 0); if (IS_ERR(newxprt->sc_pd)) { - dprintk("svcrdma: error creating PD for connect request\n"); + trace_svcrdma_pd_err(newxprt, PTR_ERR(newxprt->sc_pd)); goto errout; } newxprt->sc_sq_cq = ib_alloc_cq_any(dev, newxprt, newxprt->sc_sq_depth, IB_POLL_WORKQUEUE); - if (IS_ERR(newxprt->sc_sq_cq)) { - dprintk("svcrdma: error creating SQ CQ for connect request\n"); + if (IS_ERR(newxprt->sc_sq_cq)) goto errout; - } newxprt->sc_rq_cq = ib_alloc_cq_any(dev, newxprt, rq_depth, IB_POLL_WORKQUEUE); - if (IS_ERR(newxprt->sc_rq_cq)) { - dprintk("svcrdma: error creating RQ CQ for connect request\n"); + if (IS_ERR(newxprt->sc_rq_cq)) goto errout; - } memset(&qp_attr, 0, sizeof qp_attr); qp_attr.event_handler = qp_event_handler; @@ -476,7 +469,7 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) ret = rdma_create_qp(newxprt->sc_cm_id, newxprt->sc_pd, &qp_attr); if (ret) { - dprintk("svcrdma: failed to create QP, ret=%d\n", ret); + trace_svcrdma_qp_err(newxprt, ret); goto errout; } newxprt->sc_qp = newxprt->sc_cm_id->qp; @@ -484,8 +477,10 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) if (!(dev->attrs.device_cap_flags & IB_DEVICE_MEM_MGT_EXTENSIONS)) newxprt->sc_snd_w_inv = false; if (!rdma_protocol_iwarp(dev, newxprt->sc_port_num) && - !rdma_ib_or_roce(dev, newxprt->sc_port_num)) + !rdma_ib_or_roce(dev, newxprt->sc_port_num)) { + trace_svcrdma_fabric_err(newxprt, -EINVAL); goto errout; + } if (!svc_rdma_post_recvs(newxprt)) goto errout; @@ -507,15 +502,17 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) conn_param.initiator_depth = min_t(int, newxprt->sc_ord, dev->attrs.max_qp_init_rd_atom); if (!conn_param.initiator_depth) { - dprintk("svcrdma: invalid ORD setting\n"); ret = -EINVAL; + trace_svcrdma_initdepth_err(newxprt, ret); goto errout; } conn_param.private_data = &pmsg; conn_param.private_data_len = sizeof(pmsg); ret = rdma_accept(newxprt->sc_cm_id, &conn_param); - if (ret) + if (ret) { + trace_svcrdma_accept_err(newxprt, ret); goto errout; + } #if IS_ENABLED(CONFIG_SUNRPC_DEBUG) dprintk("svcrdma: new connection %p accepted:\n", newxprt);