From patchwork Fri May 1 17:37:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11523037 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7FE3681 for ; Fri, 1 May 2020 17:37:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 696462173E for ; Fri, 1 May 2020 17:37:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fYt52uLy" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729602AbgEARhx (ORCPT ); Fri, 1 May 2020 13:37:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729572AbgEARhx (ORCPT ); Fri, 1 May 2020 13:37:53 -0400 Received: from mail-qv1-xf44.google.com (mail-qv1-xf44.google.com [IPv6:2607:f8b0:4864:20::f44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 560F3C061A0C for ; Fri, 1 May 2020 10:37:53 -0700 (PDT) Received: by mail-qv1-xf44.google.com with SMTP id v38so5073621qvf.6 for ; Fri, 01 May 2020 10:37:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=w1iUtLJSPCUIGZsawUGcopkfcHeFfCERO+QIffqI47g=; b=fYt52uLyomqz+WPeRaXI4ohWZCIi8gJGMS1Y8hqoN6w7P//IozfepiEIR8qRjJj6Kp vbGwRYSwjE06Gv5KrGj+tXSUzYO9Ia6UFurPczScwPtBa5H6usz5vTxSWU2S4tjvh2Xx YRB8KuMM0LNCqSOnbWNRBbiUUjUqOwjTfDY+dTXmzJs3ILR5f9B22QU/vpRVr3JzLOAJ VxK9VqRLoiri8n0D4bzxoTIyH0YI7d09motOoBMyarLH2QHjM3gx9aH0CPGHMWN+VOzf apBdzrNQQpyR+XAlrrDDU9ZTpFoWg0dcZL/r6OWl+g+1QVttTdPNM6mVqkeYf5hCcThd 0pVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=w1iUtLJSPCUIGZsawUGcopkfcHeFfCERO+QIffqI47g=; b=CCEjzOupLDqN1RavtyULzNR0gmVQX7+4U+RnzMt6WpGZBB+8+SIYb03m6QPZUCA1zW VDNmr21w5FF8RbjeXNgJayzWFb8fZhf0rgXl18gIXtuSIMWB7MgPDSxK5po8Epar3ywh fdJodgddUIwPiJj6KBCl7d3nKKKXhGij6TCALQRUvEo5AQ7wuziA79uc5v9orLjtTEgY 2CF0MAUSsing8ejWs2JBVWd87/5P7h9on99PE++bRdK9GJ/NWoxcclYl3ojQJMHMTkqB Al2aVIcHIJpLFPtFVBH2N95vvFZOyhsxtbOXf54TsJeKOTdQtYNy2ieRZbX5lqIoWYwu TpDQ== X-Gm-Message-State: AGi0PuajWBSVeKQH1nxXW+/Vkgn8642hDidNGoH7PefjmSbzgUDxHbeb Uu/NL1jElX3c0fDzph2oPIUb8/QZ X-Google-Smtp-Source: APiQypJQVEsLYrKuothng1OwUJDerQwmghMkKP8XAH7qcAA7XdJKEqJyIO/TFrCWipCTeNOg700bmA== X-Received: by 2002:a05:6214:bc6:: with SMTP id ff6mr5161473qvb.43.1588354672451; Fri, 01 May 2020 10:37:52 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id h13sm3129376qti.32.2020.05.01.10.37.51 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:37:51 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HboJL026728 for ; Fri, 1 May 2020 17:37:50 GMT Subject: [PATCH v1 1/8] SUNRPC: Remove "#include " From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Fri, 01 May 2020 13:37:50 -0400 Message-ID: <20200501173750.3868.17508.stgit@klimt.1015granger.net> In-Reply-To: <20200501173526.3868.96971.stgit@klimt.1015granger.net> References: <20200501173526.3868.96971.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: Commit 850cbaddb52d ("udp: use it's own memory accounting schema") removed the last skb trace event from svcsock.c, so it is no longer necessary to include trace/events/skb.h. Signed-off-by: Chuck Lever --- net/sunrpc/svcsock.c | 1 - 1 file changed, 1 deletion(-) diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index cf4bd198c19d..1c4d0aacc531 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -45,7 +45,6 @@ #include #include #include -#include #include #include From patchwork Fri May 1 17:37:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11523041 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A11F781 for ; Fri, 1 May 2020 17:38:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8434F2173E for ; Fri, 1 May 2020 17:38:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NiR6LNxY" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729434AbgEARiA (ORCPT ); Fri, 1 May 2020 13:38:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33324 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729040AbgEARiA (ORCPT ); Fri, 1 May 2020 13:38:00 -0400 Received: from mail-qv1-xf42.google.com (mail-qv1-xf42.google.com [IPv6:2607:f8b0:4864:20::f42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F97BC061A0C for ; Fri, 1 May 2020 10:38:00 -0700 (PDT) Received: by mail-qv1-xf42.google.com with SMTP id p13so5059844qvt.12 for ; Fri, 01 May 2020 10:38:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=vb4DcyADCr/rwXplOufRwBAwRVk61kowYNusZULLRXY=; b=NiR6LNxYLn6tkFfIsyqnXzCZY7HXqwPq5ulGjBsNvZNEuscYroODPPoPZxkXuZH1o3 fuz7maiPNqqF1V41jnIX0y9DLvqtCl319dCmJf28uCQ7e9rej0Dr3dwszo2QwbEcGl/I tC/Hz6XMpvaFrd8TI6QX4tr1yFivVOPEZmfAGXSd25hPl7qgFjqA3FEhcepTmJEHryBW 3I4Y1kflRJE30rqLpoX3dyuYgqZeBerchq5l5WRbfpZ/JosGkFBO2Z3iOjopy+wGMlaD EwRjpPUmN7Bpa0xD2KWbANN50VU5e5ibLyThiQfewUb/90ECChWwENJRk6vvZqEHDhxE BLDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=vb4DcyADCr/rwXplOufRwBAwRVk61kowYNusZULLRXY=; b=a5ot5si6DaeLInA8krvKID248rJhKMugzE6sDCbY0mEsEhr7iEsvjx7wh0dAxa7cj4 hs8fbyOZ1JxHYg02OdfhQPUwFj6Ov+whJ4SzZLCjR+wHmb1YcHxdnYhD036gziSQGB9i emtPoKsoKUWmKnHeIS9qfrTshtohI4hEZyUAZstdlX1NOuVIMjY1KFL+E7wEL9cm5JBS 1Ux8BDs6rQYRJedrPdNIOyXxr37bqF45fLxkdVAxkICdwYr8L5WdTDFGJukgL1vSP1bO vxpAf5ecGcjHtFpiJj+7TysD+bMML/dz4gMlaseqFe9Lhc2aXIBZjMhk0RmQLqpD5Fqo u1pA== X-Gm-Message-State: AGi0PuZMmQyFKr0Yn8J9BbL9GAaWiZK2U3uX40N0mavN2cw2tOA9cUX+ XYm9y52xMtO6x4RIsfzHwCBOVyyP X-Google-Smtp-Source: APiQypKx5y6i+qQRuuxddiBzTHaHlefyUxbHIwQ7LohRMQ4YumrGm+iYRDhF+++ImYZitDBPpPe1Dw== X-Received: by 2002:a0c:e204:: with SMTP id q4mr5026621qvl.16.1588354678082; Fri, 01 May 2020 10:37:58 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id b10sm3035607qkl.19.2020.05.01.10.37.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:37:57 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HbuNE026731 for ; Fri, 1 May 2020 17:37:56 GMT Subject: [PATCH v1 2/8] SUNRPC: Add more svcsock tracepoints From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Fri, 01 May 2020 13:37:56 -0400 Message-ID: <20200501173756.3868.76227.stgit@klimt.1015granger.net> In-Reply-To: <20200501173526.3868.96971.stgit@klimt.1015granger.net> References: <20200501173526.3868.96971.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org In addition to tracing recently-updated socket sendto events, this commit adds a trace event class that can be used for additional svcsock-related tracepoints in subsequent patches. Commit 850cbaddb52d ("udp: use it's own memory accounting schema") removed the last skb trace event from svcsock.c, so it is no longer necessary to include trace/events/skb.h. Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 97 +++++++++++++++++++++++++++++++++++++++++ net/sunrpc/svcsock.c | 30 ++++--------- 2 files changed, 107 insertions(+), 20 deletions(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index 561a438bcc66..95ee3d1a49c2 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -14,6 +14,39 @@ #include #include +TRACE_DEFINE_ENUM(SOCK_STREAM); +TRACE_DEFINE_ENUM(SOCK_DGRAM); +TRACE_DEFINE_ENUM(SOCK_RAW); +TRACE_DEFINE_ENUM(SOCK_RDM); +TRACE_DEFINE_ENUM(SOCK_SEQPACKET); +TRACE_DEFINE_ENUM(SOCK_DCCP); +TRACE_DEFINE_ENUM(SOCK_PACKET); + +#define show_socket_type(type) \ + __print_symbolic(type, \ + { SOCK_STREAM, "STREAM" }, \ + { SOCK_DGRAM, "DGRAM" }, \ + { SOCK_RAW, "RAW" }, \ + { SOCK_RDM, "RDM" }, \ + { SOCK_SEQPACKET, "SEQPACKET" }, \ + { SOCK_DCCP, "DCCP" }, \ + { SOCK_PACKET, "PACKET" }) + +/* This list is known to be incomplete, add new enums as needed. */ +TRACE_DEFINE_ENUM(AF_UNSPEC); +TRACE_DEFINE_ENUM(AF_UNIX); +TRACE_DEFINE_ENUM(AF_LOCAL); +TRACE_DEFINE_ENUM(AF_INET); +TRACE_DEFINE_ENUM(AF_INET6); + +#define rpc_show_address_family(family) \ + __print_symbolic(family, \ + { AF_UNSPEC, "AF_UNSPEC" }, \ + { AF_UNIX, "AF_UNIX" }, \ + { AF_LOCAL, "AF_LOCAL" }, \ + { AF_INET, "AF_INET" }, \ + { AF_INET6, "AF_INET6" }) + DECLARE_EVENT_CLASS(xdr_buf_class, TP_PROTO( const struct xdr_buf *xdr @@ -1400,6 +1433,70 @@ DEFINE_SVC_DEFERRED_EVENT(drop); DEFINE_SVC_DEFERRED_EVENT(queue); DEFINE_SVC_DEFERRED_EVENT(recv); +TRACE_EVENT(svcsock_new_socket, + TP_PROTO( + const struct socket *socket + ), + + TP_ARGS(socket), + + TP_STRUCT__entry( + __field(unsigned long, type) + __field(unsigned long, family) + __field(bool, listener) + ), + + TP_fast_assign( + __entry->type = socket->type; + __entry->family = socket->sk->sk_family; + __entry->listener = (socket->sk->sk_state == TCP_LISTEN); + ), + + TP_printk("type=%s family=%s%s", + show_socket_type(__entry->type), + rpc_show_address_family(__entry->family), + __entry->listener ? " (listener)" : "" + ) +); + +DECLARE_EVENT_CLASS(svcsock_class, + TP_PROTO( + const struct svc_xprt *xprt, + ssize_t result + ), + + TP_ARGS(xprt, result), + + TP_STRUCT__entry( + __field(ssize_t, result) + __field(unsigned long, flags) + __string(addr, xprt->xpt_remotebuf) + ), + + TP_fast_assign( + __entry->result = result; + __entry->flags = xprt->xpt_flags; + __assign_str(addr, xprt->xpt_remotebuf); + ), + + TP_printk("addr=%s result=%zd flags=%s", __get_str(addr), + __entry->result, show_svc_xprt_flags(__entry->flags) + ) +); + +#define DEFINE_SVCSOCK_EVENT(name) \ + DEFINE_EVENT(svcsock_class, svcsock_##name, \ + TP_PROTO( \ + const struct svc_xprt *xprt, \ + ssize_t result \ + ), \ + TP_ARGS(xprt, result)) + +DEFINE_SVCSOCK_EVENT(udp_send); +DEFINE_SVCSOCK_EVENT(tcp_send); +DEFINE_SVCSOCK_EVENT(data_ready); +DEFINE_SVCSOCK_EVENT(write_space); + DECLARE_EVENT_CLASS(cache_event, TP_PROTO( const struct cache_detail *cd, diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 1c4d0aacc531..758b835ad4ce 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -54,6 +54,8 @@ #include #include +#include + #include "socklib.h" #include "sunrpc.h" @@ -281,13 +283,10 @@ static void svc_data_ready(struct sock *sk) struct svc_sock *svsk = (struct svc_sock *)sk->sk_user_data; if (svsk) { - dprintk("svc: socket %p(inet %p), busy=%d\n", - svsk, sk, - test_bit(XPT_BUSY, &svsk->sk_xprt.xpt_flags)); - /* Refer to svc_setup_socket() for details. */ rmb(); svsk->sk_odata(sk); + trace_svcsock_data_ready(&svsk->sk_xprt, 0); if (!test_and_set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags)) svc_xprt_enqueue(&svsk->sk_xprt); } @@ -301,11 +300,9 @@ static void svc_write_space(struct sock *sk) struct svc_sock *svsk = (struct svc_sock *)(sk->sk_user_data); if (svsk) { - dprintk("svc: socket %p(inet %p), write_space busy=%d\n", - svsk, sk, test_bit(XPT_BUSY, &svsk->sk_xprt.xpt_flags)); - /* Refer to svc_setup_socket() for details. */ rmb(); + trace_svcsock_write_space(&svsk->sk_xprt, 0); svsk->sk_owspace(sk); svc_xprt_enqueue(&svsk->sk_xprt); } @@ -545,6 +542,7 @@ static int svc_udp_sendto(struct svc_rqst *rqstp) err = xprt_sock_sendmsg(svsk->sk_sock, &msg, xdr, 0, 0, &sent); xdr_free_bvec(xdr); } + trace_svcsock_udp_send(xprt, err); mutex_unlock(&xprt->xpt_mutex); if (err < 0) @@ -616,7 +614,7 @@ static struct svc_xprt_class svc_udp_class = { static void svc_udp_init(struct svc_sock *svsk, struct svc_serv *serv) { - int err, level, optname, one = 1; + int level, optname, one = 1; svc_xprt_init(sock_net(svsk->sk_sock->sk), &svc_udp_class, &svsk->sk_xprt, serv); @@ -647,9 +645,8 @@ static void svc_udp_init(struct svc_sock *svsk, struct svc_serv *serv) default: BUG(); } - err = kernel_setsockopt(svsk->sk_sock, level, optname, - (char *)&one, sizeof(one)); - dprintk("svc: kernel_setsockopt returned %d\n", err); + kernel_setsockopt(svsk->sk_sock, level, optname, (char *)&one, + sizeof(one)); } /* @@ -1100,6 +1097,7 @@ static int svc_tcp_sendto(struct svc_rqst *rqstp) goto out_notconn; err = xprt_sock_sendmsg(svsk->sk_sock, &msg, xdr, 0, marker, &sent); xdr_free_bvec(xdr); + trace_svcsock_tcp_send(xprt, err < 0 ? err : sent); if (err < 0 || sent != (xdr->len + sizeof(marker))) goto out_close; mutex_unlock(&xprt->xpt_mutex); @@ -1170,13 +1168,11 @@ static void svc_tcp_init(struct svc_sock *svsk, struct svc_serv *serv) set_bit(XPT_CACHE_AUTH, &svsk->sk_xprt.xpt_flags); set_bit(XPT_CONG_CTRL, &svsk->sk_xprt.xpt_flags); if (sk->sk_state == TCP_LISTEN) { - dprintk("setting up TCP socket for listening\n"); strcpy(svsk->sk_xprt.xpt_remotebuf, "listener"); set_bit(XPT_LISTENER, &svsk->sk_xprt.xpt_flags); sk->sk_data_ready = svc_tcp_listen_data_ready; set_bit(XPT_CONN, &svsk->sk_xprt.xpt_flags); } else { - dprintk("setting up TCP socket for reading\n"); sk->sk_state_change = svc_tcp_state_change; sk->sk_data_ready = svc_data_ready; sk->sk_write_space = svc_write_space; @@ -1226,7 +1222,6 @@ static struct svc_sock *svc_setup_socket(struct svc_serv *serv, int pmap_register = !(flags & SVC_SOCK_ANONYMOUS); int err = 0; - dprintk("svc: svc_setup_socket %p\n", sock); svsk = kzalloc(sizeof(*svsk), GFP_KERNEL); if (!svsk) return ERR_PTR(-ENOMEM); @@ -1263,12 +1258,7 @@ static struct svc_sock *svc_setup_socket(struct svc_serv *serv, else svc_tcp_init(svsk, serv); - dprintk("svc: svc_setup_socket created %p (inet %p), " - "listen %d close %d\n", - svsk, svsk->sk_sk, - test_bit(XPT_LISTENER, &svsk->sk_xprt.xpt_flags), - test_bit(XPT_CLOSE, &svsk->sk_xprt.xpt_flags)); - + trace_svcsock_new_socket(sock); return svsk; } From patchwork Fri May 1 17:38:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11523043 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 876DF913 for ; Fri, 1 May 2020 17:38:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6A8E12173E for ; Fri, 1 May 2020 17:38:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mcIDFDsi" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729705AbgEARiE (ORCPT ); Fri, 1 May 2020 13:38:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33336 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729040AbgEARiE (ORCPT ); Fri, 1 May 2020 13:38:04 -0400 Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 031F4C061A0C for ; Fri, 1 May 2020 10:38:04 -0700 (PDT) Received: by mail-qt1-x843.google.com with SMTP id g16so4505889qtp.11 for ; Fri, 01 May 2020 10:38:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=QuMsnTvS/IK7TWa2P2FA2ktebEJNuKiUIuY0v+J4dSk=; b=mcIDFDsifngYlOX6I7vVJ3wsWp9OLhmPxv5/Lky01JvXVqLH6ZW+P8iL6o+FGPXrnA p/FVw5pazNpE6WMts+CBxL7Iq4MZWyoCiIXiMsmONPQugN80g9cqhRUPv21BvkPWOQH6 IaRqGCIol8IX3bZmfFymfltx2cXDEI+vdWVlvoV+4FNMfbQ05uAMHqcAUhvZIMKlyDVH 2KOsSRR4FAB7tMyQXNLhen6JbaUIBNJA4r+Dg/8KTlCPYFbGZGzWn50yQCa4VtWNDH0N MRyP6+rTCAL2Am0gHthFG7KI1Ti5c55MtNy3I8LB1hn0OjwSBAGR8T0MAT1AXlhgGpXr 07Mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=QuMsnTvS/IK7TWa2P2FA2ktebEJNuKiUIuY0v+J4dSk=; b=s94Bqx6djVCYPcXDAxDSOlCsXynq8B7K99A6ul1YD/FnaOXzlnPNmPiqZBaa1yV3HT mTxQesNPmJRORp5KOg4ufzCzQ0TOxK0xfuy6hGp7HpbOwF71aGRJGZFuWvGv5f6o1nez eG5EJ3pELhiVjDNnSUMXzqmHuCzdBknpigzpYWqoUDcS/4t+WN//WnmC2n49xWi/+61/ jPl0vR4cdNpISPGRkUps1jsyqw1/RDoQWA8DcjvDPpEpzA3TuN68ZbT7qeQFzrn4rPIp mYgMfdKpTKfGB7OncJEbLgAbRrwCfgsW5RlUH/Q8NLz0xv0+tpH+cy7SO7wrEkkH6kAp 1foQ== X-Gm-Message-State: AGi0PuaI4K5uyuJms2NIusJaxt7mZqPU1VhoyPwAdUpQtH/nbA2dJ1KX ZqiXY6SupSyUhv3RkswbkGImGGxi X-Google-Smtp-Source: APiQypIR4fVNgYa4FTSyD7v/e5M6PJlsIeMw4NSH3RiEIckjxlbeVsoLHLWki+JS4DEPiNyFjJCe+w== X-Received: by 2002:aed:34c3:: with SMTP id x61mr4621312qtd.333.1588354682959; Fri, 01 May 2020 10:38:02 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id e186sm3144411qkb.40.2020.05.01.10.38.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:38:02 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041Hc19l026734 for ; Fri, 1 May 2020 17:38:01 GMT Subject: [PATCH v1 3/8] SUNRPC: Replace dprintk call sites in TCP state change callouts From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Fri, 01 May 2020 13:38:01 -0400 Message-ID: <20200501173801.3868.35817.stgit@klimt.1015granger.net> In-Reply-To: <20200501173526.3868.96971.stgit@klimt.1015granger.net> References: <20200501173526.3868.96971.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Report TCP socket state changes and accept failures via tracepoints, replacing dprintk() call sites. No tracepoint is added in svc_tcp_listen_data_ready. There's no information available there that isn't also reported by the svcsock_new_socket and the accept failure tracepoints. Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 67 +++++++++++++++++++++++++++++++++++++++++ net/sunrpc/svcsock.c | 35 ++++----------------- 2 files changed, 73 insertions(+), 29 deletions(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index 95ee3d1a49c2..cb839ceba89e 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1497,6 +1497,73 @@ DEFINE_SVCSOCK_EVENT(tcp_send); DEFINE_SVCSOCK_EVENT(data_ready); DEFINE_SVCSOCK_EVENT(write_space); +TRACE_EVENT(svcsock_tcp_state, + TP_PROTO( + const struct svc_xprt *xprt, + const struct socket *socket + ), + + TP_ARGS(xprt, socket), + + TP_STRUCT__entry( + __field(unsigned long, socket_state) + __field(unsigned long, sock_state) + __field(unsigned long, flags) + __string(addr, xprt->xpt_remotebuf) + ), + + TP_fast_assign( + __entry->socket_state = socket->state; + __entry->sock_state = socket->sk->sk_state; + __entry->flags = xprt->xpt_flags; + __assign_str(addr, xprt->xpt_remotebuf); + ), + + TP_printk("addr=%s state=%s sk_state=%s flags=%s", __get_str(addr), + rpc_show_socket_state(__entry->socket_state), + rpc_show_sock_state(__entry->sock_state), + show_svc_xprt_flags(__entry->flags) + ) +); + +DECLARE_EVENT_CLASS(svcsock_accept_class, + TP_PROTO( + const struct svc_xprt *xprt, + const char *service, + long status + ), + + TP_ARGS(xprt, service, status), + + TP_STRUCT__entry( + __field(long, status) + __string(service, service) + __array(unsigned char, addr, sizeof(struct sockaddr_in6)) + ), + + TP_fast_assign( + __entry->status = status; + __assign_str(service, service); + memcpy(__entry->addr, &xprt->xpt_local, sizeof(__entry->addr)); + ), + + TP_printk("listener=%pISpc service=%s status=%ld", + __entry->addr, __get_str(service), __entry->status + ) +); + +#define DEFINE_ACCEPT_EVENT(name) \ + DEFINE_EVENT(svcsock_accept_class, svcsock_##name##_err, \ + TP_PROTO( \ + const struct svc_xprt *xprt, \ + const char *service, \ + long status \ + ), \ + TP_ARGS(xprt, service, status)) + +DEFINE_ACCEPT_EVENT(accept); +DEFINE_ACCEPT_EVENT(getpeername); + DECLARE_EVENT_CLASS(cache_event, TP_PROTO( const struct cache_detail *cd, diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 758b835ad4ce..4ac1180c6306 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -657,9 +657,6 @@ static void svc_tcp_listen_data_ready(struct sock *sk) { struct svc_sock *svsk = (struct svc_sock *)sk->sk_user_data; - dprintk("svc: socket %p TCP (listen) state change %d\n", - sk, sk->sk_state); - if (svsk) { /* Refer to svc_setup_socket() for details. */ rmb(); @@ -680,8 +677,7 @@ static void svc_tcp_listen_data_ready(struct sock *sk) if (svsk) { set_bit(XPT_CONN, &svsk->sk_xprt.xpt_flags); svc_xprt_enqueue(&svsk->sk_xprt); - } else - printk("svc: socket %p: no user data\n", sk); + } } } @@ -692,15 +688,11 @@ static void svc_tcp_state_change(struct sock *sk) { struct svc_sock *svsk = (struct svc_sock *)sk->sk_user_data; - dprintk("svc: socket %p TCP (connected) state change %d (svsk %p)\n", - sk, sk->sk_state, sk->sk_user_data); - - if (!svsk) - printk("svc: socket %p: no user data\n", sk); - else { + if (svsk) { /* Refer to svc_setup_socket() for details. */ rmb(); svsk->sk_ostate(sk); + trace_svcsock_tcp_state(&svsk->sk_xprt, svsk->sk_sock); if (sk->sk_state != TCP_ESTABLISHED) { set_bit(XPT_CLOSE, &svsk->sk_xprt.xpt_flags); svc_xprt_enqueue(&svsk->sk_xprt); @@ -721,7 +713,6 @@ static struct svc_xprt *svc_tcp_accept(struct svc_xprt *xprt) struct socket *newsock; struct svc_sock *newsvsk; int err, slen; - RPC_IFDEBUG(char buf[RPC_MAX_ADDRBUFLEN]); if (!sock) return NULL; @@ -735,30 +726,18 @@ static struct svc_xprt *svc_tcp_accept(struct svc_xprt *xprt) else if (err != -EAGAIN) net_warn_ratelimited("%s: accept failed (err %d)!\n", serv->sv_name, -err); + trace_svcsock_accept_err(xprt, serv->sv_name, err); return NULL; } set_bit(XPT_CONN, &svsk->sk_xprt.xpt_flags); err = kernel_getpeername(newsock, sin); if (err < 0) { - net_warn_ratelimited("%s: peername failed (err %d)!\n", - serv->sv_name, -err); + trace_svcsock_getpeername_err(xprt, serv->sv_name, err); goto failed; /* aborted connection or whatever */ } slen = err; - /* Ideally, we would want to reject connections from unauthorized - * hosts here, but when we get encryption, the IP of the host won't - * tell us anything. For now just warn about unpriv connections. - */ - if (!svc_port_is_privileged(sin)) { - dprintk("%s: connect from unprivileged port: %s\n", - serv->sv_name, - __svc_print_addr(sin, buf, sizeof(buf))); - } - dprintk("%s: connect from %s\n", serv->sv_name, - __svc_print_addr(sin, buf, sizeof(buf))); - /* Reset the inherited callbacks before calling svc_setup_socket */ newsock->sk->sk_state_change = svsk->sk_ostate; newsock->sk->sk_data_ready = svsk->sk_odata; @@ -776,10 +755,8 @@ static struct svc_xprt *svc_tcp_accept(struct svc_xprt *xprt) svc_xprt_set_remote(&newsvsk->sk_xprt, sin, slen); err = kernel_getsockname(newsock, sin); slen = err; - if (unlikely(err < 0)) { - dprintk("svc_tcp_accept: kernel_getsockname error %d\n", -err); + if (unlikely(err < 0)) slen = offsetof(struct sockaddr, sa_data); - } svc_xprt_set_local(&newsvsk->sk_xprt, sin, slen); if (sock_is_loopback(newsock->sk)) From patchwork Fri May 1 17:38:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11523045 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 77CE1913 for ; Fri, 1 May 2020 17:38:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 60CC32173E for ; Fri, 1 May 2020 17:38:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Wr2ScTnx" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730252AbgEARiK (ORCPT ); Fri, 1 May 2020 13:38:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33354 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729040AbgEARiJ (ORCPT ); Fri, 1 May 2020 13:38:09 -0400 Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5946AC061A0C for ; Fri, 1 May 2020 10:38:09 -0700 (PDT) Received: by mail-qk1-x735.google.com with SMTP id k81so7234794qke.5 for ; Fri, 01 May 2020 10:38:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=j0961NufbplOY1PjHPKeEZy3fMqEqoB5Z/dSp67mIHs=; b=Wr2ScTnxcJadagI5gkoQGtjgsmqRYIa2vHzZcRyamWp+mdR0Q7FP80V04RMW0b1jFG HKdHjNi++dnJ7GCzO8l8q6EG1fSl7alcez00DjC+uXUwZdQc09v+NiWdFy3DuY2SK35H lgJwxycjY7cvktBmpMbuTcGxfsZ32COrBkqlsR6neEVEa+w4lw9QRwPplFJ29KxpNabe mKBJidISfOCKIGMWzSVz6qhs8gm1inlegpgWv6MOjkMPgsfY4n8voyD6xsQOWcMXmCqp rcnUAcgAxcJ3/CBYGMaR7u+BlYlUeJEmMFSpqCh/1yGbORrDAG9QMhwgQCU3fnGeqCB+ PBrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=j0961NufbplOY1PjHPKeEZy3fMqEqoB5Z/dSp67mIHs=; b=h0URgr1jtV37e8A3S3G7PbjeyXYV2fSmWmxx6q2tdmpxq3eozctioleBEvmmfCNsM3 QcaxpBUv/J3aVpLiEd4yJJBZnP0l1zZw2djCOeVJssQqkakSQ7dSe6KpoGKZgnEO2xEy vaUwTBRFBwVzu72BIOX5AL0dxB9iIDulJ++Wq0zL6VejiwkaLRzDeMRCoAZ1g9ZPqcwQ l7eia3hU2tgBnEg8bfQbishzu1MriLehedTS2ndGuXh09Xe960Jus4sbbUNvZoIyaLqn F1AKCkUsL1Wks1d6bHe7EDM3EVP/D8BbufT7lEAl0GRPfgLycAmy/zDqlZromtKS3mW5 mZQA== X-Gm-Message-State: AGi0PuYoY5g/HQdBmhyQhWTu/jrZQZrmiw7DH3d5zkn5iCbW/LxyaJm1 rM2D2AA53iI6rhDe80N3zp/X7DXo X-Google-Smtp-Source: APiQypJTvFlWvlL6a1QR0hxIO6K8UaSo3KXZML3qbJ0eluC169Ufgdi2+EyDNtyZeCwN2/vqh5HH3Q== X-Received: by 2002:a37:9344:: with SMTP id v65mr2261619qkd.366.1588354688184; Fri, 01 May 2020 10:38:08 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id u17sm2976201qtv.56.2020.05.01.10.38.07 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:38:07 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041Hc6n4026737 for ; Fri, 1 May 2020 17:38:06 GMT Subject: [PATCH v1 4/8] SUNRPC: Trace server-side rpcbind registration events From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Fri, 01 May 2020 13:38:06 -0400 Message-ID: <20200501173806.3868.85854.stgit@klimt.1015granger.net> In-Reply-To: <20200501173526.3868.96971.stgit@klimt.1015granger.net> References: <20200501173526.3868.96971.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 80 +++++++++++++++++++++++++++++++++++++++++ net/sunrpc/svc.c | 15 ++------ 2 files changed, 83 insertions(+), 12 deletions(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index cb839ceba89e..32200745f1b8 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1597,6 +1597,86 @@ DEFINE_CACHE_EVENT(cache_entry_update); DEFINE_CACHE_EVENT(cache_entry_make_negative); DEFINE_CACHE_EVENT(cache_entry_no_listener); +DECLARE_EVENT_CLASS(register_class, + TP_PROTO( + const char *program, + const u32 version, + const int family, + const unsigned short protocol, + const unsigned short port, + int error + ), + + TP_ARGS(program, version, family, protocol, port, error), + + TP_STRUCT__entry( + __field(u32, version) + __field(unsigned long, family) + __field(unsigned short, protocol) + __field(unsigned short, port) + __field(int, error) + __string(program, program) + ), + + TP_fast_assign( + __entry->version = version; + __entry->family = family; + __entry->protocol = protocol; + __entry->port = port; + __entry->error = error; + __assign_str(program, program); + ), + + TP_printk("program=%sv%u proto=%s port=%u family=%s error=%d", + __get_str(program), __entry->version, + __entry->protocol == IPPROTO_UDP ? "udp" : "tcp", + __entry->port, rpc_show_address_family(__entry->family), + __entry->error + ) +); + +#define DEFINE_REGISTER_EVENT(name) \ + DEFINE_EVENT(register_class, svc_##name, \ + TP_PROTO( \ + const char *program, \ + const u32 version, \ + const int family, \ + const unsigned short protocol, \ + const unsigned short port, \ + int error \ + ), \ + TP_ARGS(program, version, family, protocol, \ + port, error)) + +DEFINE_REGISTER_EVENT(register); +DEFINE_REGISTER_EVENT(noregister); + +TRACE_EVENT(svc_unregister, + TP_PROTO( + const char *program, + const u32 version, + int error + ), + + TP_ARGS(program, version, error), + + TP_STRUCT__entry( + __field(u32, version) + __field(int, error) + __string(program, program) + ), + + TP_fast_assign( + __entry->version = version; + __entry->error = error; + __assign_str(program, program); + ), + + TP_printk("program=%sv%u error=%d", + __get_str(program), __entry->version, __entry->error + ) +); + #endif /* _TRACE_SUNRPC_H */ #include diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index 9ed3126600ce..61473fcde92b 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -991,6 +991,7 @@ static int __svc_register(struct net *net, const char *progname, #endif } + trace_svc_register(progname, version, protocol, port, family, error); return error; } @@ -1000,11 +1001,6 @@ int svc_rpcbind_set_version(struct net *net, unsigned short proto, unsigned short port) { - dprintk("svc: svc_register(%sv%d, %s, %u, %u)\n", - progp->pg_name, version, - proto == IPPROTO_UDP? "udp" : "tcp", - port, family); - return __svc_register(net, progp->pg_name, progp->pg_prog, version, family, proto, port); @@ -1024,11 +1020,7 @@ int svc_generic_rpcbind_set(struct net *net, return 0; if (vers->vs_hidden) { - dprintk("svc: svc_register(%sv%d, %s, %u, %u)" - " (but not telling portmap)\n", - progp->pg_name, version, - proto == IPPROTO_UDP? "udp" : "tcp", - port, family); + trace_svc_noregister(progp->pg_name, version, proto, port, family, 0); return 0; } @@ -1106,8 +1098,7 @@ static void __svc_unregister(struct net *net, const u32 program, const u32 versi if (error == -EPROTONOSUPPORT) error = rpcb_register(net, program, version, 0, 0); - dprintk("svc: %s(%sv%u), error %d\n", - __func__, progname, version, error); + trace_svc_unregister(progname, version, error); } /* From patchwork Fri May 1 17:38:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11523047 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 56DB281 for ; Fri, 1 May 2020 17:38:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3FB792173E for ; Fri, 1 May 2020 17:38:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="rp5tYDyS" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729221AbgEARiP (ORCPT ); Fri, 1 May 2020 13:38:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729040AbgEARiO (ORCPT ); Fri, 1 May 2020 13:38:14 -0400 Received: from mail-qv1-xf42.google.com (mail-qv1-xf42.google.com [IPv6:2607:f8b0:4864:20::f42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4C06C061A0C for ; Fri, 1 May 2020 10:38:14 -0700 (PDT) Received: by mail-qv1-xf42.google.com with SMTP id 59so5045123qva.13 for ; Fri, 01 May 2020 10:38:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=AJMwKyVCLJ7t1pONDCzPyqQfa9MtjCvSaNEq/HttZ2U=; b=rp5tYDySU/MKhaSQVLQgjWjPttY2vSxICS3cb2Qv8qdRZTVIooQod9yx3/8JcTkkKk j1bBa0Iin9iW3DyQGBieQ1JBpOYMH4dpFoumbmNYSx7kgQbQb7jw35FM9YdSH9syHu3j QC79oLz+7bhZgKApCgf0A2L3Jb5F4DXM6JqQWxZQ5sWgkAhpJ5ceSrOIuyujzOiG0H/k ea66gtDhnXd+IY8b9554Nn8zFhdCzr3urg4I9Zsdn/eVBveOfubrUQyPrckgJPMLr2PS IsTqeWTCsne5olbkaoMtqAND44OxUnYj5ZfcOWt6kDx2YDv/DCj95clb4WzUIt2Bs4BT 9vTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=AJMwKyVCLJ7t1pONDCzPyqQfa9MtjCvSaNEq/HttZ2U=; b=lncC9+38/RJuVUSlT1GIkblhebBukRifXYmGBxRy7rBSyHCYTqXqDDDNqnQLG1J7TV X16o5I7IannunELZh0U/hIm/0Bqr5oA23/1Xcb4ObdB7iUgJ9YoDla7pGvpRXn+SPOZY XtzTRoKNfEOoUDBOHhpzx8Yfm7ESnErQYK5ihiTwynt4XHBm0MulM4O8Pz9xyHT+OrMr fBUj4XXVxl5+baYnPwlJuwsdIuOigJoxzqgzXdiJ+SAWQX2XJSQVlj+C7IKI5/jHQzaF IV04/oS0Nt1On+RMrKEfHEzZ4YkC9HBNPTss3X38oLDUQq0ICUQaFh8uZYQN5Do3ygDs EUTg== X-Gm-Message-State: AGi0PubXWoft205nfwSPReIqZFTvjmKb24qX4Avn5BQxs9ml545GZVOx s9WPJjSfU9Bl8DIq3/2vZ0rrEqZo X-Google-Smtp-Source: APiQypLK60HWENJ4GWe0XmXR7neOk2jLfuSeKkE77d1AVoZiDD5roGQuKtJvmex67RHwyrfqWT4jkA== X-Received: by 2002:a0c:c190:: with SMTP id n16mr5220362qvh.107.1588354693456; Fri, 01 May 2020 10:38:13 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id d4sm2935408qtw.25.2020.05.01.10.38.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:38:13 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HcBJP026740 for ; Fri, 1 May 2020 17:38:11 GMT Subject: [PATCH v1 5/8] SUNRPC: Clean up: Rename svc_sock::sk_reclen From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Fri, 01 May 2020 13:38:11 -0400 Message-ID: <20200501173811.3868.85517.stgit@klimt.1015granger.net> In-Reply-To: <20200501173526.3868.96971.stgit@klimt.1015granger.net> References: <20200501173526.3868.96971.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org I find the name of the svc_sock::sk_reclen field confusing, so I've changed it to better reflect its function. This field is not read directly to get the record length. Rather, it is a buffer containing a record marker that needs to be decoded. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svcsock.h | 6 +++--- net/sunrpc/svcsock.c | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index 771baadaee9d..b7ac7fe68306 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -28,7 +28,7 @@ struct svc_sock { /* private TCP part */ /* On-the-wire fragment header: */ - __be32 sk_reclen; + __be32 sk_marker; /* As we receive a record, this includes the length received so * far (including the fragment header): */ u32 sk_tcplen; @@ -41,12 +41,12 @@ struct svc_sock { static inline u32 svc_sock_reclen(struct svc_sock *svsk) { - return ntohl(svsk->sk_reclen) & RPC_FRAGMENT_SIZE_MASK; + return be32_to_cpu(svsk->sk_marker) & RPC_FRAGMENT_SIZE_MASK; } static inline u32 svc_sock_final_rec(struct svc_sock *svsk) { - return ntohl(svsk->sk_reclen) & RPC_LAST_STREAM_FRAGMENT; + return be32_to_cpu(svsk->sk_marker) & RPC_LAST_STREAM_FRAGMENT; } /* diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 4ac1180c6306..d63b21f3f207 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -841,7 +841,7 @@ static int svc_tcp_recv_record(struct svc_sock *svsk, struct svc_rqst *rqstp) struct kvec iov; want = sizeof(rpc_fraghdr) - svsk->sk_tcplen; - iov.iov_base = ((char *) &svsk->sk_reclen) + svsk->sk_tcplen; + iov.iov_base = ((char *)&svsk->sk_marker) + svsk->sk_tcplen; iov.iov_len = want; len = svc_recvfrom(rqstp, &iov, 1, want, 0); if (len < 0) @@ -938,7 +938,7 @@ static void svc_tcp_fragment_received(struct svc_sock *svsk) svc_sock_final_rec(svsk) ? "final" : "nonfinal", svc_sock_reclen(svsk)); svsk->sk_tcplen = 0; - svsk->sk_reclen = 0; + svsk->sk_marker = xdr_zero; } /* @@ -1154,7 +1154,7 @@ static void svc_tcp_init(struct svc_sock *svsk, struct svc_serv *serv) sk->sk_data_ready = svc_data_ready; sk->sk_write_space = svc_write_space; - svsk->sk_reclen = 0; + svsk->sk_marker = xdr_zero; svsk->sk_tcplen = 0; svsk->sk_datalen = 0; memset(&svsk->sk_pages[0], 0, sizeof(svsk->sk_pages)); From patchwork Fri May 1 17:38:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11523049 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5366B81 for ; Fri, 1 May 2020 17:38:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3BF792173E for ; Fri, 1 May 2020 17:38:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ki5VScw4" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729552AbgEARiU (ORCPT ); Fri, 1 May 2020 13:38:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33386 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729040AbgEARiT (ORCPT ); Fri, 1 May 2020 13:38:19 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8FA93C061A0C for ; Fri, 1 May 2020 10:38:19 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id f83so2572541qke.13 for ; Fri, 01 May 2020 10:38:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=1d5HyKY/8aMXAU82uYfN4p45cMe4PgXtVEIUFVnr5I0=; b=ki5VScw42L4N0APqlNmUYVJhf7TuwAdc2vAj+Pmnn8U39trHWU3od09XOKW8ofxJMR wK+QkKLMu0kYePACfxdDDJAr8bPkVmV4Lhxgx2b8FIe6pPE2U2BFAKLQaK+DVODiayYH vfQtMIC/7SddNVZMpWQGEqM7JIeLo+tFe3e6IaZFqFJz11HG97bDs1Qw7ifP+11xAk55 NM1ZiCYM6fsLNlqo/3RXa0FtRztx1LjGyhcSaqmP3UfznBF1DGVtV7Fa4NvdjHHeswEF zDS7v+MW9F/sOeC87RbJM5YmuCmLA/txbM0n+mkx3mC5bcF8swAWchqOkwVSGx1kRnTr dMSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=1d5HyKY/8aMXAU82uYfN4p45cMe4PgXtVEIUFVnr5I0=; b=GNngFWp+iX8hW3ZPFouWL5bKGFBccFCTv0jTKQargrPnrBWADU2y9nQw0hhOavWpdC k34TfhWCfoQ3GuqB5SGpugiGqUERvcc8/zW+JS6/gzdz/kVjXhiBGF7sIKzAxJsgi2i/ G3ufMunL5pcHQ89lbVRYt0BW68N45I6S6qjWeNA6pO4s/njC1uY4sVnJiuVqMmCCG7kb yJ6FPBcqGutqpPXXglZ5axTmwHHT/H/IF3W8D0eB1+S8/ue1XPbE1Mms+zRAGh19z7tq nm1yx5bBKT6cvpizvfbHEwpDkLHi8OdORVk6ttqI7gYsYveJUhahqi4afb0uDRGUWA4j ApYQ== X-Gm-Message-State: AGi0PuYAE/k6V42JEtHBNqbte3LkoQ85evu3TuMfTe43SAgyreRPkexa dLikNoYSuheKB9w+71R2V0/wOUPo X-Google-Smtp-Source: APiQypL+peqYcLcU+2jz9+Zvuab72LcNwuTaTAk610qw12PaX2L4exoNVZ2L94oeGc14DyLxrbeTKw== X-Received: by 2002:a37:9f4a:: with SMTP id i71mr4683838qke.132.1588354698542; Fri, 01 May 2020 10:38:18 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id a195sm1966411qkc.87.2020.05.01.10.38.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:38:18 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HcHnb026743 for ; Fri, 1 May 2020 17:38:17 GMT Subject: [PATCH v1 6/8] SUNRPC: Restructure svc_tcp_recv_record() From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Fri, 01 May 2020 13:38:17 -0400 Message-ID: <20200501173817.3868.97075.stgit@klimt.1015granger.net> In-Reply-To: <20200501173526.3868.96971.stgit@klimt.1015granger.net> References: <20200501173526.3868.96971.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Refactor: svc_recvfrom() is going to be converted to read into bio_vecs in a moment. Unhook the only other caller, svc_tcp_recv_record(), which just wants to read the 4-byte stream record marker into a kvec. While we're in the area, streamline this helper by straight-lining the hot path, replace dprintk call sites with tracepoints, and reduce the number of atomic bit operations in this path. Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 24 +++++++++++++++++++ net/sunrpc/svcsock.c | 51 ++++++++++++++++++++--------------------- 2 files changed, 49 insertions(+), 26 deletions(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index 32200745f1b8..fd8073d1f3b8 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1459,6 +1459,30 @@ TRACE_EVENT(svcsock_new_socket, ) ); +TRACE_EVENT(svcsock_marker, + TP_PROTO( + const struct svc_xprt *xprt, + __be32 marker + ), + + TP_ARGS(xprt, marker), + + TP_STRUCT__entry( + __field(unsigned int, length) + __field(bool, last) + __string(addr, xprt->xpt_remotebuf) + ), + + TP_fast_assign( + __entry->length = be32_to_cpu(marker) & RPC_FRAGMENT_SIZE_MASK; + __entry->last = be32_to_cpu(marker) & RPC_LAST_STREAM_FRAGMENT; + __assign_str(addr, xprt->xpt_remotebuf); + ), + + TP_printk("addr=%s length=%u%s", __get_str(addr), + __entry->length, __entry->last ? " (last)" : "") +); + DECLARE_EVENT_CLASS(svcsock_class, TP_PROTO( const struct svc_xprt *xprt, diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index d63b21f3f207..9c1eb13aa9b8 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -828,47 +828,45 @@ static void svc_tcp_clear_pages(struct svc_sock *svsk) } /* - * Receive fragment record header. - * If we haven't gotten the record length yet, get the next four bytes. + * Receive fragment record header into sk_marker. */ -static int svc_tcp_recv_record(struct svc_sock *svsk, struct svc_rqst *rqstp) +static ssize_t svc_tcp_read_marker(struct svc_sock *svsk, + struct svc_rqst *rqstp) { - struct svc_serv *serv = svsk->sk_xprt.xpt_server; - unsigned int want; - int len; + ssize_t want, len; + /* If we haven't gotten the record length yet, + * get the next four bytes. + */ if (svsk->sk_tcplen < sizeof(rpc_fraghdr)) { + struct msghdr msg = { NULL }; struct kvec iov; want = sizeof(rpc_fraghdr) - svsk->sk_tcplen; iov.iov_base = ((char *)&svsk->sk_marker) + svsk->sk_tcplen; iov.iov_len = want; - len = svc_recvfrom(rqstp, &iov, 1, want, 0); + iov_iter_kvec(&msg.msg_iter, READ, &iov, 1, want); + len = sock_recvmsg(svsk->sk_sock, &msg, MSG_DONTWAIT); if (len < 0) - goto error; + return len; svsk->sk_tcplen += len; - if (len < want) { - dprintk("svc: short recvfrom while reading record " - "length (%d of %d)\n", len, want); - return -EAGAIN; + /* call again to read the remaining bytes */ + goto err_short; } - - dprintk("svc: TCP record, %d bytes\n", svc_sock_reclen(svsk)); + trace_svcsock_marker(&svsk->sk_xprt, svsk->sk_marker); if (svc_sock_reclen(svsk) + svsk->sk_datalen > - serv->sv_max_mesg) { - net_notice_ratelimited("RPC: fragment too large: %d\n", - svc_sock_reclen(svsk)); - goto err_delete; - } + svsk->sk_xprt.xpt_server->sv_max_mesg) + goto err_too_large; } - return svc_sock_reclen(svsk); -error: - dprintk("RPC: TCP recv_record got %d\n", len); - return len; -err_delete: + +err_too_large: + net_notice_ratelimited("svc: %s %s RPC fragment too large: %d\n", + __func__, svsk->sk_xprt.xpt_server->sv_name, + svc_sock_reclen(svsk)); set_bit(XPT_CLOSE, &svsk->sk_xprt.xpt_flags); +err_short: return -EAGAIN; } @@ -961,12 +959,13 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) test_bit(XPT_CONN, &svsk->sk_xprt.xpt_flags), test_bit(XPT_CLOSE, &svsk->sk_xprt.xpt_flags)); - len = svc_tcp_recv_record(svsk, rqstp); + clear_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); + len = svc_tcp_read_marker(svsk, rqstp); if (len < 0) goto error; base = svc_tcp_restore_pages(svsk, rqstp); - want = svc_sock_reclen(svsk) - (svsk->sk_tcplen - sizeof(rpc_fraghdr)); + want = len - (svsk->sk_tcplen - sizeof(rpc_fraghdr)); vec = rqstp->rq_vec; From patchwork Fri May 1 17:38:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11523051 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C054981 for ; Fri, 1 May 2020 17:38:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9516624953 for ; Fri, 1 May 2020 17:38:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gcoXnFZU" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729660AbgEARi0 (ORCPT ); Fri, 1 May 2020 13:38:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729570AbgEARiZ (ORCPT ); Fri, 1 May 2020 13:38:25 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6B16BC061A0C for ; Fri, 1 May 2020 10:38:25 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id c63so9962010qke.2 for ; Fri, 01 May 2020 10:38:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=QVSCmi4mEYvlS/H0QTdzPUn0Mhv6H9f+bTK1TkYx4zA=; b=gcoXnFZUeC/IjICvM+7JELquQJqy/1Yt4mOQJGsyo+R4xvoONqOkU5dvUX2iEVyNFE OAR65+8lkTgdGVDeLq/MnrwFckubKIUib2EuEwQFDkeQprMXY+hfUed3fK6q5qJwy6zd 6DUkg8RQL6MmPacg+I0wZXrY0E7HCA3ZMwqFZjmoXttdrYYodiYseBZjI9eSFtAn9DuH gEJW6h3tk6g0L2BmUFWYNphIV8fc13NiowCRLHfvTwb5wvQWltUcN7aoX64IY1car8EU xA4vCCk9XCasnw0p/QE5XfT8PIRmHMqvptC/Tgw1rZYosEDZgWc5d3vMCaLdIX0P3Tzg ULzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=QVSCmi4mEYvlS/H0QTdzPUn0Mhv6H9f+bTK1TkYx4zA=; b=WCXxXX+qunlVNuqLQcJD0SSW2CvneKGxzNsGusl8o0LbpEBqnQ5zTnceuBNb2FfNWm kPjyNeIqoQ8wCfrcxtkb7ZIuZAra+NaMokmeim00/zC1oZ+K5rPaXOtKNP3GOb4rzfbs j7uVMpMQrUqYihv5qiXM/Ka+OU2OOmwELc8xKTk5oSQ6f6aNeGlwS1lEE/bsDq4PyLBh kKNB57Pm0jsmjsDtr3Cv+9w53M6AWJByFUx7i39gGIPy+YEcTkp13lY/4r4HU/PU8UJY xlnZ5CrXWYHjgncej/kflxMCR9SXziHh2DSsU2s+0vLjJHqAIfEAfzAAd1ubpWU3lRJw fPGw== X-Gm-Message-State: AGi0PuZw9JNDfCEAQs2jGBF8HkrT/xHgDXbQvr9Ml/xb0epceAKJy0Di i2PsThyVRUbhF1gQD3jToRJqgMC0 X-Google-Smtp-Source: APiQypI6I6CUDucFqQhveFZSyAELIYmJkDTSrcmoXm5HqhbKDcIGXNm+ts39FDoT8H7cuJzHENnXPA== X-Received: by 2002:ae9:ebc8:: with SMTP id b191mr4508461qkg.1.1588354704128; Fri, 01 May 2020 10:38:24 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id d123sm1009655qkb.28.2020.05.01.10.38.23 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:38:23 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HcMxn026746 for ; Fri, 1 May 2020 17:38:22 GMT Subject: [PATCH v1 7/8] SUNRPC: Refactor svc_recvfrom() From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Fri, 01 May 2020 13:38:22 -0400 Message-ID: <20200501173822.3868.44940.stgit@klimt.1015granger.net> In-Reply-To: <20200501173526.3868.96971.stgit@klimt.1015granger.net> References: <20200501173526.3868.96971.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org This function is not currently "generic" so remove the documenting comment and rename it appropriately. Its internals are converted to use bio_vecs for reading from the transport socket. In existing typical sunrpc uses of bio_vecs, the bio_vec array is allocated dynamically. Here, instead, an array of bio_vecs is added to svc_rqst. The lifetime of this array can be greater than one call to xpo_recvfrom(): - Multiple calls to xpo_recvfrom() might be needed to read an RPC message completely. - At some later point, rq_arg.bvecs will point to this array and it will carry the received message into svc_process(). I also expect that a future optimization will remove either the rq_vec or rq_pages array in favor of rq_bvec, thus conserving the size of struct svc_rqst. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc.h | 1 include/trace/events/sunrpc.h | 1 net/sunrpc/svcsock.c | 190 +++++++++++++++++++++-------------------- 3 files changed, 99 insertions(+), 93 deletions(-) diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h index fd390894a584..05da19a0516d 100644 --- a/include/linux/sunrpc/svc.h +++ b/include/linux/sunrpc/svc.h @@ -254,6 +254,7 @@ struct svc_rqst { struct page * *rq_page_end; /* one past the last page */ struct kvec rq_vec[RPCSVC_MAXPAGES]; /* generally useful.. */ + struct bio_vec rq_bvec[RPCSVC_MAXPAGES]; __be32 rq_xid; /* transmission id */ u32 rq_prog; /* program number */ diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index fd8073d1f3b8..844c0642fcbb 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1518,6 +1518,7 @@ DECLARE_EVENT_CLASS(svcsock_class, DEFINE_SVCSOCK_EVENT(udp_send); DEFINE_SVCSOCK_EVENT(tcp_send); +DEFINE_SVCSOCK_EVENT(tcp_recv); DEFINE_SVCSOCK_EVENT(data_ready); DEFINE_SVCSOCK_EVENT(write_space); diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 9c1eb13aa9b8..f482cfd0d49d 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -109,19 +109,15 @@ static void svc_reclassify_socket(struct socket *sock) } #endif -/* - * Release an skbuff after use - */ -static void svc_release_skb(struct svc_rqst *rqstp) +static void svc_tcp_release_rqst(struct svc_rqst *rqstp) { struct sk_buff *skb = rqstp->rq_xprt_ctxt; if (skb) { struct svc_sock *svsk = container_of(rqstp->rq_xprt, struct svc_sock, sk_xprt); - rqstp->rq_xprt_ctxt = NULL; - dprintk("svc: service %p, releasing skb %p\n", rqstp, skb); + rqstp->rq_xprt_ctxt = NULL; skb_free_datagram_locked(svsk->sk_sk, skb); } } @@ -219,34 +215,60 @@ static int svc_one_sock_name(struct svc_sock *svsk, char *buf, int remaining) return len; } +#if ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE +static void svc_flush_bvec(const struct bio_vec *bvec, size_t size, size_t seek) +{ + struct bvec_iter bi = { + .bi_size = size, + }; + struct bio_vec bv; + + bvec_iter_advance(bvec, &bi, seek & PAGE_MASK); + for_each_bvec(bv, bvec, bi, bi) + flush_dcache_page(bv.bv_page); +} +#else +static inline void svc_flush_bvec(const struct bio_vec *bvec, size_t size, + size_t seek) +{ +} +#endif + /* - * Generic recvfrom routine. + * Read from @rqstp's transport socket. The incoming message fills whole + * pages in @rqstp's rq_pages array until the last page of the message + * has been received into a partial page. */ -static ssize_t svc_recvfrom(struct svc_rqst *rqstp, struct kvec *iov, - unsigned int nr, size_t buflen, unsigned int base) +static ssize_t svc_tcp_read_msg(struct svc_rqst *rqstp, size_t buflen, + size_t seek) { struct svc_sock *svsk = container_of(rqstp->rq_xprt, struct svc_sock, sk_xprt); + struct bio_vec *bvec = rqstp->rq_bvec; struct msghdr msg = { NULL }; + unsigned int i; ssize_t len; + size_t t; - rqstp->rq_xprt_hlen = 0; + for (i = 0, t = 0; t < buflen; i++, t += PAGE_SIZE) { + bvec[i].bv_page = rqstp->rq_pages[i]; + bvec[i].bv_len = PAGE_SIZE; + bvec[i].bv_offset = 0; + } + rqstp->rq_respages = &rqstp->rq_pages[i]; + rqstp->rq_next_page = rqstp->rq_respages + 1; - clear_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); - iov_iter_kvec(&msg.msg_iter, READ, iov, nr, buflen); - if (base != 0) { - iov_iter_advance(&msg.msg_iter, base); - buflen -= base; + iov_iter_bvec(&msg.msg_iter, READ, bvec, i, buflen); + if (seek) { + iov_iter_advance(&msg.msg_iter, seek); + buflen -= seek; } len = sock_recvmsg(svsk->sk_sock, &msg, MSG_DONTWAIT); - /* If we read a full record, then assume there may be more - * data to read (stream based sockets only!) - */ - if (len == buflen) - set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); + if (len < 0) + return len; - dprintk("svc: socket %p recvfrom(%p, %zu) = %zd\n", - svsk, iov[0].iov_base, iov[0].iov_len, len); + svc_flush_bvec(bvec, len, seek); + trace_svcsock_tcp_recv(&svsk->sk_xprt, len); return len; } @@ -773,18 +795,18 @@ static struct svc_xprt *svc_tcp_accept(struct svc_xprt *xprt) return NULL; } -static unsigned int svc_tcp_restore_pages(struct svc_sock *svsk, struct svc_rqst *rqstp) +static size_t svc_tcp_restore_pages(struct svc_sock *svsk, + struct svc_rqst *rqstp) { - unsigned int i, len, npages; + size_t len = svsk->sk_datalen; + unsigned int i, npages; - if (svsk->sk_datalen == 0) - return 0; - len = svsk->sk_datalen; + if (!len) + return len; npages = (len + PAGE_SIZE - 1) >> PAGE_SHIFT; for (i = 0; i < npages; i++) { if (rqstp->rq_pages[i] != NULL) put_page(rqstp->rq_pages[i]); - BUG_ON(svsk->sk_pages[i] == NULL); rqstp->rq_pages[i] = svsk->sk_pages[i]; svsk->sk_pages[i] = NULL; } @@ -915,49 +937,40 @@ static int receive_cb_reply(struct svc_sock *svsk, struct svc_rqst *rqstp) return -EAGAIN; } -static int copy_pages_to_kvecs(struct kvec *vec, struct page **pages, int len) -{ - int i = 0; - int t = 0; - - while (t < len) { - vec[i].iov_base = page_address(pages[i]); - vec[i].iov_len = PAGE_SIZE; - i++; - t += PAGE_SIZE; - } - return i; -} - static void svc_tcp_fragment_received(struct svc_sock *svsk) { /* If we have more data, signal svc_xprt_enqueue() to try again */ - dprintk("svc: TCP %s record (%d bytes)\n", - svc_sock_final_rec(svsk) ? "final" : "nonfinal", - svc_sock_reclen(svsk)); svsk->sk_tcplen = 0; svsk->sk_marker = xdr_zero; } -/* - * Receive data from a TCP socket. +/** + * svc_tcp_recvfrom - Receive data from a TCP socket + * @rqstp: request structure into which to receive an RPC Call + * + * Called in a loop when XPT_DATA has been set. + * + * Read the 4-byte stream record marker, then use the record length + * in that marker to set up exactly the resources needed to receive + * the next RPC message into @rqstp. + * + * Returns: + * On success, the number of bytes in a received RPC Call, or + * %0 if a complete RPC Call message was not ready to return + * + * The zero return case handles partial receives and callback Replies. + * The state of a partial receive is preserved in the svc_sock for + * the next call to svc_tcp_recvfrom. */ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) { struct svc_sock *svsk = container_of(rqstp->rq_xprt, struct svc_sock, sk_xprt); struct svc_serv *serv = svsk->sk_xprt.xpt_server; - int len; - struct kvec *vec; - unsigned int want, base; + size_t want, base; + ssize_t len; __be32 *p; __be32 calldir; - int pnum; - - dprintk("svc: tcp_recv %p data %d conn %d close %d\n", - svsk, test_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags), - test_bit(XPT_CONN, &svsk->sk_xprt.xpt_flags), - test_bit(XPT_CLOSE, &svsk->sk_xprt.xpt_flags)); clear_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); len = svc_tcp_read_marker(svsk, rqstp); @@ -966,38 +979,19 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) base = svc_tcp_restore_pages(svsk, rqstp); want = len - (svsk->sk_tcplen - sizeof(rpc_fraghdr)); - - vec = rqstp->rq_vec; - - pnum = copy_pages_to_kvecs(&vec[0], &rqstp->rq_pages[0], base + want); - - rqstp->rq_respages = &rqstp->rq_pages[pnum]; - rqstp->rq_next_page = rqstp->rq_respages + 1; - - /* Now receive data */ - len = svc_recvfrom(rqstp, vec, pnum, base + want, base); + len = svc_tcp_read_msg(rqstp, base + want, base); if (len >= 0) { svsk->sk_tcplen += len; svsk->sk_datalen += len; } - if (len != want || !svc_sock_final_rec(svsk)) { - svc_tcp_save_pages(svsk, rqstp); - if (len < 0 && len != -EAGAIN) - goto err_delete; - if (len == want) - svc_tcp_fragment_received(svsk); - else - dprintk("svc: incomplete TCP record (%d of %d)\n", - (int)(svsk->sk_tcplen - sizeof(rpc_fraghdr)), - svc_sock_reclen(svsk)); - goto err_noclose; - } - - if (svsk->sk_datalen < 8) { - svsk->sk_datalen = 0; - goto err_delete; /* client is nuts. */ - } + if (len != want || !svc_sock_final_rec(svsk)) + goto err_incomplete; + if (svsk->sk_datalen < 8) + goto err_nuts; + /* + * At this point, a full stream RPC record has been read. + */ rqstp->rq_arg.len = svsk->sk_datalen; rqstp->rq_arg.page_base = 0; if (rqstp->rq_arg.len <= rqstp->rq_arg.head[0].iov_len) { @@ -1005,7 +999,6 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) rqstp->rq_arg.page_len = 0; } else rqstp->rq_arg.page_len = rqstp->rq_arg.len - rqstp->rq_arg.head[0].iov_len; - rqstp->rq_xprt_ctxt = NULL; rqstp->rq_prot = IPPROTO_TCP; if (test_bit(XPT_LOCAL, &svsk->sk_xprt.xpt_flags)) @@ -1018,7 +1011,13 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) if (calldir) len = receive_cb_reply(svsk, rqstp); - /* Reset TCP read info */ + /* + * Force another call to svc_tcp_recvfrom to check for + * more data waiting on the socket. + */ + set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); + + /* Reset TCP read info to prepare for the next record */ svsk->sk_datalen = 0; svc_tcp_fragment_received(svsk); @@ -1028,20 +1027,25 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) svc_xprt_copy_addrs(rqstp, &svsk->sk_xprt); if (serv->sv_stats) serv->sv_stats->nettcpcnt++; - return rqstp->rq_arg.len; +err_incomplete: + svc_tcp_save_pages(svsk, rqstp); + if (len < 0 && len != -EAGAIN) + goto err_delete; + if (len == want) + svc_tcp_fragment_received(svsk); + return 0; error: if (len != -EAGAIN) goto err_delete; - dprintk("RPC: TCP recvfrom got EAGAIN\n"); return 0; +err_nuts: + svsk->sk_datalen = 0; err_delete: - printk(KERN_NOTICE "%s: recvfrom returned errno %d\n", - svsk->sk_xprt.xpt_server->sv_name, -len); + pr_notice("%s: recvfrom returns %zd\n", serv->sv_name, len); set_bit(XPT_CLOSE, &svsk->sk_xprt.xpt_flags); -err_noclose: - return 0; /* record not complete */ + return 0; } /** @@ -1066,7 +1070,7 @@ static int svc_tcp_sendto(struct svc_rqst *rqstp) unsigned int uninitialized_var(sent); int err; - svc_release_skb(rqstp); + svc_tcp_release_rqst(rqstp); mutex_lock(&xprt->xpt_mutex); if (svc_xprt_is_dead(xprt)) @@ -1106,7 +1110,7 @@ static const struct svc_xprt_ops svc_tcp_ops = { .xpo_recvfrom = svc_tcp_recvfrom, .xpo_sendto = svc_tcp_sendto, .xpo_read_payload = svc_sock_read_payload, - .xpo_release_rqst = svc_release_skb, + .xpo_release_rqst = svc_tcp_release_rqst, .xpo_detach = svc_tcp_sock_detach, .xpo_free = svc_sock_free, .xpo_has_wspace = svc_tcp_has_wspace, From patchwork Fri May 1 17:38:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11523053 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5B09A81 for ; Fri, 1 May 2020 17:38:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3D56F2173E for ; Fri, 1 May 2020 17:38:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XwT81lWd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729115AbgEARia (ORCPT ); Fri, 1 May 2020 13:38:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33416 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729040AbgEARia (ORCPT ); Fri, 1 May 2020 13:38:30 -0400 Received: from mail-qv1-xf43.google.com (mail-qv1-xf43.google.com [IPv6:2607:f8b0:4864:20::f43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1BF23C061A0C for ; Fri, 1 May 2020 10:38:30 -0700 (PDT) Received: by mail-qv1-xf43.google.com with SMTP id ck5so5046036qvb.11 for ; Fri, 01 May 2020 10:38:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=r8RQUAU4uIGHsg9iyj0Y/YjWaq9ew2ES00P4l5K0nqQ=; b=XwT81lWdjMqxHPbG7gMTUGQNqjMG61uHlbz6asRwfVS0SrZ6msIY02GHy9VQlas8B4 ZA4kVOLv9Y9kISM5wsupAFgj3dnbw19712c+qBstpKI0zcfegxEDJS+kB7V9GCyiP65o PoxIO1Xd7HQsAvi1pZD+Huv9J3rwOcZDdlAIUDZWkmaAiymfM6n5k3tyn0BGBn0zJjqx tHhusBEl8mNxqPrbVkOSwgXrJrKyXc4bgo20ok7GYBxUwAB9RDTl6kscQlPD8jVy5ev6 4cmB8ucR3jP+LskXkM4gWOOuYUwIpnioKu8MGcJfawD6Dy96hlj22Dy8gE3dRiEHxcrK hMAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=r8RQUAU4uIGHsg9iyj0Y/YjWaq9ew2ES00P4l5K0nqQ=; b=Jz+7nOqH4dwMIKZkBnmLf+0IxOiQXCAyOpLdrZX3fMe/8m+nyceqgbrceuqxcw5CzW Jay9shMwd2JtHcq3zm67nI3QBdeqQGNgWl4wCsnJBySWvaDQLM8lkEqvExZb084b9wEL TjmXNJBLxF9NGvn0cHhm38LmobY+dCTPhItELzz30DXWYCwZNTft13DxOirDO+y8zVks VDFyDq9LXggVkj+im6PLfg2XrDwCvshKDUSF6AKXexZo/9Ayio0FDTjwZe7ipE/Z/EZ1 H8DF6Mk3o2fs4jio3pYD490NTn9QTBd9EWYQdjzTkCiyedIAX0opElTLTOwc9qpOYnwg Lsow== X-Gm-Message-State: AGi0PuaJU5MSXnUfusuPcOS684aZlj+PLkLu5o/kYDVjnyTALBdPZrf+ ZtCDJ95kkgg7949WafHGBcDkN71h X-Google-Smtp-Source: APiQypI0LfjIRzYU35UZmg6A0CRScLqayulXaZAdI6yCYihxdX/qc9VPKr4HkVxhTBM3Cen8XuMH/w== X-Received: by 2002:a0c:9e6d:: with SMTP id z45mr5114170qve.206.1588354709124; Fri, 01 May 2020 10:38:29 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id 10sm3502489qtp.4.2020.05.01.10.38.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:38:28 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HcRri026749 for ; Fri, 1 May 2020 17:38:27 GMT Subject: [PATCH v1 8/8] SUNRPC: Restructure svc_udp_recvfrom() From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Fri, 01 May 2020 13:38:27 -0400 Message-ID: <20200501173827.3868.3139.stgit@klimt.1015granger.net> In-Reply-To: <20200501173526.3868.96971.stgit@klimt.1015granger.net> References: <20200501173526.3868.96971.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org At this point, we are not ready yet to support bio_vecs in the UDP transport. However, we can clean up svc_udp_recvfrom to match the tracing and straight-lining changes made in svc_tcp_recvfrom. Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 1 + net/sunrpc/svcsock.c | 68 +++++++++++++++++++++++------------------ 2 files changed, 39 insertions(+), 30 deletions(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index 844c0642fcbb..b15b5c5dad16 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -1517,6 +1517,7 @@ DECLARE_EVENT_CLASS(svcsock_class, TP_ARGS(xprt, result)) DEFINE_SVCSOCK_EVENT(udp_send); +DEFINE_SVCSOCK_EVENT(udp_recv); DEFINE_SVCSOCK_EVENT(tcp_send); DEFINE_SVCSOCK_EVENT(tcp_recv); DEFINE_SVCSOCK_EVENT(data_ready); diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index f482cfd0d49d..d31a807c2c39 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -122,14 +122,12 @@ static void svc_tcp_release_rqst(struct svc_rqst *rqstp) } } -static void svc_release_udp_skb(struct svc_rqst *rqstp) +static void svc_udp_release_rqst(struct svc_rqst *rqstp) { struct sk_buff *skb = rqstp->rq_xprt_ctxt; if (skb) { rqstp->rq_xprt_ctxt = NULL; - - dprintk("svc: service %p, releasing skb %p\n", rqstp, skb); consume_skb(skb); } } @@ -409,8 +407,15 @@ static int svc_udp_get_dest_address(struct svc_rqst *rqstp, return 0; } -/* - * Receive a datagram from a UDP socket. +/** + * svc_udp_recvfrom - Receive a datagram from a UDP socket. + * @rqstp: request structure into which to receive an RPC Call + * + * Called in a loop when XPT_DATA has been set. + * + * Returns: + * On success, the number of bytes in a received RPC Call, or + * %0 if a complete RPC Call message was not ready to return */ static int svc_udp_recvfrom(struct svc_rqst *rqstp) { @@ -444,20 +449,14 @@ static int svc_udp_recvfrom(struct svc_rqst *rqstp) svc_sock_setbufsize(svsk, serv->sv_nrthreads + 3); clear_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); - skb = NULL; err = kernel_recvmsg(svsk->sk_sock, &msg, NULL, 0, 0, MSG_PEEK | MSG_DONTWAIT); - if (err >= 0) - skb = skb_recv_udp(svsk->sk_sk, 0, 1, &err); - - if (skb == NULL) { - if (err != -EAGAIN) { - /* possibly an icmp error */ - dprintk("svc: recvfrom returned error %d\n", -err); - set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); - } - return 0; - } + if (err < 0) + goto out_recv_err; + skb = skb_recv_udp(svsk->sk_sk, 0, 1, &err); + if (!skb) + goto out_recv_err; + len = svc_addr_len(svc_addr(rqstp)); rqstp->rq_addrlen = len; if (skb->tstamp == 0) { @@ -468,26 +467,21 @@ static int svc_udp_recvfrom(struct svc_rqst *rqstp) sock_write_timestamp(svsk->sk_sk, skb->tstamp); set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); /* there may be more data... */ - len = skb->len; + len = skb->len; rqstp->rq_arg.len = len; + trace_svcsock_udp_recv(&svsk->sk_xprt, len); rqstp->rq_prot = IPPROTO_UDP; - if (!svc_udp_get_dest_address(rqstp, cmh)) { - net_warn_ratelimited("svc: received unknown control message %d/%d; dropping RPC reply datagram\n", - cmh->cmsg_level, cmh->cmsg_type); - goto out_free; - } + if (!svc_udp_get_dest_address(rqstp, cmh)) + goto out_cmsg_err; rqstp->rq_daddrlen = svc_addr_len(svc_daddr(rqstp)); if (skb_is_nonlinear(skb)) { /* we have to copy */ local_bh_disable(); - if (csum_partial_copy_to_xdr(&rqstp->rq_arg, skb)) { - local_bh_enable(); - /* checksum error */ - goto out_free; - } + if (csum_partial_copy_to_xdr(&rqstp->rq_arg, skb)) + goto out_bh_enable; local_bh_enable(); consume_skb(skb); } else { @@ -515,6 +509,20 @@ static int svc_udp_recvfrom(struct svc_rqst *rqstp) serv->sv_stats->netudpcnt++; return len; + +out_recv_err: + if (err != -EAGAIN) { + /* possibly an icmp error */ + set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags); + } + trace_svcsock_udp_recv(&svsk->sk_xprt, err); + return 0; +out_cmsg_err: + net_warn_ratelimited("svc: received unknown control message %d/%d; dropping RPC reply datagram\n", + cmh->cmsg_level, cmh->cmsg_type); + goto out_free; +out_bh_enable: + local_bh_enable(); out_free: kfree_skb(skb); return 0; @@ -548,7 +556,7 @@ static int svc_udp_sendto(struct svc_rqst *rqstp) unsigned int uninitialized_var(sent); int err; - svc_release_udp_skb(rqstp); + svc_udp_release_rqst(rqstp); svc_set_cmsg_data(rqstp, cmh); @@ -617,7 +625,7 @@ static const struct svc_xprt_ops svc_udp_ops = { .xpo_recvfrom = svc_udp_recvfrom, .xpo_sendto = svc_udp_sendto, .xpo_read_payload = svc_sock_read_payload, - .xpo_release_rqst = svc_release_udp_skb, + .xpo_release_rqst = svc_udp_release_rqst, .xpo_detach = svc_sock_detach, .xpo_free = svc_sock_free, .xpo_has_wspace = svc_udp_has_wspace,