From patchwork Thu Sep 30 01:03:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 12527073 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 142D0C433EF for ; Thu, 30 Sep 2021 01:03:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EEEE5615E2 for ; Thu, 30 Sep 2021 01:03:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347561AbhI3BFV (ORCPT ); Wed, 29 Sep 2021 21:05:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48480 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346422AbhI3BFV (ORCPT ); Wed, 29 Sep 2021 21:05:21 -0400 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C80EC06161C for ; Wed, 29 Sep 2021 18:03:39 -0700 (PDT) Received: by mail-pj1-x102f.google.com with SMTP id r7so2969180pjo.3 for ; Wed, 29 Sep 2021 18:03:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=agm9urRkkd7RnPZ1IYFk64CAVDW3XAeYjrfZvS7GRT4=; b=aGEfkAYBo78UQCT4EcUlV2HZN2dgoLRDuTmv3sxShjYw9DbLYeltN+JAJ9NB+m9YFP a+FL2vaQgjNhSV3fixAaxeEjKG2wzjmONpDt70LvYJ6bbZEsYmK2r48oOe8GycbvczYl mU832IF6DGu2UoqJgtQZj/JYnEX5Z65Lsx+r1lZzfen1tktV+9Q6QI7jOvarpMY8adXZ BCuPFvroR6YC1MmPuDf0c+s/q9hJMQ/R20z9CaReI7kDwdPkf6ewFgZpY14JzgofSfId Zb/aRB1qwHyjClOyJB5P44gRHp62PFWSZLK1rcv6CyV/JAKaykkPVUH1v6HBlWtS16wX IjTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=agm9urRkkd7RnPZ1IYFk64CAVDW3XAeYjrfZvS7GRT4=; b=W4xYOty7fUCoGV49pZr5vOS7rZEYVdr7wKzTbJYUI7M3Lp56CLKyTgkEhnb8gef6vc 5sUSt6CJTZFCcHa/5TTwxwY6GiJQFtvCsOdJRwbVrbUqzkf53Q5dMN64paPBowKmImho HOhNkJN/bF2pee4pxlkBWyaB51Kz29oM4Td5OwEkWApLAqp8yq8W6YT1Ska+eRpppHIc Wx3m+8Tv/w/rBAbiFeHAhP4VWWzcTUMD1xkpVLHzgk3lHIxCrTk020LYeV/ibnHvydd7 7btG90NXf/uYN4OwmsnKT+2oXXExHf7RYJbe5GaSEookQAdHQTCeraNX9LlqystGAeTm tfAg== X-Gm-Message-State: AOAM5305hLiwTIEzFQn4aNphjn81Q75P2l0TSHB3MosMTjYPNnivN1+L mNtBx60NEAHv3j/Vb+rltrk= X-Google-Smtp-Source: ABdhPJwL1o41O+6nWd34IeRzwVc+pwDRq9FyBoVNdJYGq+7noNQN7hZfFUgMpr3qwjp6GzukgL6vxw== X-Received: by 2002:a17:90b:1105:: with SMTP id gi5mr9855986pjb.100.1632963818998; Wed, 29 Sep 2021 18:03:38 -0700 (PDT) Received: from edumazet1.svl.corp.google.com ([2620:15c:2c4:201:8858:a0b7:dcc9:9a3b]) by smtp.gmail.com with ESMTPSA id p17sm711695pju.34.2021.09.29.18.03.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Sep 2021 18:03:38 -0700 (PDT) From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski Cc: netdev , Eric Dumazet , Eric Dumazet , Mat Martineau , Matthieu Baerts Subject: [PATCH net-next 1/2] net: snmp: inline snmp_get_cpu_field() Date: Wed, 29 Sep 2021 18:03:32 -0700 Message-Id: <20210930010333.2625706-2-eric.dumazet@gmail.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930010333.2625706-1-eric.dumazet@gmail.com> References: <20210930010333.2625706-1-eric.dumazet@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Eric Dumazet This trivial function is called ~90,000 times on 256 cpus hosts, when reading /proc/net/netstat. And this number keeps inflating. Inlining it saves many cycles. Signed-off-by: Eric Dumazet --- include/net/ip.h | 6 +++++- net/ipv4/af_inet.c | 6 ------ 2 files changed, 5 insertions(+), 7 deletions(-) diff --git a/include/net/ip.h b/include/net/ip.h index 9192444f2964ebb59454a7dfa5ddf3b19dea04c9..cf229a53119428307da898af4b0dc23e1cecc053 100644 --- a/include/net/ip.h +++ b/include/net/ip.h @@ -291,7 +291,11 @@ void ip_send_unicast_reply(struct sock *sk, struct sk_buff *skb, #define NET_ADD_STATS(net, field, adnd) SNMP_ADD_STATS((net)->mib.net_statistics, field, adnd) #define __NET_ADD_STATS(net, field, adnd) __SNMP_ADD_STATS((net)->mib.net_statistics, field, adnd) -u64 snmp_get_cpu_field(void __percpu *mib, int cpu, int offct); +static inline u64 snmp_get_cpu_field(void __percpu *mib, int cpu, int offt) +{ + return *(((unsigned long *)per_cpu_ptr(mib, cpu)) + offt); +} + unsigned long snmp_fold_field(void __percpu *mib, int offt); #if BITS_PER_LONG==32 u64 snmp_get_cpu_field64(void __percpu *mib, int cpu, int offct, diff --git a/net/ipv4/af_inet.c b/net/ipv4/af_inet.c index 40558033f857c0ca7d98b778f70487e194f3d066..967926c1bf56cfc915258b0969914b11f24c1e16 100644 --- a/net/ipv4/af_inet.c +++ b/net/ipv4/af_inet.c @@ -1662,12 +1662,6 @@ int inet_ctl_sock_create(struct sock **sk, unsigned short family, } EXPORT_SYMBOL_GPL(inet_ctl_sock_create); -u64 snmp_get_cpu_field(void __percpu *mib, int cpu, int offt) -{ - return *(((unsigned long *)per_cpu_ptr(mib, cpu)) + offt); -} -EXPORT_SYMBOL_GPL(snmp_get_cpu_field); - unsigned long snmp_fold_field(void __percpu *mib, int offt) { unsigned long res = 0; From patchwork Thu Sep 30 01:03:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 12527075 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BDFAC433EF for ; Thu, 30 Sep 2021 01:03:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6B393615A4 for ; Thu, 30 Sep 2021 01:03:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347565AbhI3BFY (ORCPT ); Wed, 29 Sep 2021 21:05:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236932AbhI3BFW (ORCPT ); Wed, 29 Sep 2021 21:05:22 -0400 Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 45009C06161C for ; Wed, 29 Sep 2021 18:03:41 -0700 (PDT) Received: by mail-pl1-x62a.google.com with SMTP id x4so2785167pln.5 for ; Wed, 29 Sep 2021 18:03:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7nEP9G45spq5xv8tXXluO1vEopMiEfhTEHqUciYsJoQ=; b=XeiGtRSNN7kLqmXoTCGBCgAuTrNsFwY+2iwPfjpXPP3JiCGzQ8zoIKtOIvYFmgXiCD UXHP4Y8bAQayRKLDwhhjcSizc6G9HF5Kuvv1eWHO98r97t+B1toMikTVSlpUq7QoSIP5 WPpvav4HEvesY2vZMkqcVVWjseash35medEQyatfRkNEk1AhCplffOCXXd64qwq2Gv55 mmiBe6R1qpgxdrkerzg+Uzq8ceRdaPXLDAc10RDXEaHSn7mbKkVJX3CwIiVafIPmBn47 cAlYlCShxFIxOMhcigdI9Fe3GWEK7TmHXR9JbqNIf7Eb7AosMUVQgaMVHExa6/WJU4yr GuDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7nEP9G45spq5xv8tXXluO1vEopMiEfhTEHqUciYsJoQ=; b=QVJAfHOb66xFlQkRNuSrQVXrgycuZsvBYPTIKgiTesGKDB5X+eNuA7cpVDwfdZe3At Y3IB5fykgCyVUDUoaW6CGiE7bJBmR/bxzbyHzTMConVa7hkVDMy518IBQxTXk2/cjVGj JZ/Oc8qA7EnkE4riioDGuer9L/41DRCtQiBzqNTlThQa30iNPHgLfoHF/J6PxQmw3HKZ gEYUuGZwdsKjc5qxHnDFRzEjIiNnA5Vneh80AvgVfKw+DQ/7Xhd+hXly5FarnyUD2UdW S4/XiheZc+j23qPvD1BZ75Qh5WbDbc0IHVJ/z6bHTdF9kaZDFKWuB8F6cmlP/uNT5L4a jPHg== X-Gm-Message-State: AOAM530EH9lg6/nJs7xaJmqbsFCMPjxBIOg4GmSo2mFccUW1MZrMViB0 UrotAi5Wpqa4LM/nnaa3sBQ= X-Google-Smtp-Source: ABdhPJwz8ds5UtCZn+EHoPCBJNtaemK/BakF9eqioXRv8qPbQb7PFpOwbRxkdCpC9l8bKtRD9sG5mg== X-Received: by 2002:a17:90b:911:: with SMTP id bo17mr7744343pjb.232.1632963820852; Wed, 29 Sep 2021 18:03:40 -0700 (PDT) Received: from edumazet1.svl.corp.google.com ([2620:15c:2c4:201:8858:a0b7:dcc9:9a3b]) by smtp.gmail.com with ESMTPSA id p17sm711695pju.34.2021.09.29.18.03.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Sep 2021 18:03:40 -0700 (PDT) From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski Cc: netdev , Eric Dumazet , Eric Dumazet , Mat Martineau , Matthieu Baerts Subject: [PATCH net-next 2/2] mptcp: use batch snmp operations in mptcp_seq_show() Date: Wed, 29 Sep 2021 18:03:33 -0700 Message-Id: <20210930010333.2625706-3-eric.dumazet@gmail.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930010333.2625706-1-eric.dumazet@gmail.com> References: <20210930010333.2625706-1-eric.dumazet@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Eric Dumazet Using snmp_get_cpu_field_batch() allows for better cpu cache utilization, especially on hosts with large number of cpus. Also remove special handling when mptcp mibs where not yet allocated. I chose to use temporary storage on the stack to keep this patch simple. We might in the future use the storage allocated in netstat_seq_show(). Combined with prior patch (inlining snmp_get_cpu_field) time to fetch and output mptcp counters on a 256 cpu host [1] goes from 75 usec to 16 usec. [1] L1 cache size is 32KB, it is not big enough to hold all dataset. Signed-off-by: Eric Dumazet --- net/mptcp/mib.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/net/mptcp/mib.c b/net/mptcp/mib.c index b21ff9be04c61772c8a7648d9540b48907de9115..3240b72271a7f3724577907bc99bdfe41a97f9de 100644 --- a/net/mptcp/mib.c +++ b/net/mptcp/mib.c @@ -72,6 +72,7 @@ bool mptcp_mib_alloc(struct net *net) void mptcp_seq_show(struct seq_file *seq) { + unsigned long sum[ARRAY_SIZE(mptcp_snmp_list) - 1]; struct net *net = seq->private; int i; @@ -81,17 +82,13 @@ void mptcp_seq_show(struct seq_file *seq) seq_puts(seq, "\nMPTcpExt:"); - if (!net->mib.mptcp_statistics) { - for (i = 0; mptcp_snmp_list[i].name; i++) - seq_puts(seq, " 0"); - - seq_putc(seq, '\n'); - return; - } + memset(sum, 0, sizeof(sum)); + if (net->mib.mptcp_statistics) + snmp_get_cpu_field_batch(sum, mptcp_snmp_list, + net->mib.mptcp_statistics); for (i = 0; mptcp_snmp_list[i].name; i++) - seq_printf(seq, " %lu", - snmp_fold_field(net->mib.mptcp_statistics, - mptcp_snmp_list[i].entry)); + seq_printf(seq, " %lu", sum[i]); + seq_putc(seq, '\n'); }