From patchwork Tue Jun 7 23:36:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 12872617 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F47AC433EF for ; Wed, 8 Jun 2022 00:23:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1385096AbiFHAU4 (ORCPT ); Tue, 7 Jun 2022 20:20:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1588651AbiFGXyx (ORCPT ); Tue, 7 Jun 2022 19:54:53 -0400 Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82C821B184B for ; Tue, 7 Jun 2022 16:36:32 -0700 (PDT) Received: by mail-pg1-x52d.google.com with SMTP id f65so7113363pgc.7 for ; Tue, 07 Jun 2022 16:36:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8sLwMqlPzXnNNEKic0159pnQtK8wgFlPa/p2A5Zl72c=; b=IRFM9sDeEq8C8ZH4d1Zdyw7XB5Lr0EkoOPWSPfUCgFVxEsEiFzofFudKZfYd3r3hjV cp3kzN8pDDDBVVwyr0ovd87pr2myIEL5tefJF8qTTeYndYxKQoMvIlJ4hd7FmYQ8D04h oh8Yl7zFEVut6XzJ+ENT88Y+QsjX5i9d8lxTZUwFwaqwtjt3hT2FC9n5qUzby9qjO+H1 cD/+NBjtQQvmsyrmFO2/rgT4+9R48YLLuOMBZ6q6dPMZbBeOAwqKwgX9LHUX+dXWfK83 n0KVBJ4r6C+Ib8tmlaWwzyf90m1DeDaKOOBwXFoCuvw+Ja8qIpVSTjliJzyCl2jvVQij MnwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8sLwMqlPzXnNNEKic0159pnQtK8wgFlPa/p2A5Zl72c=; b=tsvBg6DG6OC+AzW4GzvudUNdVJ7SjAaQZeT4QQusJjGwz9Ts7cf0XQJkJWoCv3YxHH KqhFCci44X6FBmj1LmbVW/dPskZcYAbGUBI3Fn6Ix1oxfALhVjcaOetxfEgvykzYVXe9 ZqrQGEG8lpxjj+cUcf2L0llXLUYw6mBHhtPes2C74oYv3CQ2Pi5gup4wWn1nPtopxNTW cxe4hP+qdnCLEqOm8tCL72QRZMu6y08NVqyIRYz5FtWHUHd6RuORDNefmPUz6mzevobG MEvtYIDp4pZEC/9SWS/jzK5IOHrs6lirghDxUoaB7kPAf6Ztr0Sxm48EHyewE4JMdAcN LJAQ== X-Gm-Message-State: AOAM532MXdyxhi8rbr80InALS/MAdbwdXh79CQX5Jg+yJFHqyDJVByMD EsuC4JNwUuMeTSBXduhPoqg= X-Google-Smtp-Source: ABdhPJwBRXbcrcQZM8llMGBKarnnC8OdIV+aJ7WnZ2j822udT1Nyv/Yli6f8xd7kkjWZTCAJIt5Byg== X-Received: by 2002:a65:6b8a:0:b0:3fc:4c06:8a8d with SMTP id d10-20020a656b8a000000b003fc4c068a8dmr27958162pgw.83.1654644991797; Tue, 07 Jun 2022 16:36:31 -0700 (PDT) Received: from edumazet1.svl.corp.google.com ([2620:15c:2c4:201:191a:13a7:b80a:f36e]) by smtp.gmail.com with ESMTPSA id u79-20020a627952000000b0051ba7515e0dsm13550947pfc.54.2022.06.07.16.36.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jun 2022 16:36:31 -0700 (PDT) From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: netdev , "Jason A . Donenfeld" , Eric Dumazet , Eric Dumazet Subject: [PATCH net-next 6/9] net: adopt u64_stats_t in struct pcpu_sw_netstats Date: Tue, 7 Jun 2022 16:36:11 -0700 Message-Id: <20220607233614.1133902-7-eric.dumazet@gmail.com> X-Mailer: git-send-email 2.36.1.255.ge46751e96f-goog In-Reply-To: <20220607233614.1133902-1-eric.dumazet@gmail.com> References: <20220607233614.1133902-1-eric.dumazet@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Eric Dumazet As explained in commit 316580b69d0a ("u64_stats: provide u64_stats_t type") we should use u64_stats_t and related accessors to avoid load/store tearing. Signed-off-by: Eric Dumazet --- drivers/net/macsec.c | 8 ++++---- drivers/net/usb/usbnet.c | 8 ++++---- drivers/net/vxlan/vxlan_core.c | 8 ++++---- include/linux/netdevice.h | 16 +++++++-------- include/net/ip_tunnels.h | 4 ++-- net/bridge/br_netlink.c | 8 ++++---- net/bridge/br_vlan.c | 36 +++++++++++++++++++--------------- net/core/dev.c | 18 ++++++++--------- net/dsa/slave.c | 8 ++++---- 9 files changed, 59 insertions(+), 55 deletions(-) diff --git a/drivers/net/macsec.c b/drivers/net/macsec.c index 817577e713d709fb2961b3bdf195879234d08183..96e8d5fd90b2dba92b833de8b2dc7e682ef47a76 100644 --- a/drivers/net/macsec.c +++ b/drivers/net/macsec.c @@ -523,8 +523,8 @@ static void count_tx(struct net_device *dev, int ret, int len) struct pcpu_sw_netstats *stats = this_cpu_ptr(dev->tstats); u64_stats_update_begin(&stats->syncp); - stats->tx_packets++; - stats->tx_bytes += len; + u64_stats_inc(&stats->tx_packets); + u64_stats_add(&stats->tx_bytes, len); u64_stats_update_end(&stats->syncp); } } @@ -825,8 +825,8 @@ static void count_rx(struct net_device *dev, int len) struct pcpu_sw_netstats *stats = this_cpu_ptr(dev->tstats); u64_stats_update_begin(&stats->syncp); - stats->rx_packets++; - stats->rx_bytes += len; + u64_stats_inc(&stats->rx_packets); + u64_stats_add(&stats->rx_bytes, len); u64_stats_update_end(&stats->syncp); } diff --git a/drivers/net/usb/usbnet.c b/drivers/net/usb/usbnet.c index 36b24ec1165043def00bfd151eb1b60929995deb..4409d6b24101075cb2c3a2eaaefe260b50cb36ad 100644 --- a/drivers/net/usb/usbnet.c +++ b/drivers/net/usb/usbnet.c @@ -337,8 +337,8 @@ void usbnet_skb_return (struct usbnet *dev, struct sk_buff *skb) skb->protocol = eth_type_trans (skb, dev->net); flags = u64_stats_update_begin_irqsave(&stats64->syncp); - stats64->rx_packets++; - stats64->rx_bytes += skb->len; + u64_stats_inc(&stats64->rx_packets); + u64_stats_add(&stats64->rx_bytes, skb->len); u64_stats_update_end_irqrestore(&stats64->syncp, flags); netif_dbg(dev, rx_status, dev->net, "< rx, len %zu, type 0x%x\n", @@ -1258,8 +1258,8 @@ static void tx_complete (struct urb *urb) unsigned long flags; flags = u64_stats_update_begin_irqsave(&stats64->syncp); - stats64->tx_packets += entry->packets; - stats64->tx_bytes += entry->length; + u64_stats_add(&stats64->tx_packets, entry->packets); + u64_stats_add(&stats64->tx_bytes, entry->length); u64_stats_update_end_irqrestore(&stats64->syncp, flags); } else { dev->net->stats.tx_errors++; diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c index 265d4a0245e7fcb31f469061eaaf46baa47ebbab..8b0710b576c2109c135e86515ded416360383535 100644 --- a/drivers/net/vxlan/vxlan_core.c +++ b/drivers/net/vxlan/vxlan_core.c @@ -2385,15 +2385,15 @@ static void vxlan_encap_bypass(struct sk_buff *skb, struct vxlan_dev *src_vxlan, vxlan_snoop(dev, &loopback, eth_hdr(skb)->h_source, 0, vni); u64_stats_update_begin(&tx_stats->syncp); - tx_stats->tx_packets++; - tx_stats->tx_bytes += len; + u64_stats_inc(&tx_stats->tx_packets); + u64_stats_add(&tx_stats->tx_bytes, len); u64_stats_update_end(&tx_stats->syncp); vxlan_vnifilter_count(src_vxlan, vni, NULL, VXLAN_VNI_STATS_TX, len); if (__netif_rx(skb) == NET_RX_SUCCESS) { u64_stats_update_begin(&rx_stats->syncp); - rx_stats->rx_packets++; - rx_stats->rx_bytes += len; + u64_stats_inc(&rx_stats->rx_packets); + u64_stats_add(&rx_stats->rx_bytes, len); u64_stats_update_end(&rx_stats->syncp); vxlan_vnifilter_count(dst_vxlan, vni, NULL, VXLAN_VNI_STATS_RX, len); diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index f615a66c89e98b5d58e1b23d6674fa142106fb6e..a07fae3ef10817c77bdec59660fa0feb7cdeb406 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -2636,10 +2636,10 @@ struct packet_offload { /* often modified stats are per-CPU, other are shared (netdev->stats) */ struct pcpu_sw_netstats { - u64 rx_packets; - u64 rx_bytes; - u64 tx_packets; - u64 tx_bytes; + u64_stats_t rx_packets; + u64_stats_t rx_bytes; + u64_stats_t tx_packets; + u64_stats_t tx_bytes; struct u64_stats_sync syncp; } __aligned(4 * sizeof(u64)); @@ -2656,8 +2656,8 @@ static inline void dev_sw_netstats_rx_add(struct net_device *dev, unsigned int l struct pcpu_sw_netstats *tstats = this_cpu_ptr(dev->tstats); u64_stats_update_begin(&tstats->syncp); - tstats->rx_bytes += len; - tstats->rx_packets++; + u64_stats_add(&tstats->rx_bytes, len); + u64_stats_inc(&tstats->rx_packets); u64_stats_update_end(&tstats->syncp); } @@ -2668,8 +2668,8 @@ static inline void dev_sw_netstats_tx_add(struct net_device *dev, struct pcpu_sw_netstats *tstats = this_cpu_ptr(dev->tstats); u64_stats_update_begin(&tstats->syncp); - tstats->tx_bytes += len; - tstats->tx_packets += packets; + u64_stats_add(&tstats->tx_bytes, len); + u64_stats_add(&tstats->tx_packets, packets); u64_stats_update_end(&tstats->syncp); } diff --git a/include/net/ip_tunnels.h b/include/net/ip_tunnels.h index c24fa934221dde1c59ae6519cee783233d19af48..70cbc4a726691de160e89e92dfb0700c77d3097b 100644 --- a/include/net/ip_tunnels.h +++ b/include/net/ip_tunnels.h @@ -456,8 +456,8 @@ static inline void iptunnel_xmit_stats(struct net_device *dev, int pkt_len) struct pcpu_sw_netstats *tstats = get_cpu_ptr(dev->tstats); u64_stats_update_begin(&tstats->syncp); - tstats->tx_bytes += pkt_len; - tstats->tx_packets++; + u64_stats_add(&tstats->tx_bytes, pkt_len); + u64_stats_inc(&tstats->tx_packets); u64_stats_update_end(&tstats->syncp); put_cpu_ptr(tstats); } else { diff --git a/net/bridge/br_netlink.c b/net/bridge/br_netlink.c index bb01776d2d88c46ac29ba94c2647fe26ff3468df..1ef14a099c6b023dcf2b54e8ef8c9eeb6820e0e4 100644 --- a/net/bridge/br_netlink.c +++ b/net/bridge/br_netlink.c @@ -1770,10 +1770,10 @@ static int br_fill_linkxstats(struct sk_buff *skb, if (v->vid == pvid) vxi.flags |= BRIDGE_VLAN_INFO_PVID; br_vlan_get_stats(v, &stats); - vxi.rx_bytes = stats.rx_bytes; - vxi.rx_packets = stats.rx_packets; - vxi.tx_bytes = stats.tx_bytes; - vxi.tx_packets = stats.tx_packets; + vxi.rx_bytes = u64_stats_read(&stats.rx_bytes); + vxi.rx_packets = u64_stats_read(&stats.rx_packets); + vxi.tx_bytes = u64_stats_read(&stats.tx_bytes); + vxi.tx_packets = u64_stats_read(&stats.tx_packets); if (nla_put(skb, BRIDGE_XSTATS_VLAN, sizeof(vxi), &vxi)) goto nla_put_failure; diff --git a/net/bridge/br_vlan.c b/net/bridge/br_vlan.c index 0f5e75ccac7957184b18acb8dc53876c12663dc0..6e53dc991409429f26316d2c407e01c50c47c664 100644 --- a/net/bridge/br_vlan.c +++ b/net/bridge/br_vlan.c @@ -505,8 +505,8 @@ struct sk_buff *br_handle_vlan(struct net_bridge *br, if (br_opt_get(br, BROPT_VLAN_STATS_ENABLED)) { stats = this_cpu_ptr(v->stats); u64_stats_update_begin(&stats->syncp); - stats->tx_bytes += skb->len; - stats->tx_packets++; + u64_stats_add(&stats->tx_bytes, skb->len); + u64_stats_inc(&stats->tx_packets); u64_stats_update_end(&stats->syncp); } @@ -624,8 +624,8 @@ static bool __allowed_ingress(const struct net_bridge *br, if (br_opt_get(br, BROPT_VLAN_STATS_ENABLED)) { stats = this_cpu_ptr(v->stats); u64_stats_update_begin(&stats->syncp); - stats->rx_bytes += skb->len; - stats->rx_packets++; + u64_stats_add(&stats->rx_bytes, skb->len); + u64_stats_inc(&stats->rx_packets); u64_stats_update_end(&stats->syncp); } @@ -1379,16 +1379,16 @@ void br_vlan_get_stats(const struct net_bridge_vlan *v, cpu_stats = per_cpu_ptr(v->stats, i); do { start = u64_stats_fetch_begin_irq(&cpu_stats->syncp); - rxpackets = cpu_stats->rx_packets; - rxbytes = cpu_stats->rx_bytes; - txbytes = cpu_stats->tx_bytes; - txpackets = cpu_stats->tx_packets; + rxpackets = u64_stats_read(&cpu_stats->rx_packets); + rxbytes = u64_stats_read(&cpu_stats->rx_bytes); + txbytes = u64_stats_read(&cpu_stats->tx_bytes); + txpackets = u64_stats_read(&cpu_stats->tx_packets); } while (u64_stats_fetch_retry_irq(&cpu_stats->syncp, start)); - stats->rx_packets += rxpackets; - stats->rx_bytes += rxbytes; - stats->tx_bytes += txbytes; - stats->tx_packets += txpackets; + u64_stats_add(&stats->rx_packets, rxpackets); + u64_stats_add(&stats->rx_bytes, rxbytes); + u64_stats_add(&stats->tx_bytes, txbytes); + u64_stats_add(&stats->tx_packets, txpackets); } } @@ -1779,14 +1779,18 @@ static bool br_vlan_stats_fill(struct sk_buff *skb, return false; br_vlan_get_stats(v, &stats); - if (nla_put_u64_64bit(skb, BRIDGE_VLANDB_STATS_RX_BYTES, stats.rx_bytes, + if (nla_put_u64_64bit(skb, BRIDGE_VLANDB_STATS_RX_BYTES, + u64_stats_read(&stats.rx_bytes), BRIDGE_VLANDB_STATS_PAD) || nla_put_u64_64bit(skb, BRIDGE_VLANDB_STATS_RX_PACKETS, - stats.rx_packets, BRIDGE_VLANDB_STATS_PAD) || - nla_put_u64_64bit(skb, BRIDGE_VLANDB_STATS_TX_BYTES, stats.tx_bytes, + u64_stats_read(&stats.rx_packets), + BRIDGE_VLANDB_STATS_PAD) || + nla_put_u64_64bit(skb, BRIDGE_VLANDB_STATS_TX_BYTES, + u64_stats_read(&stats.tx_bytes), BRIDGE_VLANDB_STATS_PAD) || nla_put_u64_64bit(skb, BRIDGE_VLANDB_STATS_TX_PACKETS, - stats.tx_packets, BRIDGE_VLANDB_STATS_PAD)) + u64_stats_read(&stats.tx_packets), + BRIDGE_VLANDB_STATS_PAD)) goto out_err; nla_nest_end(skb, nest); diff --git a/net/core/dev.c b/net/core/dev.c index 08ce317fcec89609f6f8e9335b3d9f57e813024d..6ed775459f45e8df1d14233aed92b90d1a84cca2 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -10459,23 +10459,23 @@ void dev_fetch_sw_netstats(struct rtnl_link_stats64 *s, int cpu; for_each_possible_cpu(cpu) { + u64 rx_packets, rx_bytes, tx_packets, tx_bytes; const struct pcpu_sw_netstats *stats; - struct pcpu_sw_netstats tmp; unsigned int start; stats = per_cpu_ptr(netstats, cpu); do { start = u64_stats_fetch_begin_irq(&stats->syncp); - tmp.rx_packets = stats->rx_packets; - tmp.rx_bytes = stats->rx_bytes; - tmp.tx_packets = stats->tx_packets; - tmp.tx_bytes = stats->tx_bytes; + rx_packets = u64_stats_read(&stats->rx_packets); + rx_bytes = u64_stats_read(&stats->rx_bytes); + tx_packets = u64_stats_read(&stats->tx_packets); + tx_bytes = u64_stats_read(&stats->tx_bytes); } while (u64_stats_fetch_retry_irq(&stats->syncp, start)); - s->rx_packets += tmp.rx_packets; - s->rx_bytes += tmp.rx_bytes; - s->tx_packets += tmp.tx_packets; - s->tx_bytes += tmp.tx_bytes; + s->rx_packets += rx_packets; + s->rx_bytes += rx_bytes; + s->tx_packets += tx_packets; + s->tx_bytes += tx_bytes; } } EXPORT_SYMBOL_GPL(dev_fetch_sw_netstats); diff --git a/net/dsa/slave.c b/net/dsa/slave.c index 801a5d445833c0f6b0328fc1adb9508d79244e90..2e1ac638d135e8b83cd80f83a67736e31f387afa 100644 --- a/net/dsa/slave.c +++ b/net/dsa/slave.c @@ -935,10 +935,10 @@ static void dsa_slave_get_ethtool_stats(struct net_device *dev, s = per_cpu_ptr(dev->tstats, i); do { start = u64_stats_fetch_begin_irq(&s->syncp); - tx_packets = s->tx_packets; - tx_bytes = s->tx_bytes; - rx_packets = s->rx_packets; - rx_bytes = s->rx_bytes; + tx_packets = u64_stats_read(&s->tx_packets); + tx_bytes = u64_stats_read(&s->tx_bytes); + rx_packets = u64_stats_read(&s->rx_packets); + rx_bytes = u64_stats_read(&s->rx_bytes); } while (u64_stats_fetch_retry_irq(&s->syncp, start)); data[0] += tx_packets; data[1] += tx_bytes;