From patchwork Wed Jan 6 14:42:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sieng-Piaw Liew X-Patchwork-Id: 12001845 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9F81C433DB for ; Wed, 6 Jan 2021 14:45:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8E7FC2311B for ; Wed, 6 Jan 2021 14:45:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726499AbhAFOny (ORCPT ); Wed, 6 Jan 2021 09:43:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59220 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726011AbhAFOnx (ORCPT ); Wed, 6 Jan 2021 09:43:53 -0500 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C75BCC061358; Wed, 6 Jan 2021 06:43:12 -0800 (PST) Received: by mail-pj1-x1031.google.com with SMTP id lj6so1655293pjb.0; Wed, 06 Jan 2021 06:43:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1jYNLvaZPXWnnMKmqFijAtk3wCJypCnIEErPxeILOoQ=; b=etftZ6kOn5Ec3/YvWRT/0S0dVxzc8TqdLkBXJCyGh9hlG/yi8IYlOkjSskH8BKEDz2 7rRmZClbEx36W9sDTajXoYQqsxDmhSlLVyHJW+pdAqZWpBgujtPZXPDX90JELHIYM3vW BeOuWiIVQnTXqnzgcn/KvrndrQ0pVVghzXTq+R9SL3jsNJXLHEO9pUapZnTOEyh2hQw2 jydNNlCT3zLIfIhhCxofcMTrHWWvRObtUQcrLREOYdJTTbu6FN/iOcxauwWN7eyYcVLz /Z7MjLg/IIpZ0WMKlPfCVztBkpibXxfVlBWmx2x/UTbV4h5sq1HEDufTSO6KFPCjVQdg NLrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1jYNLvaZPXWnnMKmqFijAtk3wCJypCnIEErPxeILOoQ=; b=rOAtYKBpw4HN/EVDHWxNiz2BmGmLcKGYiVaYg2fXL3vVGSeS6r7dcsYuoXzPpvnCiH lHpL9B99jw3VMOvgZ0en07CGWYXgs1D7NWMAmFELmfNisZOeuYokihC3udnpAsyTNLnn 5a7mwoFMYsc8gmZuuDZ+Qw+3Zj6J8Cij0PNo+wHlOgOiqqv+S5/oxYr7KVqQdqO93MgF xArtVWoyTLbO7sQDJxhN2enO7hEKzY7QIdTw+cNPSDy8dh+cgU62Kik8Vedtcd7QB2Oj +55vr5Pz9Y/qdr7P3MoTmbxHN7mDHGO/klIXKXzby7oE97m43T+d8KN93BiOXW/d7wGT HZyA== X-Gm-Message-State: AOAM5316v2k/opxIrQ3Q4qL2/zC7YFucU85pfF/ePhwsBTbfrFnDPWYF eDATilWDDA6ph4/V33ftq3nOGjqz3AM= X-Google-Smtp-Source: ABdhPJwBkD64n6wJ4zsenbZnKQOtt9mVoT9cbikyC3Zc9lxS+Nbe8MXUSHR7tOpBXvVM0AVTAAIqAA== X-Received: by 2002:a17:902:8643:b029:da:d5f9:28f6 with SMTP id y3-20020a1709028643b02900dad5f928f6mr4627912plt.8.1609944192439; Wed, 06 Jan 2021 06:43:12 -0800 (PST) Received: from DESKTOP-8REGVGF.localdomain ([124.13.157.5]) by smtp.gmail.com with ESMTPSA id h8sm3076774pjc.2.2021.01.06.06.43.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Jan 2021 06:43:12 -0800 (PST) From: Sieng Piaw Liew To: Florian Fainelli Cc: bcm-kernel-feedback-list@broadcom.com, Sieng Piaw Liew , netdev@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v3 1/7] bcm63xx_enet: batch process rx path Date: Wed, 6 Jan 2021 22:42:02 +0800 Message-Id: <20210106144208.1935-2-liew.s.piaw@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210106144208.1935-1-liew.s.piaw@gmail.com> References: <20210106144208.1935-1-liew.s.piaw@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Use netif_receive_skb_list to batch process rx skb. Tested on BCM6328 320 MHz using iperf3 -M 512, increasing performance by 12.5%. Before: [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-30.00 sec 120 MBytes 33.7 Mbits/sec 277 sender [ 4] 0.00-30.00 sec 120 MBytes 33.5 Mbits/sec receiver After: [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-30.00 sec 136 MBytes 37.9 Mbits/sec 203 sender [ 4] 0.00-30.00 sec 135 MBytes 37.7 Mbits/sec receiver Signed-off-by: Sieng Piaw Liew Acked-by: Florian Fainelli --- drivers/net/ethernet/broadcom/bcm63xx_enet.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/broadcom/bcm63xx_enet.c b/drivers/net/ethernet/broadcom/bcm63xx_enet.c index 916824cca3fd..b82b7805c36a 100644 --- a/drivers/net/ethernet/broadcom/bcm63xx_enet.c +++ b/drivers/net/ethernet/broadcom/bcm63xx_enet.c @@ -297,10 +297,12 @@ static void bcm_enet_refill_rx_timer(struct timer_list *t) static int bcm_enet_receive_queue(struct net_device *dev, int budget) { struct bcm_enet_priv *priv; + struct list_head rx_list; struct device *kdev; int processed; priv = netdev_priv(dev); + INIT_LIST_HEAD(&rx_list); kdev = &priv->pdev->dev; processed = 0; @@ -391,10 +393,12 @@ static int bcm_enet_receive_queue(struct net_device *dev, int budget) skb->protocol = eth_type_trans(skb, dev); dev->stats.rx_packets++; dev->stats.rx_bytes += len; - netif_receive_skb(skb); + list_add_tail(&skb->list, &rx_list); } while (--budget > 0); + netif_receive_skb_list(&rx_list); + if (processed || !priv->rx_desc_count) { bcm_enet_refill_rx(dev); From patchwork Wed Jan 6 14:42:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sieng-Piaw Liew X-Patchwork-Id: 12001841 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD5D8C43331 for ; Wed, 6 Jan 2021 14:44:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 81A1E23110 for ; Wed, 6 Jan 2021 14:44:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726603AbhAFOn6 (ORCPT ); Wed, 6 Jan 2021 09:43:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59226 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726011AbhAFOnz (ORCPT ); Wed, 6 Jan 2021 09:43:55 -0500 Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C87CC061359; Wed, 6 Jan 2021 06:43:15 -0800 (PST) Received: by mail-pl1-x630.google.com with SMTP id g3so1655760plp.2; Wed, 06 Jan 2021 06:43:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=W6/+FtIew+DXRW3+N/9J9qYZ49HugSJ5H2PNvgyUorQ=; b=KjUbm8X+6VcOdIetuU0rQ2YE+NfSHkfxjz5dwEoD2cUl207fgHqKpCeXQYe2TdZdPL mUOj/Tthh2YNsxqbeBaPr/TpTnf3oZ54ugcJmiycrcRVxDCGHRTtBiJKxRBlBTq1qxRk u+bIsDLWE6zUbWbfsgbEVTEG8jeUb4ALs0rWGW2MyPEtGoj8K4izS+fzto7QkPDBBWXK KkR0+R4MtEiJvZkbQxGypfWCKC0WWlEEVuTG0X+QBCLutvXxWOvDPhuefdvLf5w1VRdb DdqHANXaupWNqnJ8lHSbpTWSzPUwJShGJBKAdksJ0LAr1gYUaDOzu3JsWuQtvHL1uNeo tugg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=W6/+FtIew+DXRW3+N/9J9qYZ49HugSJ5H2PNvgyUorQ=; b=pRsFda9su1cykWBL8epmxQiCdZn0qcP9TD3qdbfA8s80Uzg0Ij2zJ9hfQ0o4+aMS3Z u2d27uM/Ih4zRDNXRNphOuMtuIZgJEEZ2pkvNfY4uR+eQTmLkCvgIOM2PdjRrJi/37c2 OodeZiTXBL88m1qm1oC23z0yqt/ze2QR7JDvHc90lCupX/X8It/f+QOOg/NDMse92od6 rvTqwOIT+G8/I65lT7ZePXpc2rDwpsKtQUqk3Sbyb2z5wTjuJ+LQBugVCoVqLQTBjHbZ 4PLp57jVFUK5lNwTKo6Jyy6W3mACwJO9xdENjW5w2Xy3bvr1NXA58wMrwQBRI+eJAGZR 18DA== X-Gm-Message-State: AOAM530rRzG5x6K0YydoDm6v+zJ6sCoM6hfnqjMDSRB/HZ2PJBb6dkLn G0SqNstt87F0Y5S4rfwlG18= X-Google-Smtp-Source: ABdhPJwPN0va3Ov9iieIunuBdRIokxU+5WWs72jb0mtHcFaUH0aV4MvRKjItzE4QRCueqmdGBXIyDA== X-Received: by 2002:a17:902:ed11:b029:da:3137:2695 with SMTP id b17-20020a170902ed11b02900da31372695mr4584837pld.1.1609944194844; Wed, 06 Jan 2021 06:43:14 -0800 (PST) Received: from DESKTOP-8REGVGF.localdomain ([124.13.157.5]) by smtp.gmail.com with ESMTPSA id h8sm3076774pjc.2.2021.01.06.06.43.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Jan 2021 06:43:14 -0800 (PST) From: Sieng Piaw Liew To: Florian Fainelli Cc: bcm-kernel-feedback-list@broadcom.com, Sieng Piaw Liew , netdev@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v3 2/7] bcm63xx_enet: add BQL support Date: Wed, 6 Jan 2021 22:42:03 +0800 Message-Id: <20210106144208.1935-3-liew.s.piaw@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210106144208.1935-1-liew.s.piaw@gmail.com> References: <20210106144208.1935-1-liew.s.piaw@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add Byte Queue Limits support to reduce/remove bufferbloat in bcm63xx_enet. Signed-off-by: Sieng Piaw Liew Acked-by: Florian Fainelli --- drivers/net/ethernet/broadcom/bcm63xx_enet.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/net/ethernet/broadcom/bcm63xx_enet.c b/drivers/net/ethernet/broadcom/bcm63xx_enet.c index b82b7805c36a..90f8214b4d22 100644 --- a/drivers/net/ethernet/broadcom/bcm63xx_enet.c +++ b/drivers/net/ethernet/broadcom/bcm63xx_enet.c @@ -417,9 +417,11 @@ static int bcm_enet_receive_queue(struct net_device *dev, int budget) static int bcm_enet_tx_reclaim(struct net_device *dev, int force) { struct bcm_enet_priv *priv; + unsigned int bytes; int released; priv = netdev_priv(dev); + bytes = 0; released = 0; while (priv->tx_desc_count < priv->tx_ring_size) { @@ -456,10 +458,13 @@ static int bcm_enet_tx_reclaim(struct net_device *dev, int force) if (desc->len_stat & DMADESC_UNDER_MASK) dev->stats.tx_errors++; + bytes += skb->len; dev_kfree_skb(skb); released++; } + netdev_completed_queue(dev, released, bytes); + if (netif_queue_stopped(dev) && released) netif_wake_queue(dev); @@ -626,6 +631,8 @@ bcm_enet_start_xmit(struct sk_buff *skb, struct net_device *dev) desc->len_stat = len_stat; wmb(); + netdev_sent_queue(dev, skb->len); + /* kick tx dma */ enet_dmac_writel(priv, priv->dma_chan_en_mask, ENETDMAC_CHANCFG, priv->tx_chan); @@ -1169,6 +1176,7 @@ static int bcm_enet_stop(struct net_device *dev) kdev = &priv->pdev->dev; netif_stop_queue(dev); + netdev_reset_queue(dev); napi_disable(&priv->napi); if (priv->has_phy) phy_stop(dev->phydev); @@ -2338,6 +2346,7 @@ static int bcm_enetsw_stop(struct net_device *dev) del_timer_sync(&priv->swphy_poll); netif_stop_queue(dev); + netdev_reset_queue(dev); napi_disable(&priv->napi); del_timer_sync(&priv->rx_timeout); From patchwork Wed Jan 6 14:42:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sieng-Piaw Liew X-Patchwork-Id: 12001849 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 332F3C43381 for ; Wed, 6 Jan 2021 14:45:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0AEF023120 for ; Wed, 6 Jan 2021 14:45:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727023AbhAFOo1 (ORCPT ); Wed, 6 Jan 2021 09:44:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726527AbhAFOn5 (ORCPT ); Wed, 6 Jan 2021 09:43:57 -0500 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 932BDC06134C; Wed, 6 Jan 2021 06:43:17 -0800 (PST) Received: by mail-pj1-x102a.google.com with SMTP id w1so2642882pjc.0; Wed, 06 Jan 2021 06:43:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=DJYt8hiCkCeD9GXS/w4FMJ4EVxAjdbezO9NsUhxWdrw=; b=aD4qk+qHK5m9iZjRQ3AdyWO4znFZkqE9fJ+Qjh1CAEgC5qZVuEh1sHz041A8LklSuX BXME0Tg5Ovolk5FVTsfCl2HOjCdF3tiJAlfsdftJiT25BX+9FyHVYa5tJMbrqqDZNvOr PWc7kt7Vv8U+Dn00c2L+LGJ5YZYrkoIYh1mTIP2UsTLPnF5RvxtsQX9ONUksARe12/r8 gCI26sapJNh0REmriKqZyYUmjenRipVdU/OtX362tLEVf9NHaK8q1Qfkkqi6BRmwzfOF 2mKXPZGFoWggpRjNcwQWWgTyxeEpi4qfh1fvHwbrr+bvYOdV6ZLBLBTlSlaIVr4RoPh5 5zfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=DJYt8hiCkCeD9GXS/w4FMJ4EVxAjdbezO9NsUhxWdrw=; b=EBu34f+hstwwU/QzhRChuHeFnUP+laWkoFXxWLvXJJj6pqJ7avB23nV+iWSPNEKLpa himB2wrECGNBErNn9PQQf2P9i7f6JRmYHJAE/rRPlr3rHTLomm0XPYjm4dJYLz6j3vtT 9/+M6dD9CELV9QzcZMbUVlM6o8tVDlCUIRnQVXFk7BuwukCLdM/duW0y7ZUnfdzEpDM9 iDUgGnioUMMlcxcX2LhJg30/O1ldefSyzH5pkuA/TlUKesE9NR/aVzxiANrBYE7S4IrV fXIQXeNM0seDlFsyBYZ37hWvOqMraeWSp0gddXwyZUTG4qVbP4jixY2/VYJYeYHqvN1q HZ/g== X-Gm-Message-State: AOAM532CU2Ct6c2X+2Y0ZEjfnEZIBJCZyri1TtCLCdH77nKiPZxJr4pw B1PbQCPwHa4RqhxfjCAu3B+sorvS/gg= X-Google-Smtp-Source: ABdhPJxNz97H1pOSKsSbdx2hxJ8m6C/YDYdG5fbC+d2ew200pgpPSGIb8ilU3sGTmYIm6bxjQXLhvA== X-Received: by 2002:a17:90a:ba88:: with SMTP id t8mr4566800pjr.229.1609944197246; Wed, 06 Jan 2021 06:43:17 -0800 (PST) Received: from DESKTOP-8REGVGF.localdomain ([124.13.157.5]) by smtp.gmail.com with ESMTPSA id h8sm3076774pjc.2.2021.01.06.06.43.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Jan 2021 06:43:16 -0800 (PST) From: Sieng Piaw Liew To: Florian Fainelli Cc: bcm-kernel-feedback-list@broadcom.com, Sieng Piaw Liew , netdev@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v3 3/7] bcm63xx_enet: add xmit_more support Date: Wed, 6 Jan 2021 22:42:04 +0800 Message-Id: <20210106144208.1935-4-liew.s.piaw@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210106144208.1935-1-liew.s.piaw@gmail.com> References: <20210106144208.1935-1-liew.s.piaw@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Support bulking hardware TX queue by using netdev_xmit_more(). Signed-off-by: Sieng Piaw Liew Acked-by: Florian Fainelli --- drivers/net/ethernet/broadcom/bcm63xx_enet.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/broadcom/bcm63xx_enet.c b/drivers/net/ethernet/broadcom/bcm63xx_enet.c index 90f8214b4d22..21744dae30ce 100644 --- a/drivers/net/ethernet/broadcom/bcm63xx_enet.c +++ b/drivers/net/ethernet/broadcom/bcm63xx_enet.c @@ -634,7 +634,8 @@ bcm_enet_start_xmit(struct sk_buff *skb, struct net_device *dev) netdev_sent_queue(dev, skb->len); /* kick tx dma */ - enet_dmac_writel(priv, priv->dma_chan_en_mask, + if (!netdev_xmit_more() || !priv->tx_desc_count) + enet_dmac_writel(priv, priv->dma_chan_en_mask, ENETDMAC_CHANCFG, priv->tx_chan); /* stop queue if no more desc available */ From patchwork Wed Jan 6 14:42:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sieng-Piaw Liew X-Patchwork-Id: 12001837 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7DBCC433E6 for ; Wed, 6 Jan 2021 14:44:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 76AA52311B for ; Wed, 6 Jan 2021 14:44:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726698AbhAFOoB (ORCPT ); Wed, 6 Jan 2021 09:44:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59242 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726011AbhAFOoA (ORCPT ); Wed, 6 Jan 2021 09:44:00 -0500 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EDA4BC06135A; Wed, 6 Jan 2021 06:43:19 -0800 (PST) Received: by mail-pf1-x42d.google.com with SMTP id a188so1808681pfa.11; Wed, 06 Jan 2021 06:43:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=z/gY4eGbQAG4g6JUH+BYAEEr7QAfURZLBW/Uf2NFIK0=; b=IPSEZARh30rxUqSwqq9rqaIe+PW5H7ek0sb4Cvr2MhQy/Q7VrQiNqNAmwOHN6KE3Ez V2qxLWygzb2tkHtBoUgd0C98YF0LP7ZAEp0+0ZKt9e5a72RmfjDIeGj0D61ZLYVrOszO hBc1gs3QxNcRtgnIKhZuWAWMZNiBbEDVtfxva7NL5d68NGi/PPXrWLG49Jk69oswLPRW 2y/n+dYiBIelPuO4ztRcRVqJ3rGM9KMkg/ymbvUaXXdI/gxymspZ2Shgy5jsw2EsahXZ 8jSugzs94fTkHhvcqd32M/Xw53UZRngMtZxXqx8SryuuYBpwy/GWoQjFLWkawuENpR5R K3vA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=z/gY4eGbQAG4g6JUH+BYAEEr7QAfURZLBW/Uf2NFIK0=; b=CbZFstZvAFN9HUjyMEsW8CHROc3PBD+rXb9SwH2RNDLJ1sfOPg+VKx//S2ixfsg2Wa mtmIdiYjfgsUUocovMtg37cDdPubd0cN6X++pH8ij9dUGbVtfQ8ovrlXjFEPtkYSbB+N hMBCfDfD0LJ4iMMM/+4l4PbrcoJiH3ibhaxATZhiMrDRuF0z9YcsC9XLGJf9S/Cn9q+1 r1Jti+cwZiCHsU3CtTwQifmefWDlyGaVigp0KMVzboBX8yCtvcuPfXv4SG236E0toC4V A0dFOHiEtseb6K/Gz9seq0WgIrAb7fqyTbsSV65xijlSt7WtddUlcX0cWY7ctcOUdxYZ R26w== X-Gm-Message-State: AOAM532IqgSSGC0N46U41/JTrxSE9tNswjRcyYQuUTqCZS42vK9jcuuE T8JQVheL/7W09/EYACEUSMP1QP2J7js= X-Google-Smtp-Source: ABdhPJxQ+dFA/PivecFA1EwNbyd2csrbQ97GYfNKqFdTGd4st0fhLl9XtfPRwR9LsgyLitEw4MngwQ== X-Received: by 2002:aa7:82cc:0:b029:19e:1328:d039 with SMTP id f12-20020aa782cc0000b029019e1328d039mr4283565pfn.70.1609944199597; Wed, 06 Jan 2021 06:43:19 -0800 (PST) Received: from DESKTOP-8REGVGF.localdomain ([124.13.157.5]) by smtp.gmail.com with ESMTPSA id h8sm3076774pjc.2.2021.01.06.06.43.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Jan 2021 06:43:19 -0800 (PST) From: Sieng Piaw Liew To: Florian Fainelli Cc: bcm-kernel-feedback-list@broadcom.com, Sieng Piaw Liew , netdev@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v3 4/7] bcm63xx_enet: alloc rx skb with NET_IP_ALIGN Date: Wed, 6 Jan 2021 22:42:05 +0800 Message-Id: <20210106144208.1935-5-liew.s.piaw@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210106144208.1935-1-liew.s.piaw@gmail.com> References: <20210106144208.1935-1-liew.s.piaw@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Use netdev_alloc_skb_ip_align on newer SoCs with integrated switch (enetsw) when refilling RX. Increases packet processing performance by 30% (with netif_receive_skb_list). Non-enetsw SoCs cannot function with the extra pad so continue to use the regular netdev_alloc_skb. Tested on BCM6328 320 MHz and iperf3 -M 512 to measure packet/sec performance. Before: [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-30.00 sec 120 MBytes 33.7 Mbits/sec 277 sender [ 4] 0.00-30.00 sec 120 MBytes 33.5 Mbits/sec receiver After (+netif_receive_skb_list): [ 4] 0.00-30.00 sec 155 MBytes 43.3 Mbits/sec 354 sender [ 4] 0.00-30.00 sec 154 MBytes 43.1 Mbits/sec receiver Signed-off-by: Sieng Piaw Liew Acked-by: Florian Fainelli --- drivers/net/ethernet/broadcom/bcm63xx_enet.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/broadcom/bcm63xx_enet.c b/drivers/net/ethernet/broadcom/bcm63xx_enet.c index 21744dae30ce..96d56c3e2cc9 100644 --- a/drivers/net/ethernet/broadcom/bcm63xx_enet.c +++ b/drivers/net/ethernet/broadcom/bcm63xx_enet.c @@ -237,7 +237,10 @@ static int bcm_enet_refill_rx(struct net_device *dev) desc = &priv->rx_desc_cpu[desc_idx]; if (!priv->rx_skb[desc_idx]) { - skb = netdev_alloc_skb(dev, priv->rx_skb_size); + if (priv->enet_is_sw) + skb = netdev_alloc_skb_ip_align(dev, priv->rx_skb_size); + else + skb = netdev_alloc_skb(dev, priv->rx_skb_size); if (!skb) break; priv->rx_skb[desc_idx] = skb; From patchwork Wed Jan 6 14:42:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sieng-Piaw Liew X-Patchwork-Id: 12001839 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AB44C433E9 for ; Wed, 6 Jan 2021 14:44:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B865C23110 for ; Wed, 6 Jan 2021 14:44:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726794AbhAFOoI (ORCPT ); Wed, 6 Jan 2021 09:44:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59262 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726730AbhAFOoH (ORCPT ); Wed, 6 Jan 2021 09:44:07 -0500 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6EB67C06135B; Wed, 6 Jan 2021 06:43:22 -0800 (PST) Received: by mail-pj1-x1030.google.com with SMTP id iq13so1646304pjb.3; Wed, 06 Jan 2021 06:43:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=d5vcGavR4PydjqkNXvyu8THbVXAEosqI/1jZwqLL0vk=; b=GYu6qkg+JpoEsxTAWQhgpr3Al9D3h56YGDPI5tnln1e6CEoIMSxWiTvrAwoOahlIno Rxwn6l0HHnRVzDYaHF0yR2C9QB3E0v6rqMcqvZ+U6yhjdZSq8ACpf405nSTAvTuZ0GZW bgJFz123BjLlRKJR0E4sWmIoC8mK0Ms8EHC5u402sborc+V2LDFXHvjaQ+ouUz8e0e6G tDqqPxqJ07fhIOeFQLb28cEDDvReJa12zwi1evOKo9GN6eL4q2Trs9EnbcPIn+SVQJqJ eR++caeE+ExKsGdBEdtO0ihCDK7t6pb3QuDdR4OeNj+2GqDy431yhbBGtWJup0ENbjGQ ABFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=d5vcGavR4PydjqkNXvyu8THbVXAEosqI/1jZwqLL0vk=; b=oX2QRMqPjfOAmD/cqGlepjuIXgAT0LVidfxte9RWhMxReqFLlVGYNgmYX8eiv0z9Em Bmfx4KjxTW1v3ExTLwpiIEgF5TQi8Id7mXydmBenTW6VFmvcZh7rj5TH0WYhjHhJ9HKD dVGJ11nG83Pu4SktUenDA2QhQAKTsFWguu3jF/j+lIsqhSx0m/ycwhQTrr1NwobKSzrB AER8v6ckedhJtZEFVNIHBflOSve6kpqluik2sGYpu2Vu0ib+O0qH9WMfnJbJXXlHLRab SITXQNQql0HuqlDzjHEg3Yb364mj4rDRCsINwKJNIv1i9ajr8lnURk3isFRVXLmoyKtn etfw== X-Gm-Message-State: AOAM5307H4WSuxAFq6fELePZFjIb3qKK3Z+eiYpNYWjrt9uCYRqfk6/H TH60TYW2YveBm85iZOe7PbM= X-Google-Smtp-Source: ABdhPJw4i1kMKdzTxsDJaO0rPt7U1lhItGo5ZPMgO7u3ZT71armudO+xZGvfVINDdSuKY2AZqUR0cw== X-Received: by 2002:a17:902:654f:b029:da:347d:7af3 with SMTP id d15-20020a170902654fb02900da347d7af3mr4873136pln.18.1609944201986; Wed, 06 Jan 2021 06:43:21 -0800 (PST) Received: from DESKTOP-8REGVGF.localdomain ([124.13.157.5]) by smtp.gmail.com with ESMTPSA id h8sm3076774pjc.2.2021.01.06.06.43.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Jan 2021 06:43:21 -0800 (PST) From: Sieng Piaw Liew To: Florian Fainelli Cc: bcm-kernel-feedback-list@broadcom.com, Sieng Piaw Liew , netdev@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v3 5/7] bcm63xx_enet: consolidate rx SKB ring cleanup code Date: Wed, 6 Jan 2021 22:42:06 +0800 Message-Id: <20210106144208.1935-6-liew.s.piaw@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210106144208.1935-1-liew.s.piaw@gmail.com> References: <20210106144208.1935-1-liew.s.piaw@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org The rx SKB ring use the same code for cleanup at various points. Combine them into a function to reduce lines of code. Signed-off-by: Sieng Piaw Liew Acked-by: Florian Fainelli --- drivers/net/ethernet/broadcom/bcm63xx_enet.c | 72 ++++++-------------- 1 file changed, 22 insertions(+), 50 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bcm63xx_enet.c b/drivers/net/ethernet/broadcom/bcm63xx_enet.c index 96d56c3e2cc9..e34b05b10e43 100644 --- a/drivers/net/ethernet/broadcom/bcm63xx_enet.c +++ b/drivers/net/ethernet/broadcom/bcm63xx_enet.c @@ -860,6 +860,24 @@ static void bcm_enet_adjust_link(struct net_device *dev) priv->pause_tx ? "tx" : "off"); } +static void bcm_enet_free_rx_skb_ring(struct device *kdev, struct bcm_enet_priv *priv) +{ + int i; + + for (i = 0; i < priv->rx_ring_size; i++) { + struct bcm_enet_desc *desc; + + if (!priv->rx_skb[i]) + continue; + + desc = &priv->rx_desc_cpu[i]; + dma_unmap_single(kdev, desc->address, priv->rx_skb_size, + DMA_FROM_DEVICE); + kfree_skb(priv->rx_skb[i]); + } + kfree(priv->rx_skb); +} + /* * open callback, allocate dma rings & buffers and start rx operation */ @@ -1084,18 +1102,7 @@ static int bcm_enet_open(struct net_device *dev) return 0; out: - for (i = 0; i < priv->rx_ring_size; i++) { - struct bcm_enet_desc *desc; - - if (!priv->rx_skb[i]) - continue; - - desc = &priv->rx_desc_cpu[i]; - dma_unmap_single(kdev, desc->address, priv->rx_skb_size, - DMA_FROM_DEVICE); - kfree_skb(priv->rx_skb[i]); - } - kfree(priv->rx_skb); + bcm_enet_free_rx_skb_ring(kdev, priv); out_free_tx_skb: kfree(priv->tx_skb); @@ -1174,7 +1181,6 @@ static int bcm_enet_stop(struct net_device *dev) { struct bcm_enet_priv *priv; struct device *kdev; - int i; priv = netdev_priv(dev); kdev = &priv->pdev->dev; @@ -1203,20 +1209,9 @@ static int bcm_enet_stop(struct net_device *dev) bcm_enet_tx_reclaim(dev, 1); /* free the rx skb ring */ - for (i = 0; i < priv->rx_ring_size; i++) { - struct bcm_enet_desc *desc; - - if (!priv->rx_skb[i]) - continue; - - desc = &priv->rx_desc_cpu[i]; - dma_unmap_single(kdev, desc->address, priv->rx_skb_size, - DMA_FROM_DEVICE); - kfree_skb(priv->rx_skb[i]); - } + bcm_enet_free_rx_skb_ring(kdev, priv); /* free remaining allocated memory */ - kfree(priv->rx_skb); kfree(priv->tx_skb); dma_free_coherent(kdev, priv->rx_desc_alloc_size, priv->rx_desc_cpu, priv->rx_desc_dma); @@ -2303,18 +2298,7 @@ static int bcm_enetsw_open(struct net_device *dev) return 0; out: - for (i = 0; i < priv->rx_ring_size; i++) { - struct bcm_enet_desc *desc; - - if (!priv->rx_skb[i]) - continue; - - desc = &priv->rx_desc_cpu[i]; - dma_unmap_single(kdev, desc->address, priv->rx_skb_size, - DMA_FROM_DEVICE); - kfree_skb(priv->rx_skb[i]); - } - kfree(priv->rx_skb); + bcm_enet_free_rx_skb_ring(kdev, priv); out_free_tx_skb: kfree(priv->tx_skb); @@ -2343,7 +2327,6 @@ static int bcm_enetsw_stop(struct net_device *dev) { struct bcm_enet_priv *priv; struct device *kdev; - int i; priv = netdev_priv(dev); kdev = &priv->pdev->dev; @@ -2366,20 +2349,9 @@ static int bcm_enetsw_stop(struct net_device *dev) bcm_enet_tx_reclaim(dev, 1); /* free the rx skb ring */ - for (i = 0; i < priv->rx_ring_size; i++) { - struct bcm_enet_desc *desc; - - if (!priv->rx_skb[i]) - continue; - - desc = &priv->rx_desc_cpu[i]; - dma_unmap_single(kdev, desc->address, priv->rx_skb_size, - DMA_FROM_DEVICE); - kfree_skb(priv->rx_skb[i]); - } + bcm_enet_free_rx_skb_ring(kdev, priv); /* free remaining allocated memory */ - kfree(priv->rx_skb); kfree(priv->tx_skb); dma_free_coherent(kdev, priv->rx_desc_alloc_size, priv->rx_desc_cpu, priv->rx_desc_dma); From patchwork Wed Jan 6 14:42:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sieng-Piaw Liew X-Patchwork-Id: 12001843 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 848EBC43332 for ; Wed, 6 Jan 2021 14:44:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 53EC723100 for ; Wed, 6 Jan 2021 14:44:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726925AbhAFOoS (ORCPT ); Wed, 6 Jan 2021 09:44:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59264 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726731AbhAFOoH (ORCPT ); Wed, 6 Jan 2021 09:44:07 -0500 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A855C06135C; Wed, 6 Jan 2021 06:43:25 -0800 (PST) Received: by mail-pf1-x42a.google.com with SMTP id x126so1823880pfc.7; Wed, 06 Jan 2021 06:43:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=XftEdDCTyWGddNDIbdQhc19SIuraUjREfl71qMF7tjk=; b=L/TbdpbLXo2XcNE+MK8cUiQi2xRaIy8RYbnbBwDSmv5m/3CakH4SR80jIBeyrUv+Fc zHr5lEQ2U+Nzd2WmqSyo1Cskbv5jZTRQbkhOZbJ/c1MF64DJcIrGZMWVQ7oAV2SObgmD beQVLz5HHpGK1qB7HrKNFluiH7Sq7bzu0fTlM81Df/w2xAJSMXwIJpCsBgN9SrbgRuG9 5LcqvC5xf3mbDmoh154SP1fdMZs12fVMLYdTs2Dn+fMjGZwYJ4bjvku+uvzgLw2Oa9zE FND4DNhneCQU8P1pE4njXMyFjzHidngl6ctUOeodBpHNt3BuKADnyfUwX3+jdfcaIWBh Vp8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=XftEdDCTyWGddNDIbdQhc19SIuraUjREfl71qMF7tjk=; b=G23YhwrDvOC0R1vicI1oQYXGhDI54hwsdqkNEG9aPaZeVIYGSVMMmJTx912zIFWEXX CQF+B/7z4FoFYRmXGnnwdtRaBHciycxrW8CymHeU6q6EhNLuzTFqBIHD0jiLu6kX9gt8 zy6z9G7uWcHGY7mpjqVJASiZMB3KC251EbnCyfJzDZIf4sa5l4mONxyzzn5tdiJ6wz7D WEh6zJu+vFeCIpNWmxzosownFE99XYnAfHcOsR/2Usu9cabe+TEJoazGbOkXTe7A06zt NB1hsjdBXgxq5jz3lB7frOup8z2ILUqrEfDfh7/njgPMbQzrR+UHaHiY7njgzEgByCJ2 eFdQ== X-Gm-Message-State: AOAM530SugYgIpK19pdQEz3OO5VzyyJvzbDYR8vAVeSgD+mNQGAmd9IP 87P2ECHxqwOmKnCDWM6JV3t2xoFRKTE= X-Google-Smtp-Source: ABdhPJyoyM3gNVTNy+Exywyho72xpwIy/5cwhBNnzWibG1ZCfUhQ/RpDbUkbzRUl+pn89MEP0dMPsw== X-Received: by 2002:a62:7a91:0:b029:19e:55db:9ddc with SMTP id v139-20020a627a910000b029019e55db9ddcmr4418635pfc.68.1609944204566; Wed, 06 Jan 2021 06:43:24 -0800 (PST) Received: from DESKTOP-8REGVGF.localdomain ([124.13.157.5]) by smtp.gmail.com with ESMTPSA id h8sm3076774pjc.2.2021.01.06.06.43.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Jan 2021 06:43:24 -0800 (PST) From: Sieng Piaw Liew To: Florian Fainelli Cc: bcm-kernel-feedback-list@broadcom.com, Sieng Piaw Liew , netdev@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v3 6/7] bcm63xx_enet: convert to build_skb Date: Wed, 6 Jan 2021 22:42:07 +0800 Message-Id: <20210106144208.1935-7-liew.s.piaw@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210106144208.1935-1-liew.s.piaw@gmail.com> References: <20210106144208.1935-1-liew.s.piaw@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org We can increase the efficiency of rx path by using buffers to receive packets then build SKBs around them just before passing into the network stack. In contrast, preallocating SKBs too early reduces CPU cache efficiency. Check if we're in NAPI context when refilling RX. Normally we're almost always running in NAPI context. Dispatch to napi_alloc_frag directly instead of relying on netdev_alloc_frag which does the same but with the overhead of local_bh_disable/enable. Tested on BCM6328 320 MHz and iperf3 -M 512 to measure packet/sec performance. Included netif_receive_skb_list and NET_IP_ALIGN optimizations. Before: [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 49.9 MBytes 41.9 Mbits/sec 197 sender [ 4] 0.00-10.00 sec 49.3 MBytes 41.3 Mbits/sec receiver After: [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-30.00 sec 171 MBytes 47.8 Mbits/sec 272 sender [ 4] 0.00-30.00 sec 170 MBytes 47.6 Mbits/sec receiver Signed-off-by: Sieng Piaw Liew Acked-by: Florian Fainelli --- drivers/net/ethernet/broadcom/bcm63xx_enet.c | 111 ++++++++++--------- drivers/net/ethernet/broadcom/bcm63xx_enet.h | 14 ++- 2 files changed, 71 insertions(+), 54 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bcm63xx_enet.c b/drivers/net/ethernet/broadcom/bcm63xx_enet.c index e34b05b10e43..c11491429ed2 100644 --- a/drivers/net/ethernet/broadcom/bcm63xx_enet.c +++ b/drivers/net/ethernet/broadcom/bcm63xx_enet.c @@ -220,7 +220,7 @@ static void bcm_enet_mdio_write_mii(struct net_device *dev, int mii_id, /* * refill rx queue */ -static int bcm_enet_refill_rx(struct net_device *dev) +static int bcm_enet_refill_rx(struct net_device *dev, bool napi_mode) { struct bcm_enet_priv *priv; @@ -228,29 +228,29 @@ static int bcm_enet_refill_rx(struct net_device *dev) while (priv->rx_desc_count < priv->rx_ring_size) { struct bcm_enet_desc *desc; - struct sk_buff *skb; - dma_addr_t p; int desc_idx; u32 len_stat; desc_idx = priv->rx_dirty_desc; desc = &priv->rx_desc_cpu[desc_idx]; - if (!priv->rx_skb[desc_idx]) { - if (priv->enet_is_sw) - skb = netdev_alloc_skb_ip_align(dev, priv->rx_skb_size); + if (!priv->rx_buf[desc_idx]) { + void *buf; + + if (likely(napi_mode)) + buf = napi_alloc_frag(priv->rx_frag_size); else - skb = netdev_alloc_skb(dev, priv->rx_skb_size); - if (!skb) + buf = netdev_alloc_frag(priv->rx_frag_size); + if (unlikely(!buf)) break; - priv->rx_skb[desc_idx] = skb; - p = dma_map_single(&priv->pdev->dev, skb->data, - priv->rx_skb_size, - DMA_FROM_DEVICE); - desc->address = p; + priv->rx_buf[desc_idx] = buf; + desc->address = dma_map_single(&priv->pdev->dev, + buf + priv->rx_buf_offset, + priv->rx_buf_size, + DMA_FROM_DEVICE); } - len_stat = priv->rx_skb_size << DMADESC_LENGTH_SHIFT; + len_stat = priv->rx_buf_size << DMADESC_LENGTH_SHIFT; len_stat |= DMADESC_OWNER_MASK; if (priv->rx_dirty_desc == priv->rx_ring_size - 1) { len_stat |= (DMADESC_WRAP_MASK >> priv->dma_desc_shift); @@ -290,7 +290,7 @@ static void bcm_enet_refill_rx_timer(struct timer_list *t) struct net_device *dev = priv->net_dev; spin_lock(&priv->rx_lock); - bcm_enet_refill_rx(dev); + bcm_enet_refill_rx(dev, false); spin_unlock(&priv->rx_lock); } @@ -320,6 +320,7 @@ static int bcm_enet_receive_queue(struct net_device *dev, int budget) int desc_idx; u32 len_stat; unsigned int len; + void *buf; desc_idx = priv->rx_curr_desc; desc = &priv->rx_desc_cpu[desc_idx]; @@ -365,16 +366,14 @@ static int bcm_enet_receive_queue(struct net_device *dev, int budget) } /* valid packet */ - skb = priv->rx_skb[desc_idx]; + buf = priv->rx_buf[desc_idx]; len = (len_stat & DMADESC_LENGTH_MASK) >> DMADESC_LENGTH_SHIFT; /* don't include FCS */ len -= 4; if (len < copybreak) { - struct sk_buff *nskb; - - nskb = napi_alloc_skb(&priv->napi, len); - if (!nskb) { + skb = napi_alloc_skb(&priv->napi, len); + if (unlikely(!skb)) { /* forget packet, just rearm desc */ dev->stats.rx_dropped++; continue; @@ -382,14 +381,21 @@ static int bcm_enet_receive_queue(struct net_device *dev, int budget) dma_sync_single_for_cpu(kdev, desc->address, len, DMA_FROM_DEVICE); - memcpy(nskb->data, skb->data, len); + memcpy(skb->data, buf + priv->rx_buf_offset, len); dma_sync_single_for_device(kdev, desc->address, len, DMA_FROM_DEVICE); - skb = nskb; } else { - dma_unmap_single(&priv->pdev->dev, desc->address, - priv->rx_skb_size, DMA_FROM_DEVICE); - priv->rx_skb[desc_idx] = NULL; + dma_unmap_single(kdev, desc->address, + priv->rx_buf_size, DMA_FROM_DEVICE); + priv->rx_buf[desc_idx] = NULL; + + skb = build_skb(buf, priv->rx_frag_size); + if (unlikely(!skb)) { + skb_free_frag(buf); + dev->stats.rx_dropped++; + continue; + } + skb_reserve(skb, priv->rx_buf_offset); } skb_put(skb, len); @@ -403,7 +409,7 @@ static int bcm_enet_receive_queue(struct net_device *dev, int budget) netif_receive_skb_list(&rx_list); if (processed || !priv->rx_desc_count) { - bcm_enet_refill_rx(dev); + bcm_enet_refill_rx(dev, true); /* kick rx dma */ enet_dmac_writel(priv, priv->dma_chan_en_mask, @@ -860,22 +866,22 @@ static void bcm_enet_adjust_link(struct net_device *dev) priv->pause_tx ? "tx" : "off"); } -static void bcm_enet_free_rx_skb_ring(struct device *kdev, struct bcm_enet_priv *priv) +static void bcm_enet_free_rx_buf_ring(struct device *kdev, struct bcm_enet_priv *priv) { int i; for (i = 0; i < priv->rx_ring_size; i++) { struct bcm_enet_desc *desc; - if (!priv->rx_skb[i]) + if (!priv->rx_buf[i]) continue; desc = &priv->rx_desc_cpu[i]; - dma_unmap_single(kdev, desc->address, priv->rx_skb_size, + dma_unmap_single(kdev, desc->address, priv->rx_buf_size, DMA_FROM_DEVICE); - kfree_skb(priv->rx_skb[i]); + skb_free_frag(priv->rx_buf[i]); } - kfree(priv->rx_skb); + kfree(priv->rx_buf); } /* @@ -987,10 +993,10 @@ static int bcm_enet_open(struct net_device *dev) priv->tx_curr_desc = 0; spin_lock_init(&priv->tx_lock); - /* init & fill rx ring with skbs */ - priv->rx_skb = kcalloc(priv->rx_ring_size, sizeof(struct sk_buff *), + /* init & fill rx ring with buffers */ + priv->rx_buf = kcalloc(priv->rx_ring_size, sizeof(void *), GFP_KERNEL); - if (!priv->rx_skb) { + if (!priv->rx_buf) { ret = -ENOMEM; goto out_free_tx_skb; } @@ -1007,8 +1013,8 @@ static int bcm_enet_open(struct net_device *dev) enet_dmac_writel(priv, ENETDMA_BUFALLOC_FORCE_MASK | 0, ENETDMAC_BUFALLOC, priv->rx_chan); - if (bcm_enet_refill_rx(dev)) { - dev_err(kdev, "cannot allocate rx skb queue\n"); + if (bcm_enet_refill_rx(dev, false)) { + dev_err(kdev, "cannot allocate rx buffer queue\n"); ret = -ENOMEM; goto out; } @@ -1102,7 +1108,7 @@ static int bcm_enet_open(struct net_device *dev) return 0; out: - bcm_enet_free_rx_skb_ring(kdev, priv); + bcm_enet_free_rx_buf_ring(kdev, priv); out_free_tx_skb: kfree(priv->tx_skb); @@ -1208,8 +1214,8 @@ static int bcm_enet_stop(struct net_device *dev) /* force reclaim of all tx buffers */ bcm_enet_tx_reclaim(dev, 1); - /* free the rx skb ring */ - bcm_enet_free_rx_skb_ring(kdev, priv); + /* free the rx buffer ring */ + bcm_enet_free_rx_buf_ring(kdev, priv); /* free remaining allocated memory */ kfree(priv->tx_skb); @@ -1633,9 +1639,12 @@ static int bcm_enet_change_mtu(struct net_device *dev, int new_mtu) * align rx buffer size to dma burst len, account FCS since * it's appended */ - priv->rx_skb_size = ALIGN(actual_mtu + ETH_FCS_LEN, + priv->rx_buf_size = ALIGN(actual_mtu + ETH_FCS_LEN, priv->dma_maxburst * 4); + priv->rx_frag_size = SKB_DATA_ALIGN(priv->rx_buf_offset + priv->rx_buf_size) + + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)); + dev->mtu = new_mtu; return 0; } @@ -1720,6 +1729,7 @@ static int bcm_enet_probe(struct platform_device *pdev) priv->enet_is_sw = false; priv->dma_maxburst = BCMENET_DMA_MAXBURST; + priv->rx_buf_offset = NET_SKB_PAD; ret = bcm_enet_change_mtu(dev, dev->mtu); if (ret) @@ -2137,7 +2147,7 @@ static int bcm_enetsw_open(struct net_device *dev) priv->tx_skb = kcalloc(priv->tx_ring_size, sizeof(struct sk_buff *), GFP_KERNEL); if (!priv->tx_skb) { - dev_err(kdev, "cannot allocate rx skb queue\n"); + dev_err(kdev, "cannot allocate tx skb queue\n"); ret = -ENOMEM; goto out_free_tx_ring; } @@ -2147,11 +2157,11 @@ static int bcm_enetsw_open(struct net_device *dev) priv->tx_curr_desc = 0; spin_lock_init(&priv->tx_lock); - /* init & fill rx ring with skbs */ - priv->rx_skb = kcalloc(priv->rx_ring_size, sizeof(struct sk_buff *), + /* init & fill rx ring with buffers */ + priv->rx_buf = kcalloc(priv->rx_ring_size, sizeof(void *), GFP_KERNEL); - if (!priv->rx_skb) { - dev_err(kdev, "cannot allocate rx skb queue\n"); + if (!priv->rx_buf) { + dev_err(kdev, "cannot allocate rx buffer queue\n"); ret = -ENOMEM; goto out_free_tx_skb; } @@ -2198,8 +2208,8 @@ static int bcm_enetsw_open(struct net_device *dev) enet_dma_writel(priv, ENETDMA_BUFALLOC_FORCE_MASK | 0, ENETDMA_BUFALLOC_REG(priv->rx_chan)); - if (bcm_enet_refill_rx(dev)) { - dev_err(kdev, "cannot allocate rx skb queue\n"); + if (bcm_enet_refill_rx(dev, false)) { + dev_err(kdev, "cannot allocate rx buffer queue\n"); ret = -ENOMEM; goto out; } @@ -2298,7 +2308,7 @@ static int bcm_enetsw_open(struct net_device *dev) return 0; out: - bcm_enet_free_rx_skb_ring(kdev, priv); + bcm_enet_free_rx_buf_ring(kdev, priv); out_free_tx_skb: kfree(priv->tx_skb); @@ -2348,8 +2358,8 @@ static int bcm_enetsw_stop(struct net_device *dev) /* force reclaim of all tx buffers */ bcm_enet_tx_reclaim(dev, 1); - /* free the rx skb ring */ - bcm_enet_free_rx_skb_ring(kdev, priv); + /* free the rx buffer ring */ + bcm_enet_free_rx_buf_ring(kdev, priv); /* free remaining allocated memory */ kfree(priv->tx_skb); @@ -2648,6 +2658,7 @@ static int bcm_enetsw_probe(struct platform_device *pdev) priv->rx_ring_size = BCMENET_DEF_RX_DESC; priv->tx_ring_size = BCMENET_DEF_TX_DESC; priv->dma_maxburst = BCMENETSW_DMA_MAXBURST; + priv->rx_buf_offset = NET_SKB_PAD + NET_IP_ALIGN; pd = dev_get_platdata(&pdev->dev); if (pd) { diff --git a/drivers/net/ethernet/broadcom/bcm63xx_enet.h b/drivers/net/ethernet/broadcom/bcm63xx_enet.h index 1d3c917eb830..78f1830fb3cb 100644 --- a/drivers/net/ethernet/broadcom/bcm63xx_enet.h +++ b/drivers/net/ethernet/broadcom/bcm63xx_enet.h @@ -230,11 +230,17 @@ struct bcm_enet_priv { /* next dirty rx descriptor to refill */ int rx_dirty_desc; - /* size of allocated rx skbs */ - unsigned int rx_skb_size; + /* size of allocated rx buffers */ + unsigned int rx_buf_size; - /* list of skb given to hw for rx */ - struct sk_buff **rx_skb; + /* allocated rx buffer offset */ + unsigned int rx_buf_offset; + + /* size of allocated rx frag */ + unsigned int rx_frag_size; + + /* list of buffer given to hw for rx */ + void **rx_buf; /* used when rx skb allocation failed, so we defer rx queue * refill */ From patchwork Wed Jan 6 14:42:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sieng-Piaw Liew X-Patchwork-Id: 12001851 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 542F3C4332D for ; Wed, 6 Jan 2021 14:45:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2A1BD23100 for ; Wed, 6 Jan 2021 14:45:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727085AbhAFOoc (ORCPT ); Wed, 6 Jan 2021 09:44:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59326 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726527AbhAFOob (ORCPT ); Wed, 6 Jan 2021 09:44:31 -0500 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6234CC06134D; Wed, 6 Jan 2021 06:43:27 -0800 (PST) Received: by mail-pf1-x429.google.com with SMTP id m6so1824706pfm.6; Wed, 06 Jan 2021 06:43:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vZbmDTb0LxwuuPGyUB5ba2z6ziLQcYOI0K/25hO2Dx0=; b=TMMOJGKDAEV2jGJkqOX+5Rt1Zi+fJkf/XKL7dYcMgkDy9ZFH+5JoGHKmqdej30REEv 7gcogGi33Cq2mYRFJowHB2lNDtzi6AbyPuvdWr1I8ToBfZ2Pu0hmbMMt123YY1l1tSGe d6P7HCiYRXtQx2NJ4YBsElIxTGMgAWe/aXwNx8+723/XJMjiIyFcoZzl/bkEamqugp8O P1h+bK2g5uujntQpHMjdBDI/wC+u9mAdgGRo5gscPbxnscfQp/3esjDmoOtbqRvBuO4b R+zmoIqKH5oeqZbXx0F3cfM4lIlGQfmh28HoZnArDWrGIupKwQCPeaAa25e1UtLnbWtO 8zwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vZbmDTb0LxwuuPGyUB5ba2z6ziLQcYOI0K/25hO2Dx0=; b=iaX8ja7WDaDsloWqNFb+5dMyUENnMUUfv62UhUUhiI+SNrqia5HOhgipA1HdXkC/Gz dIwn1LMUitSwaQa92Yr0qb19rhLZBX6y5cdo3xEwBmq467ElO1TWqGeZN5b+WIiXNDrQ fB24uU5AMuV72ocibTVt9q8+EQCNEATlyRH/+oDpDOkPA40T05Kt8DSg8IjxgKNokkmf NwXtqSIw2FDbRYihIW48asutFvyABuXQD16qafyDFtopIsbMqbJhh59eTY5U8PKBWB6F pviXwtJ8scQcdd5m1Hi+Z8Zsa3RCkap+YzbHA7n17CpU4EuzYKnCWZuP0AeTCe+9ivFk 5FQQ== X-Gm-Message-State: AOAM530xKKo6BSCrAcIDvXXxpspoFdiCA/wg/3BX8IiyLxOt2gZDBOL8 pT5woPJtGmQVIbQo4PVtZMM7bXMVzOk= X-Google-Smtp-Source: ABdhPJxEchh+BCJzCpA0Qbvipmo23B3PLsu7S5X1UJRT/vlffhz3vdgqLGZKUJwZAIotOpcUp1qfgw== X-Received: by 2002:aa7:81d6:0:b029:19e:2987:7465 with SMTP id c22-20020aa781d60000b029019e29877465mr4204058pfn.29.1609944206977; Wed, 06 Jan 2021 06:43:26 -0800 (PST) Received: from DESKTOP-8REGVGF.localdomain ([124.13.157.5]) by smtp.gmail.com with ESMTPSA id h8sm3076774pjc.2.2021.01.06.06.43.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Jan 2021 06:43:26 -0800 (PST) From: Sieng Piaw Liew To: Florian Fainelli Cc: bcm-kernel-feedback-list@broadcom.com, Sieng Piaw Liew , netdev@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v3 7/7] bcm63xx_enet: improve rx loop Date: Wed, 6 Jan 2021 22:42:08 +0800 Message-Id: <20210106144208.1935-8-liew.s.piaw@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210106144208.1935-1-liew.s.piaw@gmail.com> References: <20210106144208.1935-1-liew.s.piaw@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Use existing rx processed count to track against budget, thereby making budget decrement operation redundant. rx_desc_count can be calculated outside the rx loop, making the loop a bit smaller. Signed-off-by: Sieng Piaw Liew Acked-by: Florian Fainelli --- drivers/net/ethernet/broadcom/bcm63xx_enet.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bcm63xx_enet.c b/drivers/net/ethernet/broadcom/bcm63xx_enet.c index c11491429ed2..fd8767213165 100644 --- a/drivers/net/ethernet/broadcom/bcm63xx_enet.c +++ b/drivers/net/ethernet/broadcom/bcm63xx_enet.c @@ -339,7 +339,6 @@ static int bcm_enet_receive_queue(struct net_device *dev, int budget) priv->rx_curr_desc++; if (priv->rx_curr_desc == priv->rx_ring_size) priv->rx_curr_desc = 0; - priv->rx_desc_count--; /* if the packet does not have start of packet _and_ * end of packet flag set, then just recycle it */ @@ -404,9 +403,10 @@ static int bcm_enet_receive_queue(struct net_device *dev, int budget) dev->stats.rx_bytes += len; list_add_tail(&skb->list, &rx_list); - } while (--budget > 0); + } while (processed < budget); netif_receive_skb_list(&rx_list); + priv->rx_desc_count -= processed; if (processed || !priv->rx_desc_count) { bcm_enet_refill_rx(dev, true);