From patchwork Tue Jun 4 07:28:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Barker X-Patchwork-Id: 13684831 X-Patchwork-Delegate: kuba@kernel.org Received: from relmlie5.idc.renesas.com (relmlor1.renesas.com [210.160.252.171]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9BCE3137C5A; Tue, 4 Jun 2024 07:28:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.160.252.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717486131; cv=none; b=O5jKIPqokCPixUqu0NUKHDixEfN5/cxeNJQv45rooCjk30chQejOO+EeYA7QshPtroDhkLuX7d8tloVMR9q1AECX42u4khY3uaihxtTBf8UG/9gmW5I6Sf7cTPgT2WiLXiyuxdZQvA7EA2IH3kO2oz9xGPzZbLAanAoMU8MbT7k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717486131; c=relaxed/simple; bh=cbgZ+pq+U2matu3Zc7+gDYS3VKVi1lfiwdDgZsp8wOM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pRHZsKjSc9Eb4G2pKXiZ0EZNKgvENaIpgrEDk8K4kTFSXF2o74/7L/W8RPsmMrUexm5wlB9V4WYySrLOjf5f4K+NfjKzxI/1qsQU6kFGFIdgQWpZ/M+KxM6iSFIts2XRkaRrz1422htH+RGtt8gev0e8sYKMmIAij8Mb6PU+cYQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bp.renesas.com; spf=pass smtp.mailfrom=bp.renesas.com; arc=none smtp.client-ip=210.160.252.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bp.renesas.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bp.renesas.com X-IronPort-AV: E=Sophos;i="6.08,213,1712588400"; d="scan'208";a="206651303" Received: from unknown (HELO relmlir6.idc.renesas.com) ([10.200.68.152]) by relmlie5.idc.renesas.com with ESMTP; 04 Jun 2024 16:28:44 +0900 Received: from renesas-deb12.mshome.net (unknown [10.226.92.239]) by relmlir6.idc.renesas.com (Postfix) with ESMTP id 1E1C741FBB3D; Tue, 4 Jun 2024 16:28:39 +0900 (JST) From: Paul Barker To: Sergey Shtylyov , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , =?utf-8?q?Niklas_S=C3=B6derlund?= , Simon Horman Cc: Paul Barker , Biju Das , Claudiu Beznea , Yoshihiro Shimoda , netdev@vger.kernel.org, linux-renesas-soc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [net-next PATCH v5 1/7] net: ravb: Simplify poll & receive functions Date: Tue, 4 Jun 2024 08:28:19 +0100 Message-Id: <20240604072825.7490-2-paul.barker.ct@bp.renesas.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240604072825.7490-1-paul.barker.ct@bp.renesas.com> References: <20240604072825.7490-1-paul.barker.ct@bp.renesas.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org We don't need to pass the work budget to ravb_rx() by reference, it's cleaner to pass this by value and return the amount of work done. This allows us to simplify the ravb_poll() function and use the common `work_done` variable name seen in other network drivers for consistency and ease of understanding. This is a pure refactor and should not affect behaviour. Signed-off-by: Paul Barker Reviewed-by: Sergey Shtylyov --- Changes v4->v5: * Added Sergey's Reviewed-by tag. Changes v3->v4: * Fixed update of stats->rx_packets in ravb_rx_gbeth() and ravb_rx_rcar(). drivers/net/ethernet/renesas/ravb.h | 2 +- drivers/net/ethernet/renesas/ravb_main.c | 27 +++++++++++------------- 2 files changed, 13 insertions(+), 16 deletions(-) diff --git a/drivers/net/ethernet/renesas/ravb.h b/drivers/net/ethernet/renesas/ravb.h index b48935ec7e28..71de2a7aa27c 100644 --- a/drivers/net/ethernet/renesas/ravb.h +++ b/drivers/net/ethernet/renesas/ravb.h @@ -1039,7 +1039,7 @@ struct ravb_ptp { }; struct ravb_hw_info { - bool (*receive)(struct net_device *ndev, int *quota, int q); + int (*receive)(struct net_device *ndev, int budget, int q); void (*set_rate)(struct net_device *ndev); int (*set_feature)(struct net_device *ndev, netdev_features_t features); int (*dmac_init)(struct net_device *ndev); diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c index 4d100283c30f..193ad05383a8 100644 --- a/drivers/net/ethernet/renesas/ravb_main.c +++ b/drivers/net/ethernet/renesas/ravb_main.c @@ -759,7 +759,7 @@ static struct sk_buff *ravb_get_skb_gbeth(struct net_device *ndev, int entry, } /* Packet receive function for Gigabit Ethernet */ -static bool ravb_rx_gbeth(struct net_device *ndev, int *quota, int q) +static int ravb_rx_gbeth(struct net_device *ndev, int budget, int q) { struct ravb_private *priv = netdev_priv(ndev); const struct ravb_hw_info *info = priv->info; @@ -781,7 +781,7 @@ static bool ravb_rx_gbeth(struct net_device *ndev, int *quota, int q) for (i = 0; i < limit; i++, priv->cur_rx[q]++) { entry = priv->cur_rx[q] % priv->num_rx_ring[q]; desc = &priv->rx_ring[q].desc[entry]; - if (rx_packets == *quota || desc->die_dt == DT_FEMPTY) + if (rx_packets == budget || desc->die_dt == DT_FEMPTY) break; /* Descriptor type must be checked before all other reads */ @@ -882,12 +882,11 @@ static bool ravb_rx_gbeth(struct net_device *ndev, int *quota, int q) } stats->rx_packets += rx_packets; - *quota -= rx_packets; - return *quota == 0; + return rx_packets; } /* Packet receive function for Ethernet AVB */ -static bool ravb_rx_rcar(struct net_device *ndev, int *quota, int q) +static int ravb_rx_rcar(struct net_device *ndev, int budget, int q) { struct ravb_private *priv = netdev_priv(ndev); const struct ravb_hw_info *info = priv->info; @@ -906,7 +905,7 @@ static bool ravb_rx_rcar(struct net_device *ndev, int *quota, int q) for (i = 0; i < limit; i++, priv->cur_rx[q]++) { entry = priv->cur_rx[q] % priv->num_rx_ring[q]; desc = &priv->rx_ring[q].ex_desc[entry]; - if (rx_packets == *quota || desc->die_dt == DT_FEMPTY) + if (rx_packets == budget || desc->die_dt == DT_FEMPTY) break; /* Descriptor type must be checked before all other reads */ @@ -992,17 +991,16 @@ static bool ravb_rx_rcar(struct net_device *ndev, int *quota, int q) } stats->rx_packets += rx_packets; - *quota -= rx_packets; - return *quota == 0; + return rx_packets; } /* Packet receive function for Ethernet AVB */ -static bool ravb_rx(struct net_device *ndev, int *quota, int q) +static int ravb_rx(struct net_device *ndev, int budget, int q) { struct ravb_private *priv = netdev_priv(ndev); const struct ravb_hw_info *info = priv->info; - return info->receive(ndev, quota, q); + return info->receive(ndev, budget, q); } static void ravb_rcv_snd_disable(struct net_device *ndev) @@ -1319,13 +1317,12 @@ static int ravb_poll(struct napi_struct *napi, int budget) unsigned long flags; int q = napi - priv->napi; int mask = BIT(q); - int quota = budget; - bool unmask; + int work_done; /* Processing RX Descriptor Ring */ /* Clear RX interrupt */ ravb_write(ndev, ~(mask | RIS0_RESERVED), RIS0); - unmask = !ravb_rx(ndev, "a, q); + work_done = ravb_rx(ndev, budget, q); /* Processing TX Descriptor Ring */ spin_lock_irqsave(&priv->lock, flags); @@ -1344,7 +1341,7 @@ static int ravb_poll(struct napi_struct *napi, int budget) if (priv->rx_fifo_errors != ndev->stats.rx_fifo_errors) ndev->stats.rx_fifo_errors = priv->rx_fifo_errors; - if (!unmask) + if (work_done == budget) goto out; napi_complete(napi); @@ -1361,7 +1358,7 @@ static int ravb_poll(struct napi_struct *napi, int budget) spin_unlock_irqrestore(&priv->lock, flags); out: - return budget - quota; + return work_done; } static void ravb_set_duplex_gbeth(struct net_device *ndev) From patchwork Tue Jun 4 07:28:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Barker X-Patchwork-Id: 13684832 X-Patchwork-Delegate: kuba@kernel.org Received: from relmlie5.idc.renesas.com (relmlor1.renesas.com [210.160.252.171]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DF5AA1419BC; Tue, 4 Jun 2024 07:28:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.160.252.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717486134; cv=none; b=A785RBHVc2GjRYHmPXmQmAu8bHPL7qJHSAE6C1gYSVJAVFF6VLvC5hEe5rk7myUxqQL2s1biI7JoMx69if6+UnHgCtJnPpbBfQlU0L/082J3Uawb8OhgjdLqY90xvBWEZJahI/0wbpTUIWEcZvkA1N+JgtnZk5UR+bXfdIs1ec0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717486134; c=relaxed/simple; bh=RSBGKdOzXdRtmVAKwVSFdjPnJ9eoE/CJ2FGAoU3DE7E=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KnJZnogPPjzBe6geiU7Ao1zYJfeP0cwUgH88a4GN9SiRo3UAVd+ez4iOpLbG36LubfC2XUhJ1S2QnvUGcsj6uvIsm/bTcqxP4R1Mxrvco2O/qZDVEwa0Yusg1i4HNUmOj0sL3IdboEzIbs1PwBH/DNzqrhV14/piwEfHqjLhCDM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bp.renesas.com; spf=pass smtp.mailfrom=bp.renesas.com; arc=none smtp.client-ip=210.160.252.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bp.renesas.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bp.renesas.com X-IronPort-AV: E=Sophos;i="6.08,213,1712588400"; d="scan'208";a="206651310" Received: from unknown (HELO relmlir6.idc.renesas.com) ([10.200.68.152]) by relmlie5.idc.renesas.com with ESMTP; 04 Jun 2024 16:28:50 +0900 Received: from renesas-deb12.mshome.net (unknown [10.226.92.239]) by relmlir6.idc.renesas.com (Postfix) with ESMTP id 7E3D541FB3BD; Tue, 4 Jun 2024 16:28:45 +0900 (JST) From: Paul Barker To: Sergey Shtylyov , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , =?utf-8?q?Niklas_S=C3=B6derlund?= , Simon Horman Cc: Paul Barker , Biju Das , Claudiu Beznea , Yoshihiro Shimoda , netdev@vger.kernel.org, linux-renesas-soc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [net-next PATCH v5 2/7] net: ravb: Align poll function with NAPI docs Date: Tue, 4 Jun 2024 08:28:20 +0100 Message-Id: <20240604072825.7490-3-paul.barker.ct@bp.renesas.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240604072825.7490-1-paul.barker.ct@bp.renesas.com> References: <20240604072825.7490-1-paul.barker.ct@bp.renesas.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Align ravb_poll() with the documentation in `Documentation/networking/kapi.rst` and `Documentation/networking/napi.rst`. The documentation says that we should prefer napi_complete_done() over napi_complete(), and using the former allows us to properly support busy polling. We should ensure that napi_complete_done() is only called if the work budget has not been exhausted, and we should only re-arm interrupts if it returns true. Signed-off-by: Paul Barker Reviewed-by: Sergey Shtylyov --- Changes v4->v5: * Re-wrote the commit message to make things clearer. Changes v3->v4: * Used Niklas' suggested commit message. drivers/net/ethernet/renesas/ravb_main.c | 26 ++++++++++-------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c index 193ad05383a8..472aa80002be 100644 --- a/drivers/net/ethernet/renesas/ravb_main.c +++ b/drivers/net/ethernet/renesas/ravb_main.c @@ -1341,23 +1341,19 @@ static int ravb_poll(struct napi_struct *napi, int budget) if (priv->rx_fifo_errors != ndev->stats.rx_fifo_errors) ndev->stats.rx_fifo_errors = priv->rx_fifo_errors; - if (work_done == budget) - goto out; - - napi_complete(napi); - - /* Re-enable RX/TX interrupts */ - spin_lock_irqsave(&priv->lock, flags); - if (!info->irq_en_dis) { - ravb_modify(ndev, RIC0, mask, mask); - ravb_modify(ndev, TIC, mask, mask); - } else { - ravb_write(ndev, mask, RIE0); - ravb_write(ndev, mask, TIE); + if (work_done < budget && napi_complete_done(napi, work_done)) { + /* Re-enable RX/TX interrupts */ + spin_lock_irqsave(&priv->lock, flags); + if (!info->irq_en_dis) { + ravb_modify(ndev, RIC0, mask, mask); + ravb_modify(ndev, TIC, mask, mask); + } else { + ravb_write(ndev, mask, RIE0); + ravb_write(ndev, mask, TIE); + } + spin_unlock_irqrestore(&priv->lock, flags); } - spin_unlock_irqrestore(&priv->lock, flags); -out: return work_done; } From patchwork Tue Jun 4 07:28:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Barker X-Patchwork-Id: 13684833 X-Patchwork-Delegate: kuba@kernel.org Received: from relmlie5.idc.renesas.com (relmlor1.renesas.com [210.160.252.171]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C35BC83CB4; Tue, 4 Jun 2024 07:28:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.160.252.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717486141; cv=none; b=ODFQ6fu21fVBwX7rU3MQ6ptjBYBkrAnWid36GvwTjVnaZl0ZBoFe8vgG5k1SgGrxnLws2jcVNe2/63StoC0PAl7Zy/vIEiXzYRfKMMOK/FXtZxDSRwbfQDkLyL28c63Ka2m1cFW1IVuKLY2bT3NhOsR414xQTIJlKmx7K84FzoM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717486141; c=relaxed/simple; bh=RvBmkc/0Z0rEHBdLhh543QGElROuM7v8GOvtolnFabg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rzFkbwe9mh9J5AEI00d1uOeq1b88wmxuMgggQ3xxr7sbqUW9yTyohtG/4CtlvNI7wdY31+jZr/FqYtiMKO4iZgNpepS28WkzHPVVnlA2uF0UmiamxeIJUqlng2YdPPv3zza6AzbTwacN2hsZNpNW54GVAKnb1ktzG/0ieQkgbnc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bp.renesas.com; spf=pass smtp.mailfrom=bp.renesas.com; arc=none smtp.client-ip=210.160.252.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bp.renesas.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bp.renesas.com X-IronPort-AV: E=Sophos;i="6.08,213,1712588400"; d="scan'208";a="206651317" Received: from unknown (HELO relmlir6.idc.renesas.com) ([10.200.68.152]) by relmlie5.idc.renesas.com with ESMTP; 04 Jun 2024 16:28:55 +0900 Received: from renesas-deb12.mshome.net (unknown [10.226.92.239]) by relmlir6.idc.renesas.com (Postfix) with ESMTP id E545441FBB3D; Tue, 4 Jun 2024 16:28:50 +0900 (JST) From: Paul Barker To: Sergey Shtylyov , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , =?utf-8?q?Niklas_S=C3=B6derlund?= , Simon Horman Cc: Paul Barker , Biju Das , Claudiu Beznea , Yoshihiro Shimoda , netdev@vger.kernel.org, linux-renesas-soc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [net-next PATCH v5 3/7] net: ravb: Refactor RX ring refill Date: Tue, 4 Jun 2024 08:28:21 +0100 Message-Id: <20240604072825.7490-4-paul.barker.ct@bp.renesas.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240604072825.7490-1-paul.barker.ct@bp.renesas.com> References: <20240604072825.7490-1-paul.barker.ct@bp.renesas.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org To reduce code duplication, we add a new RX ring refill function which can handle both the initial RX ring population (which was split between ravb_ring_init() and ravb_ring_format()) and the RX ring refill after polling (in ravb_rx()). Signed-off-by: Paul Barker Reviewed-by: Sergey Shtylyov --- Changes v4->v5: * Re-wrapped to 80 cols, except where this would significantly impact readability. * Added Sergey's Reviewed-by tag. Changes v3->v4: * Avoided unnecessarily re-ordering code in ravb_ring_init() and ravb_ring_format(). drivers/net/ethernet/renesas/ravb_main.c | 150 +++++++++-------------- 1 file changed, 57 insertions(+), 93 deletions(-) diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c index 472aa80002be..63d03929e3f5 100644 --- a/drivers/net/ethernet/renesas/ravb_main.c +++ b/drivers/net/ethernet/renesas/ravb_main.c @@ -317,35 +317,43 @@ static void ravb_ring_free(struct net_device *ndev, int q) priv->tx_skb[q] = NULL; } -static void ravb_rx_ring_format(struct net_device *ndev, int q) +static u32 +ravb_rx_ring_refill(struct net_device *ndev, int q, u32 count, gfp_t gfp_mask) { struct ravb_private *priv = netdev_priv(ndev); + const struct ravb_hw_info *info = priv->info; struct ravb_rx_desc *rx_desc; - unsigned int rx_ring_size; dma_addr_t dma_addr; - unsigned int i; + u32 i, entry; - rx_ring_size = priv->info->rx_desc_size * priv->num_rx_ring[q]; - memset(priv->rx_ring[q].raw, 0, rx_ring_size); - /* Build RX ring buffer */ - for (i = 0; i < priv->num_rx_ring[q]; i++) { - /* RX descriptor */ - rx_desc = ravb_rx_get_desc(priv, q, i); - rx_desc->ds_cc = cpu_to_le16(priv->info->rx_max_desc_use); - dma_addr = dma_map_single(ndev->dev.parent, priv->rx_skb[q][i]->data, - priv->info->rx_max_frame_size, - DMA_FROM_DEVICE); - /* We just set the data size to 0 for a failed mapping which - * should prevent DMA from happening... - */ - if (dma_mapping_error(ndev->dev.parent, dma_addr)) - rx_desc->ds_cc = cpu_to_le16(0); - rx_desc->dptr = cpu_to_le32(dma_addr); + for (i = 0; i < count; i++) { + entry = (priv->dirty_rx[q] + i) % priv->num_rx_ring[q]; + rx_desc = ravb_rx_get_desc(priv, q, entry); + rx_desc->ds_cc = cpu_to_le16(info->rx_max_desc_use); + + if (!priv->rx_skb[q][entry]) { + priv->rx_skb[q][entry] = ravb_alloc_skb(ndev, info, + gfp_mask); + if (!priv->rx_skb[q][entry]) + break; + dma_addr = dma_map_single(ndev->dev.parent, + priv->rx_skb[q][entry]->data, + priv->info->rx_max_frame_size, + DMA_FROM_DEVICE); + skb_checksum_none_assert(priv->rx_skb[q][entry]); + /* We just set the data size to 0 for a failed mapping + * which should prevent DMA from happening... + */ + if (dma_mapping_error(ndev->dev.parent, dma_addr)) + rx_desc->ds_cc = cpu_to_le16(0); + rx_desc->dptr = cpu_to_le32(dma_addr); + } + /* Descriptor type must be set after all the above writes */ + dma_wmb(); rx_desc->die_dt = DT_FEMPTY; } - rx_desc = ravb_rx_get_desc(priv, q, i); - rx_desc->dptr = cpu_to_le32((u32)priv->rx_desc_dma[q]); - rx_desc->die_dt = DT_LINKFIX; /* type */ + + return i; } /* Format skb and descriptor buffer for Ethernet AVB */ @@ -353,6 +361,7 @@ static void ravb_ring_format(struct net_device *ndev, int q) { struct ravb_private *priv = netdev_priv(ndev); unsigned int num_tx_desc = priv->num_tx_desc; + struct ravb_rx_desc *rx_desc; struct ravb_tx_desc *tx_desc; struct ravb_desc *desc; unsigned int tx_ring_size = sizeof(*tx_desc) * priv->num_tx_ring[q] * @@ -364,7 +373,13 @@ static void ravb_ring_format(struct net_device *ndev, int q) priv->dirty_rx[q] = 0; priv->dirty_tx[q] = 0; - ravb_rx_ring_format(ndev, q); + /* Regular RX descriptors have already been initialized by + * ravb_rx_ring_refill(), we just need to initialize the final link + * descriptor. + */ + rx_desc = ravb_rx_get_desc(priv, q, priv->num_rx_ring[q]); + rx_desc->dptr = cpu_to_le32((u32)priv->rx_desc_dma[q]); + rx_desc->die_dt = DT_LINKFIX; /* type */ memset(priv->tx_ring[q], 0, tx_ring_size); /* Build TX ring buffer */ @@ -408,11 +423,9 @@ static void *ravb_alloc_rx_desc(struct net_device *ndev, int q) static int ravb_ring_init(struct net_device *ndev, int q) { struct ravb_private *priv = netdev_priv(ndev); - const struct ravb_hw_info *info = priv->info; unsigned int num_tx_desc = priv->num_tx_desc; unsigned int ring_size; - struct sk_buff *skb; - unsigned int i; + u32 num_filled; /* Allocate RX and TX skb rings */ priv->rx_skb[q] = kcalloc(priv->num_rx_ring[q], @@ -422,12 +435,18 @@ static int ravb_ring_init(struct net_device *ndev, int q) if (!priv->rx_skb[q] || !priv->tx_skb[q]) goto error; - for (i = 0; i < priv->num_rx_ring[q]; i++) { - skb = ravb_alloc_skb(ndev, info, GFP_KERNEL); - if (!skb) - goto error; - priv->rx_skb[q][i] = skb; - } + /* Allocate all RX descriptors. */ + if (!ravb_alloc_rx_desc(ndev, q)) + goto error; + + /* Populate RX ring buffer. */ + priv->dirty_rx[q] = 0; + ring_size = priv->info->rx_desc_size * priv->num_rx_ring[q]; + memset(priv->rx_ring[q].raw, 0, ring_size); + num_filled = ravb_rx_ring_refill(ndev, q, priv->num_rx_ring[q], + GFP_KERNEL); + if (num_filled != priv->num_rx_ring[q]) + goto error; if (num_tx_desc > 1) { /* Allocate rings for the aligned buffers */ @@ -437,12 +456,6 @@ static int ravb_ring_init(struct net_device *ndev, int q) goto error; } - /* Allocate all RX descriptors. */ - if (!ravb_alloc_rx_desc(ndev, q)) - goto error; - - priv->dirty_rx[q] = 0; - /* Allocate all TX descriptors. */ ring_size = sizeof(struct ravb_tx_desc) * (priv->num_tx_ring[q] * num_tx_desc + 1); @@ -762,11 +775,9 @@ static struct sk_buff *ravb_get_skb_gbeth(struct net_device *ndev, int entry, static int ravb_rx_gbeth(struct net_device *ndev, int budget, int q) { struct ravb_private *priv = netdev_priv(ndev); - const struct ravb_hw_info *info = priv->info; struct net_device_stats *stats; struct ravb_rx_desc *desc; struct sk_buff *skb; - dma_addr_t dma_addr; int rx_packets = 0; u8 desc_status; u16 desc_len; @@ -854,32 +865,9 @@ static int ravb_rx_gbeth(struct net_device *ndev, int budget, int q) } /* Refill the RX ring buffers. */ - for (; priv->cur_rx[q] - priv->dirty_rx[q] > 0; priv->dirty_rx[q]++) { - entry = priv->dirty_rx[q] % priv->num_rx_ring[q]; - desc = &priv->rx_ring[q].desc[entry]; - desc->ds_cc = cpu_to_le16(priv->info->rx_max_desc_use); - - if (!priv->rx_skb[q][entry]) { - skb = ravb_alloc_skb(ndev, info, GFP_ATOMIC); - if (!skb) - break; - dma_addr = dma_map_single(ndev->dev.parent, - skb->data, - priv->info->rx_max_frame_size, - DMA_FROM_DEVICE); - skb_checksum_none_assert(skb); - /* We just set the data size to 0 for a failed mapping - * which should prevent DMA from happening... - */ - if (dma_mapping_error(ndev->dev.parent, dma_addr)) - desc->ds_cc = cpu_to_le16(0); - desc->dptr = cpu_to_le32(dma_addr); - priv->rx_skb[q][entry] = skb; - } - /* Descriptor type must be set after all the above writes */ - dma_wmb(); - desc->die_dt = DT_FEMPTY; - } + priv->dirty_rx[q] += ravb_rx_ring_refill(ndev, q, + priv->cur_rx[q] - priv->dirty_rx[q], + GFP_ATOMIC); stats->rx_packets += rx_packets; return rx_packets; @@ -889,12 +877,10 @@ static int ravb_rx_gbeth(struct net_device *ndev, int budget, int q) static int ravb_rx_rcar(struct net_device *ndev, int budget, int q) { struct ravb_private *priv = netdev_priv(ndev); - const struct ravb_hw_info *info = priv->info; struct net_device_stats *stats = &priv->stats[q]; struct ravb_ex_rx_desc *desc; unsigned int limit, i; struct sk_buff *skb; - dma_addr_t dma_addr; struct timespec64 ts; int rx_packets = 0; u8 desc_status; @@ -964,31 +950,9 @@ static int ravb_rx_rcar(struct net_device *ndev, int budget, int q) } /* Refill the RX ring buffers. */ - for (; priv->cur_rx[q] - priv->dirty_rx[q] > 0; priv->dirty_rx[q]++) { - entry = priv->dirty_rx[q] % priv->num_rx_ring[q]; - desc = &priv->rx_ring[q].ex_desc[entry]; - desc->ds_cc = cpu_to_le16(priv->info->rx_max_desc_use); - - if (!priv->rx_skb[q][entry]) { - skb = ravb_alloc_skb(ndev, info, GFP_ATOMIC); - if (!skb) - break; /* Better luck next round. */ - dma_addr = dma_map_single(ndev->dev.parent, skb->data, - priv->info->rx_max_frame_size, - DMA_FROM_DEVICE); - skb_checksum_none_assert(skb); - /* We just set the data size to 0 for a failed mapping - * which should prevent DMA from happening... - */ - if (dma_mapping_error(ndev->dev.parent, dma_addr)) - desc->ds_cc = cpu_to_le16(0); - desc->dptr = cpu_to_le32(dma_addr); - priv->rx_skb[q][entry] = skb; - } - /* Descriptor type must be set after all the above writes */ - dma_wmb(); - desc->die_dt = DT_FEMPTY; - } + priv->dirty_rx[q] += ravb_rx_ring_refill(ndev, q, + priv->cur_rx[q] - priv->dirty_rx[q], + GFP_ATOMIC); stats->rx_packets += rx_packets; return rx_packets; From patchwork Tue Jun 4 07:28:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Barker X-Patchwork-Id: 13684834 X-Patchwork-Delegate: kuba@kernel.org Received: from relmlie6.idc.renesas.com (relmlor2.renesas.com [210.160.252.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 606AD142900; Tue, 4 Jun 2024 07:29:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.160.252.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717486144; cv=none; b=LFnHr6aasUdpZNGdnCPwiKHo5SJrVfNcR+OAoWcE/KR39ywOFP/YjWFZKVwOb5Tk8amWcGbfBxGv3f/h1hYZqOxTa9DkA4qdu+w3ROXg46Hb5ZqceL0SwH65nBJom90NS7eZyZGshDIVclRFvNKKDc+SoHi/8T2Xxw+X/ycgWNQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717486144; c=relaxed/simple; bh=S9rcrkNCIr3Jy7WR55wdfEGflMtmySEn/dQxylxKq+I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=VTarzdkiwYaNdILtY4TaxzUo83CYSvtPg1Iye1jMDwRlkOvAFIHjTNMGfiOjHkInBZsAJ/6wrrdoSMTTO/Jlw0F61WIYajZPeOcN2KYEblQEUECQoJp8ZLygToL7s7TAdliCUUbvlaUdd6r21SgjpODwuLqZTiWO3stACd1HYi4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bp.renesas.com; spf=pass smtp.mailfrom=bp.renesas.com; arc=none smtp.client-ip=210.160.252.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bp.renesas.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bp.renesas.com X-IronPort-AV: E=Sophos;i="6.08,213,1712588400"; d="scan'208";a="210610445" Received: from unknown (HELO relmlir6.idc.renesas.com) ([10.200.68.152]) by relmlie6.idc.renesas.com with ESMTP; 04 Jun 2024 16:29:01 +0900 Received: from renesas-deb12.mshome.net (unknown [10.226.92.239]) by relmlir6.idc.renesas.com (Postfix) with ESMTP id 5042B41FC779; Tue, 4 Jun 2024 16:28:56 +0900 (JST) From: Paul Barker To: Sergey Shtylyov , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , =?utf-8?q?Niklas_S=C3=B6derlund?= , Simon Horman Cc: Paul Barker , Biju Das , Claudiu Beznea , Yoshihiro Shimoda , netdev@vger.kernel.org, linux-renesas-soc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [net-next PATCH v5 4/7] net: ravb: Refactor GbEth RX code path Date: Tue, 4 Jun 2024 08:28:22 +0100 Message-Id: <20240604072825.7490-5-paul.barker.ct@bp.renesas.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240604072825.7490-1-paul.barker.ct@bp.renesas.com> References: <20240604072825.7490-1-paul.barker.ct@bp.renesas.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org We can reduce code duplication in ravb_rx_gbeth(). Signed-off-by: Paul Barker Reviewed-by: Sergey Shtylyov --- Changes v4->v5: * Re-wrapped to 80 cols, except where this would significantly impact readability. * Use lower case `skb` consistently in comments. Changes v3->v4: * Used switch statements instead of if statements in ravb_rx_gbeth(). drivers/net/ethernet/renesas/ravb_main.c | 71 +++++++++++++----------- 1 file changed, 38 insertions(+), 33 deletions(-) diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c index 63d03929e3f5..e8c3f48b213d 100644 --- a/drivers/net/ethernet/renesas/ravb_main.c +++ b/drivers/net/ethernet/renesas/ravb_main.c @@ -819,47 +819,52 @@ static int ravb_rx_gbeth(struct net_device *ndev, int budget, int q) stats->rx_missed_errors++; } else { die_dt = desc->die_dt & 0xF0; + skb = ravb_get_skb_gbeth(ndev, entry, desc); switch (die_dt) { case DT_FSINGLE: - skb = ravb_get_skb_gbeth(ndev, entry, desc); + case DT_FSTART: + /* Start of packet: Set initial data length. */ skb_put(skb, desc_len); + + /* Save this skb if the packet spans multiple + * descriptors. + */ + if (die_dt == DT_FSTART) + priv->rx_1st_skb = skb; + break; + + case DT_FMID: + case DT_FEND: + /* Continuing a packet: Move data into the saved + * skb. + */ + skb_copy_to_linear_data_offset(priv->rx_1st_skb, + priv->rx_1st_skb->len, + skb->data, + desc_len); + skb_put(priv->rx_1st_skb, desc_len); + dev_kfree_skb(skb); + + /* Set skb to point at the whole packet so that + * we only need one code path for finishing a + * packet. + */ + skb = priv->rx_1st_skb; + } + + switch (die_dt) { + case DT_FSINGLE: + case DT_FEND: + /* Finishing a packet: Determine protocol & + * checksum, hand off to NAPI and update our + * stats. + */ skb->protocol = eth_type_trans(skb, ndev); if (ndev->features & NETIF_F_RXCSUM) ravb_rx_csum_gbeth(skb); + stats->rx_bytes += skb->len; napi_gro_receive(&priv->napi[q], skb); rx_packets++; - stats->rx_bytes += desc_len; - break; - case DT_FSTART: - priv->rx_1st_skb = ravb_get_skb_gbeth(ndev, entry, desc); - skb_put(priv->rx_1st_skb, desc_len); - break; - case DT_FMID: - skb = ravb_get_skb_gbeth(ndev, entry, desc); - skb_copy_to_linear_data_offset(priv->rx_1st_skb, - priv->rx_1st_skb->len, - skb->data, - desc_len); - skb_put(priv->rx_1st_skb, desc_len); - dev_kfree_skb(skb); - break; - case DT_FEND: - skb = ravb_get_skb_gbeth(ndev, entry, desc); - skb_copy_to_linear_data_offset(priv->rx_1st_skb, - priv->rx_1st_skb->len, - skb->data, - desc_len); - skb_put(priv->rx_1st_skb, desc_len); - dev_kfree_skb(skb); - priv->rx_1st_skb->protocol = - eth_type_trans(priv->rx_1st_skb, ndev); - if (ndev->features & NETIF_F_RXCSUM) - ravb_rx_csum_gbeth(priv->rx_1st_skb); - stats->rx_bytes += priv->rx_1st_skb->len; - napi_gro_receive(&priv->napi[q], - priv->rx_1st_skb); - rx_packets++; - break; } } } From patchwork Tue Jun 4 07:28:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Barker X-Patchwork-Id: 13684835 X-Patchwork-Delegate: kuba@kernel.org Received: from relmlie5.idc.renesas.com (relmlor1.renesas.com [210.160.252.171]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CE1D5130A68; Tue, 4 Jun 2024 07:29:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.160.252.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717486149; cv=none; b=TRq1G4crEPjpBSNHq6VSRRyFJrU1ucH3sE4H9WTGoe3xfFpwiyEeU9tayMyZ/xcVHq8mPK8f9A6QjBOgK0UiS5U++nAAMLpeAV7GHEpGKW0elFFTnMJAA10NXu7WesKiiHyCmj/9yaTTn/4RE1U+XDehTyXPwTIAuf4r/PhZRGU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717486149; c=relaxed/simple; bh=3gEZCTL28/ZgT3H/CqmcdTDxG82gz3LtvnvdDELxE4w=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jzsj9tDACEuKSCUcH0o5ycCloOCKZqunrdMR7GDMk5vgbdvxBkuf10KZCmicHi5XSzaPp0e1WAwPNtLZzW0ry+/XrvpXBBNGPf0BtEZXkLcwm3/GINNpSbyQLLdsKBqlkEr99WKSvRejiqqa6d87E7y5PEJINASkiAwJCRrqg94= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bp.renesas.com; spf=pass smtp.mailfrom=bp.renesas.com; arc=none smtp.client-ip=210.160.252.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bp.renesas.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bp.renesas.com X-IronPort-AV: E=Sophos;i="6.08,213,1712588400"; d="scan'208";a="206651367" Received: from unknown (HELO relmlir6.idc.renesas.com) ([10.200.68.152]) by relmlie5.idc.renesas.com with ESMTP; 04 Jun 2024 16:29:06 +0900 Received: from renesas-deb12.mshome.net (unknown [10.226.92.239]) by relmlir6.idc.renesas.com (Postfix) with ESMTP id BF8E741FCFB9; Tue, 4 Jun 2024 16:29:01 +0900 (JST) From: Paul Barker To: Sergey Shtylyov , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , =?utf-8?q?Niklas_S=C3=B6derlund?= , Simon Horman Cc: Paul Barker , Biju Das , Claudiu Beznea , Yoshihiro Shimoda , netdev@vger.kernel.org, linux-renesas-soc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [net-next PATCH v5 5/7] net: ravb: Enable SW IRQ Coalescing for GbEth Date: Tue, 4 Jun 2024 08:28:23 +0100 Message-Id: <20240604072825.7490-6-paul.barker.ct@bp.renesas.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240604072825.7490-1-paul.barker.ct@bp.renesas.com> References: <20240604072825.7490-1-paul.barker.ct@bp.renesas.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Software IRQ Coalescing is required to improve network stack performance in the RZ/G2L SoC family and the RZ/G3S SoC, i.e. the SoCs which use the GbEth IP. This patch gives the following improvements during testing with iperf3: * RZ/G2L: * TCP RX: same bandwidth with -6% CPU load (76% -> 71%) * UDP RX: same bandwidth with -10% CPU load (99% -> 89%) * RZ/G2UL: * UDP RX: +4200% bandwidth (1.23Mbps -> 53Mbps) * RZ/G3S: * UDP RX: +425% bandwidth (1.23Mbps -> 6.46Mbps) The improvement of UDP RX bandwidth for the single core SoCs (RZ/G2UL & RZ/G3S) is particularly critical. Signed-off-by: Paul Barker Reviewed-by: Sergey Shtylyov --- No changes v4->v5 Changes v3->v4: * Renamed needs_irq_coalesce -> coalesce_irqs. * Added Sergey's Reviewed-by tag. drivers/net/ethernet/renesas/ravb.h | 1 + drivers/net/ethernet/renesas/ravb_main.c | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/drivers/net/ethernet/renesas/ravb.h b/drivers/net/ethernet/renesas/ravb.h index 71de2a7aa27c..6a7aa7dd17e6 100644 --- a/drivers/net/ethernet/renesas/ravb.h +++ b/drivers/net/ethernet/renesas/ravb.h @@ -1054,6 +1054,7 @@ struct ravb_hw_info { u32 rx_max_desc_use; u32 rx_desc_size; unsigned aligned_tx: 1; + unsigned coalesce_irqs:1; /* Needs software IRQ coalescing */ /* hardware features */ unsigned internal_delay:1; /* AVB-DMAC has internal delays */ diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c index e8c3f48b213d..2c24cb0271af 100644 --- a/drivers/net/ethernet/renesas/ravb_main.c +++ b/drivers/net/ethernet/renesas/ravb_main.c @@ -2667,6 +2667,7 @@ static const struct ravb_hw_info gbeth_hw_info = { .rx_max_desc_use = 4080, .rx_desc_size = sizeof(struct ravb_rx_desc), .aligned_tx = 1, + .coalesce_irqs = 1, .tx_counters = 1, .carrier_counters = 1, .half_duplex = 1, @@ -2943,6 +2944,9 @@ static int ravb_probe(struct platform_device *pdev) if (info->nc_queues) netif_napi_add(ndev, &priv->napi[RAVB_NC], ravb_poll); + if (info->coalesce_irqs) + netdev_sw_irq_coalesce_default_on(ndev); + /* Network device register */ error = register_netdev(ndev); if (error) From patchwork Tue Jun 4 07:28:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Barker X-Patchwork-Id: 13684836 X-Patchwork-Delegate: kuba@kernel.org Received: from relmlie5.idc.renesas.com (relmlor1.renesas.com [210.160.252.171]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 35CB31411F4; Tue, 4 Jun 2024 07:29:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.160.252.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717486156; cv=none; b=PjL7kMTFZMNV/5QzndEVv673lwUOaenLphr5iQDejwfJqEOvxlnRaZltxRyfHnDGDEUHUhIqIKccHm5dPnivoL/ARLwGO+XNwrmsOA3HBfrGDd88/Clt2HEfQmj7aDub/VaU/4kb8/pULVkgYEeU4C7kblF95PtCNAi9kXkDRm0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717486156; c=relaxed/simple; bh=XuVlm0U5SW4Dli6mAr7+sut314HZYUNxyu/rQnq8fJA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DVb7jlGkBSRtVQ/EKufYsTmurjNqMslF0W9ZpnhU+Na0AmMHethGdlg6HrvjmLO6VKNWv/E0s1jFM9GSA8SmlZtsUtih0koUmEydB8sBt5QCWLe5nHfX7vCmsXWgC8+66eLw0OM5vA02RZxqVZK5Eamf4NBmbzGBstPHorrf7E0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bp.renesas.com; spf=pass smtp.mailfrom=bp.renesas.com; arc=none smtp.client-ip=210.160.252.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bp.renesas.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bp.renesas.com X-IronPort-AV: E=Sophos;i="6.08,213,1712588400"; d="scan'208";a="206651387" Received: from unknown (HELO relmlir6.idc.renesas.com) ([10.200.68.152]) by relmlie5.idc.renesas.com with ESMTP; 04 Jun 2024 16:29:12 +0900 Received: from renesas-deb12.mshome.net (unknown [10.226.92.239]) by relmlir6.idc.renesas.com (Postfix) with ESMTP id 36CA141FCFCA; Tue, 4 Jun 2024 16:29:06 +0900 (JST) From: Paul Barker To: Sergey Shtylyov , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , =?utf-8?q?Niklas_S=C3=B6derlund?= , Simon Horman Cc: Paul Barker , Biju Das , Claudiu Beznea , Yoshihiro Shimoda , netdev@vger.kernel.org, linux-renesas-soc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [net-next PATCH v5 6/7] net: ravb: Use NAPI threaded mode on 1-core CPUs with GbEth IP Date: Tue, 4 Jun 2024 08:28:24 +0100 Message-Id: <20240604072825.7490-7-paul.barker.ct@bp.renesas.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240604072825.7490-1-paul.barker.ct@bp.renesas.com> References: <20240604072825.7490-1-paul.barker.ct@bp.renesas.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org NAPI Threaded mode (along with the previously enabled SW IRQ Coalescing) is required to improve network stack performance for single core SoCs using the GbEth IP (currently the RZ/G2L SoC family and the RZ/G3S SoC). This patch gives the following improvements during testing with iperf3. * RZ/G2UL: * TCP TX: +32% bandwidth (638Mbps -> 841Mbps) * TXP RX: +8.8% bandwidth (667Mbps -> 726Mbps) * UDP RX: +104% bandwidth (53Mbps -> 108Mbps) * RZ/G3S: * TCP TX: 29% bandwidth (529Mbps -> 681Mbps) * UDP RX: +1290% bandwidth (6.46Mbps -> 90Mbps) * RZ/Five: * UDP RX: Test no longer crashes (0 -> 20 Mbps) This patch gives the following reductions in performance in the same testing: * RZ/G2UL: * UDP TX: -7.5% bandwidth (594Mbps -> 549Mbps) * RZ/G3S: * UDP TX: -5% bandwidth (625Mbps -> 594Mbps) These losses are considered acceptable given the benefits shown above. If UDP TX bandwidth must be maximised for a particular use case, NAPI threaded mode can be disabled at runtime via sysfs writes. The improvement of UDP RX bandwidth for the single core SoCs (RZ/G2UL & RZ/G3S) is particularly critical. Signed-off-by: Paul Barker Reviewed-by: Sergey Shtylyov --- No changes v4->v5 Changes v3->v4: * Added Sergey's Reviewed-by tag. drivers/net/ethernet/renesas/ravb_main.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c index 2c24cb0271af..c89bb38492b4 100644 --- a/drivers/net/ethernet/renesas/ravb_main.c +++ b/drivers/net/ethernet/renesas/ravb_main.c @@ -2944,8 +2944,11 @@ static int ravb_probe(struct platform_device *pdev) if (info->nc_queues) netif_napi_add(ndev, &priv->napi[RAVB_NC], ravb_poll); - if (info->coalesce_irqs) + if (info->coalesce_irqs) { netdev_sw_irq_coalesce_default_on(ndev); + if (num_present_cpus() == 1) + dev_set_threaded(ndev, true); + } /* Network device register */ error = register_netdev(ndev); From patchwork Tue Jun 4 07:28:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Barker X-Patchwork-Id: 13684837 X-Patchwork-Delegate: kuba@kernel.org Received: from relmlie5.idc.renesas.com (relmlor1.renesas.com [210.160.252.171]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1E5BA14198E; Tue, 4 Jun 2024 07:29:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.160.252.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717486160; cv=none; b=QzJK64PnwF5RSzQm0lG71tK0712DsDBhjM97PibF5SmUH6y12/bZQ9ftdDBl7/fdkGZSDgn+BzHhP8SazcjNCX1cEtDreDzvaFfhRHdE2qDJiunPysy8fWo32X0OOD5Me+yIZM+sotK5lssLMYkATKf5I5icxE0V38Bt/+QiX5s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717486160; c=relaxed/simple; bh=UILWbnWLJi8QKNzh4xrmQEEB/svoX2/buW0DpPwS4hc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DR0WbDUIAAKRBiVNF50kmjtrkVbd4V2BRGZfUljccsg2e5BpVFC2hJNKY36tLL3zaxqL/vlV0JuG73Hs9cLip1eaF6IPlFzDHIiMSWGFkBsCe2KP0OSmifYXQw2ZMwVC9B68cJrt8n6AZ5chtn+DQlYFe3KldZU1OWRnazBMBDY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bp.renesas.com; spf=pass smtp.mailfrom=bp.renesas.com; arc=none smtp.client-ip=210.160.252.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bp.renesas.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bp.renesas.com X-IronPort-AV: E=Sophos;i="6.08,213,1712588400"; d="scan'208";a="206651400" Received: from unknown (HELO relmlir6.idc.renesas.com) ([10.200.68.152]) by relmlie5.idc.renesas.com with ESMTP; 04 Jun 2024 16:29:17 +0900 Received: from renesas-deb12.mshome.net (unknown [10.226.92.239]) by relmlir6.idc.renesas.com (Postfix) with ESMTP id 976A241FCFCA; Tue, 4 Jun 2024 16:29:12 +0900 (JST) From: Paul Barker To: Sergey Shtylyov , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , =?utf-8?q?Niklas_S=C3=B6derlund?= , Simon Horman Cc: Paul Barker , Biju Das , Claudiu Beznea , Yoshihiro Shimoda , netdev@vger.kernel.org, linux-renesas-soc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [net-next PATCH v5 7/7] net: ravb: Allocate RX buffers via page pool Date: Tue, 4 Jun 2024 08:28:25 +0100 Message-Id: <20240604072825.7490-8-paul.barker.ct@bp.renesas.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240604072825.7490-1-paul.barker.ct@bp.renesas.com> References: <20240604072825.7490-1-paul.barker.ct@bp.renesas.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org This patch makes multiple changes that can't be separated: 1) Allocate plain RX buffers via a page pool instead of allocating SKBs, then use build_skb() when a packet is received. 2) For GbEth IP, reduce the RX buffer size to 2kB. 3) For GbEth IP, merge packets which span more than one RX descriptor as SKB fragments instead of copying data. Implementing (1) without (2) would require the use of an order-1 page pool (instead of an order-0 page pool split into page fragments) for GbEth. Implementing (2) without (3) would leave us no space to re-assemble packets which span more than one RX descriptor. Implementing (3) without (1) would not be possible as the network stack expects to use put_page() or page_pool_put_page() to free SKB fragments after an SKB is consumed. RX checksum offload support is adjusted to handle both linear and nonlinear (fragmented) packets. This patch gives the following improvements during testing with iperf3. * RZ/G2L: * TCP RX: same bandwidth at -43% CPU load (70% -> 40%) * UDP RX: same bandwidth at -17% CPU load (88% -> 74%) * RZ/G2UL: * TCP RX: +30% bandwidth (726Mbps -> 941Mbps) * UDP RX: +417% bandwidth (108Mbps -> 558Mbps) * RZ/G3S: * TCP RX: +64% bandwidth (562Mbps -> 920Mbps) * UDP RX: +420% bandwidth (90Mbps -> 468Mbps) * RZ/Five: * TCP RX: +217% bandwidth (145Mbps -> 459Mbps) * UDP RX: +470% bandwidth (20Mbps -> 114Mbps) There is no significant impact on bandwidth or CPU load in testing on RZ/G2H or R-Car M3N. Signed-off-by: Paul Barker Reviewed-by: Sergey Shtylyov --- Changes v4->v5: * Re-wrapped to 80 cols, except where this would significantly impact readability. * Included in ravb.h. * Moved rx_buffer_size so it is in the same place in ravb_hw_info as rx_max_desc_use was previously. * Used reverse xmas tree ordering in variable declarations. * Split lines after binary operators, instead of before. * Factor subtraction of sizeof(__sum16) out of the if condition in ravb_rx_csum_gbeth(). * Add blank lines after variable declarations where needed. * Used goto instead of break to handle napi_build_skb() failure in ravb_rx_gbeth(). Break was incorrectly scoped to the surrounding switch statement, when it's the outer loop we really want to break out of. * Used continue instead of break to handle NULL priv->rx_1st_skb in ravb_rx_gbeth() as we may still be able to process further descriptors. * Unconditionally set priv->rx_1st_skb = NULL after processing a packet in ravb_rx_gbeth(). We don't need to check die_dt as this will be a no-op for single descriptor packets. * Moved napi_build_skb() call after dma_sync_single_for_cpu() in ravb_rx_rcar() to align the order of operations with ravb_rx_gbeth() and ensure the data is sync'd before it is accessed. * Moved zeroing of rx_buff->page to the end of packet processing in ravb_rx_rcar() to align the order of operations with ravb_rx_gbeth(). Changes v3->v4: * Used a separate page pool for each RX queue. * Passed struct ravb_rx_desc to ravb_alloc_rx_buffer() so that we can simplify the calling function. * Explained the calculation of rx_desc->ds_cc. * Added handling of nonlinear SKBs in ravb_rx_csum_gbeth(). drivers/net/ethernet/renesas/ravb.h | 11 +- drivers/net/ethernet/renesas/ravb_main.c | 258 +++++++++++++++-------- 2 files changed, 174 insertions(+), 95 deletions(-) diff --git a/drivers/net/ethernet/renesas/ravb.h b/drivers/net/ethernet/renesas/ravb.h index 6a7aa7dd17e6..6b2444d31fcc 100644 --- a/drivers/net/ethernet/renesas/ravb.h +++ b/drivers/net/ethernet/renesas/ravb.h @@ -19,6 +19,7 @@ #include #include #include +#include #define BE_TX_RING_SIZE 64 /* TX ring size for Best Effort */ #define BE_RX_RING_SIZE 1024 /* RX ring size for Best Effort */ @@ -1051,7 +1052,7 @@ struct ravb_hw_info { int stats_len; u32 tccr_mask; u32 rx_max_frame_size; - u32 rx_max_desc_use; + u32 rx_buffer_size; u32 rx_desc_size; unsigned aligned_tx: 1; unsigned coalesce_irqs:1; /* Needs software IRQ coalescing */ @@ -1071,6 +1072,11 @@ struct ravb_hw_info { unsigned half_duplex:1; /* E-MAC supports half duplex mode */ }; +struct ravb_rx_buffer { + struct page *page; + unsigned int offset; +}; + struct ravb_private { struct net_device *ndev; struct platform_device *pdev; @@ -1094,7 +1100,8 @@ struct ravb_private { struct ravb_tx_desc *tx_ring[NUM_TX_QUEUE]; void *tx_align[NUM_TX_QUEUE]; struct sk_buff *rx_1st_skb; - struct sk_buff **rx_skb[NUM_RX_QUEUE]; + struct page_pool *rx_pool[NUM_RX_QUEUE]; + struct ravb_rx_buffer *rx_buffers[NUM_RX_QUEUE]; struct sk_buff **tx_skb[NUM_TX_QUEUE]; u32 rx_over_errors; u32 rx_fifo_errors; diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c index c89bb38492b4..c1546b916e4e 100644 --- a/drivers/net/ethernet/renesas/ravb_main.c +++ b/drivers/net/ethernet/renesas/ravb_main.c @@ -30,6 +30,7 @@ #include #include #include +#include #include "ravb.h" @@ -113,25 +114,6 @@ static void ravb_set_rate_rcar(struct net_device *ndev) } } -static struct sk_buff * -ravb_alloc_skb(struct net_device *ndev, const struct ravb_hw_info *info, - gfp_t gfp_mask) -{ - struct sk_buff *skb; - u32 reserve; - - skb = __netdev_alloc_skb(ndev, info->rx_max_frame_size + RAVB_ALIGN - 1, - gfp_mask); - if (!skb) - return NULL; - - reserve = (unsigned long)skb->data & (RAVB_ALIGN - 1); - if (reserve) - skb_reserve(skb, RAVB_ALIGN - reserve); - - return skb; -} - /* Get MAC address from the MAC address registers * * Ethernet AVB device doesn't have ROM for MAC address. @@ -257,21 +239,10 @@ static void ravb_rx_ring_free(struct net_device *ndev, int q) { struct ravb_private *priv = netdev_priv(ndev); unsigned int ring_size; - unsigned int i; if (!priv->rx_ring[q].raw) return; - for (i = 0; i < priv->num_rx_ring[q]; i++) { - struct ravb_rx_desc *desc = ravb_rx_get_desc(priv, q, i); - - if (!dma_mapping_error(ndev->dev.parent, - le32_to_cpu(desc->dptr))) - dma_unmap_single(ndev->dev.parent, - le32_to_cpu(desc->dptr), - priv->info->rx_max_frame_size, - DMA_FROM_DEVICE); - } ring_size = priv->info->rx_desc_size * (priv->num_rx_ring[q] + 1); dma_free_coherent(ndev->dev.parent, ring_size, priv->rx_ring[q].raw, priv->rx_desc_dma[q]); @@ -298,13 +269,16 @@ static void ravb_ring_free(struct net_device *ndev, int q) priv->tx_ring[q] = NULL; } - /* Free RX skb ringbuffer */ - if (priv->rx_skb[q]) { - for (i = 0; i < priv->num_rx_ring[q]; i++) - dev_kfree_skb(priv->rx_skb[q][i]); + /* Free RX buffers */ + for (i = 0; i < priv->num_rx_ring[q]; i++) { + if (priv->rx_buffers[q][i].page) + page_pool_put_page(priv->rx_pool[q], + priv->rx_buffers[q][i].page, + 0, true); } - kfree(priv->rx_skb[q]); - priv->rx_skb[q] = NULL; + kfree(priv->rx_buffers[q]); + priv->rx_buffers[q] = NULL; + page_pool_destroy(priv->rx_pool[q]); /* Free aligned TX buffers */ kfree(priv->tx_align[q]); @@ -317,36 +291,57 @@ static void ravb_ring_free(struct net_device *ndev, int q) priv->tx_skb[q] = NULL; } +static int +ravb_alloc_rx_buffer(struct net_device *ndev, int q, u32 entry, gfp_t gfp_mask, + struct ravb_rx_desc *rx_desc) +{ + struct ravb_private *priv = netdev_priv(ndev); + const struct ravb_hw_info *info = priv->info; + struct ravb_rx_buffer *rx_buff; + dma_addr_t dma_addr; + unsigned int size; + + rx_buff = &priv->rx_buffers[q][entry]; + size = info->rx_buffer_size; + rx_buff->page = page_pool_alloc(priv->rx_pool[q], &rx_buff->offset, + &size, gfp_mask); + if (unlikely(!rx_buff->page)) { + /* We just set the data size to 0 for a failed mapping which + * should prevent DMA from happening... + */ + rx_desc->ds_cc = cpu_to_le16(0); + return -ENOMEM; + } + + dma_addr = page_pool_get_dma_addr(rx_buff->page) + rx_buff->offset; + dma_sync_single_for_device(ndev->dev.parent, dma_addr, + info->rx_buffer_size, DMA_FROM_DEVICE); + rx_desc->dptr = cpu_to_le32(dma_addr); + + /* The end of the RX buffer is used to store skb shared data, so we need + * to ensure that the hardware leaves enough space for this. + */ + rx_desc->ds_cc = cpu_to_le16(info->rx_buffer_size - + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) - + ETH_FCS_LEN + sizeof(__sum16)); + return 0; +} + static u32 ravb_rx_ring_refill(struct net_device *ndev, int q, u32 count, gfp_t gfp_mask) { struct ravb_private *priv = netdev_priv(ndev); - const struct ravb_hw_info *info = priv->info; struct ravb_rx_desc *rx_desc; - dma_addr_t dma_addr; u32 i, entry; for (i = 0; i < count; i++) { entry = (priv->dirty_rx[q] + i) % priv->num_rx_ring[q]; rx_desc = ravb_rx_get_desc(priv, q, entry); - rx_desc->ds_cc = cpu_to_le16(info->rx_max_desc_use); - if (!priv->rx_skb[q][entry]) { - priv->rx_skb[q][entry] = ravb_alloc_skb(ndev, info, - gfp_mask); - if (!priv->rx_skb[q][entry]) + if (!priv->rx_buffers[q][entry].page) { + if (unlikely(ravb_alloc_rx_buffer(ndev, q, entry, + gfp_mask, rx_desc))) break; - dma_addr = dma_map_single(ndev->dev.parent, - priv->rx_skb[q][entry]->data, - priv->info->rx_max_frame_size, - DMA_FROM_DEVICE); - skb_checksum_none_assert(priv->rx_skb[q][entry]); - /* We just set the data size to 0 for a failed mapping - * which should prevent DMA from happening... - */ - if (dma_mapping_error(ndev->dev.parent, dma_addr)) - rx_desc->ds_cc = cpu_to_le16(0); - rx_desc->dptr = cpu_to_le32(dma_addr); } /* Descriptor type must be set after all the above writes */ dma_wmb(); @@ -424,15 +419,32 @@ static int ravb_ring_init(struct net_device *ndev, int q) { struct ravb_private *priv = netdev_priv(ndev); unsigned int num_tx_desc = priv->num_tx_desc; + struct page_pool_params params = { + .order = 0, + .flags = PP_FLAG_DMA_MAP, + .pool_size = priv->num_rx_ring[q], + .nid = NUMA_NO_NODE, + .dev = ndev->dev.parent, + .dma_dir = DMA_FROM_DEVICE, + }; unsigned int ring_size; u32 num_filled; - /* Allocate RX and TX skb rings */ - priv->rx_skb[q] = kcalloc(priv->num_rx_ring[q], - sizeof(*priv->rx_skb[q]), GFP_KERNEL); + /* Allocate RX page pool and buffers */ + priv->rx_pool[q] = page_pool_create(¶ms); + if (IS_ERR(priv->rx_pool[q])) + goto error; + + /* Allocate RX buffers */ + priv->rx_buffers[q] = kcalloc(priv->num_rx_ring[q], + sizeof(*priv->rx_buffers[q]), GFP_KERNEL); + if (!priv->rx_buffers[q]) + goto error; + + /* Allocate TX skb rings */ priv->tx_skb[q] = kcalloc(priv->num_tx_ring[q], sizeof(*priv->tx_skb[q]), GFP_KERNEL); - if (!priv->rx_skb[q] || !priv->tx_skb[q]) + if (!priv->tx_skb[q]) goto error; /* Allocate all RX descriptors. */ @@ -719,7 +731,9 @@ static void ravb_get_tx_tstamp(struct net_device *ndev) static void ravb_rx_csum_gbeth(struct sk_buff *skb) { + struct skb_shared_info *shinfo = skb_shinfo(skb); __wsum csum_ip_hdr, csum_proto; + skb_frag_t *last_frag; u8 *hw_csum; /* The hardware checksum status is contained in sizeof(__sum16) * 2 = 4 @@ -729,12 +743,24 @@ static void ravb_rx_csum_gbeth(struct sk_buff *skb) if (unlikely(skb->len < sizeof(__sum16) * 2)) return; - hw_csum = skb_tail_pointer(skb) - sizeof(__sum16); + if (skb_is_nonlinear(skb)) { + last_frag = &shinfo->frags[shinfo->nr_frags - 1]; + hw_csum = skb_frag_address(last_frag) + + skb_frag_size(last_frag); + } else { + hw_csum = skb_tail_pointer(skb); + } + + hw_csum -= sizeof(__sum16); csum_proto = csum_unfold((__force __sum16)get_unaligned_le16(hw_csum)); hw_csum -= sizeof(__sum16); csum_ip_hdr = csum_unfold((__force __sum16)get_unaligned_le16(hw_csum)); - skb_trim(skb, skb->len - 2 * sizeof(__sum16)); + + if (skb_is_nonlinear(skb)) + skb_frag_size_sub(last_frag, 2 * sizeof(__sum16)); + else + skb_trim(skb, skb->len - 2 * sizeof(__sum16)); /* TODO: IPV6 Rx checksum */ if (skb->protocol == htons(ETH_P_IP) && !csum_ip_hdr && !csum_proto) @@ -756,25 +782,11 @@ static void ravb_rx_csum(struct sk_buff *skb) skb_trim(skb, skb->len - sizeof(__sum16)); } -static struct sk_buff *ravb_get_skb_gbeth(struct net_device *ndev, int entry, - struct ravb_rx_desc *desc) -{ - struct ravb_private *priv = netdev_priv(ndev); - struct sk_buff *skb; - - skb = priv->rx_skb[RAVB_BE][entry]; - priv->rx_skb[RAVB_BE][entry] = NULL; - dma_unmap_single(ndev->dev.parent, le32_to_cpu(desc->dptr), - ALIGN(priv->info->rx_max_frame_size, 16), - DMA_FROM_DEVICE); - - return skb; -} - /* Packet receive function for Gigabit Ethernet */ static int ravb_rx_gbeth(struct net_device *ndev, int budget, int q) { struct ravb_private *priv = netdev_priv(ndev); + const struct ravb_hw_info *info = priv->info; struct net_device_stats *stats; struct ravb_rx_desc *desc; struct sk_buff *skb; @@ -818,12 +830,30 @@ static int ravb_rx_gbeth(struct net_device *ndev, int budget, int q) if (desc_status & MSC_CEEF) stats->rx_missed_errors++; } else { + struct ravb_rx_buffer *rx_buff; + void *rx_addr; + + rx_buff = &priv->rx_buffers[q][entry]; + rx_addr = page_address(rx_buff->page) + rx_buff->offset; die_dt = desc->die_dt & 0xF0; - skb = ravb_get_skb_gbeth(ndev, entry, desc); + dma_sync_single_for_cpu(ndev->dev.parent, + le32_to_cpu(desc->dptr), + desc_len, DMA_FROM_DEVICE); + switch (die_dt) { case DT_FSINGLE: case DT_FSTART: /* Start of packet: Set initial data length. */ + skb = napi_build_skb(rx_addr, + info->rx_buffer_size); + if (unlikely(!skb)) { + stats->rx_errors++; + page_pool_put_page(priv->rx_pool[q], + rx_buff->page, 0, + true); + goto refill; + } + skb_mark_for_recycle(skb); skb_put(skb, desc_len); /* Save this skb if the packet spans multiple @@ -835,15 +865,30 @@ static int ravb_rx_gbeth(struct net_device *ndev, int budget, int q) case DT_FMID: case DT_FEND: - /* Continuing a packet: Move data into the saved - * skb. + /* Continuing a packet: Add this buffer as an RX + * frag. */ - skb_copy_to_linear_data_offset(priv->rx_1st_skb, - priv->rx_1st_skb->len, - skb->data, - desc_len); - skb_put(priv->rx_1st_skb, desc_len); - dev_kfree_skb(skb); + + /* rx_1st_skb will be NULL if napi_build_skb() + * failed for the first descriptor of a + * multi-descriptor packet. + */ + if (unlikely(!priv->rx_1st_skb)) { + stats->rx_errors++; + page_pool_put_page(priv->rx_pool[q], + rx_buff->page, 0, + true); + + /* We may find a DT_FSINGLE or DT_FSTART + * descriptor in the queue which we can + * process, so don't give up yet. + */ + continue; + } + skb_add_rx_frag(priv->rx_1st_skb, + skb_shinfo(priv->rx_1st_skb)->nr_frags, + rx_buff->page, rx_buff->offset, + desc_len, info->rx_buffer_size); /* Set skb to point at the whole packet so that * we only need one code path for finishing a @@ -865,10 +910,19 @@ static int ravb_rx_gbeth(struct net_device *ndev, int budget, int q) stats->rx_bytes += skb->len; napi_gro_receive(&priv->napi[q], skb); rx_packets++; + + /* Clear rx_1st_skb so that it will only be + * non-NULL when valid. + */ + priv->rx_1st_skb = NULL; } + + /* Mark this RX buffer as consumed. */ + rx_buff->page = NULL; } } +refill: /* Refill the RX ring buffers. */ priv->dirty_rx[q] += ravb_rx_ring_refill(ndev, q, priv->cur_rx[q] - priv->dirty_rx[q], @@ -882,6 +936,7 @@ static int ravb_rx_gbeth(struct net_device *ndev, int budget, int q) static int ravb_rx_rcar(struct net_device *ndev, int budget, int q) { struct ravb_private *priv = netdev_priv(ndev); + const struct ravb_hw_info *info = priv->info; struct net_device_stats *stats = &priv->stats[q]; struct ravb_ex_rx_desc *desc; unsigned int limit, i; @@ -924,12 +979,23 @@ static int ravb_rx_rcar(struct net_device *ndev, int budget, int q) stats->rx_missed_errors++; } else { u32 get_ts = priv->tstamp_rx_ctrl & RAVB_RXTSTAMP_TYPE; + struct ravb_rx_buffer *rx_buff; + void *rx_addr; - skb = priv->rx_skb[q][entry]; - priv->rx_skb[q][entry] = NULL; - dma_unmap_single(ndev->dev.parent, le32_to_cpu(desc->dptr), - priv->info->rx_max_frame_size, - DMA_FROM_DEVICE); + rx_buff = &priv->rx_buffers[q][entry]; + rx_addr = page_address(rx_buff->page) + rx_buff->offset; + dma_sync_single_for_cpu(ndev->dev.parent, + le32_to_cpu(desc->dptr), + pkt_len, DMA_FROM_DEVICE); + + skb = napi_build_skb(rx_addr, info->rx_buffer_size); + if (unlikely(!skb)) { + stats->rx_errors++; + page_pool_put_page(priv->rx_pool[q], + rx_buff->page, 0, true); + break; + } + skb_mark_for_recycle(skb); get_ts &= (q == RAVB_NC) ? RAVB_RXTSTAMP_TYPE_V2_L2_EVENT : ~RAVB_RXTSTAMP_TYPE_V2_L2_EVENT; @@ -951,6 +1017,9 @@ static int ravb_rx_rcar(struct net_device *ndev, int budget, int q) napi_gro_receive(&priv->napi[q], skb); rx_packets++; stats->rx_bytes += pkt_len; + + /* Mark this RX buffer as consumed. */ + rx_buff->page = NULL; } } @@ -2596,7 +2665,8 @@ static const struct ravb_hw_info ravb_gen3_hw_info = { .stats_len = ARRAY_SIZE(ravb_gstrings_stats), .tccr_mask = TCCR_TSRQ0 | TCCR_TSRQ1 | TCCR_TSRQ2 | TCCR_TSRQ3, .rx_max_frame_size = SZ_2K, - .rx_max_desc_use = SZ_2K - ETH_FCS_LEN + sizeof(__sum16), + .rx_buffer_size = SZ_2K + + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)), .rx_desc_size = sizeof(struct ravb_ex_rx_desc), .internal_delay = 1, .tx_counters = 1, @@ -2620,7 +2690,8 @@ static const struct ravb_hw_info ravb_gen2_hw_info = { .stats_len = ARRAY_SIZE(ravb_gstrings_stats), .tccr_mask = TCCR_TSRQ0 | TCCR_TSRQ1 | TCCR_TSRQ2 | TCCR_TSRQ3, .rx_max_frame_size = SZ_2K, - .rx_max_desc_use = SZ_2K - ETH_FCS_LEN + sizeof(__sum16), + .rx_buffer_size = SZ_2K + + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)), .rx_desc_size = sizeof(struct ravb_ex_rx_desc), .aligned_tx = 1, .gptp = 1, @@ -2641,7 +2712,8 @@ static const struct ravb_hw_info ravb_rzv2m_hw_info = { .stats_len = ARRAY_SIZE(ravb_gstrings_stats), .tccr_mask = TCCR_TSRQ0 | TCCR_TSRQ1 | TCCR_TSRQ2 | TCCR_TSRQ3, .rx_max_frame_size = SZ_2K, - .rx_max_desc_use = SZ_2K - ETH_FCS_LEN + sizeof(__sum16), + .rx_buffer_size = SZ_2K + + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)), .rx_desc_size = sizeof(struct ravb_ex_rx_desc), .multi_irqs = 1, .err_mgmt_irqs = 1, @@ -2664,7 +2736,7 @@ static const struct ravb_hw_info gbeth_hw_info = { .stats_len = ARRAY_SIZE(ravb_gstrings_stats_gbeth), .tccr_mask = TCCR_TSRQ0, .rx_max_frame_size = SZ_8K, - .rx_max_desc_use = 4080, + .rx_buffer_size = SZ_2K, .rx_desc_size = sizeof(struct ravb_rx_desc), .aligned_tx = 1, .coalesce_irqs = 1,