From patchwork Thu Oct 3 16:06:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 13821212 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 582E319C56A; Thu, 3 Oct 2024 16:06:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727971605; cv=none; b=BRBGYyXZZnWPHc7cac9NSRwonqN4IWzWyHODR174vdgBSPUXbxRC/VVO2CZOayBaSTbBavHOLX1k/Oi5RZsHr2lJAZETu+a8LwAQL+D52i/5knp0HP7ONBWGGL/Oe6z8xLW1ipxfMmdwEaNKq6JBec0K22habYFrRsIHl4nWs/E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727971605; c=relaxed/simple; bh=LROGOsI6l21g70Z0CUMCYBUUYkcGifM2fot8zgS7BJQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=b/ac4Z6gMyLtfQstrGNh6Fl3hHRQ2wPR+NMZbasifwnLVw3kUfDRQJYD2UsemMeTG9ylqTLIwmVYo4Pf7tCGEBmamokE00mjCztEY6H/zP3e3t5TLflVB/bVaTQgPVj4bh3j4ZSf+fgaj2ZG7wG/i6A/8mexF8f3NTX8E4LxF/Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MwdwTVId; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MwdwTVId" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-20b01da232aso9427255ad.1; Thu, 03 Oct 2024 09:06:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727971603; x=1728576403; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=x7YWexRb2GNKLIhxJSHSvpF2GBCzOZF2JoATS6OF5uQ=; b=MwdwTVIdA0XcIs5VfeA1UBWVA7UfPqayso7vVw3c3UDoOGGrTLKwcOVkIZEhNpTr7I RcGNk0OdCRNJlhM+LcL60SH74hOBA8bc+tzUst9BnyGVZELBbStYIImELAG1Epj6zsQY 2SL6ebLbcgeXDS2o1jrMgYJd2kBLdCcf/u1Apu+mrzhjP/0dwvWYadtedeSA8nFIIdV6 HpoMQlK/FpPzpWn+B4I3Sldmt1e9cuadKYiRLVp/sopqXDcaimk7+goF2qUuapgI0agd YXIvoVRAzhT1wqlgehTBsrhMLt/NAG0vVMkklhFeK4H1LIwDkyv8D3bgCI8HKVhTOo/9 qZAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727971603; x=1728576403; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=x7YWexRb2GNKLIhxJSHSvpF2GBCzOZF2JoATS6OF5uQ=; b=IOFJwEPOwoO/hDlEpmktuDHp5KSRtBpYPWfOqy8hOCo2cLVUIsaRahgYdmf5lLC82V mJpIhXacpkId79uTIxBTmJdN/W4tEM+VjfpQJPZZoZql3T0sQmVbPrrQJcCFm3m2ZEA1 /5WwlHZLSt45wWClYaSCOIR3jq3V39D5rkWdrC/0Vr8Xt8XBG/yF9Xh+GewQ6LQVtBds m++qX4/o+y7nWONZPhDHX6O0y4v8djADX9m/r1hLk7qI11lLF/WuRpe1oc5Uos8OEdgE o984nODGOSyAG0EQ22iFDshlte5TDNsL7hbYf0Q2ikDllSaq66RM98LlSfM63ikV6Bcq 4HKA== X-Forwarded-Encrypted: i=1; AJvYcCUATMUVXJp/VTVTCqY2tz+8mlp9wn/eu5ct9fQen2Y9jb5ypyJnaCFYAQAN4MD3ZcYowXlmrBOjG98=@vger.kernel.org, AJvYcCUeC10HYJqITfVeWo/wSGSxIaBv+4/H6TCLlFM9xy7RP9IW5FY6D854V/taeHdV+94Xj1RCY/0D@vger.kernel.org X-Gm-Message-State: AOJu0YwSOqi23XffZfUGjVHUr/Zx+jayvCd0iQRtInVSQsVx6r5XbIY5 9CoVdcw3QAryAovw5JBzPdsSc033VkOz4QeXLVIilYV9PFfIl3BT X-Google-Smtp-Source: AGHT+IFer07+3UYCIIJll9yJkjxLQ2LGQN6AlFzvSwcHyI0C7bBaIbAWppXX6ST+hWdOUU5tT3Mk/A== X-Received: by 2002:a17:902:d2ca:b0:206:9dfb:3e9e with SMTP id d9443c01a7336-20be189892dmr40712985ad.10.1727971603186; Thu, 03 Oct 2024 09:06:43 -0700 (PDT) Received: from ap.. ([182.213.254.91]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20bef7071f1sm10425435ad.292.2024.10.03.09.06.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Oct 2024 09:06:42 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, almasrymina@google.com, netdev@vger.kernel.org, linux-doc@vger.kernel.org, donald.hunter@gmail.com, corbet@lwn.net, michael.chan@broadcom.com Cc: kory.maincent@bootlin.com, andrew@lunn.ch, maxime.chevallier@bootlin.com, danieller@nvidia.com, hengqi@linux.alibaba.com, ecree.xilinx@gmail.com, przemyslaw.kitszel@intel.com, hkallweit1@gmail.com, ahmed.zaki@intel.com, paul.greenwalt@intel.com, rrameshbabu@nvidia.com, idosch@nvidia.com, asml.silence@gmail.com, kaiyuanz@google.com, willemb@google.com, aleksander.lobakin@intel.com, dw@davidwei.uk, sridhar.samudrala@intel.com, bcreeley@amd.com, ap420073@gmail.com Subject: [PATCH net-next v3 1/7] bnxt_en: add support for rx-copybreak ethtool command Date: Thu, 3 Oct 2024 16:06:14 +0000 Message-Id: <20241003160620.1521626-2-ap420073@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241003160620.1521626-1-ap420073@gmail.com> References: <20241003160620.1521626-1-ap420073@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org The bnxt_en driver supports rx-copybreak, but it couldn't be set by userspace. Only the default value(256) has worked. This patch makes the bnxt_en driver support following command. `ethtool --set-tunable rx-copybreak ` and `ethtool --get-tunable rx-copybreak`. Signed-off-by: Taehee Yoo Reviewed-by: Brett Creeley --- v3: - Update copybreak value before closing nic. v2: - Define max/vim rx_copybreak value. drivers/net/ethernet/broadcom/bnxt/bnxt.c | 24 +++++---- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 6 ++- .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 49 ++++++++++++++++++- 3 files changed, 68 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 6e422e24750a..8da211e083a4 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -81,7 +81,6 @@ MODULE_DESCRIPTION("Broadcom NetXtreme network driver"); #define BNXT_RX_OFFSET (NET_SKB_PAD + NET_IP_ALIGN) #define BNXT_RX_DMA_OFFSET NET_SKB_PAD -#define BNXT_RX_COPY_THRESH 256 #define BNXT_TX_PUSH_THRESH 164 @@ -1330,13 +1329,13 @@ static struct sk_buff *bnxt_copy_data(struct bnxt_napi *bnapi, u8 *data, if (!skb) return NULL; - dma_sync_single_for_cpu(&pdev->dev, mapping, bp->rx_copy_thresh, + dma_sync_single_for_cpu(&pdev->dev, mapping, bp->rx_copybreak, bp->rx_dir); memcpy(skb->data - NET_IP_ALIGN, data - NET_IP_ALIGN, len + NET_IP_ALIGN); - dma_sync_single_for_device(&pdev->dev, mapping, bp->rx_copy_thresh, + dma_sync_single_for_device(&pdev->dev, mapping, bp->rx_copybreak, bp->rx_dir); skb_put(skb, len); @@ -1829,7 +1828,7 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp, return NULL; } - if (len <= bp->rx_copy_thresh) { + if (len <= bp->rx_copybreak) { skb = bnxt_copy_skb(bnapi, data_ptr, len, mapping); if (!skb) { bnxt_abort_tpa(cpr, idx, agg_bufs); @@ -1931,6 +1930,7 @@ static void bnxt_deliver_skb(struct bnxt *bp, struct bnxt_napi *bnapi, bnxt_vf_rep_rx(bp, skb); return; } + skb_record_rx_queue(skb, bnapi->index); napi_gro_receive(&bnapi->napi, skb); } @@ -2162,7 +2162,7 @@ static int bnxt_rx_pkt(struct bnxt *bp, struct bnxt_cp_ring_info *cpr, } } - if (len <= bp->rx_copy_thresh) { + if (len <= bp->rx_copybreak) { if (!xdp_active) skb = bnxt_copy_skb(bnapi, data_ptr, len, dma_addr); else @@ -4451,6 +4451,11 @@ void bnxt_set_tpa_flags(struct bnxt *bp) bp->flags |= BNXT_FLAG_GRO; } +static void bnxt_init_ring_params(struct bnxt *bp) +{ + bp->rx_copybreak = BNXT_DEFAULT_RX_COPYBREAK; +} + /* bp->rx_ring_size, bp->tx_ring_size, dev->mtu, BNXT_FLAG_{G|L}RO flags must * be set on entry. */ @@ -4465,7 +4470,6 @@ void bnxt_set_ring_params(struct bnxt *bp) rx_space = rx_size + ALIGN(max(NET_SKB_PAD, XDP_PACKET_HEADROOM), 8) + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)); - bp->rx_copy_thresh = BNXT_RX_COPY_THRESH; ring_size = bp->rx_ring_size; bp->rx_agg_ring_size = 0; bp->rx_agg_nr_pages = 0; @@ -4510,7 +4514,8 @@ void bnxt_set_ring_params(struct bnxt *bp) ALIGN(max(NET_SKB_PAD, XDP_PACKET_HEADROOM), 8) - SKB_DATA_ALIGN(sizeof(struct skb_shared_info)); } else { - rx_size = SKB_DATA_ALIGN(BNXT_RX_COPY_THRESH + NET_IP_ALIGN); + rx_size = SKB_DATA_ALIGN(bp->rx_copybreak + + NET_IP_ALIGN); rx_space = rx_size + NET_SKB_PAD + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)); } @@ -6424,8 +6429,8 @@ static int bnxt_hwrm_vnic_set_hds(struct bnxt *bp, struct bnxt_vnic_info *vnic) VNIC_PLCMODES_CFG_REQ_FLAGS_HDS_IPV6); req->enables |= cpu_to_le32(VNIC_PLCMODES_CFG_REQ_ENABLES_HDS_THRESHOLD_VALID); - req->jumbo_thresh = cpu_to_le16(bp->rx_copy_thresh); - req->hds_threshold = cpu_to_le16(bp->rx_copy_thresh); + req->jumbo_thresh = cpu_to_le16(bp->rx_copybreak); + req->hds_threshold = cpu_to_le16(bp->rx_copybreak); } req->vnic_id = cpu_to_le32(vnic->fw_vnic_id); return hwrm_req_send(bp, req); @@ -15864,6 +15869,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) bnxt_init_l2_fltr_tbl(bp); bnxt_set_rx_skb_mode(bp, false); bnxt_set_tpa_flags(bp); + bnxt_init_ring_params(bp); bnxt_set_ring_params(bp); bnxt_rdma_aux_device_init(bp); rc = bnxt_set_dflt_rings(bp, true); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 69231e85140b..cff031993223 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -34,6 +34,10 @@ #include #endif +#define BNXT_DEFAULT_RX_COPYBREAK 256 +#define BNXT_MIN_RX_COPYBREAK 65 +#define BNXT_MAX_RX_COPYBREAK 1024 + extern struct list_head bnxt_block_cb_list; struct page_pool; @@ -2299,7 +2303,7 @@ struct bnxt { enum dma_data_direction rx_dir; u32 rx_ring_size; u32 rx_agg_ring_size; - u32 rx_copy_thresh; + u32 rx_copybreak; u32 rx_ring_mask; u32 rx_agg_ring_mask; int rx_nr_pages; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c index f71cc8188b4e..fdecdf8894b3 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c @@ -4319,6 +4319,51 @@ static int bnxt_get_eee(struct net_device *dev, struct ethtool_keee *edata) return 0; } +static int bnxt_set_tunable(struct net_device *dev, + const struct ethtool_tunable *tuna, + const void *data) +{ + struct bnxt *bp = netdev_priv(dev); + u32 rx_copybreak; + + switch (tuna->id) { + case ETHTOOL_RX_COPYBREAK: + rx_copybreak = *(u32 *)data; + if (rx_copybreak < BNXT_MIN_RX_COPYBREAK || + rx_copybreak > BNXT_MAX_RX_COPYBREAK) + return -EINVAL; + if (rx_copybreak != bp->rx_copybreak) { + if (netif_running(dev)) { + bnxt_close_nic(bp, false, false); + bp->rx_copybreak = rx_copybreak; + bnxt_set_ring_params(bp); + bnxt_open_nic(bp, false, false); + } else { + bp->rx_copybreak = rx_copybreak; + } + } + return 0; + default: + return -EOPNOTSUPP; + } +} + +static int bnxt_get_tunable(struct net_device *dev, + const struct ethtool_tunable *tuna, void *data) +{ + struct bnxt *bp = netdev_priv(dev); + + switch (tuna->id) { + case ETHTOOL_RX_COPYBREAK: + *(u32 *)data = bp->rx_copybreak; + break; + default: + return -EOPNOTSUPP; + } + + return 0; +} + static int bnxt_read_sfp_module_eeprom_info(struct bnxt *bp, u16 i2c_addr, u16 page_number, u8 bank, u16 start_addr, u16 data_length, @@ -4769,7 +4814,7 @@ static int bnxt_run_loopback(struct bnxt *bp) cpr = &rxr->bnapi->cp_ring; if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) cpr = rxr->rx_cpr; - pkt_size = min(bp->dev->mtu + ETH_HLEN, bp->rx_copy_thresh); + pkt_size = min(bp->dev->mtu + ETH_HLEN, bp->rx_copybreak); skb = netdev_alloc_skb(bp->dev, pkt_size); if (!skb) return -ENOMEM; @@ -5342,6 +5387,8 @@ const struct ethtool_ops bnxt_ethtool_ops = { .get_link_ext_stats = bnxt_get_link_ext_stats, .get_eee = bnxt_get_eee, .set_eee = bnxt_set_eee, + .get_tunable = bnxt_get_tunable, + .set_tunable = bnxt_set_tunable, .get_module_info = bnxt_get_module_info, .get_module_eeprom = bnxt_get_module_eeprom, .get_module_eeprom_by_page = bnxt_get_module_eeprom_by_page, From patchwork Thu Oct 3 16:06:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 13821213 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B44F1A4F30; Thu, 3 Oct 2024 16:06:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727971613; cv=none; b=dQLQ6hXfuttgCGochVNanIwEzzrxBzA64t7eLtLH5U1yD1nozg6aegQh1W7ad/Jic0Me/x7SnglyN2icE7KRBDsA4fWoD5DWe+AiMk2umUth3uWBBGKARKZqjiFQyQ5tcdYexbXlwArPQ6clTzQapEDA2NyV1z8T84IJXrJ2oqc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727971613; c=relaxed/simple; bh=jNSUloLXTW7XNo6ZIiysdZCvwrXEXTkS6KjsJozFq6g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YVIPpFxbrAqYJT6vCBaJaku3Og2WCIAx8LVJ+6adpPdqrm43rriSXENXikW3voHHLflG9HA8Gveo7eYmmEAELhugm+zhWjb9dKzFIdiktvf2bbz3YeRQlkZ84F5XYCFsD3its9P1SQL7n1nv8Mp3J1bURz3vR75Ir1R0GBuxEvI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=R41VqN47; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="R41VqN47" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-20b6458ee37so12282735ad.1; Thu, 03 Oct 2024 09:06:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727971611; x=1728576411; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CrdRZ6PxOosogiFJCfq5RDtTFnNKOZUdm2yfBEXM0OI=; b=R41VqN47CGXs8mlADR6fNG9GF9Q0wO5NwbdcCVIiprw10tKg/dDq0emFizMKH4Rxaf bzzUEwXsrYp96YJniY5jftTLDzEfxxXVFg2/OKlVXR437xmBz7qh6e51xd2zr94UCz/Q SfKAxJRG6h+D4nCbCZ54xH5E8iMp+NV+YwiUKQHBexwbSgqMauGXbpkcW30sdTFgIkgn ZUQvmTiIzPpJkP215eGjpEcLq1CFS32EfXfjBrXzhS5jDQI3/8zPLeAQyU/Sn8l7t0/I wMJMi92XyuQ5+juq5TAWqFvGYEghf4NQiUOAu+IZYC5SFrVDNRDYyQAVAk9elxoMwjRS jgxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727971611; x=1728576411; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CrdRZ6PxOosogiFJCfq5RDtTFnNKOZUdm2yfBEXM0OI=; b=GVwbk7bj2/IgA3RIB+ec3JoDQmgMhGrpIWX2t6/Fy94okirEEEpjV2fyFFoGZmSsU0 7Y1MCyFW9+aEJMV3hWI7wUrZaf0w/M4W+0qajKSTFXxqoZripC3omQcsyt6amr2ulvHY dU4fVYVZLlpP1JcCsRH4Phk8TeFrOJkZqJMT/ku5qQTq7R0Mn9Qb9zJVXLytxMI+ivd3 UwBiw1EcmqzMWWGaglRC/+05tQsHTqcdMHMPuV0N2K48xOaXKwp7iStsYxefxmnd1fQM biGmo/VBADRGAnER8zdiwA+PO1Oi1UK8DcFkVewkMzjyL4FeRjFs/p4wg6Roqo+H44sO vxyA== X-Forwarded-Encrypted: i=1; AJvYcCU/KML/4dgY1nh1ONHSvSbouaGnEg3zSZfUoIPIL5cJQFiUC//q+ruQMNgPRv+TcttNrt5jgjti@vger.kernel.org, AJvYcCW6rErzaln+pkurHjm/BiBIG6eyHs+mkSwy3r7tt/qq8Wj17Ak8a8vuXVdw9gsmidw5CxdoxL3/gnc=@vger.kernel.org X-Gm-Message-State: AOJu0Ywe2UoX80oCk0knH/YzDncNchNMHW3+TBctPNAjJLbVIeSZziec vQqZFV/dkhI055Dh9RQekpk2fSjH2zT64+cwDQCP49BZIL0kn8XO X-Google-Smtp-Source: AGHT+IGK8beT/Um9fHuskDtoU6uwH7Ywmp1rWC395Q/iU2v8bBZutSYeZe5TEQ1WyeHgB/tJLdoRNQ== X-Received: by 2002:a17:903:22d2:b0:20b:751f:c9ca with SMTP id d9443c01a7336-20bc59ae37emr105030175ad.5.1727971610624; Thu, 03 Oct 2024 09:06:50 -0700 (PDT) Received: from ap.. ([182.213.254.91]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20bef7071f1sm10425435ad.292.2024.10.03.09.06.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Oct 2024 09:06:49 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, almasrymina@google.com, netdev@vger.kernel.org, linux-doc@vger.kernel.org, donald.hunter@gmail.com, corbet@lwn.net, michael.chan@broadcom.com Cc: kory.maincent@bootlin.com, andrew@lunn.ch, maxime.chevallier@bootlin.com, danieller@nvidia.com, hengqi@linux.alibaba.com, ecree.xilinx@gmail.com, przemyslaw.kitszel@intel.com, hkallweit1@gmail.com, ahmed.zaki@intel.com, paul.greenwalt@intel.com, rrameshbabu@nvidia.com, idosch@nvidia.com, asml.silence@gmail.com, kaiyuanz@google.com, willemb@google.com, aleksander.lobakin@intel.com, dw@davidwei.uk, sridhar.samudrala@intel.com, bcreeley@amd.com, ap420073@gmail.com Subject: [PATCH net-next v3 2/7] bnxt_en: add support for tcp-data-split ethtool command Date: Thu, 3 Oct 2024 16:06:15 +0000 Message-Id: <20241003160620.1521626-3-ap420073@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241003160620.1521626-1-ap420073@gmail.com> References: <20241003160620.1521626-1-ap420073@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org NICs that uses bnxt_en driver supports tcp-data-split feature by the name of HDS(header-data-split). But there is no implementation for the HDS to enable or disable by ethtool. Only getting the current HDS status is implemented and The HDS is just automatically enabled only when either LRO, HW-GRO, or JUMBO is enabled. The hds_threshold follows rx-copybreak value. and it was unchangeable. This implements `ethtool -G tcp-data-split ` command option. The value can be , , and but the will be automatically changed to . HDS feature relies on the aggregation ring. So, if HDS is enabled, the bnxt_en driver initializes the aggregation ring. This is the reason why BNXT_FLAG_AGG_RINGS contains HDS condition. Signed-off-by: Taehee Yoo --- v3: - No changes. v2: - Do not set hds_threshold to 0. drivers/net/ethernet/broadcom/bnxt/bnxt.c | 9 +++---- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 5 ++-- .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 25 +++++++++++++++++-- 3 files changed, 30 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 8da211e083a4..f046478dfd2a 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -4454,6 +4454,7 @@ void bnxt_set_tpa_flags(struct bnxt *bp) static void bnxt_init_ring_params(struct bnxt *bp) { bp->rx_copybreak = BNXT_DEFAULT_RX_COPYBREAK; + bp->flags |= BNXT_FLAG_HDS; } /* bp->rx_ring_size, bp->tx_ring_size, dev->mtu, BNXT_FLAG_{G|L}RO flags must @@ -4474,7 +4475,7 @@ void bnxt_set_ring_params(struct bnxt *bp) bp->rx_agg_ring_size = 0; bp->rx_agg_nr_pages = 0; - if (bp->flags & BNXT_FLAG_TPA) + if (bp->flags & BNXT_FLAG_TPA || bp->flags & BNXT_FLAG_HDS) agg_factor = min_t(u32, 4, 65536 / BNXT_RX_PAGE_SIZE); bp->flags &= ~BNXT_FLAG_JUMBO; @@ -6421,15 +6422,13 @@ static int bnxt_hwrm_vnic_set_hds(struct bnxt *bp, struct bnxt_vnic_info *vnic) req->flags = cpu_to_le32(VNIC_PLCMODES_CFG_REQ_FLAGS_JUMBO_PLACEMENT); req->enables = cpu_to_le32(VNIC_PLCMODES_CFG_REQ_ENABLES_JUMBO_THRESH_VALID); + req->jumbo_thresh = cpu_to_le16(bp->rx_buf_use_size); - if (BNXT_RX_PAGE_MODE(bp)) { - req->jumbo_thresh = cpu_to_le16(bp->rx_buf_use_size); - } else { + if (bp->flags & BNXT_FLAG_HDS) { req->flags |= cpu_to_le32(VNIC_PLCMODES_CFG_REQ_FLAGS_HDS_IPV4 | VNIC_PLCMODES_CFG_REQ_FLAGS_HDS_IPV6); req->enables |= cpu_to_le32(VNIC_PLCMODES_CFG_REQ_ENABLES_HDS_THRESHOLD_VALID); - req->jumbo_thresh = cpu_to_le16(bp->rx_copybreak); req->hds_threshold = cpu_to_le16(bp->rx_copybreak); } req->vnic_id = cpu_to_le32(vnic->fw_vnic_id); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index cff031993223..35601c71dfe9 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -2202,8 +2202,6 @@ struct bnxt { #define BNXT_FLAG_TPA (BNXT_FLAG_LRO | BNXT_FLAG_GRO) #define BNXT_FLAG_JUMBO 0x10 #define BNXT_FLAG_STRIP_VLAN 0x20 - #define BNXT_FLAG_AGG_RINGS (BNXT_FLAG_JUMBO | BNXT_FLAG_GRO | \ - BNXT_FLAG_LRO) #define BNXT_FLAG_RFS 0x100 #define BNXT_FLAG_SHARED_RINGS 0x200 #define BNXT_FLAG_PORT_STATS 0x400 @@ -2224,6 +2222,9 @@ struct bnxt { #define BNXT_FLAG_ROCE_MIRROR_CAP 0x4000000 #define BNXT_FLAG_TX_COAL_CMPL 0x8000000 #define BNXT_FLAG_PORT_STATS_EXT 0x10000000 + #define BNXT_FLAG_HDS 0x20000000 + #define BNXT_FLAG_AGG_RINGS (BNXT_FLAG_JUMBO | BNXT_FLAG_GRO | \ + BNXT_FLAG_LRO | BNXT_FLAG_HDS) #define BNXT_FLAG_ALL_CONFIG_FEATS (BNXT_FLAG_TPA | \ BNXT_FLAG_RFS | \ diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c index fdecdf8894b3..e9ef65dd2e7b 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c @@ -829,12 +829,16 @@ static void bnxt_get_ringparam(struct net_device *dev, if (bp->flags & BNXT_FLAG_AGG_RINGS) { ering->rx_max_pending = BNXT_MAX_RX_DESC_CNT_JUM_ENA; ering->rx_jumbo_max_pending = BNXT_MAX_RX_JUM_DESC_CNT; - kernel_ering->tcp_data_split = ETHTOOL_TCP_DATA_SPLIT_ENABLED; } else { ering->rx_max_pending = BNXT_MAX_RX_DESC_CNT; ering->rx_jumbo_max_pending = 0; - kernel_ering->tcp_data_split = ETHTOOL_TCP_DATA_SPLIT_DISABLED; } + + if (bp->flags & BNXT_FLAG_HDS) + kernel_ering->tcp_data_split = ETHTOOL_TCP_DATA_SPLIT_ENABLED; + else + kernel_ering->tcp_data_split = ETHTOOL_TCP_DATA_SPLIT_DISABLED; + ering->tx_max_pending = BNXT_MAX_TX_DESC_CNT; ering->rx_pending = bp->rx_ring_size; @@ -854,9 +858,25 @@ static int bnxt_set_ringparam(struct net_device *dev, (ering->tx_pending < BNXT_MIN_TX_DESC_CNT)) return -EINVAL; + if (kernel_ering->tcp_data_split != ETHTOOL_TCP_DATA_SPLIT_DISABLED && + BNXT_RX_PAGE_MODE(bp)) { + NL_SET_ERR_MSG_MOD(extack, "tcp-data-split can not be enabled with XDP"); + return -EINVAL; + } + if (netif_running(dev)) bnxt_close_nic(bp, false, false); + switch (kernel_ering->tcp_data_split) { + case ETHTOOL_TCP_DATA_SPLIT_UNKNOWN: + case ETHTOOL_TCP_DATA_SPLIT_ENABLED: + bp->flags |= BNXT_FLAG_HDS; + break; + case ETHTOOL_TCP_DATA_SPLIT_DISABLED: + bp->flags &= ~BNXT_FLAG_HDS; + break; + } + bp->rx_ring_size = ering->rx_pending; bp->tx_ring_size = ering->tx_pending; bnxt_set_ring_params(bp); @@ -5346,6 +5366,7 @@ const struct ethtool_ops bnxt_ethtool_ops = { ETHTOOL_COALESCE_STATS_BLOCK_USECS | ETHTOOL_COALESCE_USE_ADAPTIVE_RX | ETHTOOL_COALESCE_USE_CQE, + .supported_ring_params = ETHTOOL_RING_USE_TCP_DATA_SPLIT, .get_link_ksettings = bnxt_get_link_ksettings, .set_link_ksettings = bnxt_set_link_ksettings, .get_fec_stats = bnxt_get_fec_stats, From patchwork Thu Oct 3 16:06:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 13821214 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 31130145B1F; Thu, 3 Oct 2024 16:06:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727971622; cv=none; b=Clhst4cV4N7hKmnGBFDsl46BH1jJRzgRx9weVoP9ygHAMBc8zlCgN6R1uZtCrdtZ4KzpJXt7A1eGOBBfi813BTdELUbKozYpFPR+z8Wjg7hWKjmwqKCS4QNkxPudCBq36EknQu5oNSf6uL20uusF19AQG1mUgoUvPsIt7wC6HnY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727971622; c=relaxed/simple; bh=TYrrp4+fXsd603T661sXMxk71f8OLHQCKwGfYnBVUgk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=XYap1KrETYqEkrzH30yWYSBsFuv0mdUOgpN3e8JNpM61vq3UE2jkIu2bO4U6M/hpbSNIGBOWvzVZ5Ew8QesA9IqNDzIbQsocRlELeZ9dLuOmJN0iSOzUeX0MEVa7rCAFOQYz9yLJtBKZ5ufEGM8086egc6Z+w2PohB2i4zwCwbk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=RRIRDp74; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RRIRDp74" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-20b0b5cdb57so16168635ad.1; Thu, 03 Oct 2024 09:06:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727971618; x=1728576418; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Fq3GJkLw5Uy4hCUSonWNYYQStI5yWMbLaCyxN/9ygUc=; b=RRIRDp743ccD3+iuDzos4UMrAbtGLdAhS7NtmTmpvtRG2yqxyJgzuf/VO2oeIQtGWM oHrL//4OOu1oTUSeLU62EncmhIFXIN6snyh1BEyYPy9F562JCXjnbUJAi8MG4Uxm2p/z sFrrzypJIrweuun7UwTJCwck+V0eRvpEZpCY+5ZlzuL8DJXvBgs3yOIapCx4H0InQJi1 ZTlrVYQuDxgL7NrOX1e//gUBvdHBn1WluIP9H61Im515Yi55EB+8JAYtJCQ9uYLVzNX7 97vNgHLlmL0jUCJbqL1AlQ+QPKlrkLgui0CjolZiSqy/8meXbjw2J1RXoQ3uEA5+VoIT Eheg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727971618; x=1728576418; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Fq3GJkLw5Uy4hCUSonWNYYQStI5yWMbLaCyxN/9ygUc=; b=LTv2QHpwniPQjWPcuZWhBBqeQHhwI7ThHDFgzGzE58MjeapbKVn8zYmVVhzp6iVY4R Qb2Z1Bs9D7qArZ8KM4bT3ghHTmYaU90JJODzEMtEQ1EY3oIKmL619suCEwY8QF8sn4Io PgO+QkviO2fYmC79/6qANISjBezbH9UYA4U0KMr9rNgqejFb1hUOFPVodRRY4xOwIXfZ Ol6AahvENNSE4cHzcHVXxQAn1YjSQVbgbQc/lv//GKQGVECrOVzJ5cT5E7SfHJLSPsx1 vOHHMJrR7v7AMk+fDBzq1/eZ3s24/qUx2OIdO6fEVBbTj52gmsAaQ2uM+AjZym/ON/f1 kYPg== X-Forwarded-Encrypted: i=1; AJvYcCV1d23ETZO0XEvitj6zdhAAoEHslECQWdBLxzytkp6cUjeMRkwcoMHc3NYHWIvZwK2yZKzYficOf10=@vger.kernel.org, AJvYcCWodnlW6AEoo7AGbpiKTIF2WTRgeNSFNO8jKI1+sV012d/Zut2GgYZeAaSeBZYW+M2MQQHvleLo@vger.kernel.org X-Gm-Message-State: AOJu0YxsCv4VAeExEyC9m1o0hOI63qUiHZGkXcT0/jYrmsYwYeEANwGp bxjwJC9YjqYjqhGylO9yAvAfxXdPDxdmr5IEbO44KA05Y/rm7EHw X-Google-Smtp-Source: AGHT+IFlMetfCalg1cSjT6oOoUcIoJEoH7BNTou9ObkVeJ4wj8mnMTaANI6c+K4Sf3PADLe700fMaQ== X-Received: by 2002:a17:902:da81:b0:20b:a8ad:9b0c with SMTP id d9443c01a7336-20be18c544amr60852385ad.3.1727971618108; Thu, 03 Oct 2024 09:06:58 -0700 (PDT) Received: from ap.. ([182.213.254.91]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20bef7071f1sm10425435ad.292.2024.10.03.09.06.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Oct 2024 09:06:57 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, almasrymina@google.com, netdev@vger.kernel.org, linux-doc@vger.kernel.org, donald.hunter@gmail.com, corbet@lwn.net, michael.chan@broadcom.com Cc: kory.maincent@bootlin.com, andrew@lunn.ch, maxime.chevallier@bootlin.com, danieller@nvidia.com, hengqi@linux.alibaba.com, ecree.xilinx@gmail.com, przemyslaw.kitszel@intel.com, hkallweit1@gmail.com, ahmed.zaki@intel.com, paul.greenwalt@intel.com, rrameshbabu@nvidia.com, idosch@nvidia.com, asml.silence@gmail.com, kaiyuanz@google.com, willemb@google.com, aleksander.lobakin@intel.com, dw@davidwei.uk, sridhar.samudrala@intel.com, bcreeley@amd.com, ap420073@gmail.com Subject: [PATCH net-next v3 3/7] net: ethtool: add support for configuring tcp-data-split-thresh Date: Thu, 3 Oct 2024 16:06:16 +0000 Message-Id: <20241003160620.1521626-4-ap420073@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241003160620.1521626-1-ap420073@gmail.com> References: <20241003160620.1521626-1-ap420073@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org The tcp-data-split-thresh option configures the threshold value of the tcp-data-split. If a received packet size is larger than this threshold value, a packet will be split into header and payload. The header indicates TCP header, but it depends on driver spec. The bnxt_en driver supports HDS(Header-Data-Split) configuration at FW level, affecting TCP and UDP too. So, like the tcp-data-split option, If tcp-data-split-thresh is set, it affects UDP and TCP packets. The tcp-data-split-thresh has a dependency, that is tcp-data-split option. This threshold value can be get/set only when tcp-data-split option is enabled. Example: # ethtool -G tcp-data-split-thresh # ethtool -G enp14s0f0np0 tcp-data-split on tcp-data-split-thresh 256 # ethtool -g enp14s0f0np0 Ring parameters for enp14s0f0np0: Pre-set maximums: ... TCP data split thresh: 256 Current hardware settings: ... TCP data split: on TCP data split thresh: 256 The tcp-data-split is not enabled, the tcp-data-split-thresh will not be used and can't be configured. # ethtool -G enp14s0f0np0 tcp-data-split off # ethtool -g enp14s0f0np0 Ring parameters for enp14s0f0np0: Pre-set maximums: ... TCP data split thresh: 256 Current hardware settings: ... TCP data split: off TCP data split thresh: n/a The default/min/max values are not defined in the ethtool so the drivers should define themself. The 0 value means that all TCP and UDP packets' header and payload will be split. Users should consider the overhead due to this feature. Signed-off-by: Taehee Yoo --- v3: - Fix documentation and ynl - Update error messages - Validate configuration of tcp-data-split and tcp-data-split-thresh v2: - Patch added. Documentation/netlink/specs/ethtool.yaml | 8 +++ Documentation/networking/ethtool-netlink.rst | 75 ++++++++++++-------- include/linux/ethtool.h | 4 ++ include/uapi/linux/ethtool_netlink.h | 2 + net/ethtool/netlink.h | 2 +- net/ethtool/rings.c | 46 ++++++++++-- 6 files changed, 102 insertions(+), 35 deletions(-) diff --git a/Documentation/netlink/specs/ethtool.yaml b/Documentation/netlink/specs/ethtool.yaml index 6a050d755b9c..96298fe5ed43 100644 --- a/Documentation/netlink/specs/ethtool.yaml +++ b/Documentation/netlink/specs/ethtool.yaml @@ -215,6 +215,12 @@ attribute-sets: - name: tx-push-buf-len-max type: u32 + - + name: tcp-data-split-thresh + type: u32 + - + name: tcp-data-split-thresh-max + type: u32 - name: mm-stat @@ -1393,6 +1399,8 @@ operations: - rx-push - tx-push-buf-len - tx-push-buf-len-max + - tcp-data-split-thresh + - tcp-data-split-thresh-max dump: *ring-get-op - name: rings-set diff --git a/Documentation/networking/ethtool-netlink.rst b/Documentation/networking/ethtool-netlink.rst index 295563e91082..f0cd918dbe7e 100644 --- a/Documentation/networking/ethtool-netlink.rst +++ b/Documentation/networking/ethtool-netlink.rst @@ -875,24 +875,32 @@ Request contents: Kernel response contents: - ======================================= ====== =========================== - ``ETHTOOL_A_RINGS_HEADER`` nested reply header - ``ETHTOOL_A_RINGS_RX_MAX`` u32 max size of RX ring - ``ETHTOOL_A_RINGS_RX_MINI_MAX`` u32 max size of RX mini ring - ``ETHTOOL_A_RINGS_RX_JUMBO_MAX`` u32 max size of RX jumbo ring - ``ETHTOOL_A_RINGS_TX_MAX`` u32 max size of TX ring - ``ETHTOOL_A_RINGS_RX`` u32 size of RX ring - ``ETHTOOL_A_RINGS_RX_MINI`` u32 size of RX mini ring - ``ETHTOOL_A_RINGS_RX_JUMBO`` u32 size of RX jumbo ring - ``ETHTOOL_A_RINGS_TX`` u32 size of TX ring - ``ETHTOOL_A_RINGS_RX_BUF_LEN`` u32 size of buffers on the ring - ``ETHTOOL_A_RINGS_TCP_DATA_SPLIT`` u8 TCP header / data split - ``ETHTOOL_A_RINGS_CQE_SIZE`` u32 Size of TX/RX CQE - ``ETHTOOL_A_RINGS_TX_PUSH`` u8 flag of TX Push mode - ``ETHTOOL_A_RINGS_RX_PUSH`` u8 flag of RX Push mode - ``ETHTOOL_A_RINGS_TX_PUSH_BUF_LEN`` u32 size of TX push buffer - ``ETHTOOL_A_RINGS_TX_PUSH_BUF_LEN_MAX`` u32 max size of TX push buffer - ======================================= ====== =========================== + ============================================= ====== ======================= + ``ETHTOOL_A_RINGS_HEADER`` nested reply header + ``ETHTOOL_A_RINGS_RX_MAX`` u32 max size of RX ring + ``ETHTOOL_A_RINGS_RX_MINI_MAX`` u32 max size of RX mini + ring + ``ETHTOOL_A_RINGS_RX_JUMBO_MAX`` u32 max size of RX jumbo + ring + ``ETHTOOL_A_RINGS_TX_MAX`` u32 max size of TX ring + ``ETHTOOL_A_RINGS_RX`` u32 size of RX ring + ``ETHTOOL_A_RINGS_RX_MINI`` u32 size of RX mini ring + ``ETHTOOL_A_RINGS_RX_JUMBO`` u32 size of RX jumbo ring + ``ETHTOOL_A_RINGS_TX`` u32 size of TX ring + ``ETHTOOL_A_RINGS_RX_BUF_LEN`` u32 size of buffers on the + ring + ``ETHTOOL_A_RINGS_TCP_DATA_SPLIT`` u8 TCP header / data split + ``ETHTOOL_A_RINGS_CQE_SIZE`` u32 Size of TX/RX CQE + ``ETHTOOL_A_RINGS_TX_PUSH`` u8 flag of TX Push mode + ``ETHTOOL_A_RINGS_RX_PUSH`` u8 flag of RX Push mode + ``ETHTOOL_A_RINGS_TX_PUSH_BUF_LEN`` u32 size of TX push buffer + ``ETHTOOL_A_RINGS_TX_PUSH_BUF_LEN_MAX`` u32 max size of TX push + buffer + ``ETHTOOL_A_RINGS_TCP_DATA_SPLIT_THRESH`` u32 threshold of + TCP header / data split + ``ETHTOOL_A_RINGS_TCP_DATA_SPLIT_THRESH_MAX`` u32 max threshold of + TCP header / data split + ============================================= ====== ======================= ``ETHTOOL_A_RINGS_TCP_DATA_SPLIT`` indicates whether the device is usable with page-flipping TCP zero-copy receive (``getsockopt(TCP_ZEROCOPY_RECEIVE)``). @@ -927,18 +935,21 @@ Sets ring sizes like ``ETHTOOL_SRINGPARAM`` ioctl request. Request contents: - ==================================== ====== =========================== - ``ETHTOOL_A_RINGS_HEADER`` nested reply header - ``ETHTOOL_A_RINGS_RX`` u32 size of RX ring - ``ETHTOOL_A_RINGS_RX_MINI`` u32 size of RX mini ring - ``ETHTOOL_A_RINGS_RX_JUMBO`` u32 size of RX jumbo ring - ``ETHTOOL_A_RINGS_TX`` u32 size of TX ring - ``ETHTOOL_A_RINGS_RX_BUF_LEN`` u32 size of buffers on the ring - ``ETHTOOL_A_RINGS_CQE_SIZE`` u32 Size of TX/RX CQE - ``ETHTOOL_A_RINGS_TX_PUSH`` u8 flag of TX Push mode - ``ETHTOOL_A_RINGS_RX_PUSH`` u8 flag of RX Push mode - ``ETHTOOL_A_RINGS_TX_PUSH_BUF_LEN`` u32 size of TX push buffer - ==================================== ====== =========================== + ========================================= ====== ======================= + ``ETHTOOL_A_RINGS_HEADER`` nested reply header + ``ETHTOOL_A_RINGS_RX`` u32 size of RX ring + ``ETHTOOL_A_RINGS_RX_MINI`` u32 size of RX mini ring + ``ETHTOOL_A_RINGS_RX_JUMBO`` u32 size of RX jumbo ring + ``ETHTOOL_A_RINGS_TX`` u32 size of TX ring + ``ETHTOOL_A_RINGS_RX_BUF_LEN`` u32 size of buffers on the ring + ``ETHTOOL_A_RINGS_TCP_DATA_SPLIT`` u8 TCP header / data split + ``ETHTOOL_A_RINGS_CQE_SIZE`` u32 Size of TX/RX CQE + ``ETHTOOL_A_RINGS_TX_PUSH`` u8 flag of TX Push mode + ``ETHTOOL_A_RINGS_RX_PUSH`` u8 flag of RX Push mode + ``ETHTOOL_A_RINGS_TX_PUSH_BUF_LEN`` u32 size of TX push buffer + ``ETHTOOL_A_RINGS_TCP_DATA_SPLIT_THRESH`` u32 threshold of + TCP header / data split + ========================================= ====== ======================= Kernel checks that requested ring sizes do not exceed limits reported by driver. Driver may impose additional constraints and may not support all @@ -954,6 +965,10 @@ A bigger CQE can have more receive buffer pointers, and in turn the NIC can transfer a bigger frame from wire. Based on the NIC hardware, the overall completion queue size can be adjusted in the driver if CQE size is modified. +``ETHTOOL_A_RINGS_TCP_DATA_SPLIT_THRESH`` specifies the threshold value of +tcp data split feature. If tcp-data-split is enabled and a received packet +size is larger than this threshold value, header and data will be split. + CHANNELS_GET ============ diff --git a/include/linux/ethtool.h b/include/linux/ethtool.h index 12f6dc567598..891f55b0f6aa 100644 --- a/include/linux/ethtool.h +++ b/include/linux/ethtool.h @@ -78,6 +78,8 @@ enum { * @cqe_size: Size of TX/RX completion queue event * @tx_push_buf_len: Size of TX push buffer * @tx_push_buf_max_len: Maximum allowed size of TX push buffer + * @tcp_data_split_thresh: Threshold value of tcp-data-split + * @tcp_data_split_thresh_max: Maximum allowed threshold of tcp-data-split-threshold */ struct kernel_ethtool_ringparam { u32 rx_buf_len; @@ -87,6 +89,8 @@ struct kernel_ethtool_ringparam { u32 cqe_size; u32 tx_push_buf_len; u32 tx_push_buf_max_len; + u32 tcp_data_split_thresh; + u32 tcp_data_split_thresh_max; }; /** diff --git a/include/uapi/linux/ethtool_netlink.h b/include/uapi/linux/ethtool_netlink.h index 283305f6b063..20fe6065b7ba 100644 --- a/include/uapi/linux/ethtool_netlink.h +++ b/include/uapi/linux/ethtool_netlink.h @@ -364,6 +364,8 @@ enum { ETHTOOL_A_RINGS_RX_PUSH, /* u8 */ ETHTOOL_A_RINGS_TX_PUSH_BUF_LEN, /* u32 */ ETHTOOL_A_RINGS_TX_PUSH_BUF_LEN_MAX, /* u32 */ + ETHTOOL_A_RINGS_TCP_DATA_SPLIT_THRESH, /* u32 */ + ETHTOOL_A_RINGS_TCP_DATA_SPLIT_THRESH_MAX, /* u32 */ /* add new constants above here */ __ETHTOOL_A_RINGS_CNT, diff --git a/net/ethtool/netlink.h b/net/ethtool/netlink.h index 203b08eb6c6f..8bea47a26605 100644 --- a/net/ethtool/netlink.h +++ b/net/ethtool/netlink.h @@ -455,7 +455,7 @@ extern const struct nla_policy ethnl_features_set_policy[ETHTOOL_A_FEATURES_WANT extern const struct nla_policy ethnl_privflags_get_policy[ETHTOOL_A_PRIVFLAGS_HEADER + 1]; extern const struct nla_policy ethnl_privflags_set_policy[ETHTOOL_A_PRIVFLAGS_FLAGS + 1]; extern const struct nla_policy ethnl_rings_get_policy[ETHTOOL_A_RINGS_HEADER + 1]; -extern const struct nla_policy ethnl_rings_set_policy[ETHTOOL_A_RINGS_TX_PUSH_BUF_LEN_MAX + 1]; +extern const struct nla_policy ethnl_rings_set_policy[ETHTOOL_A_RINGS_TCP_DATA_SPLIT_THRESH_MAX + 1]; extern const struct nla_policy ethnl_channels_get_policy[ETHTOOL_A_CHANNELS_HEADER + 1]; extern const struct nla_policy ethnl_channels_set_policy[ETHTOOL_A_CHANNELS_COMBINED_COUNT + 1]; extern const struct nla_policy ethnl_coalesce_get_policy[ETHTOOL_A_COALESCE_HEADER + 1]; diff --git a/net/ethtool/rings.c b/net/ethtool/rings.c index b7865a14fdf8..c7824515857f 100644 --- a/net/ethtool/rings.c +++ b/net/ethtool/rings.c @@ -61,7 +61,9 @@ static int rings_reply_size(const struct ethnl_req_info *req_base, nla_total_size(sizeof(u8)) + /* _RINGS_TX_PUSH */ nla_total_size(sizeof(u8))) + /* _RINGS_RX_PUSH */ nla_total_size(sizeof(u32)) + /* _RINGS_TX_PUSH_BUF_LEN */ - nla_total_size(sizeof(u32)); /* _RINGS_TX_PUSH_BUF_LEN_MAX */ + nla_total_size(sizeof(u32)) + /* _RINGS_TX_PUSH_BUF_LEN_MAX */ + nla_total_size(sizeof(u32)) + /* _RINGS_TCP_DATA_SPLIT_THRESH */ + nla_total_size(sizeof(u32)); /* _RINGS_TCP_DATA_SPLIT_THRESH_MAX */ } static int rings_fill_reply(struct sk_buff *skb, @@ -108,7 +110,13 @@ static int rings_fill_reply(struct sk_buff *skb, (nla_put_u32(skb, ETHTOOL_A_RINGS_TX_PUSH_BUF_LEN_MAX, kr->tx_push_buf_max_len) || nla_put_u32(skb, ETHTOOL_A_RINGS_TX_PUSH_BUF_LEN, - kr->tx_push_buf_len)))) + kr->tx_push_buf_len))) || + (kr->tcp_data_split == ETHTOOL_TCP_DATA_SPLIT_ENABLED && + (nla_put_u32(skb, ETHTOOL_A_RINGS_TCP_DATA_SPLIT_THRESH, + kr->tcp_data_split_thresh))) || + (kr->tcp_data_split == ETHTOOL_TCP_DATA_SPLIT_ENABLED && + (nla_put_u32(skb, ETHTOOL_A_RINGS_TCP_DATA_SPLIT_THRESH_MAX, + kr->tcp_data_split_thresh_max)))) return -EMSGSIZE; return 0; @@ -130,6 +138,7 @@ const struct nla_policy ethnl_rings_set_policy[] = { [ETHTOOL_A_RINGS_TX_PUSH] = NLA_POLICY_MAX(NLA_U8, 1), [ETHTOOL_A_RINGS_RX_PUSH] = NLA_POLICY_MAX(NLA_U8, 1), [ETHTOOL_A_RINGS_TX_PUSH_BUF_LEN] = { .type = NLA_U32 }, + [ETHTOOL_A_RINGS_TCP_DATA_SPLIT_THRESH] = { .type = NLA_U32 }, }; static int @@ -155,6 +164,14 @@ ethnl_set_rings_validate(struct ethnl_req_info *req_info, return -EOPNOTSUPP; } + if (tb[ETHTOOL_A_RINGS_TCP_DATA_SPLIT_THRESH] && + !(ops->supported_ring_params & ETHTOOL_RING_USE_TCP_DATA_SPLIT)) { + NL_SET_ERR_MSG_ATTR(info->extack, + tb[ETHTOOL_A_RINGS_TCP_DATA_SPLIT_THRESH], + "setting tcp-data-split-thresh is not supported"); + return -EOPNOTSUPP; + } + if (tb[ETHTOOL_A_RINGS_CQE_SIZE] && !(ops->supported_ring_params & ETHTOOL_RING_USE_CQE_SIZE)) { NL_SET_ERR_MSG_ATTR(info->extack, @@ -196,9 +213,9 @@ ethnl_set_rings(struct ethnl_req_info *req_info, struct genl_info *info) struct kernel_ethtool_ringparam kernel_ringparam = {}; struct ethtool_ringparam ringparam = {}; struct net_device *dev = req_info->dev; + bool mod = false, thresh_mod = false; struct nlattr **tb = info->attrs; const struct nlattr *err_attr; - bool mod = false; int ret; dev->ethtool_ops->get_ringparam(dev, &ringparam, @@ -222,9 +239,30 @@ ethnl_set_rings(struct ethnl_req_info *req_info, struct genl_info *info) tb[ETHTOOL_A_RINGS_RX_PUSH], &mod); ethnl_update_u32(&kernel_ringparam.tx_push_buf_len, tb[ETHTOOL_A_RINGS_TX_PUSH_BUF_LEN], &mod); - if (!mod) + ethnl_update_u32(&kernel_ringparam.tcp_data_split_thresh, + tb[ETHTOOL_A_RINGS_TCP_DATA_SPLIT_THRESH], + &thresh_mod); + if (!mod && !thresh_mod) return 0; + if (kernel_ringparam.tcp_data_split == ETHTOOL_TCP_DATA_SPLIT_DISABLED && + thresh_mod) { + NL_SET_ERR_MSG_ATTR(info->extack, + tb[ETHTOOL_A_RINGS_TCP_DATA_SPLIT_THRESH], + "tcp-data-split-thresh can not be updated while tcp-data-split is disabled"); + return -EINVAL; + } + + if (kernel_ringparam.tcp_data_split_thresh > + kernel_ringparam.tcp_data_split_thresh_max) { + NL_SET_ERR_MSG_ATTR_FMT(info->extack, + tb[ETHTOOL_A_RINGS_TCP_DATA_SPLIT_THRESH_MAX], + "Requested tcp-data-split-thresh exceeds the maximum of %u", + kernel_ringparam.tcp_data_split_thresh_max); + + return -EINVAL; + } + /* ensure new ring parameters are within limits */ if (ringparam.rx_pending > ringparam.rx_max_pending) err_attr = tb[ETHTOOL_A_RINGS_RX]; From patchwork Thu Oct 3 16:06:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 13821215 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C3F6145B1F; Thu, 3 Oct 2024 16:07:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727971627; cv=none; b=f/UmWQVIM89Y/NEZeOt91R2X+5Ib9OWWprJCUB+M/iGYLhT3NfJGCLlKEb7NjkGQRTJinRjE1IFNm59LdEm7xBSkVW6h1Df43TWwkzw7diTDGDI7DWjFYwc+wkSkiszri0K2a/P8gCZYwdYecmXAUtrINn3O1nQN91QYpwrfrnc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727971627; c=relaxed/simple; bh=dJyNekBNc8xXABNnImPmso0iUEUZlWUfmjCiYWP14Bg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=N8Oe7PVvquTw0Gfa1NFBtlTdk0oGO4UoyHoJbX3N8k02GNrUkdqZhvQbuk47MuI+bOg+8JN25mtZ3ycUzsZhoZO5AkwS2MYFMRsSdh5hy7E667Z3xMvmNRb+iYZU9D9TTS/Eu+7M44wfg4MzpwG84TM+a1f+Q3c9FnLM7RNeFqE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ImjvmkAR; arc=none smtp.client-ip=209.85.216.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ImjvmkAR" Received: by mail-pj1-f48.google.com with SMTP id 98e67ed59e1d1-2e09a276ec6so971859a91.0; Thu, 03 Oct 2024 09:07:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727971625; x=1728576425; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PGWzXPKXZdtX1z7l0g6rkXvuhx9rs3LPkv24/Z68kAY=; b=ImjvmkAROcmdwT9U+q8Wrfus0nHywqDlsmb9RmMMZ1EwN5D6bnEES2z8lRdbwx9Cdj vhQFcvPcPY2jRGPVMxaWOBcK4CxgjKLzp2PwYRARnXs29nBx0Ah87UKC/nFzxcxjuva+ vxdg7G8ZUF1QvEFnqzHednSYqekyZJdYw9yNgjtBrgVTr/1FD+U2ba3KsBtzXcwhS6cX 5viSw7XeKXv1AToWeUzg6X5BLFVbLAt7p92NAfurAhceMRWn7k84pV0SZisAFM5Q0Kba UvGBBeW8fpoMPYteIKvxfySlxpsBRdRt/CBJzaV2JZE1gHyMCHBPSztHm/UDUcCXyb6r Oxdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727971625; x=1728576425; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PGWzXPKXZdtX1z7l0g6rkXvuhx9rs3LPkv24/Z68kAY=; b=HRfe2Rlhd8MGwuL6u12rl1mey5QzNAHsKC8OKeH4IPEczbMUA9csnez6uuVvWqbITf WFMo1xiJRXyhq204sNz3fXVWxuTSEvzaTlQTVt56r4aZZXNppnJ6gBnSxFO2cFA08TI9 QGQRcZMOOFxn6T8Z34jPQy8NrtGkGVOJe8wTLkjdjv8e6FKLNn75I8EyztbPuyoGX2Lx rKGOhF931ajVaxxodvYE462snjdG6PXrS41CwCMQqWZOv69JLK+zzkKM1niJFL8tD3D8 QSh+vIqaPH3m6Cn88pPnKLOAyR74X3oY/Dnr2o0mmyIT6YXlwhU/vjBvEOYpcsDieVem i8dQ== X-Forwarded-Encrypted: i=1; AJvYcCUmrICRXO1fCICF4TT178I93M/q9xEb7tP/dv8SITFJRQWvYC+j2MkUQvLo9VbrfqOiKhK62Dy7@vger.kernel.org, AJvYcCWl5UCKNlQCFv6wK+7uJ5J+RHqbiBYXBGnWtFYe7U5zs9beeOijjUsCNaoCVYJRCqEiL4TWzINEjNc=@vger.kernel.org X-Gm-Message-State: AOJu0YyNdt4eksfJtFNufTKRMN73v8PE/bdZHum5jLJIH16oXzz7N+2S F4qCRFgEa6kzM1+chmopd0o+VSpQttQaHyUofU/PWrWzbjvxbcPD X-Google-Smtp-Source: AGHT+IHd88tWg+GAZM5L8SwqHpasZ2OsB82J9zIx3y/+bYDhetGMZ+mwaqGfs63UwgPG4L1vlqE/Gw== X-Received: by 2002:a17:90b:3908:b0:2e0:7d60:759 with SMTP id 98e67ed59e1d1-2e18456b525mr8751634a91.3.1727971625398; Thu, 03 Oct 2024 09:07:05 -0700 (PDT) Received: from ap.. ([182.213.254.91]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20bef7071f1sm10425435ad.292.2024.10.03.09.06.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Oct 2024 09:07:04 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, almasrymina@google.com, netdev@vger.kernel.org, linux-doc@vger.kernel.org, donald.hunter@gmail.com, corbet@lwn.net, michael.chan@broadcom.com Cc: kory.maincent@bootlin.com, andrew@lunn.ch, maxime.chevallier@bootlin.com, danieller@nvidia.com, hengqi@linux.alibaba.com, ecree.xilinx@gmail.com, przemyslaw.kitszel@intel.com, hkallweit1@gmail.com, ahmed.zaki@intel.com, paul.greenwalt@intel.com, rrameshbabu@nvidia.com, idosch@nvidia.com, asml.silence@gmail.com, kaiyuanz@google.com, willemb@google.com, aleksander.lobakin@intel.com, dw@davidwei.uk, sridhar.samudrala@intel.com, bcreeley@amd.com, ap420073@gmail.com Subject: [PATCH net-next v3 4/7] bnxt_en: add support for tcp-data-split-thresh ethtool command Date: Thu, 3 Oct 2024 16:06:17 +0000 Message-Id: <20241003160620.1521626-5-ap420073@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241003160620.1521626-1-ap420073@gmail.com> References: <20241003160620.1521626-1-ap420073@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org The bnxt_en driver has configured the hds_threshold value automatically when TPA is enabled based on the rx-copybreak default value. Now the tcp-data-split-thresh ethtool command is added, so it adds an implementation of tcp-data-split-thresh option. Configuration of the tcp-data-split-thresh is allowed only when the tcp-data-split is enabled. The default value of tcp-data-split-thresh is 256, which is the default value of rx-copybreak, which used to be the hds_thresh value. # Example: # ethtool -G enp14s0f0np0 tcp-data-split on tcp-data-split-thresh 256 # ethtool -g enp14s0f0np0 Ring parameters for enp14s0f0np0: Pre-set maximums: ... TCP data split thresh: 256 Current hardware settings: ... TCP data split: on TCP data split thresh: 256 It enables tcp-data-split and sets tcp-data-split-thresh value to 256. # ethtool -G enp14s0f0np0 tcp-data-split off # ethtool -g enp14s0f0np0 Ring parameters for enp14s0f0np0: Pre-set maximums: ... TCP data split thresh: 256 Current hardware settings: ... TCP data split: off TCP data split thresh: n/a Signed-off-by: Taehee Yoo --- v3: - Drop validation logic tcp-data-split and tcp-data-split-thresh. v2: - Patch added. drivers/net/ethernet/broadcom/bnxt/bnxt.c | 3 ++- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 ++ drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 4 ++++ 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index f046478dfd2a..872b15842b11 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -4455,6 +4455,7 @@ static void bnxt_init_ring_params(struct bnxt *bp) { bp->rx_copybreak = BNXT_DEFAULT_RX_COPYBREAK; bp->flags |= BNXT_FLAG_HDS; + bp->hds_threshold = BNXT_DEFAULT_RX_COPYBREAK; } /* bp->rx_ring_size, bp->tx_ring_size, dev->mtu, BNXT_FLAG_{G|L}RO flags must @@ -6429,7 +6430,7 @@ static int bnxt_hwrm_vnic_set_hds(struct bnxt *bp, struct bnxt_vnic_info *vnic) VNIC_PLCMODES_CFG_REQ_FLAGS_HDS_IPV6); req->enables |= cpu_to_le32(VNIC_PLCMODES_CFG_REQ_ENABLES_HDS_THRESHOLD_VALID); - req->hds_threshold = cpu_to_le16(bp->rx_copybreak); + req->hds_threshold = cpu_to_le16(bp->hds_threshold); } req->vnic_id = cpu_to_le32(vnic->fw_vnic_id); return hwrm_req_send(bp, req); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 35601c71dfe9..48f390519c35 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -2311,6 +2311,8 @@ struct bnxt { int rx_agg_nr_pages; int rx_nr_rings; int rsscos_nr_ctxs; +#define BNXT_HDS_THRESHOLD_MAX 256 + u16 hds_threshold; u32 tx_ring_size; u32 tx_ring_mask; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c index e9ef65dd2e7b..af6ed492f688 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c @@ -839,6 +839,9 @@ static void bnxt_get_ringparam(struct net_device *dev, else kernel_ering->tcp_data_split = ETHTOOL_TCP_DATA_SPLIT_DISABLED; + kernel_ering->tcp_data_split_thresh = bp->hds_threshold; + kernel_ering->tcp_data_split_thresh_max = BNXT_HDS_THRESHOLD_MAX; + ering->tx_max_pending = BNXT_MAX_TX_DESC_CNT; ering->rx_pending = bp->rx_ring_size; @@ -871,6 +874,7 @@ static int bnxt_set_ringparam(struct net_device *dev, case ETHTOOL_TCP_DATA_SPLIT_UNKNOWN: case ETHTOOL_TCP_DATA_SPLIT_ENABLED: bp->flags |= BNXT_FLAG_HDS; + bp->hds_threshold = (u16)kernel_ering->tcp_data_split_thresh; break; case ETHTOOL_TCP_DATA_SPLIT_DISABLED: bp->flags &= ~BNXT_FLAG_HDS; From patchwork Thu Oct 3 16:06:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 13821216 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 983B11A7040; Thu, 3 Oct 2024 16:07:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727971635; cv=none; b=nkgMF8uSJ/fOiefcqlvnqQ08MPfuVfoWFNTSQ+nYITId+CvAPrlXzDN52+/Cm09zAP9PkzByGb9t9EczrlKoyuY76t3PD/KpUtqf1e/OgIH7GPn/jQcB4LyqhTvp57jt/cvwEJPyWlW9f6CuW978qfk49x1Zgqs1+ZcuJIrK5e8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727971635; c=relaxed/simple; bh=GUxAVAdWls6nGx92bT6aTgRzlCPXr8vGEGOFwsQv93k=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Sd07S95CaTsTWV9dHuziY7Jk12R7fY4r1XnV0HhSk2ML548cb9zAs8OXRSo6VPFr86Fh1tQt6/HRZlx1OAB+f8pUFksk39Ri6jTl7otDcy1+w/bNbmzp7V24D8OXk8oKHmLRmxLeF6fvwPE8w0mlSW2Jk2YMtdVsVZx0HvXeylo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=EfZMf15I; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="EfZMf15I" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-20b58f2e1f4so7639255ad.2; Thu, 03 Oct 2024 09:07:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727971633; x=1728576433; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1wQX2LX/wv/RIoIQmgU/BP3FU/uSb/iTRAGvr1uJUqQ=; b=EfZMf15ISynde7OndwRhwtQZHYiwhB+LzvStU+DpMYxRu69Ob9X3CTCspCirII5Ovw whQpViIob3DOgBSLqPcl8uR8Vdbmz5XNgeLCV4MeOPVrlRec1tQfRFEieJsqAJWiW56v AZ5lUsUtIak+GQ+x6V/uMufaF9PMQziaqrP4J+1nWe+sLYUWRKYQQyCIjv8r2EaQnnd0 jcKin2v/Ea6n7ZErCqmkPdDomf+w9S5aML8iaKWzptLou6n+Oi2j+ThmiYKMjnWoPrW0 YX0XOcj/QEiSf4cqVxn83x414fpGmpPrhT1jlizyY2fAMKZgCJ0EvFtvpgY7uoQTuq/i qSMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727971633; x=1728576433; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1wQX2LX/wv/RIoIQmgU/BP3FU/uSb/iTRAGvr1uJUqQ=; b=PXkReaP+Q0tmzGB1iCo7apP4leS8ZX3QHw9OeoB5rCkZmBdC+VH6UnJFbcsN5e87/L 9iDtzeRCzysONOxt9jzhyltXz6A3v333dcAX+V5a+ADwxLuJHaXcIVd8PwUg11h866dh NMAzZ+HY1cHGqoATFaPWAEob3K3nlkZT5c9480cMwlMoVdYk9a6ExZG7iY+epwX5iFIW 1G+ssFYRsYDb5vG/czIJDktV/C/69wvLcEaWkT5hWxzFy5KvnYpCU0/J9dwUSplZVaU9 ZvYq4rlg/UGZgdYHvz4XVCftVnpnej4U00aJ++hmopPQyi9bD7p1Bd5GezCPaVGr0Hy9 m3cg== X-Forwarded-Encrypted: i=1; AJvYcCWtZiVzg2RilFjBmSgdmZexWBE1y/qg6s1lYL/ttKgdK6RMNDaEiutZgaJ9CLmVpjal1lb3bv20yGA=@vger.kernel.org, AJvYcCXJ0GkAUzf43s25ycT67oHCCW49T/v5hHQKyA68h7UFOaJ/GUg4sThTFMU4ytxqukGgWAR/HXVE@vger.kernel.org X-Gm-Message-State: AOJu0YzE3BqWtXl9yMZ1i+ockVlQeKz9/AHEiCsc+9/KGg4JG/yprD1D QHlgvLHgvvosoqGNEzwwqLnz2GbLvUB5HQH+Qv6xGzyCjvDXvdTf X-Google-Smtp-Source: AGHT+IGORnD4UEkFETTgSfUXqkR36wqi49XJV4ePNDbruOhZezv8WXJnkngJcBstkSHEYtCnAmG5/A== X-Received: by 2002:a17:902:ea01:b0:20b:9062:7b00 with SMTP id d9443c01a7336-20bc54c7086mr89253895ad.0.1727971632807; Thu, 03 Oct 2024 09:07:12 -0700 (PDT) Received: from ap.. ([182.213.254.91]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20bef7071f1sm10425435ad.292.2024.10.03.09.07.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Oct 2024 09:07:11 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, almasrymina@google.com, netdev@vger.kernel.org, linux-doc@vger.kernel.org, donald.hunter@gmail.com, corbet@lwn.net, michael.chan@broadcom.com Cc: kory.maincent@bootlin.com, andrew@lunn.ch, maxime.chevallier@bootlin.com, danieller@nvidia.com, hengqi@linux.alibaba.com, ecree.xilinx@gmail.com, przemyslaw.kitszel@intel.com, hkallweit1@gmail.com, ahmed.zaki@intel.com, paul.greenwalt@intel.com, rrameshbabu@nvidia.com, idosch@nvidia.com, asml.silence@gmail.com, kaiyuanz@google.com, willemb@google.com, aleksander.lobakin@intel.com, dw@davidwei.uk, sridhar.samudrala@intel.com, bcreeley@amd.com, ap420073@gmail.com Subject: [PATCH net-next v3 5/7] net: devmem: add ring parameter filtering Date: Thu, 3 Oct 2024 16:06:18 +0000 Message-Id: <20241003160620.1521626-6-ap420073@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241003160620.1521626-1-ap420073@gmail.com> References: <20241003160620.1521626-1-ap420073@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org If driver doesn't support ring parameter or tcp-data-split configuration is not sufficient, the devmem should not be set up. Before setup the devmem, tcp-data-split should be ON and tcp-data-split-thresh value should be 0. Signed-off-by: Taehee Yoo --- v3: - Patch added. net/core/devmem.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/net/core/devmem.c b/net/core/devmem.c index 11b91c12ee11..a9e9b15028e0 100644 --- a/net/core/devmem.c +++ b/net/core/devmem.c @@ -8,6 +8,8 @@ */ #include +#include +#include #include #include #include @@ -131,6 +133,8 @@ int net_devmem_bind_dmabuf_to_queue(struct net_device *dev, u32 rxq_idx, struct net_devmem_dmabuf_binding *binding, struct netlink_ext_ack *extack) { + struct kernel_ethtool_ringparam kernel_ringparam = {}; + struct ethtool_ringparam ringparam = {}; struct netdev_rx_queue *rxq; u32 xa_idx; int err; @@ -146,6 +150,20 @@ int net_devmem_bind_dmabuf_to_queue(struct net_device *dev, u32 rxq_idx, return -EEXIST; } + if (!dev->ethtool_ops->get_ringparam) { + NL_SET_ERR_MSG(extack, "can't get ringparam"); + return -EINVAL; + } + + dev->ethtool_ops->get_ringparam(dev, &ringparam, + &kernel_ringparam, extack); + if (kernel_ringparam.tcp_data_split != ETHTOOL_TCP_DATA_SPLIT_ENABLED || + kernel_ringparam.tcp_data_split_thresh) { + NL_SET_ERR_MSG(extack, + "tcp-header-data-split is disabled or threshold is not zero"); + return -EINVAL; + } + #ifdef CONFIG_XDP_SOCKETS if (rxq->pool) { NL_SET_ERR_MSG(extack, "designated queue already in use by AF_XDP"); From patchwork Thu Oct 3 16:06:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 13821217 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 074101A4F3A; Thu, 3 Oct 2024 16:07:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727971643; cv=none; b=VhDSaOBqxuxpLdQdh2vREIzSODvXv26Kq8gtklj2aqFPPXgKVTdruEmleL6xaTWYVgJDNAje9lEJpVJGZYT/5sonB2cfX/5kcdu3SC9hyyQ9qr90i1hfVnhJ/Ial3U0MXa4JFPpzrJRuGy/NmmhNV1PRBbOF2RoBeQkoltxtxNw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727971643; c=relaxed/simple; bh=fH0dchgKMq3fXQkGCCVLAyu8UF9Vd8KdI0rToMEDtzI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mi5NyVktrID6J7pa2k6PHzsZ/tzI1nN1KUn6jE64ls9nn5/egVOdauGoMnZNBOB7XndewO6Rj+RskOj1T8SVgxmZ05ueUOZiJT/9jyywWG9/NKQ+bI/Iu+F9PdY8Avg0921PACSLdkPvwp+6BxbynYv2H31V9wAfQBtn4yd132s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XBy4Ub8A; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XBy4Ub8A" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-208cf673b8dso11623125ad.3; Thu, 03 Oct 2024 09:07:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727971641; x=1728576441; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FctRudtrWsv/Ogj24COLxGVifRWLvWISYZk/oegPxYQ=; b=XBy4Ub8AMFYZ0nAUz9k7Q2iQ5+sF4VYwonEz2yqHNmaFrQB4GqriYa2kj437l9TQVA F5GepmFpwc4uDSkSbo5en/020JavRxGb+TWQwhygLqcoI4UtWVNjcJmXIfzmHvhKtcqL 8owi4tbtzI6a/YcYW27VAVwWw/ksVGpLnDJXiIRlK0v5Trjtb7b75PK3xSLIp/Rxw8/A 5Wu3tUW7PHxWQP9uy0BwrB5tg3ELk/xtU8OGYdH0EERO6A2LsgotmCNCnSPAuFJqp0wv QsDQbj999U4jmPU7+CXYs2/MSNitpZbXD8rEOxUAUk8k7KeQSd1MlQJwj1Mt8bZmchW7 p+uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727971641; x=1728576441; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FctRudtrWsv/Ogj24COLxGVifRWLvWISYZk/oegPxYQ=; b=KAknthu8McZkJVgIiN3kILhJe08pgEP0fHpqI+u3ybLvyBosJLtrKwXoZzw8/xDNYT yS6Nr4CTBCnkXxqJf/JKUUXlWGP+11FIyIb1I57j+bErIYuPx7w+a3iCTKZFQ7+34zfa JCUgHr8wAjEMnM0M5ujDiXrSPmNpS+yEiXTrsEPJXVDFmfl6RBBluoJFi2w2+f4g4BCa lFJmtB7hS1ZXCD0XDAgWlPSiYbCS4/hyPLJVTBVwM6oD7caxfZ90fdJz868N7E9bAPMK mPVtEDBTTrWMlG27aE0DPH5rUaI4bTxmxG5cCBfh4GGdZZaFRlHziwH5KPEMU5ooLjXr naXA== X-Forwarded-Encrypted: i=1; AJvYcCU3Jc6hL21UlIP6PakowOJhKN54NZHJEP5xwGp0aL/b5hQ7cwsXO3RSYOD62XvRaWig9PXAB/mB@vger.kernel.org, AJvYcCXjIklLjejbH6hexl5OMUppeucoPp2N108BK/poJwKSwhlv1UMCuSb/tcxgHVwaajziit37glxtnzk=@vger.kernel.org X-Gm-Message-State: AOJu0Yxk2kvMDfgv0FYOhbPlsa8Od5jNmzsmK6eZifV0ZQnDXL9kZ53M 9/7CKtjYMRNNirwNZQnCz/uZIgpRKS7NR3vfbVUSokMEMaiIdHjo X-Google-Smtp-Source: AGHT+IFe1RdKceRPkdoydM6HfGZtMob484p2+vuyVmOBY7Xvrz2gi0sZ74GT4bc8jGHdE5wpqb7cKg== X-Received: by 2002:a17:902:f552:b0:20b:9822:66fa with SMTP id d9443c01a7336-20bc5a0b178mr117385425ad.16.1727971640145; Thu, 03 Oct 2024 09:07:20 -0700 (PDT) Received: from ap.. ([182.213.254.91]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20bef7071f1sm10425435ad.292.2024.10.03.09.07.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Oct 2024 09:07:19 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, almasrymina@google.com, netdev@vger.kernel.org, linux-doc@vger.kernel.org, donald.hunter@gmail.com, corbet@lwn.net, michael.chan@broadcom.com Cc: kory.maincent@bootlin.com, andrew@lunn.ch, maxime.chevallier@bootlin.com, danieller@nvidia.com, hengqi@linux.alibaba.com, ecree.xilinx@gmail.com, przemyslaw.kitszel@intel.com, hkallweit1@gmail.com, ahmed.zaki@intel.com, paul.greenwalt@intel.com, rrameshbabu@nvidia.com, idosch@nvidia.com, asml.silence@gmail.com, kaiyuanz@google.com, willemb@google.com, aleksander.lobakin@intel.com, dw@davidwei.uk, sridhar.samudrala@intel.com, bcreeley@amd.com, ap420073@gmail.com Subject: [PATCH net-next v3 6/7] net: ethtool: add ring parameter filtering Date: Thu, 3 Oct 2024 16:06:19 +0000 Message-Id: <20241003160620.1521626-7-ap420073@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241003160620.1521626-1-ap420073@gmail.com> References: <20241003160620.1521626-1-ap420073@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org While the devmem is running, the tcp-data-split and tcp-data-split-thresh configuration should not be changed. If user tries to change tcp-data-split and threshold value while the devmem is running, it fails and shows extack message. Signed-off-by: Taehee Yoo --- v3: - Patch added net/ethtool/common.h | 1 + net/ethtool/rings.c | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/net/ethtool/common.h b/net/ethtool/common.h index d55d5201b085..beebd4db3e10 100644 --- a/net/ethtool/common.h +++ b/net/ethtool/common.h @@ -5,6 +5,7 @@ #include #include +#include #define ETHTOOL_DEV_FEATURE_WORDS DIV_ROUND_UP(NETDEV_FEATURE_COUNT, 32) diff --git a/net/ethtool/rings.c b/net/ethtool/rings.c index c7824515857f..0afc6b29a229 100644 --- a/net/ethtool/rings.c +++ b/net/ethtool/rings.c @@ -216,7 +216,8 @@ ethnl_set_rings(struct ethnl_req_info *req_info, struct genl_info *info) bool mod = false, thresh_mod = false; struct nlattr **tb = info->attrs; const struct nlattr *err_attr; - int ret; + struct netdev_rx_queue *rxq; + int ret, i; dev->ethtool_ops->get_ringparam(dev, &ringparam, &kernel_ringparam, info->extack); @@ -263,6 +264,18 @@ ethnl_set_rings(struct ethnl_req_info *req_info, struct genl_info *info) return -EINVAL; } + if (kernel_ringparam.tcp_data_split != ETHTOOL_TCP_DATA_SPLIT_ENABLED || + kernel_ringparam.tcp_data_split_thresh) { + for (i = 0; i < dev->real_num_rx_queues; i++) { + rxq = __netif_get_rx_queue(dev, i); + if (rxq->mp_params.mp_priv) { + NL_SET_ERR_MSG(info->extack, + "tcp-header-data-split is disabled or threshold is not zero"); + return -EINVAL; + } + } + } + /* ensure new ring parameters are within limits */ if (ringparam.rx_pending > ringparam.rx_max_pending) err_attr = tb[ETHTOOL_A_RINGS_RX]; From patchwork Thu Oct 3 16:06:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 13821218 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A3EA19B3EC; Thu, 3 Oct 2024 16:07:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727971650; cv=none; b=WXef55Wh64vPXtgUEdIhWAeIn4Fl7hviFlCQhmrmf7eWSKw5aIvGz3Aas/srilhGOQJ0q9k1PZ9oIjmoTERwIRCCSvxewLp0HpeVeXoMPdJE/E3KdxL1W9mjCC/r0dIaO7G7CZoTaHqaWcJmTKCHDgPC4lJaB3bhJBmISC/u8v0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727971650; c=relaxed/simple; bh=kixhTS1AA9SXQOw8x/CDCJKsyHJEF22BuRq0XIJItzE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZAIi0FwPmbo+mTy8uX93C+aCPBCa9ThrSEtyEkg7ALl1+6+bsJiCGYUaVyuP4RRtGsQxSo0ZV8/fwQbQOG59p6QTLywJyIoSiQENoBRDOPqgkqDvPFROJTDREXQiQMgTirks5CYpE09D/6MMqzgfR9hVe1i5pt7z2REY2brYZm4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=VkDGils3; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="VkDGils3" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-20b0b5cdb57so16174825ad.1; Thu, 03 Oct 2024 09:07:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727971648; x=1728576448; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=182z2bo7BWUlXrjZkZWyVgtJ1djGqSkG83itq9HK/1w=; b=VkDGils3oc7t2nrmmulqrAyllqxJLaauEoc4Rr/KSSamxbBqu3XVjx9zpbN/XGcGeg RQm6dayYjCjKF5nSP1K5i2Uu7bkK/lfUIMOJ+lUfCDScLaK+vKS7DQA1x74NFBaS1yFp 2FxYDnGJaQb8F6ashMSCLCu5Ua8u1zYGoFg6oNNBRNKxRGNGXY+Yw/stK4Mx0R1ybIUY jGq30fcadoyhjJCEnj9hrhOqk55VfEB8zkIlEMzk2zlRlKP7n0UnGnppjr4zPJsERamH qh0lNnJcnAJ4WZP6ulgGcknImdMptrfOGaMTiv8EhZIrJUwPCEHII0rOamPlioR8aUhH jzfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727971648; x=1728576448; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=182z2bo7BWUlXrjZkZWyVgtJ1djGqSkG83itq9HK/1w=; b=eQaHC1cbl7nurGP0cKBXvGD2T//kiyghxwkQ0Nl+OvuESq9ceugAxUMR8VVOcL5hTc S1Dc888V/QFU1DhK/SOgtnIixrFczi5/0HKD9kSoMAy+HL2vXFgOEtar704wyibyMgfX 5/592DTXb2Ycm3bxZovr7bA4+9+qEpayAGzY0iO6U6TduXKHa6Dsb1hJJmegJTemBTM6 /zaKB8rB+JIPaBZ7GQXB+qZYVfGQp3WHBuBTW/jcbEbMvwCwpy3JGv6WwTbYU1wB1wwA LtLYSwGSrLLF7hBaMpn4Sfu/M5xcplnSXQlHs9TowqcVAaVu1A9OevKgo4gfYnXVCUo8 Kb3g== X-Forwarded-Encrypted: i=1; AJvYcCVDbTrpqN4nnAGFCC/J8jjWC1xszLqAfq3o6oUUAlffmbBUHZ+/vfikOxqFlY0WU0LxbnfY8ecw5BM=@vger.kernel.org, AJvYcCX6bY+aujTW46nsdqcP8y4ggESxeRwzLtbrCNq2kPbf1s2SrZdX4OD9yiQd/NA70eYRZi4eE11I@vger.kernel.org X-Gm-Message-State: AOJu0YxoyNtBG4adjYoK37Fb1Mj6q1b7pBM1mJWIGNn+a2WLHzSJecR1 8rpfj5opqs69vPSFHWNX66j3SF0cs2FkWZN2AMQFeqbl+sN4sOb7 X-Google-Smtp-Source: AGHT+IFQ1OkKWIjY+turuwkrvJjYcvvWSVlCZnwNwemE1FJzNwmKVnLwHK5EmdO4/fp6UFltloCaxw== X-Received: by 2002:a17:903:11c3:b0:20b:5ef8:10a6 with SMTP id d9443c01a7336-20be18fa810mr59429155ad.8.1727971647530; Thu, 03 Oct 2024 09:07:27 -0700 (PDT) Received: from ap.. ([182.213.254.91]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20bef7071f1sm10425435ad.292.2024.10.03.09.07.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Oct 2024 09:07:26 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, almasrymina@google.com, netdev@vger.kernel.org, linux-doc@vger.kernel.org, donald.hunter@gmail.com, corbet@lwn.net, michael.chan@broadcom.com Cc: kory.maincent@bootlin.com, andrew@lunn.ch, maxime.chevallier@bootlin.com, danieller@nvidia.com, hengqi@linux.alibaba.com, ecree.xilinx@gmail.com, przemyslaw.kitszel@intel.com, hkallweit1@gmail.com, ahmed.zaki@intel.com, paul.greenwalt@intel.com, rrameshbabu@nvidia.com, idosch@nvidia.com, asml.silence@gmail.com, kaiyuanz@google.com, willemb@google.com, aleksander.lobakin@intel.com, dw@davidwei.uk, sridhar.samudrala@intel.com, bcreeley@amd.com, ap420073@gmail.com Subject: [PATCH net-next v3 7/7] bnxt_en: add support for device memory tcp Date: Thu, 3 Oct 2024 16:06:20 +0000 Message-Id: <20241003160620.1521626-8-ap420073@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241003160620.1521626-1-ap420073@gmail.com> References: <20241003160620.1521626-1-ap420073@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Currently, bnxt_en driver satisfies the requirements of Device memory TCP, which is tcp-data-split. So, it implements Device memory TCP for bnxt_en driver. From now on, the aggregation ring handles netmem_ref instead of page regardless of the on/off of netmem. So, for the aggregation ring, memory will be handled with the netmem page_pool API instead of generic page_pool API. If Devmem is enabled, netmem_ref is used as-is and if Devmem is not enabled, netmem_ref will be converted to page and that is used. Driver recognizes whether the devmem is set or unset based on the mp_params.mp_priv is not NULL. Only if devmem is set, it passes PP_FLAG_ALLOW_UNREADABLE_NETMEM. Signed-off-by: Taehee Yoo --- v3: - Patch added drivers/net/ethernet/broadcom/Kconfig | 1 + drivers/net/ethernet/broadcom/bnxt/bnxt.c | 98 +++++++++++++++-------- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 +- 3 files changed, 66 insertions(+), 35 deletions(-) diff --git a/drivers/net/ethernet/broadcom/Kconfig b/drivers/net/ethernet/broadcom/Kconfig index 75ca3ddda1f5..f37ff12d4746 100644 --- a/drivers/net/ethernet/broadcom/Kconfig +++ b/drivers/net/ethernet/broadcom/Kconfig @@ -211,6 +211,7 @@ config BNXT select FW_LOADER select LIBCRC32C select NET_DEVLINK + select NET_DEVMEM select PAGE_POOL select DIMLIB select AUXILIARY_BUS diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 872b15842b11..64e07d247f97 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -55,6 +55,7 @@ #include #include #include +#include #include "bnxt_hsi.h" #include "bnxt.h" @@ -863,6 +864,22 @@ static void bnxt_tx_int(struct bnxt *bp, struct bnxt_napi *bnapi, int budget) bnapi->events &= ~BNXT_TX_CMP_EVENT; } +static netmem_ref __bnxt_alloc_rx_netmem(struct bnxt *bp, dma_addr_t *mapping, + struct bnxt_rx_ring_info *rxr, + unsigned int *offset, + gfp_t gfp) +{ + netmem_ref netmem; + + netmem = page_pool_alloc_netmem(rxr->page_pool, GFP_ATOMIC); + if (!netmem) + return 0; + *offset = 0; + + *mapping = page_pool_get_dma_addr_netmem(netmem) + *offset; + return netmem; +} + static struct page *__bnxt_alloc_rx_page(struct bnxt *bp, dma_addr_t *mapping, struct bnxt_rx_ring_info *rxr, unsigned int *offset, @@ -972,21 +989,21 @@ static inline u16 bnxt_find_next_agg_idx(struct bnxt_rx_ring_info *rxr, u16 idx) return next; } -static inline int bnxt_alloc_rx_page(struct bnxt *bp, - struct bnxt_rx_ring_info *rxr, - u16 prod, gfp_t gfp) +static inline int bnxt_alloc_rx_netmem(struct bnxt *bp, + struct bnxt_rx_ring_info *rxr, + u16 prod, gfp_t gfp) { struct rx_bd *rxbd = &rxr->rx_agg_desc_ring[RX_AGG_RING(bp, prod)][RX_IDX(prod)]; struct bnxt_sw_rx_agg_bd *rx_agg_buf; - struct page *page; + netmem_ref netmem; dma_addr_t mapping; u16 sw_prod = rxr->rx_sw_agg_prod; unsigned int offset = 0; - page = __bnxt_alloc_rx_page(bp, &mapping, rxr, &offset, gfp); + netmem = __bnxt_alloc_rx_netmem(bp, &mapping, rxr, &offset, gfp); - if (!page) + if (!netmem) return -ENOMEM; if (unlikely(test_bit(sw_prod, rxr->rx_agg_bmap))) @@ -996,7 +1013,7 @@ static inline int bnxt_alloc_rx_page(struct bnxt *bp, rx_agg_buf = &rxr->rx_agg_ring[sw_prod]; rxr->rx_sw_agg_prod = RING_RX_AGG(bp, NEXT_RX_AGG(sw_prod)); - rx_agg_buf->page = page; + rx_agg_buf->netmem = netmem; rx_agg_buf->offset = offset; rx_agg_buf->mapping = mapping; rxbd->rx_bd_haddr = cpu_to_le64(mapping); @@ -1044,7 +1061,7 @@ static void bnxt_reuse_rx_agg_bufs(struct bnxt_cp_ring_info *cpr, u16 idx, struct rx_agg_cmp *agg; struct bnxt_sw_rx_agg_bd *cons_rx_buf, *prod_rx_buf; struct rx_bd *prod_bd; - struct page *page; + netmem_ref netmem; if (p5_tpa) agg = bnxt_get_tpa_agg_p5(bp, rxr, idx, start + i); @@ -1061,11 +1078,11 @@ static void bnxt_reuse_rx_agg_bufs(struct bnxt_cp_ring_info *cpr, u16 idx, cons_rx_buf = &rxr->rx_agg_ring[cons]; /* It is possible for sw_prod to be equal to cons, so - * set cons_rx_buf->page to NULL first. + * set cons_rx_buf->netmem to 0 first. */ - page = cons_rx_buf->page; - cons_rx_buf->page = NULL; - prod_rx_buf->page = page; + netmem = cons_rx_buf->netmem; + cons_rx_buf->netmem = 0; + prod_rx_buf->netmem = netmem; prod_rx_buf->offset = cons_rx_buf->offset; prod_rx_buf->mapping = cons_rx_buf->mapping; @@ -1192,6 +1209,7 @@ static struct sk_buff *bnxt_rx_skb(struct bnxt *bp, static u32 __bnxt_rx_agg_pages(struct bnxt *bp, struct bnxt_cp_ring_info *cpr, + struct sk_buff *skb, struct skb_shared_info *shinfo, u16 idx, u32 agg_bufs, bool tpa, struct xdp_buff *xdp) @@ -1211,7 +1229,7 @@ static u32 __bnxt_rx_agg_pages(struct bnxt *bp, u16 cons, frag_len; struct rx_agg_cmp *agg; struct bnxt_sw_rx_agg_bd *cons_rx_buf; - struct page *page; + netmem_ref netmem; dma_addr_t mapping; if (p5_tpa) @@ -1223,9 +1241,15 @@ static u32 __bnxt_rx_agg_pages(struct bnxt *bp, RX_AGG_CMP_LEN) >> RX_AGG_CMP_LEN_SHIFT; cons_rx_buf = &rxr->rx_agg_ring[cons]; - skb_frag_fill_page_desc(frag, cons_rx_buf->page, - cons_rx_buf->offset, frag_len); - shinfo->nr_frags = i + 1; + if (skb) { + skb_add_rx_frag_netmem(skb, i, cons_rx_buf->netmem, + cons_rx_buf->offset, frag_len, + BNXT_RX_PAGE_SIZE); + } else { + skb_frag_fill_page_desc(frag, netmem_to_page(cons_rx_buf->netmem), + cons_rx_buf->offset, frag_len); + shinfo->nr_frags = i + 1; + } __clear_bit(cons, rxr->rx_agg_bmap); /* It is possible for bnxt_alloc_rx_page() to allocate @@ -1233,15 +1257,15 @@ static u32 __bnxt_rx_agg_pages(struct bnxt *bp, * need to clear the cons entry now. */ mapping = cons_rx_buf->mapping; - page = cons_rx_buf->page; - cons_rx_buf->page = NULL; + netmem = cons_rx_buf->netmem; + cons_rx_buf->netmem = 0; - if (xdp && page_is_pfmemalloc(page)) + if (xdp && page_is_pfmemalloc(netmem_to_page(netmem))) xdp_buff_set_frag_pfmemalloc(xdp); - if (bnxt_alloc_rx_page(bp, rxr, prod, GFP_ATOMIC) != 0) { + if (bnxt_alloc_rx_netmem(bp, rxr, prod, GFP_ATOMIC) != 0) { --shinfo->nr_frags; - cons_rx_buf->page = page; + cons_rx_buf->netmem = netmem; /* Update prod since possibly some pages have been * allocated already. @@ -1269,7 +1293,7 @@ static struct sk_buff *bnxt_rx_agg_pages_skb(struct bnxt *bp, struct skb_shared_info *shinfo = skb_shinfo(skb); u32 total_frag_len = 0; - total_frag_len = __bnxt_rx_agg_pages(bp, cpr, shinfo, idx, + total_frag_len = __bnxt_rx_agg_pages(bp, cpr, skb, shinfo, idx, agg_bufs, tpa, NULL); if (!total_frag_len) { skb_mark_for_recycle(skb); @@ -1277,9 +1301,6 @@ static struct sk_buff *bnxt_rx_agg_pages_skb(struct bnxt *bp, return NULL; } - skb->data_len += total_frag_len; - skb->len += total_frag_len; - skb->truesize += BNXT_RX_PAGE_SIZE * agg_bufs; return skb; } @@ -1294,7 +1315,7 @@ static u32 bnxt_rx_agg_pages_xdp(struct bnxt *bp, if (!xdp_buff_has_frags(xdp)) shinfo->nr_frags = 0; - total_frag_len = __bnxt_rx_agg_pages(bp, cpr, shinfo, + total_frag_len = __bnxt_rx_agg_pages(bp, cpr, NULL, shinfo, idx, agg_bufs, tpa, xdp); if (total_frag_len) { xdp_buff_set_frags_flag(xdp); @@ -3342,15 +3363,15 @@ static void bnxt_free_one_rx_agg_ring(struct bnxt *bp, struct bnxt_rx_ring_info for (i = 0; i < max_idx; i++) { struct bnxt_sw_rx_agg_bd *rx_agg_buf = &rxr->rx_agg_ring[i]; - struct page *page = rx_agg_buf->page; + netmem_ref netmem = rx_agg_buf->netmem; - if (!page) + if (!netmem) continue; - rx_agg_buf->page = NULL; + rx_agg_buf->netmem = 0; __clear_bit(i, rxr->rx_agg_bmap); - page_pool_recycle_direct(rxr->page_pool, page); + page_pool_put_full_netmem(rxr->page_pool, netmem, true); } } @@ -3608,9 +3629,11 @@ static void bnxt_free_rx_rings(struct bnxt *bp) static int bnxt_alloc_rx_page_pool(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, + int queue_idx, int numa_node) { struct page_pool_params pp = { 0 }; + struct netdev_rx_queue *rxq; pp.pool_size = bp->rx_agg_ring_size; if (BNXT_RX_PAGE_MODE(bp)) @@ -3621,8 +3644,15 @@ static int bnxt_alloc_rx_page_pool(struct bnxt *bp, pp.dev = &bp->pdev->dev; pp.dma_dir = bp->rx_dir; pp.max_len = PAGE_SIZE; - pp.flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV; + pp.order = 0; + + rxq = __netif_get_rx_queue(bp->dev, queue_idx); + if (rxq->mp_params.mp_priv) + pp.flags = PP_FLAG_DMA_MAP | PP_FLAG_ALLOW_UNREADABLE_NETMEM; + else + pp.flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV; + pp.queue_idx = queue_idx; rxr->page_pool = page_pool_create(&pp); if (IS_ERR(rxr->page_pool)) { int err = PTR_ERR(rxr->page_pool); @@ -3655,7 +3685,7 @@ static int bnxt_alloc_rx_rings(struct bnxt *bp) cpu_node = cpu_to_node(cpu); netdev_dbg(bp->dev, "Allocating page pool for rx_ring[%d] on numa_node: %d\n", i, cpu_node); - rc = bnxt_alloc_rx_page_pool(bp, rxr, cpu_node); + rc = bnxt_alloc_rx_page_pool(bp, rxr, i, cpu_node); if (rc) return rc; @@ -4154,7 +4184,7 @@ static void bnxt_alloc_one_rx_ring_page(struct bnxt *bp, prod = rxr->rx_agg_prod; for (i = 0; i < bp->rx_agg_ring_size; i++) { - if (bnxt_alloc_rx_page(bp, rxr, prod, GFP_KERNEL)) { + if (bnxt_alloc_rx_netmem(bp, rxr, prod, GFP_KERNEL)) { netdev_warn(bp->dev, "init'ed rx ring %d with %d/%d pages only\n", ring_nr, i, bp->rx_ring_size); break; @@ -15063,7 +15093,7 @@ static int bnxt_queue_mem_alloc(struct net_device *dev, void *qmem, int idx) clone->rx_sw_agg_prod = 0; clone->rx_next_cons = 0; - rc = bnxt_alloc_rx_page_pool(bp, clone, rxr->page_pool->p.nid); + rc = bnxt_alloc_rx_page_pool(bp, clone, idx, rxr->page_pool->p.nid); if (rc) return rc; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 48f390519c35..3cf57a3c7664 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -895,7 +895,7 @@ struct bnxt_sw_rx_bd { }; struct bnxt_sw_rx_agg_bd { - struct page *page; + netmem_ref netmem; unsigned int offset; dma_addr_t mapping; };