From patchwork Mon Jan 6 13:56:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Yoong Siang X-Patchwork-Id: 13927481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 163B4E77188 for ; Mon, 6 Jan 2025 13:59:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=FbImh3rew9F11otkTECgCTloPAzNFac328y4t/QaBb0=; b=mLoz4MoR2ru42roz3qvXbhbWlK 0X+0RisI0BrIeNJI0A5Ah+i6lV4QL7SWoK3VbGlxZ3+PFVpuJqw5PVPl2KAKQlcBNwrMxpV//HmFZ Bh1in3wMut/EudwyqtXrFSRmf6GMsPm1vTaq5bEpFS0p8uZPnjw0TldaLQ7M+LTlN9GSdz4+DTk2y vkURJOVJdgSzIjs8MeWUKuOYV2YaVUsRTVkyncyxXBZ6ttwrFZozln928esRQKxNpT9qEyAVEZZv6 P8zolW4t9EUYRwbY2bQ4KctDytewkI97BELZ0rdbcH1Zp+B+pxT+eAxHqJGxbf6cWYevQGGpBLsfB zXdk0VPg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tUndH-00000001Tyk-00Be; Mon, 06 Jan 2025 13:59:03 +0000 Received: from mgamail.intel.com ([198.175.65.14]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tUnag-00000001TLg-3IsV for linux-arm-kernel@lists.infradead.org; Mon, 06 Jan 2025 13:56:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1736171783; x=1767707783; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=RE390wSjYMqV/JAg33I90B6BgKfiEgWfan1O6Kb0phQ=; b=CE/V4Fr2WC+qj5C7UJ64+Z4Wepne8ffCW/YVe5OWZbidpv9+g/iCNP4K 50+8f4M6WkYXkMY3c6y2kXnY6ET+/I8VgOTM3RAaU8+RVRzmx+5YHnB4M Mssv+bqe95LrJ1AX0nMFiBlW/WK4AUZ08IF3+L1gQr+9LfF+CUg3YQG0Z d7wXhsF0cd+ceZ1/cNIXlpUz5AQz7CKMv2odkHZMu+HfLvy/SPfQ2n890 S+L1gcW9H7JacjsIlB9J6ESKKyrXAmFXuFKsaiqCgbm6D4AqaIjYuYOL+ SOud0ugwxNWiFEW963tpKZGa970cYlM1srWBCjTucDVoezKpH2qDCNMAv w==; X-CSE-ConnectionGUID: M/R2mw/eQzSI2lta364Wig== X-CSE-MsgGUID: RLetlU0+TciJqOd1W+kjrA== X-IronPort-AV: E=McAfee;i="6700,10204,11307"; a="40088805" X-IronPort-AV: E=Sophos;i="6.12,292,1728975600"; d="scan'208";a="40088805" Received: from orviesa010.jf.intel.com ([10.64.159.150]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Jan 2025 05:56:22 -0800 X-CSE-ConnectionGUID: 2fBglCCoQjGjjWIOWDO7yQ== X-CSE-MsgGUID: soy+YUYQRMyPZTG9Ic16Pg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="102305073" Received: from unknown (HELO P12ILL20yoongsia.png.intel.com) ([10.88.227.38]) by orviesa010.jf.intel.com with ESMTP; 06 Jan 2025 05:56:10 -0800 From: Song Yoong Siang To: "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Willem de Bruijn , Florian Bezdeka , Donald Hunter , Jonathan Corbet , Bjorn Topel , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Andrew Lunn , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Joe Damato , Stanislav Fomichev , Xuan Zhuo , Mina Almasry , Daniel Jurgens , Song Yoong Siang , Amritha Nambiar , Andrii Nakryiko , Eduard Zingerman , Mykola Lysenko , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Hao Luo , Jiri Olsa , Shuah Khan , Alexandre Torgue , Jose Abreu , Maxime Coquelin , Tony Nguyen , Przemek Kitszel Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-stm32@st-md-mailman.stormreply.com, linux-arm-kernel@lists.infradead.org, intel-wired-lan@lists.osuosl.org, xdp-hints@xdp-project.net Subject: [PATCH bpf-next v4 1/4] xsk: Add launch time hardware offload support to XDP Tx metadata Date: Mon, 6 Jan 2025 21:56:06 +0800 Message-Id: <20250106135606.9704-1-yoong.siang.song@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250106_055622_898190_1CC9C153 X-CRM114-Status: GOOD ( 28.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Extend the XDP Tx metadata framework so that user can requests launch time hardware offload, where the Ethernet device will schedule the packet for transmission at a pre-determined time called launch time. The value of launch time is communicated from user space to Ethernet driver via launch_time field of struct xsk_tx_metadata. Suggested-by: Stanislav Fomichev Signed-off-by: Song Yoong Siang --- Documentation/netlink/specs/netdev.yaml | 4 ++ Documentation/networking/xsk-tx-metadata.rst | 64 ++++++++++++++++++++ include/net/xdp_sock.h | 10 +++ include/net/xdp_sock_drv.h | 1 + include/uapi/linux/if_xdp.h | 10 +++ include/uapi/linux/netdev.h | 3 + net/core/netdev-genl.c | 2 + net/xdp/xsk.c | 3 + tools/include/uapi/linux/if_xdp.h | 10 +++ tools/include/uapi/linux/netdev.h | 3 + 10 files changed, 110 insertions(+) diff --git a/Documentation/netlink/specs/netdev.yaml b/Documentation/netlink/specs/netdev.yaml index cbb544bd6c84..e59c8a14f7d1 100644 --- a/Documentation/netlink/specs/netdev.yaml +++ b/Documentation/netlink/specs/netdev.yaml @@ -70,6 +70,10 @@ definitions: name: tx-checksum doc: L3 checksum HW offload is supported by the driver. + - + name: tx-launch-time + doc: + Launch time HW offload is supported by the driver. - name: queue-type type: enum diff --git a/Documentation/networking/xsk-tx-metadata.rst b/Documentation/networking/xsk-tx-metadata.rst index e76b0cfc32f7..3cec089747ce 100644 --- a/Documentation/networking/xsk-tx-metadata.rst +++ b/Documentation/networking/xsk-tx-metadata.rst @@ -50,6 +50,10 @@ The flags field enables the particular offload: checksum. ``csum_start`` specifies byte offset of where the checksumming should start and ``csum_offset`` specifies byte offset where the device should store the computed checksum. +- ``XDP_TXMD_FLAGS_LAUNCH_TIME``: requests the device to schedule the + packet for transmission at a pre-determined time called launch time. The + value of launch time is indicated by ``launch_time`` field of + ``union xsk_tx_metadata``. Besides the flags above, in order to trigger the offloads, the first packet's ``struct xdp_desc`` descriptor should set ``XDP_TX_METADATA`` @@ -65,6 +69,65 @@ In this case, when running in ``XDK_COPY`` mode, the TX checksum is calculated on the CPU. Do not enable this option in production because it will negatively affect performance. +Launch Time +=========== + +The value of the requested launch time should be based on the device's PTP +Hardware Clock (PHC) to ensure accuracy. AF_XDP takes a different data path +compared to the ETF queuing discipline, which organizes packets and delays +their transmission. Instead, AF_XDP immediately hands off the packets to +the device driver without rearranging their order or holding them prior to +transmission. In scenarios where the launch time offload feature is +disabled, the device driver is expected to disregard the launch time +request. For correct interpretation and meaningful operation, the launch +time should never be set to a value larger than the farthest programmable +time in the future (the horizon). Different devices have different hardware +limitations on the launch time offload feature. + +stmmac driver +------------- + +For stmmac, TSO and launch time (TBS) features are mutually exclusive for +each individual Tx Queue. By default, the driver configures Tx Queue 0 to +support TSO and the rest of the Tx Queues to support TBS. The launch time +hardware offload feature can be enabled or disabled by using the tc-etf +command to call the driver's ndo_setup_tc() callback. + +The value of the launch time that is programmed in the Enhanced Normal +Transmit Descriptors is a 32-bit value, where the most significant 8 bits +represent the time in seconds and the remaining 24 bits represent the time +in 256 ns increments. The programmed launch time is compared against the +PTP time (bits[39:8]) and rolls over after 256 seconds. Therefore, the +horizon of the launch time for dwmac4 and dwxlgmac2 is 128 seconds in the +future. + +The stmmac driver maintains FIFO behavior and does not perform packet +reordering. This means that a packet with a launch time request will block +other packets in the same Tx Queue until it is transmitted. + +igc driver +---------- + +For igc, all four Tx Queues support the launch time feature. The launch +time hardware offload feature can be enabled or disabled by using the +tc-etf command to call the driver's ndo_setup_tc() callback. When entering +TSN mode, the igc driver will reset the device and create a default Qbv +schedule with a 1-second cycle time, with all Tx Queues open at all times. + +The value of the launch time that is programmed in the Advanced Transmit +Context Descriptor is a relative offset to the starting time of the Qbv +transmission window of the queue. The Frst flag of the descriptor can be +set to schedule the packet for the next Qbv cycle. Therefore, the horizon +of the launch time for i225 and i226 is the ending time of the next cycle +of the Qbv transmission window of the queue. For example, when the Qbv +cycle time is set to 1 second, the horizon of the launch time ranges +from 1 second to 2 seconds, depending on where the Qbv cycle is currently +running. + +The igc driver maintains FIFO behavior and does not perform packet +reordering. This means that a packet with a launch time request will block +other packets in the same Tx Queue until it is transmitted. + Querying Device Capabilities ============================ @@ -74,6 +137,7 @@ Refer to ``xsk-flags`` features bitmask in - ``tx-timestamp``: device supports ``XDP_TXMD_FLAGS_TIMESTAMP`` - ``tx-checksum``: device supports ``XDP_TXMD_FLAGS_CHECKSUM`` +- ``tx-launch-time``: device supports ``XDP_TXMD_FLAGS_LAUNCH_TIME`` See ``tools/net/ynl/samples/netdev.c`` on how to query this information. diff --git a/include/net/xdp_sock.h b/include/net/xdp_sock.h index bfe625b55d55..a58ae7589d12 100644 --- a/include/net/xdp_sock.h +++ b/include/net/xdp_sock.h @@ -110,11 +110,16 @@ struct xdp_sock { * indicates position where checksumming should start. * csum_offset indicates position where checksum should be stored. * + * void (*tmo_request_launch_time)(u64 launch_time, void *priv) + * Called when AF_XDP frame requested launch time HW offload support. + * launch_time indicates the PTP time at which the device can schedule the + * packet for transmission. */ struct xsk_tx_metadata_ops { void (*tmo_request_timestamp)(void *priv); u64 (*tmo_fill_timestamp)(void *priv); void (*tmo_request_checksum)(u16 csum_start, u16 csum_offset, void *priv); + void (*tmo_request_launch_time)(u64 launch_time, void *priv); }; #ifdef CONFIG_XDP_SOCKETS @@ -162,6 +167,11 @@ static inline void xsk_tx_metadata_request(const struct xsk_tx_metadata *meta, if (!meta) return; + if (ops->tmo_request_launch_time) + if (meta->flags & XDP_TXMD_FLAGS_LAUNCH_TIME) + ops->tmo_request_launch_time(meta->request.launch_time, + priv); + if (ops->tmo_request_timestamp) if (meta->flags & XDP_TXMD_FLAGS_TIMESTAMP) ops->tmo_request_timestamp(priv); diff --git a/include/net/xdp_sock_drv.h b/include/net/xdp_sock_drv.h index 40085afd9160..78af371bc002 100644 --- a/include/net/xdp_sock_drv.h +++ b/include/net/xdp_sock_drv.h @@ -198,6 +198,7 @@ static inline void *xsk_buff_raw_get_data(struct xsk_buff_pool *pool, u64 addr) #define XDP_TXMD_FLAGS_VALID ( \ XDP_TXMD_FLAGS_TIMESTAMP | \ XDP_TXMD_FLAGS_CHECKSUM | \ + XDP_TXMD_FLAGS_LAUNCH_TIME | \ 0) static inline bool xsk_buff_valid_tx_metadata(struct xsk_tx_metadata *meta) diff --git a/include/uapi/linux/if_xdp.h b/include/uapi/linux/if_xdp.h index 42ec5ddaab8d..42869770776e 100644 --- a/include/uapi/linux/if_xdp.h +++ b/include/uapi/linux/if_xdp.h @@ -127,6 +127,12 @@ struct xdp_options { */ #define XDP_TXMD_FLAGS_CHECKSUM (1 << 1) +/* Request launch time hardware offload. The device will schedule the packet for + * transmission at a pre-determined time called launch time. The value of + * launch time is communicated via launch_time field of struct xsk_tx_metadata. + */ +#define XDP_TXMD_FLAGS_LAUNCH_TIME (1 << 2) + /* AF_XDP offloads request. 'request' union member is consumed by the driver * when the packet is being transmitted. 'completion' union member is * filled by the driver when the transmit completion arrives. @@ -142,6 +148,10 @@ struct xsk_tx_metadata { __u16 csum_start; /* Offset from csum_start where checksum should be stored. */ __u16 csum_offset; + + /* XDP_TXMD_FLAGS_LAUNCH_TIME */ + /* Launch time in nanosecond against the PTP HW Clock */ + __u64 launch_time; } request; struct { diff --git a/include/uapi/linux/netdev.h b/include/uapi/linux/netdev.h index e4be227d3ad6..5ab85f4af009 100644 --- a/include/uapi/linux/netdev.h +++ b/include/uapi/linux/netdev.h @@ -59,10 +59,13 @@ enum netdev_xdp_rx_metadata { * by the driver. * @NETDEV_XSK_FLAGS_TX_CHECKSUM: L3 checksum HW offload is supported by the * driver. + * @NETDEV_XSK_FLAGS_LAUNCH_TIME: Launch Time HW offload is supported by the + * driver. */ enum netdev_xsk_flags { NETDEV_XSK_FLAGS_TX_TIMESTAMP = 1, NETDEV_XSK_FLAGS_TX_CHECKSUM = 2, + NETDEV_XSK_FLAGS_LAUNCH_TIME = 4, }; enum netdev_queue_type { diff --git a/net/core/netdev-genl.c b/net/core/netdev-genl.c index 9527dd46e4dc..e2515cf9190f 100644 --- a/net/core/netdev-genl.c +++ b/net/core/netdev-genl.c @@ -52,6 +52,8 @@ XDP_METADATA_KFUNC_xxx xsk_features |= NETDEV_XSK_FLAGS_TX_TIMESTAMP; if (netdev->xsk_tx_metadata_ops->tmo_request_checksum) xsk_features |= NETDEV_XSK_FLAGS_TX_CHECKSUM; + if (netdev->xsk_tx_metadata_ops->tmo_request_launch_time) + xsk_features |= NETDEV_XSK_FLAGS_LAUNCH_TIME; } if (nla_put_u32(rsp, NETDEV_A_DEV_IFINDEX, netdev->ifindex) || diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index 3fa70286c846..8feaa0e86f07 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -743,6 +743,9 @@ static struct sk_buff *xsk_build_skb(struct xdp_sock *xs, goto free_err; } } + + if (meta->flags & XDP_TXMD_FLAGS_LAUNCH_TIME) + skb->skb_mstamp_ns = meta->request.launch_time; } } diff --git a/tools/include/uapi/linux/if_xdp.h b/tools/include/uapi/linux/if_xdp.h index 2f082b01ff22..67719f8966c2 100644 --- a/tools/include/uapi/linux/if_xdp.h +++ b/tools/include/uapi/linux/if_xdp.h @@ -127,6 +127,12 @@ struct xdp_options { */ #define XDP_TXMD_FLAGS_CHECKSUM (1 << 1) +/* Request launch time hardware offload. The device will schedule the packet for + * transmission at a pre-determined time called launch time. The value of + * launch time is communicated via launch_time field of struct xsk_tx_metadata. + */ +#define XDP_TXMD_FLAGS_LAUNCH_TIME (1 << 2) + /* AF_XDP offloads request. 'request' union member is consumed by the driver * when the packet is being transmitted. 'completion' union member is * filled by the driver when the transmit completion arrives. @@ -142,6 +148,10 @@ struct xsk_tx_metadata { __u16 csum_start; /* Offset from csum_start where checksum should be stored. */ __u16 csum_offset; + + /* XDP_TXMD_FLAGS_LAUNCH_TIME */ + /* Launch time in nanosecond against the PTP HW Clock */ + __u64 launch_time; } request; struct { diff --git a/tools/include/uapi/linux/netdev.h b/tools/include/uapi/linux/netdev.h index e4be227d3ad6..5ab85f4af009 100644 --- a/tools/include/uapi/linux/netdev.h +++ b/tools/include/uapi/linux/netdev.h @@ -59,10 +59,13 @@ enum netdev_xdp_rx_metadata { * by the driver. * @NETDEV_XSK_FLAGS_TX_CHECKSUM: L3 checksum HW offload is supported by the * driver. + * @NETDEV_XSK_FLAGS_LAUNCH_TIME: Launch Time HW offload is supported by the + * driver. */ enum netdev_xsk_flags { NETDEV_XSK_FLAGS_TX_TIMESTAMP = 1, NETDEV_XSK_FLAGS_TX_CHECKSUM = 2, + NETDEV_XSK_FLAGS_LAUNCH_TIME = 4, }; enum netdev_queue_type { From patchwork Mon Jan 6 13:56:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Yoong Siang X-Patchwork-Id: 13927483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1A6CBE77188 for ; Mon, 6 Jan 2025 14:00:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=8DG1m2MO4VbtmjqRQgAFewTSJOIUlZggw031ix+qRLg=; b=qIdfFvVKTuQvsWBixXTAWu0NeH 47t49dUlKfoCEU9WMk7pdnEfU/pTMtgi7sqIZ8u8LnfCWAlEH+IFHXK9Zn3juiFBNb8A5VqOtmzKx bwJ0au3RQB3HfBAYhjeBglPfkE7Ke661n1ynRO4nU6Is82K6IWC63hD3zW8rprqYF/yfjI5T5lLlb jMHOYmmFtw0ZqLexYN2az5WGbR9AdjRAMcz0FUbUxUcKB5Snkr5Yt7rfnwUI/BgJTEMpv8b1/WY10 G4WiqXf8jFlLwWCuJ5eojtmdAdnQP1Hbk+ACCOB/9cEEoJKBgrcfHvmMkG0IE92fmcEhgMX5/ZSEi /haeQqGA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tUneP-00000001U63-2TOn; Mon, 06 Jan 2025 14:00:13 +0000 Received: from mgamail.intel.com ([192.198.163.17]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tUnbB-00000001TTn-2yge for linux-arm-kernel@lists.infradead.org; Mon, 06 Jan 2025 13:56:54 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1736171814; x=1767707814; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=j5yMpN4jtRU0ARlnwODyqOYe3Fo7KApfyuihjMBXRfI=; b=k7t/m7RMySmSU4JvPK/IuVz3FwBKj0G+iRMRfi4qmfhuRsUYJD0DB1z0 pPS9iL1xmum59wDNV/vjnRk1T0gtMYyvHCGwYYwcxYLe5YBEb/Hv/ytZY 6qEUCVddbvZy9NF0K48wOeaioQ0F63mn+dSQSA5iJwMy0fafBALqVdQSY cOMV/zkHMHfJFD1ow1m59nnb00YBQPrfiEv5tJH5EJH5Zn4625gJSq/eY hjypGAQPl65JOTtmwMuKQzNyzwEU2tsZqdFFPdmE0F99Zr+GTuJApZlUY XGCW5NDRgrCtS2SwseLRBBnBfqqjZTuskZRcup1WQlqwC4QceuaIJTMKC g==; X-CSE-ConnectionGUID: yUg6JYGbQT2RSmcOssEc8g== X-CSE-MsgGUID: +SOvOsOiQzCkP0CNf/qzhQ== X-IronPort-AV: E=McAfee;i="6700,10204,11307"; a="36214356" X-IronPort-AV: E=Sophos;i="6.12,292,1728975600"; d="scan'208";a="36214356" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Jan 2025 05:56:53 -0800 X-CSE-ConnectionGUID: Pwm2JC3KQPCA4v3oc7WvIQ== X-CSE-MsgGUID: Hg6VOARaR0Sq8Cl4XWiMEw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="125728687" Received: from unknown (HELO P12ILL20yoongsia.png.intel.com) ([10.88.227.38]) by fmviesa002.fm.intel.com with ESMTP; 06 Jan 2025 05:56:42 -0800 From: Song Yoong Siang To: "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Willem de Bruijn , Florian Bezdeka , Donald Hunter , Jonathan Corbet , Bjorn Topel , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Andrew Lunn , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Joe Damato , Stanislav Fomichev , Xuan Zhuo , Mina Almasry , Daniel Jurgens , Song Yoong Siang , Amritha Nambiar , Andrii Nakryiko , Eduard Zingerman , Mykola Lysenko , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Hao Luo , Jiri Olsa , Shuah Khan , Alexandre Torgue , Jose Abreu , Maxime Coquelin , Tony Nguyen , Przemek Kitszel Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-stm32@st-md-mailman.stormreply.com, linux-arm-kernel@lists.infradead.org, intel-wired-lan@lists.osuosl.org, xdp-hints@xdp-project.net Subject: [PATCH bpf-next v4 2/4] selftests/bpf: Add Launch Time request to xdp_hw_metadata Date: Mon, 6 Jan 2025 21:56:38 +0800 Message-Id: <20250106135638.9719-1-yoong.siang.song@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250106_055653_764085_CDC7E0B9 X-CRM114-Status: GOOD ( 15.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add Launch Time hw offload request to xdp_hw_metadata. User can configure the delta of launch time to HW RX-time by using "-l" argument. The default delta is 100,000,000 nanosecond. Signed-off-by: Song Yoong Siang --- tools/testing/selftests/bpf/xdp_hw_metadata.c | 30 +++++++++++++++++-- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/bpf/xdp_hw_metadata.c b/tools/testing/selftests/bpf/xdp_hw_metadata.c index 6f7b15d6c6ed..795c1d14e02d 100644 --- a/tools/testing/selftests/bpf/xdp_hw_metadata.c +++ b/tools/testing/selftests/bpf/xdp_hw_metadata.c @@ -13,6 +13,7 @@ * - UDP 9091 packets trigger TX reply * - TX HW timestamp is requested and reported back upon completion * - TX checksum is requested + * - TX launch time HW offload is requested for transmission */ #include @@ -64,6 +65,8 @@ int rxq; bool skip_tx; __u64 last_hw_rx_timestamp; __u64 last_xdp_rx_timestamp; +__u64 last_launch_time; +__u64 launch_time_delta_to_hw_rx_timestamp = 100000000; /* 0.1 second */ void test__fail(void) { /* for network_helpers.c */ } @@ -298,6 +301,8 @@ static bool complete_tx(struct xsk *xsk, clockid_t clock_id) if (meta->completion.tx_timestamp) { __u64 ref_tstamp = gettime(clock_id); + print_tstamp_delta("HW Launch-time", "HW TX-complete-time", + last_launch_time, meta->completion.tx_timestamp); print_tstamp_delta("HW TX-complete-time", "User TX-complete-time", meta->completion.tx_timestamp, ref_tstamp); print_tstamp_delta("XDP RX-time", "User TX-complete-time", @@ -395,6 +400,14 @@ static void ping_pong(struct xsk *xsk, void *rx_packet, clockid_t clock_id) xsk, ntohs(udph->check), ntohs(want_csum), meta->request.csum_start, meta->request.csum_offset); + /* Set the value of launch time */ + meta->flags |= XDP_TXMD_FLAGS_LAUNCH_TIME; + meta->request.launch_time = last_hw_rx_timestamp + + launch_time_delta_to_hw_rx_timestamp; + last_launch_time = meta->request.launch_time; + print_tstamp_delta("HW RX-time", "HW Launch-time", last_hw_rx_timestamp, + meta->request.launch_time); + memcpy(data, rx_packet, len); /* don't share umem chunk for simplicity */ tx_desc->options |= XDP_TX_METADATA; tx_desc->len = len; @@ -402,10 +415,14 @@ static void ping_pong(struct xsk *xsk, void *rx_packet, clockid_t clock_id) xsk_ring_prod__submit(&xsk->tx, 1); } +#define SLEEP_PER_ITERATION_IN_US 10 +#define SLEEP_PER_ITERATION_IN_NS (SLEEP_PER_ITERATION_IN_US * 1000) +#define MAX_ITERATION(x) (((x) / SLEEP_PER_ITERATION_IN_NS) + 500) static int verify_metadata(struct xsk *rx_xsk, int rxq, int server_fd, clockid_t clock_id) { const struct xdp_desc *rx_desc; struct pollfd fds[rxq + 1]; + int max_iterations; __u64 comp_addr; __u64 addr; __u32 idx = 0; @@ -418,6 +435,9 @@ static int verify_metadata(struct xsk *rx_xsk, int rxq, int server_fd, clockid_t fds[i].revents = 0; } + /* Calculate max iterations to wait for transmit completion */ + max_iterations = MAX_ITERATION(launch_time_delta_to_hw_rx_timestamp); + fds[rxq].fd = server_fd; fds[rxq].events = POLLIN; fds[rxq].revents = 0; @@ -477,10 +497,10 @@ static int verify_metadata(struct xsk *rx_xsk, int rxq, int server_fd, clockid_t if (ret) printf("kick_tx ret=%d\n", ret); - for (int j = 0; j < 500; j++) { + for (int j = 0; j < max_iterations; j++) { if (complete_tx(xsk, clock_id)) break; - usleep(10); + usleep(SLEEP_PER_ITERATION_IN_US); } } } @@ -608,6 +628,7 @@ static void print_usage(void) " -h Display this help and exit\n\n" " -m Enable multi-buffer XDP for larger MTU\n" " -r Don't generate AF_XDP reply (rx metadata only)\n" + " -l Delta of launch time to HW RX-time in ns (default: 100,000,000ns)\n" "Generate test packets on the other machine with:\n" " echo -n xdp | nc -u -q1 9091\n"; @@ -618,7 +639,7 @@ static void read_args(int argc, char *argv[]) { int opt; - while ((opt = getopt(argc, argv, "chmr")) != -1) { + while ((opt = getopt(argc, argv, "chmrl:")) != -1) { switch (opt) { case 'c': bind_flags &= ~XDP_USE_NEED_WAKEUP; @@ -634,6 +655,9 @@ static void read_args(int argc, char *argv[]) case 'r': skip_tx = true; break; + case 'l': + launch_time_delta_to_hw_rx_timestamp = atoll(optarg); + break; case '?': if (isprint(optopt)) fprintf(stderr, "Unknown option: -%c\n", optopt); From patchwork Mon Jan 6 13:56:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Yoong Siang X-Patchwork-Id: 13927484 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 79C1AE77198 for ; Mon, 6 Jan 2025 14:01:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=oOOSCdK8/MdEd5mLIWruc0Y4cjtDY8rdZJwUNZdP9r0=; b=tRMuINNNtqhrSHRG4ekcp7yquC jH7dOccQ+CxDx7k0Hj77tboSjjQ+5KMPKgJzRE20hboxfqHPUJ97lM0Baxe03/WlEzjJ6TJuKGkEm pSzaGqVAgrmXEUeidUIdpR2zn/u0bQJBYW8ZloJSsIK0hNyTPDHHQFxtvn+R3FjD6MevpfRJJDKGw IXhGPI+vrqNv5l8ynMsHBtsn2XQufCesNfi8Q7LlPMeJlaD3L3HKg7nVal60/Q388Nbd1QgOyphQq 0tZanEJvKcMtmbGXqAvxk01y7vmfyVf3cOaz6j2YvEgnWdvaNoqMieSlt94DANNQ/P3YMfDCAVPsj PxGXj57w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tUnfZ-00000001UJr-1KS7; Mon, 06 Jan 2025 14:01:25 +0000 Received: from mgamail.intel.com ([192.198.163.17]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tUnbT-00000001TYz-2HBI for linux-arm-kernel@lists.infradead.org; Mon, 06 Jan 2025 13:57:12 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1736171831; x=1767707831; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=/+i8FYAafrrfqRSuGKBUIfPXgTCEk4S697hIJh7teK4=; b=kabvQL0ofaeFTlOgNLeSACBCPjX2su1ZjgoP1HPXVS2jOjmgIPQnQP5c hE1V/5vnek5fY8wVtwt2c8DpwD+pdasFCCjOmgdCkjqRdYY2dMK2KMaxI t3dgaQK+LfxYptTHLA6uPo1qmJ3xjh6IWdYLoIFrE8k4RYbcOPPlxB9dh QDbTY0esZWGt6NbOJEH7l0KOLRVQY+RyD7ETGUD3AOVRzT1/0+CfShHji TId/i0qbgwcII7B4fOSXlqqfLl4h2tGqcaZlL+I/U8hNyWhT8bC1GX5kt v8XFCMYm/BDymWbzqXQ4An99iKw9e9tmORAm6/XygTRTQzDZmmWTSlcC0 g==; X-CSE-ConnectionGUID: XiAyvj6kR6i14JUM4y0Qaw== X-CSE-MsgGUID: iPixkZ73RDi1iTilOMAb5Q== X-IronPort-AV: E=McAfee;i="6700,10204,11307"; a="36214394" X-IronPort-AV: E=Sophos;i="6.12,292,1728975600"; d="scan'208";a="36214394" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Jan 2025 05:57:11 -0800 X-CSE-ConnectionGUID: M6+cb+MjRTC9JvS0857Mbg== X-CSE-MsgGUID: FJybESsMTdaPa83cjklrFw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="125728751" Received: from unknown (HELO P12ILL20yoongsia.png.intel.com) ([10.88.227.38]) by fmviesa002.fm.intel.com with ESMTP; 06 Jan 2025 05:57:01 -0800 From: Song Yoong Siang To: "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Willem de Bruijn , Florian Bezdeka , Donald Hunter , Jonathan Corbet , Bjorn Topel , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Andrew Lunn , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Joe Damato , Stanislav Fomichev , Xuan Zhuo , Mina Almasry , Daniel Jurgens , Song Yoong Siang , Amritha Nambiar , Andrii Nakryiko , Eduard Zingerman , Mykola Lysenko , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Hao Luo , Jiri Olsa , Shuah Khan , Alexandre Torgue , Jose Abreu , Maxime Coquelin , Tony Nguyen , Przemek Kitszel Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-stm32@st-md-mailman.stormreply.com, linux-arm-kernel@lists.infradead.org, intel-wired-lan@lists.osuosl.org, xdp-hints@xdp-project.net Subject: [PATCH bpf-next v4 3/4] net: stmmac: Add launch time support to XDP ZC Date: Mon, 6 Jan 2025 21:56:58 +0800 Message-Id: <20250106135658.9734-1-yoong.siang.song@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250106_055711_594397_6281213D X-CRM114-Status: GOOD ( 12.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Enable launch time (Time-Based Scheduling) support to XDP zero copy via XDP Tx metadata framework. This patch is tested with tools/testing/selftests/bpf/xdp_hw_metadata on Intel Tiger Lake platform. Below are the test steps and result. Test Steps: 1. Add mqprio qdisc: $ sudo tc qdisc add dev enp0s30f4 handle 8001: parent root mqprio num_tc 4 map 0 1 2 3 3 3 3 3 3 3 3 3 3 3 3 3 queues 1@0 1@1 1@2 1@3 hw 0 2. Enable launch time hardware offload on hardware queue 1: $ sudo tc qdisc replace dev enp0s30f4 parent 8001:2 etf offload clockid CLOCK_TAI delta 500000 3. Add an ingress qdisc: $ sudo tc qdisc add dev enp0s30f4 ingress 4. Add a flower filter to route incoming packet with VLAN priority 1 into hardware queue 1: $ sudo tc filter add dev enp0s30f4 parent ffff: protocol 802.1Q flower vlan_prio 1 hw_tc 1 5. Enable VLAN tag stripping: $ sudo ethtool -K enp0s30f4 rxvlan on 6. Start xdp_hw_metadata selftest application: $ sudo ./xdp_hw_metadata enp0s30f4 -l 1000000000 7. Send an UDP packet with VLAN priority 1 to port 9091 of DUT. When launch time is set to 1s in the future, the delta between launch time and transmit hardware timestamp is equal to 16.963us, as shown in result below: 0x55b5864717a8: rx_desc[4]->addr=88100 addr=88100 comp_addr=88100 EoP No rx_hash, err=-95 HW RX-time: 1734579065767717328 (sec:1734579065.7677) delta to User RX-time sec:0.0004 (375.624 usec) XDP RX-time: 1734579065768004454 (sec:1734579065.7680) delta to User RX-time sec:0.0001 (88.498 usec) No rx_vlan_tci or rx_vlan_proto, err=-95 0x55b5864717a8: ping-pong with csum=5619 (want 0000) csum_start=34 csum_offset=6 HW RX-time: 1734579065767717328 (sec:1734579065.7677) delta to HW Launch-time sec:1.0000 (1000000.000 usec) 0x55b5864717a8: complete tx idx=4 addr=4018 HW Launch-time: 1734579066767717328 (sec:1734579066.7677) delta to HW TX-complete-time sec:0.0000 (16.963 usec) HW TX-complete-time: 1734579066767734291 (sec:1734579066.7677) delta to User TX-complete-time sec:0.0001 (130.408 usec) XDP RX-time: 1734579065768004454 (sec:1734579065.7680) delta to User TX-complete-time sec:0.9999 (999860.245 usec) HW RX-time: 1734579065767717328 (sec:1734579065.7677) delta to HW TX-complete-time sec:1.0000 (1000016.963 usec) 0x55b5864717a8: complete rx idx=132 addr=88100 Signed-off-by: Song Yoong Siang --- drivers/net/ethernet/stmicro/stmmac/stmmac.h | 2 ++ drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 13 +++++++++++++ 2 files changed, 15 insertions(+) diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac.h b/drivers/net/ethernet/stmicro/stmmac/stmmac.h index 1d86439b8a14..c80462d42989 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac.h +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac.h @@ -106,6 +106,8 @@ struct stmmac_metadata_request { struct stmmac_priv *priv; struct dma_desc *tx_desc; bool *set_ic; + struct dma_edesc *edesc; + int tbs; }; struct stmmac_xsk_tx_complete { diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index c81ea8cdfe6e..3a083e3684ed 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -2445,9 +2445,20 @@ static u64 stmmac_xsk_fill_timestamp(void *_priv) return 0; } +static void stmmac_xsk_request_launch_time(u64 launch_time, void *_priv) +{ + struct stmmac_metadata_request *meta_req = _priv; + struct timespec64 ts = ns_to_timespec64(launch_time); + + if (meta_req->tbs & STMMAC_TBS_EN) + stmmac_set_desc_tbs(meta_req->priv, meta_req->edesc, ts.tv_sec, + ts.tv_nsec); +} + static const struct xsk_tx_metadata_ops stmmac_xsk_tx_metadata_ops = { .tmo_request_timestamp = stmmac_xsk_request_timestamp, .tmo_fill_timestamp = stmmac_xsk_fill_timestamp, + .tmo_request_launch_time = stmmac_xsk_request_launch_time, }; static bool stmmac_xdp_xmit_zc(struct stmmac_priv *priv, u32 queue, u32 budget) @@ -2531,6 +2542,8 @@ static bool stmmac_xdp_xmit_zc(struct stmmac_priv *priv, u32 queue, u32 budget) meta_req.priv = priv; meta_req.tx_desc = tx_desc; meta_req.set_ic = &set_ic; + meta_req.tbs = tx_q->tbs; + meta_req.edesc = &tx_q->dma_entx[entry]; xsk_tx_metadata_request(meta, &stmmac_xsk_tx_metadata_ops, &meta_req); if (set_ic) { From patchwork Mon Jan 6 13:57:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Yoong Siang X-Patchwork-Id: 13927485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9617CE77198 for ; Mon, 6 Jan 2025 14:02:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=4C5FyZ9zpL0WI0kqjxT0tLMHStA5KPm50HM8ww9BAbI=; b=EF2sM6iEwNhQMl0pLuM/ThW+zN n+ygNvWnOtpVOpHiZw8d1D4bwSdOdAkiLYBsb2bU08VNixk4yCOKNOnozj7LGoR8h3gPlNdUbgha6 3zed/H8ihQPK/k2xC6iYX+QWUpF16eu/aCiQXetjmLuUk9RUDITEQfuPvAGR0HfVDHAOuV6focqv/ Rnfz+jp64gpQpl5NAdepIc4iG9MIgCQ37qqGhpAQ4PsWYPXieMTP6mYMBFr0I1m8DQN/KpYI0m9uH T0jHukL2hxnssWSi8GXI3xEqTYDozamcKBHJ5DVKXCoS+6vBtO0rFHbSg7xwZ/sJXykLUtiwr3oTg TaI021Nw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tUngi-00000001UdT-49Sn; Mon, 06 Jan 2025 14:02:36 +0000 Received: from mgamail.intel.com ([198.175.65.16]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tUnbv-00000001Th4-0jtf for linux-arm-kernel@lists.infradead.org; Mon, 06 Jan 2025 13:57:40 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1736171860; x=1767707860; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=qGCwaypJFSzHfD7SuLIe+AzV2lEwzPgKj63Q4p347so=; b=bUNl52Evms9UTlJzGmkGzX65iJF65THJZQp/fYVjNSxmVQbUli4+od2t fA4xhQQ0xYWzZSO2NI2kVb8kmpR4wcSBjz9t1v+Gn3xWSvNfk12jsEx6R vx4mM8LEaG56qiodJsKgzMBmBzHHgFj5zHGf5sJ9VCWZyvAj80v8TZF/5 /2QJa8tbK2OFQiP0soAj/QTKq7YlFZ98ZUrWdVrDbxoa6tvM1L7RkGUek /iIViblxphAbS2bLNWvwXSF9NUfofceED2C44MNahZlReOfQfEOH7L7Y8 7nYmxABiF8xOZNE0a7QGYlWugbQvrSAbn6ObZmfEcI8ouMo19SVHGr3Bl g==; X-CSE-ConnectionGUID: al0tOawgSKGKmWb9/zZWMA== X-CSE-MsgGUID: TKFOHsm+T3KI5Qi5PTOOMQ== X-IronPort-AV: E=McAfee;i="6700,10204,11307"; a="36473842" X-IronPort-AV: E=Sophos;i="6.12,292,1728975600"; d="scan'208";a="36473842" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Jan 2025 05:57:39 -0800 X-CSE-ConnectionGUID: 5mNEh76+SQOPHNjCosOwZg== X-CSE-MsgGUID: 7Q1uLqm2TRCloWfgYIj/Pg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,292,1728975600"; d="scan'208";a="107408167" Received: from unknown (HELO P12ILL20yoongsia.png.intel.com) ([10.88.227.38]) by orviesa004.jf.intel.com with ESMTP; 06 Jan 2025 05:57:27 -0800 From: Song Yoong Siang To: "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Willem de Bruijn , Florian Bezdeka , Donald Hunter , Jonathan Corbet , Bjorn Topel , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Andrew Lunn , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Joe Damato , Stanislav Fomichev , Xuan Zhuo , Mina Almasry , Daniel Jurgens , Song Yoong Siang , Amritha Nambiar , Andrii Nakryiko , Eduard Zingerman , Mykola Lysenko , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Hao Luo , Jiri Olsa , Shuah Khan , Alexandre Torgue , Jose Abreu , Maxime Coquelin , Tony Nguyen , Przemek Kitszel Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-stm32@st-md-mailman.stormreply.com, linux-arm-kernel@lists.infradead.org, intel-wired-lan@lists.osuosl.org, xdp-hints@xdp-project.net Subject: [PATCH bpf-next v4 4/4] igc: Add launch time support to XDP ZC Date: Mon, 6 Jan 2025 21:57:24 +0800 Message-Id: <20250106135724.9749-1-yoong.siang.song@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250106_055739_261816_A7A99017 X-CRM114-Status: GOOD ( 16.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Enable Launch Time Control (LTC) support to XDP zero copy via XDP Tx metadata framework. This patch is tested with tools/testing/selftests/bpf/xdp_hw_metadata on Intel Tiger Lake platform. Below are the test steps and result. Test Steps: 1. Add mqprio qdisc: $ sudo tc qdisc add dev enp2s0 handle 8001: parent root mqprio num_tc 4 map 3 2 1 0 0 0 0 0 0 0 0 0 0 0 0 0 queues 1@0 1@1 1@2 1@3 hw 0 2. Enable launch time hardware offload on hardware queue 1: $ sudo tc qdisc replace dev enp2s0 parent 8001:2 etf offload clockid CLOCK_TAI delta 500000 3. Change RSS to route all incoming IP packets into hardware queue 1: $ sudo ethtool -X enp2s0 start 1 equal 1 4. Start xdp_hw_metadata selftest application: $ sudo ./xdp_hw_metadata enp2s0 -l 1000000000 5. Send an UDP packet to port 9091 of DUT. $ echo -n xdp | nc -u -q0 169.254.1.1 9091 When launch time is set to 1s in the future, the delta between launch time and transmit hardware timestamp is equal to 0.016us, as shown in result below: 0x562ff5dc8880: rx_desc[4]->addr=84110 addr=84110 comp_addr=84110 EoP rx_hash: 0xE343384 with RSS type:0x1 HW RX-time: 1734578015467548904 (sec:1734578015.4675) delta to User RX-time sec:0.0002 (183.103 usec) XDP RX-time: 1734578015467651698 (sec:1734578015.4677) delta to User RX-time sec:0.0001 (80.309 usec) No rx_vlan_tci or rx_vlan_proto, err=-95 0x562ff5dc8880: ping-pong with csum=561c (want c7dd) csum_start=34 csum_offset=6 HW RX-time: 1734578015467548904 (sec:1734578015.4675) delta to HW Launch-time sec:1.0000 (1000000.000 usec) 0x562ff5dc8880: complete tx idx=4 addr=4018 HW Launch-time: 1734578016467548904 (sec:1734578016.4675) delta to HW TX-complete-time sec:0.0000 (0.016 usec) HW TX-complete-time: 1734578016467548920 (sec:1734578016.4675) delta to User TX-complete-time sec:0.0000 (32.546 usec) XDP RX-time: 1734578015467651698 (sec:1734578015.4677) delta to User TX-complete-time sec:0.9999 (999929.768 usec) HW RX-time: 1734578015467548904 (sec:1734578015.4675) delta to HW TX-complete-time sec:1.0000 (1000000.016 usec) 0x562ff5dc8880: complete rx idx=132 addr=84110 Signed-off-by: Song Yoong Siang --- drivers/net/ethernet/intel/igc/igc_main.c | 78 ++++++++++++++++------- 1 file changed, 56 insertions(+), 22 deletions(-) diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 27872bdea9bd..6857f5f5b4b2 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -1566,6 +1566,26 @@ static bool igc_request_tx_tstamp(struct igc_adapter *adapter, struct sk_buff *s return false; } +static void igc_insert_empty_packet(struct igc_ring *tx_ring) +{ + struct igc_tx_buffer *empty_info; + struct sk_buff *empty; + void *data; + + empty_info = &tx_ring->tx_buffer_info[tx_ring->next_to_use]; + empty = alloc_skb(IGC_EMPTY_FRAME_SIZE, GFP_ATOMIC); + if (!empty) + return; + + data = skb_put(empty, IGC_EMPTY_FRAME_SIZE); + memset(data, 0, IGC_EMPTY_FRAME_SIZE); + + igc_tx_ctxtdesc(tx_ring, 0, false, 0, 0, 0); + + if (igc_init_tx_empty_descriptor(tx_ring, empty, empty_info) < 0) + dev_kfree_skb_any(empty); +} + static netdev_tx_t igc_xmit_frame_ring(struct sk_buff *skb, struct igc_ring *tx_ring) { @@ -1603,26 +1623,8 @@ static netdev_tx_t igc_xmit_frame_ring(struct sk_buff *skb, skb->tstamp = ktime_set(0, 0); launch_time = igc_tx_launchtime(tx_ring, txtime, &first_flag, &insert_empty); - if (insert_empty) { - struct igc_tx_buffer *empty_info; - struct sk_buff *empty; - void *data; - - empty_info = &tx_ring->tx_buffer_info[tx_ring->next_to_use]; - empty = alloc_skb(IGC_EMPTY_FRAME_SIZE, GFP_ATOMIC); - if (!empty) - goto done; - - data = skb_put(empty, IGC_EMPTY_FRAME_SIZE); - memset(data, 0, IGC_EMPTY_FRAME_SIZE); - - igc_tx_ctxtdesc(tx_ring, 0, false, 0, 0, 0); - - if (igc_init_tx_empty_descriptor(tx_ring, - empty, - empty_info) < 0) - dev_kfree_skb_any(empty); - } + if (insert_empty) + igc_insert_empty_packet(tx_ring); done: /* record the location of the first descriptor for this packet */ @@ -2955,9 +2957,33 @@ static u64 igc_xsk_fill_timestamp(void *_priv) return *(u64 *)_priv; } +static void igc_xsk_request_launch_time(u64 launch_time, void *_priv) +{ + struct igc_metadata_request *meta_req = _priv; + struct igc_ring *tx_ring = meta_req->tx_ring; + __le32 launch_time_offset; + bool insert_empty = false; + bool first_flag = false; + + if (!tx_ring->launchtime_enable) + return; + + launch_time_offset = igc_tx_launchtime(tx_ring, + ns_to_ktime(launch_time), + &first_flag, &insert_empty); + if (insert_empty) { + igc_insert_empty_packet(tx_ring); + meta_req->tx_buffer = + &tx_ring->tx_buffer_info[tx_ring->next_to_use]; + } + + igc_tx_ctxtdesc(tx_ring, launch_time_offset, first_flag, 0, 0, 0); +} + const struct xsk_tx_metadata_ops igc_xsk_tx_metadata_ops = { .tmo_request_timestamp = igc_xsk_request_timestamp, .tmo_fill_timestamp = igc_xsk_fill_timestamp, + .tmo_request_launch_time = igc_xsk_request_launch_time, }; static void igc_xdp_xmit_zc(struct igc_ring *ring) @@ -2980,7 +3006,7 @@ static void igc_xdp_xmit_zc(struct igc_ring *ring) ntu = ring->next_to_use; budget = igc_desc_unused(ring); - while (xsk_tx_peek_desc(pool, &xdp_desc) && budget--) { + while (xsk_tx_peek_desc(pool, &xdp_desc) && budget >= 4) { struct igc_metadata_request meta_req; struct xsk_tx_metadata *meta = NULL; struct igc_tx_buffer *bi; @@ -3004,6 +3030,12 @@ static void igc_xdp_xmit_zc(struct igc_ring *ring) xsk_tx_metadata_request(meta, &igc_xsk_tx_metadata_ops, &meta_req); + /* xsk_tx_metadata_request() may have updated next_to_use */ + ntu = ring->next_to_use; + + /* xsk_tx_metadata_request() may have updated Tx buffer info */ + bi = meta_req.tx_buffer; + tx_desc = IGC_TX_DESC(ring, ntu); tx_desc->read.cmd_type_len = cpu_to_le32(meta_req.cmd_type); tx_desc->read.olinfo_status = cpu_to_le32(olinfo_status); @@ -3021,9 +3053,11 @@ static void igc_xdp_xmit_zc(struct igc_ring *ring) ntu++; if (ntu == ring->count) ntu = 0; + + ring->next_to_use = ntu; + budget = igc_desc_unused(ring); } - ring->next_to_use = ntu; if (tx_desc) { igc_flush_tx_descriptors(ring); xsk_tx_release(pool);