From patchwork Wed Mar 5 14:31:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002823 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9DBC124A07C; Wed, 5 Mar 2025 14:33:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185203; cv=none; b=M5jjRs6Tqn3nxLN6H2YCaIuLX76fgwhLEKdGUKaD1HBBa2F+1g0fldi7RH3ej4qPsMikYJjtHK3CHRIEMHHaovesOFclA/V/J54hf2sIKp7neAP6oGFogjeSDzriPGOrq319Bh7PbIPaXw2FV6BmLBudOllVb7kZgjlz+SRpPuQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185203; c=relaxed/simple; bh=qmVEeJKaSlHB670aWXno8J+LHkCC6GUzC2bJyHLdzAw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=MxXIGtrSa+iJSXG18D1Ymah/8VBjVn9m5+FEWhVTJkU4RVR16e6rW1iX07MJeinQOClmZLEav2AFEdXbuDjMiN+jIuiescMgXj6UPabNqKzZ7CTCqcauaAYP8T44FJ9lYnNqwf1RsxwS+qGE1bTVzY8pU8QsGv5c/9ZiPVWIti4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=ToFLQZdE; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=C6dho88Y; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="ToFLQZdE"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="C6dho88Y" Received: from phl-compute-06.internal (phl-compute-06.phl.internal [10.202.2.46]) by mailfhigh.phl.internal (Postfix) with ESMTP id 84F901140122; Wed, 5 Mar 2025 09:33:19 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-06.internal (MEProxy); Wed, 05 Mar 2025 09:33:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185199; x=1741271599; bh=HrkdRASLIyznwcZXcpFIW79Iwc9mintT sa0Nyk4khtQ=; b=ToFLQZdEx/q/2oldYjrr3eXwtRs0YAVRgoOCGedZi6alyp0U s/I6PS4GS7DLGmuK7oAe0ROURuvPzQY5bBA6XizHouJ6VJWnNgQ4IFD4tdtFvHE3 RvDz46C2+RKDpBKktUVsuMadXgW+PNKzblQj/Vp45PJ4Wa86hI4sjFOamll1M7df ZwiQscLtL8dB7w8xueNiQDPm8IhPPuzjk36xXw3uwpN1J7xGUruFJHH6AF3Gedgr 9OxVMcgG7GkNMJ9LlFHdXN4x4pjNbgHmv3dxDAF0ZO8bC8jGs8JJ7hjwDx6MMcSH tJgag4dJ2ObNLiQYyko/1L8LDBYadErC/n53Pw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185199; x= 1741271599; bh=HrkdRASLIyznwcZXcpFIW79Iwc9mintTsa0Nyk4khtQ=; b=C 6dho88YEoyL3wA2nE1106mc6pt1tDCh5GnyfWkcOxhYObW1EOYViZqfQir7JJS9Z qN/qaQ+WOr2HEGIWoe0nHSEWMuCRl0gK2xoZbdfA0VROLeCMgvJEm1aP+1NAaiyU Y48sZFT3UHxpPo6YQ5QBW3XS8MboDa2sbA3iioR3YEBzuKzv9ffYvRFPbHu4xh/5 aCeTp2my+jfuehMKkjCdYtLfY7DWJ3cHuw0hBdlLxtCs2teVsprEY+SDa7oLXudt caCgh7mm/I8rJfnJHs2mAYh+xriIbFkMnqH626hn2prQOh5594I9ndhCTzkH3TbF nKge6nX48t57BJRwVTQvg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtkeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeefkefhgeevhfeiffegfefgheeijeevgfehfeelhffguefgudelveej geffuefhvdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:17 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:31:58 +0100 Subject: [PATCH RFC bpf-next 01/20] trait: limited KV store for packet metadata Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-1-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre A (very limited) KV store to support storing KVs in the packet headroom, with: - 64 keys (0-63). - 2, 4 or 8 byte values. - O(1) lookup - O(n) insertion - A fixed 16 byte header. Values are stored ordered by key, immediately following the fixed header. This could be extended in the future, for now it implements the smallest possible API. The API intentionally uses u64 keys to not impose restrictions on the implementation in the future. I picked 2ΒΈ 4, and 8 bytes arbitrarily. We could also support 0 sized values for use as flags. A 16 byte value could be useful to store UUIDs and IPv6 addresses. If we want more than 3 sizes, we can add a word to the header (for a total of 24 bytes) to support 7 sizes. We could also allow users to set several consecutive keys in one trait_set() call to support storing larger values. Implemented in the header file so functions are always inlinable. Signed-off-by: Arthur Fabre --- include/net/trait.h | 243 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 243 insertions(+) diff --git a/include/net/trait.h b/include/net/trait.h new file mode 100644 index 0000000000000000000000000000000000000000..536b8a17dbbc091b4d1a4d7b4b21c1e36adea86a --- /dev/null +++ b/include/net/trait.h @@ -0,0 +1,243 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2025 Arthur Fabre, Cloudflare */ +#ifndef __LINUX_NET_TRAIT_H__ +#define __LINUX_NET_TRAIT_H__ + +#include +#include +#include +#include + +/* Traits are a very limited KV store, with: + * - 64 keys (0-63). + * - 2, 4 or 8 byte values. + * - O(1) lookup + * - O(n) insertion + * - A fixed 16 byte header. + */ +struct __trait_hdr { + /* Values are stored ordered by key, immediately after the header. + * + * The size of each value set is stored in the header as two bits: + * - 00: Not set. + * - 01: 2 bytes. + * - 10: 4 bytes. + * - 11: 8 bytes. + * + * Naively storing the size of each value (eg packing 4 of them in a byte) + * doesn't let us efficiently calculate the offset of the value of an arbitrary key: + * we'd have to mask out and sum the sizes of all preceding values. + * + * Instead, store the high and low bits of the size in two separate words: + * - A high bit in the high word. + * - A low bit in the low word. + * The bit position of each word, LSb 0, is the key. + * + * To calculate the offset of the value of a key: + * - Mask out subsequent keys from both words. + * - Calculate the hamming weight / population count of each word. + * Single instruction on modern amd64 CPUs. + * - hweight(low) + hweight(high)<<1 is offset. + */ + u64 high; + u64 low; +}; + +static __always_inline bool __trait_valid_len(u64 len) +{ + return len == 2 || len == 4 || len == 8; +} + +static __always_inline bool __trait_valid_key(u64 key) +{ + return key < 64; +} + +static __always_inline int __trait_total_length(struct __trait_hdr h) +{ + return (hweight64(h.low) << 1) + (hweight64(h.high) << 2) + // For size 8, we only get 4+2=6. Add another 2 in. + + (hweight64(h.high & h.low) << 1); +} + +static __always_inline struct __trait_hdr __trait_and(struct __trait_hdr h, u64 mask) +{ + return (struct __trait_hdr){ + h.high & mask, + h.low & mask, + }; +} + +static __always_inline int __trait_offset(struct __trait_hdr h, u64 key) +{ + /* Calculate total length of previous keys by masking out keys after. */ + return sizeof(struct __trait_hdr) + __trait_total_length(__trait_and(h, ~(~0llu << key))); +} + +/** + * traits_init() - Initialize a trait store. + * @traits: Start of trait store area. + * @hard_end: Hard limit the trait store can currently grow up against. + * Can change dynamically after initialization, as long as it + * does not overwrite any area already used (see traits_size()). + * + * Return: + * * %0 - Success. + * * %-ENOMEM - Not enough room to store any traits. + */ +static __always_inline int traits_init(void *traits, void *hard_end) +{ + if (traits + sizeof(struct __trait_hdr) > hard_end) + return -ENOMEM; + + memset(traits, 0, sizeof(struct __trait_hdr)); + return 0; +} + +/** + * traits_size() - Total size currently used by a trait store. + * @traits: Start of trait store area. + * + * Return: Size in bytes. + */ +static __always_inline int traits_size(void *traits) +{ + return sizeof(struct __trait_hdr) + __trait_total_length(*(struct __trait_hdr *)traits); +} + +/** + * trait_set() - Set a trait key. + * @traits: Start of trait store area. + * @hard_end: Hard limit the trait store can currently grow up against. + * @key: The key to set. + * @val: The value to set. + * @len: The length of the value. + * @flags: Unused for now. Should be 0. + * + * Return: + * * %0 - Success. + * * %-EINVAL - Key or length invalid. + * * %-EBUSY - Key previously set with different length. + * * %-ENOSPC - Not enough room left to store value. + */ +static __always_inline +int trait_set(void *traits, void *hard_end, u64 key, const void *val, u64 len, u64 flags) +{ + if (!__trait_valid_key(key) || !__trait_valid_len(len)) + return -EINVAL; + + struct __trait_hdr *h = (struct __trait_hdr *)traits; + + /* Offset of value of this key. */ + int off = __trait_offset(*h, key); + + if ((h->high & (1ull << key)) || (h->low & (1ull << key))) { + /* Key is already set, but with a different length */ + if (__trait_total_length(__trait_and(*h, (1ull << key))) != len) + return -EBUSY; + } else { + /* Figure out if we have enough room left: total length of everything now. */ + if (traits + sizeof(struct __trait_hdr) + __trait_total_length(*h) + len > hard_end) + return -ENOSPC; + + /* Memmove all the kvs after us over. */ + if (traits_size(traits) > off) + memmove(traits + off + len, traits + off, traits_size(traits) - off); + } + + /* Set our value. */ + memcpy(traits + off, val, len); + + /* Store our length in header. */ + u64 encode_len = 0; + + switch (len) { + case 2: + encode_len = 1; + break; + case 4: + encode_len = 2; + break; + case 8: + encode_len = 3; + break; + } + h->high |= (encode_len >> 1) << key; + h->low |= (encode_len & 1) << key; + return 0; +} + +/** + * trait_get() - Get a trait key. + * @traits: Start of trait store area. + * @key: The key to get. + * @val: Where to put stored value. + * @val_len: The length of val. + * + * Return: + * * %>0 - Actual size of value. + * * %-EINVAL - Key or length invalid. + * * %-ENOENT - Key has not been set with trait_set() previously. + * * %-ENOSPC - Val is not big enough to hold stored value. + */ +static __always_inline +int trait_get(void *traits, u64 key, void *val, u64 val_len) +{ + if (!__trait_valid_key(key)) + return -EINVAL; + + struct __trait_hdr h = *(struct __trait_hdr *)traits; + + /* Check key is set */ + if (!((h.high & (1ull << key)) || (h.low & (1ull << key)))) + return -ENOENT; + + /* Offset of value of this key */ + int off = __trait_offset(h, key); + + /* Figure out our length */ + int real_len = __trait_total_length(__trait_and(h, (1ull << key))); + + if (real_len > val_len) + return -ENOSPC; + + memcpy(val, traits + off, real_len); + return real_len; +} + +/** + * trait_del() - Delete a trait key. + * @traits: Start of trait store area. + * @key: The key to delete. + * + * Return: + * * %0 - Success. + * * %-EINVAL - Key or length invalid. + * * %-ENOENT - Key has not been set with trait_set() previously. + */ +static __always_inline int trait_del(void *traits, u64 key) +{ + if (!__trait_valid_key(key)) + return -EINVAL; + + struct __trait_hdr *h = (struct __trait_hdr *)traits; + + /* Check key is set */ + if (!((h->high & (1ull << key)) || (h->low & (1ull << key)))) + return -ENOENT; + + /* Offset and length of value of this key */ + int off = __trait_offset(*h, key); + int len = __trait_total_length(__trait_and(*h, (1ull << key))); + + /* Memmove all the kvs after us over */ + if (traits_size(traits) > off + len) + memmove(traits + off, traits + off + len, traits_size(traits) - off - len); + + /* Clear our length in header */ + h->high &= ~(1ull << key); + h->low &= ~(1ull << key); + return 0; +} + +#endif /* __LINUX_NET_TRAIT_H__ */ From patchwork Wed Mar 5 14:31:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002824 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fout-a2-smtp.messagingengine.com (fout-a2-smtp.messagingengine.com [103.168.172.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41B85242911; Wed, 5 Mar 2025 14:33:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.145 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185204; cv=none; b=DzCGZRe4crhT5aZPJoa5c9Nu3aJbOSiwXqhUMRub85YJpMNlo0XVmU9wirXW77qsXYawqjqqlNHNe0J4G5ELzlUPmVEEae4KEEeZIrj8+GOpbgC1r41gbcN/90JAC5dorOM3f3FkHYjEwBRrUySD/pi7V6KusdjqYEBs4ceaDE8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185204; c=relaxed/simple; bh=EWejSudzCkg0ro7qEMrk6AxjjLTb7NgqepbrvsLhCDc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=JFxMZe5A2t2YhafVt9Ur9bb1PuDMjvHEUmiJiQGRA3eZ2BifZDpcG7qpgIWvOwn84vFuSJja1YkJrdVNFkbr5UCa0P/ithZSdXNql+wHWdzY5Ug9ClvdKI1o6dU0x1UrLQlOaNobR72IPA8Oh/Oj1qlle1eDzYqC0atrLI63aB4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=bogVGdbs; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=Gqh8aa7V; arc=none smtp.client-ip=103.168.172.145 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="bogVGdbs"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="Gqh8aa7V" Received: from phl-compute-12.internal (phl-compute-12.phl.internal [10.202.2.52]) by mailfout.phl.internal (Postfix) with ESMTP id 5530513826AD; Wed, 5 Mar 2025 09:33:21 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-12.internal (MEProxy); Wed, 05 Mar 2025 09:33:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185201; x=1741271601; bh=Kuu2Ja/7iSsvMscNuv79YgPPg+bC65qw g3QSGHAcr2Q=; b=bogVGdbs+xD4mRgMBy+o9prlxwVXC02zYaSjcrxzMV6qw2fR tPf/axl7BJBtlbhlGgZdjhGPu6Q5Mn6o5+xmkbT7apgfulkA57FRKivzhiWgYR3C JKglBQvgY5PV9qmHtuQ4uiBPOtkQCkPrZSPantAbZGRaRuPtp0OJBVh3jD4A/QyS zknhfsTmrZHfp45sK6BvkvbJ/r4ldnsMBcONpBJDDQEP57D8HDtXdnG6feLug2d/ 6pJCT/pu7wOTe6wxfNIknTJx/ssADxEWaB/tkGV0zQL3LN1WR9ZkbRH/EJfECeO/ e2ax5exFyWO0ykI/Vpv3mb56dkuNh3wtsTDrEg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185201; x= 1741271601; bh=Kuu2Ja/7iSsvMscNuv79YgPPg+bC65qwg3QSGHAcr2Q=; b=G qh8aa7VOD3VYgb6QuxkS8aQymit89Ag5kNBbad6mxilixpjWQvHmU4yFlzNyPfO5 yF0FejXeX+gliLQSU7gweqFLsc1kMBmnZWkbRXduyQYaD1NJv2yWoEpYIC0dZwJy 8EMDqqcj8Nh+U1cHjeN5JImIzI6/XXR1FWX9QC+O5ToAxAme0EI5LQFqvfGErmQb L7Gf0Ukno4/xcvPXr0jQcjaQm8D/3Vvwq8XR37bmITqCDi/8Tkrj1/k5130+Ybt6 lmPZSrADTHFlBO65n2r+ibmn5+IZPVP1ckl+U29rltk/OySezR+X98QNI9cDM2Vk ek9/B7F/w9nSHpFz9ihxA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffleekuefguedtiedtjeeuudejjeegfefgieelffejjefftefgfedu udejheevteenucffohhmrghinhepkhgvrhhnvghlrdhorhhgnecuvehluhhsthgvrhfuih iivgeptdenucfrrghrrghmpehmrghilhhfrhhomheprghrthhhuhhrsegrrhhthhhurhhf rggsrhgvrdgtohhmpdhnsggprhgtphhtthhopeelpdhmohguvgepshhmthhpohhuthdprh gtphhtthhopehthhhoihhlrghnugesrhgvughhrghtrdgtohhmpdhrtghpthhtoheplhgs ihgrnhgtohhnsehrvgguhhgrthdrtghomhdprhgtphhtthhopehhrgifkheskhgvrhhnvg hlrdhorhhgpdhrtghpthhtohepsghpfhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgt phhtthhopegrfhgrsghrvgestghlohhuughflhgrrhgvrdgtohhmpdhrtghpthhtohepjh grkhhusgestghlohhuughflhgrrhgvrdgtohhmpdhrtghpthhtohephigrnhestghlohhu ughflhgrrhgvrdgtohhmpdhrtghpthhtohepnhgvthguvghvsehvghgvrhdrkhgvrhhnvg hlrdhorhhgpdhrtghpthhtohepjhgsrhgrnhguvggsuhhrghestghlohhuughflhgrrhgv rdgtohhm X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:19 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:31:59 +0100 Subject: [PATCH RFC bpf-next 02/20] trait: XDP support Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-2-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre Store the trait KV store in the xdp_buff headroom, right after the xdp_frame. This ensures it can be used at the same time as XDP metadata, runtime checks prevent either one from overlapping. It also avoids having to move the trait KV store on every xdp_adjust_head() call. Support for traits is gated on support for metadata: to propagate the traits to the skb layer, drivers will have to explicitly signal support like the existing skb_metadata_set() call. This also makes bounds checks simpler: we can rely on xdp->data_meta always being a valid hard_end for traits. This assumes there is enough headroom to store the traits header. That avoids having to check in the hot-path, where all we could do anyways is BUG_ON(). Defining _XDP_FRAME_SIZE in skbuff.h seems awkward, but we'll need it to access the traits in the skb layer. Might as well use it now, to avoid having to move the definition of struct xdp_frame. Signed-off-by: Arthur Fabre --- include/linux/skbuff.h | 3 +++ include/net/xdp.h | 20 ++++++++++++++++++++ net/core/filter.c | 7 +++---- net/core/xdp.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 76 insertions(+), 4 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index bb2b751d274acff931281a72e8b4b0c699b4e8af..03553c2200ab1c3750b799edb94fa3b94e11a5f1 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -274,6 +274,9 @@ SKB_DATA_ALIGN(sizeof(struct sk_buff)) + \ SKB_DATA_ALIGN(sizeof(struct skb_shared_info))) +/* From xdp.h, to avoid indirectly including skbuff.h */ +#define _XDP_FRAME_SIZE (40) + struct ahash_request; struct net_device; struct scatterlist; diff --git a/include/net/xdp.h b/include/net/xdp.h index 4dafc5e021f13688f0bf69a21bff58d394d1ac28..58019fa299b56dbd45c104fdfa807f73af6e4fa4 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -10,6 +10,7 @@ #include #include #include /* skb_shared_info */ +#include #include @@ -115,6 +116,11 @@ static __always_inline void xdp_buff_set_frag_pfmemalloc(struct xdp_buff *xdp) xdp->flags |= XDP_FLAGS_FRAGS_PF_MEMALLOC; } +static __always_inline void *xdp_buff_traits(const struct xdp_buff *xdp) +{ + return xdp->data_hard_start + _XDP_FRAME_SIZE; +} + static __always_inline void xdp_init_buff(struct xdp_buff *xdp, u32 frame_sz, struct xdp_rxq_info *rxq) { @@ -133,6 +139,13 @@ xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start, xdp->data = data; xdp->data_end = data + data_len; xdp->data_meta = meta_valid ? data : data + 1; + + if (meta_valid) { + /* We assume drivers reserve enough headroom to store xdp_frame + * and the traits header. + */ + traits_init(xdp_buff_traits(xdp), xdp->data_meta); + } } /* Reserve memory area at end-of data area. @@ -267,6 +280,8 @@ struct xdp_frame { u32 flags; /* supported values defined in xdp_buff_flags */ }; +static_assert(sizeof(struct xdp_frame) == _XDP_FRAME_SIZE); + static __always_inline bool xdp_frame_has_frags(const struct xdp_frame *frame) { return !!(frame->flags & XDP_FLAGS_HAS_FRAGS); @@ -517,6 +532,11 @@ static inline bool xdp_metalen_invalid(unsigned long metalen) return !IS_ALIGNED(metalen, sizeof(u32)) || metalen > meta_max; } +static __always_inline void *xdp_meta_hard_start(const struct xdp_buff *xdp) +{ + return xdp_buff_traits(xdp) + traits_size(xdp_buff_traits(xdp)); +} + struct xdp_attachment_info { struct bpf_prog *prog; u32 flags; diff --git a/net/core/filter.c b/net/core/filter.c index dcc53ac5c5458f67a422453134665d43d466a02e..79b78e7cd57fd78c6cc8443da54ae96408c496b0 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -85,6 +85,7 @@ #include #include #include +#include #include "dev.h" @@ -3935,9 +3936,8 @@ static unsigned long xdp_get_metalen(const struct xdp_buff *xdp) BPF_CALL_2(bpf_xdp_adjust_head, struct xdp_buff *, xdp, int, offset) { - void *xdp_frame_end = xdp->data_hard_start + sizeof(struct xdp_frame); unsigned long metalen = xdp_get_metalen(xdp); - void *data_start = xdp_frame_end + metalen; + void *data_start = xdp_meta_hard_start(xdp) + metalen; void *data = xdp->data + offset; if (unlikely(data < data_start || @@ -4228,13 +4228,12 @@ static const struct bpf_func_proto bpf_xdp_adjust_tail_proto = { BPF_CALL_2(bpf_xdp_adjust_meta, struct xdp_buff *, xdp, int, offset) { - void *xdp_frame_end = xdp->data_hard_start + sizeof(struct xdp_frame); void *meta = xdp->data_meta + offset; unsigned long metalen = xdp->data - meta; if (xdp_data_meta_unsupported(xdp)) return -ENOTSUPP; - if (unlikely(meta < xdp_frame_end || + if (unlikely(meta < xdp_meta_hard_start(xdp) || meta > xdp->data)) return -EINVAL; if (unlikely(xdp_metalen_invalid(metalen))) diff --git a/net/core/xdp.c b/net/core/xdp.c index 2c6ab6fb452f7b90d85125ae17fef96cfc9a8576..2e87f82aa5f835f60295d859a524e40bd47c42ee 100644 --- a/net/core/xdp.c +++ b/net/core/xdp.c @@ -1032,3 +1032,53 @@ void xdp_features_clear_redirect_target(struct net_device *dev) xdp_set_features_flag(dev, val); } EXPORT_SYMBOL_GPL(xdp_features_clear_redirect_target); + +__bpf_kfunc_start_defs(); + +__bpf_kfunc int bpf_xdp_trait_set(const struct xdp_buff *xdp, u64 key, + const void *val, u64 val__sz, u64 flags) +{ + if (xdp_data_meta_unsupported(xdp)) + return -EOPNOTSUPP; + + return trait_set(xdp_buff_traits(xdp), xdp->data_meta, key, + val, val__sz, flags); +} + +__bpf_kfunc int bpf_xdp_trait_get(const struct xdp_buff *xdp, u64 key, + void *val, u64 val__sz) +{ + if (xdp_data_meta_unsupported(xdp)) + return -EOPNOTSUPP; + + return trait_get(xdp_buff_traits(xdp), key, val, val__sz); +} + +__bpf_kfunc int bpf_xdp_trait_del(const struct xdp_buff *xdp, u64 key) +{ + if (xdp_data_meta_unsupported(xdp)) + return -EOPNOTSUPP; + + return trait_del(xdp_buff_traits(xdp), key); +} + +__bpf_kfunc_end_defs(); + +BTF_KFUNCS_START(xdp_trait) +// TODO - should we use KF_TRUSTED_ARGS? https://www.kernel.org/doc/html/next/bpf/kfuncs.html#kf-trusted-args-flag +BTF_ID_FLAGS(func, bpf_xdp_trait_set) +BTF_ID_FLAGS(func, bpf_xdp_trait_get) +BTF_ID_FLAGS(func, bpf_xdp_trait_del) +BTF_KFUNCS_END(xdp_trait) + +static const struct btf_kfunc_id_set xdp_trait_kfunc_set = { + .owner = THIS_MODULE, + .set = &xdp_trait, +}; + +static int xdp_trait_init(void) +{ + return register_btf_kfunc_id_set(BPF_PROG_TYPE_XDP, + &xdp_trait_kfunc_set); +} +late_initcall(xdp_trait_init); From patchwork Wed Mar 5 14:32:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002825 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fout-a2-smtp.messagingengine.com (fout-a2-smtp.messagingengine.com [103.168.172.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E561E24E4AD; Wed, 5 Mar 2025 14:33:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.145 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185205; cv=none; b=V/lhYWbooTnY2u41CoTlrv8QUaINLF1ZmPtxl2Smsnu1yGWf3HR4LqDaWKWMOtT4WwvEF6bP0DejtIA10dgK2u9uNbjg6asRgOMZxXdaBaAKFLBLeNaAo9HyKHc+ynLaCw5KCS1XF9yCRGlTsRTVzzvFCEpnd0ayM25YFC3Y9oc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185205; c=relaxed/simple; bh=dwv28XE92+KFkMambh3tbhhkQP8E/Fx8xkgHETK/91k=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=hHnAEO1uJfP4/ae4ItHCR1PzzF0diLVYdjLKn5uM/WhBruQriHCr+9SIPlLK1o/XutTWFtM3dLJe/4gK+BPcWV0KBQHFRLQuKBaYRUeEjLBUqCPW15PbfQUSA7+1iQR4Is0KmvGbEy1b4FDVa0XRjZOhK02ff6+sbkJDxaj8Se4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=mh3YLQou; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=jwrandt0; arc=none smtp.client-ip=103.168.172.145 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="mh3YLQou"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="jwrandt0" Received: from phl-compute-08.internal (phl-compute-08.phl.internal [10.202.2.48]) by mailfout.phl.internal (Postfix) with ESMTP id F21CA13826E6; Wed, 5 Mar 2025 09:33:22 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-08.internal (MEProxy); Wed, 05 Mar 2025 09:33:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185202; x=1741271602; bh=dAehco09VFBKaKPWvptrONpyGISZqDUV oIUUtWKA6As=; b=mh3YLQousOW4VoHH1hhxC3tEQNUkSugDzPgLvsiyTGS9eYj3 tzQHTCMacN5GjqXX9gFIMjaDLWbBHOXiZZfMWNBwn+CiGMgBY/ijYsJLp7ZBIAFN Mi1OHnGJzzHMZEYSb6+aVS7mKMMqaUX+2nEfRkVzr16rPL44fMLTGxDZ3IvuWm2b i80VjEjkyv/hT4UFZeCLqDI004dqdUGNszfaG1pTf4bNCszTy6Uml3VWQvpZhyep 8AMbB62zKHSWpgP32CSVjHWc/b6OvX8j63uP8M5RKcN1Ej3/2bknMqb5sGo+MZZQ fYNk/IqAMy7yhlWHti/n+kSeSAaxxM28OUBl8w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185202; x= 1741271602; bh=dAehco09VFBKaKPWvptrONpyGISZqDUVoIUUtWKA6As=; b=j wrandt0A1YR84+H28Rvvs6CWeIycZsMHsjWD2OI77x/FrcyATKsBjLz3Y+1aH5Hq pOU4AoR1fL6zL5FH/AmNKOaVlxq97VaP+aGaWXzpdFZhLHkOQBV8n2GYcMDwYl+d QoMpBm835e7uBic24UBLR9pBsC7EjIe0AzyH/vieqNUFxXa9xfzH+e9MIhUOeNC1 ePndSE1QdVinNqX1RjCeEb129nPjIH8eMTJd/NIyymerWXT+6ZlR2UjmM34qB++F Hez6ng4TsHvOpBvWYbAxvjSjoRHgWmrZbViIM/MsqyUGdy5ddtrWKlzzHyjxySgu R3IrvPNSUB7VdErotTHrg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:21 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:00 +0100 Subject: [PATCH RFC bpf-next 03/20] trait: basic XDP selftest Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-3-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre Doesn't test interop with the "data_meta" area yet, or error cases. Signed-off-by: Arthur Fabre --- .../testing/selftests/bpf/prog_tests/xdp_traits.c | 35 ++++++++ .../testing/selftests/bpf/progs/test_xdp_traits.c | 94 ++++++++++++++++++++++ 2 files changed, 129 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_traits.c b/tools/testing/selftests/bpf/prog_tests/xdp_traits.c new file mode 100644 index 0000000000000000000000000000000000000000..4175b28d45e91e82435e646e5edd783980d5fe70 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/xdp_traits.c @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include + +static void _test_xdp_traits(void) +{ + const char *file = "./test_xdp_traits.bpf.o"; + struct bpf_object *obj; + int err, prog_fd; + char buf[128]; + + LIBBPF_OPTS(bpf_test_run_opts, topts, + .data_in = &pkt_v4, + .data_size_in = sizeof(pkt_v4), + .data_out = buf, + .data_size_out = sizeof(buf), + .repeat = 1, + ); + + err = bpf_prog_test_load(file, BPF_PROG_TYPE_XDP, &obj, &prog_fd); + if (!ASSERT_OK(err, "test_xdp_traits")) + return; + + err = bpf_prog_test_run_opts(prog_fd, &topts); + ASSERT_OK(err, "prog_run"); + ASSERT_EQ(topts.retval, XDP_PASS, "retval"); + + bpf_object__close(obj); +} + +void test_xdp_traits(void) +{ + if (test__start_subtest("xdp_traits")) + _test_xdp_traits(); +} diff --git a/tools/testing/selftests/bpf/progs/test_xdp_traits.c b/tools/testing/selftests/bpf/progs/test_xdp_traits.c new file mode 100644 index 0000000000000000000000000000000000000000..79e0b0dedb5c05792579861975e8caf290f5f42b --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_xdp_traits.c @@ -0,0 +1,94 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +extern int bpf_xdp_trait_set(const struct xdp_md *xdp, __u64 key, + const void *val, __u64 val__sz, + __u64 flags) __ksym; +extern int bpf_xdp_trait_get(const struct xdp_md *xdp, __u64 key, void *val, + __u64 val__sz) __ksym; +extern int bpf_xdp_trait_del(const struct xdp_md *xdp, __u64 key) __ksym; + +SEC("xdp") +int _xdp_traits(struct xdp_md *xdp) +{ + int ret; + __u16 val, got, want; + + // No keys to start. + for (int i = 0; i < 64; i++) { + ret = bpf_xdp_trait_get(xdp, i, &val, sizeof(val)); + if (ret != -ENOENT) { + bpf_printk("get(%d) ret %d", i, ret); + return XDP_DROP; + } + } + + // Set 64 2 byte KVs. + for (int i = 0; i < 64; i++) { + val = i << 8 | i; + ret = bpf_xdp_trait_set(xdp, i, &val, sizeof(val), 0); + if (ret < 0) { + bpf_printk("set(%d) ret %d\n", i, ret); + return XDP_DROP; + } + bpf_printk("set(%d, 0x%04x)\n", i, val); + } + + // Check we can get the 64 2 byte KVs back out. + for (int i = 0; i < 64; i++) { + ret = bpf_xdp_trait_get(xdp, i, &got, sizeof(got)); + if (ret != 2) { + bpf_printk("get(%d) ret %d", i, ret); + return XDP_DROP; + } + want = (i << 8) | i; + if (got != want) { + bpf_printk("get(%d) got 0x%04x want 0x%04x\n", i, got, + want); + return XDP_DROP; + } + bpf_printk("get(%d) 0x%04x\n", i, got); + } + + // Overwrite all 64 2 byte KVs. + for (int i = 0; i < 64; i++) { + val = i << 9 | i << 1; + ret = bpf_xdp_trait_set(xdp, i, &val, sizeof(val), 0); + if (ret < 0) { + bpf_printk("set(%d) ret %d\n", i, ret); + return XDP_DROP; + } + bpf_printk("set(%d, 0x%04x)\n", i, val); + } + + // Delete all the even KVs. + for (int i = 0; i < 64; i += 2) { + ret = bpf_xdp_trait_del(xdp, i); + if (ret < 0) { + bpf_printk("del(%d) ret %d\n", i, ret); + return XDP_DROP; + } + } + + // Read out all the odd KVs again. + for (int i = 1; i < 63; i += 2) { + ret = bpf_xdp_trait_get(xdp, i, &got, sizeof(got)); + if (ret != 2) { + bpf_printk("get(%d) ret %d", i, ret); + return XDP_DROP; + } + want = (i << 9) | i << 1; + if (got != want) { + bpf_printk("get(%d) got 0x%04x want 0x%04x\n", i, got, + want); + return XDP_DROP; + } + bpf_printk("get(%d) 0x%04x\n", i, got); + } + + return XDP_PASS; +} + +char _license[] SEC("license") = "GPL"; From patchwork Wed Mar 5 14:32:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002826 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fout-a2-smtp.messagingengine.com (fout-a2-smtp.messagingengine.com [103.168.172.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0E9E24EAB6; Wed, 5 Mar 2025 14:33:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.145 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185207; cv=none; b=mVl0DHMEMtUgHtxhVkMbASLXgbrE8U/b0goKWd+HK73bZTId6Up1v9UpyJYziek1ZQ9ZxAHbd9Fm4X0Z8SUe9Tsft2MRbutbEOI9Zl0Av8vaAD+cwLDGPpcpUZkMAQ+xnfMvHTbfKkRgLIw8R/byKHleXfe6hFidwAITeq+B8nk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185207; c=relaxed/simple; bh=KczKMFMXFRk+1JUF9WxMHIQEBSxpXI06Ae/1D2VS6n0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=BjhhdWfzOwd4SFZx9DnPfvQzLVzK5fG/K7yLJG+MXMtxe1f5MEpBVhGlpJT565GQdlgFopdDc7mahb9QZ0RrMnX1bcNMNUFikZ0ssl9TQyH+cRUviXhXSznZ3yU72ebXEqXPoIxYnqA/KIn3y+w/rfV2gqLpa37p8URZer0K4sc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=viztjMbF; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=dY5RhGJo; arc=none smtp.client-ip=103.168.172.145 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="viztjMbF"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="dY5RhGJo" Received: from phl-compute-05.internal (phl-compute-05.phl.internal [10.202.2.45]) by mailfout.phl.internal (Postfix) with ESMTP id D654813810B2; Wed, 5 Mar 2025 09:33:24 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-05.internal (MEProxy); Wed, 05 Mar 2025 09:33:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185204; x=1741271604; bh=MzgzHgd06C01r695iBlGOIJt3yonQRko SowreVZfXbA=; b=viztjMbF57g689QIrsYfhTdI33HYEJ6h0qCc+6NbvncfHFF+ lQ5U+nSEyQLX3lIayPaFSHeZDxMOhgQzS8jyglapafmQA8CNfyzJV7GRRTgu24+B gcT4/nSLNTvNcuXo58niUjL5h6Aa8rVEIz14jdLRxoTHA6kScsXFW7gOqxB5NW5x zDMyfwBNf4SoUvCn9PNvAiPx3oQKMrid0WZFqoD4g16Sar0KCwmOgn3uVPdCEgEp 1wQ8i4RTrywgkwS6yC/wCQuNjTqxesYAWbv/B2nimF3F4USbPZKJwmyTjlj3eLDj Z9UouAdfgvnzsn4ieWlhgXEbKuWzlD/9VUXZpA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185204; x= 1741271604; bh=MzgzHgd06C01r695iBlGOIJt3yonQRkoSowreVZfXbA=; b=d Y5RhGJorN1SoSWU3qMtbMTiB55nkdkKZ/va4eOpHLCW7P5IPJCZzwYqO0+WReXZJ 1748qlZGhZO8UczyUPw0/0S6vksOOJjaotx6Mfd2aLCzJ9thsknlTM/2dvxMM5sv oOXe3CEOOB0gqK7p2h9Fo3IHiUnrHzolsdYNRhpVithwZPP6YlkHSWt2acdRiqhx 1psoPmqqrrrYsFCmGkaaLrvzER5tkzCUykD5kNTh+l3nwz8R8UnBTgJOR90s5S/t X7o0u31vavvqCM2Vn4FsbYhqhViQTHKV5jn/Gpz8ptY9R4KFWRqfoCb0EQB/uQqp 1CYlZQqvusuXKvKRPOiVA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:23 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:01 +0100 Subject: [PATCH RFC bpf-next 04/20] trait: basic XDP benchmark Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-4-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre Basic benchmarks for: * Getting a trait. * Setting a trait, with no traits already stored after it. * Moving traits, when setting or deleting a trait with traits stored after it. Signed-off-by: Arthur Fabre --- tools/testing/selftests/bpf/Makefile | 2 + tools/testing/selftests/bpf/bench.c | 8 ++ .../selftests/bpf/benchs/bench_xdp_traits.c | 160 +++++++++++++++++++++ .../testing/selftests/bpf/progs/bench_xdp_traits.c | 131 +++++++++++++++++ 4 files changed, 301 insertions(+) diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index e6a02d5b87d123cef7e6b41bfbc96d34083f38e1..6b7a7351664cf611ce72fb76b308b7392e3811ba 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -815,6 +815,7 @@ $(OUTPUT)/bench_local_storage_create.o: $(OUTPUT)/bench_local_storage_create.ske $(OUTPUT)/bench_bpf_hashmap_lookup.o: $(OUTPUT)/bpf_hashmap_lookup.skel.h $(OUTPUT)/bench_htab_mem.o: $(OUTPUT)/htab_mem_bench.skel.h $(OUTPUT)/bench_bpf_crypto.o: $(OUTPUT)/crypto_bench.skel.h +$(OUTPUT)/bench_xdp_traits.o: $(OUTPUT)/bench_xdp_traits.skel.h $(OUTPUT)/bench.o: bench.h testing_helpers.h $(BPFOBJ) $(OUTPUT)/bench: LDLIBS += -lm $(OUTPUT)/bench: $(OUTPUT)/bench.o \ @@ -835,6 +836,7 @@ $(OUTPUT)/bench: $(OUTPUT)/bench.o \ $(OUTPUT)/bench_local_storage_create.o \ $(OUTPUT)/bench_htab_mem.o \ $(OUTPUT)/bench_bpf_crypto.o \ + $(OUTPUT)/bench_xdp_traits.o \ # $(call msg,BINARY,,$@) $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $(filter %.a %.o,$^) $(LDLIBS) -o $@ diff --git a/tools/testing/selftests/bpf/bench.c b/tools/testing/selftests/bpf/bench.c index 1bd403a5ef7b3401f501d790453c7be9ed289cb5..4678b928fc6ad2f0a870a25d9b10c75a1f6d77ba 100644 --- a/tools/testing/selftests/bpf/bench.c +++ b/tools/testing/selftests/bpf/bench.c @@ -283,6 +283,7 @@ extern struct argp bench_local_storage_create_argp; extern struct argp bench_htab_mem_argp; extern struct argp bench_trigger_batch_argp; extern struct argp bench_crypto_argp; +extern struct argp bench_trait_argp; static const struct argp_child bench_parsers[] = { { &bench_ringbufs_argp, 0, "Ring buffers benchmark", 0 }, @@ -297,6 +298,7 @@ static const struct argp_child bench_parsers[] = { { &bench_htab_mem_argp, 0, "hash map memory benchmark", 0 }, { &bench_trigger_batch_argp, 0, "BPF triggering benchmark", 0 }, { &bench_crypto_argp, 0, "bpf crypto benchmark", 0 }, + { &bench_trait_argp, 0, "XDP trait benchmark", 0 }, {}, }; @@ -549,6 +551,9 @@ extern const struct bench bench_local_storage_create; extern const struct bench bench_htab_mem; extern const struct bench bench_crypto_encrypt; extern const struct bench bench_crypto_decrypt; +extern const struct bench bench_xdp_trait_get; +extern const struct bench bench_xdp_trait_set; +extern const struct bench bench_xdp_trait_move; static const struct bench *benchs[] = { &bench_count_global, @@ -609,6 +614,9 @@ static const struct bench *benchs[] = { &bench_htab_mem, &bench_crypto_encrypt, &bench_crypto_decrypt, + &bench_xdp_trait_get, + &bench_xdp_trait_set, + &bench_xdp_trait_move, }; static void find_benchmark(void) diff --git a/tools/testing/selftests/bpf/benchs/bench_xdp_traits.c b/tools/testing/selftests/bpf/benchs/bench_xdp_traits.c new file mode 100644 index 0000000000000000000000000000000000000000..0fbcd49edd825f53e6957319d3f05efc218dfb02 --- /dev/null +++ b/tools/testing/selftests/bpf/benchs/bench_xdp_traits.c @@ -0,0 +1,160 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include "bench.h" +#include "bench_xdp_traits.skel.h" + +static struct trait_ctx { + struct bench_xdp_traits *skel; + int pfd; +} ctx; + +static struct trait_args { + u32 trait_len; +} args = { + .trait_len = 2, +}; + +enum { + ARG_TRAIT_LEN = 6000, +}; + +static const struct argp_option opts[] = { + { "trait-len", ARG_TRAIT_LEN, "TRAIT_LEN", 0, + "Set the length of the trait set/get" }, + {}, +}; + +static error_t trait_parse_arg(int key, char *arg, struct argp_state *state) +{ + switch (key) { + case ARG_TRAIT_LEN: + args.trait_len = strtoul(arg, NULL, 10); + if (args.trait_len != 2 && args.trait_len != 4 && args.trait_len != 8) { + fprintf(stderr, "Invalid trait length\n"); + argp_usage(state); + } + break; + default: + return ARGP_ERR_UNKNOWN; + } + + return 0; +} + +const struct argp bench_trait_argp = { + .options = opts, + .parser = trait_parse_arg, +}; + +static void trait_validate(void) +{ + if (env.consumer_cnt != 0) { + fprintf(stderr, "bpf trait benchmark doesn't support consumer!\n"); + exit(1); + } +} + +static void trait_setup(void) +{ + int err; + + setup_libbpf(); + + ctx.skel = bench_xdp_traits__open(); + if (!ctx.skel) { + fprintf(stderr, "failed to open skeleton\n"); + exit(1); + } + + ctx.skel->bss->trait_len = args.trait_len; + + err = bench_xdp_traits__load(ctx.skel); + if (err) { + fprintf(stderr, "failed to load skeleton\n"); + bench_xdp_traits__destroy(ctx.skel); + exit(1); + } +} + +static void trait_get_setup(void) +{ + trait_setup(); + ctx.pfd = bpf_program__fd(ctx.skel->progs.trait_get); +} + +static void trait_set_setup(void) +{ + trait_setup(); + ctx.pfd = bpf_program__fd(ctx.skel->progs.trait_set); +} + +static void trait_move_setup(void) +{ + trait_setup(); + ctx.pfd = bpf_program__fd(ctx.skel->progs.trait_move); +} + +static void trait_measure(struct bench_res *res) +{ + res->hits = atomic_swap(&ctx.skel->bss->hits, 0); +} + +static void *trait_producer(void *unused) +{ + static char in[14]; + int err; + + LIBBPF_OPTS(bpf_test_run_opts, opts, + .data_in = in, + .data_size_in = sizeof(in), + .repeat = 256, // max + ); + + while (true) { + err = bpf_prog_test_run_opts(ctx.pfd, &opts); + if (err != 0) { + fprintf(stderr, "failed to prog_run: %d\n", err); + return NULL; + } + if (opts.retval != 0) { + fprintf(stderr, "prog didn't return 0: %d\n", opts.retval); + return NULL; + } + } + + return NULL; +} + +const struct bench bench_xdp_trait_get = { + .name = "xdp-trait-get", + .argp = &bench_trait_argp, + .validate = trait_validate, + .setup = trait_get_setup, + .producer_thread = trait_producer, + .measure = trait_measure, + .report_progress = ops_report_progress, + .report_final = ops_report_final, +}; + +const struct bench bench_xdp_trait_set = { + .name = "xdp-trait-set", + .argp = &bench_trait_argp, + .validate = trait_validate, + .setup = trait_set_setup, + .producer_thread = trait_producer, + .measure = trait_measure, + .report_progress = ops_report_progress, + .report_final = ops_report_final, +}; + +const struct bench bench_xdp_trait_move = { + .name = "xdp-trait-move", + .argp = &bench_trait_argp, + .validate = trait_validate, + .setup = trait_move_setup, + .producer_thread = trait_producer, + .measure = trait_measure, + .report_progress = ops_report_progress, + .report_final = ops_report_final, +}; diff --git a/tools/testing/selftests/bpf/progs/bench_xdp_traits.c b/tools/testing/selftests/bpf/progs/bench_xdp_traits.c new file mode 100644 index 0000000000000000000000000000000000000000..558c1ab22e09990d3eb0f78f916fd02de3def749 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/bench_xdp_traits.c @@ -0,0 +1,131 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +extern int bpf_xdp_trait_set(const struct xdp_md *xdp, __u64 key, + const void *val, __u64 val__sz, + __u64 flags) __ksym; +extern int bpf_xdp_trait_get(const struct xdp_md *xdp, __u64 key, void *val, + __u64 val__sz) __ksym; +extern int bpf_xdp_trait_del(const struct xdp_md *xdp, __u64 key) __ksym; + +__u32 trait_len; +long hits = 0; + +#define ITERATIONS (10000) + +SEC("xdp") +int trait_get(struct xdp_md *xdp) +{ + int ret, i; + __u16 val2 = 0xdead; + __u32 val4 = 0xdeadbeef; + __u64 val8 = 0xdeadbeefafafcfcf; + +#define BENCH_GET(val, size) do { \ + ret = bpf_xdp_trait_set(xdp, 32, &val, sizeof(val), 0); \ + if (ret != 0) \ + return ret; \ + for (i = 0; i < ITERATIONS; i++) \ + ret = bpf_xdp_trait_get(xdp, 32, &val, sizeof(val)); \ + if (ret != size) \ + return ret; \ + } while (0) + + switch (trait_len) { + case 2: + BENCH_GET(val2, 2); + break; + case 4: + BENCH_GET(val4, 4); + break; + case 8: + BENCH_GET(val8, 8); + break; + } + + __sync_add_and_fetch(&hits, ITERATIONS); + return 0; +} + +SEC("xdp") +int trait_set(struct xdp_md *xdp) +{ + int ret, i; + __u16 val2 = 0xdead; + __u32 val4 = 0xdeadbeef; + __u64 val8 = 0xdeadbeefafafcfcf; + +#define BENCH_SET(val) do { \ + for (i = 0; i < ITERATIONS; i++) \ + ret = bpf_xdp_trait_set(xdp, 32, &val, sizeof(val), 0); \ + } while (0) + + switch (trait_len) { + case 2: + BENCH_SET(val2); + break; + case 4: + BENCH_SET(val4); + break; + case 8: + BENCH_SET(val8); + break; + } + + if (ret != 0) + return ret; + + __sync_add_and_fetch(&hits, ITERATIONS); + return 0; +} + +SEC("xdp") +int trait_move(struct xdp_md *xdp) +{ + int ret, ret_del, i; + __u16 val2 = 0xdead; + __u32 val4 = 0xdeadbeef; + __u64 val8 = 0xdeadbeefafafcfcf; + +#define BENCH_MOVE(val) do { \ + for (i = 0; i < 8; i++) { \ + ret = bpf_xdp_trait_set(xdp, 40+i, &val8, sizeof(val8), 0); \ + if (ret != 0) \ + return ret; \ + } \ + /* We do two operations per iteration, so do half as many to make it + * vaguely comparable with other benchmarks + */ \ + for (i = 0; i < ITERATIONS/2; i++) { \ + /* Need to delete after, otherwise we'll just overwrite an + * existing trait and there will be nothing to move. + */ \ + ret = bpf_xdp_trait_set(xdp, 32, &val, sizeof(val), 0); \ + ret_del = bpf_xdp_trait_del(xdp, 32); \ + } \ + } while (0) + + switch (trait_len) { + case 2: + BENCH_MOVE(val2); + break; + case 4: + BENCH_MOVE(val4); + break; + case 8: + BENCH_MOVE(val8); + break; + } + + if (ret != 0) + return ret; + if (ret_del != 0) + return ret_del; + + __sync_add_and_fetch(&hits, ITERATIONS); + return 0; +} + +char _license[] SEC("license") = "GPL"; From patchwork Wed Mar 5 14:32:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002827 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9251724EA95; Wed, 5 Mar 2025 14:33:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185209; cv=none; b=LsxT89jole9hNBH8VxAww3swKW4+IEdbobumve+SXj5XvhbGh6xrRS7vMHK11c79BzY0uhZLfhvTprlteuttNpWsAM45Z4WlFJaZ63VoOeWnXPJx1lvrFhnAC+KJLB8ZPbgBTo69BhPQ0YVuUYXW9vHaUaewBGYMZHGbNOTziDc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185209; c=relaxed/simple; bh=qxv1LsAKRqsKUxDL/Dxta7IuAJCwzGHmYrXlzUsT8vc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=aLZIPElRYk1D1TcSyvI3jJ0xHBo4fmgdpuGDZBMmwV6gAjaYzDFxL4e5YdYieWtkTfH2uAaPJwbMogxWsTgS0re19ZjuY2zNcJtatVo/q2SYYap7qXOUEmLkmhlSnn7CSd85FqvoImdbx9eHLckzHCsyOLRuWEKHggJGSRZEtT4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=bbbDaCzR; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=x96j8RKd; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="bbbDaCzR"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="x96j8RKd" Received: from phl-compute-08.internal (phl-compute-08.phl.internal [10.202.2.48]) by mailfhigh.phl.internal (Postfix) with ESMTP id 9B1051140210; Wed, 5 Mar 2025 09:33:26 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-08.internal (MEProxy); Wed, 05 Mar 2025 09:33:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185206; x=1741271606; bh=qelSNgB8wnspupq2rYPj+q9VuLNr5Cac v5miRIDC3NU=; b=bbbDaCzRKMzXPakKx1VApcuRpbKYY7Xt0EBXmHIF21TK8LRL z8MI62ykQ2Ll90xh1oC3gCEA0IsSzIT+nz0ldhHTUtD/fd4lL0KG2iN7/5g5Iyyl pTYGyz3mnuarquMnIsRD0t33+7Bc6hTPKxMR8hPEXOqJz1D+6Cg4+TvoMOp7dPT4 pDvojuL8XAB1Bo5+di/E2S4tAeA6GLI5n0DO+K22NYXL9r7HIMjCUC/+EfwEfHjb DT7WTeA5rMn5C1lhsPFw77KqEbCBhvplFXFbbzRM5YQL73SwW7RefKEM42GsmdI5 EKxRzCz4KGM+aTwVZrvinkTTd65qd2XQIvtr5w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185206; x= 1741271606; bh=qelSNgB8wnspupq2rYPj+q9VuLNr5Cacv5miRIDC3NU=; b=x 96j8RKdl54gaKPLioY7H/qxkoVMGuSc85WKZ5jMKBLdP91jq29aRT02tdRJg3hyg u2ckM39vq7m2pT226KqXqLYiBB8cvpY9+fa8s5kPPdBWzYZsPQmhWdxqfOCXsijS Cg2GjkQrj3yZy5RiQcIBNcPB1oetDZZAyq+biFoBDoEpXhKRalTsHb+DzXTdOELZ 8gU487rQ0yiLXDlj4HSjrJNFMxMevEuHATgmnSw2BpNAuXJM0OC8v8CJ5Rl+nEfr oRZjHe8Uv8RF4IJzKGCsh4338dfKgExrXfZRDjbOBJ0ynfpRkcMw4MqlT80KB4nK BDySLTtXA6YFlpEwAdIiw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:24 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:02 +0100 Subject: [PATCH RFC bpf-next 05/20] trait: Replace memcpy calls with inline copies Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-5-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre When copying trait values to or from the caller, the size isn't a constant so memcpy() ends up being a function call. Replace it with an inline implementation that only handles the sizes we support. We store values "packed", so they won't necessarily be 4 or 8 byte aligned. Setting and getting traits is roughly ~40% faster. Signed-off-by: Arthur Fabre --- include/net/trait.h | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/include/net/trait.h b/include/net/trait.h index 536b8a17dbbc091b4d1a4d7b4b21c1e36adea86a..d4581a877bd57a32e2ad032147c906764d6d37f8 100644 --- a/include/net/trait.h +++ b/include/net/trait.h @@ -7,6 +7,7 @@ #include #include #include +#include /* Traits are a very limited KV store, with: * - 64 keys (0-63). @@ -145,23 +146,23 @@ int trait_set(void *traits, void *hard_end, u64 key, const void *val, u64 len, u memmove(traits + off + len, traits + off, traits_size(traits) - off); } - /* Set our value. */ - memcpy(traits + off, val, len); - - /* Store our length in header. */ u64 encode_len = 0; - switch (len) { case 2: + /* Values are least two bytes, so they'll be two byte aligned */ + *(u16 *)(traits + off) = *(u16 *)val; encode_len = 1; break; case 4: + put_unaligned(*(u32 *)val, (u32 *)(traits + off)); encode_len = 2; break; case 8: + put_unaligned(*(u64 *)val, (u64 *)(traits + off)); encode_len = 3; break; } + h->high |= (encode_len >> 1) << key; h->low |= (encode_len & 1) << key; return 0; @@ -201,7 +202,19 @@ int trait_get(void *traits, u64 key, void *val, u64 val_len) if (real_len > val_len) return -ENOSPC; - memcpy(val, traits + off, real_len); + switch (real_len) { + case 2: + /* Values are least two bytes, so they'll be two byte aligned */ + *(u16 *)val = *(u16 *)(traits + off); + break; + case 4: + *(u32 *)val = get_unaligned((u32 *)(traits + off)); + break; + case 8: + *(u64 *)val = get_unaligned((u64 *)(traits + off)); + break; + } + return real_len; } From patchwork Wed Mar 5 14:32:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002828 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fout-a2-smtp.messagingengine.com (fout-a2-smtp.messagingengine.com [103.168.172.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3284624EF77; Wed, 5 Mar 2025 14:33:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.145 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185210; cv=none; b=lbj/YIBjhYUNgIgiZnqi2BsM7rgU8nBzd4aGBunLBTTJa87vY0uymz8Cdxz2+NnpuQG7YQQvtcs3CajpXC3vMjWO/vH7xrXiJO9imDqqs1UJwgDEwRaOsefoU9rmNQENvTTTcX+WtsZbe4dRZ4+5b529ewq27uKKq0WYzYPEWvo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185210; c=relaxed/simple; bh=AXqsGUMZjJ3nfR3/zpLp0QNglAVCz66tV2016FOkt0w=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ueEbcBPSZlMgLWt7Q8Rcxx3Sx8fHOIe3gEEaNV8ylGSePdJn0JDnq4NNo2iWHDaN69XMqYdxDUkK6ylGBrOX21qmYVHc3iyx8krMH2B+25T8/Z3+Myl+ZvsHcPqVF0VhRtoY+ESJM1cwEu0JRtu//TwUQCGk+0EZpYYY7dOasig= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=SOHan5ky; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=DANHmaR9; arc=none smtp.client-ip=103.168.172.145 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="SOHan5ky"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="DANHmaR9" Received: from phl-compute-06.internal (phl-compute-06.phl.internal [10.202.2.46]) by mailfout.phl.internal (Postfix) with ESMTP id 405DF13826E8; Wed, 5 Mar 2025 09:33:28 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-06.internal (MEProxy); Wed, 05 Mar 2025 09:33:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185208; x=1741271608; bh=fNt401R7YNbzRDUFKtOB/YgJSARNk28u a1w/BTuFA1g=; b=SOHan5kyO0cEfhLA6U9WbjJxxREzaRlc51s56z3R1Vhcr0Vs Fgtw0o72cDdIwNWvRfbT/AtyL7OSn6Lo2boyrZDaECoOHuesQmdkhz+P1R0819NV ylIDnXCQG9AYot9O/by4zArKlHgCONzQVUrv1uzMw7QSuOlRALumrII2pTdcL0L/ ist4KkmR7pGtl1BcRsB/JJ0lgDSynzsblJzis8zfkDE3aTd7lwmo1hcQVahi2Gzk LY5GYo6B046DqoRTfH/SVMaHFrbuRBpjFNiq28Cnox8xWnWQp609Uyey3+r47DhO OWj6ocA82X3vCdnb01jIRR3nCvv4YL/f/wzKlw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185208; x= 1741271608; bh=fNt401R7YNbzRDUFKtOB/YgJSARNk28ua1w/BTuFA1g=; b=D ANHmaR9f0SH8udbtIKois+MoYtoMiNftseEsCiNTBXzgrO56IwP225iDkIGdgxkg 9TBFV+KU2JRtklbqMl6kM7hfcDl9K1eFk4x9rD02O0qCLsfwbPmYTqNdv1Ol18m2 HI3ArQZPJpLBp4EOZZ2pabd0DARhyl+YdKj6MWffDqOfb909wxmujIdgj5A1j6G8 MAj280qMEJWznwwUwpGnrQrIBjA6925o7Fxy5bbaDHI5ea7PvGriupMjVs+mbiFp ayaG0Q2IXgu6lDOxp+IcLO08ol3BAThdk5Fr5IM7c/fFX8IxP+/l44UISuuAoiKO 61Q0Tc3/k39AWKO3oLVmw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:26 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:03 +0100 Subject: [PATCH RFC bpf-next 06/20] trait: Replace memmove calls with inline move Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-6-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre When inserting or deleting traits, we need to move any subsequent traits over. Replace it with an inline implementation to avoid the function call overhead. This is especially expensive on AMD with SRSO. In practice we shouldn't have too much data to move around, and we're naturally limited to 238 bytes max, so a dumb implementation should hopefully be fast enough. Jesper Brouer kindly ran benchmarks on real hardware with three configs: - Intel: E5-1650 v4 - AMD SRSO: 9684X SRSO - AMD IBPB: 9684X SRSO=IBPB Intel AMD IBPB AMD SRSO xdp-trait-get 5.530 3.901 9.188 (ns/op) xdp-trait-set 7.538 4.941 10.050 (ns/op) xdp-trait-move 14.245 8.865 14.834 (ns/op) function call 1.319 1.359 5.703 (ns/op) indirect call 8.922 6.251 10.329 (ns/op) Signed-off-by: Arthur Fabre --- include/net/trait.h | 40 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 36 insertions(+), 4 deletions(-) diff --git a/include/net/trait.h b/include/net/trait.h index d4581a877bd57a32e2ad032147c906764d6d37f8..111ce766345cef45fd95e562eda6a7d01c6b76da 100644 --- a/include/net/trait.h +++ b/include/net/trait.h @@ -75,6 +75,40 @@ static __always_inline int __trait_offset(struct __trait_hdr h, u64 key) return sizeof(struct __trait_hdr) + __trait_total_length(__trait_and(h, ~(~0llu << key))); } +/* Avoid overhead of memmove() function call when possible. */ +static __always_inline void __trait_move(void *src, int off, size_t n) +{ + if (n == 0) + return; + + if (!IS_ENABLED(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) || BITS_PER_LONG != 64) { + memmove(src + off, src, n); + return; + } + + /* Need to move in reverse to handle overlap. */ + if (off > 0) + src += n; + +#define ___trait_move(op) do { \ + src -= (off > 0) ? sizeof(u##op) : 0; \ + *(u##op *)(src + off) = *(u##op *)src; \ + src += (off < 0) ? sizeof(u##op) : 0; \ + } while (0) + + for (int w = 0; w < n / 8; w++) + ___trait_move(64); + + if (n & 4) + ___trait_move(32); + + if (n & 2) + ___trait_move(16); + + if (n & 1) + ___trait_move(8); +} + /** * traits_init() - Initialize a trait store. * @traits: Start of trait store area. @@ -142,8 +176,7 @@ int trait_set(void *traits, void *hard_end, u64 key, const void *val, u64 len, u return -ENOSPC; /* Memmove all the kvs after us over. */ - if (traits_size(traits) > off) - memmove(traits + off + len, traits + off, traits_size(traits) - off); + __trait_move(traits + off, len, traits_size(traits) - off); } u64 encode_len = 0; @@ -244,8 +277,7 @@ static __always_inline int trait_del(void *traits, u64 key) int len = __trait_total_length(__trait_and(*h, (1ull << key))); /* Memmove all the kvs after us over */ - if (traits_size(traits) > off + len) - memmove(traits + off, traits + off + len, traits_size(traits) - off - len); + __trait_move(traits + off + len, -len, traits_size(traits) - off - len); /* Clear our length in header */ h->high &= ~(1ull << key); From patchwork Wed Mar 5 14:32:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002829 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB42E24F594; Wed, 5 Mar 2025 14:33:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185212; cv=none; b=IRLvRmqXm0xAt4EjaW7HOWdlj74ejRnc2+yCuCwSmNm9YKfy0gyfaILFj1+tTNlJqQlYIYtEXSwlzR3vEW0TS1DjEdpj3t6U+dSn29qh7bzc2Z2fXEv21fw8DdbG4U18l2H/Rg2j/0HPSGKIas4eH2umrGCvpfxmETSBsc9KINQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185212; c=relaxed/simple; bh=jmCFStIua7DVXWNMLIbQuvMCokmSZJGjNJ0nWEu4LRk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=nvHFT6lYkCvPsMeYyk+aWy9fy1DbMKYJW46VdyCUlnhHqlFsQhweAYp21LWox+SGIyNQp5Q2qAENlrHeVhimtuWCoIhbSfFKh3OKPOx1EjM04n2cRN8kSXMoizDFJGEh1LeiVyzR5WGpdw9I1Vyo6cIFirkW+OKK4lvUxw3Y600= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=WYshwkPb; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=ILHHS5Yb; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="WYshwkPb"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="ILHHS5Yb" Received: from phl-compute-01.internal (phl-compute-01.phl.internal [10.202.2.41]) by mailfhigh.phl.internal (Postfix) with ESMTP id D904E11401C7; Wed, 5 Mar 2025 09:33:29 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-01.internal (MEProxy); Wed, 05 Mar 2025 09:33:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185209; x=1741271609; bh=89iuU0MvqzP2Mt3xe3b+4GMNgzhWxZUG Ue/kBsSAJEg=; b=WYshwkPbwkYDBAzOHyuVeoctcXj9JH18gxFIBW1CpXdSVmSg 1fD7U98zTNT65qNdHwmyIm8X9it8UKt0IM7RbbH6ZKas9foW0vx9KNXjayd0dtNG vLYPbHaAILWqN6G3O/SFjS4h64elwEt+eW0rMtc9sfS+dhQSQzIbvC4zqzGYCQ4Q jPtgT3D4RtxKPSbimeTcg6l9featacQIqR43kiaFL93/+7Kh4OG2whCyJZlxAnAQ ie/p5MAKharK1RgxqWiuHmkHaufIRie8/NcTdvDkB0yJ+c27XRZwODAoYNMz05sm U/+WaxkMSBsMlhMB5kBzjNqSt92EHOHoE8pIWQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185209; x= 1741271609; bh=89iuU0MvqzP2Mt3xe3b+4GMNgzhWxZUGUe/kBsSAJEg=; b=I LHHS5YbM9lH8/JrLqdfIvnFDpjtLW0cbHbX9KXvqEpuaYIfr2Iy3UvEA9K1fphXE w42Dd72ZLMklZ9PLcZUXr3YM7UME00IHAB47NGOzR2W07WWCJPDBLNWc+blKqBux 74/bpsnYdmKdp7hYwwF/Ckv8E8Vp/FdkIEFmx3Y9jXbOKPokbpimdpAqqIJ1wQM4 zIjXqyRu7b1167LAdFD0Nw8q/Zs3mOXwjkkAgeMEFq1qkIGwyUWehZk6YMFxq6h8 icQSSv7b/cVPNwk6Jle+9hgM+LiQOPWm4LrvYEFe4kx8Yn8XkPsqsti5Lih6xgko mT4Ss0zpn8IZZ/F8vfVbw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:28 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:04 +0100 Subject: [PATCH RFC bpf-next 07/20] xdp: Track if metadata is supported in xdp_frame <> xdp_buff conversions Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-7-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre xdp_buff stores whether metadata is supported by a NIC by setting data_meta to be greater than data. But xdp_frame only stores the metadata size (as metasize), so converting between xdp_frame and xdp_buff is lossy. Steal an unused bit in xdp_frame to track whether metadata is supported or not. This will lets us have "generic" functions for setting skb fields from either xdp_frame or xdp_buff from drivers. Signed-off-by: Arthur Fabre --- include/net/xdp.h | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/include/net/xdp.h b/include/net/xdp.h index 58019fa299b56dbd45c104fdfa807f73af6e4fa4..84afe07d09efdb2ab0cb78b904f02cb74f9a56b6 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -116,6 +116,9 @@ static __always_inline void xdp_buff_set_frag_pfmemalloc(struct xdp_buff *xdp) xdp->flags |= XDP_FLAGS_FRAGS_PF_MEMALLOC; } +static bool xdp_data_meta_unsupported(const struct xdp_buff *xdp); +static void xdp_set_data_meta_invalid(struct xdp_buff *xdp); + static __always_inline void *xdp_buff_traits(const struct xdp_buff *xdp) { return xdp->data_hard_start + _XDP_FRAME_SIZE; @@ -270,7 +273,9 @@ struct xdp_frame { void *data; u32 len; u32 headroom; - u32 metasize; /* uses lower 8-bits */ + u32 :23, /* unused */ + meta_unsupported:1, + metasize:8; /* Lifetime of xdp_rxq_info is limited to NAPI/enqueue time, * while mem_type is valid on remote CPU. */ @@ -369,6 +374,8 @@ void xdp_convert_frame_to_buff(const struct xdp_frame *frame, xdp->data = frame->data; xdp->data_end = frame->data + frame->len; xdp->data_meta = frame->data - frame->metasize; + if (frame->meta_unsupported) + xdp_set_data_meta_invalid(xdp); xdp->frame_sz = frame->frame_sz; xdp->flags = frame->flags; } @@ -396,6 +403,7 @@ int xdp_update_frame_from_buff(const struct xdp_buff *xdp, xdp_frame->len = xdp->data_end - xdp->data; xdp_frame->headroom = headroom - sizeof(*xdp_frame); xdp_frame->metasize = metasize; + xdp_frame->meta_unsupported = xdp_data_meta_unsupported(xdp); xdp_frame->frame_sz = xdp->frame_sz; xdp_frame->flags = xdp->flags; From patchwork Wed Mar 5 14:32:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002830 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fout-a2-smtp.messagingengine.com (fout-a2-smtp.messagingengine.com [103.168.172.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 892CB24FC0A; Wed, 5 Mar 2025 14:33:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.145 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185214; cv=none; b=nSajZp2PYiWJdUTHkFoRCxYomxwk6SxKqEVeVM5+EvuF/t/RlZFgVzHOxhASUGDvuQyQJLP5GRBvmOzjRsapcqyE+wdr3ybOmQk5TyUtuCLiL+hLZk9ZWiwxf5q9xtMfuHrWPDzbaVfcVsOhS2zb7Mny7XjDFvWzbiJrateFIUE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185214; c=relaxed/simple; bh=mfUsoO14IQldAwz5DqUWP3X3/mCdwxyhS+11WgRQ2tY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=oS7dSZmfUwzbnkEVvsOmnpN/EBiG/TCPlCWlpMEFGUX19oMh8NOw61zwhYb6rwz/IW9I/YVgmsuVAL6IxLSFHRLjw1J7Uwwf3jmDLoBpEIchCcXugOEbU/VB1Dp2vzZZy/8zrXdwRw9uqEOeOXG7RIFY/TqyiMyWtM/4RCaQG0w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=uU6UodM0; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=ugSko2eY; arc=none smtp.client-ip=103.168.172.145 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="uU6UodM0"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="ugSko2eY" Received: from phl-compute-13.internal (phl-compute-13.phl.internal [10.202.2.53]) by mailfout.phl.internal (Postfix) with ESMTP id 8E08913826DD; Wed, 5 Mar 2025 09:33:31 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-13.internal (MEProxy); Wed, 05 Mar 2025 09:33:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185211; x=1741271611; bh=+eymztBpYrvjHP3XdFqYJYpwv+edl3BS 3I768fbcNog=; b=uU6UodM03rWydn7xtKs6ywIw/TsLldGy+LHIegx3kEJhUmyH CQ7Is4o/AfDzlgL8WaEncB+Tcaej8qFgD9xkGwmRyD8H+3HM79sZaMlJD3duidZ7 4qbCSvLwTtmkT3qDJqoVXrT/qI+gebrCz4AlJuYX4KwttI/GIW+FKfkXmKsUgZra jY3D9gtVgq9f6/2Jq9XIfAhgur5unZhz3hCmIvDxaslpKxJh8JXu+G7DmRrX2LGc edSXgDT7Fuvh7+CJc23Dzs3xG2mVOGxmBGS29coxPz9ZoLAC797QhG7j62b9uJ6R f1U6F+F1t5a6idtHlFsRqgO7TZvicBYjDGAdcg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185211; x= 1741271611; bh=+eymztBpYrvjHP3XdFqYJYpwv+edl3BS3I768fbcNog=; b=u gSko2eYyNIs7gdt2C8sLhjMQB6v3gfuWXUwuothMtQMz4ligJCM0+RSfjeDSdG7/ wlOS8zsGF+pGXnZQmFXDD/H84h+UA9v+pyq6w+C7KRDrLinAMVmVfZhOsn4f1c91 h9V7U8iw/8/ztkoBzbPuU/hrxH7sjlbRsCxAkJwAZIsU9FEwZ5Wpwzfaba2go7PR nQYd7YRuwSk/55s7WkqsygtIUmYu70uXOgSsq4PVy8USn5SDzl8DrIi1bdcy+9bp Fukf04Oh/i2A/QNOEN0hAToXM1qzLq6myHA8NFIKBmIkj6TUSIgyCaxoyHUhM2o8 pnuAubSqXmcbzlfsKsZow== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:29 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:05 +0100 Subject: [PATCH RFC bpf-next 08/20] trait: Propagate presence of traits to sk_buff Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-8-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre Use a bit in sk_buff to track whether or not an skb stores traits in it's headroom. It's tempting to store it in skb_shared_info like the XDP metadata, but storing it in the skb allows us to more easily handle cases such as skb clones in a few patches. (I couldn't find an existing hole to use in sk_buff, so this makes a 4 byte hole... any pointers to a free bit?) The following patches add support for handful of drivers, in the final form we'd like to cover all drivers that currently support XDP metadata. Signed-off-by: Arthur Fabre --- include/linux/skbuff.h | 7 +++++++ include/net/xdp.h | 12 ++++++++++++ 2 files changed, 19 insertions(+) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 03553c2200ab1c3750b799edb94fa3b94e11a5f1..d7dfee152ebd26ce87a230222e94076aca793adc 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -725,6 +725,12 @@ enum skb_tstamp_type { __SKB_CLOCK_MAX = SKB_CLOCK_TAI, }; +enum skb_traits_type { + SKB_TRAITS_NONE, + /* Trait store in headroom, offset by sizeof(struct xdp_frame) */ + SKB_TRAITS_AFTER_XDP, +}; + /** * DOC: Basic sk_buff geometry * @@ -1023,6 +1029,7 @@ struct sk_buff { __u8 csum_not_inet:1; #endif __u8 unreadable:1; + __u8 traits_type:1; /* See enum skb_traits_type */ #if defined(CONFIG_NET_SCHED) || defined(CONFIG_NET_XGRESS) __u16 tc_index; /* traffic control index */ #endif diff --git a/include/net/xdp.h b/include/net/xdp.h index 84afe07d09efdb2ab0cb78b904f02cb74f9a56b6..e2e7c270efa3cbd11bddd234010b91daee5009a2 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -119,6 +119,12 @@ static __always_inline void xdp_buff_set_frag_pfmemalloc(struct xdp_buff *xdp) static bool xdp_data_meta_unsupported(const struct xdp_buff *xdp); static void xdp_set_data_meta_invalid(struct xdp_buff *xdp); +static __always_inline void xdp_buff_update_skb(struct xdp_buff *xdp, struct sk_buff *skb) +{ + if (!xdp_data_meta_unsupported(xdp)) + skb->traits_type = SKB_TRAITS_AFTER_XDP; +} + static __always_inline void *xdp_buff_traits(const struct xdp_buff *xdp) { return xdp->data_hard_start + _XDP_FRAME_SIZE; @@ -298,6 +304,12 @@ xdp_frame_is_frag_pfmemalloc(const struct xdp_frame *frame) return !!(frame->flags & XDP_FLAGS_FRAGS_PF_MEMALLOC); } +static __always_inline void xdp_frame_update_skb(struct xdp_frame *frame, struct sk_buff *skb) +{ + if (!frame->meta_unsupported) + skb->traits_type = SKB_TRAITS_AFTER_XDP; +} + #define XDP_BULK_QUEUE_SIZE 16 struct xdp_frame_bulk { int count; From patchwork Wed Mar 5 14:32:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002831 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fout-a2-smtp.messagingengine.com (fout-a2-smtp.messagingengine.com [103.168.172.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D90F1632EF; Wed, 5 Mar 2025 14:33:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.145 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185215; cv=none; b=sSbF0RByW/NYGn9FQyZbL39n5daxyTjpv+meQgPqiIScylUx9I52tsiyvWHeh4BAEWMQIOjI35Wt/5EwYIAuOWXhEjLMmRNkbDaWv87Bvxu1zvNAvphkFzMQ1RPSfcp8Jt3HDyuzS/C5sZLa/40Sopp/id03yq6/C6na5s3Oyzg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185215; c=relaxed/simple; bh=88EbAv9wA20xdpCHpwGY7DA8RnfPSAiZ3mf1GSiOwp8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=rCSsAfPQGH5BZuIjh3hL2090asfxwbLmuQb2mNSz2PlYZnfbaNd/QdBQgYt7U/1Hq/hRuK3mZLS3Lw0At1AvTvzWYC6l1sTUww8Mwf5VkQm79uKQoCB0TG0ZbXIaPlpmm2v9XAt+iDm1CLu9MDpZa3bmMp6+ujFJKpe0ZJpqCf8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=cKr1s9UP; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=T8GsC2r5; arc=none smtp.client-ip=103.168.172.145 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="cKr1s9UP"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="T8GsC2r5" Received: from phl-compute-06.internal (phl-compute-06.phl.internal [10.202.2.46]) by mailfout.phl.internal (Postfix) with ESMTP id 4B66C13826A7; Wed, 5 Mar 2025 09:33:33 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-06.internal (MEProxy); Wed, 05 Mar 2025 09:33:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185213; x=1741271613; bh=nAdEtDrkHqUTwsOjy7M1AaSyH7jAnptn 4P9ellHBC8A=; b=cKr1s9UPWvzfrMh7rHnucKeA3GoZl93s8iWZVTze0pmQyHFi 9butLGG+/MtmIdAhamX2UZOGJuCVTyQ2gnx3650o8FtulYGOW0F3qwVfM01Rzn7b 0LtExYNZGCd3NC2r63z9oEYmuS5uHvV332OS58JRMUQ/oNIJ4KCX9mw5HmDAXw99 kBjqP8bvOrDpAYbN8Me1JBPvyL8Q0JjoRy9XlQtcUlXFYlxxxL6XB6qcZKB4z5BQ EWw6j13JhB9a4IYYL3248pOvEyR/df6EMC/Th8S2zAbCh5ImnXgTxF3nSGcG0fGz u371yIfTD3iToWNG3dcl5QJQHScpOsTnUpXZcQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185213; x= 1741271613; bh=nAdEtDrkHqUTwsOjy7M1AaSyH7jAnptn4P9ellHBC8A=; b=T 8GsC2r5hQTPsgg2HdqDv8yR97mO9Un8Wm5Ls8pmDyQJl6F0sTGeMzjjkf1YktoDD zUlLCRhPTs4C9JOPbDxwvdR8/f1+ki07x4wmglGZh3/h0d5Uy7W+yJYDPWAaSdgn Ob0091JqRSLAHH8N/XXf5A+KO65Xpb+fxfepRJ1BfakFtye6HYX2vZPuzWedBUpB McO4vNLKhyve2pYdbspr62/VzyPrixifya3b+e9aN/cVapvPsHqtOn1nawNo6cJd PfSop1eIPpsOxbVvQiAS7PH2w4fzOrN+kR3jzL07GR9cl4v50YqfphECjSJfW+VF 75b6r5CW2udoJvX57H41Q== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:31 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:06 +0100 Subject: [PATCH RFC bpf-next 09/20] bnxt: Propagate trait presence to skb Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-9-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre Call the common xdp_buff_update_skb() helper. Signed-off-by: Arthur Fabre --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 7b8b5b39c7bbe8885543a7c612567f7ff55a4fca..7e22804ba09b5a12384a0f8125db42f79b187d42 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -2273,6 +2273,10 @@ static int bnxt_rx_pkt(struct bnxt *bp, struct bnxt_cp_ring_info *cpr, } } } + + if (xdp_active) + xdp_buff_update_skb(&xdp, skb); + bnxt_deliver_skb(bp, bnapi, skb); rc = 1; From patchwork Wed Mar 5 14:32:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002832 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 04FAC2505B2; Wed, 5 Mar 2025 14:33:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185217; cv=none; b=JZQ/srAE1NfvboMugoMCJh0qirdrAqSbvhFi5EtQnKsad8oEaxwmC3SAUtAJsk7wcu9Ef4Bk13YvyKakSWSSewQAQGRxjfzUP3kV2mQ7I8JLlu76pAbayghZQM5hqm0C7zPGtrJ9Co+Juzr44vNnx7g6pD2kTdIekWuxvn7b1C0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185217; c=relaxed/simple; bh=IrVawBUn1RmxC0Wuya9HD1bemcOwOHctPP/GE0nmpgM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=XE9Pu9oDZotSW8T+6vOUkf6IIacA2o/vbGAgqjC4+Z7DlGYT2vvxtwUSI8I7OJZzSo5M1yzebeOtJzI7klGHQZoM2LiLF4beEVyP0alaDygmcVGOjJTR4oGGrIjqghjqu/2ofkvtiL31az0K8QC7dmcriLOeY9z/pBlSegLQgik= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=wfXRA0O9; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=fY8qQSo8; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="wfXRA0O9"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="fY8qQSo8" Received: from phl-compute-09.internal (phl-compute-09.phl.internal [10.202.2.49]) by mailfhigh.phl.internal (Postfix) with ESMTP id 1A1121140156; Wed, 5 Mar 2025 09:33:35 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-09.internal (MEProxy); Wed, 05 Mar 2025 09:33:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185215; x=1741271615; bh=qru1O9zdUmjKZ1p2yUXAEyuttmotw+de 6bIlQUpfaCs=; b=wfXRA0O9omTwhjqjVCD5BIlQ5XH1sy4OyW/BuS0fI3asS5g5 FtW+izSKj4LAFpjnDTZHdI0v57QboOIQHo4Uq6PGdWiMScUJe/JUG+0zZFTUPPZY UIV8NGawtnV4JqduR6JiT/z9qW53YX23wDL8oqNmvKjtuEXdA222MU70Vjtu2U/H z/RSvxZFv3PTGEtvyvbSm6dDZwssYTLkBpsg5LhXsUcvqdxfVwrVcMIoj1PflENU KWN13QcTk1iYFuormX/HhfBLA3F9Art5Y6AONUshy+oMvpNnVqxOKzQWECMX2nfe 1P7ll78UpONr1k1JnwDPlu/35pIxLCLvYrpHWw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185215; x= 1741271615; bh=qru1O9zdUmjKZ1p2yUXAEyuttmotw+de6bIlQUpfaCs=; b=f Y8qQSo8qUBI9Klltrj8vuDWQ6yXtQk5XNGLTZDQdyenWuyuSQo+zOcXLHL1ANW8m 9mBhHZ07fIsmqRGHupz4G8yIM2MyraZIA5VeiBxhl3CF0aqIesCz8azdpbQTwdiJ TZKSBKRbuaaldeEeBGAj1t+rOh4AYTt8wTxUeaK67FqrxCb9A7FU9pB2ZE3jki7H SxZhbSVGMNMWuy/CWOsV9MjbUwagTId31bHJ8q9JbDHAYMlU6ld1MFQX+KIX+8hU y3Q4s8h/gk+X2Xu4MFP4xzxNQBQj88YykGSwpXSng+rD/lpJNc74Z8BNucGOk1rH SObJsWikVnGZxhhRL4ELw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdekucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:33 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:07 +0100 Subject: [PATCH RFC bpf-next 10/20] ice: Propagate trait presence to skb Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-10-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre Call the common xdp_buff_update_skb() helper. Signed-off-by: Arthur Fabre --- drivers/net/ethernet/intel/ice/ice_txrx.c | 4 ++++ drivers/net/ethernet/intel/ice/ice_xsk.c | 2 ++ 2 files changed, 6 insertions(+) diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c index 9c9ea4c1b93b7fb5ac9bd5599ac91cf326fb2527..70101fc365b17a13f06ff7c46610d11fbf81313a 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx.c +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c @@ -1011,6 +1011,8 @@ ice_build_skb(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp) nr_frags * xdp->frame_sz, xdp_buff_is_frag_pfmemalloc(xdp)); + xdp_buff_update_skb(xdp, skb); + return skb; } @@ -1092,6 +1094,8 @@ ice_construct_skb(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp) xdp_buff_is_frag_pfmemalloc(xdp)); } + xdp_buff_update_skb(xdp, skb); + return skb; } diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c index 8975d2971bc3778aad8898e334ce1d759618fd32..a34c7a69c58677958974b84b7fb6e5643c5e29b2 100644 --- a/drivers/net/ethernet/intel/ice/ice_xsk.c +++ b/drivers/net/ethernet/intel/ice/ice_xsk.c @@ -572,6 +572,8 @@ ice_construct_skb_zc(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp) __skb_pull(skb, metasize); } + xdp_buff_update_skb(xdp, skb); + if (likely(!xdp_buff_has_frags(xdp))) goto out; From patchwork Wed Mar 5 14:32:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002833 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3FD32505A3; Wed, 5 Mar 2025 14:33:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185219; cv=none; b=Rr3xBUfo+uJxpNFJ0NX6ibENrCd4NUvXrVVeuh2kPqXQd/9PYF+4LcxO3qzl+aGHreHnSQ+/Og8orGFFasty4NlzXnbyLfB1CsXN3Y0D0Y3XNHB35K/3afJc0n4c9UkdUfc+1eYQSAYhXnqOd6FJl6Vx59Ou5sRP4u3h0w0Nh1Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185219; c=relaxed/simple; bh=5A2BC89iPT2R5CsY1G6WWz1zdKXqY9Sc7caSn5kzKnw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=HvqFI0IR3UGE+7TAHauZoyae4N02m3vsITVMyh+Dh0j27a4niMnip6bTPbLr0sNVuKmiyUMH79Z3wL0TOC+Szrq6mj/Y0YQrZc0Vm9WN/Tz8cGvOOg62TVcV92DLt3B3m1OK6y60FVX9B/I8izkHl1RYcfUsy05BophcsdDoP+U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=YHCL16gK; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=l6L4eZlD; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="YHCL16gK"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="l6L4eZlD" Received: from phl-compute-12.internal (phl-compute-12.phl.internal [10.202.2.52]) by mailfhigh.phl.internal (Postfix) with ESMTP id D271511401F8; Wed, 5 Mar 2025 09:33:36 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-12.internal (MEProxy); Wed, 05 Mar 2025 09:33:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185216; x=1741271616; bh=T1TDGy+aGcL5GXLTKfJf+nXwA5yAxBek JCkVmMjW4vg=; b=YHCL16gKwqO9rU15UUxLcvEo8CCLTbTTuqUqynkNpi8tFSyT s/m2A/9goz1Ar0NWiFFrUfmj81M9R+l4Usn0AlxpRHXWYL3vf6gAFegbHv8nCro0 XXnNUOrfyuFN5xu3EYvLwvAwYfGm2baEY3Dx5vB8Ze5osXIxiqdWT6ssDtCyb49Z tk1fi9LRJLw6b2Xs5HKpq7LFX0l2UKbx9rP2wepwW05GiayU9zCZ/232MLUNwEdY IqJ4xjPBeZ+ZJybNcTuvGZXTmxdEurkdiolmES9JaP4rNkzmcL7xX2cG8ph9He+s C6bWM2jpEb2qOkG1L2BkUbxTr5EzYRmJvvhVuA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185216; x= 1741271616; bh=T1TDGy+aGcL5GXLTKfJf+nXwA5yAxBekJCkVmMjW4vg=; b=l 6L4eZlDsSHA9mN+DO2BseRLK83gHChJlVt1YGhpYF9+QO8lH4CMcsknI4bP2jYnl WCQU3K5jE+J56ptngb9LTnVIHXTZ5oKTlqHmbBWeqdxl2cvZULOBQkX4aBG7gA3D 4pNrXyXQuCk6vy4i8uA/5m6wE5OUnZbVdCW8iGPNFpJLJiU35VL4QRTh9dfrmxF0 Xg40AojCCgFAnfgAwHsXO7GFzilESVsU3LX3ChxVPaj8uiHWEmX3u5B1HjdgAqF0 lSr4onfOPbjkxwPoaYpmmLL6uWfVKHpOdnu9vxgEVWxYnXxrVpJI0vMIE4yrwmBO x7QJgYyD4XhSF/sDpyy4g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:35 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:08 +0100 Subject: [PATCH RFC bpf-next 11/20] veth: Propagate trait presence to skb Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-11-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre Call the common xdp_buff_update_skb() / xdp_frame_update_skb() helpers. Signed-off-by: Arthur Fabre --- drivers/net/veth.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/net/veth.c b/drivers/net/veth.c index 01251868a9c27592f0dfbcdc32c0afbd7e9baafc..186ea9e09efbcb6abb01effcec5f7d03a186cefe 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -703,6 +703,8 @@ static void veth_xdp_rcv_bulk_skb(struct veth_rq *rq, void **frames, stats->rx_drops++; continue; } + + xdp_frame_update_skb(frames[i], skb); napi_gro_receive(&rq->xdp_napi, skb); } } @@ -855,6 +857,8 @@ static struct sk_buff *veth_xdp_rcv_skb(struct veth_rq *rq, metalen = xdp->data - xdp->data_meta; if (metalen) skb_metadata_set(skb, metalen); + + xdp_buff_update_skb(xdp, skb); out: return skb; drop: From patchwork Wed Mar 5 14:32:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002834 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fout-a2-smtp.messagingengine.com (fout-a2-smtp.messagingengine.com [103.168.172.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 27B3C2505DB; Wed, 5 Mar 2025 14:33:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.145 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185220; cv=none; b=rJcNa5a+X0aXauuMVWSYYMhxliDGaB434pfgJ/vi+utb/s2pB1JC0w+4AfyQeQtfLpLPq4e6X3WYlkGaoR9Ml49qF6HsavwEfjG6xUQVnuDVbK1EnSejdqij4SkTfV3JMzAvil4b+q/Z+9VaFhwi0Ipauah5zxQx6UyVFRKknCU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185220; c=relaxed/simple; bh=AI1Oa6AqU8eYEoTFzUFnBnF62DR+wac6Et9+mHw/w+Q=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=jI82kPiCVXcFlOGbFFH/8Dg/NiBm7UVT5Bai9/Qws94yvEyr80b925wvTZT85pxiQlOukbiRAHiSybMtcbiGH5ENSblSmN47Kl0Y6u/iH1tMj7HSZobnp44Xt7lCJCLr6bKoAXXW3zEAYh5HidVqXe9f5R3hbJQNRuFdmsqV4po= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=JHTxCAjR; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=7T6TH9ZM; arc=none smtp.client-ip=103.168.172.145 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="JHTxCAjR"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="7T6TH9ZM" Received: from phl-compute-09.internal (phl-compute-09.phl.internal [10.202.2.49]) by mailfout.phl.internal (Postfix) with ESMTP id 8001613814DA; Wed, 5 Mar 2025 09:33:38 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-09.internal (MEProxy); Wed, 05 Mar 2025 09:33:38 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185218; x=1741271618; bh=LivPnBBVA3IvtDu0uUqXT//4QAqc88XT GfchXNR1bbQ=; b=JHTxCAjRUnuQk9OjYuNhd+VIT9JUFNwAOLs29vufn+HIunRC gSL9LMmV1gSb/ebrJIZbnWZeG3kcOu37/INYsibB/DnlkZlA9RPFDS1kg6sAkGoD zFuYkrRC2b1zJAAR5LEnznphxPJ3erbthS9cpT/it1rqZkMUneAmA6mC5oRwkf7H jH5lA2wiwAhjQLuywXw5zrQeXb4CgoPZ4pxLOBgiMOI0Rtkb3/pTmNVNBrMvUBD1 xnPTIWi68nI/AQlL/rb+EgwyYrGPTLyDHa4lJvYPalzqglreGVDvMWIzciKloOAc 6wghS2GakWAzDwNNfG9X9OCW5ScGTOGmT605AQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185218; x= 1741271618; bh=LivPnBBVA3IvtDu0uUqXT//4QAqc88XTGfchXNR1bbQ=; b=7 T6TH9ZM1qRvmSS86hHZctkTVC9sKK/+uGA6oJHEwLj/XdRpu6FJ3eKMgxcdVPOB8 BsEH2OH9TtzCy0bwNph48KyqcojWSxBcME6Hokg/YtPDMiI+w33S1YGdQaNlujJo QssxFNpxoFt+L2J7NaQOfl0ViZ0KFZzuuZ1a73dxGTv+gWmCDN9DZvGBywivv2H5 DEE7cLans8cV3hmBhJr2xDqhbPnYLf9mS4GmhXYtvd7VV6hfxWe/y8mkFAUrcvgX Cf7b/G0QZBZEVn7HB7vO5HNrQwtbR+Hi/y8m1Z0mWycPxe00lcnbo3zeiWdFYA+l Prh4+ic2Xa0EW3UATQz3A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdekucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:36 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:09 +0100 Subject: [PATCH RFC bpf-next 12/20] virtio_net: Propagate trait presence to skb Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-12-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre Call the common xdp_buff_update_skb() / xdp_frame_update_skb() helpers. Signed-off-by: Arthur Fabre --- drivers/net/virtio_net.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 7646ddd9bef70cf2a0833c7db7c0491884fb7ab5..85798c2bc3446c47deaef66dbb96270e51c9076c 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -1190,6 +1190,7 @@ static struct sk_buff *virtnet_receive_xsk_small(struct net_device *dev, struct struct virtnet_rq_stats *stats) { struct bpf_prog *prog; + struct sk_buff *skb; u32 ret; ret = XDP_PASS; @@ -1201,7 +1202,9 @@ static struct sk_buff *virtnet_receive_xsk_small(struct net_device *dev, struct switch (ret) { case XDP_PASS: - return xsk_construct_skb(rq, xdp); + skb = xsk_construct_skb(rq, xdp); + xdp_buff_update_skb(xdp, skb); + return skb; case XDP_TX: case XDP_REDIRECT: @@ -1323,6 +1326,7 @@ static struct sk_buff *virtnet_receive_xsk_merge(struct net_device *dev, struct skb = xsk_construct_skb(rq, xdp); if (!skb) goto drop_bufs; + xdp_buff_update_skb(xdp, skb); if (xsk_append_merge_buffer(vi, rq, skb, num_buf, hdr, stats)) { dev_kfree_skb(skb); @@ -1956,6 +1960,7 @@ static struct sk_buff *receive_small_xdp(struct net_device *dev, skb = virtnet_build_skb(buf, buflen, xdp.data - buf, len); if (unlikely(!skb)) goto err; + xdp_buff_update_skb(&xdp, skb); if (metasize) skb_metadata_set(skb, metasize); @@ -2324,6 +2329,7 @@ static struct sk_buff *receive_mergeable_xdp(struct net_device *dev, head_skb = build_skb_from_xdp_buff(dev, vi, &xdp, xdp_frags_truesz); if (unlikely(!head_skb)) break; + xdp_buff_update_skb(&xdp, head_skb); return head_skb; case XDP_TX: From patchwork Wed Mar 5 14:32:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002835 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3730A250BF1; Wed, 5 Mar 2025 14:33:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185223; cv=none; b=JWAHKgIZE7OinmGJ0P2EtJcEQczpRjqgzu0e4Nrq/XdPK5L3SV4IakZya44nv17fAvYDuht8oATOJ995/Ljo6L9RM2wng6zgs3Xpc5ZEtoYYEND2yWujU+1a1CoXECHjNUJNBvReD9pi5f3+YJ+6kPlxqNO79ezWdHuqJs0jsqg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185223; c=relaxed/simple; bh=tpM0cB5vXCNmMtGFfTMEtumsE+4YNdQFGV+GNpq71Xs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=G2NMpcJi1Fm57AuVEW/k3rlOlvfX6kdJ+0H45kguPLEGDoXJXkyXmICuNNQ4K/J9HpkOJHtV9TuyHTs8r+BglbrPD9EiB4JDjetjuMWiy6c22pT1tFdIquxdvvG2vx3PY2od2uXmQrhPmCbSEwQFQwC4Xm6x+xhxr7mAW8ahgvU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=hf1dc0L4; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=3nUXE0B8; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="hf1dc0L4"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="3nUXE0B8" Received: from phl-compute-01.internal (phl-compute-01.phl.internal [10.202.2.41]) by mailfhigh.phl.internal (Postfix) with ESMTP id 431F91140219; Wed, 5 Mar 2025 09:33:40 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-01.internal (MEProxy); Wed, 05 Mar 2025 09:33:40 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185220; x=1741271620; bh=fTGDU7O2879ZtUCbBbFzOgm2l0L85u2g SYZCEXfVXFA=; b=hf1dc0L4Gqc6F4T7PM4/I01MOaIdUV8Kl4b74J5jQDBmFGtI dlQr3Nft1l/PmcLOEtGLOjtSviZ4oWLt1Xq9Z01H/ncOEZKSipcRBze3zdsqC6O7 hvFLWsCIqJc4jZjwXF4czsKKt6f05xV4Iu8WC9xOgzN+APCUmkYk33pbaXSQYXfH pCZqNhm1ARqQUhOoUxIHY3k5kuky6FHEfGUeQXhfMlT15fYmvaZG38G7/D/nfaqY nQ8W4u5KX2ev5o3wK4VGBocvAx/h2A6bqY7cMvDJ5NmUTvIcPO3zz2OGuHX64/VV d3dQtF+z7yNnKjpzqWayf6XaAGk08+ztw6A5rg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185220; x= 1741271620; bh=fTGDU7O2879ZtUCbBbFzOgm2l0L85u2gSYZCEXfVXFA=; b=3 nUXE0B8TsA8lZwRhllXC7w3r5UYhX1hEzqf5Z+DZqNSJtBddWxY7N4Wac/oZOhxg QiozLtm4OWToIu+1U2Pth/YvXeY68/IIXJ7nlZA2ubZRu7RtD70nAkw1uJ3PKxnG oibRmltay/oZyg5XlfCGLyukUEs39E4VagzNT61l3jfrbJdI0YmG7alJ5CrmwoI3 V3y0rTy2zLRDLaDrMrAKe/Y4hsqs8kRwEK5FsK6EMjUKtgHFnt0+DncuhQjsA52M BM0V2k19D/Em3qmXOoodIHrvoBJtu5+8rApPbNw6wKyyBOB85qDT8uSXMft1OsHX 7hE1pRCh7afYM+w4kfRgw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpedvtdefhedtgeehleffledvtedtkefhgeevffduheekvdetledtfeek hfefudetieenucffohhmrghinhepgiguphdruggrthgrnecuvehluhhsthgvrhfuihiivg eptdenucfrrghrrghmpehmrghilhhfrhhomheprghrthhhuhhrsegrrhhthhhurhhfrggs rhgvrdgtohhmpdhnsggprhgtphhtthhopeelpdhmohguvgepshhmthhpohhuthdprhgtph htthhopehthhhoihhlrghnugesrhgvughhrghtrdgtohhmpdhrtghpthhtoheplhgsihgr nhgtohhnsehrvgguhhgrthdrtghomhdprhgtphhtthhopehhrgifkheskhgvrhhnvghlrd horhhgpdhrtghpthhtohepsghpfhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphht thhopegrfhgrsghrvgestghlohhuughflhgrrhgvrdgtohhmpdhrtghpthhtohepjhgrkh husgestghlohhuughflhgrrhgvrdgtohhmpdhrtghpthhtohephigrnhestghlohhuughf lhgrrhgvrdgtohhmpdhrtghpthhtohepnhgvthguvghvsehvghgvrhdrkhgvrhhnvghlrd horhhgpdhrtghpthhtohepjhgsrhgrnhguvggsuhhrghestghlohhuughflhgrrhgvrdgt ohhm X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:38 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:10 +0100 Subject: [PATCH RFC bpf-next 13/20] mlx5: move xdp_buff scope one level up Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-13-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Jesper Dangaard Brouer This is in preparation for changes. Signed-off-by: Jesper Dangaard Brouer --- drivers/net/ethernet/mellanox/mlx5/core/en.h | 6 +- .../net/ethernet/mellanox/mlx5/core/en/xsk/rx.c | 6 +- .../net/ethernet/mellanox/mlx5/core/en/xsk/rx.h | 6 +- drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 103 +++++++++++---------- 4 files changed, 66 insertions(+), 55 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h index 979fc56205e1fe7b473ad0849cf84f189d09fd4f..9bed146806a8d8b2c0afb14b2417a4a95cc09dcb 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -581,14 +581,16 @@ struct mlx5e_mpw_info { #define MLX5E_MAX_RX_FRAGS 4 struct mlx5e_rq; +struct mlx5e_xdp_buff; typedef void (*mlx5e_fp_handle_rx_cqe)(struct mlx5e_rq*, struct mlx5_cqe64*); typedef struct sk_buff * (*mlx5e_fp_skb_from_cqe_mpwrq)(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, struct mlx5_cqe64 *cqe, u16 cqe_bcnt, - u32 head_offset, u32 page_idx); + u32 head_offset, u32 page_idx, + struct mlx5e_xdp_buff *mxbuf); typedef struct sk_buff * (*mlx5e_fp_skb_from_cqe)(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi, - struct mlx5_cqe64 *cqe, u32 cqe_bcnt); + struct mlx5_cqe64 *cqe, u32 cqe_bcnt, struct mlx5e_xdp_buff *mxbuf); typedef bool (*mlx5e_fp_post_rx_wqes)(struct mlx5e_rq *rq); typedef void (*mlx5e_fp_dealloc_wqe)(struct mlx5e_rq*, u16); typedef void (*mlx5e_fp_shampo_dealloc_hd)(struct mlx5e_rq*, u16, u16, bool); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c index 1b7132fa70de2805a81b878fe3fa308ca9d4de6f..4dacaa61e1061960e09dccd6f97bc2f2d02ffbb8 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c @@ -249,7 +249,8 @@ struct sk_buff *mlx5e_xsk_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe, u16 cqe_bcnt, u32 head_offset, - u32 page_idx) + u32 page_idx, + struct mlx5e_xdp_buff *mxbuf_) { struct mlx5e_xdp_buff *mxbuf = xsk_buff_to_mxbuf(wi->alloc_units.xsk_buffs[page_idx]); struct bpf_prog *prog; @@ -304,7 +305,8 @@ struct sk_buff *mlx5e_xsk_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct sk_buff *mlx5e_xsk_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi, struct mlx5_cqe64 *cqe, - u32 cqe_bcnt) + u32 cqe_bcnt, + struct mlx5e_xdp_buff *mxbuf_) { struct mlx5e_xdp_buff *mxbuf = xsk_buff_to_mxbuf(*wi->xskp); struct bpf_prog *prog; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h index cefc0ef6105d24705bdd450cfd7857435a9d0c67..0890c975042c7f58e512a61fc538f21b5e37c6b4 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h @@ -16,10 +16,12 @@ struct sk_buff *mlx5e_xsk_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe, u16 cqe_bcnt, u32 head_offset, - u32 page_idx); + u32 page_idx, + struct mlx5e_xdp_buff *mxbuf_); struct sk_buff *mlx5e_xsk_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi, struct mlx5_cqe64 *cqe, - u32 cqe_bcnt); + u32 cqe_bcnt, + struct mlx5e_xdp_buff *mxbuf_); #endif /* __MLX5_EN_XSK_RX_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index 1963bc5adb1887af5a2cadb3febf24bef0ae3338..77bace3b212ae18c420a11312a5e3043b5e3f4ae 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -63,11 +63,11 @@ static struct sk_buff * mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, struct mlx5_cqe64 *cqe, u16 cqe_bcnt, u32 head_offset, - u32 page_idx); + u32 page_idx, struct mlx5e_xdp_buff *mxbuf); static struct sk_buff * mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, struct mlx5_cqe64 *cqe, u16 cqe_bcnt, u32 head_offset, - u32 page_idx); + u32 page_idx, struct mlx5e_xdp_buff *mxbuf); static void mlx5e_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe); static void mlx5e_handle_rx_cqe_mpwrq(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe); static void mlx5e_handle_rx_cqe_mpwrq_shampo(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe); @@ -1662,7 +1662,8 @@ static void mlx5e_fill_mxbuf(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe, static struct sk_buff * mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi, - struct mlx5_cqe64 *cqe, u32 cqe_bcnt) + struct mlx5_cqe64 *cqe, u32 cqe_bcnt, + struct mlx5e_xdp_buff *mxbuf) { struct mlx5e_frag_page *frag_page = wi->frag_page; u16 rx_headroom = rq->buff.headroom; @@ -1684,17 +1685,15 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi, prog = rcu_dereference(rq->xdp_prog); if (prog) { - struct mlx5e_xdp_buff mxbuf; - net_prefetchw(va); /* xdp_frame data area */ mlx5e_fill_mxbuf(rq, cqe, va, rx_headroom, rq->buff.frame0_sz, - cqe_bcnt, &mxbuf); - if (mlx5e_xdp_handle(rq, prog, &mxbuf)) + cqe_bcnt, mxbuf); + if (mlx5e_xdp_handle(rq, prog, mxbuf)) return NULL; /* page/packet was consumed by XDP */ - rx_headroom = mxbuf.xdp.data - mxbuf.xdp.data_hard_start; - metasize = mxbuf.xdp.data - mxbuf.xdp.data_meta; - cqe_bcnt = mxbuf.xdp.data_end - mxbuf.xdp.data; + rx_headroom = mxbuf->xdp.data - mxbuf->xdp.data_hard_start; + metasize = mxbuf->xdp.data - mxbuf->xdp.data_meta; + cqe_bcnt = mxbuf->xdp.data_end - mxbuf->xdp.data; } frag_size = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt); skb = mlx5e_build_linear_skb(rq, va, frag_size, rx_headroom, cqe_bcnt, metasize); @@ -1710,14 +1709,14 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi, static struct sk_buff * mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi, - struct mlx5_cqe64 *cqe, u32 cqe_bcnt) + struct mlx5_cqe64 *cqe, u32 cqe_bcnt, + struct mlx5e_xdp_buff *mxbuf) { struct mlx5e_rq_frag_info *frag_info = &rq->wqe.info.arr[0]; struct mlx5e_wqe_frag_info *head_wi = wi; u16 rx_headroom = rq->buff.headroom; struct mlx5e_frag_page *frag_page; struct skb_shared_info *sinfo; - struct mlx5e_xdp_buff mxbuf; u32 frag_consumed_bytes; struct bpf_prog *prog; struct sk_buff *skb; @@ -1737,8 +1736,8 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi net_prefetch(va + rx_headroom); mlx5e_fill_mxbuf(rq, cqe, va, rx_headroom, rq->buff.frame0_sz, - frag_consumed_bytes, &mxbuf); - sinfo = xdp_get_shared_info_from_buff(&mxbuf.xdp); + frag_consumed_bytes, mxbuf); + sinfo = xdp_get_shared_info_from_buff(&mxbuf->xdp); truesize = 0; cqe_bcnt -= frag_consumed_bytes; @@ -1750,7 +1749,7 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi frag_consumed_bytes = min_t(u32, frag_info->frag_size, cqe_bcnt); - mlx5e_add_skb_shared_info_frag(rq, sinfo, &mxbuf.xdp, frag_page, + mlx5e_add_skb_shared_info_frag(rq, sinfo, &mxbuf->xdp, frag_page, wi->offset, frag_consumed_bytes); truesize += frag_info->frag_stride; @@ -1760,7 +1759,7 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi } prog = rcu_dereference(rq->xdp_prog); - if (prog && mlx5e_xdp_handle(rq, prog, &mxbuf)) { + if (prog && mlx5e_xdp_handle(rq, prog, mxbuf)) { if (__test_and_clear_bit(MLX5E_RQ_FLAG_XDP_XMIT, rq->flags)) { struct mlx5e_wqe_frag_info *pwi; @@ -1770,21 +1769,21 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi return NULL; /* page/packet was consumed by XDP */ } - skb = mlx5e_build_linear_skb(rq, mxbuf.xdp.data_hard_start, rq->buff.frame0_sz, - mxbuf.xdp.data - mxbuf.xdp.data_hard_start, - mxbuf.xdp.data_end - mxbuf.xdp.data, - mxbuf.xdp.data - mxbuf.xdp.data_meta); + skb = mlx5e_build_linear_skb(rq, mxbuf->xdp.data_hard_start, rq->buff.frame0_sz, + mxbuf->xdp.data - mxbuf->xdp.data_hard_start, + mxbuf->xdp.data_end - mxbuf->xdp.data, + mxbuf->xdp.data - mxbuf->xdp.data_meta); if (unlikely(!skb)) return NULL; skb_mark_for_recycle(skb); head_wi->frag_page->frags++; - if (xdp_buff_has_frags(&mxbuf.xdp)) { + if (xdp_buff_has_frags(&mxbuf->xdp)) { /* sinfo->nr_frags is reset by build_skb, calculate again. */ xdp_update_skb_shared_info(skb, wi - head_wi - 1, sinfo->xdp_frags_size, truesize, - xdp_buff_is_frag_pfmemalloc(&mxbuf.xdp)); + xdp_buff_is_frag_pfmemalloc(&mxbuf->xdp)); for (struct mlx5e_wqe_frag_info *pwi = head_wi + 1; pwi < wi; pwi++) pwi->frag_page->frags++; @@ -1815,6 +1814,7 @@ static void mlx5e_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe) { struct mlx5_wq_cyc *wq = &rq->wqe.wq; struct mlx5e_wqe_frag_info *wi; + struct mlx5e_xdp_buff mxbuf; struct sk_buff *skb; u32 cqe_bcnt; u16 ci; @@ -1832,7 +1832,7 @@ static void mlx5e_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe) mlx5e_skb_from_cqe_linear, mlx5e_skb_from_cqe_nonlinear, mlx5e_xsk_skb_from_cqe_linear, - rq, wi, cqe, cqe_bcnt); + rq, wi, cqe, cqe_bcnt, &mxbuf); if (!skb) { /* probably for XDP */ if (__test_and_clear_bit(MLX5E_RQ_FLAG_XDP_XMIT, rq->flags)) @@ -1863,6 +1863,7 @@ static void mlx5e_handle_rx_cqe_rep(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe) struct mlx5_eswitch_rep *rep = rpriv->rep; struct mlx5_wq_cyc *wq = &rq->wqe.wq; struct mlx5e_wqe_frag_info *wi; + struct mlx5e_xdp_buff mxbuf; struct sk_buff *skb; u32 cqe_bcnt; u16 ci; @@ -1879,7 +1880,7 @@ static void mlx5e_handle_rx_cqe_rep(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe) skb = INDIRECT_CALL_2(rq->wqe.skb_from_cqe, mlx5e_skb_from_cqe_linear, mlx5e_skb_from_cqe_nonlinear, - rq, wi, cqe, cqe_bcnt); + rq, wi, cqe, cqe_bcnt, &mxbuf); if (!skb) { /* probably for XDP */ if (__test_and_clear_bit(MLX5E_RQ_FLAG_XDP_XMIT, rq->flags)) @@ -1907,6 +1908,7 @@ static void mlx5e_handle_rx_cqe_mpwrq_rep(struct mlx5e_rq *rq, struct mlx5_cqe64 u32 wqe_offset = stride_ix << rq->mpwqe.log_stride_sz; u32 head_offset = wqe_offset & ((1 << rq->mpwqe.page_shift) - 1); u32 page_idx = wqe_offset >> rq->mpwqe.page_shift; + struct mlx5e_xdp_buff mxbuf; struct mlx5e_rx_wqe_ll *wqe; struct mlx5_wq_ll *wq; struct sk_buff *skb; @@ -1932,7 +1934,7 @@ static void mlx5e_handle_rx_cqe_mpwrq_rep(struct mlx5e_rq *rq, struct mlx5_cqe64 skb = INDIRECT_CALL_2(rq->mpwqe.skb_from_cqe_mpwrq, mlx5e_skb_from_cqe_mpwrq_linear, mlx5e_skb_from_cqe_mpwrq_nonlinear, - rq, wi, cqe, cqe_bcnt, head_offset, page_idx); + rq, wi, cqe, cqe_bcnt, head_offset, page_idx, &mxbuf); if (!skb) goto mpwrq_cqe_out; @@ -1979,7 +1981,7 @@ mlx5e_shampo_fill_skb_data(struct sk_buff *skb, struct mlx5e_rq *rq, static struct sk_buff * mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, struct mlx5_cqe64 *cqe, u16 cqe_bcnt, u32 head_offset, - u32 page_idx) + u32 page_idx, struct mlx5e_xdp_buff *mxbuf) { struct mlx5e_frag_page *frag_page = &wi->alloc_units.frag_pages[page_idx]; u16 headlen = min_t(u16, MLX5E_RX_MAX_HEAD, cqe_bcnt); @@ -1987,7 +1989,6 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w u32 frag_offset = head_offset; u32 byte_cnt = cqe_bcnt; struct skb_shared_info *sinfo; - struct mlx5e_xdp_buff mxbuf; unsigned int truesize = 0; struct bpf_prog *prog; struct sk_buff *skb; @@ -2033,9 +2034,9 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w } } - mlx5e_fill_mxbuf(rq, cqe, va, linear_hr, linear_frame_sz, linear_data_len, &mxbuf); + mlx5e_fill_mxbuf(rq, cqe, va, linear_hr, linear_frame_sz, linear_data_len, mxbuf); - sinfo = xdp_get_shared_info_from_buff(&mxbuf.xdp); + sinfo = xdp_get_shared_info_from_buff(&mxbuf->xdp); while (byte_cnt) { /* Non-linear mode, hence non-XSK, which always uses PAGE_SIZE. */ @@ -2046,7 +2047,7 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w else truesize += ALIGN(pg_consumed_bytes, BIT(rq->mpwqe.log_stride_sz)); - mlx5e_add_skb_shared_info_frag(rq, sinfo, &mxbuf.xdp, frag_page, frag_offset, + mlx5e_add_skb_shared_info_frag(rq, sinfo, &mxbuf->xdp, frag_page, frag_offset, pg_consumed_bytes); byte_cnt -= pg_consumed_bytes; frag_offset = 0; @@ -2054,7 +2055,7 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w } if (prog) { - if (mlx5e_xdp_handle(rq, prog, &mxbuf)) { + if (mlx5e_xdp_handle(rq, prog, mxbuf)) { if (__test_and_clear_bit(MLX5E_RQ_FLAG_XDP_XMIT, rq->flags)) { struct mlx5e_frag_page *pfp; @@ -2067,10 +2068,10 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w return NULL; /* page/packet was consumed by XDP */ } - skb = mlx5e_build_linear_skb(rq, mxbuf.xdp.data_hard_start, + skb = mlx5e_build_linear_skb(rq, mxbuf->xdp.data_hard_start, linear_frame_sz, - mxbuf.xdp.data - mxbuf.xdp.data_hard_start, 0, - mxbuf.xdp.data - mxbuf.xdp.data_meta); + mxbuf->xdp.data - mxbuf->xdp.data_hard_start, 0, + mxbuf->xdp.data - mxbuf->xdp.data_meta); if (unlikely(!skb)) { mlx5e_page_release_fragmented(rq, &wi->linear_page); return NULL; @@ -2080,13 +2081,13 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w wi->linear_page.frags++; mlx5e_page_release_fragmented(rq, &wi->linear_page); - if (xdp_buff_has_frags(&mxbuf.xdp)) { + if (xdp_buff_has_frags(&mxbuf->xdp)) { struct mlx5e_frag_page *pagep; /* sinfo->nr_frags is reset by build_skb, calculate again. */ xdp_update_skb_shared_info(skb, frag_page - head_page, sinfo->xdp_frags_size, truesize, - xdp_buff_is_frag_pfmemalloc(&mxbuf.xdp)); + xdp_buff_is_frag_pfmemalloc(&mxbuf->xdp)); pagep = head_page; do @@ -2097,12 +2098,12 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w } else { dma_addr_t addr; - if (xdp_buff_has_frags(&mxbuf.xdp)) { + if (xdp_buff_has_frags(&mxbuf->xdp)) { struct mlx5e_frag_page *pagep; xdp_update_skb_shared_info(skb, sinfo->nr_frags, sinfo->xdp_frags_size, truesize, - xdp_buff_is_frag_pfmemalloc(&mxbuf.xdp)); + xdp_buff_is_frag_pfmemalloc(&mxbuf->xdp)); pagep = frag_page - sinfo->nr_frags; do @@ -2124,7 +2125,7 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w static struct sk_buff * mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, struct mlx5_cqe64 *cqe, u16 cqe_bcnt, u32 head_offset, - u32 page_idx) + u32 page_idx, struct mlx5e_xdp_buff *mxbuf) { struct mlx5e_frag_page *frag_page = &wi->alloc_units.frag_pages[page_idx]; u16 rx_headroom = rq->buff.headroom; @@ -2152,20 +2153,19 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, prog = rcu_dereference(rq->xdp_prog); if (prog) { - struct mlx5e_xdp_buff mxbuf; net_prefetchw(va); /* xdp_frame data area */ mlx5e_fill_mxbuf(rq, cqe, va, rx_headroom, rq->buff.frame0_sz, - cqe_bcnt, &mxbuf); - if (mlx5e_xdp_handle(rq, prog, &mxbuf)) { + cqe_bcnt, mxbuf); + if (mlx5e_xdp_handle(rq, prog, mxbuf)) { if (__test_and_clear_bit(MLX5E_RQ_FLAG_XDP_XMIT, rq->flags)) frag_page->frags++; return NULL; /* page/packet was consumed by XDP */ } - rx_headroom = mxbuf.xdp.data - mxbuf.xdp.data_hard_start; - metasize = mxbuf.xdp.data - mxbuf.xdp.data_meta; - cqe_bcnt = mxbuf.xdp.data_end - mxbuf.xdp.data; + rx_headroom = mxbuf->xdp.data - mxbuf->xdp.data_hard_start; + metasize = mxbuf->xdp.data - mxbuf->xdp.data_meta; + cqe_bcnt = mxbuf->xdp.data_end - mxbuf->xdp.data; } frag_size = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt); skb = mlx5e_build_linear_skb(rq, va, frag_size, rx_headroom, cqe_bcnt, metasize); @@ -2288,12 +2288,14 @@ static void mlx5e_handle_rx_cqe_mpwrq_shampo(struct mlx5e_rq *rq, struct mlx5_cq bool flush = cqe->shampo.flush; bool match = cqe->shampo.match; struct mlx5e_rq_stats *stats = rq->stats; + struct mlx5e_xdp_buff mxbuf; struct mlx5e_rx_wqe_ll *wqe; struct mlx5e_mpw_info *wi; struct mlx5_wq_ll *wq; wi = mlx5e_get_mpw_info(rq, wqe_id); wi->consumed_strides += cstrides; + mxbuf.xdp.flags = 0; if (unlikely(MLX5E_RX_ERR_CQE(cqe))) { mlx5e_handle_rx_err_cqe(rq, cqe); @@ -2316,7 +2318,7 @@ static void mlx5e_handle_rx_cqe_mpwrq_shampo(struct mlx5e_rq *rq, struct mlx5_cq *skb = mlx5e_skb_from_cqe_shampo(rq, wi, cqe, header_index); else *skb = mlx5e_skb_from_cqe_mpwrq_nonlinear(rq, wi, cqe, cqe_bcnt, - data_offset, page_idx); + data_offset, page_idx, &mxbuf); if (unlikely(!*skb)) goto free_hd_entry; @@ -2377,6 +2379,7 @@ static void mlx5e_handle_rx_cqe_mpwrq(struct mlx5e_rq *rq, struct mlx5_cqe64 *cq u32 wqe_offset = stride_ix << rq->mpwqe.log_stride_sz; u32 head_offset = wqe_offset & ((1 << rq->mpwqe.page_shift) - 1); u32 page_idx = wqe_offset >> rq->mpwqe.page_shift; + struct mlx5e_xdp_buff mxbuf; struct mlx5e_rx_wqe_ll *wqe; struct mlx5_wq_ll *wq; struct sk_buff *skb; @@ -2404,7 +2407,7 @@ static void mlx5e_handle_rx_cqe_mpwrq(struct mlx5e_rq *rq, struct mlx5_cqe64 *cq mlx5e_skb_from_cqe_mpwrq_nonlinear, mlx5e_xsk_skb_from_cqe_mpwrq_linear, rq, wi, cqe, cqe_bcnt, head_offset, - page_idx); + page_idx, &mxbuf); if (!skb) goto mpwrq_cqe_out; @@ -2632,6 +2635,7 @@ static void mlx5i_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe) { struct mlx5_wq_cyc *wq = &rq->wqe.wq; struct mlx5e_wqe_frag_info *wi; + struct mlx5e_xdp_buff mxbuf; struct sk_buff *skb; u32 cqe_bcnt; u16 ci; @@ -2648,7 +2652,7 @@ static void mlx5i_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe) skb = INDIRECT_CALL_2(rq->wqe.skb_from_cqe, mlx5e_skb_from_cqe_linear, mlx5e_skb_from_cqe_nonlinear, - rq, wi, cqe, cqe_bcnt); + rq, wi, cqe, cqe_bcnt, &mxbuf); if (!skb) goto wq_cyc_pop; @@ -2722,6 +2726,7 @@ static void mlx5e_trap_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe { struct mlx5_wq_cyc *wq = &rq->wqe.wq; struct mlx5e_wqe_frag_info *wi; + struct mlx5e_xdp_buff mxbuf; struct sk_buff *skb; u32 cqe_bcnt; u16 trap_id; @@ -2737,7 +2742,7 @@ static void mlx5e_trap_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe goto wq_cyc_pop; } - skb = mlx5e_skb_from_cqe_nonlinear(rq, wi, cqe, cqe_bcnt); + skb = mlx5e_skb_from_cqe_nonlinear(rq, wi, cqe, cqe_bcnt, &mxbuf); if (!skb) goto wq_cyc_pop; From patchwork Wed Mar 5 14:32:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002836 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E6EF24CEEC; Wed, 5 Mar 2025 14:33:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185224; cv=none; b=JPRZzwfyeohTayTq2dwonBB5iDZwbH+Yl6fA47zgP3Ju0OtNsztpsIcGm+vuWr2egTXtKllda8U+e0S5d4RLjMZ/ryfr1ZsSGq64ktlS848erdb6hFMisd5kIyYc6rVBSC57aa7+U2du2HrAxQJfh8J/cQRy0yPYgHI1NEG8B14= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185224; c=relaxed/simple; bh=JvyiaBoZ3tLM0Ebt8VBzodGkOeIPyWTP9kDQYbzbrEs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=OuOoBV8w1zLRcXT/uFX79CTw0iKyBRpqFYTs0L4MXUtBNm4OKb1PquGg1F6VNEY5tplUkFQK9JOrKxHIDLGq3nqcPjWpP9w1OqgcTboFOUWfSaeRPvvrOxsak2Rk6kTI82R8otBEPWlYwGPceJJaGhzRQdbQZ7rGGkVRe5DBazI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=RG6ezzxJ; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=2K++8eaj; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="RG6ezzxJ"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="2K++8eaj" Received: from phl-compute-11.internal (phl-compute-11.phl.internal [10.202.2.51]) by mailfhigh.phl.internal (Postfix) with ESMTP id 1E83F1140245; Wed, 5 Mar 2025 09:33:42 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-11.internal (MEProxy); Wed, 05 Mar 2025 09:33:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185222; x=1741271622; bh=Z2fR/H00aT8ZpO51TEqSeEcY9pKr00k8 tFgJuylz4bY=; b=RG6ezzxJOAmbAqi9JpJhLeNjS4xVVm4Fve4PzUg47vCv7vN/ D43nU3kU9po7J2+XsyS6n8JQOobQbkiC9cnqzmO+9yE+pQ7kVQ8EUkIbn4zyjYGL RrsI1qJqF0POgtcaHfpvXZcZp3Vf3g6bc0qltxSD904VSXF/7WY0Jard2UNVovhd iY4JXlJqgW3R+jc1owUKYvT0KLHPGvqiJIVkm9owQs8blUtHjlMEj3bJwgZFat75 XAmPuY2PfX9DEyWATWRJ6lPG2IgQsC24+WaAPlm0t+jKXBHd81NfGEUP6Rc+ZhPr 19QBwzEC/iVNOdRemaY6Hi1q+6jI2B1PdqLTrg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185222; x= 1741271622; bh=Z2fR/H00aT8ZpO51TEqSeEcY9pKr00k8tFgJuylz4bY=; b=2 K++8eajDttg3KCucbhm9iIMRqlpeiMTo8TWYsjap5fPX8AWKVhebFkkWEtHEDFnp fNY96IwafQUQ2EkGO2VbtdfLxfr4UTYA0zSbZHqFiymrJirQsK2ArqysxXrjmfiR ELBtBg1WckTRtsdRdXTm1uKm2ceHb78DVsoZjLQaj+iJhgWORrh+CQ9tHsSFUn/i HL6cMHs0k54bDN6epoJbQPkZSDNGN4u66aO0o0xybaTLMZHLd2Pmy72czkIy+v+p GtEFY3kFvbOb6Jlil+3fbPJ8Pc6WbkOH6ewE6zayZtUpRd05FQoOk12SXKvJAPmH 7X4lXI0g9H/Jt6omEkH2w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:40 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:11 +0100 Subject: [PATCH RFC bpf-next 14/20] mlx5: Propagate trait presence to skb Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-14-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre Call the common xdp_buff_update_skb() helper. Signed-off-by: Arthur Fabre --- drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index 77bace3b212ae18c420a11312a5e3043b5e3f4ae..4ced9109a8f2a047992ab96fa533ad2a7283bb91 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -1632,7 +1632,8 @@ static inline void mlx5e_complete_rx_cqe(struct mlx5e_rq *rq, static inline struct sk_buff *mlx5e_build_linear_skb(struct mlx5e_rq *rq, void *va, u32 frag_size, u16 headroom, - u32 cqe_bcnt, u32 metasize) + u32 cqe_bcnt, u32 metasize, + struct mlx5e_xdp_buff *mxbuf) { struct sk_buff *skb = napi_build_skb(va, frag_size); @@ -1646,6 +1647,8 @@ struct sk_buff *mlx5e_build_linear_skb(struct mlx5e_rq *rq, void *va, if (metasize) skb_metadata_set(skb, metasize); + if (mxbuf) + xdp_buff_update_skb(&mxbuf->xdp, skb); return skb; } @@ -1696,7 +1699,7 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi, cqe_bcnt = mxbuf->xdp.data_end - mxbuf->xdp.data; } frag_size = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt); - skb = mlx5e_build_linear_skb(rq, va, frag_size, rx_headroom, cqe_bcnt, metasize); + skb = mlx5e_build_linear_skb(rq, va, frag_size, rx_headroom, cqe_bcnt, metasize, mxbuf); if (unlikely(!skb)) return NULL; @@ -1772,7 +1775,7 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi skb = mlx5e_build_linear_skb(rq, mxbuf->xdp.data_hard_start, rq->buff.frame0_sz, mxbuf->xdp.data - mxbuf->xdp.data_hard_start, mxbuf->xdp.data_end - mxbuf->xdp.data, - mxbuf->xdp.data - mxbuf->xdp.data_meta); + mxbuf->xdp.data - mxbuf->xdp.data_meta, mxbuf); if (unlikely(!skb)) return NULL; @@ -2071,7 +2074,7 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w skb = mlx5e_build_linear_skb(rq, mxbuf->xdp.data_hard_start, linear_frame_sz, mxbuf->xdp.data - mxbuf->xdp.data_hard_start, 0, - mxbuf->xdp.data - mxbuf->xdp.data_meta); + mxbuf->xdp.data - mxbuf->xdp.data_meta, mxbuf); if (unlikely(!skb)) { mlx5e_page_release_fragmented(rq, &wi->linear_page); return NULL; @@ -2168,7 +2171,7 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, cqe_bcnt = mxbuf->xdp.data_end - mxbuf->xdp.data; } frag_size = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt); - skb = mlx5e_build_linear_skb(rq, va, frag_size, rx_headroom, cqe_bcnt, metasize); + skb = mlx5e_build_linear_skb(rq, va, frag_size, rx_headroom, cqe_bcnt, metasize, mxbuf); if (unlikely(!skb)) return NULL; @@ -2202,7 +2205,7 @@ mlx5e_skb_from_cqe_shampo(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, dma_sync_single_range_for_cpu(rq->pdev, dma_addr, 0, frag_size, rq->buff.map_dir); net_prefetchw(hdr); net_prefetch(data); - skb = mlx5e_build_linear_skb(rq, hdr, frag_size, rx_headroom, head_size, 0); + skb = mlx5e_build_linear_skb(rq, hdr, frag_size, rx_headroom, head_size, 0, NULL); if (unlikely(!skb)) return NULL; From patchwork Wed Mar 5 14:32:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002837 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78CE524E4BD; Wed, 5 Mar 2025 14:33:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185226; cv=none; b=qdhPm8QJ/fEYa21dBdCz8LUdIWUuVmQt+a7dcJ7VA8c6CMpjZXd1fpgwj4SIiOFS/6DnCga8Pe8Aj07KGeYhrhO6zu1W8SbQtt2DOCTP8xxtUpL6u4c1Co0wKQIw/H+He51I4Pl5cCHImbtlmF+eBwx3pVM4pj0ag53HDqueK+U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185226; c=relaxed/simple; bh=opUvoUUN/i+dA0gP6MLpRSGC8WGZHIofqJRtStODJ4c=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=eWpWmFqOk1lBTLyAr+uWR8y9jZ1c9qADjGt+kiZ/BDskzGFhRfFTUs6SGcQh6erJ5Z+21bKN6OFGkGW1r9Idg+K4PClwmzf8Efiu8OX7ouIU6e/rRq6vrElMtq3CtZpyFbvY1551zR8Cxxytrh2LuqoN0bvCfnu4NUQmnfVXQnE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=jOoZ5ByZ; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=hPfZRctx; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="jOoZ5ByZ"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="hPfZRctx" Received: from phl-compute-09.internal (phl-compute-09.phl.internal [10.202.2.49]) by mailfhigh.phl.internal (Postfix) with ESMTP id C60A9114020B; Wed, 5 Mar 2025 09:33:43 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-09.internal (MEProxy); Wed, 05 Mar 2025 09:33:43 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185223; x=1741271623; bh=mxfNcIt84UiMaaAtDgllbXKyl8R/X/9y mdJeTAyEdEo=; b=jOoZ5ByZ+uopIJRSW0mU3lYLAS5Zz/cukTjPrC6tZGRELHvT 11NOrp9766ESIx48OP31VIqTeP0mempanDggU8hWI8+CgajhFZYp03nuSf2pHswr j+5O7bKjb3noTLHzPQhg6rK3XllOIezueUTO/e2heCE4T77GElpdQ8nivu+jFdwH M+WerUGP7AgnSZIUuOy9STvVJFBUAyS8zwWUG/cNnvrrlO1IOP8V0JnaNwr3CvwZ NaBTYZHl5IPMu1jL+5/cl9EiwaEkLtsOZUdiTzv3RPEL2kcNGiPdZSvzj6nZTB0l eIyCtRUepq6D8qYIRnHZ6RbZCAmaYBQLish2sg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185223; x= 1741271623; bh=mxfNcIt84UiMaaAtDgllbXKyl8R/X/9ymdJeTAyEdEo=; b=h PfZRctxP6eRJABjT0Gl+0fVvfs+/vh0e9KQ9CuBL2osVyDr/hvzYtqYZhKEPZYNL cx7TDaz0N8OgDLoz0xwKJ+61tXkBRzzXPiBj8vDo4PBUUvCImsp95FrgAyL28A/C tZGNIy+r0bbi5CTvCeh4VifiaShsFRVqmw1Su/DZzj8eXXoiYepKcI+66kvyqbBJ Jv0VPr7nQN+EuRnOdsrcW0eYdi9AipDsyTYJmsu+j+oqxuCZMUfBiP9/gqlcxy1a 4OnH49HuRI5qPVEh/pWw6efTfbuMX+nVyo2Wm5C9kBF87q/kJ0H2W+8MePa4vBUA uAfrjuEG6rmwIbCtFCfYg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdekucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedvnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:42 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:12 +0100 Subject: [PATCH RFC bpf-next 15/20] xdp generic: Propagate trait presence to skb Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-15-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre Call the common xdp_buff_update_skb() / xdp_frame_update_skb() helpers. Signed-off-by: Arthur Fabre --- net/core/dev.c | 1 + 1 file changed, 1 insertion(+) diff --git a/net/core/dev.c b/net/core/dev.c index 1b252e9459fdbde42f6fb71dc146692c7f7ec17a..ecf994374b5a8721c1aacc6e4267a91775be3a25 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -5247,6 +5247,7 @@ u32 bpf_prog_run_generic_xdp(struct sk_buff *skb, struct xdp_buff *xdp, metalen = xdp->data - xdp->data_meta; if (metalen) skb_metadata_set(skb, metalen); + xdp_buff_update_skb(xdp, skb); break; } From patchwork Wed Mar 5 14:32:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002838 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fout-a2-smtp.messagingengine.com (fout-a2-smtp.messagingengine.com [103.168.172.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E5202512EB; Wed, 5 Mar 2025 14:33:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.145 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185227; cv=none; b=qf9+BgiyU2nFegDBSByfEu3I0Q0kIgEtDX+WoqKXZbz9LEPMNddif7Wp7bVOc5Rx58X4N8idOrvJd73gKglqArUJ/6DpvcFG2TFST2UVcUkXsiqq8BJeNIFYYgX+THjw9ufQNbydJ3sQPdES9vQDBqaM//gIUs5DnVnYpk0PQoo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185227; c=relaxed/simple; bh=aAWiz1/Yhp/NcPIDekOVc0qpgZDJ2ocfHx18Eg04Eos=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=c3e9ptTlegrhsz1gmq1lzBUkYhDIWTRr7AykE1fkzjJ8wNtO90cvQG4tEYjlqY0mwu/BLuDZkqNi0s9zj2r9oyEXY3uiJxwvmHgmk7qgBGgcmV7uRLdheUmHOg1fa0fMrewHfY/tZpXpLQU9+O7Z/FDOdmqW+ZvI9QGeihPnSkw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=e/z4DcnZ; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=w79b3t6k; arc=none smtp.client-ip=103.168.172.145 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="e/z4DcnZ"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="w79b3t6k" Received: from phl-compute-09.internal (phl-compute-09.phl.internal [10.202.2.49]) by mailfout.phl.internal (Postfix) with ESMTP id 829F813826FD; Wed, 5 Mar 2025 09:33:45 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-09.internal (MEProxy); Wed, 05 Mar 2025 09:33:45 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185225; x=1741271625; bh=P8eFBWeyD2bEzgGdiWCgHYqErqW8eiFg 374J4YEEeKg=; b=e/z4DcnZRHGAJsSfIPZ3ecQeNwHNrmBlfzqn7N7Wn01MWvvT 0Sy2TgyrXDp3GL4ZOUwn4pTeegiHddpPAjrQ8k+9ZbYRjqdm9Zo+WX8yndP4acel PRYbSd5GhARqGMWrRSJcG60awibNySQi65tIH4FyqmTi46wBEd/3K7pk6L67ofnP HAPQ+3Qtpd8UbnUhmp6ltT0u9MJQrlP7WklKPvxeZh84O8YtMl+rOGTp7cyrW6sN pUj4x8RLj4LVeLYA8RytpQcMLM5lW4BkDJORf5+vlc8IPHz1D6S0ObhoSe8NFfQc /EwVL9po/wy22qNXbw6PRIsDSPj/dxPvKNv8xg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185225; x= 1741271625; bh=P8eFBWeyD2bEzgGdiWCgHYqErqW8eiFg374J4YEEeKg=; b=w 79b3t6ki4fV4QlhttHJSMth0t0Q7j8VWdzIgdds53DRhohgoHNAqpvcv2ZG7e9+1 S+GXXHqYqVuG8OPlctBtj2MKygkfkEcgzqUNjvvMP7GcVO9ohgoOKsnU4Sor3mly VukusamhVKMQTaEgxRM1YAK81AKBgBJh8D/14nk8X21bhNycJ4CQPwHbgpJORNPE v4rTwt2bl4Jr/hllaKcHWky8IA3DkaRaff1dz2IQdjBoBIN1B4aEhgQQymgzcy8Y WYtuL8xRR6xRNf2bvKLvminU38tPKabHGw/T40stDZfWjGLjhQg7PKxpBFxg7mUh 9NOQ1DDJPVeV2uzSlmI4Q== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdekucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedvnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:43 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:13 +0100 Subject: [PATCH RFC bpf-next 16/20] trait: Support sk_buffs Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-16-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre Hide the space used by traits from skb_headroom(): that space isn't actually usable. Preserve the trait store in pskb_expand_head() by copying it ahead of the new headroom. The struct xdp_frame at the start of the headroom isn't needed anymore, so we can overwrite it with traits, and introduce a new flag to indicate traits are stored at the start of the headroom. Cloned skbs share the same packet data and headroom as the original skb, so changes to traits in one would be reflected in the other. Is that ok? Are there cases where we would want a clone to have different traits? For now, prevent clones from using traits. Signed-off-by: Arthur Fabre --- include/linux/skbuff.h | 25 +++++++++++++++++++++++-- net/core/skbuff.c | 25 +++++++++++++++++++++++-- 2 files changed, 46 insertions(+), 4 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index d7dfee152ebd26ce87a230222e94076aca793adc..886537508789202339c925b5613574de67b7e43c 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -39,6 +39,7 @@ #include #include #include +#include /** * DOC: skb checksums @@ -729,6 +730,8 @@ enum skb_traits_type { SKB_TRAITS_NONE, /* Trait store in headroom, offset by sizeof(struct xdp_frame) */ SKB_TRAITS_AFTER_XDP, + /* Trait store at start of headroom */ + SKB_TRAITS_AT_HEAD, }; /** @@ -1029,7 +1032,7 @@ struct sk_buff { __u8 csum_not_inet:1; #endif __u8 unreadable:1; - __u8 traits_type:1; /* See enum skb_traits_type */ + __u8 traits_type:2; /* See enum skb_traits_type */ #if defined(CONFIG_NET_SCHED) || defined(CONFIG_NET_XGRESS) __u16 tc_index; /* traffic control index */ #endif @@ -2836,6 +2839,18 @@ static inline void *pskb_pull(struct sk_buff *skb, unsigned int len) void skb_condense(struct sk_buff *skb); +static inline void *skb_traits(const struct sk_buff *skb) +{ + switch (skb->traits_type) { + case SKB_TRAITS_AFTER_XDP: + return skb->head + _XDP_FRAME_SIZE; + case SKB_TRAITS_AT_HEAD: + return skb->head; + default: + return NULL; + } +} + /** * skb_headroom - bytes at buffer head * @skb: buffer to check @@ -2844,7 +2859,13 @@ void skb_condense(struct sk_buff *skb); */ static inline unsigned int skb_headroom(const struct sk_buff *skb) { - return skb->data - skb->head; + int trait_size = 0; + void *traits = skb_traits(skb); + + if (traits) + trait_size = traits_size(traits); + + return skb->data - skb->head - trait_size; } /** diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 7b03b64fdcb276f68ce881d1d8da8e4c6b897efc..83f58517738e8ff12990c28b09336ed44f4be32a 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -1515,6 +1515,19 @@ static struct sk_buff *__skb_clone(struct sk_buff *n, struct sk_buff *skb) atomic_inc(&(skb_shinfo(skb)->dataref)); skb->cloned = 1; + /* traits would end up shared with the clone, + * and edits would be reflected there. + * + * Is that ok? What if the original skb and the clone take different paths? + * Does that even happen? + * + * If that's not ok, we could copy the traits and store them in an extension header + * for clones. + * + * For now, pretend the clone doesn't have any traits. + */ + skb->traits_type = SKB_TRAITS_NONE; + return n; #undef C } @@ -2170,7 +2183,7 @@ int pskb_expand_head(struct sk_buff *skb, int nhead, int ntail, unsigned int osize = skb_end_offset(skb); unsigned int size = osize + nhead + ntail; long off; - u8 *data; + u8 *data, *head; int i; BUG_ON(nhead < 0); @@ -2187,10 +2200,18 @@ int pskb_expand_head(struct sk_buff *skb, int nhead, int ntail, goto nodata; size = SKB_WITH_OVERHEAD(size); + head = skb->head; + if (skb->traits_type != SKB_TRAITS_NONE) { + head = skb_traits(skb) + traits_size(skb_traits(skb)); + /* struct xdp_frame isn't needed in the headroom, drop it */ + memcpy(data, skb_traits(skb), traits_size(skb_traits(skb))); + skb->traits_type = SKB_TRAITS_AT_HEAD; + } + /* Copy only real data... and, alas, header. This should be * optimized for the cases when header is void. */ - memcpy(data + nhead, skb->head, skb_tail_pointer(skb) - skb->head); + memcpy(data + nhead, head, skb_tail_pointer(skb) - head); memcpy((struct skb_shared_info *)(data + size), skb_shinfo(skb), From patchwork Wed Mar 5 14:32:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002839 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0057F251796; Wed, 5 Mar 2025 14:33:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185229; cv=none; b=R6ebViwLPIFgtqw+0u46JlRtd4imM6TsRhTUMejYAi2fbosI3jFMJYg6p0/6YCGEaTCjeOtopu6KdsXRusi5TS/ZQJiyYNtBkbkl1BQPXdcbKuu5aAYD6C4ar8M3D1VZ1NiWXpAI+UWyLugobZBjvae5bpvKWx8d4Gowcu8EBv0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185229; c=relaxed/simple; bh=RcDbTovihb8NIypJgTsn+XIWRQ7sPppDg7Sd/mECLjw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=nANRAV2K7pTP6g1J8pznCLbq3rcosiV9DJnmT3c+09kbreisze5GBrmlOTNS8kgZN8W0fnh4qulVI0ZCAxO5ob9pLUZzywHRwJjFW61FXVeQd41VNAr+oHXRbUXuKeCkDZxH0qyx1u3p7W6IEZEjcjdMpw3fYbvnBX1Gfb3JNag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=jvEzpgfG; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=NQijt9bL; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="jvEzpgfG"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="NQijt9bL" Received: from phl-compute-08.internal (phl-compute-08.phl.internal [10.202.2.48]) by mailfhigh.phl.internal (Postfix) with ESMTP id 29281114024E; Wed, 5 Mar 2025 09:33:47 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-08.internal (MEProxy); Wed, 05 Mar 2025 09:33:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185227; x=1741271627; bh=H3Q3mi5j6Eq1sW1Wo8iuFfIlsWGGiFrH jtw00R758tk=; b=jvEzpgfGGddXOe+RXC06NBvj77/3ApIUM3Kn6BWExI0r3O63 9KwmdRYdMy/9Hy7nkGa/URAihXSnxJeU4qMPT4OywfFm4yjnObB7U1QNgfx9GDVF jiWP0dLhJhnoFhc1ltyFX3NsFZfXuhNTzEhzIxqrmbGf7FLt5KJddo+24GBzQZdF vP0W/k7YbNDj3F0GdwYbs6lOOpsBEa2EMH4ClzcumNKE+ZoEEU1G2ljW8BM6dmEq /XoYjpNolMqU4Pu6MsE+oxY1K0j2XEtlQy2dZsDOT1a4vLSHUU8Vfuxpd9j5Cyp6 YYIA97wve/AHSyB4QHZxPj2Wk7+gCU+1Kk7gCw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185227; x= 1741271627; bh=H3Q3mi5j6Eq1sW1Wo8iuFfIlsWGGiFrHjtw00R758tk=; b=N Qijt9bLdnAGskPWmyPMyDAwpHvjmoFjstzqBwNMlr/IYPls0a3H6QGichYS7I0Rk 2aF+QunffpEtVZd8JODckhC2/9ErVupf8MTbK+fVqLvH8CfIvOK+IiGqadRLQCss xk0pa9n70UoVjps/MePn+ZWfmpvuyFI7UXYZ3Bj+5uVkIkT94ooPn7G3M1UxmXvW XWGtaf8UbFaAtOMqSh463FbuTHDwuCHj8rq4N4hsPBiilT/lqyiaRFzic1DCfaY5 zCg/qO62A9GQ18Qc1zUQ1d4jkRFWssQCcPxmdhqVmI0shWVeDUh8Jbm50hDjzXCR d4rNdcDXEA8lyg6vYtLUA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedvnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:45 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:14 +0100 Subject: [PATCH RFC bpf-next 17/20] trait: Allow socket filters to access traits Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-17-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre Add kfuncs to allow socket filter programs access to traits in an skb. Signed-off-by: Arthur Fabre --- net/core/filter.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/net/core/filter.c b/net/core/filter.c index 79b78e7cd57fd78c6cc8443da54ae96408c496b0..47f18fb0e23c2c19d26f9b408653c6756a5110c7 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -12063,6 +12063,39 @@ __bpf_kfunc int bpf_sk_assign_tcp_reqsk(struct __sk_buff *s, struct sock *sk, #endif } +__bpf_kfunc int bpf_skb_trait_set(const struct sk_buff *skb, u64 key, + const void *val, u64 val__sz, u64 flags) +{ + void *traits = skb_traits(skb); + + if (!traits) + return -EOPNOTSUPP; + + return trait_set(traits, skb_metadata_end(skb) - skb_metadata_len(skb), + key, val, val__sz, flags); +} + +__bpf_kfunc int bpf_skb_trait_get(const struct sk_buff *skb, u64 key, + void *val, u64 val__sz) +{ + void *traits = skb_traits(skb); + + if (!traits) + return -EOPNOTSUPP; + + return trait_get(traits, key, val, val__sz); +} + +__bpf_kfunc int bpf_skb_trait_del(const struct sk_buff *skb, u64 key) +{ + void *traits = skb_traits(skb); + + if (!traits) + return -EOPNOTSUPP; + + return trait_del(traits, key); +} + __bpf_kfunc_end_defs(); int bpf_dynptr_from_skb_rdonly(struct __sk_buff *skb, u64 flags, @@ -12082,6 +12115,9 @@ int bpf_dynptr_from_skb_rdonly(struct __sk_buff *skb, u64 flags, BTF_KFUNCS_START(bpf_kfunc_check_set_skb) BTF_ID_FLAGS(func, bpf_dynptr_from_skb, KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_skb_trait_set) +BTF_ID_FLAGS(func, bpf_skb_trait_get) +BTF_ID_FLAGS(func, bpf_skb_trait_del) BTF_KFUNCS_END(bpf_kfunc_check_set_skb) BTF_KFUNCS_START(bpf_kfunc_check_set_xdp) From patchwork Wed Mar 5 14:32:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002840 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fout-a2-smtp.messagingengine.com (fout-a2-smtp.messagingengine.com [103.168.172.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DCDD72517B6; Wed, 5 Mar 2025 14:33:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.145 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185231; cv=none; b=ingz+4bw1Vqfa3Z8TZtMTUq3gBe1jCx8lWIoC0CD0EzSVaIXgUIZ1EpV9ayCDzNAKQhmtTpQ8itCUzt4la1v3oodHhO6m7mXIG37dKumv+t+/IgHRXMVGP7gI4UtlUkOq8Uo5s97316of0qfW9wZnlZNtGPMCHkSWbU9WQRYYv8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185231; c=relaxed/simple; bh=HicSASNYdZDA0RtzA5b9cOjfR7//E6FUJk6ywDhapTA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Ei/TDAdTx5YWziJm6CMgl/mSzaXRToT3a4t/QbqZVg9oyoV7EwaBU70VTuqVJY3IDe0TaXDzLW+AMMv/v9UHg4yPGQFYuoCUyfGbWPUxTypsBxoyctTTN36xGYnkSOTqSf/XRoUvfP6kmYVXwDPRm7Zewr0l832eSC3M62TZ/+Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=AlTOl8v4; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=VJr5D2xb; arc=none smtp.client-ip=103.168.172.145 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="AlTOl8v4"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="VJr5D2xb" Received: from phl-compute-06.internal (phl-compute-06.phl.internal [10.202.2.46]) by mailfout.phl.internal (Postfix) with ESMTP id DDD2C1382714; Wed, 5 Mar 2025 09:33:48 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-06.internal (MEProxy); Wed, 05 Mar 2025 09:33:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185228; x=1741271628; bh=1yiODElVD2neYVmTMEcUZKN6LweCz1z0 YkWtPJecTq8=; b=AlTOl8v4ziM2qlT6JwwOu/+iCiu56Pt3q4/sES1Q53otv8RK bdxa2kSPz0kZYTxY7jsNkUufBI9oC+ZoNV0bElIUH7FDrlP31M1T2NINkRov2Q7S uk2O4aeBdjvceFM18hjVaFyczLgDG+7e6KLBE8kVVs3zOZH2kbwVvf/P9XYh7hSH 3ZDxIULJQ8qkUGFSPAQIfLcZmeLoP92T9zN761xxgd8wM3U7khbzwB8gMGS6OOQO uGFWs+5oznxxAs8tQow/myoMkA+obgrWlblGB56qSnG9RWVvOfc8XblNiAtFjg3U R+GPRfZcGDxVlsKdiQ58WJTLWkhMQRJSLY+xHQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185228; x= 1741271628; bh=1yiODElVD2neYVmTMEcUZKN6LweCz1z0YkWtPJecTq8=; b=V Jr5D2xbBDkJuFCkEXnIIRf5wZkH+At/ATKXpY8cdWAPR1elcpgkbmCSesSwt966w XEIbiTJMn47iBo+qupAKUH6XehftOG8UkbZjEJN2gTt71r3mkIfbalA0g4RKF5kz t1MddtDqKAlHDT3eC+87ckPQgv87vzEHj9Cn7iP1Sm83rXoHVhV0EYBYkJuKWsax zszA1wGtxPuVwzmKF9gsYkPt9aetyW/pIJ1hkWoa0KmOiSZAKqR3KlK9D8bNn5bf AjTm1ZGAWGJZv5Lomj5U2GLfHXB6+lQ9idcTh6YQc+vZ2iqUYqj+XWOPgItvqYHq BG9LkGfL8kkNclVKf/5gg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedvnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:47 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:15 +0100 Subject: [PATCH RFC bpf-next 18/20] trait: registration API Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-18-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre The "raw" key values (0-63) of traits could be exposed to userspace, and used directly in BPF programs. Similarly to allocating TCP or UDP ports to services, services that use traits could make the IDs they use configurable in their config file, and the system administrator could decided which services use which IDs. However, unlike TCP or UDP ports, if two services hardcode and use the same trait ID, we wouldn't be able to detect this and fail fast with EADDRINUSE. The services would instead overwrite each other's traits. This is intended more to kick off a discussion rather than be a fully fleshed out patch. One solution is to have the kernel provide an API for services to register traits they want to use. If the trait is already registered, we can return an appropriate error. This could take many forms (in order of increasing complexity): - A syscall to "reserve" a specific trait ID / key for use. - A syscall to dynamically request a free trait ID / key. The keys could be passed to BPF programs as constants. (what this patch implements). - Hide numerical trait IDs / keys from userspace entirely, and have them use strings. The verifier could allocate IDs for each unique string, and rewrite programs to use them (or the loader). In all cases, it isn't obvious how to deal with the lifetime of registered traits. Services need to be able to unregister traits so they can restart cleanly. But that's complicated: - Do we need to track which BPF programs use which traits, to prevent traits in use from being unregistered? How will this work if we extend netfilter and routing to support traits? - If a service unregisters a key, what happens to in flight packets that have a trait with that ID set? If the same ID is re-used, it could be mis-interpreted. On the other hand, it would also lets us specify and store per trait / key flags. This could let us handle cases like GRO by specifying policies for how to "merge" traits of GRO'd packets, eg: - TRAIT_KEEP_FIRST: keep the value of the first packet. - TRAIT_KEEP_LAST: keep the value of the last packet. - TRAIT_SUM: sum the values of all packets. This would avoid disabling GRO as soon as packets have different traits, which unfortunately happens today with XDP metadata. Many other aspects of traits can be decided later on: how to guarantee enough headroom without XDP, how to deal with GRO, skb clones, IP fragmentation... because we can implement the most restrictive option for now. That's not the case for a registration API: we can't add one once we allow users to use raw trait IDs. Signed-off-by: Arthur Fabre --- include/linux/bpf-netns.h | 12 ++++++++++ include/net/net_namespace.h | 6 +++++ include/net/netns/trait.h | 22 ++++++++++++++++++ include/uapi/linux/bpf.h | 26 ++++++++++++++++++++++ kernel/bpf/net_namespace.c | 54 +++++++++++++++++++++++++++++++++++++++++++++ kernel/bpf/syscall.c | 26 ++++++++++++++++++++++ kernel/bpf/verifier.c | 39 +++++++++++++++++++++++++++++++- 7 files changed, 184 insertions(+), 1 deletion(-) diff --git a/include/linux/bpf-netns.h b/include/linux/bpf-netns.h index 413cfa5e4b0710806f5ed56824c7488b2586e19b..775f9de5e3e2931103ca66b3f168637437872480 100644 --- a/include/linux/bpf-netns.h +++ b/include/linux/bpf-netns.h @@ -33,6 +33,8 @@ int netns_bpf_prog_attach(const union bpf_attr *attr, int netns_bpf_prog_detach(const union bpf_attr *attr, enum bpf_prog_type ptype); int netns_bpf_link_create(const union bpf_attr *attr, struct bpf_prog *prog); +int netns_bpf_trait_register(const union bpf_attr *attr); +int netns_bpf_trait_unregister(const union bpf_attr *attr); #else static inline int netns_bpf_prog_query(const union bpf_attr *attr, union bpf_attr __user *uattr) @@ -57,6 +59,16 @@ static inline int netns_bpf_link_create(const union bpf_attr *attr, { return -EOPNOTSUPP; } + +static inline int netns_bpf_trait_register(const union bpf_attr *attr) +{ + return -EOPNOTSUPP; +} + +static inline int netns_bpf_trait_unregister(const union bpf_attr *attr) +{ + return -EOPNOTSUPP; +} #endif #endif /* _BPF_NETNS_H */ diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h index f467a66abc6b16b690a99037a3dea2e355910661..6f35593c91627c2793f7ced3e6c69c6fb4409665 100644 --- a/include/net/net_namespace.h +++ b/include/net/net_namespace.h @@ -37,6 +37,7 @@ #include #include #include +#include #include #include #include @@ -193,6 +194,11 @@ struct net { /* Move to a better place when the config guard is removed. */ struct mutex rtnl_mutex; #endif + /* Traits probably shouldn't be per namespace - otherwise we'd have to explicitly clear + * them. And packet tracing should work across namespaces. + * I just didn't know where to put this. + */ + struct netns_traits traits; } __randomize_layout; #include diff --git a/include/net/netns/trait.h b/include/net/netns/trait.h new file mode 100644 index 0000000000000000000000000000000000000000..5b1d2ad4572c447dea80c285feb4da69d15693d6 --- /dev/null +++ b/include/net/netns/trait.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Traits registered to a network namespace. + */ + +#ifndef __NETNS_TRAIT_H__ +#define __NETNS_TRAIT_H__ + +#include +#include + +struct registered_trait { + bool used; + char name[BPF_OBJ_NAME_LEN]; + u32 flags; +}; + +struct netns_traits { + struct registered_trait traits[64]; +}; + +#endif /* __NETNS_TRAIT_H__ */ diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index bb37897c039398dd3568cd007586d9b088ddeb32..748ab5a1cbe0d29d890b874aacfc4ee66b082058 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -906,6 +906,21 @@ union bpf_iter_link_info { * A new file descriptor (a nonnegative integer), or -1 if an * error occurred (in which case, *errno* is set appropriately). * + * BPF_REGISTER_TRAIT + * Description + * Register a trait. Docs to make bpf_doc.py not error out. + * Return + * Registered trait key. + * + * BPF_UNREGISTER_TRAIT + * Description + * Unregister a trait. Needed so services registering traits + * can restart. + * But what happens if a trait is currently being used? + * And to in flight packets? + * Return + * -1 if an error occurred. + * * NOTES * eBPF objects (maps and programs) can be shared between processes. * @@ -961,6 +976,8 @@ enum bpf_cmd { BPF_LINK_DETACH, BPF_PROG_BIND_MAP, BPF_TOKEN_CREATE, + BPF_REGISTER_TRAIT, + BPF_UNREGISTER_TRAIT, __MAX_BPF_CMD, }; @@ -1841,6 +1858,15 @@ union bpf_attr { __u32 bpffs_fd; } token_create; + struct { /* struct used by BPF_REGISTER_TRAIT command */ + char name[BPF_OBJ_NAME_LEN]; + __u32 flags; + } register_trait; + + struct { /* struct used by BPF_UNREGISTER_TRAIT command */ + __u64 trait; + } unregister_trait; + } __attribute__((aligned(8))); /* The description below is an attempt at providing documentation to eBPF diff --git a/kernel/bpf/net_namespace.c b/kernel/bpf/net_namespace.c index 868cc2c43899713b5dbd0ac92f06d297be4b4270..0828310b45f12d1feb8c26aad2cf9cbb0fb5cdf9 100644 --- a/kernel/bpf/net_namespace.c +++ b/kernel/bpf/net_namespace.c @@ -526,6 +526,60 @@ int netns_bpf_link_create(const union bpf_attr *attr, struct bpf_prog *prog) return err; } +int netns_bpf_trait_register(const union bpf_attr *attr) +{ + struct net *net; + int i; + int ret; + + net = current->nsproxy->net_ns; + mutex_lock(&netns_bpf_mutex); + + for (i = 0; i < 64; i++) { + if (net->traits.traits[i].used) + continue; + + net->traits.traits[i].used = true; + net->traits.traits[i].flags = attr->register_trait.flags; + memcpy(net->traits.traits[i].name, attr->register_trait.name, + sizeof(attr->register_trait.name)); + + ret = i; + goto out_unlock; + } + ret = -ENOSPC; + +out_unlock: + mutex_unlock(&netns_bpf_mutex); + + return ret; +} + +int netns_bpf_trait_unregister(const union bpf_attr *attr) +{ + struct net *net; + int ret; + + if (attr->unregister_trait.trait > 63) + return -EINVAL; + + net = current->nsproxy->net_ns; + mutex_lock(&netns_bpf_mutex); + + if (!net->traits.traits[attr->unregister_trait.trait].used) { + ret = -ENOENT; + goto out_unlock; + } + + net->traits.traits[attr->unregister_trait.trait].used = false; + ret = 0; + +out_unlock: + mutex_unlock(&netns_bpf_mutex); + + return ret; +} + static int __net_init netns_bpf_pernet_init(struct net *net) { int type; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 57a438706215936b75c957f95f5324371aa8f436..4ae401dbb5e332343f729dee0fa7bdbe6d216f16 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -5769,6 +5769,26 @@ static int token_create(union bpf_attr *attr) return bpf_token_create(attr); } +#define BPF_REGISTER_TRAIT_LAST_FIELD register_trait.flags + +static int register_trait(union bpf_attr *attr) +{ + if (CHECK_ATTR(BPF_REGISTER_TRAIT)) + return -EINVAL; + + return netns_bpf_trait_register(attr); +} + +#define BPF_UNREGISTER_TRAIT_LAST_FIELD unregister_trait.trait + +static int unregister_trait(union bpf_attr *attr) +{ + if (CHECK_ATTR(BPF_UNREGISTER_TRAIT)) + return -EINVAL; + + return netns_bpf_trait_unregister(attr); +} + static int __sys_bpf(enum bpf_cmd cmd, bpfptr_t uattr, unsigned int size) { union bpf_attr attr; @@ -5905,6 +5925,12 @@ static int __sys_bpf(enum bpf_cmd cmd, bpfptr_t uattr, unsigned int size) case BPF_TOKEN_CREATE: err = token_create(&attr); break; + case BPF_REGISTER_TRAIT: + err = register_trait(&attr); + break; + case BPF_UNREGISTER_TRAIT: + err = unregister_trait(&attr); + break; default: err = -EINVAL; break; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 4edb2db0f889c5b236416ce0016d74fda2e09a18..da41a54731fb1eccafb295ea1a0bc1d27d92c726 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -30,6 +30,7 @@ #include #include #include +#include #include "disasm.h" @@ -12007,6 +12008,8 @@ enum special_kfunc_type { KF_bpf_iter_num_destroy, KF_bpf_set_dentry_xattr, KF_bpf_remove_dentry_xattr, + KF_bpf_xdp_trait_set, + KF_bpf_skb_trait_set, }; BTF_SET_START(special_kfunc_set) @@ -12040,6 +12043,8 @@ BTF_ID(func, bpf_iter_css_task_new) BTF_ID(func, bpf_set_dentry_xattr) BTF_ID(func, bpf_remove_dentry_xattr) #endif +BTF_ID(func, bpf_xdp_trait_set) +BTF_ID(func, bpf_skb_trait_set) BTF_SET_END(special_kfunc_set) BTF_ID_LIST(special_kfunc_list) @@ -12096,6 +12101,8 @@ BTF_ID(func, bpf_remove_dentry_xattr) BTF_ID_UNUSED BTF_ID_UNUSED #endif +BTF_ID(func, bpf_xdp_trait_set) +BTF_ID(func, bpf_skb_trait_set) static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta) { @@ -13288,7 +13295,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, { bool sleepable, rcu_lock, rcu_unlock, preempt_disable, preempt_enable; u32 i, nargs, ptr_type_id, release_ref_obj_id; - struct bpf_reg_state *regs = cur_regs(env); + struct bpf_reg_state *regs = cur_regs(env), *reg; const char *func_name, *ptr_type_name; const struct btf_type *t, *ptr_type; struct bpf_kfunc_call_arg_meta meta; @@ -13297,6 +13304,8 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, const struct btf_param *args; const struct btf_type *ret_t; struct btf *desc_btf; + struct net *net; + bool trait_used; /* skip for now, but return error when we find this in fixup_kfunc_call */ if (!insn->imm) @@ -13463,6 +13472,34 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, } } + if (meta.func_id == special_kfunc_list[KF_bpf_xdp_trait_set] || + meta.func_id == special_kfunc_list[KF_bpf_skb_trait_set]) { + reg = &cur_regs(env)[BPF_REG_2]; + if (reg->type != SCALAR_VALUE || !tnum_is_const(reg->var_off)) { + verbose(env, "trait_set() key is not a known constant\n"); + return -EINVAL; + } + + if (reg->var_off.value > 63) { + verbose(env, "trait_set() key %llu invalid\n", reg->var_off.value); + return -EINVAL; + } + + net = current->nsproxy->net_ns; + mutex_lock(&netns_bpf_mutex); + trait_used = net->traits.traits[reg->var_off.value].used; + mutex_unlock(&netns_bpf_mutex); + + /* Checking in the verifier is good for runtime performance, but what happens if + * a trait is unregistered? + * Should we track which traits are used by BPF programs and prevent it? + */ + if (!trait_used) { + verbose(env, "trait_set() key %llu is not registered\n", reg->var_off.value); + return -EINVAL; + } + } + for (i = 0; i < CALLER_SAVED_REGS; i++) mark_reg_not_init(env, regs, caller_saved[i]); From patchwork Wed Mar 5 14:32:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002841 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fout-a2-smtp.messagingengine.com (fout-a2-smtp.messagingengine.com [103.168.172.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E5B524EA91; Wed, 5 Mar 2025 14:33:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.145 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185233; cv=none; b=UkQv2/7ewKebOuIuP4O0Y5cp6gHzkVHzLIigtyhtBqXZrIbBhEcu+FQXzT3HhhaWnh7AXMJsI7HfNG1k/wNPz+zDqSG09lpSPICFngIlSBQlYUGahPz51NRohOzXfu2e+YHL61KhmpYUMztFWZyICuy2qDYk65IYGS9bTdDsKwU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185233; c=relaxed/simple; bh=ALNDmKMBD9mXTWg2ZSDgD5TqaU58kfSRMxImvMs65hA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=aJxkzVVxjMmGMrvFO91ATpGCGebpphmc9ijkJEez3G/rWGbHMT5WS60lvS2bpO++6x0yI3AywJDWEcNMpcWZPhkuh3XioVbS+mGTyX5BUJdQ2HudQX8kp7shLIOsEBaZUarSO4a0XTfj+gWt/jPchwtjU2gQ/7hVGYdPII5x1og= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=NEw5t0hH; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=8Pci5FnV; arc=none smtp.client-ip=103.168.172.145 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="NEw5t0hH"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="8Pci5FnV" Received: from phl-compute-08.internal (phl-compute-08.phl.internal [10.202.2.48]) by mailfout.phl.internal (Postfix) with ESMTP id 9182413826F7; Wed, 5 Mar 2025 09:33:50 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-08.internal (MEProxy); Wed, 05 Mar 2025 09:33:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185230; x=1741271630; bh=NE5bh1kmLcgI9x0jYIzVg9x+KrpOUoIj lroPwL9ii4A=; b=NEw5t0hHy4BO1ftm2Qvb7lPBIzBT82yXQa9VCZv71Xfn3cAv DGWrPpB0CGngj5Wo6+y5TBWeQX7Y8hfMRyp+3ZiYyqkzOKGRsXMG7zmKqt/IIwk4 568CaZy6Y/RjjwgV5lGhcbGno4IWF7m7EDIpnKulDyzT7li3RtwZOieUkgFO1a17 ScgSjrT77BV/zE3SlewiDlz+gF2bR+pNvSoaOedAd931TVcEG5/uJ9lnLB5vM0H7 ggMvSaQCDZQ13DL5WzNROmvU2S1WjN8xAswxR2Bwj7MW8jhheoCTcETerc3E39hM 4g+35XvYAj3UViACmdkcjOeh34jN/OmAhbOfGg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185230; x= 1741271630; bh=NE5bh1kmLcgI9x0jYIzVg9x+KrpOUoIjlroPwL9ii4A=; b=8 Pci5FnVGMmjC7SOzLD4smYLolqas6nkjEuJwWiooFm80rG4Wwx/P9b4roNKxGNRf 68SAedXjtEXIPUjzQWfJfSc0oMt5THRdj2lHJ2P+q7aE9RsJFkoZvN30u718DJYB 3zdoExiK7aFNsjJnS+0/3BfIiiG1tKwZqaJEsoEnR9nJ7mkBnqRvStQeZs29P1St sUtcv5xvU/Ij3rxu5LNQh26PjWjXnmganAG4muAmYNICx1hK7N5OBcXz1o3qxdov DyIqWS4Htjb4ZhdhVDxqEpukf7DHYmJMUfSY9E360AEQDaVo1KCAKWxQi3ZjNXEx uGnPiPOycfgFRBA2A9Bxw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpeefnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:49 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:16 +0100 Subject: [PATCH RFC bpf-next 19/20] trait: Sync linux/bpf.h to tools/ for trait registration Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-19-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre Now that traits are required to be registered, the benchmarks and selftests will need to register them. Signed-off-by: Arthur Fabre --- tools/include/uapi/linux/bpf.h | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index bb37897c039398dd3568cd007586d9b088ddeb32..748ab5a1cbe0d29d890b874aacfc4ee66b082058 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -906,6 +906,21 @@ union bpf_iter_link_info { * A new file descriptor (a nonnegative integer), or -1 if an * error occurred (in which case, *errno* is set appropriately). * + * BPF_REGISTER_TRAIT + * Description + * Register a trait. Docs to make bpf_doc.py not error out. + * Return + * Registered trait key. + * + * BPF_UNREGISTER_TRAIT + * Description + * Unregister a trait. Needed so services registering traits + * can restart. + * But what happens if a trait is currently being used? + * And to in flight packets? + * Return + * -1 if an error occurred. + * * NOTES * eBPF objects (maps and programs) can be shared between processes. * @@ -961,6 +976,8 @@ enum bpf_cmd { BPF_LINK_DETACH, BPF_PROG_BIND_MAP, BPF_TOKEN_CREATE, + BPF_REGISTER_TRAIT, + BPF_UNREGISTER_TRAIT, __MAX_BPF_CMD, }; @@ -1841,6 +1858,15 @@ union bpf_attr { __u32 bpffs_fd; } token_create; + struct { /* struct used by BPF_REGISTER_TRAIT command */ + char name[BPF_OBJ_NAME_LEN]; + __u32 flags; + } register_trait; + + struct { /* struct used by BPF_UNREGISTER_TRAIT command */ + __u64 trait; + } unregister_trait; + } __attribute__((aligned(8))); /* The description below is an attempt at providing documentation to eBPF From patchwork Wed Mar 5 14:32:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arthur Fabre X-Patchwork-Id: 14002842 X-Patchwork-Delegate: bpf@iogearbox.net Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2382C252903; Wed, 5 Mar 2025 14:33:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185234; cv=none; b=k/kwlzZAXODwRYwQGvcx9lyipvzcj3nVVjH5c/LBu+oa8V9aiYfSVEg9gIcHvyM6vhuPrpcvDa9bqk0TkYDgdjpTFAnUJ7Tf3EZeZV4o4b2ysNFIZO4NFHxCr35XuDmGXAcvU1hdZ+ceTyH9qU75Gf79ePvpNdE0MO82SAIxuc4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741185234; c=relaxed/simple; bh=UyzCQ+1TEf63QXbFVHZfqRUApOl37dvhkdRu2DAyPbM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dQ4iRbG/dtliojw2cVcMjKbpVRgiBNkxWCh0bWmD54atAcmiWCzh04ywXcyj6xd0hcNTZr7Llg5OR66sxSsviM+miV6ttlTLMr7R1kGSwuW8E/eQoRt6vmM9DcjV8n6Z0L0kzH1ciOcIAO1NcLltO4/etWIQe/N4PANoqLh3L24= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com; spf=pass smtp.mailfrom=arthurfabre.com; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b=I2MnrdAO; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=Bo0nl3QL; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arthurfabre.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=arthurfabre.com header.i=@arthurfabre.com header.b="I2MnrdAO"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="Bo0nl3QL" Received: from phl-compute-13.internal (phl-compute-13.phl.internal [10.202.2.53]) by mailfhigh.phl.internal (Postfix) with ESMTP id 3E76B1140252; Wed, 5 Mar 2025 09:33:52 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-13.internal (MEProxy); Wed, 05 Mar 2025 09:33:52 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arthurfabre.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1741185232; x=1741271632; bh=k1YGAnSYA9FoYxracqWXHS34OxsoexW0 9UyZ0uIaTJ8=; b=I2MnrdAOZkZaNH/iUnOV4F55V1G2OXXm5GUBNHQ6mX4rk3Ye VynOd2PPBNDRnEQfE3vtWwwTuKaDj/qs4U56SZDnZ8ipZO4TM9ECDlIp1I2I5FB9 2zOoXLIrtN4feG3VVcjOQJH9jFIzLuc2abLILsLcfmwN30atvHjFPTGyFgXGolWO +slZ9/vqu0ZxNMIBGsbPaU1okhfvVQGEkcwF6p6J/YpOdoVKwEP50XwI4IJHjtxp Ft26UZrUmkUqmZBlHwtZAV9LbakPKe/s92VaZRERHbeKVHntDTKB4ONflbq36Zh3 doTJufMdV3qVWdmWdJcDnadQoiW87llH+T4nag== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1741185232; x= 1741271632; bh=k1YGAnSYA9FoYxracqWXHS34OxsoexW09UyZ0uIaTJ8=; b=B o0nl3QLVpVkTz5M4HklP6ZI0WywbMyLM6mJP7T8DorZ0cUJP6YTOUwrf+wTfVPpV RyY6/Y44vmwk5yO4KPu+3W+AVVsxFjPyYDYN0Lss2iIwHj6vjiY+DIaCRxsgEoRH BqouuuvCOvDlQtXgWLQEYCNKgOUFTm77S3jTvf3NhfCAAM4iIxAnlt5n1eVVeP6/ LIMRfDNigzGPiUAW1ve6jA8XizruXOH5myHjEZHT07BkMtvfUc65eZ0FuCVEqhwf QBrZkbSIo7zU9do/y1KYgrCaiMTIDzCvPjGfUnQ8wSRD9WCmZS8Qb/+0npKA23oE y4mLu6W4RjWzEQBgjOmCA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdehtdejucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhenucggtf frrghtthgvrhhnpeffueehtddtkeetgfelteejledvjeekgeduleffjeetfeekveeggffh fefhvdegffenucevlhhushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhroh hmpegrrhhthhhurhesrghrthhhuhhrfhgrsghrvgdrtghomhdpnhgspghrtghpthhtohep ledpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepthhhohhilhgrnhgusehrvgguhh grthdrtghomhdprhgtphhtthhopehlsghirghntghonhesrhgvughhrghtrdgtohhmpdhr tghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvgh gvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghfrggsrhgvsegtlhhouhgufhhl rghrvgdrtghomhdprhgtphhtthhopehjrghkuhgssegtlhhouhgufhhlrghrvgdrtghomh dprhgtphhtthhopeihrghnsegtlhhouhgufhhlrghrvgdrtghomhdprhgtphhtthhopehn vghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehjsghrrghnug gvsghurhhgsegtlhhouhgufhhlrghrvgdrtghomh X-ME-Proxy: Feedback-ID: i25f1493c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Mar 2025 09:33:50 -0500 (EST) From: arthur@arthurfabre.com Date: Wed, 05 Mar 2025 15:32:17 +0100 Subject: [PATCH RFC bpf-next 20/20] trait: register traits in benchmarks and tests Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250305-afabre-traits-010-rfc2-v1-20-d0ecfb869797@cloudflare.com> References: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> In-Reply-To: <20250305-afabre-traits-010-rfc2-v1-0-d0ecfb869797@cloudflare.com> To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: jakub@cloudflare.com, hawk@kernel.org, yan@cloudflare.com, jbrandeburg@cloudflare.com, thoiland@redhat.com, lbiancon@redhat.com, Arthur Fabre X-Mailer: b4 0.14.2 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Arthur Fabre Otherwise the verifier rejects the programs now. Signed-off-by: Arthur Fabre --- tools/testing/selftests/bpf/bench.c | 3 ++ tools/testing/selftests/bpf/bench.h | 1 + .../selftests/bpf/benchs/bench_xdp_traits.c | 33 +++++++++++++++++++++- .../testing/selftests/bpf/prog_tests/xdp_traits.c | 18 +++++++++++- 4 files changed, 53 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/bpf/bench.c b/tools/testing/selftests/bpf/bench.c index 4678b928fc6ad2f0a870a25d9b10c75a1f6d77ba..0961cb71ddf1d682ca61e512e9f4b2df3606747c 100644 --- a/tools/testing/selftests/bpf/bench.c +++ b/tools/testing/selftests/bpf/bench.c @@ -752,5 +752,8 @@ int main(int argc, char **argv) bench->report_final(state.results + env.warmup_sec, state.res_cnt - env.warmup_sec); + if (bench->cleanup) + bench->cleanup(); + return 0; } diff --git a/tools/testing/selftests/bpf/bench.h b/tools/testing/selftests/bpf/bench.h index 005c401b3e2275030d1d489cd77423cb1fb652ad..6a94af31df17e7cc35bb1432c4b205eb96b631f2 100644 --- a/tools/testing/selftests/bpf/bench.h +++ b/tools/testing/selftests/bpf/bench.h @@ -58,6 +58,7 @@ struct bench { void (*measure)(struct bench_res* res); void (*report_progress)(int iter, struct bench_res* res, long delta_ns); void (*report_final)(struct bench_res res[], int res_cnt); + void (*cleanup)(void); }; struct counter { diff --git a/tools/testing/selftests/bpf/benchs/bench_xdp_traits.c b/tools/testing/selftests/bpf/benchs/bench_xdp_traits.c index 0fbcd49edd825f53e6957319d3f05efc218dfb02..b6fa2d8f2504dd9c35f8fb9dc1f1099b55a55ac6 100644 --- a/tools/testing/selftests/bpf/benchs/bench_xdp_traits.c +++ b/tools/testing/selftests/bpf/benchs/bench_xdp_traits.c @@ -57,7 +57,18 @@ static void trait_validate(void) static void trait_setup(void) { - int err; + int err, i, key; + union bpf_attr attr; + + /* Register all keys so we can use them all. */ + bzero(&attr, sizeof(attr)); + for (i = 0; i < 64; i++) { + key = syscall(__NR_bpf, BPF_REGISTER_TRAIT, &attr, sizeof(attr)); + if (key < 0) { + fprintf(stderr, "couldn't register trait: %d\n", key); + exit(1); + } + } setup_libbpf(); @@ -77,6 +88,23 @@ static void trait_setup(void) } } +static void trait_cleanup(void) +{ + int err, i; + union bpf_attr attr; + + /* Unregister all keys so we can run again. */ + bzero(&attr, sizeof(attr)); + for (i = 0; i < 64; i++) { + attr.unregister_trait.trait = i; + err = syscall(__NR_bpf, BPF_UNREGISTER_TRAIT, &attr, sizeof(attr)); + if (err < 0) { + fprintf(stderr, "couldn't unregister trait %d: %d\n", i, err); + exit(1); + } + } +} + static void trait_get_setup(void) { trait_setup(); @@ -135,6 +163,7 @@ const struct bench bench_xdp_trait_get = { .measure = trait_measure, .report_progress = ops_report_progress, .report_final = ops_report_final, + .cleanup = trait_cleanup, }; const struct bench bench_xdp_trait_set = { @@ -146,6 +175,7 @@ const struct bench bench_xdp_trait_set = { .measure = trait_measure, .report_progress = ops_report_progress, .report_final = ops_report_final, + .cleanup = trait_cleanup, }; const struct bench bench_xdp_trait_move = { @@ -157,4 +187,5 @@ const struct bench bench_xdp_trait_move = { .measure = trait_measure, .report_progress = ops_report_progress, .report_final = ops_report_final, + .cleanup = trait_cleanup, }; diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_traits.c b/tools/testing/selftests/bpf/prog_tests/xdp_traits.c index 4175b28d45e91e82435e646e5edd783980d5fe70..1c1eff235a6159d377a5e8b9e0a4d956c4540e8e 100644 --- a/tools/testing/selftests/bpf/prog_tests/xdp_traits.c +++ b/tools/testing/selftests/bpf/prog_tests/xdp_traits.c @@ -6,8 +6,16 @@ static void _test_xdp_traits(void) { const char *file = "./test_xdp_traits.bpf.o"; struct bpf_object *obj; - int err, prog_fd; + int err, prog_fd, i, key; char buf[128]; + union bpf_attr attr; + + /* Register all keys so we can use them all. */ + bzero(&attr, sizeof(attr)); + for (i = 0; i < 64; i++) { + key = syscall(__NR_bpf, BPF_REGISTER_TRAIT, &attr, sizeof(attr)); + ASSERT_OK_FD(key, "test_xdp_traits"); + } LIBBPF_OPTS(bpf_test_run_opts, topts, .data_in = &pkt_v4, @@ -26,6 +34,14 @@ static void _test_xdp_traits(void) ASSERT_EQ(topts.retval, XDP_PASS, "retval"); bpf_object__close(obj); + + /* Unregister all keys so we can run again. */ + bzero(&attr, sizeof(attr)); + for (i = 0; i < 64; i++) { + attr.unregister_trait.trait = i; + err = syscall(__NR_bpf, BPF_UNREGISTER_TRAIT, &attr, sizeof(attr)); + ASSERT_OK(err, "test_xdp_traits"); + } } void test_xdp_traits(void)