From patchwork Fri Mar 3 13:32:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 13158781 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1648CC7EE30 for ; Fri, 3 Mar 2023 13:33:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229754AbjCCNdm (ORCPT ); Fri, 3 Mar 2023 08:33:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41112 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230153AbjCCNdl (ORCPT ); Fri, 3 Mar 2023 08:33:41 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 839511043A; Fri, 3 Mar 2023 05:33:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677850420; x=1709386420; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gkP8YSGL/x3nylTZPZvFPOewgh3s/bhW36zHGcB17/E=; b=nxNH7L6mzCBtkYrbzxNrwD9cf92XXOuwtwl9tt96rCJZZU5bqHDzrsNx Gu1JWkM2Y2D1Al2uiDMVvePdavALlx74Tqwz79gJsNgVpDA1rCUVgF/WS OzBxc+ufUCwMSfzF5gkewAbGKabfsbAYe/uoTc/GkMeQCZ+CYMaoHRek9 3ASnkg/lYhjnrlxFVMFSHQZcvWWG+P3wg6D1aKGRBCFzNT5A+D2esqw+q nKw9GrCq7a6VgZHIdOTPXOVX3VwrH6OvGhMWHj0/BVcneI+DuVe0a1Plx FdEQQSSfYpJ66fI9Mh0mxSnJpe7IdjZsc1FI6jl89lT9J3zfmkrYzufd+ g==; X-IronPort-AV: E=McAfee;i="6500,9779,10637"; a="421314262" X-IronPort-AV: E=Sophos;i="5.98,230,1673942400"; d="scan'208";a="421314262" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Mar 2023 05:33:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10637"; a="625347421" X-IronPort-AV: E=Sophos;i="5.98,230,1673942400"; d="scan'208";a="625347421" Received: from irvmail002.ir.intel.com ([10.43.11.120]) by orsmga003.jf.intel.com with ESMTP; 03 Mar 2023 05:33:35 -0800 Received: from newjersey.igk.intel.com (newjersey.igk.intel.com [10.102.20.203]) by irvmail002.ir.intel.com (Postfix) with ESMTP id 6DAE535FBF; Fri, 3 Mar 2023 13:33:34 +0000 (GMT) From: Alexander Lobakin To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau Cc: Alexander Lobakin , Maciej Fijalkowski , Larysa Zaremba , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Song Liu , Jesper Dangaard Brouer , Menglong Dong , Jakub Kicinski , Eric Dumazet , bpf@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, kernel test robot Subject: [PATCH bpf-next v2 1/3] net: page_pool, skbuff: make skb_mark_for_recycle() always available Date: Fri, 3 Mar 2023 14:32:30 +0100 Message-Id: <20230303133232.2546004-2-aleksander.lobakin@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230303133232.2546004-1-aleksander.lobakin@intel.com> References: <20230303133232.2546004-1-aleksander.lobakin@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net skb_mark_for_recycle() is guarded with CONFIG_PAGE_POOL, this creates unneeded complication when using it in the generic code. For now, it's only used in the drivers always selecting Page Pool, so this works. Move the guards so that preprocessor will cut out only the operation itself and the function will still be a noop on !PAGE_POOL systems, but available there as well. No functional changes. Reported-by: kernel test robot Link: https://lore.kernel.org/oe-kbuild-all/202303020342.Wi2PRFFH-lkp@intel.com Signed-off-by: Alexander Lobakin --- include/linux/skbuff.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index ff7ad331fb82..5cc1b4606b69 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -5071,12 +5071,12 @@ static inline u64 skb_get_kcov_handle(struct sk_buff *skb) #endif } -#ifdef CONFIG_PAGE_POOL static inline void skb_mark_for_recycle(struct sk_buff *skb) { +#ifdef CONFIG_PAGE_POOL skb->pp_recycle = 1; -} #endif +} #endif /* __KERNEL__ */ #endif /* _LINUX_SKBUFF_H */ From patchwork Fri Mar 3 13:32:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 13158780 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 265D4C7EE2F for ; Fri, 3 Mar 2023 13:33:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230484AbjCCNdo (ORCPT ); Fri, 3 Mar 2023 08:33:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230157AbjCCNdm (ORCPT ); Fri, 3 Mar 2023 08:33:42 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 72904125B9; Fri, 3 Mar 2023 05:33:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677850421; x=1709386421; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WV2QgxuHT3p8j+KGI77slUqYCqNGD1cnP+Xz71qqoTc=; b=AKpmQbjXCc8Rar4jKAXmEZimQ+DCo29uvDBo9gbeQdFr+v9QZLeQYUvq P5/M4wUxJlC7uRDaTSNjXQA+7EWPZJhK32EWTspcyUGtKCty8gCQbVoHg LIpInb8p8zRXQsI6g85IqE7puRgRNdiyrM9bEppBDcYbzuhksT8lTnZnL ysafeQd0o3hpvOeur+E4z+3v9PG60a93kD1z7vrILxEw5SPVV9J+7LY4j i9KL74Ca4qYp0cROGa33SiNO05X1A2wZlAGI8Ch3VC8BunVFl5FgE6YId UDr6v2EQvmV6jjHHxdSS4d4m28U1rwsv+eq+zwL0PlJTk/LYWBNXt16h3 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10637"; a="421314280" X-IronPort-AV: E=Sophos;i="5.98,230,1673942400"; d="scan'208";a="421314280" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Mar 2023 05:33:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10637"; a="625347429" X-IronPort-AV: E=Sophos;i="5.98,230,1673942400"; d="scan'208";a="625347429" Received: from irvmail002.ir.intel.com ([10.43.11.120]) by orsmga003.jf.intel.com with ESMTP; 03 Mar 2023 05:33:36 -0800 Received: from newjersey.igk.intel.com (newjersey.igk.intel.com [10.102.20.203]) by irvmail002.ir.intel.com (Postfix) with ESMTP id 6E9E4365A0; Fri, 3 Mar 2023 13:33:35 +0000 (GMT) From: Alexander Lobakin To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau Cc: Alexander Lobakin , Maciej Fijalkowski , Larysa Zaremba , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Song Liu , Jesper Dangaard Brouer , Menglong Dong , Jakub Kicinski , Eric Dumazet , bpf@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v2 2/3] xdp: recycle Page Pool backed skbs built from XDP frames Date: Fri, 3 Mar 2023 14:32:31 +0100 Message-Id: <20230303133232.2546004-3-aleksander.lobakin@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230303133232.2546004-1-aleksander.lobakin@intel.com> References: <20230303133232.2546004-1-aleksander.lobakin@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net __xdp_build_skb_from_frame() state(d): /* Until page_pool get SKB return path, release DMA here */ Page Pool got skb pages recycling in April 2021, but missed this function. xdp_release_frame() is relevant only for Page Pool backed frames and it detaches the page from the corresponding page_pool in order to make it freeable via page_frag_free(). It can instead just mark the output skb as eligible for recycling if the frame is backed by a pp. No change for other memory model types (the same condition check as before). cpumap redirect and veth on Page Pool drivers now become zero-alloc (or almost). Signed-off-by: Alexander Lobakin --- net/core/xdp.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/net/core/xdp.c b/net/core/xdp.c index 8c92fc553317..a2237cfca8e9 100644 --- a/net/core/xdp.c +++ b/net/core/xdp.c @@ -658,8 +658,8 @@ struct sk_buff *__xdp_build_skb_from_frame(struct xdp_frame *xdpf, * - RX ring dev queue index (skb_record_rx_queue) */ - /* Until page_pool get SKB return path, release DMA here */ - xdp_release_frame(xdpf); + if (xdpf->mem.type == MEM_TYPE_PAGE_POOL) + skb_mark_for_recycle(skb); /* Allow SKB to reuse area used by xdp_frame */ xdp_scrub_frame(xdpf); From patchwork Fri Mar 3 13:32:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 13158782 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0BE5CC7EE37 for ; Fri, 3 Mar 2023 13:33:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229956AbjCCNdq (ORCPT ); Fri, 3 Mar 2023 08:33:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230463AbjCCNdn (ORCPT ); Fri, 3 Mar 2023 08:33:43 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BED8A12F38; Fri, 3 Mar 2023 05:33:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677850421; x=1709386421; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZFpBMYKp7Vej9uaBP1GUTHm12MmkAD4Atk8fjSWwcN0=; b=LlFb9nbwUdavui7NHS5Z9QqyFuAAgQvJA5Ng82px7e4Yi6dJ1NQ8WUcW OnO8/gYa1IezLfy/wy0bmxnaCQhiTqJlZk59KTbOMV+p1oLTzsfcFIOoW WuDSllu2cl7gPHZ7ZYFzqTvHXSAh/jDCC+Ow//33aKUUik5xlGSYlrbTu Ksl2OO/Ttpa/tobGfqjpqqxg693XP2rjwsyu6fw8clHNBDmdYI6GGMKPB CJ+OL8HkNJ6lYvou+7dINkJejgp45XiYaqEUSvnYI44kDRCmN/p/jo50M 8Ow32p1JpJjwvVTnpRfqW/4/ta8MrdRHV5PM01VkpfNeMlZT7kE072szN A==; X-IronPort-AV: E=McAfee;i="6500,9779,10637"; a="421314288" X-IronPort-AV: E=Sophos;i="5.98,230,1673942400"; d="scan'208";a="421314288" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Mar 2023 05:33:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10637"; a="625347432" X-IronPort-AV: E=Sophos;i="5.98,230,1673942400"; d="scan'208";a="625347432" Received: from irvmail002.ir.intel.com ([10.43.11.120]) by orsmga003.jf.intel.com with ESMTP; 03 Mar 2023 05:33:37 -0800 Received: from newjersey.igk.intel.com (newjersey.igk.intel.com [10.102.20.203]) by irvmail002.ir.intel.com (Postfix) with ESMTP id 63689365AE; Fri, 3 Mar 2023 13:33:36 +0000 (GMT) From: Alexander Lobakin To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau Cc: Alexander Lobakin , Maciej Fijalkowski , Larysa Zaremba , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Song Liu , Jesper Dangaard Brouer , Menglong Dong , Jakub Kicinski , Eric Dumazet , bpf@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v2 3/3] xdp: remove unused {__,}xdp_release_frame() Date: Fri, 3 Mar 2023 14:32:32 +0100 Message-Id: <20230303133232.2546004-4-aleksander.lobakin@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230303133232.2546004-1-aleksander.lobakin@intel.com> References: <20230303133232.2546004-1-aleksander.lobakin@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net __xdp_build_skb_from_frame() was the last user of {__,}xdp_release_frame(), which detaches pages from the page_pool. All the consumers now recycle Page Pool skbs and page, except mlx5, stmmac and tsnep drivers, which use page_pool_release_page() directly (might change one day). It's safe to assume this functionality is not needed anymore and can be removed (in favor of recycling). Signed-off-by: Alexander Lobakin --- include/net/xdp.h | 29 ----------------------------- net/core/xdp.c | 15 --------------- 2 files changed, 44 deletions(-) diff --git a/include/net/xdp.h b/include/net/xdp.h index d517bfac937b..5393b3ebe56e 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -317,35 +317,6 @@ void xdp_flush_frame_bulk(struct xdp_frame_bulk *bq); void xdp_return_frame_bulk(struct xdp_frame *xdpf, struct xdp_frame_bulk *bq); -/* When sending xdp_frame into the network stack, then there is no - * return point callback, which is needed to release e.g. DMA-mapping - * resources with page_pool. Thus, have explicit function to release - * frame resources. - */ -void __xdp_release_frame(void *data, struct xdp_mem_info *mem); -static inline void xdp_release_frame(struct xdp_frame *xdpf) -{ - struct xdp_mem_info *mem = &xdpf->mem; - struct skb_shared_info *sinfo; - int i; - - /* Curr only page_pool needs this */ - if (mem->type != MEM_TYPE_PAGE_POOL) - return; - - if (likely(!xdp_frame_has_frags(xdpf))) - goto out; - - sinfo = xdp_get_shared_info_from_frame(xdpf); - for (i = 0; i < sinfo->nr_frags; i++) { - struct page *page = skb_frag_page(&sinfo->frags[i]); - - __xdp_release_frame(page_address(page), mem); - } -out: - __xdp_release_frame(xdpf->data, mem); -} - static __always_inline unsigned int xdp_get_frame_len(struct xdp_frame *xdpf) { struct skb_shared_info *sinfo; diff --git a/net/core/xdp.c b/net/core/xdp.c index a2237cfca8e9..8d3ad315f18d 100644 --- a/net/core/xdp.c +++ b/net/core/xdp.c @@ -531,21 +531,6 @@ void xdp_return_buff(struct xdp_buff *xdp) } EXPORT_SYMBOL_GPL(xdp_return_buff); -/* Only called for MEM_TYPE_PAGE_POOL see xdp.h */ -void __xdp_release_frame(void *data, struct xdp_mem_info *mem) -{ - struct xdp_mem_allocator *xa; - struct page *page; - - rcu_read_lock(); - xa = rhashtable_lookup(mem_id_ht, &mem->id, mem_id_rht_params); - page = virt_to_head_page(data); - if (xa) - page_pool_release_page(xa->page_pool, page); - rcu_read_unlock(); -} -EXPORT_SYMBOL_GPL(__xdp_release_frame); - void xdp_attachment_setup(struct xdp_attachment_info *info, struct netdev_bpf *bpf) {