From patchwork Wed Jan 27 20:11:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 12050961 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E646C433DB for ; Wed, 27 Jan 2021 20:11:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 10F6564D9E for ; Wed, 27 Jan 2021 20:11:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 10F6564D9E Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=pm.me Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 99BA96B006C; Wed, 27 Jan 2021 15:11:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 970F66B006E; Wed, 27 Jan 2021 15:11:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8388F6B0070; Wed, 27 Jan 2021 15:11:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0251.hostedemail.com [216.40.44.251]) by kanga.kvack.org (Postfix) with ESMTP id 7063D6B006C for ; Wed, 27 Jan 2021 15:11:10 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 3C451824999B for ; Wed, 27 Jan 2021 20:11:10 +0000 (UTC) X-FDA: 77752649100.28.death61_4602e7227599 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin28.hostedemail.com (Postfix) with ESMTP id 11D646C2D for ; Wed, 27 Jan 2021 20:11:10 +0000 (UTC) X-HE-Tag: death61_4602e7227599 X-Filterd-Recvd-Size: 3239 Received: from mail-40131.protonmail.ch (mail-40131.protonmail.ch [185.70.40.131]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Wed, 27 Jan 2021 20:11:09 +0000 (UTC) Date: Wed, 27 Jan 2021 20:11:01 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1611778267; bh=VZmsQgsGD/2KxVsDk2CgN0UPiozU/OwIb092ldM9UwI=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=Mo13LuMcS9CANwW7XuPVKscyWcsv8NM3kVvpGhzPxTMduDaUWjfj+6u2OIMbiUlxX V7R/Svp3OhhbWYVrlgVHaAHBbgTOFaAa2FLdiWj3zioyWEbs0aZqN5XhVPsbIaV3MF hY5Ej1L7XHAeoBmTFAX89Wp4Q5mMiPBVizMZfZ4xc7ioivzPJduyXXC/7Wa6Bbbos9 j9WN55nUjya6an1T8YCwh2N6VPYUFGoEqZi12595euDTtVf8I21OTxOQHHRUCasEOz jQEJTl9cfC0sFN0x0gtkXrYC0I9F0zWBs0/UmSOpMeRxXiiZ/mfnngKkxxPkEE2Z0W 8efS2tykmkesg== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: David Rientjes , Yisen Zhuang , Salil Mehta , Jesse Brandeburg , Tony Nguyen , Saeed Mahameed , Leon Romanovsky , Andrew Morton , Jesper Dangaard Brouer , Ilias Apalodimas , Jonathan Lemon , Willem de Bruijn , Randy Dunlap , Pablo Neira Ayuso , Dexuan Cui , Jakub Sitnicki , Marco Elver , Paolo Abeni , Alexander Lobakin , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, intel-wired-lan@lists.osuosl.org, linux-rdma@vger.kernel.org, linux-mm@kvack.org Reply-To: Alexander Lobakin Subject: [PATCH v2 net-next 1/4] mm: constify page_is_pfmemalloc() argument Message-ID: <20210127201031.98544-2-alobakin@pm.me> In-Reply-To: <20210127201031.98544-1-alobakin@pm.me> References: <20210127201031.98544-1-alobakin@pm.me> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The function only tests for page->index, so its argument should be const. Signed-off-by: Alexander Lobakin Acked-by: David Rientjes --- include/linux/mm.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index ecdf8a8cd6ae..078633d43af9 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1584,7 +1584,7 @@ struct address_space *page_mapping_file(struct page *page); * ALLOC_NO_WATERMARKS and the low watermark was not * met implying that the system is under some pressure. */ -static inline bool page_is_pfmemalloc(struct page *page) +static inline bool page_is_pfmemalloc(const struct page *page) { /* * Page index cannot be this large so this must be From patchwork Wed Jan 27 20:11:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 12050963 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50BFFC433DB for ; Wed, 27 Jan 2021 20:11:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3AAA364D9E for ; Wed, 27 Jan 2021 20:11:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3AAA364D9E Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=pm.me Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C341A6B006E; Wed, 27 Jan 2021 15:11:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C088A6B0070; Wed, 27 Jan 2021 15:11:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B45AF6B0071; Wed, 27 Jan 2021 15:11:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0098.hostedemail.com [216.40.44.98]) by kanga.kvack.org (Postfix) with ESMTP id 948666B006E for ; Wed, 27 Jan 2021 15:11:16 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 3A0EF363F for ; Wed, 27 Jan 2021 20:11:16 +0000 (UTC) X-FDA: 77752649352.03.brake27_270f20627599 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id 129B728A4EB for ; Wed, 27 Jan 2021 20:11:16 +0000 (UTC) X-HE-Tag: brake27_270f20627599 X-Filterd-Recvd-Size: 3443 Received: from mail-40131.protonmail.ch (mail-40131.protonmail.ch [185.70.40.131]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Wed, 27 Jan 2021 20:11:15 +0000 (UTC) Date: Wed, 27 Jan 2021 20:11:12 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1611778273; bh=WGn3Y4XXY7ltWJBzRL8oxPGlTjtHLW5fnpeQGERPzl4=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=kLg7qi7avVJ9qLl4flp/CXLvM3HChtTuCeAiof3s0V/YfAxu8HF+otf0ApeHUvgUG MxeMACV+4kgXuidZB8WCoonEBt21RadqT+/dYt3uW92WeHHN0G2sxWnxIgfJgZoWtk Mss+4kom9kQFB4kRGjcfD4efRioZ8gHF2oWgGpH7UIErdsXpGvrbYsYd9/I7XG3kS5 snI3iiHPi1hnxKGFqTltxOuLP1IzszKFhMc6FqGCwoyD9UafrD4G3jveOBC6wdyXO/ B8GRHXAA2t24hiBkfVeQfkTNi/I0gA/qIPvgPb0NLVaKDx1gNFkm/tlHcJMHS5c+bW IimfDh7I9ScXQ== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: David Rientjes , Yisen Zhuang , Salil Mehta , Jesse Brandeburg , Tony Nguyen , Saeed Mahameed , Leon Romanovsky , Andrew Morton , Jesper Dangaard Brouer , Ilias Apalodimas , Jonathan Lemon , Willem de Bruijn , Randy Dunlap , Pablo Neira Ayuso , Dexuan Cui , Jakub Sitnicki , Marco Elver , Paolo Abeni , Alexander Lobakin , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, intel-wired-lan@lists.osuosl.org, linux-rdma@vger.kernel.org, linux-mm@kvack.org Reply-To: Alexander Lobakin Subject: [PATCH v2 net-next 2/4] skbuff: constify skb_propagate_pfmemalloc() "page" argument Message-ID: <20210127201031.98544-3-alobakin@pm.me> In-Reply-To: <20210127201031.98544-1-alobakin@pm.me> References: <20210127201031.98544-1-alobakin@pm.me> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The function doesn't write anything to the page struct itself, so this argument can be const. Misc: align second argument to the brace while at it. Signed-off-by: Alexander Lobakin Acked-by: David Rientjes --- include/linux/skbuff.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 9313b5aaf45b..b027526da4f9 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -2943,8 +2943,8 @@ static inline struct page *dev_alloc_page(void) * @page: The page that was allocated from skb_alloc_page * @skb: The skb that may need pfmemalloc set */ -static inline void skb_propagate_pfmemalloc(struct page *page, - struct sk_buff *skb) +static inline void skb_propagate_pfmemalloc(const struct page *page, + struct sk_buff *skb) { if (page_is_pfmemalloc(page)) skb->pfmemalloc = true; From patchwork Wed Jan 27 20:11:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 12050965 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58AA5C433E0 for ; Wed, 27 Jan 2021 20:11:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CBFE064DA5 for ; Wed, 27 Jan 2021 20:11:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CBFE064DA5 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=pm.me Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5F14D6B0070; Wed, 27 Jan 2021 15:11:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5C7296B0071; Wed, 27 Jan 2021 15:11:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4DB466B0072; Wed, 27 Jan 2021 15:11:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0100.hostedemail.com [216.40.44.100]) by kanga.kvack.org (Postfix) with ESMTP id 29ED56B0070 for ; Wed, 27 Jan 2021 15:11:33 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id DA5611EFD for ; Wed, 27 Jan 2021 20:11:32 +0000 (UTC) X-FDA: 77752650024.05.patch87_000489d27599 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id B32081804CB2C for ; Wed, 27 Jan 2021 20:11:32 +0000 (UTC) X-HE-Tag: patch87_000489d27599 X-Filterd-Recvd-Size: 16086 Received: from mail-40133.protonmail.ch (mail-40133.protonmail.ch [185.70.40.133]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Wed, 27 Jan 2021 20:11:31 +0000 (UTC) Date: Wed, 27 Jan 2021 20:11:23 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1611778289; bh=qxnDWFVEqg5VGfDWPP9D9tEqnWHHalzlBI7RQuSD1Vc=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=Jm5Eiw1SE4FqIosGATgxGGliy7MW7yjcwsjhgvAwQ+PggG0Bw0ZeQWCyFoxMM5jak GGjzN6mdBg72k4wtOOTQG1zPOwmrFcg+gjFodALcTEbRRa99WEUo7ELS4VnspMRlxe CNx5jcoVWopBkmLe1LBb4rT5pbNvOBAZ08YoK80unO0YA6n0zs/FLxIFVS6jVpP0ot XVNP3ryyb5vzVnDcahUDWe+SmSOwNFJzywpMD764BaOv0uU+gPiIm3DGLDcBw7VLRs Li7kLE4U6Ei40YVQE3kUwBa2ommgbaVrgxgjg2s6U7XoHsnV6gE6jo2iCFa5LFrshn osxMqD+ejTCbg== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: David Rientjes , Yisen Zhuang , Salil Mehta , Jesse Brandeburg , Tony Nguyen , Saeed Mahameed , Leon Romanovsky , Andrew Morton , Jesper Dangaard Brouer , Ilias Apalodimas , Jonathan Lemon , Willem de Bruijn , Randy Dunlap , Pablo Neira Ayuso , Dexuan Cui , Jakub Sitnicki , Marco Elver , Paolo Abeni , Alexander Lobakin , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, intel-wired-lan@lists.osuosl.org, linux-rdma@vger.kernel.org, linux-mm@kvack.org Reply-To: Alexander Lobakin Subject: [PATCH v2 net-next 3/4] net: introduce common dev_page_is_reserved() Message-ID: <20210127201031.98544-4-alobakin@pm.me> In-Reply-To: <20210127201031.98544-1-alobakin@pm.me> References: <20210127201031.98544-1-alobakin@pm.me> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A bunch of drivers test the page before reusing/recycling for two common conditions: - if a page was allocated under memory pressure (pfmemalloc page); - if a page was allocated at a distant memory node (to exclude slowdowns). Introduce and use a new common function for doing this and eliminate all functions-duplicates from drivers. Suggested-by: David Rientjes Signed-off-by: Alexander Lobakin Reviewed-by: Jesse Brandeburg Acked-by: David Rientjes --- drivers/net/ethernet/hisilicon/hns3/hns3_enet.c | 10 ++-------- drivers/net/ethernet/intel/fm10k/fm10k_main.c | 9 ++------- drivers/net/ethernet/intel/i40e/i40e_txrx.c | 15 +-------------- drivers/net/ethernet/intel/iavf/iavf_txrx.c | 15 +-------------- drivers/net/ethernet/intel/ice/ice_txrx.c | 11 +---------- drivers/net/ethernet/intel/igb/igb_main.c | 7 +------ drivers/net/ethernet/intel/igc/igc_main.c | 7 +------ drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 7 +------ drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 7 +------ drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 7 +------ include/linux/skbuff.h | 15 +++++++++++++++ 11 files changed, 27 insertions(+), 83 deletions(-) diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c index 512080640cbc..f71e3d963750 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c @@ -2800,12 +2800,6 @@ static void hns3_nic_alloc_rx_buffers(struct hns3_enet_ring *ring, writel(i, ring->tqp->io_base + HNS3_RING_RX_RING_HEAD_REG); } -static bool hns3_page_is_reusable(struct page *page) -{ - return page_to_nid(page) == numa_mem_id() && - !page_is_pfmemalloc(page); -} - static bool hns3_can_reuse_page(struct hns3_desc_cb *cb) { return (page_count(cb->priv) - cb->pagecnt_bias) == 1; @@ -2826,7 +2820,7 @@ static void hns3_nic_reuse_page(struct sk_buff *skb, int i, /* Avoid re-using remote pages, or the stack is still using the page * when page_offset rollback to zero, flag default unreuse */ - if (unlikely(!hns3_page_is_reusable(desc_cb->priv)) || + if (unlikely(dev_page_is_reserved(desc_cb->priv)) || (!desc_cb->page_offset && !hns3_can_reuse_page(desc_cb))) { __page_frag_cache_drain(desc_cb->priv, desc_cb->pagecnt_bias); return; @@ -3084,7 +3078,7 @@ static int hns3_alloc_skb(struct hns3_enet_ring *ring, unsigned int length, memcpy(__skb_put(skb, length), va, ALIGN(length, sizeof(long))); /* We can reuse buffer as-is, just make sure it is local */ - if (likely(hns3_page_is_reusable(desc_cb->priv))) + if (likely(!dev_page_is_reserved(desc_cb->priv))) desc_cb->reuse_flag = 1; else /* This page cannot be reused so discard it */ __page_frag_cache_drain(desc_cb->priv, diff --git a/drivers/net/ethernet/intel/fm10k/fm10k_main.c b/drivers/net/ethernet/intel/fm10k/fm10k_main.c index 99b8252eb969..9f547dd8c914 100644 --- a/drivers/net/ethernet/intel/fm10k/fm10k_main.c +++ b/drivers/net/ethernet/intel/fm10k/fm10k_main.c @@ -194,17 +194,12 @@ static void fm10k_reuse_rx_page(struct fm10k_ring *rx_ring, DMA_FROM_DEVICE); } -static inline bool fm10k_page_is_reserved(struct page *page) -{ - return (page_to_nid(page) != numa_mem_id()) || page_is_pfmemalloc(page); -} - static bool fm10k_can_reuse_rx_page(struct fm10k_rx_buffer *rx_buffer, struct page *page, unsigned int __maybe_unused truesize) { /* avoid re-using remote pages */ - if (unlikely(fm10k_page_is_reserved(page))) + if (unlikely(dev_page_is_reserved(page))) return false; #if (PAGE_SIZE < 8192) @@ -266,7 +261,7 @@ static bool fm10k_add_rx_frag(struct fm10k_rx_buffer *rx_buffer, memcpy(__skb_put(skb, size), va, ALIGN(size, sizeof(long))); /* page is not reserved, we can reuse buffer as-is */ - if (likely(!fm10k_page_is_reserved(page))) + if (likely(!dev_page_is_reserved(page))) return true; /* this page cannot be reused so discard it */ diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c index 2574e78f7597..4c295671aa09 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c @@ -1843,19 +1843,6 @@ static bool i40e_cleanup_headers(struct i40e_ring *rx_ring, struct sk_buff *skb, return false; } -/** - * i40e_page_is_reusable - check if any reuse is possible - * @page: page struct to check - * - * A page is not reusable if it was allocated under low memory - * conditions, or it's not in the same NUMA node as this CPU. - */ -static inline bool i40e_page_is_reusable(struct page *page) -{ - return (page_to_nid(page) == numa_mem_id()) && - !page_is_pfmemalloc(page); -} - /** * i40e_can_reuse_rx_page - Determine if this page can be reused by * the adapter for another receive @@ -1891,7 +1878,7 @@ static bool i40e_can_reuse_rx_page(struct i40e_rx_buffer *rx_buffer, struct page *page = rx_buffer->page; /* Is any reuse possible? */ - if (unlikely(!i40e_page_is_reusable(page))) + if (unlikely(dev_page_is_reserved(page))) return false; #if (PAGE_SIZE < 8192) diff --git a/drivers/net/ethernet/intel/iavf/iavf_txrx.c b/drivers/net/ethernet/intel/iavf/iavf_txrx.c index 256fa07d54d5..a6b552465f03 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_txrx.c +++ b/drivers/net/ethernet/intel/iavf/iavf_txrx.c @@ -1141,19 +1141,6 @@ static void iavf_reuse_rx_page(struct iavf_ring *rx_ring, new_buff->pagecnt_bias = old_buff->pagecnt_bias; } -/** - * iavf_page_is_reusable - check if any reuse is possible - * @page: page struct to check - * - * A page is not reusable if it was allocated under low memory - * conditions, or it's not in the same NUMA node as this CPU. - */ -static inline bool iavf_page_is_reusable(struct page *page) -{ - return (page_to_nid(page) == numa_mem_id()) && - !page_is_pfmemalloc(page); -} - /** * iavf_can_reuse_rx_page - Determine if this page can be reused by * the adapter for another receive @@ -1187,7 +1174,7 @@ static bool iavf_can_reuse_rx_page(struct iavf_rx_buffer *rx_buffer) struct page *page = rx_buffer->page; /* Is any reuse possible? */ - if (unlikely(!iavf_page_is_reusable(page))) + if (unlikely(dev_page_is_reserved(page))) return false; #if (PAGE_SIZE < 8192) diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c index 422f53997c02..623bbd27870f 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx.c +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c @@ -728,15 +728,6 @@ bool ice_alloc_rx_bufs(struct ice_ring *rx_ring, u16 cleaned_count) return !!cleaned_count; } -/** - * ice_page_is_reserved - check if reuse is possible - * @page: page struct to check - */ -static bool ice_page_is_reserved(struct page *page) -{ - return (page_to_nid(page) != numa_mem_id()) || page_is_pfmemalloc(page); -} - /** * ice_rx_buf_adjust_pg_offset - Prepare Rx buffer for reuse * @rx_buf: Rx buffer to adjust @@ -776,7 +767,7 @@ ice_can_reuse_rx_page(struct ice_rx_buf *rx_buf, int rx_buf_pgcnt) struct page *page = rx_buf->page; /* avoid re-using remote pages */ - if (unlikely(ice_page_is_reserved(page))) + if (unlikely(dev_page_is_reserved(page))) return false; #if (PAGE_SIZE < 8192) diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index 84d4284b8b32..f3020a73323d 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -8215,18 +8215,13 @@ static void igb_reuse_rx_page(struct igb_ring *rx_ring, new_buff->pagecnt_bias = old_buff->pagecnt_bias; } -static inline bool igb_page_is_reserved(struct page *page) -{ - return (page_to_nid(page) != numa_mem_id()) || page_is_pfmemalloc(page); -} - static bool igb_can_reuse_rx_page(struct igb_rx_buffer *rx_buffer) { unsigned int pagecnt_bias = rx_buffer->pagecnt_bias; struct page *page = rx_buffer->page; /* avoid re-using remote pages */ - if (unlikely(igb_page_is_reserved(page))) + if (unlikely(dev_page_is_reserved(page))) return false; #if (PAGE_SIZE < 8192) diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 43aec42e6d9d..8d51fb86be2e 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -1648,18 +1648,13 @@ static void igc_reuse_rx_page(struct igc_ring *rx_ring, new_buff->pagecnt_bias = old_buff->pagecnt_bias; } -static inline bool igc_page_is_reserved(struct page *page) -{ - return (page_to_nid(page) != numa_mem_id()) || page_is_pfmemalloc(page); -} - static bool igc_can_reuse_rx_page(struct igc_rx_buffer *rx_buffer) { unsigned int pagecnt_bias = rx_buffer->pagecnt_bias; struct page *page = rx_buffer->page; /* avoid re-using remote pages */ - if (unlikely(igc_page_is_reserved(page))) + if (unlikely(dev_page_is_reserved(page))) return false; #if (PAGE_SIZE < 8192) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index e08c01525fd2..37cc117c3aa6 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -1940,11 +1940,6 @@ static void ixgbe_reuse_rx_page(struct ixgbe_ring *rx_ring, new_buff->pagecnt_bias = old_buff->pagecnt_bias; } -static inline bool ixgbe_page_is_reserved(struct page *page) -{ - return (page_to_nid(page) != numa_mem_id()) || page_is_pfmemalloc(page); -} - static bool ixgbe_can_reuse_rx_page(struct ixgbe_rx_buffer *rx_buffer, int rx_buffer_pgcnt) { @@ -1952,7 +1947,7 @@ static bool ixgbe_can_reuse_rx_page(struct ixgbe_rx_buffer *rx_buffer, struct page *page = rx_buffer->page; /* avoid re-using remote pages */ - if (unlikely(ixgbe_page_is_reserved(page))) + if (unlikely(dev_page_is_reserved(page))) return false; #if (PAGE_SIZE < 8192) diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c index a14e55e7fce8..24caa0d2d572 100644 --- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c +++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c @@ -781,18 +781,13 @@ static void ixgbevf_reuse_rx_page(struct ixgbevf_ring *rx_ring, new_buff->pagecnt_bias = old_buff->pagecnt_bias; } -static inline bool ixgbevf_page_is_reserved(struct page *page) -{ - return (page_to_nid(page) != numa_mem_id()) || page_is_pfmemalloc(page); -} - static bool ixgbevf_can_reuse_rx_page(struct ixgbevf_rx_buffer *rx_buffer) { unsigned int pagecnt_bias = rx_buffer->pagecnt_bias; struct page *page = rx_buffer->page; /* avoid re-using remote pages */ - if (unlikely(ixgbevf_page_is_reserved(page))) + if (unlikely(dev_page_is_reserved(page))) return false; #if (PAGE_SIZE < 8192) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index dec93d57542f..5b9fb979adc6 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -212,11 +212,6 @@ static inline u32 mlx5e_decompress_cqes_start(struct mlx5e_rq *rq, return mlx5e_decompress_cqes_cont(rq, wq, 1, budget_rem) - 1; } -static inline bool mlx5e_page_is_reserved(struct page *page) -{ - return page_is_pfmemalloc(page) || page_to_nid(page) != numa_mem_id(); -} - static inline bool mlx5e_rx_cache_put(struct mlx5e_rq *rq, struct mlx5e_dma_info *dma_info) { @@ -229,7 +224,7 @@ static inline bool mlx5e_rx_cache_put(struct mlx5e_rq *rq, return false; } - if (unlikely(mlx5e_page_is_reserved(dma_info->page))) { + if (unlikely(dev_page_is_reserved(dma_info->page))) { stats->cache_waive++; return false; } diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index b027526da4f9..688782513d09 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -2938,6 +2938,21 @@ static inline struct page *dev_alloc_page(void) return dev_alloc_pages(0); } +/** + * dev_page_is_reserved - check whether a page can be reused for network Rx + * @page: the page to test + * + * A page shouldn't be considered for reusing/recycling if it was allocated + * under memory pressure or at a distant memory node. + * + * Returns true if this page should be returned to page allocator, false + * otherwise. + */ +static inline bool dev_page_is_reserved(const struct page *page) +{ + return page_is_pfmemalloc(page) || page_to_nid(page) != numa_mem_id(); +} + /** * skb_propagate_pfmemalloc - Propagate pfmemalloc if skb is allocated after RX page * @page: The page that was allocated from skb_alloc_page From patchwork Wed Jan 27 20:11:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 12050967 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8F34C433DB for ; Wed, 27 Jan 2021 20:11:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 85D5664DA4 for ; Wed, 27 Jan 2021 20:11:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 85D5664DA4 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=pm.me Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 183D16B0071; Wed, 27 Jan 2021 15:11:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 133B86B0072; Wed, 27 Jan 2021 15:11:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 047B46B0073; Wed, 27 Jan 2021 15:11:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E43936B0071 for ; Wed, 27 Jan 2021 15:11:35 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A7ADD8249980 for ; Wed, 27 Jan 2021 20:11:35 +0000 (UTC) X-FDA: 77752650150.07.use18_1d1452127599 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id 8C19E18046C6D for ; Wed, 27 Jan 2021 20:11:35 +0000 (UTC) X-HE-Tag: use18_1d1452127599 X-Filterd-Recvd-Size: 4276 Received: from mail-40131.protonmail.ch (mail-40131.protonmail.ch [185.70.40.131]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Wed, 27 Jan 2021 20:11:34 +0000 (UTC) Date: Wed, 27 Jan 2021 20:11:31 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1611778293; bh=9qyrYwHZiBXvDz842Cw5/bsnWtdqL9n+12n4P+yiMMc=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=f4zd97APJyHcue9fD9L3vZ6ObTvDAUTssdCjkJFUURwJpo26PCZZKFy2l1suO3zVz 8NNsQtnAaEkc6VyEiMYabnXmIjFugh7vzLUt/ZHQUBGYplrJAw0WE/jH6ruBux/j0J 04OvKQeW+CcjBPC+F95iJgzpVnAdr90tWgHt29Cn4UgcLXhZXrCE2IrTp91BlVRDhj F1M6wnpbadsj3dAwxbGoBMbFmPE/4fIQUF7/PWpstXw/eoUNHlhSOFPybnBQj0SvjA BP4f3UBxUnZ9eTZqjzIpcVhlufshszPAf0klO9GAvl9KV5JB4USz1IaKHK3mFxkwmI 1jDc+G2UaWtqg== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: David Rientjes , Yisen Zhuang , Salil Mehta , Jesse Brandeburg , Tony Nguyen , Saeed Mahameed , Leon Romanovsky , Andrew Morton , Jesper Dangaard Brouer , Ilias Apalodimas , Jonathan Lemon , Willem de Bruijn , Randy Dunlap , Pablo Neira Ayuso , Dexuan Cui , Jakub Sitnicki , Marco Elver , Paolo Abeni , Alexander Lobakin , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, intel-wired-lan@lists.osuosl.org, linux-rdma@vger.kernel.org, linux-mm@kvack.org Reply-To: Alexander Lobakin Subject: [PATCH v2 net-next 4/4] net: page_pool: simplify page recycling condition tests Message-ID: <20210127201031.98544-5-alobakin@pm.me> In-Reply-To: <20210127201031.98544-1-alobakin@pm.me> References: <20210127201031.98544-1-alobakin@pm.me> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: pool_page_reusable() is a leftover from pre-NUMA-aware times. For now, this function is just a redundant wrapper over page_is_pfmemalloc(), so Inline it into its sole call site. Signed-off-by: Alexander Lobakin Acked-by: Jesper Dangaard Brouer Reviewed-by: Ilias Apalodimas Acked-by: David Rientjes --- net/core/page_pool.c | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index f3c690b8c8e3..ad8b0707af04 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -350,14 +350,6 @@ static bool page_pool_recycle_in_cache(struct page *page, return true; } -/* page is NOT reusable when: - * 1) allocated when system is under some pressure. (page_is_pfmemalloc) - */ -static bool pool_page_reusable(struct page_pool *pool, struct page *page) -{ - return !page_is_pfmemalloc(page); -} - /* If the page refcnt == 1, this will try to recycle the page. * if PP_FLAG_DMA_SYNC_DEV is set, we'll try to sync the DMA area for * the configured size min(dma_sync_size, pool->max_len). @@ -373,9 +365,11 @@ __page_pool_put_page(struct page_pool *pool, struct page *page, * regular page allocator APIs. * * refcnt == 1 means page_pool owns page, and can recycle it. + * + * page is NOT reusable when allocated when system is under + * some pressure. (page_is_pfmemalloc) */ - if (likely(page_ref_count(page) == 1 && - pool_page_reusable(pool, page))) { + if (likely(page_ref_count(page) == 1 && !page_is_pfmemalloc(page))) { /* Read barrier done in page_ref_count / READ_ONCE */ if (pool->p.flags & PP_FLAG_DMA_SYNC_DEV)