From patchwork Sat Mar 2 07:00:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13579441 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 601E5C5478C for ; Sat, 2 Mar 2024 07:00:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E935B6B009F; Sat, 2 Mar 2024 02:00:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E42016B00A0; Sat, 2 Mar 2024 02:00:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE3506B00A1; Sat, 2 Mar 2024 02:00:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B62376B009F for ; Sat, 2 Mar 2024 02:00:48 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 99E991A0614 for ; Sat, 2 Mar 2024 07:00:48 +0000 (UTC) X-FDA: 81851201376.13.7DF0EE6 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf27.hostedemail.com (Postfix) with ESMTP id EB61340020 for ; Sat, 2 Mar 2024 07:00:46 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=N34Y3B6g; dmarc=none; spf=none (imf27.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709362847; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Q4z2pCHKerviP2W/GaaEWBolh6vlgWiXDx5cDUrL/+8=; b=lVGmD8KFWVjwY3Kq0b7VHoXGXUPEfnIsEiTQJjDvucG6JgGc506GlLHnbp3hYjkHnmnWrH GUzuEPQsI2Dvdxvxv35hTqAp2CppNchE1kwPSzh4A/gDF///0WY2B9oX79jrY3S8HWk/ki a6Nt0mC7M+Q1qTWH00xUdnF7zZ9VbCw= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=N34Y3B6g; dmarc=none; spf=none (imf27.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709362847; a=rsa-sha256; cv=none; b=DDa6XftSU9XnGeiILL8I2wmO5GUQHo+uj92QyEOgUsJsE63NihQinXXq4cz5ur8av5iRQF 5NL/MsKAswa94STUypw5oni0GyJpiuvyKb+t0nWJ9WZL1lsaeJnaTeFPNth+vJeRoYGJS9 cqm+imQDQdEwW5hSulyE3+E/icw6psA= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Q4z2pCHKerviP2W/GaaEWBolh6vlgWiXDx5cDUrL/+8=; b=N34Y3B6g74GhM2XhLX22H3kscZ KLfYWAGBqrM68cets6KKAkvjMBmCC0aV4Gdgsvb9S75BjBEQfku6aeI+y48efuSn1DQd6psqCrv/w gOhsPPaLJKm23kDCKr7hqV39E8rwBeVql5ieDfEAIs2APjQW9FTpg6HEKX83Zh/0N91tlrbB0HAPu aaQSTvteoszZb0I2zyV6fyD8vROU8aA9wG0WXDcE+nuOIdISvmpVq3mE5FMvHjVChAZXoL4snLQUf +xQeLzuj3LlJLFmozInBqU5afPJNdBp3WgAs5BzwajR+E1zVGvrX+w1xAxWGNdcsyFxup2MWP5Rlh TMXD2+1g==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rgJMH-0000000D2fg-2JuH; Sat, 02 Mar 2024 07:00:33 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , Kent Overstreet Subject: [PATCH 2/4] page_alloc: Combine free_the_page() and free_unref_page() Date: Sat, 2 Mar 2024 07:00:28 +0000 Message-ID: <20240302070031.3108523-3-willy@infradead.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240302070031.3108523-1-willy@infradead.org> References: <20240302070031.3108523-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: EB61340020 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 6rzkzmg8zrbcirp1z4gttibceyekxkk9 X-HE-Tag: 1709362846-928208 X-HE-Meta: U2FsdGVkX18YXrXbIDi2Dck4xXHqlwvfstxkC0qUnX3q5BoguIJYI3faVbBmWFuNibuJoP4FHDyXHEnbay9c3H1wXlLFNJtfXupBYX7ASsCrfk22Nv5ar3PMQdbFKsZQqDwThzo7Va+YjA2oJq5G1A4D2ILDNWVjMjV60uLg6iJ/QkTUD3D8RQmRFeTLi79eg73zERpTZMUAoIr8wsrOROsGEZgR9w3fEV524vA8CYANj0aHWeifr0eY50aJ7hgpf/Y3TxvsE0Xx48w/9JYVK3vlvQ71bIfLlv3RGWXKsGOvPcqc9h+ybNOfzxb9BsazrE+d0wUxqL9W1S6yIopdZ+s+8JKRm96sm0G1MHJ89m+3O3HQK8jexGXDe/16/zatPcrGQTHgAMR1I65Kb+LgXRi4omr04/LRUQWHTCN0pxg+E2zN+u1vsYp2GwKIMrMjvpYHKJM/MTHoOlKMsKfCp0wdpv/TEhLXqIC/c7hiYFwv7mD+Zku57JmU8jflXB+WsDC8W9HcgSW6NRvI8rKPq0AI3OFktw8v4hLClPEAH8LXDWIHwPAmAj6rJQEb3SRcs1dDeo2IvD2Di6IgFVYqfj5dZjcHhEhRX1Bl5spLNVuxtF0y4biw0bIzMGVPmcXJIY+sjLF/8RbGhZcyPullcbjEy3m5ToDQxnel0dNdilOvwBsWJfqgPhF4mu1N+OS2mkQ1q5xQuFHmSoxdmrhQnkA1i/iVvnr1Ag/JovLYKyZ7gWYTel8Q+Pyxu2iKBP6fOmgsbEJHlLR4nHICNEMGqBncPhq3xHpV3/DtHy8uQE3DgoBUl541yqaABXkCYYkOtSYg3CD/TUql7i77OqdGIOCBkkza7lAGrAKB0okhw2cmcCSa69gAoymaz/4nn3UO8r6mDc+/8J+/TLkrwycaJHAoNk7s64xeiKJpWW9JYftuvVLrE7glMjOk5L0n/ueXXLglCC1GfqkZk+aWj5W aBihU1tw fBX8HKSk3ClEzeGRB6NZ1a4WTmfeo0Gktn44lPdAuwWfu599AYKnYRL/9rHRa2rKjMxgv6ArIyOD3n7enAEwnmFmYMvzkFSIX4CNMJWprQE7DDasAm7nH4HgCY7l6/fv5Yf4Z9f14ZLo4xr0Y8Lrc26rLcEleIfR4L7NfvBcV2moeJGN1aPTOpEDQIVQl2yTTm3MzYTswM4BuI5zq7r+y/BPkfEJIFUGqqKxJ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The pcp_allowed_order() check was only being skipped by __folio_put_small() which is about to be rearranged. Signed-off-by: Matthew Wilcox (Oracle) --- mm/page_alloc.c | 25 +++++++++++-------------- 1 file changed, 11 insertions(+), 14 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 16241906a368..a51cbae62501 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -558,14 +558,6 @@ static inline bool pcp_allowed_order(unsigned int order) return false; } -static inline void free_the_page(struct page *page, unsigned int order) -{ - if (pcp_allowed_order(order)) /* Via pcp? */ - free_unref_page(page, order); - else - __free_pages_ok(page, order, FPI_NONE); -} - /* * Higher-order pages are called "compound pages". They are structured thusly: * @@ -601,7 +593,7 @@ void destroy_large_folio(struct folio *folio) folio_undo_large_rmappable(folio); mem_cgroup_uncharge(folio); - free_the_page(&folio->page, folio_order(folio)); + free_unref_page(&folio->page, folio_order(folio)); } static inline void set_buddy_order(struct page *page, unsigned int order) @@ -2520,6 +2512,11 @@ void free_unref_page(struct page *page, unsigned int order) unsigned long pfn = page_to_pfn(page); int migratetype, pcpmigratetype; + if (!pcp_allowed_order(order)) { + __free_pages_ok(page, order, FPI_NONE); + return; + } + if (!free_unref_page_prepare(page, pfn, order)) return; @@ -4694,10 +4691,10 @@ void __free_pages(struct page *page, unsigned int order) int head = PageHead(page); if (put_page_testzero(page)) - free_the_page(page, order); + free_unref_page(page, order); else if (!head) while (order-- > 0) - free_the_page(page + (1 << order), order); + free_unref_page(page + (1 << order), order); } EXPORT_SYMBOL(__free_pages); @@ -4748,7 +4745,7 @@ void __page_frag_cache_drain(struct page *page, unsigned int count) VM_BUG_ON_PAGE(page_ref_count(page) == 0, page); if (page_ref_sub_and_test(page, count)) - free_the_page(page, compound_order(page)); + free_unref_page(page, compound_order(page)); } EXPORT_SYMBOL(__page_frag_cache_drain); @@ -4789,7 +4786,7 @@ void *page_frag_alloc_align(struct page_frag_cache *nc, goto refill; if (unlikely(nc->pfmemalloc)) { - free_the_page(page, compound_order(page)); + free_unref_page(page, compound_order(page)); goto refill; } @@ -4833,7 +4830,7 @@ void page_frag_free(void *addr) struct page *page = virt_to_head_page(addr); if (unlikely(put_page_testzero(page))) - free_the_page(page, compound_order(page)); + free_unref_page(page, compound_order(page)); } EXPORT_SYMBOL(page_frag_free);