From patchwork Thu Mar 21 14:24:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13598780 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A484C54E58 for ; Thu, 21 Mar 2024 14:25:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A60C6B009A; Thu, 21 Mar 2024 10:25:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 908516B009B; Thu, 21 Mar 2024 10:25:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7315C6B009C; Thu, 21 Mar 2024 10:25:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 6461A6B009A for ; Thu, 21 Mar 2024 10:25:21 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 409BFA15BB for ; Thu, 21 Mar 2024 14:25:21 +0000 (UTC) X-FDA: 81921268842.09.679A7BB Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf05.hostedemail.com (Postfix) with ESMTP id CC5A1100028 for ; Thu, 21 Mar 2024 14:25:18 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=BmyYt3Mb; dmarc=none; spf=none (imf05.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711031118; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TFqb/icTqpLESKsoUJZg7vHbrRZsZRhvgU73HCkwJw0=; b=2SVnH5HXDVvgS5FxA4LvXSlbIqhljrbRQsBTM9Lt2MhdfPQubXDYLGZq4KLYgXendqdlZ7 SP7AwmCq/9atpuXuTEkbV1GAFhdR8hJeVdOkGRA+WxFFtRZhbWfMJ6jHWNqGXr34d6oE/b bDaDegF2El0ZvE7t+Y0ClCQAYt2H9js= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=BmyYt3Mb; dmarc=none; spf=none (imf05.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711031118; a=rsa-sha256; cv=none; b=RpxO2fM+Ox3yGuOvjS9PS3juIxrIxwRUecbwRUCNrTE/N5+nH//We8dm+0WPGya4n+90V3 CUC8pjr/HB2tOt0PiO3IEZ+AfaBC2GHVdhpRoZC9HxHn9d1Rpsfh36RV7pQ3nl4FoSrUre pdaoly34HcAsi3BdPJ6MkLQnb4Lu9Rk= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=TFqb/icTqpLESKsoUJZg7vHbrRZsZRhvgU73HCkwJw0=; b=BmyYt3Mb/S+Cgpx6rWtSMumZKO fleGe/ejJ9rAdj3cWbRXQbXtvm0aDnnv8QEQjlfXMtz0q1klIfO6cidZ4KRHsy8EOx/ukFdervB+v h5o7md1FznAVlf3+MJ6k2cKJJ/yqhYlM8oi8/N1VzIamj6EMo2x+uV4mLWyK9NronAMc20TsRlK0h 9hO2iukEOAYqNBEwZVjk+ZmAA4usbtx0tO5wlc7/aNyHDyqdp6uOIUjpWCV47ikbLWBufk+0BQEz2 J70tQjQ2tqKHb6nOOGpH8B7XLwVcNPcdXGuHUNw/esGTf5d3s4GeXqxlCDqGgljC/AXlJ3XtujC6V hxBY14hw==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnJLf-00000006u3u-1QqD; Thu, 21 Mar 2024 14:24:51 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, David Hildenbrand , Vlastimil Babka , Miaohe Lin , Muchun Song , Oscar Salvador Subject: [PATCH 1/9] mm: Always initialise folio->_deferred_list Date: Thu, 21 Mar 2024 14:24:39 +0000 Message-ID: <20240321142448.1645400-2-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240321142448.1645400-1-willy@infradead.org> References: <20240321142448.1645400-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: CC5A1100028 X-Stat-Signature: 1z4rnptidsqba7uwwtwarkuoc974euhc X-HE-Tag: 1711031118-722627 X-HE-Meta: U2FsdGVkX18gmpHh6Xfw0H7SC5PwBasixz9clrVuCHYq63ucyPJqwFrQaVjRfJ0nMuacSc764YPuZROsrQdkLlPCSEgr2tFNS5z+gaHB40LZtRX/KH6SkXAsVWG/lsN+vYoz6IlX0Ib31pjZP0f+EW9cX9pvD6Y295887jw+CRK++uRfVDD4+vfoUh/1Hnki/b31CAEgM/RgXBmofb3F3Uc0EyM+uAE6eWEq9StXn6ieAdcK2FU+uPzcrYu9SB/ZIQVvl/r1ZnN3PKdPh1NpehotC/Nk2ft07Cu7vxAQ1OVNs3UImfBn9RYKWMFFNoUzIox9s2F123G9dArs1aqvUGJ6Lg3mQxU0yzEDIK/swoGBMWYISc6QDbKrWEJqtwiYCYxAubbWsWOhmClxecGaId9UL45NSHvlFfdHmx6O0vJsF6L+k/3mCBYe+FvxhmBBYdss02P/jzsNwxmHc03DvbpCTFs3vCbf5+iZ+vlFMBqyQVaR3NbYqmYRd3+4YChr8YvQGokk75QnZWelIywezfkXDCiLx+2+jkVc452A8oQrsTMbhjhFu66qCiVAgMrsCyHekb4VVUyclX56h86qegaUjBbney448uyohI77ko9+0KDsTT9ErygelLglQIkmqts/yi/HABycvWVf1659+2WluSQQlSD2hL6csfb3Ddk9hoQcXYyy2ZD6PHfbCR2FXJCKKigBZbIgRR38x2FPQrM8vyQzthpG3BcqLAJ5ezbukJB1FjCBVDnhdqwABQiIJJYCirhXcbE4PXbTJaqVsMgz6Od7lEhyzei75Uiz6Ty0SUEeKNlhTPG9sQ6b/pkdT1Zm02qDCWASD9oqD/j3rgXKvBJxYpPghz2vctvkalJcGXJZ5BmRs6hp8qefExnAbws6BTxh3UVvGPjH9GCrLbtauoZ36b+7 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For compound pages which are at least order-2 (and hence have a deferred_list), initialise it and then we can check at free that the page is not part of a deferred list. We recently found this useful to rule out a source of corruption. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Vlastimil Babka Reviewed-by: David Hildenbrand --- mm/huge_memory.c | 2 -- mm/hugetlb.c | 3 ++- mm/internal.h | 2 ++ mm/memcontrol.c | 2 ++ mm/page_alloc.c | 9 +++++---- 5 files changed, 11 insertions(+), 7 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 9859aa4f7553..04fb994a7b0b 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -792,8 +792,6 @@ void folio_prep_large_rmappable(struct folio *folio) { if (!folio || !folio_test_large(folio)) return; - if (folio_order(folio) > 1) - INIT_LIST_HEAD(&folio->_deferred_list); folio_set_large_rmappable(folio); } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 23ef240ba48a..7e9a766059aa 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1796,7 +1796,8 @@ static void __update_and_free_hugetlb_folio(struct hstate *h, destroy_compound_gigantic_folio(folio, huge_page_order(h)); free_gigantic_folio(folio, huge_page_order(h)); } else { - __free_pages(&folio->page, huge_page_order(h)); + INIT_LIST_HEAD(&folio->_deferred_list); + folio_put(folio); } } diff --git a/mm/internal.h b/mm/internal.h index 7e486f2c502c..10895ec52546 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -525,6 +525,8 @@ static inline void prep_compound_head(struct page *page, unsigned int order) atomic_set(&folio->_entire_mapcount, -1); atomic_set(&folio->_nr_pages_mapped, 0); atomic_set(&folio->_pincount, 0); + if (order > 1) + INIT_LIST_HEAD(&folio->_deferred_list); } static inline void prep_compound_tail(struct page *head, int tail_idx) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index fabce2b50c69..a2a74d4ca0b1 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -7448,6 +7448,8 @@ static void uncharge_folio(struct folio *folio, struct uncharge_gather *ug) struct obj_cgroup *objcg; VM_BUG_ON_FOLIO(folio_test_lru(folio), folio); + VM_BUG_ON_FOLIO(folio_order(folio) > 1 && + !list_empty(&folio->_deferred_list), folio); /* * Nobody should be changing or seriously looking at diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 14d39f34d336..4301146a5bf4 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1006,10 +1006,11 @@ static int free_tail_page_prepare(struct page *head_page, struct page *page) } break; case 2: - /* - * the second tail page: ->mapping is - * deferred_list.next -- ignore value. - */ + /* the second tail page: deferred_list overlaps ->mapping */ + if (unlikely(!list_empty(&folio->_deferred_list))) { + bad_page(page, "on deferred list"); + goto out; + } break; default: if (page->mapping != TAIL_MAPPING) { From patchwork Thu Mar 21 14:24:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13598776 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72335C54E68 for ; Thu, 21 Mar 2024 14:25:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CD8B16B0095; Thu, 21 Mar 2024 10:25:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C8A996B0096; Thu, 21 Mar 2024 10:25:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB7E16B0098; Thu, 21 Mar 2024 10:25:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 9A9786B0095 for ; Thu, 21 Mar 2024 10:25:08 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 7045412136D for ; Thu, 21 Mar 2024 14:25:08 +0000 (UTC) X-FDA: 81921268296.03.E4CC5F2 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf19.hostedemail.com (Postfix) with ESMTP id CB1431A0016 for ; Thu, 21 Mar 2024 14:25:06 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=tmDboZEe; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711031106; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UUO66vHKvPfAbmInHFgL3UfGdliS6xZEbrFBwuiOeG8=; b=rCEIN+5DL2GNn0RWGDZpm9QJe08nPR1fGtR0pTJAlH0vN7dlzKwkKxYN3fIf8Zu90JCCCf X/TOSPsykk04pBM6DYy2hweWQUc5XmrSM7UztJSTUApn1QhiaCHBxv/OtqwOpbQRxdVzFC vJM4ggnQQxhGKfHulmZa1Kz3kAIonYs= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=tmDboZEe; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711031106; a=rsa-sha256; cv=none; b=cUfJzNHze2oZYg6fSfRAzXy0WYz29gUbQu3L40q6r2mKYeKrOG3qA2sYl0WoEsUGZaFlbf xAiMrm6gT2Ual8jTRi323VC0dnA56Ki9YbFc+aYJt+PeOTDIRaosaOQ51+u5UWJrVmy9UE CvJQuqCBj8D2Pe5WAStVVFz8VsdtJd0= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=UUO66vHKvPfAbmInHFgL3UfGdliS6xZEbrFBwuiOeG8=; b=tmDboZEe6hbgb0MtG11/QNaq53 G8mq2iMFrX8QvESdWj5cIlc1Ze98teSkAPw+2X3ixhqtJMimYzDwaHpxmLR4bBgZRAemoMDbsxoGC KB+yWfBBIn6s1qpEzT5yJMxcDNt4BRubre7FLlnv0RUg3TRGnUkkFsts84w4wCDTzuC7mplU9OAyW TpZMSyqBpnwkmoaH/h8Q9Xx5nBwru0AwFzVmEQTDDlH40+jUK80OYcwjsfROi+uy4YuiWIgVpZvQf Wm1w/7o5nlPHRxYA20uDRRLonpmXbli6+BFmthT2HwP07X76dL4LWvLmI2eQarCtYMZ1xHbcCQv9V GPD9SIDQ==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnJLf-00000006u3w-1o9u; Thu, 21 Mar 2024 14:24:51 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, David Hildenbrand , Vlastimil Babka , Miaohe Lin , Muchun Song , Oscar Salvador Subject: [PATCH 2/9] mm: Create FOLIO_FLAG_FALSE and FOLIO_TYPE_OPS macros Date: Thu, 21 Mar 2024 14:24:40 +0000 Message-ID: <20240321142448.1645400-3-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240321142448.1645400-1-willy@infradead.org> References: <20240321142448.1645400-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: CB1431A0016 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: u4mmgpo1kiq6psukzo9ks5sc19hhg7at X-HE-Tag: 1711031106-429606 X-HE-Meta: U2FsdGVkX19yCoS/w0575yjG4kpxTfztPPxv4GDqZrM0NLsB+7Y87EqMWek2D93V/+AcS0gUPSz9mN/8n8KEDkWnsCroSR4mauZRlWsZ5Vbvc7LUsPumPjSVjgVQWYJ1QO4FYJ5Qf2yqyCK+F0akjde5+JoGHSSsQO81t9mP3REvSod1Xz7Bwaa3VoLXk0cCUf+WMaw5i/59FZ3m6/crmZImQ9K3MLkfo8hq8qjYKYeVrkYqEhmhdLXdOJl6eADYLOqu8c7X+m4p1G4MGad26ejxxPROfXBC8eTyoOTjxpdOaK2EIWrmoGRGXEOq9Xp6IsurRBDWOj3NUnqvs8I21x/1SR1USWPPzEWwvnRZJG5zyS3DsvztG9oUssG9fXxAF454PzoMcvxVV1DKUWi8Rmd6F1cYKZUZGQ7VesbRVJXG/2m2an/vzPnOkWTSGiHEzc2AYCWBybtyyjY83hWfxwUXvEceRrBBHra69jkHmjeIHoEpNOxnw8tbbePwXPqEmOB9tlV7zLqeWfQf8ui9qm2opfrfHoiBMfUn2WAtxncm1vIq/VYoeUlX0Zcz/1fRlYx1byssqbgaLnKgEhbzuG75wSPdRcKHmu/51Epfxe6/OvA1qiwBtgVva5xp9IhDkkpPDmGEnOplaUa0vxD9UoqgUaWepjZONtrtdS1V0KBovCSypYUo2ubmOPjLwWc+4o10lIpKiVxqZGS9nrW3CVJh6cxmXr42uP1637fiSPKXb+zAl4q7BWs7p/82JyFyLEOsdk/X0dEUEcVaBxG2xhE/kDFGAzAeTb1eIS3oklj3s+PrHMjsDr0fop1hgyonP6Z1N9nys1Qq/R07vdklntoKlnt5poTfC8A5Suk+mp7M4eAHe+Lhxckj+9o0GpKI+ZgSRnpubJcb/B1LUboyHRvVAUlKFsiEZD5eh9DqXqJFRSHJ0H92RktNHo61yVL02XNYHfZNEZXujk9II/h RpYXFXhD UohyAl/+CIgdeLbQRJMxOg7G83bN8VTHKXJib48Q1ALjGjvLbC/4xR6P5DfzYlTwQ20e9jhjEAp6YHIcBt4QNG6tGnEtgfK9L/lz460F2t9myD/DDhI5A34EvWbUHi/h2smAcVUJ+HiFjZ9bYc9NTCZXAyWcvwmMrBh0MGhGVYE7q4zCOxEM94MoYJ/rShRUlQ68E9a63aj0+yeMtrF7rRJNEqk9WlJRyuuAb X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Following the separation of FOLIO_FLAGS from PAGEFLAGS, separate FOLIO_FLAG_FALSE from PAGEFLAG_FALSE and FOLIO_TYPE_OPS from PAGE_TYPE_OPS. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Vlastimil Babka --- include/linux/page-flags.h | 70 +++++++++++++++++++++++++------------- 1 file changed, 47 insertions(+), 23 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 652d77805e99..dc1607f1415e 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -458,30 +458,51 @@ static __always_inline int TestClearPage##uname(struct page *page) \ TESTSETFLAG(uname, lname, policy) \ TESTCLEARFLAG(uname, lname, policy) +#define FOLIO_TEST_FLAG_FALSE(name) \ +static inline bool folio_test_##name(const struct folio *folio) \ +{ return false; } +#define FOLIO_SET_FLAG_NOOP(name) \ +static inline void folio_set_##name(struct folio *folio) { } +#define FOLIO_CLEAR_FLAG_NOOP(name) \ +static inline void folio_clear_##name(struct folio *folio) { } +#define __FOLIO_SET_FLAG_NOOP(name) \ +static inline void __folio_set_##name(struct folio *folio) { } +#define __FOLIO_CLEAR_FLAG_NOOP(name) \ +static inline void __folio_clear_##name(struct folio *folio) { } +#define FOLIO_TEST_SET_FLAG_FALSE(name) \ +static inline bool folio_test_set_##name(struct folio *folio) \ +{ return false; } +#define FOLIO_TEST_CLEAR_FLAG_FALSE(name) \ +static inline bool folio_test_clear_##name(struct folio *folio) \ +{ return false; } + +#define FOLIO_FLAG_FALSE(name) \ +FOLIO_TEST_FLAG_FALSE(name) \ +FOLIO_SET_FLAG_NOOP(name) \ +FOLIO_CLEAR_FLAG_NOOP(name) + #define TESTPAGEFLAG_FALSE(uname, lname) \ -static inline bool folio_test_##lname(const struct folio *folio) { return false; } \ +FOLIO_TEST_FLAG_FALSE(lname) \ static inline int Page##uname(const struct page *page) { return 0; } #define SETPAGEFLAG_NOOP(uname, lname) \ -static inline void folio_set_##lname(struct folio *folio) { } \ +FOLIO_SET_FLAG_NOOP(lname) \ static inline void SetPage##uname(struct page *page) { } #define CLEARPAGEFLAG_NOOP(uname, lname) \ -static inline void folio_clear_##lname(struct folio *folio) { } \ +FOLIO_CLEAR_FLAG_NOOP(lname) \ static inline void ClearPage##uname(struct page *page) { } #define __CLEARPAGEFLAG_NOOP(uname, lname) \ -static inline void __folio_clear_##lname(struct folio *folio) { } \ +__FOLIO_CLEAR_FLAG_NOOP(lname) \ static inline void __ClearPage##uname(struct page *page) { } #define TESTSETFLAG_FALSE(uname, lname) \ -static inline bool folio_test_set_##lname(struct folio *folio) \ -{ return 0; } \ +FOLIO_TEST_SET_FLAG_FALSE(lname) \ static inline int TestSetPage##uname(struct page *page) { return 0; } #define TESTCLEARFLAG_FALSE(uname, lname) \ -static inline bool folio_test_clear_##lname(struct folio *folio) \ -{ return 0; } \ +FOLIO_TEST_CLEAR_FLAG_FALSE(lname) \ static inline int TestClearPage##uname(struct page *page) { return 0; } #define PAGEFLAG_FALSE(uname, lname) TESTPAGEFLAG_FALSE(uname, lname) \ @@ -977,35 +998,38 @@ static inline int page_has_type(const struct page *page) return page_type_has_type(page->page_type); } +#define FOLIO_TYPE_OPS(lname, fname) \ +static __always_inline bool folio_test_##fname(const struct folio *folio)\ +{ \ + return folio_test_type(folio, PG_##lname); \ +} \ +static __always_inline void __folio_set_##fname(struct folio *folio) \ +{ \ + VM_BUG_ON_FOLIO(!folio_test_type(folio, 0), folio); \ + folio->page.page_type &= ~PG_##lname; \ +} \ +static __always_inline void __folio_clear_##fname(struct folio *folio) \ +{ \ + VM_BUG_ON_FOLIO(!folio_test_##fname(folio), folio); \ + folio->page.page_type |= PG_##lname; \ +} + #define PAGE_TYPE_OPS(uname, lname, fname) \ +FOLIO_TYPE_OPS(lname, fname) \ static __always_inline int Page##uname(const struct page *page) \ { \ return PageType(page, PG_##lname); \ } \ -static __always_inline int folio_test_##fname(const struct folio *folio)\ -{ \ - return folio_test_type(folio, PG_##lname); \ -} \ static __always_inline void __SetPage##uname(struct page *page) \ { \ VM_BUG_ON_PAGE(!PageType(page, 0), page); \ page->page_type &= ~PG_##lname; \ } \ -static __always_inline void __folio_set_##fname(struct folio *folio) \ -{ \ - VM_BUG_ON_FOLIO(!folio_test_type(folio, 0), folio); \ - folio->page.page_type &= ~PG_##lname; \ -} \ static __always_inline void __ClearPage##uname(struct page *page) \ { \ VM_BUG_ON_PAGE(!Page##uname(page), page); \ page->page_type |= PG_##lname; \ -} \ -static __always_inline void __folio_clear_##fname(struct folio *folio) \ -{ \ - VM_BUG_ON_FOLIO(!folio_test_##fname(folio), folio); \ - folio->page.page_type |= PG_##lname; \ -} \ +} /* * PageBuddy() indicates that the page is free and in the buddy system From patchwork Thu Mar 21 14:24:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13598774 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D902C54E58 for ; Thu, 21 Mar 2024 14:25:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 251B76B0092; Thu, 21 Mar 2024 10:25:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2028D6B0093; Thu, 21 Mar 2024 10:25:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED4A76B0095; Thu, 21 Mar 2024 10:25:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D352D6B0092 for ; Thu, 21 Mar 2024 10:25:05 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7AC4F41611 for ; Thu, 21 Mar 2024 14:25:05 +0000 (UTC) X-FDA: 81921268170.11.40F3A8C Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf03.hostedemail.com (Postfix) with ESMTP id DC4642000E for ; Thu, 21 Mar 2024 14:25:03 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=CKN43ljK; dmarc=none; spf=none (imf03.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711031104; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hEgT0R9QcI74Q3oBDbLmRkwAKlSYxK4VScRXFQScAqA=; b=e/XGyLrfGhb3vl0DDqEN3+hF71WPTmeWvjVTm+0EPx+eErhH0p6d540SQg/Fybdf5BWPVo jmBNMfdo7NMOjf/gerV+MBlBA2c3Xjpdyn9n4NV3omdu8R2+c+m+NbZdFWwwM0vtrJnMvh 9C09Hqizsz7I3kcutR44jr1nXd7/UGk= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=CKN43ljK; dmarc=none; spf=none (imf03.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711031104; a=rsa-sha256; cv=none; b=bgbjXGPG4OIpRppNqexEvTVbJgiFn1po3L9TeUWVvhJHkW7aGJBYJTW01OFOG4vjJFhi+M Jkd8fCwxu3lWEAeJKNOHlZBuOg+vtjeQk4RLmhX6sZXz16wzP/VjBuXQmDif4fGYYVvdZi 4j/F1TR2nsr55IAwg9qArr9jxNrirjM= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=hEgT0R9QcI74Q3oBDbLmRkwAKlSYxK4VScRXFQScAqA=; b=CKN43ljKVIYmMNstqOZ4ij6FmH 7oZwcwxHAaU9mdhiOY2quY1rITzaqjShTeq2aGv2jS8qkOC8boKGUgjutgaM9FBfaViEbi0T3DDDh l8697w92T+yF4Qp5Key5jPVQjq6Nc1XR3RKELfvUiEk/Sk+bytJwHnes9yWz/sUGN/PpdmxedB62R seNMCDUm+HhZnEkJO1xvM3MKXz1hiZScuEAHi2LpmNYBdIW4AuAz3aYZKU/0D5fo8MAnoT/YaTJNC zP9IC3TGL8uesYxI4zVJoX7tiNZhEr4v3KFkFuYiNxlKX38D4omwq6otc+hKB6YMNss+ROqrtPoM9 9zcji3oQ==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnJLf-00000006u3y-26CJ; Thu, 21 Mar 2024 14:24:51 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, David Hildenbrand , Vlastimil Babka , Miaohe Lin , Muchun Song , Oscar Salvador Subject: [PATCH 3/9] mm: Remove folio_prep_large_rmappable() Date: Thu, 21 Mar 2024 14:24:41 +0000 Message-ID: <20240321142448.1645400-4-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240321142448.1645400-1-willy@infradead.org> References: <20240321142448.1645400-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: jpsxp8afq4xhhozxz1qaf8ehnkpjmbq6 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: DC4642000E X-HE-Tag: 1711031103-583990 X-HE-Meta: U2FsdGVkX1+BSSYMDaP9yk8KFpkBRFUKE/8IlqPE2UcJuF8vArKNzGJL27SreF3g1ouJcXpmyFP67DahionnZ0Pg4t1dq3ynhBhKp0fmMpq8Gf+fdr9+x9d4hONEDGArkEgRN2tbaQUMF+MGJDYr43JVLePJ7jM7UFAhxg1/IqO+E9yNvwUvfPja9SKQRu6HlKGVm+UtH0VLtMp9Gc/wtlN85uG2pRmUzg++xi5R1QIrrBCrxA8CqWVYJP0fQIjCM6b0LCFXNvOqK52G07YPDxVkOonbung13FxB9FItD1jlHf1hpnRpnEW7WPvN9yiUl2PFpgxp40qAgMkeuhHUrZoHulzmqb99S9MjL30s6mFi5VNYdzafAUVlTiULCKE+2u+WCFET+7pvHbSFzELK0Jd2Tnstb3Sn9WXf2XARFb92aNzy/f2YQKv3s8StwgccmP/Mq9IvRYExV5lhQQqlvX9LflI66MRFfkSVg0DUuam05l2madlAx07cWGegwbaskQSVn2wYAuk9ycNetm9gU5JfP/+K6yrp7tV/7FsQ8S5I4LWa2+PvENzFRo2MNxS1Tavms3W3TLMU0TbJ2HnTXbK6Fjz2PMaqqrcRSOR5nDvW7mI87jpWoOsiumLYDPw36+GMuSS+6f4dEH53tvs8Rg3F5RXJCxn1AOeKcS/bNfMOOCsyXe5UnmzIiJHtWJDaGqvqc5tYTygTKtUl4DIyBrKuvnuZgAMXJfX3LPcJqtscj4OXCP42le/rTTI0bO2BZFdz70BBJZgYZDtMnm5D4Bm9U8GqWwAmhVCrpGClT+0LwrsBDNlA6NfUUnhKSJWYamNtfkPestuZo3DB5CFOlcb7QhOlFQ7vomFjnxi27c1SKdUDkxFC/B44aTAqYklE9lzq7bQ+8i3ck/DTGy6yeOt1a7KhH2L/P7LAEWSqRyNaVWPJVYSM8KhNCAhtZ0u8g3PEWXpoWzfG013WSdC I9STV9QF w0PJZWl7aKjFfXw7sdzSna/nyXXFRnhnlgdCjA2p7xQ9EGn9/31NxMEsyksvreOgUOEn29FijRR4BpTR+xgSN4DGHvD98BX9YfGnM+2SvuLHB+pi0PYoCm1cxbyl66+2KatX0/dgJJV00UCMlis79hLIPxeE47UuTY73Q9zGngesBObYsx17CjErarQWpB0YjwVekS+E4f69AkmSDKLQXEIf4gi/VGaKvPJIn X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now that prep_compound_page() initialises folio->_deferred_list, folio_prep_large_rmappable()'s only purpose is to set the large_rmappable flag, so inline it into the two callers. Take the opportunity to convert the large_rmappable definition from PAGEFLAG to FOLIO_FLAG and remove the existance of PageTestLargeRmappable and friends. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Vlastimil Babka Reviewed-by: David Hildenbrand --- include/linux/huge_mm.h | 3 --- include/linux/page-flags.h | 4 ++-- mm/huge_memory.c | 9 +-------- mm/internal.h | 3 ++- 4 files changed, 5 insertions(+), 14 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index de0c89105076..0e16451adaba 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -263,7 +263,6 @@ unsigned long thp_vma_allowable_orders(struct vm_area_struct *vma, unsigned long thp_get_unmapped_area(struct file *filp, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags); -void folio_prep_large_rmappable(struct folio *folio); bool can_split_folio(struct folio *folio, int *pextra_pins); int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, unsigned int new_order); @@ -411,8 +410,6 @@ static inline unsigned long thp_vma_allowable_orders(struct vm_area_struct *vma, return 0; } -static inline void folio_prep_large_rmappable(struct folio *folio) {} - #define transparent_hugepage_flags 0UL #define thp_get_unmapped_area NULL diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index dc1607f1415e..8d0e6ce25ca2 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -869,9 +869,9 @@ static inline void ClearPageCompound(struct page *page) BUG_ON(!PageHead(page)); ClearPageHead(page); } -PAGEFLAG(LargeRmappable, large_rmappable, PF_SECOND) +FOLIO_FLAG(large_rmappable, FOLIO_SECOND_PAGE) #else -TESTPAGEFLAG_FALSE(LargeRmappable, large_rmappable) +FOLIO_FLAG_FALSE(large_rmappable) #endif #define PG_head_mask ((1UL << PG_head)) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 04fb994a7b0b..5cb025341d52 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -788,13 +788,6 @@ struct deferred_split *get_deferred_split_queue(struct folio *folio) } #endif -void folio_prep_large_rmappable(struct folio *folio) -{ - if (!folio || !folio_test_large(folio)) - return; - folio_set_large_rmappable(folio); -} - static inline bool is_transparent_hugepage(struct folio *folio) { if (!folio_test_large(folio)) @@ -2861,7 +2854,7 @@ static void __split_huge_page_tail(struct folio *folio, int tail, clear_compound_head(page_tail); if (new_order) { prep_compound_page(page_tail, new_order); - folio_prep_large_rmappable(new_folio); + folio_set_large_rmappable(new_folio); } /* Finally unfreeze refcount. Additional reference from page cache. */ diff --git a/mm/internal.h b/mm/internal.h index 10895ec52546..ee669963db15 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -513,7 +513,8 @@ static inline struct folio *page_rmappable_folio(struct page *page) { struct folio *folio = (struct folio *)page; - folio_prep_large_rmappable(folio); + if (folio && folio_test_large(folio)) + folio_set_large_rmappable(folio); return folio; } From patchwork Thu Mar 21 14:24:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13598781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A777AC54E68 for ; Thu, 21 Mar 2024 14:25:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CD8F16B009B; Thu, 21 Mar 2024 10:25:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C61186B009C; Thu, 21 Mar 2024 10:25:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB4776B009D; Thu, 21 Mar 2024 10:25:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 93E3C6B009B for ; Thu, 21 Mar 2024 10:25:23 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7475940E33 for ; Thu, 21 Mar 2024 14:25:23 +0000 (UTC) X-FDA: 81921268926.09.9968D42 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf19.hostedemail.com (Postfix) with ESMTP id CF5911A001E for ; Thu, 21 Mar 2024 14:25:21 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=JCNRc3BX; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711031121; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ebFx3o4GQh90ODzh7/NEYBCyas5aF8eLEGHSG28Zkf0=; b=ABMzHthUpMx7jCZvb2/QLmKq+VBhFbSPuCMa6bGCV5GJ5hpRY3FKAWCmqehFJusk6zSdMW eyTAo5Xv9Rs0s41BHkqyU+I4y1SNvAAG+PhwtYRidE8Vm1mmRdckVybiBXShqU+Ze55qME GTCH4PKrVHC7mITPDNzpF9W/3MLDHok= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=JCNRc3BX; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711031121; a=rsa-sha256; cv=none; b=oDHvd2ebok/2vPAa0y+OXWDYisw1sfgrH61F6qsD4/PJUjhgO1Xg44LhvKTiQM7ZQ+7sqa EMn6j4XNS9scLMBsQbja82eUpM2XYAiXzOqm2TNi33PgNYy4hsK9JjQQ5gIrtFlcSzWF1Q U0CXeNJUBcm4b2FX4jEMFEYzqrFHgIc= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ebFx3o4GQh90ODzh7/NEYBCyas5aF8eLEGHSG28Zkf0=; b=JCNRc3BXIymRdQ66QtS5p7JXs+ 5nFG3PhEYXXoxZNox2WCtwEK8yv6McFh44JjwsraSj+ZlkirVHd2hmanGe43kcwRz7xIk4BnDocfG Ika0RNfUnFA9urD6lq9jMYrozLWB5+Iue8XWyqA4lavfzeWJM1LQDj4iTny3R8hsfTZthhkj2pTmE CvHQUZOj8SXD2ipnVxxlh8JpeLu3+aaEv0zs282c22FRbIjY6+E4ygW7NL2ItAZB4quNmctrzqhzj ewup2uV6yz6gEhq3XHhl+2qHwX4QWpABiL0IW155qe8y2oOpRBXJQEUsF+Gl+Ap73iHDr7AhAeoKp /90iMcyg==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnJLf-00000006u40-2Pvk; Thu, 21 Mar 2024 14:24:51 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, David Hildenbrand , Vlastimil Babka , Miaohe Lin , Muchun Song , Oscar Salvador Subject: [PATCH 4/9] mm: Support page_mapcount() on page_has_type() pages Date: Thu, 21 Mar 2024 14:24:42 +0000 Message-ID: <20240321142448.1645400-5-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240321142448.1645400-1-willy@infradead.org> References: <20240321142448.1645400-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: CF5911A001E X-Stat-Signature: 61c1eibtirrno1gkqcu69rz1h77oopbe X-Rspam-User: X-HE-Tag: 1711031121-325227 X-HE-Meta: U2FsdGVkX18l5xX49/Jbr7gxLFh5VwDjslXXwIsB0Syx/3aSVRuXxTMd9WixHYcMFUGKGeSUbIEcv6PKf2ZoMHdRFhBfgxubAbm62l4tw55A1MtuQSoVrUkl0ml7/gTud0KtJkSaIutF11rBfUjDXDHu7We6PMmzybhiYNHXnUCwA8rnT1O7LkgBHJ8Q9p+GnGG/JKyFrwlyFXFu4gsZ820KKAwO5jf3IEjxft2yB1TJZRV7lhK6sDuimTK8ClZTAesH+cssG+8M5h9CqteF1fUnLygKhlyJ+r+wCDLvrAwrT+MiT2f033ZBrPEUZ59BF17SGsslwhu8f0WEj0VRVkK3G+oGaFjuXts6LaCiOjPMreEPga8Bckn8qkpCuBIKFrdcaDZjPmLZuXSuLctpYx2u7hz4sY9jGCL6aEMfV6n4cRK6EqonHDFO+OawJEAMP1W43GZA7Vw2b2GiEIrlv1Qgm3hr5zqKT4jcyjoV9LgDDEa+PhGzyc43oy/kjA7vdq7AV1TBignobmxInK9XpBiFaMckG1n28+0LIO4HW370ibBeOz4bjP6rRYO6ENqyN9zo9fQEB87Y0wKKHdy0vf9Vk2J+c2xwyYQClaYqsFus1EB0lbvBPqE1JAAdBB06ucr1vD09KEnWZ6+pAe6k3C3rucksX1xNrMjE1wDYBS1XptDwSGQvHh0kHmQYdj+PNlqzmfcMz+zsiuKMrDgFgf2VNn4a/Wsaxa68hJa5XQN+PwfDXk2y6d5P+UGZIb+C+ko9KadxcslHTtkQxBJqHI2jjntvO67Aiv8ERG8faUPOzegD5EsL5Hy6PopQcIOheFBU3HvUGw0wcYrtkgIEt3HrxeHNjGnu902VNbTZPwGUbUOo1v/7ve1wHBK45h87a9BHRmXeNCO1cUCXSGwNMpM8f/zR4cU2CkoHgnxuHE9K4FJIqgee8rPtDV6+Wsp0/03W1RqHWEWysSdzSVn EGNBPaAR DI639d03g0npC4D2lbISF4dgYdY/t6d8DzZUnv9KksPOozsiRRq5ce4miRmUtVZY8K5lWI1NvFEFSS8/fln8qEL74AO4mHQHdmJeT+Am2ipH5OuvDgyRDgQT6FvLL8HrObbjwsYVQAvuUbACMl7j699AuFGfE0zsYTzwQt8P3QPYyDzeVD/4aCqF1vC98i2UIoZz8s7ggivVtJ+Aa1IBBbyljUm+DWoJA/G1d X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Return 0 for pages which can't be mapped. This matches how page_mapped() works. It is more convenient for users to not have to filter out these pages. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Vlastimil Babka Reviewed-by: David Hildenbrand --- fs/proc/page.c | 7 ++----- include/linux/mm.h | 8 +++++--- include/linux/page-flags.h | 4 ++-- 3 files changed, 9 insertions(+), 10 deletions(-) diff --git a/fs/proc/page.c b/fs/proc/page.c index 195b077c0fac..9223856c934b 100644 --- a/fs/proc/page.c +++ b/fs/proc/page.c @@ -67,7 +67,7 @@ static ssize_t kpagecount_read(struct file *file, char __user *buf, */ ppage = pfn_to_online_page(pfn); - if (!ppage || PageSlab(ppage) || page_has_type(ppage)) + if (!ppage) pcount = 0; else pcount = page_mapcount(ppage); @@ -124,11 +124,8 @@ u64 stable_page_flags(struct page *page) /* * pseudo flags for the well known (anonymous) memory mapped pages - * - * Note that page->_mapcount is overloaded in SLAB, so the - * simple test in page_mapped() is not enough. */ - if (!PageSlab(page) && page_mapped(page)) + if (page_mapped(page)) u |= 1 << KPF_MMAP; if (PageAnon(page)) u |= 1 << KPF_ANON; diff --git a/include/linux/mm.h b/include/linux/mm.h index 0436b919f1c7..5ff3d687bc6c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1223,14 +1223,16 @@ static inline void page_mapcount_reset(struct page *page) * a large folio, it includes the number of times this page is mapped * as part of that folio. * - * The result is undefined for pages which cannot be mapped into userspace. - * For example SLAB or special types of pages. See function page_has_type(). - * They use this field in struct page differently. + * Will report 0 for pages which cannot be mapped into userspace, eg + * slab, page tables and similar. */ static inline int page_mapcount(struct page *page) { int mapcount = atomic_read(&page->_mapcount) + 1; + /* Handle page_has_type() pages */ + if (mapcount < 0) + mapcount = 0; if (unlikely(PageCompound(page))) mapcount += folio_entire_mapcount(page_folio(page)); diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 8d0e6ce25ca2..5852f967c640 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -971,12 +971,12 @@ static inline bool is_page_hwpoison(struct page *page) * page_type may be used. Because it is initialised to -1, we invert the * sense of the bit, so __SetPageFoo *clears* the bit used for PageFoo, and * __ClearPageFoo *sets* the bit used for PageFoo. We reserve a few high and - * low bits so that an underflow or overflow of page_mapcount() won't be + * low bits so that an underflow or overflow of _mapcount won't be * mistaken for a page type value. */ #define PAGE_TYPE_BASE 0xf0000000 -/* Reserve 0x0000007f to catch underflows of page_mapcount */ +/* Reserve 0x0000007f to catch underflows of _mapcount */ #define PAGE_MAPCOUNT_RESERVE -128 #define PG_buddy 0x00000080 #define PG_offline 0x00000100 From patchwork Thu Mar 21 14:24:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13598779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04769C54E58 for ; Thu, 21 Mar 2024 14:25:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 585CA6B0099; Thu, 21 Mar 2024 10:25:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 50FEF6B009A; Thu, 21 Mar 2024 10:25:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2EE566B009B; Thu, 21 Mar 2024 10:25:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 19D886B0099 for ; Thu, 21 Mar 2024 10:25:18 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D7C2BC160F for ; Thu, 21 Mar 2024 14:25:17 +0000 (UTC) X-FDA: 81921268674.23.4F18708 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf02.hostedemail.com (Postfix) with ESMTP id 1289A80005 for ; Thu, 21 Mar 2024 14:25:15 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=rG6FPrBm; dmarc=none; spf=none (imf02.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711031116; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YSXRYNjFOMopeMCyQudR+fTuGGBUc9i8jkm6+jE9EkQ=; b=Y+Et24czPn70/TMyAdfx5gPUeqYrJOYrsT2Gtv0kwhHClj+2eBBh5uhn4k1Gk3AxlRUQuN atEGB0zYLEAibmXgI+xehQinvoTmjk4g78rf3otcdRRBClnWDsG2BYJOhdckPUaSRQbJvs aVM6qxO6vEzALClr6MwHyMFTHHNYlmQ= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=rG6FPrBm; dmarc=none; spf=none (imf02.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711031116; a=rsa-sha256; cv=none; b=qlqZ2b4KaJ9YAefX7i1UwtXC4hPlgBXzFBxUY79LLHNB8BB+mo5eDYQodGZm2r1qCsZHDt rNLWcilfqGSeE3xfsFGshNbKzr9s3u5YdIv4XivVm/PiZp++r6u5cOuusooNMgqfassyUC D3JYHg2R386EsSg36G8BqJMNdGIJG1k= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=YSXRYNjFOMopeMCyQudR+fTuGGBUc9i8jkm6+jE9EkQ=; b=rG6FPrBmd1O5jT8zp4YGxsfqjK pJAqGO1dH5yRnC8dzPWMuh63NMP2CcMYqUjGsrT4iyIeyhisq3To8X/GRrI2WZrcjsBfaSxHaJUb7 WGUzkL0Yon/lAtdP/M2WyzSzxPYkY1PQmz5KtHaX1glDMUabi11CZ+fDruOEZjP2EFR4D+LYLLUYL VF3D9Esoo6Y8HV8rPanb/ViJ1LhqKbPDtMdIROHhP4yssmRTk3enfbiak2nWGWN+rjyKy4vIcAF47 pCEloN8T0O/IHN1zrDeIaaFYI351F94ZOr6wveP88RtgBJ7u1VYXrtViINApocLYHGx+0BMJu0cJQ ffubHvtw==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnJLf-00000006u42-2m9a; Thu, 21 Mar 2024 14:24:51 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, David Hildenbrand , Vlastimil Babka , Miaohe Lin , Muchun Song , Oscar Salvador Subject: [PATCH 5/9] mm: Turn folio_test_hugetlb into a PageType Date: Thu, 21 Mar 2024 14:24:43 +0000 Message-ID: <20240321142448.1645400-6-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240321142448.1645400-1-willy@infradead.org> References: <20240321142448.1645400-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: h1o3zgffaru8gmp6f8do5bdnwiefddyq X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1289A80005 X-HE-Tag: 1711031115-264285 X-HE-Meta: U2FsdGVkX1/I8qqhZs2z/8iQ+mj4VfjFZrzRAtUxTWov537jzz1Iig2PkQTqo+Nvxr/3ncVRk71J3ve3sUQk3y3EZK0AazNHIPsdDXmqtxk0/h6BZh1GFpgavpMHtXInOMn7xHOIpx3qcRJz0UbWf5JVKqJq59tGP5zlpGxLInBzLjoxhwwM8o66ZyphDXp0HvYAvmfNMOfzYZ9iuKbjE6RvLkQR/LzJ/MIyoTLxFEjfsIhnG3+I4CWx0iSfUgNOgn3o699DMW5ek4lI/mLGn85Neq9xp5FH86qVijG0CL0QgI6dz5CnugYgMa12zCivivVdCqoMGBR4/5+jnbAuWK4w9gYf9SzLoZgDH96E7zaPXvi3SVFT0a1Ju8HoUeUYWnJKE5akIZNIhaFuflSSX/yffbg1O14Jihueex52Bf+5rVmlOIqt6kWSIb3AyikLJVjvpUqWNHx1B6nH8Fxx3wYIyTrap0gtxutxGM4kAO6rfUTgTmmfa/5bfCTJ2zKX/1SLoEHPuXdpgMiBeZXDaTFHy6+grnrzF/9chXUYAZEGdU+JXSCI8+dwbnvZ2Xe4jj4f92VSmL6MfSbmVPZi4SLaBvayMVS3BymL3XsOFBLVjISnIdsWckbiRos3u9eD2Gu5WE/K0AQ0Y5GdI4PLc+dXdEHrZXHmDpF56BcVJ0PzQuMQI9xrwVdrhHK8UD0a21bFyjHLfKqkGcT7gXIjBlc09Jh3GzqYUaoEHWWRZcpzEMHNCw9YOAh9nVfg2XPRhLOlh1fi45vdQ/FX/6XL5GaB2GbHCPgt3zS4USlcV6szvPgPF51Ng0raXIcLbI5/lYS2z77M3slWceqLU5IjGQFVyNQu7eNP/nM2FhZWZjhg4DU8gp7VRYRroS0MUcpA531r5gcGRGjgQ/K3NbTMHLlRPS3RU+x5C3N+sG3Z1rkXrvjkGqMXQgB6svssJpVYPAp2ZITvV7ew+c0WPVS 6orS5efy k0Ct8ddNbiVwAC0PUbLFd6w6fMe1rK3zrVmKAEcarTkBLpVfI+JydWOZkFu3qF4IAYlFJw9sZsslQ/FEbWHrtXZ8/21B959Mnr88DTzXllrTjMneva7jXKFAMtuetvxvkdwb5t8v0dybJCDmnC2vYmF0eV/n3bzGYNHDY6UFYJ6u/eUm6wvyxLMF1/ibbg5OWKk2t29M5Yvgr/rnkDA64OsBiKK93jB8mRec77xeHTHoWjRFkbTdN5CKfX6fLNjcXCaIB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The current folio_test_hugetlb() can be fooled by a concurrent folio split into returning true for a folio which has never belonged to hugetlbfs. This can't happen if the caller holds a refcount on it, but we have a few places (memory-failure, compaction, procfs) which do not and should not take a speculative reference. Since hugetlb pages do not use individual page mapcounts (they are always fully mapped and use the entire_mapcount field to record the number of mappings), the PageType field is available now that page_mapcount() ignores the value in this field. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: David Hildenbrand Acked-by: Vlastimil Babka Reported-by: Luis Chamberlain --- include/linux/page-flags.h | 70 ++++++++++++++++------------------ include/trace/events/mmflags.h | 1 + mm/hugetlb.c | 22 ++--------- 3 files changed, 37 insertions(+), 56 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 5852f967c640..6fb3cd42ee59 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -190,7 +190,6 @@ enum pageflags { /* At least one page in this folio has the hwpoison flag set */ PG_has_hwpoisoned = PG_error, - PG_hugetlb = PG_active, PG_large_rmappable = PG_workingset, /* anon or file-backed */ }; @@ -876,29 +875,6 @@ FOLIO_FLAG_FALSE(large_rmappable) #define PG_head_mask ((1UL << PG_head)) -#ifdef CONFIG_HUGETLB_PAGE -int PageHuge(const struct page *page); -SETPAGEFLAG(HugeTLB, hugetlb, PF_SECOND) -CLEARPAGEFLAG(HugeTLB, hugetlb, PF_SECOND) - -/** - * folio_test_hugetlb - Determine if the folio belongs to hugetlbfs - * @folio: The folio to test. - * - * Context: Any context. Caller should have a reference on the folio to - * prevent it from being turned into a tail page. - * Return: True for hugetlbfs folios, false for anon folios or folios - * belonging to other filesystems. - */ -static inline bool folio_test_hugetlb(const struct folio *folio) -{ - return folio_test_large(folio) && - test_bit(PG_hugetlb, const_folio_flags(folio, 1)); -} -#else -TESTPAGEFLAG_FALSE(Huge, hugetlb) -#endif - #ifdef CONFIG_TRANSPARENT_HUGEPAGE /* * PageHuge() only returns true for hugetlbfs pages, but not for @@ -954,18 +930,6 @@ PAGEFLAG_FALSE(HasHWPoisoned, has_hwpoisoned) TESTSCFLAG_FALSE(HasHWPoisoned, has_hwpoisoned) #endif -/* - * Check if a page is currently marked HWPoisoned. Note that this check is - * best effort only and inherently racy: there is no way to synchronize with - * failing hardware. - */ -static inline bool is_page_hwpoison(struct page *page) -{ - if (PageHWPoison(page)) - return true; - return PageHuge(page) && PageHWPoison(compound_head(page)); -} - /* * For pages that are never mapped to userspace (and aren't PageSlab), * page_type may be used. Because it is initialised to -1, we invert the @@ -982,6 +946,7 @@ static inline bool is_page_hwpoison(struct page *page) #define PG_offline 0x00000100 #define PG_table 0x00000200 #define PG_guard 0x00000400 +#define PG_hugetlb 0x00000800 #define PageType(page, flag) \ ((page->page_type & (PAGE_TYPE_BASE | flag)) == PAGE_TYPE_BASE) @@ -1076,6 +1041,37 @@ PAGE_TYPE_OPS(Table, table, pgtable) */ PAGE_TYPE_OPS(Guard, guard, guard) +#ifdef CONFIG_HUGETLB_PAGE +FOLIO_TYPE_OPS(hugetlb, hugetlb) +#else +FOLIO_TEST_FLAG_FALSE(hugetlb) +#endif + +/** + * PageHuge - Determine if the page belongs to hugetlbfs + * @page: The page to test. + * + * Context: Any context. + * Return: True for hugetlbfs pages, false for anon pages or pages + * belonging to other filesystems. + */ +static inline bool PageHuge(const struct page *page) +{ + return folio_test_hugetlb(page_folio(page)); +} + +/* + * Check if a page is currently marked HWPoisoned. Note that this check is + * best effort only and inherently racy: there is no way to synchronize with + * failing hardware. + */ +static inline bool is_page_hwpoison(struct page *page) +{ + if (PageHWPoison(page)) + return true; + return PageHuge(page) && PageHWPoison(compound_head(page)); +} + extern bool is_free_buddy_page(struct page *page); PAGEFLAG(Isolated, isolated, PF_ANY); @@ -1142,7 +1138,7 @@ static __always_inline void __ClearPageAnonExclusive(struct page *page) */ #define PAGE_FLAGS_SECOND \ (0xffUL /* order */ | 1UL << PG_has_hwpoisoned | \ - 1UL << PG_hugetlb | 1UL << PG_large_rmappable) + 1UL << PG_large_rmappable) #define PAGE_FLAGS_PRIVATE \ (1UL << PG_private | 1UL << PG_private_2) diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index d801409b33cf..d55e53ac91bd 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -135,6 +135,7 @@ IF_HAVE_PG_ARCH_X(arch_3) #define DEF_PAGETYPE_NAME(_name) { PG_##_name, __stringify(_name) } #define __def_pagetype_names \ + DEF_PAGETYPE_NAME(hugetlb), \ DEF_PAGETYPE_NAME(offline), \ DEF_PAGETYPE_NAME(guard), \ DEF_PAGETYPE_NAME(table), \ diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 7e9a766059aa..bdcbb62096cf 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1624,7 +1624,7 @@ static inline void __clear_hugetlb_destructor(struct hstate *h, { lockdep_assert_held(&hugetlb_lock); - folio_clear_hugetlb(folio); + __folio_clear_hugetlb(folio); } /* @@ -1711,7 +1711,7 @@ static void add_hugetlb_folio(struct hstate *h, struct folio *folio, h->surplus_huge_pages_node[nid]++; } - folio_set_hugetlb(folio); + __folio_set_hugetlb(folio); folio_change_private(folio, NULL); /* * We have to set hugetlb_vmemmap_optimized again as above @@ -2050,7 +2050,7 @@ static void __prep_account_new_huge_page(struct hstate *h, int nid) static void init_new_hugetlb_folio(struct hstate *h, struct folio *folio) { - folio_set_hugetlb(folio); + __folio_set_hugetlb(folio); INIT_LIST_HEAD(&folio->lru); hugetlb_set_folio_subpool(folio, NULL); set_hugetlb_cgroup(folio, NULL); @@ -2160,22 +2160,6 @@ static bool prep_compound_gigantic_folio_for_demote(struct folio *folio, return __prep_compound_gigantic_folio(folio, order, true); } -/* - * PageHuge() only returns true for hugetlbfs pages, but not for normal or - * transparent huge pages. See the PageTransHuge() documentation for more - * details. - */ -int PageHuge(const struct page *page) -{ - const struct folio *folio; - - if (!PageCompound(page)) - return 0; - folio = page_folio(page); - return folio_test_hugetlb(folio); -} -EXPORT_SYMBOL_GPL(PageHuge); - /* * Find and lock address space (mapping) in write mode. * From patchwork Thu Mar 21 14:24:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13598777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 011AEC54E68 for ; Thu, 21 Mar 2024 14:25:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 251776B0096; Thu, 21 Mar 2024 10:25:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1661F6B0098; Thu, 21 Mar 2024 10:25:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EAE946B0099; Thu, 21 Mar 2024 10:25:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D2CBB6B0096 for ; Thu, 21 Mar 2024 10:25:11 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id AB1D1A15AD for ; Thu, 21 Mar 2024 14:25:11 +0000 (UTC) X-FDA: 81921268422.29.127A754 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf07.hostedemail.com (Postfix) with ESMTP id C9C2F40023 for ; Thu, 21 Mar 2024 14:25:09 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="rB/KMdaV"; dmarc=none; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711031109; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pVgu8F+HUxFYXHQ8q7hUsgbZUWBjtZzUu+JH6Hd85qY=; b=lM3T/2U1/glNGabzFaW79w2UaPaSREz5FzGA5e4I96Vg2HiE4qhGjhyz6eC7qIXWIW2pwp ivlGHsyE9jAGFSERG+HfR/TjP9SSDAsNTo51mhZpQjz0NmxnxsylY/7YryaxGxFqVb5USc KT0yJrtD/0ID7KU4ePfx49LyJyEds50= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="rB/KMdaV"; dmarc=none; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711031109; a=rsa-sha256; cv=none; b=4Vzp2rOUP4xrxO+yJmut63DUoJVItziq7rS+/rIHSAfvxksAfmqPcTSo3qGMjBlgygNNGI ludSlYBs6jxnAw/mUxLw2j5F0AVybAvbPKfPiy6Kr1BRxPmEE58nrxWuWW2ndDBsom0MJn 3G7KCPMb9/+ixcAzxxQtxA7U8RzR3hc= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=pVgu8F+HUxFYXHQ8q7hUsgbZUWBjtZzUu+JH6Hd85qY=; b=rB/KMdaVGxjLcMzjJL7/JMK9u+ nNkmy/TutiWXmphtt9Ro/xQrlv1yEeYIyl0IB3ljZprvmo+i+Cj5ksiFALBXKQ+601sqsHwomNWic SMTXrXdPoSTfJHDCZSzYQzLQqa6RzFzQ62t1+Edjih9GC4p0/7gX3281eAefY0bc5DoXj6kWSCKM2 1A8/JrMeb/7HDV+wPZFW+t3yXZnCS1Wvk3BTOM2YHcVzOnowJC7zHHS7hlXNbDvc2DBsGY57WBvXI 88vS0zxStbdKzEFRlMo37uaTtIgsEQLtnBk567x/m8cve0KPz6cXEClhEtTcet1RdNfuYczucF1Oe 2y4k3+BQ==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnJLf-00000006u44-39lp; Thu, 21 Mar 2024 14:24:51 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, David Hildenbrand , Vlastimil Babka , Miaohe Lin , Muchun Song , Oscar Salvador Subject: [PATCH 6/9] mm: Remove a call to compound_head() from is_page_hwpoison() Date: Thu, 21 Mar 2024 14:24:44 +0000 Message-ID: <20240321142448.1645400-7-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240321142448.1645400-1-willy@infradead.org> References: <20240321142448.1645400-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: C9C2F40023 X-Stat-Signature: fzah93irxjidwj7soeup3etjhr848w6u X-Rspam-User: X-HE-Tag: 1711031109-516111 X-HE-Meta: U2FsdGVkX18knQRTV2D1a+AC6G3fedRw5Q9iIkjKS/G5Y3iRsJy/0rNidsIqhiC0QkbQqtx5sNS8daeQrJdMzW8B0t1lpBlZtjs4NTsnuzyig2Smw0FvbSrbDWgKIPP3yyUbO9jZiTaxPESQmfYu8NKJ++cHSaHN5NriYiIz9UdwmEhOYbc0Y0INo/nzZ41YNw2wNhmAUl5MAij/1D/g38ps/Q+NSWgTyjIVkSc3EMuRFo1Bn8y+l0QzdSiQdYvH1SUT9ikWGpCqc9FSr1AVKS5qzzszfAdsNlG3VlLXn2mwfuvJhVDhjwTdS3vTraAeCTxLanfgD+HeD0PMIpC9pOxZr2JkLHS41QhNJiSGGfQGjsiRdkWhn5oBFsV7SxMyTxrxjed3Gq+mOTbU4GzzjmKFh8vyQMw/jU2NNaVvV5hY4OJqmZ6mlOtg69KhE3I0a/aT6iQWLB8PCvUdcotv0wsmgGsLrbp6l6VTi2f4582dQr/FdmmoPzhuCFeAyA0ZJuuYsogpTgCyLKyduRGr2cBUAAf75hynzq6OnQQA4+9m7snpPQeYDDgWhGOkdYVcZIFdjtBupwCtK98qEEifHOOTYqw65DVIjrNbzDbXTXZq5LGvbwC8YxFSayMRHofbtzdOFpfwqqxBjh/v2dWwY0PLvgc59pVWaoqXMpSxb4jnMqqiKCxjfWziuODjXs2apNzySBg1FzrIxEw9NgIbpsx0kUXOrdzYux9EwaZijnjGyK5T2y7DsD+ZPrb9IdVhFcYrXzXpAQQ7DA14cvlJ0hHZ5ROQpMxiNILATUaUNBpjzXeUuIbxKtFp9ptci+KWAT80k8ERTY7U+BH/V5K6/SUBI+I7gnXL22+3kg5bTWLILfLeKCzTUmhMu7GAsPOIXId4yhVZD/PIN6GKZsbAXwOAFel8VFF4fsdAgeqjqqfDhPeXcVUJYQCt9DAz5Y6s1p7LTXVjTbYZ4eidtOB ARiApsmb MnvRfCVbY//oj+wqtBsH/xvyrvcrWmRF770p8/mwgGC4GTC3ccddA73QFcb9C/QTsvXAlGwd3KzQTo1NJoH8Spw0vlOF1+njyo29qrGod6wFRgV9Bva278AghDBLhh+dUBtaJtoo4r2PUh2muij8EqNT97dKO9M7aOnMunj0imLSqHjk750U6YgUAe4O/PkU9CAp4G9kPBDJI/+1puSgeoH3ygCCfsPZEB+yWVnMlmj1IILAlwhdhQCli+Al5rWjAgcEGqjl/OhXFwenqzzYpNFv76w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We can call it only once instead of twice. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: David Hildenbrand Reviewed-by: Miaohe Lin Acked-by: Vlastimil Babka --- include/linux/page-flags.h | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 6fb3cd42ee59..94eb8a11a321 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -1065,11 +1065,14 @@ static inline bool PageHuge(const struct page *page) * best effort only and inherently racy: there is no way to synchronize with * failing hardware. */ -static inline bool is_page_hwpoison(struct page *page) +static inline bool is_page_hwpoison(const struct page *page) { + const struct folio *folio; + if (PageHWPoison(page)) return true; - return PageHuge(page) && PageHWPoison(compound_head(page)); + folio = page_folio(page); + return folio_test_hugetlb(folio) && PageHWPoison(&folio->page); } extern bool is_free_buddy_page(struct page *page); From patchwork Thu Mar 21 14:24:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13598778 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75DD4C54E58 for ; Thu, 21 Mar 2024 14:25:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EFB2E6B0098; Thu, 21 Mar 2024 10:25:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E83E06B0099; Thu, 21 Mar 2024 10:25:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB11F6B009A; Thu, 21 Mar 2024 10:25:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B35A66B0098 for ; Thu, 21 Mar 2024 10:25:14 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 78ABB804BB for ; Thu, 21 Mar 2024 14:25:14 +0000 (UTC) X-FDA: 81921268548.01.E9470AC Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf15.hostedemail.com (Postfix) with ESMTP id D4350A001B for ; Thu, 21 Mar 2024 14:25:12 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=J4T7BUhH; spf=none (imf15.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711031112; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=H59Aoy3vN+vh0NMsD3DjW5sbwztCBqX+xLBKaHRsFkw=; b=nsrmAufzrZMgmfjdTGUvQSFFLEMrFsqPMoIjX0uWbNWIL/Nrm2uFdHELO8D3n2R5K5wxUx ZRKUrfvzFIjF4DStS7gw8aD/iRkRjyvfSviJH1EMWC8u69D3vHXPquEjxjVoSgmjeWOj1C X3XPCqY9Yx/q5j5QbRCYXFGSfabDfFI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711031112; a=rsa-sha256; cv=none; b=oJkquw92LIBzYx+yF2WXmVsgq26QAK8/SkzRRgLaWcZwmVj+lmzWTGNfWlRt5SI62tkFPX jghONjTYZIo/ozCWKxl6/zFQ7P9ijS3l23yxJFnPuCuVZvmF57ckJDEkHfSavHQW8edHkQ G8Dy811oPaBAp96w93HOzhfsmc01wPk= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=J4T7BUhH; spf=none (imf15.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=H59Aoy3vN+vh0NMsD3DjW5sbwztCBqX+xLBKaHRsFkw=; b=J4T7BUhH0G+A9Dl7iHQbYUR64L xfcfXa6qDJNyj5/LEDXd3O7dQAULqO7j0j76FCHQpZOECwB7wS87KlVcrAALEWaxKV6TxSNHEWgey 4xKuQ9o0LYynbztzkFbschADJTF9mO87MozrdmQTtMNsWWwxfKS81KyjFmIrnjXexymUXUvhU/gxX CrVSpVJkKXDsAjSbYyb4B1r7t63Q5THlU3GHGsOrLhetDeG7ICph3PItO6HzWcHnFmr/VhXeA76Bd eFjum8m8Fa7UB3ACu4h4D6xMsQFUFSihvSdJ/nSSbnFW0DPQD0UOKrbIzz8HMahjDOR5je98RMadj JGC0GwHg==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnJLf-00000006u46-3Xaa; Thu, 21 Mar 2024 14:24:51 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, David Hildenbrand , Vlastimil Babka , Miaohe Lin , Muchun Song , Oscar Salvador Subject: [PATCH 7/9] mm: Free up PG_slab Date: Thu, 21 Mar 2024 14:24:45 +0000 Message-ID: <20240321142448.1645400-8-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240321142448.1645400-1-willy@infradead.org> References: <20240321142448.1645400-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: D4350A001B X-Rspam-User: X-Stat-Signature: p3gstd9praeykptnpko4h8ya7f936fgz X-Rspamd-Server: rspam03 X-HE-Tag: 1711031112-727641 X-HE-Meta: U2FsdGVkX1+XY8/PThLnZLUmaN1+2M67nrSF8EULcyQuni9jHCya0DcvXf23IrbAabvjgJTUUlNU1/6qm9gBKcON+O/eLlrxyRyIe/Hsq0gKZHjIwrQUForj4qa4BZPwW/IE1NkEc6f+igt+64WsNcX3uM2+BPAAU9rhr9kZIA1khpxki5fYUdTEGAFjPv3L3Km/vQYiE7V4E9L10Gsbtyl4nHypqS9LIT1HkLleRTHtQxRiUML/kXvCGenVBgOp17fLbeX3OI/sReUQoKQlSM9hqmVMcqIINvDC0NsdGv137IO1ewF8l7Y14rkMJqPvQsi48GmGbsVbUhxwHzQMHY02QFIeymBafojXS89uFCTYO4hg9Fi6RzFuTknGSnmkKgpw+Rls5urgx6PdvmuP5Qby5++Yyil+e1IW/BEOvAlohDN5TbjL5Z1z8FffYh0KfbqUpfuItSQ5A5OitXHR63BKJJLl7SN06m6Ic0MB+8TEAjxYOAN1lXurtsnH9IsWmhuzA71znQDUp0AwG32Dr7zAp7hObS1xjtq+J5q7qg8dHIs1adnZ0rLW4WzM+e/Uv494h+qiGZFgwqoH2GSscrCZ+W2Y5fVIZR54GnnJs5HYpZ3aOspJG2L5zmZ5/hJqIO8eMDV0FkVsBx0kE0Y3jeqo+LcahKhcdrHR2xWnJr9TmtW3rKwzezYCO+OnNKiqone+T7+Ea8KTrcHTV8NvGmyFSFZNeZwB4drH6COLvv/6ASv1RLe7GoUGauYR6/oujjlxIrXUvcFk+sS7Xsi/edfk7pX+nuI68K9/2kqPM6qN0p1utFsdK12S34VnPjcDxWjNs1cEWwVLOM5CwativVCSC0FK22XC4Pmf86K0BN3UrBiraizBiylRePWAUQE73iIf8aXSflaTsFHI3j83TchvuKs9aIOE1r0ULwkx5W/hTmQi9DmL6dYAvIwwDXmKSRtH1qNEVfsHYYTlZ2O v2BGg7ES N6i6yrPQKJGHDdcaOycM2j8vHLhwJ89lJGcqegpBeghkJu8OS7yzhr5kTva96VYnJ4178un8zQm0gpFIM1y23B6mcNWWx4UWT8ByUUH1Rwh1aaOs6AQa/t9/DJfDpbhJnpTXBuvGgHekPMuE8iVpkS3Pw+d63U03Gc9MJ+1/q+emYYXtJ4DwCDlLeiB+ktctUxp7qJ/F9Y7LVrOf05UHOlW3gcbIRFE2kTtl/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Reclaim the Slab page flag by using a spare bit in PageType. We are perennially short of page flags for various purposes, and now that the original SLAB allocator has been retired, SLUB does not use the mapcount/page_type field. This lets us remove a number of special cases for ignoring mapcount on Slab pages. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Vlastimil Babka Acked-by: David Hildenbrand --- include/linux/page-flags.h | 21 +++++++++++++++++---- include/trace/events/mmflags.h | 2 +- mm/memory-failure.c | 9 --------- mm/slab.h | 2 +- 4 files changed, 19 insertions(+), 15 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 94eb8a11a321..73e0b17c7728 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -109,7 +109,6 @@ enum pageflags { PG_active, PG_workingset, PG_error, - PG_slab, PG_owner_priv_1, /* Owner use. If pagecache, fs may use*/ PG_arch_1, PG_reserved, @@ -524,7 +523,6 @@ PAGEFLAG(Active, active, PF_HEAD) __CLEARPAGEFLAG(Active, active, PF_HEAD) TESTCLEARFLAG(Active, active, PF_HEAD) PAGEFLAG(Workingset, workingset, PF_HEAD) TESTCLEARFLAG(Workingset, workingset, PF_HEAD) -__PAGEFLAG(Slab, slab, PF_NO_TAIL) PAGEFLAG(Checked, checked, PF_NO_COMPOUND) /* Used by some filesystems */ /* Xen */ @@ -931,7 +929,7 @@ PAGEFLAG_FALSE(HasHWPoisoned, has_hwpoisoned) #endif /* - * For pages that are never mapped to userspace (and aren't PageSlab), + * For pages that are never mapped to userspace, * page_type may be used. Because it is initialised to -1, we invert the * sense of the bit, so __SetPageFoo *clears* the bit used for PageFoo, and * __ClearPageFoo *sets* the bit used for PageFoo. We reserve a few high and @@ -947,6 +945,7 @@ PAGEFLAG_FALSE(HasHWPoisoned, has_hwpoisoned) #define PG_table 0x00000200 #define PG_guard 0x00000400 #define PG_hugetlb 0x00000800 +#define PG_slab 0x00001000 #define PageType(page, flag) \ ((page->page_type & (PAGE_TYPE_BASE | flag)) == PAGE_TYPE_BASE) @@ -1041,6 +1040,20 @@ PAGE_TYPE_OPS(Table, table, pgtable) */ PAGE_TYPE_OPS(Guard, guard, guard) +FOLIO_TYPE_OPS(slab, slab) + +/** + * PageSlab - Determine if the page belongs to the slab allocator + * @page: The page to test. + * + * Context: Any context. + * Return: True for slab pages, false for any other kind of page. + */ +static inline bool PageSlab(const struct page *page) +{ + return folio_test_slab(page_folio(page)); +} + #ifdef CONFIG_HUGETLB_PAGE FOLIO_TYPE_OPS(hugetlb, hugetlb) #else @@ -1121,7 +1134,7 @@ static __always_inline void __ClearPageAnonExclusive(struct page *page) (1UL << PG_lru | 1UL << PG_locked | \ 1UL << PG_private | 1UL << PG_private_2 | \ 1UL << PG_writeback | 1UL << PG_reserved | \ - 1UL << PG_slab | 1UL << PG_active | \ + 1UL << PG_active | \ 1UL << PG_unevictable | __PG_MLOCKED | LRU_GEN_MASK) /* diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index d55e53ac91bd..e46d6e82765e 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -107,7 +107,6 @@ DEF_PAGEFLAG_NAME(lru), \ DEF_PAGEFLAG_NAME(active), \ DEF_PAGEFLAG_NAME(workingset), \ - DEF_PAGEFLAG_NAME(slab), \ DEF_PAGEFLAG_NAME(owner_priv_1), \ DEF_PAGEFLAG_NAME(arch_1), \ DEF_PAGEFLAG_NAME(reserved), \ @@ -135,6 +134,7 @@ IF_HAVE_PG_ARCH_X(arch_3) #define DEF_PAGETYPE_NAME(_name) { PG_##_name, __stringify(_name) } #define __def_pagetype_names \ + DEF_PAGETYPE_NAME(slab), \ DEF_PAGETYPE_NAME(hugetlb), \ DEF_PAGETYPE_NAME(offline), \ DEF_PAGETYPE_NAME(guard), \ diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 9349948f1abf..1cb41ba7870c 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1239,7 +1239,6 @@ static int me_huge_page(struct page_state *ps, struct page *p) #define mlock (1UL << PG_mlocked) #define lru (1UL << PG_lru) #define head (1UL << PG_head) -#define slab (1UL << PG_slab) #define reserved (1UL << PG_reserved) static struct page_state error_states[] = { @@ -1249,13 +1248,6 @@ static struct page_state error_states[] = { * PG_buddy pages only make a small fraction of all free pages. */ - /* - * Could in theory check if slab page is free or if we can drop - * currently unused objects without touching them. But just - * treat it as standard kernel for now. - */ - { slab, slab, MF_MSG_SLAB, me_kernel }, - { head, head, MF_MSG_HUGE, me_huge_page }, { sc|dirty, sc|dirty, MF_MSG_DIRTY_SWAPCACHE, me_swapcache_dirty }, @@ -1282,7 +1274,6 @@ static struct page_state error_states[] = { #undef mlock #undef lru #undef head -#undef slab #undef reserved static void update_per_node_mf_stats(unsigned long pfn, diff --git a/mm/slab.h b/mm/slab.h index d2bc9b191222..457b15da2a6b 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -84,8 +84,8 @@ struct slab { }; struct rcu_head rcu_head; }; - unsigned int __unused; + unsigned int __page_type; atomic_t __page_refcount; #ifdef CONFIG_MEMCG unsigned long memcg_data; From patchwork Thu Mar 21 14:24:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13598775 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF8C3CD11BF for ; Thu, 21 Mar 2024 14:25:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2A1BB6B0093; Thu, 21 Mar 2024 10:25:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 202E06B0095; Thu, 21 Mar 2024 10:25:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 054646B0096; Thu, 21 Mar 2024 10:25:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E64C16B0093 for ; Thu, 21 Mar 2024 10:25:06 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id AC6C9141556 for ; Thu, 21 Mar 2024 14:25:06 +0000 (UTC) X-FDA: 81921268212.28.EA20812 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf25.hostedemail.com (Postfix) with ESMTP id C545EA0019 for ; Thu, 21 Mar 2024 14:25:00 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=QNUqMrSN; dmarc=none; spf=none (imf25.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711031100; a=rsa-sha256; cv=none; b=JcjVw+X+qiZqTs2T3QmyRHb5BDW7X8A63fCdAz/Hr0AE5uHwL5eWxm7mHjx32ihgE/8g9V WuI8iWv04amP6PYIaRrBQkUFQXNCZcJUa12zwxePmk1BY233B2WfrieviRdVjLlzgsDUFp ffBVptlaUg638kRaDf9/mV5u/M87FzY= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=QNUqMrSN; dmarc=none; spf=none (imf25.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711031100; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ke+FbGYMGX/p8C8dwkFxfgnKBjd/N6i/0WYgiRXaFMY=; b=EuGdebkrnIQkeQV1MT2Y9ri82Si/P+WxXSwuxw1pXKjxDn+lTuK0UlvQ3eeKcebPGLaDJn 6REtG+p+SYMEUKDthd+YoafZQXwRIi6fwf4t3xjXOSOPlGmSeVOLttCLZ8KLDzHvuK/09n st4e1AcbC0gCgU9y8FNHg2alBHdvo7w= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Ke+FbGYMGX/p8C8dwkFxfgnKBjd/N6i/0WYgiRXaFMY=; b=QNUqMrSNsBxYOQZgsgGr7hHPiz mWvZc11iIoz3LkXOJOpE5X+uuvnfRJsyVmLB9NPblc/7BRDO9fg/v9wdj3oDt1gXd5GSCTwaO7u6N 2Ld7hTX5KQr07Jgbth3bMFtqciNzM/xsCmiCIaP1DRdMN+Kw7xWFgK2Mlsi6meiVgL52t2sHTHJJE 0AgwGrNLxRSBGXXKs6a1nkXY0uJZDO9zGcIFdSeSSJ4TXXzyX0KSxKWmJbWShuhV9XppI+XO/GtOe EdqG9qXyXPk7xrzs24dN9WVnPJgrcU0Jmj0qQq9eBctEy7SWqOZ2oWX7OL5uPBGii2qCL7i6IC7gS dy85vsIw==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnJLf-00000006u48-3pIq; Thu, 21 Mar 2024 14:24:51 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, David Hildenbrand , Vlastimil Babka , Miaohe Lin , Muchun Song , Oscar Salvador Subject: [PATCH 8/9] mm: Improve dumping of mapcount and page_type Date: Thu, 21 Mar 2024 14:24:46 +0000 Message-ID: <20240321142448.1645400-9-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240321142448.1645400-1-willy@infradead.org> References: <20240321142448.1645400-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: C545EA0019 X-Stat-Signature: pxr3pqk7uxyz4gmp4fbiktmzw3ewq6ne X-HE-Tag: 1711031100-254725 X-HE-Meta: U2FsdGVkX1/IgZxiofyApKk+lBbXRyyhz3Htm6zmbmcFyRrWUC6NZmOcKiOFANWwLz4NhuXdE3N+b5jIrCql1B+LbVQZJXU34zo9J55JHEd3vTbiZAUDtdR3lEj0QdIt6wQdqrRyIbPppW7/hQuri5V/nQwVEQ9We66D385J2NO+priEwSCzl6ZJnXJlLx910Xp+RxfwGMWEfeo/tjVPhVuDYZOUIZ/WceGwNaGqqB0urZoLIY1DD1Fwm8mMLqZGWJdj5Yz3ngOPOTLRKmlfdZchhbmSV9A0QIuMmgj79aJUy9nbxUfe9R3aPHFe41uTxOjkeG1tH/xQYglOFfVzfjGLZj3nvxhmq3XfjU2DTOqp/RQTFUq2WrXuCrDIABAj0CM1b4zJnCIscvTItfpS/uf/dbRQNoPVOF/0scrR8i+5SrM+r1Vgn2xhfNdDwy/yRIM7g58eqKV2mwUuievgHlLbVuq/cVEthbjnASppOSmj7p/X+0USnAcQGXs6uZbFTWGWdscVP0wYObWBq3jDg2CnfjwXYcr9MivhsDIajaUcvr8Xr5kOL4an2UzYGD28guoVKTSjbIm3LbBTPpHW2FyWm+tXr5Fn+cQt5Cw+y3IHogZ1Rt8ahsNaYAHJ+5dyqtgpOZ8gO4xbWSIfvQjze8rnRT7fJeW1W35jTDRR03QnekGlHQqh8v2glokHvirGhXYqIn0aOd/Qtzc7KPWSecUGLKEvfuC2RpoNfhVh1yQ+GWRye3ABuM5+XUIOgV1B4wGp73mTvtYLftxrnNFU8uaAy60ioYY3M+FdmHgmF3D0o59QcoRLSScZRRcTBWunLGOZKjauuhl/zNrUPlRzhiLAjg81LbkZMDtEX2bF2BOP0RpIBkR9DEWgBLCiYb0MHZ28pVis8his0VnyHyIG8joR/6JazNxC8D3jPuB1D8PSMuLbe9jdv3Q2bfos5K4FPRDhOTbGOHMddxREyzb 0OZkjT+R q/3iR3ytlY/w6vaRFtOBZrOlU5RChNFIM1O9n4aFxP5SiG2B/zcV/KHcU9T/2kubYQ6o0XaIozH7x/vqQujrS0p+xMEfB4r4HN03Ig5t9IhWOJdD5rHFH9fbPkC4R3WH9II76NyAHOj/GGtB0hAQ+wUYqrLImUWfU4nH2ZS50NsS2kgursw/m6VmZaDJ8Jez+xWMRXf3t5C5PZsKMX3VNff3UHVFTCbub1usw X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For pages that have a page_type, set the mapcount to 0, which will reduce the confusion in people reading page dumps ("Why does this page have a mapcount of -128?"). Now that hugetlbfs is a page_type, read the entire_mapcount for any large folio; this is fine for all folios as no user reuses the entire_mapcount field. For pages which do not have a page type, do not print it to reduce clutter. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Vlastimil Babka Reviewed-by: David Hildenbrand --- mm/debug.c | 19 ++++++++----------- 1 file changed, 8 insertions(+), 11 deletions(-) diff --git a/mm/debug.c b/mm/debug.c index c1c1a6a484e4..e8a96b8b7197 100644 --- a/mm/debug.c +++ b/mm/debug.c @@ -55,18 +55,14 @@ static void __dump_folio(struct folio *folio, struct page *page, unsigned long pfn, unsigned long idx) { struct address_space *mapping = folio_mapping(folio); - int mapcount = 0; + int mapcount = atomic_read(&page->_mapcount) + 1; char *type = ""; - /* - * page->_mapcount space in struct page is used by slab pages to - * encode own info, and we must avoid calling page_folio() again. - */ - if (!folio_test_slab(folio)) { - mapcount = atomic_read(&page->_mapcount) + 1; - if (folio_test_large(folio)) - mapcount += folio_entire_mapcount(folio); - } + /* Open-code page_mapcount() to avoid looking up a stale folio */ + if (mapcount < 0) + mapcount = 0; + if (folio_test_large(folio)) + mapcount += folio_entire_mapcount(folio); pr_warn("page: refcount:%d mapcount:%d mapping:%p index:%#lx pfn:%#lx\n", folio_ref_count(folio), mapcount, mapping, @@ -99,7 +95,8 @@ static void __dump_folio(struct folio *folio, struct page *page, */ pr_warn("%sflags: %pGp%s\n", type, &folio->flags, is_migrate_cma_folio(folio, pfn) ? " CMA" : ""); - pr_warn("page_type: %pGt\n", &folio->page.page_type); + if (page_has_type(&folio->page)) + pr_warn("page_type: %pGt\n", &folio->page.page_type); print_hex_dump(KERN_WARNING, "raw: ", DUMP_PREFIX_NONE, 32, sizeof(unsigned long), page, From patchwork Thu Mar 21 14:24:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13598772 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 900F7C54E68 for ; Thu, 21 Mar 2024 14:25:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4D3F6B008A; Thu, 21 Mar 2024 10:25:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CD5BC6B008C; Thu, 21 Mar 2024 10:25:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B4FBD6B0092; Thu, 21 Mar 2024 10:25:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A02006B008A for ; Thu, 21 Mar 2024 10:25:01 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id C691C1A0260 for ; Thu, 21 Mar 2024 14:25:00 +0000 (UTC) X-FDA: 81921267960.19.A258214 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf19.hostedemail.com (Postfix) with ESMTP id 863D11A0016 for ; Thu, 21 Mar 2024 14:24:58 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ukRKoFAj; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711031099; a=rsa-sha256; cv=none; b=X6O3WKleNVpjZLK8ONvMqsDbhP+9rdpzEkrjSQeNDosnQjxeBVTe2ratBOXxG6L/a+EARd vgAG1hO8e3+rN2YkMt4U4Z3gTalNrjxrojG9MP0Em4ThzfzIGJmKmt2HihoRYeCR21GSO4 PB4AuyJ2iJovt5yS8Aat3cCEoxPTmV8= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ukRKoFAj; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711031099; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kMnaFf5hqRgc9vfkJ4p3HF/QxzQ4aaa8vBIiVKvak3s=; b=UeuMekWw3mlRtuwtSeJ36u+MVdHWUSaI/a1ExXsZyn35slbGhx1FgvrgPnG25VLt7ODqb1 JKe1K4xfDT7/5u5av7bvJRzTmc/lllBn7ejwYN8BrGapzuvNBSfSRSAhtm0qus1EkTiF97 F9IQSPDu1/Iu0+86LHdfg/K63lDnTGA= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=kMnaFf5hqRgc9vfkJ4p3HF/QxzQ4aaa8vBIiVKvak3s=; b=ukRKoFAjG0tsH+qK3TeHNEtGeG KQw4Djf3QaezfNAHr7FgtNdye0j+Et3V9i/iTrZ6Kql4uf+VYlj0zKFh2EQ/KM27O8dV3FbvGBvr9 e5XMSeBH4EZqyrrXr7td/vs5VO0sSS/0/hPkmZ5+D64DoAQw8/t9VTScDmbHJNYlF5Hzib+WH1luZ 5VzWKzwEgpmdVlL7XgIVEmT/aSIhRSNQX6rnR52CvX3B6Lzu92zSwWW52ydgIG9Z05+n2boCPYoqC NnzNd6NEKBcRngsFwIzGr+2e9uH3zd5r9Wbh60mLVMzg777VyFLe2LsDztusnh85G1qYR8UBYbYpE 4Kx/lB9w==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnJLf-00000006u4A-4938; Thu, 21 Mar 2024 14:24:52 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, David Hildenbrand , Vlastimil Babka , Miaohe Lin , Muchun Song , Oscar Salvador Subject: [PATCH 9/9] hugetlb: Remove mention of destructors Date: Thu, 21 Mar 2024 14:24:47 +0000 Message-ID: <20240321142448.1645400-10-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240321142448.1645400-1-willy@infradead.org> References: <20240321142448.1645400-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 863D11A0016 X-Stat-Signature: cztx8ti3np9mdo9ypg3xc41j8py5tyg7 X-HE-Tag: 1711031098-399038 X-HE-Meta: U2FsdGVkX18aeGGrNS2hQTo8kwVwDIDxa1FM488QiXU2i7J9QDA+0HaK9jWcY9P782qFGE9vu7hQJDaFBgh9YEgm4XJYY09SUwxCV1Tjh4zV0CawbuE7ArCnqlVHz6ftSHvHbXEKUgQaiA404OSIYziH15tTts5E2r5khT6gC2l5EcViJaMpY89sIFxSdS4ZhjEnMC16GinI6rbkQzYOk+VrD8nh+Knz3bSreQlMYjQgtGQo2mL0Dbt5GMLxxLkiB6LV37DytcpD/SCPL6bLWm6t5CyQhkb68E2NowVWhAKjc9hLqOPN0mU1DrwodV4eWsZAOcTyk1i+S25gZWNKPBTvJX496UxNOeY7ccGHwnucm/0HEQF6xe7G2QIC5AwHNsLsRlmrn4E1OOMLmxmDKwsnvJa/Qzg36hrU76Z3RLTyK92T+1hutYpht9cCIre2jJlUkkdmaGeRO/sYlQmZW6v2SZopqoyF76M2iiEn+WuCNr1gcnFaUDN3L2WKkcWf2N45kJBGKWe1CCqipCjBRfOKNJHbXC02BQLeguud0ibSl+9k8Qks9qrSM+dNfzkFyyNLv3hmG6oeRQWBkx+I5+0FvbhztavL+EpfUJvuPEP2O/c0C9I0Ga3jOvnD9C/hywtOYeeF3se7ybWitrfY7iPzNfdhJ7Se5zD7iAJgEmPQi0SO1PW8SIERRXqpCCjehwZPHfV0ZBltNnudksxwe4ZlzOW3dI2OODf6ilDPtqaYeg9JDW7D40RtOe8MLxkz7EVJ3re0gewOi00aVdtV41iYVSo5Rj6/1OwCaKuUVv48kOjlAZSSkyODvvaf023+lqOcg2HdBchaM2k8ZpqPDzhL9VK9fKt2x7TFPMmrMyhQgS2HOynF+3jiEov9XEVHOQVKg18iV4cFzO7V4dLypxuFPTtqO9f1+t2KffWqVxvGn55RR5q+eBHYvk0gvv8yCEeIrABIhBU7LxSzzDT t9kegqtA AuJKBB77u0P7AIp6xBVdVqKkp/PrT03EQ1dYUN2JHSS3TU2MG3BoOEDu83Ee3Fl5RvKELTefxUoDNcSJBQyNaxiu82ZFCW1ZcMU61rylyoBX8U6NH0y/273IYyp/sWy3rBLSme4bUUVvJd4uL4Z7bH8SMGyXPOubmvW7psLep+Ty6bKte98jKDyr6ww== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We no longer have destructors or dtors, merely a page flag (technically a page type flag, but that's an implementation detail). Remove __clear_hugetlb_destructor, fix up comments and the occasional variable name. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Vlastimil Babka Reviewed-by: David Hildenbrand --- mm/hugetlb.c | 42 +++++++++++++++++------------------------- 1 file changed, 17 insertions(+), 25 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index bdcbb62096cf..6ca9ac90ad35 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1619,19 +1619,11 @@ static inline void destroy_compound_gigantic_folio(struct folio *folio, unsigned int order) { } #endif -static inline void __clear_hugetlb_destructor(struct hstate *h, - struct folio *folio) -{ - lockdep_assert_held(&hugetlb_lock); - - __folio_clear_hugetlb(folio); -} - /* * Remove hugetlb folio from lists. - * If vmemmap exists for the folio, update dtor so that the folio appears - * as just a compound page. Otherwise, wait until after allocating vmemmap - * to update dtor. + * If vmemmap exists for the folio, clear the hugetlb flag so that the + * folio appears as just a compound page. Otherwise, wait until after + * allocating vmemmap to clear the flag. * * A reference is held on the folio, except in the case of demote. * @@ -1662,12 +1654,12 @@ static void __remove_hugetlb_folio(struct hstate *h, struct folio *folio, } /* - * We can only clear the hugetlb destructor after allocating vmemmap + * We can only clear the hugetlb flag after allocating vmemmap * pages. Otherwise, someone (memory error handling) may try to write * to tail struct pages. */ if (!folio_test_hugetlb_vmemmap_optimized(folio)) - __clear_hugetlb_destructor(h, folio); + __folio_clear_hugetlb(folio); /* * In the case of demote we do not ref count the page as it will soon @@ -1741,7 +1733,7 @@ static void add_hugetlb_folio(struct hstate *h, struct folio *folio, static void __update_and_free_hugetlb_folio(struct hstate *h, struct folio *folio) { - bool clear_dtor = folio_test_hugetlb_vmemmap_optimized(folio); + bool clear_flag = folio_test_hugetlb_vmemmap_optimized(folio); if (hstate_is_gigantic(h) && !gigantic_page_runtime_supported()) return; @@ -1754,11 +1746,11 @@ static void __update_and_free_hugetlb_folio(struct hstate *h, return; /* - * If folio is not vmemmap optimized (!clear_dtor), then the folio + * If folio is not vmemmap optimized (!clear_flag), then the folio * is no longer identified as a hugetlb page. hugetlb_vmemmap_restore_folio * can only be passed hugetlb pages and will BUG otherwise. */ - if (clear_dtor && hugetlb_vmemmap_restore_folio(h, folio)) { + if (clear_flag && hugetlb_vmemmap_restore_folio(h, folio)) { spin_lock_irq(&hugetlb_lock); /* * If we cannot allocate vmemmap pages, just refuse to free the @@ -1779,11 +1771,11 @@ static void __update_and_free_hugetlb_folio(struct hstate *h, /* * If vmemmap pages were allocated above, then we need to clear the - * hugetlb destructor under the hugetlb lock. + * hugetlb flag under the hugetlb lock. */ - if (clear_dtor) { + if (clear_flag) { spin_lock_irq(&hugetlb_lock); - __clear_hugetlb_destructor(h, folio); + __folio_clear_hugetlb(folio); spin_unlock_irq(&hugetlb_lock); } @@ -1885,7 +1877,7 @@ static void bulk_vmemmap_restore_error(struct hstate *h, list_for_each_entry_safe(folio, t_folio, non_hvo_folios, lru) { list_del(&folio->lru); spin_lock_irq(&hugetlb_lock); - __clear_hugetlb_destructor(h, folio); + __folio_clear_hugetlb(folio); spin_unlock_irq(&hugetlb_lock); update_and_free_hugetlb_folio(h, folio, false); cond_resched(); @@ -1910,7 +1902,7 @@ static void bulk_vmemmap_restore_error(struct hstate *h, } else { list_del(&folio->lru); spin_lock_irq(&hugetlb_lock); - __clear_hugetlb_destructor(h, folio); + __folio_clear_hugetlb(folio); spin_unlock_irq(&hugetlb_lock); update_and_free_hugetlb_folio(h, folio, false); cond_resched(); @@ -1943,14 +1935,14 @@ static void update_and_free_pages_bulk(struct hstate *h, * should only be pages on the non_hvo_folios list. * Do note that the non_hvo_folios list could be empty. * Without HVO enabled, ret will be 0 and there is no need to call - * __clear_hugetlb_destructor as this was done previously. + * __folio_clear_hugetlb as this was done previously. */ VM_WARN_ON(!list_empty(folio_list)); VM_WARN_ON(ret < 0); if (!list_empty(&non_hvo_folios) && ret) { spin_lock_irq(&hugetlb_lock); list_for_each_entry(folio, &non_hvo_folios, lru) - __clear_hugetlb_destructor(h, folio); + __folio_clear_hugetlb(folio); spin_unlock_irq(&hugetlb_lock); } @@ -1975,7 +1967,7 @@ void free_huge_folio(struct folio *folio) { /* * Can't pass hstate in here because it is called from the - * compound page destructor. + * generic mm code. */ struct hstate *h = folio_hstate(folio); int nid = folio_nid(folio); @@ -2125,7 +2117,7 @@ static bool __prep_compound_gigantic_folio(struct folio *folio, set_compound_head(p, &folio->page); } __folio_set_head(folio); - /* we rely on prep_new_hugetlb_folio to set the destructor */ + /* we rely on prep_new_hugetlb_folio to set the hugetlb flag */ folio_set_order(folio, order); atomic_set(&folio->_entire_mapcount, -1); atomic_set(&folio->_nr_pages_mapped, 0);