From patchwork Thu Sep 12 18:55:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13802489 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C71B4EED63A for ; Thu, 12 Sep 2024 18:56:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E05756B0092; Thu, 12 Sep 2024 14:56:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DB62C6B0093; Thu, 12 Sep 2024 14:56:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C7CA46B0095; Thu, 12 Sep 2024 14:56:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id ABFFB6B0092 for ; Thu, 12 Sep 2024 14:56:10 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id EF478A018B for ; Thu, 12 Sep 2024 18:56:09 +0000 (UTC) X-FDA: 82556991258.30.584D50A Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf11.hostedemail.com (Postfix) with ESMTP id D6AE840006 for ; Thu, 12 Sep 2024 18:56:06 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=C1R1Ld3V; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726167227; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=qAUhcutSpNzHMfGoUnNFYBLH5l/c9SO2zzfu+lWTDuI=; b=v40j287Hh4f0Dx6dmHI13GQbHNEc9qRRtwquHJQwJQkX9CxPx/QcSHOHVwyG+16CruTrxA zEjdmiDX4MF2cajQ8TRGfRP52iHHzg1tJ6/6UmYXT86ejnhI3rmYQWdAfchCvL1opfBLs0 qD8jTkY5xMrm0R+NL8gv1QcFURDWXXY= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=C1R1Ld3V; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726167227; a=rsa-sha256; cv=none; b=OUux6J/8pGrJlOEzUCApiwmOWPT8spwl6FREaGhPKCtXZvnubkVv0HkLbNrmbbaJjYTABa Zx6ianUcv/k+CXUvkqvEvFbkomYRIA7rqt07+WkD7UeobK7Blyuq6vIhfnhSWmSwcv4msl x/ElCQZXMf8in1a6Vz7nIeCYxsfST4I= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:In-Reply-To:References; bh=qAUhcutSpNzHMfGoUnNFYBLH5l/c9SO2zzfu+lWTDuI=; b=C1R1Ld3VYh9GSDxuU899rofH+0 tu56EkTWq6Vj5mhH5evor4xYbDy+xQeZu3LudjISaW5EO7VaD5zZZBEBRrEy/Fmx1B/CJPo+SppvD cfoRssxTeEWJNqgkSZPVg008n/WQxpn5V3Ztdw+IbwGwP0KhWk7R4CBCtZU19qae7rOtgQkAfgUY9 nMMSAuXnHjjmsrrvtUcHX7JBUCOgpaxv5VqPOZZSvqgTjR00rAvSQOfhWWXoE0pWi2LDqeQmL+o0Q f8VQmCyP636+64jtq4GiPEs24J3g7IXWAYr1BdetaVbmS+4HI7hrJo+Q78XHRqfD27UWizAmFqna1 nArsBYUQ==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1sooz6-00000009pJ4-1zR7; Thu, 12 Sep 2024 18:56:04 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , x86@kernel.org, Mike Rapoport Subject: [PATCH] bootmem: Stop using page->index Date: Thu, 12 Sep 2024 19:55:59 +0100 Message-ID: <20240912185602.2342148-1-willy@infradead.org> X-Mailer: git-send-email 2.46.0 MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: D6AE840006 X-Stat-Signature: qq6onuzd39rsmyczjcxe8xb7jq5x91ye X-Rspam-User: X-HE-Tag: 1726167366-574654 X-HE-Meta: U2FsdGVkX18m4df8KO6tfPfue28p5ddzh3CBpOcAJxKAAp7RF790pX5nFuWrsqPJXC067MK0Lm0Q8JVeAxqVT63iaNvh88KK6qY0YcSS7sAIZzgZKvoMzov1ZRZX+kDJSOKMGYBBxeKW87ui7OYXB4Lcr/QqcI16OxPHk+FmwtGAwfAinLbdAoU0ETxlSXbNegy0p2+zOppLJH9NA+IrvL5GaX0iP4e2jPl2mJUWTetpP/SipPoDLDZ1Q5+cBpjC2eOV5kY3E19A+KVDywjZwBzvvOWRAWd5mKu8M5HMHipCmDoYo7BtpVR0Eb+nSy/LBN+mv9d8yyPZoDRP4lJJoW1K6bFSbOJGIhBXkBmovFL3GQdz9iOxqW9/WL6ONRMQb6xc6IwF9HYPwGhLxgd8C4019Ai1Z2MtDH+IPliisnnaYzycUJ8enqZN+FTORZLUiXl/GMoXp/OI9EdbH/cj0r+J39ytZC67MTwir6DlVOK+xSTmBmw38mDJ1rT8r0UStF3UtHDrwcqxK4h8A8L8DHdWxaEBN5pIMSJNA67Bi4KGPF5qiMqeH2cLJXuXhDI7P1jGnohVIvAbCpzFCRpK5PznHmAOjXLlyQcK/MDrGt5pTUYdHF92nrGseNFfD/wRnOmFaOPJI1Lnw6lstZSzNBXnLIAZ0z9PrCYoUMbUw6rjTqODrz668uWCA+v2nLrw723d8Ge7XUkqXZIU8mpHh4TcMRR5P8mSSrtUOMFY/coDWnNqo4u6KH0Hnx0pUrZUMghjKbESmxK3Kyi3He/3sshtIxz0BElGMZkqPpCxtFlVUFDRcLuCg+7uadSWDnm800XjL9UZhucWJOiA6SwNVmwIZF4Lb9Vecp28ribFLBQ9zjI8RhEELhhsEir0b65unDiseen/E0EWpmC7nqg9CvdFJ0f6bLjnB7LBNVCpJQSPwsCVr+YVao/EeJ6mXFHkc6mQOo23Cjmd/wTJuqC aHzkTYe1 OIoOb8KE3xdep4ByWfkrLlxlc16Kl98tU/6a9rZ/PgQ9Bas94IOqDZW8itxWAIHAlQOumyaMP2h5GQZrhSKmBlJvVgySR5L1p1QPCipWeKtecCpUcY8rbnILMoSh2tah5o+Ho8Xebsd4e6TjNLF/MXXL4rQxAkPWqvyFqVTfMiE5qTWZApaMJ134/fA7Sl5yW2EstGDuIwOuYsb0GO9lfHRWjvsOhI/kPg686rU2vl1AnywU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Encode the type into the bottom four bits of page->private and the info into the remaining bits. Also turn the bootmem type into a named enum. Signed-off-by: Matthew Wilcox (Oracle) --- arch/x86/mm/init_64.c | 9 ++++----- include/linux/bootmem_info.h | 25 +++++++++++++++++-------- mm/bootmem_info.c | 11 ++++++----- mm/sparse.c | 8 ++++---- 4 files changed, 31 insertions(+), 22 deletions(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index ff253648706f..4d5fde324136 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -987,13 +987,12 @@ int arch_add_memory(int nid, u64 start, u64 size, static void __meminit free_pagetable(struct page *page, int order) { - unsigned long magic; - unsigned int nr_pages = 1 << order; - /* bootmem page has reserved flag */ if (PageReserved(page)) { - magic = page->index; - if (magic == SECTION_INFO || magic == MIX_SECTION_INFO) { + enum bootmem_type type = bootmem_type(page); + unsigned long nr_pages = 1 << order; + + if (type == SECTION_INFO || type == MIX_SECTION_INFO) { while (nr_pages--) put_page_bootmem(page++); } else diff --git a/include/linux/bootmem_info.h b/include/linux/bootmem_info.h index cffa38a73618..e2fe5de93dcc 100644 --- a/include/linux/bootmem_info.h +++ b/include/linux/bootmem_info.h @@ -6,11 +6,10 @@ #include /* - * Types for free bootmem stored in page->lru.next. These have to be in - * some random range in unsigned long space for debugging purposes. + * Types for free bootmem stored in the low bits of page->private. */ -enum { - MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE = 12, +enum bootmem_type { + MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE = 1, SECTION_INFO = MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE, MIX_SECTION_INFO, NODE_INFO, @@ -21,9 +20,19 @@ enum { void __init register_page_bootmem_info_node(struct pglist_data *pgdat); void get_page_bootmem(unsigned long info, struct page *page, - unsigned long type); + enum bootmem_type type); void put_page_bootmem(struct page *page); +static inline enum bootmem_type bootmem_type(const struct page *page) +{ + return (unsigned long)page->private & 0xf; +} + +static inline unsigned long bootmem_info(const struct page *page) +{ + return (unsigned long)page->private >> 4; +} + /* * Any memory allocated via the memblock allocator and not via the * buddy will be marked reserved already in the memmap. For those @@ -31,7 +40,7 @@ void put_page_bootmem(struct page *page); */ static inline void free_bootmem_page(struct page *page) { - unsigned long magic = page->index; + enum bootmem_type type = bootmem_type(page); /* * The reserve_bootmem_region sets the reserved flag on bootmem @@ -39,7 +48,7 @@ static inline void free_bootmem_page(struct page *page) */ VM_BUG_ON_PAGE(page_ref_count(page) != 2, page); - if (magic == SECTION_INFO || magic == MIX_SECTION_INFO) + if (type == SECTION_INFO || type == MIX_SECTION_INFO) put_page_bootmem(page); else VM_BUG_ON_PAGE(1, page); @@ -54,7 +63,7 @@ static inline void put_page_bootmem(struct page *page) } static inline void get_page_bootmem(unsigned long info, struct page *page, - unsigned long type) + enum bootmem_type type) { } diff --git a/mm/bootmem_info.c b/mm/bootmem_info.c index fa7cb0c87c03..95f288169a38 100644 --- a/mm/bootmem_info.c +++ b/mm/bootmem_info.c @@ -14,23 +14,24 @@ #include #include -void get_page_bootmem(unsigned long info, struct page *page, unsigned long type) +void get_page_bootmem(unsigned long info, struct page *page, + enum bootmem_type type) { - page->index = type; + BUG_ON(type > 0xf); + BUG_ON(info > (ULONG_MAX >> 4)); SetPagePrivate(page); - set_page_private(page, info); + set_page_private(page, info << 4 | type); page_ref_inc(page); } void put_page_bootmem(struct page *page) { - unsigned long type = page->index; + enum bootmem_type type = bootmem_type(page); BUG_ON(type < MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE || type > MEMORY_HOTPLUG_MAX_BOOTMEM_TYPE); if (page_ref_dec_return(page) == 1) { - page->index = 0; ClearPagePrivate(page); set_page_private(page, 0); INIT_LIST_HEAD(&page->lru); diff --git a/mm/sparse.c b/mm/sparse.c index dc38539f8560..6ba5354cf2e1 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -720,19 +720,19 @@ static void depopulate_section_memmap(unsigned long pfn, unsigned long nr_pages, static void free_map_bootmem(struct page *memmap) { unsigned long maps_section_nr, removing_section_nr, i; - unsigned long magic, nr_pages; + unsigned long type, nr_pages; struct page *page = virt_to_page(memmap); nr_pages = PAGE_ALIGN(PAGES_PER_SECTION * sizeof(struct page)) >> PAGE_SHIFT; for (i = 0; i < nr_pages; i++, page++) { - magic = page->index; + type = bootmem_type(page); - BUG_ON(magic == NODE_INFO); + BUG_ON(type == NODE_INFO); maps_section_nr = pfn_to_section_nr(page_to_pfn(page)); - removing_section_nr = page_private(page); + removing_section_nr = bootmem_info(page); /* * When this function is called, the removing section is