From patchwork Tue Jul 23 15:34:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13740175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B513DC3DA49 for ; Tue, 23 Jul 2024 15:35:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 26D5A6B00BA; Tue, 23 Jul 2024 11:35:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1F3B56B00BB; Tue, 23 Jul 2024 11:35:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 01E6A6B00BC; Tue, 23 Jul 2024 11:35:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D7E106B00BA for ; Tue, 23 Jul 2024 11:35:15 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 656E91401FE for ; Tue, 23 Jul 2024 15:35:15 +0000 (UTC) X-FDA: 82371416190.19.F8D611E Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf19.hostedemail.com (Postfix) with ESMTP id 82FF71A001D for ; Tue, 23 Jul 2024 15:35:13 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="mgbs/kdS"; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721748861; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ls/nGiqrMN6eSAAmt4FhCK20E6yJjNxuF4jZ+2UM+B0=; b=BghWDtH2S2XMwI9zMq6qmKjK+UL6dSnf2MoKAJ75BsGm7+qxyZJ7swGJnpzkwdOWO2mb4i QRYOU2RlcfW+Es4fA8FTuh5BN8Kqawrfs3H4rCQOEzr47yO8EvvLRxpXqckpckEb12K9ko hJn4HuJhuMsXGqSf/EqXJVAMYQ3EKz0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721748861; a=rsa-sha256; cv=none; b=p8mvwHBv7rxj06C/j2hPWYc6PpkBXbMZe/b4V1fZKCVAeOIseCCFZou6nZJ8gPnflRY/kG 1hQleP+TtkTFncwEmu034cjISnueVHuC6z+f1VSoViUYr6u2v+RGrOSAX2v2v1dEpZjVSY WQ1kS+MqW+vxxQ7yY9cRG9v1vBSYzlQ= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="mgbs/kdS"; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Ls/nGiqrMN6eSAAmt4FhCK20E6yJjNxuF4jZ+2UM+B0=; b=mgbs/kdSkqd6aBDyv5jfK177NU YGpnBG6y917w8L1ue6DG4cwxXIdz+WEqvTmQW/t6akSGIXXW2u1jK13LcLRCcalqK2nlL69GbT1f8 rrK9vogldMYB8quOj84uxm90Yo0EzF6JxwEApuRDNBQAGAF2ozfzSGrCidGWO2ZrlExWTnY/tqR5y T3dwii11efm4uO3c5S3DILvAWwoCChhKpbOXguTXpBiveTGhV3k9+B+wCvfDlSy3510l1X1GHI6NZ GXF3XwalC4l7+YeM8hRJj0uKZllQDVgKRCOuAUmQXbqjq0EaWngF43RYC/JQ6HUqiH7VaL7bTTxEt DzjJ7CiA==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1sWHXd-000000070LG-0Mdl; Tue, 23 Jul 2024 15:35:05 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 1/6] bootmem: Stop using page->index Date: Tue, 23 Jul 2024 16:34:56 +0100 Message-ID: <20240723153503.1669586-2-willy@infradead.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240723153503.1669586-1-willy@infradead.org> References: <20240723153503.1669586-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 82FF71A001D X-Stat-Signature: sfg5gg8x7jwwibfx6ogdwxeog1re4f3x X-Rspam-User: X-HE-Tag: 1721748913-651064 X-HE-Meta: U2FsdGVkX1/fb8o0ethVoW7V5wlgYg+ctWS0lNsz12tVbnpPf5CsVfyUtAamF8kVqgtHUqWL/pH8eUIurJV+UQa3NmbGVZuvDpkD89z0kexmY1bDYQzbOBwq7ouIVlYl0clgQWXwlnYNl8fq+mz7vIlAyp6SgnLyNNE5VQ92YtAIDAwpY5d3Hpmv4pApfHC5lKz9E73hQ7vxlYs2n0hk/XHRJsjT9qSWQ7Gb59xSWXrG5b8aFhkyv2lMNLWRUCjLG78l8jvs+fdhqUkp3LDhQ/oDzMzBAoFZsU6PAfidyggiaDB5u3gjPQxvFSoDsEMGH3tFKTfD3gW8lEZZaUB/wL405G9qVeYoVDnNs3MOvF7Y/aFL4sKI1dTpu8SXbtt9GOdwkHHhcNAQeh2zND3o5AOLNt3yGhDW6OlhrAOhfubQCIPxIAwjURKS8iHyKcdowe9cRGF+4iLREWSVBlrMbihZkzW9McBKPS4Fgm4E71JGDONpY5D0FogmhJNcyErBQ1iUZISUDJc+iUgssAu7Mvle/P7Zk+lG6nPrhSEoKVTd4NKbsfUmDxtEeeeExb+nPWxvW2jJuIvfZOoAL5iq9R4SpfwDo5tUvMk0rKGxlwPZ+NyUvxg28Ek+bLHtt3fCBN98G1NQrir4rs2lju6cLWHJV7jrbbPXmXB6tIoiWbXHmpJb/9Gtdha9m5Kv5Tta+Gt8fKCeA6/CkHNc7zxkdlFvddKIOJKbMQvV0eQtAIBXDq7D557EcXOsXmROgmAZk9DYN0lz1AvOI+NriyMUWwNeEgAE2hEcZbxUWRSurkKJpz2dyTL6hvgi7soBJR6hyF/JoZMUjbIDK2JRN2zmD7aO/quxgZfdDoiFyTpfngoTdId9A8UozGmAklxlsn1H2i8+VoP596Xm+OTM3DvjNX8+SxIDK+JznNEPEin6PUsDUPNyLJajM9Cwo6EGHryHTInZJlPVeGq4dKraj1P CTGRIgPy vrrxDrkVGvp9zWyg+lblwb1U5pbTB5TMNl0stbDL/w8Rw5F5XZYuS+iwDnZqkIFI6tWUoD01+zoFakij7AdCgIcyS0eqswFCgIIky8sLDfvv7RvdWwxt+CEvbdARcsKKpUEhEAsOiHgKprKPU6jPU7zMNfIymuFhdHryC5CMB8ho3bvopsYrxrpLpP+ikETUze662REXiwExAep6N3JlPfNhPcA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Encode the type into the bottom four bits of page->private and the info into the remaining bits. Also turn the bootmem type into a named enum. Signed-off-by: Matthew Wilcox (Oracle) --- arch/x86/mm/init_64.c | 9 ++++----- include/linux/bootmem_info.h | 25 +++++++++++++++++-------- mm/bootmem_info.c | 11 ++++++----- mm/sparse.c | 8 ++++---- 4 files changed, 31 insertions(+), 22 deletions(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index d8dbeac8b206..d77f22850aa2 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -983,13 +983,12 @@ int arch_add_memory(int nid, u64 start, u64 size, static void __meminit free_pagetable(struct page *page, int order) { - unsigned long magic; - unsigned int nr_pages = 1 << order; - /* bootmem page has reserved flag */ if (PageReserved(page)) { - magic = page->index; - if (magic == SECTION_INFO || magic == MIX_SECTION_INFO) { + enum bootmem_type type = bootmem_type(page); + unsigned long nr_pages = 1 << order; + + if (type == SECTION_INFO || type == MIX_SECTION_INFO) { while (nr_pages--) put_page_bootmem(page++); } else diff --git a/include/linux/bootmem_info.h b/include/linux/bootmem_info.h index cffa38a73618..e2fe5de93dcc 100644 --- a/include/linux/bootmem_info.h +++ b/include/linux/bootmem_info.h @@ -6,11 +6,10 @@ #include /* - * Types for free bootmem stored in page->lru.next. These have to be in - * some random range in unsigned long space for debugging purposes. + * Types for free bootmem stored in the low bits of page->private. */ -enum { - MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE = 12, +enum bootmem_type { + MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE = 1, SECTION_INFO = MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE, MIX_SECTION_INFO, NODE_INFO, @@ -21,9 +20,19 @@ enum { void __init register_page_bootmem_info_node(struct pglist_data *pgdat); void get_page_bootmem(unsigned long info, struct page *page, - unsigned long type); + enum bootmem_type type); void put_page_bootmem(struct page *page); +static inline enum bootmem_type bootmem_type(const struct page *page) +{ + return (unsigned long)page->private & 0xf; +} + +static inline unsigned long bootmem_info(const struct page *page) +{ + return (unsigned long)page->private >> 4; +} + /* * Any memory allocated via the memblock allocator and not via the * buddy will be marked reserved already in the memmap. For those @@ -31,7 +40,7 @@ void put_page_bootmem(struct page *page); */ static inline void free_bootmem_page(struct page *page) { - unsigned long magic = page->index; + enum bootmem_type type = bootmem_type(page); /* * The reserve_bootmem_region sets the reserved flag on bootmem @@ -39,7 +48,7 @@ static inline void free_bootmem_page(struct page *page) */ VM_BUG_ON_PAGE(page_ref_count(page) != 2, page); - if (magic == SECTION_INFO || magic == MIX_SECTION_INFO) + if (type == SECTION_INFO || type == MIX_SECTION_INFO) put_page_bootmem(page); else VM_BUG_ON_PAGE(1, page); @@ -54,7 +63,7 @@ static inline void put_page_bootmem(struct page *page) } static inline void get_page_bootmem(unsigned long info, struct page *page, - unsigned long type) + enum bootmem_type type) { } diff --git a/mm/bootmem_info.c b/mm/bootmem_info.c index fa7cb0c87c03..95f288169a38 100644 --- a/mm/bootmem_info.c +++ b/mm/bootmem_info.c @@ -14,23 +14,24 @@ #include #include -void get_page_bootmem(unsigned long info, struct page *page, unsigned long type) +void get_page_bootmem(unsigned long info, struct page *page, + enum bootmem_type type) { - page->index = type; + BUG_ON(type > 0xf); + BUG_ON(info > (ULONG_MAX >> 4)); SetPagePrivate(page); - set_page_private(page, info); + set_page_private(page, info << 4 | type); page_ref_inc(page); } void put_page_bootmem(struct page *page) { - unsigned long type = page->index; + enum bootmem_type type = bootmem_type(page); BUG_ON(type < MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE || type > MEMORY_HOTPLUG_MAX_BOOTMEM_TYPE); if (page_ref_dec_return(page) == 1) { - page->index = 0; ClearPagePrivate(page); set_page_private(page, 0); INIT_LIST_HEAD(&page->lru); diff --git a/mm/sparse.c b/mm/sparse.c index e4b830091d13..ad89ce5d9d28 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -721,19 +721,19 @@ static void depopulate_section_memmap(unsigned long pfn, unsigned long nr_pages, static void free_map_bootmem(struct page *memmap) { unsigned long maps_section_nr, removing_section_nr, i; - unsigned long magic, nr_pages; + unsigned long type, nr_pages; struct page *page = virt_to_page(memmap); nr_pages = PAGE_ALIGN(PAGES_PER_SECTION * sizeof(struct page)) >> PAGE_SHIFT; for (i = 0; i < nr_pages; i++, page++) { - magic = page->index; + type = bootmem_type(page); - BUG_ON(magic == NODE_INFO); + BUG_ON(type == NODE_INFO); maps_section_nr = pfn_to_section_nr(page_to_pfn(page)); - removing_section_nr = page_private(page); + removing_section_nr = bootmem_info(page); /* * When this function is called, the removing section is