From patchwork Wed Dec 9 21:43:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 11962709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8405C4361B for ; Wed, 9 Dec 2020 21:43:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6A28123C91 for ; Wed, 9 Dec 2020 21:43:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6A28123C91 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0FBEF8D0031; Wed, 9 Dec 2020 16:43:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 036066B013A; Wed, 9 Dec 2020 16:43:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E66298D0031; Wed, 9 Dec 2020 16:43:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0135.hostedemail.com [216.40.44.135]) by kanga.kvack.org (Postfix) with ESMTP id C35526B0139 for ; Wed, 9 Dec 2020 16:43:17 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 83DFE362D for ; Wed, 9 Dec 2020 21:43:17 +0000 (UTC) X-FDA: 77575070034.25.use30_4703e18273f3 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin25.hostedemail.com (Postfix) with ESMTP id 702721804E3A0 for ; Wed, 9 Dec 2020 21:43:17 +0000 (UTC) X-HE-Tag: use30_4703e18273f3 X-Filterd-Recvd-Size: 4472 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Wed, 9 Dec 2020 21:43:16 +0000 (UTC) From: Mike Rapoport Authentication-Results: mail.kernel.org; dkim=permerror (bad message/signature format) To: Andrew Morton Cc: Andrea Arcangeli , Baoquan He , David Hildenbrand , Mel Gorman , Michal Hocko , Mike Rapoport , Mike Rapoport , Qian Cai , Vlastimil Babka , linux-kernel@vger.kernel.org, linux-mm@kvack.org, stable@vger.kernel.org Subject: [PATCH v2 1/2] mm: memblock: enforce overlap of memory.memblock and memory.reserved Date: Wed, 9 Dec 2020 23:43:03 +0200 Message-Id: <20201209214304.6812-2-rppt@kernel.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201209214304.6812-1-rppt@kernel.org> References: <20201209214304.6812-1-rppt@kernel.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Rapoport memblock does not require that the reserved memory ranges will be a subset of memblock.memory. As the result there maybe reserved pages that are not in the range of any zone or node because zone and node boundaries are detected based on memblock.memory and pages that only present in memblock.reserved are not taken into account during zone/node size detection. Make sure that all ranges in memblock.reserved are added to memblock.memory before calculating node and zone boundaries. Fixes: 73a6e474cb37 ("mm: memmap_init: iterate over memblock regions rather that check each PFN") Reported-by: Andrea Arcangeli Signed-off-by: Mike Rapoport --- include/linux/memblock.h | 1 + mm/memblock.c | 24 ++++++++++++++++++++++++ mm/page_alloc.c | 7 +++++++ 3 files changed, 32 insertions(+) diff --git a/include/linux/memblock.h b/include/linux/memblock.h index ef131255cedc..e64dae2dd1ce 100644 --- a/include/linux/memblock.h +++ b/include/linux/memblock.h @@ -120,6 +120,7 @@ int memblock_clear_nomap(phys_addr_t base, phys_addr_t size); unsigned long memblock_free_all(void); void reset_node_managed_pages(pg_data_t *pgdat); void reset_all_zones_managed_pages(void); +void memblock_enforce_memory_reserved_overlap(void); /* Low level functions */ void __next_mem_range(u64 *idx, int nid, enum memblock_flags flags, diff --git a/mm/memblock.c b/mm/memblock.c index b68ee86788af..9277aca642b2 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -1857,6 +1857,30 @@ void __init_memblock memblock_trim_memory(phys_addr_t align) } } +/** + * memblock_enforce_memory_reserved_overlap - make sure every range in + * @memblock.reserved is covered by @memblock.memory + * + * The data in @memblock.memory is used to detect zone and node boundaries + * during initialization of the memory map and the page allocator. Make + * sure that every memory range present in @memblock.reserved is also added + * to @memblock.memory even if the architecture specific memory + * initialization failed to do so + */ +void __init memblock_enforce_memory_reserved_overlap(void) +{ + phys_addr_t start, end; + int nid; + u64 i; + + __for_each_mem_range(i, &memblock.reserved, &memblock.memory, + NUMA_NO_NODE, MEMBLOCK_NONE, &start, &end, &nid) { + pr_warn("memblock: reserved range [%pa-%pa] is not in memory\n", + &start, &end); + memblock_add_node(start, (end - start), nid); + } +} + void __init_memblock memblock_set_current_limit(phys_addr_t limit) { memblock.current_limit = limit; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index eaa227a479e4..dbc57dbbacd8 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -7436,6 +7436,13 @@ void __init free_area_init(unsigned long *max_zone_pfn) memset(arch_zone_highest_possible_pfn, 0, sizeof(arch_zone_highest_possible_pfn)); + /* + * Some architectures (e.g. x86) have reserved pages outside of + * memblock.memory. Make sure these pages are taken into account + * when detecting zone and node boundaries + */ + memblock_enforce_memory_reserved_overlap(); + start_pfn = find_min_pfn_with_active_regions(); descending = arch_has_descending_max_zone_pfns(); From patchwork Wed Dec 9 21:43:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 11962711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B015C433FE for ; Wed, 9 Dec 2020 21:43:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7CC8A23CD1 for ; Wed, 9 Dec 2020 21:43:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7CC8A23CD1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0AB128D0059; Wed, 9 Dec 2020 16:43:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 050DF6B013A; Wed, 9 Dec 2020 16:43:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E19B68D0059; Wed, 9 Dec 2020 16:43:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0127.hostedemail.com [216.40.44.127]) by kanga.kvack.org (Postfix) with ESMTP id C39B96B0139 for ; Wed, 9 Dec 2020 16:43:21 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 7CF3E1EE6 for ; Wed, 9 Dec 2020 21:43:21 +0000 (UTC) X-FDA: 77575070202.22.shade99_0410d44273f3 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id 5621C18038E60 for ; Wed, 9 Dec 2020 21:43:21 +0000 (UTC) X-HE-Tag: shade99_0410d44273f3 X-Filterd-Recvd-Size: 9208 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Wed, 9 Dec 2020 21:43:20 +0000 (UTC) From: Mike Rapoport Authentication-Results: mail.kernel.org; dkim=permerror (bad message/signature format) To: Andrew Morton Cc: Andrea Arcangeli , Baoquan He , David Hildenbrand , Mel Gorman , Michal Hocko , Mike Rapoport , Mike Rapoport , Qian Cai , Vlastimil Babka , linux-kernel@vger.kernel.org, linux-mm@kvack.org, stable@vger.kernel.org Subject: [PATCH v2 2/2] mm: fix initialization of struct page for holes in memory layout Date: Wed, 9 Dec 2020 23:43:04 +0200 Message-Id: <20201209214304.6812-3-rppt@kernel.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201209214304.6812-1-rppt@kernel.org> References: <20201209214304.6812-1-rppt@kernel.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Rapoport There could be struct pages that are not backed by actual physical memory. This can happen when the actual memory bank is not a multiple of SECTION_SIZE or when an architecture does not register memory holes reserved by the firmware as memblock.memory. Such pages are currently initialized using init_unavailable_mem() function that iterated through PFNs in holes in memblock.memory and if there is a struct page corresponding to a PFN, the fields if this page are set to default values and it is marked as Reserved. init_unavailable_mem() does not take into account zone and node the page belongs to and sets both zone and node links in struct page to zero. On a system that has firmware reserved holes in a zone above ZONE_DMA, for instance in a configuration below: # grep -A1 E820 /proc/iomem 7a17b000-7a216fff : Unknown E820 type 7a217000-7bffffff : System RAM unset zone link in struct page will trigger VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page); because there are pages in both ZONE_DMA32 and ZONE_DMA (unset zone link in struct page) in the same pageblock. Interleave initialization of pages that correspond to holes with the initialization of memory map, so that zone and node information will be properly set on such pages. Fixes: 73a6e474cb37 ("mm: memmap_init: iterate over memblock regions rather that check each PFN") Reported-by: Andrea Arcangeli Signed-off-by: Mike Rapoport --- mm/page_alloc.c | 152 +++++++++++++++++++++--------------------------- 1 file changed, 65 insertions(+), 87 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index dbc57dbbacd8..ea5aefef0004 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6185,24 +6185,85 @@ static void __meminit zone_init_free_lists(struct zone *zone) } } -void __meminit __weak memmap_init(unsigned long size, int nid, - unsigned long zone, - unsigned long range_start_pfn) +#if !defined(CONFIG_FLAT_NODE_MEM_MAP) +/* + * Only struct pages that are backed by physical memory available to the + * kernel are zeroed and initialized by memmap_init_zone(). + * But, there are some struct pages that are either reserved by firmware or + * do not correspond to physical page frames becuase the actual memory bank + * is not a multiple of SECTION_SIZE. + * Fields of those struct pages may be accessed (for example page_to_pfn() + * on some configuration accesses page flags) so we must explicitly + * initialize those struct pages. + */ +static u64 __init init_unavailable_range(unsigned long spfn, unsigned long epfn, + int zone, int node) { - unsigned long start_pfn, end_pfn; + unsigned long pfn; + u64 pgcnt = 0; + + for (pfn = spfn; pfn < epfn; pfn++) { + if (!pfn_valid(ALIGN_DOWN(pfn, pageblock_nr_pages))) { + pfn = ALIGN_DOWN(pfn, pageblock_nr_pages) + + pageblock_nr_pages - 1; + continue; + } + __init_single_page(pfn_to_page(pfn), pfn, zone, node); + __SetPageReserved(pfn_to_page(pfn)); + pgcnt++; + } + + return pgcnt; +} +#else +static inline u64 init_unavailable_range(unsigned long spfn, unsigned long epfn, + int zone, int node) +{ + return 0; +} +#endif + +void __init __weak memmap_init(unsigned long size, int nid, + unsigned long zone, + unsigned long range_start_pfn) +{ + unsigned long start_pfn, end_pfn, hole_start_pfn = 0; unsigned long range_end_pfn = range_start_pfn + size; + u64 pgcnt = 0; int i; for_each_mem_pfn_range(i, nid, &start_pfn, &end_pfn, NULL) { start_pfn = clamp(start_pfn, range_start_pfn, range_end_pfn); end_pfn = clamp(end_pfn, range_start_pfn, range_end_pfn); + hole_start_pfn = clamp(hole_start_pfn, range_start_pfn, + range_end_pfn); if (end_pfn > start_pfn) { size = end_pfn - start_pfn; memmap_init_zone(size, nid, zone, start_pfn, MEMINIT_EARLY, NULL, MIGRATE_MOVABLE); } + + if (hole_start_pfn < start_pfn) + pgcnt += init_unavailable_range(hole_start_pfn, + start_pfn, zone, nid); + hole_start_pfn = end_pfn; } + + /* + * Early sections always have a fully populated memmap for the whole + * section - see pfn_valid(). If the last section has holes at the + * end and that section is marked "online", the memmap will be + * considered initialized. Make sure that memmap has a well defined + * state. + */ + if (hole_start_pfn < range_end_pfn) + pgcnt += init_unavailable_range(hole_start_pfn, range_end_pfn, + zone, nid); + + if (pgcnt) + pr_info("%s: Zeroed struct page in unavailable ranges: %lld\n", + zone_names[zone], pgcnt); } static int zone_batchsize(struct zone *zone) @@ -6995,88 +7056,6 @@ void __init free_area_init_memoryless_node(int nid) free_area_init_node(nid); } -#if !defined(CONFIG_FLAT_NODE_MEM_MAP) -/* - * Initialize all valid struct pages in the range [spfn, epfn) and mark them - * PageReserved(). Return the number of struct pages that were initialized. - */ -static u64 __init init_unavailable_range(unsigned long spfn, unsigned long epfn) -{ - unsigned long pfn; - u64 pgcnt = 0; - - for (pfn = spfn; pfn < epfn; pfn++) { - if (!pfn_valid(ALIGN_DOWN(pfn, pageblock_nr_pages))) { - pfn = ALIGN_DOWN(pfn, pageblock_nr_pages) - + pageblock_nr_pages - 1; - continue; - } - /* - * Use a fake node/zone (0) for now. Some of these pages - * (in memblock.reserved but not in memblock.memory) will - * get re-initialized via reserve_bootmem_region() later. - */ - __init_single_page(pfn_to_page(pfn), pfn, 0, 0); - __SetPageReserved(pfn_to_page(pfn)); - pgcnt++; - } - - return pgcnt; -} - -/* - * Only struct pages that are backed by physical memory are zeroed and - * initialized by going through __init_single_page(). But, there are some - * struct pages which are reserved in memblock allocator and their fields - * may be accessed (for example page_to_pfn() on some configuration accesses - * flags). We must explicitly initialize those struct pages. - * - * This function also addresses a similar issue where struct pages are left - * uninitialized because the physical address range is not covered by - * memblock.memory or memblock.reserved. That could happen when memblock - * layout is manually configured via memmap=, or when the highest physical - * address (max_pfn) does not end on a section boundary. - */ -static void __init init_unavailable_mem(void) -{ - phys_addr_t start, end; - u64 i, pgcnt; - phys_addr_t next = 0; - - /* - * Loop through unavailable ranges not covered by memblock.memory. - */ - pgcnt = 0; - for_each_mem_range(i, &start, &end) { - if (next < start) - pgcnt += init_unavailable_range(PFN_DOWN(next), - PFN_UP(start)); - next = end; - } - - /* - * Early sections always have a fully populated memmap for the whole - * section - see pfn_valid(). If the last section has holes at the - * end and that section is marked "online", the memmap will be - * considered initialized. Make sure that memmap has a well defined - * state. - */ - pgcnt += init_unavailable_range(PFN_DOWN(next), - round_up(max_pfn, PAGES_PER_SECTION)); - - /* - * Struct pages that do not have backing memory. This could be because - * firmware is using some of this memory, or for some other reasons. - */ - if (pgcnt) - pr_info("Zeroed struct page in unavailable ranges: %lld pages", pgcnt); -} -#else -static inline void __init init_unavailable_mem(void) -{ -} -#endif /* !CONFIG_FLAT_NODE_MEM_MAP */ - #if MAX_NUMNODES > 1 /* * Figure out the number of possible node ids. @@ -7507,7 +7486,6 @@ void __init free_area_init(unsigned long *max_zone_pfn) /* Initialise every node */ mminit_verify_pageflags_layout(); setup_nr_node_ids(); - init_unavailable_mem(); for_each_online_node(nid) { pg_data_t *pgdat = NODE_DATA(nid); free_area_init_node(nid);