From patchwork Thu Aug 1 06:08:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 13749638 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8494DC3DA4A for ; Thu, 1 Aug 2024 06:10:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 107116B00A1; Thu, 1 Aug 2024 02:10:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0B8E16B00A2; Thu, 1 Aug 2024 02:10:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E75326B00A3; Thu, 1 Aug 2024 02:10:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id CB0296B00A1 for ; Thu, 1 Aug 2024 02:10:35 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 4FD90A08C9 for ; Thu, 1 Aug 2024 06:10:35 +0000 (UTC) X-FDA: 82402652430.29.C80B075 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf11.hostedemail.com (Postfix) with ESMTP id A276140003 for ; Thu, 1 Aug 2024 06:10:33 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=lobL3BnM; spf=pass (imf11.hostedemail.com: domain of rppt@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722492558; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3BWryMfEZo/KzQD2cwpXs2RdGDzCNUDXDq0IaVP7hD0=; b=JACBUlDCjPU0BQ65eah5tjIM58nwZfCx6FYNLJNl8RBENhKCxG+zaSGBQI4SHsAWBIFFik St316LFrJXNt2VhatuzVY68FLBoGeIb0O1xhOsREZZ8/LS5wuJ1aAHL0R1CC8SwRtSGClB 1G904wBC+e5fjTgroy1hSb8HKYu3i3c= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=lobL3BnM; spf=pass (imf11.hostedemail.com: domain of rppt@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722492558; a=rsa-sha256; cv=none; b=VxZSBf/3Kqe8otVvBrxhkexEV9cQn1GjG7ggR/OGP7C443ghpsSGo47MMu5P2u3Qko352j +W9L0hImR4+A1zrVVHrONUwzaiv7cln0+mYFh8/BGG07BR0YqnK1Pc5zPOR4DYqYpggAzt bF/GeTqu53ipjTrp9F8xBwlkKsGnY00= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id CB96A62502; Thu, 1 Aug 2024 06:10:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4287CC4AF0D; Thu, 1 Aug 2024 06:10:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722492632; bh=M/jYHd7+70Yn3Ls9O55R+dDWEJ+Dj73xIHNTwCldMLA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lobL3BnM6c0xUSDE/gmQjqGCSw8foPLZVeo/NqQGptvjCBs0CojEBbM5yOHo2j+aQ C280b1MW1QsteD4Ix5pl+2lhaQ+XLtVM08fbLP1K+hBhpRRq6swIJG6YHeNhAjdnms ch7cxpXn3X3djchL2x0KfD+Bf2Si42dLA9NqaBmFIs3VhvBo8ZVTa0kw/jUKs+fc8t rpigYlgV6IJUNN4UjhtvhLvSASsiID2KsgGPMx8bhBhSDRmJvfFpSKmsLkKRzZLNsu ZdaxIPHAXeoK8nW7bGDairHDEi/zUsyBohOU7c6IkMNBpuGosnlFSdRH4lwb/W2FNs wJ6HDjot6JSGA== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Alexander Gordeev , Andreas Larsson , Andrew Morton , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Christophe Leroy , Dan Williams , Dave Hansen , David Hildenbrand , "David S. Miller" , Davidlohr Bueso , Greg Kroah-Hartman , Heiko Carstens , Huacai Chen , Ingo Molnar , Jiaxun Yang , John Paul Adrian Glaubitz , Jonathan Cameron , Jonathan Corbet , Michael Ellerman , Mike Rapoport , Palmer Dabbelt , "Rafael J. Wysocki" , Rob Herring , Samuel Holland , Thomas Bogendoerfer , Thomas Gleixner , Vasily Gorbik , Will Deacon , Zi Yan , devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-cxl@vger.kernel.org, linux-doc@vger.kernel.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, nvdimm@lists.linux.dev, sparclinux@vger.kernel.org, x86@kernel.org, Jonathan Cameron Subject: [PATCH v3 09/26] arch, mm: pull out allocation of NODE_DATA to generic code Date: Thu, 1 Aug 2024 09:08:09 +0300 Message-ID: <20240801060826.559858-10-rppt@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240801060826.559858-1-rppt@kernel.org> References: <20240801060826.559858-1-rppt@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: A276140003 X-Stat-Signature: ex351bmw85w6d6te88gisjeziipttspd X-Rspam-User: X-HE-Tag: 1722492633-310455 X-HE-Meta: U2FsdGVkX18J9taBwbMpddIY+Sl21fF8ppXiAc0Pm00e2jH75KZKCNUjSQ6T8Ui8JKydNjIF27JjUQKlBTxOizGmud3rS9FPbBSYEhIDd7vo73WYNyeGrFA0fXHMCDjf4Dn8/mL6O82ZV9J5FVKCJ0F+lYY/xkYXm1vY4KbPIdJcBl95lshVJiKRnT0LY+nHkqjB3b3lZy9usIhcdBU/Msf4sMgZoEVwCHR/NJSeVfeSEkj2fTpVs2IxitbFOdd1eSlKIT3vEOBrPmFTwUiJUJcsdvIZJX8kqMJsEtPNh6DZx2bXppEeNmOiE6Ae+wA5BGAtC1cc6tKYcuF+DuP3qMGaTlpDgFhJAvyTgB58nBk7dkA4nVqvfJg837dNB3u3jLukI6nx/PFiehIABtF8WKsiuUFFh8Ia65deb+cAdvv169RgrMmsAazSDZslYHuoAl12Qr09EigIyi6H43kIgcHWzkvoCkhh7/5f3dbFru4ANavWmaDf6B2NV2bCfFMQZd4+esWUm2OzrSp5bBnemPCH3cQdnZfEEKxmktcZHFZYwuJ30d7zLaPJkgnopa9xtLNvPNtSNLy7surFL+a3WSPK4Cz+KCxfFhpeTvkXP4dlVSnx2OqP5K8CuKuAQey9UzoCb2Ap0fPQWWBkh9qIu7l5ie+oAPjgaADITIYjjmJGTAmlgbkcWWQreZVpwcMOmzKE1QmLoPe0o0SUU+52DD27b9anIQdL7EIp39Pyr6vD6LagX614KumuB/ZLvZt1my3a8kf364mBmImjF0TWxU56PJO0+KQB11OKWiQ6G22n1y01jBPn47314uWyCVIJk65EncuuAwXITnX/+BGivk05/O8evxFBwzOeCN9RM1pYm2vzYMLbJnNddf+g0Afh751ClRWERC+WP53jwryyru0TtJ9N/EIaip9G/d8GmW1Uk3RRFXp1Kp/EfFVbAkAcjuI6d5OStWU/A1UoRHV HYFSeLTc Iu3T+rhmKMCVV91FaNpQiBvm75irQf5Xhl2Y7EbqvR276t9fpFdBbJeUgBq16qYNFntrA0AKJCJIr3emhX1WHqRZSmhhi4e5D9sPy3Qu9wK5glcHDN6xQCpwOkBmfIhN+QYkym7vJtMDt67J6obU6nkey2Nghrk4QHra8As/45SBKFw0LbXfobYwX1lyBtUXo6trONww1rCqAJoXi9cEv8ICVQI5IP5kdJ/0rOTTBJEAwLXG1VFKS9MUz6LyuGlfQaNVlFFulA8nHaBKaHuH5yqf51aHtExuyfVujCTNJAQWoYNeDRVG8e++ClUfEH2GP7N2C23/55U2SuwzpYgGBUJfjWNuzP1M22hQIsvTdBG4KhJiNMPJ+5xmBSIaDERCbaEkX1eUkqCAihQw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Mike Rapoport (Microsoft)" Architectures that support NUMA duplicate the code that allocates NODE_DATA on the node-local memory with slight variations in reporting of the addresses where the memory was allocated. Use x86 version as the basis for the generic alloc_node_data() function and call this function in architecture specific numa initialization. Round up node data size to SMP_CACHE_BYTES rather than to PAGE_SIZE like x86 used to do since the bootmem era when allocation granularity was PAGE_SIZE anyway. Signed-off-by: Mike Rapoport (Microsoft) Acked-by: David Hildenbrand Reviewed-by: Jonathan Cameron Tested-by: Zi Yan # for x86_64 and arm64 --- arch/loongarch/kernel/numa.c | 18 ------------------ arch/mips/loongson64/numa.c | 16 ++-------------- arch/powerpc/mm/numa.c | 24 +++--------------------- arch/sh/mm/init.c | 7 +------ arch/sparc/mm/init_64.c | 9 ++------- arch/x86/mm/numa.c | 34 +--------------------------------- drivers/base/arch_numa.c | 21 +-------------------- include/linux/numa.h | 2 ++ mm/numa.c | 27 +++++++++++++++++++++++++++ 9 files changed, 39 insertions(+), 119 deletions(-) diff --git a/arch/loongarch/kernel/numa.c b/arch/loongarch/kernel/numa.c index acada671e020..84fe7f854820 100644 --- a/arch/loongarch/kernel/numa.c +++ b/arch/loongarch/kernel/numa.c @@ -187,24 +187,6 @@ int __init numa_add_memblk(int nid, u64 start, u64 end) return numa_add_memblk_to(nid, start, end, &numa_meminfo); } -static void __init alloc_node_data(int nid) -{ - void *nd; - unsigned long nd_pa; - size_t nd_sz = roundup(sizeof(pg_data_t), PAGE_SIZE); - - nd_pa = memblock_phys_alloc_try_nid(nd_sz, SMP_CACHE_BYTES, nid); - if (!nd_pa) { - pr_err("Cannot find %zu Byte for node_data (initial node: %d)\n", nd_sz, nid); - return; - } - - nd = __va(nd_pa); - - node_data[nid] = nd; - memset(nd, 0, sizeof(pg_data_t)); -} - static void __init node_mem_init(unsigned int node) { unsigned long start_pfn, end_pfn; diff --git a/arch/mips/loongson64/numa.c b/arch/mips/loongson64/numa.c index d56238745744..8388400d052f 100644 --- a/arch/mips/loongson64/numa.c +++ b/arch/mips/loongson64/numa.c @@ -81,12 +81,8 @@ static void __init init_topology_matrix(void) static void __init node_mem_init(unsigned int node) { - struct pglist_data *nd; unsigned long node_addrspace_offset; unsigned long start_pfn, end_pfn; - unsigned long nd_pa; - int tnid; - const size_t nd_size = roundup(sizeof(pg_data_t), SMP_CACHE_BYTES); node_addrspace_offset = nid_to_addrbase(node); pr_info("Node%d's addrspace_offset is 0x%lx\n", @@ -96,16 +92,8 @@ static void __init node_mem_init(unsigned int node) pr_info("Node%d: start_pfn=0x%lx, end_pfn=0x%lx\n", node, start_pfn, end_pfn); - nd_pa = memblock_phys_alloc_try_nid(nd_size, SMP_CACHE_BYTES, node); - if (!nd_pa) - panic("Cannot allocate %zu bytes for node %d data\n", - nd_size, node); - nd = __va(nd_pa); - memset(nd, 0, sizeof(struct pglist_data)); - tnid = early_pfn_to_nid(nd_pa >> PAGE_SHIFT); - if (tnid != node) - pr_info("NODE_DATA(%d) on node %d\n", node, tnid); - node_data[node] = nd; + alloc_node_data(node); + NODE_DATA(node)->node_start_pfn = start_pfn; NODE_DATA(node)->node_spanned_pages = end_pfn - start_pfn; diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c index 0744a9a2944b..3c1da08304d0 100644 --- a/arch/powerpc/mm/numa.c +++ b/arch/powerpc/mm/numa.c @@ -1093,27 +1093,9 @@ void __init dump_numa_cpu_topology(void) static void __init setup_node_data(int nid, u64 start_pfn, u64 end_pfn) { u64 spanned_pages = end_pfn - start_pfn; - const size_t nd_size = roundup(sizeof(pg_data_t), SMP_CACHE_BYTES); - u64 nd_pa; - void *nd; - int tnid; - - nd_pa = memblock_phys_alloc_try_nid(nd_size, SMP_CACHE_BYTES, nid); - if (!nd_pa) - panic("Cannot allocate %zu bytes for node %d data\n", - nd_size, nid); - - nd = __va(nd_pa); - - /* report and initialize */ - pr_info(" NODE_DATA [mem %#010Lx-%#010Lx]\n", - nd_pa, nd_pa + nd_size - 1); - tnid = early_pfn_to_nid(nd_pa >> PAGE_SHIFT); - if (tnid != nid) - pr_info(" NODE_DATA(%d) on node %d\n", nid, tnid); - - node_data[nid] = nd; - memset(NODE_DATA(nid), 0, sizeof(pg_data_t)); + + alloc_node_data(nid); + NODE_DATA(nid)->node_id = nid; NODE_DATA(nid)->node_start_pfn = start_pfn; NODE_DATA(nid)->node_spanned_pages = spanned_pages; diff --git a/arch/sh/mm/init.c b/arch/sh/mm/init.c index d1fe90b2f5ff..2a88b0c9e70f 100644 --- a/arch/sh/mm/init.c +++ b/arch/sh/mm/init.c @@ -212,12 +212,7 @@ void __init allocate_pgdat(unsigned int nid) get_pfn_range_for_nid(nid, &start_pfn, &end_pfn); #ifdef CONFIG_NUMA - NODE_DATA(nid) = memblock_alloc_try_nid( - sizeof(struct pglist_data), - SMP_CACHE_BYTES, MEMBLOCK_LOW_LIMIT, - MEMBLOCK_ALLOC_ACCESSIBLE, nid); - if (!NODE_DATA(nid)) - panic("Can't allocate pgdat for node %d\n", nid); + alloc_node_data(nid); #endif NODE_DATA(nid)->node_start_pfn = start_pfn; diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c index c6c7f43cb1e8..21f8cbbd0581 100644 --- a/arch/sparc/mm/init_64.c +++ b/arch/sparc/mm/init_64.c @@ -1075,14 +1075,9 @@ static void __init allocate_node_data(int nid) { struct pglist_data *p; unsigned long start_pfn, end_pfn; -#ifdef CONFIG_NUMA - NODE_DATA(nid) = memblock_alloc_node(sizeof(struct pglist_data), - SMP_CACHE_BYTES, nid); - if (!NODE_DATA(nid)) { - prom_printf("Cannot allocate pglist_data for nid[%d]\n", nid); - prom_halt(); - } +#ifdef CONFIG_NUMA + alloc_node_data(nid); NODE_DATA(nid)->node_id = nid; #endif diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c index 7de725d6bb05..5e1dde26674b 100644 --- a/arch/x86/mm/numa.c +++ b/arch/x86/mm/numa.c @@ -191,39 +191,6 @@ int __init numa_add_memblk(int nid, u64 start, u64 end) return numa_add_memblk_to(nid, start, end, &numa_meminfo); } -/* Allocate NODE_DATA for a node on the local memory */ -static void __init alloc_node_data(int nid) -{ - const size_t nd_size = roundup(sizeof(pg_data_t), PAGE_SIZE); - u64 nd_pa; - void *nd; - int tnid; - - /* - * Allocate node data. Try node-local memory and then any node. - * Never allocate in DMA zone. - */ - nd_pa = memblock_phys_alloc_try_nid(nd_size, SMP_CACHE_BYTES, nid); - if (!nd_pa) { - pr_err("Cannot find %zu bytes in any node (initial node: %d)\n", - nd_size, nid); - return; - } - nd = __va(nd_pa); - - /* report and initialize */ - printk(KERN_INFO "NODE_DATA(%d) allocated [mem %#010Lx-%#010Lx]\n", nid, - nd_pa, nd_pa + nd_size - 1); - tnid = early_pfn_to_nid(nd_pa >> PAGE_SHIFT); - if (tnid != nid) - printk(KERN_INFO " NODE_DATA(%d) on node %d\n", nid, tnid); - - node_data[nid] = nd; - memset(NODE_DATA(nid), 0, sizeof(pg_data_t)); - - node_set_online(nid); -} - /** * numa_cleanup_meminfo - Cleanup a numa_meminfo * @mi: numa_meminfo to clean up @@ -571,6 +538,7 @@ static int __init numa_register_memblks(struct numa_meminfo *mi) continue; alloc_node_data(nid); + node_set_online(nid); } /* Dump memblock with node info and return. */ diff --git a/drivers/base/arch_numa.c b/drivers/base/arch_numa.c index ceac5b59bf2b..b6af7475ec44 100644 --- a/drivers/base/arch_numa.c +++ b/drivers/base/arch_numa.c @@ -216,30 +216,11 @@ int __init numa_add_memblk(int nid, u64 start, u64 end) */ static void __init setup_node_data(int nid, u64 start_pfn, u64 end_pfn) { - const size_t nd_size = roundup(sizeof(pg_data_t), SMP_CACHE_BYTES); - u64 nd_pa; - void *nd; - int tnid; - if (start_pfn >= end_pfn) pr_info("Initmem setup node %d []\n", nid); - nd_pa = memblock_phys_alloc_try_nid(nd_size, SMP_CACHE_BYTES, nid); - if (!nd_pa) - panic("Cannot allocate %zu bytes for node %d data\n", - nd_size, nid); - - nd = __va(nd_pa); - - /* report and initialize */ - pr_info("NODE_DATA [mem %#010Lx-%#010Lx]\n", - nd_pa, nd_pa + nd_size - 1); - tnid = early_pfn_to_nid(nd_pa >> PAGE_SHIFT); - if (tnid != nid) - pr_info("NODE_DATA(%d) on node %d\n", nid, tnid); + alloc_node_data(nid); - node_data[nid] = nd; - memset(NODE_DATA(nid), 0, sizeof(pg_data_t)); NODE_DATA(nid)->node_id = nid; NODE_DATA(nid)->node_start_pfn = start_pfn; NODE_DATA(nid)->node_spanned_pages = end_pfn - start_pfn; diff --git a/include/linux/numa.h b/include/linux/numa.h index e5841d4057ab..3b12d8ca0afd 100644 --- a/include/linux/numa.h +++ b/include/linux/numa.h @@ -33,6 +33,8 @@ static inline bool numa_valid_node(int nid) extern struct pglist_data *node_data[]; #define NODE_DATA(nid) (node_data[nid]) +void __init alloc_node_data(int nid); + /* Generic implementation available */ int numa_nearest_node(int node, unsigned int state); diff --git a/mm/numa.c b/mm/numa.c index 8c157d41c026..67a0d7734a98 100644 --- a/mm/numa.c +++ b/mm/numa.c @@ -1,11 +1,38 @@ // SPDX-License-Identifier: GPL-2.0-or-later +#include #include #include struct pglist_data *node_data[MAX_NUMNODES]; EXPORT_SYMBOL(node_data); +/* Allocate NODE_DATA for a node on the local memory */ +void __init alloc_node_data(int nid) +{ + const size_t nd_size = roundup(sizeof(pg_data_t), SMP_CACHE_BYTES); + u64 nd_pa; + void *nd; + int tnid; + + /* Allocate node data. Try node-local memory and then any node. */ + nd_pa = memblock_phys_alloc_try_nid(nd_size, SMP_CACHE_BYTES, nid); + if (!nd_pa) + panic("Cannot allocate %zu bytes for node %d data\n", + nd_size, nid); + nd = __va(nd_pa); + + /* report and initialize */ + pr_info("NODE_DATA(%d) allocated [mem %#010Lx-%#010Lx]\n", nid, + nd_pa, nd_pa + nd_size - 1); + tnid = early_pfn_to_nid(nd_pa >> PAGE_SHIFT); + if (tnid != nid) + pr_info(" NODE_DATA(%d) on node %d\n", nid, tnid); + + node_data[nid] = nd; + memset(NODE_DATA(nid), 0, sizeof(pg_data_t)); +} + /* Stub functions: */ #ifndef memory_add_physaddr_to_nid