From patchwork Mon May 29 02:19:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13257997 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 138A0C77B7E for ; Mon, 29 May 2023 02:20:12 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.540439.842210 (Exim 4.92) (envelope-from ) id 1q3SUF-0006uc-4a; Mon, 29 May 2023 02:19:55 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 540439.842210; Mon, 29 May 2023 02:19:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q3SUF-0006uB-19; Mon, 29 May 2023 02:19:55 +0000 Received: by outflank-mailman (input) for mailman id 540439; Mon, 29 May 2023 02:19:53 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1q3SUD-0006GL-ON for xen-devel@lists.xenproject.org; Mon, 29 May 2023 02:19:53 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 4af984a0-fdc7-11ed-8611-37d641c3527e; Mon, 29 May 2023 04:19:51 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8E789C14; Sun, 28 May 2023 19:20:36 -0700 (PDT) Received: from a015966.shanghai.arm.com (a015966.shanghai.arm.com [10.169.190.5]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id F38EB3F64C; Sun, 28 May 2023 19:19:48 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4af984a0-fdc7-11ed-8611-37d641c3527e From: Henry Wang To: xen-devel@lists.xenproject.org Cc: Wei Chen , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Henry Wang Subject: [PATCH v5 05/17] xen/arm: build NUMA cpu_to_node map in dt_smp_init_cpus Date: Mon, 29 May 2023 10:19:09 +0800 Message-Id: <20230529021921.2606623-6-Henry.Wang@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230529021921.2606623-1-Henry.Wang@arm.com> References: <20230529021921.2606623-1-Henry.Wang@arm.com> MIME-Version: 1.0 From: Wei Chen NUMA implementation has a cpu_to_node array to store CPU to NODE map. Xen is using CPU logical ID in runtime components, so we use CPU logical ID as CPU index in cpu_to_node. In device tree case, cpu_logical_map is created in dt_smp_init_cpus. So, when NUMA is enabled, dt_smp_init_cpus will fetch CPU NUMA id at the same time for cpu_to_node. Signed-off-by: Wei Chen Signed-off-by: Henry Wang --- v4 -> v5: 1. No change. v3 -> v4: 1. No change. v2 -> v3: 1. No change. v1 -> v2: 1. Use static inline to replace macros to perform function paramerters type check. 2. Add numa_disabled to gate the numa-node-id check for CONFIG_NUMA on but numa disabled user case. 3. Use macro instead of static inline function to stub numa_set_node. --- xen/arch/arm/include/asm/numa.h | 4 ++++ xen/arch/arm/smpboot.c | 36 +++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+) diff --git a/xen/arch/arm/include/asm/numa.h b/xen/arch/arm/include/asm/numa.h index 96c856a9f7..97d4a67dea 100644 --- a/xen/arch/arm/include/asm/numa.h +++ b/xen/arch/arm/include/asm/numa.h @@ -68,6 +68,10 @@ static inline bool arch_numa_broken(void) return true; } +static inline void numa_set_node(unsigned int cpu, nodeid_t node) +{ +} + #endif #define arch_want_default_dmazone() (false) diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c index e107b86b7b..7506085540 100644 --- a/xen/arch/arm/smpboot.c +++ b/xen/arch/arm/smpboot.c @@ -118,7 +118,12 @@ static void __init dt_smp_init_cpus(void) { [0 ... NR_CPUS - 1] = MPIDR_INVALID }; + static nodeid_t node_map[NR_CPUS] __initdata = + { + [0 ... NR_CPUS - 1] = NUMA_NO_NODE + }; bool bootcpu_valid = false; + unsigned int nid = 0; int rc; mpidr = system_cpuinfo.mpidr.bits & MPIDR_HWID_MASK; @@ -169,6 +174,28 @@ static void __init dt_smp_init_cpus(void) continue; } + if ( IS_ENABLED(CONFIG_NUMA) ) + { + /* + * When CONFIG_NUMA is set, try to fetch numa infomation + * from CPU dts node, otherwise the nid is always 0. + */ + if ( !dt_property_read_u32(cpu, "numa-node-id", &nid) ) + { + printk(XENLOG_WARNING + "cpu[%d] dts path: %s: doesn't have numa information!\n", + cpuidx, dt_node_full_name(cpu)); + /* + * During the early stage of NUMA initialization, when Xen + * found any CPU dts node doesn't have numa-node-id info, the + * NUMA will be treated as off, all CPU will be set to a FAKE + * node 0. So if we get numa-node-id failed here, we should + * set nid to 0. + */ + nid = 0; + } + } + /* * 8 MSBs must be set to 0 in the DT since the reg property * defines the MPIDR[23:0] @@ -228,9 +255,13 @@ static void __init dt_smp_init_cpus(void) { printk("cpu%d init failed (hwid %"PRIregister"): %d\n", i, hwid, rc); tmp_map[i] = MPIDR_INVALID; + node_map[i] = NUMA_NO_NODE; } else + { tmp_map[i] = hwid; + node_map[i] = nid; + } } if ( !bootcpu_valid ) @@ -246,6 +277,11 @@ static void __init dt_smp_init_cpus(void) continue; cpumask_set_cpu(i, &cpu_possible_map); cpu_logical_map(i) = tmp_map[i]; + + nid = node_map[i]; + if ( nid >= MAX_NUMNODES ) + nid = 0; + numa_set_node(i, nid); } }