From patchwork Tue Mar 28 15:53:23 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vijay Kilari X-Patchwork-Id: 9650077 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 65B25602C8 for ; Tue, 28 Mar 2017 15:57:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 56E2126E56 for ; Tue, 28 Mar 2017 15:57:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4B4A6283C9; Tue, 28 Mar 2017 15:57:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D08E726E56 for ; Tue, 28 Mar 2017 15:57:31 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cstSZ-0007Wj-Ii; Tue, 28 Mar 2017 15:55:03 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cstSY-0007Ue-Jf for xen-devel@lists.xen.org; Tue, 28 Mar 2017 15:55:02 +0000 Received: from [193.109.254.147] by server-5.bemta-6.messagelabs.com id 9C/90-27545-5578AD85; Tue, 28 Mar 2017 15:55:01 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrAIsWRWlGSWpSXmKPExsXiVRvsrBvafiv C4MNjXoslHxezODB6HN39mymAMYo1My8pvyKBNePO57KCpRoVr+5fZWtgvCDfxcjFISQwgVGi YdcdZhCHReAli8S/mRNZQRwJgX5WiZ3bdjJ2MXICOTESc65cg7KrJDqf3GECsYUEVCU6/r9lh xj1jVHi+ObXzCAJNgEZiSPPD4AViQhIS1z7fJkRpIhZoJlJom/3EjaQhLCArcT/y8/YQWwWoE mXPq4Ai/MKuEvs/HWMpYuRA2ibgsScSTYgYU6g8KYna5khFrtJ3D5zkXECo8ACRoZVjBrFqUV lqUW6RgZ6SUWZ6RkluYmZObqGBmZ6uanFxYnpqTmJScV6yfm5mxiBocUABDsYfy0LOMQoycGk JMr7IehWhBBfUn5KZUZicUZ8UWlOavEhRhkODiUJ3sQ2oJxgUWp6akVaZg4wyGHSEhw8SiK8q 0HSvMUFibnFmekQqVOMuhwf+g+/YRJiycvPS5US5w0DKRIAKcoozYMbAYu4S4yyUsK8jEBHCf EUpBblZpagyr9iFOdgVBLmVQaZwpOZVwK36RXQEUxAR4jbgB1RkoiQkmpg9NnwKmDalMMi15p V1X5LHuON2jJnktFdpzffJ8tObMmMt5zRtXKmWmfX+cicK71+8zmWzH9y1zHvF+fvZQYZygdl C0x+3+2Pkec84OK27M+05mSBKWIzYj6eZK9x/L01aIKh5NL0M0vrhPyyL6W5zDRauaA/zLTMt p9/+3vGJouiDx7fuvfGKbEUZyQaajEXFScCAGh2RpmzAgAA X-Env-Sender: vijay.kilari@gmail.com X-Msg-Ref: server-6.tower-27.messagelabs.com!1490716500!94332565!1 X-Originating-IP: [74.125.83.67] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.2.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 27972 invoked from network); 28 Mar 2017 15:55:00 -0000 Received: from mail-pg0-f67.google.com (HELO mail-pg0-f67.google.com) (74.125.83.67) by server-6.tower-27.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 28 Mar 2017 15:55:00 -0000 Received: by mail-pg0-f67.google.com with SMTP id 79so22369699pgf.0 for ; Tue, 28 Mar 2017 08:55:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vgZX89eUfbknLRdBdnOa2BMCWWszMc84kD+YHD6S5aE=; b=EIF3SQLUjoYeHXWjAQ2PTDd6k6WtTqBRDFFAtnvhXFqGnXJ7+LYDJOhN9x6LIFPQ+K UXyjcSatZ85xGmmlfBArKd5Ci634oHld9+xqtwwDNcNz82yHlXGphkS7dtAdPnamDTPX OfxIxcweUEOlxtbZHwWMsiXvxavrlF2v6yKadCVbbWO87oLndvlzgVD8S4YMPhsjiAkA wI13/m/pt8w9wCyPWoFRZpMj1thMRsJg+WTkPeScdGpDg/9Ppg+tyTX5Hmrqcv/tMJib U0ujxLpYme0BPHkzFQIw0g1XzwcNeJ9xC8edzRglUloHthVMIOkIEO0P7XNEY1VJYgxX +u0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vgZX89eUfbknLRdBdnOa2BMCWWszMc84kD+YHD6S5aE=; b=cs7J1xos5OSh28U6Xq1CBrbQjuTaMx12uPSNe1rjCvKpMMf+zrdoOg+1K4S5d7IeMI VJ5jK4Rt3ZqNBmlh8prNlLskuzIm4CN+03lvrpiP++52qzriSPJ8UnfvcLl0cBTaIFb2 641aB+pON0YcCol0TC6bLwsCmE/Y78RL6jx3G583xHKzbS+PkPSt2mavTHWu4cQfaUty OXv5OOmEK7rormMchLMrV1BC3UQV/R80mJxajhacwtW64kSCX16yc7Sb1Bs66FZNXLED ktJhG/fNXAzb6luD79B9pqruhLOFC6NHUZ+EremC1Jtbw8RNHbBLto/TeAZAKTauFZJ/ k1jg== X-Gm-Message-State: AFeK/H2I3zzrxf+u9NzmhsS/V0JCqQZVRUq8NuWeeA3ItLNFEJwITNK2jMh022QZ2gxkjw== X-Received: by 10.98.198.138 with SMTP id x10mr32982542pfk.85.1490716499526; Tue, 28 Mar 2017 08:54:59 -0700 (PDT) Received: from localhost.localdomain ([14.140.2.178]) by smtp.gmail.com with ESMTPSA id y7sm8298867pfk.93.2017.03.28.08.54.55 (version=TLS1_1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 28 Mar 2017 08:54:59 -0700 (PDT) From: vijay.kilari@gmail.com To: xen-devel@lists.xen.org Date: Tue, 28 Mar 2017 21:23:23 +0530 Message-Id: <1490716413-19796-16-git-send-email-vijay.kilari@gmail.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1490716413-19796-1-git-send-email-vijay.kilari@gmail.com> References: <1490716413-19796-1-git-send-email-vijay.kilari@gmail.com> Cc: sstabellini@kernel.org, wei.liu2@citrix.com, George.Dunlap@eu.citrix.com, andrew.cooper3@citrix.com, ian.jackson@eu.citrix.com, tim@xen.org, julien.grall@arm.com, jbeulich@suse.com, Vijaya Kumar K Subject: [Xen-devel] [RFC PATCH v2 15/25] ARM: NUMA: Add CPU NUMA support X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Vijaya Kumar K For each cpu, update cpu_to_node[] with node id from the numa-node-id DT property. Also, initialize cpu_to_node[] with node 0. Add macros to access cpu_to_node[] information. Signed-off-by: Vijaya Kumar K --- xen/arch/arm/numa/numa.c | 21 +++++++++++++++++++++ xen/arch/arm/smpboot.c | 25 ++++++++++++++++++++++++- xen/include/asm-arm/numa.h | 24 ++++++++++++++++++++++++ 3 files changed, 69 insertions(+), 1 deletion(-) diff --git a/xen/arch/arm/numa/numa.c b/xen/arch/arm/numa/numa.c index 0ee89da..eef5870 100644 --- a/xen/arch/arm/numa/numa.c +++ b/xen/arch/arm/numa/numa.c @@ -28,6 +28,25 @@ static uint8_t (*node_distance_fn)(nodeid_t a, nodeid_t b); extern nodemask_t processor_nodes_parsed; static bool_t dt_numa = 1; +/* + * Setup early cpu_to_node. + */ +void __init init_cpu_to_node(void) +{ + int i; + + for ( i = 0; i < NR_CPUS; i++ ) + numa_set_node(i, 0); +} + +void __init numa_set_cpu_node(int cpu, unsigned int nid) +{ + if ( !node_isset(nid, processor_nodes_parsed) || nid >= MAX_NUMNODES ) + nid = 0; + + numa_set_node(cpu, nid); +} + uint8_t __node_distance(nodeid_t a, nodeid_t b) { if ( node_distance_fn != NULL); @@ -48,6 +67,7 @@ void numa_failed(void) dt_numa = 0; init_dt_numa_distance(); node_distance_fn = NULL; + init_cpu_to_node(); } void __init numa_init(void) @@ -55,6 +75,7 @@ void __init numa_init(void) int ret = 0; nodes_clear(processor_nodes_parsed); + init_cpu_to_node(); init_dt_numa_distance(); if ( is_numa_off() ) goto no_numa; diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c index 32e8722..bf7ddaf 100644 --- a/xen/arch/arm/smpboot.c +++ b/xen/arch/arm/smpboot.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -106,6 +107,7 @@ static void __init dt_smp_init_cpus(void) [0 ... NR_CPUS - 1] = MPIDR_INVALID }; bool_t bootcpu_valid = 0; + nodeid_t *cpu_to_nodemap; int rc; mpidr = boot_cpu_data.mpidr.bits & MPIDR_HWID_MASK; @@ -117,11 +119,18 @@ static void __init dt_smp_init_cpus(void) return; } + cpu_to_nodemap = xzalloc_array(nodeid_t, NR_CPUS); + if ( !cpu_to_nodemap ) + { + printk(XENLOG_WARNING "Failed to allocate memory for cpu_to_nodemap\n"); + return; + } + dt_for_each_child_node( cpus, cpu ) { const __be32 *prop; u64 addr; - u32 reg_len; + u32 reg_len, nid; register_t hwid; if ( !dt_device_type_is_equal(cpu, "cpu") ) @@ -146,6 +155,15 @@ static void __init dt_smp_init_cpus(void) continue; } + if ( !dt_property_read_u32(cpu, "numa-node-id", &nid) ) + { + printk(XENLOG_WARNING "cpu node `%s`: numa-node-id not found\n", + dt_node_full_name(cpu)); + nid = 0; + } + + cpu_to_nodemap[cpuidx] = nid; + addr = dt_read_number(prop, dt_n_addr_cells(cpu)); hwid = addr; @@ -224,6 +242,7 @@ static void __init dt_smp_init_cpus(void) { printk(XENLOG_WARNING "DT missing boot CPU MPIDR[23:0]\n" "Using only 1 CPU\n"); + xfree(cpu_to_nodemap); return; } @@ -233,7 +252,10 @@ static void __init dt_smp_init_cpus(void) continue; cpumask_set_cpu(i, &cpu_possible_map); cpu_logical_map(i) = tmp_map[i]; + numa_set_cpu_node(i, cpu_to_nodemap[i]); } + + xfree(cpu_to_nodemap); } void __init smp_init_cpus(void) @@ -313,6 +335,7 @@ void start_secondary(unsigned long boot_phys_offset, */ smp_wmb(); + numa_add_cpu(cpuid); /* Now report this CPU is up */ cpumask_set_cpu(cpuid, &cpu_online_map); diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h index c390a0e..65bdd5e 100644 --- a/xen/include/asm-arm/numa.h +++ b/xen/include/asm-arm/numa.h @@ -14,12 +14,36 @@ extern uint8_t __node_distance(nodeid_t a, nodeid_t b); #ifdef CONFIG_NUMA extern void numa_init(void); extern int dt_numa_init(void); +extern void numa_set_cpu_node(int cpu, unsigned int nid); +extern void numa_add_cpu(int cpu); + +extern nodeid_t cpu_to_node[NR_CPUS]; +extern cpumask_t node_to_cpumask[]; +/* Simple perfect hash to map pdx to node numbers */ +extern unsigned int memnode_shift; +extern uint8_t *memnodemap; + +#define cpu_to_node(cpu) (cpu_to_node[cpu]) +#define parent_node(node) (node) +#define node_to_first_cpu(node) (__ffs(node_to_cpumask[node])) +#define node_to_cpumask(node) (node_to_cpumask[node]) + #else static inline void numa_init(void) { return; } +static inline void numa_set_cpu_node(int cpu, unsigned int nid) +{ + return; +} + +static inline void numa_add_cpu(int cpu) +{ + return; +} + /* Fake one node for now. See also node_online_map. */ #define cpu_to_node(cpu) 0 #define node_to_cpumask(node) (cpu_online_map)