From patchwork Tue Jul 18 11:41:37 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vijay Kilari X-Patchwork-Id: 9848085 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9CD3D60392 for ; Tue, 18 Jul 2017 11:44:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 898EE28462 for ; Tue, 18 Jul 2017 11:44:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7E4EF2857F; Tue, 18 Jul 2017 11:44:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.1 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,RCVD_IN_DNSWL_MED,RCVD_IN_SORBS_SPAM, RCVD_IN_SORBS_WEB,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 03A2428462 for ; Tue, 18 Jul 2017 11:44:50 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dXQuS-0007Be-1Z; Tue, 18 Jul 2017 11:43:24 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dXQuR-000790-0k for xen-devel@lists.xen.org; Tue, 18 Jul 2017 11:43:23 +0000 Received: from [85.158.139.211] by server-13.bemta-5.messagelabs.com id 85/5C-01732-954FD695; Tue, 18 Jul 2017 11:43:21 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrNIsWRWlGSWpSXmKPExsXiVRvspBv2JTf S4OJOXoslHxezODB6HN39mymAMYo1My8pvyKBNePNyVfMBZP1Krq/T2BvYHys0sXIxSEkMJFR Yueqj2wgDovASxaJhl+7WUAcCYF+VolDGw4zdzFyAjlxEpuXb2GBsCsl7u5cBxYXElCV6Pj/l h3CbmOSaDlSBWKzCchIHHl+gAnEFhGQlrj2+TIjyFBmgW1MEpuuTwVrEBZwlPj5oYu1i5EDaL WqROumYpAwr4CbxIMLv8HCEgIKEnMm2YCEOYHCi7ouQa1ylbjRt5J1AqPAAkaGVYwaxalFZal FukYGeklFmekZJbmJmTm6hgamermpxcWJ6ak5iUnFesn5uZsYgYFVz8DAuIOxcbbfIUZJDiYl Ud7VB3IjhfiS8lMqMxKLM+KLSnNSiw8xynBwKEnwJn0GygkWpaanVqRl5gBDHCYtwcGjJMJ78 xNQmre4IDG3ODMdInWK0ZLjTt+GL0wckw5sB5KvJvz/xiTEkpeflyolzisKMk8ApCGjNA9uHC wOLzHKSgnzMjIwMAjxFKQW5WaWoMq/YhTnYFQS5l0HspYnM68EbusroIOYgA4S9s0BOagkESE l1cBoVNmcMUXjRewB9+iMru75LA+uRke88M5fFXF4rczW3/Fytrxhv2andZyLq53S8Fs/cavw +y1zN5nHPHvJ/0xWaEvOcvaGN0mReyY43sy2OZF5fGq84u3XRy8d6rm8J+zH3hx/5ymc2sv2v OJliftweo5LbHH1z1CmyVOyeG89zZHdfCdadlenEktxRqKhFnNRcSIABjA6Ab4CAAA= X-Env-Sender: vijay.kilari@gmail.com X-Msg-Ref: server-8.tower-206.messagelabs.com!1500378196!103254694!1 X-Originating-IP: [74.125.83.66] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.4.25; banners=-,-,- X-VirusChecked: Checked Received: (qmail 24073 invoked from network); 18 Jul 2017 11:43:17 -0000 Received: from mail-pg0-f66.google.com (HELO mail-pg0-f66.google.com) (74.125.83.66) by server-8.tower-206.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 18 Jul 2017 11:43:17 -0000 Received: by mail-pg0-f66.google.com with SMTP id d193so2535748pgc.2 for ; Tue, 18 Jul 2017 04:43:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=PE/mzgBuJYld+U+qBrsijOW/HcS3ymeWMkKmMl9C2Co=; b=UoVDVNHYfJGelQPaoSpQJrRlT7qRV+llHwFYh//DSuGuPdaXnEFqxF50DgGMsI0mTt 7Pt7hIKRBFJcz2bijyqRVVCsSkO2jv9SUqmEX/wS+5UMFi0nLAwvrxMpPZ7RsTpkKtcW R0WPaSylHStB++LUFzXkenqD8apyl33c3AyKkXoQ6n7BCwVHCm0h8QKPUWiNiSMvvhiT 1+lI2UET4Pv2SZ2aBp3nrUTmd3EKPmqQRSUjzsw1QhGINnA++Ta6M17v4WcZ5CM652B0 C5/g/ssr89UDgmshpNQNb8KftKB0BeL+n/tGC6C3IBj8a1pGkO/krLOfz3U5saEb+XQ+ 6d9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=PE/mzgBuJYld+U+qBrsijOW/HcS3ymeWMkKmMl9C2Co=; b=SMcnCWI4U6TcaMrJ8PTDy6fQHmumGMhfe6SuE+/Sgqp92VYqRdvHgkKqPEgnQajVKK vaVOMIrn4oA5NQT89X+M0jagFlmJESyFYgytFhYZF791Fs30Y5Dtp7EoU9i+095sqZnY oUcxwgjWg8ZpkGhcXCaAhYhTlEmDigclcEzGtzh+JnpkVEzFz5vma8NaFAWxTqjVl5S8 h0OijFAcjrN9gfmy2ZRw0/4kwn7CzEUlymWL+xgTiftohAaC4mZyevS7ejaGAvcBQm30 Zezdayc2Gnt/N28+6KS8zmY4sXurTC/4Wt+ZPB8+pD4Tat9aCBy/OUKNLXo0ut3czQqB 4Vtg== X-Gm-Message-State: AIVw110fJmFy5VkLceeKkPzTqo50fOzL109IxTnfr8AbcQTcSjKJrmrW GftUEcZbq/nf3KXC X-Received: by 10.99.54.205 with SMTP id d196mr1226765pga.79.1500378196053; Tue, 18 Jul 2017 04:43:16 -0700 (PDT) Received: from cavium-Vostro-2520.caveonetworks.com ([111.93.218.67]) by smtp.gmail.com with ESMTPSA id r27sm4879707pfe.0.2017.07.18.04.43.11 (version=TLS1_1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 18 Jul 2017 04:43:15 -0700 (PDT) From: vijay.kilari@gmail.com To: xen-devel@lists.xen.org Date: Tue, 18 Jul 2017 17:11:37 +0530 Message-Id: <1500378106-2620-16-git-send-email-vijay.kilari@gmail.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1500378106-2620-1-git-send-email-vijay.kilari@gmail.com> References: <1500378106-2620-1-git-send-email-vijay.kilari@gmail.com> Cc: kevin.tian@intel.com, sstabellini@kernel.org, wei.liu2@citrix.com, George.Dunlap@eu.citrix.com, andrew.cooper3@citrix.com, dario.faggioli@citrix.com, ian.jackson@eu.citrix.com, tim@xen.org, julien.grall@arm.com, jbeulich@suse.com, Vijaya Kumar K Subject: [Xen-devel] [RFC PATCH v3 15/24] ARM: NUMA: DT: Add CPU NUMA support X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Vijaya Kumar K For each cpu, update cpu_to_node[] with node id from the numa-node-id DT property. Also, initialize cpu_to_node[] with node 0. Add macros to access cpu_to_node[] information. Signed-off-by: Vijaya Kumar K --- v3: - Dropped numa_add_cpu declaration from asm-arm/numa.h - Dropped stale declarations - Call numa_add_cpu for cpu0 --- xen/arch/arm/numa/numa.c | 21 +++++++++++++++++++++ xen/arch/arm/setup.c | 2 ++ xen/arch/arm/smpboot.c | 25 ++++++++++++++++++++++++- xen/include/asm-arm/numa.h | 7 +++++++ xen/include/asm-x86/numa.h | 1 - xen/include/xen/numa.h | 1 + 6 files changed, 55 insertions(+), 2 deletions(-) diff --git a/xen/arch/arm/numa/numa.c b/xen/arch/arm/numa/numa.c index c00b92c..dc80aa5 100644 --- a/xen/arch/arm/numa/numa.c +++ b/xen/arch/arm/numa/numa.c @@ -22,11 +22,31 @@ static uint8_t (*node_distance_fn)(nodeid_t a, nodeid_t b); +/* + * Setup early cpu_to_node. + */ +void __init init_cpu_to_node(void) +{ + int i; + + for ( i = 0; i < NR_CPUS; i++ ) + numa_set_node(i, 0); +} + void numa_failed(void) { numa_off = true; init_dt_numa_distance(); node_distance_fn = NULL; + init_cpu_to_node(); +} + +void __init numa_set_cpu_node(int cpu, unsigned int nid) +{ + if ( !node_isset(nid, processor_nodes_parsed) || nid >= MAX_NUMNODES ) + nid = 0; + + numa_set_node(cpu, nid); } uint8_t __node_distance(nodeid_t a, nodeid_t b) @@ -49,6 +69,7 @@ void __init numa_init(void) int ret = 0; nodes_clear(processor_nodes_parsed); + init_cpu_to_node(); init_dt_numa_distance(); if ( numa_off ) diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index a6d1499..b9c8b0d 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -787,6 +787,8 @@ void __init start_xen(unsigned long boot_phys_offset, processor_id(); + numa_add_cpu(0); + smp_init_cpus(); cpus = smp_get_max_cpus(); printk(XENLOG_INFO "SMP: Allowing %u CPUs\n", cpus); diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c index 32e8722..fcf9afc 100644 --- a/xen/arch/arm/smpboot.c +++ b/xen/arch/arm/smpboot.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -106,6 +107,7 @@ static void __init dt_smp_init_cpus(void) [0 ... NR_CPUS - 1] = MPIDR_INVALID }; bool_t bootcpu_valid = 0; + nodeid_t *cpu_to_nodemap; int rc; mpidr = boot_cpu_data.mpidr.bits & MPIDR_HWID_MASK; @@ -117,11 +119,18 @@ static void __init dt_smp_init_cpus(void) return; } + cpu_to_nodemap = xzalloc_array(nodeid_t, NR_CPUS); + if ( !cpu_to_nodemap ) + { + printk(XENLOG_WARNING "Failed to allocate memory for cpu_to_nodemap\n"); + return; + } + dt_for_each_child_node( cpus, cpu ) { const __be32 *prop; u64 addr; - u32 reg_len; + uint32_t reg_len, nid; register_t hwid; if ( !dt_device_type_is_equal(cpu, "cpu") ) @@ -146,6 +155,15 @@ static void __init dt_smp_init_cpus(void) continue; } + if ( !dt_property_read_u32(cpu, "numa-node-id", &nid) ) + { + printk(XENLOG_WARNING "cpu node `%s`: numa-node-id not found\n", + dt_node_full_name(cpu)); + nid = 0; + } + + cpu_to_nodemap[cpuidx] = nid; + addr = dt_read_number(prop, dt_n_addr_cells(cpu)); hwid = addr; @@ -224,6 +242,7 @@ static void __init dt_smp_init_cpus(void) { printk(XENLOG_WARNING "DT missing boot CPU MPIDR[23:0]\n" "Using only 1 CPU\n"); + xfree(cpu_to_nodemap); return; } @@ -233,7 +252,10 @@ static void __init dt_smp_init_cpus(void) continue; cpumask_set_cpu(i, &cpu_possible_map); cpu_logical_map(i) = tmp_map[i]; + numa_set_cpu_node(i, cpu_to_nodemap[i]); } + + xfree(cpu_to_nodemap); } void __init smp_init_cpus(void) @@ -313,6 +335,7 @@ void start_secondary(unsigned long boot_phys_offset, */ smp_wmb(); + numa_add_cpu(cpuid); /* Now report this CPU is up */ cpumask_set_cpu(cpuid, &cpu_online_map); diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h index d1dc83a..0d3146c 100644 --- a/xen/include/asm-arm/numa.h +++ b/xen/include/asm-arm/numa.h @@ -10,12 +10,19 @@ void init_dt_numa_distance(void); #ifdef CONFIG_NUMA void numa_init(void); int dt_numa_init(void); +void numa_set_cpu_node(int cpu, unsigned int nid); + #else static inline void numa_init(void) { return; } +static inline void numa_set_cpu_node(int cpu, unsigned int nid) +{ + return; +} + /* Fake one node for now. See also node_online_map. */ #define cpu_to_node(cpu) 0 #define node_to_cpumask(node) (cpu_online_map) diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h index ca0a2a6..fc4747f 100644 --- a/xen/include/asm-x86/numa.h +++ b/xen/include/asm-x86/numa.h @@ -15,7 +15,6 @@ extern nodeid_t acpi_setup_node(unsigned int pxm); extern void srat_detect_node(int cpu); extern nodeid_t apicid_to_node[]; -extern void init_cpu_to_node(void); void srat_parse_regions(paddr_t addr); unsigned int arch_get_dma_bitsize(void); diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h index 10ef4c4..8a306e7 100644 --- a/xen/include/xen/numa.h +++ b/xen/include/xen/numa.h @@ -30,6 +30,7 @@ extern s8 acpi_numa; void numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn); int srat_disabled(void); int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node); +void init_cpu_to_node(void); #ifdef CONFIG_NUMA #define cpu_to_node(cpu) (cpu_to_node[cpu])