From patchwork Thu Feb 9 16:59:59 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vijay Kilari X-Patchwork-Id: 9565189 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5E02160216 for ; Thu, 9 Feb 2017 17:02:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4DDC728538 for ; Thu, 9 Feb 2017 17:02:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 42F812853B; Thu, 9 Feb 2017 17:02:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CFFEB28538 for ; Thu, 9 Feb 2017 17:02:54 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cbs4j-00082E-Rd; Thu, 09 Feb 2017 17:00:05 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cbs4j-0007x2-5k for xen-devel@lists.xenproject.org; Thu, 09 Feb 2017 17:00:05 +0000 Received: from [85.158.139.211] by server-11.bemta-5.messagelabs.com id 0F/0F-01711-410AC985; Thu, 09 Feb 2017 17:00:04 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrMIsWRWlGSWpSXmKPExsVyMfSWo67ggjk RBrNm2Fh83zKZyYHR4/CHKywBjFGsmXlJ+RUJrBl7v05jL7ivWDFv9VS2BsYVUl2MXBxCAtMZ JdZOPsMC4rAINLBKnOhaxwriSAjMYZW4+/MgUIYTyMmRmLpnByOEXShxs+somM0rIChxcuYTs BohAU+JjUtng9mcAtYSL5a1sULESyQObf0AFmcT0Ja4tOYxM4jNIqAicaV/LzvEnACJ5euXgs 0UBup9uPI0WL2IgLrEnwsTGEEOYha4xyhxa+8poAQHkKMusX6e0ARGgVlIzpiFkFnAyLSKUaM 4tagstUjXyFIvqSgzPaMkNzEzR9fQwFQvN7W4ODE9NScxqVgvOT93EyMwEOsZGBh3MF7e4neI UZKDSUmUV7ZgToQQX1J+SmVGYnFGfFFpTmrxIUYZDg4lCd6Z84BygkWp6akVaZk5wJiASUtw8 CiJ8KqCpHmLCxJzizPTIVKnGC05erpOv2TiOHUDRO7ZdfklkxBLXn5eqpQ47xSQBgGQhozSPL hxsLi9xCgrJczLyMDAIMRTkFqUm1mCKv+KUZyDUUmYtwFkCk9mXgnc1ldABzEBHXT99CyQg0o SEVJSDYxps7WlQ9dpBzTf3lIqcuWbu+AyZtN5uTdso8SSrbf9Wzql9flmFW11g8c7Fm26mTrZ /hz/5tkb/i3cOuGqX962Z7vfqMpGfMkvWle8Jdnmy1TFJBfzWQcF9QrC5l3eLvx610qNpVUdT nY3rxcatcief8HdfPudAKNp577X4nucAkpqfNsXF8xRYinOSDTUYi4qTgQAfA5xadYCAAA= X-Env-Sender: vijay.kilari@gmail.com X-Msg-Ref: server-2.tower-206.messagelabs.com!1486659600!63527670!1 X-Originating-IP: [209.85.218.65] X-SpamReason: No, hits=0.3 required=7.0 tests=RCVD_BY_IP X-StarScan-Received: X-StarScan-Version: 9.1.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 18858 invoked from network); 9 Feb 2017 17:00:01 -0000 Received: from mail-oi0-f65.google.com (HELO mail-oi0-f65.google.com) (209.85.218.65) by server-2.tower-206.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 9 Feb 2017 17:00:01 -0000 Received: by mail-oi0-f65.google.com with SMTP id u143so684371oif.3 for ; Thu, 09 Feb 2017 09:00:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=VdGgjsuKeVvO+7Lx/Fd8GH2UJm6uiKD9mZTTlPUhWxU=; b=eyX0uwykiHyAU1R7e+W7oHfJRbVat0LPwyQy3blWaONgUqtmynZY4BqGhPUACh8RDf 6/jVPm0Er0ms7cAfy/hmrTssiDIvMdAtzJfXmnJFK3RiFFSMVc3vXMyPX2V33zAJbRA6 cRym+z6yp7TaIKjhBEUkKg0h1z2CX7T6ELpW/JpOwPkOMbwO0JPiyViSl4zGZpfL1JCt FfDRlT0RjALNTimALm9+msjlExQDkWuGIRLE7LTl1vmF5U3Lzq6LGqkpTt8+noOtcwr5 Sm2oBH3qU0KTGGL91QZZPdNYCvWOzo1nRV3L9tOmr5R6hdpG8dRkcXP4D3VGzFKYVkNM sj4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=VdGgjsuKeVvO+7Lx/Fd8GH2UJm6uiKD9mZTTlPUhWxU=; b=nYXhfSd23fNUPbltuE1aH+7y2I+LS/EAjAJLRDtrCa98/PN7ji+ANI1lOFHq+e/xRz C27+daic+1+5wtUDBd9MXEV5nD27Kk3uPm5URyHKk+FzaVL3QYy53Xu0YY0H7/gitkWh qanUiuwlfKhIVNgNSjyuH3EWkTLBUyYZZFNTF9uK8KTPnr696JNculk8Rs1rG9yAK2P/ hAbeZqfbvf3kvYzNauklLeanURMASdenuiNt0jtrhV+tpunyc/QMsMewNebnTtFDql6e MAH23ovHwrUmGNQCNgrBHf8ag1/JOcW/a65/i4t29edruWd9dObZiLpyuu2/C4NdFZ1Z mw6g== X-Gm-Message-State: AMke39kMQohitIQs5cYHZuGRZnCVMRhT9kzrRPp0MHiks6hctZ9gSp7QEXWToeJL9gJxWerEdXD8Gey4d6BXfw== X-Received: by 10.202.56.84 with SMTP id f81mr1909249oia.21.1486659600436; Thu, 09 Feb 2017 09:00:00 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.37.84 with HTTP; Thu, 9 Feb 2017 08:59:59 -0800 (PST) In-Reply-To: <0326b5f8-b646-7e40-0494-020979373ce7@arm.com> References: <1486655834-9708-1-git-send-email-vijay.kilari@gmail.com> <0326b5f8-b646-7e40-0494-020979373ce7@arm.com> From: Vijay Kilari Date: Thu, 9 Feb 2017 22:29:59 +0530 Message-ID: To: Julien Grall Cc: Andre Przywara , Dario Faggioli , Stefano Stabellini , Vijaya Kumar K , xen-devel Subject: Re: [Xen-devel] [RFC PATCH v1 00/21] ARM: Add Xen NUMA support X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Hi Julien, On Thu, Feb 9, 2017 at 10:01 PM, Julien Grall wrote: > Hi Vijay, > > On 02/09/2017 03:56 PM, vijay.kilari@gmail.com wrote: >> >> Note: Please use this patch series only for review. >> For testing, patch to boot allocator is required. Which will >> be sent outside this series. > > > Can you expand here? Is this patch a NUMA specific? Yes it is NUMA specific, which I have reported here. I have workaround for this. Need to prepare a patch. ( I hope till now there is no patch from anyone else for this issue) https://www.mail-archive.com/xen-devel@lists.xen.org/msg92093.html > > Also in a previous thread you mentioned issue to boot Xen with NUMA on Xen > unstable. So how did you test it? This issue (panic in page_alloc.c) that I reported is seen when I boot plain unstable xen on NUMA board without any NUMA or ITS patches. This issue is seen only with on NUMA board with DT I have tested this series with ACPI using unstable version and DT on 4.7 version. Also, I have prepared a small patch as below (just adhoc way), where in I called cpu_to_node() for all cpus and print phys_to_nid() to see if the node id is correct or not. ----------------------------------------------------------------------------------------- > > Cheers, > > -- > Julien Grall diff --git a/xen/arch/arm/numa.c b/xen/arch/arm/numa.c index d296523..d28e6bf 100644 --- a/xen/arch/arm/numa.c +++ b/xen/arch/arm/numa.c @@ -43,9 +43,11 @@ void __init numa_set_cpu_node(int cpu, unsigned long hwid) unsigned node; node = hwid >> 16 & 0xf; + printk("In %s cpu %d node %d\n",__func__, cpu, node); if ( !node_isset(node, numa_nodes_parsed) || node == MAX_NUMNODES ) node = 0; + printk("In %s cpu %d node %d\n",__func__, cpu, node); numa_set_node(cpu, node); numa_add_cpu(cpu); } @@ -245,3 +247,52 @@ int __init arch_numa_setup(char *opt) { return 1; } + +struct mem_list { + u64 start; + u64 end; +}; + +void numa_test(void) +{ + int i; + + struct mem_list ml[] = + { + { 0x0000000001400000, 0x00000000fffecfff }, + { 0x0000000100000000 , 0x0000000ff7ffffff }, + { 0x0000000ff8000000 , 0x0000000ff801ffff }, + { 0x0000000ff8020000 , 0x0000000fffa9cfff }, + { 0x0000000fffa9d000 , 0x0000000fffffffff }, + { 0x0000010000400000 , 0x0000010ff57b2fff }, + { 0x0000010ff6618000 , 0x0000010ff6ff0fff }, + { 0x0000010ff6ff1000 , 0x0000010ff724ffff }, + { 0x0000010ff734b000 , 0x0000010ff73defff }, + { 0x0000010ff73f0000 , 0x0000010ff73fbfff }, + { 0x0000010ff73fc000 , 0x0000010ff74defff }, + { 0x0000010ff74df000 , 0x0000010ff9718fff }, + { 0x0000010ff97a2000 , 0x0000010ff97acfff }, + { 0x0000010ff97ad000 , 0x0000010ff97b3fff }, + { 0x0000010ff97b5000 , 0x0000010ff9813fff }, + { 0x0000010ff9814000 , 0x0000010ff9819fff }, + { 0x0000010ff981a000 , 0x0000010ff984afff }, + { 0x0000010ff984c000 , 0x0000010ff9851fff }, + { 0x0000010ff9935000 , 0x0000010ffaeb5fff }, + { 0x0000010ffaff5000 , 0x0000010ffb008fff }, + { 0x0000010ffb009000 , 0x0000010fffe28fff }, + { 0x0000010fffe29000 , 0x0000010fffe70fff }, + { 0x0000010fffe71000 , 0x0000010ffffb8fff }, + { 0x0000010ffffff000 , 0x0000010fffffffff }, + }; + + for ( i = 0; i < 23; i++ ) + { + printk("NUMA MEM TEST: start 0x%lx in node %d end 0x%lx in node %d\n", + ml[i].start, phys_to_nid(ml[i].start), ml[i].end, phys_to_nid(ml[i].end)); + } + + for ( i = 0; i < NR_CPUS; i++) + { + printk("NUMA CPU TEST: cpu %d in node %d\n", i, cpu_to_node(i)); + } +} diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index 5612ba6..0598672 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -698,6 +698,7 @@ void __init setup_cache(void) cacheline_bytes = 1U << (4 + (ccsid & 0x7)); } +extern void numa_test(void); /* C entry point for boot CPU */ void __init start_xen(unsigned long boot_phys_offset, unsigned long fdt_paddr, @@ -825,6 +826,7 @@ void __init start_xen(unsigned long boot_phys_offset, } } + numa_test(); printk("Brought up %ld CPUs\n", (long)num_online_cpus()); /* TODO: smp_cpus_done(); */