From patchwork Thu Feb 9 15:57:05 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vijay Kilari X-Patchwork-Id: 9565043 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id EF34F6020C for ; Thu, 9 Feb 2017 16:02:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DD5D627F81 for ; Thu, 9 Feb 2017 16:02:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D225F281E1; Thu, 9 Feb 2017 16:02:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EFC1427F95 for ; Thu, 9 Feb 2017 16:02:27 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cbr8M-0008Hr-9l; Thu, 09 Feb 2017 15:59:46 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cbr8K-0008FE-EC for xen-devel@lists.xenproject.org; Thu, 09 Feb 2017 15:59:44 +0000 Received: from [193.109.254.147] by server-11.bemta-6.messagelabs.com id 68/22-04971-FE19C985; Thu, 09 Feb 2017 15:59:43 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrMIsWRWlGSWpSXmKPExsVyMfTAId33E+d EGCxqZLb4vmUykwOjx+EPV1gCGKNYM/OS8isSWDPm/JzIVvAkq6J1wSfmBsbTYV2MnBxCAtMY Je71qHcxcnGwCLxkkVjzZzsziCMh0M8qMaFzNwtIlYRAjMSFI4+YIewqiXctq5kgulUlOv6/Z QdpEBL4wShx6twMsASbgIzEkecHwGwRgTSJY6vXsoHYzAK+Ei8XTQQaxMEhLBAssfKkIUiYBW jO3E3/wMp5BdwkbjW+YQUpkRBQkJgzyQYkzAkUXr7nEgvEWleJB91trBMYBRYwMqxiVC9OLSp LLdK10EsqykzPKMlNzMzRNTQw08tNLS5OTE/NSUwq1kvOz93ECAwqBiDYwTj7sv8hRkkOJiVR XtmCORFCfEn5KZUZicUZ8UWlOanFhxhlODiUJHhNgEEqJFiUmp5akZaZAwxvmLQEB4+SCK8lS Jq3uCAxtzgzHSJ1itGYo6fr9Esmjj27Lr9kEmLJy89LlRLn5QYpFQApzSjNgxsEi7tLjLJSwr yMQKcJ8RSkFuVmlqDKv2IU52BUEuaVAJnCk5lXArfvFdApTECnXD89C+SUkkSElFQDo1X2+mf PWES7I8KvV845f9nu36XNwZKvZ2rfuysmH/Bly55tx296f5ZPCNJ1Vfk/9c6sRW18mzd+TBL4 +Lfxet8/0bCqjRafBBOn/J5grTG53+nIn8rL8WvKNyya+rwiP95qt2CwxlTBCXq2GVzVMyobP nP6nxH4c4g38b/NYv/e0odPc2a/u6zEUpyRaKjFXFScCADLV+yutgIAAA== X-Env-Sender: vijay.kilari@gmail.com X-Msg-Ref: server-16.tower-27.messagelabs.com!1486655981!85949041!1 X-Originating-IP: [209.85.192.194] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.1.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 59871 invoked from network); 9 Feb 2017 15:59:42 -0000 Received: from mail-pf0-f194.google.com (HELO mail-pf0-f194.google.com) (209.85.192.194) by server-16.tower-27.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 9 Feb 2017 15:59:42 -0000 Received: by mail-pf0-f194.google.com with SMTP id e4so482128pfg.0 for ; Thu, 09 Feb 2017 07:59:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=N+VpHRsB9hPzG71S/qKb+mRzPkqe6qj/3GUvP+ujIss=; b=LRoC5dAXQJKXf8iNo4zO+NiIgmS58YJQQwIBj89rR424apUPixBbHCB54/Iu4GCl7n uMSNL60BeCHX2nugHOGCvxc/sISB2QkNdtSTiQIICNck3R/aTjujtXOoKseJKqL/ipl7 +Xwi6ndVV3R9POb4ugU7IlYVuaCQfk5p8xarz0Nj2DVzyXsIYUxUEke0ndb9nUFzHPvc SdogkLRGtfCq3SOxaRwly6admPEPvrVFIu0Dorkmp6/3gYn75kv00xPqRIONs85x+ogZ qvZzQzYs2tVDIwsil5xVAOtPmcidpmQdiaYeFSWdQy+zRcsPe8aNl/JxK7rqOqkm67OD Zlog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=N+VpHRsB9hPzG71S/qKb+mRzPkqe6qj/3GUvP+ujIss=; b=H4G6vS+A6hRF6kH1vgn6HSSrZClxQzyzQqXdKTqN4sIgQZleBu11Bfv0fxBfQtwe6I tCBMDNdb/OzzjXmHBIcX49SXD6cCDUMWPzO9I0F+ZaPTPKLBLvkMoT5YTlgtbQvV+aue 5yPwdNi2YR3tHHTBDXkt+zTuYW4tvZl011XwGloulHgXJHjA2lio6L4sMEZQbrXOaJ2x jFBgHj0t8+vlCjSOXsNrsR/7PibVF6v6HL2Ktg8v516NP3OLEqKPBYeOXKLuwaCPxk4R tmuLZttoeF/SbLTxVK1H0L51zZREDyIiLEhSjZCUEpGK05T1pIj3itS4guj5bbxikJ2e XpJA== X-Gm-Message-State: AMke39mAhRkEfd2icGA8F9mGFQqYN1zdKSPoXJId8i+J7PVkQoby2cxyd6WGrD/FejRreA== X-Received: by 10.84.196.164 with SMTP id l33mr5099979pld.158.1486655981087; Thu, 09 Feb 2017 07:59:41 -0800 (PST) Received: from localhost.localdomain ([14.140.2.178]) by smtp.gmail.com with ESMTPSA id m29sm30093842pfi.54.2017.02.09.07.59.38 (version=TLS1_1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 09 Feb 2017 07:59:40 -0800 (PST) From: vijay.kilari@gmail.com To: julien.grall@arm.com, sstabellini@kernel.org, andre.przywara@arm.com, dario.faggioli@citrix.com Date: Thu, 9 Feb 2017 21:27:05 +0530 Message-Id: <1486655834-9708-14-git-send-email-vijay.kilari@gmail.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1486655834-9708-1-git-send-email-vijay.kilari@gmail.com> References: <1486655834-9708-1-git-send-email-vijay.kilari@gmail.com> Cc: xen-devel@lists.xenproject.org, Vijaya Kumar K Subject: [Xen-devel] [RFC PATCH v1 13/21] ACPI: Refactor acpi SRAT and SLIT table handling code X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Vijaya Kumar K Move SRAT handling code which is common across architecture is moved to new file xen/commom/srat.c from xen/arch/x86/srat.c file. New header file srat.h is introduced. Signed-off-by: Vijaya Kumar --- xen/arch/x86/domain_build.c | 1 + xen/arch/x86/numa.c | 1 + xen/arch/x86/physdev.c | 1 + xen/arch/x86/setup.c | 1 + xen/arch/x86/smpboot.c | 1 + xen/arch/x86/srat.c | 129 +------------------------------ xen/arch/x86/x86_64/mm.c | 1 + xen/common/Makefile | 1 + xen/common/srat.c | 150 ++++++++++++++++++++++++++++++++++++ xen/drivers/passthrough/vtd/iommu.c | 1 + xen/include/asm-x86/numa.h | 2 - xen/include/xen/numa.h | 1 - xen/include/xen/srat.h | 13 ++++ 13 files changed, 173 insertions(+), 130 deletions(-) diff --git a/xen/arch/x86/domain_build.c b/xen/arch/x86/domain_build.c index 243df96..4d7795b 100644 --- a/xen/arch/x86/domain_build.c +++ b/xen/arch/x86/domain_build.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c index 28d1891..58de324 100644 --- a/xen/arch/x86/numa.c +++ b/xen/arch/x86/numa.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c index 5a49796..2184d62 100644 --- a/xen/arch/x86/physdev.c +++ b/xen/arch/x86/physdev.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 176ee74..ee7a368 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #include diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index 9b390b8..6c61114 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include #include diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c index 58dee09..af12e26 100644 --- a/xen/arch/x86/srat.c +++ b/xen/arch/x86/srat.c @@ -18,91 +18,20 @@ #include #include #include +#include #include #include -static struct acpi_table_slit *__read_mostly acpi_slit; +extern struct acpi_table_slit *__read_mostly acpi_slit; static nodemask_t memory_nodes_parsed __initdata; static nodemask_t processor_nodes_parsed __initdata; extern struct node nodes[MAX_NUMNODES] __initdata; - -struct pxm2node { - unsigned pxm; - nodeid_t node; -}; -static struct pxm2node __read_mostly pxm2node[MAX_NUMNODES] = - { [0 ... MAX_NUMNODES - 1] = {.node = NUMA_NO_NODE} }; - -static unsigned node_to_pxm(nodeid_t n); - extern int num_node_memblks; extern struct node node_memblk_range[NR_NODE_MEMBLKS]; extern nodeid_t memblk_nodeid[NR_NODE_MEMBLKS]; static __initdata DECLARE_BITMAP(memblk_hotplug, NR_NODE_MEMBLKS); -static inline bool_t node_found(unsigned idx, unsigned pxm) -{ - return ((pxm2node[idx].pxm == pxm) && - (pxm2node[idx].node != NUMA_NO_NODE)); -} - -nodeid_t pxm_to_node(unsigned pxm) -{ - unsigned i; - - if ((pxm < ARRAY_SIZE(pxm2node)) && node_found(pxm, pxm)) - return pxm2node[pxm].node; - - for (i = 0; i < ARRAY_SIZE(pxm2node); i++) - if (node_found(i, pxm)) - return pxm2node[i].node; - - return NUMA_NO_NODE; -} - -nodeid_t setup_node(unsigned pxm) -{ - nodeid_t node; - unsigned idx; - static bool_t warned; - static unsigned nodes_found; - - BUILD_BUG_ON(MAX_NUMNODES >= NUMA_NO_NODE); - - if (pxm < ARRAY_SIZE(pxm2node)) { - if (node_found(pxm, pxm)) - return pxm2node[pxm].node; - - /* Try to maintain indexing of pxm2node by pxm */ - if (pxm2node[pxm].node == NUMA_NO_NODE) { - idx = pxm; - goto finish; - } - } - - for (idx = 0; idx < ARRAY_SIZE(pxm2node); idx++) - if (pxm2node[idx].node == NUMA_NO_NODE) - goto finish; - - if (!warned) { - printk(KERN_WARNING "SRAT: Too many proximity domains (%#x)\n", - pxm); - warned = 1; - } - - return NUMA_NO_NODE; - - finish: - node = nodes_found++; - if (node >= MAX_NUMNODES) - return NUMA_NO_NODE; - pxm2node[idx].pxm = pxm; - pxm2node[idx].node = node; - - return node; -} - static __init void bad_srat(void) { int i; @@ -115,48 +44,6 @@ static __init void bad_srat(void) mem_hotplug = 0; } -/* - * A lot of BIOS fill in 10 (= no distance) everywhere. This messes - * up the NUMA heuristics which wants the local node to have a smaller - * distance than the others. - * Do some quick checks here and only use the SLIT if it passes. - */ -static __init int slit_valid(struct acpi_table_slit *slit) -{ - int i, j; - int d = slit->locality_count; - for (i = 0; i < d; i++) { - for (j = 0; j < d; j++) { - u8 val = slit->entry[d*i + j]; - if (i == j) { - if (val != 10) - return 0; - } else if (val <= 10) - return 0; - } - } - return 1; -} - -/* Callback for SLIT parsing */ -void __init acpi_numa_slit_init(struct acpi_table_slit *slit) -{ - unsigned long mfn; - if (!slit_valid(slit)) { - printk(KERN_INFO "ACPI: SLIT table looks invalid. " - "Not used.\n"); - return; - } - mfn = alloc_boot_pages(PFN_UP(slit->header.length), 1); - if (!mfn) { - printk(KERN_ERR "ACPI: Unable to allocate memory for " - "saving ACPI SLIT numa information.\n"); - return; - } - acpi_slit = mfn_to_virt(mfn); - memcpy(acpi_slit, slit, slit->header.length); -} - /* Callback for Proximity Domain -> x2APIC mapping */ void __init acpi_numa_x2apic_affinity_init(const struct acpi_srat_x2apic_cpu_affinity *pa) @@ -456,18 +343,6 @@ int __init acpi_scan_nodes(u64 start, u64 end) return 0; } -static unsigned node_to_pxm(nodeid_t n) -{ - unsigned i; - - if ((n < ARRAY_SIZE(pxm2node)) && (pxm2node[n].node == n)) - return pxm2node[n].pxm; - for (i = 0; i < ARRAY_SIZE(pxm2node); i++) - if (pxm2node[i].node == n) - return pxm2node[i].pxm; - return 0; -} - u8 __node_distance(nodeid_t a, nodeid_t b) { unsigned index; diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c index 9ead02e..f823fb3 100644 --- a/xen/arch/x86/x86_64/mm.c +++ b/xen/arch/x86/x86_64/mm.c @@ -27,6 +27,7 @@ asm(".file \"" __FILE__ "\""); #include #include #include +#include #include #include #include diff --git a/xen/common/Makefile b/xen/common/Makefile index c1bd2ff..a668094 100644 --- a/xen/common/Makefile +++ b/xen/common/Makefile @@ -64,6 +64,7 @@ obj-bin-y += warning.init.o obj-$(CONFIG_XENOPROF) += xenoprof.o obj-y += xmalloc_tlsf.o obj-y += numa.o +obj-y += srat.o obj-bin-$(CONFIG_X86) += $(foreach n,decompress bunzip2 unxz unlzma unlzo unlz4 earlycpio,$(n).init.o) diff --git a/xen/common/srat.c b/xen/common/srat.c new file mode 100644 index 0000000..cf50c78 --- /dev/null +++ b/xen/common/srat.c @@ -0,0 +1,150 @@ +/* + * ACPI 3.0 based NUMA setup + * Copyright 2004 Andi Kleen, SuSE Labs. + * + * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs. + * + * Called from acpi_numa_init while reading the SRAT and SLIT tables. + * Assumes all memory regions belonging to a single proximity domain + * are in one chunk. Holes between them will be included in the node. + * + * Adapted for Xen: Ryan Harper + * + * Moved this generic code from xen/arch/x86/srat.c for other arch usage + * by Vijaya Kumar K + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct acpi_table_slit *__read_mostly acpi_slit; +extern struct node nodes[MAX_NUMNODES] __initdata; + +struct pxm2node __read_mostly pxm2node[MAX_NUMNODES] = + { [0 ... MAX_NUMNODES - 1] = {.node = NUMA_NO_NODE} }; + +static inline bool_t node_found(unsigned idx, unsigned pxm) +{ + return ((pxm2node[idx].pxm == pxm) && + (pxm2node[idx].node != NUMA_NO_NODE)); +} + +nodeid_t pxm_to_node(unsigned pxm) +{ + unsigned i; + + if ( (pxm < ARRAY_SIZE(pxm2node)) && node_found(pxm, pxm) ) + return pxm2node[pxm].node; + + for ( i = 0; i < ARRAY_SIZE(pxm2node); i++ ) + if ( node_found(i, pxm) ) + return pxm2node[i].node; + + return NUMA_NO_NODE; +} + +nodeid_t setup_node(unsigned pxm) +{ + nodeid_t node; + unsigned idx; + static bool_t warned; + static unsigned nodes_found; + + BUILD_BUG_ON(MAX_NUMNODES >= NUMA_NO_NODE); + + if ( pxm < ARRAY_SIZE(pxm2node) ) { + if (node_found(pxm, pxm)) + return pxm2node[pxm].node; + + /* Try to maintain indexing of pxm2node by pxm */ + if ( pxm2node[pxm].node == NUMA_NO_NODE ) { + idx = pxm; + goto finish; + } + } + + for ( idx = 0; idx < ARRAY_SIZE(pxm2node); idx++ ) + if ( pxm2node[idx].node == NUMA_NO_NODE ) + goto finish; + + if ( !warned ) { + printk(KERN_WARNING "SRAT: Too many proximity domains (%#x)\n", + pxm); + warned = 1; + } + + return NUMA_NO_NODE; + + finish: + node = nodes_found++; + if (node >= MAX_NUMNODES) + return NUMA_NO_NODE; + pxm2node[idx].pxm = pxm; + pxm2node[idx].node = node; + + return node; +} + +/* + * A lot of BIOS fill in 10 (= no distance) everywhere. This messes + * up the NUMA heuristics which wants the local node to have a smaller + * distance than the others. + * Do some quick checks here and only use the SLIT if it passes. + */ +static __init int slit_valid(struct acpi_table_slit *slit) +{ + int i, j; + int d = slit->locality_count; + + for ( i = 0; i < d; i++ ) { + for ( j = 0; j < d; j++ ) { + u8 val = slit->entry[d*i + j]; + if ( i == j ) { + if (val != 10) + return 0; + } else if ( val <= 10 ) + return 0; + } + } + + return 1; +} + +/* Callback for SLIT parsing */ +void __init acpi_numa_slit_init(struct acpi_table_slit *slit) +{ + unsigned long mfn; + + if ( !slit_valid(slit) ) { + printk(KERN_INFO "ACPI: SLIT table looks invalid. " + "Not used.\n"); + return; + } + mfn = alloc_boot_pages(PFN_UP(slit->header.length), 1); + if ( !mfn ) { + printk(KERN_ERR "ACPI: Unable to allocate memory for " + "saving ACPI SLIT numa information.\n"); + return; + } + acpi_slit = mfn_to_virt(mfn); + memcpy(acpi_slit, slit, slit->header.length); +} + +unsigned node_to_pxm(nodeid_t n) +{ + unsigned i; + + if ( (n < ARRAY_SIZE(pxm2node)) && (pxm2node[n].node == n) ) + return pxm2node[n].pxm; + for ( i = 0; i < ARRAY_SIZE(pxm2node); i++ ) + if ( pxm2node[i].node == n ) + return pxm2node[i].pxm; + return 0; +} diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c index a5c61c6..7a4463d 100644 --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -30,6 +30,7 @@ #include #include #include +#include #include #include #include diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h index 659ff6a..79a445c 100644 --- a/xen/include/asm-x86/numa.h +++ b/xen/include/asm-x86/numa.h @@ -17,8 +17,6 @@ extern cpumask_t node_to_cpumask[]; #define node_to_first_cpu(node) (__ffs(node_to_cpumask[node])) #define node_to_cpumask(node) (node_to_cpumask[node]) -extern nodeid_t pxm_to_node(unsigned int pxm); - #define ZONE_ALIGN (1UL << (MAX_ORDER+PAGE_SHIFT)) extern void numa_init_array(void); diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h index 4f04ab4..eb18380 100644 --- a/xen/include/xen/numa.h +++ b/xen/include/xen/numa.h @@ -73,7 +73,6 @@ extern int valid_numa_range(u64 start, u64 end, nodeid_t node); extern int conflicting_memblks(u64 start, u64 end); extern void cutoff_node(int i, u64 start, u64 end); extern void numa_add_cpu(int cpu); -extern nodeid_t setup_node(unsigned int pxm); extern void numa_set_node(int cpu, nodeid_t node); extern void setup_node_bootmem(nodeid_t nodeid, u64 start, u64 end); extern int compute_hash_shift(struct node *nodes, int numnodes, diff --git a/xen/include/xen/srat.h b/xen/include/xen/srat.h new file mode 100644 index 0000000..978f1e8 --- /dev/null +++ b/xen/include/xen/srat.h @@ -0,0 +1,13 @@ +#ifndef __XEN_SRAT_H__ +#define __XEN_SRAT_H__ + +struct pxm2node { + unsigned pxm; + nodeid_t node; +}; + +extern struct pxm2node __read_mostly pxm2node[MAX_NUMNODES]; +nodeid_t pxm_to_node(unsigned pxm); +nodeid_t setup_node(unsigned pxm); +unsigned node_to_pxm(nodeid_t n); +#endif /* __XEN_SRAT_H__ */