From patchwork Thu Feb 9 15:57:00 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vijay Kilari X-Patchwork-Id: 9565015 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 139066020C for ; Thu, 9 Feb 2017 16:01:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 00D30237A5 for ; Thu, 9 Feb 2017 16:01:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E810127F81; Thu, 9 Feb 2017 16:01:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E4344237A5 for ; Thu, 9 Feb 2017 16:01:48 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cbr87-00080W-LQ; Thu, 09 Feb 2017 15:59:31 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cbr86-0007zM-Ec for xen-devel@lists.xenproject.org; Thu, 09 Feb 2017 15:59:30 +0000 Received: from [193.109.254.147] by server-10.bemta-6.messagelabs.com id 83/DB-13192-1E19C985; Thu, 09 Feb 2017 15:59:29 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprLIsWRWlGSWpSXmKPExsXiVRvsrPtw4pw Ig73PWCy+b5nM5MDocfjDFZYAxijWzLyk/IoE1ozOC78YC25qVux7846lgfGfbBcjB4eQwDRG iSazLkYuDhaBlywSLzY8YAFxJAT6WSWWfnnF1sXICeTESPy/sYwFpEFCoFJi+hUlkLCQgKpEx /+37CD1QgI/GCWezFjOCpJgE5CROPL8ABOILSKQJnFs9VqwOcwCvhIvF01kBrGFBTwkZi3vA6 thARq0fdYFdhCbV8BV4sC7dWwQuxQk5kyyAQlzCrhJLN9ziQVir6vEg+421gmMAgsYGVYxahS nFpWlFukaWeglFWWmZ5TkJmbm6BoamOnlphYXJ6an5iQmFesl5+duYgQGFQMQ7GA8vzbwEKMk B5OSKK9swZwIIb6k/JTKjMTijPii0pzU4kOMMhwcShK8KyYA5QSLUtNTK9Iyc4DhDZOW4OBRE uH9C5LmLS5IzC3OTIdInWLU5di16/JLJiGWvPy8VClxXm5gtAgJgBRllObBjYDF2iVGWSlhXk ago4R4ClKLcjNLUOVfMYpzMCoJ894FWcWTmVcCt+kV0BFMQEdcPz0L5IiSRISUVAMj58bHNau LHv4KshB/7Sxl+ctEedraDr/fCztn8i2aVb9V8ttSzVf/9bYcb8opldf93bFuou52h6cLNqxr bJd49CVrv4+ri/IxFa7Pa62UP9ip/d6pXhC7dveN0xUmC5jX6s3fwnzy1SPzFWvPvfk5559W0 JcVIZ8PHvLjdYv3NWrzMe1yC1I8q8RSnJFoqMVcVJwIAFdNw9KwAgAA X-Env-Sender: vijay.kilari@gmail.com X-Msg-Ref: server-9.tower-27.messagelabs.com!1486655967!86361537!1 X-Originating-IP: [74.125.83.67] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.1.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 3260 invoked from network); 9 Feb 2017 15:59:28 -0000 Received: from mail-pg0-f67.google.com (HELO mail-pg0-f67.google.com) (74.125.83.67) by server-9.tower-27.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 9 Feb 2017 15:59:28 -0000 Received: by mail-pg0-f67.google.com with SMTP id 194so629096pgd.0 for ; Thu, 09 Feb 2017 07:59:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=50RF+khugPpBVOrjrj+IuDGV00MHYELNz8sm6NHlUwk=; b=GCjX6pkfQjC9bHCv43Ad1UTUyimRE6zRlxLzmzJdHLmZPPTHzAPyKW4pSq8S5GXkZD SOYTriOUC7iGo+1ZJ1kLsyCWh2n8JDYExv80tSppiqxXIUa8IqHyQSWlPksDkSjXe7Uf xywjFYqFUn/1++6dJRcLia2J0PPJL9HUApnRcNC1Grwsbt22E6sLsqH7L9psD8y9OCjJ 1sUesbohm6fEsYnvwUyMluX8eVbayuNIF5L0OoyLgkjkV/tHJjjzctyuf/rsZ8FRg+9P S5zs2Vx37NN2vUgw02iCMbMWRwV7ysH8aSHSOHxxbS/Ma+skNanEFtoW4rJrBnCO629h 7vBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=50RF+khugPpBVOrjrj+IuDGV00MHYELNz8sm6NHlUwk=; b=q1gIaJZd6b03ZxiLNpJTAXnMf7GIpchjaBnas+e26xMkmmRBPEXVu/9uJVGofO3pnv pjsqcx/o8ppTcAdm8Nr9hXiLaPwIU8ejpBv7Lbsm4fBc7ty8PqpHug6DZ4wXXljUs7Mt XE8YW8LFROWwgwvt2jWBFSqUdGEfNiwkyJewGaI8Go9oTJwoC6bxuxD7xRtlnafkwnqT Hx9BL/0+WPAUFV77Xg/oLb7ZmLDtMq3SYTPsaaoP2iHIFDC2Z9KLlcbMXD30j0KDp10V UYtGYnrJ33DmH18TqtdnWf7MuLYYW7Hj70R9XZ3Qd0watK0gZEINO+n8eMBzYST2EL4h tpKw== X-Gm-Message-State: AMke39nrpn/2REWOuaCXdq8R8QTdRebBdvXUlb8qD0Wu5a++tTCJCIWDJZc63fC7iWrgrA== X-Received: by 10.84.233.136 with SMTP id l8mr5083124plk.169.1486655967212; Thu, 09 Feb 2017 07:59:27 -0800 (PST) Received: from localhost.localdomain ([14.140.2.178]) by smtp.gmail.com with ESMTPSA id m29sm30093842pfi.54.2017.02.09.07.59.24 (version=TLS1_1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 09 Feb 2017 07:59:26 -0800 (PST) From: vijay.kilari@gmail.com To: julien.grall@arm.com, sstabellini@kernel.org, andre.przywara@arm.com, dario.faggioli@citrix.com Date: Thu, 9 Feb 2017 21:27:00 +0530 Message-Id: <1486655834-9708-9-git-send-email-vijay.kilari@gmail.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1486655834-9708-1-git-send-email-vijay.kilari@gmail.com> References: <1486655834-9708-1-git-send-email-vijay.kilari@gmail.com> Cc: xen-devel@lists.xenproject.org, Vijaya Kumar K Subject: [Xen-devel] [RFC PATCH v1 08/21] ARM: NUMA: Parse NUMA distance information X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Vijaya Kumar K Parse distance-matrix and fetch node distance information. Store distance information in node_distance[]. Signed-off-by: Vijaya Kumar K --- xen/arch/arm/dt_numa.c | 90 ++++++++++++++++++++++++++++++++++++++++++++++ xen/arch/arm/numa.c | 19 +++++++++- xen/include/asm-arm/numa.h | 1 + 3 files changed, 109 insertions(+), 1 deletion(-) diff --git a/xen/arch/arm/dt_numa.c b/xen/arch/arm/dt_numa.c index fce9e67..8979612 100644 --- a/xen/arch/arm/dt_numa.c +++ b/xen/arch/arm/dt_numa.c @@ -28,6 +28,19 @@ nodemask_t numa_nodes_parsed; extern struct node node_memblk_range[NR_NODE_MEMBLKS]; +extern int _node_distance[MAX_NUMNODES * 2]; +extern int *node_distance; + +static int numa_set_distance(u32 nodea, u32 nodeb, u32 distance) +{ + if ( nodea >= MAX_NUMNODES || nodeb >= MAX_NUMNODES ) + return -EINVAL; + + _node_distance[(nodea * MAX_NUMNODES) + nodeb] = distance; + node_distance = &_node_distance[0]; + + return 0; +} /* * Even though we connect cpus to numa domains later in SMP @@ -112,6 +125,66 @@ static int __init dt_numa_process_memory_node(const void *fdt, int node, return 0; } +static int __init dt_numa_parse_distance_map(const void *fdt, int node, + const char *name, + u32 address_cells, + u32 size_cells) +{ + const struct fdt_property *prop; + const __be32 *matrix; + int entry_count, len, i; + + printk(XENLOG_INFO "NUMA: parsing numa-distance-map\n"); + + prop = fdt_get_property(fdt, node, "distance-matrix", &len); + if ( !prop ) + { + printk(XENLOG_WARNING + "NUMA: No distance-matrix property in distance-map\n"); + + return -EINVAL; + } + + if ( len % sizeof(u32) != 0 ) + { + printk(XENLOG_WARNING + "distance-matrix in node is not a multiple of u32\n"); + + return -EINVAL; + } + + entry_count = len / sizeof(u32); + if ( entry_count <= 0 ) + { + printk(XENLOG_WARNING "NUMA: Invalid distance-matrix\n"); + + return -EINVAL; + } + + matrix = (const __be32 *)prop->data; + for ( i = 0; i + 2 < entry_count; i += 3 ) + { + u32 nodea, nodeb, distance; + + nodea = dt_read_number(matrix, 1); + matrix++; + nodeb = dt_read_number(matrix, 1); + matrix++; + distance = dt_read_number(matrix, 1); + matrix++; + + numa_set_distance(nodea, nodeb, distance); + printk(XENLOG_INFO "NUMA: distance[node%d -> node%d] = %d\n", + nodea, nodeb, distance); + + /* Set default distance of node B->A same as A->B */ + if ( nodeb > nodea ) + numa_set_distance(nodeb, nodea, distance); + } + + return 0; +} + static int __init dt_numa_scan_cpu_node(const void *fdt, int node, const char *name, int depth, u32 address_cells, u32 size_cells, @@ -136,6 +209,18 @@ static int __init dt_numa_scan_memory_node(const void *fdt, int node, return 0; } +static int __init dt_numa_scan_distance_node(const void *fdt, int node, + const char *name, int depth, + u32 address_cells, u32 size_cells, + void *data) +{ + if ( device_tree_node_matches(fdt, node, "distance-map") ) + return dt_numa_parse_distance_map(fdt, node, name, address_cells, + size_cells); + + return 0; +} + int __init dt_numa_init(void) { int ret; @@ -149,6 +234,11 @@ int __init dt_numa_init(void) ret = device_tree_for_each_node((void *)device_tree_flattened, dt_numa_scan_memory_node, NULL); + if ( ret ) + return ret; + + ret = device_tree_for_each_node((void *)device_tree_flattened, + dt_numa_scan_distance_node, NULL); return ret; } diff --git a/xen/arch/arm/numa.c b/xen/arch/arm/numa.c index 9a7f0bb..11d100b 100644 --- a/xen/arch/arm/numa.c +++ b/xen/arch/arm/numa.c @@ -22,14 +22,31 @@ #include #include #include +#include + +int _node_distance[MAX_NUMNODES * 2]; +int *node_distance; + +u8 __node_distance(nodeid_t a, nodeid_t b) +{ + if ( !node_distance ) + return a == b ? 10 : 20; + + return _node_distance[a * MAX_NUMNODES + b]; +} + +EXPORT_SYMBOL(__node_distance); int __init numa_init(void) { - int ret = 0; + int i, ret = 0; if ( numa_off ) goto no_numa; + for ( i = 0; i < MAX_NUMNODES * 2; i++ ) + _node_distance[i] = 0; + ret = dt_numa_init(); no_numa: diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h index cdfeecd..b8857e2 100644 --- a/xen/include/asm-arm/numa.h +++ b/xen/include/asm-arm/numa.h @@ -11,6 +11,7 @@ typedef u8 nodeid_t; int arch_numa_setup(char *opt); int __init numa_init(void); int __init dt_numa_init(void); +u8 __node_distance(nodeid_t a, nodeid_t b); #else static inline int arch_numa_setup(char *opt) {