From patchwork Sun Nov 17 17:46:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 11248591 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 68661930 for ; Sun, 17 Nov 2019 18:00:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 366E92075C for ; Sun, 17 Nov 2019 18:00:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 366E92075C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 570856B0281; Sun, 17 Nov 2019 13:00:26 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 547896B0282; Sun, 17 Nov 2019 13:00:26 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 45DC46B0283; Sun, 17 Nov 2019 13:00:26 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0246.hostedemail.com [216.40.44.246]) by kanga.kvack.org (Postfix) with ESMTP id 2E5276B0281 for ; Sun, 17 Nov 2019 13:00:26 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id DB1135839 for ; Sun, 17 Nov 2019 18:00:25 +0000 (UTC) X-FDA: 76166534010.30.alarm51_5afce0b8f2f5c X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,dan.j.williams@intel.com,:linux-nvdimm@lists.01.org:dave.hansen@linux.intel.com:luto@kernel.org:peterz@infradead.org:tglx@linutronix.de:mingo@redhat.com:bp@alien8.de:hpa@zytor.com:x86@kernel.org:akpm@linux-foundation.org:david@redhat.com:mhocko@suse.com:lkp@intel.com:vishal.l.verma@intel.com:hch@lst.de:linux-kernel@vger.kernel.org::linux-acpi@vger.kernel.org:dan.j.williams@intel.com,RULES_HIT:30054:30064:30070:30075:30080:30090,0,RBL:134.134.136.100:@intel.com:.lbl8.mailshell.net-62.18.0.100 64.95.201.95,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: alarm51_5afce0b8f2f5c X-Filterd-Recvd-Size: 8157 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Sun, 17 Nov 2019 18:00:25 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Nov 2019 10:00:23 -0800 X-IronPort-AV: E=Sophos;i="5.68,317,1569308400"; d="scan'208";a="195907454" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.16]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Nov 2019 10:00:23 -0800 Subject: [PATCH v2 17/18] x86/numa: Provide a range-to-target_node lookup facility From: Dan Williams To: linux-nvdimm@lists.01.org Cc: Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org, Andrew Morton , David Hildenbrand , Michal Hocko , kbuild test robot , vishal.l.verma@intel.com, hch@lst.de, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 17 Nov 2019 09:46:07 -0800 Message-ID: <157401276776.43284.12396353118982684546.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <157401267421.43284.2135775608523385279.stgit@dwillia2-desk3.amr.corp.intel.com> References: <157401267421.43284.2135775608523385279.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The DEV_DAX_KMEM facility is a generic mechanism to allow device-dax instances, fronting performance-differentiated-memory like pmem, to be added to the System RAM pool. The numa node for that hot-added memory is derived from the device-dax instance's 'target_node' attribute. Recall that the 'target_node' is the ACPI-PXM-to-node translation for memory when it comes online whereas the 'numa_node' attribute of the device represents the closest online cpu node. Presently useful target_node information from the ACPI SRAT is discarded with the expectation that "Reserved" memory will never be onlined. Now, DEV_DAX_KMEM violates that assumption, there is a need to retain the translation. Move, rather than discard, numa_memblk data to a secondary array that memory_add_physaddr_to_target_node() may consider at a later point in time. Note that memory_add_physaddr_to_nid() is currently only available on CONFIG_MEMORY_HOTPLUG=y platforms whereas the target node information may be useful on CONFIG_MEMORY_HOTPLUG=n builds, hence why it is calling phys_to_target_node() and optionally defined by asm/io.h rather than a memory_add_physaddr_to_target_nid() helper that lives in include/linux/memory_hotplug.h. Cc: Dave Hansen Cc: Andy Lutomirski Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: Cc: Andrew Morton Cc: David Hildenbrand Cc: Michal Hocko Reported-by: kbuild test robot Signed-off-by: Dan Williams --- arch/x86/mm/numa.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++--- include/linux/numa.h | 8 +++++ mm/mempolicy.c | 5 +++ 3 files changed, 83 insertions(+), 6 deletions(-) diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c index 4123100e0eaf..f4f02ac0c465 100644 --- a/arch/x86/mm/numa.c +++ b/arch/x86/mm/numa.c @@ -31,6 +31,24 @@ __initdata #endif ; +/* + * Presently, DEV_DAX_KMEM is the only kernel facility that might + * convert Reserved or Soft Reserved memory to System RAM. + */ +#if IS_ENABLED(CONFIG_DEV_DAX_KMEM) +static struct numa_meminfo __numa_reserved_meminfo; + +static struct numa_meminfo *numa_reserved_meminfo(void) +{ + return &__numa_reserved_meminfo; +} +#else +static struct numa_meminfo *numa_reserved_meminfo(void) +{ + return NULL; +} +#endif + static int numa_distance_cnt; static u8 *numa_distance; @@ -168,6 +186,26 @@ void __init numa_remove_memblk_from(int idx, struct numa_meminfo *mi) (mi->nr_blks - idx) * sizeof(mi->blk[0])); } +/** + * numa_move_memblk - Move one numa_memblk from one numa_meminfo to another + * @dst: numa_meminfo to move block to + * @idx: Index of memblk to remove + * @src: numa_meminfo to remove memblk from + * + * If @dst is non-NULL add it at the @dst->nr_blks index and increment + * @dst->nr_blks, then remove it from @src. + */ +static void __init numa_move_memblk(struct numa_meminfo *dst, int idx, + struct numa_meminfo *src) +{ + if (dst) { + memcpy(&dst->blk[dst->nr_blks], &src->blk[idx], + sizeof(struct numa_memblk)); + dst->nr_blks++; + } + numa_remove_memblk_from(idx, src); +} + /** * numa_add_memblk - Add one numa_memblk to numa_meminfo * @nid: NUMA node ID of the new memblk @@ -245,7 +283,7 @@ int __init numa_cleanup_meminfo(struct numa_meminfo *mi) if (bi->start >= bi->end || !memblock_overlaps_region(&memblock.memory, bi->start, bi->end - bi->start)) - numa_remove_memblk_from(i--, mi); + numa_move_memblk(numa_reserved_meminfo(), i--, mi); } /* merge neighboring / overlapping entries */ @@ -881,16 +919,44 @@ EXPORT_SYMBOL(cpumask_of_node); #endif /* !CONFIG_DEBUG_PER_CPU_MAPS */ +static int meminfo_to_nid(struct numa_meminfo *mi, u64 start, int *nid) +{ + int i; + + for (i = 0; mi && i < mi->nr_blks; i++) + if (mi->blk[i].start <= start && mi->blk[i].end > start) { + *nid = mi->blk[i].nid; + break; + } + return i; +} + +int phys_to_target_node(phys_addr_t start) +{ + struct numa_meminfo *mi = &numa_meminfo; + int nid = mi->blk[0].nid; + int i = meminfo_to_nid(mi, start, &nid); + + /* + * Prefer online nodes, but if reserved memory might be + * hot-added continue the search with reserved ranges. + */ + if (i < mi->nr_blks) + return nid; + + mi = numa_reserved_meminfo(); + meminfo_to_nid(mi, start, &nid); + return nid; +} +EXPORT_SYMBOL_GPL(phys_to_target_node); + #ifdef CONFIG_MEMORY_HOTPLUG int memory_add_physaddr_to_nid(u64 start) { struct numa_meminfo *mi = &numa_meminfo; int nid = mi->blk[0].nid; - int i; - for (i = 0; i < mi->nr_blks; i++) - if (mi->blk[i].start <= start && mi->blk[i].end > start) - nid = mi->blk[i].nid; + meminfo_to_nid(mi, start, &nid); return nid; } EXPORT_SYMBOL_GPL(memory_add_physaddr_to_nid); diff --git a/include/linux/numa.h b/include/linux/numa.h index 20f4e44b186c..941790a0765b 100644 --- a/include/linux/numa.h +++ b/include/linux/numa.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ #ifndef _LINUX_NUMA_H #define _LINUX_NUMA_H - +#include #ifdef CONFIG_NODES_SHIFT #define NODES_SHIFT CONFIG_NODES_SHIFT @@ -15,11 +15,17 @@ #ifdef CONFIG_NUMA int numa_map_to_online_node(int node); +int phys_to_target_node(phys_addr_t addr); #else static inline int numa_map_to_online_node(int node) { return NUMA_NO_NODE; } + +static inline int phys_to_target_node(phys_addr_t addr) +{ + return NUMA_NO_NODE; +} #endif #endif /* _LINUX_NUMA_H */ diff --git a/mm/mempolicy.c b/mm/mempolicy.c index d618121bcc17..0db8b446e23e 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2996,3 +2996,8 @@ void mpol_to_str(char *buffer, int maxlen, struct mempolicy *pol) p += scnprintf(p, buffer + maxlen - p, ":%*pbl", nodemask_pr_args(&nodes)); } + +__weak int phys_to_target_node(phys_addr_t addr) +{ + return NUMA_NO_NODE; +}