From patchwork Thu Nov 7 03:57:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 11231847 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6003D1575 for ; Thu, 7 Nov 2019 04:12:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1E550222C9 for ; Thu, 7 Nov 2019 04:12:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1E550222C9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 46C786B027A; Wed, 6 Nov 2019 23:12:12 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 41D806B027B; Wed, 6 Nov 2019 23:12:12 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3348B6B027C; Wed, 6 Nov 2019 23:12:12 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0250.hostedemail.com [216.40.44.250]) by kanga.kvack.org (Postfix) with ESMTP id 193D76B027A for ; Wed, 6 Nov 2019 23:12:12 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id B65FC2497 for ; Thu, 7 Nov 2019 04:12:11 +0000 (UTC) X-FDA: 76128158862.30.jail18_877ef2858740e X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,dan.j.williams@intel.com,:linux-nvdimm@lists.01.org:dave.hansen@linux.intel.com:luto@kernel.org:peterz@infradead.org:tglx@linutronix.de:mingo@redhat.com:bp@alien8.de:hpa@zytor.com:x86@kernel.org:akpm@linux-foundation.org:david@redhat.com:mhocko@suse.com:vishal.l.verma@intel.com:linux-kernel@vger.kernel.org::dan.j.williams@intel.com,RULES_HIT:30054:30064:30070:30075:30080:30090,0,RBL:192.55.52.88:@intel.com:.lbl8.mailshell.net-62.18.0.100 64.95.201.95,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: jail18_877ef2858740e X-Filterd-Recvd-Size: 7030 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by imf10.hostedemail.com (Postfix) with ESMTP for ; Thu, 7 Nov 2019 04:12:10 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Nov 2019 20:12:09 -0800 X-IronPort-AV: E=Sophos;i="5.68,276,1569308400"; d="scan'208";a="196430523" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.16]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Nov 2019 20:12:09 -0800 Subject: [PATCH 14/16] x86/numa: Provide a range-to-target_node lookup facility From: Dan Williams To: linux-nvdimm@lists.01.org Cc: Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org, Andrew Morton , David Hildenbrand , Michal Hocko , vishal.l.verma@intel.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Date: Wed, 06 Nov 2019 19:57:53 -0800 Message-ID: <157309907296.1582359.7986676987778026949.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <157309899529.1582359.15358067933360719580.stgit@dwillia2-desk3.amr.corp.intel.com> References: <157309899529.1582359.15358067933360719580.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-2-gc94f MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The DEV_DAX_KMEM facility is a generic mechanism to allow device-dax instances, fronting performance-differentiated-memory like pmem, to be added to the System RAM pool. The numa node for that hot-added memory is derived from the device-dax instance's 'target_node' attribute. Recall that the 'target_node' is the ACPI-PXM-to-node translation for memory when it comes online whereas the 'numa_node' attribute of the device represents the closest online cpu node. Presently useful target_node information from the ACPI SRAT is discarded with the expectation that "Reserved" memory will never be onlined. Now, DEV_DAX_KMEM violates that assumption, there is a need to retain the translation. Move, rather than discard, numa_memblk data to a secondary array that memory_add_physaddr_to_target_node() may consider at a later point in time. Cc: Dave Hansen Cc: Andy Lutomirski Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: Cc: Andrew Morton Cc: David Hildenbrand Cc: Michal Hocko Signed-off-by: Dan Williams --- arch/x86/mm/numa.c | 72 +++++++++++++++++++++++++++++++++++++--- include/linux/memory_hotplug.h | 6 +++ 2 files changed, 73 insertions(+), 5 deletions(-) diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c index 4123100e0eaf..3bbae90b3197 100644 --- a/arch/x86/mm/numa.c +++ b/arch/x86/mm/numa.c @@ -31,6 +31,20 @@ __initdata #endif ; +#if IS_ENABLED(CONFIG_DEV_DAX_KMEM) +static struct numa_meminfo __numa_reserved_meminfo; + +static struct numa_meminfo *numa_reserved_meminfo(void) +{ + return &__numa_reserved_meminfo; +} +#else +static struct numa_meminfo *numa_reserved_meminfo(void) +{ + return NULL; +} +#endif + static int numa_distance_cnt; static u8 *numa_distance; @@ -168,6 +182,26 @@ void __init numa_remove_memblk_from(int idx, struct numa_meminfo *mi) (mi->nr_blks - idx) * sizeof(mi->blk[0])); } +/** + * numa_move_memblk - Move one numa_memblk from one numa_meminfo to another + * @dst: numa_meminfo to move block to + * @idx: Index of memblk to remove + * @src: numa_meminfo to remove memblk from + * + * If @dst is non-NULL add it at the @dst->nr_blks index and increment + * @dst->nr_blks, then remove it from @src. + */ +void __init numa_move_memblk(struct numa_meminfo *dst, int idx, + struct numa_meminfo *src) +{ + if (dst) { + memcpy(&dst->blk[dst->nr_blks], &src->blk[idx], + sizeof(struct numa_memblk)); + dst->nr_blks++; + } + numa_remove_memblk_from(idx, src); +} + /** * numa_add_memblk - Add one numa_memblk to numa_meminfo * @nid: NUMA node ID of the new memblk @@ -245,7 +279,7 @@ int __init numa_cleanup_meminfo(struct numa_meminfo *mi) if (bi->start >= bi->end || !memblock_overlaps_region(&memblock.memory, bi->start, bi->end - bi->start)) - numa_remove_memblk_from(i--, mi); + numa_move_memblk(numa_reserved_meminfo(), i--, mi); } /* merge neighboring / overlapping entries */ @@ -882,15 +916,43 @@ EXPORT_SYMBOL(cpumask_of_node); #endif /* !CONFIG_DEBUG_PER_CPU_MAPS */ #ifdef CONFIG_MEMORY_HOTPLUG +static int meminfo_to_nid(struct numa_meminfo *mi, u64 start, int *nid) +{ + int i; + + for (i = 0; mi && i < mi->nr_blks; i++) + if (mi->blk[i].start <= start && mi->blk[i].end > start) { + *nid = mi->blk[i].nid; + break; + } + return i; +} + +int memory_add_physaddr_to_target_node(u64 start) +{ + struct numa_meminfo *mi = &numa_meminfo; + int nid = mi->blk[0].nid; + int i = meminfo_to_nid(mi, start, &nid); + + /* + * Prefer online nodes, but if reserved memory might be + * hot-added continue the search with reserved ranges. + */ + if (i < mi->nr_blks) + return nid; + + mi = numa_reserved_meminfo(); + meminfo_to_nid(mi, start, &nid); + return nid; +} +EXPORT_SYMBOL_GPL(memory_add_physaddr_to_target_node); + int memory_add_physaddr_to_nid(u64 start) { struct numa_meminfo *mi = &numa_meminfo; int nid = mi->blk[0].nid; - int i; - for (i = 0; i < mi->nr_blks; i++) - if (mi->blk[i].start <= start && mi->blk[i].end > start) - nid = mi->blk[i].nid; + meminfo_to_nid(mi, start, &nid); return nid; } EXPORT_SYMBOL_GPL(memory_add_physaddr_to_nid); diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h index f46ea71b4ffd..84efb0f20f7e 100644 --- a/include/linux/memory_hotplug.h +++ b/include/linux/memory_hotplug.h @@ -145,11 +145,17 @@ int add_pages(int nid, unsigned long start_pfn, unsigned long nr_pages, #ifdef CONFIG_NUMA extern int memory_add_physaddr_to_nid(u64 start); +extern int memory_add_physaddr_to_target_node(u64 start); #else static inline int memory_add_physaddr_to_nid(u64 start) { return 0; } + +static inline int memory_add_physaddr_to_target_node(u64 start) +{ + return 0; +} #endif #ifdef CONFIG_HAVE_ARCH_NODEDATA_EXTENSION