From patchwork Thu Jun 4 23:48:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11588661 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 363EC618 for ; Thu, 4 Jun 2020 23:48:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EAC58208C3 for ; Thu, 4 Jun 2020 23:48:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="hsh56h9w" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EAC58208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EE7D628002F; Thu, 4 Jun 2020 19:48:50 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E990B280005; Thu, 4 Jun 2020 19:48:50 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD65D28002F; Thu, 4 Jun 2020 19:48:50 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0225.hostedemail.com [216.40.44.225]) by kanga.kvack.org (Postfix) with ESMTP id C6553280005 for ; Thu, 4 Jun 2020 19:48:50 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 988D43749 for ; Thu, 4 Jun 2020 23:48:50 +0000 (UTC) X-FDA: 76893172020.15.rod90_29116c526d9b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin15.hostedemail.com (Postfix) with ESMTP id 7B3651814B0CC for ; Thu, 4 Jun 2020 23:48:50 +0000 (UTC) X-Spam-Summary: 2,0,0,caf75e1e4490ba4e,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:2:41:355:379:800:960:966:967:968:973:988:989:1260:1263:1345:1359:1381:1431:1437:1535:1605:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2563:2682:2685:2693:2859:2902:2903:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4050:4120:4250:4321:4385:4605:5007:6238:6261:6653:6737:7514:7576:7875:7903:8599:9025:9163:9545:10004:10913:11026:11473:11658:11914:12043:12048:12114:12291:12296:12297:12438:12517:12519:12555:12679:12783:12986:13149:13230:13846:13870:13972:14093:14096:21080:21220:21324:21325:21451:21627:21939:21987:21990:30003:30029:30054:30055:30064:30070,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:1,LU A_SUMMAR X-HE-Tag: rod90_29116c526d9b X-Filterd-Recvd-Size: 9112 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Thu, 4 Jun 2020 23:48:49 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C899720885; Thu, 4 Jun 2020 23:48:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1591314529; bh=jgkjQV/RHon6MiB9iB4bWs7LFX5/h1UYJaJPrFY0Z+I=; h=Date:From:To:Subject:In-Reply-To:From; b=hsh56h9wbl7evEjma1o6dkdSvwwXO7gSqhfYK4pa7kfp0T+ZeSHjE7IiY5mZCmSi7 aPXzSbnE/ZpEN4GrophLjLpgeydkKXslOHa51zaBvnWC+WPJjgIqN6dz9AQSLg3uVE T9m2QBXftmMguyaWmJCjFjeOVafRbAdKVce782So= Date: Thu, 04 Jun 2020 16:48:48 -0700 From: Andrew Morton To: akpm@linux-foundation.org, bhe@redhat.com, dan.j.williams@intel.com, dave.hansen@linux.intel.com, david@redhat.com, ebiederm@xmission.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, pankaj.gupta.linux@gmail.com, pasha.tatashin@soleen.com, richard.weiyang@gmail.com, torvalds@linux-foundation.org Subject: [patch 049/127] device-dax: add memory via add_memory_driver_managed() Message-ID: <20200604234848.vK7nX6kk1%akpm@linux-foundation.org> In-Reply-To: <20200604164523.e15f3177f4b69dcb4f2534a1@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Queue-Id: 7B3651814B0CC X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: device-dax: add memory via add_memory_driver_managed() Currently, when adding memory, we create entries in /sys/firmware/memmap/ as "System RAM". This will lead to kexec-tools to add that memory to the fixed-up initial memmap for a kexec kernel (loaded via kexec_load()). The memory will be considered initial System RAM by the kexec'd kernel and can no longer be reconfigured. This is not what happens during a real reboot. Let's add our memory via add_memory_driver_managed() now, so we won't create entries in /sys/firmware/memmap/ and indicate the memory as "System RAM (kmem)" in /proc/iomem. This allows everybody (especially kexec-tools) to identify that this memory is special and has to be treated differently than ordinary (hotplugged) System RAM. Before configuring the namespace: [root@localhost ~]# cat /proc/iomem ... 140000000-33fffffff : Persistent Memory 140000000-33fffffff : namespace0.0 3280000000-32ffffffff : PCI Bus 0000:00 After configuring the namespace: [root@localhost ~]# cat /proc/iomem ... 140000000-33fffffff : Persistent Memory 140000000-1481fffff : namespace0.0 148200000-33fffffff : dax0.0 3280000000-32ffffffff : PCI Bus 0000:00 After loading kmem before this change: [root@localhost ~]# cat /proc/iomem ... 140000000-33fffffff : Persistent Memory 140000000-1481fffff : namespace0.0 150000000-33fffffff : dax0.0 150000000-33fffffff : System RAM 3280000000-32ffffffff : PCI Bus 0000:00 After loading kmem after this change: [root@localhost ~]# cat /proc/iomem ... 140000000-33fffffff : Persistent Memory 140000000-1481fffff : namespace0.0 150000000-33fffffff : dax0.0 150000000-33fffffff : System RAM (kmem) 3280000000-32ffffffff : PCI Bus 0000:00 After a proper reboot: [root@localhost ~]# cat /proc/iomem ... 140000000-33fffffff : Persistent Memory 140000000-1481fffff : namespace0.0 148200000-33fffffff : dax0.0 3280000000-32ffffffff : PCI Bus 0000:00 Within the kexec kernel before this change: [root@localhost ~]# cat /proc/iomem ... 140000000-33fffffff : Persistent Memory 140000000-1481fffff : namespace0.0 150000000-33fffffff : System RAM 3280000000-32ffffffff : PCI Bus 0000:00 Within the kexec kernel after this change: [root@localhost ~]# cat /proc/iomem ... 140000000-33fffffff : Persistent Memory 140000000-1481fffff : namespace0.0 148200000-33fffffff : dax0.0 3280000000-32ffffffff : PCI Bus 0000:00 /sys/firmware/memmap/ before this change: 0000000000000000-000000000009fc00 (System RAM) 000000000009fc00-00000000000a0000 (Reserved) 00000000000f0000-0000000000100000 (Reserved) 0000000000100000-00000000bffdf000 (System RAM) 00000000bffdf000-00000000c0000000 (Reserved) 00000000feffc000-00000000ff000000 (Reserved) 00000000fffc0000-0000000100000000 (Reserved) 0000000100000000-0000000140000000 (System RAM) 0000000150000000-0000000340000000 (System RAM) /sys/firmware/memmap/ after a proper reboot: 0000000000000000-000000000009fc00 (System RAM) 000000000009fc00-00000000000a0000 (Reserved) 00000000000f0000-0000000000100000 (Reserved) 0000000000100000-00000000bffdf000 (System RAM) 00000000bffdf000-00000000c0000000 (Reserved) 00000000feffc000-00000000ff000000 (Reserved) 00000000fffc0000-0000000100000000 (Reserved) 0000000100000000-0000000140000000 (System RAM) /sys/firmware/memmap/ after this change: 0000000000000000-000000000009fc00 (System RAM) 000000000009fc00-00000000000a0000 (Reserved) 00000000000f0000-0000000000100000 (Reserved) 0000000000100000-00000000bffdf000 (System RAM) 00000000bffdf000-00000000c0000000 (Reserved) 00000000feffc000-00000000ff000000 (Reserved) 00000000fffc0000-0000000100000000 (Reserved) 0000000100000000-0000000140000000 (System RAM) kexec-tools already seem to basically ignore any System RAM that's not on top level when searching for areas to place kexec images - but also for determining crash areas to dump via kdump. Changing the resource name won't have an impact. Handle unloading of the driver after memory hotremove failed properly, by duplicating the string if necessary. Link: http://lkml.kernel.org/r/20200508084217.9160-5-david@redhat.com Signed-off-by: David Hildenbrand Acked-by: Pankaj Gupta Cc: Michal Hocko Cc: Pankaj Gupta Cc: Wei Yang Cc: Baoquan He Cc: Dave Hansen Cc: Eric Biederman Cc: Pavel Tatashin Cc: Dan Williams Signed-off-by: Andrew Morton --- drivers/dax/dax-private.h | 1 + drivers/dax/kmem.c | 28 ++++++++++++++++++++++++++-- 2 files changed, 27 insertions(+), 2 deletions(-) --- a/drivers/dax/dax-private.h~device-dax-add-memory-via-add_memory_driver_managed +++ a/drivers/dax/dax-private.h @@ -44,6 +44,7 @@ struct dax_region { * @dev - device core * @pgmap - pgmap for memmap setup / lifetime (driver owned) * @dax_mem_res: physical address range of hotadded DAX memory + * @dax_mem_name: name for hotadded DAX memory via add_memory_driver_managed() */ struct dev_dax { struct dax_region *region; --- a/drivers/dax/kmem.c~device-dax-add-memory-via-add_memory_driver_managed +++ a/drivers/dax/kmem.c @@ -14,6 +14,11 @@ #include "dax-private.h" #include "bus.h" +/* Memory resource name used for add_memory_driver_managed(). */ +static const char *kmem_name; +/* Set if any memory will remain added when the driver will be unloaded. */ +static bool any_hotremove_failed; + int dev_dax_kmem_probe(struct device *dev) { struct dev_dax *dev_dax = to_dev_dax(dev); @@ -70,7 +75,12 @@ int dev_dax_kmem_probe(struct device *de */ new_res->flags = IORESOURCE_SYSTEM_RAM; - rc = add_memory(numa_node, new_res->start, resource_size(new_res)); + /* + * Ensure that future kexec'd kernels will not treat this as RAM + * automatically. + */ + rc = add_memory_driver_managed(numa_node, new_res->start, + resource_size(new_res), kmem_name); if (rc) { release_resource(new_res); kfree(new_res); @@ -100,6 +110,7 @@ static int dev_dax_kmem_remove(struct de */ rc = remove_memory(dev_dax->target_node, kmem_start, kmem_size); if (rc) { + any_hotremove_failed = true; dev_err(dev, "DAX region %pR cannot be hotremoved until the next reboot\n", res); @@ -124,6 +135,7 @@ static int dev_dax_kmem_remove(struct de * permanently pinned as reserved by the unreleased * request_mem_region(). */ + any_hotremove_failed = true; return 0; } #endif /* CONFIG_MEMORY_HOTREMOVE */ @@ -137,12 +149,24 @@ static struct dax_device_driver device_d static int __init dax_kmem_init(void) { - return dax_driver_register(&device_dax_kmem_driver); + int rc; + + /* Resource name is permanently allocated if any hotremove fails. */ + kmem_name = kstrdup_const("System RAM (kmem)", GFP_KERNEL); + if (!kmem_name) + return -ENOMEM; + + rc = dax_driver_register(&device_dax_kmem_driver); + if (rc) + kfree_const(kmem_name); + return rc; } static void __exit dax_kmem_exit(void) { dax_driver_unregister(&device_dax_kmem_driver); + if (!any_hotremove_failed) + kfree_const(kmem_name); } MODULE_AUTHOR("Intel Corporation");