From patchwork Fri Aug 21 10:34:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11728829 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BD4FA16B1 for ; Fri, 21 Aug 2020 10:35:22 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 979B5207DA for ; Fri, 21 Aug 2020 10:35:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UwfRSjv+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 979B5207DA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k94Nw-0000Y8-Gi; Fri, 21 Aug 2020 10:35:00 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k94Nv-0000Xn-1a for xen-devel@lists.xenproject.org; Fri, 21 Aug 2020 10:34:59 +0000 X-Inumbo-ID: b8054a08-e193-454a-9e84-8032e221f063 Received: from us-smtp-delivery-1.mimecast.com (unknown [207.211.31.81]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id b8054a08-e193-454a-9e84-8032e221f063; Fri, 21 Aug 2020 10:34:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598006091; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XJmXFP26TlaoILesjgfmIBQIbNPk9EKlqFijZvcbNh0=; b=UwfRSjv+lN1AWLxzo1bC0dZJb9o6BtxA8Os0rqZCPbgnn06rOeNMvrktmgYt7LukaADDue E2S1aN4bKFeOMC8bRWUzRoW0gWvlv0dUIyPO8ZJgtEeUrsr2b+GTAl5esUv6BnfAl2Sbyk Yu9QJioER3grpS41DIwM2mWWRT4qorI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-229-6z0QkybENJy6Io-WetcOng-1; Fri, 21 Aug 2020 06:34:49 -0400 X-MC-Unique: 6z0QkybENJy6Io-WetcOng-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A6DAF640CA; Fri, 21 Aug 2020 10:34:47 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-87.ams2.redhat.com [10.36.114.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id CDF5460BF1; Fri, 21 Aug 2020 10:34:44 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: virtualization@lists.linux-foundation.org, linux-mm@kvack.org, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, David Hildenbrand , Andrew Morton , Michal Hocko , Dan Williams , Jason Gunthorpe , Kees Cook , Ard Biesheuvel , Pankaj Gupta , Baoquan He , Wei Yang Subject: [PATCH v1 1/5] kernel/resource: make release_mem_region_adjustable() never fail Date: Fri, 21 Aug 2020 12:34:27 +0200 Message-Id: <20200821103431.13481-2-david@redhat.com> In-Reply-To: <20200821103431.13481-1-david@redhat.com> References: <20200821103431.13481-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Let's make sure splitting a resource on memory hotunplug will never fail. This will become more relevant once we merge selected system ram resources - then, we'll trigger that case more frequently on memory hotunplug. In general, this function is already unlikely to fail. When we remove memory, we free up quite a lot of metadata (memmap, page tables, memory block device, etc.). The only way it could really fail currently is when injecting allocation errors. All other error cases inside release_mem_region_adjustable() seem to be sanity checks if the function would be abused in different context - let's add WARN_ON_ONCE(). Cc: Andrew Morton Cc: Michal Hocko Cc: Dan Williams Cc: Jason Gunthorpe Cc: Kees Cook Cc: Ard Biesheuvel Cc: Pankaj Gupta Cc: Baoquan He Cc: Wei Yang Signed-off-by: David Hildenbrand --- include/linux/ioport.h | 4 ++-- kernel/resource.c | 49 ++++++++++++++++++++++++------------------ mm/memory_hotplug.c | 22 +------------------ 3 files changed, 31 insertions(+), 44 deletions(-) diff --git a/include/linux/ioport.h b/include/linux/ioport.h index 6c2b06fe8beb7..52a91f5fa1a36 100644 --- a/include/linux/ioport.h +++ b/include/linux/ioport.h @@ -248,8 +248,8 @@ extern struct resource * __request_region(struct resource *, extern void __release_region(struct resource *, resource_size_t, resource_size_t); #ifdef CONFIG_MEMORY_HOTREMOVE -extern int release_mem_region_adjustable(struct resource *, resource_size_t, - resource_size_t); +extern void release_mem_region_adjustable(struct resource *, resource_size_t, + resource_size_t); #endif /* Wrappers for managed devices */ diff --git a/kernel/resource.c b/kernel/resource.c index 841737bbda9e5..1dcef5d53d76e 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -1255,21 +1255,28 @@ EXPORT_SYMBOL(__release_region); * assumes that all children remain in the lower address entry for * simplicity. Enhance this logic when necessary. */ -int release_mem_region_adjustable(struct resource *parent, - resource_size_t start, resource_size_t size) +void release_mem_region_adjustable(struct resource *parent, + resource_size_t start, resource_size_t size) { + struct resource *new_res = NULL; + bool alloc_nofail = false; struct resource **p; struct resource *res; - struct resource *new_res; resource_size_t end; - int ret = -EINVAL; end = start + size - 1; - if ((start < parent->start) || (end > parent->end)) - return ret; + if (WARN_ON_ONCE((start < parent->start) || (end > parent->end))) + return; - /* The alloc_resource() result gets checked later */ - new_res = alloc_resource(GFP_KERNEL); + /* + * We free up quite a lot of memory on memory hotunplug (esp., memap), + * just before releasing the region. This is highly unlikely to + * fail - let's play save and make it never fail as the caller cannot + * perform any error handling (e.g., trying to re-add memory will fail + * similarly). + */ +retry: + new_res = alloc_resource(GFP_KERNEL | alloc_nofail ? __GFP_NOFAIL : 0); p = &parent->child; write_lock(&resource_lock); @@ -1295,7 +1302,6 @@ int release_mem_region_adjustable(struct resource *parent, * so if we are dealing with them, let us just back off here. */ if (!(res->flags & IORESOURCE_SYSRAM)) { - ret = 0; break; } @@ -1312,20 +1318,23 @@ int release_mem_region_adjustable(struct resource *parent, /* free the whole entry */ *p = res->sibling; free_resource(res); - ret = 0; } else if (res->start == start && res->end != end) { /* adjust the start */ - ret = __adjust_resource(res, end + 1, - res->end - end); + WARN_ON_ONCE(__adjust_resource(res, end + 1, + res->end - end)); } else if (res->start != start && res->end == end) { /* adjust the end */ - ret = __adjust_resource(res, res->start, - start - res->start); + WARN_ON_ONCE(__adjust_resource(res, res->start, + start - res->start)); } else { - /* split into two entries */ + /* split into two entries - we need a new resource */ if (!new_res) { - ret = -ENOMEM; - break; + new_res = alloc_resource(GFP_ATOMIC); + if (!new_res) { + alloc_nofail = true; + write_unlock(&resource_lock); + goto retry; + } } new_res->name = res->name; new_res->start = end + 1; @@ -1336,9 +1345,8 @@ int release_mem_region_adjustable(struct resource *parent, new_res->sibling = res->sibling; new_res->child = NULL; - ret = __adjust_resource(res, res->start, - start - res->start); - if (ret) + if (WARN_ON_ONCE(__adjust_resource(res, res->start, + start - res->start))) break; res->sibling = new_res; new_res = NULL; @@ -1349,7 +1357,6 @@ int release_mem_region_adjustable(struct resource *parent, write_unlock(&resource_lock); free_resource(new_res); - return ret; } #endif /* CONFIG_MEMORY_HOTREMOVE */ diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 35d56cbd3e45b..f77fd1810e9f5 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1700,26 +1700,6 @@ void try_offline_node(int nid) } EXPORT_SYMBOL(try_offline_node); -static void __release_memory_resource(resource_size_t start, - resource_size_t size) -{ - int ret; - - /* - * When removing memory in the same granularity as it was added, - * this function never fails. It might only fail if resources - * have to be adjusted or split. We'll ignore the error, as - * removing of memory cannot fail. - */ - ret = release_mem_region_adjustable(&iomem_resource, start, size); - if (ret) { - resource_size_t endres = start + size - 1; - - pr_warn("Unable to release resource <%pa-%pa> (%d)\n", - &start, &endres, ret); - } -} - static int __ref try_remove_memory(int nid, u64 start, u64 size) { int rc = 0; @@ -1753,7 +1733,7 @@ static int __ref try_remove_memory(int nid, u64 start, u64 size) memblock_remove(start, size); } - __release_memory_resource(start, size); + release_mem_region_adjustable(&iomem_resource, start, size); try_offline_node(nid); From patchwork Fri Aug 21 10:34:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11728837 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4784317CF for ; Fri, 21 Aug 2020 10:35:54 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2314F204EC for ; Fri, 21 Aug 2020 10:35:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="FrULWNGU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2314F204EC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k94O0-0000Z8-RF; Fri, 21 Aug 2020 10:35:04 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k94O0-0000Xn-1u for xen-devel@lists.xenproject.org; Fri, 21 Aug 2020 10:35:04 +0000 X-Inumbo-ID: bd76f3b0-5cf1-4146-a27d-d4d7d6cace23 Received: from us-smtp-delivery-1.mimecast.com (unknown [207.211.31.120]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id bd76f3b0-5cf1-4146-a27d-d4d7d6cace23; Fri, 21 Aug 2020 10:35:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598006101; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=G68oLT4JVMoIG5DqaYBfCZBI9EqNtAD2sZmp6ATBUeA=; b=FrULWNGUo4md0VuZQnM9ccGnoG1bCbfjiS10prNeVMG1jZJcfWOHOQp2SojYbRZ/jqDhRz C23r2E074N3A9wlN4jJ3WBEA/AHaLAAfaPvN0kobbB+PqCnBG9JwugXDn87cdbxFZyQog7 jv/eNDQ+cSqQy3JTqAmhqocPO0epvzg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-561-i2qdkbQ-NhGC9y6PSLFkzg-1; Fri, 21 Aug 2020 06:34:57 -0400 X-MC-Unique: i2qdkbQ-NhGC9y6PSLFkzg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E7F3A8030D0; Fri, 21 Aug 2020 10:34:54 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-87.ams2.redhat.com [10.36.114.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 05299756D8; Fri, 21 Aug 2020 10:34:47 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: virtualization@lists.linux-foundation.org, linux-mm@kvack.org, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, David Hildenbrand , Andrew Morton , Michal Hocko , Dan Williams , Jason Gunthorpe , Kees Cook , Ard Biesheuvel , Thomas Gleixner , "K. Y. Srinivasan" , Haiyang Zhang , Stephen Hemminger , Wei Liu , Boris Ostrovsky , Juergen Gross , Stefano Stabellini , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Julien Grall , Pankaj Gupta , Baoquan He , Wei Yang Subject: [PATCH v1 2/5] kernel/resource: merge_system_ram_resources() to merge resources after hotplug Date: Fri, 21 Aug 2020 12:34:28 +0200 Message-Id: <20200821103431.13481-3-david@redhat.com> In-Reply-To: <20200821103431.13481-1-david@redhat.com> References: <20200821103431.13481-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Some add_memory*() users add memory in small, contiguous memory blocks. Examples include virtio-mem, hyper-v balloon, and the XEN balloon. This can quickly result in a lot of memory resources, whereby the actual resource boundaries are not of interest (e.g., it might be relevant for DIMMs, exposed via /proc/iomem to user space). We really want to merge added resources in this scenario where possible. Let's provide an interface to trigger merging of applicable child resources. It will be, for example, used by virtio-mem to trigger merging of system ram resources it added to its resource container, but also by XEN and Hyper-V to trigger merging of system ram resources in iomem_resource. Note: We really want to merge after the whole operation succeeded, not directly when adding a resource to the resource tree (it would break add_memory_resource() and require splitting resources again when the operation failed - e.g., due to -ENOMEM). Cc: Andrew Morton Cc: Michal Hocko Cc: Dan Williams Cc: Jason Gunthorpe Cc: Kees Cook Cc: Ard Biesheuvel Cc: Thomas Gleixner Cc: "K. Y. Srinivasan" Cc: Haiyang Zhang Cc: Stephen Hemminger Cc: Wei Liu Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini Cc: Roger Pau Monné Cc: Julien Grall Cc: Pankaj Gupta Cc: Baoquan He Cc: Wei Yang Signed-off-by: David Hildenbrand --- include/linux/ioport.h | 3 +++ kernel/resource.c | 52 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/include/linux/ioport.h b/include/linux/ioport.h index 52a91f5fa1a36..3bb0020cd6ddc 100644 --- a/include/linux/ioport.h +++ b/include/linux/ioport.h @@ -251,6 +251,9 @@ extern void __release_region(struct resource *, resource_size_t, extern void release_mem_region_adjustable(struct resource *, resource_size_t, resource_size_t); #endif +#ifdef CONFIG_MEMORY_HOTPLUG +extern void merge_system_ram_resources(struct resource *res); +#endif /* Wrappers for managed devices */ struct device; diff --git a/kernel/resource.c b/kernel/resource.c index 1dcef5d53d76e..b4e0963edadd2 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -1360,6 +1360,58 @@ void release_mem_region_adjustable(struct resource *parent, } #endif /* CONFIG_MEMORY_HOTREMOVE */ +#ifdef CONFIG_MEMORY_HOTPLUG +static bool system_ram_resources_mergeable(struct resource *r1, + struct resource *r2) +{ + return r1->flags == r2->flags && r1->end + 1 == r2->start && + r1->name == r2->name && r1->desc == r2->desc && + !r1->child && !r2->child; +} + +/* + * merge_system_ram_resources - try to merge contiguous system ram resources + * @parent: parent resource descriptor + * + * This interface is intended for memory hotplug, whereby lots of contiguous + * system ram resources are added (e.g., via add_memory*()) by a driver, and + * the actual resource boundaries are not of interest (e.g., it might be + * relevant for DIMMs). Only immediate child resources that are busy and + * don't have any children are considered. All applicable child resources + * must be immutable during the request. + * + * Note: + * - The caller has to make sure that no pointers to resources that might + * get merged are held anymore. Callers should only trigger merging of child + * resources when they are the only one adding system ram resources to the + * parent (besides during boot). + * - release_mem_region_adjustable() will split on demand on memory hotunplug + */ +void merge_system_ram_resources(struct resource *parent) +{ + const unsigned long flags = IORESOURCE_SYSTEM_RAM | IORESOURCE_BUSY; + struct resource *cur, *next; + + write_lock(&resource_lock); + + cur = parent->child; + while (cur && cur->sibling) { + next = cur->sibling; + if ((cur->flags & flags) == flags && + system_ram_resources_mergeable(cur, next)) { + cur->end = next->end; + cur->sibling = next->sibling; + free_resource(next); + next = cur->sibling; + } + cur = next; + } + + write_unlock(&resource_lock); +} +EXPORT_SYMBOL(merge_system_ram_resources); +#endif /* CONFIG_MEMORY_HOTPLUG */ + /* * Managed region resource */ From patchwork Fri Aug 21 10:34:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11728841 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F209516B1 for ; Fri, 21 Aug 2020 10:36:27 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CDE9120738 for ; Fri, 21 Aug 2020 10:36:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Z+whIEeF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CDE9120738 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k94O8-0000bE-Bd; Fri, 21 Aug 2020 10:35:12 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k94O7-0000aw-Hq for xen-devel@lists.xenproject.org; Fri, 21 Aug 2020 10:35:11 +0000 X-Inumbo-ID: 7897e7dd-32db-4c44-949f-acfa0afcd6a6 Received: from us-smtp-delivery-1.mimecast.com (unknown [205.139.110.120]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id 7897e7dd-32db-4c44-949f-acfa0afcd6a6; Fri, 21 Aug 2020 10:35:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598006110; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=a286TBwbZ7ci6RWhKZMWDqKWC78sTVuqH/ttlMVxfUs=; b=Z+whIEeFsJbcZWTC14b1kwV1djiU7QE3/k/MTYSHgu9WuhqsgaFcoOUmQqIzzp55Nc0tqt +JdPHKGFUXjNI5Q8vBZQWYliivKyzHj4ynaBeqK3QS1A8YOWJ/PfNbfNlMmENeFHgk07He JPJh/lRWGKfkImCDkViiVGFEq2N1J/Y= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-144-Gn4EHG4jOouXnffn-SpIAw-1; Fri, 21 Aug 2020 06:35:05 -0400 X-MC-Unique: Gn4EHG4jOouXnffn-SpIAw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4707F18A2241; Fri, 21 Aug 2020 10:35:04 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-87.ams2.redhat.com [10.36.114.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 449F960BF1; Fri, 21 Aug 2020 10:34:55 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: virtualization@lists.linux-foundation.org, linux-mm@kvack.org, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, David Hildenbrand , Andrew Morton , Michal Hocko , Dan Williams , "Michael S . Tsirkin" , Jason Wang , Pankaj Gupta , Baoquan He , Wei Yang Subject: [PATCH v1 3/5] virtio-mem: try to merge system ram resources Date: Fri, 21 Aug 2020 12:34:29 +0200 Message-Id: <20200821103431.13481-4-david@redhat.com> In-Reply-To: <20200821103431.13481-1-david@redhat.com> References: <20200821103431.13481-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" virtio-mem adds memory in memory block granularity, to be able to remove it in the same granularity again later, and to grow slowly on demand. This, however, results in quite a lot of resources when adding a lot of memory. Resources are effectively stored in a list-based tree. Having a lot of resources not only wastes memory, it also makes traversing that tree more expensive, and makes /proc/iomem explode in size (e.g., requiring kexec-tools to manually merge resources later when e.g., trying to create a kdump header). Before this patch, we get (/proc/iomem) when hotplugging 2G via virtio-mem on x86-64: [...] 100000000-13fffffff : System RAM 140000000-33fffffff : virtio0 140000000-147ffffff : System RAM (virtio_mem) 148000000-14fffffff : System RAM (virtio_mem) 150000000-157ffffff : System RAM (virtio_mem) 158000000-15fffffff : System RAM (virtio_mem) 160000000-167ffffff : System RAM (virtio_mem) 168000000-16fffffff : System RAM (virtio_mem) 170000000-177ffffff : System RAM (virtio_mem) 178000000-17fffffff : System RAM (virtio_mem) 180000000-187ffffff : System RAM (virtio_mem) 188000000-18fffffff : System RAM (virtio_mem) 190000000-197ffffff : System RAM (virtio_mem) 198000000-19fffffff : System RAM (virtio_mem) 1a0000000-1a7ffffff : System RAM (virtio_mem) 1a8000000-1afffffff : System RAM (virtio_mem) 1b0000000-1b7ffffff : System RAM (virtio_mem) 1b8000000-1bfffffff : System RAM (virtio_mem) 3280000000-32ffffffff : PCI Bus 0000:00 With this patch, we get (/proc/iomem): [...] fffc0000-ffffffff : Reserved 100000000-13fffffff : System RAM 140000000-33fffffff : virtio0 140000000-1bfffffff : System RAM (virtio_mem) 3280000000-32ffffffff : PCI Bus 0000:00 Of course, with more hotplugged memory, it gets worse. When unplugging memory blocks again, try_remove_memory() (via offline_and_remove_memory()) will properly split the resource up again. Cc: Andrew Morton Cc: Michal Hocko Cc: Dan Williams Cc: Michael S. Tsirkin Cc: Jason Wang Cc: Pankaj Gupta Cc: Baoquan He Cc: Wei Yang Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 834b7c13ef3dc..3aae0f87073a8 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -407,6 +407,7 @@ static int virtio_mem_mb_add(struct virtio_mem *vm, unsigned long mb_id) { const uint64_t addr = virtio_mem_mb_id_to_phys(mb_id); int nid = vm->nid; + int rc; if (nid == NUMA_NO_NODE) nid = memory_add_physaddr_to_nid(addr); @@ -423,8 +424,17 @@ static int virtio_mem_mb_add(struct virtio_mem *vm, unsigned long mb_id) } dev_dbg(&vm->vdev->dev, "adding memory block: %lu\n", mb_id); - return add_memory_driver_managed(nid, addr, memory_block_size_bytes(), - vm->resource_name); + rc = add_memory_driver_managed(nid, addr, memory_block_size_bytes(), + vm->resource_name); + if (!rc) { + /* + * Try to reduce the number of system ram resources in our + * resource container. The memory removal path will properly + * split them up again. + */ + merge_system_ram_resources(vm->parent_resource); + } + return rc; } /* From patchwork Fri Aug 21 10:34:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11728845 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 10AEF16B1 for ; Fri, 21 Aug 2020 10:36:36 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E0AA120738 for ; Fri, 21 Aug 2020 10:36:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="X9QaBfV9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E0AA120738 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k94OE-0000eW-LA; Fri, 21 Aug 2020 10:35:18 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k94OC-0000do-R7 for xen-devel@lists.xenproject.org; Fri, 21 Aug 2020 10:35:16 +0000 X-Inumbo-ID: 87800420-2e38-46e1-a254-1baa901fe22f Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id 87800420-2e38-46e1-a254-1baa901fe22f; Fri, 21 Aug 2020 10:35:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598006116; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OKy/5qBO5o//LYJ+b8xL+LjmCbIRLl1gm4kTDHiwBRU=; b=X9QaBfV9UtkGKENQvWpDhiboujweGYZTE92dUhMcI/ow+4kw4SKTEvUD4+lG+ZRU3RVQBE uHaq9/jM2temy7+sp2JRu3w4AFeaqLl8quzrEOlKuQEkw6EXpR4KUYAdlSxEcWIRqnr3zF 6wIzfzwAtpBRoxJtUHL4bW9Ha0CwYbk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-133-2VGBU1yXNXyePjxSO5BEig-1; Fri, 21 Aug 2020 06:35:12 -0400 X-MC-Unique: 2VGBU1yXNXyePjxSO5BEig-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 002AE80733A; Fri, 21 Aug 2020 10:35:10 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-87.ams2.redhat.com [10.36.114.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9887C756DD; Fri, 21 Aug 2020 10:35:04 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: virtualization@lists.linux-foundation.org, linux-mm@kvack.org, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, David Hildenbrand , Andrew Morton , Michal Hocko , Boris Ostrovsky , Juergen Gross , Stefano Stabellini , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Julien Grall , Pankaj Gupta , Baoquan He , Wei Yang Subject: [PATCH v1 4/5] xen/balloon: try to merge system ram resources Date: Fri, 21 Aug 2020 12:34:30 +0200 Message-Id: <20200821103431.13481-5-david@redhat.com> In-Reply-To: <20200821103431.13481-1-david@redhat.com> References: <20200821103431.13481-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Let's reuse the new mechanism to merge system ram resources below the root. We are the only one hotplugging system ram (e.g., DIMMs don't apply), so this is safe to be used. Cc: Andrew Morton Cc: Michal Hocko Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini Cc: Roger Pau Monné Cc: Julien Grall Cc: Pankaj Gupta Cc: Baoquan He Cc: Wei Yang Signed-off-by: David Hildenbrand --- drivers/xen/balloon.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c index 37ffccda8bb87..5ec73f752b8a7 100644 --- a/drivers/xen/balloon.c +++ b/drivers/xen/balloon.c @@ -338,6 +338,10 @@ static enum bp_state reserve_additional_memory(void) if (rc) { pr_warn("Cannot add additional memory (%i)\n", rc); goto err; + } else { + resource = NULL; + /* Try to reduce the number of system ram resources. */ + merge_system_ram_resources(&iomem_resource); } balloon_stats.total_pages += balloon_hotplug; From patchwork Fri Aug 21 10:34:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11728843 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CD5361731 for ; Fri, 21 Aug 2020 10:36:29 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A9DCA20738 for ; Fri, 21 Aug 2020 10:36:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="f4atkxJj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A9DCA20738 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k94OI-0000gN-V0; Fri, 21 Aug 2020 10:35:22 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k94OH-0000fs-QW for xen-devel@lists.xenproject.org; Fri, 21 Aug 2020 10:35:21 +0000 X-Inumbo-ID: 4ac470bc-c306-4431-b891-acfeb222db68 Received: from us-smtp-1.mimecast.com (unknown [207.211.31.81]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id 4ac470bc-c306-4431-b891-acfeb222db68; Fri, 21 Aug 2020 10:35:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598006121; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AeL7eQMcuzDtJ58y3twR8woldx1f2g3cujVSMwGjGgg=; b=f4atkxJjlh77DKACdy22VoaPcQ4vopSeQEfqAo5J39GRHCWAzLROVSfcT8RWbIR7mWwxSV XgVcYhG9gGdInbUWja9cYVT545jqAjeR2C7lUpEbK40+SZUdrvt/aV8hMoMTDPbxhsK+n5 49BUXBFodelVyqVOsqjgwmGd/7TEq/Y= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-353-Fg8UITSqMl-wo432KovFmA-1; Fri, 21 Aug 2020 06:35:17 -0400 X-MC-Unique: Fg8UITSqMl-wo432KovFmA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2DF0281F02D; Fri, 21 Aug 2020 10:35:15 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-87.ams2.redhat.com [10.36.114.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5684C60BF1; Fri, 21 Aug 2020 10:35:10 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: virtualization@lists.linux-foundation.org, linux-mm@kvack.org, linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org, David Hildenbrand , Andrew Morton , Michal Hocko , "K. Y. Srinivasan" , Haiyang Zhang , Stephen Hemminger , Wei Liu , Pankaj Gupta , Baoquan He , Wei Yang Subject: [PATCH v1 5/5] hv_balloon: try to merge system ram resources Date: Fri, 21 Aug 2020 12:34:31 +0200 Message-Id: <20200821103431.13481-6-david@redhat.com> In-Reply-To: <20200821103431.13481-1-david@redhat.com> References: <20200821103431.13481-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Let's use the new mechanism to merge system ram resources below the root. We are the only one hotplugging system ram, e.g., DIMMs don't apply, so this is safe to be used. Cc: Andrew Morton Cc: Michal Hocko Cc: "K. Y. Srinivasan" Cc: Haiyang Zhang Cc: Stephen Hemminger Cc: Wei Liu Cc: Pankaj Gupta Cc: Baoquan He Cc: Wei Yang Signed-off-by: David Hildenbrand --- drivers/hv/hv_balloon.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/hv/hv_balloon.c b/drivers/hv/hv_balloon.c index 32e3bc0aa665a..49a6305f0fb73 100644 --- a/drivers/hv/hv_balloon.c +++ b/drivers/hv/hv_balloon.c @@ -745,6 +745,9 @@ static void hv_mem_hot_add(unsigned long start, unsigned long size, has->covered_end_pfn -= processed_pfn; spin_unlock_irqrestore(&dm_device.ha_lock, flags); break; + } else { + /* Try to reduce the number of system ram resources. */ + merge_system_ram_resources(&iomem_resource); } /*