From patchwork Wed Apr 3 04:30:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 10882775 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7475F17E1 for ; Wed, 3 Apr 2019 04:30:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5EEFF28613 for ; Wed, 3 Apr 2019 04:30:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5029C28750; Wed, 3 Apr 2019 04:30:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EB84628613 for ; Wed, 3 Apr 2019 04:30:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1D4096B0286; Wed, 3 Apr 2019 00:30:21 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 183F56B0288; Wed, 3 Apr 2019 00:30:21 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 04C9C6B0289; Wed, 3 Apr 2019 00:30:21 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id ABB356B0286 for ; Wed, 3 Apr 2019 00:30:20 -0400 (EDT) Received: by mail-ed1-f69.google.com with SMTP id p90so4225309edp.11 for ; Tue, 02 Apr 2019 21:30:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=B0Km0z7J00nzfFzzN3rWUsWibL2B59QvurZv2eHIpkY=; b=j90vDvbr+fUK6OKpxARpWQNhNHLVw1+4sV/bDAavHU+0E7P6LJ7+DURo6BlJEoQvnP czKIBWbEZuEPMKEdywIw9rZDpKi0sLRJyVcB3A7DNK/nAN9xxo7mAoluUf9lv6QdaVz9 vHtclpUqpyXU72eNRtJ17VhbxcZ/2ZpLgUyrdq+jgQyBdBm2CWc7MakR1Iasj+5IjRkS RQLtsin18V8fM6uOYFpdQWUlJ2T7Dbv9W+Z5czuZacMQeL7N3aqjSFcm8233Rxk/W1PV moy8P5gp9ylnL04OWZbrRVs54zYnmWGVU6WOjQeM/IvsXG3hmMoK3Zf0xUGOvpcnEtt8 jpGw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com X-Gm-Message-State: APjAAAUYOdJvvJEtWYa6nMK/WVgwqf/+zqZcnOi9jx1Jpi3RGb7AWEcR J9toi7ooNvYRnb/6v0KfOl1s8rKl6kG6/6w22oDy9Iwa5DTBUxbGCTFZhM8iDgWNY1V6lZdvQIS f8xaAzgTFgOOW269o1WzYff98ad2yv+vFU9E98nhZsLs6OVf2M2rh8tGxr64SkmFXmw== X-Received: by 2002:a50:86c3:: with SMTP id 3mr49754936edu.143.1554265820221; Tue, 02 Apr 2019 21:30:20 -0700 (PDT) X-Google-Smtp-Source: APXvYqy/j/yfugsr9g4uHqGukZCP6ilftc3StDPXATFmhpZbl3zWyfsAOlZ9dg9WWVmyH6eL9HdF X-Received: by 2002:a50:86c3:: with SMTP id 3mr49754887edu.143.1554265819145; Tue, 02 Apr 2019 21:30:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554265819; cv=none; d=google.com; s=arc-20160816; b=zXzw4HpCAnf3ZHYdeTtCrq6JuFsiHTkjejaxmFIucuo5jPtkzMmILM7vtE15l9fgNS sZaMp5ICG9apALjVkV1afJC9DVqPP66RIdbtGjvk+0IHlR6do0xJq5U49Gba0cm33MMF dRGCUFkNQ+oogU62G3IOnJ0A5IGEvg9kOpNYHQsXGdAxDJKqv9dBRlsNw30mA5ZwFHsf WvZRwNoD/By/9AMiwcp0VMq5vNaBHuE1xGSZlmHvzUmb3x7Rx0a2mPe78ywbnFoVeCNJ N5pebiULFPRT5JpnMXWGLWTfMhL7QxKzeimX4hd8NOzyv+/a0ZZVnrC5I04bhZywmssq HsQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=B0Km0z7J00nzfFzzN3rWUsWibL2B59QvurZv2eHIpkY=; b=rKi6jafma1+MmikBMD1GGEiSuw4KRFsij2kcyEtf3LrEzrxqNwRqX4WVIN/NdX9jrq mn+QYpg0kUJzn0oPPIhrF0U4eJljpkxQED/P8YohVcq3vDQCMQC5qjQBQlBI3eJJ5RgP q+raQHL3meWB6JKsH9caJPNHBgMO3W80iobNojAK6khQhCYAKxswvH5ZDbqKMQsdwJgP CC2pM4wP0LMwU2lwwLzJGjNRd7UAkE6Ew52m8CekjTOS3ZGJ80XdMBKUionkz9G3FDoE Uw2KXHY7vkvqbRrWA46cybqy8MoEiC/VN4NUGr74YoR3GBSHiRiNfrV8GV9s1dAkCIS8 WsnA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id dx25si155583ejb.64.2019.04.02.21.30.18 for ; Tue, 02 Apr 2019 21:30:19 -0700 (PDT) Received-SPF: pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 13D5F15AD; Tue, 2 Apr 2019 21:30:18 -0700 (PDT) Received: from p8cg001049571a15.arm.com (unknown [10.163.1.97]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B6BCE3F721; Tue, 2 Apr 2019 21:30:12 -0700 (PDT) From: Anshuman Khandual To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, akpm@linux-foundation.org, will.deacon@arm.com, catalin.marinas@arm.com Cc: mhocko@suse.com, mgorman@techsingularity.net, james.morse@arm.com, mark.rutland@arm.com, robin.murphy@arm.com, cpandya@codeaurora.org, arunks@codeaurora.org, dan.j.williams@intel.com, osalvador@suse.de, logang@deltatee.com, pasha.tatashin@oracle.com, david@redhat.com, cai@lca.pw Subject: [PATCH 1/6] arm64/mm: Enable sysfs based memory hot add interface Date: Wed, 3 Apr 2019 10:00:01 +0530 Message-Id: <1554265806-11501-2-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1554265806-11501-1-git-send-email-anshuman.khandual@arm.com> References: <1554265806-11501-1-git-send-email-anshuman.khandual@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Sysfs memory probe interface (/sys/devices/system/memory/probe) can accept starting physical address of an entire memory block to be hot added into the kernel. This is in addition to the existing ACPI based interface. This just enables it with the required config CONFIG_ARCH_MEMORY_PROBE. Signed-off-by: Anshuman Khandual --- arch/arm64/Kconfig | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 7e34b9e..a2418fb 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -266,6 +266,15 @@ config HAVE_GENERIC_GUP config ARCH_ENABLE_MEMORY_HOTPLUG def_bool y +config ARCH_MEMORY_PROBE + bool "Enable /sys/devices/system/memory/probe interface" + depends on MEMORY_HOTPLUG + help + This option enables a sysfs /sys/devices/system/memory/probe + interface for testing. See Documentation/memory-hotplug.txt + for more information. If you are unsure how to answer this + question, answer N. + config SMP def_bool y From patchwork Wed Apr 3 04:30:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 10882779 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8F1ED17E1 for ; Wed, 3 Apr 2019 04:30:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7445728613 for ; Wed, 3 Apr 2019 04:30:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 681B828913; Wed, 3 Apr 2019 04:30:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7B83028613 for ; Wed, 3 Apr 2019 04:30:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8B1536B0288; Wed, 3 Apr 2019 00:30:28 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 862596B028A; Wed, 3 Apr 2019 00:30:28 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 729FF6B028B; Wed, 3 Apr 2019 00:30:28 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id 34C736B0288 for ; Wed, 3 Apr 2019 00:30:28 -0400 (EDT) Received: by mail-ed1-f69.google.com with SMTP id m31so6856665edm.4 for ; Tue, 02 Apr 2019 21:30:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=qnLEgMq1dxXlnwV474xu1ZsWxRBbzAkfiGYmeqB2/MM=; b=hswdLAcmQmkYI3WB95t5OIPLUFJm+VT5uA/PtalksabK0+RF1MWalUdvrUkjeUNgO+ 607s/MhoytuGT1VUG1XlJlpfN4nzbEydJxfyrTalbL82DLshlXG+PnLHQDAMfzKwmsYf +t6fs14hHYHsyosz+AC3mTv/T76X0Ww6L0KBsHJH+tnkIEWmo614fBWZ3AEmLjM+5uj2 5l1p3RDTB5ZA/HZvbJpBKqDxa/RGkG2av/A6MZGeDMtJfy/WhvHqXkw67wBw+XbDbX+I 7+XJ0IP1mMSDDxoDk9vTtjOOToEgjsCssgvVR0shKvCsp6kjhgpyo4UD7B0GEGnyPu7f UwkQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com X-Gm-Message-State: APjAAAWulynvD4ml3bBebUeCj+B8bVt3cFej2un+RaVwGWEq6DejJVKD 2ZDRv3B5bXY2YQ0GQkmi2hxSoco4eoJUZExltGIWuroChsIr9x3ktxBNjVlkH04JDX9rm1+b2Z6 GqGAMWWChAEk1b6KOMJdeEsEIU+Ptb2BK4b7o4lXe7LR8UQLlSdBcZoQGL9nDoWHyiQ== X-Received: by 2002:a50:a7e5:: with SMTP id i92mr50484486edc.181.1554265827680; Tue, 02 Apr 2019 21:30:27 -0700 (PDT) X-Google-Smtp-Source: APXvYqzO/21SsDnJ+iuMsovWWiHksEtf4p+fMpycKbI2a5obzo+NlJiPHeKq4ICf7CYcIHU4s8ue X-Received: by 2002:a50:a7e5:: with SMTP id i92mr50484411edc.181.1554265825810; Tue, 02 Apr 2019 21:30:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554265825; cv=none; d=google.com; s=arc-20160816; b=QZDd8xn95K1yJxi+gVtD+vuq1nvxBIkFneQVvedRmUOZcL03+Pp70e22Tg1eQr6Oh7 L67DIJ1Wgv7adBJinnauWkjKxJXKjuGgLFmdVPypgjqQc89RomiUNL5utjZmeq9+bMbm e3inbd968K3mg4YQbVvWRt0fwsiR01p3PRIy4rPNSQJmU6RMBK3gbQL9qL0uXbxxInKX FxE2ja55TKIEmtWrsaf/QZyEigNaKCwl6QYajL8Hn9K+bupju1xFttjJe8W+gRYWf/Ac 3P9T7i562kD32zHKzsvTPfiHqMhig90bvpcMPcwanGTEY7SiFPpP4vc6u60nP7hN9h2C Pe2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=qnLEgMq1dxXlnwV474xu1ZsWxRBbzAkfiGYmeqB2/MM=; b=XEezv5AEe5PGI6LD2OcBydsxwFzbCAPiPs9G/PaCwaB9VsOzknzHq8mqgCH44p/Qnr Jcs2QM7ANdFRt2//QpfTyObZYI4nEw6XH4U2phdjAxQk+9ZLKRzHKANEVyD9tJyi15pe avdCixlyGrt1ew6//wmH+pdz/yGBozIq4DjDH6mR7Z3EegUhPsy68xMgZF5nT2ZrI2By nKhkdA39aXNfQlfWVjqEa5n9e39AtwFuMldbcCTS56hfrnVjzYS10y6fxxLhovtjZKx7 N+nWZEBULQHxhCqSNpXkFWesaF0T8e+qQlbNvamjRjlAfG5Pejg5n10/YvEs3HcvsOCQ Z51Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id gr15si3469172ejb.302.2019.04.02.21.30.25 for ; Tue, 02 Apr 2019 21:30:25 -0700 (PDT) Received-SPF: pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A55EA1596; Tue, 2 Apr 2019 21:30:24 -0700 (PDT) Received: from p8cg001049571a15.arm.com (unknown [10.163.1.97]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7F0253F721; Tue, 2 Apr 2019 21:30:18 -0700 (PDT) From: Anshuman Khandual To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, akpm@linux-foundation.org, will.deacon@arm.com, catalin.marinas@arm.com Cc: mhocko@suse.com, mgorman@techsingularity.net, james.morse@arm.com, mark.rutland@arm.com, robin.murphy@arm.com, cpandya@codeaurora.org, arunks@codeaurora.org, dan.j.williams@intel.com, osalvador@suse.de, logang@deltatee.com, pasha.tatashin@oracle.com, david@redhat.com, cai@lca.pw Subject: [PATCH 2/6] arm64/mm: Enable memory hot remove Date: Wed, 3 Apr 2019 10:00:02 +0530 Message-Id: <1554265806-11501-3-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1554265806-11501-1-git-send-email-anshuman.khandual@arm.com> References: <1554265806-11501-1-git-send-email-anshuman.khandual@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Memory removal from an arch perspective involves tearing down two different kernel based mappings i.e vmemmap and linear while releasing related page table pages allocated for the physical memory range to be removed. Define a common kernel page table tear down helper remove_pagetable() which can be used to unmap given kernel virtual address range. In effect it can tear down both vmemap or kernel linear mappings. This new helper is called from both vmemamp_free() and ___remove_pgd_mapping() during memory removal. The argument 'direct' here identifies kernel linear mappings. Vmemmap mappings page table pages are allocated through sparse mem helper functions like vmemmap_alloc_block() which does not cycle the pages through pgtable_page_ctor() constructs. Hence while removing it skips corresponding destructor construct pgtable_page_dtor(). While here update arch_add_mempory() to handle __add_pages() failures by just unmapping recently added kernel linear mapping. Now enable memory hot remove on arm64 platforms by default with ARCH_ENABLE_MEMORY_HOTREMOVE. This implementation is overall inspired from kernel page table tear down procedure on X86 architecture. Signed-off-by: Anshuman Khandual --- arch/arm64/Kconfig | 3 + arch/arm64/include/asm/pgtable.h | 14 +++ arch/arm64/mm/mmu.c | 227 ++++++++++++++++++++++++++++++++++++++- 3 files changed, 241 insertions(+), 3 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a2418fb..db3e625 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -266,6 +266,9 @@ config HAVE_GENERIC_GUP config ARCH_ENABLE_MEMORY_HOTPLUG def_bool y +config ARCH_ENABLE_MEMORY_HOTREMOVE + def_bool y + config ARCH_MEMORY_PROBE bool "Enable /sys/devices/system/memory/probe interface" depends on MEMORY_HOTPLUG diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index de70c1e..858098e 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -355,6 +355,18 @@ static inline int pmd_protnone(pmd_t pmd) } #endif +#if (CONFIG_PGTABLE_LEVELS > 2) +#define pmd_large(pmd) (pmd_val(pmd) && !(pmd_val(pmd) & PMD_TABLE_BIT)) +#else +#define pmd_large(pmd) 0 +#endif + +#if (CONFIG_PGTABLE_LEVELS > 3) +#define pud_large(pud) (pud_val(pud) && !(pud_val(pud) & PUD_TABLE_BIT)) +#else +#define pud_large(pmd) 0 +#endif + /* * THP definitions. */ @@ -555,6 +567,7 @@ static inline phys_addr_t pud_page_paddr(pud_t pud) #else +#define pmd_index(addr) 0 #define pud_page_paddr(pud) ({ BUILD_BUG(); 0; }) /* Match pmd_offset folding in */ @@ -612,6 +625,7 @@ static inline phys_addr_t pgd_page_paddr(pgd_t pgd) #else +#define pud_index(adrr) 0 #define pgd_page_paddr(pgd) ({ BUILD_BUG(); 0;}) /* Match pud_offset folding in */ diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index e97f018..ae0777b 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -714,6 +714,198 @@ int kern_addr_valid(unsigned long addr) return pfn_valid(pte_pfn(pte)); } + +#ifdef CONFIG_MEMORY_HOTPLUG +static void __meminit free_pagetable(struct page *page, int order) +{ + unsigned long magic; + unsigned int nr_pages = 1 << order; + + if (PageReserved(page)) { + __ClearPageReserved(page); + + magic = (unsigned long)page->freelist; + if (magic == SECTION_INFO || magic == MIX_SECTION_INFO) { + while (nr_pages--) + put_page_bootmem(page++); + } else + while (nr_pages--) + free_reserved_page(page++); + } else + free_pages((unsigned long)page_address(page), order); +} + +#if (CONFIG_PGTABLE_LEVELS > 2) +static void __meminit free_pte_table(pte_t *pte_start, pmd_t *pmd, bool direct) +{ + pte_t *pte; + int i; + + for (i = 0; i < PTRS_PER_PTE; i++) { + pte = pte_start + i; + if (!pte_none(*pte)) + return; + } + + if (direct) + pgtable_page_dtor(pmd_page(*pmd)); + free_pagetable(pmd_page(*pmd), 0); + spin_lock(&init_mm.page_table_lock); + pmd_clear(pmd); + spin_unlock(&init_mm.page_table_lock); +} +#else +static void __meminit free_pte_table(pte_t *pte_start, pmd_t *pmd, bool direct) +{ +} +#endif + +#if (CONFIG_PGTABLE_LEVELS > 3) +static void __meminit free_pmd_table(pmd_t *pmd_start, pud_t *pud, bool direct) +{ + pmd_t *pmd; + int i; + + for (i = 0; i < PTRS_PER_PMD; i++) { + pmd = pmd_start + i; + if (!pmd_none(*pmd)) + return; + } + + if (direct) + pgtable_page_dtor(pud_page(*pud)); + free_pagetable(pud_page(*pud), 0); + spin_lock(&init_mm.page_table_lock); + pud_clear(pud); + spin_unlock(&init_mm.page_table_lock); +} + +static void __meminit free_pud_table(pud_t *pud_start, pgd_t *pgd, bool direct) +{ + pud_t *pud; + int i; + + for (i = 0; i < PTRS_PER_PUD; i++) { + pud = pud_start + i; + if (!pud_none(*pud)) + return; + } + + if (direct) + pgtable_page_dtor(pgd_page(*pgd)); + free_pagetable(pgd_page(*pgd), 0); + spin_lock(&init_mm.page_table_lock); + pgd_clear(pgd); + spin_unlock(&init_mm.page_table_lock); +} +#else +static void __meminit free_pmd_table(pmd_t *pmd_start, pud_t *pud, bool direct) +{ +} + +static void __meminit free_pud_table(pud_t *pud_start, pgd_t *pgd, bool direct) +{ +} +#endif + +static void __meminit +remove_pte_table(pte_t *pte_start, unsigned long addr, + unsigned long end, bool direct) +{ + pte_t *pte; + + pte = pte_start + pte_index(addr); + for (; addr < end; addr += PAGE_SIZE, pte++) { + if (!pte_present(*pte)) + continue; + + if (!direct) + free_pagetable(pte_page(*pte), 0); + spin_lock(&init_mm.page_table_lock); + pte_clear(&init_mm, addr, pte); + spin_unlock(&init_mm.page_table_lock); + } +} + +static void __meminit +remove_pmd_table(pmd_t *pmd_start, unsigned long addr, + unsigned long end, bool direct) +{ + unsigned long next; + pte_t *pte_base; + pmd_t *pmd; + + pmd = pmd_start + pmd_index(addr); + for (; addr < end; addr = next, pmd++) { + next = pmd_addr_end(addr, end); + if (!pmd_present(*pmd)) + continue; + + if (pmd_large(*pmd)) { + if (!direct) + free_pagetable(pmd_page(*pmd), + get_order(PMD_SIZE)); + spin_lock(&init_mm.page_table_lock); + pmd_clear(pmd); + spin_unlock(&init_mm.page_table_lock); + continue; + } + pte_base = pte_offset_kernel(pmd, 0UL); + remove_pte_table(pte_base, addr, next, direct); + free_pte_table(pte_base, pmd, direct); + } +} + +static void __meminit +remove_pud_table(pud_t *pud_start, unsigned long addr, + unsigned long end, bool direct) +{ + unsigned long next; + pmd_t *pmd_base; + pud_t *pud; + + pud = pud_start + pud_index(addr); + for (; addr < end; addr = next, pud++) { + next = pud_addr_end(addr, end); + if (!pud_present(*pud)) + continue; + + if (pud_large(*pud)) { + if (!direct) + free_pagetable(pud_page(*pud), + get_order(PUD_SIZE)); + spin_lock(&init_mm.page_table_lock); + pud_clear(pud); + spin_unlock(&init_mm.page_table_lock); + continue; + } + pmd_base = pmd_offset(pud, 0UL); + remove_pmd_table(pmd_base, addr, next, direct); + free_pmd_table(pmd_base, pud, direct); + } +} + +static void __meminit +remove_pagetable(unsigned long start, unsigned long end, bool direct) +{ + unsigned long addr, next; + pud_t *pud_base; + pgd_t *pgd; + + for (addr = start; addr < end; addr = next) { + next = pgd_addr_end(addr, end); + pgd = pgd_offset_k(addr); + if (!pgd_present(*pgd)) + continue; + + pud_base = pud_offset(pgd, 0UL); + remove_pud_table(pud_base, addr, next, direct); + free_pud_table(pud_base, pgd, direct); + } + flush_tlb_kernel_range(start, end); +} +#endif + #ifdef CONFIG_SPARSEMEM_VMEMMAP #if !ARM64_SWAPPER_USES_SECTION_MAPS int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, @@ -758,9 +950,12 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, return 0; } #endif /* CONFIG_ARM64_64K_PAGES */ -void vmemmap_free(unsigned long start, unsigned long end, +void __ref vmemmap_free(unsigned long start, unsigned long end, struct vmem_altmap *altmap) { +#ifdef CONFIG_MEMORY_HOTPLUG + remove_pagetable(start, end, false); +#endif } #endif /* CONFIG_SPARSEMEM_VMEMMAP */ @@ -1046,10 +1241,16 @@ int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) } #ifdef CONFIG_MEMORY_HOTPLUG +static void __remove_pgd_mapping(pgd_t *pgdir, unsigned long start, u64 size) +{ + WARN_ON(pgdir != init_mm.pgd); + remove_pagetable(start, start + size, true); +} + int arch_add_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap, bool want_memblock) { - int flags = 0; + int flags = 0, ret = 0; if (rodata_full || debug_pagealloc_enabled()) flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; @@ -1057,7 +1258,27 @@ int arch_add_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap, __create_pgd_mapping(swapper_pg_dir, start, __phys_to_virt(start), size, PAGE_KERNEL, pgd_pgtable_alloc, flags); - return __add_pages(nid, start >> PAGE_SHIFT, size >> PAGE_SHIFT, + ret = __add_pages(nid, start >> PAGE_SHIFT, size >> PAGE_SHIFT, altmap, want_memblock); + if (ret) + __remove_pgd_mapping(swapper_pg_dir, + __phys_to_virt(start), size); + return ret; } + +#ifdef CONFIG_MEMORY_HOTREMOVE +int arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap) +{ + unsigned long start_pfn = start >> PAGE_SHIFT; + unsigned long nr_pages = size >> PAGE_SHIFT; + struct zone *zone = page_zone(pfn_to_page(start_pfn)); + int ret; + + ret = __remove_pages(zone, start_pfn, nr_pages, altmap); + if (!ret) + __remove_pgd_mapping(swapper_pg_dir, + __phys_to_virt(start), size); + return ret; +} +#endif #endif From patchwork Wed Apr 3 04:30:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 10882781 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9E91017E1 for ; Wed, 3 Apr 2019 04:30:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 82DEA28913 for ; Wed, 3 Apr 2019 04:30:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7702D289BF; Wed, 3 Apr 2019 04:30:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D9F7F28913 for ; Wed, 3 Apr 2019 04:30:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E57D56B028A; Wed, 3 Apr 2019 00:30:33 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E05EA6B028C; Wed, 3 Apr 2019 00:30:33 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD3806B028D; Wed, 3 Apr 2019 00:30:33 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id 7E9BC6B028A for ; Wed, 3 Apr 2019 00:30:33 -0400 (EDT) Received: by mail-ed1-f72.google.com with SMTP id h27so6735518eda.8 for ; Tue, 02 Apr 2019 21:30:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=KerfpmvxM83iM3jUFKjZ5pNEafPHXBEirlMZMbxybmU=; b=sVC+CJxCU41tM8b81e1Nko4SKPlUZm7UXSGE4pSMVnSQIWq7wSLdAEi0c24FI5GBcB POdu+FzMHNbMoeONHhiBzCLkDfFH9r2pCkE2ePApcrAAyZpcAwCGTFLeyYtUZ4DKPq+j PTC6/hFDdmrFitFt0ZrtAHq36pWvUfLQvgWDdteHq892DQQjCQE5ZaSnzH1JxuBioLs+ 7y/807OEG/VDKjg337bSprTIW/qU9feDQBuJ0wPfcmg1HI1FNVWuEvdc1ybhV+WXxdOU HsA5LHk5gobtTL2W7azvSWsqdaNTnWEjjwdXabDHAtqWGFNgNHe7OZwnep42RfvMI/XG HV7g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com X-Gm-Message-State: APjAAAVdGI0fXxpzHDm0pmbLwP/NSsZim3Rm6Y6nQlAF1caZkkQmdiJi dn44qprJAZL+z8FV1/QF0pcpKhrYZrqVwin30DD2Qg4Dp4oWQfKVpZDoHeyQ5/EfNkcChLNNrFw Pfzc9jYJwtq0bEkOaW+h/sk2PUqKmg+Z3hosZtMEAjzoRr4Tk3TxLUVCTYIiEXk5gNA== X-Received: by 2002:a50:a705:: with SMTP id h5mr39742294edc.226.1554265833026; Tue, 02 Apr 2019 21:30:33 -0700 (PDT) X-Google-Smtp-Source: APXvYqx42/iMCculpK3p5aN50aIRe3QIts0lxdTcJyI5AGykw5n/JXxRdd8+mEwO3PCtA+HBX7wd X-Received: by 2002:a50:a705:: with SMTP id h5mr39742226edc.226.1554265831610; Tue, 02 Apr 2019 21:30:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554265831; cv=none; d=google.com; s=arc-20160816; b=axe+NUlzJ+yBvAnzGeoyi+EvYkH5DCue+QgmeVCdu8fBmJpLsDLCZJYv2O2sRMdngO axsq4qCAKBfmChSH7J3Vqk/6aFVNTwiKGcuoczrDvs8h85kV9OwrHMcdCXSWoK4LWk6W gTW+bkAuhzbtU22RBErqvuk/F8kVbIsHRu+QaRLOPe9U7h96rhJk13ydq28dytpZMr9z b+NqJUz1+XCEuVh53RERBgxRGgvmFqd1nTwvOgOyzCFzWMW2oIuGNRDpV+ekwDOA/54s WYulEpElilz+OXIP6Xe2ozW0KBYlxFnYq3r0fJysqCfbbIfJH83ZPuaN1s57h09Kwcyd /8BA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=KerfpmvxM83iM3jUFKjZ5pNEafPHXBEirlMZMbxybmU=; b=q1+4Fszj46RFGG8WZw7yBs8LVxzNycpQvgFT5tv92A2BvfLMtOlqZsROWeqdrsKG9q eESTvenXVFwDrdpM6eE4AhB4vuIcBa4Kx0Om0nCjux9fMChPhLGY9BD2Qorhfe8fj/a9 3Z9veVWHlmBGu5pZ0/1EEFXrfPJc1MaMuBLgpvm1lsNQjSqCXsv2KKF1eS/SEItT7Ask 7Dn8/4tTeQ+UWjAZSA0gMxftW2Vrik0kri4CrDdF7ktR+cX6JWnOi6SV+FK6JMHtKrt+ /R/GHH+Estyl6AlgJxB2RazoHGdfH1FvTWMrhzXj2OPG9CzbBnIUsluLS6kAgXc+ZZQ1 p14g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id d3si4018170ejc.309.2019.04.02.21.30.31 for ; Tue, 02 Apr 2019 21:30:31 -0700 (PDT) Received-SPF: pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 85EA8168F; Tue, 2 Apr 2019 21:30:30 -0700 (PDT) Received: from p8cg001049571a15.arm.com (unknown [10.163.1.97]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 2D9863F721; Tue, 2 Apr 2019 21:30:24 -0700 (PDT) From: Anshuman Khandual To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, akpm@linux-foundation.org, will.deacon@arm.com, catalin.marinas@arm.com Cc: mhocko@suse.com, mgorman@techsingularity.net, james.morse@arm.com, mark.rutland@arm.com, robin.murphy@arm.com, cpandya@codeaurora.org, arunks@codeaurora.org, dan.j.williams@intel.com, osalvador@suse.de, logang@deltatee.com, pasha.tatashin@oracle.com, david@redhat.com, cai@lca.pw Subject: [PATCH 3/6] arm64/mm: Enable struct page allocation from device memory Date: Wed, 3 Apr 2019 10:00:03 +0530 Message-Id: <1554265806-11501-4-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1554265806-11501-1-git-send-email-anshuman.khandual@arm.com> References: <1554265806-11501-1-git-send-email-anshuman.khandual@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP ZONE_DEVICE based device memory like persistent memory would typically be more than available system RAM and can have size in TBs. Allocating struct pages from system RAM for these vast range of device memory will reduce amount of system RAM available for other purposes. There is a mechanism with struct vmem_altmap which reserves range of device memory to be used for it's own struct pages. On arm64 platforms this enables vmemmap_populate() & vmemmap_free() which creates & destroys struct page mapping to accommodate a given instance of struct vmem_altmap. Signed-off-by: Anshuman Khandual --- arch/arm64/mm/mmu.c | 41 +++++++++++++++++++++++++++-------------- 1 file changed, 27 insertions(+), 14 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index ae0777b..4b25b75 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -735,6 +735,15 @@ static void __meminit free_pagetable(struct page *page, int order) free_pages((unsigned long)page_address(page), order); } +static void __meminit free_huge_pagetable(struct page *page, int order, + struct vmem_altmap *altmap) +{ + if (altmap) + vmem_altmap_free(altmap, (1UL << order)); + else + free_pagetable(page, order); +} + #if (CONFIG_PGTABLE_LEVELS > 2) static void __meminit free_pte_table(pte_t *pte_start, pmd_t *pmd, bool direct) { @@ -828,8 +837,8 @@ remove_pte_table(pte_t *pte_start, unsigned long addr, } static void __meminit -remove_pmd_table(pmd_t *pmd_start, unsigned long addr, - unsigned long end, bool direct) +remove_pmd_table(pmd_t *pmd_start, unsigned long addr, unsigned long end, + bool direct, struct vmem_altmap *altmap) { unsigned long next; pte_t *pte_base; @@ -843,8 +852,8 @@ remove_pmd_table(pmd_t *pmd_start, unsigned long addr, if (pmd_large(*pmd)) { if (!direct) - free_pagetable(pmd_page(*pmd), - get_order(PMD_SIZE)); + free_huge_pagetable(pmd_page(*pmd), + get_order(PMD_SIZE), altmap); spin_lock(&init_mm.page_table_lock); pmd_clear(pmd); spin_unlock(&init_mm.page_table_lock); @@ -857,8 +866,8 @@ remove_pmd_table(pmd_t *pmd_start, unsigned long addr, } static void __meminit -remove_pud_table(pud_t *pud_start, unsigned long addr, - unsigned long end, bool direct) +remove_pud_table(pud_t *pud_start, unsigned long addr, unsigned long end, + bool direct, struct vmem_altmap *altmap) { unsigned long next; pmd_t *pmd_base; @@ -872,21 +881,22 @@ remove_pud_table(pud_t *pud_start, unsigned long addr, if (pud_large(*pud)) { if (!direct) - free_pagetable(pud_page(*pud), - get_order(PUD_SIZE)); + free_huge_pagetable(pud_page(*pud), + get_order(PUD_SIZE), altmap); spin_lock(&init_mm.page_table_lock); pud_clear(pud); spin_unlock(&init_mm.page_table_lock); continue; } pmd_base = pmd_offset(pud, 0UL); - remove_pmd_table(pmd_base, addr, next, direct); + remove_pmd_table(pmd_base, addr, next, direct, altmap); free_pmd_table(pmd_base, pud, direct); } } static void __meminit -remove_pagetable(unsigned long start, unsigned long end, bool direct) +remove_pagetable(unsigned long start, unsigned long end, + bool direct, struct vmem_altmap *altmap) { unsigned long addr, next; pud_t *pud_base; @@ -899,7 +909,7 @@ remove_pagetable(unsigned long start, unsigned long end, bool direct) continue; pud_base = pud_offset(pgd, 0UL); - remove_pud_table(pud_base, addr, next, direct); + remove_pud_table(pud_base, addr, next, direct, altmap); free_pud_table(pud_base, pgd, direct); } flush_tlb_kernel_range(start, end); @@ -938,7 +948,10 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, if (pmd_none(READ_ONCE(*pmdp))) { void *p = NULL; - p = vmemmap_alloc_block_buf(PMD_SIZE, node); + if (altmap) + p = altmap_alloc_block_buf(PMD_SIZE, altmap); + else + p = vmemmap_alloc_block_buf(PMD_SIZE, node); if (!p) return -ENOMEM; @@ -954,7 +967,7 @@ void __ref vmemmap_free(unsigned long start, unsigned long end, struct vmem_altmap *altmap) { #ifdef CONFIG_MEMORY_HOTPLUG - remove_pagetable(start, end, false); + remove_pagetable(start, end, false, altmap); #endif } #endif /* CONFIG_SPARSEMEM_VMEMMAP */ @@ -1244,7 +1257,7 @@ int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) static void __remove_pgd_mapping(pgd_t *pgdir, unsigned long start, u64 size) { WARN_ON(pgdir != init_mm.pgd); - remove_pagetable(start, start + size, true); + remove_pagetable(start, start + size, true, NULL); } int arch_add_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap, From patchwork Wed Apr 3 04:30:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 10882785 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B9BC217E9 for ; Wed, 3 Apr 2019 04:30:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A166C28913 for ; Wed, 3 Apr 2019 04:30:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 94C79289BF; Wed, 3 Apr 2019 04:30:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B6260289A2 for ; Wed, 3 Apr 2019 04:30:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AACAC6B028C; Wed, 3 Apr 2019 00:30:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A5CA56B028E; Wed, 3 Apr 2019 00:30:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 924F86B028F; Wed, 3 Apr 2019 00:30:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id 439C76B028C for ; Wed, 3 Apr 2019 00:30:39 -0400 (EDT) Received: by mail-ed1-f69.google.com with SMTP id n24so6818022edd.21 for ; Tue, 02 Apr 2019 21:30:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=0tmsegy5nwxeJ1f1itRRHuF3L7n3Cp7BKyoDWcbyACM=; b=EOGEzWKfISYO5S1P/lIAwFXx2ySKI/1PjjEJ88B6ZMvuviJQVOrIA9h4lp2ompMCZq IlAQ8Vvoce2Caby3JP/36rrCMDct4xtUMnFx8bM73IVhVtHVx6aT6WFaz3WvY7hxFgGW Ndkw9pMUJfuTZesuNRGUjhB8SFawlzNdIk+FdBjVHw3F6mGYY42VFR2HTysdPUV+nJyy 08D2DBe36o+dazIo5FMkknXDOyav4CKvc3srnAwwph6fUel+hlJHjqbUD/bw+mmkeQ3E o1jjg6Iz9pYvfkbr1dZ6lCK7c9YBaOkZv/0x92xoAwM6oxbH/vrE1uF1w7tHLD4SVZxj Fvhw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com X-Gm-Message-State: APjAAAXHYrRMy+5VlQQWPZ7TdsrBSdnu0HTRcGGchTmHZe3kmajnwAxB hcgwWGCFYwLySFM8tVVSBU5KdLobRZXSnv9lsKSjACnE99sIT3F9eO1EE5mE+DBCO6ZFsBJ9new t7KcgRWPIDDtKMb4p6GEllRf7Z4KPSfZeZnLZPNZBx8SQJvsy8HguC3F4giBPvJtCSA== X-Received: by 2002:a17:906:eb96:: with SMTP id mh22mr16017032ejb.186.1554265838773; Tue, 02 Apr 2019 21:30:38 -0700 (PDT) X-Google-Smtp-Source: APXvYqzC7VJ//Net1Cvt9zNZG5Moa9yejZdqDviyC6FZ/mzFfITGTou8EwHwv71Ju6JDHXQHPuD1 X-Received: by 2002:a17:906:eb96:: with SMTP id mh22mr16016999ejb.186.1554265837729; Tue, 02 Apr 2019 21:30:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554265837; cv=none; d=google.com; s=arc-20160816; b=Kr7vFewF8zuHyq8DnL4FSRNSVDcTopZN8++ezNcNt5KGL8ZYcZjoRRcOn6HPlXQjQ8 3/anH3D9cqlJJqm9TuVGjYLrr0riOyh44qzhrei6doX0q/lgeth0l+3HUEbn/BtePRWK K8Ad1r8uoGW77jfpB31SNjcZetCIKVyGTruLQwBGvE9JhdqNMa0Z3htWYaKTxs2pAoKr JbHc47ckRHUH3N0CjyKXWjlQPvtJ/WXHzaGJRhHRPL3HhERvcExJjgJDR5GElsFaTaI+ assAdwnRojeUpM6ZT4RmT4ycLLESLObifV4/hb3G0XvWZqsIeFNTAp6J0L71lQGeXpwG AdWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=0tmsegy5nwxeJ1f1itRRHuF3L7n3Cp7BKyoDWcbyACM=; b=yQbAwoyHdDYHDxZ9TopL3PKNF3TeavkqRM1kqxvusCH7PjpBCwUOS03KjzNnxNTtQ5 f6N6X9+8tLlNHAkdmt6TcQHpQO7uiPQxnqRvYUKUwy6TgtUKefeefaT7JF4wAOgKCGts VyUBs/C5oL/fCqreyvtqbeC4fK1sH3RCtVfyg1iCVVsYNY3u29nVBWaOLSiYne2xaJap zTmpnAwqBAd8R3uQidHuvEdXGsJexTEJ67d5wCA0ISTHOXxvR9amdzKjyecsDdx1w6/O 3GTmUNbxMfHROZE+EHItqkDE1z/SY7jKzghW3XHspXr+0ARV5RgIcEH3PnOIwmbNIaim xUAw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id x22si2315220ejf.205.2019.04.02.21.30.37 for ; Tue, 02 Apr 2019 21:30:37 -0700 (PDT) Received-SPF: pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8B98F1993; Tue, 2 Apr 2019 21:30:36 -0700 (PDT) Received: from p8cg001049571a15.arm.com (unknown [10.163.1.97]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id EF1B43F721; Tue, 2 Apr 2019 21:30:30 -0700 (PDT) From: Anshuman Khandual To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, akpm@linux-foundation.org, will.deacon@arm.com, catalin.marinas@arm.com Cc: mhocko@suse.com, mgorman@techsingularity.net, james.morse@arm.com, mark.rutland@arm.com, robin.murphy@arm.com, cpandya@codeaurora.org, arunks@codeaurora.org, dan.j.williams@intel.com, osalvador@suse.de, logang@deltatee.com, pasha.tatashin@oracle.com, david@redhat.com, cai@lca.pw Subject: [PATCH 4/6] mm/hotplug: Reorder arch_remove_memory() call in __remove_memory() Date: Wed, 3 Apr 2019 10:00:04 +0530 Message-Id: <1554265806-11501-5-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1554265806-11501-1-git-send-email-anshuman.khandual@arm.com> References: <1554265806-11501-1-git-send-email-anshuman.khandual@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Memory hot remove uses get_nid_for_pfn() while tearing down linked sysfs entries between memory block and node. It first checks pfn validity with pfn_valid_within() before fetching nid. With CONFIG_HOLES_IN_ZONE config (arm64 has this enabled) pfn_valid_within() calls pfn_valid(). pfn_valid() is an arch implementation on arm64 (CONFIG_HAVE_ARCH_PFN_VALID) which scans all mapped memblock regions with memblock_is_map_memory(). This creates a problem in memory hot remove path which has already removed given memory range from memory block with memblock_[remove|free] before arriving at unregister_mem_sect_under_nodes(). Hence get_nid_for_pfn() returns -1 skipping subsequent sysfs_remove_link() calls leaving node <-> memory block sysfs entries as is. Subsequent memory add operation hits BUG_ON() because of existing sysfs entries. [ 62.007176] NUMA: Unknown node for memory at 0x680000000, assuming node 0 [ 62.052517] ------------[ cut here ]------------ [ 62.053211] kernel BUG at mm/memory_hotplug.c:1143! [ 62.053868] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP [ 62.054589] Modules linked in: [ 62.054999] CPU: 19 PID: 3275 Comm: bash Not tainted 5.1.0-rc2-00004-g28cea40b2683 #41 [ 62.056274] Hardware name: linux,dummy-virt (DT) [ 62.057166] pstate: 40400005 (nZcv daif +PAN -UAO) [ 62.058083] pc : add_memory_resource+0x1cc/0x1d8 [ 62.058961] lr : add_memory_resource+0x10c/0x1d8 [ 62.059842] sp : ffff0000168b3ce0 [ 62.060477] x29: ffff0000168b3ce0 x28: ffff8005db546c00 [ 62.061501] x27: 0000000000000000 x26: 0000000000000000 [ 62.062509] x25: ffff0000111ef000 x24: ffff0000111ef5d0 [ 62.063520] x23: 0000000000000000 x22: 00000006bfffffff [ 62.064540] x21: 00000000ffffffef x20: 00000000006c0000 [ 62.065558] x19: 0000000000680000 x18: 0000000000000024 [ 62.066566] x17: 0000000000000000 x16: 0000000000000000 [ 62.067579] x15: ffffffffffffffff x14: ffff8005e412e890 [ 62.068588] x13: ffff8005d6b105d8 x12: 0000000000000000 [ 62.069610] x11: ffff8005d6b10490 x10: 0000000000000040 [ 62.070615] x9 : ffff8005e412e898 x8 : ffff8005e412e890 [ 62.071631] x7 : ffff8005d6b105d8 x6 : ffff8005db546c00 [ 62.072640] x5 : 0000000000000001 x4 : 0000000000000002 [ 62.073654] x3 : ffff8005d7049480 x2 : 0000000000000002 [ 62.074666] x1 : 0000000000000003 x0 : 00000000ffffffef [ 62.075685] Process bash (pid: 3275, stack limit = 0x00000000d754280f) [ 62.076930] Call trace: [ 62.077411] add_memory_resource+0x1cc/0x1d8 [ 62.078227] __add_memory+0x70/0xa8 [ 62.078901] probe_store+0xa4/0xc8 [ 62.079561] dev_attr_store+0x18/0x28 [ 62.080270] sysfs_kf_write+0x40/0x58 [ 62.080992] kernfs_fop_write+0xcc/0x1d8 [ 62.081744] __vfs_write+0x18/0x40 [ 62.082400] vfs_write+0xa4/0x1b0 [ 62.083037] ksys_write+0x5c/0xc0 [ 62.083681] __arm64_sys_write+0x18/0x20 [ 62.084432] el0_svc_handler+0x88/0x100 [ 62.085177] el0_svc+0x8/0xc Re-ordering arch_remove_memory() with memblock_[free|remove] solves the problem on arm64 as pfn_valid() behaves correctly and returns positive as memblock for the address range still exists. arch_remove_memory() removes applicable memory sections from zone with __remove_pages() and tears down kernel linear mapping. Removing memblock regions afterwards is consistent. Signed-off-by: Anshuman Khandual Reviewed-by: Oscar Salvador Acked-by: Michal Hocko Reviewed-by: David Hildenbrand --- mm/memory_hotplug.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 0082d69..71d0d79 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1872,11 +1872,10 @@ void __ref __remove_memory(int nid, u64 start, u64 size) /* remove memmap entry */ firmware_map_remove(start, start + size, "System RAM"); + arch_remove_memory(nid, start, size, NULL); memblock_free(start, size); memblock_remove(start, size); - arch_remove_memory(nid, start, size, NULL); - try_offline_node(nid); mem_hotplug_done(); From patchwork Wed Apr 3 04:30:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 10882787 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4411717E1 for ; Wed, 3 Apr 2019 04:30:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 28C3A28913 for ; Wed, 3 Apr 2019 04:30:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1C350289BF; Wed, 3 Apr 2019 04:30:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9123828913 for ; Wed, 3 Apr 2019 04:30:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 83D7A6B028E; Wed, 3 Apr 2019 00:30:45 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7EB516B0290; Wed, 3 Apr 2019 00:30:45 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6B44A6B0291; Wed, 3 Apr 2019 00:30:45 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by kanga.kvack.org (Postfix) with ESMTP id 1D5B76B028E for ; Wed, 3 Apr 2019 00:30:45 -0400 (EDT) Received: by mail-ed1-f70.google.com with SMTP id d2so6714608edo.23 for ; Tue, 02 Apr 2019 21:30:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=EkSxpVM4Bmt2ySq+et1fZS12TxWCefQIMkki4BSNv7E=; b=Vngz61cAKspbkDFXLke0UW3OQjoiuw2KgMN41ZEJ/bkksoo4AtL2NFtvoNIrNjvO8/ vrejxD5iSNw2VMYeUiIq9LY6Q6daCuMkBTC4cUXybL/hQNAGxjHDXe3fjaIemr61qKea 0qii3rF9W83BsLVYKAkzjNXo0FDq7ivL535+JWQe6y4JslH1JAITX2fUueqmS4oU612l K6MmPb8WLqHOM4dENHzAdAkMz5mq2hIyW5i8CKewTF9Tz1pR0VrRuPt4WQJKltpN1cc4 3oUWd/ZcbbzQc9U7Ga2W/zleDZ6geP8KTqjHrENu1GEGk77jJfHKphJHKnYic4ebUISE bxIg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com X-Gm-Message-State: APjAAAUjUxn1mbzLjML2R51YPxCE6BvKR93bM7IjcDaNAIHrJ9ftl2m0 n4qTImsTxqjnPoCKFfbwdDgw+TxpUX5x5HTFExjcvY7GSXjf60yGWTmEQTIy3WgLpMT7Log1mX9 Z4wTZKnE80mlI+ba1/RvcS9JSXPWS8jiCU40qMa/w/JLFmtI5Tey+1MBmbn4HHGUFdw== X-Received: by 2002:a17:906:b80e:: with SMTP id dv14mr33097088ejb.157.1554265844573; Tue, 02 Apr 2019 21:30:44 -0700 (PDT) X-Google-Smtp-Source: APXvYqxOwjRszB9q7wGddKeDk1Pt2j7iEvUc/9HbX6WZNUjMUAs0eKi7eZUk/SgJojyt7zZFg7HH X-Received: by 2002:a17:906:b80e:: with SMTP id dv14mr33097058ejb.157.1554265843619; Tue, 02 Apr 2019 21:30:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554265843; cv=none; d=google.com; s=arc-20160816; b=Mum5CtJEBPoVSue2lIPja7WkuUQ5m+nBs8+z7fhB37ArDQZs7Ag+TZUtkMWXguCjVE dHdqkjihP2a9LZ0Mil6xTiXyPT65EO3QNDwYPGD/2skNnQmVVfjNhZVXDIhLR9hfRzIH 9fLghZdMekRkfxi3QSXHCVC3fJogpDXl66pMPLHq2Pjt+I4sHLE5Kcq+7qpUv5lOioDc 1sGKdJUa/p2sx5LofZD6p8mbc8So1lz63clBRNvS/F6SW3PeE+QGOKN6c8IsINy34BMZ Xymh2yH1pktUSpRnUPeU38Z/zjmzo3BPeCcfMM9GepYrfARrVOgIcEjnBSs5LKKFUjDK 17xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=EkSxpVM4Bmt2ySq+et1fZS12TxWCefQIMkki4BSNv7E=; b=WsnW6vzg35jXPwsB6D/DQRQIYzjXB2EudekPcn5llXuP8VJu+nSOJEc01P7k7ngmfH jk+ZBEqAAWJnqUSFdv2QLg8euLmkvBqEgiBpcsVDrHzF8tlC7HsdkN7jOf6J/NPx6ry4 Czysj/xtwcizEhr9trtdKf1onu2Nc8ijwDnFCBjcjioM2VJOmkKXQrbbE0NBZnMunxgK 92x81kOakD0Y7QJFzuGYiPvcPBwxUVE7FpuqY2Q2kjlahUEKvN6J0k+iEmYffkgxsIkO bMN2qNTXLw1yEj29cDjCi1hkrcHpWrIT2hyXHuxvhVu9zUt2gb6gJhCtNvG4/b1tpIo7 qQNw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id x28si4076547edm.210.2019.04.02.21.30.43 for ; Tue, 02 Apr 2019 21:30:43 -0700 (PDT) Received-SPF: pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7FAC780D; Tue, 2 Apr 2019 21:30:42 -0700 (PDT) Received: from p8cg001049571a15.arm.com (unknown [10.163.1.97]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0E79E3F721; Tue, 2 Apr 2019 21:30:36 -0700 (PDT) From: Anshuman Khandual To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, akpm@linux-foundation.org, will.deacon@arm.com, catalin.marinas@arm.com Cc: mhocko@suse.com, mgorman@techsingularity.net, james.morse@arm.com, mark.rutland@arm.com, robin.murphy@arm.com, cpandya@codeaurora.org, arunks@codeaurora.org, dan.j.williams@intel.com, osalvador@suse.de, logang@deltatee.com, pasha.tatashin@oracle.com, david@redhat.com, cai@lca.pw Subject: [PATCH 5/6] mm/memremap: Rename and consolidate SECTION_SIZE Date: Wed, 3 Apr 2019 10:00:05 +0530 Message-Id: <1554265806-11501-6-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1554265806-11501-1-git-send-email-anshuman.khandual@arm.com> References: <1554265806-11501-1-git-send-email-anshuman.khandual@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Robin Murphy Enabling ZONE_DEVICE (through ARCH_HAS_ZONE_DEVICE) for arm64 reveals that memremap's internal helpers for sparsemem sections conflict with arm64's definitions for hugepages which inherit the name of "sections" from earlier versions of the ARM architecture. Disambiguate memremap by propagating sparsemem's PA_ prefix, to clarify that these values are in terms of addresses rather than PFNs (and because it's a heck of a lot easier than changing all the arch code). SECTION_MASK is unused, so it can just go. While here consolidate single instance of PA_SECTION_SIZE from mm/hmm.c as well. [anshuman: Consolidated mm/hmm.c instance and updated the commit message] Signed-off-by: Robin Murphy Signed-off-by: Anshuman Khandual Acked-by: Michal Hocko Reviewed-by: David Hildenbrand --- include/linux/mmzone.h | 1 + kernel/memremap.c | 10 ++++------ mm/hmm.c | 2 -- 3 files changed, 5 insertions(+), 8 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index fba7741..ed7dd27 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -1081,6 +1081,7 @@ static inline unsigned long early_pfn_to_nid(unsigned long pfn) * PFN_SECTION_SHIFT pfn to/from section number */ #define PA_SECTION_SHIFT (SECTION_SIZE_BITS) +#define PA_SECTION_SIZE (1UL << PA_SECTION_SHIFT) #define PFN_SECTION_SHIFT (SECTION_SIZE_BITS - PAGE_SHIFT) #define NR_MEM_SECTIONS (1UL << SECTIONS_SHIFT) diff --git a/kernel/memremap.c b/kernel/memremap.c index a856cb5..dda1367 100644 --- a/kernel/memremap.c +++ b/kernel/memremap.c @@ -14,8 +14,6 @@ #include static DEFINE_XARRAY(pgmap_array); -#define SECTION_MASK ~((1UL << PA_SECTION_SHIFT) - 1) -#define SECTION_SIZE (1UL << PA_SECTION_SHIFT) #if IS_ENABLED(CONFIG_DEVICE_PRIVATE) vm_fault_t device_private_entry_fault(struct vm_area_struct *vma, @@ -98,8 +96,8 @@ static void devm_memremap_pages_release(void *data) put_page(pfn_to_page(pfn)); /* pages are dead and unused, undo the arch mapping */ - align_start = res->start & ~(SECTION_SIZE - 1); - align_size = ALIGN(res->start + resource_size(res), SECTION_SIZE) + align_start = res->start & ~(PA_SECTION_SIZE - 1); + align_size = ALIGN(res->start + resource_size(res), PA_SECTION_SIZE) - align_start; nid = page_to_nid(pfn_to_page(align_start >> PAGE_SHIFT)); @@ -154,8 +152,8 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap) if (!pgmap->ref || !pgmap->kill) return ERR_PTR(-EINVAL); - align_start = res->start & ~(SECTION_SIZE - 1); - align_size = ALIGN(res->start + resource_size(res), SECTION_SIZE) + align_start = res->start & ~(PA_SECTION_SIZE - 1); + align_size = ALIGN(res->start + resource_size(res), PA_SECTION_SIZE) - align_start; align_end = align_start + align_size - 1; diff --git a/mm/hmm.c b/mm/hmm.c index fe1cd87..ef9e4e6 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -33,8 +33,6 @@ #include #include -#define PA_SECTION_SIZE (1UL << PA_SECTION_SHIFT) - #if IS_ENABLED(CONFIG_HMM_MIRROR) static const struct mmu_notifier_ops hmm_mmu_notifier_ops; From patchwork Wed Apr 3 04:30:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 10882789 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0854D17E1 for ; Wed, 3 Apr 2019 04:30:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E5FED28613 for ; Wed, 3 Apr 2019 04:30:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DA070289A2; Wed, 3 Apr 2019 04:30:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7EF9128613 for ; Wed, 3 Apr 2019 04:30:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 703B76B0290; Wed, 3 Apr 2019 00:30:51 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6B4916B0292; Wed, 3 Apr 2019 00:30:51 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A3656B0293; Wed, 3 Apr 2019 00:30:51 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by kanga.kvack.org (Postfix) with ESMTP id 09C616B0290 for ; Wed, 3 Apr 2019 00:30:51 -0400 (EDT) Received: by mail-ed1-f70.google.com with SMTP id p88so6799967edd.17 for ; Tue, 02 Apr 2019 21:30:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=CXTPUJH63sQhSZ8X2JP4briWNtesBoJzD59pdvE9buM=; b=pbXmPuxU1kCTa2/9YEvTSfNFZ8PTYEYigIaOdAG++2DqJtcs9fBHFoeCil3XhtjoTC uOFCz5yoeApoHEoOZG7G6NKEY0dlRjwZGap836yqVJ99BabAq5WAsp9T0Gqmdh9KNXkg VaaJtm1VnRa9CFycnU81FW0uOpWWE0ZuXVKZS7Lv36KXcFV/MOswwe2v5jH1/BDXTTRG vVfn9e2ZkbDrKBigOcZplv+t/cxZslPF0MypNYFLgnXeQVLd62XcGrk35TnxeHLrfOWq hyJBYUfsg1YWbakJIgTsjAivYg4J4dWyw4KrM8i4KDs4FXX5lkKX1cXvJSDsnpqW8DU7 wrQw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com X-Gm-Message-State: APjAAAWmP9rYqglomiLOCChMi7OsB2G6VVOlFNREiLAYLcp4w2jKWHVi hDBHshaESGa19kx3CQgGLlMddAbXF+HV94ZiTIFRogV00n9Un3ZCjSW39k1ibEbl94U34rBPqrb /mVGBpLH9EKUHq/sVxEYNZbK8U5sQ1pFHVBf1zHbH35yLHpKqA1AcbvudpB9zHi8wKw== X-Received: by 2002:a17:906:c9c6:: with SMTP id hk6mr686442ejb.113.1554265850553; Tue, 02 Apr 2019 21:30:50 -0700 (PDT) X-Google-Smtp-Source: APXvYqxLxqBGk0m832ebxF0phJIxShQ8X7bD2YiZtq8c9jUfUljLxBp0A+nthCTJi4jQ1PZo2B5Y X-Received: by 2002:a17:906:c9c6:: with SMTP id hk6mr686399ejb.113.1554265849393; Tue, 02 Apr 2019 21:30:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554265849; cv=none; d=google.com; s=arc-20160816; b=t8a4qeqL2ftuf65Cl3ilNZ/TYl+b5iNCXA7ONsm5rqra2mtuqOB4Mpxi8Poooceetp X8scg/MF5wRUFdEfTIOdOcGASnQMyQ35nqVb9Mt1ALtCpW0H3Q1uH+03XvHsu+cH+PKN CNtZkZkOclgHsTMIvVPf2dE/BSrAtCO/8ReqancAGi3d4NBIUOcSHZ4XOXlGXLGP6lZX DfSlUnscuoD2N0ub0qtTOv52jpDPMPeesHM3Ki9AvdOKahthdomNX0aoyxUrL1+pbfUz o5P7Hk7GLf9+hqXSEDnrGFAdFs0WpoRl4BLq2GAOCwTwv5ZzzrT7COnc934duS3j0feI yLsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=CXTPUJH63sQhSZ8X2JP4briWNtesBoJzD59pdvE9buM=; b=ufIJRRFei1U4WXVxZIVoX+S1f0DNl2b1rgJI5gTrBtB3roJbAvI7ni92zIpccnWmnq 0dKPlS9zePhg5owyM26fYhjvOukhy6TFYLQuV7m2nljJZFXRk5i/5Piw5mvJ4CA9BZyY uwTbGgQ5rQ9sxIFvXX8XkVS3ZxCYzP3BV4i9EgEyoTZGar/JDgPjxtmHbcnD/PO3cvn0 Hgdj0Ox/ZCTe4vKJC/sM4h6I5WBVEeWp9ksMEMKQqhXSHgXZRq34euYx29BG48mMZXIQ s1xvGhWRImIc23tMkaZx+sTT+3G9TSgVRDLpCMY6tVaci0dimtCMPxsv5ReqsZsb7Uao Ts5Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id f11si1534763edt.109.2019.04.02.21.30.49 for ; Tue, 02 Apr 2019 21:30:49 -0700 (PDT) Received-SPF: pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 82E851596; Tue, 2 Apr 2019 21:30:48 -0700 (PDT) Received: from p8cg001049571a15.arm.com (unknown [10.163.1.97]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0EB103F721; Tue, 2 Apr 2019 21:30:42 -0700 (PDT) From: Anshuman Khandual To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, akpm@linux-foundation.org, will.deacon@arm.com, catalin.marinas@arm.com Cc: mhocko@suse.com, mgorman@techsingularity.net, james.morse@arm.com, mark.rutland@arm.com, robin.murphy@arm.com, cpandya@codeaurora.org, arunks@codeaurora.org, dan.j.williams@intel.com, osalvador@suse.de, logang@deltatee.com, pasha.tatashin@oracle.com, david@redhat.com, cai@lca.pw Subject: [PATCH 6/6] arm64/mm: Enable ZONE_DEVICE Date: Wed, 3 Apr 2019 10:00:06 +0530 Message-Id: <1554265806-11501-7-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1554265806-11501-1-git-send-email-anshuman.khandual@arm.com> References: <1554265806-11501-1-git-send-email-anshuman.khandual@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Arch implementation for functions which create or destroy vmemmap mapping (vmemmap_populate, vmemmap_free) can comprehend and allocate from inside device memory range through driver provided vmem_altmap structure which fulfils all requirements to enable ZONE_DEVICE on the platform. Hence just enable ZONE_DEVICE by subscribing to ARCH_HAS_ZONE_DEVICE. But this is only applicable for ARM64_4K_PAGES (ARM64_SWAPPER_USES_SECTION_MAPS) only which creates vmemmap section mappings and utilize vmem_altmap structure. Signed-off-by: Anshuman Khandual --- arch/arm64/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index db3e625..b5d8cf5 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -31,6 +31,7 @@ config ARM64 select ARCH_HAS_SYSCALL_WRAPPER select ARCH_HAS_TEARDOWN_DMA_OPS if IOMMU_SUPPORT select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST + select ARCH_HAS_ZONE_DEVICE if ARM64_4K_PAGES select ARCH_HAVE_NMI_SAFE_CMPXCHG select ARCH_INLINE_READ_LOCK if !PREEMPT select ARCH_INLINE_READ_LOCK_BH if !PREEMPT