From patchwork Wed Apr 3 04:30:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 10882791 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 684791708 for ; Wed, 3 Apr 2019 04:30:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 497C428613 for ; Wed, 3 Apr 2019 04:30:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3C8D6289A2; Wed, 3 Apr 2019 04:30:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A31BD28613 for ; Wed, 3 Apr 2019 04:30:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=8GlYSAT5nHNyNkaO9HFdqmFDM2hRS8ECOPVudlwnc0U=; b=gv9CFP/zrbF05+3zpa83bCh2tW TkJ+eBpVMULYyhRqIBSo1ucRnzr4wN9qSkYvjfNPd0l3fMuH3DrJiSA1V5D5RTf+MeuTyDIVJtxfq TtFhSwPZBTu4LYsBaavh2w51IABEim8Pz3DmFoONjHLnsxnKlpFuK4EwEoKTG4Tsws9K4PAwK93+j IVbrYhJPZamaXZeLLsFqrflrhUFdUBJ3AlsKvwsRfvcwNQ5qQAjN4tC8VoAWOHB1kEdh9CXrSiPgF zwmRryNvOyC8HZtVpzzJOGoy7fCzvIcF1OuF/XnCnxqYiHZ+biJM+6+Y0A2pF1exA06MQRov1v1Nk kLsKG3Mw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hBXY4-0004If-D2; Wed, 03 Apr 2019 04:30:52 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hBXXi-0003qG-QS for linux-arm-kernel@lists.infradead.org; Wed, 03 Apr 2019 04:30:37 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 85EA8168F; Tue, 2 Apr 2019 21:30:30 -0700 (PDT) Received: from p8cg001049571a15.arm.com (unknown [10.163.1.97]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 2D9863F721; Tue, 2 Apr 2019 21:30:24 -0700 (PDT) From: Anshuman Khandual To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, akpm@linux-foundation.org, will.deacon@arm.com, catalin.marinas@arm.com Subject: [PATCH 3/6] arm64/mm: Enable struct page allocation from device memory Date: Wed, 3 Apr 2019 10:00:03 +0530 Message-Id: <1554265806-11501-4-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1554265806-11501-1-git-send-email-anshuman.khandual@arm.com> References: <1554265806-11501-1-git-send-email-anshuman.khandual@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190402_213031_724698_A221CB16 X-CRM114-Status: GOOD ( 12.99 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, mhocko@suse.com, david@redhat.com, robin.murphy@arm.com, cai@lca.pw, pasha.tatashin@oracle.com, logang@deltatee.com, james.morse@arm.com, cpandya@codeaurora.org, arunks@codeaurora.org, dan.j.williams@intel.com, mgorman@techsingularity.net, osalvador@suse.de MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP ZONE_DEVICE based device memory like persistent memory would typically be more than available system RAM and can have size in TBs. Allocating struct pages from system RAM for these vast range of device memory will reduce amount of system RAM available for other purposes. There is a mechanism with struct vmem_altmap which reserves range of device memory to be used for it's own struct pages. On arm64 platforms this enables vmemmap_populate() & vmemmap_free() which creates & destroys struct page mapping to accommodate a given instance of struct vmem_altmap. Signed-off-by: Anshuman Khandual --- arch/arm64/mm/mmu.c | 41 +++++++++++++++++++++++++++-------------- 1 file changed, 27 insertions(+), 14 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index ae0777b..4b25b75 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -735,6 +735,15 @@ static void __meminit free_pagetable(struct page *page, int order) free_pages((unsigned long)page_address(page), order); } +static void __meminit free_huge_pagetable(struct page *page, int order, + struct vmem_altmap *altmap) +{ + if (altmap) + vmem_altmap_free(altmap, (1UL << order)); + else + free_pagetable(page, order); +} + #if (CONFIG_PGTABLE_LEVELS > 2) static void __meminit free_pte_table(pte_t *pte_start, pmd_t *pmd, bool direct) { @@ -828,8 +837,8 @@ remove_pte_table(pte_t *pte_start, unsigned long addr, } static void __meminit -remove_pmd_table(pmd_t *pmd_start, unsigned long addr, - unsigned long end, bool direct) +remove_pmd_table(pmd_t *pmd_start, unsigned long addr, unsigned long end, + bool direct, struct vmem_altmap *altmap) { unsigned long next; pte_t *pte_base; @@ -843,8 +852,8 @@ remove_pmd_table(pmd_t *pmd_start, unsigned long addr, if (pmd_large(*pmd)) { if (!direct) - free_pagetable(pmd_page(*pmd), - get_order(PMD_SIZE)); + free_huge_pagetable(pmd_page(*pmd), + get_order(PMD_SIZE), altmap); spin_lock(&init_mm.page_table_lock); pmd_clear(pmd); spin_unlock(&init_mm.page_table_lock); @@ -857,8 +866,8 @@ remove_pmd_table(pmd_t *pmd_start, unsigned long addr, } static void __meminit -remove_pud_table(pud_t *pud_start, unsigned long addr, - unsigned long end, bool direct) +remove_pud_table(pud_t *pud_start, unsigned long addr, unsigned long end, + bool direct, struct vmem_altmap *altmap) { unsigned long next; pmd_t *pmd_base; @@ -872,21 +881,22 @@ remove_pud_table(pud_t *pud_start, unsigned long addr, if (pud_large(*pud)) { if (!direct) - free_pagetable(pud_page(*pud), - get_order(PUD_SIZE)); + free_huge_pagetable(pud_page(*pud), + get_order(PUD_SIZE), altmap); spin_lock(&init_mm.page_table_lock); pud_clear(pud); spin_unlock(&init_mm.page_table_lock); continue; } pmd_base = pmd_offset(pud, 0UL); - remove_pmd_table(pmd_base, addr, next, direct); + remove_pmd_table(pmd_base, addr, next, direct, altmap); free_pmd_table(pmd_base, pud, direct); } } static void __meminit -remove_pagetable(unsigned long start, unsigned long end, bool direct) +remove_pagetable(unsigned long start, unsigned long end, + bool direct, struct vmem_altmap *altmap) { unsigned long addr, next; pud_t *pud_base; @@ -899,7 +909,7 @@ remove_pagetable(unsigned long start, unsigned long end, bool direct) continue; pud_base = pud_offset(pgd, 0UL); - remove_pud_table(pud_base, addr, next, direct); + remove_pud_table(pud_base, addr, next, direct, altmap); free_pud_table(pud_base, pgd, direct); } flush_tlb_kernel_range(start, end); @@ -938,7 +948,10 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, if (pmd_none(READ_ONCE(*pmdp))) { void *p = NULL; - p = vmemmap_alloc_block_buf(PMD_SIZE, node); + if (altmap) + p = altmap_alloc_block_buf(PMD_SIZE, altmap); + else + p = vmemmap_alloc_block_buf(PMD_SIZE, node); if (!p) return -ENOMEM; @@ -954,7 +967,7 @@ void __ref vmemmap_free(unsigned long start, unsigned long end, struct vmem_altmap *altmap) { #ifdef CONFIG_MEMORY_HOTPLUG - remove_pagetable(start, end, false); + remove_pagetable(start, end, false, altmap); #endif } #endif /* CONFIG_SPARSEMEM_VMEMMAP */ @@ -1244,7 +1257,7 @@ int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) static void __remove_pgd_mapping(pgd_t *pgdir, unsigned long start, u64 size) { WARN_ON(pgdir != init_mm.pgd); - remove_pagetable(start, start + size, true); + remove_pagetable(start, start + size, true, NULL); } int arch_add_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap,