From patchwork Sun Apr 25 14:12:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12223327 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E352C43461 for ; Sun, 25 Apr 2021 14:15:39 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B34E061369 for ; Sun, 25 Apr 2021 14:15:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B34E061369 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=XHDrNnuCcW+ke01fcTy0kdtG8Cm63Gb0eKlRpsK1kyA=; b=dpIMjVDaGjW2+D0o9/9FBLlwA jJiLQJ1Fux89x28yyVMXv2LLf+vv17TtXrH2sSqcL21uDpUgC3OhXW1wULQR0quGpwSziowLqX1n0 nY2MXOawldIaath9YrfQGxTvD7PlW5CP8y+lKZ5ucgVZ5B15JlweYOQfJwEXbRtLaINHrJL/NQ41K Q8qYWzRCEjAGyeYAPeqZ7nS6nwJjqTNQT27jzSylIbcwkmE5JkpPeTTeAE1zmMHYj2fglscth2tFX UfepggYMHexyNUtpTmyCC/+uFaoKwwbOZxtrozE15eeqh9MuTQNVAHa4L7e0ks/aM3xi+WMXdnAOS ZfI8gt3Wg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lafWP-005iI1-0o; Sun, 25 Apr 2021 14:14:05 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lafW9-005iGS-10 for linux-arm-kernel@desiato.infradead.org; Sun, 25 Apr 2021 14:13:49 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=U5gbanVTJ9Aw8b1zBkgEbWv2enAd86V77nRaPs+4B70=; b=YCohjPtbEam9W1wfo9y24PNcz8 zjgBKhcoK/Q9Ntmcr7yoDZyDrb57vH+9h+5aHK2gY63BoJ7IihAv1fRG/crLnh6AFnOnl2iJxY23D HPocDvpgxy5itJbqh6dV2zj2JcoaZMa8UU2shv1DgIpZAsPRV56neaTBGrWNymkxEdcxmMXt4MZG0 02RYinAlTC1bio59lwDXm5k3sCvXsyhurx0Yr812lYFmvKzN0RJgCIDT5u7dPPSXQUI5fKFi7+NA1 zhBDcwCU6/YUHUx9cRnYK0w1YhebsR9b+ffxO4Numgb7hhwTLZvnsvp7Ba3D5JBEm797Hl/FLPeya jT1avw/w==; Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lafW5-00FT4f-Te for linux-arm-kernel@lists.infradead.org; Sun, 25 Apr 2021 14:13:47 +0000 Received: by mail-pf1-x42a.google.com with SMTP id e15so453642pfv.10 for ; Sun, 25 Apr 2021 07:13:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=U5gbanVTJ9Aw8b1zBkgEbWv2enAd86V77nRaPs+4B70=; b=APYcP23C3nQSUnPnF2xSxmB84yPSKF4Rpg2Pjdru057jxnglVILgvEAw5mdyXG6N9y jvZis4aW6tSs3SYrHu6B27wcdrGKBNuA1RjyfdlTIxAmLwpgYOqffg5510+XL7V9JGqA SI/pPUdyTHSle32HaVtx/5CC2MZxAvDu709HmSuBcvjrf/c1vWCfJMW7xNuq7ifgeSjd tTfLqCxjZwcd68+CaoX+IRFoheyyhhr9bMhX1BA7Kp02+nKW2Ch3Vxjw1OBxZ49NlUGc oifZK+V2Hb0Gz+/o2PwzPHiylMmwO0spkYLxQSxsKOmr5ENEhsE+JsmVEtiig48UA8k6 aonw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=U5gbanVTJ9Aw8b1zBkgEbWv2enAd86V77nRaPs+4B70=; b=PWpklxREIvotKsuKspwvtGC1xHxKIoqgtfJxyzu78bcGul84hNsi/rIjLsNCzvLD62 eg7RtHEjtZtJvgwr5v7a4j5azj7cUT72pVxFrRzwbjatt88IN1cmTZ4cSxl9HJZPaKJR fLE4Ks5dnu05xMc1vT34kSrgTh3wUZVpze+LnlSpjE41NZZXvM2epqV5FP1Rrv0LQ7vD ypiuz2oCn/IQGCYYE5LouBlxH9Wr07Me5hFMbcnoyF6ec0VAxSWi74A+Zwg0gZxW4mfX 8Z7YdjCN7xRapDKZcufuNHvEy085kCJUgiA8+ZfgyhQoSHJa2aG6XX0ZDw7sSv9TXzl1 u/Vw== X-Gm-Message-State: AOAM531jIYe1TkJ1j1AmZ2XYkYW6AKzcRQIkHWZi7xUYOLDRrmTzlAEA A+oF79OrbiUGnmHDbLvLmY6WOFo1Fw== X-Google-Smtp-Source: ABdhPJybdQ2fg+d64YE4YVdQ3LK2jNNIUM4hFnaOZ0Z9NuxhMvz01Ns1jl5Ffp+Zgq5W7JKguzr3JA== X-Received: by 2002:a63:91c9:: with SMTP id l192mr12692090pge.339.1619360024841; Sun, 25 Apr 2021 07:13:44 -0700 (PDT) Received: from x1pad.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id r32sm1007139pgm.49.2021.04.25.07.13.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Apr 2021 07:13:44 -0700 (PDT) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org Cc: Pingfan Liu , Catalin Marinas , Will Deacon , Marc Zyngier , Kristina Martsenko , James Morse , Steven Price , Jonathan Cameron , Pavel Tatashin , Anshuman Khandual , Atish Patra , Mike Rapoport , Logan Gunthorpe , Mark Brown Subject: [PATCHv2 03/10] arm64/mm: change __create_pgd_mapping() to accept extra parameter for allocator Date: Sun, 25 Apr 2021 22:12:57 +0800 Message-Id: <20210425141304.32721-4-kernelfans@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210425141304.32721-1-kernelfans@gmail.com> References: <20210425141304.32721-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210425_071345_989754_EBA2DA6A X-CRM114-Status: GOOD ( 16.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The current pgtable allocator only accepts the depth of pgtable as parameter. And the allocator function itself determines the memory pool info. But incoming pgtable allocator needs an extra param to get local pool info, which directs the allocation. Here preparing the prototype for the incoming change. Signed-off-by: Pingfan Liu Cc: Catalin Marinas Cc: Will Deacon Cc: Marc Zyngier Cc: Kristina Martsenko Cc: James Morse Cc: Steven Price Cc: Jonathan Cameron Cc: Pavel Tatashin Cc: Anshuman Khandual Cc: Atish Patra Cc: Mike Rapoport Cc: Logan Gunthorpe Cc: Mark Brown To: linux-arm-kernel@lists.infradead.org --- arch/arm64/include/asm/pgalloc.h | 5 +++- arch/arm64/mm/idmap_mmu.c | 5 ++-- arch/arm64/mm/mmu.c | 31 +++++++++++++------------ arch/arm64/mm/mmu_include.c | 39 +++++++++++++++++++------------- 4 files changed, 46 insertions(+), 34 deletions(-) diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h index 555792921af0..42f602528b90 100644 --- a/arch/arm64/include/asm/pgalloc.h +++ b/arch/arm64/include/asm/pgalloc.h @@ -83,11 +83,14 @@ pmd_populate(struct mm_struct *mm, pmd_t *pmdp, pgtable_t ptep) } #define pmd_pgtable(pmd) pmd_page(pmd) +typedef phys_addr_t (*pgtable_alloc)(unsigned long shift, void *data); + extern void __create_pgd_mapping_extend(pgd_t *pgdir, unsigned int entries_cnt, phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + pgtable_alloc allocator, + void *info, int flags); #endif diff --git a/arch/arm64/mm/idmap_mmu.c b/arch/arm64/mm/idmap_mmu.c index bff1bffee321..4477cc2704a7 100644 --- a/arch/arm64/mm/idmap_mmu.c +++ b/arch/arm64/mm/idmap_mmu.c @@ -27,11 +27,12 @@ void __create_pgd_mapping_extend(pgd_t *pgdir, unsigned long virt, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + pgtable_alloc allocator, + void *info, int flags) { __create_pgd_mapping(pgdir, entries_cnt, phys, virt, size, prot, - pgtable_alloc, flags); + allocator, info, flags); } #endif diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 70a5a7b032dc..520738c43874 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -86,7 +86,7 @@ pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, } EXPORT_SYMBOL(phys_mem_access_prot); -static phys_addr_t __init early_pgtable_alloc(int shift) +static phys_addr_t __init early_pgtable_alloc(unsigned long unused_a, void *unused_b) { phys_addr_t phys; void *ptr; @@ -113,7 +113,7 @@ static phys_addr_t __init early_pgtable_alloc(int shift) return phys; } -static phys_addr_t __pgd_pgtable_alloc(int shift) +static phys_addr_t __pgd_pgtable_alloc(unsigned long unused_a, void *unused_b) { void *ptr = (void *)__get_free_page(GFP_PGTABLE_KERNEL); BUG_ON(!ptr); @@ -123,9 +123,9 @@ static phys_addr_t __pgd_pgtable_alloc(int shift) return __pa(ptr); } -static phys_addr_t pgd_pgtable_alloc(int shift) +static phys_addr_t pgd_pgtable_alloc(unsigned long shift, void *unused) { - phys_addr_t pa = __pgd_pgtable_alloc(shift); + phys_addr_t pa = __pgd_pgtable_alloc(shift, unused); /* * Call proper page table ctor in case later we need to @@ -154,7 +154,8 @@ int idmap_extend_pgtable; void create_idmap(pgd_t *pgdir, phys_addr_t phys, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + pgtable_alloc allocator, + void *info, int flags) { u64 ptrs_per_pgd = idmap_ptrs_per_pgd; @@ -162,13 +163,13 @@ void create_idmap(pgd_t *pgdir, phys_addr_t phys, #ifdef CONFIG_IDMAP_PGTABLE_EXPAND if (idmap_extend_pgtable) __create_pgd_mapping_extend(pgdir, ptrs_per_pgd, - phys, phys, size, prot, pgtable_alloc, flags); + phys, phys, size, prot, allocator, info, flags); else __create_pgd_mapping(pgdir, ptrs_per_pgd, - phys, phys, size, prot, pgtable_alloc, flags); + phys, phys, size, prot, allocator, info, flags); #else __create_pgd_mapping(pgdir, ptrs_per_pgd, - phys, phys, size, prot, pgtable_alloc, flags); + phys, phys, size, prot, allocator, info, flags); #endif } @@ -186,7 +187,7 @@ static void __init create_mapping_noalloc(phys_addr_t phys, unsigned long virt, return; } __create_pgd_mapping(init_mm.pgd, PTRS_PER_PGD, phys, virt, size, prot, NULL, - NO_CONT_MAPPINGS); + NULL, NO_CONT_MAPPINGS); } void __init create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, @@ -201,7 +202,7 @@ void __init create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; __create_pgd_mapping(mm->pgd, PTRS_PER_PGD, phys, virt, size, prot, - pgd_pgtable_alloc, flags); + pgd_pgtable_alloc, NULL, flags); } static void update_mapping_prot(phys_addr_t phys, unsigned long virt, @@ -214,7 +215,7 @@ static void update_mapping_prot(phys_addr_t phys, unsigned long virt, } __create_pgd_mapping(init_mm.pgd, PTRS_PER_PGD, phys, virt, size, prot, NULL, - NO_CONT_MAPPINGS); + NULL, NO_CONT_MAPPINGS); /* flush the TLBs after updating live kernel mappings */ flush_tlb_kernel_range(virt, virt + size); @@ -224,7 +225,7 @@ static void __init __map_memblock(pgd_t *pgdp, phys_addr_t start, phys_addr_t end, pgprot_t prot, int flags) { __create_pgd_mapping(pgdp, PTRS_PER_PGD, start, __phys_to_virt(start), end - start, - prot, early_pgtable_alloc, flags); + prot, early_pgtable_alloc, NULL, flags); } void __init mark_linear_text_alias_ro(void) @@ -325,7 +326,7 @@ static void __init map_kernel_segment(pgd_t *pgdp, void *va_start, void *va_end, BUG_ON(!PAGE_ALIGNED(size)); __create_pgd_mapping(pgdp, PTRS_PER_PGD, pa_start, (unsigned long)va_start, size, prot, - early_pgtable_alloc, flags); + early_pgtable_alloc, NULL, flags); if (!(vm_flags & VM_NO_GUARD)) size += PAGE_SIZE; @@ -369,7 +370,7 @@ static int __init map_entry_trampoline(void) /* Map only the text into the trampoline page table */ memset(tramp_pg_dir, 0, PGD_SIZE); __create_pgd_mapping(tramp_pg_dir, PTRS_PER_PGD, pa_start, TRAMP_VALIAS, PAGE_SIZE, - prot, __pgd_pgtable_alloc, 0); + prot, __pgd_pgtable_alloc, NULL, 0); /* Map both the text and data into the kernel page table */ __set_fixmap(FIX_ENTRY_TRAMP_TEXT, pa_start, prot); @@ -1261,7 +1262,7 @@ int arch_add_memory(int nid, u64 start, u64 size, flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; __create_pgd_mapping(swapper_pg_dir, PTRS_PER_PGD, start, __phys_to_virt(start), - size, params->pgprot, __pgd_pgtable_alloc, + size, params->pgprot, __pgd_pgtable_alloc, NULL, flags); memblock_clear_nomap(start, size); diff --git a/arch/arm64/mm/mmu_include.c b/arch/arm64/mm/mmu_include.c index be51689d1133..732e603fe3fc 100644 --- a/arch/arm64/mm/mmu_include.c +++ b/arch/arm64/mm/mmu_include.c @@ -69,7 +69,8 @@ static void init_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + pgtable_alloc allocator, + void *info, int flags) { unsigned long next; @@ -79,8 +80,8 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, if (pmd_none(pmd)) { phys_addr_t pte_phys; - BUG_ON(!pgtable_alloc); - pte_phys = pgtable_alloc(PAGE_SHIFT); + BUG_ON(!allocator); + pte_phys = allocator(PAGE_SHIFT, info); __pmd_populate(pmdp, pte_phys, PMD_TYPE_TABLE); pmd = READ_ONCE(*pmdp); } @@ -104,7 +105,9 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), int flags) + pgtable_alloc allocator, + void *info, + int flags) { unsigned long next; pmd_t *pmdp; @@ -128,7 +131,7 @@ static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end, READ_ONCE(pmd_val(*pmdp)))); } else { alloc_init_cont_pte(pmdp, addr, next, phys, prot, - pgtable_alloc, flags); + allocator, info, flags); BUG_ON(pmd_val(old_pmd) != 0 && pmd_val(old_pmd) != READ_ONCE(pmd_val(*pmdp))); @@ -142,7 +145,9 @@ static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end, static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), int flags) + pgtable_alloc allocator, + void *info, + int flags) { unsigned long next; pud_t pud = READ_ONCE(*pudp); @@ -154,8 +159,8 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, if (pud_none(pud)) { phys_addr_t pmd_phys; - BUG_ON(!pgtable_alloc); - pmd_phys = pgtable_alloc(PMD_SHIFT); + BUG_ON(!allocator); + pmd_phys = allocator(PMD_SHIFT, info); __pud_populate(pudp, pmd_phys, PUD_TYPE_TABLE); pud = READ_ONCE(*pudp); } @@ -171,7 +176,7 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, (flags & NO_CONT_MAPPINGS) == 0) __prot = __pgprot(pgprot_val(prot) | PTE_CONT); - init_pmd(pudp, addr, next, phys, __prot, pgtable_alloc, flags); + init_pmd(pudp, addr, next, phys, __prot, allocator, info, flags); phys += next - addr; } while (addr = next, addr != end); @@ -191,7 +196,8 @@ static inline bool use_1G_block(unsigned long addr, unsigned long next, static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + pgtable_alloc allocator, + void *info, int flags) { unsigned long next; @@ -202,8 +208,8 @@ static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, if (p4d_none(p4d)) { phys_addr_t pud_phys; - BUG_ON(!pgtable_alloc); - pud_phys = pgtable_alloc(PUD_SHIFT); + BUG_ON(!allocator); + pud_phys = allocator(PUD_SHIFT, info); __p4d_populate(p4dp, pud_phys, PUD_TYPE_TABLE); p4d = READ_ONCE(*p4dp); } @@ -230,7 +236,7 @@ static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, READ_ONCE(pud_val(*pudp)))); } else { alloc_init_cont_pmd(pudp, addr, next, phys, prot, - pgtable_alloc, flags); + allocator, info, flags); BUG_ON(pud_val(old_pud) != 0 && pud_val(old_pud) != READ_ONCE(pud_val(*pudp))); @@ -244,7 +250,8 @@ static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, static void __create_pgd_mapping(pgd_t *pgdir, unsigned int entries_cnt, phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + pgtable_alloc allocator, + void *info, int flags) { unsigned long addr, end, next; @@ -268,8 +275,8 @@ static void __create_pgd_mapping(pgd_t *pgdir, unsigned int entries_cnt, phys_ad do { next = pgd_addr_end(addr, end); - alloc_init_pud(pgdp, addr, next, phys, prot, pgtable_alloc, - flags); + alloc_init_pud(pgdp, addr, next, phys, prot, allocator, + info, flags); phys += next - addr; } while (pgdp++, addr = next, addr != end); }