From patchwork Tue Apr 8 09:52:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14042564 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A7BBC3600C for ; Tue, 8 Apr 2025 09:53:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 586B6280004; Tue, 8 Apr 2025 05:53:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 534B2280001; Tue, 8 Apr 2025 05:53:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3FDC5280004; Tue, 8 Apr 2025 05:53:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 2298A280001 for ; Tue, 8 Apr 2025 05:53:18 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B8F3016068A for ; Tue, 8 Apr 2025 09:53:17 +0000 (UTC) X-FDA: 83310413634.02.BCB64D5 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf19.hostedemail.com (Postfix) with ESMTP id 203781A000A for ; Tue, 8 Apr 2025 09:53:15 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744105996; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H3XPZUFeB3j0oUqZhyoJ7SKF30TvT5YWB8oVd5GbDSs=; b=iPRWODv5BVzsZfzUdEHfPfpJZC/Fb9Jc6VNhci2fFvXwY7WMTVp+Ry+B4gQ6EXhMAxB2Sl 58MhpwJsb6bFm6L+yt1gOHOjAjhZ41yD61t9/0KPGXMSB7GcIV0FV6hp8hZ5UaAv9lSyFQ h1vVhLfYyX00AjEcd7v5kTKo8JBcvqY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744105996; a=rsa-sha256; cv=none; b=p+qszfdxcgGLmpXvMowMDyHaPhVeFR+LT/e/vSkezqYPMR3M25ecY1XpDTz72gZ5k1IY1C K2Pr7TvDRePVQ3dAy89QZxn5xLPobvHPpYNMigRqtIVBQoG1Gu8+XgB0Et+FAtQ+ZD4Mpa L1gY1cgEuTM8MIOBi3Ap8AnfS2ub9AE= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 021EA1691; Tue, 8 Apr 2025 02:53:16 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 98B8B3F6A8; Tue, 8 Apr 2025 02:53:10 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v2 01/12] mm: Pass mm down to pagetable_{pte,pmd}_ctor Date: Tue, 8 Apr 2025 10:52:11 +0100 Message-ID: <20250408095222.860601-2-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250408095222.860601-1-kevin.brodsky@arm.com> References: <20250408095222.860601-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 203781A000A X-Stat-Signature: n3d7643do8k9oostpoprx7ywpkz9mn56 X-HE-Tag: 1744105995-167886 X-HE-Meta: U2FsdGVkX1+IZgR8aj3hwwbwG3CJqrNL4T24sSev0iPoSGgjXbE8ijn0/xAKOHhSqYrxbr6alp9LR4e0BcX8tmLuSnLe1E79oVICV3hsa9qvn9j/cNudSOZ5XyhBWo05lhpv5qu9X2/wQ7etzYQgfInO30ndj116wGXW7kseSszPpV/w1rkZhPQ4jvCOuylR7k0xnf8bXQS7M4PSx7POvP70Rj47kSgWSbiVS4dQXfZqfBGIfJPzY4cO/4bFTvyVcDIsCwl0DXRZguOdrDQL9RRh55kr5OFeOvu2CNcHpH3PgMVPxeMWhoKziapJmA33dGYQiKMwvmFHRLFnhzSmmS1Ldp8cEs1GnW2PAyyCTU6jwb4NLdirkxtHyPlqOhUwmAT9srLpkGWxu6U175jvhUyvYjTAbdgLx0U0dIitLf7DEJIGy8nNsmXmC+CfCbkvy/d3k7stZN9KOxmBXTajvbincZbTSvyrZ3MsKB1TuOw5upYDlVI2hUOduF7aYUL1oEKh4IQA5Ej35A9YM4wzxjOur6lsZKZf4vsymHjDYYVxhz96Exd1fDJ6WKfQkcEMiBa9kv2hkhMwezCOvHZc//ikYMJTvyyzhGYM9+mQbb+9g4P4pMHiHw/qkpa9nAZuCf8BXZ0XGTbHB4FY/DyM49bcdlcl/8uMql3AgOTjkB8uuOWKG+66CXgVOOcMDNmyTe9w1LyYOipQlcYA2CYiZcqwnR5dVlOrnQ5n6GP3qgMoC8vB1LKMhzxD10pqh+f5y5CnZdR2o5fhu7xuHg2mfOrGInBOeoR5wVOpktCESX8YefVFP0b/j7ktkF64ghLV2wdkDJGBf3cf7HtctkCa/rN4OkQrZyM29CYOC5J4t+Tgw+KW8riHB9Mivfj6flMa6DEAXMQIhJHj+gAEfio49P2Fl4q2vsj1pigbMLPK5w1f6Ym0Gxr5eOdaIMIdqDEEDAayNZUF9CXlbbIz30l USrGpxyU qisl66/Up9gm7XGwJ+Eq/tkxE/3UpkBo0At4eo/PaZ1mAVKnGeeu3bZxePmquKr6HtdYuSR4Nsj4dO9TS+heeWhX7KfC+DW2Zv4FWbzHPoG1G7aJ2OjL3qCJrg92zvkICzgLbVivU443iynO8QOz3RCjXOAYFOe5J7Bq2eQd6a2dj7b0W8TB71jFVRyXuM8tRFtm15Xwmg7hII1o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In preparation for calling constructors for all kernel page tables while eliding unnecessary ptlock initialisation, let's pass down the associated mm to the PTE/PMD level ctors. (These are the two levels where ptlocks are used.) In most cases the mm is already around at the point of calling the ctor so we simply pass it down. This is however not the case for special page table allocators: * arch/arm/mm/mmu.c * arch/arm64/mm/mmu.c * arch/riscv/mm/init.c In those cases, the page tables being allocated are either for standard kernel memory (init_mm) or special page directories, which may not be associated to any mm. For now let's pass NULL as mm; this will be refined where possible in future patches. No functional change in this patch. Signed-off-by: Kevin Brodsky --- arch/arm/mm/mmu.c | 2 +- arch/arm64/mm/mmu.c | 4 ++-- arch/loongarch/include/asm/pgalloc.h | 2 +- arch/m68k/include/asm/mcf_pgalloc.h | 2 +- arch/m68k/include/asm/motorola_pgalloc.h | 10 +++++----- arch/m68k/mm/motorola.c | 6 +++--- arch/mips/include/asm/pgalloc.h | 2 +- arch/parisc/include/asm/pgalloc.h | 2 +- arch/powerpc/mm/book3s64/pgtable.c | 2 +- arch/powerpc/mm/pgtable-frag.c | 2 +- arch/riscv/mm/init.c | 4 ++-- arch/s390/include/asm/pgalloc.h | 2 +- arch/s390/mm/pgalloc.c | 2 +- arch/sparc/mm/init_64.c | 2 +- arch/sparc/mm/srmmu.c | 2 +- arch/x86/mm/pgtable.c | 2 +- include/asm-generic/pgalloc.h | 4 ++-- include/linux/mm.h | 6 ++++-- 18 files changed, 30 insertions(+), 28 deletions(-) diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c index f02f872ea8a9..edb7f56b7c91 100644 --- a/arch/arm/mm/mmu.c +++ b/arch/arm/mm/mmu.c @@ -735,7 +735,7 @@ static void *__init late_alloc(unsigned long sz) void *ptdesc = pagetable_alloc(GFP_PGTABLE_KERNEL & ~__GFP_HIGHMEM, get_order(sz)); - if (!ptdesc || !pagetable_pte_ctor(ptdesc)) + if (!ptdesc || !pagetable_pte_ctor(NULL, ptdesc)) BUG(); return ptdesc_to_virt(ptdesc); } diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index ea6695d53fb9..8c5c471cfb06 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -494,9 +494,9 @@ static phys_addr_t pgd_pgtable_alloc(int shift) * folded, and if so pagetable_pte_ctor() becomes nop. */ if (shift == PAGE_SHIFT) - BUG_ON(!pagetable_pte_ctor(ptdesc)); + BUG_ON(!pagetable_pte_ctor(NULL, ptdesc)); else if (shift == PMD_SHIFT) - BUG_ON(!pagetable_pmd_ctor(ptdesc)); + BUG_ON(!pagetable_pmd_ctor(NULL, ptdesc)); return pa; } diff --git a/arch/loongarch/include/asm/pgalloc.h b/arch/loongarch/include/asm/pgalloc.h index b58f587f0f0a..1c63a9d9a6d3 100644 --- a/arch/loongarch/include/asm/pgalloc.h +++ b/arch/loongarch/include/asm/pgalloc.h @@ -69,7 +69,7 @@ static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address) if (!ptdesc) return NULL; - if (!pagetable_pmd_ctor(ptdesc)) { + if (!pagetable_pmd_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/m68k/include/asm/mcf_pgalloc.h b/arch/m68k/include/asm/mcf_pgalloc.h index 4c648b51e7fd..465a71101b7d 100644 --- a/arch/m68k/include/asm/mcf_pgalloc.h +++ b/arch/m68k/include/asm/mcf_pgalloc.h @@ -48,7 +48,7 @@ static inline pgtable_t pte_alloc_one(struct mm_struct *mm) if (!ptdesc) return NULL; - if (!pagetable_pte_ctor(ptdesc)) { + if (!pagetable_pte_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/m68k/include/asm/motorola_pgalloc.h b/arch/m68k/include/asm/motorola_pgalloc.h index 5abe7da8ac5a..1091fb0affbe 100644 --- a/arch/m68k/include/asm/motorola_pgalloc.h +++ b/arch/m68k/include/asm/motorola_pgalloc.h @@ -15,7 +15,7 @@ enum m68k_table_types { }; extern void init_pointer_table(void *table, int type); -extern void *get_pointer_table(int type); +extern void *get_pointer_table(struct mm_struct *mm, int type); extern int free_pointer_table(void *table, int type); /* @@ -26,7 +26,7 @@ extern int free_pointer_table(void *table, int type); static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) { - return get_pointer_table(TABLE_PTE); + return get_pointer_table(mm, TABLE_PTE); } static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) @@ -36,7 +36,7 @@ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) static inline pgtable_t pte_alloc_one(struct mm_struct *mm) { - return get_pointer_table(TABLE_PTE); + return get_pointer_table(mm, TABLE_PTE); } static inline void pte_free(struct mm_struct *mm, pgtable_t pgtable) @@ -53,7 +53,7 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pgtable, static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address) { - return get_pointer_table(TABLE_PMD); + return get_pointer_table(mm, TABLE_PMD); } static inline int pmd_free(struct mm_struct *mm, pmd_t *pmd) @@ -75,7 +75,7 @@ static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd) static inline pgd_t *pgd_alloc(struct mm_struct *mm) { - return get_pointer_table(TABLE_PGD); + return get_pointer_table(mm, TABLE_PGD); } diff --git a/arch/m68k/mm/motorola.c b/arch/m68k/mm/motorola.c index 73651e093c4d..6ab3ef39ba7a 100644 --- a/arch/m68k/mm/motorola.c +++ b/arch/m68k/mm/motorola.c @@ -139,7 +139,7 @@ void __init init_pointer_table(void *table, int type) return; } -void *get_pointer_table(int type) +void *get_pointer_table(struct mm_struct *mm, int type) { ptable_desc *dp = ptable_list[type].next; unsigned int mask = list_empty(&ptable_list[type]) ? 0 : PD_MARKBITS(dp); @@ -164,10 +164,10 @@ void *get_pointer_table(int type) * m68k doesn't have SPLIT_PTE_PTLOCKS for not having * SMP. */ - pagetable_pte_ctor(virt_to_ptdesc(page)); + pagetable_pte_ctor(mm, virt_to_ptdesc(page)); break; case TABLE_PMD: - pagetable_pmd_ctor(virt_to_ptdesc(page)); + pagetable_pmd_ctor(mm, virt_to_ptdesc(page)); break; case TABLE_PGD: pagetable_pgd_ctor(virt_to_ptdesc(page)); diff --git a/arch/mips/include/asm/pgalloc.h b/arch/mips/include/asm/pgalloc.h index bbca420c96d3..942af87f1cdd 100644 --- a/arch/mips/include/asm/pgalloc.h +++ b/arch/mips/include/asm/pgalloc.h @@ -62,7 +62,7 @@ static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address) if (!ptdesc) return NULL; - if (!pagetable_pmd_ctor(ptdesc)) { + if (!pagetable_pmd_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/parisc/include/asm/pgalloc.h b/arch/parisc/include/asm/pgalloc.h index 2ca74a56415c..3b84ee93edaa 100644 --- a/arch/parisc/include/asm/pgalloc.h +++ b/arch/parisc/include/asm/pgalloc.h @@ -39,7 +39,7 @@ static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address) ptdesc = pagetable_alloc(gfp, PMD_TABLE_ORDER); if (!ptdesc) return NULL; - if (!pagetable_pmd_ctor(ptdesc)) { + if (!pagetable_pmd_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/pgtable.c index 8f7d41ce2ca1..a282233c8785 100644 --- a/arch/powerpc/mm/book3s64/pgtable.c +++ b/arch/powerpc/mm/book3s64/pgtable.c @@ -422,7 +422,7 @@ static pmd_t *__alloc_for_pmdcache(struct mm_struct *mm) ptdesc = pagetable_alloc(gfp, 0); if (!ptdesc) return NULL; - if (!pagetable_pmd_ctor(ptdesc)) { + if (!pagetable_pmd_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/powerpc/mm/pgtable-frag.c b/arch/powerpc/mm/pgtable-frag.c index 713268ccb1a0..387e9b1fe12c 100644 --- a/arch/powerpc/mm/pgtable-frag.c +++ b/arch/powerpc/mm/pgtable-frag.c @@ -61,7 +61,7 @@ static pte_t *__alloc_for_ptecache(struct mm_struct *mm, int kernel) ptdesc = pagetable_alloc(PGALLOC_GFP | __GFP_ACCOUNT, 0); if (!ptdesc) return NULL; - if (!pagetable_pte_ctor(ptdesc)) { + if (!pagetable_pte_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index ab475ec6ca42..e5ef693fc778 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -442,7 +442,7 @@ static phys_addr_t __meminit alloc_pte_late(uintptr_t va) { struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL & ~__GFP_HIGHMEM, 0); - BUG_ON(!ptdesc || !pagetable_pte_ctor(ptdesc)); + BUG_ON(!ptdesc || !pagetable_pte_ctor(NULL, ptdesc)); return __pa((pte_t *)ptdesc_address(ptdesc)); } @@ -522,7 +522,7 @@ static phys_addr_t __meminit alloc_pmd_late(uintptr_t va) { struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL & ~__GFP_HIGHMEM, 0); - BUG_ON(!ptdesc || !pagetable_pmd_ctor(ptdesc)); + BUG_ON(!ptdesc || !pagetable_pmd_ctor(NULL, ptdesc)); return __pa((pmd_t *)ptdesc_address(ptdesc)); } diff --git a/arch/s390/include/asm/pgalloc.h b/arch/s390/include/asm/pgalloc.h index 005497ffebda..5345398df653 100644 --- a/arch/s390/include/asm/pgalloc.h +++ b/arch/s390/include/asm/pgalloc.h @@ -97,7 +97,7 @@ static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long vmaddr) if (!table) return NULL; crst_table_init(table, _SEGMENT_ENTRY_EMPTY); - if (!pagetable_pmd_ctor(virt_to_ptdesc(table))) { + if (!pagetable_pmd_ctor(mm, virt_to_ptdesc(table))) { crst_table_free(mm, table); return NULL; } diff --git a/arch/s390/mm/pgalloc.c b/arch/s390/mm/pgalloc.c index e3a6f8ae156c..619d6917e3b7 100644 --- a/arch/s390/mm/pgalloc.c +++ b/arch/s390/mm/pgalloc.c @@ -145,7 +145,7 @@ unsigned long *page_table_alloc(struct mm_struct *mm) ptdesc = pagetable_alloc(GFP_KERNEL, 0); if (!ptdesc) return NULL; - if (!pagetable_pte_ctor(ptdesc)) { + if (!pagetable_pte_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c index 760818950464..5c8eabda1d17 100644 --- a/arch/sparc/mm/init_64.c +++ b/arch/sparc/mm/init_64.c @@ -2895,7 +2895,7 @@ pgtable_t pte_alloc_one(struct mm_struct *mm) if (!ptdesc) return NULL; - if (!pagetable_pte_ctor(ptdesc)) { + if (!pagetable_pte_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c index dd32711022f5..f8fb4911d360 100644 --- a/arch/sparc/mm/srmmu.c +++ b/arch/sparc/mm/srmmu.c @@ -350,7 +350,7 @@ pgtable_t pte_alloc_one(struct mm_struct *mm) page = pfn_to_page(__nocache_pa((unsigned long)ptep) >> PAGE_SHIFT); spin_lock(&mm->page_table_lock); if (page_ref_inc_return(page) == 2 && - !pagetable_pte_ctor(page_ptdesc(page))) { + !pagetable_pte_ctor(mm, page_ptdesc(page))) { page_ref_dec(page); ptep = NULL; } diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index a05fcddfc811..7930f234c5f6 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -205,7 +205,7 @@ static int preallocate_pmds(struct mm_struct *mm, pmd_t *pmds[], int count) if (!ptdesc) failed = true; - if (ptdesc && !pagetable_pmd_ctor(ptdesc)) { + if (ptdesc && !pagetable_pmd_ctor(mm, ptdesc)) { pagetable_free(ptdesc); ptdesc = NULL; failed = true; diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h index 892ece4558a2..e164ca66f0f6 100644 --- a/include/asm-generic/pgalloc.h +++ b/include/asm-generic/pgalloc.h @@ -70,7 +70,7 @@ static inline pgtable_t __pte_alloc_one_noprof(struct mm_struct *mm, gfp_t gfp) ptdesc = pagetable_alloc_noprof(gfp, 0); if (!ptdesc) return NULL; - if (!pagetable_pte_ctor(ptdesc)) { + if (!pagetable_pte_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } @@ -137,7 +137,7 @@ static inline pmd_t *pmd_alloc_one_noprof(struct mm_struct *mm, unsigned long ad ptdesc = pagetable_alloc_noprof(gfp, 0); if (!ptdesc) return NULL; - if (!pagetable_pmd_ctor(ptdesc)) { + if (!pagetable_pmd_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/include/linux/mm.h b/include/linux/mm.h index b7f13f087954..f9b793cce2c1 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3100,7 +3100,8 @@ static inline void pagetable_dtor_free(struct ptdesc *ptdesc) pagetable_free(ptdesc); } -static inline bool pagetable_pte_ctor(struct ptdesc *ptdesc) +static inline bool pagetable_pte_ctor(struct mm_struct *mm, + struct ptdesc *ptdesc) { if (!ptlock_init(ptdesc)) return false; @@ -3206,7 +3207,8 @@ static inline spinlock_t *pmd_lock(struct mm_struct *mm, pmd_t *pmd) return ptl; } -static inline bool pagetable_pmd_ctor(struct ptdesc *ptdesc) +static inline bool pagetable_pmd_ctor(struct mm_struct *mm, + struct ptdesc *ptdesc) { if (!pmd_ptlock_init(ptdesc)) return false; From patchwork Tue Apr 8 09:52:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14042565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4325DC3600C for ; Tue, 8 Apr 2025 09:53:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5E0E7280005; Tue, 8 Apr 2025 05:53:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5675A280001; Tue, 8 Apr 2025 05:53:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E0A1280005; Tue, 8 Apr 2025 05:53:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1FE65280001 for ; Tue, 8 Apr 2025 05:53:22 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 3D4D0C09EE for ; Tue, 8 Apr 2025 09:53:22 +0000 (UTC) X-FDA: 83310413844.18.21D10E4 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf09.hostedemail.com (Postfix) with ESMTP id 9C03114000C for ; Tue, 8 Apr 2025 09:53:20 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf09.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744106000; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3dDgWlNxtleZVCX3MWpmcRMjX6SMpusMABcu8Cw32FM=; b=3a4GXXNUJeO7Dkb2aYopqgkoaV8rb1VBISfYM3q+Z4JVcKACF2LmXMqew48iHG2S/rQRfc TGW0oJ/pPZuwE12gR7qYj1Z0K3/IRfXIKAwpo8M9QdSUS2Gq2PyukSwEKZzIYjl7GTi0VX u6mGgIqW3wJhMCSefZ54phBGqmqTdfw= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf09.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744106000; a=rsa-sha256; cv=none; b=qYW+ECe0bF8yQY1//Gz3oiJJDQ7RDprFiyijLYmln8U3JJodxsKcuvcxOkjlmyRPl3O/Tb OJwEAAh/43D+Xos0eUFFScEfSsAF8iJZkNRzMj/Vw58BdO9TTF/gibXh0BV46JnrcC0x2I 5gntZYeDzqAIXXCIo7gofM51p2sCgi8= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AE8B616A3; Tue, 8 Apr 2025 02:53:20 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6F7EA3F6A8; Tue, 8 Apr 2025 02:53:15 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v2 02/12] x86: pgtable: Always use pte_free_kernel() Date: Tue, 8 Apr 2025 10:52:12 +0100 Message-ID: <20250408095222.860601-3-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250408095222.860601-1-kevin.brodsky@arm.com> References: <20250408095222.860601-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Stat-Signature: tspb4y7qcijeixbu19xa7scftwfaxrsn X-Rspam-User: X-Rspamd-Queue-Id: 9C03114000C X-HE-Tag: 1744106000-594716 X-HE-Meta: U2FsdGVkX1+8MHt/UTZu/lIHdJdrENfMjmaW9VBP8Vw4VKNQ965JdDlCCCxwTqXWzUXXBuRoOLOABkPpnPxdls7vsPkgwoLiXor2DIG3R2vRNg6DpJ1i0+IHPhCPytZpaMbkX6idbibhJaYTokYRsNJEA9QpoZndiyBkocsSf8rzNR0Ha7zekkJesrti9LBlnggHGmc8m876q1yHj1mxsWu6KCMm2KYc65b/ULq0sD3sgJNS0m80/KxYTk9Y4MInlinZHzLpl90o6ze8XXQkroIauQzLa0qSt8KXAYuLN9TxHv4G28Y643UkBbeSkjYAIlOje3w9hSdt7un+cER6cHlfIAUdbpbe0qihTqOV3B2xUyeURuOBYbDd67DnLSMbBkM2RlntxCCfsPhsmkJaCxGae/8KIipRPEK+DUh63qZBKE9k6Nmg/aL1hbvXzptW5Qzuo3tzM3V8+zs7imlr+nIFnjs+F05jgAsdWqd0JGQfZakSwHjTEYovONMTZW1Dt/TtmHZP57hyF3+ysOF4+iB0L6OJPTCQb9JtvpZFkSQ7zIP+65DzRCrX/WfdyZL3FqBywIi+517/cMj8oRbJ2LT+X30+3YPNfKdYCJBLkfGPcqaL+rc+bJOU+qT/UykwF8DyqH9DhkC5xRHOOfuLr2n/hb85qKyOM667+avL61DHS1+tJPXoVtRFr5OrZgm+Pky366Av5fjpXB/iy/TWuFiPsQ/mM4cRPBpdNpJi5kDwe+Ucg89spEh7TB5QsWiDkEuFYjk6zIOOqxGDhex4F4xCkxr5ahFE7xHLRWrg6qXtRndbdVHHVcto7COIHG+cbhZT8HI46UbPCSpB8nU+vmD+sa6nuHNASP4RUC9V3anozzqVKtynt++ZVbPl0Siiy5rYRi06Y05Y9ywycvnCn1QoAk0T6AP4uIRVqc/8HYOAZYHTSB2XqVqAWiPPXzUAKZXRwnRdT4XPGuVnDzh vN16ajlQ aCYxhsx26Ty2WovkpGlMyELvBZDRXIKNH0CDrcL6NN9DmrCdBNJxul1C/pMWqqx3JziHAy5TGlja5eFbfo4jlO9BshVFs6xpbkPywiJLvrNdSiAA+WDCSwfIKaBITmhdkIeJ96al9PucqviNkzGWS5pKFDdUYVp9DraDifCGC+psYo9sfh3/9JYZHZU2goVVB4bbfQMIORTurj1s= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Page table pages are normally freed using the appropriate helper for the given page table level. On x86, pud_free_pmd_page() and pmd_free_pte_page() are an exception to the rule: they call free_page() directly. Constructor/destructor calls are about to be introduced for kernel PTEs. To avoid missing dtor calls in those helpers, free the PTE pages using pte_free_kernel() instead of free_page(). While at it also use pmd_free() instead of calling pagetable_dtor() explicitly at the PMD level. Signed-off-by: Kevin Brodsky Acked-by: Dave Hansen --- arch/x86/mm/pgtable.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index 7930f234c5f6..1dee9bdbeea5 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -809,14 +809,13 @@ int pud_free_pmd_page(pud_t *pud, unsigned long addr) for (i = 0; i < PTRS_PER_PMD; i++) { if (!pmd_none(pmd_sv[i])) { pte = (pte_t *)pmd_page_vaddr(pmd_sv[i]); - free_page((unsigned long)pte); + pte_free_kernel(&init_mm, pte); } } free_page((unsigned long)pmd_sv); - pagetable_dtor(virt_to_ptdesc(pmd)); - free_page((unsigned long)pmd); + pmd_free(&init_mm, pmd); return 1; } @@ -839,7 +838,7 @@ int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) /* INVLPG to clear all paging-structure caches */ flush_tlb_kernel_range(addr, addr + PAGE_SIZE-1); - free_page((unsigned long)pte); + pte_free_kernel(&init_mm, pte); return 1; } From patchwork Tue Apr 8 09:52:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14042566 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF3A1C369A1 for ; Tue, 8 Apr 2025 09:53:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E50D7280006; Tue, 8 Apr 2025 05:53:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DFDEE280001; Tue, 8 Apr 2025 05:53:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C78A5280006; Tue, 8 Apr 2025 05:53:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id AAAAA280001 for ; Tue, 8 Apr 2025 05:53:26 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id C9C07140991 for ; Tue, 8 Apr 2025 09:53:26 +0000 (UTC) X-FDA: 83310414012.26.47CFE24 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf17.hostedemail.com (Postfix) with ESMTP id 35AC740004 for ; Tue, 8 Apr 2025 09:53:25 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744106005; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EU3IKg5gNWi8e73Hj4jkz8q3EV56066wj6qzlOZeFZQ=; b=bSwla4hqP7TWG1gGJSJoYASeBMT5Xgx023hTnbp47Xs9cR/YthnETwEYi5RSlsFh3QS6ye NNEZK4niUdEysGRBXmsEOSdw8KwdBRW2CP+BvZf0MFzuthSveITFrWzkgVkpPCEsuhXq6F WLrZ3mbkIdNilrnhxR6DUvzol4GYHHI= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744106005; a=rsa-sha256; cv=none; b=2Cj4kI41ofxyy0pDqNS8zirtf5ur/FXobYk6WUTEZZDZ6gN0hItz+2+kkjzlZFCsN1BV2F lsbVFtXMW4GAHppP3f6SED4mT6V6OfzBnAdFxoHSumLM1Q34K92PYxnryPgoFkpyyuiYae F2gI4gBPiHvoROXV367dD/SD1e9iKkI= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 66F6A22FA; Tue, 8 Apr 2025 02:53:25 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 27BC83F6A8; Tue, 8 Apr 2025 02:53:20 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v2 03/12] mm: Call ctor/dtor for kernel PTEs Date: Tue, 8 Apr 2025 10:52:13 +0100 Message-ID: <20250408095222.860601-4-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250408095222.860601-1-kevin.brodsky@arm.com> References: <20250408095222.860601-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 35AC740004 X-Rspamd-Server: rspam05 X-Rspam-User: X-Stat-Signature: ban94fgdmx3i9i74f1fczyfbcbw9i7de X-HE-Tag: 1744106005-260331 X-HE-Meta: U2FsdGVkX19SJRGgeE9qQXr90Syv+ulUgFlZ+I10TIGRmabVs2Kb2mLRaVg9y0IZ+6xUIrY1acwbApieKiznr3SlWtDVYwCwUO/Y2KBfZZSvJxUADdTa/fu6y/SjlNx/nAaIrRQjIB9uX+Y6qDCQr1ZR+cCAbwgcmRwXXYMr/nfeHZ/B4d7A8SCsKQLBwRHJaTFRTdzn4/rDLXt6tEdc8q4MSgihU5yNvG/0w32JRrJk2ejMIxyBD7VCzMgppTmxG7Q+Z6utGOgkzmM5eLOo1lV5khUsMuizXISI1UrfkXl1yylUyIWJQhyqNh9pCezuKpNAhQvLOh2SFyuzG7vTsTHR+09oixBw1gGro5FO110UrEPj5xfP3hw4VjAEvh2xvYMutlNePsksFTAY/TKTZ7TUMDCCZF1u+r6aX9mF2xT+0A8hD+rf6E6E1yau5b3v9u1GkiAUnWWUU52FaXnmpdSexTd+SQjhv3+z6hodT6e0uNhNb5ikWqlasz83EoqL9j5PfjA7BTJDiGowDrorXUJMOFPK6Krc2/BYsJlrJV7mYuvJo0FfUkOHxERlhMW5Vpryce4SAwWEsFzZhKBjvlAvLMR+ratRWu9eXhtptzOatBzJ7BrMUCQb6ZLo0wIJeVyeKq8R1nfTm6vTPGypqbGjllZA3Z0qDcJF/ZMqqN9wRf1nSL46YO/usggxte5CMaVQxfGzv/h5lCYTWciYP36TOxRIFzdI9r17Uzmsj67tBOgxDhUcKT1qDI3qhu4QZQfUyRgQICtJy/l+v8tEirSa17fdnqbHmm66azMCVOyJzU/iDiIRH0HeRKfC3O16Xak9+2BD61DccpX8DKhHB0Mma0O6+ZmhPA7OTaRrpr8mmjXAHq3cNZi+NsuV689C9CTLVIUEX8LlrIoouFUS3Tk1iWPq/bjn7N8FnvKDt98BDTi92Ut8D2rOsvy9Mtirq/B35LoRzPhpeh9OwMT SVVwdrBw GAG/Hh/gpiHFscvSkAV9BdSBW3HYY38TH2RQzSNfhSA8X2cKlB4VHDOwLWSUtT535uxiQUFz9ec19+L2QEIQHmjAIJK1Pu7vGPxU9QYKLeUaEKqvaTDRHaiDxAn+Jxg+8Vv808cPCFU/JZ4IMz2DKBTFgOECj+oITuZQvikw1H1uo1Lee2jQc/8aR0A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since [1], constructors/destructors are expected to be called for all page table pages, at all levels and for both user and kernel pgtables. There is however one glaring exception: kernel PTEs are managed via separate helpers (pte_alloc_kernel/pte_free_kernel), which do not call the [cd]tor, at least not in the generic implementation. The most obvious reason for this anomaly is that init_mm is special-cased not to use split page table locks. As a result calling ptlock_init() for PTEs associated with init_mm would be wasteful, potentially resulting in dynamic memory allocation. However, pgtable [cd]tors perform other actions - currently related to accounting/statistics, and potentially more functionally significant in the future. Now that pagetable_pte_ctor() is passed the associated mm, we can make it skip the call to ptlock_init() for init_mm; this allows us to call the ctor from pte_alloc_one_kernel() too. This is matched by a call to the pgtable destructor in pte_free_kernel(); no special-casing is needed on that path, as ptlock_free() is already called unconditionally. (ptlock_free() is a no-op unless a ptlock was allocated for the given PTP.) This patch ensures that all architectures that rely on call the [cd]tor for kernel PTEs. pte_free_kernel() cannot be overridden so changing the generic implementation is sufficient. pte_alloc_one_kernel() can be overridden using __HAVE_ARCH_PTE_ALLOC_ONE_KERNEL, and a few architectures implement it by calling the page allocator directly. We amend those so that they call the generic __pte_alloc_one_kernel() instead, if possible, ensuring that the ctor is called. A few architectures do not use ; those will be taken care of separately. [1] https://lore.kernel.org/linux-mm/20250103184415.2744423-1-kevin.brodsky@arm.com/ Signed-off-by: Kevin Brodsky --- arch/csky/include/asm/pgalloc.h | 2 +- arch/microblaze/mm/pgtable.c | 2 +- arch/openrisc/mm/ioremap.c | 2 +- include/asm-generic/pgalloc.h | 7 ++++++- include/linux/mm.h | 2 +- 5 files changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/csky/include/asm/pgalloc.h b/arch/csky/include/asm/pgalloc.h index 11055c574968..9ed2b15ffd94 100644 --- a/arch/csky/include/asm/pgalloc.h +++ b/arch/csky/include/asm/pgalloc.h @@ -29,7 +29,7 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) pte_t *pte; unsigned long i; - pte = (pte_t *) __get_free_page(GFP_KERNEL); + pte = __pte_alloc_one_kernel(mm); if (!pte) return NULL; diff --git a/arch/microblaze/mm/pgtable.c b/arch/microblaze/mm/pgtable.c index 9f73265aad4e..e96dd1b7aba4 100644 --- a/arch/microblaze/mm/pgtable.c +++ b/arch/microblaze/mm/pgtable.c @@ -245,7 +245,7 @@ unsigned long iopa(unsigned long addr) __ref pte_t *pte_alloc_one_kernel(struct mm_struct *mm) { if (mem_init_done) - return (pte_t *)__get_free_page(GFP_KERNEL | __GFP_ZERO); + return __pte_alloc_one_kernel(mm); else return memblock_alloc_try_nid(PAGE_SIZE, PAGE_SIZE, MEMBLOCK_LOW_LIMIT, diff --git a/arch/openrisc/mm/ioremap.c b/arch/openrisc/mm/ioremap.c index 8e63e86251ca..3b352f97fecb 100644 --- a/arch/openrisc/mm/ioremap.c +++ b/arch/openrisc/mm/ioremap.c @@ -36,7 +36,7 @@ pte_t __ref *pte_alloc_one_kernel(struct mm_struct *mm) pte_t *pte; if (likely(mem_init_done)) { - pte = (pte_t *)get_zeroed_page(GFP_KERNEL); + pte = __pte_alloc_one_kernel(mm); } else { pte = memblock_alloc_or_panic(PAGE_SIZE, PAGE_SIZE); } diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h index e164ca66f0f6..3c8ec3bfea44 100644 --- a/include/asm-generic/pgalloc.h +++ b/include/asm-generic/pgalloc.h @@ -23,6 +23,11 @@ static inline pte_t *__pte_alloc_one_kernel_noprof(struct mm_struct *mm) if (!ptdesc) return NULL; + if (!pagetable_pte_ctor(mm, ptdesc)) { + pagetable_free(ptdesc); + return NULL; + } + return ptdesc_address(ptdesc); } #define __pte_alloc_one_kernel(...) alloc_hooks(__pte_alloc_one_kernel_noprof(__VA_ARGS__)) @@ -48,7 +53,7 @@ static inline pte_t *pte_alloc_one_kernel_noprof(struct mm_struct *mm) */ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) { - pagetable_free(virt_to_ptdesc(pte)); + pagetable_dtor_free(virt_to_ptdesc(pte)); } /** diff --git a/include/linux/mm.h b/include/linux/mm.h index f9b793cce2c1..3f48e449574a 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3103,7 +3103,7 @@ static inline void pagetable_dtor_free(struct ptdesc *ptdesc) static inline bool pagetable_pte_ctor(struct mm_struct *mm, struct ptdesc *ptdesc) { - if (!ptlock_init(ptdesc)) + if (mm != &init_mm && !ptlock_init(ptdesc)) return false; __pagetable_ctor(ptdesc); return true; From patchwork Tue Apr 8 09:52:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14042567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77067C369A1 for ; Tue, 8 Apr 2025 09:53:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 81822280007; Tue, 8 Apr 2025 05:53:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7C913280001; Tue, 8 Apr 2025 05:53:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6BA32280007; Tue, 8 Apr 2025 05:53:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 52B08280001 for ; Tue, 8 Apr 2025 05:53:31 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 722B1C0A1A for ; Tue, 8 Apr 2025 09:53:31 +0000 (UTC) X-FDA: 83310414222.25.8A1B6FE Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf14.hostedemail.com (Postfix) with ESMTP id DC21A100012 for ; Tue, 8 Apr 2025 09:53:29 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf14.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744106009; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tEBbZMuK7OXxaJSao4EKWfZ44UXf7uN/ZmWU7nnmgv8=; b=GyWprvhXXrGSu6/MDfe12+H9GyNSsw1f5GkOJAm80Yi3ePPFNhC72uMwEuAHn3INYojdVQ fq45whJcsTssaMRDqDVw2wJcz1R9oBWtdkS9hE8yHQnULj6CoyM1BuGxwMo8HSvbRne1TA GtkKRQchbYvQCcS6aBPFDzkdQVTwAwk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744106010; a=rsa-sha256; cv=none; b=cyvu1uETbGSjpzUxB4lNwpqqqDZ3ehMIpfK6tXGtyMpNtqcvXRmktAVAMoLBL4KT9GYP/c YjvSOxiOSj7bqZWowhH7HsmjAP5EU+olAFVMXovd/YpuBtqWacgRl0Zb6x+QIx/YnB5Gwa FPL4OzAFb0fHAlHHOCel4+CArBiJGjc= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf14.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 211432308; Tue, 8 Apr 2025 02:53:30 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D3DDB3F6A8; Tue, 8 Apr 2025 02:53:24 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v2 04/12] m68k: mm: Call ctor/dtor for kernel PTEs Date: Tue, 8 Apr 2025 10:52:14 +0100 Message-ID: <20250408095222.860601-5-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250408095222.860601-1-kevin.brodsky@arm.com> References: <20250408095222.860601-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: DC21A100012 X-Stat-Signature: mzxsxa3fkxasasn185cktj9mis3odr1i X-Rspam-User: X-HE-Tag: 1744106009-669890 X-HE-Meta: U2FsdGVkX18gHzUkHJ6iDtNndhet8/I+uqkCmF7fIb6bdwnjul4zHTf3cvlMKxu0ZxDJ8dpSbFR0lqA9v312KKghnuRiGnE/nPduM/Xxkzcl+V82Tb/erMz8ZGT1bbqKcGrNr/b5MRliYxAXoCDf+Ti787XA08C+hK6XOsHlZ2Bz8dRu0XYzLW3ua6XJICNTXEadq3wQtcOAEw/oBx7rL3gWurfEqYHo1Orq6UBiTYjCZ2CHW6qU+yhtg5LEVk75HgMU76Ip4AS1TFg9b/dQLPR46G/b+llbvISpxAUsRaHCNbg/DfAWK/jFIa1RHxyywciRZ/+K4BOnhrpUgaQTQKNZcjLsforwaN+gbxla1I+wLTc5vyeN9Yul/hDZSZWIC/Z2hFxDQkyCwgtAjaMEIa78RFd+8zUCYwLtamGZoujrZ8wqRf8PVswuo8SeNt4vsEGwpyBTWq8W+wsQ4e9yI4A9WzZuMvn2hUb2xuWR6mgV0NBcBPm/Yqy58XVke2Q02CGp/57NnEA3Ma5PVBKYIf9ElOKve/Qb+LeoCcYvJMx6Lgpw/ou6RKLyCnQznP0j8md7h+wsvSqAJV/1FNMf5BrwqmED80GT167uFpWMBVj7HkImqaJQhtfaHur+Uc2Iqz7+CH6Pcg0uO8tYsipze2uAP66kabB+Nziha9p4epPDDLKZeGuS523PWzQg8XDSYOdpaNAkkxN5gRtsc/e7+6pmXtagupMDSoJW5jcxYUVpz7WPIj0KkWb3UXnDDWeXLyeDclv9wFo/BAeyZQdEj1kc63x0Z09KnLx1JKUTRELyA/ehoUtwzfNoaQVwk0NoZKBx6/EkZDuYkBF6R5lN9j0n577XreRuVlJ7d1jCVqlzZSXS4AJGhLsBQd8qXFjRVrUtDmGKDhQYDOht/RxDzRTdJiDtnrbZewOCR3cC3nRI2/8fJtOPcMLE2DQNQUj6RN2PjZdBnhKuOisJL0l H8XmdXMO 1/8G0O8H7iJW2feBQjoAHuhZJBR4n5R9on4PFnNFbco7/LUaaZTD3msTH/HJRRWMziG9jCL1ns4GfmbfaN6BGcvzMeulqPVO517I3LxLkVhL7WRJClqa8HjG7/lvqMkEe01+Au97Kcu+k4ZiqzdcZqwUtAw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The generic implementation of pte_{alloc_one,free}_kernel now calls the [cd]tor. Align the m68k/ColdFire implementation of those functions by calling the [cd]tor explicitly. Signed-off-by: Kevin Brodsky --- arch/m68k/include/asm/mcf_pgalloc.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/m68k/include/asm/mcf_pgalloc.h b/arch/m68k/include/asm/mcf_pgalloc.h index 465a71101b7d..fc5454d37da3 100644 --- a/arch/m68k/include/asm/mcf_pgalloc.h +++ b/arch/m68k/include/asm/mcf_pgalloc.h @@ -7,7 +7,7 @@ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) { - pagetable_free(virt_to_ptdesc(pte)); + pagetable_dtor_free(virt_to_ptdesc(pte)); } extern const char bad_pmd_string[]; @@ -19,6 +19,10 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) if (!ptdesc) return NULL; + if (!pagetable_pte_ctor(mm, ptdesc)) { + pagetable_free(ptdesc); + return NULL; + } return ptdesc_address(ptdesc); } From patchwork Tue Apr 8 09:52:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14042568 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CCF1C369A1 for ; Tue, 8 Apr 2025 09:53:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 565D7280008; Tue, 8 Apr 2025 05:53:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 515C8280001; Tue, 8 Apr 2025 05:53:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3DD68280008; Tue, 8 Apr 2025 05:53:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 1FF2A280001 for ; Tue, 8 Apr 2025 05:53:36 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 4420A160953 for ; Tue, 8 Apr 2025 09:53:36 +0000 (UTC) X-FDA: 83310414432.06.EC7063F Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf29.hostedemail.com (Postfix) with ESMTP id 95FF612000B for ; Tue, 8 Apr 2025 09:53:34 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf29.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744106014; a=rsa-sha256; cv=none; b=QypjVpp3DvwgraylonoSZcrt0Bez/cporGco8j8KP6oYp2fX5iCnutZbbUtnG8K/I7xtrt qGTylO+xk6xFEZMX9uvZFfK15Fpc//l3bwJKZIcHl/8eFaH9u9EQElgpyqQQdnO8FhzSev YqdjBFA4ArlIsC+Zz2/w0t8nRJW92V8= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf29.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744106014; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/BkpJ0cTKq4ChrXalGvhAQpS6e0Bp2XA+877e60Q9Ds=; b=M9SR2aqaVOrRm74lvLlr1e6UM+MnPC7HI/1sr3s7I3iIivjVROCniyfPP7tXUZL6xIwR++ DsBT5JO+EP+GoOeyqGe0a+6VwC1MVaKcontYJLHbLtg5kTm1fq6UetijLGITCLXtE951KB LIVBdeb78aHIO6+0bUPAhax+J+y9SrU= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CCE4822FC; Tue, 8 Apr 2025 02:53:34 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8D5873F6A8; Tue, 8 Apr 2025 02:53:29 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v2 05/12] powerpc: mm: Call ctor/dtor for kernel PTEs Date: Tue, 8 Apr 2025 10:52:15 +0100 Message-ID: <20250408095222.860601-6-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250408095222.860601-1-kevin.brodsky@arm.com> References: <20250408095222.860601-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 95FF612000B X-Stat-Signature: accfmqq4ytru31nyu1q5nx4e8e5kqz3c X-HE-Tag: 1744106014-981398 X-HE-Meta: U2FsdGVkX19pv6jQBzSqrCql2lfAJys9Bk+RR0iM0E3VHSYzj7otzM97YD94+4oRCcaNBoIZXpEvYVSPN2ALkU9c0D3PjxV8t7NRvIxLcXi0LugKEsIcl0ZRAY/tEVzJFF9Pqygtv0THtymqE1pVUVTNndpFNQ1N+HAY1CcLDfPODLjcCe0EjBY+Tsf0cxnostzjMCg3xkf95xiE8cPbK2dEh01Fom6kIuAQv8jTUfB3wJZloF65lbOfI48V5TEfaTu07ZfSBh2/Uo67C/st6Zh3arPD/SzJG34bdolqhd/ywYT9enLo5k4ogHGVm7pEoU/p12OoVBqKgkgCoz9lk39/cJm0OP71nWJLvyOUlY7ztqpUu9xWQWeQjcN9S7HK1Gq614fLWdLVZWz60Kgbxddtgx0+qdVPhRAuUjN51y85uiNy6MVAHZTSPqZ27itU8SRpSDuQdrmpgpz38PLlh0MO2BgznUAVOcXDQq0Ooma9226j2jBCAETkcmIm0l1+0MP6e1c3++U6C9ZU8TKjtvEnsTaM156ba9VwxL7+zlzij4MOPWomA8WVsuk41Ve05F+8mC5QMgAMdxnS/BzwTa44O91JqB83dDrNt96t08TTqR4bCO80zdBJCSCeVAJOIn9O+gK7kyeYu/QMulcVpfL91ixAKqLfufQpVVuC7hnNnQ++WEv7nujXRr93WDa4hYdNTmkI8llfEx8fmUg5/zHtOh8iqMWngZXImAPXaG0Bwv01FGorqUw+qj3t0G6xUDGy/zN0R4myaLmXLJAdbgl8uZkGUs2DBelq7sXxGyRBRxry/iLtB9mug5pDrWwXuAMZkVmc2LuO59tPVGcHoxhMd4F5cuLwnjwnIYVNrxD4gj14FtqQkb+k+hZz8VkW13LJxo6lUr9hLonF5BLWqYlWykNdxMBGGHLFNYXItLRVqqMR1SHCJfZPN9X89OqP1solIXZy1iVckX6D3kF jkLxeYpS 54m9UD3eH0OtNCVUSnPt3b0mMW2oAYj9seFFoF8F7orfBVLyqqrC3VHKml0glCZv1GO2bhy2O/4ONp//2h4cXJ3NB8PyNseuekHFJnkQtct5eFXrk6Bs3tCxl27kovAU7QaJpjmmR8UIrIXjr1A9b+v9YTw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The generic implementation of pte_{alloc_one,free}_kernel now calls the [cd]tor, without initialising the ptlock needlessly as pagetable_pte_ctor() skips it for init_mm. On powerpc, all functions related to PTE allocation are implemented by common helpers, which are passed a boolean to differentiate user from kernel pgtables. This patch aligns the powerpc implementation with the generic one by calling pagetable_pte_[cd]tor() unconditionally in those helpers. Signed-off-by: Kevin Brodsky --- arch/powerpc/mm/pgtable-frag.c | 30 +++++++++++++----------------- 1 file changed, 13 insertions(+), 17 deletions(-) diff --git a/arch/powerpc/mm/pgtable-frag.c b/arch/powerpc/mm/pgtable-frag.c index 387e9b1fe12c..77e55eac16e4 100644 --- a/arch/powerpc/mm/pgtable-frag.c +++ b/arch/powerpc/mm/pgtable-frag.c @@ -56,19 +56,17 @@ static pte_t *__alloc_for_ptecache(struct mm_struct *mm, int kernel) { void *ret = NULL; struct ptdesc *ptdesc; + gfp_t gfp = PGALLOC_GFP; - if (!kernel) { - ptdesc = pagetable_alloc(PGALLOC_GFP | __GFP_ACCOUNT, 0); - if (!ptdesc) - return NULL; - if (!pagetable_pte_ctor(mm, ptdesc)) { - pagetable_free(ptdesc); - return NULL; - } - } else { - ptdesc = pagetable_alloc(PGALLOC_GFP, 0); - if (!ptdesc) - return NULL; + if (!kernel) + gfp |= __GFP_ACCOUNT; + + ptdesc = pagetable_alloc(gfp, 0); + if (!ptdesc) + return NULL; + if (!pagetable_pte_ctor(mm, ptdesc)) { + pagetable_free(ptdesc); + return NULL; } atomic_set(&ptdesc->pt_frag_refcount, 1); @@ -124,12 +122,10 @@ void pte_fragment_free(unsigned long *table, int kernel) BUG_ON(atomic_read(&ptdesc->pt_frag_refcount) <= 0); if (atomic_dec_and_test(&ptdesc->pt_frag_refcount)) { - if (kernel) - pagetable_free(ptdesc); - else if (folio_test_clear_active(ptdesc_folio(ptdesc))) - call_rcu(&ptdesc->pt_rcu_head, pte_free_now); - else + if (kernel || !folio_test_clear_active(ptdesc_folio(ptdesc))) pte_free_now(&ptdesc->pt_rcu_head); + else + call_rcu(&ptdesc->pt_rcu_head, pte_free_now); } } From patchwork Tue Apr 8 09:52:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14042569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2ACE6C369A1 for ; Tue, 8 Apr 2025 09:53:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 26452280003; Tue, 8 Apr 2025 05:53:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23AB7280001; Tue, 8 Apr 2025 05:53:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 12978280003; Tue, 8 Apr 2025 05:53:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E9260280001 for ; Tue, 8 Apr 2025 05:53:40 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id EB98C1A0A09 for ; Tue, 8 Apr 2025 09:53:40 +0000 (UTC) X-FDA: 83310414600.21.A5BE6FD Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf10.hostedemail.com (Postfix) with ESMTP id 521F5C0003 for ; Tue, 8 Apr 2025 09:53:39 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf10.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744106019; a=rsa-sha256; cv=none; b=kbEG8y8LBQ1fMVLCeNfGq7loXPwEjOkbqXmwq+lkO4+disNXWSPA2OupoHge3UM4oYJw2E Lqn7VS5O/6+n/1pmvihEc3fBE1u7OuS26gPgM198zCNog4qEwYZfAyvxZ5ZacIU1S+0Y+g G1gjIjrRBsRLGy8+zIYvEbM8DbAst+4= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf10.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744106019; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mhR2Iq+beAhq3GZKKAxztcVBqkxIucxufOP5/zFYWuY=; b=FdGoAK2vCiCs9U5X30gp8u3lzxVFwCanNW1QBq4WQyOB4XaCs1Hj96tkS52cJBIQXZPvxR W1P1F4Du9Lik0V8QRTPYvqGSX2ux78F2oaXqzqRh3Gwru+MK1GmwRQ7BzUL9dQj8rCZZ98 PlTV7seagIZtF7J3AgA4Ddu2loZ+fUw= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 861E82309; Tue, 8 Apr 2025 02:53:39 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4566F3F6A8; Tue, 8 Apr 2025 02:53:34 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v2 06/12] sparc64: mm: Call ctor/dtor for kernel PTEs Date: Tue, 8 Apr 2025 10:52:16 +0100 Message-ID: <20250408095222.860601-7-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250408095222.860601-1-kevin.brodsky@arm.com> References: <20250408095222.860601-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 521F5C0003 X-Stat-Signature: fxecdotyh83wa7y751nrxzgq8sgg6yrf X-Rspam-User: X-HE-Tag: 1744106019-985248 X-HE-Meta: U2FsdGVkX18rhu5G97DxXJYXWKDMD6N2cJNCqOwlUVjBwWvJfv7Zx4Uyaa81igmIhyQ05tL+3dkZhlyYmkHSiQv3lirpHQkKF03z12s2imV3WIxECv7A4sc61IDgAEX7wXHPXvizjJ2r1XZ87euxUDTBFwDWyISAY3bopuymCF50LpSMEGa2oCWCXJsaTmmaBp3vFEnlPkmjhmrldkjiKCYBBvcw3KExG2j3nJuKa6lU45R3KXUy3J6wTAO4erCqLkmazms8daNZZvH2GCap7CdFwo2rnv+cDtNHojUuZRcUSITajn6+bfKXa+2ybHL6Divp2iAEDm5kRF5DhoBJTNr88PdNp05DZ1V3dHcHMDU+Ik/3Vl13e304eUkh+FBiDqcI7PJo4I+BNOv6eUaj1kZO1rlVqrac1EAeW4PysmHkTfWq7nPaJdPbxQIZAl8YCjciCOxgfFhp0UMPysmbkXfzRUk4pQIjy4e0dQrzQwWb1xC+VkzDAx9ahhoRCBpr4hh+waM2nd3qtGeEhxk9TDAvCFfe/awNYaIAODORUHaitUQnuvnwapGPnp2E2hSiwZPdqTcb45Z65mKSEev4RuCA7iQ/FMuZPZDQkRfMCxlwRn+reVo5QTwfqyFTPtRqixCLdWIItONXaQfoaT6JjUE1LfYDvVOvyCrrEZDarVG8Kv8UBaO9NPwKxqlvIpXVByeQEXFmdX91i9skaHF+pEneeuQCBhY343jLEm2X38tOqR7Rbx1RtUWOKmiZqe1pE5t+s4GECIWs2bzi8cOfVa2fyrZtyJkKBpd4Rj84j4PcJvv5hUgI6uKdPzjhYDRUtACIxglBA09PnNZNi1EcaDKfbKzrifjbY3L6FktWzyAzXd5YhtekunvhQ6BlLexNvP6WZaIdSnc+lkm309jWI8tkEGxws+7dNRU5hWdt5KnQCjwE8gxIq0B7qjOIyc+wA4nw+agAL7tqgnA5yvm M9W1DoXq yVlgod52HyGs0T1/a9lqYKvyjUMt6eJ3MUfd5+6t+5Aa4dEo1kOMD7brUizCLS3pcnNdRrSQNRNhFnxbLeSLT/nQIEs4Tirqbi6+I1uR8admAoOi0byg4PNZZpFqyvcX7CMhtABnq7/t4aHC43Yt943E+0w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The generic implementation of pte_{alloc_one,free}_kernel now calls the [cd]tor, without initialising the ptlock needlessly as pagetable_pte_ctor() skips it for init_mm. Align sparc64 with the generic implementation by ensuring pagetable_pte_[cd]tor() are called for kernel PTEs. As a result the kernel and user alloc/free functions have the same implementation, and since pgtable_t is defined as pte_t *, we can have both call a common helper. Signed-off-by: Kevin Brodsky --- arch/sparc/mm/init_64.c | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c index 5c8eabda1d17..25ae4c897aae 100644 --- a/arch/sparc/mm/init_64.c +++ b/arch/sparc/mm/init_64.c @@ -2878,18 +2878,7 @@ void __flush_tlb_all(void) : : "r" (pstate)); } -pte_t *pte_alloc_one_kernel(struct mm_struct *mm) -{ - struct page *page = alloc_page(GFP_KERNEL | __GFP_ZERO); - pte_t *pte = NULL; - - if (page) - pte = (pte_t *) page_address(page); - - return pte; -} - -pgtable_t pte_alloc_one(struct mm_struct *mm) +static pte_t *__pte_alloc_one(struct mm_struct *mm) { struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL | __GFP_ZERO, 0); @@ -2902,9 +2891,14 @@ pgtable_t pte_alloc_one(struct mm_struct *mm) return ptdesc_address(ptdesc); } -void pte_free_kernel(struct mm_struct *mm, pte_t *pte) +pte_t *pte_alloc_one_kernel(struct mm_struct *mm) { - free_page((unsigned long)pte); + return __pte_alloc_one(mm); +} + +pgtable_t pte_alloc_one(struct mm_struct *mm) +{ + return __pte_alloc_one(mm); } static void __pte_free(pgtable_t pte) @@ -2915,6 +2909,11 @@ static void __pte_free(pgtable_t pte) pagetable_free(ptdesc); } +void pte_free_kernel(struct mm_struct *mm, pte_t *pte) +{ + __pte_free(pte); +} + void pte_free(struct mm_struct *mm, pgtable_t pte) { __pte_free(pte); From patchwork Tue Apr 8 09:52:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14042589 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CAA5C369A1 for ; Tue, 8 Apr 2025 09:53:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A38F1280009; Tue, 8 Apr 2025 05:53:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A0E00280001; Tue, 8 Apr 2025 05:53:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D76F280009; Tue, 8 Apr 2025 05:53:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 718B6280001 for ; Tue, 8 Apr 2025 05:53:45 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A8676BA168 for ; Tue, 8 Apr 2025 09:53:45 +0000 (UTC) X-FDA: 83310414810.16.6056DFB Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf21.hostedemail.com (Postfix) with ESMTP id 164441C000C for ; Tue, 8 Apr 2025 09:53:43 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf21.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744106024; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AoW5uPcncPp3axMC3OLXSZhaQ2vhQ+ldUmkYdrG/FME=; b=woF241aK9RyUBqlmRq5XnASnngU4K98O7+EHu8RDAGcmQgMvuqHUi5zuhdNUXreXLpKV61 1Z7HLjtCHRcfzxATU2oREc0q8LT83RgG5HndF+fhldkm72/KLCa4CJ6ehGvTduzfOqFBJL hHhhifB1jD6eziBKHWIhDmQ1KdblDig= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf21.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744106024; a=rsa-sha256; cv=none; b=JZpGPw//+EQuskHJ94WJe02RPrTuV1/Yvg4CXlkhjII2s62uTU/e8tmWCtUXKKy+Kc8BlB x+lTnuaNxg1OyEfmvCRUs2m1sMR922CdaVYdmoLr/Z3VBVNyDz5hp2xN69y3ucxMDqo7R+ I4hI7ZfoThUuVrD1lsyAfMHG8QtLwXA= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3EB2B2328; Tue, 8 Apr 2025 02:53:44 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F29D83F6A8; Tue, 8 Apr 2025 02:53:38 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v2 07/12] mm: Skip ptlock_init() for kernel PMDs Date: Tue, 8 Apr 2025 10:52:17 +0100 Message-ID: <20250408095222.860601-8-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250408095222.860601-1-kevin.brodsky@arm.com> References: <20250408095222.860601-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 164441C000C X-Rspamd-Server: rspam05 X-Rspam-User: X-Stat-Signature: mf5wsatq6z65c4d6y61gm6ijhyjokham X-HE-Tag: 1744106023-713978 X-HE-Meta: U2FsdGVkX199cGuOejEAvNSFtsbKXpC354a6Asaiat4y9BQ5AiLZtwnceYycJbXVggIfWsL84aE0UFcuSbTcFD+9vGLKknML03d2qfm8E6fKUPKmkz5gxZvNs4cfnKdt8IubSAamGlldkfmUfSmfuY0WVQiq+vbLVbTHn6tRxRI+XeIstr9MaXsZlsbQ+wkNdktoSloWlHswKxC6zc1AFebyGi3FgCN1/suRdRa3WpdRU7vw39oMVqikF0rm6BEnvxiE55Pjm/Cl02NGRjZncalWOJE2x1GfC5LciHfvnz3DRMcwheAJL2DBXBaVzX4JDPZQWdTqY0fn0W+D1tGIJ7+cm9syYVmRZceDh84vtn7haMS4NyuJsU40oSAmFP1PHR5hHNbiBIaeMBnvUY+mTXGg8ee5o/BzWC6V81xNIJXYJXrR7HZhcVtYALi/Mk01jLV0iemynLO4yXELTWkcgl0EteCvodigHKTlOWTRDT2KnfDyQIMbkxbMQRb1eKD4fD7kxjk+7pQDa7jukQ1qB0OLK4iT+LQLXTBaZLC798+JXJnQDq/YQhnpBQ0J4qrfjvpb2YGy9abCfIIQlR4p0mMo+slgP2Iy77u0mju9OqOI3Bi41lZhX1uQ4Il9UwTaBHkxLd1YfxgOeDrge9YP20Kjs/i1vGUPoyTS5j9MZDC9opBoPDiaQ7G3C4nGf7klo+wEEvDEUkmg2bsSBOGlWW5M+jND8cxckROhmHggF5I5RMoaR78CRLYpJEzFJz9lY+A2Qw84MGSgNpg6xBRAj3wZ1JcbU4rkI+/egxvfHhd1wfShaeYyVYsvDtN4SuKOuSCjxYyGIzSlfshKyJDVLYoZyM9XgaM0Vbip2tsgKH1LXe7/FndMAv6d1nZpgoRECxNQeqQe+cP0wawxOGpiTigcmN2UZVkgrk/nHju/SF2u2DpqUBk96hoLzz+TRWFp8UIlLz3oiAOU+Ic++NR dFgSnPQw M15MZAhb9Rg2YLxhfbpDTK55HDishW/ZGRXIKKYca6CNpTO2lsc54hKDl7UJlb16ZCymLN5rwHGTLP1EQRF8DZx4xFvlRsQovcoaXX4lY1zlBxSyJvsPECkCAjBkXOQ5tCLUUVqRR+Vrio4/vdcdjD0Fb0DJaPEwfi8szgqquHyaa5wAwj6WhZWzV3MlidritT4URhfHC+gFxO8M= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Split page table locks are not used for pgtables associated to init_mm, at any level. pte_alloc_kernel() does not call ptlock_init() as a result. There is however no separate alloc/free functions for kernel PMDs, and pmd_ptlock_init() is called unconditionally. When ALLOC_SPLIT_PTLOCKS is true (e.g. 32-bit architectures or if CONFIG_PREEMPT_RT is selected), this results in unnecessary dynamic memory allocation every time a kernel PMD is allocated. Now that pagetable_pmd_ctor() is passed the associated mm, we can easily remove this overhead by skipping pmd_ptlock_init() if the pgtable is associated to init_mm. No special-casing is needed on the dtor path, as ptlock_free() is already called unconditionally for all levels. (ptlock_free() is a no-op unless a ptlock was allocated for the given PTP.) Signed-off-by: Kevin Brodsky --- include/linux/mm.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 3f48e449574a..ef420f4dc72c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3210,7 +3210,7 @@ static inline spinlock_t *pmd_lock(struct mm_struct *mm, pmd_t *pmd) static inline bool pagetable_pmd_ctor(struct mm_struct *mm, struct ptdesc *ptdesc) { - if (!pmd_ptlock_init(ptdesc)) + if (mm != &init_mm && !pmd_ptlock_init(ptdesc)) return false; ptdesc_pmd_pts_init(ptdesc); __pagetable_ctor(ptdesc); From patchwork Tue Apr 8 09:52:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14042590 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 653ABC369A1 for ; Tue, 8 Apr 2025 09:53:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6265428000A; Tue, 8 Apr 2025 05:53:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D599280001; Tue, 8 Apr 2025 05:53:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 49EEF28000A; Tue, 8 Apr 2025 05:53:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2B030280001 for ; Tue, 8 Apr 2025 05:53:50 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 720CEBA15A for ; Tue, 8 Apr 2025 09:53:50 +0000 (UTC) X-FDA: 83310415020.29.FDE7D2B Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf17.hostedemail.com (Postfix) with ESMTP id CD76040004 for ; Tue, 8 Apr 2025 09:53:48 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744106028; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+0pYQ1qH+UpfJYLoe9vNgfctFN5Ww4sCZmrF9jkxtoc=; b=IPvtSOiASTRGoFwQxYjwlz65t+/YYBkRdC9ufqvFqB2fNZsmvqNYPyA3nt8hMYwXdgw0gH 6jj8TInf36OAPgnMZuORH2HypXPpI0PsI/wO8tpAAEDSdCivtvsjxOjAPJnyYzx+fbcDxv cPybBy5Ek4+3eaQFSY3ssI1ZBhLsETA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744106028; a=rsa-sha256; cv=none; b=meS/KRbC7m1X5rEqbcgQ9aQR8pxMDW0qUdMFY0I6urIjZ7xBOCkJQqnt1xjASuNkhfWqdc j/swBhpUbidBnHKBjYAzXngbME8Ws2YnR7+ezhQo4NN6C4wQoniVMoojyknBe0SpUsZJOg 3vBc8g0SETjx7Cm60DbUznFH9Pt/f9Y= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EC0D71688; Tue, 8 Apr 2025 02:53:48 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id ABBF53F6A8; Tue, 8 Apr 2025 02:53:43 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v2 08/12] arm64: mm: Use enum to identify pgtable level instead of *_SHIFT Date: Tue, 8 Apr 2025 10:52:18 +0100 Message-ID: <20250408095222.860601-9-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250408095222.860601-1-kevin.brodsky@arm.com> References: <20250408095222.860601-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: CD76040004 X-Stat-Signature: oecqrwh1ua9jxt45ombac4jdkcqhe6jh X-Rspam-User: X-HE-Tag: 1744106028-689387 X-HE-Meta: U2FsdGVkX182YWMbNk4wWqX9WSs5CPMK1r9DFRtoay62Utg9PETpEfpr0W8DPX75vL40dQzKtZxw1oVif8l2DVNjjr3NJnQncTC0BOnFbvU6TU1yTxERNq96LPYTlsb1nSf5N/SK9QXskO7jl2J8R9XldebHSBEVMt7V93Z0Nww215ZpCyW99x5wgx5aX1bYBpVDdu/stkyAYkyz4BSZTmaElsKLRZrC4mNcSscPYnTI/eRR/CUcN3ssJ7Cz+05/h05XYKRxx6bwo34i2UgswJhRmXrkbh9yOfbxBdaSoRha3oAy0PiX2CxYXJDHXXBHewuvmineie8kzVorX10F4gPlb91HF1ZVPsDlpFav8r6vYHqgxy3GpZHFkdGTuHlx/wReTXi9dVguaAlxcZSQ/TV6NfYU8NFsvkMwT18MzLGb7VBrXCp+3sGvM2xaKCdWl9i1UhWEy0MYptyvtyhi0QWs1HJyb4RqCpZtsbRlbCyo2lB2GpXq8HgT+bLzFCCVknApYZpOTovQ7r1KzYlSqgnQIv6ejl7149gHDd5cbH039RLKAChXzpnUoFuGy3XDbn+J3y+q5pbHtXBYKDkYdVtq2CKCOc5RYJev9GD5pn4mrJ4xU67z34afiW7XLtWSDYo6O4XpRAWGPsl8aDoeDlRbmAIwV1ez/pkrn9Lkp8tFrilMgbMP2xI1lchAsawW7uzsL+xCui4Rf5CrqX1aMo7BGs42fFLn6jCJFfjmuCuPwngW+2oIn2+uAkfm0CfVl5r119Nq8GUwmEQ/+f5DrI3j4DSLJqgMySfaHZSFNZY3YzuJCYOkTXoogzObgYwmQUfYFinzpjv+8gRekN2qIw5GvV+4fSJiWGeZIsYMedb9fh0/mrSBQbELxX1WMm/6pmXCbzGTdncXJmIy+SS9UhSXs7Ecfm0VA87baB6HW9/wfgpAc81FDe0jSVq+9h3HWYSdJR6LcV2DNO56KB5 vfqmv72d ry8rI+nW9BgTlMNT7rmfycade4u2exDKgow8wBag7sFTZ/057VJHCsWlL1UPQJ6bGYKEGHOe44i0SFtGxCvPndHvkzloQhbNmRDfJzU8djSFoXuKoMOCXit+j6jTl4qp7cb4g9js/BjWB4bnq/vUnOLrrIBzS6+iGuKpmFZm41cw2McCp+WQbPDa+TZ7Aj0NBkYOmJJQkYi9Y0+k= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Commit 90292aca9854 ("arm64: mm: use appropriate ctors for page tables") introduced pgtable ctor calls in pgd_pgtable_alloc(). To identify the pgtable level and call the appropriate ctor, the *_SHIFT value associated with the pgtable level is used. However, those values do not unambiguously identify a level, because if a given level is folded, the *_SHIFT value will be equal to that of the upper level (e.g. PMD_SHIFT == PUD_SHIFT if PMD is folded). As things stand, there is probably not much damaged done by calling the ctor for a different level, and ARCH_ENABLE_SPLIT_PMD_PTLOCK is only selected if PMD isn't folded (so we don't needlessly initialise pmd_ptlock). Still, this is pretty confusing, and it would get even more confusing when adding ctor calls for the remaining levels. Let's simplify all this by using an enum to identify the pgtable level instead; this way folding becomes irrelevant. This is inspired by one of the m68k pgtable allocators (arch/m68k/include/asm/motorola_pgalloc.h). Signed-off-by: Kevin Brodsky --- arch/arm64/mm/mmu.c | 54 +++++++++++++++++++++++++++------------------ 1 file changed, 33 insertions(+), 21 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 8c5c471cfb06..eca324b3a6fc 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -46,6 +46,13 @@ #define NO_CONT_MAPPINGS BIT(1) #define NO_EXEC_MAPPINGS BIT(2) /* assumes FEAT_HPDS is not used */ +enum pgtable_type { + TABLE_PTE, + TABLE_PMD, + TABLE_PUD, + TABLE_P4D, +}; + u64 kimage_voffset __ro_after_init; EXPORT_SYMBOL(kimage_voffset); @@ -107,7 +114,7 @@ pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, } EXPORT_SYMBOL(phys_mem_access_prot); -static phys_addr_t __init early_pgtable_alloc(int shift) +static phys_addr_t __init early_pgtable_alloc(enum pgtable_type pgtable_type) { phys_addr_t phys; @@ -192,7 +199,7 @@ static void init_pte(pte_t *ptep, unsigned long addr, unsigned long end, static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + phys_addr_t (*pgtable_alloc)(enum pgtable_type), int flags) { unsigned long next; @@ -207,7 +214,7 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, if (flags & NO_EXEC_MAPPINGS) pmdval |= PMD_TABLE_PXN; BUG_ON(!pgtable_alloc); - pte_phys = pgtable_alloc(PAGE_SHIFT); + pte_phys = pgtable_alloc(TABLE_PTE); ptep = pte_set_fixmap(pte_phys); init_clear_pgtable(ptep); ptep += pte_index(addr); @@ -243,7 +250,7 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, static void init_pmd(pmd_t *pmdp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), int flags) + phys_addr_t (*pgtable_alloc)(enum pgtable_type), int flags) { unsigned long next; @@ -277,7 +284,8 @@ static void init_pmd(pmd_t *pmdp, unsigned long addr, unsigned long end, static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), int flags) + phys_addr_t (*pgtable_alloc)(enum pgtable_type), + int flags) { unsigned long next; pud_t pud = READ_ONCE(*pudp); @@ -294,7 +302,7 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, if (flags & NO_EXEC_MAPPINGS) pudval |= PUD_TABLE_PXN; BUG_ON(!pgtable_alloc); - pmd_phys = pgtable_alloc(PMD_SHIFT); + pmd_phys = pgtable_alloc(TABLE_PMD); pmdp = pmd_set_fixmap(pmd_phys); init_clear_pgtable(pmdp); pmdp += pmd_index(addr); @@ -325,7 +333,7 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, static void alloc_init_pud(p4d_t *p4dp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + phys_addr_t (*pgtable_alloc)(enum pgtable_type), int flags) { unsigned long next; @@ -339,7 +347,7 @@ static void alloc_init_pud(p4d_t *p4dp, unsigned long addr, unsigned long end, if (flags & NO_EXEC_MAPPINGS) p4dval |= P4D_TABLE_PXN; BUG_ON(!pgtable_alloc); - pud_phys = pgtable_alloc(PUD_SHIFT); + pud_phys = pgtable_alloc(TABLE_PUD); pudp = pud_set_fixmap(pud_phys); init_clear_pgtable(pudp); pudp += pud_index(addr); @@ -383,7 +391,7 @@ static void alloc_init_pud(p4d_t *p4dp, unsigned long addr, unsigned long end, static void alloc_init_p4d(pgd_t *pgdp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + phys_addr_t (*pgtable_alloc)(enum pgtable_type), int flags) { unsigned long next; @@ -397,7 +405,7 @@ static void alloc_init_p4d(pgd_t *pgdp, unsigned long addr, unsigned long end, if (flags & NO_EXEC_MAPPINGS) pgdval |= PGD_TABLE_PXN; BUG_ON(!pgtable_alloc); - p4d_phys = pgtable_alloc(P4D_SHIFT); + p4d_phys = pgtable_alloc(TABLE_P4D); p4dp = p4d_set_fixmap(p4d_phys); init_clear_pgtable(p4dp); p4dp += p4d_index(addr); @@ -427,7 +435,7 @@ static void alloc_init_p4d(pgd_t *pgdp, unsigned long addr, unsigned long end, static void __create_pgd_mapping_locked(pgd_t *pgdir, phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + phys_addr_t (*pgtable_alloc)(enum pgtable_type), int flags) { unsigned long addr, end, next; @@ -455,7 +463,7 @@ static void __create_pgd_mapping_locked(pgd_t *pgdir, phys_addr_t phys, static void __create_pgd_mapping(pgd_t *pgdir, phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + phys_addr_t (*pgtable_alloc)(enum pgtable_type), int flags) { mutex_lock(&fixmap_lock); @@ -468,10 +476,11 @@ static void __create_pgd_mapping(pgd_t *pgdir, phys_addr_t phys, extern __alias(__create_pgd_mapping_locked) void create_kpti_ng_temp_pgd(pgd_t *pgdir, phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), int flags); + phys_addr_t (*pgtable_alloc)(enum pgtable_type), + int flags); #endif -static phys_addr_t __pgd_pgtable_alloc(int shift) +static phys_addr_t __pgd_pgtable_alloc(enum pgtable_type pgtable_type) { /* Page is zeroed by init_clear_pgtable() so don't duplicate effort. */ void *ptr = (void *)__get_free_page(GFP_PGTABLE_KERNEL & ~__GFP_ZERO); @@ -480,23 +489,26 @@ static phys_addr_t __pgd_pgtable_alloc(int shift) return __pa(ptr); } -static phys_addr_t pgd_pgtable_alloc(int shift) +static phys_addr_t pgd_pgtable_alloc(enum pgtable_type pgtable_type) { - phys_addr_t pa = __pgd_pgtable_alloc(shift); + phys_addr_t pa = __pgd_pgtable_alloc(pgtable_type); struct ptdesc *ptdesc = page_ptdesc(phys_to_page(pa)); /* * Call proper page table ctor in case later we need to * call core mm functions like apply_to_page_range() on * this pre-allocated page table. - * - * We don't select ARCH_ENABLE_SPLIT_PMD_PTLOCK if pmd is - * folded, and if so pagetable_pte_ctor() becomes nop. */ - if (shift == PAGE_SHIFT) + switch (pgtable_type) { + case TABLE_PTE: BUG_ON(!pagetable_pte_ctor(NULL, ptdesc)); - else if (shift == PMD_SHIFT) + break; + case TABLE_PMD: BUG_ON(!pagetable_pmd_ctor(NULL, ptdesc)); + break; + default: + break; + } return pa; } From patchwork Tue Apr 8 09:52:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14042591 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FBB2C369A4 for ; Tue, 8 Apr 2025 09:53:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4363C28000B; Tue, 8 Apr 2025 05:53:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 40C99280001; Tue, 8 Apr 2025 05:53:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D3D328000B; Tue, 8 Apr 2025 05:53:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 0D2FB280001 for ; Tue, 8 Apr 2025 05:53:55 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 49AEA1A09FC for ; Tue, 8 Apr 2025 09:53:55 +0000 (UTC) X-FDA: 83310415230.07.AFA7A0A Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf23.hostedemail.com (Postfix) with ESMTP id 8797A140003 for ; Tue, 8 Apr 2025 09:53:53 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf23.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744106033; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k9E8JC26wdK4bPf2nrNh902XnvK5CWhMXm2sdNweAJc=; b=y3iGYDwm7oVLSYtx3w+HQbKzVXdHKSG3Q6Gv0Suyo8Ep6Wj4IMD0QwFDS3HQ+AwBN5waq4 pyqOaKnvJCudoOFn6LHwFLRXP/dFxZTh/wdaaKSf349ShMoEjh9CAVk45qM9GV3F4vzHil MEWXBADBle+C2x2lghkcrkWugfwsVnM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744106033; a=rsa-sha256; cv=none; b=l/fs/0ge1ja41N5dvKCxfgHnlCe3+Om+uekUv+Nmz0dMjLCxTadMPUmvpZtYYDOJtFYikw TGJFYwbY28giuH3cnIGrTuSlkRCNQP80E1k+UtM5xaEFi/KMQXP1Av6kq1JGhhBv1QQRj4 jCcLJ0fO4otxAVZa5zC3ybBhABaq1VY= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf23.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A4D9F1691; Tue, 8 Apr 2025 02:53:53 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 64C983F6A8; Tue, 8 Apr 2025 02:53:48 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v2 09/12] arm64: mm: Always call PTE/PMD ctor in __create_pgd_mapping() Date: Tue, 8 Apr 2025 10:52:19 +0100 Message-ID: <20250408095222.860601-10-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250408095222.860601-1-kevin.brodsky@arm.com> References: <20250408095222.860601-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 8797A140003 X-Stat-Signature: xtsi31g41geca8azaq4i5tj9icrrewwj X-Rspam-User: X-HE-Tag: 1744106033-955803 X-HE-Meta: U2FsdGVkX19XcglI7DmJLDSUJLGzg5WxnQBNsgtkf13/BB4RdI47XPGSbNdhafIBkQ7ualy/2Ale1gvIASIfih/Pj0yjbuawho7cHrl45ZANYOu87G9XZCF4o5PfFHap7jNku3Z5ZcRrYAvU0S4sAkMeUEt9OqOaGKaMhJo37Tq0KR2Nu/2KABD1/O2AxxrpX6uO6pylNLwA5tX2vEGcF9roStU2Z+jX/PPr0Xz+qkKEQDCss7jrCu+fUYvQLkA4sy05L1Z6liCpd0SQJgS5fJiqEHoHzBJkyAet8QgfbUVSPI87H4kMAUK5PbS/1c59EzqOPM7IlOP5NLtWo+LkVCEn769iVaB/1wKJIuYiXySBQUClXqzsNvPCFfh/tbuXGvOOkVI4FqvWUotL0gmLSb+shIgpwMA0EZnpNI6tGHcSrOiq1LVt5orrrVezbPYKtj4SkcKeEP9p4Fe+oPLeK8X6hsR7yXNu4SixnwFbUWwyaTjuL1+WIxTjDJKRvNLz/f9+IezupK8R6DiPnc5SBLdNfpURUvFXSil/QGqEC46cDEsIQJ4BUTl77zidc7FN7IlyQ1KKor7+8ct2ZnpbHIhBecFpk9ZJCjDsw4OQ+0csSS/KLaSXLEoidpi8rAOyXtHub7+wUULC5v6wwTxbezAU9VCacOJxGiejHE9lUubBgAFUQ7d8vH6qJSiU+QZpblMywK3S1tGOQKmnbbCZmGvIsVm/9attV2gPRPlf6Q3ld//ccd8dnnAWwBomxSiWSWltq0ZpR3as+LGibQgv7Ajo3UoufgTvhkXojCx6SizPgw2/0/8UZMxnbS5af+zdL/R13YV+EAM4lWWafNClLtp2dsbgyijBNDzjlZUlPOZLfYPwN0IWTpxWf93vnXMrKHtX6bebPFeIJcaCC4d+4qsXvYMjLy0dhL2jkOl4TQmuwZ6GYF11WTb9a3deAu6Actbo9ag3MVTbDNnGK24 VYUzR8Ph 8tq0o/8XAQiu5B5mI7f/nvf+VHet9J844dwiTWIfjUXNVcLvRw0DZ9dDs+XIVQPdxTkU/rlRdvNVubdgmUy0oXM8+Ey/Fp0nKmFjyPqleWuaejBskIV9cK32zTyW3AwZKQrMsU/rmIV68CsL9GpTJ/6SzsjbFVY/vrbb52ipRPwA1iEPaiwj8Jk1362QnW4EUURIvOiV/HiqbU+aJlzbR88ApT7Dc+EwYY8g2Q3hS7Ixtv1Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: TL;DR: always call the PTE/PMD ctor, passing the appropriate mm to skip ptlock_init() if unneeded. __create_pgd_mapping() is used for creating different kinds of mappings, and may allocate page table pages if passed an allocator callback. There are currently three such cases: 1. create_pgd_mapping(), which is used to create the EFI mapping 2. arch_add_memory() 3. map_entry_trampoline() 1. uses pgd_pgtable_alloc() as allocator callback, which calls the PTE/PMD ctor, while 2. and 3. use __pgd_pgtable_alloc(), which does not. The rationale is most likely that pgtables associated with init_mm do not make use of split page table locks, and it is therefore unnecessary to initialise them by calling the ctor. 2. operates on swapper_pg_dir so the allocated pgtables are clearly associated with init_mm, this is arguably the case for 3. too (the trampoline mapping is never modified so ptlocks are anyway irrelevant). 1. corresponds to efi_mm so ptlocks need to be initialised in that case. We are now moving towards calling the ctor for all page tables, even those associated with init_mm. pagetable_{pte,pmd}_ctor() have become aware of the associated mm so that the ptlock initialisation can be skipped for init_mm. This patch therefore amends the allocator callbacks so that the PTE/PMD ctor are always called, with an appropriate mm pointer to avoid unnecessary ptlock overhead. Modifying the prototype of the allocator callbacks to take the mm and propagating that pointer all the way down would be pretty invasive. Instead: * __pgd_pgtable_alloc() (cases 2. and 3. above) is replaced with pgd_pgtable_alloc_init_mm(), resulting in the ctors being called with &init_mm. This is the main functional change in this patch; the ptlock still isn't initialised, but other ctor actions (e.g. accounting-related) are now carried out for those allocated pgtables. * pgd_pgtable_alloc() (case 1. above) is replaced with pgd_pgtable_alloc_special_mm(), resulting in the ctors being called with NULL as mm. No functional change here; NULL essentially means "not init_mm", and the ptlock is still initialised. __pgd_pgtable_alloc() is now the common implementation of those two helpers. While at it we switch it to using pagetable_alloc() like standard pgtable allocator functions and remove the comment regarding ctor calls (ctors are now always expected to be called). Signed-off-by: Kevin Brodsky --- arch/arm64/mm/mmu.c | 43 +++++++++++++++++++++++-------------------- 1 file changed, 23 insertions(+), 20 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index eca324b3a6fc..51cfc891f6a1 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -480,31 +480,22 @@ void create_kpti_ng_temp_pgd(pgd_t *pgdir, phys_addr_t phys, unsigned long virt, int flags); #endif -static phys_addr_t __pgd_pgtable_alloc(enum pgtable_type pgtable_type) +static phys_addr_t __pgd_pgtable_alloc(struct mm_struct *mm, + enum pgtable_type pgtable_type) { /* Page is zeroed by init_clear_pgtable() so don't duplicate effort. */ - void *ptr = (void *)__get_free_page(GFP_PGTABLE_KERNEL & ~__GFP_ZERO); + struct ptdesc *ptdesc = pagetable_alloc(GFP_PGTABLE_KERNEL & ~__GFP_ZERO, 0); + phys_addr_t pa; - BUG_ON(!ptr); - return __pa(ptr); -} - -static phys_addr_t pgd_pgtable_alloc(enum pgtable_type pgtable_type) -{ - phys_addr_t pa = __pgd_pgtable_alloc(pgtable_type); - struct ptdesc *ptdesc = page_ptdesc(phys_to_page(pa)); + BUG_ON(!ptdesc); + pa = page_to_phys(ptdesc_page(ptdesc)); - /* - * Call proper page table ctor in case later we need to - * call core mm functions like apply_to_page_range() on - * this pre-allocated page table. - */ switch (pgtable_type) { case TABLE_PTE: - BUG_ON(!pagetable_pte_ctor(NULL, ptdesc)); + BUG_ON(!pagetable_pte_ctor(mm, ptdesc)); break; case TABLE_PMD: - BUG_ON(!pagetable_pmd_ctor(NULL, ptdesc)); + BUG_ON(!pagetable_pmd_ctor(mm, ptdesc)); break; default: break; @@ -513,6 +504,18 @@ static phys_addr_t pgd_pgtable_alloc(enum pgtable_type pgtable_type) return pa; } +static phys_addr_t __maybe_unused +pgd_pgtable_alloc_init_mm(enum pgtable_type pgtable_type) +{ + return __pgd_pgtable_alloc(&init_mm, pgtable_type); +} + +static phys_addr_t +pgd_pgtable_alloc_special_mm(enum pgtable_type pgtable_type) +{ + return __pgd_pgtable_alloc(NULL, pgtable_type); +} + /* * This function can only be used to modify existing table entries, * without allocating new levels of table. Note that this permits the @@ -542,7 +545,7 @@ void __init create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; __create_pgd_mapping(mm->pgd, phys, virt, size, prot, - pgd_pgtable_alloc, flags); + pgd_pgtable_alloc_special_mm, flags); } static void update_mapping_prot(phys_addr_t phys, unsigned long virt, @@ -756,7 +759,7 @@ static int __init map_entry_trampoline(void) memset(tramp_pg_dir, 0, PGD_SIZE); __create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS, entry_tramp_text_size(), prot, - __pgd_pgtable_alloc, NO_BLOCK_MAPPINGS); + pgd_pgtable_alloc_init_mm, NO_BLOCK_MAPPINGS); /* Map both the text and data into the kernel page table */ for (i = 0; i < DIV_ROUND_UP(entry_tramp_text_size(), PAGE_SIZE); i++) @@ -1362,7 +1365,7 @@ int arch_add_memory(int nid, u64 start, u64 size, flags |= NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; __create_pgd_mapping(swapper_pg_dir, start, __phys_to_virt(start), - size, params->pgprot, __pgd_pgtable_alloc, + size, params->pgprot, pgd_pgtable_alloc_init_mm, flags); memblock_clear_nomap(start, size); From patchwork Tue Apr 8 09:52:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14042592 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA349C3600C for ; Tue, 8 Apr 2025 09:54:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AAABA28000C; Tue, 8 Apr 2025 05:53:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A338F280001; Tue, 8 Apr 2025 05:53:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 920B228000C; Tue, 8 Apr 2025 05:53:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 723C0280001 for ; Tue, 8 Apr 2025 05:53:59 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id B069E1A0A10 for ; Tue, 8 Apr 2025 09:53:59 +0000 (UTC) X-FDA: 83310415398.06.861BC58 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf05.hostedemail.com (Postfix) with ESMTP id 1FD7610000D for ; Tue, 8 Apr 2025 09:53:57 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744106038; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/xCdomzVQ9UKIUqSV0nOR1Orsi3U0kam6TrHTVmoiR4=; b=lupRO59ILO/HZI1+ZLLW7773BmaqQ0rs5/+NhW9K98SrcpOfcTrOu0nN/Oy1NkXh8LfJ7o nkfImwMurc0zqqBAInwcCu5jOl6q9pgUn+6eJm21oByA248wp6LC3I0GBXXEGSZNY222b+ MssC1+iC3B/yQPvVJT0+hqM4oC6n9ZM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744106038; a=rsa-sha256; cv=none; b=1sBFDXm/9a9oQUu2BmVQiOa5UR0QWpdjGr8cSYHqOCgPlkRnCmWMlWRJxJm8gDvcSw8s9o nLnz2qWME4VE/SJdwPwEUnV1lMmUfALmpi0MYNErqnceiPDCOWjX7UIpP0FQK/97NhWHk+ XZ5cwNh/ePm7erdPyteOsFgUUTuFjnY= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5D48D16A3; Tue, 8 Apr 2025 02:53:58 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1DCDA3F6A8; Tue, 8 Apr 2025 02:53:52 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v2 10/12] riscv: mm: Clarify ctor mm argument in alloc_{pte,pmd}_late Date: Tue, 8 Apr 2025 10:52:20 +0100 Message-ID: <20250408095222.860601-11-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250408095222.860601-1-kevin.brodsky@arm.com> References: <20250408095222.860601-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1FD7610000D X-Stat-Signature: tgi89zumacaswgdx8k1idezfa8j8p9ro X-Rspam-User: X-HE-Tag: 1744106037-235133 X-HE-Meta: U2FsdGVkX18QQNLpwLZrtGzetblteyASPGietcZ/wvpT+E0Vogzfb5vcQj3aknu5UboVzjIT9ILTc32MrMlx789u2wn/N/2PkHYhrxp4UUmWWeHC35VQg+mwnISjTfw58/f8kYNAWKyYrUpzxBcallzEDbTGJBQAovwdM+2O2vovNsoy5jciY00HLDAQ37LMxHlfwGZMbZ4cqpvT3mTiVX5CYumYbxEisxsSqlDPm+X9hZi8IBTPUoi45LNUKqoWddo0/+UWQvEKT+PBxYz/C9BfcTtiO1K6G6euigN6CHM/gqB0Bhs3UlSYAo+jkOUvprBK7P8VXtnCMrSbaYM+7jYnTyRLAOw4LvDtvMkJwnWznC/3vsUJIqPkIE+HoOg2UDhc7JKvT5lIKDQkGczCsagi0Z5vN+i1ZJYb7aySnliEBgxV0Iz9ZgTii4QrMNYy5b57USKAQezQ+CsOeNuECGpKi1rkxUdoYPqdwncBwBgiy1OeIb51B5ZSDyU+AWlRtmsA8U+THL8CJ2EuWuCHha/4yExK4cPTCmgbvR82G+SJu8r4+szxwQLCR6URhiGWun9zCOcUn7vDmfGEXDLAvqrU9/btX/w1AUTCENCw6+dSENXYIRhyPZJDZgqiF3DLIgOpOoU8iWN1OUSOsHfQDskk+L5DXR9iJWoGLbhrE/ikUPWw5RMAWth76k9P/nVCEcmBy18fDNxwaeOoo3HkONLJ8mAoI5jalWS5az3fXSQx4Hr7ofKkiwCxw/W0w/ADvvlvjzPZEFBHDkcadwEv+PQvK4WJiX55R7hkLRfQWoFfL9OQaN5a/l/IW8ScLKlriARAodgwRhCH3Vi6x0FKdoP14gocaeZNuy/w4i4/AVIY0rzfZR6UmXlDMzzgA+Wp7V5GAuMF7UKICuwrbuUDix/wS0DxpfRN9D2b7h52RDbf8hXKRUFW7QgShZOZqA4gv5VtajRv20Os9qsJZXk HJGVDTeJ wxSIEkyXGCYAR15nahrRj181o11L9v5sR08F75wtBdl0cnbBwpKv33WHfwxBXnXc85Jo+iAWeinKsGr3J+CXIQfdq6nw54Oq9hjkVpHeg0qgslk9X59McXFpT6uq6y2XFBVcX4dUne0OaSuc/0RR5X4wES2es2ZR3rb4T2iG1O2mI2ow7KsH0GyRLbDlPUEfMfG34+UQW0wJAukk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: pagetable_{pte,pmd}_ctor(mm, ptdesc) skip the ptlock initialisation if mm is &init_mm. To avoid unnecessary overhead, it is therefore preferable to pass the actual mm associated to the PTE/PMD. Unfortunately, this proves challenging for alloc_{pte,pmd}_late() as the associated mm is not available at the point where they are called - in fact not even top-level functions like create_pgd_mapping() are passed the mm. As a result they both call the ctor with NULL as mm; this is safe but potentially wasteful. This is not a new situation, but let's add a couple of comments to clarify it. Signed-off-by: Kevin Brodsky --- arch/riscv/mm/init.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index e5ef693fc778..59a982f88908 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -442,6 +442,11 @@ static phys_addr_t __meminit alloc_pte_late(uintptr_t va) { struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL & ~__GFP_HIGHMEM, 0); + /* + * We do not know which mm the PTE page is associated to at this point. + * Passing NULL to the ctor is the safe option, though it may result + * in unnecessary work (e.g. initialising the ptlock for init_mm). + */ BUG_ON(!ptdesc || !pagetable_pte_ctor(NULL, ptdesc)); return __pa((pte_t *)ptdesc_address(ptdesc)); } @@ -522,6 +527,7 @@ static phys_addr_t __meminit alloc_pmd_late(uintptr_t va) { struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL & ~__GFP_HIGHMEM, 0); + /* See comment in alloc_pte_late() regarding NULL passed the ctor */ BUG_ON(!ptdesc || !pagetable_pmd_ctor(NULL, ptdesc)); return __pa((pmd_t *)ptdesc_address(ptdesc)); } From patchwork Tue Apr 8 09:52:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14042593 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A21BC3600C for ; Tue, 8 Apr 2025 09:54:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7F6E2280002; Tue, 8 Apr 2025 05:54:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7A763280001; Tue, 8 Apr 2025 05:54:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 66FC8280002; Tue, 8 Apr 2025 05:54:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 47B1E280001 for ; Tue, 8 Apr 2025 05:54:04 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 77AA9120A73 for ; Tue, 8 Apr 2025 09:54:04 +0000 (UTC) X-FDA: 83310415608.13.0645DD9 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf25.hostedemail.com (Postfix) with ESMTP id D9004A0008 for ; Tue, 8 Apr 2025 09:54:02 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf25.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744106042; a=rsa-sha256; cv=none; b=q99LI0SCoh5leQCrkgNQpAJMfssWR+hy+prOaqQj6rRJKLEWBC1mdkEoZtT58NPTC21dvR D8/bBH4opn40SnqqelkYDY+I5+Yj//YBZtFq2CJOCXEr4J437eBFzOiA6PHaxBsd+Irz54 CUbXcV4YLSFi6c+9OHZXz5+rJZSluGE= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf25.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744106042; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zCqJ5i1RxEJN7X5dUM1nM7yZwBGmu6xdERTmZRAkVic=; b=iBqZtVSSQwa6Zls2U7vD8D+G8q9XsTc/fU5xe8k/nTj/CKL88gZSrEfXvRFtdLnLS1NHir zLPbWdQspeRYJitP9yc2u/66TtWXWq8Ckq4jB0dnBiWR2gckg6HVrHOxh1iv5Lmkn8VOZr pquVd5B+S5XTU+DTYBlagyS80Stt1GU= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 15DC122FA; Tue, 8 Apr 2025 02:54:03 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CA25B3F6A8; Tue, 8 Apr 2025 02:53:57 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v2 11/12] arm64: mm: Call PUD/P4D ctor in __create_pgd_mapping() Date: Tue, 8 Apr 2025 10:52:21 +0100 Message-ID: <20250408095222.860601-12-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250408095222.860601-1-kevin.brodsky@arm.com> References: <20250408095222.860601-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: D9004A0008 X-Stat-Signature: yy7imnmyfg8ory5ydeid33hqkmmuew8g X-HE-Tag: 1744106042-583722 X-HE-Meta: U2FsdGVkX1/Ivcckn/6mQjIK4UqvH4+1Ene6JTVbckhn2CVcuW+2kODs2fPLFjUw9Ofzbvv2lo8BPqTFiWcaN5KpC0FaKva+cqsIzdABwMExhXXv3qlTiDf5GGdYa+TOUBTqzVSrTflN26kB7D2XxjVsg5ukCwQKGjVKlm3XSZ/RRUNiw4ilVabmqeMlZyxQe2CwR/DefcudY3lgdCVT/e10g+kbU9N/YmovMOEw77Dt6zNayzIKhliChjbFa9gdsoYNK5gqg6c90zHRRAwFgePOB8/Qkh4aVvgppdmaFMSyJNaG1kDT0pLGNo7MdFOYpGTOJoV8OHb60QG3H2zMGPtMAoMdrffGZFEvp8ZhLw18KNCJm8rxlTpyzG6O4FVld47h3Q8LJTwz1NBDjD5TAoZUep8NhLJIYLJdTJ8s2deW4VO3QroBDAERi0xQrGJ4l3nxm0wmMRG64I9U0tAb6jVPeWkwnSLQf4w3CiMAyvlBP4nM0fF/IfErduOXgQg80qR7SpwGL5+VdqYTiIITp0/G5lhVRRhIExXMZXmUXdVdik7PzvOYtokxk429bnfMvgvKkvbz4RW1jJ9MYuktfb2QhtRfYi1KQhBi8oq8+OUn2duNTdmW4cL09Q239bFzAynrYdYePrlRyAUKdSR9/ulTKeDh5u/khGy89LVQz+BCqjF6iOffDKaJqI89eW+YRcTO1deNuOKSTznb5jUxTNGNz6orrYey9BLSqLinCVNztyqevEZG39hU2qlb70MVJwseJkWa1+cxsJLk0ZNRRRGiZEz3PcFIfGiM/BDC+rKhtr6BucuQfqwJcBCV75ctuQqbOSLOJIqL738cmjbjKSBOIwfd3cIBFlX0azSqeVCkQZxvBl4l9qYz+zJGEf58+rG0rqOZFtO79DCPDvvzetJiTfY3qdkFnkFNj1K12CpDaW8byf+sG+XhIpVMYLm/AtnAXv5rjDphINHleKI tlSckeVQ n6rc2mJ+++Djs5IC7wd4q6lgaxgMvo31dCWZu3Mv6V6GpQIxmQKnrAY3c7V/k8FHTumLaicQqwWRn/r1Zmrm3xF21idERsoDf1vlKxfe76u8XYRxfEBg7wxz7rgeOjqu54tJZdki447jAX1Hl76hmh6hHentSWO05I0+zA4qvGahp70y2byXPtczBqc5ID2375X1aPDB7PD9XlLo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Constructors for PUD/P4D-level pgtables were recently introduced. They should be called for all pgtables; make sure they are called for special kernel mappings created by __create_pgd_mapping() too. Signed-off-by: Kevin Brodsky --- arch/arm64/mm/mmu.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 51cfc891f6a1..8fcf59ba39db 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -497,7 +497,11 @@ static phys_addr_t __pgd_pgtable_alloc(struct mm_struct *mm, case TABLE_PMD: BUG_ON(!pagetable_pmd_ctor(mm, ptdesc)); break; - default: + case TABLE_PUD: + pagetable_pud_ctor(ptdesc); + break; + case TABLE_P4D: + pagetable_p4d_ctor(ptdesc); break; } From patchwork Tue Apr 8 09:52:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14042594 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C7DFC3600C for ; Tue, 8 Apr 2025 09:54:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6302628000D; Tue, 8 Apr 2025 05:54:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5DFA8280001; Tue, 8 Apr 2025 05:54:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4578D28000D; Tue, 8 Apr 2025 05:54:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2560B280001 for ; Tue, 8 Apr 2025 05:54:09 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 24B9EA85AB for ; Tue, 8 Apr 2025 09:54:09 +0000 (UTC) X-FDA: 83310415818.30.DC6BE96 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf24.hostedemail.com (Postfix) with ESMTP id 8F789180005 for ; Tue, 8 Apr 2025 09:54:07 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf24.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744106047; a=rsa-sha256; cv=none; b=zROf5TBgtCgkuTSeeoU7WlpWFlbkxZneJ0ssL/9ZmXApJ6urtR5XFb08T9yrFSuJjwCdar SzqHuwgWQiNddjXhhIr9cgcLPulrWwxPaL5reDq3KQbm2oBW7S745O2II+OBsjdVfUCxzD iKPM27EtkPa+dTQqnz0i3TtmpoyNYpk= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf24.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744106047; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=08E//Tnzm4Otbmgjv5PkMZzFPCICf9ItkLK9Ykz94vQ=; b=zTV6Crc7dH7WMxAtgz+IDJoJPfKPnKQS3t6RVCo1FxhK3BbyO+QJkeAx0xnGtpuNvvXmxN wPvP8TPB2P1a+Uj4/ewUT6122rpTIR5r5GHl+zCEoRnzLnE9gA06WVjVFztyf6a515BQtA C45cArNGhAARpa5f5MGKBsMArdI6ACA= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C22912308; Tue, 8 Apr 2025 02:54:07 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8289A3F6A8; Tue, 8 Apr 2025 02:54:02 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v2 12/12] riscv: mm: Call PUD/P4D ctor in special kernel pgtable alloc Date: Tue, 8 Apr 2025 10:52:22 +0100 Message-ID: <20250408095222.860601-13-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250408095222.860601-1-kevin.brodsky@arm.com> References: <20250408095222.860601-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 8F789180005 X-Stat-Signature: 3zumbw4bhgr73zzxc5fo9qi3jjybkx19 X-HE-Tag: 1744106047-182198 X-HE-Meta: U2FsdGVkX19HkImlr6fp+Um/dx/0ICoOY+KG9a7XxQ3y8Zcg7L6KYvS2fCg07MfOERAVz8pxIbK6/O0FxVC6hRww7iZ8wgjAVopl8yajmhk3tyfj3DY97iW0xWOoWpHSiZnBCJh9+6esGEj1DX456maMtsBZVYn3Z0QDwHAcmq8hbD+UQfyz30HxrJAlvWlSt+62RSDK+HDs3v9pY8uWO6jN69FM5Q4/Fh9/X9rYdMLfa5sLZMf4P6C4uHCxmlgZ92pcsDVx7aakxffTONp0btbUTkT/Ia6i0ERjwiOv913LLnTmx/ynBegUu2TLqfoAL9KWMjeGZtq4P2mYAEwSlX3TpOtnyXxp/lHVMWsA/toenCI9tl53KDeGYXgd8ZhEb6eucgnV5QjoHrMciuFo9xhE9+mVcKD3RaQq9+31Srqtzdl5hTvlizFUnhKp0fh+LtLsQH3GTKb/yY/ZRLoI5vSbo+252NF5s/sWYp/yPkTpgB2/ftd96hBu7Cfoh3gpWMKa6FYC3LdvmFXLgLHodE5gv/yEyub+JUNnXBiziYqau5YnlcBOOk2RZfwFYqlhntgEcS54eCsbln4IlSkZdt6QF5PTlag5F3qg4PMi540Prkt6wYzys0l4j+GWIZFE8iX66OOLBctSAfRihz7nVJVCsOI9Yyhvb5vDfPEOrceOq7Jf+zM4HmQ/9AAROfjl6de7AbQrKfs5rGn3Ebg26QFbM4jMmFJu8GgrcO2CFxout5f772UPf9leMyb5Z+u/IuHXHxhF1JJc1Gvej8j9amYnz37U2c0HceduJDi2IdLL6KTTtQiBJdIYNNU9lQXCLA0yKGADsHWJLGYWiaFNhqvSk+zTqIBknMDFkFPikCZZt/TrDTvUGdn7ZlIUYc/4yqMbdbCrtiMA4p2olpSUzvPDdbUGYwhmJQB8J64i2Eybxgu8Eec0V0A1ySNxpqpEwHe8UFJiF2c7Syvmdod ouQICpuc L/yOGRbcB0UfcI370NAygENtwsgPBt0hpCKVl/8eG/RKpRQWTrBvgL8baDk8cn3POQ+UjbSombrcDbokm2f1nRy49I8vXWwk/in8E8eAJfIOR7iWHGb6XBRFEsKXp7eGyKhLdrx8KqACmEpILBEx8HRKm6w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Constructors for PUD/P4D-level pgtables were recently introduced. They should be called for all pgtables; make sure they are called for special kernel mappings created by create_pgd_mapping() too. While at it also switch to using pagetable_alloc() like in alloc_{pte,pmd}_late(). Signed-off-by: Kevin Brodsky --- arch/riscv/mm/init.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 59a982f88908..8d0374d7ce8e 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -590,11 +590,11 @@ static phys_addr_t __init alloc_pud_fixmap(uintptr_t va) static phys_addr_t __meminit alloc_pud_late(uintptr_t va) { - unsigned long vaddr; + struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL, 0); - vaddr = __get_free_page(GFP_KERNEL); - BUG_ON(!vaddr); - return __pa(vaddr); + BUG_ON(!ptdesc); + pagetable_pud_ctor(ptdesc); + return __pa((pud_t *)ptdesc_address(ptdesc)); } static p4d_t *__init get_p4d_virt_early(phys_addr_t pa) @@ -628,11 +628,11 @@ static phys_addr_t __init alloc_p4d_fixmap(uintptr_t va) static phys_addr_t __meminit alloc_p4d_late(uintptr_t va) { - unsigned long vaddr; + struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL, 0); - vaddr = __get_free_page(GFP_KERNEL); - BUG_ON(!vaddr); - return __pa(vaddr); + BUG_ON(!ptdesc); + pagetable_p4d_ctor(ptdesc); + return __pa((p4d_t *)ptdesc_address(ptdesc)); } static void __meminit create_pud_mapping(pud_t *pudp, uintptr_t va, phys_addr_t pa, phys_addr_t sz,