From patchwork Tue Apr 8 09:52:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14042566 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF3A1C369A1 for ; Tue, 8 Apr 2025 09:53:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E50D7280006; Tue, 8 Apr 2025 05:53:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DFDEE280001; Tue, 8 Apr 2025 05:53:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C78A5280006; Tue, 8 Apr 2025 05:53:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id AAAAA280001 for ; Tue, 8 Apr 2025 05:53:26 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id C9C07140991 for ; Tue, 8 Apr 2025 09:53:26 +0000 (UTC) X-FDA: 83310414012.26.47CFE24 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf17.hostedemail.com (Postfix) with ESMTP id 35AC740004 for ; Tue, 8 Apr 2025 09:53:25 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744106005; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EU3IKg5gNWi8e73Hj4jkz8q3EV56066wj6qzlOZeFZQ=; b=bSwla4hqP7TWG1gGJSJoYASeBMT5Xgx023hTnbp47Xs9cR/YthnETwEYi5RSlsFh3QS6ye NNEZK4niUdEysGRBXmsEOSdw8KwdBRW2CP+BvZf0MFzuthSveITFrWzkgVkpPCEsuhXq6F WLrZ3mbkIdNilrnhxR6DUvzol4GYHHI= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744106005; a=rsa-sha256; cv=none; b=2Cj4kI41ofxyy0pDqNS8zirtf5ur/FXobYk6WUTEZZDZ6gN0hItz+2+kkjzlZFCsN1BV2F lsbVFtXMW4GAHppP3f6SED4mT6V6OfzBnAdFxoHSumLM1Q34K92PYxnryPgoFkpyyuiYae F2gI4gBPiHvoROXV367dD/SD1e9iKkI= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 66F6A22FA; Tue, 8 Apr 2025 02:53:25 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 27BC83F6A8; Tue, 8 Apr 2025 02:53:20 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v2 03/12] mm: Call ctor/dtor for kernel PTEs Date: Tue, 8 Apr 2025 10:52:13 +0100 Message-ID: <20250408095222.860601-4-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250408095222.860601-1-kevin.brodsky@arm.com> References: <20250408095222.860601-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 35AC740004 X-Rspamd-Server: rspam05 X-Rspam-User: X-Stat-Signature: ban94fgdmx3i9i74f1fczyfbcbw9i7de X-HE-Tag: 1744106005-260331 X-HE-Meta: U2FsdGVkX19SJRGgeE9qQXr90Syv+ulUgFlZ+I10TIGRmabVs2Kb2mLRaVg9y0IZ+6xUIrY1acwbApieKiznr3SlWtDVYwCwUO/Y2KBfZZSvJxUADdTa/fu6y/SjlNx/nAaIrRQjIB9uX+Y6qDCQr1ZR+cCAbwgcmRwXXYMr/nfeHZ/B4d7A8SCsKQLBwRHJaTFRTdzn4/rDLXt6tEdc8q4MSgihU5yNvG/0w32JRrJk2ejMIxyBD7VCzMgppTmxG7Q+Z6utGOgkzmM5eLOo1lV5khUsMuizXISI1UrfkXl1yylUyIWJQhyqNh9pCezuKpNAhQvLOh2SFyuzG7vTsTHR+09oixBw1gGro5FO110UrEPj5xfP3hw4VjAEvh2xvYMutlNePsksFTAY/TKTZ7TUMDCCZF1u+r6aX9mF2xT+0A8hD+rf6E6E1yau5b3v9u1GkiAUnWWUU52FaXnmpdSexTd+SQjhv3+z6hodT6e0uNhNb5ikWqlasz83EoqL9j5PfjA7BTJDiGowDrorXUJMOFPK6Krc2/BYsJlrJV7mYuvJo0FfUkOHxERlhMW5Vpryce4SAwWEsFzZhKBjvlAvLMR+ratRWu9eXhtptzOatBzJ7BrMUCQb6ZLo0wIJeVyeKq8R1nfTm6vTPGypqbGjllZA3Z0qDcJF/ZMqqN9wRf1nSL46YO/usggxte5CMaVQxfGzv/h5lCYTWciYP36TOxRIFzdI9r17Uzmsj67tBOgxDhUcKT1qDI3qhu4QZQfUyRgQICtJy/l+v8tEirSa17fdnqbHmm66azMCVOyJzU/iDiIRH0HeRKfC3O16Xak9+2BD61DccpX8DKhHB0Mma0O6+ZmhPA7OTaRrpr8mmjXAHq3cNZi+NsuV689C9CTLVIUEX8LlrIoouFUS3Tk1iWPq/bjn7N8FnvKDt98BDTi92Ut8D2rOsvy9Mtirq/B35LoRzPhpeh9OwMT SVVwdrBw GAG/Hh/gpiHFscvSkAV9BdSBW3HYY38TH2RQzSNfhSA8X2cKlB4VHDOwLWSUtT535uxiQUFz9ec19+L2QEIQHmjAIJK1Pu7vGPxU9QYKLeUaEKqvaTDRHaiDxAn+Jxg+8Vv808cPCFU/JZ4IMz2DKBTFgOECj+oITuZQvikw1H1uo1Lee2jQc/8aR0A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since [1], constructors/destructors are expected to be called for all page table pages, at all levels and for both user and kernel pgtables. There is however one glaring exception: kernel PTEs are managed via separate helpers (pte_alloc_kernel/pte_free_kernel), which do not call the [cd]tor, at least not in the generic implementation. The most obvious reason for this anomaly is that init_mm is special-cased not to use split page table locks. As a result calling ptlock_init() for PTEs associated with init_mm would be wasteful, potentially resulting in dynamic memory allocation. However, pgtable [cd]tors perform other actions - currently related to accounting/statistics, and potentially more functionally significant in the future. Now that pagetable_pte_ctor() is passed the associated mm, we can make it skip the call to ptlock_init() for init_mm; this allows us to call the ctor from pte_alloc_one_kernel() too. This is matched by a call to the pgtable destructor in pte_free_kernel(); no special-casing is needed on that path, as ptlock_free() is already called unconditionally. (ptlock_free() is a no-op unless a ptlock was allocated for the given PTP.) This patch ensures that all architectures that rely on call the [cd]tor for kernel PTEs. pte_free_kernel() cannot be overridden so changing the generic implementation is sufficient. pte_alloc_one_kernel() can be overridden using __HAVE_ARCH_PTE_ALLOC_ONE_KERNEL, and a few architectures implement it by calling the page allocator directly. We amend those so that they call the generic __pte_alloc_one_kernel() instead, if possible, ensuring that the ctor is called. A few architectures do not use ; those will be taken care of separately. [1] https://lore.kernel.org/linux-mm/20250103184415.2744423-1-kevin.brodsky@arm.com/ Signed-off-by: Kevin Brodsky --- arch/csky/include/asm/pgalloc.h | 2 +- arch/microblaze/mm/pgtable.c | 2 +- arch/openrisc/mm/ioremap.c | 2 +- include/asm-generic/pgalloc.h | 7 ++++++- include/linux/mm.h | 2 +- 5 files changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/csky/include/asm/pgalloc.h b/arch/csky/include/asm/pgalloc.h index 11055c574968..9ed2b15ffd94 100644 --- a/arch/csky/include/asm/pgalloc.h +++ b/arch/csky/include/asm/pgalloc.h @@ -29,7 +29,7 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) pte_t *pte; unsigned long i; - pte = (pte_t *) __get_free_page(GFP_KERNEL); + pte = __pte_alloc_one_kernel(mm); if (!pte) return NULL; diff --git a/arch/microblaze/mm/pgtable.c b/arch/microblaze/mm/pgtable.c index 9f73265aad4e..e96dd1b7aba4 100644 --- a/arch/microblaze/mm/pgtable.c +++ b/arch/microblaze/mm/pgtable.c @@ -245,7 +245,7 @@ unsigned long iopa(unsigned long addr) __ref pte_t *pte_alloc_one_kernel(struct mm_struct *mm) { if (mem_init_done) - return (pte_t *)__get_free_page(GFP_KERNEL | __GFP_ZERO); + return __pte_alloc_one_kernel(mm); else return memblock_alloc_try_nid(PAGE_SIZE, PAGE_SIZE, MEMBLOCK_LOW_LIMIT, diff --git a/arch/openrisc/mm/ioremap.c b/arch/openrisc/mm/ioremap.c index 8e63e86251ca..3b352f97fecb 100644 --- a/arch/openrisc/mm/ioremap.c +++ b/arch/openrisc/mm/ioremap.c @@ -36,7 +36,7 @@ pte_t __ref *pte_alloc_one_kernel(struct mm_struct *mm) pte_t *pte; if (likely(mem_init_done)) { - pte = (pte_t *)get_zeroed_page(GFP_KERNEL); + pte = __pte_alloc_one_kernel(mm); } else { pte = memblock_alloc_or_panic(PAGE_SIZE, PAGE_SIZE); } diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h index e164ca66f0f6..3c8ec3bfea44 100644 --- a/include/asm-generic/pgalloc.h +++ b/include/asm-generic/pgalloc.h @@ -23,6 +23,11 @@ static inline pte_t *__pte_alloc_one_kernel_noprof(struct mm_struct *mm) if (!ptdesc) return NULL; + if (!pagetable_pte_ctor(mm, ptdesc)) { + pagetable_free(ptdesc); + return NULL; + } + return ptdesc_address(ptdesc); } #define __pte_alloc_one_kernel(...) alloc_hooks(__pte_alloc_one_kernel_noprof(__VA_ARGS__)) @@ -48,7 +53,7 @@ static inline pte_t *pte_alloc_one_kernel_noprof(struct mm_struct *mm) */ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) { - pagetable_free(virt_to_ptdesc(pte)); + pagetable_dtor_free(virt_to_ptdesc(pte)); } /** diff --git a/include/linux/mm.h b/include/linux/mm.h index f9b793cce2c1..3f48e449574a 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3103,7 +3103,7 @@ static inline void pagetable_dtor_free(struct ptdesc *ptdesc) static inline bool pagetable_pte_ctor(struct mm_struct *mm, struct ptdesc *ptdesc) { - if (!ptlock_init(ptdesc)) + if (mm != &init_mm && !ptlock_init(ptdesc)) return false; __pagetable_ctor(ptdesc); return true;