From patchwork Fri Jan 3 18:44:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13925790 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63098E77188 for ; Fri, 3 Jan 2025 18:44:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E48406B0093; Fri, 3 Jan 2025 13:44:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DF4216B0095; Fri, 3 Jan 2025 13:44:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C95A26B0096; Fri, 3 Jan 2025 13:44:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id AC7736B0093 for ; Fri, 3 Jan 2025 13:44:36 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 25B9CC0A49 for ; Fri, 3 Jan 2025 18:44:36 +0000 (UTC) X-FDA: 82967016552.27.2663F86 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf07.hostedemail.com (Postfix) with ESMTP id 6AA0540002 for ; Fri, 3 Jan 2025 18:44:34 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735929874; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vSoOMdn+/GfqJnDjyLF0mh7cPRiC2LKGG844rQUUfqE=; b=GE7CdlvY9uZmPZuBBScBhLp4ggHYgtPGqOGciI+0xi4nb6fjJFj5KKvhuN22Fmp4IPTb4j 6dJUTWIz84Er8pya51ZQ+2nJPy3uFzV3lEkO0L9rulSvOU63YlWeQPeMoGcs0vETPgeLt9 1s/w6d7lTcCbxBO/PC0sugvjTFEu0ac= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735929874; a=rsa-sha256; cv=none; b=GawkmYZd20NUBP7f15Lw7wPrFX8jDJEH8C3YIM5orj6a8z8htdHU3ZNU2vYhvxtN+zxgVg NytKPuDdEj9W60fryZhzZnL6lIej1kOmDqwL8jTWzzoZVDG4NgfwP3DfJVA+GU/d/iPauS hr+q4wzbCoG4BtA+lDlCRQnVOUCZ9cg= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A16E7150C; Fri, 3 Jan 2025 10:45:01 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6B3F53F673; Fri, 3 Jan 2025 10:44:29 -0800 (PST) From: Kevin Brodsky To: linux-mm@kvack.org Cc: Kevin Brodsky , Andrew Morton , Catalin Marinas , Dave Hansen , Linus Walleij , Andy Lutomirski , Peter Zijlstra , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-alpha@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-kernel@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-um@lists.infradead.org, loongarch@lists.linux.dev, x86@kernel.org Subject: [PATCH v2 1/6] mm: Move common part of pagetable_*_ctor to helper Date: Fri, 3 Jan 2025 18:44:10 +0000 Message-ID: <20250103184415.2744423-2-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250103184415.2744423-1-kevin.brodsky@arm.com> References: <20250103184415.2744423-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 6AA0540002 X-Stat-Signature: 6w63xg4x4qjxxi6bqd5w1cnzrkm8jug4 X-Rspam-User: X-HE-Tag: 1735929874-506660 X-HE-Meta: U2FsdGVkX1+pOyNbPYNlm7q+RHWbUb1l+HDx9O4bVsgnflK/ffv5vVgQD0w64AYg9UOXGtVPQc9z2Egc4TNRzf1Vx/G7XejqWKNg1KTX0eSARHsz8J17IdsAVmvPlc0fj3k51d5Geu+nmlt4KBTj3kMpkSIG8J9t8s7I4A78s5zElqkcumTtMEEPcOiUc6s3qq3v28UALX7ooJ644nUt1pbN7HSvpmzzlNzELILpKeNWm5AHY+lFHfXvPpIflOdTXhs6xplLUxAC4we0GwsKzdSvDNb/gwjLdxydcH8mfPrG5RXGENotQVoOL97yTvKLmmaubQq2YRa8Poez3svSPLTKhd1cJHh1uQaTw0x2FLKdTWSvr014dgXBBTuLafyNcqYOs2ZRt4iAzRrY/jaqHKcfGKtUJNF6StKmMKJvYezHYl5MogcWlrHSkrLe2k+ByBA/OPKXQjdqEwNi69qX3Jr23bknwIneOXhwo8Qnk8cClxg7XFzWaARglFj2qwWvx3kIl7bwIJ1K12xb3QsFPRL54nnGJO4geD4N8il5jLDKi1B/imCu5Nt2+k/HROC3wlnEmmfSdOn5kUwi4YuIsltPUoaGrMKfRA2R9eKJpjK18oNMg07TPjyP06W94dhLi+nZOffvwTcvcE1EiGp3c2zZucDSNqPV6OwDe2/0RLwJyI1D69h7FuCCfwt9mgbCFivGW/0vyaA5PHdJvOyGwcgO0Vdxi0BN2YS+9NewmQUVcg916MCFXbHrO1yDu7TOp290a2VtPPKYRtqpDaKwcNBnOQfScY5LSlJ2TsEsydskD01QPdxH7iQ23yCwfU3kKTlmQ+Y6gIieVbAud3stkdrKo+YFc5MZBH2RrZRIcTAgXBVtfj3WaTA/NMGkeSjxsk6MvHR9lZ0T8zSRX7Du0YXkdPk7qoGgSA3TmZd1Rv4bI7szcQxe0SHXvuJSQXfstqzpzUPaRvw9mNGw7GQ EhggSLbK FMD5Q/Ow1NYTtHZY86ss8Vbflv+xJj58pFBu0Uo91FVwIS56jRcF8+TaRGOMIovwgGdieEdlNS3xPXX11JGovDaqxtr+BVwCUwr9mLokaHPZnYrsQKDKjKrcx3x513bHE6tvBYpjy5Ze1VF20URCMNVoSH4SB5VsX9lwumyh7sjWoPItvgeKq4VFLoHnjoy1gUaSlleHdCGA+8DSIoVjjceXM9zXrNd1JdSO0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: pagetable_*_ctor all have the same basic implementation. Move the common part to a helper to reduce duplication. Acked-by: Dave Hansen Signed-off-by: Kevin Brodsky --- include/linux/mm.h | 28 ++++++++++++---------------- 1 file changed, 12 insertions(+), 16 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 1a11f9df5c2d..065fa9449d03 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3076,6 +3076,14 @@ static inline bool ptlock_init(struct ptdesc *ptdesc) { return true; } static inline void ptlock_free(struct ptdesc *ptdesc) {} #endif /* defined(CONFIG_SPLIT_PTE_PTLOCKS) */ +static inline void __pagetable_ctor(struct ptdesc *ptdesc) +{ + struct folio *folio = ptdesc_folio(ptdesc); + + __folio_set_pgtable(folio); + lruvec_stat_add_folio(folio, NR_PAGETABLE); +} + static inline void pagetable_dtor(struct ptdesc *ptdesc) { struct folio *folio = ptdesc_folio(ptdesc); @@ -3093,12 +3101,9 @@ static inline void pagetable_dtor_free(struct ptdesc *ptdesc) static inline bool pagetable_pte_ctor(struct ptdesc *ptdesc) { - struct folio *folio = ptdesc_folio(ptdesc); - if (!ptlock_init(ptdesc)) return false; - __folio_set_pgtable(folio); - lruvec_stat_add_folio(folio, NR_PAGETABLE); + __pagetable_ctor(ptdesc); return true; } @@ -3202,13 +3207,10 @@ static inline spinlock_t *pmd_lock(struct mm_struct *mm, pmd_t *pmd) static inline bool pagetable_pmd_ctor(struct ptdesc *ptdesc) { - struct folio *folio = ptdesc_folio(ptdesc); - if (!pmd_ptlock_init(ptdesc)) return false; - __folio_set_pgtable(folio); ptdesc_pmd_pts_init(ptdesc); - lruvec_stat_add_folio(folio, NR_PAGETABLE); + __pagetable_ctor(ptdesc); return true; } @@ -3233,18 +3235,12 @@ static inline spinlock_t *pud_lock(struct mm_struct *mm, pud_t *pud) static inline void pagetable_pud_ctor(struct ptdesc *ptdesc) { - struct folio *folio = ptdesc_folio(ptdesc); - - __folio_set_pgtable(folio); - lruvec_stat_add_folio(folio, NR_PAGETABLE); + __pagetable_ctor(ptdesc); } static inline void pagetable_p4d_ctor(struct ptdesc *ptdesc) { - struct folio *folio = ptdesc_folio(ptdesc); - - __folio_set_pgtable(folio); - lruvec_stat_add_folio(folio, NR_PAGETABLE); + __pagetable_ctor(ptdesc); } extern void __init pagecache_init(void); From patchwork Fri Jan 3 18:44:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13925791 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43627E77188 for ; Fri, 3 Jan 2025 18:44:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B10B96B008C; Fri, 3 Jan 2025 13:44:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AC17F6B0095; Fri, 3 Jan 2025 13:44:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E0A06B0096; Fri, 3 Jan 2025 13:44:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8102A6B008C for ; Fri, 3 Jan 2025 13:44:40 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 337B1160A2B for ; Fri, 3 Jan 2025 18:44:40 +0000 (UTC) X-FDA: 82967016720.08.47A6268 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf12.hostedemail.com (Postfix) with ESMTP id 9065340006 for ; Fri, 3 Jan 2025 18:44:38 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf12.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735929878; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/ZOoCqxUOeXyIAoNGnuNc/b+IIwNReB87jcTC0KeBzY=; b=YZvxIavV6amSsnI6AL3l/mB4B0EVFXg5S3wbiudI1b7Nh6IZIQr7E0PvwzXFkK9GwWTm2C GStBPjp+he21gCPuuHKuoDJAuam7En35JGpdCUS4vl/Wxxpfxua2D9zrbuAG4bZsvwyfiD cvBia0l9d0qxTPWYaECtWxnx7P+idcA= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf12.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735929878; a=rsa-sha256; cv=none; b=RklmVuvA8vxJMj9sEcFBGuNABazSeB855Fupsl875fN66yn7LsdTxJVNW0B+Q+bGnfUwSP If5UMeflSw/43WOeKHpVtFElqayoXsPVfQ4eIpOL/d7HTVtKE98+8qZFKi51YeLin+rVou pG/w8DhIc0jnG0/04cBipHBdj79M9zc= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id ED50C1C14; Fri, 3 Jan 2025 10:45:05 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B7C5C3F673; Fri, 3 Jan 2025 10:44:33 -0800 (PST) From: Kevin Brodsky To: linux-mm@kvack.org Cc: Kevin Brodsky , Andrew Morton , Catalin Marinas , Dave Hansen , Linus Walleij , Andy Lutomirski , Peter Zijlstra , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-alpha@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-kernel@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-um@lists.infradead.org, loongarch@lists.linux.dev, x86@kernel.org Subject: [PATCH v2 2/6] parisc: mm: Ensure pagetable_pmd_[cd]tor are called Date: Fri, 3 Jan 2025 18:44:11 +0000 Message-ID: <20250103184415.2744423-3-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250103184415.2744423-1-kevin.brodsky@arm.com> References: <20250103184415.2744423-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 9065340006 X-Stat-Signature: 83t3hfyy6ica618cyekd6urxakwmf78q X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1735929878-632087 X-HE-Meta: U2FsdGVkX197/nHKEgNW3CHWLfixj4qCbQbyF2mDRY8xMqbmLg2ql0ixfZCSC5eYL6HnaqwVY+InhkZynNIkd0fjDXYTHwkYEZPDgKgnOEfYjEhKulr0mD/s8/n30OaXjQsFtXJUYmL9EqeZ6pN0VTfEim66/zKihUbuoWmAx0XSpeKiuLZP7E5lWrEHY2g00udGtJfQ9wZW/h88o4sYfpT3ZsZwCPOprcgP1rflxXjVJvP5/8wmR590f012BorKz7aZJ4hSBnNDNTIGYHlO+JkFqAAu7Wr4nN0SVqT9uUf9jPMl8nhSzg359xdNSjT53ZSCYdOTCVTyYyl6Lz6mk7txHK+jvKLKQ24hV7BoJhypDgKLAClPKzHqKJZ3JT4UDR34GPGEOracHkRQNnmeVLSnbPj0KnkAUSLFn4z/wCaEtAcmWrxSXIDnfYAjPMZvz9H3zxGU5W8mpOx9/McHL0ef4JKEHhyB5Qj1+aQLNIDDkF78v+dQ2s9K4wLyMFIZA4YAkq8olN8whaZCrjyghorYQ0XktV9kd8yA2C0UW30HKtf3cxe2RvNF+ZD/S8DPO+9QS4OCDMgEcHUsfxeg0clNzeSr0lIjOZdStFiGr9+yusZbDQAWubXyVGzb/tLAEbgoVFq2SiKDP6t5/ot7wp5UFoWLWYVgW61Wejo4Um48HAGH4sF7vaZgAycSEizi1isg7Ve/E3NLZ5XlIQDG4GAd7wZUQG2Gr6B86ODyvp2bc+yhLApYsVFeEhpvjTS7+qTa8wQwapiju8XZXbQxrKraGAHv1C6juisT9dHLsetF5dAzOUy8UDSusebuqGQHzEy22j7TUY/RTZVQA3sKMF529rDdeDHQ0Rh0p/cw342sVIPQF7nop+4IsMSXSept0GIYd9aKGYwkg10J38T0vm05bVvWhJEimGEGrqrK/ny6l59K/YJov3tRIF2kX8yM8OPFBkaaetPZAqKgv2n OIMHTIjG XH+AI5/u2CfLFVHiV01P5l9kAPmugrxvjYe8N7Edaenmuo7FghCPwH/ehw/Dp6Ss9ByQqAL4hrnAHj57auSzvIhf3J5k2DPNlD7VpDj/Qzx4Ujo2sLLeMys3aTxys3r9R4pZPOH9qOSN3QersswCTKkKgoNnSUVrVa6xxiy8XAqWFv4WCkX4ejNV+7tMAvH6hHD+fkGFAXYvHUlk9iqiYxThy4YwtIebSgjcS X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The implementation of pmd_{alloc_one,free} on parisc requires a non-zero allocation order, but is completely standard aside from that. Let's reuse the generic implementation of pmd_alloc_one(). Explicit zeroing is not needed as GFP_PGTABLE_KERNEL includes __GFP_ZERO. The generic pmd_free() can handle higher allocation orders so we don't need to define our own. These changes ensure that pagetable_pmd_[cd]tor are called, improving the accounting of page table pages. Acked-by: Dave Hansen Signed-off-by: Kevin Brodsky --- arch/parisc/include/asm/pgalloc.h | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/arch/parisc/include/asm/pgalloc.h b/arch/parisc/include/asm/pgalloc.h index e3e142b1c5c5..3e8dbd79670b 100644 --- a/arch/parisc/include/asm/pgalloc.h +++ b/arch/parisc/include/asm/pgalloc.h @@ -11,7 +11,6 @@ #include #define __HAVE_ARCH_PMD_ALLOC_ONE -#define __HAVE_ARCH_PMD_FREE #define __HAVE_ARCH_PGD_FREE #include @@ -46,17 +45,19 @@ static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd) static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address) { - pmd_t *pmd; + struct ptdesc *ptdesc; + gfp_t gfp = GFP_PGTABLE_USER; - pmd = (pmd_t *)__get_free_pages(GFP_PGTABLE_KERNEL, PMD_TABLE_ORDER); - if (likely(pmd)) - memset ((void *)pmd, 0, PAGE_SIZE << PMD_TABLE_ORDER); - return pmd; -} - -static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd) -{ - free_pages((unsigned long)pmd, PMD_TABLE_ORDER); + if (mm == &init_mm) + gfp = GFP_PGTABLE_KERNEL; + ptdesc = pagetable_alloc(gfp, PMD_TABLE_ORDER); + if (!ptdesc) + return NULL; + if (!pagetable_pmd_ctor(ptdesc)) { + pagetable_free(ptdesc); + return NULL; + } + return ptdesc_address(ptdesc); } #endif From patchwork Fri Jan 3 18:44:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13925792 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92AE3E77188 for ; Fri, 3 Jan 2025 18:44:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2367A6B0096; Fri, 3 Jan 2025 13:44:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 20D606B0098; Fri, 3 Jan 2025 13:44:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0AEC26B0099; Fri, 3 Jan 2025 13:44:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id E33E76B0096 for ; Fri, 3 Jan 2025 13:44:44 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 8EC551209F0 for ; Fri, 3 Jan 2025 18:44:44 +0000 (UTC) X-FDA: 82967016888.10.5F82E9F Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf01.hostedemail.com (Postfix) with ESMTP id DFB1540006 for ; Fri, 3 Jan 2025 18:44:42 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735929883; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2At99XcZDFcXlkIf+qv97R+a7TzNSQFTbGSOrVu3ygw=; b=DgY7jPWjAPOapXuK5LWqVdEqw+iGti48dJvfrC4nwkg9QT6RnYq7JV4cdLjpPcEHOsOaYF ybOT7/7WFkJjPMyeVQgvBl5vqjvbeHpqjLyakJ93aspPy8RlyOEzN7d0J+itVDn29qJFGT 4Z7CkWZGwD57B+V0MYWXJC0DICVh2Lo= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735929883; a=rsa-sha256; cv=none; b=EyMC0VdX/ipC668JcFhLa+QFPGBGh5CZFhRujJAXCHweylWkuf2f3zGGMwXaVvL9Y55JcF UXFLED4ceo633YPe7h/neg1zFcwVb0JHaUbsS744cLRR/ie/zBODM2wyj8tvUBuJzaAn+q 3pmlvRK0VlY/AxR450vQJ2//mM5jhiE= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 47F661C25; Fri, 3 Jan 2025 10:45:10 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0FCC13F673; Fri, 3 Jan 2025 10:44:37 -0800 (PST) From: Kevin Brodsky To: linux-mm@kvack.org Cc: Kevin Brodsky , Andrew Morton , Catalin Marinas , Dave Hansen , Linus Walleij , Andy Lutomirski , Peter Zijlstra , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-alpha@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-kernel@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-um@lists.infradead.org, loongarch@lists.linux.dev, x86@kernel.org Subject: [PATCH v2 3/6] m68k: mm: Add calls to pagetable_pmd_[cd]tor Date: Fri, 3 Jan 2025 18:44:12 +0000 Message-ID: <20250103184415.2744423-4-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250103184415.2744423-1-kevin.brodsky@arm.com> References: <20250103184415.2744423-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Stat-Signature: t54f36por3oxb15xfny4wtco6jb3gykd X-Rspamd-Queue-Id: DFB1540006 X-Rspam-User: X-HE-Tag: 1735929882-478457 X-HE-Meta: U2FsdGVkX1/4TvCi2FqJMj3NBc1EVl15l6yB6BFCRYuTrPGhLOYeIzSJ/FPsMT1TfX633XdLurKWzb1kmtypVZL6Uw6PRB+fEud30Aw3VNuMomHL7srDvl8PjH2JqLweunPFSsS74vq+qW8gsqdBrCpCbwFFFMgOiaJEArtsn4LJOEZmTH1IVsMR7auFBFyZsY9kDANZncbkHyldB8ABWc4ZFMbOFvpvB3DMeGpqcprHE/VQvKsnInK0NIr3wXW+/qupeuc1UzX+1uDx8qGyPdp5R60o23FMq+RIU6ap3jz0OH62MqSukAPKYm//Bcp3HzL90EHT++5HoCGWsPQ1Ptf/5ajp8Umqjk0OoUsuvBmdUpNw3Dk/wPJR+5tanK/7iMYfNDawBwg8EQQHrsKChDhWNcF08aJK8H0NeaTV3DNOaGqA5yT7k/ETKPWP1dkZn7z3RFkHzA9LQeSvWZRq6IY0pO87oK6PqnCvxogXTktuRdrHu9UrQ54j6TL+kxFjXNUeW+KmV3mbQlz5PTqMRdCMQ/VwEdLUjKKyZV9f6HHq9ogYPo2Ul6WfYWmibPOIPz00TR/F+DAib/MIL10NjcQZJTZlSkgFvQyaFSrBqNHcSO0Itw+JLUJ+vIMgSaaYI8H5JP10HB0Y72vyBtUitG1B8ZqeCYcD9azkOvuUlDJQMuwrgPRL+toJzlMLbDhDMFF7IqU7T2+oOzdZqff8jk1j/9Td1e/abCMOittHOXZWpfdHjXBUIBlXzcWEqOJk5ZL+7T7RTrc6R1NfvUpZSSrVZ+8IMLZbatfkgR7HLQdFMNeuO6n82VSAKvfa3quE1g9ixRCp+eoTNETQmuR7aXmxTv9xQF84ER9nNYKACwx5ek/bqjzpTI9I/QpCv7Hy9p92HmIuojqFYpiyyPjwelR/mDDrzQRfmAGHqTvowcTMtgxtubC09Q574nOMiW+k22769phBlyLRSypO2oH ycmEQRcD GcxiuJS0cQL4eanIU8JNzr3P0sEYxJArGPjaIDfstctmyqPvwxwClPedlFnjHg9mRhBaI0lwo/5K0KgWexvxQ8zU4o760pxbCLpKU9JlGGj/tLA/v4E+F7IK2yPGqWoLocy7Cy0x4Q4JTiH/gq2HTG7OZeC8Ezko1R7+mHr3I5PUh3yfKBPdR8fql29iRnrXyd+JnFShagA3DCZLBVt3MpbBVDi6598LmSyAg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: get_pointer_table() and free_pointer_table() already special-case TABLE_PTE to call pagetable_pte_[cd]tor. Let's do the same at PMD level to improve accounting further. TABLE_PGD and TABLE_PMD are currently defined to the same value, so we first need to separate them. That also implies separating ptable_list for PMD/PGD levels. Acked-by: Dave Hansen Signed-off-by: Kevin Brodsky --- arch/m68k/include/asm/motorola_pgalloc.h | 6 +++--- arch/m68k/mm/motorola.c | 17 ++++++++++++----- 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/arch/m68k/include/asm/motorola_pgalloc.h b/arch/m68k/include/asm/motorola_pgalloc.h index 74a817d9387f..5abe7da8ac5a 100644 --- a/arch/m68k/include/asm/motorola_pgalloc.h +++ b/arch/m68k/include/asm/motorola_pgalloc.h @@ -9,9 +9,9 @@ extern void mmu_page_ctor(void *page); extern void mmu_page_dtor(void *page); enum m68k_table_types { - TABLE_PGD = 0, - TABLE_PMD = 0, /* same size as PGD */ - TABLE_PTE = 1, + TABLE_PGD, + TABLE_PMD, + TABLE_PTE, }; extern void init_pointer_table(void *table, int type); diff --git a/arch/m68k/mm/motorola.c b/arch/m68k/mm/motorola.c index eab50dda14ee..6c09ccb72e8b 100644 --- a/arch/m68k/mm/motorola.c +++ b/arch/m68k/mm/motorola.c @@ -97,17 +97,19 @@ void mmu_page_dtor(void *page) typedef struct list_head ptable_desc; -static struct list_head ptable_list[2] = { +static struct list_head ptable_list[3] = { LIST_HEAD_INIT(ptable_list[0]), LIST_HEAD_INIT(ptable_list[1]), + LIST_HEAD_INIT(ptable_list[2]), }; #define PD_PTABLE(page) ((ptable_desc *)&(virt_to_page((void *)(page))->lru)) #define PD_PAGE(ptable) (list_entry(ptable, struct page, lru)) #define PD_MARKBITS(dp) (*(unsigned int *)&PD_PAGE(dp)->index) -static const int ptable_shift[2] = { - 7+2, /* PGD, PMD */ +static const int ptable_shift[3] = { + 7+2, /* PGD */ + 7+2, /* PMD */ 6+2, /* PTE */ }; @@ -156,12 +158,17 @@ void *get_pointer_table(int type) if (!(page = (void *)get_zeroed_page(GFP_KERNEL))) return NULL; - if (type == TABLE_PTE) { + switch (type) { + case TABLE_PTE: /* * m68k doesn't have SPLIT_PTE_PTLOCKS for not having * SMP. */ pagetable_pte_ctor(virt_to_ptdesc(page)); + break; + case TABLE_PMD: + pagetable_pmd_ctor(virt_to_ptdesc(page)); + break; } mmu_page_ctor(page); @@ -200,7 +207,7 @@ int free_pointer_table(void *table, int type) /* all tables in page are free, free page */ list_del(dp); mmu_page_dtor((void *)page); - if (type == TABLE_PTE) + if (type == TABLE_PTE || type == TABLE_PMD) pagetable_dtor(virt_to_ptdesc((void *)page)); free_page (page); return 1; From patchwork Fri Jan 3 18:44:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13925793 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6E52E7718F for ; Fri, 3 Jan 2025 18:44:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5DFCE6B0098; Fri, 3 Jan 2025 13:44:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 58FD36B0099; Fri, 3 Jan 2025 13:44:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 457FF6B009B; Fri, 3 Jan 2025 13:44:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 28C706B0098 for ; Fri, 3 Jan 2025 13:44:49 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id A7BAFC0A55 for ; Fri, 3 Jan 2025 18:44:48 +0000 (UTC) X-FDA: 82967017056.07.E9FDB61 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf04.hostedemail.com (Postfix) with ESMTP id 2447440012 for ; Fri, 3 Jan 2025 18:44:46 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf04.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735929887; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UgvT7pAeV3IIDBflFGuCxx0F4tXh8w8e+aOL3RS3IYM=; b=Ek8JrOzIzBEqGNHi3RyKW2t5Pd2O0rYiy8EdrH6//ABRYhBHy+238Bieahiwu4ZNbhoohk Q+NMvnT1CMHNfJbSBWxJbgp04WRjn/QyY8Djf5CaCoGL3nlI5jBJBrNOXBlN12u4V0wa23 zMMT/c1MBRj83bo2gHvHg6hpU58/JT0= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf04.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735929887; a=rsa-sha256; cv=none; b=70w62wJMNEhks82eBl6onNT1nVlRpM04YCZW7qoc4sqYCXljwgmg5EFMMQS79EoWm6LppC Xr0CL54RBQXuVpMFE6SPXs2dYOjBwPmR0nbRAmN0kLfM6DvFkq+21DRhI9LI3SRUw8PJDv RLCqKiKI2juCYI55i2tQbfqB41NWLVI= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 92BE9150C; Fri, 3 Jan 2025 10:45:14 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5CBB03F673; Fri, 3 Jan 2025 10:44:42 -0800 (PST) From: Kevin Brodsky To: linux-mm@kvack.org Cc: Kevin Brodsky , Andrew Morton , Catalin Marinas , Dave Hansen , Linus Walleij , Andy Lutomirski , Peter Zijlstra , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-alpha@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-kernel@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-um@lists.infradead.org, loongarch@lists.linux.dev, x86@kernel.org Subject: [PATCH v2 4/6] ARM: mm: Rename PGD helpers Date: Fri, 3 Jan 2025 18:44:13 +0000 Message-ID: <20250103184415.2744423-5-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250103184415.2744423-1-kevin.brodsky@arm.com> References: <20250103184415.2744423-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 2447440012 X-Stat-Signature: x57esuxw5ztuz15dqqm4eh3rqaziy6ca X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1735929886-290539 X-HE-Meta: U2FsdGVkX19EYD4lW/2ROe6ImHU4ac/FNigaHX7AYwFsQ5CWxxdyTU2JKe4Bhco9RmLmll3O4fP2wM+VTTnzk5SGCy9DqxXHPKopyl5u+FNtlMJcjv1pzIxD/PMoAOkzciXWLoKfF+QAZ5+KTSwVQ7KYGz1YZdrZxAODNlpqjXR2N31egw3TCX+HC1OVtWKA0e091Zkeafgm/06Mt0GwfzraWSvDZ8+1U+MFd7rD8bp+FpIqrGjz7CLsVdoZAbHcjns4uyc4XHVHurMXclkJbzEMpisqsYXplVQsuntRx8BrT9cDT+iiRGQGWNIDw7Y1tqtJR5Vjti/xKF9fka6OK46lT90FpXc0T2RM6GGz7NZsiR7FujpQs1BdrFaRlXGwjrIu02leA5AmfV+vUHV30RZif9O9jNV1CmLtSyQZvoOnVs20KWWGaPlUcGpG3Wq91f34FTNMvV9VHgSVyjNP3n6eu7s5CVeyWekZpqzG6Vgq3y59fDKvgNBVhFx3yvoJzDgkLXnJBb+z8FY2UkEdGtbwwUzTGqMquPWNQS0mSCaiOnF+dK1ZTKpQXVceEILZkavmUajx1Y+5Ggg1JXrdGuAqqnZ5fQ4vWwoqY5J8B9Z/5gyrkd6W381hDU/JtkuEgOry7k7I+xcpSxJuCqcYJXCsrPk/QtV3A7BKiQtLYmF/SmynsheH/2NdPsoBRKmRdHCCJ6eRGdZ5FtDtc4uW8xoZDwInHuJyUCd8suVA/QAY2LfGo//S57P8ytNivF032cMA+aID8QyMk23N4ycE9xOl0r5yhpVeJZDLkgXeuzXQTVvz/Y2MftWzGSVQ/jPluUx29CvI38ZhDkaVYV7z63TfucRGTyGTkSxmUYLC5CpKQoJLzByllURdQA0d8Yfb9PlcxczQ8f+FmOMnbXXLYXhHX6a9mH1zqk4L8zf/4fqcMcLsKvcIWMcgHgHkCsOk/spdSeisbfnWMXX7kO8 ffhfjr7f 3mEb8ZvWO1ulWhCIvLig3/E+1omy49YqUZ7bzaatDWAdKewqk9mgUlCsRlu6MOrLj69JmAFLUg1AX0d/+PJvhJ2n1aFMANe18tu+zQ2Iqxim0o7yQM3LWoe92INYoZe2Ld9lNhdk5d7MsKggHe7UN84+wvTU+I+zVr2FTQGnoADwwc2r6mrZ8BDcMqQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Generic implementations of __pgd_alloc and __pgd_free are about to be introduced. Rename the macros in arch/arm/mm/pgd.c to avoid clashes. While we're at it, also pass down the mm as argument to those helpers, as it will be needed to call the generic __pgd_{alloc,free}. Acked-by: Dave Hansen Signed-off-by: Kevin Brodsky --- arch/arm/mm/pgd.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/arch/arm/mm/pgd.c b/arch/arm/mm/pgd.c index f8e9bc58a84f..2a1077747758 100644 --- a/arch/arm/mm/pgd.c +++ b/arch/arm/mm/pgd.c @@ -17,11 +17,11 @@ #include "mm.h" #ifdef CONFIG_ARM_LPAE -#define __pgd_alloc() kmalloc_array(PTRS_PER_PGD, sizeof(pgd_t), GFP_KERNEL) -#define __pgd_free(pgd) kfree(pgd) +#define _pgd_alloc(mm) kmalloc_array(PTRS_PER_PGD, sizeof(pgd_t), GFP_KERNEL) +#define _pgd_free(mm, pgd) kfree(pgd) #else -#define __pgd_alloc() (pgd_t *)__get_free_pages(GFP_KERNEL, 2) -#define __pgd_free(pgd) free_pages((unsigned long)pgd, 2) +#define _pgd_alloc(mm) (pgd_t *)__get_free_pages(GFP_KERNEL, 2) +#define _pgd_free(mm, pgd) free_pages((unsigned long)pgd, 2) #endif /* @@ -35,7 +35,7 @@ pgd_t *pgd_alloc(struct mm_struct *mm) pmd_t *new_pmd, *init_pmd; pte_t *new_pte, *init_pte; - new_pgd = __pgd_alloc(); + new_pgd = _pgd_alloc(mm); if (!new_pgd) goto no_pgd; @@ -134,7 +134,7 @@ pgd_t *pgd_alloc(struct mm_struct *mm) no_pud: p4d_free(mm, new_p4d); no_p4d: - __pgd_free(new_pgd); + _pgd_free(mm, new_pgd); no_pgd: return NULL; } @@ -207,5 +207,5 @@ void pgd_free(struct mm_struct *mm, pgd_t *pgd_base) p4d_free(mm, p4d); } #endif - __pgd_free(pgd_base); + _pgd_free(mm, pgd_base); } From patchwork Fri Jan 3 18:44:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13925794 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 469DAE7718F for ; Fri, 3 Jan 2025 18:44:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D332D6B0099; Fri, 3 Jan 2025 13:44:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CE3656B009B; Fri, 3 Jan 2025 13:44:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B83F96B009D; Fri, 3 Jan 2025 13:44:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 964E36B0099 for ; Fri, 3 Jan 2025 13:44:53 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 1E87EC0A37 for ; Fri, 3 Jan 2025 18:44:53 +0000 (UTC) X-FDA: 82967017266.10.4491EC2 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf13.hostedemail.com (Postfix) with ESMTP id 7979420002 for ; Fri, 3 Jan 2025 18:44:51 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf13.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735929891; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3Kb4bi7Q+lzjbTMQstNvMPI2lg8VWLpLeXhMyxFtfvs=; b=xpXSFJ9V5oE9uhRD5xKNk3xhJj8n+ZeQGUWl4DoxxuWIRWXrritm9c6frLDacLob68B8UG hXt4oko3bo9PBREuaLz2lpAWPzZlSR3otLwOonMjGSsxm9ECfLZ6NnwKE2vf7IyIxSN+GZ 1xmQPLpd+lw3TakXwrIv2Rvuz9r/MVY= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf13.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735929891; a=rsa-sha256; cv=none; b=5ZXDLkxS2o8sIskAvyAoGrzrRjzm2ipJhAhETa3Fc9e+gVacjkRAzdKafT71yokyqe76vg HfzRjx/FsU5w84KkAJLK6kgSQWWCUQYc0AwQa4KnkN3e9qcVXI0H+wPj/yQ0c1zS/cBj+X 24wkEl4nzl8CcVdZuMKf/DECdIaRo4g= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0B18B1480; Fri, 3 Jan 2025 10:45:19 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A9F483F673; Fri, 3 Jan 2025 10:44:46 -0800 (PST) From: Kevin Brodsky To: linux-mm@kvack.org Cc: Kevin Brodsky , Andrew Morton , Catalin Marinas , Dave Hansen , Linus Walleij , Andy Lutomirski , Peter Zijlstra , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-alpha@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-kernel@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-um@lists.infradead.org, loongarch@lists.linux.dev, x86@kernel.org Subject: [PATCH v2 5/6] asm-generic: pgalloc: Provide generic __pgd_{alloc,free} Date: Fri, 3 Jan 2025 18:44:14 +0000 Message-ID: <20250103184415.2744423-6-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250103184415.2744423-1-kevin.brodsky@arm.com> References: <20250103184415.2744423-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 7979420002 X-Stat-Signature: xp83toyawihk58ctn3gyrxy7zjnyj4eu X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1735929891-118730 X-HE-Meta: U2FsdGVkX18DyBI5PUzRyA+bLvwxKiUQ/8ZKClGqwQv+bZ3Y/yFlNy4uOnSFeKuCec7RujI0QkBfK4yV4MtSt10dNUzf5LfoXpz9/L+j8IcOFD7TY+yw+G56rZpnzOQlcScInnJEK3G1dZmwG037+n026wS7o7pECvo9+91R26jJrEWxRNAkLjDTD5NwlK0zTciABSTnMOKDhI34o6xNyYhHDu+f/reUvd/RQnKrVJCzmrG0zyMSjg8rOAzqLVhy5e/t4nXK/aUOMwoTXqBazXa1sPR3sSoLuynaDzRgXWa1wObIafXSjmA8N93SzYlGRir9cx8JOwy+8sEmu81zaHyhk8VE3M9TvONyCS7U3AShQtjYjP56y1AfR1Wtpyo5NjPnF/2bqU2dDbECHJOq5hiIbiTH7cl1ATfRfqsDD/lbtcgCX2TdHVA6B45ntH4xPkWkiGeFDYu03FL/7CK5FE0UGfyUgkRnp3mmfdmA4sijgf1ZIvVbOi5n2AWIa2wltz+k9Dr6W9DXEU/B/dFTMf+YmDUvbXzl2ZiYcw1Px/y1jPbH1G65tRE5v9brG80vFPyAxqrDBhCye7NF/sDSMNrt6x9cXfyrRf91v+b2ETWfkJwvVWU4kRlq1hIMTiT8ChjoQ5shd25ExWtwBWKh469aoNtutRwwFoXrm20Zt+ac+j60mx/C7ZYMXmkXRP/XNmV7VDZtbaAduNKsH5SpnxH1JFmQfPPBmicZYE+DDIwpu1gWpPV8tuh0OKuQi83jHNXQRfcc9iVcZJ/uqodjZHRtkHOgmfLE3fCcWS7SoSNSippAgeQF0Sl/hKVYaSUKJ80Qgv5Piak9o2JcgUK+7r42/iP+e/2HOa9l3HQRKg4AebAPxqlZW9hGInjgEy72Yxu9GPJKY+n70R6SOnA0U7rGE/Mn5Y7PxGxP+WBGJZjhOQHL3GN62zAzrE8uPE8R5i1MiZVRLq00jYlkWEx mFkIyFAT 4ZYt/bCLh88jjT9W5JLx5DlRCtLHx7J1vsm1vzt4xsKHGWAbx/3p3y9Wf75rjdIaDYkyZ0tTnC7EyWxSBsGd2l3gtXdZUwxn03DGCW1E7vp3YkXmFB8wJ8dBbAWoSE75n/Zjr5sYTv6kP1TcNhsM3a28CgXsI+LVXlzwPQM3Oul/5u47dyrADZfmc4AaKCane7MGWc9OE0z5xlkXQKVe88evx8LhrNIy5jMFp X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We already have a generic implementation of alloc/free up to P4D level, as well as pgd_free(). Let's finish the work and add a generic PGD-level alloc helper as well. Unlike at lower levels, almost all architectures need some specific magic at PGD level (typically initialising PGD entries), so introducing a generic pgd_alloc() isn't worth it. Instead we introduce two new helpers, __pgd_alloc() and __pgd_free(), and make use of them in the arch-specific pgd_alloc() and pgd_free() wherever possible. To accommodate as many arch as possible, __pgd_alloc() takes a page allocation order. Because pagetable_alloc() allocates zeroed pages, explicit zeroing in pgd_alloc() becomes redundant and we can get rid of it. Some trivial implementations of pgd_free() also become unnecessary once __pgd_alloc() is used; remove them. Another small improvement is consistent accounting of PGD pages by using GFP_PGTABLE_{USER,KERNEL} as appropriate. Not all PGD allocations can be handled by the generic helpers. In particular, multiple architectures allocate PGDs from a kmem_cache, and those PGDs may not be page-sized. Acked-by: Dave Hansen Signed-off-by: Kevin Brodsky --- arch/alpha/mm/init.c | 2 +- arch/arc/include/asm/pgalloc.h | 9 ++------- arch/arm/mm/pgd.c | 8 +++----- arch/arm64/mm/pgd.c | 4 ++-- arch/csky/include/asm/pgalloc.h | 2 +- arch/hexagon/include/asm/pgalloc.h | 2 +- arch/loongarch/mm/pgtable.c | 7 +++---- arch/m68k/include/asm/sun3_pgalloc.h | 2 +- arch/microblaze/include/asm/pgalloc.h | 7 +------ arch/mips/include/asm/pgalloc.h | 6 ------ arch/mips/mm/pgtable.c | 8 +++----- arch/nios2/mm/pgtable.c | 3 ++- arch/openrisc/include/asm/pgalloc.h | 6 ++---- arch/parisc/include/asm/pgalloc.h | 16 +--------------- arch/riscv/include/asm/pgalloc.h | 3 +-- arch/um/kernel/mem.c | 7 +++---- arch/x86/mm/pgtable.c | 24 +++++++++++------------- arch/xtensa/include/asm/pgalloc.h | 2 +- include/asm-generic/pgalloc.h | 27 ++++++++++++++++++++++++++- 19 files changed, 65 insertions(+), 80 deletions(-) diff --git a/arch/alpha/mm/init.c b/arch/alpha/mm/init.c index 4fe618446e4c..61c2198b1359 100644 --- a/arch/alpha/mm/init.c +++ b/arch/alpha/mm/init.c @@ -42,7 +42,7 @@ pgd_alloc(struct mm_struct *mm) { pgd_t *ret, *init; - ret = (pgd_t *)__get_free_page(GFP_KERNEL | __GFP_ZERO); + ret = __pgd_alloc(mm, 0); init = pgd_offset(&init_mm, 0UL); if (ret) { #ifdef CONFIG_ALPHA_LARGE_VMALLOC diff --git a/arch/arc/include/asm/pgalloc.h b/arch/arc/include/asm/pgalloc.h index 096b8ef58edb..dfae070fe8d5 100644 --- a/arch/arc/include/asm/pgalloc.h +++ b/arch/arc/include/asm/pgalloc.h @@ -53,19 +53,14 @@ static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd, pgtable_t pte_ static inline pgd_t *pgd_alloc(struct mm_struct *mm) { - pgd_t *ret = (pgd_t *) __get_free_page(GFP_KERNEL); + pgd_t *ret = __pgd_alloc(mm, 0); if (ret) { int num, num2; - num = USER_PTRS_PER_PGD + USER_KERNEL_GUTTER / PGDIR_SIZE; - memzero(ret, num * sizeof(pgd_t)); + num = USER_PTRS_PER_PGD + USER_KERNEL_GUTTER / PGDIR_SIZE; num2 = VMALLOC_SIZE / PGDIR_SIZE; memcpy(ret + num, swapper_pg_dir + num, num2 * sizeof(pgd_t)); - - memzero(ret + num + num2, - (PTRS_PER_PGD - num - num2) * sizeof(pgd_t)); - } return ret; } diff --git a/arch/arm/mm/pgd.c b/arch/arm/mm/pgd.c index 2a1077747758..4eb81b7ed03a 100644 --- a/arch/arm/mm/pgd.c +++ b/arch/arm/mm/pgd.c @@ -17,11 +17,11 @@ #include "mm.h" #ifdef CONFIG_ARM_LPAE -#define _pgd_alloc(mm) kmalloc_array(PTRS_PER_PGD, sizeof(pgd_t), GFP_KERNEL) +#define _pgd_alloc(mm) kmalloc_array(PTRS_PER_PGD, sizeof(pgd_t), GFP_KERNEL | __GFP_ZERO) #define _pgd_free(mm, pgd) kfree(pgd) #else -#define _pgd_alloc(mm) (pgd_t *)__get_free_pages(GFP_KERNEL, 2) -#define _pgd_free(mm, pgd) free_pages((unsigned long)pgd, 2) +#define _pgd_alloc(mm) __pgd_alloc(mm, 2) +#define _pgd_free(mm, pgd) __pgd_free(mm, pgd) #endif /* @@ -39,8 +39,6 @@ pgd_t *pgd_alloc(struct mm_struct *mm) if (!new_pgd) goto no_pgd; - memset(new_pgd, 0, USER_PTRS_PER_PGD * sizeof(pgd_t)); - /* * Copy over the kernel and IO PGD entries */ diff --git a/arch/arm64/mm/pgd.c b/arch/arm64/mm/pgd.c index 0c501cabc238..8160cff35089 100644 --- a/arch/arm64/mm/pgd.c +++ b/arch/arm64/mm/pgd.c @@ -33,7 +33,7 @@ pgd_t *pgd_alloc(struct mm_struct *mm) gfp_t gfp = GFP_PGTABLE_USER; if (pgdir_is_page_size()) - return (pgd_t *)__get_free_page(gfp); + return __pgd_alloc(mm, 0); else return kmem_cache_alloc(pgd_cache, gfp); } @@ -41,7 +41,7 @@ pgd_t *pgd_alloc(struct mm_struct *mm) void pgd_free(struct mm_struct *mm, pgd_t *pgd) { if (pgdir_is_page_size()) - free_page((unsigned long)pgd); + __pgd_free(mm, pgd); else kmem_cache_free(pgd_cache, pgd); } diff --git a/arch/csky/include/asm/pgalloc.h b/arch/csky/include/asm/pgalloc.h index f1ce5b7b28f2..bf8400c28b5a 100644 --- a/arch/csky/include/asm/pgalloc.h +++ b/arch/csky/include/asm/pgalloc.h @@ -44,7 +44,7 @@ static inline pgd_t *pgd_alloc(struct mm_struct *mm) pgd_t *ret; pgd_t *init; - ret = (pgd_t *) __get_free_page(GFP_KERNEL); + ret = __pgd_alloc(mm, 0); if (ret) { init = pgd_offset(&init_mm, 0UL); pgd_init((unsigned long *)ret); diff --git a/arch/hexagon/include/asm/pgalloc.h b/arch/hexagon/include/asm/pgalloc.h index 40e42a0e7167..1ee5f5f157ca 100644 --- a/arch/hexagon/include/asm/pgalloc.h +++ b/arch/hexagon/include/asm/pgalloc.h @@ -22,7 +22,7 @@ static inline pgd_t *pgd_alloc(struct mm_struct *mm) { pgd_t *pgd; - pgd = (pgd_t *)__get_free_page(GFP_KERNEL | __GFP_ZERO); + pgd = __pgd_alloc(mm, 0); /* * There may be better ways to do this, but to ensure diff --git a/arch/loongarch/mm/pgtable.c b/arch/loongarch/mm/pgtable.c index 3fa69b23ff84..22a94bb3e6e8 100644 --- a/arch/loongarch/mm/pgtable.c +++ b/arch/loongarch/mm/pgtable.c @@ -23,11 +23,10 @@ EXPORT_SYMBOL(tlb_virt_to_page); pgd_t *pgd_alloc(struct mm_struct *mm) { - pgd_t *init, *ret = NULL; - struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL & ~__GFP_HIGHMEM, 0); + pgd_t *init, *ret; - if (ptdesc) { - ret = (pgd_t *)ptdesc_address(ptdesc); + ret = __pgd_alloc(mm, 0); + if (ret) { init = pgd_offset(&init_mm, 0UL); pgd_init(ret); memcpy(ret + USER_PTRS_PER_PGD, init + USER_PTRS_PER_PGD, diff --git a/arch/m68k/include/asm/sun3_pgalloc.h b/arch/m68k/include/asm/sun3_pgalloc.h index 2b626cb3ad0a..f1ae4ed890db 100644 --- a/arch/m68k/include/asm/sun3_pgalloc.h +++ b/arch/m68k/include/asm/sun3_pgalloc.h @@ -43,7 +43,7 @@ static inline pgd_t * pgd_alloc(struct mm_struct *mm) { pgd_t *new_pgd; - new_pgd = (pgd_t *)get_zeroed_page(GFP_KERNEL); + new_pgd = __pgd_alloc(mm, 0); memcpy(new_pgd, swapper_pg_dir, PAGE_SIZE); memset(new_pgd, 0, (PAGE_OFFSET >> PGDIR_SHIFT)); return new_pgd; diff --git a/arch/microblaze/include/asm/pgalloc.h b/arch/microblaze/include/asm/pgalloc.h index 6c33b05f730f..084a8a0dc239 100644 --- a/arch/microblaze/include/asm/pgalloc.h +++ b/arch/microblaze/include/asm/pgalloc.h @@ -21,12 +21,7 @@ extern void __bad_pte(pmd_t *pmd); -static inline pgd_t *get_pgd(void) -{ - return (pgd_t *)__get_free_pages(GFP_KERNEL|__GFP_ZERO, 0); -} - -#define pgd_alloc(mm) get_pgd() +#define pgd_alloc(mm) __pgd_alloc(mm, 0) extern pte_t *pte_alloc_one_kernel(struct mm_struct *mm); diff --git a/arch/mips/include/asm/pgalloc.h b/arch/mips/include/asm/pgalloc.h index 36d9805033c4..26c7a6ede983 100644 --- a/arch/mips/include/asm/pgalloc.h +++ b/arch/mips/include/asm/pgalloc.h @@ -15,7 +15,6 @@ #define __HAVE_ARCH_PMD_ALLOC_ONE #define __HAVE_ARCH_PUD_ALLOC_ONE -#define __HAVE_ARCH_PGD_FREE #include static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, @@ -49,11 +48,6 @@ static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd) extern void pgd_init(void *addr); extern pgd_t *pgd_alloc(struct mm_struct *mm); -static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd) -{ - pagetable_free(virt_to_ptdesc(pgd)); -} - #define __pte_free_tlb(tlb, pte, address) \ do { \ pagetable_dtor(page_ptdesc(pte)); \ diff --git a/arch/mips/mm/pgtable.c b/arch/mips/mm/pgtable.c index 1506e458040d..10835414819f 100644 --- a/arch/mips/mm/pgtable.c +++ b/arch/mips/mm/pgtable.c @@ -10,12 +10,10 @@ pgd_t *pgd_alloc(struct mm_struct *mm) { - pgd_t *init, *ret = NULL; - struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL & ~__GFP_HIGHMEM, - PGD_TABLE_ORDER); + pgd_t *init, *ret; - if (ptdesc) { - ret = ptdesc_address(ptdesc); + ret = __pgd_alloc(mm, PGD_TABLE_ORDER); + if (ret) { init = pgd_offset(&init_mm, 0UL); pgd_init(ret); memcpy(ret + USER_PTRS_PER_PGD, init + USER_PTRS_PER_PGD, diff --git a/arch/nios2/mm/pgtable.c b/arch/nios2/mm/pgtable.c index 7c76e8a7447a..6470ed378782 100644 --- a/arch/nios2/mm/pgtable.c +++ b/arch/nios2/mm/pgtable.c @@ -11,6 +11,7 @@ #include #include +#include /* pteaddr: * ptbase | vpn* | zero @@ -54,7 +55,7 @@ pgd_t *pgd_alloc(struct mm_struct *mm) { pgd_t *ret, *init; - ret = (pgd_t *) __get_free_page(GFP_KERNEL); + ret = __pgd_alloc(mm, 0); if (ret) { init = pgd_offset(&init_mm, 0UL); pgd_init(ret); diff --git a/arch/openrisc/include/asm/pgalloc.h b/arch/openrisc/include/asm/pgalloc.h index 596e2355824e..3372f4e6ab4b 100644 --- a/arch/openrisc/include/asm/pgalloc.h +++ b/arch/openrisc/include/asm/pgalloc.h @@ -41,15 +41,13 @@ static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd, */ static inline pgd_t *pgd_alloc(struct mm_struct *mm) { - pgd_t *ret = (pgd_t *)__get_free_page(GFP_KERNEL); + pgd_t *ret = __pgd_alloc(mm, 0); - if (ret) { - memset(ret, 0, USER_PTRS_PER_PGD * sizeof(pgd_t)); + if (ret) memcpy(ret + USER_PTRS_PER_PGD, swapper_pg_dir + USER_PTRS_PER_PGD, (PTRS_PER_PGD - USER_PTRS_PER_PGD) * sizeof(pgd_t)); - } return ret; } diff --git a/arch/parisc/include/asm/pgalloc.h b/arch/parisc/include/asm/pgalloc.h index 3e8dbd79670b..2ca74a56415c 100644 --- a/arch/parisc/include/asm/pgalloc.h +++ b/arch/parisc/include/asm/pgalloc.h @@ -11,26 +11,12 @@ #include #define __HAVE_ARCH_PMD_ALLOC_ONE -#define __HAVE_ARCH_PGD_FREE #include /* Allocate the top level pgd (page directory) */ static inline pgd_t *pgd_alloc(struct mm_struct *mm) { - pgd_t *pgd; - - pgd = (pgd_t *) __get_free_pages(GFP_KERNEL, PGD_TABLE_ORDER); - if (unlikely(pgd == NULL)) - return NULL; - - memset(pgd, 0, PAGE_SIZE << PGD_TABLE_ORDER); - - return pgd; -} - -static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd) -{ - free_pages((unsigned long)pgd, PGD_TABLE_ORDER); + return __pgd_alloc(mm, PGD_TABLE_ORDER); } #if CONFIG_PGTABLE_LEVELS == 3 diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h index c8907b831711..3e2aebea6312 100644 --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -130,9 +130,8 @@ static inline pgd_t *pgd_alloc(struct mm_struct *mm) { pgd_t *pgd; - pgd = (pgd_t *)__get_free_page(GFP_KERNEL); + pgd = __pgd_alloc(mm, 0); if (likely(pgd != NULL)) { - memset(pgd, 0, USER_PTRS_PER_PGD * sizeof(pgd_t)); /* Copy kernel mappings */ sync_kernel_mappings(pgd); } diff --git a/arch/um/kernel/mem.c b/arch/um/kernel/mem.c index 53248ed04771..d98812907493 100644 --- a/arch/um/kernel/mem.c +++ b/arch/um/kernel/mem.c @@ -214,14 +214,13 @@ void free_initmem(void) pgd_t *pgd_alloc(struct mm_struct *mm) { - pgd_t *pgd = (pgd_t *)__get_free_page(GFP_KERNEL); + pgd_t *pgd = __pgd_alloc(mm, 0); - if (pgd) { - memset(pgd, 0, USER_PTRS_PER_PGD * sizeof(pgd_t)); + if (pgd) memcpy(pgd + USER_PTRS_PER_PGD, swapper_pg_dir + USER_PTRS_PER_PGD, (PTRS_PER_PGD - USER_PTRS_PER_PGD) * sizeof(pgd_t)); - } + return pgd; } diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index a0b0e501ba66..29b0196ae49c 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -395,15 +395,14 @@ void __init pgtable_cache_init(void) SLAB_PANIC, NULL); } -static inline pgd_t *_pgd_alloc(void) +static inline pgd_t *_pgd_alloc(struct mm_struct *mm) { /* * If no SHARED_KERNEL_PMD, PAE kernel is running as a Xen domain. * We allocate one page for pgd. */ if (!SHARED_KERNEL_PMD) - return (pgd_t *)__get_free_pages(GFP_PGTABLE_USER, - PGD_ALLOCATION_ORDER); + return __pgd_alloc(mm, PGD_ALLOCATION_ORDER); /* * Now PAE kernel is not running as a Xen domain. We can allocate @@ -412,24 +411,23 @@ static inline pgd_t *_pgd_alloc(void) return kmem_cache_alloc(pgd_cache, GFP_PGTABLE_USER); } -static inline void _pgd_free(pgd_t *pgd) +static inline void _pgd_free(struct mm_struct *mm, pgd_t *pgd) { if (!SHARED_KERNEL_PMD) - free_pages((unsigned long)pgd, PGD_ALLOCATION_ORDER); + __pgd_free(mm, pgd); else kmem_cache_free(pgd_cache, pgd); } #else -static inline pgd_t *_pgd_alloc(void) +static inline pgd_t *_pgd_alloc(struct mm_struct *mm) { - return (pgd_t *)__get_free_pages(GFP_PGTABLE_USER, - PGD_ALLOCATION_ORDER); + return __pgd_alloc(mm, PGD_ALLOCATION_ORDER); } -static inline void _pgd_free(pgd_t *pgd) +static inline void _pgd_free(struct mm_struct *mm, pgd_t *pgd) { - free_pages((unsigned long)pgd, PGD_ALLOCATION_ORDER); + __pgd_free(mm, pgd); } #endif /* CONFIG_X86_PAE */ @@ -439,7 +437,7 @@ pgd_t *pgd_alloc(struct mm_struct *mm) pmd_t *u_pmds[MAX_PREALLOCATED_USER_PMDS]; pmd_t *pmds[MAX_PREALLOCATED_PMDS]; - pgd = _pgd_alloc(); + pgd = _pgd_alloc(mm); if (pgd == NULL) goto out; @@ -482,7 +480,7 @@ pgd_t *pgd_alloc(struct mm_struct *mm) if (sizeof(pmds) != 0) free_pmds(mm, pmds, PREALLOCATED_PMDS); out_free_pgd: - _pgd_free(pgd); + _pgd_free(mm, pgd); out: return NULL; } @@ -492,7 +490,7 @@ void pgd_free(struct mm_struct *mm, pgd_t *pgd) pgd_mop_up_pmds(mm, pgd); pgd_dtor(pgd); paravirt_pgd_free(mm, pgd); - _pgd_free(pgd); + _pgd_free(mm, pgd); } /* diff --git a/arch/xtensa/include/asm/pgalloc.h b/arch/xtensa/include/asm/pgalloc.h index 7fc0f9126dd3..1919ee9c3dd6 100644 --- a/arch/xtensa/include/asm/pgalloc.h +++ b/arch/xtensa/include/asm/pgalloc.h @@ -29,7 +29,7 @@ static inline pgd_t* pgd_alloc(struct mm_struct *mm) { - return (pgd_t*) __get_free_page(GFP_KERNEL | __GFP_ZERO); + return __pgd_alloc(mm, 0); } static inline void ptes_clear(pte_t *ptep) diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h index e3977ddca15e..de4df14158e6 100644 --- a/include/asm-generic/pgalloc.h +++ b/include/asm-generic/pgalloc.h @@ -258,10 +258,35 @@ static inline void p4d_free(struct mm_struct *mm, p4d_t *p4d) #endif /* CONFIG_PGTABLE_LEVELS > 4 */ +static inline pgd_t *__pgd_alloc_noprof(struct mm_struct *mm, unsigned int order) +{ + gfp_t gfp = GFP_PGTABLE_USER; + struct ptdesc *ptdesc; + + if (mm == &init_mm) + gfp = GFP_PGTABLE_KERNEL; + gfp &= ~__GFP_HIGHMEM; + + ptdesc = pagetable_alloc_noprof(gfp, order); + if (!ptdesc) + return NULL; + + return ptdesc_address(ptdesc); +} +#define __pgd_alloc(...) alloc_hooks(__pgd_alloc_noprof(__VA_ARGS__)) + +static inline void __pgd_free(struct mm_struct *mm, pgd_t *pgd) +{ + struct ptdesc *ptdesc = virt_to_ptdesc(pgd); + + BUG_ON((unsigned long)pgd & (PAGE_SIZE-1)); + pagetable_free(ptdesc); +} + #ifndef __HAVE_ARCH_PGD_FREE static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd) { - pagetable_free(virt_to_ptdesc(pgd)); + __pgd_free(mm, pgd); } #endif From patchwork Fri Jan 3 18:44:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13925795 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57ED3E7718F for ; Fri, 3 Jan 2025 18:44:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E6CE36B009B; Fri, 3 Jan 2025 13:44:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DF56C6B009D; Fri, 3 Jan 2025 13:44:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE3626B009F; Fri, 3 Jan 2025 13:44:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B1B236B009B for ; Fri, 3 Jan 2025 13:44:57 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 78AB9C0A37 for ; Fri, 3 Jan 2025 18:44:57 +0000 (UTC) X-FDA: 82967017434.25.E804846 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf15.hostedemail.com (Postfix) with ESMTP id BDEE9A000D for ; Fri, 3 Jan 2025 18:44:55 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; spf=pass (imf15.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735929895; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bDYv+lD8IINsE762iomZ4bq5tahQltx5gkKZxMzCtrk=; b=mONYJI8RMpWhCNMyJYElV1jnY6GfiZLacy2zbw7KXFqobWlSDVEWw6VvBlEfnHNk04O58F GDhLJvstlM0QClGoZxkpuxFrs8Ky9bT1lL04/Z+xYu5gBq9PAPGAshwth9iTN+fYzCcb5r LNXFQzBOwIh+N6J+h510uPbizhoWTX8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735929895; a=rsa-sha256; cv=none; b=T/4zwqwLHBfYexJTBkaVLbNzBCi3JuEPKk6B9hF2yQJ2cub06TSMw9Kj128lZsYK8GdsOP BI3AKU8RmvdlmKDiKdYghEHK3uSPyYceS4YRp9AE8A8f6d4gjJ0Vfp3YhyjnW14M6bHj/M MAZ3PdlMojMmZtYL66sTDLOtNpifNys= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; spf=pass (imf15.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 575781C14; Fri, 3 Jan 2025 10:45:23 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2191A3F673; Fri, 3 Jan 2025 10:44:51 -0800 (PST) From: Kevin Brodsky To: linux-mm@kvack.org Cc: Kevin Brodsky , Andrew Morton , Catalin Marinas , Dave Hansen , Linus Walleij , Andy Lutomirski , Peter Zijlstra , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-alpha@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-kernel@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-um@lists.infradead.org, loongarch@lists.linux.dev, x86@kernel.org Subject: [PATCH v2 6/6] mm: Introduce ctor/dtor at PGD level Date: Fri, 3 Jan 2025 18:44:15 +0000 Message-ID: <20250103184415.2744423-7-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250103184415.2744423-1-kevin.brodsky@arm.com> References: <20250103184415.2744423-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: BDEE9A000D X-Stat-Signature: ng4gg64xfsy5o6m8xsb7igwfn9fcckp1 X-Rspam-User: X-HE-Tag: 1735929895-36704 X-HE-Meta: U2FsdGVkX1+M73J/N4kWibgzGhUU3GoxEFEK6ibvH6i+hWZO7zLtpIEOzCSACy7wJqr6uecWlwU9YohsPO/2TyeI+obGBd1dV+YM91yThM2pbz6ITI/GIkEU47UUvmeBL0Z1mJCf8yLkt+9njULx7wNdveS8sRs6zMTiI2gZDwKn102SAf2I9mnu2fNTkGJ6jlD/VUa6WNNfL+3mZzyujO6/aRdsVtDx4o8jn/VUvDH2jmALR1TCjGnoE5V/zlZyAyXaSlvxpC0as36Ju1tpZ2aJNCM66yyA5Pnvjh0+M2mk/oBENodGw24n/kP6xJb0Ahg6N0EK1WBQZQr9QriZiGuOnNSm2SH0zC1qcXSHRi22ziRODom3ssR0Dj9w/M/GDWciwh+YB0SKlk4Q5DlYUct84k4G+UvtENlqV0Cu0zqWCW+lcrRi7ULx8LnwJBhYAEpVNWBAolUDyqSo1bDdo4F3Ox1/dledN+bSRv4yYXTIy5zW7HZOgGPuFkD99wXwl9dyZSqGjj7MYByL0jSGOI1hY27f43MbCRJajtNeLsKSVP/PoY1fReXtgmEdOsFYK/1W7f24yEBIdgIxMt6QUN8m6YWHHiVRRyEvUqaas4jfE5uCdbZLK9f+lP9MRxd3B8LCLvBTcJQdkRMERStDc4CGuakrhjR0RBLoN/bT2OkVoefPeNcCuvYLNiZ2GpPRLCw+rNJlctOzquqJLEmdLk3n2FrlSdMWfYOPRyBcvkicPOVSY2BeAo5CU/OqcPaDTQcJQNfIll/6YRd72ZsVV37x4WKQEmc1XIB1VjMZ2/qPQKzjtMBrF3TUoT+n0U9LWOO0bwpJdIyqhQ6q6ypOR6uDEfVyFIDWrsNNP+L/JBjz1Hw67ir1juWh8LVQK65LYxEDw9xzMYqIdPOBa9aYac4ptkkEOIZdPGASMpyFs4ewFyd0egFtuaeMUsAdq2y0srb2oJj66vWk16pJtKA FrhP1kaS La44GFHACgkKnoBXjAPwi/8O4Z7vxT0eY3ynmyvENsJPvERhGl7xmRl33ZtVCEFo3P3sbLMsOqQhw8jm6Up500hSJqS/sx5Lvpqt+xy3GoukKLn0ebMkYTjfXE7cu5e5gvDfd/S25GfyyQJOUKzn87Me18MYaLNXgRZPgXE/FjEYc0DUHRsKAx3JOyQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Following on from the introduction of P4D-level ctor/dtor, let's finish the job and introduce ctor/dtor at PGD level. The incurred improvement in page accounting is minimal - the main motivation is to create a single, generic place where construction/destruction hooks can be added for all page table pages. This patch should cover all architectures and all configurations where PGDs are one or more regular pages. This excludes any configuration where PGDs are allocated from a kmem_cache object. Acked-by: Dave Hansen Signed-off-by: Kevin Brodsky --- arch/m68k/include/asm/mcf_pgalloc.h | 3 ++- arch/m68k/mm/motorola.c | 6 ++++-- arch/s390/include/asm/pgalloc.h | 9 ++++++++- include/asm-generic/pgalloc.h | 3 ++- include/linux/mm.h | 5 +++++ 5 files changed, 21 insertions(+), 5 deletions(-) diff --git a/arch/m68k/include/asm/mcf_pgalloc.h b/arch/m68k/include/asm/mcf_pgalloc.h index 22d6c1fcabfb..4c648b51e7fd 100644 --- a/arch/m68k/include/asm/mcf_pgalloc.h +++ b/arch/m68k/include/asm/mcf_pgalloc.h @@ -73,7 +73,7 @@ static inline void pte_free(struct mm_struct *mm, pgtable_t pgtable) static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd) { - pagetable_free(virt_to_ptdesc(pgd)); + pagetable_dtor_free(virt_to_ptdesc(pgd)); } static inline pgd_t *pgd_alloc(struct mm_struct *mm) @@ -84,6 +84,7 @@ static inline pgd_t *pgd_alloc(struct mm_struct *mm) if (!ptdesc) return NULL; + pagetable_pgd_ctor(ptdesc); new_pgd = ptdesc_address(ptdesc); memcpy(new_pgd, swapper_pg_dir, PTRS_PER_PGD * sizeof(pgd_t)); diff --git a/arch/m68k/mm/motorola.c b/arch/m68k/mm/motorola.c index 6c09ccb72e8b..73651e093c4d 100644 --- a/arch/m68k/mm/motorola.c +++ b/arch/m68k/mm/motorola.c @@ -169,6 +169,9 @@ void *get_pointer_table(int type) case TABLE_PMD: pagetable_pmd_ctor(virt_to_ptdesc(page)); break; + case TABLE_PGD: + pagetable_pgd_ctor(virt_to_ptdesc(page)); + break; } mmu_page_ctor(page); @@ -207,8 +210,7 @@ int free_pointer_table(void *table, int type) /* all tables in page are free, free page */ list_del(dp); mmu_page_dtor((void *)page); - if (type == TABLE_PTE || type == TABLE_PMD) - pagetable_dtor(virt_to_ptdesc((void *)page)); + pagetable_dtor(virt_to_ptdesc((void *)page)); free_page (page); return 1; } else if (ptable_list[type].next != dp) { diff --git a/arch/s390/include/asm/pgalloc.h b/arch/s390/include/asm/pgalloc.h index 5fced6d3c36b..b19b6ed2ab53 100644 --- a/arch/s390/include/asm/pgalloc.h +++ b/arch/s390/include/asm/pgalloc.h @@ -130,11 +130,18 @@ static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd) static inline pgd_t *pgd_alloc(struct mm_struct *mm) { - return (pgd_t *) crst_table_alloc(mm); + unsigned long *table = crst_table_alloc(mm); + + if (!table) + return NULL; + pagetable_pgd_ctor(virt_to_ptdesc(table)); + + return (pgd_t *) table; } static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd) { + pagetable_dtor(virt_to_ptdesc(pgd)); crst_table_free(mm, (unsigned long *) pgd); } diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h index de4df14158e6..892ece4558a2 100644 --- a/include/asm-generic/pgalloc.h +++ b/include/asm-generic/pgalloc.h @@ -271,6 +271,7 @@ static inline pgd_t *__pgd_alloc_noprof(struct mm_struct *mm, unsigned int order if (!ptdesc) return NULL; + pagetable_pgd_ctor(ptdesc); return ptdesc_address(ptdesc); } #define __pgd_alloc(...) alloc_hooks(__pgd_alloc_noprof(__VA_ARGS__)) @@ -280,7 +281,7 @@ static inline void __pgd_free(struct mm_struct *mm, pgd_t *pgd) struct ptdesc *ptdesc = virt_to_ptdesc(pgd); BUG_ON((unsigned long)pgd & (PAGE_SIZE-1)); - pagetable_free(ptdesc); + pagetable_dtor_free(ptdesc); } #ifndef __HAVE_ARCH_PGD_FREE diff --git a/include/linux/mm.h b/include/linux/mm.h index 065fa9449d03..486638d22fc6 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3243,6 +3243,11 @@ static inline void pagetable_p4d_ctor(struct ptdesc *ptdesc) __pagetable_ctor(ptdesc); } +static inline void pagetable_pgd_ctor(struct ptdesc *ptdesc) +{ + __pagetable_ctor(ptdesc); +} + extern void __init pagecache_init(void); extern void free_initmem(void);