From patchwork Mon Jun 24 14:45:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 13709625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9035BC2BD09 for ; Mon, 24 Jun 2024 14:46:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1FA706B01A3; Mon, 24 Jun 2024 10:46:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A9D56B03E5; Mon, 24 Jun 2024 10:46:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 04C1C6B03E8; Mon, 24 Jun 2024 10:46:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id D41476B01A3 for ; Mon, 24 Jun 2024 10:46:54 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 85709161137 for ; Mon, 24 Jun 2024 14:46:54 +0000 (UTC) X-FDA: 82266059148.27.D615C2C Received: from pegase1.c-s.fr (pegase1.c-s.fr [93.17.236.30]) by imf17.hostedemail.com (Postfix) with ESMTP id 631964000F for ; Mon, 24 Jun 2024 14:46:52 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.236.30 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu; dmarc=pass (policy=quarantine) header.from=csgroup.eu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719240398; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Xnap6QF/MKf6QAqysV3GvrdUulr8iCU56pdyhGUPre8=; b=mAMrrDVKYx5jBZViIGMOOOiSgH41fwFOQjnqkZkztv2ATYFBXZgFwQ5wcImm5ohAVRNEwg wVCD3QYBQhvuLhHAFkN84vmGBYiHYL4bZptdn02GZgy2gQhdOev5DKvuIJHUqNV7w4YpV9 Nq9szdT0dwIVPNOHaTsOVMsSRjEeIb8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719240398; a=rsa-sha256; cv=none; b=3grpzdtvEinfrJyP2OWM93WFc67U/DjSjnlnVHi3V0RFVfnLRSEne0geuQoUvL5RtTkVW/ cO76CVhqIMk7BGA7BXzMmreuRNpWO7YiQtjatvD1QjVqPyqnhJl092kWQRCGzvdsqSPPz5 DjuTsbfopRHHjHVTuTOnaQ9IbTK+Qy0= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.236.30 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu; dmarc=pass (policy=quarantine) header.from=csgroup.eu Received: from localhost (mailhub3.si.c-s.fr [192.168.12.233]) by localhost (Postfix) with ESMTP id 4W79kr2bkHz9vGV; Mon, 24 Jun 2024 16:46:12 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from pegase1.c-s.fr ([192.168.12.234]) by localhost (pegase1.c-s.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mljZycpqFe68; Mon, 24 Jun 2024 16:46:12 +0200 (CEST) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 4W79kh6WXLz9vFn; Mon, 24 Jun 2024 16:46:04 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id DA7A08B766; Mon, 24 Jun 2024 16:46:04 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id Ku0Pcz5vOux0; Mon, 24 Jun 2024 16:46:04 +0200 (CEST) Received: from PO20335.idsi0.si.c-s.fr (unknown [192.168.233.33]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 559438B768; Mon, 24 Jun 2024 16:46:04 +0200 (CEST) From: Christophe Leroy To: Andrew Morton , Jason Gunthorpe , Peter Xu , Oscar Salvador , Michael Ellerman , Nicholas Piggin Cc: Christophe Leroy , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org Subject: [PATCH v6 11/23] powerpc/mm: Allow hugepages without hugepd Date: Mon, 24 Jun 2024 16:45:37 +0200 Message-ID: X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1719240334; l=6864; i=christophe.leroy@csgroup.eu; s=20211009; h=from:subject:message-id; bh=OHpbr2EW8Ehxc4cGWtKirj+0lOuzXeICYvz9BBmzSDA=; b=+ATNSpAzf8isw3Aa1tw0C+3ybmb6G6UdnToqaxV++RMET3/Bj6zUlz52UQeYITJq6fgQFXx8k A/zzeYXxEPBBe3ywKbg7K+BqBLBUCN71bTwMi0wpCmc77aFh81g1bbO X-Developer-Key: i=christophe.leroy@csgroup.eu; a=ed25519; pk=HIzTzUj91asvincQGOFx6+ZF5AoUuP9GdOtQChs7Mm0= X-Stat-Signature: jzry59pnazcsxzdme53u5jy8dq4ac4cq X-Rspamd-Queue-Id: 631964000F X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1719240412-292033 X-HE-Meta: U2FsdGVkX1861C6L482y02FWslGB+L1R4rTa1Ycjupv9Usdvx8MLm4at49U7LZkROrgc9kQdtmuKs9YdJdbTh+q5S6Pc1O7OI6fHpsBOivAuoYSBHJcYHjlqeSFUS18NlkJ1MNPOx/Rwqw2TeW9/w0ljf+5FoF3APPIz+3ez5MBh5kqn4hLQeGm2hcW7FgBBKgCjU4QJcjUVzdkWBlA8q8aEIje96oVTFKtCis4s8gPlrjBO+YvIgzaYKUBaVj5a1pjNC1p36p5f7bLdej6SQFFsjGj8tl6U/ED8zmRq3f687lvXUfQmylorMVYxhF6912bB0cmgmF3oL8U+elJVNbyDwaLyYzC0RAEb/Ghs3hZIF4Z+BTAKlmuM8mNC27crpEVkAo3pJROy5bUPy4Hw5F91gizF1IQfixm+Hsl9gA8rM74fG2bv+cTVJcGWzNfIns9FqxqnXbTVH//NLZGCm+QOX61CeaN158WO/wpHs8zkbw6g4UqEv2FfsXHJlTcW2sZTmwCAtXau/nv7bLH2ElvATjGLHOneTxJIaTQuOma5j+rc37r9kfBBgEpbZW6xpWtqlR2J2uwbvB2npCpdwGg0UlXl00pH6B8PdMKIEWzMhIyMJ8+q/zSCqp5a8+DYR9XJpHwp5s2KoaKWLsyFXOxHV4JxndN8HtzWHe6B+Gh3l38472Cjifv7Meslrf9Xn4MYwr9tpJPZ70Cbqh7WKNfU7WjjPNAwy60K0KxOpw5EiMknwp1XlyqwXnssSAAQlZruJT/MUkLOqVEu1Bswg0CmGbxbqBN3cHlyoJOrU9jML7SyrHvyw9PLRGixHY6zEF6F6x0MqOmSh5rdUB4YXx1ilHyN4/8Mo6qD9DOQJteZPzFJRIIxC9wEH/IbwCIqMkciGPrPH4hj6LkRs0unA+QOCr+BCBT7ExnoVswdm/lORnCy5kdj8q8TktqEPGJTQWrRmtRL3HoXIFNs/uf /AlGfINj q1ztKA3+m9qfN/gTHMV1BZomfD8IecxfOi3iuGQjmk98jZeODP49qgZkzpA8et4LngyZPieg302gPIs3BZg3j4+gsd4zhz/JymnP4Qme1CF4cu+fHkRB09wOxY+gLh6ld8BPpgjeii38LqjvL7pkj54/hnmxesM+hARpG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In preparation of implementing huge pages on powerpc 8xx without hugepd, enclose hugepd related code inside an ifdef CONFIG_ARCH_HAS_HUGEPD This also allows removing some stubs. Signed-off-by: Christophe Leroy Reviewed-by: Oscar Salvador --- v3: - Prepare huge_pte_alloc() for full standard topology, not only for 2-level - Reordered last part of huge_pte_alloc() v4: - Rebased of v6.10-rc1 v5: - Moved the Kconfig split in this patch. --- arch/powerpc/Kconfig | 1 - arch/powerpc/include/asm/book3s/32/pgalloc.h | 2 -- arch/powerpc/include/asm/hugetlb.h | 10 ++---- arch/powerpc/include/asm/nohash/pgtable.h | 2 +- arch/powerpc/mm/hugetlbpage.c | 33 ++++++++++++++++++++ arch/powerpc/mm/pgtable.c | 2 ++ arch/powerpc/platforms/Kconfig.cputype | 3 ++ 7 files changed, 41 insertions(+), 12 deletions(-) diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index c88c6d46a5bc..b60b6e991227 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -135,7 +135,6 @@ config PPC select ARCH_HAS_DMA_MAP_DIRECT if PPC_PSERIES select ARCH_HAS_FORTIFY_SOURCE select ARCH_HAS_GCOV_PROFILE_ALL - select ARCH_HAS_HUGEPD if HUGETLB_PAGE select ARCH_HAS_KCOV select ARCH_HAS_KERNEL_FPU_SUPPORT if PPC64 && PPC_FPU select ARCH_HAS_MEMBARRIER_CALLBACKS diff --git a/arch/powerpc/include/asm/book3s/32/pgalloc.h b/arch/powerpc/include/asm/book3s/32/pgalloc.h index dc5c039eb28e..dd4eb3063175 100644 --- a/arch/powerpc/include/asm/book3s/32/pgalloc.h +++ b/arch/powerpc/include/asm/book3s/32/pgalloc.h @@ -47,8 +47,6 @@ static inline void pgtable_free(void *table, unsigned index_size) } } -#define get_hugepd_cache_index(x) (x) - static inline void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int shift) { diff --git a/arch/powerpc/include/asm/hugetlb.h b/arch/powerpc/include/asm/hugetlb.h index ea71f7245a63..79176a499763 100644 --- a/arch/powerpc/include/asm/hugetlb.h +++ b/arch/powerpc/include/asm/hugetlb.h @@ -30,10 +30,12 @@ static inline int is_hugepage_only_range(struct mm_struct *mm, } #define is_hugepage_only_range is_hugepage_only_range +#ifdef CONFIG_ARCH_HAS_HUGEPD #define __HAVE_ARCH_HUGETLB_FREE_PGD_RANGE void hugetlb_free_pgd_range(struct mmu_gather *tlb, unsigned long addr, unsigned long end, unsigned long floor, unsigned long ceiling); +#endif #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR static inline pte_t huge_ptep_get_and_clear(struct mm_struct *mm, @@ -67,14 +69,6 @@ static inline void flush_hugetlb_page(struct vm_area_struct *vma, { } -#define hugepd_shift(x) 0 -static inline pte_t *hugepte_offset(hugepd_t hpd, unsigned long addr, - unsigned pdshift) -{ - return NULL; -} - - static inline void __init gigantic_hugetlb_cma_reserve(void) { } diff --git a/arch/powerpc/include/asm/nohash/pgtable.h b/arch/powerpc/include/asm/nohash/pgtable.h index f5f39d4f03c8..e7fc1314c23e 100644 --- a/arch/powerpc/include/asm/nohash/pgtable.h +++ b/arch/powerpc/include/asm/nohash/pgtable.h @@ -340,7 +340,7 @@ static inline void __set_pte_at(struct mm_struct *mm, unsigned long addr, #define pgprot_writecombine pgprot_noncached_wc -#ifdef CONFIG_HUGETLB_PAGE +#ifdef CONFIG_ARCH_HAS_HUGEPD static inline int hugepd_ok(hugepd_t hpd) { #ifdef CONFIG_PPC_8xx diff --git a/arch/powerpc/mm/hugetlbpage.c b/arch/powerpc/mm/hugetlbpage.c index 594a4b7b2ca2..20fad59ff9f5 100644 --- a/arch/powerpc/mm/hugetlbpage.c +++ b/arch/powerpc/mm/hugetlbpage.c @@ -42,6 +42,7 @@ pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr, unsigned long s return __find_linux_pte(mm->pgd, addr, NULL, NULL); } +#ifdef CONFIG_ARCH_HAS_HUGEPD static int __hugepte_alloc(struct mm_struct *mm, hugepd_t *hpdp, unsigned long address, unsigned int pdshift, unsigned int pshift, spinlock_t *ptl) @@ -193,6 +194,36 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, return hugepte_offset(*hpdp, addr, pdshift); } +#else +pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, + unsigned long addr, unsigned long sz) +{ + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + + addr &= ~(sz - 1); + + p4d = p4d_offset(pgd_offset(mm, addr), addr); + if (!mm_pud_folded(mm) && sz >= P4D_SIZE) + return (pte_t *)p4d; + + pud = pud_alloc(mm, p4d, addr); + if (!pud) + return NULL; + if (!mm_pmd_folded(mm) && sz >= PUD_SIZE) + return (pte_t *)pud; + + pmd = pmd_alloc(mm, pud, addr); + if (!pmd) + return NULL; + + if (sz >= PMD_SIZE) + return (pte_t *)pmd; + + return pte_alloc_huge(mm, pmd, addr); +} +#endif #ifdef CONFIG_PPC_BOOK3S_64 /* @@ -248,6 +279,7 @@ int __init alloc_bootmem_huge_page(struct hstate *h, int nid) return __alloc_bootmem_huge_page(h, nid); } +#ifdef CONFIG_ARCH_HAS_HUGEPD #ifndef CONFIG_PPC_BOOK3S_64 #define HUGEPD_FREELIST_SIZE \ ((PAGE_SIZE - sizeof(struct hugepd_freelist)) / sizeof(pte_t)) @@ -505,6 +537,7 @@ void hugetlb_free_pgd_range(struct mmu_gather *tlb, } } while (addr = next, addr != end); } +#endif bool __init arch_hugetlb_valid_size(unsigned long size) { diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c index bce8a8619589..9010973f036c 100644 --- a/arch/powerpc/mm/pgtable.c +++ b/arch/powerpc/mm/pgtable.c @@ -496,8 +496,10 @@ pte_t *__find_linux_pte(pgd_t *pgdir, unsigned long ea, if (!hpdp) return NULL; +#ifdef CONFIG_ARCH_HAS_HUGEPD ret_pte = hugepte_offset(*hpdp, ea, pdshift); pdshift = hugepd_shift(*hpdp); +#endif out: if (hpage_shift) *hpage_shift = pdshift; diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype index b2d8c0da2ad9..0c3c69ebf19f 100644 --- a/arch/powerpc/platforms/Kconfig.cputype +++ b/arch/powerpc/platforms/Kconfig.cputype @@ -37,6 +37,7 @@ config PPC_85xx config PPC_8xx bool "Freescale 8xx" + select ARCH_HAS_HUGEPD if HUGETLB_PAGE select ARCH_SUPPORTS_HUGETLBFS select FSL_SOC select PPC_KUEP @@ -98,6 +99,7 @@ config PPC_BOOK3S_64 select ARCH_ENABLE_HUGEPAGE_MIGRATION if HUGETLB_PAGE && MIGRATION select ARCH_ENABLE_SPLIT_PMD_PTLOCK select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE + select ARCH_HAS_HUGEPD if HUGETLB_PAGE select ARCH_SUPPORTS_HUGETLBFS select ARCH_SUPPORTS_NUMA_BALANCING select HAVE_MOVE_PMD @@ -290,6 +292,7 @@ config PPC_BOOK3S config PPC_E500 select FSL_EMB_PERFMON bool + select ARCH_HAS_HUGEPD if HUGETLB_PAGE select ARCH_SUPPORTS_HUGETLBFS if PHYS_64BIT || PPC64 select PPC_SMP_MUXED_IPI select PPC_DOORBELL