From patchwork Tue Jul 30 07:27:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13746821 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CA5FC3DA49 for ; Tue, 30 Jul 2024 07:23:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BAFA86B0098; Tue, 30 Jul 2024 03:23:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B5F4B6B0099; Tue, 30 Jul 2024 03:23:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 98B0F6B009A; Tue, 30 Jul 2024 03:23:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 70FEC6B0098 for ; Tue, 30 Jul 2024 03:23:19 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id E92AE160122 for ; Tue, 30 Jul 2024 07:23:18 +0000 (UTC) X-FDA: 82395578076.09.7450CFF Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf04.hostedemail.com (Postfix) with ESMTP id A26E74000B for ; Tue, 30 Jul 2024 07:23:16 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=HHL0szqj; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf04.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722324153; a=rsa-sha256; cv=none; b=DAJoNm/J347+qTQvMkk24AeUC62ZnQC6fXfYN86fk+316PqoNnr+FdYHqdQaf7kYaCx24M pDzEXdlmMgYDz3vaRPcOfCfs+U6HqoOeJKHq6TfFEysm3sXCRqqqd3G/tu9crbM6LXdFG5 w9xlmW+MpI6rPrCLHI0fmhXW+sA/0Ls= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=HHL0szqj; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf04.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722324153; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LogEJFOPEHoxxpDndZ0xRU9l/InF3XirabhdOqCiICk=; b=n7C7NLiEr8+q1w+ahM3B510s4DUBsp5X47/OmKaGS3trpbXUcY43rp5C9ArSkQA12eRWVL +nPZ1y/byZ6xQ0DMUwuQGe0fRxsXWgcbx5AjcbTv3dUrgdWjZ1ODxDbZpwNhrgnaJPtsgI pXE67a0zbkZ+YPFBEaX739mkSH6yg94= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id C538CCE0EBE; Tue, 30 Jul 2024 07:23:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B21F9C4AF16; Tue, 30 Jul 2024 07:22:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722324192; bh=+91Y+/MLNk1WBxXTysrg77L3vkr/WGy2+/WN+U1aHfM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HHL0szqjb9e2c7ISow55+gL3zMTJ7Hi70dgqGvtn04IgFlKSoCDbSjvq5pKd3SALU BNdZFk3RcqpWHqICH7MEB7lMmy5GUPrdSGvEDrygCOQcEoLTdfx6YJDQCsmkTky0Ql Dd6ra4zjUVvYJyB9u9IbpBdyrSpsi7fah8Mpxsshed3fhBinvJRbysQV62PvMQ39yN malm4RfG9bAhGyLJJu3pI2OdmRtcqaC1PB2x/qfzEDiHAkmrq0gPhjyLfii9l1DWIZ wHmNuR5GcmhGyMxkdgUgoHi1zrgdIL924cLjlxVp5VoiwZM31oi9eGVEUa4eTSyk8D bxsp/h7xWID4w== From: alexs@kernel.org To: Will Deacon , "Aneesh Kumar K . V" , Nick Piggin , Peter Zijlstra , Russell King , Catalin Marinas , Brian Cain , WANG Xuerui , Geert Uytterhoeven , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Michael Ellerman , Naveen N Rao , Paul Walmsley , Albert Ou , Thomas Gleixner , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Bibo Mao , Baolin Wang , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Qi Zheng , Vishal Moola , "Aneesh Kumar K . V" , Kemeng Shi , Lance Yang , Peter Xu , Barry Song , linux-s390@vger.kernel.org Cc: Guo Ren , Christophe Leroy , Palmer Dabbelt , Mike Rapoport , Oscar Salvador , Alexandre Ghiti , Jisheng Zhang , Samuel Holland , Anup Patel , Josh Poimboeuf , Breno Leitao , Alexander Gordeev , Gerald Schaefer , Hugh Dickins , David Hildenbrand , Ryan Roberts , Matthew Wilcox , Alex Shi , "Naveen N . Rao" , nvdimm@lists.linux.dev, linux-fsdevel@vger.kernel.org, sparclinux@vger.kernel.org, Kinsey Ho , Ingo Molnar , Christian Brauner , Alexander Viro , Jan Kara , Dan Williams , Andreas Larsson , "David S . Miller" , Jason Gunthorpe Subject: [RFC PATCH 14/18] mm/pgtable: use ptdesc in pgtable_trans_huge_deposit Date: Tue, 30 Jul 2024 15:27:15 +0800 Message-ID: <20240730072719.3715016-4-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240730072719.3715016-1-alexs@kernel.org> References: <20240730064712.3714387-1-alexs@kernel.org> <20240730072719.3715016-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: A26E74000B X-Stat-Signature: who8emzxo77x6c3a611myfzc8yrk1iwo X-Rspam-User: X-HE-Tag: 1722324196-270305 X-HE-Meta: U2FsdGVkX18dfvgDvKeh4pyjw6Ne5mpIj1qNQ6Vlg7se4NOjjTTXwFPQSJ5KqbqvEIWToSXXV3JBkDj7ytEva6XwubDpo3uMbAKnTYWan192WmC70t8lPtWs1fKkUX0+0ob95esSIic1SHKo2Bc44O+M7+d4fyZPpf9XHZJr5tSzEIpcaD1p0b80pvHKDTlG7iLSeFJaiYkuIRnZl848Ikej5aiDC65WQ9I8WrGrVY88FE3Yo5RCGwIwaOHm8Lw8T1M7JxICStJj5TVUEASuxUVBuArDT0tvWexd0PLNrVUrFbHoduqQzVSDRLnHCOYl6vtt4E79f91lL6owArfShCLXozZtOsa/JlN72aTApH4ziC8vKgHvksEe9OoHnxDjPgXCHIJhYjDvAgS7cN15iN0gt1hgQ4OIRSXLYuc4bmYNmv6klZikZHVP76T0uZta0c1RLLz+lIdyrdrlRbhlajVtb8KpRQT2jr/4ETJ0UTkmu4Ozumtu7l6NHiESvpdYS8PDeLapXDmIXCYmz8oCwesR2rS4xPX1xP2rqHllOJ08xe04YL9Ai+muRk3xI/X6/edj4/I4ps7BNgOJ4SxoJ2kPaj+Igwx2bKLVP+nOm6/nZpvL2fA4z82RlkkZBO3ZHRIZ0A/HwCnrGV+hgyfrhgBEbgR/IRwMVTo0Qpc+UxN6hAB4U9wdrVk/tFnoOi1nOskw5x6w1fqzMuCTC8zziNujVStJXRNU0A/PX4rfObBsTXfkjpi+hVe3PIAbFCCwNYXoIc+t+s27QcuvWy6qFazL2zRtzcWuN24IfIG/n45MdyWcjBsOBPQT+fBaRCwpDudUjv33o0OZbyI2KRYAyV1r65PwDb948BC01B4WNJn1oYR5kQU3Vs3h3bYfhh6li2Q184BRBmv4uN9eG7OllCAGyQVa6jGBaSnEngAgDhbHl9QlrpJIRD4yT/Q3ge69B99PG+vaAn2jp5c2nBa vasp71VN vbn3sEGKaNZAzqx/vpizm9WGv5WLnuK9K3oikjoCJEfTYOxhZEZRINW1Qrk4OKdZ1LoYbHM6aMqma5zg0B1AcGyTZiwol6SgNokjeDzswOFG8s806yTnMs0NHe5iIAUdgLTJqfVeFfb4q9mvE+GyNFDK7IF8cSDpy2o6V7oX45K4gf9EEOGZD0O1Ceevt6AG6vgWDYKFmZHheWvqb3Z0n0T7iP/j18egV98zWreIS4WBLP7xGlrA58WZyKxOP26GLuChZuB5Kb92DhVweRfM/x0Wlc7xDT1x84heS2OzY7Km7tNDFcFqId2BjVFQZhj+pIliwd26uQxN6ncH30nRQXmRVjlyS8fFQ3kRKdqSgijVpanSZ81XCBLPFBudtxp3dfoACGnlmT2DlW3LDZfIilcnRpneiMMPTkPoil5DtT5aDvMOtM+pdShLVNIpfLyIIIry7JwQcR181ZUS8P46zLXgkqUHTd1YUliqe2YA8l9x60tBUty59qeFmv33jAZQWgizoXe6fHV4rmvOxOlLNEMzMcgrz7fWCRe3A3rar2JYjkcGn/HLH3E75BrNDA2Xa2q8Q7tSxLl6l+GlVXgJhg6H8GmWeSqlffcEyc6c+6QLh+GKJtnFRuuwIDTVlgUbE4HDvBCkMVMf8xp0XiSlMuuOjMjsOi1xxgQqyf29IR6e81gatciBK5P+I4eW5coT97J3y4DjKOdppKCdx+Ab4JkqsBDxphCKbQbJhMMu6SLgkvBH1x3i7JlGlNIF/9kmCSEpa4ebkyN54cWz4VbG+lncKjXjtfjT+ka6HKscIYNtNPRhCbu9kB9qYZvFneNOZws9WvVzAfBE0BEk+HtuAq0sRxPpju64UtRc7eZnrTNZT3v4F8NIpgKNemBLHj1Lm4d8TLa8fG3Sa5y/hDyJrN7Di8GiLmtPkny4KX1j0hbGc53HCCA3R1geqfMOQN4w5c8mePzI+NJHc+UMtfBeTlfo/HUxM AoAKhUlM FLZRnG3ZUmIvVcFa6GF0EA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi A step to replace pgtable_t to struct ptdesc. Signed-off-by: Alex Shi Cc: linux-mm@kvack.org Cc: nvdimm@lists.linux.dev Cc: linux-fsdevel@vger.kernel.org Cc: sparclinux@vger.kernel.org Cc: linux-s390@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: linuxppc-dev@lists.ozlabs.org Cc: Barry Song Cc: Lance Yang Cc: Hugh Dickins Cc: Kinsey Ho Cc: Ingo Molnar Cc: Aneesh Kumar K.V Cc: Christian Brauner Cc: Alexander Viro Cc: Jan Kara Cc: Dan Williams Cc: Andreas Larsson Cc: David S. Miller Cc: Sven Schnelle Cc: Christian Borntraeger Cc: Vasily Gorbik Cc: Heiko Carstens Cc: Gerald Schaefer Cc: Alexander Gordeev Cc: Naveen N. Rao Cc: Nicholas Piggin Cc: Ryan Roberts Cc: David Hildenbrand Cc: Jason Gunthorpe Cc: Aneesh Kumar K.V Cc: Mike Rapoport Cc: Peter Xu Cc: Matthew Wilcox Cc: Christophe Leroy Cc: Michael Ellerman --- arch/powerpc/include/asm/book3s/64/pgtable.h | 6 +++--- arch/powerpc/mm/book3s64/hash_pgtable.c | 6 +++--- arch/powerpc/mm/book3s64/radix_pgtable.c | 6 +++--- arch/s390/include/asm/pgtable.h | 2 +- arch/s390/mm/pgtable.c | 6 +++--- arch/sparc/include/asm/pgtable_64.h | 2 +- arch/sparc/mm/tlb.c | 6 +++--- fs/dax.c | 2 +- include/linux/pgtable.h | 2 +- mm/debug_vm_pgtable.c | 2 +- mm/huge_memory.c | 14 +++++++------- mm/khugepaged.c | 2 +- mm/memory.c | 2 +- mm/pgtable-generic.c | 8 ++++---- 14 files changed, 33 insertions(+), 33 deletions(-) diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index 0ee440b819d7..cf44e2440825 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -1365,11 +1365,11 @@ pud_t pudp_huge_get_and_clear_full(struct vm_area_struct *vma, #define __HAVE_ARCH_PGTABLE_DEPOSIT static inline void pgtable_trans_huge_deposit(struct mm_struct *mm, - pmd_t *pmdp, pgtable_t pgtable) + pmd_t *pmdp, struct ptdesc *ptdesc) { if (radix_enabled()) - return radix__pgtable_trans_huge_deposit(mm, pmdp, pgtable); - return hash__pgtable_trans_huge_deposit(mm, pmdp, pgtable); + return radix__pgtable_trans_huge_deposit(mm, pmdp, ptdesc); + return hash__pgtable_trans_huge_deposit(mm, pmdp, ptdesc); } #define __HAVE_ARCH_PGTABLE_WITHDRAW diff --git a/arch/powerpc/mm/book3s64/hash_pgtable.c b/arch/powerpc/mm/book3s64/hash_pgtable.c index 35562d1f4267..8fd2c833dc3d 100644 --- a/arch/powerpc/mm/book3s64/hash_pgtable.c +++ b/arch/powerpc/mm/book3s64/hash_pgtable.c @@ -265,16 +265,16 @@ pmd_t hash__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long addres * the base page size hptes */ void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable) + struct ptdesc *ptdesc) { - pgtable_t *pgtable_slot; + pte_t **pgtable_slot; assert_spin_locked(pmd_lockptr(mm, pmdp)); /* * we store the pgtable in the second half of PMD */ pgtable_slot = (pgtable_t *)pmdp + PTRS_PER_PMD; - *pgtable_slot = pgtable; + *pgtable_slot = (pte_t)ptdesc; /* * expose the deposited pgtable to other cpus. * before we set the hugepage PTE at pmd level diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c index 3b9bb19510e3..c33e860966ad 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -1478,9 +1478,9 @@ pmd_t radix__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long addre * list_head memory area. */ void radix__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable) + struct ptdesc *ptdesc) { - struct list_head *lh = (struct list_head *) pgtable; + struct list_head *lh = (struct list_head *)ptdesc; assert_spin_locked(pmd_lockptr(mm, pmdp)); @@ -1489,7 +1489,7 @@ void radix__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, INIT_LIST_HEAD(lh); else list_add(lh, (struct list_head *) pmd_huge_pte(mm, pmdp)); - pmd_huge_pte(mm, pmdp) = pgtable; + pmd_huge_pte(mm, pmdp) = ptdesc_page(ptdesc); } struct ptdesc *radix__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h index cf0baf4bfe5c..d7b635f5e1e7 100644 --- a/arch/s390/include/asm/pgtable.h +++ b/arch/s390/include/asm/pgtable.h @@ -1735,7 +1735,7 @@ pud_t pudp_xchg_direct(struct mm_struct *, unsigned long, pud_t *, pud_t); #define __HAVE_ARCH_PGTABLE_DEPOSIT void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable); + struct ptdesc *ptdesc); #define __HAVE_ARCH_PGTABLE_WITHDRAW struct ptdesc *pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index b9016ee145cb..cf1a6aeb66d4 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -563,9 +563,9 @@ EXPORT_SYMBOL(pudp_xchg_direct); #ifdef CONFIG_TRANSPARENT_HUGEPAGE void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable) + struct ptdesc *ptdesc) { - struct list_head *lh = (struct list_head *) pgtable; + struct list_head *lh = (struct list_head *)ptdesc; assert_spin_locked(pmd_lockptr(mm, pmdp)); @@ -574,7 +574,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, INIT_LIST_HEAD(lh); else list_add(lh, (struct list_head *) pmd_huge_pte(mm, pmdp)); - pmd_huge_pte(mm, pmdp) = (struct ptdesc *)pgtable; + pmd_huge_pte(mm, pmdp) = ptdesc; } struct ptdesc *pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) diff --git a/arch/sparc/include/asm/pgtable_64.h b/arch/sparc/include/asm/pgtable_64.h index bfefd678e220..c71be5ef8b06 100644 --- a/arch/sparc/include/asm/pgtable_64.h +++ b/arch/sparc/include/asm/pgtable_64.h @@ -995,7 +995,7 @@ extern pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address, #define __HAVE_ARCH_PGTABLE_DEPOSIT void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable); + struct ptdesc *ptdesc); #define __HAVE_ARCH_PGTABLE_WITHDRAW struct ptdesc *pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); diff --git a/arch/sparc/mm/tlb.c b/arch/sparc/mm/tlb.c index bd2d3b1f6ba3..eeed4427f524 100644 --- a/arch/sparc/mm/tlb.c +++ b/arch/sparc/mm/tlb.c @@ -267,9 +267,9 @@ pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address, } void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable) + struct ptdesc *ptdesc) { - struct list_head *lh = (struct list_head *) pgtable; + struct list_head *lh = (struct list_head *)ptdesc; assert_spin_locked(&mm->page_table_lock); @@ -278,7 +278,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, INIT_LIST_HEAD(lh); else list_add(lh, (struct list_head *) pmd_huge_pte(mm, pmdp)); - pmd_huge_pte(mm, pmdp) = (struct ptdesc *)pgtable; + pmd_huge_pte(mm, pmdp) = ptdesc; } struct ptdesc *pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) diff --git a/fs/dax.c b/fs/dax.c index 61b9bd5200da..4b4e6acb0efc 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -1234,7 +1234,7 @@ static vm_fault_t dax_pmd_load_hole(struct xa_state *xas, struct vm_fault *vmf, } if (ptdesc) { - pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, ptdesc_page(ptdesc)); + pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, ptdesc); mm_inc_nr_ptes(vma->vm_mm); } pmd_entry = mk_pmd(&zero_folio->page, vmf->vma->vm_page_prot); diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 3fa7b93580a3..9d256c548f5e 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -925,7 +925,7 @@ static inline pmd_t pmdp_collapse_flush(struct vm_area_struct *vma, #ifndef __HAVE_ARCH_PGTABLE_DEPOSIT extern void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable); + struct ptdesc *ptdesc); #endif #ifndef __HAVE_ARCH_PGTABLE_WITHDRAW diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index f256bc816744..8550eec32aba 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -225,7 +225,7 @@ static void __init pmd_advanced_tests(struct pgtable_debug_args *args) /* Align the address wrt HPAGE_PMD_SIZE */ vaddr &= HPAGE_PMD_MASK; - pgtable_trans_huge_deposit(args->mm, args->pmdp, args->start_ptep); + pgtable_trans_huge_deposit(args->mm, args->pmdp, page_ptdesc(args->start_ptep)); pmd = pfn_pmd(args->pmd_pfn, args->page_prot); set_pmd_at(args->mm, vaddr, args->pmdp, pmd); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 4dc36910c8aa..aac67e8a8cc8 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -997,7 +997,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma); folio_add_new_anon_rmap(folio, vma, haddr, RMAP_EXCLUSIVE); folio_add_lru_vma(folio, vma); - pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, ptdesc_page(ptdesc)); + pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, ptdesc); set_pmd_at(vma->vm_mm, haddr, vmf->pmd, entry); update_mmu_cache_pmd(vma, vmf->address, vmf->pmd); add_mm_counter(vma->vm_mm, MM_ANONPAGES, HPAGE_PMD_NR); @@ -1064,7 +1064,7 @@ static void set_huge_zero_folio(struct ptdesc *ptdesc, struct mm_struct *mm, return; entry = mk_pmd(&zero_folio->page, vma->vm_page_prot); entry = pmd_mkhuge(entry); - pgtable_trans_huge_deposit(mm, pmd, ptdesc_page(ptdesc)); + pgtable_trans_huge_deposit(mm, pmd, ptdesc); set_pmd_at(mm, haddr, pmd, entry); mm_inc_nr_ptes(mm); } @@ -1167,7 +1167,7 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, } if (ptdesc) { - pgtable_trans_huge_deposit(mm, pmd, ptdesc_page(ptdesc)); + pgtable_trans_huge_deposit(mm, pmd, ptdesc); mm_inc_nr_ptes(mm); ptdesc = NULL; } @@ -1404,7 +1404,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, } add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR); mm_inc_nr_ptes(dst_mm); - pgtable_trans_huge_deposit(dst_mm, dst_pmd, ptdesc_page(ptdesc)); + pgtable_trans_huge_deposit(dst_mm, dst_pmd, ptdesc); if (!userfaultfd_wp(dst_vma)) pmd = pmd_swp_clear_uffd_wp(pmd); set_pmd_at(dst_mm, addr, dst_pmd, pmd); @@ -1449,7 +1449,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR); out_zero_page: mm_inc_nr_ptes(dst_mm); - pgtable_trans_huge_deposit(dst_mm, dst_pmd, ptdesc_page(ptdesc)); + pgtable_trans_huge_deposit(dst_mm, dst_pmd, ptdesc); pmdp_set_wrprotect(src_mm, addr, src_pmd); if (!userfaultfd_wp(dst_vma)) pmd = pmd_clear_uffd_wp(pmd); @@ -1962,7 +1962,7 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr, struct ptdesc *ptdesc; ptdesc = pgtable_trans_huge_withdraw(mm, old_pmd); - pgtable_trans_huge_deposit(mm, new_pmd, ptdesc_page(ptdesc)); + pgtable_trans_huge_deposit(mm, new_pmd, ptdesc); } pmd = move_soft_dirty_pmd(pmd); set_pmd_at(mm, new_addr, new_pmd, pmd); @@ -2236,7 +2236,7 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pm set_pmd_at(mm, dst_addr, dst_pmd, _dst_pmd); src_ptdesc = pgtable_trans_huge_withdraw(mm, src_pmd); - pgtable_trans_huge_deposit(mm, dst_pmd, ptdesc_page(src_ptdesc)); + pgtable_trans_huge_deposit(mm, dst_pmd, src_ptdesc); unlock_ptls: double_pt_unlock(src_ptl, dst_ptl); if (src_anon_vma) { diff --git a/mm/khugepaged.c b/mm/khugepaged.c index f3b3db104615..48a54269472e 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1232,7 +1232,7 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address, BUG_ON(!pmd_none(*pmd)); folio_add_new_anon_rmap(folio, vma, address, RMAP_EXCLUSIVE); folio_add_lru_vma(folio, vma); - pgtable_trans_huge_deposit(mm, pmd, pgtable); + pgtable_trans_huge_deposit(mm, pmd, page_ptdesc(pgtable)); set_pmd_at(mm, address, pmd, _pmd); update_mmu_cache_pmd(vma, address, pmd); spin_unlock(pmd_ptl); diff --git a/mm/memory.c b/mm/memory.c index 27c2f63b7487..956cfe5f644d 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4687,7 +4687,7 @@ static void deposit_prealloc_pte(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; - pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, vmf->prealloc_pte); + pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, page_ptdesc(vmf->prealloc_pte)); /* * We are going to consume the prealloc table, * count that as nr_ptes. diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index de1ed30fea16..5e763682941d 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -163,16 +163,16 @@ pud_t pudp_huge_clear_flush(struct vm_area_struct *vma, unsigned long address, #ifndef __HAVE_ARCH_PGTABLE_DEPOSIT void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable) + struct ptdesc *ptdesc) { assert_spin_locked(pmd_lockptr(mm, pmdp)); /* FIFO */ if (!pmd_huge_pte(mm, pmdp)) - INIT_LIST_HEAD(&pgtable->lru); + INIT_LIST_HEAD(&ptdesc->pt_list); else - list_add(&pgtable->lru, &pmd_huge_pte(mm, pmdp)->pt_list); - pmd_huge_pte(mm, pmdp) = page_ptdesc(pgtable); + list_add(&ptdesc->pt_list, &pmd_huge_pte(mm, pmdp)->pt_list); + pmd_huge_pte(mm, pmdp) = ptdesc; } #endif