From patchwork Mon Jan 27 09:35:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13951149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 547FCC0218F for ; Mon, 27 Jan 2025 09:44:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DB0A9280142; Mon, 27 Jan 2025 04:44:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D608628013F; Mon, 27 Jan 2025 04:44:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C00E1280142; Mon, 27 Jan 2025 04:44:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A177528013F for ; Mon, 27 Jan 2025 04:44:57 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 63C14B2985 for ; Mon, 27 Jan 2025 09:44:57 +0000 (UTC) X-FDA: 83052747834.12.6404107 Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by imf20.hostedemail.com (Postfix) with ESMTP id 7424C1C000A for ; Mon, 27 Jan 2025 09:44:55 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=eW+Kjmtu; dmarc=none; spf=pass (imf20.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.43 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737971095; a=rsa-sha256; cv=none; b=aeZWGAuIpQxjqyOHEEP9/OKU3LYAEac4dpktxCaNx38A6QWW8UUIwUOU+Ra59KYDucnB74 qWcvahw1Qln53AE9Dlh3sBSdzH3/UZmH9ST7jmLN0O+x73PMaW29wPlsfgHD5c+b03HO8g Rkyfk+9kV+yDsz6ohmOl0dyWTNuhi4Y= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=eW+Kjmtu; dmarc=none; spf=pass (imf20.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.43 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737971095; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jsGp3DbjRPLkoj2kwFjv1+EJYiVm/DSEknC/OoKntbQ=; b=rnAemXc/k7bNd379lB/p686MToCZHjNRFUEgs27/ZFh9F0kHbL945tRA/hMv4hrZAsDYtW aH6cm8Hz7QpPQJP2gEVB5zeGYjwGLLyljgP3Ik05RXZ+zIcNkpuoTL8eAG8TrlSKVE8oJz zqreBTXq32wxJhdtwKvbRnoFu3EtN1s= Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-4364a37a1d7so43285495e9.3 for ; Mon, 27 Jan 2025 01:44:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1737971094; x=1738575894; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jsGp3DbjRPLkoj2kwFjv1+EJYiVm/DSEknC/OoKntbQ=; b=eW+KjmtuQr8NlAJK/1BD+kajk7HD91lhGu3uY/MtmRTCP7ZTv1pD3bmiXh0+Pl4Jkk wL906wZ8voX/06Ustz5VpdfUmh6dBJmzSGQYVGzbPcpxNypFvuw6CspbgnXgqk92CCXy 13jo5roq7THWk6f5FGHRILpmU4QnfZMPKd25H+4AtRUw14ExmcAp15euYK0BcZT3qjCj ec/qNpRasQdaEXbVKPyoF5RieXAu7MrGqCJJnL6++8cTGkwN/nupuffZqR2O4xCX1fGj GX41zpV8pZu8J6AyWclmebHnrp0BzkcDCrHZzECj4GpWkMypXl4T9z3ZyKyLYgWIBkTi HqdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737971094; x=1738575894; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jsGp3DbjRPLkoj2kwFjv1+EJYiVm/DSEknC/OoKntbQ=; b=H8e2IJJOIL0IhyHP2WU55dHvxTmHe5L73nH7maAd9+l3/NtwdBREd5kjF8wmADFy02 bu/Mn/RQzVfXT8j6ohk6Vjz5qewixgCyKrQ500xQjN11/fh2BxAQuUhwB5peumKH4B9p 8dM2UwCPwSgv8nvbGsV5147anzpB/j8b2VZyrSVloRa0Pk0GhfBzke71vhyiYF5HHhdb TYCs6auQ3MfN7NNVmt287ZDtPeJ8nxPh7ET4aWfvzYIDhPLY7HWRn3/J1NlipryaM/ms 5M0JManRLSxMilxb7rsY8GolSvK8ejzu6xic8hwMQTT8caF+HB2IUhALbKdvULqSZmB5 X+1A== X-Forwarded-Encrypted: i=1; AJvYcCUo30HbAD+mNYp/jurZ8E3Avu9TmxLKjqTBgWz3nwsc8KYOWX12GH1ZBeGKGgeaVP3D1qZzeadYnw==@kvack.org X-Gm-Message-State: AOJu0Yw+j+rB6PRvNjq8EhMkASEnpHuQiCfRWPAiQDKsZCR6M06cCK/i CqHUYIfqueOHgKe1odPFMfbjtlq6ysAQrrZob78sO9btN/yvBn7Gh9a1Sq5gWzg= X-Gm-Gg: ASbGncuMDqioffFe/swja7NmJ3D6Wqwtz72C5+ZdT1ZIzcHyi9o4oRJ8p8E08qRuJ6U JRn1VZNlwtZIAVczZDd/8m5mF42JBq4YpuJ5uZO7gxlEmgbW8EdqeaX4d0LoJM/szrqXvjYmUJ3 VOeDzIKXLqBGnkmb6kkjbWxWjtVvaNNTTFx5T5nuHJqPnhZOSmdDNRVM5CV1RQdarb8OkKRyRZW PADxaXNTHCg2k3IQo1CZfOcWRuuLpzBQB/gncCeuCw0Mvj58m+dZKSF3g59l7KG6Llz0bvHncBq NNV1JSW77pvLfULPRveRXwx7CFdWzBw= X-Google-Smtp-Source: AGHT+IFDaK9cNMo23MROO6bkzC2nQlBGexsaacNWVhEd87bHI/6OEZGcK91epmkFjxQ01HSL43+DVA== X-Received: by 2002:a05:600c:1ca9:b0:434:f297:8e78 with SMTP id 5b1f17b1804b1-438913cb65fmr390606995e9.7.1737971094091; Mon, 27 Jan 2025 01:44:54 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3e22:3e78:ce5a:32c3]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-438bd47eecasm124006275e9.6.2025.01.27.01.44.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jan 2025 01:44:53 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v4 9/9] mm: Use common huge_ptep_clear_flush() function for riscv/arm64 Date: Mon, 27 Jan 2025 10:35:30 +0100 Message-Id: <20250127093530.19548-10-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250127093530.19548-1-alexghiti@rivosinc.com> References: <20250127093530.19548-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 7424C1C000A X-Stat-Signature: kxe7jj5epnkob5979stjq1o674pd6uqd X-HE-Tag: 1737971095-498855 X-HE-Meta: U2FsdGVkX1+xase6SCuQLMTVzfGofU4LIlcs8pbPg8g+u+jxXzzsh0b1/k1B1VSpmMr1IGo7cfvcllErWlvQLLgVZAtPQcqUxl3yF3QVJpejpy/15FPD9kAdjA5NvuWPP7s7reA3j69gbg+XdNvrtWjzyDl79s7uKFqx7N2/AymkQMZKodeueYG2L0/20+PCoeS5Nq69y6vwhFUyDRuR4Gh6oeMbhe3WvfHZLAo9c7f0xBNueiI2I+01fdIv+ks2sZwlPrYmX2MRTbcnr5Xudn87XClJxrtBl/d2PrK0lpr8dBDglBiM/zAbHaKB+ltiPxTUxqFtbVCaJQzMd+KSLrEfj8Wxks4OpW19Ln5hj6E6tK9aXmnwlbhFqJ7O0l3keSsJ/JsertbXhW5IK29VYCGH/ckAJnpoKQ7qF3qvfNMQfIzAETk1dIbEhl9sBQ1bb70ZuTVUhVdCTng8YunKJY9QHrI1hKv3CNm990G9oSZscK0w+YpIx/FQXQMb0jG+JxIYDYTl1PwVJeADzpulL+ouuO7wGY4ME/3x26GLmr3IHVkx3OvSgZwzgHu5J7KjsAMe0nBeIbD29H4tLBXS32bn0IYuS+Vz41MWS61S0fh6CzwyDObOSisMRozk56rsz2zp2Z3TUScWCHvr/uQdY2PNbC5BwxSXWyiJjDwcHkLb2chgS0r70jhnehOzuvuDLN1ygYjBjVzggHBNwGwFz1iTpWLuTFcz52WqWLweRuRsAvWEZmIzU5egUb1LfGNehgIGbMVzEW4BRxJaERgXWA/IA0a9mPHI01hYezps048CHU2vL8WuS9jsyG7soC5cipJkDty2KULLjX1B1rghD6ti1ygFGw0xwyDfQuS3dLPdvXU31BkVEIeB8bm6bC9Ilcw2aRV05FwxMhLVK/SSwkGFVLRLXsPGOdMXQ72qimO2MrECtSk8ra3JWW1PUOu/h/RyhiyJymFX2+XtiiM 7sECplVV QdQ6OJomEBn1/7rOSh06gCTSoY3aJSSwwFTCJBmx6MjR9XQ1Yxz0TNg2lWx05kM7alVU+BlaikmZfgcuJzEheuxYJm+dmqDsRKbDOnhxVn3Wh9aeD+gxqXOnTut245BitOw7fmva1LU58M47bexXyty1PT5MCBZ7Z43YjxgBylaCTD++nW/n8cVUC/5cDjG13U4pUvLCv/I5x/sYEHE80OZcjnrmXj6AAkSSmjiVv2Sui4p7oj4a9z9RYsW6gaNY7IykkGaCU93M7rvJeesEDBNBtjG0LaF1aeDPMcw3YmDNCcFiZxWtcgA2nmEfp8bprL8ykYGDS2s8TTrBBRIihyvlXNKzbVNkYlQNdjnQTpOj0vCTjMpIe6NPdG7j/BmeC/1X6ShtbS67r3MZsNtFkI7b5L4dm2JNyCLyvYfKsuv8eR6Vl45M8wdsDCwzubpWbVg6E1SWb5faG6wPl5zn9BNS3eQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 -- arch/arm64/mm/hugetlbpage.c | 61 -------------------------------- arch/riscv/include/asm/hugetlb.h | 7 +--- arch/riscv/mm/hugetlbpage.c | 51 -------------------------- include/linux/hugetlb_contpte.h | 4 +++ mm/hugetlb_contpte.c | 15 ++++++++ 6 files changed, 20 insertions(+), 121 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index fd1de0caad3f..3f79e4b76711 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -35,9 +35,6 @@ static inline void arch_clear_hugetlb_flags(struct folio *folio) pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte -#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH -extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep); void __init arm64_hugetlb_cma_reserve(void); diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 0430cb41f381..270e4580e12a 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -98,53 +98,6 @@ int find_num_contig(struct mm_struct *mm, unsigned long addr, return CONT_PTES; } -/* - * Changing some bits of contiguous entries requires us to follow a - * Break-Before-Make approach, breaking the whole contiguous set - * before we can change any entries. See ARM DDI 0487A.k_iss10775, - * "Misprogramming of the Contiguous bit", page D4-1762. - * - * This helper performs the break step. - */ -static pte_t get_clear_contig(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - pte_t orig_pte = __ptep_get(ptep); - unsigned long i; - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) { - pte_t pte = __ptep_get_and_clear(mm, addr, ptep); - - /* - * If HW_AFDBM is enabled, then the HW could turn on - * the dirty or accessed bit for any page in the set, - * so check them all. - */ - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - return orig_pte; -} - -static pte_t get_clear_contig_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - pte_t orig_pte = get_clear_contig(mm, addr, ptep, pgsize, ncontig); - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - - flush_tlb_range(&vma, addr, addr + (pgsize * ncontig)); - return orig_pte; -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned long sz) { @@ -277,20 +230,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep) -{ - struct mm_struct *mm = vma->vm_mm; - size_t pgsize; - int ncontig; - - if (!pte_cont(__ptep_get(ptep))) - return ptep_clear_flush(vma, addr, ptep); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - return get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); -} - static int __init hugetlbpage_init(void) { if (pud_sect_supported()) diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 4c692dd82779..63c7e4fa342a 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -20,14 +20,9 @@ bool arch_hugetlb_migration_supported(struct hstate *h); #endif #ifdef CONFIG_RISCV_ISA_SVNAPOT -#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep); - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte - -#endif /*CONFIG_RISCV_ISA_SVNAPOT*/ +#endif /* CONFIG_RISCV_ISA_SVNAPOT */ #include diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 8963a4e77742..ea1ae3a43d45 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -121,42 +121,6 @@ unsigned long hugetlb_mask_last_page(struct hstate *h) return 0UL; } -static pte_t get_clear_contig(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pte_num) -{ - pte_t orig_pte = ptep_get(ptep); - unsigned long i; - - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) { - pte_t pte = ptep_get_and_clear(mm, addr, ptep); - - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - - return orig_pte; -} - -static pte_t get_clear_contig_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pte_num) -{ - pte_t orig_pte = get_clear_contig(mm, addr, ptep, pte_num); - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - bool valid = !pte_none(orig_pte); - - if (valid) - flush_tlb_range(&vma, addr, addr + (PAGE_SIZE * pte_num)); - - return orig_pte; -} - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) { unsigned long order; @@ -173,21 +137,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, - pte_t *ptep) -{ - pte_t pte = ptep_get(ptep); - int pte_num; - - if (!pte_napot(pte)) - return ptep_clear_flush(vma, addr, ptep); - - pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL); - - return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); -} - static bool is_napot_size(unsigned long size) { unsigned long order; diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index 02bce0ed93d8..911b9cd4aa4d 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -31,4 +31,8 @@ extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, extern void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep); +#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH +extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index de505350ef48..d27c7599ce74 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -27,6 +27,7 @@ * - huge_ptep_get_and_clear() * - huge_ptep_set_access_flags() * - huge_ptep_set_wrprotect() + * - huge_ptep_clear_flush() */ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -254,3 +255,17 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, set_contptes(mm, addr, ptep, pte, ncontig, pgsize); } + +pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep) +{ + struct mm_struct *mm = vma->vm_mm; + size_t pgsize; + int ncontig; + + if (!pte_cont(__ptep_get(ptep))) + return ptep_clear_flush(vma, addr, ptep); + + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); + return get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); +}