From patchwork Fri Mar 21 13:06:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 14025407 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8750C36000 for ; Fri, 21 Mar 2025 13:08:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D3A5B280005; Fri, 21 Mar 2025 09:08:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CEA6A280001; Fri, 21 Mar 2025 09:08:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B8B63280005; Fri, 21 Mar 2025 09:08:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9B3CD280001 for ; Fri, 21 Mar 2025 09:08:44 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7F0D682114 for ; Fri, 21 Mar 2025 13:08:45 +0000 (UTC) X-FDA: 83245587810.26.C537ACD Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) by imf20.hostedemail.com (Postfix) with ESMTP id 925791C0036 for ; Fri, 21 Mar 2025 13:08:43 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=QUyKaCTO; dmarc=none; spf=pass (imf20.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.52 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742562523; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MHcTeoTN4G5JoOrCU+WuSXOyCDmNaouJRLSJHK0ciYw=; b=UvJn3z1S6mLloT+iWQcv34/uCqMG8X02GInSbzkyCBDxuWNFDl9uzvVvIVpf+GILL6P+nP SbiPaZtuuRcvudetIKgAoulBzQuSJbkVYLemq9adnL/DJvW18Okei4vvqBY8QtnzewhBpz pOtxp0FKd/CvAL9TA8e0Wzp9HE+bC98= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=QUyKaCTO; dmarc=none; spf=pass (imf20.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.52 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742562523; a=rsa-sha256; cv=none; b=uDl8qJ25JAE9lfxSpUXVLACMOYyJI6JXlol9e8XklQ6e4p81OVRhjrBwRJpU7UwPF38VjM NNjzv+rL6nZ03DHYgfvBRDjzMyBz1BBNGpVO+HFwRoSXr7FQibOnZcmtVEshW/vALF5fDd Gi1nSXfrZVnQRdC162O9ioV03XdSuoA= Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-43cf58eea0fso11034515e9.0 for ; Fri, 21 Mar 2025 06:08:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1742562522; x=1743167322; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MHcTeoTN4G5JoOrCU+WuSXOyCDmNaouJRLSJHK0ciYw=; b=QUyKaCTOLb1f+YV9yLx/9NQuPSnbMAHsSRy3xusAKqVM0xIBX+8kQOmr3BnaGX80qx W5QFe4uvWqjD8plzHU3vecbSEGMCq7jMQt5aaOyBoZYCfwhD7gphpPe8mKDq5/F9NOSm +6MgTEDdxLImZmAbN5S4oNMG7zmREF6Yscuwa3qPPe9kwVxmHUpbfJBEGMF9mhgFH+TJ fs0gB/vI35/VesektThi+LDnCy9BY9HDTzQzdHjAMEbZN1YHA7qoMW1tUCkMxe+Ct+MT 2TRq+KLCTVoeIeCwS2gYHhtgx30Wn+6wd3zUEWlXOj6Mwz0W2ESCM6brvl+ApSG0Hb7o fAUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742562522; x=1743167322; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MHcTeoTN4G5JoOrCU+WuSXOyCDmNaouJRLSJHK0ciYw=; b=txDiPw2KZtz4ZDrcab3BQUXc5w6oUAH9FoF7oIzuwkPBUdip9dRkSLwZ9N+7x1Ht42 F5y8xDDvbayjUxIiqm1d4t07sLEATJqRQEjTin7qM6jrCEH196yq3QnQZ8uOIu8dMJWM aZ/aaB/TRzq5IARYy8kEGdwrMA9J/ycQTmRZjkyZAKr6BT+lbOBRNqH0vkMbwKJiuNCZ Kt5gL3NkcqXXsh5vkmn+fAleYV/WKLMVvnzSVASfrJ9fqR2sCQUb7ns0vWOoW2SmFW3Q HV/9YyGa4+QlZ+OLc5A/9pdQDqG6GjSC4K0Pbw6xeUNWsoO8fB2fIiWHp0HI9Frmw1J9 5ofA== X-Forwarded-Encrypted: i=1; AJvYcCXUFfGVjSaq98ipWRAiTHchdP9hIq+GR6ZHbYjQFcXkl7sPeKZJqTDNKuJBfpIG1N7DP+OqZGZ2kA==@kvack.org X-Gm-Message-State: AOJu0YyEy1qHYPgsD8e91smlKl2DOonfhgYLnWvhXGgaHqqawotPvyEj VAX8cNPruQY7ZIGI7z05Yvg2gGPMz+CjvWRw4kQ1ihwHeI+y9L0vssGF5h10rTk= X-Gm-Gg: ASbGncsbAlJtfMkZuLSwE3ZdF6ToQyJOAATHHvwOUf19ol7soC+KzAUQU7iSaNjv2dY STLgn0hOLfLBm+/NR47kcirW/Us6qXw2EC4xecyFjwmKdHnQt+29l9us5bje3h53LBwNBKpR/s3 qp/aDHxCdG0uQU6Z9qEmL+9JJNnCQhFdUI5MT2DGvYMlC3M8j0+4yvItSrZbYa+K4JIp32j6+0i 8VIWppUfmnmrCJDODLbGaI0prCzvEr7Uj422EgDIaWFQtqABhCTIrPGrawvrsML2UEA8ouJBKS+ thNnnqbqQmwaSJuN02jIibI3e+3e5XL6JeJ+q9BcVxHFPj6gHSj6ta5MtoYtnyFThV95TqcSKKI wlXyx X-Google-Smtp-Source: AGHT+IEMJBrD5VHSiFntLl7eq4OXY9R2l0VvwmSKG3LcIwl+yD5imfUnEeI+G2u9jHEtl52hrfshfg== X-Received: by 2002:a05:600c:46d4:b0:43c:e6d1:efe7 with SMTP id 5b1f17b1804b1-43d50a3b62emr18553825e9.26.1742562521606; Fri, 21 Mar 2025 06:08:41 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3d12:52fe:c1cc:c94]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43d43fdec51sm78104015e9.27.2025.03.21.06.08.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Mar 2025 06:08:41 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Matthew Wilcox , Paul Walmsley , Palmer Dabbelt , Alexandre Ghiti , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v5 2/9] riscv: Restore the pfn in a NAPOT pte when manipulated by core mm code Date: Fri, 21 Mar 2025 14:06:28 +0100 Message-Id: <20250321130635.227011-3-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250321130635.227011-1-alexghiti@rivosinc.com> References: <20250321130635.227011-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 925791C0036 X-Stat-Signature: cd1etkgpapfkcdt3nkqa8z5gazerbc1f X-Rspam-User: X-HE-Tag: 1742562523-347164 X-HE-Meta: U2FsdGVkX19O0DDfUQUoYe8ZhJmJ32R7/aZbhTjlgUWPhR1iCpOlY7p3QRo9A4euZlsS6MKCxjqWE5cU48E7vslb2JJdt0bBO7ZGnVcOBj/VplsbyGUEWBvZNXmteiVknnz9EzdhGYJnXBARoX2eabpy1QOZYhfSnQLgPAryjOKZSGZx98DUZO8m4dx3JzItK6Nm1gGtCJUcIuiGqJ/n5hmxqqBEy37FomHcZ/bhyRo8yVewaBFDfi1df4SMERHTlWxJolvA4SSR9maIL4jATjXw7Dz1bHYheuqvarvmjtQytmwgPXIVyOESHwBf3s+W58lEh2VFkgQlqIpP/IM2OjWRlyeAdkOzt4xF1iWmagRuwGp79iT/JCrGApQ2q/H29acJfS+JrtEpmxoyqhIbadIp5BfOKXUH8aWneUfXi4pcpt9U17sgHMghoBeuzQUGVp8pdO0DgzQ8yZqTvYqzHTV9jEnRkE7bLt+/ZnREICTBmlzPGE9k5ssY8TBDgX6CQJRmi6hQqBeTygpk6a5mkL4qYsKxM0Je74x6c6OF9PUEYxConEaYTFxD7CAsGZK2nXVYlRB/qjKdsaFR3Ye3AHexMjGd4cEKeYvpYBJsf/W6rfn5kNUexSCMXZlxiWUJyHJ245YpREtVZIjvg4LO/oKR6ZzcNsow1gMLwZMD2Bb874blgE1Y3tW6DNKAP0MLNLzo/YljOEaXyM9y2Rmsd3MmNaDxMnCwHgPIyoI89JIcx9wAA7ZLlLBaQbEE9zt9CgN5gsXk4+hXJir4BMxKHGeAX6GWlQCfAhWlu9HRdAlYdp4278cCcZpq0zsm2zVJUuiLFiPLNiF4HwUq4q2e80MX7M+MbJySnXNmHSUehvFzvnIg1vzXD7fxEX6svqQ+AyFfp4oPVC6qISixyB04GZRBXBi4zppNAQsJbaA65nuajLU721lORjCPWzitm6IGuAj2AR3S0nHrosKPTg5 EE8/gssy TEgTMjgEGesNv1jVswltuXmmNkWa8NkBQZkrobGgXaF7FtQoqwhMLzxdmJoZZVrcJ5M3YTKttZisXmdAM7fqAxynOnMn863dyjPPbqRM51ZrauQPdNMbtYnUKpq36Li7dbZRYQ1qBMNnnLNBCcRdqzTG9RrFx4zd2HGaKCDAtBbV8PPhnVE02ObC25I/2mbFUMNLCPwBtvZSGCgZR64jS58GhzE2tg6oZSVxXyoAUANb+2zLv6gL6dmnhkI54iOAxu5976ao2e1Q1/fu0lvzfcDzovN5gCQIjvtw8vfD94aDAQ0T8OqdW+NmF2Fjas7nanSKjDqzK6Y1LR6bgRn4BU7W+CGC/lBMRuUlz+SJ2ci8dA2yrreUrNxVyaJuIzRBm5x3NpKNSpsskcjMjAN/1wkUkoH1tdLZhvsvPDRMiT26egkw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The core mm code expects to be able to extract the pfn from a pte. NAPOT mappings work differently since its ptes actually point to the first pfn of the mapping, the other bits being used to encode the size of the mapping. So modify ptep_get() so that it returns a pte value that contains the *real* pfn (which is then different from what the HW expects) and right before storing the ptes to the page table, reset the pfn LSBs to the size of the mapping. And make sure that all NAPOT mappings are set using set_ptes(). Signed-off-by: Alexandre Ghiti --- arch/riscv/include/asm/pgtable-64.h | 11 ++ arch/riscv/include/asm/pgtable.h | 155 ++++++++++++++++++++++++---- arch/riscv/mm/hugetlbpage.c | 15 ++- 3 files changed, 152 insertions(+), 29 deletions(-) diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h index 0897dd99ab8d..cddbe426f618 100644 --- a/arch/riscv/include/asm/pgtable-64.h +++ b/arch/riscv/include/asm/pgtable-64.h @@ -104,6 +104,17 @@ enum napot_cont_order { #define napot_cont_mask(order) (~(napot_cont_size(order) - 1UL)) #define napot_pte_num(order) BIT(order) +static inline bool is_napot_order(unsigned int order) +{ + unsigned int napot_order; + + for_each_napot_order(napot_order) + if (order == napot_order) + return true; + + return false; +} + #ifdef CONFIG_RISCV_ISA_SVNAPOT #define HUGE_MAX_HSTATE (2 + (NAPOT_ORDER_MAX - NAPOT_CONT_ORDER_BASE)) #else diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 050fdc49b5ad..2e62d7e607db 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -296,6 +296,17 @@ static inline unsigned long pte_napot(pte_t pte) return pte_val(pte) & _PAGE_NAPOT; } +#define pte_valid_napot(pte) (pte_present(pte) && pte_napot(pte)) + +/* + * contpte is what we expose to the core mm code, this is not exactly a napot + * mapping since the size is not encoded in the pfn yet. + */ +static inline pte_t pte_mkcont(pte_t pte) +{ + return __pte(pte_val(pte) | _PAGE_NAPOT); +} + static inline pte_t pte_mknapot(pte_t pte, unsigned int order) { int pos = order - 1 + _PAGE_PFN_SHIFT; @@ -305,6 +316,12 @@ static inline pte_t pte_mknapot(pte_t pte, unsigned int order) return __pte((pte_val(pte) & napot_mask) | napot_bit | _PAGE_NAPOT); } +/* pte at entry must *not* encode the mapping size in the pfn LSBs. */ +static inline pte_t pte_clear_napot(pte_t pte) +{ + return __pte(pte_val(pte) & ~_PAGE_NAPOT); +} + #else static __always_inline bool has_svnapot(void) { return false; } @@ -314,17 +331,24 @@ static inline unsigned long pte_napot(pte_t pte) return 0; } +static inline pte_t pte_clear_napot(pte_t pte) +{ + return pte; +} + +static inline pte_t pte_mknapot(pte_t pte, unsigned int order) +{ + return pte; +} + +#define pte_valid_napot(pte) false + #endif /* CONFIG_RISCV_ISA_SVNAPOT */ /* Yields the page frame number (PFN) of a page table entry */ static inline unsigned long pte_pfn(pte_t pte) { - unsigned long res = __page_val_to_pfn(pte_val(pte)); - - if (has_svnapot() && pte_napot(pte)) - res = res & (res - 1UL); - - return res; + return __page_val_to_pfn(pte_val(pte)); } #define pte_page(x) pfn_to_page(pte_pfn(x)) @@ -559,8 +583,13 @@ static inline void __set_pte_at(struct mm_struct *mm, pte_t *ptep, pte_t pteval) #define PFN_PTE_SHIFT _PAGE_PFN_SHIFT -static inline void set_ptes(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pteval, unsigned int nr) +static inline pte_t ___ptep_get(pte_t *ptep) +{ + return READ_ONCE(*ptep); +} + +static inline void ___set_ptes(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pteval, unsigned int nr) { page_table_check_ptes_set(mm, ptep, pteval, nr); @@ -569,10 +598,13 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr, if (--nr == 0) break; ptep++; + + if (unlikely(pte_valid_napot(pteval))) + continue; + pte_val(pteval) += 1 << _PAGE_PFN_SHIFT; } } -#define set_ptes set_ptes static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -587,17 +619,6 @@ extern int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addre extern int ptep_test_and_clear_young(struct vm_area_struct *vma, unsigned long address, pte_t *ptep); -#define __HAVE_ARCH_PTEP_GET_AND_CLEAR -static inline pte_t ptep_get_and_clear(struct mm_struct *mm, - unsigned long address, pte_t *ptep) -{ - pte_t pte = __pte(atomic_long_xchg((atomic_long_t *)ptep, 0)); - - page_table_check_pte_clear(mm, pte); - - return pte; -} - #define __HAVE_ARCH_PTEP_SET_WRPROTECT static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long address, pte_t *ptep) @@ -627,6 +648,100 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma, return ptep_test_and_clear_young(vma, address, ptep); } +#ifdef CONFIG_RISCV_ISA_SVNAPOT +static inline pte_t pte_napot_clear_pfn(pte_t *ptep, pte_t pte) +{ + /* + * The pte we load has the N bit set and the size of the mapping in + * the pfn LSBs: keep the N bit and replace the mapping size with + * the *real* pfn since the core mm code expects to find it there. + * The mapping size will be reset just before being written to the + * page table in set_ptes(). + */ + if (unlikely(pte_valid_napot(pte))) { + unsigned int order = napot_cont_order(pte); + int pos = order - 1 + _PAGE_PFN_SHIFT; + unsigned long napot_mask = ~GENMASK(pos, _PAGE_PFN_SHIFT); + pte_t *orig_ptep = PTR_ALIGN_DOWN(ptep, sizeof(*ptep) * napot_pte_num(order)); + + pte = __pte((pte_val(pte) & napot_mask) + ((ptep - orig_ptep) << _PAGE_PFN_SHIFT)); + } + + return pte; +} + +#define __HAVE_ARCH_PTEP_GET_AND_CLEAR +static inline pte_t ptep_get_and_clear(struct mm_struct *mm, + unsigned long address, pte_t *ptep) +{ + pte_t pte = __pte(atomic_long_xchg((atomic_long_t *)ptep, 0)); + + pte = pte_napot_clear_pfn(ptep, pte); + + page_table_check_pte_clear(mm, pte); + + return pte; +} + +static inline pte_t __ptep_get(pte_t *ptep) +{ + pte_t pte = ___ptep_get(ptep); + + return pte_napot_clear_pfn(ptep, pte); +} + +static inline void __set_ptes(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pteval, unsigned int nr) +{ + if (unlikely(pte_valid_napot(pteval))) { + unsigned int order = ilog2(nr); + + if (!is_napot_order(order)) { + /* + * Something's weird, we are given a NAPOT pte but the + * size of the mapping is not a known NAPOT mapping + * size, so clear the NAPOT bit and map this without + * NAPOT support: core mm only manipulates pte with the + * real pfn so we know the pte is valid without the N + * bit. + */ + pr_err("Incorrect NAPOT mapping, resetting.\n"); + pteval = pte_clear_napot(pteval); + } else { + /* + * NAPOT ptes that arrive here only have the N bit set + * and their pfn does not contain the mapping size, so + * set that here. + */ + pteval = pte_mknapot(pteval, order); + } + } + + ___set_ptes(mm, addr, ptep, pteval, nr); +} + +#define ptep_get __ptep_get +#define set_ptes __set_ptes +#define set_contptes(mm, addr, ptep, pte, nr, pgsize) \ + set_ptes(mm, addr, ptep, pte, nr) +#else +#define ptep_get ___ptep_get +#define __ptep_get ___ptep_get +#define set_ptes ___set_ptes +#define __set_ptes ___set_ptes + +#define __HAVE_ARCH_PTEP_GET_AND_CLEAR +static inline pte_t ptep_get_and_clear(struct mm_struct *mm, + unsigned long address, pte_t *ptep) +{ + pte_t pte = __pte(atomic_long_xchg((atomic_long_t *)ptep, 0)); + + page_table_check_pte_clear(mm, pte); + + return pte; +} +#endif /* CONFIG_RISCV_ISA_SVNAPOT */ + #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) { diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 3192ad804279..60b7e738b31a 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -190,7 +190,7 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) for_each_napot_order(order) { if (shift == napot_cont_shift(order)) { - entry = pte_mknapot(entry, order); + entry = pte_mkcont(entry); break; } } @@ -267,8 +267,7 @@ void set_huge_pte_at(struct mm_struct *mm, clear_flush(mm, addr, ptep, pgsize, pte_num); - for (i = 0; i < pte_num; i++, ptep++, addr += pgsize) - set_pte_at(mm, addr, ptep, pte); + set_ptes(mm, addr, ptep, pte, pte_num); } int huge_ptep_set_access_flags(struct vm_area_struct *vma, @@ -280,7 +279,7 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, struct mm_struct *mm = vma->vm_mm; unsigned long order; pte_t orig_pte; - int i, pte_num; + int pte_num; if (!pte_napot(pte)) return ptep_set_access_flags(vma, addr, ptep, pte, dirty); @@ -295,8 +294,7 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, if (pte_young(orig_pte)) pte = pte_mkyoung(pte); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) - set_pte_at(mm, addr, ptep, pte); + set_ptes(mm, addr, ptep, pte, pte_num); return true; } @@ -324,7 +322,7 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, pte_t pte = ptep_get(ptep); unsigned long order; pte_t orig_pte; - int i, pte_num; + int pte_num; if (!pte_napot(pte)) { ptep_set_wrprotect(mm, addr, ptep); @@ -337,8 +335,7 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, orig_pte = pte_wrprotect(orig_pte); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) - set_pte_at(mm, addr, ptep, orig_pte); + set_ptes(mm, addr, ptep, orig_pte, pte_num); } pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,