From patchwork Wed May 8 11:34:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13658596 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 261E2C19F4F for ; Wed, 8 May 2024 11:35:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B64146B0147; Wed, 8 May 2024 07:35:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AEC6C6B014B; Wed, 8 May 2024 07:35:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 93F5E6B014D; Wed, 8 May 2024 07:35:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 7221F6B0147 for ; Wed, 8 May 2024 07:35:29 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 30DF51610B3 for ; Wed, 8 May 2024 11:35:29 +0000 (UTC) X-FDA: 82095023178.03.12D62E0 Received: from mail-lj1-f174.google.com (mail-lj1-f174.google.com [209.85.208.174]) by imf04.hostedemail.com (Postfix) with ESMTP id 3A0F14000C for ; Wed, 8 May 2024 11:35:26 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=TQNgVPo0; dmarc=none; spf=pass (imf04.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.174 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715168127; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eptZ1TC6/9ugjFjSR/N9yJ6dEI/3FvKeew1j70YJt7s=; b=AKBZf0Ys+Ugjdnyuf+x/zMoDFNJh2BRK9xYa9ofzTlqCcO7PwShqBqgOr/xbQAS+VwNntz GHMZX3Cu6BxrBd8WcXo6/mFLOMoDvq1MrRpKr4DKOgTSiu0Tcl3aqni5Xr0menDRJlbb2K xrY5zJ9740lZJc77IKPWH/PMZviuguM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715168127; a=rsa-sha256; cv=none; b=GaJDerLdgj9TVXFWv6AFCBLPZuaWVKKgK9/1ee4ADFpwaEjNQWGAdcNshy9ZJip9/F7wwV HlRDRnBJmi1UjXeWvTq5vfbcdldOPJN3QKt1aWHBHNd5/8L+zRpOeSpXqPmw6XEL6/6ze4 XAFyD+CuHXKcFJcytAf9A4HUUuUw4e0= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=TQNgVPo0; dmarc=none; spf=pass (imf04.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.174 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com Received: by mail-lj1-f174.google.com with SMTP id 38308e7fff4ca-2e43c481b08so16501371fa.1 for ; Wed, 08 May 2024 04:35:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1715168125; x=1715772925; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eptZ1TC6/9ugjFjSR/N9yJ6dEI/3FvKeew1j70YJt7s=; b=TQNgVPo0rKBgCE1dD0+jvA8X5fpzaOidTs2wUbJ5b1HUu5bo3yFGeoIAzvTPAI15/s AzQelXXbu1Lkg5YPK+T6ztGWkHUEaqiAHhOd5RzSe/lfWEq63oNFRt8MVqsj5kzCMEht UBIgxqQl3ni3NsuSN94JA0fAWjeMt3FdvYIhJYCbt/vN3i/ByPuZghcpWouRDpM2wqXM Sb4Q1TN5Nfr6ULROtvGGOHE7cpbJSCnbEx1ei66Zgj8PdMC0cCPH5X7VzNvE4m2Mp9IG 8QtYFzYKVevCUfJV14kO31SpS+CpG97xjkvzGVOECEA186sY7+8PiP49JobLKcaRy0fK 58Hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715168125; x=1715772925; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eptZ1TC6/9ugjFjSR/N9yJ6dEI/3FvKeew1j70YJt7s=; b=f69bS+ahufqr5LhhmEPmCzUqpqf/mMkxyAr5TAba6tnL7ZG0UNSCvKRm4NT7AZLrfG yrWR2JwVeq+IVqeodyJM7hSlT7Pzxzbs5LwmHOmAJaICl6ypzd5bC5p0Bb55foxH0uRs S4FBgdF7ZtcGpghq/zN5Eu04B16PlQZdexjCZNiWdI7Zkt77Y9+QC1UYssZyAF+mHGW4 lCJNulMz9IT6Jfcb1MYqNvUMfkO9JapPo4O/DmypIpqNF0va/m4ztfhDeAdG7mxvK7ik ftWftdsuZXP3F8jLQ58mIyUKeI1uzoIenBeO77YRB3Ju9mhOvbHqDhnxxIsseTAI+BvX 2odA== X-Forwarded-Encrypted: i=1; AJvYcCXU7kPjR/weqzKBrOR+r3MqqzU9hGx1CdYlQtEC4YNXL6zifxJAmKL+gK0GGP/mNvYam8Ox2VyleeRTD0Ma7xWNfPc= X-Gm-Message-State: AOJu0YwCY6zL1OfSt8Z4Y3eMY4xLKMCHpez7UzZMYN+P7MciRYAmeKmC YRKiBEk9jJlX7yugnZp18G6Egup6eeKNDhoReBIu2NjjIz0L8PGokHN/aHDqmCU= X-Google-Smtp-Source: AGHT+IH3DnJuCq7AQ0FZHFVm1srR1I6oMvrSfB9MDxvIjK3/BYLldiZjiTW9K6zK+rtoMInnMaJsqQ== X-Received: by 2002:a05:6512:12c3:b0:520:9775:5d4b with SMTP id 2adb3069b0e04-5217c3714demr2111385e87.13.1715168125494; Wed, 08 May 2024 04:35:25 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-41facbd295fsm1760585e9.36.2024.05.08.04.35.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 May 2024 04:35:25 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH RESEND v2 1/9] riscv: Restore the pfn in a NAPOT pte when manipulated by core mm code Date: Wed, 8 May 2024 13:34:11 +0200 Message-Id: <20240508113419.18620-2-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240508113419.18620-1-alexghiti@rivosinc.com> References: <20240508113419.18620-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 3A0F14000C X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: u9zawetfj6mfhha4o7i6dhym9b6rdi8f X-HE-Tag: 1715168126-270025 X-HE-Meta: U2FsdGVkX19heAZ1FjOwm+2GPPNOKxGusfGHEuN59Oq/QcC2P3Hd2pilHgcsYhhBwIXVPR8Wn4xdZjmEsf88kDP/Tb1tQtl9fQY3NCY8g9NH4sCbyxlGvb1K7RB1GOpHYSKwQkYsqA0cnjEPQmKPlUyV9EjbqX/9VAw0kDv2xJK15OsTfCloLU+hx0uLFz0FmkD0lS+LulXxkbjRI9+3F41ni/zoPn8idjef+llJJ/8WN6Fi+CNYQNg2g7SYMg0oEfBGA9HjL9x1YPDyj6q6DUgwNtvMoT+zMd49/e8PXfLr4XTc72SwAaUZ4lQrtdJbx3oZc6YwGkPHTklRFcRGgZjSEADEt8+PH45e1owxXgFUGSKwXk4bBsTcSnZ4T+QiL77OHWd6CEGlzLAohEkTD9woLNIZh4+B0r+qhunJ6MWP2u8yajrB177n4Kr/oOczJHFJyczv14fRjjqnd5EY0cmgcqeDMEQ66ktmZClrgRHKpaHB2M+cb0ZSrylxpk+fTkPdcO9QEuSWZBLJPlrkMMVNpzuMfTk8itoPB8Kbns0uahgufYH3JSoYZMLfcA8/W02uLsitCJnWNQSNx5+EyfL7g21pVWFuUexAhUMkMP1GWROkhvH4RCoChSQtjkM+RVC4goIk1PvOHuurwYHTzRyT/aR0WsvrrRe3vKhG/kJPPfZJz1rypoe9oNvQz9abEU6zp1/C/ydUrM+DDdPfb6AsM8T7cIsT0qN1OiIwZlj+qOd7ZNHoPlbvz6npGLEaWoKFrcrnJs3ulQ7+Pcc9TeQoGt6U81SySoA6328zLWAe6GGcN93EorcgF4TTR6KZl3l19k2OhNhfC6BvVqJ9R0ojkIYAFuhzH4z2XHEmBI2lTG3pBqm3/ZmX5dy6syOutDL1xlewKr4qrJpgIlmL4eGv+JZSorpwrawDS3csHqIf/uXZqDbwo19L3yWcmxWkdpA98K8POPy1oAfJyOZ VuGLo9eW 37wXtunj0X6XiEMYvAAURmHTpHXzeG/WFVlOc5LeEUKAZs1n6WfG5FJCUUtwbHgRbuRx8dg6rqhxHSigicCIkdbj+J7sWCJ09Qb5gBezgZhOnATLOXxNRjRldLpSTFlxY5wqhjGBpgFiy+6/3fsGUB+Dvxq/ttMh8habHovAmouwk4yd9ZpSVNKWGfFfRpTCcn8a2t76cuQX7b/8uZDaOhYNViAYvvkj+VZmgL9foXcoSO0pAT5nKrXYb44sSyNvekjsZvGd0/xx4C+lTl9giNDH3DCs82cnFisOyWSwx392J5zt/nCH4prUL6rfmYhiXFRQTaGg42J7RDSZVJMD6t/B96lp1H7LtMfCUIuynrUvs5my4Z/CtiEs5ESO+myACGHDus3BgDt0FAUxdxeikKIqHrTdGjH+CV8W35MJUS0+hUuClcrvQdqYTO7NpJHmYqTER X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The core mm code expects to be able to extract the pfn from a pte. NAPOT mappings work differently since its ptes actually point to the first pfn of the mapping, the other bits being used to encode the size of the mapping. So modify ptep_get() so that it returns a pte value that contains the *real* pfn (which is then different from what the HW expects) and right before storing the ptes to the page table, reset the pfn LSBs to the size of the mapping. And make sure that all NAPOT mappings are set using set_ptes(). Signed-off-by: Alexandre Ghiti --- arch/riscv/include/asm/pgtable-64.h | 11 +++ arch/riscv/include/asm/pgtable.h | 105 ++++++++++++++++++++++++++-- arch/riscv/mm/hugetlbpage.c | 38 +++++----- 3 files changed, 128 insertions(+), 26 deletions(-) diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h index 221a5c1ee287..9fe076fc503e 100644 --- a/arch/riscv/include/asm/pgtable-64.h +++ b/arch/riscv/include/asm/pgtable-64.h @@ -106,6 +106,17 @@ enum napot_cont_order { #define napot_cont_mask(order) (~(napot_cont_size(order) - 1UL)) #define napot_pte_num(order) BIT(order) +static inline bool is_napot_order(unsigned int order) +{ + unsigned int napot_order; + + for_each_napot_order(napot_order) + if (order == napot_order) + return true; + + return false; +} + #ifdef CONFIG_RISCV_ISA_SVNAPOT #define HUGE_MAX_HSTATE (2 + (NAPOT_ORDER_MAX - NAPOT_CONT_ORDER_BASE)) #else diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 9f8ea0e33eb1..268c828f5152 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -297,6 +297,8 @@ static inline unsigned long pte_napot(pte_t pte) return pte_val(pte) & _PAGE_NAPOT; } +#define pte_valid_napot(pte) (pte_present(pte) && pte_napot(pte)) + static inline pte_t pte_mknapot(pte_t pte, unsigned int order) { int pos = order - 1 + _PAGE_PFN_SHIFT; @@ -306,6 +308,12 @@ static inline pte_t pte_mknapot(pte_t pte, unsigned int order) return __pte((pte_val(pte) & napot_mask) | napot_bit | _PAGE_NAPOT); } +/* pte at entry must *not* encode the mapping size in the pfn LSBs. */ +static inline pte_t pte_clear_napot(pte_t pte) +{ + return __pte(pte_val(pte) & ~_PAGE_NAPOT); +} + #else static __always_inline bool has_svnapot(void) { return false; } @@ -315,17 +323,14 @@ static inline unsigned long pte_napot(pte_t pte) return 0; } +#define pte_valid_napot(pte) false + #endif /* CONFIG_RISCV_ISA_SVNAPOT */ /* Yields the page frame number (PFN) of a page table entry */ static inline unsigned long pte_pfn(pte_t pte) { - unsigned long res = __page_val_to_pfn(pte_val(pte)); - - if (has_svnapot() && pte_napot(pte)) - res = res & (res - 1UL); - - return res; + return __page_val_to_pfn(pte_val(pte)); } #define pte_page(x) pfn_to_page(pte_pfn(x)) @@ -525,9 +530,91 @@ static inline void __set_pte_at(struct mm_struct *mm, pte_t *ptep, pte_t pteval) #define PFN_PTE_SHIFT _PAGE_PFN_SHIFT +#ifdef CONFIG_RISCV_ISA_SVNAPOT +static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, + size_t *pgsize) +{ + pte_t __pte; + + /* We must read the raw value of the pte to get the size of the mapping */ + __pte = READ_ONCE(*ptep); + + if (pgsize) { + if (size >= PGDIR_SIZE) + *pgsize = PGDIR_SIZE; + else if (size >= P4D_SIZE) + *pgsize = P4D_SIZE; + else if (size >= PUD_SIZE) + *pgsize = PUD_SIZE; + else if (size >= PMD_SIZE) + *pgsize = PMD_SIZE; + else + *pgsize = PAGE_SIZE; + } + + /* Make sure __pte is not a swap entry */ + if (pte_valid_napot(__pte)) + return napot_pte_num(napot_cont_order(__pte)); + + return 1; +} +#endif + +static inline pte_t ptep_get(pte_t *ptep) +{ + pte_t pte = READ_ONCE(*ptep); + +#ifdef CONFIG_RISCV_ISA_SVNAPOT + /* + * The pte we load has the N bit set and the size of the mapping in + * the pfn LSBs: keep the N bit and replace the mapping size with + * the *real* pfn since the core mm code expects to find it there. + * The mapping size will be reset just before being written to the + * page table in set_ptes(). + */ + if (unlikely(pte_valid_napot(pte))) { + unsigned int order = napot_cont_order(pte); + int pos = order - 1 + _PAGE_PFN_SHIFT; + unsigned long napot_mask = ~GENMASK(pos, _PAGE_PFN_SHIFT); + pte_t *orig_ptep = PTR_ALIGN_DOWN(ptep, sizeof(*ptep) * napot_pte_num(order)); + + pte = __pte((pte_val(pte) & napot_mask) + ((ptep - orig_ptep) << _PAGE_PFN_SHIFT)); + } +#endif + + return pte; +} +#define ptep_get ptep_get + static inline void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pteval, unsigned int nr) { +#ifdef CONFIG_RISCV_ISA_SVNAPOT + if (unlikely(pte_valid_napot(pteval))) { + unsigned int order = ilog2(nr); + + if (!is_napot_order(order)) { + /* + * Something's weird, we are given a NAPOT pte but the + * size of the mapping is not a known NAPOT mapping + * size, so clear the NAPOT bit and map this without + * NAPOT support: core mm only manipulates pte with the + * real pfn so we know the pte is valid without the N + * bit. + */ + pr_err("Incorrect NAPOT mapping, resetting.\n"); + pteval = pte_clear_napot(pteval); + } else { + /* + * NAPOT ptes that arrive here only have the N bit set + * and their pfn does not contain the mapping size, so + * set that here. + */ + pteval = pte_mknapot(pteval, order); + } + } +#endif + page_table_check_ptes_set(mm, ptep, pteval, nr); for (;;) { @@ -535,6 +622,12 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr, if (--nr == 0) break; ptep++; + +#ifdef CONFIG_RISCV_ISA_SVNAPOT + if (unlikely(pte_valid_napot(pteval))) + continue; +#endif + pte_val(pteval) += 1 << _PAGE_PFN_SHIFT; } } diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 5ef2a6891158..fe8067ee71b4 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -256,8 +256,7 @@ void set_huge_pte_at(struct mm_struct *mm, clear_flush(mm, addr, ptep, pgsize, pte_num); - for (i = 0; i < pte_num; i++, ptep++, addr += pgsize) - set_pte_at(mm, addr, ptep, pte); + set_ptes(mm, addr, ptep, pte, pte_num); } int huge_ptep_set_access_flags(struct vm_area_struct *vma, @@ -267,16 +266,16 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, int dirty) { struct mm_struct *mm = vma->vm_mm; - unsigned long order; + size_t pgsize; pte_t orig_pte; - int i, pte_num; + int pte_num; if (!pte_napot(pte)) return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - order = napot_cont_order(pte); - pte_num = napot_pte_num(order); - ptep = huge_pte_offset(mm, addr, napot_cont_size(order)); + pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); + ptep = huge_pte_offset(mm, addr, pte_num * pgsize); + orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); if (pte_dirty(orig_pte)) @@ -285,8 +284,7 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, if (pte_young(orig_pte)) pte = pte_mkyoung(pte); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) - set_pte_at(mm, addr, ptep, pte); + set_ptes(mm, addr, ptep, pte, pte_num); return true; } @@ -301,7 +299,7 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm, if (!pte_napot(orig_pte)) return ptep_get_and_clear(mm, addr, ptep); - pte_num = napot_pte_num(napot_cont_order(orig_pte)); + pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); return get_clear_contig(mm, addr, ptep, pte_num); } @@ -311,24 +309,23 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, pte_t *ptep) { pte_t pte = ptep_get(ptep); - unsigned long order; + size_t pgsize; pte_t orig_pte; - int i, pte_num; + int pte_num; if (!pte_napot(pte)) { ptep_set_wrprotect(mm, addr, ptep); return; } - order = napot_cont_order(pte); - pte_num = napot_pte_num(order); - ptep = huge_pte_offset(mm, addr, napot_cont_size(order)); + pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); + ptep = huge_pte_offset(mm, addr, pte_num * pgsize); + orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); orig_pte = pte_wrprotect(orig_pte); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) - set_pte_at(mm, addr, ptep, orig_pte); + set_ptes(mm, addr, ptep, orig_pte, pte_num); } pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, @@ -341,7 +338,7 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, if (!pte_napot(pte)) return ptep_clear_flush(vma, addr, ptep); - pte_num = napot_pte_num(napot_cont_order(pte)); + pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); } @@ -351,6 +348,7 @@ void huge_pte_clear(struct mm_struct *mm, pte_t *ptep, unsigned long sz) { + size_t pgsize; pte_t pte = ptep_get(ptep); int i, pte_num; @@ -359,8 +357,8 @@ void huge_pte_clear(struct mm_struct *mm, return; } - pte_num = napot_pte_num(napot_cont_order(pte)); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) + pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); + for (i = 0; i < pte_num; i++, addr += pgsize, ptep++) pte_clear(mm, addr, ptep); } From patchwork Wed May 8 11:34:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13658602 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBC00C04FFE for ; Wed, 8 May 2024 11:36:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6A9026B014E; Wed, 8 May 2024 07:36:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 632366B014F; Wed, 8 May 2024 07:36:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4D3646B0150; Wed, 8 May 2024 07:36:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 2A01B6B014E for ; Wed, 8 May 2024 07:36:30 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D8E96141465 for ; Wed, 8 May 2024 11:36:29 +0000 (UTC) X-FDA: 82095025698.12.B13501C Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) by imf18.hostedemail.com (Postfix) with ESMTP id EF9AD1C000B for ; Wed, 8 May 2024 11:36:27 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=ZkVQwMzY; dmarc=none; spf=pass (imf18.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.49 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715168188; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oM+WuwVvF178uIr+5Dc8IcJyYSbzDoUf3sQGwgQ7LEI=; b=YjF53pcaj/yGjGTpjAVQTr7MgOlbB8tP76+bW2/H4mKmolq/VvMfcrQhMpWANZ/nj1OSCV TThNYduXLe81jJkwlgjYW3wDfpLwW/GB/pYKXjbKIyIN2kwe2VdQEeKUptn9IXP/nZqle0 WXDjqKa5vMeHfcV7gIqaGV56tKQjvF0= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=ZkVQwMzY; dmarc=none; spf=pass (imf18.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.49 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715168188; a=rsa-sha256; cv=none; b=2HLRqoeUBQODC/F5aJb6LdCvcp6s5oHJHmvnxt8SUGUXR+GqkTK1lLtTcmJm6N7OeF7wqN pDns1DeBMmlpN5dvYs3RHe8MbscheKCn0eE3YHCMpYWx3eW61qri6EbxuEhKfGKMm6iJRx PE8Pk6tWTGeiKRZO7R3BtNuG+5GRqaA= Received: by mail-wr1-f49.google.com with SMTP id ffacd0b85a97d-34d7a32bdd3so2263735f8f.0 for ; Wed, 08 May 2024 04:36:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1715168186; x=1715772986; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oM+WuwVvF178uIr+5Dc8IcJyYSbzDoUf3sQGwgQ7LEI=; b=ZkVQwMzYb2OShs4hxPokEjopWEZOnpYLrZPd8QeuaDgPWhdvKJ5ufQDMHUpmXJuJlg 9PoRmQX9IUW8d+lV+sMdIRRbnbcakGlaSM5mCQ4TUCumN9uiPEJd5DgMAx8xVtOzE3mS fg61ibfckRc4+im4LaVOApRJXUo4aRa+6UCUjWeghPin5+bVsv5OLp/IXIWe/7PsXyBn QgsKjoy6vzqIuDYUQcQUlZ5cj8d3OfTj95+GomqT2i5G1DICT1jUNnbKNzAve1L4oL2G yd+LXBiuGi2lnTCUET2MMYN0+DDC6vJhalCUDiIeRXdWSLTzD3MWQDqDoKxhYsjRSpSd LrGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715168186; x=1715772986; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oM+WuwVvF178uIr+5Dc8IcJyYSbzDoUf3sQGwgQ7LEI=; b=lOPhP2WqSbUkEIa1bPP5Ew17/MmI4rTkAeScBjRy7bfrcUWSpukv+FszCCpPKnHBn6 06bRZHQ1oGlVIy/2Y7zYqQWW9XVEVtAbOaqQymKLyQ08/Dm3zNH7Nhk6/e3ybVVDJXrp dylzTCU6hOA+IYE6BtTgdiDL+Kvgmh2+OcNsUjQf8reUeQF/0/amoG1y+Ul9KH3BJsEP jI9f23VFGWuSHO5m+ucBTIcGY0NejomoYfMj1AQ4mCRCOHjgFZufuN+LFq66eZCxFpuV EyB6tdKtjydf9JrHfYe+OWKm8L8RRhprv80LG0WXyf6e1IK2lpE2jPeuu9qq4vePiG/d TuLw== X-Forwarded-Encrypted: i=1; AJvYcCXeujz+KTddTSHH7CeImLDpyS9EO0DYnkIC6cX/KX0PKGUKe1UxiRRkYpU+n2IDoCeX1cTePIEWnlC/lVN9VLfpzMg= X-Gm-Message-State: AOJu0YxvIZJSueRD/vlOxJpi5aOttLUCX23t3V1oA0QRYifTP08qY7jg DVNMmU6QdzqRBocbTGHImZcOIrYyNUWhJpsa2WVSW+BlcG/7H4z/ZsTOHEeO0Gs= X-Google-Smtp-Source: AGHT+IHVO5vcmWKipbJIGDQ+CDJ8RhNP/JYjB5jDnNbYNNxR9pALzbWj/fP/sjh1jhNLJi0qhKSTGw== X-Received: by 2002:a05:6000:508:b0:34c:77bd:2545 with SMTP id ffacd0b85a97d-34fca810737mr1911157f8f.70.1715168186464; Wed, 08 May 2024 04:36:26 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id dn15-20020a0560000c0f00b0034d9012e74bsm15283014wrb.72.2024.05.08.04.36.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 May 2024 04:36:26 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH RESEND v2 2/9] riscv: Safely remove huge_pte_offset() when manipulating NAPOT ptes Date: Wed, 8 May 2024 13:34:12 +0200 Message-Id: <20240508113419.18620-3-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240508113419.18620-1-alexghiti@rivosinc.com> References: <20240508113419.18620-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: EF9AD1C000B X-Stat-Signature: hz8r6jzth1i634nhdr1f9upgow9ptdff X-HE-Tag: 1715168187-416205 X-HE-Meta: U2FsdGVkX1/WaU7SMAPGGIm8oUZVjoz1U8wo75n4AqzhYz7pEiDspRTJvYtRYnTch7h7e373qONzqjHEyJl7TYnYHIbBLD0H0s5Sud5SmoZCMv0XbcXoAWsrmviWUgJhf39QHQbKAum+eh1dQbUGyE/JtnZmKCKnqMzjCK4VDNloYUUffKEjCvHNsHFdiQPxAqnF1KfmreYB0ijtDEHw+73YlEmS4k28QWFUMfhPqni+7eUUMIP6O6FQqRL2jpvdye96FNwkon84+jErx8oyS0A/QyjjISI3lWX0m9tFFfFrrcHtNDaU1K0QshTZkfvPrKib+b4Q7A4Nc6NyR28ecZlEHJzwIQ5WeDwKzh4xub6+i9kHITB4XZCIRtEJW+eBA9q3hSPe9nKKkCYu+01HVp0R5f6MG4qQdH1axyPsqTv/1QMP44+CQeolBB9TItRv+ot+BER+fYlZHf7K8/dcILfsLr7GNYCbNOvWkY8lMJUx5X68WUnyzIg40qp6/Jy4Uxp38ybSrIsAFmZuNSukAbUymy0FacJVqIiTTIiYwokNGYPUVXJQXLEFqK53zsHy2PDtTc/d84RceSjW9lUU/Wvbmo0iFkBmDvvsrLPP/rFhFlD54IfF4B+Gu2nQKDqmNSpnRFqMuyS5eS0EafpVC3KsvW5dXGDpMm4UngBRtO5XaO9GzTPfrEl0k+uyIDx4JL3wKOTER2VRT0ZuSg8vQd01Z8dM1lzYT80XiMvdCBICSd5MtSumegxJVKSa9fhGVerfnecjhbN4CQc/qNhzaSDfK3AQW+nA7tpwhT9KOikhiRHWt8NGWChrAbfP79S9llVmbHvvOWGmBTWXXRN1540wDBLu1BFDzIZfh0evaTt9ehcEBy/EgKozu6pWnqm9j+IMhrzUPVUP0a6rp7WMzvyALrRboPGDPCwp6RUx4d331gnxCtNel5Cam+8L9ssJgxxoEs+xRSSKPCX1GdP 5IeLWo4z DroFDnOsv6fDt3hMTmjr3OhD8g705bYhuJySkCceFx9wAGjscbQvk2K8HRzmaW2qnWnjUxhDBbwTqL3fIibkqf54+62eoiYk+kv4nv4vy9XijvlUQV4FtTYnXR1VMUNukcD1388lV/pOgeyEuBDZx94isaS2UBoOqnP6dBrGg/shRLq/R63mld6PA3CRgPrd78Q8khopPy3KgHCuPp69OCnPZMxMO9IUdFsXqDUiRhNwEb2x9ob6UWa9hG6uTJB+eSg9qT1GXNwSK+eZV2auwjqPWk0UpvSgMROaNPnF5BpS8jz6wQok745vOH3q1heY5l41+ne2x+SMDqpTInsce1msV6a+dKvkcQBCEYrZXhB9vbgZIMAspnDePidVAzZzegtKuqCr5yg58h3kBgRgI/4/uEoILr3S/n6kqGsPO6s6qSTPb7+1sjsJ4BOm4iNLd6ic39GHOIl5ImNQPBkAfM7/AxaMWBbhrT+cDkuHP/JeTgQLda8LoyMiATw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The pte_t pointer is expected to point to the first entry of the NAPOT mapping so no need to use huge_pte_offset(), similarly to what is done in arm64. Signed-off-by: Alexandre Ghiti --- arch/riscv/mm/hugetlbpage.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index fe8067ee71b4..f042f5c8bdb7 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -274,7 +274,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, return ptep_set_access_flags(vma, addr, ptep, pte, dirty); pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); - ptep = huge_pte_offset(mm, addr, pte_num * pgsize); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); @@ -319,10 +318,8 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, } pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); - ptep = huge_pte_offset(mm, addr, pte_num * pgsize); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); - orig_pte = pte_wrprotect(orig_pte); set_ptes(mm, addr, ptep, orig_pte, pte_num); From patchwork Wed May 8 11:34:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13658603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 486E9C04FFE for ; Wed, 8 May 2024 11:37:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CC48C6B0151; Wed, 8 May 2024 07:37:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C4D666B0153; Wed, 8 May 2024 07:37:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A9FDC6B0154; Wed, 8 May 2024 07:37:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 8DFD36B0151 for ; Wed, 8 May 2024 07:37:31 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4DEADA23DF for ; Wed, 8 May 2024 11:37:31 +0000 (UTC) X-FDA: 82095028302.06.4F95B20 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) by imf29.hostedemail.com (Postfix) with ESMTP id 60C18120016 for ; Wed, 8 May 2024 11:37:29 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="SeklHJb/"; spf=pass (imf29.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.52 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715168249; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YO7yDIQr+QJFeAZFjf5GxhWsbGb5WjqA2EGZTufF1Fc=; b=Q24sGOP2fJYCpzW2JJsak84qmoWCDodu/cM5tfpVlzaol3PAROX3BIv8813BIDcEdA+wWz MIAdjy1sZVYArVpO/o+leu5Ozx5S16WFKgPhCw0HaGfSObEBgGdb4N7vv+8dSm0S/Bqax/ ZQ++fAg3YZw0OYtvVA//Ec6NWTK4rDg= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="SeklHJb/"; spf=pass (imf29.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.52 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715168249; a=rsa-sha256; cv=none; b=g6TbaYVca0OSuvEC+mzHHcD/sw1cxKyLjcQEBFXE8Gg8G0dt+rpkkLsuoYhcEngmxWYONc NGtFlNXVo1NiMidvI7HOUGiEsCFkZoH/gWNTrMX+tyBydcWFCn2fSlnL1VxbCoC7cZHVou +4Bm59QAS29R7j1LwB3/f4JYVcZCfYs= Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-41b79451145so31762675e9.3 for ; Wed, 08 May 2024 04:37:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1715168248; x=1715773048; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YO7yDIQr+QJFeAZFjf5GxhWsbGb5WjqA2EGZTufF1Fc=; b=SeklHJb/wqUbhtc3iYe8cnBbjaA3vsuSBKfBVyLRa4hMIoLRHigwXzA5FKILOI9q09 znTsfyZ7pHyPFikBtJ7KXHtUg/Bn/0rrg1S82DI3oS/BL780emlYaFgmHE42EHXbvY1V kkSZB/D6fqXBkRD0e2s8rKdxF+TUVqS94x84dlxIY7DNbj8fLltZ6agkPLT7o8fMMUP8 8l38RM9XV1omsQpPJBUpTkccM6/maQBBqv7PfWyWY+Q6eVglXg5BUAbcyMFfjT1vN3KI I/p8OqAxx7nqQF8BCny1aHNEbrFF+q+ZHUWrPPiC1baWL9ruToDBslxFROLnCP38+ZKV LArg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715168248; x=1715773048; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YO7yDIQr+QJFeAZFjf5GxhWsbGb5WjqA2EGZTufF1Fc=; b=Rwht8qZ9hJ+o2X2km4wY+XiCCiZOoQJNsqvMRUgBDejAi1v8GnjFGk7alR1qvUU6PQ DXvbnwBeOoAsoy77CUgoZDys6ZsAdlxTTq1X4aCHcwlVl2hNKhYl/tVHfK0/yRuqXnN3 FuTzlNDj46yJca2S0OOlGQmoyuBN/3hVQh+ODCfszFZEaH7nZwrwydkm4In4KkNeKRJd siCb1DHuBsWibXvItMVam/FbiqFScne7mnvj8AFoLA1YrcRDpDfVSFf6oWyjaCfrYf0K kCOKpzPTAh+nJFNpyNYn2FeG4A1rOAW5EDpkj3yLEceNKku6NzbJJ+j+sgrowri6m0uH Uv2Q== X-Forwarded-Encrypted: i=1; AJvYcCW1hIVI9Z35H9yADrVFqzcoArzrZOrTzSz064h0/0mbk/Ne4lkIL6H5rESf6KvpF5BjzL10ze7WcJyjM0ONGA5o/T4= X-Gm-Message-State: AOJu0YyMoCiMuUr+kBPr5d9nQsuYug5VYJ5cIwrFUqerm5vrSBqvne9B 6sAoGxbPGL9Xy0X05wMOx4BUloZTMtFlJ48DCx0tkMSt1ELiqWujWC/pEliLdvo= X-Google-Smtp-Source: AGHT+IEExaDEn0vN6E4Lxk1Zj77Ju6cFib9bZTV4G4rk0yK8V1Cj9z0Mbupc0fzpXub0lh02TBpHRw== X-Received: by 2002:a05:600c:4f0d:b0:41a:7bbe:c3bf with SMTP id 5b1f17b1804b1-41f714f717bmr19142655e9.22.1715168247871; Wed, 08 May 2024 04:37:27 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id dd3-20020a0560001e8300b0034c9f060a14sm15085210wrb.11.2024.05.08.04.37.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 May 2024 04:37:27 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH RESEND v2 3/9] mm: Use common huge_ptep_get() function for riscv/arm64 Date: Wed, 8 May 2024 13:34:13 +0200 Message-Id: <20240508113419.18620-4-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240508113419.18620-1-alexghiti@rivosinc.com> References: <20240508113419.18620-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Stat-Signature: is3yfdcsfcdpkhb663wbhxhaqyj9ymnw X-Rspam-User: X-Rspamd-Queue-Id: 60C18120016 X-Rspamd-Server: rspam05 X-HE-Tag: 1715168249-662590 X-HE-Meta: U2FsdGVkX1+m2elhzN3CokjK7ZYyH/o+7Re0BnrxzbC9OKmMo3ZcWiMAF9F2rla/F+PV+mJeXeVJQ2+YPuG/UzuSvf5jWH+a7/eogzi+fEIxkjO43ZLBS0Qilqc2jOao68u6agpySe8o0OwjFDriLF8S/+8sCxs6iDI64x4EAxuAeUrcVLbXVYIufkiTnwbZcq/xYpY00xiFuEdzdpoCBfEvuN76MK+MNW+HPfohQh56AhnIUauz9bnhsvEquVoHtyLCx9BCOBnIEP9zH/LGzfkANCm3/7N6fDEF4JftYlnomun6JC2EKFMM93RMJAFPm3nuTBLfw7TlfjrK5L6WhyGbbCW7ZEBfkw5cv1gPL8vA5b6IWbXAUpzusjfhyhDvRPLh0FjC9SqUTG0kZuKd5Fv6tvrcFL9Sn+mZu2kARe/0h7Unn0gmMZDtU22mdBc1wGJO4AVbwKPzIwqViKVTu8z6TCL8+IxEqCFJFnuFZYYQeUZrZnjJR8uO/txDbHI+Guc/a9wm2ietx8+P8x4BpOOO8+4SDl/V9CshQWuBPA8/53F1s8gc/YlPo7C4Fv2vjjQFHnJ2qvaJArVR49Gl4TsfAjGryAE8hV6cNAUqw23XLWiAH8BJHcd7GqWmcDPMA2WpvPl1+EHKroC/fRBuuFGpBjh5XmZK/PsyU8hcoTAFJdAajaoly/0sAJlwnc+qcpaSuibhSslJBV1nPpmtqXqLAG3f3mzbiRM8GkPLIzu5D6jVDc3RHrAoIa7iaCC9PFGXyx7Eu83ougiTrjQsCqvn+luh4zsMF9jho6efmTD6GRtN/OeERmbBS19/0xeryCIQaXdAsYP1X2SwZHyA7YNWDCYbcowxy6H8Dh5BH+rRO97Hb52bTk7yIbnJOARw+idSZWiSaId9J5Oe9lG0OnpKb6kQ+QOXUK4PtBDUVrR+qhN6SDlfOXle2TIeFdQFBjWwDDnf+60DPuHbWcL w7e1Yarn O5mOUdPVqTZN0Dijsq39XXa/pJW8Zd2AiwN/LMZldYWU9qNeJhmoImAAtnT+Uj+X+6F7bLlCynPCqD0BJ6amnegBWd0jUNjphdwhBYPoD4Lw2RdW25GkGynRLA27OTZ6r/v4RH2aaoGvLgJLrbWeleehESsruIvt+HyrHIuz1rlZ0FLdk3jV8QtnD9IRm3yZYWtJcJPmfoMzif2J3Ji5wz5rohZEaG2rfwBSjHoxGgKmjSRJ4Q4wYAJCJ+XH+bjzEI3Khwjo8vA+yM6lvMAJQqXSHrtoR+hbUuCNYZXqGgBsA3DbHOz8kfp5GRCejuhVwnwSJRhPwzJqO1swQYu03Ue0+XSzA027PbFWUFLgS5AIPk/ced8hn8xqqkehqSNQkgdXUW2sldV2JnUZAkGKYbhRaphrHz5XjDr8pOO6zfhK+7HLEfbforj7763WMAL9ir0SzrmLWr+kynQz9OJKL8vWerrFNbYTfsCj+sw/ExmVofyrh+69d8152Gw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For that, we need to introduce: - a new config: ARCH_HAS_CONTPTE, - a new arch specific function which returns the number of contiguous PTE in a mapping and its base page size, - a pte_cont() helper, only introduced for riscv since we keep the arm64 naming (contpte) which is more explicit than the riscv's (napot). Signed-off-by: Alexandre Ghiti --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/pgtable.h | 30 +++++++++++++++++ arch/arm64/mm/hugetlbpage.c | 55 ++------------------------------ arch/riscv/Kconfig | 1 + arch/riscv/include/asm/hugetlb.h | 2 +- arch/riscv/include/asm/pgtable.h | 6 ++-- arch/riscv/mm/hugetlbpage.c | 24 -------------- mm/Kconfig | 3 ++ mm/Makefile | 1 + mm/contpte.c | 45 ++++++++++++++++++++++++++ 10 files changed, 88 insertions(+), 80 deletions(-) create mode 100644 mm/contpte.c diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 7b11c98b3e84..ac2f6d906cc3 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -20,6 +20,7 @@ config ARM64 select ARCH_ENABLE_SPLIT_PMD_PTLOCK if PGTABLE_LEVELS > 2 select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE select ARCH_HAS_CACHE_LINE_SIZE + select ARCH_HAS_CONTPTE select ARCH_HAS_CURRENT_STACK_POINTER select ARCH_HAS_DEBUG_VIRTUAL select ARCH_HAS_DEBUG_VM_PGTABLE diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index afdd56d26ad7..e30149a128f2 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -1638,6 +1638,36 @@ static inline int ptep_set_access_flags(struct vm_area_struct *vma, #endif /* CONFIG_ARM64_CONTPTE */ +static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, + size_t *pgsize) +{ + int contig_ptes = 0; + + *pgsize = size; + + switch (size) { +#ifndef __PAGETABLE_PMD_FOLDED + case PUD_SIZE: + if (pud_sect_supported()) + contig_ptes = 1; + break; +#endif + case PMD_SIZE: + contig_ptes = 1; + break; + case CONT_PMD_SIZE: + *pgsize = PMD_SIZE; + contig_ptes = CONT_PMDS; + break; + case CONT_PTE_SIZE: + *pgsize = PAGE_SIZE; + contig_ptes = CONT_PTES; + break; + } + + return contig_ptes; +} + #endif /* !__ASSEMBLY__ */ #endif /* __ASM_PGTABLE_H */ diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 0f0e10bb0a95..9e9c80ec6e74 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -112,57 +112,6 @@ static int find_num_contig(struct mm_struct *mm, unsigned long addr, return CONT_PTES; } -static inline int num_contig_ptes(unsigned long size, size_t *pgsize) -{ - int contig_ptes = 0; - - *pgsize = size; - - switch (size) { -#ifndef __PAGETABLE_PMD_FOLDED - case PUD_SIZE: - if (pud_sect_supported()) - contig_ptes = 1; - break; -#endif - case PMD_SIZE: - contig_ptes = 1; - break; - case CONT_PMD_SIZE: - *pgsize = PMD_SIZE; - contig_ptes = CONT_PMDS; - break; - case CONT_PTE_SIZE: - *pgsize = PAGE_SIZE; - contig_ptes = CONT_PTES; - break; - } - - return contig_ptes; -} - -pte_t huge_ptep_get(pte_t *ptep) -{ - int ncontig, i; - size_t pgsize; - pte_t orig_pte = __ptep_get(ptep); - - if (!pte_present(orig_pte) || !pte_cont(orig_pte)) - return orig_pte; - - ncontig = num_contig_ptes(page_size(pte_page(orig_pte)), &pgsize); - for (i = 0; i < ncontig; i++, ptep++) { - pte_t pte = __ptep_get(ptep); - - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - return orig_pte; -} - /* * Changing some bits of contiguous entries requires us to follow a * Break-Before-Make approach, breaking the whole contiguous set @@ -243,7 +192,7 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, unsigned long pfn, dpfn; pgprot_t hugeprot; - ncontig = num_contig_ptes(sz, &pgsize); + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); if (!pte_present(pte)) { for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) @@ -390,7 +339,7 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long addr, int i, ncontig; size_t pgsize; - ncontig = num_contig_ptes(sz, &pgsize); + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) __pte_clear(mm, addr, ptep); diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 8a0f403432e8..38d93cf44456 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -19,6 +19,7 @@ config RISCV select ARCH_ENABLE_SPLIT_PMD_PTLOCK if PGTABLE_LEVELS > 2 select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE select ARCH_HAS_BINFMT_FLAT + select ARCH_HAS_CONTPTE if RISCV_ISA_SVNAPOT select ARCH_HAS_CURRENT_STACK_POINTER select ARCH_HAS_DEBUG_VIRTUAL if MMU select ARCH_HAS_DEBUG_VM_PGTABLE diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 22deb7a2a6ec..f195f611722b 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -49,7 +49,7 @@ pte_t huge_ptep_get(pte_t *ptep); pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte -#endif /*CONFIG_RISCV_ISA_SVNAPOT*/ +#endif /* CONFIG_RISCV_ISA_SVNAPOT */ #include diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 268c828f5152..66061002ff36 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -296,6 +296,7 @@ static inline unsigned long pte_napot(pte_t pte) { return pte_val(pte) & _PAGE_NAPOT; } +#define pte_cont pte_napot #define pte_valid_napot(pte) (pte_present(pte) && pte_napot(pte)) @@ -560,7 +561,7 @@ static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, } #endif -static inline pte_t ptep_get(pte_t *ptep) +static inline pte_t __ptep_get(pte_t *ptep) { pte_t pte = READ_ONCE(*ptep); @@ -584,7 +585,6 @@ static inline pte_t ptep_get(pte_t *ptep) return pte; } -#define ptep_get ptep_get static inline void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pteval, unsigned int nr) @@ -686,6 +686,8 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma, return ptep_test_and_clear_young(vma, address, ptep); } +#define ptep_get __ptep_get + #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) { diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index f042f5c8bdb7..be129f4f1503 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -3,30 +3,6 @@ #include #ifdef CONFIG_RISCV_ISA_SVNAPOT -pte_t huge_ptep_get(pte_t *ptep) -{ - unsigned long pte_num; - int i; - pte_t orig_pte = ptep_get(ptep); - - if (!pte_present(orig_pte) || !pte_napot(orig_pte)) - return orig_pte; - - pte_num = napot_pte_num(napot_cont_order(orig_pte)); - - for (i = 0; i < pte_num; i++, ptep++) { - pte_t pte = ptep_get(ptep); - - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - - return orig_pte; -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, diff --git a/mm/Kconfig b/mm/Kconfig index b1448aa81e15..c325003d6552 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -981,6 +981,9 @@ config ARCH_HAS_CPU_CACHE_ALIASING config ARCH_HAS_CACHE_LINE_SIZE bool +config ARCH_HAS_CONTPTE + bool + config ARCH_HAS_CURRENT_STACK_POINTER bool help diff --git a/mm/Makefile b/mm/Makefile index 4abb40b911ec..605ead58403b 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -89,6 +89,7 @@ obj-$(CONFIG_MIGRATION) += migrate.o obj-$(CONFIG_NUMA) += memory-tiers.o obj-$(CONFIG_DEVICE_MIGRATION) += migrate_device.o obj-$(CONFIG_TRANSPARENT_HUGEPAGE) += huge_memory.o khugepaged.o +obj-$(CONFIG_ARCH_HAS_CONTPTE) += contpte.o obj-$(CONFIG_PAGE_COUNTER) += page_counter.o obj-$(CONFIG_MEMCG) += memcontrol.o vmpressure.o ifdef CONFIG_SWAP diff --git a/mm/contpte.c b/mm/contpte.c new file mode 100644 index 000000000000..e8574051d0b9 --- /dev/null +++ b/mm/contpte.c @@ -0,0 +1,45 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 Rivos Inc. + */ + +#include +#include +#include + +/* + * Any arch that wants to use that needs to define: + * - __ptep_get() + * - pte_cont() + * - arch_contpte_get_num_contig() + */ + +/* + * This file implements the following contpte aware API: + * - huge_ptep_get() + */ + +pte_t huge_ptep_get(pte_t *ptep) +{ + int ncontig, i; + size_t pgsize; + pte_t orig_pte = __ptep_get(ptep); + + if (!pte_present(orig_pte) || !pte_cont(orig_pte)) + return orig_pte; + + ncontig = arch_contpte_get_num_contig(ptep, + page_size(pte_page(orig_pte)), + &pgsize); + + for (i = 0; i < ncontig; i++, ptep++) { + pte_t pte = __ptep_get(ptep); + + if (pte_dirty(pte)) + orig_pte = pte_mkdirty(orig_pte); + + if (pte_young(pte)) + orig_pte = pte_mkyoung(orig_pte); + } + return orig_pte; +} From patchwork Wed May 8 11:34:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13658607 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D66C2C19F4F for ; Wed, 8 May 2024 11:38:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 153526B015C; Wed, 8 May 2024 07:38:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 041AE6B015D; Wed, 8 May 2024 07:38:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D618A6B015E; Wed, 8 May 2024 07:38:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A9E396B015C for ; Wed, 8 May 2024 07:38:32 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 58EE91A0FE8 for ; Wed, 8 May 2024 11:38:32 +0000 (UTC) X-FDA: 82095030864.04.38FDEFC Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by imf14.hostedemail.com (Postfix) with ESMTP id 671B9100011 for ; Wed, 8 May 2024 11:38:30 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=MczzrYa8; spf=pass (imf14.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.45 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715168310; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9tID4t35K2Xq8XfdKgfehJoKp8xw/RZTmJgN0BGYZu8=; b=odwpA2jA9y6gU0+GdWCjbJzAUzemeF6baJ8nzCr/ZsCvTrHiVa6v31pOXhve4O6JDb8ATN SnVHhiCt7N7DN99rZUDT/5vufxATcL3m5EOmZszu8yvuDkzH7KLsm8s5K34uqrn0mwmR3P lHKf/w1DdcfaBPG8Y2tabrkSTD8/Yck= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=MczzrYa8; spf=pass (imf14.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.45 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715168310; a=rsa-sha256; cv=none; b=DWGBgGP8AmW6fdTjXK1T9LSL2cXX8Goo0OD7VywKZVeem4kk1dsNmu6EiDkSzQ4coiATId aOAqtzFTls3dcHhqc7rEUhYphwOvEcorvP38gKTeZH65HnIs/YFHsPpz2mgAJByk6kA03r qWGVzTtyFFbnBwp8/4sIYUAk2un2xq8= Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-41ba1ba5591so31819585e9.1 for ; Wed, 08 May 2024 04:38:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1715168309; x=1715773109; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9tID4t35K2Xq8XfdKgfehJoKp8xw/RZTmJgN0BGYZu8=; b=MczzrYa8MFl218lybMg88lGI5BMlt5A4XeowDVaEEdKrWazr7X6t50YaV/juio6fo2 PtmY47sA+bX6uBoFQSsFIdDkNghzocrAvaj+G+xgj//7s9YpFAeY+80bWVyNSOZAh3kI 3RZlEcf74XlF9107AtndMAHpebSz2FSUpGLqbVrtrimAvphX2+31h4HOGkHRdiWqXgbz Hr3L4PRVDIcLeSq/LciCcGhwAMSKAwgIx6dNUKw2ypW7wQ4VzjQV6Cehp1YAV9iBnR1c 6KLrvSlOubJvJ+ELwAm+KK8BcPlHsLGMoPKni1mdS1EB2SWt0mCM6a/hE31X54XT6UO5 TyJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715168309; x=1715773109; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9tID4t35K2Xq8XfdKgfehJoKp8xw/RZTmJgN0BGYZu8=; b=OKlC+FlY+0g69C0FQe+ACbKX6xlSrnZoSdFgehHbJ/eWKj/cZNbBlXAkiXkYrCrANE +eGgV3ZgvvhQjgXFWKvmNYKwekuTef8GcJGZG/rrV3V3diI49pdU3A9654zhBoxB3oLc zNH6pjjjr9Noi1GaDFE6tNUCJxJqgrZIFrC1TGLdyHaFEXxYdZWTAAjXLbYoIugi21hj aNC+YoJkMjUT5dW2xfMIAwPILG00BUF1vn69IE3oeSmlM57EJZArosst50qihDwGyYA7 0a94ph33XLUE2eGCBBYmEFoFJKofh669A2ANJGTAN+Lfob2BBbGfhFzpPtDeyGeG2BwG tNaQ== X-Forwarded-Encrypted: i=1; AJvYcCXbdrv9gpxoq8vWobvGMkiit6psBSHnIqRq+Mr+JljMO2tChgY++LSKFQeiZIxShVH6cDxBqIFhQlLPPqt0afVRkSk= X-Gm-Message-State: AOJu0Yx1WCu5GdIgI5ts4nlNJLBcvlq6UTVjVPFgVP7S+qDLbrBWd86T +xc7UcJKeIrhDrl+waC0M7gqzcEmreo7cG00Ye41/I7HAP2+lycIsEsIEJdzlhQ= X-Google-Smtp-Source: AGHT+IFc73xLy956G8LkxzJy48Vpp254D4XfW/OPcYqIx5DwErIqgJZXPvuWYs+kahkVNUCZ42Avig== X-Received: by 2002:a05:600c:444e:b0:41b:935:24a9 with SMTP id 5b1f17b1804b1-41f714fbe0amr19186605e9.24.1715168308900; Wed, 08 May 2024 04:38:28 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-41f87d20214sm20234005e9.24.2024.05.08.04.38.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 May 2024 04:38:28 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH RESEND v2 4/9] mm: Use common set_huge_pte_at() function for riscv/arm64 Date: Wed, 8 May 2024 13:34:14 +0200 Message-Id: <20240508113419.18620-5-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240508113419.18620-1-alexghiti@rivosinc.com> References: <20240508113419.18620-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 671B9100011 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 95d58c1so8urdfsb6qwdbtkmbjcn5qj1 X-HE-Tag: 1715168310-400232 X-HE-Meta: U2FsdGVkX19NC72T0GFqulaD1D2/4GkaPT61jcIFJDu8Mv21Z59M+hGSSPz2mVBbeoN2pfVnzkxkjHthmBTO1Z2GvATMtPwAYZ0NMDera9AAaQMEivhJfxJNDDCJLXSSnvZtvU13ZcHlYk4Xk74tDQb4PwASYHlEIkglpIpKT1oY6ZqhCZEJNc+Hq8j2OLz3oR1y90DOcXzB1s5z5MggYgsmLicFqn5DbqrbtKZeb2s09fD75iiPOFv6vfh68IRDTn7mQgx2VRN4PQKGWjiAjyN05RmC9PGqnLasuNfbtDDj04O3T34PxJVj+Ij4jXuVm13083I5Fl4SFznBXaQSCJWSfkOZprRVeyYzwuxXuzTFnncFBh5UDcd0aFZOT/O5MPC2l7hk0xw3yNJaSQmnSXnLayVOVUDUmqUVTB780TtM87mT71SseJSWFzt2Tv2Hk+TDnJqpyjz2Z8Utd6i3yDJ9c3TeNXpyEUDOyG7idFgzKlJ2unK8L2pLWUY5UpIYBZuD+BzVLGadHkQd3an2W9c0PJ5tashGLMT0hIPVU9BMT6P8/32DrNMq8WvmIf4niM5f4NC6qX7/l2BJu1TiuQMxiCjZlRL7+73jZ9iqCQUj0l7aIiejr9YFYVF99jvK3aashiirPMRKalPECSDL0qRlgNcjRSLiD08BNo3JXsKgR3W9oLjuktec3h7ubE4qfKaAvp2g+RqJQ5uWYa65HQUC72+y+Aj4aUdxIbe3y7hpIZnjCKTda05P+oUDaLPABiQpKXPfMn2JViKjZ5I43CUKEjmtrhP7uRjU/HIaYD44fqnvyVzIy/cS4E4XxFYmFl59rBBv9yEB8Y/XIxlI8/NCUP10nHkbE8YA9WJyNxlKLmknw3SC4WGgYdf0J6/1/CshrfAW2tXklyTXCuW4jlq6oUGg9YfBkl4+uj4DpWxPantT0WdiKZExkwjvkgqItZe8cjQ3TVgWscdFQ05 xUe6kJPI xwXO/Yt2Out4TFMiEya3uf5UCoUAKODKL0omz8lJiP5H+lagpGTA1M9554aSHGy8FzDds/vBx0FaiLazgmHoW96bjyvtPH55rGKDjJTT5N1DvrsIZTkbRpUqcJeNR3uzQErOcht7NAr5+sRfF+35BumZxSfV1c7k9RER+YVw8DG+PGlQWUdd/a4IX9IjLK2h+NJndXDqUDGk/QAemQ62RdMCDbfJQFBgMO0nNWb3aRqGB61GSMJdWJ0absmXGppETeXauOZ5G1Xinqb2mjLKbrpl/uJnbYiEWp35/D76nJqJZxjzoPthRWCos+8tni6IMfp9QpeurFD3/ofs+3mzi7bHnh49TWadXN7HoMTpqrlNlqH5ZqjxUW5W8+Lijoi06pN0C0uJlsUAX60ODqKJEP09dVcWb5SI4vOneuOfl+U1+/nG9CmH0Wpm1K8rgxfgG5RXioXpUHhie+5JyKjmLGZCRIqSdR8gGkcHzN9d6gOkuYkeOy5tryL0tQg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/pgtable.h | 14 +++++--- arch/arm64/mm/hugetlbpage.c | 56 ----------------------------- arch/riscv/include/asm/pgtable.h | 39 +++++++++++++------- arch/riscv/mm/hugetlbpage.c | 62 -------------------------------- mm/contpte.c | 59 ++++++++++++++++++++++++++++++ 5 files changed, 95 insertions(+), 135 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index e30149a128f2..2e0415fd5083 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -372,9 +372,10 @@ static inline pte_t pte_advance_pfn(pte_t pte, unsigned long nr) return pfn_pte(pte_pfn(pte) + nr, pte_pgprot(pte)); } -static inline void __set_ptes(struct mm_struct *mm, - unsigned long __always_unused addr, - pte_t *ptep, pte_t pte, unsigned int nr) +static inline void ___set_ptes(struct mm_struct *mm, + unsigned long __always_unused addr, + pte_t *ptep, pte_t pte, unsigned int nr, + size_t pgsize) { page_table_check_ptes_set(mm, ptep, pte, nr); __sync_cache_and_tags(pte, nr); @@ -385,10 +386,15 @@ static inline void __set_ptes(struct mm_struct *mm, if (--nr == 0) break; ptep++; - pte = pte_advance_pfn(pte, 1); + pte = pte_advance_pfn(pte, pgsize >> PAGE_SHIFT); } } +#define __set_ptes(mm, addr, ptep, pte, nr) \ + ___set_ptes(mm, addr, ptep, pte, nr, PAGE_SIZE) + +#define set_contptes ___set_ptes + /* * Huge pte definitions. */ diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 9e9c80ec6e74..b8353b0a273c 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -159,62 +159,6 @@ static pte_t get_clear_contig_flush(struct mm_struct *mm, return orig_pte; } -/* - * Changing some bits of contiguous entries requires us to follow a - * Break-Before-Make approach, breaking the whole contiguous set - * before we can change any entries. See ARM DDI 0487A.k_iss10775, - * "Misprogramming of the Contiguous bit", page D4-1762. - * - * This helper performs the break step for use cases where the - * original pte is not needed. - */ -static void clear_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - unsigned long i, saddr = addr; - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) - __ptep_get_and_clear(mm, addr, ptep); - - flush_tlb_range(&vma, saddr, addr); -} - -void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte, unsigned long sz) -{ - size_t pgsize; - int i; - int ncontig; - unsigned long pfn, dpfn; - pgprot_t hugeprot; - - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); - - if (!pte_present(pte)) { - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) - __set_ptes(mm, addr, ptep, pte, 1); - return; - } - - if (!pte_cont(pte)) { - __set_ptes(mm, addr, ptep, pte, 1); - return; - } - - pfn = pte_pfn(pte); - dpfn = pgsize >> PAGE_SHIFT; - hugeprot = pte_pgprot(pte); - - clear_flush(mm, addr, ptep, pgsize, ncontig); - - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned long sz) { diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 66061002ff36..5d1d3a6c7c44 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -535,29 +535,39 @@ static inline void __set_pte_at(struct mm_struct *mm, pte_t *ptep, pte_t pteval) static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, size_t *pgsize) { + unsigned long hugepage_shift; pte_t __pte; /* We must read the raw value of the pte to get the size of the mapping */ __pte = READ_ONCE(*ptep); - if (pgsize) { - if (size >= PGDIR_SIZE) + if (size >= PGDIR_SIZE) { + if (pgsize) *pgsize = PGDIR_SIZE; - else if (size >= P4D_SIZE) + hugepage_shift = PGDIR_SHIFT; + } else if (size >= P4D_SIZE) { + if (pgsize) *pgsize = P4D_SIZE; - else if (size >= PUD_SIZE) + hugepage_shift = P4D_SHIFT; + } else if (size >= PUD_SIZE) { + if (pgsize) *pgsize = PUD_SIZE; - else if (size >= PMD_SIZE) + hugepage_shift = PUD_SHIFT; + } else if (size >= PMD_SIZE) { + if (pgsize) *pgsize = PMD_SIZE; - else + hugepage_shift = PMD_SHIFT; + } else { + if (pgsize) *pgsize = PAGE_SIZE; + hugepage_shift = PAGE_SHIFT; } /* Make sure __pte is not a swap entry */ if (pte_valid_napot(__pte)) return napot_pte_num(napot_cont_order(__pte)); - return 1; + return size >> hugepage_shift; } #endif @@ -586,8 +596,8 @@ static inline pte_t __ptep_get(pte_t *ptep) return pte; } -static inline void set_ptes(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pteval, unsigned int nr) +static inline void __set_ptes(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pteval, unsigned int nr) { #ifdef CONFIG_RISCV_ISA_SVNAPOT if (unlikely(pte_valid_napot(pteval))) { @@ -631,7 +641,8 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr, pte_val(pteval) += 1 << _PAGE_PFN_SHIFT; } } -#define set_ptes set_ptes +#define set_contptes(mm, addr, ptep, pte, nr, pgsize) \ + __set_ptes(mm, addr, ptep, pte, nr) static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -646,9 +657,8 @@ extern int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addre extern int ptep_test_and_clear_young(struct vm_area_struct *vma, unsigned long address, pte_t *ptep); -#define __HAVE_ARCH_PTEP_GET_AND_CLEAR -static inline pte_t ptep_get_and_clear(struct mm_struct *mm, - unsigned long address, pte_t *ptep) +static inline pte_t __ptep_get_and_clear(struct mm_struct *mm, + unsigned long address, pte_t *ptep) { pte_t pte = __pte(atomic_long_xchg((atomic_long_t *)ptep, 0)); @@ -687,6 +697,9 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma, } #define ptep_get __ptep_get +#define set_ptes __set_ptes +#define __HAVE_ARCH_PTEP_GET_AND_CLEAR +#define ptep_get_and_clear __ptep_get_and_clear #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index be129f4f1503..d8f07aef758b 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -173,68 +173,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -static void clear_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - unsigned long i, saddr = addr; - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) - ptep_get_and_clear(mm, addr, ptep); - - flush_tlb_range(&vma, saddr, addr); -} - -/* - * When dealing with NAPOT mappings, the privileged specification indicates that - * "if an update needs to be made, the OS generally should first mark all of the - * PTEs invalid, then issue SFENCE.VMA instruction(s) covering all 4 KiB regions - * within the range, [...] then update the PTE(s), as described in Section - * 4.2.1.". That's the equivalent of the Break-Before-Make approach used by - * arm64. - */ -void set_huge_pte_at(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - pte_t pte, - unsigned long sz) -{ - unsigned long hugepage_shift, pgsize; - int i, pte_num; - - if (sz >= PGDIR_SIZE) - hugepage_shift = PGDIR_SHIFT; - else if (sz >= P4D_SIZE) - hugepage_shift = P4D_SHIFT; - else if (sz >= PUD_SIZE) - hugepage_shift = PUD_SHIFT; - else if (sz >= PMD_SIZE) - hugepage_shift = PMD_SHIFT; - else - hugepage_shift = PAGE_SHIFT; - - pte_num = sz >> hugepage_shift; - pgsize = 1 << hugepage_shift; - - if (!pte_present(pte)) { - for (i = 0; i < pte_num; i++, ptep++, addr += pgsize) - set_ptes(mm, addr, ptep, pte, 1); - return; - } - - if (!pte_napot(pte)) { - set_ptes(mm, addr, ptep, pte, 1); - return; - } - - clear_flush(mm, addr, ptep, pgsize, pte_num); - - set_ptes(mm, addr, ptep, pte, pte_num); -} - int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, diff --git a/mm/contpte.c b/mm/contpte.c index e8574051d0b9..2320ee23478a 100644 --- a/mm/contpte.c +++ b/mm/contpte.c @@ -10,6 +10,8 @@ /* * Any arch that wants to use that needs to define: * - __ptep_get() + * - __set_ptes() + * - __ptep_get_and_clear() * - pte_cont() * - arch_contpte_get_num_contig() */ @@ -17,6 +19,7 @@ /* * This file implements the following contpte aware API: * - huge_ptep_get() + * - set_huge_pte_at() */ pte_t huge_ptep_get(pte_t *ptep) @@ -43,3 +46,59 @@ pte_t huge_ptep_get(pte_t *ptep) } return orig_pte; } + +/* + * ARM64: Changing some bits of contiguous entries requires us to follow a + * Break-Before-Make approach, breaking the whole contiguous set + * before we can change any entries. See ARM DDI 0487A.k_iss10775, + * "Misprogramming of the Contiguous bit", page D4-1762. + * + * RISCV: When dealing with NAPOT mappings, the privileged specification + * indicates that "if an update needs to be made, the OS generally should first + * mark all of the PTEs invalid, then issue SFENCE.VMA instruction(s) covering + * all 4 KiB regions within the range, [...] then update the PTE(s), as + * described in Section 4.2.1.". That's the equivalent of the Break-Before-Make + * approach used by arm64. + * + * This helper performs the break step for use cases where the + * original pte is not needed. + */ +static void clear_flush(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); + unsigned long i, saddr = addr; + + for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) + __ptep_get_and_clear(mm, addr, ptep); + + flush_tlb_range(&vma, saddr, addr); +} + +void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pte, unsigned long sz) +{ + size_t pgsize; + int i; + int ncontig; + + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + + if (!pte_present(pte)) { + for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) + __set_ptes(mm, addr, ptep, pte, 1); + return; + } + + if (!pte_cont(pte)) { + __set_ptes(mm, addr, ptep, pte, 1); + return; + } + + clear_flush(mm, addr, ptep, pgsize, ncontig); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); +} From patchwork Wed May 8 11:34:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13658608 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBCF4C19F4F for ; Wed, 8 May 2024 11:39:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 831846B015D; Wed, 8 May 2024 07:39:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E1F96B015F; Wed, 8 May 2024 07:39:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 682D36B0160; Wed, 8 May 2024 07:39:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 4377D6B015D for ; Wed, 8 May 2024 07:39:33 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 0A2FCA1457 for ; Wed, 8 May 2024 11:39:33 +0000 (UTC) X-FDA: 82095033426.30.343AF4D Received: from mail-wr1-f50.google.com (mail-wr1-f50.google.com [209.85.221.50]) by imf23.hostedemail.com (Postfix) with ESMTP id 272F5140017 for ; Wed, 8 May 2024 11:39:30 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="iEIqb/Qt"; dmarc=none; spf=pass (imf23.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.50 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715168371; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hkOJ479CewvDdU5HWPEgLgm2sAjKEVUPHaCHVUUKnDI=; b=bH9qDNltJx5FEeAnyx/Hq1qmF+aA28djMHAF/kA7w7lrec0A/X4lJ100irw+Jt59aeYJbV AGc8TFWJ4u3vjFyJuUAVQ6sIUik2ufGUi9PQbyOfhOAC3UeAGB0p3dRfy3/sK6zSg7DyI0 SGnM2eUI6/g0c5b1RIgzwbiX3H1yuLA= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="iEIqb/Qt"; dmarc=none; spf=pass (imf23.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.50 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715168371; a=rsa-sha256; cv=none; b=AQ1T/JBTYE10eDQwpYrYpCFffjJIwhSl+alFNya/mQ2AXKPeOonkugWp33TDmQKvaSoCKd VAh1VnXMLj883cgLStx6kp7zBH2SSoIcPAUn524DFC2lJ4MIGq4N8GGElzp4qCAFhDkx+n y33GKFQC5RJOra/+l5Eogpj3vpV+a/A= Received: by mail-wr1-f50.google.com with SMTP id ffacd0b85a97d-34eb52bfca3so3057946f8f.0 for ; Wed, 08 May 2024 04:39:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1715168370; x=1715773170; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hkOJ479CewvDdU5HWPEgLgm2sAjKEVUPHaCHVUUKnDI=; b=iEIqb/QtsVknN2+qtIhVsgI/aHx15THqcNdwP0s8weMszFwYXKuak3mFlSQgD3j6Zu u1KuNZpsNjq3Qyyv4dS6mgyZcR2rXfGY/Sa47nYslHI7OjpIY8uc4d2TOZc5/v2timKC f6UE/GGJLLzMxpjlvmoXNklXVBWq0rh1enpNCNIYiFhQEX9rTs1W/ZVpkrwC0FyjNs+T tNq5HrVFky8ETWqHuaHYDCDYFN03PsabG5TqpTo6ddPNA2Rby/1LIBUlZR137cUap+2B vTmmm4lfS55AW8fCvK1ywtyiMgA0bfx9fvzCAEAE4e1Nfrbuj8sUbKYhIJyefKcvg+kJ tRgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715168370; x=1715773170; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hkOJ479CewvDdU5HWPEgLgm2sAjKEVUPHaCHVUUKnDI=; b=TuwH94nx3wVGw2EPafaKdwBekxMME2H0nzQPFsasU4NyaYJe1ySuRhd5MwHUYNe1Jk fo+e6+fD9LZ1e25neFWrgF+j7emNNMXOe42oDYEvVhsTAOlkc9Yza4Sg0PGICK9jJRvN lm4PwgbWGEEEiZNoprTvKcT3zcPhIfpd11gAQ4Ks/Iem11bxF+A1cYlHOE2m+ZDHg3/d RgCghMlKkowVRDE7uZfMU7OEtVDZEV6xg1jXBn3qlI0PbWQESirt6sBNoq+prMnqc73g opQgV9nyCgWsQxK56+J4jqAEUHxREvINTTn6BC707kzs5DhqT9BkWoqsn4WhiD8UH+7S eJxA== X-Forwarded-Encrypted: i=1; AJvYcCVm8xNxlExczjlbhwA35XN+3icKFmERsASRaulE/rZSVNyTZfT3zrlMyYX8Es6TIbonG4XCXQNvC5s7l1iByWYVqRU= X-Gm-Message-State: AOJu0YxheKngI6TaL2JgaQxR3C2rpqtKGS93XBWDB/QY5zLvoeqEqK26 53PcB2NhvlbvNsJZ9qDYNMZ2R0IT5//kT4GE4AyliiCrLP1l3EhvO/OD9T09HQs= X-Google-Smtp-Source: AGHT+IEZS4ddGwIWcQuhI1g6wktPlm5wrgDKEngfcuzPllSn67aCj73nYZBEmuYd+HrBwCZhVDrutw== X-Received: by 2002:adf:e752:0:b0:34d:17f2:956d with SMTP id ffacd0b85a97d-34fca80dffcmr2356969f8f.66.1715168369782; Wed, 08 May 2024 04:39:29 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-41f88110ff8sm20241365e9.38.2024.05.08.04.39.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 May 2024 04:39:29 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH RESEND v2 5/9] mm: Use common huge_pte_clear() function for riscv/arm64 Date: Wed, 8 May 2024 13:34:15 +0200 Message-Id: <20240508113419.18620-6-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240508113419.18620-1-alexghiti@rivosinc.com> References: <20240508113419.18620-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 272F5140017 X-Stat-Signature: es783gjhawhks6abewkkwbrd8jte7ofy X-Rspam-User: X-HE-Tag: 1715168370-766685 X-HE-Meta: U2FsdGVkX18jirgLvuCizw/4yPhf7hMHCTcEYarHuiToqabtCfQnfKZMJrmu9B4vGQLCk6jpWf5pu8TesK5ukN8v29aWt5pXq3VNwzckaSjtOm3l9bYcQi4970iCai3JjE4zCI8VxMomqwLekhcdMTaxnJ5ehhOzxs+nytK2DXXjVAOFcSl6As/8em0B2mD7HYvaUUUJOJkmfc/xCEpaJHBNby1tlLwwn6ZerK3955JoSPk/hxL9Pxy8/b0WHjCjJUkLEskuWH1jsyoSDgB07KWtKS6sSkC62fldaxe/thKfQ1yb7b4t1nvFYyucl2Xa6l2kP00Abfg7VwCHZJAuaqlv5rJ2SqtmSE/d8vY94R96G5lT9bXTsu9JMi1Ze/40YvzMnmwlobsBS3NYLcv03yEwfC7/BPonGMsj7JCqCOcKuC0QwzZEm/vGgOA1T300eixfAAQIkitn2H4p4sscT6/zzYIil5wUcnjse8eylewpXzGvt82Ixm0lm8Jb4dv0TeyBTlCnEMBW/+LLNeZvf05W9xR5vxv1XZkpjxixSqMD/EBz0D9KIyoCA1EBrKeVnHNN/m1oMlf0ajafmk4SD+F8nzzavUqPGkVTHpMUHgxltpKfLtTsL6BHZbDBV6c0smIafEthM27uSTSFOrUVs21+UJ0lfs0QFPWGpuklF5PbNDYDGWUId4jrkexvxTJYBy7QbOhT7f10qjuulhrbcqTpEB0jbnqZvBgJABStoc30s+2koHA53vMbtMqRsWd7hkZNU2xFnhvU0F3XEhRhct/kI2K+62duACj3w9Wm4aP3QRiqIQlu0RBxjR/cWJLSPrsjSxSbsKbwUV6UYlGK94Uiilzu7aqu7lNphbCmIekpsRCSD+Xw1/w6h3+yilBHAyWRMJfHVVhIDGHghBlfoWqIZH2rR37QuI9MecxrFMJhciCLUfjV1ZYa3dBHjiM9NBO80XBIl0+33GOyN3i 9AtJjeUj WLMZ3Rmag4j/mbXgk167somLG3nWrSVCBH5aS6TXAR/FLlZ4xGer1+4USC0IkzRjv/zi4pu2Uthdf+8alZ1wjlxITSkzuHjmJ+eozrD5v6UlOYsAJP7gfwkmooIxeTyU07ohMW0gkrr2yc1uxX7yZPGF0NOfXJX2S7PpU8Xp+atxFJlcv05M+TsB/OFH9wG+kRTd/FUVYUXUWlNB0oz0wSS8ktryqPt0luAmLAaLddkd/Nzacn7EGc0RgrQ8yz6FKGXskQseRgKW5ZOJ0ePVuzP92rFADi4nEuNSD9txkq5dZmDkvOsCdBDVpCsGDA9/2e47289PATvUE6db/eJqEHDpA6X45yPBfYM9CNkwKMz5cW67gdmkoQG9V/sxKZtpBXrzF9WEV9IqmLZm4UXdnpgVFSKcpaiq6gphkGCctavMteSVHB5U7RkeqtMbdQ0UtRw3pZkfnbPvR6RlshSgc6adl+H7PEUvNNgUjQ+UtShVoYXPgT0oUR/BVIA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Both architectures have the same implementation so move it to generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/mm/hugetlbpage.c | 12 ------------ arch/riscv/include/asm/pgtable.h | 5 +++-- arch/riscv/mm/hugetlbpage.c | 19 ------------------- mm/contpte.c | 14 ++++++++++++++ 4 files changed, 17 insertions(+), 33 deletions(-) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index b8353b0a273c..cf44837369be 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -277,18 +277,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -void huge_pte_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz) -{ - int i, ncontig; - size_t pgsize; - - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) - __pte_clear(mm, addr, ptep); -} - pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 5d1d3a6c7c44..0847a7fb8661 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -644,8 +644,8 @@ static inline void __set_ptes(struct mm_struct *mm, unsigned long addr, #define set_contptes(mm, addr, ptep, pte, nr, pgsize) \ __set_ptes(mm, addr, ptep, pte, nr) -static inline void pte_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) +static inline void __pte_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep) { __set_pte_at(mm, ptep, __pte(0)); } @@ -700,6 +700,7 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma, #define set_ptes __set_ptes #define __HAVE_ARCH_PTEP_GET_AND_CLEAR #define ptep_get_and_clear __ptep_get_and_clear +#define pte_clear __pte_clear #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index d8f07aef758b..437b1df059eb 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -254,25 +254,6 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); } -void huge_pte_clear(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long sz) -{ - size_t pgsize; - pte_t pte = ptep_get(ptep); - int i, pte_num; - - if (!pte_napot(pte)) { - pte_clear(mm, addr, ptep); - return; - } - - pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); - for (i = 0; i < pte_num; i++, addr += pgsize, ptep++) - pte_clear(mm, addr, ptep); -} - static bool is_napot_size(unsigned long size) { unsigned long order; diff --git a/mm/contpte.c b/mm/contpte.c index 2320ee23478a..22e0de197bd3 100644 --- a/mm/contpte.c +++ b/mm/contpte.c @@ -12,6 +12,7 @@ * - __ptep_get() * - __set_ptes() * - __ptep_get_and_clear() + * - __pte_clear() * - pte_cont() * - arch_contpte_get_num_contig() */ @@ -20,6 +21,7 @@ * This file implements the following contpte aware API: * - huge_ptep_get() * - set_huge_pte_at() + * - huge_pte_clear() */ pte_t huge_ptep_get(pte_t *ptep) @@ -102,3 +104,15 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, set_contptes(mm, addr, ptep, pte, ncontig, pgsize); } + +void huge_pte_clear(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, unsigned long sz) +{ + int i, ncontig; + size_t pgsize; + + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + + for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) + __pte_clear(mm, addr, ptep); +} From patchwork Wed May 8 11:34:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13658609 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A011BC19F4F for ; Wed, 8 May 2024 11:40:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 32A746B0164; Wed, 8 May 2024 07:40:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2DB036B0165; Wed, 8 May 2024 07:40:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 17BFC6B0166; Wed, 8 May 2024 07:40:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id EDE256B0164 for ; Wed, 8 May 2024 07:40:34 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A21A61412F4 for ; Wed, 8 May 2024 11:40:34 +0000 (UTC) X-FDA: 82095035988.16.06F8DEB Received: from mail-lj1-f172.google.com (mail-lj1-f172.google.com [209.85.208.172]) by imf12.hostedemail.com (Postfix) with ESMTP id 8E94740004 for ; Wed, 8 May 2024 11:40:32 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=dQZnEK7Z; dmarc=none; spf=pass (imf12.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.172 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715168432; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=k7TbOelFCBPjD//aDEZ907vn9YCYdQxapVfhTuFqwGo=; b=msZdlL/LiD+zQoArVEFokM5d8CPy5y36vFkoGKYCHzZP2kgndvlrQKZKF4/lcT4eJIB1F0 6SBieqjUrDeRJkhesWYIJykzO7Jr2TR73sg/g0Gkkg8j8XFjhdBtj/vgqRPMXUgW5rgaUT i9jhojCiTq4zIgG1eAhvK7zVb2KJFLs= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=dQZnEK7Z; dmarc=none; spf=pass (imf12.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.172 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715168432; a=rsa-sha256; cv=none; b=afQERKgA5X7FPV70JWbLj1MaCl/sLW/Ume/TjL0pM1KWBbliK1cnXX8kw23g0L7AeL9ppe 8Df9dgL8+WQU4jm3WCR4sHYYqyszghF4oUxaxWgOObImHBoFkU9bdRsk+qGwrev1GazX4y wSLMqh02Z9HiT9oS4NkQP/GkgSUX6wY= Received: by mail-lj1-f172.google.com with SMTP id 38308e7fff4ca-2e242b1df60so51971401fa.1 for ; Wed, 08 May 2024 04:40:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1715168431; x=1715773231; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=k7TbOelFCBPjD//aDEZ907vn9YCYdQxapVfhTuFqwGo=; b=dQZnEK7ZBOM2uwDVViA1w9puvg55DYmjvO9E4GsKiqdH3MTQRddEX5f4JvZEujnSdU A3DuoHMerpp1DHCN94+6XHnr/AndztB79Dvd5qYc0Qs4sjkVDypImLc6eS4ciy3VBdWA PSNYF3humGnjjDymy5uZ/2X/+Rp7CJd3CDJV5YIykh0D6RpYTV0ilerOSjAInjYDwgKF Ne+0AKrXx1hVnD7SKC2xa05895TpY3J4dnZ+oWJL6soD3ngMPm+k0QwyvxDPVeI4Xg/b jWMMVFGEJdeKbsFRLHELsnf32OJlN5rmmLF8RRjQTGscgEJIsWxIElGHNVLfpEsPtU4m mH0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715168431; x=1715773231; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k7TbOelFCBPjD//aDEZ907vn9YCYdQxapVfhTuFqwGo=; b=GwyS174RM+DCRVtv5udNtsjX+ae7n104ixleJTr/PFwc5dUqrmUYYvxz8Q8/M2LIwo c56pEJx7P0/EBpsZKPKz7NukNpvTVQ/0/LXlTVDbon0w/HhhhN/zNA7uCRGSy56eR4XU 7XaKFhUEFVHL5q+ghuz34kkgzMgoYuDY0+5Y1ebnfk1ue675a7LKh93AW4AoLExHCIJ4 TqpbnQAMHAX70HS1/305jxmjF5VinI9ebgeYEyOKP9t2jpULDfp7+wlqGXyUPLbxXX/M a5ItO58InzAun2gUXpZTeAh5+zpOjWumgsPOUA86XG1nLtxTACrvqahQzi1J7q866Kv6 sZLA== X-Forwarded-Encrypted: i=1; AJvYcCVy5aTocpHxlsdU0fQ52Jl8HjPvgqM13IjQpUkUBiOsxxq7JvCXUJsZ9cl/OiS0UHiI5E8zFaQJVUaOPsyJhQOa76c= X-Gm-Message-State: AOJu0YzsjSrxAXowD0zy3QkLrYodHjkk4rOA6F8BwL2kwMHxY6PyXYmx UU/yqU4EkyIIy3CA2maDt0mUN36x47jTJz4QZQIg4pELIFouM7vzBiH4pyBBhkA= X-Google-Smtp-Source: AGHT+IHoKXWUN4OirZVjujYdTPRjtHlf5MTjdp4rZ05v4P+q2zGuw+P9Rq7y6u6CQwAZzw3ytsfSow== X-Received: by 2002:a2e:8749:0:b0:2e1:ae29:f28a with SMTP id 38308e7fff4ca-2e447697612mr22142151fa.34.1715168430817; Wed, 08 May 2024 04:40:30 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id b15-20020a05600c4e0f00b0041aa79f27a0sm2003785wmq.38.2024.05.08.04.40.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 May 2024 04:40:30 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH RESEND v2 6/9] mm: Use common huge_ptep_get_and_clear() function for riscv/arm64 Date: Wed, 8 May 2024 13:34:16 +0200 Message-Id: <20240508113419.18620-7-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240508113419.18620-1-alexghiti@rivosinc.com> References: <20240508113419.18620-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Stat-Signature: 9aic5hx4osz46oibp9iirok9iddp3p99 X-Rspam-User: X-Rspamd-Queue-Id: 8E94740004 X-HE-Tag: 1715168432-490911 X-HE-Meta: U2FsdGVkX19J1IVX5YiKJuqBmkN1+tCS6ClW79agJQIqvwkznrFT9O1o282rhk3suc0qPE9s7hIlGwtXgXkRVNXaJVRUc5UdHVPr8EcN3yo8xdEIl/uB7HIDi7vVdc8RLW7c6Y1Pv3JiHpS6rtDS0fodGtqWM11C82676kSAHZB4JQWjUkEC61eTT/024tmgA4mMHOl9k6dWakLJlXoETi8koVSEYXqALHE9MhvX06mX7fMVLCmougxsopN3zhL9goawONlKcYw9ZBrCLsL90ooYE08iUrUEKtfgi78rQnRdo9Nl2qkqxBHWD7gjABF06cRhIZSbioRVwm/MOXxcZSJKkXY8UGmjw+zxPB0B7y/lBdtHtvSFraMdE/iS1+smFcJDbstul8lceQc0Z2+fGXswUpmZ61OKVM7WNnlVm5TIcH0lM2OZr7K8VVX2EJYEx9ejKN0aIAK9NcLXRhnvkF9V2Q3CuFQhFya3pM91UvKiIqaIJjbl8I8b4mXD5NYpHIUCtFQcYMAzZDYyFHcehumaXjM23X+YefVjnqusXlWYmnKjeRXb8pOjUUWn+u8Wqn8aH1nM2FVDpSx3m6f4l3f0WE1H1IwRRWljBAI9oKvKQYU10fca1fYW6N00Op3Ebc3YRTUIxC3LjpOACAMXREHjpywK9onTWBaoef2vn7m0diknOhPeMyLuLpZq/dRUtkOEc9HDToAxb4PIhw5NtWbTR+hpCXmWZAy0ZP/WgPTYTq5XpBgnl106TFrDFU1kImLZfu6edSD4i2DUXSUgz+acAk34Ae+EGne4VMcBSxBTftPbo+uJ1MAH728yseZjBkWbkNO6FAoE1y8ZB+Upz3wlSkKLLllqaz5sfYnOrCFitLmxL+ybd9QV1IYoCxCMJUCCvwJxVwvL1LMMZEeIHzFjINaXati1m5Z5Vcn16k4rtBTokdtPnGZyoIWnrlMT3bl5Q+2M+uYQe8ojDz7 vwPFH8Y0 kLtfsY/yckYeSOgCV/0J0ZsItOfF81Cg71M9MDsB5mLuKaNm2OQujSLkQpy+K7ho4IkHVKiBnuhhQQMGYzufVbF+Jx9aKsu9OOI0ze9mQU+cBPzoqhrR1I1+OcAjYlKSFFGZ3zFJblLDtccvV/cgrUhQPwdlhaRVNLwK7aOf6CP9oORyXRrtQO0MbjVPDbB1fRMUFDNpXtl0SYq1iDebqG3TGFtaNA3RizJljrnheR/Z13TCbZt3C2TMpomSjrKVbch+YZ83kiJvTkBQ9pE0sBATVKiYlAFGSxRJ0Hs69METzA1LjkGUonaDTInCxXhrUG+HFFVYzorzCy0jAjh7u7XojcLXfHf/uTK1GF9ucGgkApNwbVdkuIblDGYvqgbePqioEybAf1blk5QsUe3PTuDx4jRtW0ebDa0GaQl9/OLbnOQ4dA5u/Az+dzVTJAHXxxMZi/gWHmtZheD7XQqSt9DrpXCwxLi2AuxUJIiKXNY4ATWIxQM1kNEUG1w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Note that get_clear_contig() function is duplicated in the generic and the arm64 code because it is still used by some arm64 functions that will, in the next commits, be moved to the generic code. Once all have been moved, the arm64 version will be removed. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/pgtable.h | 14 +++++++++- arch/arm64/mm/hugetlbpage.c | 19 ++----------- arch/riscv/include/asm/pgtable.h | 4 ++- arch/riscv/mm/hugetlbpage.c | 21 ++------------ mm/contpte.c | 48 ++++++++++++++++++++++++++++++-- 5 files changed, 66 insertions(+), 40 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 2e0415fd5083..7c2938cb70b9 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -1644,11 +1644,23 @@ static inline int ptep_set_access_flags(struct vm_area_struct *vma, #endif /* CONFIG_ARM64_CONTPTE */ -static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, +int find_num_contig(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, size_t *pgsize); + +static inline int arch_contpte_get_num_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, unsigned long size, size_t *pgsize) { int contig_ptes = 0; + /* + * If the size is not passed, we need to go through the page table to + * find out the number of contiguous ptes. + */ + if (size == 0) + return find_num_contig(mm, addr, ptep, pgsize); + *pgsize = size; switch (size) { diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index cf44837369be..5ace4bf7ce35 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -93,8 +93,8 @@ int pud_huge(pud_t pud) #endif } -static int find_num_contig(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, size_t *pgsize) +int find_num_contig(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, size_t *pgsize) { pgd_t *pgdp = pgd_offset(mm, addr); p4d_t *p4dp; @@ -277,21 +277,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) -{ - int ncontig; - size_t pgsize; - pte_t orig_pte = __ptep_get(ptep); - - if (!pte_cont(orig_pte)) - return __ptep_get_and_clear(mm, addr, ptep); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - - return get_clear_contig(mm, addr, ptep, pgsize, ncontig); -} - /* * huge_ptep_set_access_flags will update access flags (dirty, accesssed) * and write permission. diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 0847a7fb8661..d976113a370d 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -532,7 +532,9 @@ static inline void __set_pte_at(struct mm_struct *mm, pte_t *ptep, pte_t pteval) #define PFN_PTE_SHIFT _PAGE_PFN_SHIFT #ifdef CONFIG_RISCV_ISA_SVNAPOT -static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, +static inline int arch_contpte_get_num_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, unsigned long size, size_t *pgsize) { unsigned long hugepage_shift; diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 437b1df059eb..a757e0b2f090 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -187,7 +187,7 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, if (!pte_napot(pte)) return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); + pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, &pgsize); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); @@ -202,21 +202,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, return true; } -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep) -{ - pte_t orig_pte = ptep_get(ptep); - int pte_num; - - if (!pte_napot(orig_pte)) - return ptep_get_and_clear(mm, addr, ptep); - - pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); - - return get_clear_contig(mm, addr, ptep, pte_num); -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -231,7 +216,7 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, return; } - pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); + pte_num = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); orig_pte = pte_wrprotect(orig_pte); @@ -249,7 +234,7 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, if (!pte_napot(pte)) return ptep_clear_flush(vma, addr, ptep); - pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); + pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL); return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); } diff --git a/mm/contpte.c b/mm/contpte.c index 22e0de197bd3..68eb1634b922 100644 --- a/mm/contpte.c +++ b/mm/contpte.c @@ -22,6 +22,7 @@ * - huge_ptep_get() * - set_huge_pte_at() * - huge_pte_clear() + * - huge_ptep_get_and_clear() */ pte_t huge_ptep_get(pte_t *ptep) @@ -33,7 +34,7 @@ pte_t huge_ptep_get(pte_t *ptep) if (!pte_present(orig_pte) || !pte_cont(orig_pte)) return orig_pte; - ncontig = arch_contpte_get_num_contig(ptep, + ncontig = arch_contpte_get_num_contig(NULL, 0, ptep, page_size(pte_page(orig_pte)), &pgsize); @@ -87,7 +88,7 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, int i; int ncontig; - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, sz, &pgsize); if (!pte_present(pte)) { for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) @@ -111,8 +112,49 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long addr, int i, ncontig; size_t pgsize; - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, sz, &pgsize); for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) __pte_clear(mm, addr, ptep); } + +static pte_t get_clear_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + pte_t orig_pte = __ptep_get(ptep); + unsigned long i; + + for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) { + pte_t pte = __ptep_get_and_clear(mm, addr, ptep); + + /* + * If HW_AFDBM (arm64) or svadu (riscv) is enabled, then the HW + * could turn on the dirty or accessed bit for any page in the + * set, so check them all. + */ + if (pte_dirty(pte)) + orig_pte = pte_mkdirty(orig_pte); + + if (pte_young(pte)) + orig_pte = pte_mkyoung(orig_pte); + } + return orig_pte; +} + +pte_t huge_ptep_get_and_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep) +{ + int ncontig; + size_t pgsize; + pte_t orig_pte = __ptep_get(ptep); + + if (!pte_cont(orig_pte)) + return __ptep_get_and_clear(mm, addr, ptep); + + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); + + return get_clear_contig(mm, addr, ptep, pgsize, ncontig); +} From patchwork Wed May 8 11:34:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13658622 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44939C19F4F for ; Wed, 8 May 2024 11:41:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D8C0D8D0003; Wed, 8 May 2024 07:41:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D3C598D0001; Wed, 8 May 2024 07:41:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BDC5F8D0003; Wed, 8 May 2024 07:41:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9951B8D0001 for ; Wed, 8 May 2024 07:41:37 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 554D2A0237 for ; Wed, 8 May 2024 11:41:37 +0000 (UTC) X-FDA: 82095038634.24.0F172A7 Received: from mail-qk1-f169.google.com (mail-qk1-f169.google.com [209.85.222.169]) by imf04.hostedemail.com (Postfix) with ESMTP id 816F54000C for ; Wed, 8 May 2024 11:41:35 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=XKI1DpcW; dmarc=none; spf=pass (imf04.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.222.169 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715168495; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=p6iTAuwd3BKroPVlGkkNGENGW7uKwPOOcXR0pGzhYec=; b=hdBfElNv+nvIATyComDQ1QNyttXj4pikQxQvlP1Cr36eRjzcwyQnl7dWytJKGgzkrXGVGe gva+A2riyx8EUdlj0BuHInFCEB9NfCoNDgzSB3RPSZ25nM+QrBKoKfMrft4ddWRrz6k9iw UA0mMBVxE7DRYo/yN29QlNQtzEq0yxo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715168495; a=rsa-sha256; cv=none; b=brjkyBh6coD5JQWKS8pxMhspTLTZ6nM+yoJTHTjpQY85zLCav8mbVYuDyJcq0LQqPX1UnT lQVAaKDECPmqm0Bg1wSQAExqOGWQ8Qchaf0mzSxyC9x4oLhHz5DYUrkzbM+VSc3WRO3Rq+ gX6ofxqK624WmRseHfSO6Gi2Ijt06Nc= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=XKI1DpcW; dmarc=none; spf=pass (imf04.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.222.169 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com Received: by mail-qk1-f169.google.com with SMTP id af79cd13be357-792b8d98a56so35487985a.2 for ; Wed, 08 May 2024 04:41:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1715168494; x=1715773294; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p6iTAuwd3BKroPVlGkkNGENGW7uKwPOOcXR0pGzhYec=; b=XKI1DpcWli9wwJubD8Nu+TMq7W5ZK/lBT7/DPgDv7cjusG8UCBUhrTcbEOJY+nhS7P csBbi0w4aC893K1UDt0yDcMxjekujtDUeUY8y5/5V1MWOkvLFJpF7dA4cEOMC9Pa0JUo 1yGCEQG29VOaxsJMKc9FYvE7fsmSa61PSlyc0AgGfUmhgzA+i4V/skhnLhKuf6eFLUS4 1MBBVq/zQeED/r2jTWokh45Et1ZgoBu9/b3HJj6R5HJthMtNJidx+6N3MtMhSY8DUlWI 91ShehUsSwb7evD5zVSgeY4T5SCsmYvluLB6qC/j0+b7VeownGYaf5BlV/dr8i2Lcds+ wZfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715168494; x=1715773294; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p6iTAuwd3BKroPVlGkkNGENGW7uKwPOOcXR0pGzhYec=; b=Iz2KEgrJuTm7DQbF4BydMs1tbMv5tAbK5IYPhW0HLzJ0hmkRC3W95zIIHpS2IOa1Ty fsLqkWlP1K8E6iCTK3mmtwWDEDAExZ9d2iWQFJpTK093YBptzQoBURL86zVefW010Ly0 lSABE4fEutstgIMoBn+XooxqZfur9p2Swfcpn0CQql/SKUMB9UiBUMQjsCYMTlBsxfQN Bbr0SOk0tJy5xZusA6EdaLyycPnPOaDA3EqaQfYbrEfWjPK0OOzMIOaVU3HWQn1mVxof wImNgl7hYorw41JChqiV4hl3HFAhs3ExLjATO1uMa1M63qyrvyJosvM6MKKRGFcF62Y1 zdUg== X-Forwarded-Encrypted: i=1; AJvYcCUzriXM8mjpT5eWM0ANOrHPkeHQyHuXCA+FZCP7+dlkLP35l6IpoEQSb1H6ryO8w7zOUNSUxiNDFzgRV/vxmwCg0Cg= X-Gm-Message-State: AOJu0Yz8YuUTSMtC0p+sBx2SF91DY++bCPIMWXfSPXjv5+iJJMsJ0vnS OW4JZDSFTVXcBx/yJ9gkuhwKSsx01GvU0Gm0NzJkrkyjuY5q0fJyMHNxV7OKZyI= X-Google-Smtp-Source: AGHT+IEf9c2lQsdOpkInfpCnqc/bzKruRrtsh40+8uzVrWPXvCoLmGvCA0UiasJn6iJt4vC5DbH2zA== X-Received: by 2002:a05:620a:394b:b0:78e:df14:19dc with SMTP id af79cd13be357-792b26d4b61mr350150385a.20.1715168494636; Wed, 08 May 2024 04:41:34 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id de42-20020a05620a372a00b0078d7213de13sm5753377qkb.136.2024.05.08.04.41.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 May 2024 04:41:34 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH RESEND v2 7/9] mm: Use common huge_ptep_set_access_flags() function for riscv/arm64 Date: Wed, 8 May 2024 13:34:17 +0200 Message-Id: <20240508113419.18620-8-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240508113419.18620-1-alexghiti@rivosinc.com> References: <20240508113419.18620-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 816F54000C X-Stat-Signature: d33ta136wyij7cr14mde9js6mzmq888p X-Rspam-User: X-HE-Tag: 1715168495-51737 X-HE-Meta: U2FsdGVkX1+/vrGDzPyONMGuzTztE6BO4AMd9rJQPU082BrLLVtMOzn+sweE0iYE7lRjYDdDqmbRyfeLK7XXcwC3wF4ud3Tru1pMlOEMBrG2pRG4mBQQLBnZQ99j8kkxq5LHNezA0aBvKsGZG0JWkDczPTiosL3qhuT4YyPcqgCETRSNR+obSfySayXeFjejvXjM5w64K+CcJdEPXnfQMvw7f9o8Vd8aOK0CFiTrgVuQGJQHKhW+5BDs2kguPEPktBsLPfxI6Ym55GHZ/p4KG96YSn5Edo5WINsG4wG4O1XuUHjqewNuhPIrbqX61c9MKzXuCJtkg/KckxdOOAN5fX5u/zJwF+1S+MgvNN9ejERWdeUJFNRFoMTGjdpJja5Ow0+hKNm4zs2Tb5x9/g2jxxaHy+2pZ9/5zdduhAS5Bbjc+LdFmBFGflOWgp6QwUptr57RWKhp2M2ikiRiUbm2GUnvDzw6LnOud55Q78tZXBhXGHzC5t3PJIJue2qTKqoYDoyxRKPluQuA5QhDD+3fV/z353U3PPfFeF3JOqR+VvdC4iRLi6BhjL5Nli1mIALqTVPThFdf5e+55zr8mNWuQiRm1krCMtvunKHPmaQ73icxcYk2XXZfTkDDAynb1XS6wMuFX3euJG3nc66KNyLfSxyhZvN12aguvBD+lTO7EtWhbsu/1cvuY2L0H8+jIvVs2ceeLX0Ugz2Qqrcq0/14fjmEc4Vc8UNpvx+OJ9t5XiHH0yfW+ZTxdID52flSdmGd8unqOSVfukeIqyZMk/wZquyZltXGYFhNdEoVyttGI7ZljgRHXTqHLZeiAMtnnHoQk4wqnC3c5+zCVPXspea+3XZdRdjxhQpWtW9Xgm+rCm9eKwgPdNUEnu24rEd+6qHvCKvoPolPu51IdA+gsL3rvllrs4+WBNpFDHJSXaeVeyFQTKqbxRUsBx5e8+/GM/pv6h1CEOmokFotpUUMSiB Nhvy1S0b M0hl4RskQfMvsgCpprxlGim5/A79VFSM1adCAu2y5zfN3nTNqVZc4ez23hdK1276sS9/PJXtvhOG/FlgoLaqkGVI8O9+NDsru2SNr52UVkqRtyflGbEliBGcW8TUKUqM1uROAIs3tmaULIIWuJA0UBP23jX56ciiwrj9hs4cxgD3s7lHjTcyrGiQ7wxdTEmn1AUrWpNKJE4zNpXy7U2ZiopTRw4iK1Cvr9BNP0S8UFfBTPj4CMBkdjamRwiN9B0tuijSsAvj8JkdwKSNrDC8sbx+jGMKSySMLLK5cxqV7Ncg5OfdDwQem0BQZQqneU/w99MnJ8IFG6NE+uSKFhhs9vKKki+ItARQH7YWzjPndMnnCFwXXQckqV2RlnNxueEATtF5qrmaZAbfUefizw6tSeeL0xTp1ZLP1VHbdVWaOOG0mbcxtWFLjw4Q5hG2hZZ633oGyNh25MyCTHi8IEYR9SUkVHjSimuQ9kHBWVWYmalr1SVHB4p09AeIS2A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Both architectures have almost the same implementation: __cont_access_flags_changed() is also correct on riscv and brings the same benefits (ie don't do anything if the flags are unchanged). As in the previous commit, get_clear_contig_flush() is duplicated in both the arch and the generic codes, it will be removed from the arch code when the last reference there gets moved to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/mm/hugetlbpage.c | 65 --------------------------- arch/riscv/include/asm/pgtable.h | 7 +-- arch/riscv/mm/hugetlbpage.c | 29 ------------ arch/riscv/mm/pgtable.c | 6 +-- mm/contpte.c | 75 ++++++++++++++++++++++++++++++++ 5 files changed, 82 insertions(+), 100 deletions(-) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 5ace4bf7ce35..052a5bf2926c 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -277,71 +277,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -/* - * huge_ptep_set_access_flags will update access flags (dirty, accesssed) - * and write permission. - * - * For a contiguous huge pte range we need to check whether or not write - * permission has to change only on the first pte in the set. Then for - * all the contiguous ptes we need to check whether or not there is a - * discrepancy between dirty or young. - */ -static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig) -{ - int i; - - if (pte_write(pte) != pte_write(__ptep_get(ptep))) - return 1; - - for (i = 0; i < ncontig; i++) { - pte_t orig_pte = __ptep_get(ptep + i); - - if (pte_dirty(pte) != pte_dirty(orig_pte)) - return 1; - - if (pte_young(pte) != pte_young(orig_pte)) - return 1; - } - - return 0; -} - -int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, - pte_t pte, int dirty) -{ - int ncontig, i; - size_t pgsize = 0; - unsigned long pfn = pte_pfn(pte), dpfn; - struct mm_struct *mm = vma->vm_mm; - pgprot_t hugeprot; - pte_t orig_pte; - - if (!pte_cont(pte)) - return __ptep_set_access_flags(vma, addr, ptep, pte, dirty); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - dpfn = pgsize >> PAGE_SHIFT; - - if (!__cont_access_flags_changed(ptep, pte, ncontig)) - return 0; - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); - - /* Make sure we don't lose the dirty or young state */ - if (pte_dirty(orig_pte)) - pte = pte_mkdirty(pte); - - if (pte_young(orig_pte)) - pte = pte_mkyoung(pte); - - hugeprot = pte_pgprot(pte); - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); - - return 1; -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index d976113a370d..20f62505d0bc 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -652,9 +652,8 @@ static inline void __pte_clear(struct mm_struct *mm, __set_pte_at(mm, ptep, __pte(0)); } -#define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS /* defined in mm/pgtable.c */ -extern int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address, - pte_t *ptep, pte_t entry, int dirty); +extern int __ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address, + pte_t *ptep, pte_t entry, int dirty); #define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG /* defined in mm/pgtable.c */ extern int ptep_test_and_clear_young(struct vm_area_struct *vma, unsigned long address, pte_t *ptep); @@ -703,6 +702,8 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma, #define __HAVE_ARCH_PTEP_GET_AND_CLEAR #define ptep_get_and_clear __ptep_get_and_clear #define pte_clear __pte_clear +#define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS +#define ptep_set_access_flags __ptep_set_access_flags #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index a757e0b2f090..a59b776e9c8b 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -173,35 +173,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, - pte_t *ptep, - pte_t pte, - int dirty) -{ - struct mm_struct *mm = vma->vm_mm; - size_t pgsize; - pte_t orig_pte; - int pte_num; - - if (!pte_napot(pte)) - return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - - pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, &pgsize); - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); - - if (pte_dirty(orig_pte)) - pte = pte_mkdirty(pte); - - if (pte_young(orig_pte)) - pte = pte_mkyoung(pte); - - set_ptes(mm, addr, ptep, pte, pte_num); - - return true; -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) diff --git a/arch/riscv/mm/pgtable.c b/arch/riscv/mm/pgtable.c index 533ec9055fa0..e86df7ef193c 100644 --- a/arch/riscv/mm/pgtable.c +++ b/arch/riscv/mm/pgtable.c @@ -5,9 +5,9 @@ #include #include -int ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long address, pte_t *ptep, - pte_t entry, int dirty) +int __ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long address, pte_t *ptep, + pte_t entry, int dirty) { if (!pte_same(ptep_get(ptep), entry)) __set_pte_at(vma->vm_mm, ptep, entry); diff --git a/mm/contpte.c b/mm/contpte.c index 68eb1634b922..9c7a9f250bca 100644 --- a/mm/contpte.c +++ b/mm/contpte.c @@ -13,6 +13,7 @@ * - __set_ptes() * - __ptep_get_and_clear() * - __pte_clear() + * - __ptep_set_access_flags() * - pte_cont() * - arch_contpte_get_num_contig() */ @@ -23,6 +24,7 @@ * - set_huge_pte_at() * - huge_pte_clear() * - huge_ptep_get_and_clear() + * - huge_ptep_set_access_flags() */ pte_t huge_ptep_get(pte_t *ptep) @@ -158,3 +160,76 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm, return get_clear_contig(mm, addr, ptep, pgsize, ncontig); } + +/* + * huge_ptep_set_access_flags will update access flags (dirty, accesssed) + * and write permission. + * + * For a contiguous huge pte range we need to check whether or not write + * permission has to change only on the first pte in the set. Then for + * all the contiguous ptes we need to check whether or not there is a + * discrepancy between dirty or young. + */ +static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig) +{ + int i; + + if (pte_write(pte) != pte_write(__ptep_get(ptep))) + return 1; + + for (i = 0; i < ncontig; i++) { + pte_t orig_pte = __ptep_get(ptep + i); + + if (pte_dirty(pte) != pte_dirty(orig_pte)) + return 1; + + if (pte_young(pte) != pte_young(orig_pte)) + return 1; + } + + return 0; +} + +static pte_t get_clear_contig_flush(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + pte_t orig_pte = get_clear_contig(mm, addr, ptep, pgsize, ncontig); + struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); + + flush_tlb_range(&vma, addr, addr + (pgsize * ncontig)); + return orig_pte; +} + +int huge_ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + pte_t pte, int dirty) +{ + int ncontig; + size_t pgsize = 0; + struct mm_struct *mm = vma->vm_mm; + pte_t orig_pte; + + if (!pte_cont(pte)) + return __ptep_set_access_flags(vma, addr, ptep, pte, dirty); + + ncontig = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, &pgsize); + + if (!__cont_access_flags_changed(ptep, pte, ncontig)) + return 0; + + orig_pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); + + /* Make sure we don't lose the dirty or young state */ + if (pte_dirty(orig_pte)) + pte = pte_mkdirty(pte); + + if (pte_young(orig_pte)) + pte = pte_mkyoung(pte); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); + + return 1; +} From patchwork Wed May 8 11:34:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13658623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27B38C04FFE for ; Wed, 8 May 2024 11:42:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B91B76B00C4; Wed, 8 May 2024 07:42:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B42486B00C5; Wed, 8 May 2024 07:42:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A0A476B00C6; Wed, 8 May 2024 07:42:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 821086B00C4 for ; Wed, 8 May 2024 07:42:39 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 4432D120A7F for ; Wed, 8 May 2024 11:42:39 +0000 (UTC) X-FDA: 82095041238.24.AF5B809 Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by imf30.hostedemail.com (Postfix) with ESMTP id 5309580008 for ; Wed, 8 May 2024 11:42:37 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=PQ1TCWqQ; spf=pass (imf30.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.48 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715168557; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=p3J74NkW9ioW00Pqnp327XZ4SYUtOBKlfZoGCCQmZ+M=; b=bZIzCTWxS3e1XzAKRnsyzyXaWkUq4iCg7nONP96EQV3IPGti+r3LAbEJwqk5wXgZS9CWBz phr0PyHlibSmsJyvnAqxYBnwd5jzpBSlKX9q8UKxVEZwx696MOhdSDsXRVRGc3e9qgw83B 09mKHjtklpKJc15fZvq+qwOpWr90EYI= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=PQ1TCWqQ; spf=pass (imf30.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.48 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715168557; a=rsa-sha256; cv=none; b=ebWcgVY5+qYCEczhOkEQkwlZhigESVsBEa9cbxal+J2dynuQXyJrTo2Ci+mUP5+cKTiN9v 81NdFEhCsWKFoTA1CAQODc9oUUvi4lM0fPaeKnz6Co+3bxOziu4EQuFCDqMDdDRKgv0NA/ 6B7sA0yBu0FRO05LhnVEKR5qYvtXaP8= Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-41b2119da94so27255925e9.0 for ; Wed, 08 May 2024 04:42:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1715168556; x=1715773356; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p3J74NkW9ioW00Pqnp327XZ4SYUtOBKlfZoGCCQmZ+M=; b=PQ1TCWqQ4meEUDLzFNKus6oE0YZrU1baGf1bDRsojSDsrBZjzfBT440EWvmEYFxOTq vh+7IG3oFxT+/GLwazWYHxxfW9LRsZ9L7p2rZwo/Iv5ltsRPBjpX4R7rsLeLQLRxkawo AEXd7li3wLEcHjnAbryF12DBD84IThLIMXY8M0hEpchq4LH2RjqIPOg/y7XiF12NEWER Fbf3655IpVEMogZ5MsCWJ4003FVP//yEpYs9y/9MP6GmoqBlXpJvGcciM388waL/znUh ut+qUXcjG3YyvkUdyRsTtRlxGLyXTfB1hx9T/B2cj8kpBQm3TP+Ht/Oy/BDEpkxPLbQG JCDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715168556; x=1715773356; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p3J74NkW9ioW00Pqnp327XZ4SYUtOBKlfZoGCCQmZ+M=; b=UEvrw1K4LgUDu19x1YK0i4Ah51ZWkhkt8NcVvjMCDDi7mweGKXkcU3mUWeMJOHS3u0 NmIz5cjPylt9ZdGkgj4B5QXTI7V4lPYxUS0LItWToicFNeA1Irqwv2EWmi9cV5o8Rzgx ODlv1wplzc48I3BglRRlAULcjQgWJIkaX+pDFxAcRRHrifBnwO7yzmNfvJ2hEMNFUBtJ aQnH/7/oe11f1sTPAhsbPDsknxh7040/BjVxDwjyJ0p/FXGs1QyoivFM7jOqHR6c11zx Ydi7BOmPoslimD2jGeVD9mc+Y7lbDbJH1GmSWB5XBnTHvR0L7Cb2IFhx2OpxQLmZa12u OPMQ== X-Forwarded-Encrypted: i=1; AJvYcCXNgwSebHPke2+rPCDfk8n0h/7c/HhK519u3aKhdakZ1asM79RGJuA0Nv33elSJDY6xK9J3Jo5lyZwiMgyeZS675U8= X-Gm-Message-State: AOJu0YzmbRxyrZcrB5euYjrwqKnwPmXLbZVfnF8K0infdnWdJKBmW6CN +08t+SgzcBp3Bn8X3gy532Kd4Kgf+/eGUa56DOHH8MRtXmd58WH5qmTH4/MxVCo= X-Google-Smtp-Source: AGHT+IHE8o6Es8ZSc8TwRFtN83wex84gZARiN4tVTDQiUBBDnfszL6lfcSMLSwFM5H55V+vhK3nrvQ== X-Received: by 2002:adf:b1d1:0:b0:34f:6c9c:19a3 with SMTP id ffacd0b85a97d-34fcade2a1cmr2221817f8f.15.1715168555855; Wed, 08 May 2024 04:42:35 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id h1-20020a056000000100b0034e0b56a62bsm15118070wrx.44.2024.05.08.04.42.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 May 2024 04:42:35 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH RESEND v2 8/9] mm: Use common huge_ptep_set_wrprotect() function for riscv/arm64 Date: Wed, 8 May 2024 13:34:18 +0200 Message-Id: <20240508113419.18620-9-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240508113419.18620-1-alexghiti@rivosinc.com> References: <20240508113419.18620-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5309580008 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: a7gg3i5rpfpfartk84upzqz8cgdmbn1n X-HE-Tag: 1715168557-997871 X-HE-Meta: U2FsdGVkX1/iTPwERXM6dsTFcYomqR2l2RWHCBoCG/SjGE1dMWd3DAzsCOvaqFbGHM0dDT/wTpHf8AXmfDhAxiEA9R+Er94iNjbb/c56B3zVaFO0kwgNh7039l3bJc16fPrM1IBJWfVHTPprhpltHhyqji9F6b2CPxKZVQB/fu0P2vHXWVePcDJ7YNqmYTzfAzt+qxnIJzTHbAKa61dO/w9ljAxp5ioTDEOHHymy3351n4GK6BQwfCZpYUJmg/Zywr8sYZFoDGIX5c5u6VGIBUq6iuq1P8EG81SZcoBZMT3pmGvgEwqDx0kgsPEnABYMrYQR2W7rmS3V/vFuGjerZeLRqCIc6GtUwYGcQ8Jbnq7g/TMI4saAaDacd4K0R7VdwVS3m3ry8lJgQ1/35p9bg9rOUI9a0/ia1+goHvgRJFLwTGMnohDSAXDit5uhSiewFg4onLKg5gZjSMJv1TvhsbFPAvwHxY8ZaDPEZEtn+wdrw2Zijw6r0BZa4v94H2SxCWEabW7VgdORqQFfDygFPCIgJDvyoolVNnOGTqLQ66r4DCNkwQYxKt8ONF2jgCLVazE6QWA605SCHqvyD8HwyaijqoDgc3W7g5twok3wRNlP9Q1YRtZnWH8QNYuWHoI7QX0FTBHjogwDS7tytHFE1FgKrQN30LkV1XKNJv6zcDlllvoybJDUHaAcFInzx5llPzvx4p7ZIz81Q+mnORy/kKaw6lW091SJrJXRZGEvQ/ZsTL5P8mDd4Z/z05Fo2g4C5PV3+1ngTEB0uir0p95Qruh1vYy/o5B5WSfGWnOMKNG5Ckc9F/Lp9Z7wxo6FJmOoXyeKT8SXWCGkJyV3NnxBl28NaLHTfVKiVkELTl4igcWMCf9RkZCHi5F8uHDxa6y4VBIFpr830upgm68ngSf2I/6WoprP7qujHJ9UGy0dqBPanKVrQpjezbCPjYW03Gk+LUKQ3MyUlAZNX2jEwgl 11G4R7Ct opvKqKG6a/Xw9BJISe+G6o828lPSiY9L0QtoAQK6gjej3cm4GNNYsMypZ3HKzxg7KQwgcQExki5jWDS7tozshs5T3e1okcXUR1xmtS/oIWDmKxwNiXaiwff0LOepWE9lwnSN6An4RIvT1WSgFSsXzjt55xukEsu/hnG5yBM/oVp5YhKFnZ+ZuQxFoFHpJB/LLVhjKju5GHypTkDUNdDK2yvB9ZJB20WwzuzGfqyYcwyLdDqLkwx30zKHK4rWyM8pQdxrEjhcan55Fr5pJrb1wzfR/PI+DpdyZ2o41ij+H537T0E7BZmroqxLIWiICGY10xzT4q7KXxAZxtgLvcrmrZjjIt5QY69tYJzk4/dIEcNuayS88VSsR/IRlPd90llszswIzfmxNr0znTLZZnlCJEtH8ui6OozLVOckRyGkkYLHAe7y7Dl+yWuHCk5rF5NNVRgi0f2YeRiRbrJ7f6fUdF0Qcbjo2kOsG5r+K+bxaL1p8QYJGclUSkdkTqw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/mm/hugetlbpage.c | 27 --------------------------- arch/riscv/include/asm/pgtable.h | 7 ++++--- arch/riscv/mm/hugetlbpage.c | 22 ---------------------- mm/contpte.c | 22 ++++++++++++++++++++++ 4 files changed, 26 insertions(+), 52 deletions(-) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 052a5bf2926c..e56f2c8ec7e7 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -277,33 +277,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) -{ - unsigned long pfn, dpfn; - pgprot_t hugeprot; - int ncontig, i; - size_t pgsize; - pte_t pte; - - if (!pte_cont(__ptep_get(ptep))) { - __ptep_set_wrprotect(mm, addr, ptep); - return; - } - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - dpfn = pgsize >> PAGE_SHIFT; - - pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); - pte = pte_wrprotect(pte); - - hugeprot = pte_pgprot(pte); - pfn = pte_pfn(pte); - - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); -} - pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 20f62505d0bc..9e397935536e 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -668,9 +668,8 @@ static inline pte_t __ptep_get_and_clear(struct mm_struct *mm, return pte; } -#define __HAVE_ARCH_PTEP_SET_WRPROTECT -static inline void ptep_set_wrprotect(struct mm_struct *mm, - unsigned long address, pte_t *ptep) +static inline void __ptep_set_wrprotect(struct mm_struct *mm, + unsigned long address, pte_t *ptep) { atomic_long_and(~(unsigned long)_PAGE_WRITE, (atomic_long_t *)ptep); } @@ -704,6 +703,8 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma, #define pte_clear __pte_clear #define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS #define ptep_set_access_flags __ptep_set_access_flags +#define __HAVE_ARCH_PTEP_SET_WRPROTECT +#define ptep_set_wrprotect __ptep_set_wrprotect #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index a59b776e9c8b..440d3bde88f2 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -173,28 +173,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep) -{ - pte_t pte = ptep_get(ptep); - size_t pgsize; - pte_t orig_pte; - int pte_num; - - if (!pte_napot(pte)) { - ptep_set_wrprotect(mm, addr, ptep); - return; - } - - pte_num = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); - orig_pte = pte_wrprotect(orig_pte); - - set_ptes(mm, addr, ptep, orig_pte, pte_num); -} - pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) diff --git a/mm/contpte.c b/mm/contpte.c index 9c7a9f250bca..8ad2a3099dfd 100644 --- a/mm/contpte.c +++ b/mm/contpte.c @@ -14,6 +14,7 @@ * - __ptep_get_and_clear() * - __pte_clear() * - __ptep_set_access_flags() + * - __ptep_set_wrprotect() * - pte_cont() * - arch_contpte_get_num_contig() */ @@ -25,6 +26,7 @@ * - huge_pte_clear() * - huge_ptep_get_and_clear() * - huge_ptep_set_access_flags() + * - huge_ptep_set_wrprotect() */ pte_t huge_ptep_get(pte_t *ptep) @@ -233,3 +235,23 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, return 1; } + +void huge_ptep_set_wrprotect(struct mm_struct *mm, + unsigned long addr, pte_t *ptep) +{ + int ncontig; + size_t pgsize; + pte_t pte; + + if (!pte_cont(__ptep_get(ptep))) { + __ptep_set_wrprotect(mm, addr, ptep); + return; + } + + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); + + pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); + pte = pte_wrprotect(pte); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); +} From patchwork Wed May 8 11:34:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13658624 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AE6AC04FFE for ; Wed, 8 May 2024 11:43:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 26A176B016B; Wed, 8 May 2024 07:43:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 21B206B016C; Wed, 8 May 2024 07:43:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 10A306B016D; Wed, 8 May 2024 07:43:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E742A6B016B for ; Wed, 8 May 2024 07:43:40 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 8DFE3141449 for ; Wed, 8 May 2024 11:43:40 +0000 (UTC) X-FDA: 82095043800.14.497769D Received: from mail-lj1-f175.google.com (mail-lj1-f175.google.com [209.85.208.175]) by imf07.hostedemail.com (Postfix) with ESMTP id A5BCD40004 for ; Wed, 8 May 2024 11:43:38 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=mxjZf9pJ; dmarc=none; spf=pass (imf07.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.175 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715168618; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zkDF3kPVXJf4AqMK0ZqTYH7UgAMmNVObdoVU0G1Ynu8=; b=ly3+L2XDbK/Fa2dg6bBh19uFJ+IcAbRy0c3elnWC3dSUk6x00J5L99DLtywXBGMqFDGQQK NX3OIr1RqdhTXU6/Y7l/AvnVTV3wN7fOCZl9HxvaJ95VLPY+1wd4XhFJgcAZopD267NQtX sINJUJn1Kpz2mmlhUShfvWwO0984ArI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715168618; a=rsa-sha256; cv=none; b=hd9oA2D/5z/YqHdRFHCS4F+/RfajmYs62LyeiZtygdQnV0igsF99b3RA2nDdo0TyFg+Jxn LFXGgV8ChVwWYIznJ66mzQWDQVNLxkiJZQT/8fSqRKvPkD1NZsP6Eyu6GlJ00eJKBdJrQQ 5SyUN1T9Ve1r8i/sqaiG+rmzrAsaESw= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=mxjZf9pJ; dmarc=none; spf=pass (imf07.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.175 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com Received: by mail-lj1-f175.google.com with SMTP id 38308e7fff4ca-2db13ca0363so64193921fa.3 for ; Wed, 08 May 2024 04:43:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1715168617; x=1715773417; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zkDF3kPVXJf4AqMK0ZqTYH7UgAMmNVObdoVU0G1Ynu8=; b=mxjZf9pJOJrXYS65uoN2AV6SJNUuhmc7x9QShm7xTGboTZ/ASrHT5S9YQefaLc09EY 0uhGp0MUM08kaSeFfcp+fBZQxA6hBLBLe31cdjEl53Ceul7yEb7iLBV2fa7zAQ2lu4gl +onbIitbUogmlW5jr+AVpSFldi0VuNV2mWh7k/5d8UYq0xrEC4xfN2y4sGIGHg1pR2IZ Jobg9p8EXeeCz6zJREnd7ZwMGYQSoSQBqShRYTYi5Mp4TEFqgZelVZ1rS1A2/gHsMmgR fLhMwUz5croxSDGCPcIknt3xlb7TZ52IJegfJMCPypFzGCejv1Mc4DHAldBb9GUIM54D 3SOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715168617; x=1715773417; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zkDF3kPVXJf4AqMK0ZqTYH7UgAMmNVObdoVU0G1Ynu8=; b=W3vBuDwpEzOF0t87v+Cpu9j9nwyv7yVpmhNlxhqo6BivpOQfU1nIG9jpngU8MEHWPi GqO3Lcxmna3Nh85Ocz5ZfzbuTuE1JPfP/i5il+vRrcphtaqfvL55AY7GZpbzfEU0CbSD +Y211OhHFPUqOg1i9hpkEmc2Ui2UaPXocYvW1/b/MfU3wqYb2fJFHJ1QtdmIicvT8W5w 2/8slEx3TG2QvCig8JYXatZiyixiDvR4DonAc9SaoTO4+LdXq0qHhIqkNMfiOBGNyPkJ 3qdDhzW9xJmSIBE8L1eAT1OoAMmK4oLA6tIMO/madxQSQ6mscSWMp4BrQnvZhs2QXRA+ lOsg== X-Forwarded-Encrypted: i=1; AJvYcCV/yx/C/XgClyIYziqGsM2ojjXPdK0xGjYjhTV4vxzovVmv5khNNT0WR6Ld+c3qyj2K67KGhiut1XSOiOTeX7EEIxg= X-Gm-Message-State: AOJu0YwNRCcH0aN0OvyYNWbUb+Dso11G1Kbl4sCoPkQ0/6GuT07kRivB 6RHbY+cQ2ftVdD+Su/Gqpncpfz2YpJk6UCYk/uKBX9bIH5r+QsOary1mnQag+QU= X-Google-Smtp-Source: AGHT+IGXkVEaOMFBHx/baYrBz9derdknpbq9as3brbpiTEdPwh/hYAl2dlBH3Ljix5Ca772CLWwlVQ== X-Received: by 2002:a2e:9257:0:b0:2e1:a504:f9ec with SMTP id 38308e7fff4ca-2e44708423amr19911671fa.23.1715168616936; Wed, 08 May 2024 04:43:36 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-41f88111028sm20610245e9.33.2024.05.08.04.43.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 May 2024 04:43:36 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH RESEND v2 9/9] mm: Use common huge_ptep_clear_flush() function for riscv/arm64 Date: Wed, 8 May 2024 13:34:19 +0200 Message-Id: <20240508113419.18620-10-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240508113419.18620-1-alexghiti@rivosinc.com> References: <20240508113419.18620-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: A5BCD40004 X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: mnwxzapddr5a6n6uuticjoxstdtkyx3n X-HE-Tag: 1715168618-557157 X-HE-Meta: U2FsdGVkX1+F1vAhGQChSsIjv9gdyXDAxPtGJQbSB+wBBfnc74HvbVIMl9o/8IwlwnPXjdQXThaTefGCCm32vDyNdJVqjI/S55AbwrN4F5BkFbYfpim2Jv3rOqybMErvYHTPgDKKciVJhi9hoSR1gJX0Y2LxcJnUUMxqO10Bvnpr9DsBSyKQicJgQLYf1Mlj9tMa9ONVAkjjcpBJjp/VzDvX0ddWI9mlCJCEHD26DTO+Y/UeXgSkRCAPlGvdNtDDZuUdlMOJkgvJ5PO7wjNLJjYmquCOK/es/7lzMRkHk1dbaE3v+u4VaqPAKjPIuzNd6fQlNnc7XI82g4i1XtUnbJ+06hG4T3vRNKcUs+0GNxPyebG9BZk752k+wnsflfpKzq93AXNe1eIGiC54CCYmpzebaBZUOKf76ZlsCVDTmqK95j0CSw9g5F+JvaR25EPf+GmZ0U1KeKCOyPB7m1j2NpC8n1Og/59q54d+GvMBggICnycAbA8d/8OlXHT1dPPVE7W1FeR5uNdLV/flVktr7XR6v/8905sqDq2/9rm3pxvKPA2E3N40pHCtPoszuceAi3LcyUtw/1oHiVuZdUZ2pWadkbkoRiOPAWV3QxPxPDHZkDXHJa2/xYsZK9qQH3pA2XLQiAV34KN2hXNcDlshkNDci2Q6cEK4hx8DxPAuevedyonobVjB/fBNJ5xO7OZXTAmA5ngEho/GESwOvVmWKq85DJhqeP09wxaInnBEwSYGZqz0CeGBNIhiWjeyi0PSB2NJ4+VP9XW5oT68sNx/PUpOX3/6eCIP6qroElVsUPY+K/43cTJaL15/96mFeGMdLMITM4OxKl8lPzzt3n41N1g4g6fMcCctaJW/a8ly+AHLu7p5Zyzm0xEqyyQFY+zm+XLYMxRK/vqPsO6+6sS3Izh+o9jVB0/C0SBgsK4HNM4Z8ylUhltZixiP/kgpc+0Kh3RV98/OjtCAfeQ2zHj pIoeTqd9 0P3/15IJDi+9DFVHg73mIOHvqbYse0j3d8pB87U+tZuCtVOZ81jeFryL7a71AP4cs7WKm3bCuSXq+OoaRe0Q/RbGtJUjHwlcM1WIhmYKbWRqm4HWlwrdP47Ctva0mtUIorO1lPox9QQupR+bjjO2iVlzjpVlfx23GHGHXpcGvOwH9sN9woS+o037hRCElKWmzXqZ2FWRu+uZdrpNjMg4jL26DH2M+RYbpDMd1NcessgSaJJZbTm/6oLjc9het83Lit+I5Onf/nD6ctFUWd6+kcC3XSShpgrTo2RWa7duk0qRUbdPr8by6ZUzofFSPMa1XVAlJP8tPTz6h9inpr331CW3+xrsoeJc2QeVp6Gd3ksq91hme8HJuG/tEXDOF/QzE2182QkKD0lmea0XgrLnJAWxTz0aR/PWh0e4rHMYL911hWq5gOIxSSBQTKpppJlwERR82lAgeEI0dZ/WMVNnpiLvXuN6D5AgLwPWZt2XhVVK1zOO79qD6+VwtTg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/mm/hugetlbpage.c | 61 ------------------------------------- arch/riscv/mm/hugetlbpage.c | 51 ------------------------------- mm/contpte.c | 15 +++++++++ 3 files changed, 15 insertions(+), 112 deletions(-) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index e56f2c8ec7e7..5869f20ca28e 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -112,53 +112,6 @@ int find_num_contig(struct mm_struct *mm, unsigned long addr, return CONT_PTES; } -/* - * Changing some bits of contiguous entries requires us to follow a - * Break-Before-Make approach, breaking the whole contiguous set - * before we can change any entries. See ARM DDI 0487A.k_iss10775, - * "Misprogramming of the Contiguous bit", page D4-1762. - * - * This helper performs the break step. - */ -static pte_t get_clear_contig(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - pte_t orig_pte = __ptep_get(ptep); - unsigned long i; - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) { - pte_t pte = __ptep_get_and_clear(mm, addr, ptep); - - /* - * If HW_AFDBM is enabled, then the HW could turn on - * the dirty or accessed bit for any page in the set, - * so check them all. - */ - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - return orig_pte; -} - -static pte_t get_clear_contig_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - pte_t orig_pte = get_clear_contig(mm, addr, ptep, pgsize, ncontig); - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - - flush_tlb_range(&vma, addr, addr + (pgsize * ncontig)); - return orig_pte; -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned long sz) { @@ -277,20 +230,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep) -{ - struct mm_struct *mm = vma->vm_mm; - size_t pgsize; - int ncontig; - - if (!pte_cont(__ptep_get(ptep))) - return ptep_clear_flush(vma, addr, ptep); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - return get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); -} - static int __init hugetlbpage_init(void) { if (pud_sect_supported()) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 440d3bde88f2..47333efa2d83 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -121,42 +121,6 @@ unsigned long hugetlb_mask_last_page(struct hstate *h) return 0UL; } -static pte_t get_clear_contig(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pte_num) -{ - pte_t orig_pte = ptep_get(ptep); - unsigned long i; - - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) { - pte_t pte = ptep_get_and_clear(mm, addr, ptep); - - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - - return orig_pte; -} - -static pte_t get_clear_contig_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pte_num) -{ - pte_t orig_pte = get_clear_contig(mm, addr, ptep, pte_num); - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - bool valid = !pte_none(orig_pte); - - if (valid) - flush_tlb_range(&vma, addr, addr + (PAGE_SIZE * pte_num)); - - return orig_pte; -} - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) { unsigned long order; @@ -173,21 +137,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, - pte_t *ptep) -{ - pte_t pte = ptep_get(ptep); - int pte_num; - - if (!pte_napot(pte)) - return ptep_clear_flush(vma, addr, ptep); - - pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL); - - return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); -} - static bool is_napot_size(unsigned long size) { unsigned long order; diff --git a/mm/contpte.c b/mm/contpte.c index 8ad2a3099dfd..15791f6d9c41 100644 --- a/mm/contpte.c +++ b/mm/contpte.c @@ -27,6 +27,7 @@ * - huge_ptep_get_and_clear() * - huge_ptep_set_access_flags() * - huge_ptep_set_wrprotect() + * - huge_ptep_clear_flush() */ pte_t huge_ptep_get(pte_t *ptep) @@ -255,3 +256,17 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, set_contptes(mm, addr, ptep, pte, ncontig, pgsize); } + +pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep) +{ + struct mm_struct *mm = vma->vm_mm; + size_t pgsize; + int ncontig; + + if (!pte_cont(__ptep_get(ptep))) + return ptep_clear_flush(vma, addr, ptep); + + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); + return get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); +}