From patchwork Fri Mar 21 13:06:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 14025420 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F34F2C36000 for ; Fri, 21 Mar 2025 13:12:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C1E0E280008; Fri, 21 Mar 2025 09:12:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BD020280001; Fri, 21 Mar 2025 09:12:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A703C280008; Fri, 21 Mar 2025 09:12:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 89F6B280001 for ; Fri, 21 Mar 2025 09:12:49 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 19A915A07D for ; Fri, 21 Mar 2025 13:12:51 +0000 (UTC) X-FDA: 83245598142.22.44826F3 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) by imf03.hostedemail.com (Postfix) with ESMTP id 1EF472000D for ; Fri, 21 Mar 2025 13:12:48 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=vJSQBUQn; spf=pass (imf03.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.48 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742562769; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tW5CZGKlc6dgOhT2HYLkHYDVp73Jiwu1Hc2Rif5lDWM=; b=voNrmu5RATlDqlB3eUkuP6h+MJ8FcsHeqbC430XfcJ6DuWX90ao06ODCC+z/fQJyvAhgO4 1j8TOHeFNtvVNdzOuCCO+VoIEGs4gIY+LkI2BEgl16CPZy7UrwBA1PgPrOtNCCScqFDHiV wwtonA9ajxtokToHv7IFCsQecAwC9LY= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=vJSQBUQn; spf=pass (imf03.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.48 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742562769; a=rsa-sha256; cv=none; b=iPcj4MbCXTvI8IDQwpSWYUB8E5X044BL0XhrueEMhG3XRy9mXGTtVLVbqxnwIH/zG20CwE qWMimd3I5P1mwtOoPaR/xjKw+kUGCd4h6KzdhUfBTwiSF6gO5xOqPAEGTnXLfK5qMzFD/y z5Bcv36W1seYZAJfc5HUPXmWMcDirkE= Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-3913fdd0120so1168100f8f.0 for ; Fri, 21 Mar 2025 06:12:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1742562768; x=1743167568; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tW5CZGKlc6dgOhT2HYLkHYDVp73Jiwu1Hc2Rif5lDWM=; b=vJSQBUQnyLzaPHbtsym4gg98xwszuu3W5884jO3DvZ3AVFYik1C4bAWPn17M0KsPAL xHvdbGwMWH5VblHMDoPy/al6jG9c7Ti5D18lpJ/nB/jItu0ZBsW1lr/muQNXB9jknkg3 DKl9O+rMt51Quf4UC63oOgImeiqTJd/rcf1blF7TGHouUZBz1EKPzBgV2zZevq2Lao6a uX3ytN70rzro8NeK/nW4hXYVayDAzd9BX+MutXQVBVziOWAfXU6P1Wg0vuJwhNTMLK9o uWPnF9qdKKCj2EpBJwpbBjUk58i6bnoQ7eDzMwI+r+Z3FabYgdiI41zP3+LRcD6fOh9l tVIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742562768; x=1743167568; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tW5CZGKlc6dgOhT2HYLkHYDVp73Jiwu1Hc2Rif5lDWM=; b=WyHC9DAjrkJroeEzMz1CJ+VPDbvu4+Sygns2FkvKxDWaIyeVBAUQRi11wPQdRPU5xi 7iIu1Cb6G/CXXC3L6kz5rYc8wFsnB7XKtl7QyTy/dLuKIFCKJS/hynRWU5092vpkG2ER 5Ntyb38mc8zH2Tp/aFBsjyP1g7xMSylrKzfSoOja3heS8M5N7IxMMYYqFSRuISRiSp4F ypj3Ai0E71GUHMZgIPhgxSJDIXWMI/CzszRs5LG40khVTxNJLJIfgvlrUHgP1Dp93ylc Rdnu67m8dRBksCxubpKCxRY6BRWICAclF97RM3bERaouP+uFc6TS7hQi+eMLddc8As/4 5vEQ== X-Forwarded-Encrypted: i=1; AJvYcCV4fPnZAxRn/lVOR/YWTiiyu/mAMpMU273NA27qjZVX527nvd86FHwfEoivQVtvYiAoderJoUku1A==@kvack.org X-Gm-Message-State: AOJu0Yxfzj4rPR7YTnGucAgb2CmapZ66h5vFiAeavPcidsoBVs4z9LfU tonEAVRdxEqjAEQSu7jWIEvpu5daWw/mX3WByKBjPXUMxuhug+VD9XJ6xbCO8t8= X-Gm-Gg: ASbGncvracIXUuefk0Pq+YftQQSmMwF9ZYRGc9MKbiwXKPoZ9roQK8I63EIeGvvYeg5 OU2+hg903RT7KmhrZ6cTp+MdfPd4RI2iG2Dvr6iGl70dwPtF7Yyi8T+x1ywav5xO0APt6c0OztH 1S16YLmaIHNg5SX8YgjhAqBNTFjFZK2q3LeM+v4l+OVGTpguXfLvwddYYL92sLToQ4LVSo3u7S8 iBL+jJbAJJl1EyX8jo8GZ35YvW+VyZ1/vBugc781LZzciggT3ktfhWPeEG4ehvTq9wOhjECdZ9c 76Au+RXqx1OZLjtAeA7HpiMIJmhS2T4/n0e2APDgOPG0ZzXhpIWa3x/mG1Ojj4pa5j6M7w== X-Google-Smtp-Source: AGHT+IEUtROIBbu/cy/pImYWjl+q4lRngYo9z7idT6KT9+6QFPXCkP6BZJtyCAPZK3CSSAem4Gc0LA== X-Received: by 2002:a5d:64af:0:b0:391:2db0:2961 with SMTP id ffacd0b85a97d-3997f932efcmr3394631f8f.38.1742562767493; Fri, 21 Mar 2025 06:12:47 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com ([2001:861:3382:ef90:3d12:52fe:c1cc:c94]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3997f9e66c7sm2356167f8f.79.2025.03.21.06.12.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Mar 2025 06:12:47 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Matthew Wilcox , Paul Walmsley , Palmer Dabbelt , Alexandre Ghiti , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v5 6/9] mm: Use common huge_ptep_get_and_clear() function for riscv/arm64 Date: Fri, 21 Mar 2025 14:06:32 +0100 Message-Id: <20250321130635.227011-7-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250321130635.227011-1-alexghiti@rivosinc.com> References: <20250321130635.227011-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 1EF472000D X-Stat-Signature: 6geea4zbh8gbiyb9dw7hro5egjat59ga X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1742562768-723644 X-HE-Meta: U2FsdGVkX18NiDG0t9rIXpS6QGlbhKJNSJ0tjn8NILwBlxQRFr/181bm30TKWpVy2Kt/nR7Pd8lSUymeGUcnnybkFwI2lnJPxNiYPK5qfvvxxWQruD9iscNOcTghDBaCa6CkTdnxpr1VPwDmy9BSb0j6naT3YkjEE8ATlF3I7M2R0b6jMil35Bns05mlDzvjm5MtLiKPxCpZ0FuSTUXbiAzTlp760HYznKigat5gQVL6KFEAR/tEdX3Zxyg7lKOtogXLCH8T4z1/XHGLGpAO5kLM6bzrnW9LygHnKQUvez3d22kNrCeMM5ZSwyxz/79Hh3ObNz/LNs+UDOt1I64eAud8FOV5qq6QQYFrakZQi2HorN4F6apA9mFBHe8Kw702d7zQq9TOsvt3XTQtwc0hMvvUJtnefTy+wgH3+qBfloHkZd55aBaga3pff/AxfHruw3vEFvTI9214691glk/H02QI5Fn9x4s9NTFyI1egq6AsM/PJrrwvSs4uLkTrDIwSTKPL3jm3lWhf322QH2qvGPoWKRwuYCeOSOJBub3WKs6KsPgWYFxY0JVb/s56TzVZwuVeYZEdlGARFJTkmw94aKfPGsS5qg1beYhHC1DFt5PlhHlU9aR+2iHV2+B6uw17PfgSEit/p6EV4n2m65MBAivgcvOY7NAuCWrBPXlQyYjDLS+k/3+ddNi2yTFFnDhOCoUrFwoMLEqSZ41gbhgsQ5hePvob/omKAGEdlsnmN7VjKEFiE+l5dT9StkaV1jrc4LTRjCUC0O1SY3V+xMP9AkASgKlLGRMP0/ECKvkXf3D5NCw94GUpowIvnPrWgNPBaLplcH9Dv9ZBDD8PfgdhXM8bssw7tOkijEdVdLRusGWIbo4A/+w213xOfzywo+X6o9IJgZp1FQivZZSt8FjhH7setCo1V9B3ucPP8GIT1eZUCfuUmzY8AOwptKulk7gzi2u+Eerfdr1j+7sbEZp AkMtFj/k Wo9zLefsPYtqBVJu+uxfIbrFBe7RVQQQPlvuX8HhNsKluzAgnTmuYZNRaa9WHoMA7n1hbSSC9VQ2ydcqU5PhsqPS3pSsThrbxeLa3WoRQHVqkHIkYjU+O8Vmzg/7h4i6VjDX8TfycORfflTXpvvWWhlcv+5WABp5ZYWlSiaGOVei12EP9eLMtE4apg/Kt5BdIyVQlnivMdRYIcVSEzdZxKgT5SHYbLzsjIxvWQD0+RYvpfMr+Jy2mQPUgEiXS1hrmKR1ZSc5LrmSdWJferhATrUxC7tNyZ5w+Q+LF5K06mRVnMp4yHegIerbqrMJmwNXfat/0Hi12sWjtw+2TXbtVhnoRiQX+Tj9Y83J+71bA2CyZ5fCR0EixhsQdOgspVhMnzAihVx5Rol/qSo8fgdq6V2+h4n1UByVVNji5T0n1BdrpmPoZsWYMIu/N3YAEXpe3eBsA4zU+zhYYxg95tvvVVAbziw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Note that get_clear_contig() function is duplicated in the generic and the arm64 code because it is still used by some arm64 functions that will, in the next commits, be moved to the generic code. Once all have been moved, the arm64 version will be removed. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/hugetlb.h | 3 -- arch/arm64/mm/hugetlbpage.c | 10 ------- arch/riscv/include/asm/hugetlb.h | 5 ---- arch/riscv/mm/hugetlbpage.c | 15 ---------- include/linux/hugetlb_contpte.h | 5 ++++ mm/hugetlb_contpte.c | 51 ++++++++++++++++++++++++++++++++ 6 files changed, 56 insertions(+), 33 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index ed75631ad63c..9b1c25775bea 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -39,9 +39,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t pte, int dirty); -#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR -extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz); #define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT extern void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep); diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 99728b02a3ca..62a66ce2b2fe 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -260,16 +260,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz) -{ - int ncontig; - size_t pgsize; - - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); - return get_clear_contig(mm, addr, ptep, pgsize, ncontig); -} - /* * huge_ptep_set_access_flags will update access flags (dirty, accesssed) * and write permission. diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 467bc30c2153..0fbb6b19df79 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -20,11 +20,6 @@ bool arch_hugetlb_migration_supported(struct hstate *h); #endif #ifdef CONFIG_RISCV_ISA_SVNAPOT -#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep, - unsigned long sz); - #define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index fe82284c3dc4..87168123d4a2 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -203,21 +203,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, return true; } -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, unsigned long sz) -{ - pte_t orig_pte = ptep_get(ptep); - int pte_num; - - if (!pte_napot(orig_pte)) - return ptep_get_and_clear(mm, addr, ptep); - - pte_num = arch_contpte_get_num_contig(ptep, sz, NULL); - - return get_clear_contig(mm, addr, ptep, pte_num); -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpte.h index e6aa9befa78c..1c8f46ff95ea 100644 --- a/include/linux/hugetlb_contpte.h +++ b/include/linux/hugetlb_contpte.h @@ -18,4 +18,9 @@ extern void set_huge_pte_at(struct mm_struct *mm, extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, unsigned long sz); +#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR +extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep, + unsigned long sz); + #endif /* _LINUX_HUGETLB_CONTPTE_H */ diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c index e881b302dd63..82f49eb79ffb 100644 --- a/mm/hugetlb_contpte.c +++ b/mm/hugetlb_contpte.c @@ -98,3 +98,54 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long addr, for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) __pte_clear(mm, addr, ptep); } + +/* + * ARM: Changing some bits of contiguous entries requires us to follow a + * Break-Before-Make approach, breaking the whole contiguous set + * before we can change any entries. See ARM DDI 0487A.k_iss10775, + * "Misprogramming of the Contiguous bit", page D4-1762. + * + * RISCV: When dealing with NAPOT mappings, the privileged specification + * indicates that "if an update needs to be made, the OS generally should first + * mark all of the PTEs invalid, then issue SFENCE.VMA instruction(s) covering + * all 4 KiB regions within the range, [...] then update the PTE(s), as + * described in Section 4.2.1.". That's the equivalent of the Break-Before-Make + * approach used by arm64. + * + * This helper performs the break step. + */ +static pte_t get_clear_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + pte_t pte, tmp_pte; + bool present; + + pte = __ptep_get_and_clear(mm, addr, ptep); + present = pte_present(pte); + while (--ncontig) { + ptep++; + addr += pgsize; + tmp_pte = __ptep_get_and_clear(mm, addr, ptep); + if (present) { + if (pte_dirty(tmp_pte)) + pte = pte_mkdirty(pte); + if (pte_young(tmp_pte)) + pte = pte_mkyoung(pte); + } + } + return pte; +} + +pte_t huge_ptep_get_and_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep, unsigned long sz) +{ + int ncontig; + size_t pgsize; + + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + + return get_clear_contig(mm, addr, ptep, pgsize, ncontig); +}