From patchwork Tue Mar 4 15:04:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 14000865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57B1CC021B8 for ; Tue, 4 Mar 2025 15:05:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E16946B0085; Tue, 4 Mar 2025 10:05:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DC88C6B008A; Tue, 4 Mar 2025 10:05:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C8F376B008C; Tue, 4 Mar 2025 10:05:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A6C676B0085 for ; Tue, 4 Mar 2025 10:05:00 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 1BB69A0F25 for ; Tue, 4 Mar 2025 15:05:00 +0000 (UTC) X-FDA: 83184191160.13.FFEFDBC Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf17.hostedemail.com (Postfix) with ESMTP id 3C01F4002D for ; Tue, 4 Mar 2025 15:04:58 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741100698; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FRBxlI11z+iOjTr34Y+Rzr3Tsdmm0TM9w6xycECWkVo=; b=8YN5wWmHV0ntOhwGX/WsSgLVaEEo0k9K5smd1tyx88hQ8yeRy5ybkQ4qKzcI8c2TtXiX4z B4HxaIqdY6I1un9rhFvYXLkyncGTWMur/JKkN55q0v+Caq6GIpAmZ/n6S4qQvIsQo6YLvF tqv2LhaLzi0aQKcmJ9itXPid+ZO5pdw= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741100698; a=rsa-sha256; cv=none; b=eeq614NzXQB4URwA01B2l3C05omjIrfFD2MglyksiD9wyU6lOb9ESVPO1DrtMI03nOkIJV w6EHSXzs8JNrQWomJN5LxA8WYg3K9YathEs+xJ/j+uC5To7sSn27azloWrHj050iLs0EVk 0qCsY4Lacie05q51MkbA4MdnUc7nHhc= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EB4EE1007; Tue, 4 Mar 2025 07:05:10 -0800 (PST) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 659523F66E; Tue, 4 Mar 2025 07:04:55 -0800 (PST) From: Ryan Roberts To: Catalin Marinas , Will Deacon , Pasha Tatashin , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , David Hildenbrand , "Matthew Wilcox (Oracle)" , Mark Rutland , Anshuman Khandual , Alexandre Ghiti , Kevin Brodsky Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 01/11] arm64: hugetlb: Cleanup huge_pte size discovery mechanisms Date: Tue, 4 Mar 2025 15:04:31 +0000 Message-ID: <20250304150444.3788920-2-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250304150444.3788920-1-ryan.roberts@arm.com> References: <20250304150444.3788920-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: fdfgctjd86rmhe4uufw7mi19g9amcmgt X-Rspamd-Queue-Id: 3C01F4002D X-Rspamd-Server: rspam07 X-HE-Tag: 1741100698-463317 X-HE-Meta: U2FsdGVkX18sMEsy5WHAnIYil5mKsFQqMNMoCH4KqLk1pwx+6O5ec9FncV4lo3POrT3oLiyVoRBe4dbBU0oF3ZLAe1bRFV1A9tKvA8xWA/c1Q3+QKdFl/TCPau9qGjVd4ZM/i6Cr9DG6twpATL/Bn3lnZo+HPYlN1zKe3aKEs2HZXgzeaPrKoYKfdqyJA9YznXsISRvKcdkqwcUKV7iWwAriUjqVHtTTBPYbmWApZhhv/9gxLS+P5FL0CgqGfhz3AqL8n3a1a9lImA1N+pBDP39ghxkZxDPITABY5DOz56jtrbDqN1XYsjQBw8PyD/rrSiXwGRSRkjicfMcUcPABMamnaA8y7gmv694h1zZ9PdUYkaGYykn8TZi4krMmsnJ1t+SwG1wRbuo0RWUHKYuNFMtcEQB7/mqK4WTcM6ZkbrwCePazgLnEpHF0fEKahq4aozmHhgKLanoGM3UqVIo4xt4+BomncOPKX1rqEhJrKZ/MUEbwDAjNZShSg7K4qH9s7Sq2X/3/fMayJv5cXAZMiPB2AGeoHpp4NSRK/PwmDgOH6KCjv+737/fcLXV0EcPDFFilJlzOQ3UEe+dBu8LpRiyXNh6DTyzkSSC/cY0irl3SjV3B2zb+M+6xUsg5zbJ+lRj8gqhsxemAS/Jbj1JOljbLZLJQhp7gCo4kNPzhpujCmdID979dHW5BrTfxbhA3VR8flOUQ/LG3x0Wi7XKAYx4/30ew6maja5a85QC9UEw/bqK1VloH9Ne0VarNBtrezB7gVhVFFQ+w4cvnYjTHPdVh+dt+ELJ469Zy+glIqPgVTWSiih2toRcM4zC/V4xpm2NOAmN0D632l6GRWdCefxo8COk2od8Toa0p3/jGg8U6JZPI7MjAP+8Qi4HUw08bhrb9WpfHx9N8CGHvzM9jI/52Zbv8vfj+ekl99WGNuaKwe1whAtgDXEbZqbzoIfQAx4H3kj+pYvska6+fL3C +PWC3pLE FCUL0fEdkw8IXN4+jcy54Br7HtMEM4J/k490caZqVGByNFYdZhaCnPjKHAt1nxRTY9jgSDqV1fbVe2Hlh3DuiUtp+LqEpdt4ii4cka6fmug/teV6VJWuii9IKk3iMjE7oz2Hqoy3W2nPhvv2ML91feA912yBPAmw+46GKApfkd8whvvorD1ZYCn4KHwljdG8sc7/o3d7SU7s4MsQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Not all huge_pte helper APIs explicitly provide the size of the huge_pte. So the helpers have to depend on various methods to determine the size of the huge_pte. Some of these methods are dubious. Let's clean up the code to use preferred methods and retire the dubious ones. The options in order of preference: - If size is provided as parameter, use it together with num_contig_ptes(). This is explicit and works for both present and non-present ptes. - If vma is provided as a parameter, retrieve size via huge_page_size(hstate_vma(vma)) and use it together with num_contig_ptes(). This is explicit and works for both present and non-present ptes. - If the pte is present and contiguous, use find_num_contig() to walk the pgtable to find the level and infer the number of ptes from level. Only works for *present* ptes. - If the pte is present and not contiguous and you can infer from this that only 1 pte needs to be operated on. This is ok if you don't care about the absolute size, and just want to know the number of ptes. - NEVER rely on resolving the PFN of a present pte to a folio and getting the folio's size. This is fragile at best, because there is nothing to stop the core-mm from allocating a folio twice as big as the huge_pte then mapping it across 2 consecutive huge_ptes. Or just partially mapping it. Where we require that the pte is present, add warnings if not-present. Signed-off-by: Ryan Roberts --- arch/arm64/mm/hugetlbpage.c | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index b3a7fafe8892..6a2af9fb2566 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -129,7 +129,7 @@ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) if (!pte_present(orig_pte) || !pte_cont(orig_pte)) return orig_pte; - ncontig = num_contig_ptes(page_size(pte_page(orig_pte)), &pgsize); + ncontig = find_num_contig(mm, addr, ptep, &pgsize); for (i = 0; i < ncontig; i++, ptep++) { pte_t pte = __ptep_get(ptep); @@ -438,16 +438,19 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, pgprot_t hugeprot; pte_t orig_pte; + VM_WARN_ON(!pte_present(pte)); + if (!pte_cont(pte)) return __ptep_set_access_flags(vma, addr, ptep, pte, dirty); - ncontig = find_num_contig(mm, addr, ptep, &pgsize); + ncontig = num_contig_ptes(huge_page_size(hstate_vma(vma)), &pgsize); dpfn = pgsize >> PAGE_SHIFT; if (!__cont_access_flags_changed(ptep, pte, ncontig)) return 0; orig_pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); + VM_WARN_ON(!pte_present(orig_pte)); /* Make sure we don't lose the dirty or young state */ if (pte_dirty(orig_pte)) @@ -472,7 +475,10 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, size_t pgsize; pte_t pte; - if (!pte_cont(__ptep_get(ptep))) { + pte = __ptep_get(ptep); + VM_WARN_ON(!pte_present(pte)); + + if (!pte_cont(pte)) { __ptep_set_wrprotect(mm, addr, ptep); return; } @@ -496,11 +502,15 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, struct mm_struct *mm = vma->vm_mm; size_t pgsize; int ncontig; + pte_t pte; + + pte = __ptep_get(ptep); + VM_WARN_ON(!pte_present(pte)); - if (!pte_cont(__ptep_get(ptep))) + if (!pte_cont(pte)) return ptep_clear_flush(vma, addr, ptep); - ncontig = find_num_contig(mm, addr, ptep, &pgsize); + ncontig = num_contig_ptes(huge_page_size(hstate_vma(vma)), &pgsize); return get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); }