From patchwork Wed Feb 5 15:09:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13961281 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FFB1C02192 for ; Wed, 5 Feb 2025 15:10:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C06ED28001B; Wed, 5 Feb 2025 10:10:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BB8B9280001; Wed, 5 Feb 2025 10:10:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A314528001B; Wed, 5 Feb 2025 10:10:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 85DE5280001 for ; Wed, 5 Feb 2025 10:10:38 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3D8BC1601E0 for ; Wed, 5 Feb 2025 15:10:38 +0000 (UTC) X-FDA: 83086227756.30.F68E31D Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf02.hostedemail.com (Postfix) with ESMTP id 8BFF48000C for ; Wed, 5 Feb 2025 15:10:36 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738768236; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xq0gzkQCuzS6qPgu3GdElgorB/nMOeYTLDAAFULqJX0=; b=ZZLOY6tvZBKBiNxvtqKjQSnhQr5qc20KP8zCQAwICdrhSI+cDFNBv+kL9mCISpKinVyp6D kpwFiclol3OsZT5/1ABBWGbHs3qIGjXD3nf+SXDUhALCKNDdSx1dStLyOZ3cXT07w4qbLB 9aOOLUKsKpihCkWDqmXt/5FzI1dx1JM= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738768236; a=rsa-sha256; cv=none; b=hIKHNYqHOLp7w+avOzS86Z1SBeOgnIJ/uaI4vzLftuB/kf9kixCJzeLAHigjWAVA6dPBra crTf4VRY0OoKt3patObv1LxOIBaQxCQ9KHm1JCotrI0Y8YQfi/NGPGZPRxnbDwF1ADV4aU nWqBO5oDyyXCD/uP08mMHAIl1l4d6h4= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5FB141C01; Wed, 5 Feb 2025 07:10:59 -0800 (PST) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 242593F5A1; Wed, 5 Feb 2025 07:10:33 -0800 (PST) From: Ryan Roberts To: Catalin Marinas , Will Deacon , Muchun Song , Pasha Tatashin , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Mark Rutland , Ard Biesheuvel , Anshuman Khandual , Dev Jain , Alexandre Ghiti , Steve Capper , Kevin Brodsky Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org Subject: [PATCH v1 03/16] arm64: hugetlb: Fix flush_hugetlb_tlb_range() invalidation level Date: Wed, 5 Feb 2025 15:09:43 +0000 Message-ID: <20250205151003.88959-4-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250205151003.88959-1-ryan.roberts@arm.com> References: <20250205151003.88959-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8BFF48000C X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: q3chj4dfdtdark16g8ndqdqw79wss7wx X-HE-Tag: 1738768236-802592 X-HE-Meta: U2FsdGVkX1/eEc1qwREp1Ap/MZkMvXfJSZfo01529ifbh3G6TeUnrmJTdng9VTzGd9H1jEa8NsMc39bW8LAAlH6SVzcihgw9DZsvcTnFS6h68hbxC6K/9qawUAHCiI1Zj8fbQLrkccpMqwvwysfpZ3OxSJDpGLttzOOC4y5ltn56VS3hEGW7p7l619L2mCfqRlFQISW2A3f7H6k1+xH328CHcjimpH08eKaHJ7Dm4VNv1hOVkg3RsGjgVvJeg/DeDBNhaWydDhkiy9dEuGltBWrOrPUdg/LnPORYly7ZZCH05G4BlOgSkQhCv26lZq8XVOPpIjLbvl7qAUAoLW5BIP142Ei48gBQv1HCirbAp0pWXUYW4HcloAcE509khmW2CgNVgm4J1VhJ7Tii+U0nTaOXMqgxf+TGQWaDSzdw7MAwXGZ4/x0bZraS0Q1MS4seGhmuh7denrBTzhOCFYcW+1noSVfKz5sZ42fw2MuAMoFaPfqdPu6Hs8SV8LyDmg4rowOmK580ifgeLDheBkeJOp2snq0oM1SfiueC+dCR3rsiDTWP026ZRGpbgmpWUUfSMOVtjlZ9VqMKUB5WOSL1+SNSOqygZ5tsoPOQeJOcySR0wxxFVF0XECFz2xp08YZgsIRKBvgrVBklzSGTU9h4xUyLJ7a3oH0nRTe+rR9WVQwXevlW7iUNMp2mZirvBo898nmZeg6EITJBmJKqOOclRYNNlu/PMFLRl71k4s1OjogvfOITKs0wBE6/Zk9Zf0MUFup5xNmW21nxy7Iqtm0158+pTeqrSX/cKF+xJljgB0mAV9Wup4R0XgUk92FZzZfQQO3JPilMDrEz5oUERgulBWEGaEaf5fWwrz0wYZc0KoZAan8RI+oR2jRF6ULBc4MRxp70ZKXjK7pbcAwBoSzbglr/cC/yKlq6rgG31ortayCagS6A+JMO6+1Z9/gYCdA01nDZxYkUF8xQoqzgchH spgq++32 ZbZf9a1rJjgV+TNqrFluCWtD/q9Sf1ovyj9MQeoS3RWM2yRKnXZHUcLYTDeEngkDshbiTPZlZYFT87FBpp29EnVj590+G0ca6YmYUJj55wL4gwhy9yq8vjHAk3x/g39oLUZ4mhgtuFw9vCsoHZVb73g8NMYmUT4obb4LY/tbxqsMGuBrR0aLwaIJYAvT8aQjTZOnBkhX0XCKnCjUOIlIX3YRHljWjTG0ZrS1p X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: commit c910f2b65518 ("arm64/mm: Update tlb invalidation routines for FEAT_LPA2") changed the "invalidation level unknown" hint from 0 to TLBI_TTL_UNKNOWN (INT_MAX). But the fallback "unknown level" path in flush_hugetlb_tlb_range() was not updated. So as it stands, when trying to invalidate CONT_PMD_SIZE or CONT_PTE_SIZE hugetlb mappings, we will spuriously try to invalidate at level 0 on LPA2-enabled systems. Fix this so that the fallback passes TLBI_TTL_UNKNOWN, and while we are at it, explicitly use the correct stride and level for CONT_PMD_SIZE and CONT_PTE_SIZE, which should provide a minor optimization. Cc: Fixes: c910f2b65518 ("arm64/mm: Update tlb invalidation routines for FEAT_LPA2") Signed-off-by: Ryan Roberts --- arch/arm64/include/asm/hugetlb.h | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 03db9cb21ace..8ab9542d2d22 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -76,12 +76,20 @@ static inline void flush_hugetlb_tlb_range(struct vm_area_struct *vma, { unsigned long stride = huge_page_size(hstate_vma(vma)); - if (stride == PMD_SIZE) - __flush_tlb_range(vma, start, end, stride, false, 2); - else if (stride == PUD_SIZE) - __flush_tlb_range(vma, start, end, stride, false, 1); - else - __flush_tlb_range(vma, start, end, PAGE_SIZE, false, 0); + switch (stride) { + case PUD_SIZE: + __flush_tlb_range(vma, start, end, PUD_SIZE, false, 1); + break; + case CONT_PMD_SIZE: + case PMD_SIZE: + __flush_tlb_range(vma, start, end, PMD_SIZE, false, 2); + break; + case CONT_PTE_SIZE: + __flush_tlb_range(vma, start, end, PAGE_SIZE, false, 3); + break; + default: + __flush_tlb_range(vma, start, end, PAGE_SIZE, false, TLBI_TTL_UNKNOWN); + } } #endif /* __ASM_HUGETLB_H */