From patchwork Mon Oct 30 13:30:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13440590 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1795C4332F for ; Mon, 30 Oct 2023 13:33:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4A67C6B01F4; Mon, 30 Oct 2023 09:33:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 456536B01F7; Mon, 30 Oct 2023 09:33:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 31E636B01F8; Mon, 30 Oct 2023 09:33:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 2160E6B01F4 for ; Mon, 30 Oct 2023 09:33:08 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id E3B92C0595 for ; Mon, 30 Oct 2023 13:33:07 +0000 (UTC) X-FDA: 81402218814.20.FAAE5C8 Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) by imf26.hostedemail.com (Postfix) with ESMTP id 146CE140028 for ; Mon, 30 Oct 2023 13:33:05 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="Os/AAsoL"; dmarc=none; spf=pass (imf26.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.42 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698672786; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Lg+2phEYGkxNvU/hCz5W3gC6On0lJ4QhzztTYgaxDUQ=; b=Fn33FcirvhUvnSkWXcvEEqoYsVDf8ezbmtjM6hHyHb/E0bIBqs+xZrhhWm0BYenANYB2VX MAIqDsrx/U0oZWPWEtgRYEe2EiBkmh8RMIcdSfMGUZi84gDEto+w6igZgexRm8YGO0Y7gu ynBzClbxXQkjrjQSLGXfp8PERW6DUKo= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="Os/AAsoL"; dmarc=none; spf=pass (imf26.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.42 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698672786; a=rsa-sha256; cv=none; b=Idl+b/elWphjtA3M28KiZJfrGcknbceBRO97F2pbA58cluO7bpV/rHodShz/Q3T6qXKtU0 J1ohruDJ4DLEBij//rOQMNfI3dPAmF6pjpo9W2m0wnkHcrlNyMTKHZzkV44H+hgVDk/JFN tHJUvLB3gJ0N5a3b84oxxlSwXDTXA9s= Received: by mail-wr1-f42.google.com with SMTP id ffacd0b85a97d-32d834ec222so2823976f8f.0 for ; Mon, 30 Oct 2023 06:33:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1698672785; x=1699277585; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Lg+2phEYGkxNvU/hCz5W3gC6On0lJ4QhzztTYgaxDUQ=; b=Os/AAsoLzxmGyjn5KcASvHh+KCVAGpmRKtMmsUM95Zq5vl6fhqH5LNt/8XL5Qc0cxY eQaykx8GJmnSFoYENQww/K8nkYMLtCAcHOXPgXSGwAVoacNBqoIwZ+Sm/FVfx2uyJ3TN TcDBQ6thfrsVQwQ45JrQynsxBl9I7hueFJrHH8l4RkgA3XBPAlZy60THHqps06kmNRJb 3LlV/5HRQFBObjqJhw4iGvhEm1Fzudm1ryfi5r/qrhVc1deyWr1wd58JeM7F712eWOEG NlpHyZuSt0d4XhhyY76sw5fiTgoGzXxjuPdqMJSqaPFseqEeBtWgGC2LtflZBXW+Lgbx BiMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698672785; x=1699277585; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Lg+2phEYGkxNvU/hCz5W3gC6On0lJ4QhzztTYgaxDUQ=; b=Qtw6TEOvIfsnFndp+KeL1ro5RpUFDURQ76FNlKOh76Urwox4XdPijjHcvIt7lFYmM7 2ZO2nmswe4SlLI92lnw4GFjjE/0/qRkFCEb0XlC5Zyx2IHU+aY3n4A++30d3leiltQ2y mtTikhzKsNdDqiUIaoQdJMDXi/GDPaMRBF8Rzo9U2R8Lcd9MlK43yurF3g+Phmv5X4eK yQw8aLHQ6wIMk8KC7M5yQy+H7iICkzeuT79AubfzcMgDv7mVimTpHXwDEyb1jNPj47Yb dvo5nqDGBP4fr2VAuuOgKPZGMM4LlR6CXO4k11CIisCi9KMqNPitbJ7Gsapw2x49gpMT Qlaw== X-Gm-Message-State: AOJu0YxNhuliRTIcad0LY1a82lAB+gfBOcJXeKbQINvgCBByVNpySBE7 gfCITNwO4oLb7VUhJ3F/LaZbnQ== X-Google-Smtp-Source: AGHT+IFSVGIn3GkkCdVjrv9GJYab3GvRdyYNVKTc+rLB8DHUCDxVeEERoRHJ6rwLZ6Dt9ZCmv4emsA== X-Received: by 2002:adf:d1ca:0:b0:32f:7967:aa4d with SMTP id b10-20020adfd1ca000000b0032f7967aa4dmr5491624wrd.68.1698672784653; Mon, 30 Oct 2023 06:33:04 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id d25-20020adfa419000000b0032f79e55eb8sm6061601wra.16.2023.10.30.06.33.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Oct 2023 06:33:04 -0700 (PDT) From: Alexandre Ghiti To: Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra , Mayuresh Chitale , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Samuel Holland , Lad Prabhakar Cc: Alexandre Ghiti , Samuel Holland , Lad Prabhakar Subject: [PATCH v6 2/4] riscv: Improve flush_tlb_range() for hugetlb pages Date: Mon, 30 Oct 2023 14:30:26 +0100 Message-Id: <20231030133027.19542-3-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231030133027.19542-1-alexghiti@rivosinc.com> References: <20231030133027.19542-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 146CE140028 X-Stat-Signature: r5shiiwxoxz88nakgr3nmqerbi9r7xaa X-Rspam-User: X-HE-Tag: 1698672785-369857 X-HE-Meta: U2FsdGVkX199nFpxxRXlc3/hDOpFXx72QCacSwKy1v6t5nUubXg+J3iqg+SgBNIK5ZTemCkopk+8JOlim7e7PoIbEaT27PgL2B5xmVYZSK4yjxHPqnQHGxoiHjDv2FoG9UJgTLffNNAqlAanPDSYhRJsIswOIs5UlPNKRGsf0/c0W/2Lai0Yzb5IGJr/Ok7aHhWc9YKbWwawyb7vi7XkUsPk1cLkFZlAU78nDpCQUaaTGOFJOd7zgEfjcH0calWMdypUZOEWzFeetNFnBwevpETyuOjc7kREYlgBUAjPCnna6xB9XOzu+nN/9o9VMJClsJTc7AnhfT/LhBDl1kcBOU9G6P9nTut8lzInAI5bDWZ8YvfUXf4MDHI3+EYVMfaHav63kqZ/brGOXvpmRsMpV2XNkV1dntN06OQSeygQTJtVCZUeQl6cfb/svj48e2EqaQF1aG7WEnzOfr0W2BMbtc9/VBgOZfrrI/uoouCUwsJsL/hKZmoXLxb8PEBagG3yUhMBdxj/3iRl/dheVcDT8cKcCHGD8Ahao20CkB5Uki7es4sUke2Qizvkbp2iCZTN296A19OyRX0QfwHnePUrLzMNaIcng22sBNAck34CfpO6SqnXKo+XoGztLFQegDpKEp9/ufP08Ks86ptAD9QB0XMFv7l8sWPKpki+hvK5nYucOi8gRwaXdXSdfqaf1JU/P5jKlORvL50LEZmLTfmTPTb+9OwsNx+dcgMAC3yjSnzi3clFSlpHHaeRufHzpm+NoeYuraP59Xa2rAA3qVPH0B1OtrjvvhqQYyllwP6+Y5RzfWdf1pRsrAtdw+PZaNKpYEEzHNb0z3DGPhwHm3ANwYp7SBwy1XzqczoWVEKFKUdpqfuzlPQdDhyQ60+TitFdWlJ5ZHpESKbakaWM1MP1/WqdQ6ckt/op3E3Gv4dUW6ryETJGMQFqUAEr4N/I36+xPjbjYeQcQ1ZFMG3wENt zF3aeiBB tRETHTnV7YnSzPx5ULTU0lFvEk5Q7eJxXd/ree/MuMT5fHSqF8n81WYujVLrldpAWq+ZMYksnv/3gSK9ERNYv5myiAd3ZQy0CtT8WuTZSgx7DkZDGqdbSvSRBBd5YyiVFJUfLg5dSCc34p0hKuGVYh3JgAprxYK/U1ssbfPYOPop+dJw2G340JYL5OE5jygmDYiuZPVVxS11DEwfVzbF61MIx5mQXpboIKovF4cYDBOxajHfH+ui6eDRnHy/nkTWUcz912aYMk8UprCMX6d2ZApu3iGR3d6hFni/yVYCERL7a9UxcCY3UvPKBYzFV8X3O3UMsQrntgViI3W/9zyFr7L5iqaxk+vXkFiHpBDGiMwYoX72m2RUXjvk8gzYUgwIlbE6hLo1E/AX4AKfJViIfXSaS90tjXSvFfjw5H17go3VQ2al5LUVmulxVL2wc3q6oIvr4rtbmJSuQut2uhOGlV8nYU3DfxuYpYbU2TqTSAYyLZ0b35qVnHOFHGn/pcwYcLUKOQqZenMos/UJ44K8EA7qVgYj12lN+mbYtHW5XNkKVo6IqgXUouDJ2LOUxAe9n4st38AJQkHYaHl6RRzK1HJo6PXLB0pKjocKry950mG8TE/5UoDwEyqlOG6UcYhYhPlQr5zIXQoaDHiA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: flush_tlb_range() uses a fixed stride of PAGE_SIZE and in its current form, when a hugetlb mapping needs to be flushed, flush_tlb_range() flushes the whole tlb: so set a stride of the size of the hugetlb mapping in order to only flush the hugetlb mapping. However, if the hugepage is a NAPOT region, all PTEs that constitute this mapping must be invalidated, so the stride size must actually be the size of the PTE. Note that THPs are directly handled by flush_pmd_tlb_range(). Signed-off-by: Alexandre Ghiti Reviewed-by: Samuel Holland Tested-by: Lad Prabhakar # On RZ/Five SMARC --- arch/riscv/mm/tlbflush.c | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index fa03289853d8..b6d712a82306 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include @@ -147,7 +148,33 @@ void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr) void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end) { - __flush_tlb_range(vma->vm_mm, start, end - start, PAGE_SIZE); + unsigned long stride_size; + + if (!is_vm_hugetlb_page(vma)) { + stride_size = PAGE_SIZE; + } else { + stride_size = huge_page_size(hstate_vma(vma)); + + /* + * As stated in the privileged specification, every PTE in a + * NAPOT region must be invalidated, so reset the stride in that + * case. + */ + if (has_svnapot()) { + if (stride_size >= PGDIR_SIZE) + stride_size = PGDIR_SIZE; + else if (stride_size >= P4D_SIZE) + stride_size = P4D_SIZE; + else if (stride_size >= PUD_SIZE) + stride_size = PUD_SIZE; + else if (stride_size >= PMD_SIZE) + stride_size = PMD_SIZE; + else + stride_size = PAGE_SIZE; + } + } + + __flush_tlb_range(vma->vm_mm, start, end - start, stride_size); } #ifdef CONFIG_TRANSPARENT_HUGEPAGE void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start,