From patchwork Thu Oct 19 14:01:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13429272 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 919A4CDB465 for ; Thu, 19 Oct 2023 14:04:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3558C8D001F; Thu, 19 Oct 2023 10:04:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 306498D0006; Thu, 19 Oct 2023 10:04:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1CDFD8D001F; Thu, 19 Oct 2023 10:04:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0EA248D0006 for ; Thu, 19 Oct 2023 10:04:49 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D1BB81CAB85 for ; Thu, 19 Oct 2023 14:04:48 +0000 (UTC) X-FDA: 81362381856.09.AD5571B Received: from mail-lj1-f176.google.com (mail-lj1-f176.google.com [209.85.208.176]) by imf28.hostedemail.com (Postfix) with ESMTP id 1ECC2C0160 for ; Thu, 19 Oct 2023 14:04:12 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=uUQyVn0w; spf=pass (imf28.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.176 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697724253; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/ApwA47W5sfKWNkq2IymUPAqINYWJl4bGVtXFqY2oGI=; b=1m0Bh/WxLCGZvAs61GEeAEg9C31HhmZnuDtWSu+bDFOA0hCYTQNb6fC2vQp7TE1E98yA1a RObZHIgm7eVRg+Tp6bsMA+XK4ugXyvovh+3em99VN860HP0jqgDkDlms/iMeu1q8PSUeEo cyNwDlUYQtx2Br7TbdDgnCc7DdThBj0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697724253; a=rsa-sha256; cv=none; b=8VgpiSAiWsE/p8Ti0fX9ElUaVAUgWx4JoLbGQAeg0KMybraehRoTnPiOQADeJxYUMPeNZ+ PluwdFqurRORtQHPABPXB655u94Ecu4QsnGoltINebexYKBS9V6ZPONJQ+sTELV4yNWRjG d+c9+pvbIMc72v7yCTMbGhdRSG0AyNw= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=uUQyVn0w; spf=pass (imf28.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.176 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none Received: by mail-lj1-f176.google.com with SMTP id 38308e7fff4ca-2b9338e4695so109319811fa.2 for ; Thu, 19 Oct 2023 07:04:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1697724251; x=1698329051; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/ApwA47W5sfKWNkq2IymUPAqINYWJl4bGVtXFqY2oGI=; b=uUQyVn0wIrZ57tLxJQ9hF4T0XR/9tBmHMXpYR1dR5hug4q5M327mUMv0BW74EUHuCf nNWKyPPFa1fJz+XkANt5fNf/JtB1k2IBD2ko356WI7Irqt+BtsiZQkkm4e12qH2K8V27 NYIv60chBFd0dNyD3SwSkTPG2zNS8miqK5DTUQQJUFYXKvOdAV/YEwUcfPb4/XCj023q 42tu/UQioFELIbvucx0HpUvjBTW2/jzMSWz3yVUUXW/C6RjsYp3exa1nkJCKrCDARoN3 zoiY4ofmoVHkDsBcjkFLtoHrJy6YvCdjuru92lQbZKc4HULdQFARQncRXdJMbDGSGmN1 wo8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697724251; x=1698329051; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/ApwA47W5sfKWNkq2IymUPAqINYWJl4bGVtXFqY2oGI=; b=hIpE3thNkbt1XBoLzPQtv55wxet4tv9ihGEwSsTJfWWqWUuhtCXGKhNMcVYKtClAWP zv93WNO6ZoH6mEMkPdiS/6vJIDNCoknin0RGxPuhnokisRHxWzbQHyyDKz56CiMxC28u lr2rKwHI56KxsxRZZcSeqPKGY1J3tr8h02Kf7HKfcxsvaIL8dl6aUGQPAS/g6LyvlyKm 6oTYzwWVYt655YsRDVZYCCo1CpOfUlPh/hNFtY7yv7nl2DhWarUByWyJg5KQMpihBlNu ybS4Fl1csiBVxKQqh0NQl/zjwVLBG0NwyZrZnWOLd2WxfnCRO+2VLi2JIcswwDsda802 WH7g== X-Gm-Message-State: AOJu0YyqeHmJP27XanmaZDBqFqYgfVd4yjuvwNYoAmDMCbKcW+QfL2nz O+Ej5a3TdUsY8HlvQVIiQuYxMA== X-Google-Smtp-Source: AGHT+IEBYCq6FlJQnZX7dLdouPmrLmiH7GFteySmGUDpOnC8cPojpOsH5qTZKNPlCuCg8e75j80KCg== X-Received: by 2002:a05:651c:1502:b0:2c5:1ad0:e2ff with SMTP id e2-20020a05651c150200b002c51ad0e2ffmr2017092ljf.39.1697724250744; Thu, 19 Oct 2023 07:04:10 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id az15-20020a05600c600f00b00406447b798bsm4543769wmb.37.2023.10.19.07.04.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Oct 2023 07:04:05 -0700 (PDT) From: Alexandre Ghiti To: Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra , Mayuresh Chitale , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Samuel Holland , Lad Prabhakar Cc: Alexandre Ghiti Subject: [PATCH v5 2/4] riscv: Improve flush_tlb_range() for hugetlb pages Date: Thu, 19 Oct 2023 16:01:49 +0200 Message-Id: <20231019140151.21629-3-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231019140151.21629-1-alexghiti@rivosinc.com> References: <20231019140151.21629-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Stat-Signature: 1mm63pnksr3cy4148cqqgrn6w85718sn X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 1ECC2C0160 X-Rspam-User: X-HE-Tag: 1697724252-219550 X-HE-Meta: U2FsdGVkX1/PC10ZFceGUBYJ3eWsB/hYsJM6ZSRr4ZRthQJyqQ/3DC2m/d+49otA6HT6U7CpuW0e9o10FzxDJBBkkxMH2YevOWf9kECZRBEyKt4435KQDTIUAQ25jYFCGfPLHYHIHR1MJOnm31zDjsSuGjB1H2aQPZ+myhe1iwZ0u6T87gEVhA4zfnQiTi6softg5DTGUK9g3qsofkhOkhif8B/XiUunO0jEN5vnUxF2VV2wJHfl0QB5Vj94aOqVvEV64Ml2s6ESSbhF7n1R1bT7KcO0J0a95zGtC+cbrZ8PaB1p6a/hv1VnYiBYLpf2hvd87uOKNtBzcVp8JKx+tOtiMeN0kfwx5B+lDHKDBmhPyP8AOA0BMMV4sa1Gmq2GYRxhzBf/LMUbNzVbZdKOv04TBZUeDktSSajUG2W3xoZdeU+zB5pucE0Mv5AOUDi0JpahwaHoR3oiHa6ED/cniNQa6CaYZRtzaFNm7qio0sVSb72NyXzgqWvwaScbXuyc45dtWfkYlx9CwDde/fQQOpDFYA94gsO72i3T+RfdSTYx+V8tNbE1wcSlZYzBa1jpbxCex6CdNvOFSrDL6UyoZtmuSHJEOizh+Icxq59a7a/1TLbq3OyxHUcIFldfE2q2XoTKXTIhQHOl1nftloCjN08HRUvnfLdEk3bdgTUf2RiSDaaBZ/2K442nnT2pfQmBPTVl9h1UM9TO8cdFR6fZpzxVzcfxqVdNl1LrDgVr13R46qyLopolDGD+HaZKDzbG56EG6WjL975yL7e+TcsDd0L8wf0EO94yKCLTrSTw4UgbTkerAtlMd0RBxUhI29VizirUC4BpUyNNVintP8yH251d8TJYrj0INcW6mkauU+IrSuUhT2iv6c3zwKpDKz6MP3DSf2/xAa3PV6IcbsROvi+oKlG3Agk36k1S61O96XeloXAWKzepqWq/RE6sNbFUYTw0OSgBx65pdY46P59 uoszMK4g 93acEdqoYR3Q4enKnI6awT11D+5BJt5oEdDyroNF3VxQie6JQ2/qumkLSYtAKamSf7Hyke3+ounKWjpEVXToIcZDnE0OQzsm6aaPUDcQNN3k0J1T0opyi1hlASEs/im8NbyUbATlPpE7Psk+2H8KNgWWpTxP0vmh/5ujauGHwhEGHf5oWZcIpgd3uKCqEo1e0gvj85CIqqRfYcxCJafniaX4QA2dRyfg/fm9h0LWNjeeJoJCWDxi3LOi2Ezv2IAA+vz2W2kBo19t3lEOhGi3Cg3aFfnQ1SeXHMfY1+qwsW8FL93RBHj3wHhcCYrDrkK222GV8HqAjipkKpaLgoVh30fS65s+4kLmayUhC9Z9SwYKa6B8TveC+FTucl03RpSj5HPDPYiYwPox0Vx0HRv4Scngk7o/sghCw2liSJZUXIp0hmmbX9+gCD+81buYSDIkIJdXLmq1V9dgYYQ5YdBWmulgubh1YPEbpBMaYxuxwWEME0Y74OOw5gjlwlfmn14ByOI+ym1mN74r+53xf9leCgDvLEJRxqfOpdPT4136AHwCGslrYhCRZTy6gELJHzVf4FV2CR0XppbQycPDWpAGG3UM8RwuEkVHasMYd X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: flush_tlb_range() uses a fixed stride of PAGE_SIZE and in its current form, when a hugetlb mapping needs to be flushed, flush_tlb_range() flushes the whole tlb: so set a stride of the size of the hugetlb mapping in order to only flush the hugetlb mapping. However, if the hugepage is a NAPOT region, all PTEs that constitute this mapping must be invalidated, so the stride size must actually be the size of the PTE. Note that THPs are directly handled by flush_pmd_tlb_range(). Signed-off-by: Alexandre Ghiti Reviewed-by: Samuel Holland Tested-by: Lad Prabhakar # --- arch/riscv/mm/tlbflush.c | 31 ++++++++++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index fa03289853d8..5933744df91a 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include @@ -147,7 +148,35 @@ void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr) void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end) { - __flush_tlb_range(vma->vm_mm, start, end - start, PAGE_SIZE); + unsigned long stride_size; + + if (!is_vm_hugetlb_page(vma)) { + stride_size = PAGE_SIZE; + } else { + stride_size = huge_page_size(hstate_vma(vma)); + +#ifdef CONFIG_RISCV_ISA_SVNAPOT + /* + * As stated in the privileged specification, every PTE in a + * NAPOT region must be invalidated, so reset the stride in that + * case. + */ + if (has_svnapot()) { + if (stride_size >= PGDIR_SIZE) + stride_size = PGDIR_SIZE; + else if (stride_size >= P4D_SIZE) + stride_size = P4D_SIZE; + else if (stride_size >= PUD_SIZE) + stride_size = PUD_SIZE; + else if (stride_size >= PMD_SIZE) + stride_size = PMD_SIZE; + else + stride_size = PAGE_SIZE; + } +#endif + } + + __flush_tlb_range(vma->vm_mm, start, end - start, stride_size); } #ifdef CONFIG_TRANSPARENT_HUGEPAGE void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start,