From patchwork Mon Sep 11 13:12:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13379316 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3F78EEB57D for ; Mon, 11 Sep 2023 13:13:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5110C6B0293; Mon, 11 Sep 2023 09:13:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4C1C46B0294; Mon, 11 Sep 2023 09:13:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 38B0C6B0295; Mon, 11 Sep 2023 09:13:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 266E86B0293 for ; Mon, 11 Sep 2023 09:13:32 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C0EA740A4B for ; Mon, 11 Sep 2023 13:13:31 +0000 (UTC) X-FDA: 81224358222.30.CAF9A7D Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by imf04.hostedemail.com (Postfix) with ESMTP id D4A2E4002E for ; Mon, 11 Sep 2023 13:13:29 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=CBoEHvze; dmarc=none; spf=pass (imf04.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.43 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694438010; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VJvQhuwass6T7QyQKJFHTMKHPeCPiB8wQ+ZaCt6WMq4=; b=MLFv/XpdSf51yMBcvPm8GFcP7OQgqJIS04lP30LX3UyfKbFpMdRqLECnAgm8F+nyp09LqS IJD1pBnACD0H8pOXN94LWLBZLysvX72yy8EJHh+npKSWjWbV18k3ga80koNePuHdlRxUKL xxam9Z1t4VsOS2bhyUAKk4piVObDrqE= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=CBoEHvze; dmarc=none; spf=pass (imf04.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.43 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694438010; a=rsa-sha256; cv=none; b=eyEgCUlM0V708pcqGYPkI0lFDgsQD8w0zScYSsmIbXCZgY/xlorw0yS4HOZavfejZRsaZs Cmvadjp6W1btWnALQUh8oRUoi9+6OjUMy4dKd4jM7q3GNBDp6YedONQHGT/xG9CBThZzCF Xt3f/D8hJgxqQnGvzoIdMkyQt5qPY84= Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-400a087b0bfso47216695e9.2 for ; Mon, 11 Sep 2023 06:13:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1694438008; x=1695042808; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VJvQhuwass6T7QyQKJFHTMKHPeCPiB8wQ+ZaCt6WMq4=; b=CBoEHvzeHX10bKQkPqtjxM/o2Q+ljQDKQxYAs9wt73hDtt9v3HdD4Z73yqlbPgu+MH DoTOJDgTsh+Nj4ZXh7Z7crWy2pc6G1SHqmddak3U9xohyhET4mfupdIxwY50EAKQvjXr egNbzzPrLTn96y/6wb4fGQmCTslhzqZNNdwKLrIbGazNCzTq5ABvb7gXnXhcAV1qXber ON7j7U2QzcshrF62C0Wr6JXM1mvoYexJR/owiayj4N2Yk37NhYRC5s3QRj9Xy4IcplZ1 QnAv3CWjwjZzzyUDdpD8jqCjGSV8x8dnUWmscBWSIXrmzOGFiw09adEFSL1eUrRDw3Wo 2nGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694438008; x=1695042808; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VJvQhuwass6T7QyQKJFHTMKHPeCPiB8wQ+ZaCt6WMq4=; b=M4ZJA53/q3dwqX1J1aVFKDUgIcLHaqr2HaEXC1XVyGTre0KVsf29nuhRNVslilfRZt S5RKzEEEiU0ZctVeOeQfwOhCOlt2+97qJz4mB1PvHDU4ixva1hP69ehfNKFZEL3cPrN6 /YGVjXQ/Pld8lwDTWx6Db+BFqi/ypryRqa0J/VcdVjyVpRiZAN5l2P0r24VOX65cp4zO M801boWstg7ztZOOnRfAPZ99NExQ6Q77zpXtDBHmcdP0l5iEmwX0ttC5DycfTkcSvhDK 5NprBCaYIjkdGpEGklFKp6s4jjSKtB6jDsuI3FAizv0pPXXQKRAsUUIl2C11tkKaXpcV yQWA== X-Gm-Message-State: AOJu0YyV6+ZrRGv1TEop5ZcDfB4X6UUO7JmiLDobqMG2AbRYNvRXPN0q 1u6RR7Xmz/I/NWrFLW6FSnNnFg== X-Google-Smtp-Source: AGHT+IHDNb5CqLim35ose2q7WmJ+9tXpgSAwI6QOLAumXk8SMHKSu15qIwBWXUEtVo37x5UyrTFz2g== X-Received: by 2002:a05:6000:118f:b0:317:6d9d:1250 with SMTP id g15-20020a056000118f00b003176d9d1250mr7674314wrx.61.1694438008262; Mon, 11 Sep 2023 06:13:28 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id b14-20020a5d550e000000b0031f34a395e7sm10072174wrv.45.2023.09.11.06.13.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Sep 2023 06:13:27 -0700 (PDT) From: Alexandre Ghiti To: Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra , Mayuresh Chitale , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Samuel Holland , Lad Prabhakar Cc: Alexandre Ghiti , Andrew Jones Subject: [PATCH v4 1/4] riscv: Improve flush_tlb() Date: Mon, 11 Sep 2023 15:12:21 +0200 Message-Id: <20230911131224.61924-2-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230911131224.61924-1-alexghiti@rivosinc.com> References: <20230911131224.61924-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: D4A2E4002E X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: hgo6ipyjyd1t3cgxxpfybs1qrrs4nhsa X-HE-Tag: 1694438009-622289 X-HE-Meta: U2FsdGVkX198mHWjmLKgku2tqBHA9gXGBwXun9HenjuXWWYc462GME5j4ukz8pMs2Y2kzwOmOBxXlb6efXUh9PEr06uKxb0bH3x1kTArpfEEq97kvaYUiMQYISZUyT7ySKgQ2i5dNBFPso8NKcU4WOWZrKxCfPbam3vcOcobjgBu5RUQZFu3AE+SiiHURq0gJ5IeKKBi60aD54SEEVBVgmaxy6g9pa/z/gpl5Uu2RfbtvqEx+5Xcc6LGQon+PYVQWI8hPUBdX1PbsAlUO+v3Egaug1u0BkoYjnnXwG/kKbyKhLADOeVMKieZN53XAFEAmgIWpYalqWM9DlBX+q87I188mhyecTbYJjE24aC47/RNrxcm5te1aqImHJKx2JTkmuzMtlT6V6EiIWvg/eVvBhswCbt6Rg7ARPY+3YXNQy31yuw4jYzZo7HByybSomypAxx+n0CIu/ieSi98agU+Fz+aDZSxryo93jXnxICp/A9MdUtiVUUqeSwH33W3FrVi90yxXEeS/CxQ13fT2Wuw11SBI8+ZDhze6zfXIfn7o6+8swzWAOMS5dWpIZw5uCSHrMqN8KG7MMU1TaWXZS5MkfRYh9D7QVAMh+NLp2lo1zV/c49HsR1e5TpT1wjb8C2n3nIdtI76P5kGjs9WcPN0qHiv7AuHA0WEsVvHchUPE+Jm4+gWfwk3dxGm8D9zfTa22P7HUD0TAH2xkz2jBePlc1XQCILwb2JfbUo8DkyXpZvWiRm+QKQaV88Kc/Uepj5mX5KZA1JTzc0TuTnHLZ+M+sc4ciG5Te5CROx7YFhG7auDcMvLSb3f5vIgfeAskjneAvj8ZaEEsuebS01+SFHPg1w4NUYOoLyCPl5WqMuLJZkAJzL5DXLQsC0ZUxKpJPAU+3XuxUF8w5uGTV7fWYgwe28c1Anr5H90wKc2VAUj1lbdCB83CpB29/BeUJMef4LyX0Ml+9/7GNKt1g1jpd+ +Eqjcn7H 1kV4PJSbo1+oBD4xiXrHq+0hhzS5Q1+pix1BNV+xhKkyaMvTRwK+xThEipUFZbMQeV50gZYxfNpnHbNvWLv7eTx4ZUTeqKECbokouKLe8uX89H5NecfXmuHhRo76qOM//McMJ2TdaVccVdq6KvGTO3trSCH9fZ6Fb7T9jwY4AbDSE8Cr+XfFo5jbm5BKl9ggOuboV69HLzudvSZWHW+yt5jjVhHFZgoiEaPpjqxCyn60f57g5o6j0tkmP1/5nzV2avFEbQbBokOPYtKGdvLxlN6yv+5sAw99xQl5ClBdV53Qm6YajeN6BR82ZAi18yv5rdb90tq27kz5c7NcLkpG1BqI61AUJiwYlVmr+SlFmr3H2kycIfB6ywJJNhNxn1OaVwDWK0Lzq4B3K6x6AmWewZdEfdhZt7jMmiggD7Ors+QDiIdzVFVAeJiBO0l7lJmAAmb01XmDt2NU2rR6N1VuVwpDFl51FGTpCwzoIC0MaTZJZY6Dn8GmF4b5fNMblffQFKXeRWZ8PLF6O2aVAFahmJTZCuS5jVdk9+HaRegXuJJLt5KNWWdnLK4Ucu0fXes3nPWTvL1YZiMTBAzjiNppI1RpoD7jm78NTz+EhJJnpyPhisw5P3cyU1KjWRA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For now, flush_tlb() simply calls flush_tlb_mm() which results in a flush of the whole TLB. So let's use mmu_gather fields to provide a more fine-grained flush of the TLB. Signed-off-by: Alexandre Ghiti Reviewed-by: Andrew Jones Tested-by: Lad Prabhakar # Reviewed-by: Samuel Holland --- arch/riscv/include/asm/tlb.h | 8 +++++++- arch/riscv/include/asm/tlbflush.h | 3 +++ arch/riscv/mm/tlbflush.c | 7 +++++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/arch/riscv/include/asm/tlb.h b/arch/riscv/include/asm/tlb.h index 120bcf2ed8a8..1eb5682b2af6 100644 --- a/arch/riscv/include/asm/tlb.h +++ b/arch/riscv/include/asm/tlb.h @@ -15,7 +15,13 @@ static void tlb_flush(struct mmu_gather *tlb); static inline void tlb_flush(struct mmu_gather *tlb) { - flush_tlb_mm(tlb->mm); +#ifdef CONFIG_MMU + if (tlb->fullmm || tlb->need_flush_all) + flush_tlb_mm(tlb->mm); + else + flush_tlb_mm_range(tlb->mm, tlb->start, tlb->end, + tlb_get_unmap_size(tlb)); +#endif } #endif /* _ASM_RISCV_TLB_H */ diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index a09196f8de68..f5c4fb0ae642 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -32,6 +32,8 @@ static inline void local_flush_tlb_page(unsigned long addr) #if defined(CONFIG_SMP) && defined(CONFIG_MMU) void flush_tlb_all(void); void flush_tlb_mm(struct mm_struct *mm); +void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, + unsigned long end, unsigned int page_size); void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr); void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); @@ -52,6 +54,7 @@ static inline void flush_tlb_range(struct vm_area_struct *vma, } #define flush_tlb_mm(mm) flush_tlb_all() +#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() #endif /* !CONFIG_SMP || !CONFIG_MMU */ /* Flush a range of kernel pages */ diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 77be59aadc73..fa03289853d8 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -132,6 +132,13 @@ void flush_tlb_mm(struct mm_struct *mm) __flush_tlb_range(mm, 0, -1, PAGE_SIZE); } +void flush_tlb_mm_range(struct mm_struct *mm, + unsigned long start, unsigned long end, + unsigned int page_size) +{ + __flush_tlb_range(mm, start, end - start, page_size); +} + void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr) { __flush_tlb_range(vma->vm_mm, addr, PAGE_SIZE, PAGE_SIZE);