From patchwork Thu Oct 19 14:01:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13429255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76987CDB465 for ; Thu, 19 Oct 2023 14:03:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EF4848D0019; Thu, 19 Oct 2023 10:03:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E9E478D0006; Thu, 19 Oct 2023 10:03:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3FC48D0019; Thu, 19 Oct 2023 10:03:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id BF3048D0006 for ; Thu, 19 Oct 2023 10:03:32 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 8B2231405A1 for ; Thu, 19 Oct 2023 14:03:32 +0000 (UTC) X-FDA: 81362378664.29.96CAA14 Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com [209.85.167.45]) by imf15.hostedemail.com (Postfix) with ESMTP id B15A9A002E for ; Thu, 19 Oct 2023 14:03:05 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=nZB1Tkis; dmarc=none; spf=pass (imf15.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.167.45 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697724185; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hd/FLevx18XhR5UIn9smwlv93JRL3lzcOo0hNfovrwU=; b=qhhMEfYlQQ5HVd4bxzPEX2MhgyWVsR7sWk9Yk1U2/u12haEUDMHmCLNxBlpjh4dmG5++4g zjwA1fNbuSiKecuH/tC0YjV2o0UxVAusGxs+2FeD9Lhm/dOeBIxp4GItJqn5KaSwXTa98z PwpA82NEBR93ug6vMM1P6YawClRssQ8= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=nZB1Tkis; dmarc=none; spf=pass (imf15.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.167.45 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697724185; a=rsa-sha256; cv=none; b=u9mXSw1hnL8JCBanwSe1ON+E80laJrI8ENpy6vqNVzV6Xr0Q7hIhQP93OwKmrpKw4Me0wA je8PVuFu6aPHcY3RLGnZjuRa+phlVzD7fXGPHmDVITGTETJwtsSBMJqOYSXbQRsZIKpOEY pTAa2THWq7fKXDjAX+yYYatSGo8JDM4= Received: by mail-lf1-f45.google.com with SMTP id 2adb3069b0e04-507a0907896so7843986e87.2 for ; Thu, 19 Oct 2023 07:03:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1697724184; x=1698328984; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hd/FLevx18XhR5UIn9smwlv93JRL3lzcOo0hNfovrwU=; b=nZB1TkisA8tHSJUiDK4D4bKSRW9SS/NSRusXrHMWVBeeU32ACGTziRTEuRN7QqCI8E /o+ElW8HseaVQIwjeZSLx2UOlMYMUd+jA6LFHYZnivo72FtIvKyoeweT6cIgrqNbQZMa Wz5GTfgk/ilcdxJvui4hRpbltVwe2qUFeTCn5ENpfHrun0W9oIqHUSVOBJsxcwkTXCZz SjYPorGK5ThlaviUNEsu6m6Fr82vwBtOWcBPi+4M9tuxps2/Oiw/rGASattcrYnGmdvj 1RsT4R3Afg7ppfYy8RTaZKx5/MfkzLiFBmTrJwI4ddDFZgbb+F3zzrbrl7VsGQFzQlqF mesg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697724184; x=1698328984; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hd/FLevx18XhR5UIn9smwlv93JRL3lzcOo0hNfovrwU=; b=WGnxLoW+D9Qu6dTZ1b2f3UiAl1Lvr6lmGm3NmsIrQPWia6OqH9QFOL5Y97T4X6PRT7 mS+xlXAYsna4HcDrGR/ZHazXO27BHtnJIq1JMV7FLi6E3+79+YWDFnb6F7wXgFbPN6CY TAn3eUPs6Qz67Wrxdus7LpT6nyIfkNzqLi6dq7/bcKYUpUm0bcVRdtIJi9+I3OQq7cwY +lQknIqstSGvcuEY2mSdwIgc/Wm+Ti2WmM904FyXsfiwdQ2QivrMcbW/A4Ie6FoStXay CFoD+Jf3QGNHtS48XA1acSel9e7dfveROSfV/uIUkS4LONNYGekTj79QWn9v1E2KNpS3 tOZw== X-Gm-Message-State: AOJu0YxPW+iMHHJelB2WmnvVE1w/BzpnwiBlj8L7yBHC+UuQ+PIT+CQQ 70wjp8SYhNZYxi9NaQw/BTc4dQ== X-Google-Smtp-Source: AGHT+IEreDwOMoW56yjrW09GCEE/pI+6cCamWfZVDmTeqTnMJ6ObGDZTCddBTRTbchZ8BncLsOt9fw== X-Received: by 2002:ac2:5a04:0:b0:507:b993:bb86 with SMTP id q4-20020ac25a04000000b00507b993bb86mr1486713lfn.66.1697724183683; Thu, 19 Oct 2023 07:03:03 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id q9-20020a05600000c900b0032d8eecf901sm4567033wrx.3.2023.10.19.07.03.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Oct 2023 07:03:03 -0700 (PDT) From: Alexandre Ghiti To: Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra , Mayuresh Chitale , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Samuel Holland , Lad Prabhakar Cc: Alexandre Ghiti , Andrew Jones , Samuel Holland , Lad Prabhakar Subject: [PATCH v5 1/4] riscv: Improve tlb_flush() Date: Thu, 19 Oct 2023 16:01:48 +0200 Message-Id: <20231019140151.21629-2-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231019140151.21629-1-alexghiti@rivosinc.com> References: <20231019140151.21629-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: B15A9A002E X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: j55b8mt9zdi9so51x16uo3jneg6eigjy X-HE-Tag: 1697724185-395231 X-HE-Meta: U2FsdGVkX18fl2bPb7cKCffczwjxciK2ddTXhlPE5VxUy1Sh68h+WCtUlw91rt38gEecWezrXHd6DqCWvrXm1Bc9UaE9ABVxiHuNpsrPKoZ57+21zzFgxD+WRfAEDs8XdZ71rg8LA6ugdueQdCiCZ8Hw6QEEl6LmIM4LdnoaVgddubwYdwj5nWYgZ88BZ5QEFSJ6Y8JAH0GMf5nE4IEFJjPY5C+H4aj4aMnfJBGobOGccGdLBTiPazJJzfIdrjg1tux9w0q7djqO0m5nzET/r0yW6CUES8vupqtrbngrSNvlijJ1v5pqlu8oueUvAuqY2cpdBaxPcEW9jxKTCcFHhR3ATMTZpW+4P73VH1Dr+Hb/G/aWO/t3rSeTimaCV7Ylo10aTM/zIekVVSPNeCfTYqa8HcSmtVa21BDQD9jbHlQHNsOoRu4ZRgwgeIjwppFq6b7hWAkKQjEOqnngK4IyovWP3ML2uzYoFFNuS1mTF+RmvwUPaC0b50ky6J54oJ8hiebbQrQMAqc7k4LKCC6DSpy87dZDqiQpha1jhmj5R3P+CKE0MBJKfSxW1INDcj8IktkbD6NstgsejnpXGv328vUz0dioXJCvELrGXZzuLnDoWt8Up7a9dGokX+Z6muhVAUXcLrPnsqLqCFaODkPvDv0emZPU7G8sct5VRAhzNted9L8WgxofNsoqYp5PpcxboihGb5T5dJOe5aWP0aX2hjZLgOnO5Yyaaxyk0MgMxJxhmPpSbUB/FhUynAcT8++AKffaaOaPZJRkvD+v21CzYyLsMZRLQByu2XvDXHghx9XfKxHUxRaRN1gpr9L20WTDfwPhFcRG094wfHqZhAtzWwSfz2Y1tohkQXSmes6GuUv34msNCrCTgqVARIeJdap9tE0Gw5zO/ECr1mfybjV6z4sH61Us77VCTLsjk+r84Cy6iYuphkdP32OOR+TEJdbowuiqM3lBsvRgodJyGQ/ NvP+CsT8 s9p6apBLSltILARI0sE14VDzovvq4TW//qT4kQk5+YvV5UhE0S3noHi1D6ke1f9eAjXmqD2s6fEafadoXcyQXx+ROAQs7mVXnvqspA/7eICvGm4Me4P4mOKFM5kAl3e1O34+V9qvd14OHuh4lK9N1f8Zy95U7aCXrMqtZ1+xCJL+QzL31gqW0SaFggOljswr/igBM974I1kySZjNE9qj5qcyDV/DhijZPPEU5iOVEL+ygQ3dFdfmsdxLID+VQdUwuvBOeS5cctTuXGbQFP/IbgfQhh33cccGhCKz6n7JzbYlrePE8Azoq0eWiE2mRXyeHpWiUHJX7ujWaIwnH+taM3zEK+aqP5ed54c+VSHPkqnLuj1DOTpwuc8uzXqWYdS26bqHG8+zuOyJwh6UfBfB1TPC21Q6m/MZnIHBdNZRD4s9M+HVwn1UILNaMkwTVKhG/4wXs4pNBxxSRYSbPNoeqkDSAH2mtybv2wYxN6Wxdm+b43iJ2+LSP7rtEdt87Zw4i+rDm0Y58vL2iOOPUID/249lGNPqOoMWdiOIdROwoQc8yDh+nhJfDvEsDi4tSvvk/hPvVzSNt7fSLPPIl1gZ3TG2adSRHLd/x+KRgymWV1ISXA5OIMPP3Yax/418Si2hwalcWhSWbaOJtIijEC21eXCyZ0CWgWoWuZqN8s5wqcsrGQuS9u4cMzK0qcK0gdUR2CLLKXr9GtlVZ/SE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For now, tlb_flush() simply calls flush_tlb_mm() which results in a flush of the whole TLB. So let's use mmu_gather fields to provide a more fine-grained flush of the TLB. Signed-off-by: Alexandre Ghiti Reviewed-by: Andrew Jones Reviewed-by: Samuel Holland Tested-by: Lad Prabhakar # On RZ/Five SMARC --- arch/riscv/include/asm/tlb.h | 8 +++++++- arch/riscv/include/asm/tlbflush.h | 3 +++ arch/riscv/mm/tlbflush.c | 7 +++++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/arch/riscv/include/asm/tlb.h b/arch/riscv/include/asm/tlb.h index 120bcf2ed8a8..1eb5682b2af6 100644 --- a/arch/riscv/include/asm/tlb.h +++ b/arch/riscv/include/asm/tlb.h @@ -15,7 +15,13 @@ static void tlb_flush(struct mmu_gather *tlb); static inline void tlb_flush(struct mmu_gather *tlb) { - flush_tlb_mm(tlb->mm); +#ifdef CONFIG_MMU + if (tlb->fullmm || tlb->need_flush_all) + flush_tlb_mm(tlb->mm); + else + flush_tlb_mm_range(tlb->mm, tlb->start, tlb->end, + tlb_get_unmap_size(tlb)); +#endif } #endif /* _ASM_RISCV_TLB_H */ diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index a09196f8de68..f5c4fb0ae642 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -32,6 +32,8 @@ static inline void local_flush_tlb_page(unsigned long addr) #if defined(CONFIG_SMP) && defined(CONFIG_MMU) void flush_tlb_all(void); void flush_tlb_mm(struct mm_struct *mm); +void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, + unsigned long end, unsigned int page_size); void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr); void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); @@ -52,6 +54,7 @@ static inline void flush_tlb_range(struct vm_area_struct *vma, } #define flush_tlb_mm(mm) flush_tlb_all() +#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() #endif /* !CONFIG_SMP || !CONFIG_MMU */ /* Flush a range of kernel pages */ diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 77be59aadc73..fa03289853d8 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -132,6 +132,13 @@ void flush_tlb_mm(struct mm_struct *mm) __flush_tlb_range(mm, 0, -1, PAGE_SIZE); } +void flush_tlb_mm_range(struct mm_struct *mm, + unsigned long start, unsigned long end, + unsigned int page_size) +{ + __flush_tlb_range(mm, start, end - start, page_size); +} + void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr) { __flush_tlb_range(vma->vm_mm, addr, PAGE_SIZE, PAGE_SIZE);