From patchwork Thu Feb 29 23:21:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577722 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F3EE8C54798 for ; Thu, 29 Feb 2024 23:22:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=69nnaKqoZG6dYszstJGNtgfKFYNnqDQCs+OK7icrQkw=; b=YVI8Nm9TyYORRk t8PQHHNPFWOSc4Ivr5ApSvG5ToTUxlyGp6l7aebj9Oxhalu93peshCMayJe81tkM2JOsBs+d9blW9 mKCbej9ev/msr43q0Cef+u2Xfo1gvUMebwqAgP0dNwb4RzYK7XB36KZYZlWDfvfA4m+OI+KfDaxdC vGoKMfa+BB8Ivz4szeEFt1jOVzfLN4WXZoByewAUDt2k+0yQP4Dx2BYqb5Gy2Rxtg0qEk39e+r5Wq BOzjDxqZOjHdiZ7Q7gSlsdla6BbZ0OTtT6L855P3aU5AnLlOFbz0L4bise3daY+MkEBQGd7oFfpv1 CT/Va2h2cxd1A0ZonE0Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjG-0000000FZub-2DvD; Thu, 29 Feb 2024 23:22:18 +0000 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjD-0000000FZsa-1K2i for linux-riscv@lists.infradead.org; Thu, 29 Feb 2024 23:22:16 +0000 Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-6e4e7e25945so1258941b3a.1 for ; Thu, 29 Feb 2024 15:22:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248934; x=1709853734; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=O37VWqG4iCvkiZDgqVtP8oElF9rd1vMtEBQWx77Nmx0=; b=RQhbXieEgbMXntbFuRojyfTIiFTnB9ALa39+X/o1u6o0eHtS/TXMLUe1PZ5SdihWOM X9rhQpEhOmYlP2q4LVra18WhnBMRax5WhZj9qB+9GCeCW9FmrJOQ2EuZtrFwhosv6fsi MvGqM0AvAbOvUKv1ktJs2svzTUxhC9yRUjyIde2qxz0moxUrgmiQoygMato4Ir4Mx3qA 5K6W4QfKettcD/xcfcOvwU/TGSTWVVeHXDXngv+rYn4y4/IQhlYb3AO3A07urFKDpYdG kQzFG3tHNII6ETKS/LFQjFwESjEEH1KNhBUM6suCt4cwA0BXgZC7MnBmcPVzb8nmPsEE NnBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248934; x=1709853734; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=O37VWqG4iCvkiZDgqVtP8oElF9rd1vMtEBQWx77Nmx0=; b=ce8dkSUizzCWCUT7iXK2nWU3Z/3Uk3rOazwy6i+vL58w514u1DYMKaALr9zdQq7Tu0 KL0Cm89ZvQ5Od7Eub+3VtxzT6/G/n3IhVZjxjX2PP0jlJIrw4qK3nv1khDTj5RTU6nIf dVOJ+scLK5dFi2xMvctuhepxuIDHeHZun9/jM8AEh1Sa6XUdgkxwYWRlTSJqDOvh2XaR lkq6M4XF9SDYItB6ArBdoTMIxkTouQxW7I2Hj9GaSkooT4RBMnHQY/fiTudZItT4ifQu 5Tn/hQk0DShcN78+d+1QsQs4hIgwUozV/ICRzzSCOIzpYamwp/AIIbTU5GIDCwAS31yV 46kg== X-Forwarded-Encrypted: i=1; AJvYcCUDRVtsQ2QYdmMz3TkxgA/NNBZnxq1CsgMY3Lq/9yth92Q8IaoOZ2U+iLALD1gxtY5SVkbEl+MAQIVUNt7cR8qCIL7CuA/U9dgSM+FrabFf X-Gm-Message-State: AOJu0YxrQPGxiT0kDcIfbjfAB6+/g3l71YCTNfL/XwUCOWcw/HnvfiG1 chYEMore75yp0CpCOH0kR4kJ0PfXwR1WefVgADzHmCavprM7nAuq4H+80tGMQso= X-Google-Smtp-Source: AGHT+IEXcsW9CGPGmNlysP87xM+KqyMeGQDT/2i2zpW2/RfnrP1VyWuH3lyJldn2H0bz6i0E+Cfqiw== X-Received: by 2002:a05:6a00:2d1c:b0:6e3:fa52:2f26 with SMTP id fa28-20020a056a002d1c00b006e3fa522f26mr168419pfb.8.1709248934208; Thu, 29 Feb 2024 15:22:14 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:13 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 01/13] riscv: Flush the instruction cache during SMP bringup Date: Thu, 29 Feb 2024 15:21:42 -0800 Message-ID: <20240229232211.161961-2-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_152215_410149_B1496A62 X-CRM114-Status: GOOD ( 10.25 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Instruction cache flush IPIs are sent only to CPUs in cpu_online_mask, so they will not target a CPU until it calls set_cpu_online() earlier in smp_callin(). As a result, if instruction memory is modified between the CPU coming out of reset and that point, then its instruction cache may contain stale data. Therefore, the instruction cache must be flushed after the set_cpu_online() synchronization point. Fixes: 08f051eda33b ("RISC-V: Flush I$ when making a dirty page executable") Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v4) Changes in v4: - New patch for v4 arch/riscv/kernel/smpboot.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c index cfbe4b840d42..e1b612f37dd9 100644 --- a/arch/riscv/kernel/smpboot.c +++ b/arch/riscv/kernel/smpboot.c @@ -26,7 +26,7 @@ #include #include -#include +#include #include #include #include @@ -238,9 +238,10 @@ asmlinkage __visible void smp_callin(void) riscv_user_isa_enable(); /* - * Remote TLB flushes are ignored while the CPU is offline, so emit - * a local TLB flush right now just in case. + * Remote cache and TLB flushes are ignored while the CPU is offline, + * so flush them both right now just in case. */ + local_flush_icache_all(); local_flush_tlb_all(); complete(&cpu_running); /* From patchwork Thu Feb 29 23:21:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 09A08C48BF6 for ; Thu, 29 Feb 2024 23:22:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=XZ3cYWWcK+ZDOQb9EUFjBoxEMtueiRrVEYj3GrszpoU=; b=y9Wp9z8UsHEOOi z+0h+lzfCpFmjbd9m1M+zC6yntlzj4yqocdDPXfGQ2Pq2I1FkZumPSKDuZapjLWJNUP9l2MgGdoDp cHu8Daj1tRoZ32ipgknutvcD/TDtGaBXWodqks794W5jAKaqs/e+epoCeKXr1qWVBzJdefYWKn29Y kAd0emG1t5S6MORFRnVMV1KLQl2QfuvWxLLVn7IC9qx6JcBtmetTqLKMpYo4xEaZly9IKCVkE4PnT i264ad8rwzd+BL8Ioe623j6VujBRMrB4cLiQp6LsqSyNaum/R+7/tI0v+WVTJDujZr5blxQqFx6/p yxbSh09TfBFxZ66D0KFg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjP-0000000Fa0d-30Zg; Thu, 29 Feb 2024 23:22:27 +0000 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjE-0000000FZtE-1DsT for linux-riscv@lists.infradead.org; Thu, 29 Feb 2024 23:22:17 +0000 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-6e581ba25e1so1204856b3a.1 for ; Thu, 29 Feb 2024 15:22:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248935; x=1709853735; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XEa0bZw6X2PF+fvXkyrnyWuZOURSNgi5dGIyeM9k7ic=; b=BSrV18e/vNO5QDLrMnxNoGanXr1m0hCtWjg8ATFwMN3fejYfIGEbzfoJQ67Vxqx2ny 52A3a3sjxPoHrYQHEFs0ZilH+NFReTpY58qwKMLypdPyjGOnqAV42oqdlSGQ4/xTnn/8 rGj8jMdwBk0zNOHJbaC8DaCsITLKGjVhEBzO8vPU1Gz7zzfRhviJRCCrI2AFyegH8pZG 1z1/h8suNHUR+GAaijs2RA1TMClWmJFxTEO6wryqNrSa/oQM4Jpdbcu3mCit0F+UsT6d Y2ORbFj/RNvwkRSkl+1yKEMqYa00ogdLu+vjsJY4JAeHemVgMnxuqSYfDJ2agMoFRj4U 1g3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248935; x=1709853735; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XEa0bZw6X2PF+fvXkyrnyWuZOURSNgi5dGIyeM9k7ic=; b=MUfYOvo2+JGCGLCFiCYbdralS0NdlkGB0R3hukoxXT51G00JLri8Wc0+pN1pAE7Rj9 S7WV2pbBO6MFXu0g+sb8P81BeTjL0DvGmHDsMwOkQkhsbUeMdfwhLwewfJrNjpcmlc+z OrUCnlFJ7gBfpth1u6i/75g7l6NgnV9w4Q+LCVKBeeVFCAs4uEm7F0hvo+7pixtrPuWO wzLj4mmkTVjWqIvpbN50JAWL0oYavVN7czhUnVt30O9XudUIioBF3jMNrKBRBsjcTvtc guJSjW5eatNp/GYJLhdxBPM7cJNWTHk9gY7SWwyLjto6yqmrdI0f0XCTOy/hWN5XNBKA xeRA== X-Forwarded-Encrypted: i=1; AJvYcCV2nK3MZYV4FfS+5aT9koKnV68VWTmxmmP0TmT4xw4AfpBLYCH/HFuPXvdlbrlfZJ0xRAdgBJr8qSIPc+DGCCsr8SHuAcXLOXIrGIcIwirX X-Gm-Message-State: AOJu0YwW9YMSAYRftUTpfQTqJnuSO+Xd01oCJiZGkH1BkxXdsGcRARD4 VixzkrPSVCgYaO6muUIgpVaiNYuErkvCV9N2dgaaM4BwsrF+I9owkhy3FLxLU8g= X-Google-Smtp-Source: AGHT+IFdgcTr06rQ4rVKFKQwHiCzNjuzvF9/zHzBzMpbTCozBAylfcba6wC7aAVCdn9otyCllPR9tg== X-Received: by 2002:a05:6a00:2293:b0:6e5:599a:5252 with SMTP id f19-20020a056a00229300b006e5599a5252mr606918pfe.15.1709248935218; Thu, 29 Feb 2024 15:22:15 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:14 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 02/13] riscv: Factor out page table TLB synchronization Date: Thu, 29 Feb 2024 15:21:43 -0800 Message-ID: <20240229232211.161961-3-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_152216_351719_3DEC0632 X-CRM114-Status: GOOD ( 10.19 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org The logic is the same for all page table levels. See commit 69be3fb111e7 ("riscv: enable MMU_GATHER_RCU_TABLE_FREE for SMP && MMU"). Signed-off-by: Samuel Holland --- Changes in v5: - New patch for v5 arch/riscv/include/asm/pgalloc.h | 31 +++++++++++++------------------ 1 file changed, 13 insertions(+), 18 deletions(-) diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h index deaf971253a2..87468f67951a 100644 --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -88,6 +88,14 @@ static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr) return NULL; } +static inline void riscv_tlb_remove_ptdesc(struct mmu_gather *tlb, void *pt) +{ + if (riscv_use_ipi_for_rfence()) + tlb_remove_page_ptdesc(tlb, pt); + else + tlb_remove_ptdesc(tlb, pt); +} + #define pud_free pud_free static inline void pud_free(struct mm_struct *mm, pud_t *pud) { @@ -102,10 +110,7 @@ static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pud, struct ptdesc *ptdesc = virt_to_ptdesc(pud); pagetable_pud_dtor(ptdesc); - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, ptdesc); - else - tlb_remove_ptdesc(tlb, ptdesc); + riscv_tlb_remove_ptdesc(tlb, ptdesc); } } @@ -139,12 +144,8 @@ static inline void p4d_free(struct mm_struct *mm, p4d_t *p4d) static inline void __p4d_free_tlb(struct mmu_gather *tlb, p4d_t *p4d, unsigned long addr) { - if (pgtable_l5_enabled) { - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, virt_to_ptdesc(p4d)); - else - tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d)); - } + if (pgtable_l5_enabled) + riscv_tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d)); } #endif /* __PAGETABLE_PMD_FOLDED */ @@ -176,10 +177,7 @@ static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, struct ptdesc *ptdesc = virt_to_ptdesc(pmd); pagetable_pmd_dtor(ptdesc); - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, ptdesc); - else - tlb_remove_ptdesc(tlb, ptdesc); + riscv_tlb_remove_ptdesc(tlb, ptdesc); } #endif /* __PAGETABLE_PMD_FOLDED */ @@ -190,10 +188,7 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, struct ptdesc *ptdesc = page_ptdesc(pte); pagetable_pte_dtor(ptdesc); - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, ptdesc); - else - tlb_remove_ptdesc(tlb, ptdesc); + riscv_tlb_remove_ptdesc(tlb, ptdesc); } #endif /* CONFIG_MMU */ From patchwork Thu Feb 29 23:21:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 96D15C54E55 for ; Thu, 29 Feb 2024 23:22:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UNOyvpC7hg+dXhVvY+5DnwKC4yiOvXAAI+NWZeUu6qg=; b=R9FXlZq+lHrUeS iUSwWX19CLaZCYVQMPxDsxZZoOAHEdb5E5DTs8VOrAdfv9AIW7WKl2RUgHhJL7IUU+pIhnH6eQf5M e8zdyWqTIyWcKny2b0aDEchzJFNT40IpwgYfpJxzRGj08hk58MCBKpiPKxUoS8RXFxiJGIzHD5Y12 Co944eH48DRVYamuNYx96QK0d4W9Ymj6TJnzBU6hDi3jCaLrNBAW4/bi4hnlvtbqOEv2Nn9eslI++ u42+rUslJK5BSvAWyBOLy28HtPC9fyaPI1SPRR9CNj+QbG9LnSjhp41MHkvT6/Oj/TI+QNU2XXkQ9 e49Oe2yso0aL3FO4U/WQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjT-0000000Fa35-2Mqh; Thu, 29 Feb 2024 23:22:31 +0000 Received: from mail-il1-x133.google.com ([2607:f8b0:4864:20::133]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjF-0000000FZtX-1sPs for linux-riscv@lists.infradead.org; Thu, 29 Feb 2024 23:22:19 +0000 Received: by mail-il1-x133.google.com with SMTP id e9e14a558f8ab-3651c1e1022so7353655ab.0 for ; Thu, 29 Feb 2024 15:22:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248936; x=1709853736; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nL3CU8jW9vVfZeENVgIQs2Z0lmYtCUXWU5kIlhxMA+M=; b=Sk7Q4/E/9St2uAKEv0/9KU194sx4KeqdiV1g70Jy8MPaH/3TMR/nKyh/UkXt/355JQ 5DkLt6VSBf6WTiRqcLGE/7mR4QImABgebOEXyA0PjHa6kCa0LKGFCwDjZqw10WrAky/y 6/oS1FM4SwnBUiQELkdF4J3VK8iTK1fw7rjGaZOcJa6TL3zzUVQ1fRkrRxMi5BB9NNaw RL+TeZ6YEnQGO/kYzK8cZqkbywEYThWJ5pDG3o2pd668H7rtNge7+AMcFAAusfHho+rG tiDCU73z607jxiPSZkZfwAHYYqssCNY5SpVSnZg7+3cdg+9v2SEJ9wnX/r9pGXFZx6xv kIGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248936; x=1709853736; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nL3CU8jW9vVfZeENVgIQs2Z0lmYtCUXWU5kIlhxMA+M=; b=NVPH+EglpoxnUeELIvZOFgY3HLzDLKl7pVUmfm2jXPCdsgYyqVys/O1u991dchvz6J xBEGVOPlFPlU5uZPE7bFTBzlotz+W/KltzJB2GMZkDCcCLQYjqizCdlRO9hI36JamfTs dDPkpaj8MOD9kaB0j3U/gdDVlvMgDulSl+8bcs2g50fjt4GAJqnwYS8Gswr2FvbwbuCU N3yKBPFcNXqujSOnQM+dMx6Jcl9Ecq292krzZaatkWu8QzZgT5b45tX8Y3DmqLQmMzWf 1OMEmpt70tKTI/gkrw0ysm+62XKbeTHFxgQu7x3/PbCzluezD0ZDXQFd013i0aWldh1b 9onw== X-Forwarded-Encrypted: i=1; AJvYcCWIMbLw1R7NIAp8zO+V3bUYDqymH176N0/StOnbBOaT6dL2wUubWZK7foVhT+CXPdnqZoiYMDCovpkqAVoJkcAedN8xGFot0j1y7UEI/7vb X-Gm-Message-State: AOJu0YwZ6dTlsq4yl/t1GbNF1qU57JGQRhn+SaCyr+k9khRSRuB4dKWU 7GmtvEjeDVpL3vWUP8iQHtDelpmPh/6RSE+cEqpF12evfaXg7Tk9neHX2jdj+KU= X-Google-Smtp-Source: AGHT+IEuQOuQ50R8qoZP0RbIoJTNSdVCKrs4lRWLzJ+znFdHc+eSLoYj/Ypw4BmFNiQ++vK2RkPm8w== X-Received: by 2002:a05:6e02:1b8f:b0:365:ae26:5827 with SMTP id h15-20020a056e021b8f00b00365ae265827mr159548ili.5.1709248936255; Thu, 29 Feb 2024 15:22:16 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:15 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 03/13] riscv: Use IPIs for remote cache/TLB flushes by default Date: Thu, 29 Feb 2024 15:21:44 -0800 Message-ID: <20240229232211.161961-4-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_152217_590776_299CE4E8 X-CRM114-Status: GOOD ( 21.71 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org An IPI backend is always required in an SMP configuration, but an SBI implementation is not. For example, SBI will be unavailable when the kernel runs in M mode. Generally, IPIs are assumed to be faster than SBI calls due to the SBI context switch overhead. However, when SBI is used as the IPI backend, then the context switch cost must be paid anyway, and performing the cache/TLB flush directly in the SBI implementation is more efficient than inserting an interrupt to the kernel. This is the only scenario where riscv_ipi_set_virq_range()'s use_for_rfence parameter is false. Thus, it makes sense for remote fences to use IPIs by default, and make the SBI remote fence extension the special case. sbi_ipi_init() already checks riscv_ipi_have_virq_range(), so it only calls riscv_ipi_set_virq_range() when no other IPI device is available. So we can move the static key and drop the use_for_rfence parameter. Furthermore, the static branch only makes sense when CONFIG_RISCV_SBI is enabled. Optherwise, IPIs must be used. Add a fallback definition of riscv_use_sbi_for_rfence() which handles this case and removes the need to check CONFIG_RISCV_SBI elsewhere, such as in cacheflush.c. Signed-off-by: Samuel Holland Reviewed-by: Anup Patel --- Changes in v5: - Also switch to riscv_use_sbi_for_rfence() in asm/pgalloc.h Changes in v4: - New patch for v4 arch/riscv/include/asm/pgalloc.h | 7 ++++--- arch/riscv/include/asm/sbi.h | 4 ++++ arch/riscv/include/asm/smp.h | 15 ++------------- arch/riscv/kernel/sbi-ipi.c | 11 ++++++++++- arch/riscv/kernel/smp.c | 11 +---------- arch/riscv/mm/cacheflush.c | 5 ++--- arch/riscv/mm/tlbflush.c | 31 ++++++++++++++----------------- drivers/clocksource/timer-clint.c | 2 +- 8 files changed, 38 insertions(+), 48 deletions(-) diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h index 87468f67951a..6578054977ef 100644 --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -8,6 +8,7 @@ #define _ASM_RISCV_PGALLOC_H #include +#include #include #ifdef CONFIG_MMU @@ -90,10 +91,10 @@ static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr) static inline void riscv_tlb_remove_ptdesc(struct mmu_gather *tlb, void *pt) { - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, pt); - else + if (riscv_use_sbi_for_rfence()) tlb_remove_ptdesc(tlb, pt); + else + tlb_remove_page_ptdesc(tlb, pt); } #define pud_free pud_free diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h index 6e68f8dff76b..ea84392ca9d7 100644 --- a/arch/riscv/include/asm/sbi.h +++ b/arch/riscv/include/asm/sbi.h @@ -375,8 +375,12 @@ unsigned long riscv_cached_marchid(unsigned int cpu_id); unsigned long riscv_cached_mimpid(unsigned int cpu_id); #if IS_ENABLED(CONFIG_SMP) && IS_ENABLED(CONFIG_RISCV_SBI) +DECLARE_STATIC_KEY_FALSE(riscv_sbi_for_rfence); +#define riscv_use_sbi_for_rfence() \ + static_branch_unlikely(&riscv_sbi_for_rfence) void sbi_ipi_init(void); #else +static inline bool riscv_use_sbi_for_rfence(void) { return false; } static inline void sbi_ipi_init(void) { } #endif diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h index 0d555847cde6..7ac80e9f2288 100644 --- a/arch/riscv/include/asm/smp.h +++ b/arch/riscv/include/asm/smp.h @@ -49,12 +49,7 @@ void riscv_ipi_disable(void); bool riscv_ipi_have_virq_range(void); /* Set the IPI interrupt numbers for arch (called by irqchip drivers) */ -void riscv_ipi_set_virq_range(int virq, int nr, bool use_for_rfence); - -/* Check if we can use IPIs for remote FENCEs */ -DECLARE_STATIC_KEY_FALSE(riscv_ipi_for_rfence); -#define riscv_use_ipi_for_rfence() \ - static_branch_unlikely(&riscv_ipi_for_rfence) +void riscv_ipi_set_virq_range(int virq, int nr); /* Check other CPUs stop or not */ bool smp_crash_stop_failed(void); @@ -104,16 +99,10 @@ static inline bool riscv_ipi_have_virq_range(void) return false; } -static inline void riscv_ipi_set_virq_range(int virq, int nr, - bool use_for_rfence) +static inline void riscv_ipi_set_virq_range(int virq, int nr) { } -static inline bool riscv_use_ipi_for_rfence(void) -{ - return false; -} - #endif /* CONFIG_SMP */ #if defined(CONFIG_HOTPLUG_CPU) && (CONFIG_SMP) diff --git a/arch/riscv/kernel/sbi-ipi.c b/arch/riscv/kernel/sbi-ipi.c index a4559695ce62..1026e22955cc 100644 --- a/arch/riscv/kernel/sbi-ipi.c +++ b/arch/riscv/kernel/sbi-ipi.c @@ -13,6 +13,9 @@ #include #include +DEFINE_STATIC_KEY_FALSE(riscv_sbi_for_rfence); +EXPORT_SYMBOL_GPL(riscv_sbi_for_rfence); + static int sbi_ipi_virq; static void sbi_ipi_handle(struct irq_desc *desc) @@ -72,6 +75,12 @@ void __init sbi_ipi_init(void) "irqchip/sbi-ipi:starting", sbi_ipi_starting_cpu, NULL); - riscv_ipi_set_virq_range(virq, BITS_PER_BYTE, false); + riscv_ipi_set_virq_range(virq, BITS_PER_BYTE); pr_info("providing IPIs using SBI IPI extension\n"); + + /* + * Use the SBI remote fence extension to avoid + * the extra context switch needed to handle IPIs. + */ + static_branch_enable(&riscv_sbi_for_rfence); } diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c index 45dd4035416e..8e6eb64459af 100644 --- a/arch/riscv/kernel/smp.c +++ b/arch/riscv/kernel/smp.c @@ -171,10 +171,7 @@ bool riscv_ipi_have_virq_range(void) return (ipi_virq_base) ? true : false; } -DEFINE_STATIC_KEY_FALSE(riscv_ipi_for_rfence); -EXPORT_SYMBOL_GPL(riscv_ipi_for_rfence); - -void riscv_ipi_set_virq_range(int virq, int nr, bool use_for_rfence) +void riscv_ipi_set_virq_range(int virq, int nr) { int i, err; @@ -197,12 +194,6 @@ void riscv_ipi_set_virq_range(int virq, int nr, bool use_for_rfence) /* Enabled IPIs for boot CPU immediately */ riscv_ipi_enable(); - - /* Update RFENCE static key */ - if (use_for_rfence) - static_branch_enable(&riscv_ipi_for_rfence); - else - static_branch_disable(&riscv_ipi_for_rfence); } static const char * const ipi_names[] = { diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c index 55a34f2020a8..47c485bc7df0 100644 --- a/arch/riscv/mm/cacheflush.c +++ b/arch/riscv/mm/cacheflush.c @@ -21,7 +21,7 @@ void flush_icache_all(void) { local_flush_icache_all(); - if (IS_ENABLED(CONFIG_RISCV_SBI) && !riscv_use_ipi_for_rfence()) + if (riscv_use_sbi_for_rfence()) sbi_remote_fence_i(NULL); else on_each_cpu(ipi_remote_fence_i, NULL, 1); @@ -69,8 +69,7 @@ void flush_icache_mm(struct mm_struct *mm, bool local) * with flush_icache_deferred(). */ smp_mb(); - } else if (IS_ENABLED(CONFIG_RISCV_SBI) && - !riscv_use_ipi_for_rfence()) { + } else if (riscv_use_sbi_for_rfence()) { sbi_remote_fence_i(&others); } else { on_each_cpu_mask(&others, ipi_remote_fence_i, NULL, 1); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 8d12b26f5ac3..0373661bd1c4 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -78,10 +78,10 @@ static void __ipi_flush_tlb_all(void *info) void flush_tlb_all(void) { - if (riscv_use_ipi_for_rfence()) - on_each_cpu(__ipi_flush_tlb_all, NULL, 1); - else + if (riscv_use_sbi_for_rfence()) sbi_remote_sfence_vma_asid(NULL, 0, FLUSH_TLB_MAX_SIZE, FLUSH_TLB_NO_ASID); + else + on_each_cpu(__ipi_flush_tlb_all, NULL, 1); } struct flush_tlb_range_data { @@ -102,7 +102,6 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, unsigned long start, unsigned long size, unsigned long stride) { - struct flush_tlb_range_data ftd; bool broadcast; if (cpumask_empty(cmask)) @@ -118,20 +117,18 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, broadcast = true; } - if (broadcast) { - if (riscv_use_ipi_for_rfence()) { - ftd.asid = asid; - ftd.start = start; - ftd.size = size; - ftd.stride = stride; - on_each_cpu_mask(cmask, - __ipi_flush_tlb_range_asid, - &ftd, 1); - } else - sbi_remote_sfence_vma_asid(cmask, - start, size, asid); - } else { + if (!broadcast) { local_flush_tlb_range_asid(start, size, stride, asid); + } else if (riscv_use_sbi_for_rfence()) { + sbi_remote_sfence_vma_asid(cmask, start, size, asid); + } else { + struct flush_tlb_range_data ftd; + + ftd.asid = asid; + ftd.start = start; + ftd.size = size; + ftd.stride = stride; + on_each_cpu_mask(cmask, __ipi_flush_tlb_range_asid, &ftd, 1); } if (cmask != cpu_online_mask) diff --git a/drivers/clocksource/timer-clint.c b/drivers/clocksource/timer-clint.c index 09fd292eb83d..0bdd9d7ec545 100644 --- a/drivers/clocksource/timer-clint.c +++ b/drivers/clocksource/timer-clint.c @@ -251,7 +251,7 @@ static int __init clint_timer_init_dt(struct device_node *np) } irq_set_chained_handler(clint_ipi_irq, clint_ipi_interrupt); - riscv_ipi_set_virq_range(rc, BITS_PER_BYTE, true); + riscv_ipi_set_virq_range(rc, BITS_PER_BYTE); clint_clear_ipi(); #endif From patchwork Thu Feb 29 23:21:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577724 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1E804C48BF6 for ; Thu, 29 Feb 2024 23:22:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=qbD4tVatc4PnGqA+eQ5vWYpT5pSILCuLHSi1E8AV81s=; b=tpGqBakf9/LMS1 mI/axY6v8Q0cCkf1mJGEC5WokUCwv5pcdZ+2emnjjwL9tE8i9d2t7yPamdCeQRbEyEm6KBbERwmjF wvDzG/Dl49AFjaqjjlkAxGQPazVupQwxdidDSiGUhXM9XykrYEU9CeRyZHztW+sATIs5eM8+px7FG 2A8ZKh2xULwwrAJpC0VoRjXk8k6S9+pxHacXx5umX16yaZJ0jCKdATIFMTYCu7hZQVBn21d/YFX+S BU/lD3Jc0Co6hFj1LDjybEBWaS6FIZHWLV2n2uoqtfCrRqxD2qtjnauc/qIk+Hz9Z3K20Zu18l2yK wCXXNdbeSSqTmZ/bs1ng==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjV-0000000Fa52-2di1; Thu, 29 Feb 2024 23:22:33 +0000 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjF-0000000FZuD-3oPX for linux-riscv@lists.infradead.org; Thu, 29 Feb 2024 23:22:20 +0000 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-6e56d594b31so866172b3a.1 for ; Thu, 29 Feb 2024 15:22:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248937; x=1709853737; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1/3iO4NfkALka4Buc8Img0QDYdUTnf5/4SNG0fcN5rQ=; b=QD2HWwR1SuyjA5OJpKwB/3yeT2K8TVIkR7tuXfTrdkhMMR++uN3UhXIQeR1PDsD8EU NauFD2IL7t+6qn9KQw4MddRMx19jx5mXzNvek2Y+cZbQzAr1vvGT5gkrdBXl5VKwlpiE XHqyjrnXq9WAMrBIR0JvxygqgZtXROL1g5ZllyREnO9h9ufVrLAVbGVXTzUVj/qUPgkn qt5ZA2Zeq7C09ZU/C/Y4EMFgpYhOrLNkyAtOY3M3e6DFkIPiIf3JDQG2sAy58bejbZzI zDbie1J4h00IsvdEwKUBaV3zqIAoRdpQw8TKJrnHWN1J9U0TNL+LAoxNJLbNSBzxED6y ZXwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248937; x=1709853737; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1/3iO4NfkALka4Buc8Img0QDYdUTnf5/4SNG0fcN5rQ=; b=BsIF2xA7faTGUESgOItf/BGVWvo8JQ0auEYdb5pC+hKeNSPPqWMCnfRON2WbHdPihb 6ouHYKqj+RkX8G9VZlQtPyAynrJJ/j71QnRVKbFFh1Tuxh1jJJa4FSG3qputbTj2bjRq B1zC66c2Kydv3vs0pPUlLZJHKgFPyUG5edLCLq8N8AXFwoMLvdf5ynf6aevmgWXv4Llw hRgCunMawdB6EwUYFf2A4102ZmN3Xp9ImLBOEG35GawJU/kZFJKa1AEHPZMM6X2nvlp2 A7xaWMN7kAhxfm3bLAcctOkmqdc5Ssm+ZL6XR9UqURpXk7bqNJbwL43m3juknst6M/TQ 8W0g== X-Forwarded-Encrypted: i=1; AJvYcCVTQRh6EwxssRdg72Ca27vPRHD7lhdUwZaou9Q81OopyVL5al+SgmilBwcTjy8+5r0IPTxhQZgzZYBRLEcX3IXwUJrgtsX9cfA84V7xPzFY X-Gm-Message-State: AOJu0YyGf1KK8wOz4YU5a99UnfheSQpN1nX+1yZlTMJisMW8LmrT4Qs0 6devKXB25I1vDeAoh/As4utVds43NorLXzB4eLzLlgzj+ZVXpNRQ76Seii4fmC4= X-Google-Smtp-Source: AGHT+IHEWuHXVZ/LoNCuQjq9ikFwqlpYM8ivEnjNJqMVh5lQ2hbDE0NnP51hssPDjMbEUOy86+sdSg== X-Received: by 2002:a05:6a00:14cf:b0:6e5:a996:5201 with SMTP id w15-20020a056a0014cf00b006e5a9965201mr172471pfu.10.1709248937282; Thu, 29 Feb 2024 15:22:17 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:16 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 04/13] riscv: mm: Broadcast kernel TLB flushes only when needed Date: Thu, 29 Feb 2024 15:21:45 -0800 Message-ID: <20240229232211.161961-5-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_152218_044986_642677E2 X-CRM114-Status: GOOD ( 12.80 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org __flush_tlb_range() avoids broadcasting TLB flushes when an mm context is only active on the local CPU. Apply this same optimization to TLB flushes of kernel memory when only one CPU is online. This check can be constant-folded when SMP is disabled. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v4) Changes in v4: - New patch for v4 arch/riscv/mm/tlbflush.c | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 0373661bd1c4..8cdb082f00ca 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -102,22 +102,15 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, unsigned long start, unsigned long size, unsigned long stride) { - bool broadcast; + unsigned int cpu; if (cpumask_empty(cmask)) return; - if (cmask != cpu_online_mask) { - unsigned int cpuid; + cpu = get_cpu(); - cpuid = get_cpu(); - /* check if the tlbflush needs to be sent to other CPUs */ - broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids; - } else { - broadcast = true; - } - - if (!broadcast) { + /* Check if the TLB flush needs to be sent to other CPUs. */ + if (cpumask_any_but(cmask, cpu) >= nr_cpu_ids) { local_flush_tlb_range_asid(start, size, stride, asid); } else if (riscv_use_sbi_for_rfence()) { sbi_remote_sfence_vma_asid(cmask, start, size, asid); @@ -131,8 +124,7 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, on_each_cpu_mask(cmask, __ipi_flush_tlb_range_asid, &ftd, 1); } - if (cmask != cpu_online_mask) - put_cpu(); + put_cpu(); } static inline unsigned long get_mm_asid(struct mm_struct *mm) From patchwork Thu Feb 29 23:21:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D02ABC48BF6 for ; Thu, 29 Feb 2024 23:22:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xSJNB+WljEt6zSY9Y2oOakXqXXPUcaIDmq/8qWXDdTg=; b=1A4OaWDfIl8N6P QRZirAR2VN+9NBRmOFGrm7CpgYbxhnCkTya21Z6fmz6N0jSgTn9sT7DPMCMKVddVVxnJokNs484/s 3c9rlrlgrETH7DAtTrbk+xHHNbGhRiA8i2ZiNrQ/2dx6EsjrYE/KOkYo+nepvPkuhs2lHJ+iafPoQ fNwdDO4TmIc+PpPNngBTEHRzHx4Us6SkXUOx9d2GVEQuFy/t0XdKerJAC153b6vu9XqszhGPNL3fF U4HftXB6FGXnV2vxOkqpxcpW+S3MKV+iMB/QVpAETa/IkInemzIzMCgzzPhcD3Qd/nJdqvui2iYtL IJOh4im1mQ8Yu4hZx1Lg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjZ-0000000Fa8a-3YIO; Thu, 29 Feb 2024 23:22:37 +0000 Received: from mail-oi1-x234.google.com ([2607:f8b0:4864:20::234]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjH-0000000FZue-3aM0 for linux-riscv@lists.infradead.org; Thu, 29 Feb 2024 23:22:21 +0000 Received: by mail-oi1-x234.google.com with SMTP id 5614622812f47-3c19aaedfdaso723678b6e.2 for ; Thu, 29 Feb 2024 15:22:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248938; x=1709853738; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9YO6O944kqWb4oNKtEMwGtpxpVk1nDXsHdzKgYw/NyE=; b=aAX1L979Qg6qqAYiD/986t/7RQ1+yFJKjbvbBMGMG/Njmk/YGbJhrKf2wjG1taWsBq uSYh6ZcZfadWMTDdHx6qkciTmWTHACKxTcQnkOzEHxw9cypFhELcWB1QfvnbzPD+8RAJ N9qBMU89LOPRa3wBAQhFbSgox3mr38S1YlXMCA97+2EZURVi5v8j28PL+vNgd2VXwfbz NCQ3QEeOQxG4YeK3XtCaY9kq5Wfk5+bwqIzepI2VQLBGutqAFjqVsDbq+9ndYN8Q20ZK u9DzJSd2KbaBIQTUT5svbtdhe8gV2Ms14TU1JUaD1rMiWx76JaScL6i+7Shzioj9xtaY 6GvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248938; x=1709853738; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9YO6O944kqWb4oNKtEMwGtpxpVk1nDXsHdzKgYw/NyE=; b=fMynwB6yj6TwUy8rH2oeBNrnoG+Dgm95E41OA4x5pk2UAZLwMJYy4bPBN9i5BQwUES vts85EPibLGeJEGZpeVBurc0/FpuV8zCLe1aNuO+MIV7J1+YuGHECwkz57sUdfjB9mHv pH/F2BZW2a4Sr9sn6U5JeRBv1yWsy4i2Yw9m2Nmdx/m8hh8yB5CTLhcJyLXsaJ+Qv57E eXhLMa9LK6VIifbTLJZN317QnSfxc8ZAkeBk8RizFeI5MQYunXs1FAg0ErlnthSQxsKQ VUcIAm6P7AyVfwDBt/M8fuB175Q6GrWmCazvV5gH7WAmgG+Qd9+7kKDl5tQ4BVvFAL8s hTmA== X-Forwarded-Encrypted: i=1; AJvYcCVe7r7N8F0dup3hWRMqnlrjUlx/ocMc3xZUBlW0GG4pFXo671k5d8SBNRXsJwyYgEiKg9Buyr00MUBAWhfrnrh4jETQql58qbHKtdWW3olN X-Gm-Message-State: AOJu0Yyi+ORNbrQQnDIEi82MX9aNe+K0r3uM/LYitn1M0XPnw6AQpdOs N18SLj2muwt+vq/FFVGkwWgEztaEg3FKeT4DpY8Slerk0YZQFalV4kovfsJ7Gy4= X-Google-Smtp-Source: AGHT+IF/+PdA1OOJxMn23/EE1kPZ70u92KMkSqj19wYtg5glHTCWIaPOP2OtxcWfToq1z4V1DP9b6Q== X-Received: by 2002:a05:6808:8f2:b0:3c1:c37b:1a71 with SMTP id d18-20020a05680808f200b003c1c37b1a71mr62346oic.57.1709248938472; Thu, 29 Feb 2024 15:22:18 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:17 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 05/13] riscv: Only send remote fences when some other CPU is online Date: Thu, 29 Feb 2024 15:21:46 -0800 Message-ID: <20240229232211.161961-6-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_152220_097956_5D6A5B78 X-CRM114-Status: GOOD ( 11.47 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org If no other CPU is online, a local cache or TLB flush is sufficient. These checks can be constant-folded when SMP is disabled. Signed-off-by: Samuel Holland --- (no changes since v4) Changes in v4: - New patch for v4 arch/riscv/mm/cacheflush.c | 4 +++- arch/riscv/mm/tlbflush.c | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c index 47c485bc7df0..f7933ae88a55 100644 --- a/arch/riscv/mm/cacheflush.c +++ b/arch/riscv/mm/cacheflush.c @@ -21,7 +21,9 @@ void flush_icache_all(void) { local_flush_icache_all(); - if (riscv_use_sbi_for_rfence()) + if (num_online_cpus() < 2) + return; + else if (riscv_use_sbi_for_rfence()) sbi_remote_fence_i(NULL); else on_each_cpu(ipi_remote_fence_i, NULL, 1); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 8cdb082f00ca..69402c260a89 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -78,7 +78,9 @@ static void __ipi_flush_tlb_all(void *info) void flush_tlb_all(void) { - if (riscv_use_sbi_for_rfence()) + if (num_online_cpus() < 2) + local_flush_tlb_all(); + else if (riscv_use_sbi_for_rfence()) sbi_remote_sfence_vma_asid(NULL, 0, FLUSH_TLB_MAX_SIZE, FLUSH_TLB_NO_ASID); else on_each_cpu(__ipi_flush_tlb_all, NULL, 1); From patchwork Thu Feb 29 23:21:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577726 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5474BC48BF6 for ; Thu, 29 Feb 2024 23:22:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=L3Gsx2xAfN42BObcUVKQsacpvivb6HeU0mNIfhL1E4Q=; b=S+T0e5gDLvCxZ8 R//8LSBPcWm9LPe7oEQC9wmmJIIWI8Sm1n+rOdZwRl9enULFXI29O9Y6pLJVZfRYDOVQ7EJpS3DeM +F06NkfH8kIOSpe+nMGeYb9rEClTmgFXGIgs/62sows9qJmwY1Pymu9gfnPldgSALEVuweOl7NJnp xwukCfFYH5DYm1N4oMJ0NKL/BRCV2ZbRYZJsr2PvINwAy0Ks81Y6g4mbBmZ1DyVfdnpAlg/1Q7tnN WjEd1NLIZhN/b6YndzmOYg8RKK3hIQAmwx56X9WWGzbl6JP2dh2Tn/ZAQCRal+wwmozCJPQrWHegX f2PVzbKXOqXpmSIODDew==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpje-0000000FaB9-0NCz; Thu, 29 Feb 2024 23:22:42 +0000 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjI-0000000FZvD-2zK3 for linux-riscv@lists.infradead.org; Thu, 29 Feb 2024 23:22:22 +0000 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-6e4f5e84abeso1037892b3a.2 for ; Thu, 29 Feb 2024 15:22:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248939; x=1709853739; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uLB9vlYe9JfwQMhDb14nNgv57tMyr8X6yVrB4e0zcQU=; b=gGxmRQnJbnoM8OpNGlZ731YQ574YSJgcexqaLlm+wLnYrKRLQ8cbLSA8jVIjQZWhMY vMEECT6gHiLYt8zk+P10uZMRSTzeQByGBCvnJvgiBHgQLbIF5SgrMvHd0XPSrIp3ry/2 oc634tihZOPDU5rW3ed0ZRXJoLa2+BGITMXZkLT5Lncxtpvjog9Fn9ZaCykpyUKRjlTj O5/LqN2sgB7N4apwjO/2KWX78RhT5Rd8QR2eHixktV8fQML9py+PnzQHzcIBEtteg3eO W8+qsaVmRIYBQ9TTVCyZcMDTrnEXv1+2sc2lUlTncMa9kTmxkaMhbc4p2NUmodkaRbMZ kQ0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248939; x=1709853739; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uLB9vlYe9JfwQMhDb14nNgv57tMyr8X6yVrB4e0zcQU=; b=viuDAHZdJM83Ogl5gSWs9Pm7LsG2RR1fTfLPLGjpNX/cUoPE2hE1k1af0R/N0nIJRy LpGhd+4v3/uaUCIMob1kJm8t0XL1nrqg7rhlyCVDDFeKQHMiBMKJPHAMbD+uMjhfeSpG EsH+rq3uY0z7/TI2uV/fBKqwOLITKeb07HJiaTldWkRt6VlnH9VWNz2webNRQBXzZIhk 5WE4ROI/hgKl75hbjatspmkIT1gIlo0Jl5PC3Ua+3ABasAe8uLHKme8KDGkE3rQpOyBl 4afXlGD2hMK3cQOhJLnVxjKlb++Y1h4yS80+W09M2W43FotLLwONiYR5pQMHsXfXK1Ue aaZA== X-Forwarded-Encrypted: i=1; AJvYcCVsvXZTM9bNz0cs4G6KuUofTRvT7CeNu0Sz7jleSbr2Ym099wDN6M85mU68DukjTwqfRaXSSX7WrYsZBxnQbN0ReHvhBUoQHP2bwFe359fp X-Gm-Message-State: AOJu0YwXWa2XL9irD8T6aiXMZCN+AuW8tWBLEpp3Vex6IFjQQjzcuINO v4tu4JHyJQwdUFbdi2EzP6Ab8vn0D/KeDGqGSebcyuWQJu+GJ8J+FLsprEQmhYg= X-Google-Smtp-Source: AGHT+IFNMqThgNRh63sPPm1+a+RZC7oBqAsJKeQJ9ncYhwlwuRqR+WMAL4ZrZQk20TebpNdIS4mveg== X-Received: by 2002:a05:6a00:2354:b0:6e5:80a4:2ff2 with SMTP id j20-20020a056a00235400b006e580a42ff2mr152068pfj.30.1709248939492; Thu, 29 Feb 2024 15:22:19 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:19 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 06/13] riscv: mm: Combine the SMP and UP TLB flush code Date: Thu, 29 Feb 2024 15:21:47 -0800 Message-ID: <20240229232211.161961-7-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_152220_876973_93947F93 X-CRM114-Status: GOOD ( 13.17 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org In SMP configurations, all TLB flushing narrower than flush_tlb_all() goes through __flush_tlb_range(). Do the same in UP configurations. This allows UP configurations to take advantage of recent improvements to the code in tlbflush.c, such as support for huge pages and flushing multiple-page ranges. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v4) Changes in v4: - Merge the two copies of __flush_tlb_range() and rely on the compiler to optimize out the broadcast path (both clang and gcc do this) - Merge the two copies of flush_tlb_all() and rely on constant folding Changes in v2: - Move the SMP/UP merge earlier in the series to avoid build issues - Make a copy of __flush_tlb_range() instead of adding ifdefs inside - local_flush_tlb_all() is the only function used on !MMU (smpboot.c) arch/riscv/Kconfig | 2 +- arch/riscv/include/asm/tlbflush.h | 30 +++--------------------------- arch/riscv/mm/Makefile | 5 +---- 3 files changed, 5 insertions(+), 32 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 0bfcfec67ed5..de9b6f2279ff 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -60,7 +60,7 @@ config RISCV select ARCH_USE_MEMTEST select ARCH_USE_QUEUED_RWLOCKS select ARCH_USES_CFI_TRAPS if CFI_CLANG - select ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH if SMP && MMU + select ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH if MMU select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_GENERAL_HUGETLB if !RISCV_ISA_SVNAPOT diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 928f096dca21..4f86424b1ba5 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -27,12 +27,7 @@ static inline void local_flush_tlb_page(unsigned long addr) { ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); } -#else /* CONFIG_MMU */ -#define local_flush_tlb_all() do { } while (0) -#define local_flush_tlb_page(addr) do { } while (0) -#endif /* CONFIG_MMU */ -#if defined(CONFIG_SMP) && defined(CONFIG_MMU) void flush_tlb_all(void); void flush_tlb_mm(struct mm_struct *mm); void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, @@ -54,27 +49,8 @@ void arch_tlbbatch_add_pending(struct arch_tlbflush_unmap_batch *batch, unsigned long uaddr); void arch_flush_tlb_batched_pending(struct mm_struct *mm); void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); - -#else /* CONFIG_SMP && CONFIG_MMU */ - -#define flush_tlb_all() local_flush_tlb_all() -#define flush_tlb_page(vma, addr) local_flush_tlb_page(addr) - -static inline void flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) -{ - local_flush_tlb_all(); -} - -/* Flush a range of kernel pages */ -static inline void flush_tlb_kernel_range(unsigned long start, - unsigned long end) -{ - local_flush_tlb_all(); -} - -#define flush_tlb_mm(mm) flush_tlb_all() -#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() -#endif /* !CONFIG_SMP || !CONFIG_MMU */ +#else /* CONFIG_MMU */ +#define local_flush_tlb_all() do { } while (0) +#endif /* CONFIG_MMU */ #endif /* _ASM_RISCV_TLBFLUSH_H */ diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index 2c869f8026a8..cbe4d775ef56 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -13,14 +13,11 @@ endif KCOV_INSTRUMENT_init.o := n obj-y += init.o -obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o pgtable.o +obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o pgtable.o tlbflush.o obj-y += cacheflush.o obj-y += context.o obj-y += pmem.o -ifeq ($(CONFIG_MMU),y) -obj-$(CONFIG_SMP) += tlbflush.o -endif obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_KASAN) += kasan_init.o From patchwork Thu Feb 29 23:21:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577800 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 63ED2C48BF6 for ; Fri, 1 Mar 2024 00:32:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=qpwWx15bh9fvjLMGBGYrY1WRs6WjpaDiQUANxiP9XYs=; b=DA9JnJZSYRnmhw rC2w2/Q0WuAq4rxViNON3O//zNPw0CfiIuafozrDR3LRA9F4tOSLLWzGTpHIqBpAn/qlSvgawxKu+ 5X60FQt2IohHr9ICTWsbMYSROVuIl0Ubn7/VgbxH7WyJG/7y8/FFt6jAjDT+P2spkOGuWHHaKpqbb r1RxoQKx5idz9+ikXWnkFJFNaQd1oBFqyHBqho9RMQsXoWwJ+p0v2jB5zAmCkBlywrs4wOPA30RvF 7XfPYKOfxft2o0BeQ/qS2Ruw54iAJrxwjUmRsEmmEtHor0eiAVQSvDTyf9xV5KnMEQPNH5IrB/p1p 9oYneZcjp40XnT8EPvcg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfqou-0000000FkuS-42YI; Fri, 01 Mar 2024 00:32:12 +0000 Received: from mail-ot1-x332.google.com ([2607:f8b0:4864:20::332]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjK-0000000FZwI-0P0M for linux-riscv@lists.infradead.org; Thu, 29 Feb 2024 23:22:24 +0000 Received: by mail-ot1-x332.google.com with SMTP id 46e09a7af769-6e49a5b1bbfso739423a34.3 for ; Thu, 29 Feb 2024 15:22:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248940; x=1709853740; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4jEC1X5CZdJa4ryFg9hWmhSsIovSPBi+ZbblNH/6mwk=; b=N3eCpfQq1r7ZBDIwOaS/gX6qVdZF+MPSgJWTtvghWvpwkOZmljpXXDzpu8lL/WUlzA Mzef/UhcuiOzB/L+c3H0wTHvTNQDIVeamm0DPiLJI3BfmoFu120c7EF19lB7Blr2ZuGi 6w84krWzVAWZa9lI8NmAo+7ly6vJYbkKwwlW6dhH+Z0xVtWF7p+nhnbxmxCWb2OIxu/D sVx2iswEh2ddVb8VaI+5Z3V0nuOPBRzbkHjCqOgM/X/L60g5kljAxSeordZsHUded3K1 k+e/6O7PPRbzDKIpoqugKuxikUIm4oe1jIXJC39dfBUH1XR76CqyOm2sRpCJhzjGDVrz uFYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248940; x=1709853740; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4jEC1X5CZdJa4ryFg9hWmhSsIovSPBi+ZbblNH/6mwk=; b=AiOjXmO5XazKl2fyPePK0BwICERwkPKH4kYWBO7QArOge4oSHtEnfcRQsiTwmxWVtb 8RNctFqjzhc8Ho2+hi5HWfbUE0FvYeyRMhwOJrOV8Xf1GBgxOCszGLEE662umfKfQfs+ 1MrvmUcXkCjPwvYRa7I6s2OIO4fECt8MjfZx3eAAiDe/hhRBr/od75kT1DFamJDGyqGN Iym7Da/hjswvXd+KwyoIJgkpCxY+bS3snTMcoDNHJxmaC8rJaxzjhP7Jkf5UzHrhL2vN bjW4PCVdP7iwgPHqLmEnXgjI/OBlOjQyI2tjt80SLKGqF8LoNqxf8lKOrGpB0zUi81sZ eOvw== X-Forwarded-Encrypted: i=1; AJvYcCUJ5QfHbB0zTroUirAEkeETpb6ansgUeI1E/XKQfTVGnSLIXtvrMcFlQO/SB4K2FkV/jXcwCcaDrLEe7i5eO8ohTM3ihxXdiN+lQlI1lJCg X-Gm-Message-State: AOJu0YzdWIECVQ+4fCYdF9bTu0MS9HswftUZhESFr0d7bD/dq2AkKe24 SREyNdVXIWDt1t/fur9JAZ3/MtuXEF1zaOLoiL1z0p5MKULWkpekPsFhGqOoGQfO8MsL+3wAW2h n X-Google-Smtp-Source: AGHT+IGX4/HV5W58fetZh8Yw57gYr75gd/4aFWyaezvIp/BsV5LakI2yn+mW+sbVnuOSeLnU1moubA== X-Received: by 2002:a05:6870:ac22:b0:21e:dd7a:2d3e with SMTP id kw34-20020a056870ac2200b0021edd7a2d3emr4104469oab.22.1709248940592; Thu, 29 Feb 2024 15:22:20 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:20 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 07/13] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Date: Thu, 29 Feb 2024 15:21:48 -0800 Message-ID: <20240229232211.161961-8-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_152222_366526_026AB0B4 X-CRM114-Status: GOOD ( 13.73 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org commit 3f1e782998cd ("riscv: add ASID-based tlbflushing methods") added calls to the sfence.vma instruction with rs2 != x0. These single-ASID instruction variants are also affected by SiFive errata CIP-1200. Until now, the errata workaround was not needed for the single-ASID sfence.vma variants, because they were only used when the ASID allocator was enabled, and the affected SiFive platforms do not support multiple ASIDs. However, we are going to start using those sfence.vma variants regardless of ASID support, so now we need alternatives covering them. Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Rebase on Alexandre's "riscv: tlb flush improvements" series v5 arch/riscv/include/asm/errata_list.h | 12 +++++++++++- arch/riscv/include/asm/tlbflush.h | 19 ++++++++++++++++++- arch/riscv/mm/tlbflush.c | 23 ----------------------- 3 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index ea33288f8a25..6e2762101968 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -43,11 +43,21 @@ ALTERNATIVE(__stringify(RISCV_PTR do_page_fault), \ CONFIG_ERRATA_SIFIVE_CIP_453) #else /* !__ASSEMBLY__ */ -#define ALT_FLUSH_TLB_PAGE(x) \ +#define ALT_SFENCE_VMA_ASID(asid) \ +asm(ALTERNATIVE("sfence.vma x0, %0", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (asid) : "memory") + +#define ALT_SFENCE_VMA_ADDR(addr) \ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \ ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ : : "r" (addr) : "memory") +#define ALT_SFENCE_VMA_ADDR_ASID(addr, asid) \ +asm(ALTERNATIVE("sfence.vma %0, %1", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (addr), "r" (asid) : "memory") + /* * _val is marked as "will be overwritten", so need to set it to 0 * in the default case. diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 4f86424b1ba5..463b615d7728 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -22,10 +22,27 @@ static inline void local_flush_tlb_all(void) __asm__ __volatile__ ("sfence.vma" : : : "memory"); } +static inline void local_flush_tlb_all_asid(unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ASID(asid); + else + local_flush_tlb_all(); +} + /* Flush one page from local TLB */ static inline void local_flush_tlb_page(unsigned long addr) { - ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); + ALT_SFENCE_VMA_ADDR(addr); +} + +static inline void local_flush_tlb_page_asid(unsigned long addr, + unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ADDR_ASID(addr, asid); + else + local_flush_tlb_page(addr); } void flush_tlb_all(void); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 69402c260a89..365e0a0e4725 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -7,29 +7,6 @@ #include #include -static inline void local_flush_tlb_all_asid(unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma x0, %0" - : - : "r" (asid) - : "memory"); - else - local_flush_tlb_all(); -} - -static inline void local_flush_tlb_page_asid(unsigned long addr, - unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma %0, %1" - : - : "r" (addr), "r" (asid) - : "memory"); - else - local_flush_tlb_page(addr); -} - /* * Flush entire TLB if number of entries to be flushed is greater * than the threshold below. From patchwork Thu Feb 29 23:21:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5B9A9C54798 for ; Thu, 29 Feb 2024 23:22:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=WpEDhXIF8JMzpBV5mT6dS8lMs7DdOAo958v3Ukr6Co0=; b=wJRRTU/kFCkJeY ta23JwvW9LYQycOKEjrYsTsd+MEqPbtBO5inDEhkdQZ4G06UuX8Qal1Y68FBLF+8P0Gw4y/zynv8p h4mh5CJaUvAdObpOs35NbVVcFXTCOlIBPv4b2NBtQZBg91pktZABu5hEwyWAqprbO8reox9hZsYfp bvDit4/AqeUW1zkOYR7XNO3uCWkTIl8OCkRHnzRJ/p/7sxT6I7xq4tA37MmqFv1px35JQfZZcOUyo 4nZuHXjZj1khop8NfYHetnTdiM7GxjegBnSnuVGzBqPPes/65wZbm5anehCVWLGjl90Wh6Yyz5knW 7AhQBWxKe+AGH+WqSKSA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjk-0000000FaG3-0Zod; Thu, 29 Feb 2024 23:22:48 +0000 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjK-0000000FZx4-1JUk for linux-riscv@lists.infradead.org; Thu, 29 Feb 2024 23:22:24 +0000 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-6e58d259601so911850b3a.3 for ; Thu, 29 Feb 2024 15:22:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248942; x=1709853742; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wHMbuRyYmPRZ8L34XsarFB0wBhmLHt6PApeEi+FQBPY=; b=Dcz9I2Y8SfIwBajL/i1hdnInmExPwVquCghDviZFtBsU+mQ5/W7V9rN+xNphrV//+x uMQanPv2jEundL9a5IOA70D87OgR2sW6zF7KNWOZSigkhE89CaUzjZZR+sAxIDhtxdFB xU3PViqhISQSGg44/NOFvbXqEJ5okDmL3JYU0pZMn+8XOsPN0JKiWFViUbavPMNGtJQL eTYvtxD/XZLj+4kMmIBz5smDJ84bh9dtvn6+lUP7MzVs+mDYT8MOAdL2B/9v0CWQYjg2 7ZqU8vU/W1KJZa5FjQSBOZeB+wXUdhWM4gF7wupq8Y8Xjxf9Ln9+wB60M7qfER2ArYFR TGMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248942; x=1709853742; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wHMbuRyYmPRZ8L34XsarFB0wBhmLHt6PApeEi+FQBPY=; b=a2/l5BwWnwUKaRS2J94uuPrWeL4HdM3pubRrWWQA4dtHXgIKg2SwlrB3eo21/p5kOB /I0jqc6V/e3t98ukItxBDDPnlBTSauoAeMbgL1xU6lNyn4nw85SSKm8xshagPIThK+u4 B6Jwhh2cWhxKpLNtjEIUJ0fc6jHiyVFnQYGat3H2eUUfFUUHreaV178Q3gaJJP+EkRQG 5d9SXlQqAbntMi/kj11bfw2i5fSzpebPrns+p6V/37xBSGatHsU0YMg4pzV7o8/+7wMT +RW4NZDXvGtVF6sUKk1aj4LxW4ajSPI5wl5G26WcIv3AYM0WtyfmP2yLoUO3OMJ9jLDC Iatw== X-Forwarded-Encrypted: i=1; AJvYcCU4Ot0D3MBsH+p2LoTndu9YlMWG72MTdDeAaKRo+kFN+3pwIS0Ko2oiGUGtNs5uUVbE8HAciVcEBqfBiZ1BqExEZV7JyY4bl0Bl6LC+HZuM X-Gm-Message-State: AOJu0YxAvfyE8G4rRyWaoNe6XwC8F+Wr8UYlHN6JHhikLL0OVnLarhbt uBvz9EeX7sIMxeJgdk/HImimEVPSL4YoEsRbxC9f/3FNMlIiWQLNukS53VMppLQ= X-Google-Smtp-Source: AGHT+IE0KT3RuIOZoiwK4WAulPfaiFPRjhF12PREr/S+YlSI1LgFKc32EbVKWDUbhqJfm1349m8bOA== X-Received: by 2002:a05:6a00:1955:b0:6e4:eb29:357 with SMTP id s21-20020a056a00195500b006e4eb290357mr165166pfk.12.1709248941678; Thu, 29 Feb 2024 15:22:21 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:21 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 08/13] riscv: Avoid TLB flush loops when affected by SiFive CIP-1200 Date: Thu, 29 Feb 2024 15:21:49 -0800 Message-ID: <20240229232211.161961-9-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_152222_495812_7F190381 X-CRM114-Status: GOOD ( 12.80 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Since implementations affected by SiFive errata CIP-1200 always use the global variant of the sfence.vma instruction, they only need to execute the instruction once. The range-based loop only hurts performance. Signed-off-by: Samuel Holland --- (no changes since v4) Changes in v4: - Only set tlb_flush_all_threshold when CONFIG_MMU=y. Changes in v3: - New patch for v3 arch/riscv/errata/sifive/errata.c | 5 +++++ arch/riscv/include/asm/tlbflush.h | 2 ++ arch/riscv/mm/tlbflush.c | 2 +- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c index 3d9a32d791f7..716cfedad3a2 100644 --- a/arch/riscv/errata/sifive/errata.c +++ b/arch/riscv/errata/sifive/errata.c @@ -42,6 +42,11 @@ static bool errata_cip_1200_check_func(unsigned long arch_id, unsigned long imp return false; if ((impid & 0xffffff) > 0x200630 || impid == 0x1200626) return false; + +#ifdef CONFIG_MMU + tlb_flush_all_threshold = 0; +#endif + return true; } diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 463b615d7728..8e329721375b 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -66,6 +66,8 @@ void arch_tlbbatch_add_pending(struct arch_tlbflush_unmap_batch *batch, unsigned long uaddr); void arch_flush_tlb_batched_pending(struct mm_struct *mm); void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); + +extern unsigned long tlb_flush_all_threshold; #else /* CONFIG_MMU */ #define local_flush_tlb_all() do { } while (0) #endif /* CONFIG_MMU */ diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 365e0a0e4725..22870f213188 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -11,7 +11,7 @@ * Flush entire TLB if number of entries to be flushed is greater * than the threshold below. */ -static unsigned long tlb_flush_all_threshold __read_mostly = 64; +unsigned long tlb_flush_all_threshold __read_mostly = 64; static void local_flush_tlb_range_threshold_asid(unsigned long start, unsigned long size, From patchwork Thu Feb 29 23:21:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577799 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6A895C48BF6 for ; Fri, 1 Mar 2024 00:32:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=z0IbOwokXgmxVZaz4iN34hpmtQ2UZ3zQv4QXGOfEgjY=; b=u6Z2atLrF8BLxE MRxFHgYY2JyvQzYDczBOO3cThEBwVHaG8YJ1CvNmrK1LF8LjDZmU9PmYrifWq9OpMf1uqEBtZgRYR M81t4tdWTxrWzExmtCl+ceCPtfTyvi4r4OIpNSR0tzLgBEflS6RznjMlfcbARwqhvtOzEgwjYFisK UfEl8nMcYBb0BNXhYwcA0toF8R17wCW51OaaPyzPW5UhANUHxK1Q4vqUyMMpJqwKPghZxfCL0H+zx tx+8XF4KE+IiRHXtLhneUyrpXghEs91GrXUzSuDRcqqhcW35NMNZPIzJdQpFp/4N+FGJfyl5WOlkk q0iyj3cfQBQt8rzGhQAg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfqov-0000000Fkud-20Gp; Fri, 01 Mar 2024 00:32:13 +0000 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjL-0000000FZxh-3tcW for linux-riscv@lists.infradead.org; Thu, 29 Feb 2024 23:22:26 +0000 Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-6e58a984ea1so1084434b3a.2 for ; Thu, 29 Feb 2024 15:22:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248943; x=1709853743; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lukLV3IzObY+fmX3TTuFURPx1t0s+LwQl+LTSkieoBQ=; b=ncD/diID9xPBjoJxhi1UzumSLcXd7YhsqbEYTZpX/2hn7GpIoiH7TYsHub1CH38NiW aNEIosgMDyiMycFTVAIws9bXWPY7JWwbjbll4+nCx8ooZBuGu3dl2y0SptfjDR2AvBCF l2fyaMJX2Ip85y1ICON9XoY/aUQsQGyeyAd+iBUBbvbCLdG4Gh02Z8feR0Jk6YHmkW46 oqVtx53Bf1QAheSgph6Q+BVZsvolsP0nnv3Ri8jp3HqA4004WlXbe2McsUJXeUdfG7DH wIy4p5u5w2ucVtSeJPGJBWLxRHFoBICWZBOXEeoE6mwj/mG61q/ECk0Ni4ZaSJymv4S3 FxVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248943; x=1709853743; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lukLV3IzObY+fmX3TTuFURPx1t0s+LwQl+LTSkieoBQ=; b=oHL8xjqiVAPa1sKOjYZFcerX6OasGawxLeq3H4QTTOQfdOsXxfQKWJE+ySOy9h4pIi fmSdW0Q4REHBNqLsprZpaSIhskmEKgnZr4cdalcUZiQED3qY2bLAa9g4Q1dw2f3FPc9a FlTRP2QaAMpy9e3Eu1ekq3kD8sOt8G4qx7UBNj9O+xPLoYjqPUirPEzCwdrU30vpmF1B I+GoNG+NDcEOLdPbSXQTuylnvNI4RFxTCtCJPu5jLbAiynDGGdEWeGotVRzETAg877UC zZ766s7skHnOdLdR/7RqemVVUDMsKSSfGV6Lg4KYetUGLrCSyCl3xUHK5lg+HAroIy0L 5GLw== X-Forwarded-Encrypted: i=1; AJvYcCUSwO8J4mwidA5vtFKpntjGutioswlo/dsWa5VkatNb9i4iOv/M0A2LBukigjYqaKnRb1qVa6ksByr/lYhLpVg0LdtpkDwBNDwkgrSr2L4Z X-Gm-Message-State: AOJu0YxKRYNOS4g5SyTsUXXd3DbEPNK2GL0dfgP1GRPXwWlsYx25y2XD 4WM6Z+re7flJP3pzrGjCYU/koTxc98MCJ1CnUH8lkZ9F0f0Xi4RNwlI0MIGAvUEyFABaz4/Jmpl r X-Google-Smtp-Source: AGHT+IFn2rYTQCqOuHVpuGhcF7BC7asS4ROedWcgIAd4N2h6ux4X4gjzLV3woCw85UyqGNmjtKX//g== X-Received: by 2002:a05:6a00:1955:b0:6e5:5116:2cb5 with SMTP id s21-20020a056a00195500b006e551162cb5mr172054pfk.15.1709248942678; Thu, 29 Feb 2024 15:22:22 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:22 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 09/13] riscv: mm: Introduce cntx2asid/cntx2version helper macros Date: Thu, 29 Feb 2024 15:21:50 -0800 Message-ID: <20240229232211.161961-10-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_152224_244260_FD3424E9 X-CRM114-Status: GOOD ( 13.78 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org When using the ASID allocator, the MM context ID contains two values: the ASID in the lower bits, and the allocator version number in the remaining bits. Use macros to make this separation more obvious. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/include/asm/mmu.h | 3 +++ arch/riscv/mm/context.c | 12 ++++++------ arch/riscv/mm/tlbflush.c | 2 +- 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/arch/riscv/include/asm/mmu.h b/arch/riscv/include/asm/mmu.h index 355504b37f8e..a550fbf770be 100644 --- a/arch/riscv/include/asm/mmu.h +++ b/arch/riscv/include/asm/mmu.h @@ -26,6 +26,9 @@ typedef struct { #endif } mm_context_t; +#define cntx2asid(cntx) ((cntx) & asid_mask) +#define cntx2version(cntx) ((cntx) & ~asid_mask) + void __init create_pgd_mapping(pgd_t *pgdp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, pgprot_t prot); #endif /* __ASSEMBLY__ */ diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index ba8eb3944687..b562b3c44487 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -81,7 +81,7 @@ static void __flush_context(void) if (cntx == 0) cntx = per_cpu(reserved_context, i); - __set_bit(cntx & asid_mask, context_asid_map); + __set_bit(cntx2asid(cntx), context_asid_map); per_cpu(reserved_context, i) = cntx; } @@ -102,7 +102,7 @@ static unsigned long __new_context(struct mm_struct *mm) lockdep_assert_held(&context_lock); if (cntx != 0) { - unsigned long newcntx = ver | (cntx & asid_mask); + unsigned long newcntx = ver | cntx2asid(cntx); /* * If our current CONTEXT was active during a rollover, we @@ -115,7 +115,7 @@ static unsigned long __new_context(struct mm_struct *mm) * We had a valid CONTEXT in a previous life, so try to * re-use it if possible. */ - if (!__test_and_set_bit(cntx & asid_mask, context_asid_map)) + if (!__test_and_set_bit(cntx2asid(cntx), context_asid_map)) return newcntx; } @@ -168,7 +168,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) */ old_active_cntx = atomic_long_read(&per_cpu(active_context, cpu)); if (old_active_cntx && - ((cntx & ~asid_mask) == atomic_long_read(¤t_version)) && + (cntx2version(cntx) == atomic_long_read(¤t_version)) && atomic_long_cmpxchg_relaxed(&per_cpu(active_context, cpu), old_active_cntx, cntx)) goto switch_mm_fast; @@ -177,7 +177,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) /* Check that our ASID belongs to the current_version. */ cntx = atomic_long_read(&mm->context.id); - if ((cntx & ~asid_mask) != atomic_long_read(¤t_version)) { + if (cntx2version(cntx) != atomic_long_read(¤t_version)) { cntx = __new_context(mm); atomic_long_set(&mm->context.id, cntx); } @@ -191,7 +191,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) switch_mm_fast: csr_write(CSR_SATP, virt_to_pfn(mm->pgd) | - ((cntx & asid_mask) << SATP_ASID_SHIFT) | + (cntx2asid(cntx) << SATP_ASID_SHIFT) | satp_mode); if (need_flush_tlb) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 22870f213188..e194e14e5b2b 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -109,7 +109,7 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, static inline unsigned long get_mm_asid(struct mm_struct *mm) { return static_branch_unlikely(&use_asid_allocator) ? - atomic_long_read(&mm->context.id) & asid_mask : FLUSH_TLB_NO_ASID; + cntx2asid(atomic_long_read(&mm->context.id)) : FLUSH_TLB_NO_ASID; } void flush_tlb_mm(struct mm_struct *mm) From patchwork Thu Feb 29 23:21:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A147CC54798 for ; Fri, 1 Mar 2024 00:32:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=qcvpCEED0It/EbtQ04L50Y4KCybYxBmXEmNm2oR6zQM=; b=oNykJ1qgdfbBPp MNMJglUOGrLsO0wLaIoiCg5YbbuduMefUauFvY7E8GsCQuQKTDwms5YwEbEBcvyVa7Hs3lIWUCYhz QSMe+MWonkPj2KPh1Lsy+ccIa5hxGKXG6jg9GFyVj9hw9snTBebefCT+K1AnbT67ey6m/gDoJFS0U bsN0qv8igGNpdd6SCS12H5ng4/U10Yp6fm7WZ3sb+9Bg8r0lXhf2sDyG7f74HY7V0OZyIUtsfCviY 90mln7EVoxONcfvi55opOAJVlIhSWalOW6+U/qTJ7swZBtbhaS64FwdOamW2sRlppJGHvzZP9YARO 8+pQuBWNxHLUXYmYl8Bg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfqow-0000000Fkur-0Ofh; Fri, 01 Mar 2024 00:32:14 +0000 Received: from mail-oi1-x22e.google.com ([2607:f8b0:4864:20::22e]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjN-0000000FZyG-17Uu for linux-riscv@lists.infradead.org; Thu, 29 Feb 2024 23:22:27 +0000 Received: by mail-oi1-x22e.google.com with SMTP id 5614622812f47-3c1c44a9554so655685b6e.0 for ; Thu, 29 Feb 2024 15:22:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248943; x=1709853743; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=A+Ju3+5NsQ7zqLi78mVdBh82QtKzTZZzrfFOA+rFDFs=; b=XeNX/u4z+54uPAGf0rmPCwpkq82gp6g8aPBrz7ONE/WtxjOuvUeCi47LfMmg3+8kCz SpHBaCT93I8IA/9rDEqME4cYd3ofh/m5hEsNlST/QtKOKs/5esEuaX+itbt8HiMGGXS8 uTNobT4U0bzzyprwuYKTXwGIimFsSkPB6Mv0eHSQTePS5jiOgirGuByxFerattp87rE+ 5LVzIoWnf5zfUZvr8Ji65F8a8LRoxUXsN9DLkqKyo1RyfcuAw8etIf8Bukx7GR9knC6r QZUpWQ23kg8XFNYqdT8mpXyjRpEhZhyM82nBuCdz3tUILBl1bMfiJ6+dcey6ehN/AFpS vMiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248943; x=1709853743; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=A+Ju3+5NsQ7zqLi78mVdBh82QtKzTZZzrfFOA+rFDFs=; b=iCMJDCOxosFYWc+6sJNDRx1ad+Mhc+97CUevDNzh784rRcJgN840pWkEuPEk6X590X 9iG/ru8wIEjCqrA5M39P4l4Iik36EpNjXrUGDWAQaCOd0qQlV1iyCGlGz/RvL3JAqDFh BsJRhgej41jar5DEVrvrBTUCvCdqLvP2Dh2NaoG+kFN5R9MLRJaLVBg7GjDChdr4DBIU JTdyL6ntz+RSZFZMduxvHb8Zi7JOhRiARmA/IqqVPZ41D4EnHes5Y3Z57qIa4S1tKEXT r66/r2VkJ+sgkbaBdVIhfO4A+LW2xIHmP2LkozrklCTJN6K7pYFWh6H1R/eK1zPhRoy9 4GXA== X-Forwarded-Encrypted: i=1; AJvYcCVviZPstAvPmYFo4um8MA/meJWtw0fhjxY0c1gHAf55ldKndtweJC/k1QjPXKnRGnm2PgEiqcrQrROWOrTsiGDB/9t41riZKt9dEBIYHJ4o X-Gm-Message-State: AOJu0Yygyl41H4K90x+6isim3irAuLymPrLGIfP0P0bH6XOYH5x6RTPR 1pKGVGb7sbhPv9JJBseQEelxIOI5E5bXtrrI2Zahd7X3P8EgXGzGyYoOz9qg9es= X-Google-Smtp-Source: AGHT+IHzbXWkwDmlfZ/eL4jDsB3ccQ2hOxgMwKxCvl8MiEGc4S4tbeeGWY8zmXhmQ74ouvbz4gXZvw== X-Received: by 2002:a05:6808:1447:b0:3c1:b28b:76f0 with SMTP id x7-20020a056808144700b003c1b28b76f0mr91074oiv.9.1709248943714; Thu, 29 Feb 2024 15:22:23 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:23 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 10/13] riscv: mm: Use a fixed layout for the MM context ID Date: Thu, 29 Feb 2024 15:21:51 -0800 Message-ID: <20240229232211.161961-11-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_152225_464679_02F3913B X-CRM114-Status: GOOD ( 11.91 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Currently, the size of the ASID field in the MM context ID dynamically depends on the number of hardware-supported ASID bits. This requires reading a global variable to extract either field from the context ID. Instead, allocate the maximum possible number of bits to the ASID field, so the layout of the context ID is known at compile-time. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/include/asm/mmu.h | 4 ++-- arch/riscv/include/asm/tlbflush.h | 2 -- arch/riscv/mm/context.c | 6 ++---- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/arch/riscv/include/asm/mmu.h b/arch/riscv/include/asm/mmu.h index a550fbf770be..dc0273f7905f 100644 --- a/arch/riscv/include/asm/mmu.h +++ b/arch/riscv/include/asm/mmu.h @@ -26,8 +26,8 @@ typedef struct { #endif } mm_context_t; -#define cntx2asid(cntx) ((cntx) & asid_mask) -#define cntx2version(cntx) ((cntx) & ~asid_mask) +#define cntx2asid(cntx) ((cntx) & SATP_ASID_MASK) +#define cntx2version(cntx) ((cntx) & ~SATP_ASID_MASK) void __init create_pgd_mapping(pgd_t *pgdp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, pgprot_t prot); diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 8e329721375b..72e559934952 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -15,8 +15,6 @@ #define FLUSH_TLB_NO_ASID ((unsigned long)-1) #ifdef CONFIG_MMU -extern unsigned long asid_mask; - static inline void local_flush_tlb_all(void) { __asm__ __volatile__ ("sfence.vma" : : : "memory"); diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index b562b3c44487..5315af06cd4d 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -22,7 +22,6 @@ DEFINE_STATIC_KEY_FALSE(use_asid_allocator); static unsigned long asid_bits; static unsigned long num_asids; -unsigned long asid_mask; static atomic_long_t current_version; @@ -128,7 +127,7 @@ static unsigned long __new_context(struct mm_struct *mm) goto set_asid; /* We're out of ASIDs, so increment current_version */ - ver = atomic_long_add_return_relaxed(num_asids, ¤t_version); + ver = atomic_long_add_return_relaxed(BIT(SATP_ASID_BITS), ¤t_version); /* Flush everything */ __flush_context(); @@ -247,7 +246,6 @@ static int __init asids_init(void) /* Pre-compute ASID details */ if (asid_bits) { num_asids = 1 << asid_bits; - asid_mask = num_asids - 1; } /* @@ -255,7 +253,7 @@ static int __init asids_init(void) * at-least twice more than CPUs */ if (num_asids > (2 * num_possible_cpus())) { - atomic_long_set(¤t_version, num_asids); + atomic_long_set(¤t_version, BIT(SATP_ASID_BITS)); context_asid_map = bitmap_zalloc(num_asids, GFP_KERNEL); if (!context_asid_map) From patchwork Thu Feb 29 23:21:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577728 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 04C1AC48BF6 for ; Thu, 29 Feb 2024 23:23:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GPO4FAsYN7bEGiElsjkzPhFKQfJ4daSR4nAocKvg4wk=; b=wKoNfRD7Dm23gr 1uElkhXLC0yUBZ78jLNRdiZzDMalY9JTlAGSn9p8aYrZUgJDGBA8+xsH6K2id1U7A3u/14Uh3B7r9 P54RaAUBFDIt8mem5EezoqPbU1n61oSOnqLdUBbqAgA+6ZbkNPZ4aVrzL1hj+PAY8cMvLvdyLKbmk hRlWzRFKuAsbqcjIeXbDGFGfh4yT/V/AjcwEH/p+d8bIEVN4GSEiCZtOyhwJtpsOnVFHFL82Qj8Gf rUSY+/NO4R6LhAIE9qVZ1D20B2oNacOcqo+TxOV5Dj4NJHph8AaHBWP+IKbj8EZ/56zypvNMryxTt YBScVEUBP89QwfYzxNbw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpkG-0000000FafN-3Rq6; Thu, 29 Feb 2024 23:23:20 +0000 Received: from mail-pf1-x42f.google.com ([2607:f8b0:4864:20::42f]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjO-0000000FZyw-0bgH for linux-riscv@lists.infradead.org; Thu, 29 Feb 2024 23:22:28 +0000 Received: by mail-pf1-x42f.google.com with SMTP id d2e1a72fcca58-6e4d48a5823so1324394b3a.1 for ; Thu, 29 Feb 2024 15:22:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248945; x=1709853745; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4wFLc5gcnBs285NRnicKVmmQVV53EnA/2rANcUM2uCA=; b=SUyEyu1+lXZ6/hlA6/PX8lQCeSzFG+wVQONL7CUtTV3fceOzw6ssXEtpPtDUWNsFGM X1ukZnrvxbSpN+xdm2/ZMoXvSZuKTOYckYQ9umwq+LSxdHWIqPnuxi1defYlYzA5DeQ0 slqfY+L4KyoPmSzph0RMZ/GlXy3lBpnNzubv5YjQ/VIgm+jBVaSYG9YrA7MGMh6ilRmS Hr2R6bO9xPe4jRC5/5K7UpsxVt1G3e4wZshngiaaGulZLgLYwHcl+Y4+227LK1zv5fDy mC4kyWri5immhdTnmFq0Iew0IzcUvzLPlMYYFxNK4yoEJoc3TfeyW4s/M2wAxd1hqsGF M5yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248945; x=1709853745; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4wFLc5gcnBs285NRnicKVmmQVV53EnA/2rANcUM2uCA=; b=VgaYEKvcU1OIvBHZK6jPnje6lVYfVw85hTz87vspqgUqu7eqio4Gi2tTXf3VOTWG/i QSldR85CHv5WaqISlsGQx0Gt8Ys+t8v8+vQlpMz929fZdj2+/K9sMsbPVbh4nyxtmXad M0wD/sRjxI773wFNrEnvrVWBeNUu6mLmZDCHUw7Nu9KZ0QMF+NfXCQcxCsZr0aM8SZ6b Nau3n6ErX1Z0fLJMOyGjeGeNI+mypuWnMVD+7HCHekAqSKnUzFcN5qeNzxVgZmeLT0Ku 4SiFNUgojQNolnxNb9FJ4y9pXDSfrDKBz5bcVuEEeGFgRw4uSOZVbSWgYZ4T/8Hh3nMm ujlg== X-Forwarded-Encrypted: i=1; AJvYcCU45UT1Rw1udcwgD6BuH0Or4dM6XDpKEqFCm22owI0iOwGClLM6L5ltr+Tox8tI7G6gaU8yGWwyMufs1BiNvy1wpzxk+fxHglnH4qsx/tUx X-Gm-Message-State: AOJu0Yzp745zBSH9M/jElsv9oQRGF2FYFR2YYT4cuue1JbXlNEkdaGlh GCvnNe2/KMuanIwMbCid27G7oW5DBIRwwbfrxuHnz3meo3HsOmgQIpYHbT4meFVuVyMu5bbun9p l X-Google-Smtp-Source: AGHT+IHQJB+ROn7vFDGIQNWMDCSQS4WpsKXvgO9ytwgYp+s+V0Upk9jrJAXqE3yMlwAjPHhZUdgCcg== X-Received: by 2002:a05:6a00:3d42:b0:6e4:eee0:5798 with SMTP id lp2-20020a056a003d4200b006e4eee05798mr179957pfb.14.1709248944849; Thu, 29 Feb 2024 15:22:24 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:24 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 11/13] riscv: mm: Make asid_bits a local variable Date: Thu, 29 Feb 2024 15:21:52 -0800 Message-ID: <20240229232211.161961-12-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_152226_477882_9F062C6D X-CRM114-Status: UNSURE ( 8.60 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This variable is only used inside asids_init(). Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/mm/context.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 5315af06cd4d..0bf6d0070a14 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -20,7 +20,6 @@ DEFINE_STATIC_KEY_FALSE(use_asid_allocator); -static unsigned long asid_bits; static unsigned long num_asids; static atomic_long_t current_version; @@ -226,7 +225,7 @@ static inline void set_mm(struct mm_struct *prev, static int __init asids_init(void) { - unsigned long old; + unsigned long asid_bits, old; /* Figure-out number of ASID bits in HW */ old = csr_read(CSR_SATP); From patchwork Thu Feb 29 23:21:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577730 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BC0C9C54798 for ; Thu, 29 Feb 2024 23:23:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=rLB3xJwyDt+V4H9HcdYD7X17kR2W/K5QMJXAG7DV0No=; b=w+wBgY+oz2D+sZ wjdY4ID0mjYhLtzxubLq6ZDCZTncT27HY4e57Jf0Yx9snPBw2h7JtvoKT37BfgmKmYL2DBmMj/eCd 2RFkg3vS1cxtdcTF0ihbcteiRvbDnHU158fmibpHTp/hZ1PJ3OKkIINi1gxP34pDi0KNTU6O4FP2Y A9ZBkVizwV75TvQOfFCZCzNtrvl4Bxvt9t5lsSDatq+NeluM2egZgqYhQmF/Dm/Hg88nA5wOGz+Yf NN0/ykR4ZBRyGiLDCnX9nDrTr7JajdlJiaWYEyHdkLFOCoEI09ZdDNYme+k2MjPkfTopI1+Vx/2XD 3wTJIxbzBJswLyfgBeEA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpkQ-0000000Falf-0ta1; Thu, 29 Feb 2024 23:23:30 +0000 Received: from mail-oi1-x22b.google.com ([2607:f8b0:4864:20::22b]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjQ-0000000FZzr-10zG for linux-riscv@lists.infradead.org; Thu, 29 Feb 2024 23:22:30 +0000 Received: by mail-oi1-x22b.google.com with SMTP id 5614622812f47-3c19e18470aso851590b6e.1 for ; Thu, 29 Feb 2024 15:22:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248946; x=1709853746; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hwpkv2OuO7YHUplKH98oFVfOa+CFbclxXNu0ysmduyU=; b=PfNG9M6gWuR1ksO6LejCbQDRd+YX7gPt+DlcfQ2QoyB2D7dHSi0HytU3855fw7Sx6F Rj60ZxXhCE/3vJ17vDkVy0HuIOUSvnfbd+A6F1YnuLJFjsAYUUM0pRyuO6jEO5Wfe9gJ pqkMoJQPudeXK0kcT29E1z9N7Gw26ek+i4j3AhvgKeNNYzDS8RgtJuVVn0/Z9tPJ5cZ0 +NXQxDop4KxuQUy0dn+BSmtzlorarKpuc+3BGh5d6+anQy7yoMoLtBilMrBWVIdzVyx4 pt5rXRYIvH6++Nn+Et3+zneN0yT8gxeXOH+tAy8LntwNj4rTiaflZ80yS9Rc1sIfGfGP x7tA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248946; x=1709853746; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hwpkv2OuO7YHUplKH98oFVfOa+CFbclxXNu0ysmduyU=; b=qbH1sxmvs0ZzdHz5CYbuvnvJ2Owk7W6i7nqKw3zH2uKB5jghCq0y/IODpyk/kHIiIT sK77qEoTqbzNvV1kQqtpBWGQZLPn5UW91CvJ79FS3MygkubFqNiRgSCPkeNNr8FC2kkr PYKn87Hw8HsEtu+2aSJsRyd0rbFqq1TLG7r9xpBUnKYy4GxQy91Agb2SjYVAXcSrtwHo gFNhaTwc4lrIqqC+QZpempoNsAquIDKc88fbZYtokml9WdbrDHishIgI4YkYL1qXBhHR 7AWI8fiET8Ekp/P2vUCuddfIfXDDY5wk/NUJT/cQDI+yIE00+J8vUKKYpWe7hhzSo/jb 3mHQ== X-Forwarded-Encrypted: i=1; AJvYcCU3eU1BPASINsv1M8sCX5e5Z+lMwz/OVN9M6LKVN5sW/nZPvBRBRlg043l7K1afhgAnXz9DAmpEkpjrwVDmTcG1MJZlBz2zm+kT/Fbr45z1 X-Gm-Message-State: AOJu0YwNZNXTU7liJGaqe3ak681MyQD1Xbm8WgnT3hWpldBc5l3r8ELv guhSTJjMy88mFsYYVqkEWrZ5qnKxBxh0dwupnULdfpJ7eXmXMiS0r8atueu+ak8= X-Google-Smtp-Source: AGHT+IHmmt9lzZLUCIFAbm5GqyPJaePE9QFXJ4x/3NAo+fPZmowQvwtei5P759brqGkqtPXCOnb8SQ== X-Received: by 2002:a05:6808:17a7:b0:3c1:af49:222b with SMTP id bg39-20020a05680817a700b003c1af49222bmr47209oib.37.1709248945851; Thu, 29 Feb 2024 15:22:25 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:25 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 12/13] riscv: mm: Preserve global TLB entries when switching contexts Date: Thu, 29 Feb 2024 15:21:53 -0800 Message-ID: <20240229232211.161961-13-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_152228_390840_FCC86B34 X-CRM114-Status: GOOD ( 14.18 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org If the CPU does not support multiple ASIDs, all MM contexts use ASID 0. In this case, it is still beneficial to flush the TLB by ASID, as the single-ASID variant of the sfence.vma instruction preserves TLB entries for global (kernel) pages. This optimization is recommended by the RISC-V privileged specification: If the implementation does not provide ASIDs, or software chooses to always use ASID 0, then after every satp write, software should execute SFENCE.VMA with rs1=x0. In the common case that no global translations have been modified, rs2 should be set to a register other than x0 but which contains the value zero, so that global translations are not flushed. It is not possible to apply this optimization when using the ASID allocator, because that code must flush the TLB for all ASIDs at once when incrementing the version number. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/mm/context.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 0bf6d0070a14..60cb0b82240e 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -200,7 +200,7 @@ static void set_mm_noasid(struct mm_struct *mm) { /* Switch the page table and blindly nuke entire local TLB */ csr_write(CSR_SATP, virt_to_pfn(mm->pgd) | satp_mode); - local_flush_tlb_all(); + local_flush_tlb_all_asid(0); } static inline void set_mm(struct mm_struct *prev, From patchwork Thu Feb 29 23:21:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 570A7C48BF6 for ; Thu, 29 Feb 2024 23:23:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=EHoe5KNagYjvSMwd5Sn2bkKbeLXl8DapvRyV1nQvDbY=; b=PfXGFHLmQuJk0Q lgCghzCEpHqXqd2zsLIebJA90h0fiaKT/cb7AXl43i+jpdxJgq3DX9qBtio0E7gErvbNbBgfH8LkY svqSbLHBOoW0IEY1//X5e9wmGosV+iFn6UCpi35Ow5yaAG5nwNlHBegHJxYuooTSlIhkY1+ckLBYS nDcfH3jNbyD2V7a9uyND1DS6l+CE9aP17iGTDSYx6f+W92f9Cp3Q62aCKIj0m9t/wvq6MAmPni0M/ vjKSU97NaMLmow3IERHrwD12LfmkEemt/evedPU1N1u8mCSXeTdlKdYkN608U8BhtOtR27AiSYPl6 VFYCkUf31dZvv+f9zphQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpkJ-0000000Fagq-0khX; Thu, 29 Feb 2024 23:23:23 +0000 Received: from mail-pf1-x433.google.com ([2607:f8b0:4864:20::433]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfpjQ-0000000Fa0b-10pY for linux-riscv@lists.infradead.org; Thu, 29 Feb 2024 23:22:30 +0000 Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-6e5a232fe80so701515b3a.0 for ; Thu, 29 Feb 2024 15:22:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248947; x=1709853747; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cJvv9umwOkXOwc0iOvLk+NQYD7Mm5kACvwQaK1YGZP0=; b=IppuWwflR69UMGvB/EuLCzKdSDpSrFMCFnJ1Q1RoB2WRv/sxezyHOkG6ALdIovxYcP OZMmattawW6jigP5yoCU8HeyJ/ki4dUAey7Gh3e+sxS1HUD3+W0Bydq632GxXqrmTVG5 Jdw8Fl8RxullLDdaYnew+gpJF8JEFF+ycumsFIx4RoATQDn7pXSr8ofdsJaaGqaiEGeA RvSgEZ3bAs2+6cbheNwC3LqfqtGHbQrmHJVbwWm5GxkvWOsIZP/7opqxEM2vNeMQ0wTm b4cEJgJbjS3RBQK6rzNieef5iq/XEu6GtidqApsiN9z61lofPK7yuMP2ngMwrbEscPYj mkDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248947; x=1709853747; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cJvv9umwOkXOwc0iOvLk+NQYD7Mm5kACvwQaK1YGZP0=; b=qm2Jj4HjYu0f4Xqsg/mwzeMjnhEyjcebaHHB7R6vNSbCZ4oEkQt68/NyQ35I8sqar8 iP7bq6ZDh/kpsxDDg60mPXcWMxWtG6qiYDpfG5tn5g5StoN3hMGJdokEAaFxHslUkYv6 LuRt9bKuyP7URrKlL6Y6yRyKYDn08PQBDdxf9dj214BtpzlvX14C7bNKyv2rxx3dcPe3 eOwr4eu0CbtxxCmfKVpZUFMVDNy/xIoZ7HydIHcoL09nvc3YIrLJS9eh/byGnmnZkld3 YT3AaoSWx8K9RUiKIm1dje9YVhOpUDsicQ9U8EyOzs0EeDu5q+HqORLE60fZdFokoYuP omoQ== X-Forwarded-Encrypted: i=1; AJvYcCVaajGZDYR/0ZeXBl8T6YGUu+ZOlDky3wxnxx11R/QesVYOYdGMDub55vtFuN5v9SHJJAA05KlOL4SNQvcsTjHqURduwDcowpdL6NJQ/Uo3 X-Gm-Message-State: AOJu0Yy9pOpMSC+gdBbDO+hn5tkYGayGpZzIgtQ01mwNOFFopDuTwe0Y AW0+fvp5jH9Q7mMzaqNSjYuYNd4g36qCNxWo6+dCJni7vzMGWS0wzRm3seKUiJgeQQ/7kJ5kCfH k X-Google-Smtp-Source: AGHT+IG000bMn11OdL3LH1zGovTy9Qg+UUgYE8CMrW4h6bNv2nzitQBp1dzsEHx4shrcmNE8zaAVpw== X-Received: by 2002:a05:6a00:27a3:b0:6e5:3dea:bf51 with SMTP id bd35-20020a056a0027a300b006e53deabf51mr562992pfb.7.1709248946831; Thu, 29 Feb 2024 15:22:26 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:26 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 13/13] riscv: mm: Always use an ASID to flush mm contexts Date: Thu, 29 Feb 2024 15:21:54 -0800 Message-ID: <20240229232211.161961-14-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_152228_629864_AFA6BC04 X-CRM114-Status: GOOD ( 10.84 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Even if multiple ASIDs are not supported, using the single-ASID variant of the sfence.vma instruction preserves TLB entries for global (kernel) pages. So it is always more efficient to use the single-ASID code path. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- Changes in v5: - Leave use_asid_allocator declared in asm/mmu_context.h Changes in v4: - There is now only one copy of __flush_tlb_range() Changes in v2: - Update both copies of __flush_tlb_range() arch/riscv/mm/tlbflush.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index e194e14e5b2b..5b473588a985 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -108,8 +108,7 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, static inline unsigned long get_mm_asid(struct mm_struct *mm) { - return static_branch_unlikely(&use_asid_allocator) ? - cntx2asid(atomic_long_read(&mm->context.id)) : FLUSH_TLB_NO_ASID; + return cntx2asid(atomic_long_read(&mm->context.id)); } void flush_tlb_mm(struct mm_struct *mm)