From patchwork Thu Feb 29 23:21:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577706 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29205C48BF6 for ; Thu, 29 Feb 2024 23:22:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AAA066B007B; Thu, 29 Feb 2024 18:22:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A0B516B007D; Thu, 29 Feb 2024 18:22:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D29E6B0093; Thu, 29 Feb 2024 18:22:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7E52E6B007B for ; Thu, 29 Feb 2024 18:22:17 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 4B23AA0146 for ; Thu, 29 Feb 2024 23:22:17 +0000 (UTC) X-FDA: 81846417114.10.3BA47DE Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by imf02.hostedemail.com (Postfix) with ESMTP id 5A7B280013 for ; Thu, 29 Feb 2024 23:22:15 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=PD7l2PYV; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf02.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709248935; a=rsa-sha256; cv=none; b=ydGvNHII7AZeFGcOAcwXYnUM/sxl0arTvZ+4uGupoLqF6OGqLxCosPz9ZO+K98OY8b2vAk fwYwETh3nurwZtIihIEkKekWZ3RL7Yf3YcRCanQg9v/p1qrtqY3laWeBOn1Zw3RlnYoNIA V5nEyYCy7pMNZtW189xp1yNjC+p46N0= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=PD7l2PYV; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf02.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709248935; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=O37VWqG4iCvkiZDgqVtP8oElF9rd1vMtEBQWx77Nmx0=; b=vw3QqKvBzu3aWXgPjrOZ7G2Tqhs5r7NL8gYZOuxroegpLzJP3NzrYDPiJrm2cbXyAs0T67 fWjG2dC1aashucTK7klEFm/AIATRUMYYAE26KWAhEXaXmOzjBGbI6nrFrlWLp8t6wBKf0k wVW8/mZiYvQhnLL8hn47dsP58m9YlWk= Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-6e4d869b019so891361b3a.0 for ; Thu, 29 Feb 2024 15:22:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248934; x=1709853734; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=O37VWqG4iCvkiZDgqVtP8oElF9rd1vMtEBQWx77Nmx0=; b=PD7l2PYVhQ3tViG6/elpYCYHfP/99+Jej+cUZ0exa7T/3q/V9AF0mYtLXoDTgreGtQ zZQ89ilD3Cy1atLNRvvrksCyutMSf6/xKFF6r1k/wriPwWq7wynVFDfkSy4Z/m6vuu6U a4EIGqkBRLxrElOP8MTfPolwBcR9B1fHy4XlPKjR00mvvt0Mkj2frqzz0L1m2hgHgS6n fMMJdaLm86HBegm5gEeBTygsumbfHIEL5iXg6FjfEbIHdZp2NRq36wPLWyOCmVJfmhOc uD18GZoc/uAsRpll9EN1Tm/4vncWFJNFFcBtSXqgEdU62A3fOj/SC3RIQ43+23cb1BV4 BWOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248934; x=1709853734; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=O37VWqG4iCvkiZDgqVtP8oElF9rd1vMtEBQWx77Nmx0=; b=F7Ij477bjX8xampWAkTaQqM69v6/bIqPsLJLEmO6K5oh34+IAKp5D3bQoqWKey0OX4 KyPQCGdtTAKp0OLkukUqH61Z67mxADXP0o8epU6B7YveOsE3C5XnyilveV6Tw2sm2XVb Sfx8hfwiFRo9FkldLbt/RgjCpWJfJWusESbSEfMuOlyFdgIoi2BRBhDuHDs1EcBhGXRK oipgbzW2JScoq58nji1NSI3EUL/RQLruPIEjmvexYPYnURM7LuMJr7PxkH9h2kVf9D0e 7vArQDnJNyiRmcpbEYBC+7TViNR7PC13VCL0AvRwT+2aOflV9QfbQZJq/rhxdULdXFQW 9P1w== X-Forwarded-Encrypted: i=1; AJvYcCV11ooT0IHdMxbYjd03QPHiJ5GF7C0jcxpOwcrbyZ24W+0hb1SVmNAnkJeXlan5YwkaqZoSC2inY+WyuLEnBW/I1Ro= X-Gm-Message-State: AOJu0YzlLVlf8gXFfPP+m6gX3n0NoNrI1eEycljwf/gVW2uUhNtmTrPO gEAaJy4Mlm/XOVS81R7Zi7klz3R4bGblESIMfdUyHU5SdzMdTZJmyT0dRHfbZnRMEDP+eGgIGMc n X-Google-Smtp-Source: AGHT+IEXcsW9CGPGmNlysP87xM+KqyMeGQDT/2i2zpW2/RfnrP1VyWuH3lyJldn2H0bz6i0E+Cfqiw== X-Received: by 2002:a05:6a00:2d1c:b0:6e3:fa52:2f26 with SMTP id fa28-20020a056a002d1c00b006e3fa522f26mr168419pfb.8.1709248934208; Thu, 29 Feb 2024 15:22:14 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:13 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 01/13] riscv: Flush the instruction cache during SMP bringup Date: Thu, 29 Feb 2024 15:21:42 -0800 Message-ID: <20240229232211.161961-2-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 5A7B280013 X-Stat-Signature: w1p7ac9uhtn1dku3q7a3y7ccmsc13dh9 X-HE-Tag: 1709248935-975649 X-HE-Meta: U2FsdGVkX1801hhbEiC5/ay3C2ubil9PY9AEOCR4zxo8Gujj4/m6InvYoClFeJP+dqedeNW0RBB9UEiZGRJUGjOZZ2S3EqDZy+zvYFeqvjUG9VK+9p6llZ4z32DU39MHSWjbOzHcxXfqzWfFRSOySsqH536w3gTM0QAyJGCYTAoTqwzfLNYFU/9J+uq6UZuPpsyM0qUw8oKvY5rJvm4sc3xb7CzXDXpfvW4jCLtqy1U0Iz3AqoBE7iDW2OSvyi/gcw6flqLbXsO38KDn8oFoqoIKbQaneU/z3cC8M14NWHrqwse+3ZcMAnRpZUldhUBm9c6xvmGgXTZspIXwgTo72Y/NU1DTfcRbPHsPxVgtY0T+4E0kgA0enULSm48lZShLUECqLYShTRGpBO6cWCgjjDw6xAQgA7qRbqLQjqUCTvq19Z7YPYuoIA2pVGk9WIetEKmt75eFD2RRUfdz1N1GpbkUD5Cp6kAhfU8n6pzth7hUPbkEsSGgc+x2GShb/k6TaFLoCH05w64jBI996REYTVJ3JHUymSPYRlmwkBcKdGcO0TsmUd1XmJzMq92bZGf6RpxnO8SS0eMQalRfFtpb5GFNSNByfC7xAK5SSJyLU2Wwv5Ywca+TKjPGbvPFA6EwU/XbnXgcuaS6azEgOK25Shu4x7+hUl6jiOMdnA8i5O4QTtNbhQrTK/gveMjMzv1xOQzUV+BpoBNO6iuhNXtdFI+1VMYTVN8wQIj2bztjbSYSKh2lnaZLO+lAr6mE2pesyS2b0BWgFw2TZe+hZxv0snANkdTGHkxAkD/CJKp9BMnvJqltruuFdMPiMqnZfG4w2N7xFPNTYEHbocNpmOLbrRm2++n6lFPbPjrjBy27awTFofu14XV/1sdw7CgOu2dZPuoYIFB0em1EZIq8l8/YU0mAEV0cUWxW5tmvrmSInlRxLzh+Dy1p85yDWZmSdJT7duj5LOPfQeC1IeI/w0H DuEVwfZb D6dmvSOvj+iSDDZ4WGbFaqHjFkIrfTRL1lIMAdjRh0QAwokTw7F2wta50FCIqiVuBmUY79dQeIeoYk5upZEIQDBVxfZ+pOYMB2k8wWIelClxKOdPlBPzAouz4sWJGBJvskN9lYMcq8MnxKuf6bN9KN3wo2jlKg2uphDpfuVQWMbAHbC1dD47N+/mHCTuaw/lxzrJnTkY9uaVdji6qUD/khX79TsuPd76ER7+c8ypFEtmnwDHBxHbLbMxbD/Vehls1aGyFcymyzPXefcelOazQTKVn4DANtzC2SJvjfQI3iGyJPQgMd/QBhyKI799lpWqVIp9VhN8TU3Zu+av1snfavJbgEvdNcnFBL59u3H5iVA6wQk6ECSBPdQowNG/8eCYpC87r/Ph2NOCYspbEe1uAV8J+gf0jOoR/RNqsrAqc5hT+xDDc71Hl5UkGdicpIOPQknvAjppB58igOGw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Instruction cache flush IPIs are sent only to CPUs in cpu_online_mask, so they will not target a CPU until it calls set_cpu_online() earlier in smp_callin(). As a result, if instruction memory is modified between the CPU coming out of reset and that point, then its instruction cache may contain stale data. Therefore, the instruction cache must be flushed after the set_cpu_online() synchronization point. Fixes: 08f051eda33b ("RISC-V: Flush I$ when making a dirty page executable") Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v4) Changes in v4: - New patch for v4 arch/riscv/kernel/smpboot.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c index cfbe4b840d42..e1b612f37dd9 100644 --- a/arch/riscv/kernel/smpboot.c +++ b/arch/riscv/kernel/smpboot.c @@ -26,7 +26,7 @@ #include #include -#include +#include #include #include #include @@ -238,9 +238,10 @@ asmlinkage __visible void smp_callin(void) riscv_user_isa_enable(); /* - * Remote TLB flushes are ignored while the CPU is offline, so emit - * a local TLB flush right now just in case. + * Remote cache and TLB flushes are ignored while the CPU is offline, + * so flush them both right now just in case. */ + local_flush_icache_all(); local_flush_tlb_all(); complete(&cpu_running); /* From patchwork Thu Feb 29 23:21:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E264BC54E49 for ; Thu, 29 Feb 2024 23:22:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A38406B0093; Thu, 29 Feb 2024 18:22:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 96DB06B0096; Thu, 29 Feb 2024 18:22:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7258C6B0098; Thu, 29 Feb 2024 18:22:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5243C6B0093 for ; Thu, 29 Feb 2024 18:22:18 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1F1B91C1034 for ; Thu, 29 Feb 2024 23:22:18 +0000 (UTC) X-FDA: 81846417156.04.D9DB16C Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by imf11.hostedemail.com (Postfix) with ESMTP id 533194001D for ; Thu, 29 Feb 2024 23:22:16 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=GjHBX7vh; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf11.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709248936; a=rsa-sha256; cv=none; b=L7MKOFJa7Cpbu2WUOhFZh3F/ti5ZNzB7VoD6MgaKOVO+e/kDt+eMpKEbptjfvslqp0Rybv OXI7+xhBTyUfPFMaKh9aqYUV3cXwpyERqPLIvHDATAc/cOvgza83yESFZT8UfTAjW/PAw9 SUssHlQ9fFpV1vwRt3gRm7IxKEm6KzM= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=GjHBX7vh; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf11.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709248936; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XEa0bZw6X2PF+fvXkyrnyWuZOURSNgi5dGIyeM9k7ic=; b=R0EJWyIAHxFPNWFjZ3gJd4ATDIeXVSJEiynvES0HsfC0vK6ird6G77boCe0pmnWWUnYEJ9 +QQlQMaioQlgWAyNzfMBbX6JOZ168M8E3RAxtp58SfzOr1ZrlFBRxh8Nl7bGHdhMZAeMZc 8PSUjatYtnrE3M+iUjuGtX4H6XK55Mg= Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-6e5a232fe80so701422b3a.0 for ; Thu, 29 Feb 2024 15:22:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248935; x=1709853735; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XEa0bZw6X2PF+fvXkyrnyWuZOURSNgi5dGIyeM9k7ic=; b=GjHBX7vhDQmk3uieuGL7DUacpJPTVEn2KmhcEq1I8CRE7TWwRqIkAJHwFpFDp6771t YxmJeG/obOn4oH+I7O7bh1ESlFZYXDqzN85XXeJR0rSTVs973bsAvHvQzpVqacd8LpK+ 9a8ZhSltc9AhEFWjHW0N7cvdlx+6wa9yaglYATKFtSghVRyRi6mX8DqBtn+UBri9YQ8O 7UjfQvSoTO1RKIFxYkffV54g4xOkH87XkRqBr+Rg0GRhcEuySW8YVXdcmra01MeLw40/ lDwkjtDBJ92QRQS+Tbjg7DeaQy+rep0wrIJYHdbpc7UuJVpNZ8IHTV5G1g1R98ckVLcg VKQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248935; x=1709853735; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XEa0bZw6X2PF+fvXkyrnyWuZOURSNgi5dGIyeM9k7ic=; b=QmTqIMk83ngvLhQFWttNbzblGjNClkrh5Um+p7yqwQYCQ27llZqPtNgmqvESyMM7i4 UjQl5cx3B2RpnQE4k9KMkDFyFODKxAWbTgFLRIPinBGUZ0Z+4U8ovxIlUbIBE4rnP/jb Y4bScmPuFvugrYjWZqcxMCsQi8j/KtLrC8C38ctnQ9b3A8RU27nBJ7qenzhbJKcGzhls +oWkUQWWy5NVZCUAjyfV0BiEz7zTvHpYBTAhuHGzstzVAd02kiSN7aO4nRV0+FoZqFCm axgehkpbDI6dq+brH29DM95LXqjUE5XkRO+okGHt2NfhUyinPjLlGSNHjLHanRwp4N+0 BIvQ== X-Forwarded-Encrypted: i=1; AJvYcCWLETTs9UXYgqGqwogv0j3q/fCp33jQYhlPfCQJ67FYGhNu0AfD4Dvm78UfOdvTa3hBAfCImjkHikavBJkLikDMCWE= X-Gm-Message-State: AOJu0YwfvabG3qDz+FBB+LwMN033craegafnbHzg4y/ZOEfGJuXRgWoI MQfp1RQnHZpf20EMCOUyf0kbwC44WqGCy+Rm5QknVHzlmC1PzUUNer1ZW6LFZOw= X-Google-Smtp-Source: AGHT+IFdgcTr06rQ4rVKFKQwHiCzNjuzvF9/zHzBzMpbTCozBAylfcba6wC7aAVCdn9otyCllPR9tg== X-Received: by 2002:a05:6a00:2293:b0:6e5:599a:5252 with SMTP id f19-20020a056a00229300b006e5599a5252mr606918pfe.15.1709248935218; Thu, 29 Feb 2024 15:22:15 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:14 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 02/13] riscv: Factor out page table TLB synchronization Date: Thu, 29 Feb 2024 15:21:43 -0800 Message-ID: <20240229232211.161961-3-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 533194001D X-Stat-Signature: ttdy1d8hg4cnsus8o3rwkyrnwg4npfc3 X-HE-Tag: 1709248936-781377 X-HE-Meta: U2FsdGVkX1/irKsFpJnQoEYvzShGqH+fNhFIJKCbXSKU7IAuxnM3k2XIQCUoyEe8ARKav7ooZkPB4GvpuoAIrefr84Fro7wBs7j7pERX4Ni+Baqo+M46HGys9URkBFPOnjvrQr9FgV80pEmocU3X9NtNohcEURm7gokGjnO0wjZa5KHX0xlER7nJi/NFAvIYJuQHjoUPVewGMnHB/a+BnsxaTKeFaJmYAKkcDchbP6T2UgEtw0RVR07KPtJCJ2j76kpqwPTIZLEnuNsz/I/tvtSWtycIzHv1W561fCwcQzsy1EWb+Bqjg88EHId1EnwD1mYuEhPfj3E1OnQFy2LEplxCnJamnhNGOLFOm+fktI3Mo79TEPvc1Ara0w4u6c9YHdcXW7Px1pyHd67ulQBb6LWGxvl9ZqK6GqJlDkEyg2urVXbOu8vXsl8K/k35+t8gjBx0qW2IQ+9P4BmNGUTzL3z5zsq+FoJn88tukDg6u5IQAXPWpyScHEZrFtf7qVj/eNmNkIOuu80ZFJEmxqTAg10gWKSaAsEp3JClzrRfhEcYCpEHdqmIJcsdP2dhodBnc8+b5QOWgwsTG1PBiKkVi/M7+skwFGGyLvvQAVe0NqOrnCWvfw+520FvESRED8lIXnFcAOB0I+JOiLGxEZguAIntX6qkb/YuPEN2O5qSCbwTvRAJKgypTArK5OywZqmKCGBlMKHK6wWVN1M0iShJ8Ka8KDJM1nLzxqM6c4kzBRwpPMsH5EePt+vlAbrr0VxKn3/3WvDBJN6ElaQi/OCV85gpdVJPBRVQ/tZsjvsa4wRQTUJxi2FuYYrWFTmneBItf9WeBc4owV7Khpahmsysoz7uW4UkzPgWop341iPAnvnbnZaBEw9PDuFz2WUyl7IamHpwvuEmMDY6D4p52DNJrHVp5WyxfV43h/MF2Vx7/VdxX7zzVOJFzV51M73N/NyvWwpgbDrQ2iVPdPRqm8G FGocI7Pp hiyqqi7eUX2S+kWG3kIcd5n6naywaTYbglj6/miH/ADPnC+OyRpqLYWSWsLVhYxZuR0tqT88c0GPc7QtOZu6ADggCjtOi4aWALUOIr+munG6PD6JALkybOYtVWw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000230, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The logic is the same for all page table levels. See commit 69be3fb111e7 ("riscv: enable MMU_GATHER_RCU_TABLE_FREE for SMP && MMU"). Signed-off-by: Samuel Holland --- Changes in v5: - New patch for v5 arch/riscv/include/asm/pgalloc.h | 31 +++++++++++++------------------ 1 file changed, 13 insertions(+), 18 deletions(-) diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h index deaf971253a2..87468f67951a 100644 --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -88,6 +88,14 @@ static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr) return NULL; } +static inline void riscv_tlb_remove_ptdesc(struct mmu_gather *tlb, void *pt) +{ + if (riscv_use_ipi_for_rfence()) + tlb_remove_page_ptdesc(tlb, pt); + else + tlb_remove_ptdesc(tlb, pt); +} + #define pud_free pud_free static inline void pud_free(struct mm_struct *mm, pud_t *pud) { @@ -102,10 +110,7 @@ static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pud, struct ptdesc *ptdesc = virt_to_ptdesc(pud); pagetable_pud_dtor(ptdesc); - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, ptdesc); - else - tlb_remove_ptdesc(tlb, ptdesc); + riscv_tlb_remove_ptdesc(tlb, ptdesc); } } @@ -139,12 +144,8 @@ static inline void p4d_free(struct mm_struct *mm, p4d_t *p4d) static inline void __p4d_free_tlb(struct mmu_gather *tlb, p4d_t *p4d, unsigned long addr) { - if (pgtable_l5_enabled) { - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, virt_to_ptdesc(p4d)); - else - tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d)); - } + if (pgtable_l5_enabled) + riscv_tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d)); } #endif /* __PAGETABLE_PMD_FOLDED */ @@ -176,10 +177,7 @@ static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, struct ptdesc *ptdesc = virt_to_ptdesc(pmd); pagetable_pmd_dtor(ptdesc); - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, ptdesc); - else - tlb_remove_ptdesc(tlb, ptdesc); + riscv_tlb_remove_ptdesc(tlb, ptdesc); } #endif /* __PAGETABLE_PMD_FOLDED */ @@ -190,10 +188,7 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, struct ptdesc *ptdesc = page_ptdesc(pte); pagetable_pte_dtor(ptdesc); - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, ptdesc); - else - tlb_remove_ptdesc(tlb, ptdesc); + riscv_tlb_remove_ptdesc(tlb, ptdesc); } #endif /* CONFIG_MMU */ From patchwork Thu Feb 29 23:21:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577708 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B2D3C54798 for ; Thu, 29 Feb 2024 23:22:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 889BB6B0099; Thu, 29 Feb 2024 18:22:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 838146B009B; Thu, 29 Feb 2024 18:22:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 661A16B009C; Thu, 29 Feb 2024 18:22:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 547D96B0099 for ; Thu, 29 Feb 2024 18:22:19 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 2ADA0C03B6 for ; Thu, 29 Feb 2024 23:22:19 +0000 (UTC) X-FDA: 81846417198.27.06C585A Received: from mail-il1-f181.google.com (mail-il1-f181.google.com [209.85.166.181]) by imf15.hostedemail.com (Postfix) with ESMTP id 365A6A0009 for ; Thu, 29 Feb 2024 23:22:17 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=QjMNTgSR; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf15.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.166.181 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709248937; a=rsa-sha256; cv=none; b=tZi7uDP6JCCgIuyJKk9ZYa8+9J9WOZsNoE3L+O+0NHnHAIYRLVdT6thA9mx8OAcQs0imyq wkcrydV2NrTXpWe9y8AtAcAsknNKJ0Bg6QfR/Ax6O42a1Qgz96U1jA8DYvwe1njVbjhbeo +vue/1R7J0sj1CcaSPAAPqbnMBkKsAY= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=QjMNTgSR; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf15.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.166.181 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709248937; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nL3CU8jW9vVfZeENVgIQs2Z0lmYtCUXWU5kIlhxMA+M=; b=c6xAj19bbjMJEvv8ikZHpZd5/MefokTkNwoLdJOw1psBCV/PYGVXPzZCWmW8LZ71Pw8tM4 ptGUMZIPv4yuOfUuItsa69XpZs2ha1tgDm/cifp7YcpSRdR3KE39ewnAwfi3Kb3yUrGdV/ I/g+LJ4p0MyJEhWqmJbRxqlXaUIDWwc= Received: by mail-il1-f181.google.com with SMTP id e9e14a558f8ab-3651c1e1022so7353665ab.0 for ; Thu, 29 Feb 2024 15:22:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248936; x=1709853736; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nL3CU8jW9vVfZeENVgIQs2Z0lmYtCUXWU5kIlhxMA+M=; b=QjMNTgSRwARh/B/NjAem7Lv33vyp9lvCU1+4XjhGyfR4IMjNfAocmqCBsrUhFr8NFH 7Ztfs6mPHE16Jf6WzYBpmOfRVWXtSS5zH8SlxtW0OjYlGVKwKfRdjV4iJDkRvc+n5K+Y XLv0CnuU7xXKHZOvcnhieT7mJ2Wu8D6hqDsVwYCc6obTIBn03KuRrZgRrUjfxBFHYoYJ EoYSb2GA54ACQVd+mKi/5ECFgK3UMjs2henoVmBJENvILf+jXu06meV02Og9wpTyvboP T0cVGSoXMnmkxmoakoY+LyHHTlK0rXPvhmSOo1aFFwtyspDJEHIY/M5+J1FmSecuK4FV OxHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248936; x=1709853736; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nL3CU8jW9vVfZeENVgIQs2Z0lmYtCUXWU5kIlhxMA+M=; b=uAunNNStMMzi1z47T3ElWPqx7wOUK1oGTLt8UYJF9afWYbLupL6NpBq36Xt6q+yPE2 sDaGmbw0YbHJEx9tLIv1vXfXTGFMPUBi2E59Xfvpu3HgC0MEp2Rc5VaF3lH5vMCj+nkq cLeWqDPhM0dDPlaGQRA0RjIi2y7bnTl38vpyuiZvoHZ2DSn7KR3sVbvj8oQ4YBnuSqyf qZGAluFLa50QherfywRXDDBGML+icLqN/QnIFhYxWSZkLzsVZsNWeaRADBB4ZqbEZ2Od ymjsx5k7DGhFU1C34248lKULeyD1RGo0lMbh5M/icVS6YfD4wTkPBBhajDnQD0OtmDKz bmSw== X-Forwarded-Encrypted: i=1; AJvYcCUCcZgUs3KXSn5k0VW9CDu6V6Vvcu7uX9M311RIWiOgX5K5/Ups9afyqO80n4NZLdtbcyTcVs8VOejGAFmGjz6mb9Y= X-Gm-Message-State: AOJu0Yw65vTlTgny9ZOATqGkZpMlRKeBsKZKDKlAsoiTpvFmL3YQCd/6 1xTsT7S5nPhL43Ko9n0BKxov8Z/XzkzuFpLZKzgYPBHYMLAKUNPZP4Q+XCw5UDw= X-Google-Smtp-Source: AGHT+IEuQOuQ50R8qoZP0RbIoJTNSdVCKrs4lRWLzJ+znFdHc+eSLoYj/Ypw4BmFNiQ++vK2RkPm8w== X-Received: by 2002:a05:6e02:1b8f:b0:365:ae26:5827 with SMTP id h15-20020a056e021b8f00b00365ae265827mr159548ili.5.1709248936255; Thu, 29 Feb 2024 15:22:16 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:15 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 03/13] riscv: Use IPIs for remote cache/TLB flushes by default Date: Thu, 29 Feb 2024 15:21:44 -0800 Message-ID: <20240229232211.161961-4-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 365A6A0009 X-Stat-Signature: 5pshztxuu6tut91ci4dh7ygkxteknouc X-HE-Tag: 1709248937-465014 X-HE-Meta: U2FsdGVkX1+0iFrkvODnhGB8+BdWAHZQc05r4rK/z7yVH376ixn5DywRUCTbvURBUoStTGrPJ2zc1cINVQTjGsDN8MJVwWGkYSyo/mJa6Tb5GmgxuZB4JKUSIIp4Eux8DyDoqd8Bxr6luxDiSHaSlVrvQmUPkPcixnDK12ZUM0PKA5q0HDWc33x1fZkhiL2C+1TFo1NklXcWEUbgtPjMsd3Cu4NZxq+I1NnGKrD+cYMDP2r+oMHt3ddXt/UOBx1wqDTezv5S+qjP8XvHo4Y3oc/aVMa57uUH1hsMGTstCT7owroJSQDsm4B6ryOIYKTmvXhMoIxO06vDPZXg8DIC1VbcHfBHW8TbnfENaijarCm05PI78kWjaCFWASeT5MN6lqvRZqlVqcJDtdpmaJdIjquNhJUD3zIay437S8cUFT4JknDfo+MtT7NaImmywGkq0RbGGQufK9ZbCMsvz6FquDXC2VjLJtDbMtg7aFQwUbkK/QiI6p0QGY3o4rmPrB+WFZjJYQwNj9ukhaTaTrbbuMKjFyGslE5qAMMKGgRhrSl7vC1R/VXXvBx95zPBwrQatMt6ZIhd5t/0NHsjJpuloco9vd04jgD0Rx3fsX/Cl3szrgSHM/0FlaJuL+uQPW5TQqJGX6FG+cB9Q3O2Xk026inADzeSan/OYOTTHQ8x3SLYDKRQuwc0kK7F6dPLJDv2LWGSqRNc+66HwTWlUIMUteXlHu1w6DnzQkRGIyla8jeWDADDRNExfneuA6GNmxtuZsVRLtTGHhsVMmYl0vfNISMYGqsCByJCgUYGcbrUFFmacBNRuJwg6R+sCCfXUt52uwdIypg30csOOM/F4DNZoVM4WkzViz8Af2U8IeFakLkKi8rtJQ3YPqY5Zj03hlS8rDz15u+MOkXZCaSwABWG2P5lpl2ka8XaCjll1VzFnOJjzpo9JJfLC+H2yKohWJjeo2L+Lb7y7aYaWxtxI1p auk00okl VpUXd85MSa0uKlAwR4vtO8MEJ9CXVTwNDnUxOyonJcd5/yz269WBLViAKA2wxd/6prm6evR4uCkOLQgTERhQy7rZ2mktaEBHyK0philbobHt7mZNe0wEngKUs0bSRj+YH/n5K7WFGCXl9tbXjHDFL3GvBgHfWWacxBKhL4iqII1rgPN80eIQEDAqJqutyX3wddjqplt0p+4ojW5tiNDXy83jawyqJiOm644LfKgsaKZE6QHio908rFuVb7TjMXSSiqHZDclgeZ9gycN79Ba9oa3y7ktrp63eq+O3iGN8Fc6oUJF4wEH9V1L/2kqhZEHU6aLdWLEBQKbxwiTZbHVpEVcf/wZH/+zfkvD+XTkF6D5mtxDylTkVuqA9A7VvjdM8ld3e7B013VbBxEQE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: An IPI backend is always required in an SMP configuration, but an SBI implementation is not. For example, SBI will be unavailable when the kernel runs in M mode. Generally, IPIs are assumed to be faster than SBI calls due to the SBI context switch overhead. However, when SBI is used as the IPI backend, then the context switch cost must be paid anyway, and performing the cache/TLB flush directly in the SBI implementation is more efficient than inserting an interrupt to the kernel. This is the only scenario where riscv_ipi_set_virq_range()'s use_for_rfence parameter is false. Thus, it makes sense for remote fences to use IPIs by default, and make the SBI remote fence extension the special case. sbi_ipi_init() already checks riscv_ipi_have_virq_range(), so it only calls riscv_ipi_set_virq_range() when no other IPI device is available. So we can move the static key and drop the use_for_rfence parameter. Furthermore, the static branch only makes sense when CONFIG_RISCV_SBI is enabled. Optherwise, IPIs must be used. Add a fallback definition of riscv_use_sbi_for_rfence() which handles this case and removes the need to check CONFIG_RISCV_SBI elsewhere, such as in cacheflush.c. Signed-off-by: Samuel Holland Reviewed-by: Anup Patel --- Changes in v5: - Also switch to riscv_use_sbi_for_rfence() in asm/pgalloc.h Changes in v4: - New patch for v4 arch/riscv/include/asm/pgalloc.h | 7 ++++--- arch/riscv/include/asm/sbi.h | 4 ++++ arch/riscv/include/asm/smp.h | 15 ++------------- arch/riscv/kernel/sbi-ipi.c | 11 ++++++++++- arch/riscv/kernel/smp.c | 11 +---------- arch/riscv/mm/cacheflush.c | 5 ++--- arch/riscv/mm/tlbflush.c | 31 ++++++++++++++----------------- drivers/clocksource/timer-clint.c | 2 +- 8 files changed, 38 insertions(+), 48 deletions(-) diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h index 87468f67951a..6578054977ef 100644 --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -8,6 +8,7 @@ #define _ASM_RISCV_PGALLOC_H #include +#include #include #ifdef CONFIG_MMU @@ -90,10 +91,10 @@ static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr) static inline void riscv_tlb_remove_ptdesc(struct mmu_gather *tlb, void *pt) { - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, pt); - else + if (riscv_use_sbi_for_rfence()) tlb_remove_ptdesc(tlb, pt); + else + tlb_remove_page_ptdesc(tlb, pt); } #define pud_free pud_free diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h index 6e68f8dff76b..ea84392ca9d7 100644 --- a/arch/riscv/include/asm/sbi.h +++ b/arch/riscv/include/asm/sbi.h @@ -375,8 +375,12 @@ unsigned long riscv_cached_marchid(unsigned int cpu_id); unsigned long riscv_cached_mimpid(unsigned int cpu_id); #if IS_ENABLED(CONFIG_SMP) && IS_ENABLED(CONFIG_RISCV_SBI) +DECLARE_STATIC_KEY_FALSE(riscv_sbi_for_rfence); +#define riscv_use_sbi_for_rfence() \ + static_branch_unlikely(&riscv_sbi_for_rfence) void sbi_ipi_init(void); #else +static inline bool riscv_use_sbi_for_rfence(void) { return false; } static inline void sbi_ipi_init(void) { } #endif diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h index 0d555847cde6..7ac80e9f2288 100644 --- a/arch/riscv/include/asm/smp.h +++ b/arch/riscv/include/asm/smp.h @@ -49,12 +49,7 @@ void riscv_ipi_disable(void); bool riscv_ipi_have_virq_range(void); /* Set the IPI interrupt numbers for arch (called by irqchip drivers) */ -void riscv_ipi_set_virq_range(int virq, int nr, bool use_for_rfence); - -/* Check if we can use IPIs for remote FENCEs */ -DECLARE_STATIC_KEY_FALSE(riscv_ipi_for_rfence); -#define riscv_use_ipi_for_rfence() \ - static_branch_unlikely(&riscv_ipi_for_rfence) +void riscv_ipi_set_virq_range(int virq, int nr); /* Check other CPUs stop or not */ bool smp_crash_stop_failed(void); @@ -104,16 +99,10 @@ static inline bool riscv_ipi_have_virq_range(void) return false; } -static inline void riscv_ipi_set_virq_range(int virq, int nr, - bool use_for_rfence) +static inline void riscv_ipi_set_virq_range(int virq, int nr) { } -static inline bool riscv_use_ipi_for_rfence(void) -{ - return false; -} - #endif /* CONFIG_SMP */ #if defined(CONFIG_HOTPLUG_CPU) && (CONFIG_SMP) diff --git a/arch/riscv/kernel/sbi-ipi.c b/arch/riscv/kernel/sbi-ipi.c index a4559695ce62..1026e22955cc 100644 --- a/arch/riscv/kernel/sbi-ipi.c +++ b/arch/riscv/kernel/sbi-ipi.c @@ -13,6 +13,9 @@ #include #include +DEFINE_STATIC_KEY_FALSE(riscv_sbi_for_rfence); +EXPORT_SYMBOL_GPL(riscv_sbi_for_rfence); + static int sbi_ipi_virq; static void sbi_ipi_handle(struct irq_desc *desc) @@ -72,6 +75,12 @@ void __init sbi_ipi_init(void) "irqchip/sbi-ipi:starting", sbi_ipi_starting_cpu, NULL); - riscv_ipi_set_virq_range(virq, BITS_PER_BYTE, false); + riscv_ipi_set_virq_range(virq, BITS_PER_BYTE); pr_info("providing IPIs using SBI IPI extension\n"); + + /* + * Use the SBI remote fence extension to avoid + * the extra context switch needed to handle IPIs. + */ + static_branch_enable(&riscv_sbi_for_rfence); } diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c index 45dd4035416e..8e6eb64459af 100644 --- a/arch/riscv/kernel/smp.c +++ b/arch/riscv/kernel/smp.c @@ -171,10 +171,7 @@ bool riscv_ipi_have_virq_range(void) return (ipi_virq_base) ? true : false; } -DEFINE_STATIC_KEY_FALSE(riscv_ipi_for_rfence); -EXPORT_SYMBOL_GPL(riscv_ipi_for_rfence); - -void riscv_ipi_set_virq_range(int virq, int nr, bool use_for_rfence) +void riscv_ipi_set_virq_range(int virq, int nr) { int i, err; @@ -197,12 +194,6 @@ void riscv_ipi_set_virq_range(int virq, int nr, bool use_for_rfence) /* Enabled IPIs for boot CPU immediately */ riscv_ipi_enable(); - - /* Update RFENCE static key */ - if (use_for_rfence) - static_branch_enable(&riscv_ipi_for_rfence); - else - static_branch_disable(&riscv_ipi_for_rfence); } static const char * const ipi_names[] = { diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c index 55a34f2020a8..47c485bc7df0 100644 --- a/arch/riscv/mm/cacheflush.c +++ b/arch/riscv/mm/cacheflush.c @@ -21,7 +21,7 @@ void flush_icache_all(void) { local_flush_icache_all(); - if (IS_ENABLED(CONFIG_RISCV_SBI) && !riscv_use_ipi_for_rfence()) + if (riscv_use_sbi_for_rfence()) sbi_remote_fence_i(NULL); else on_each_cpu(ipi_remote_fence_i, NULL, 1); @@ -69,8 +69,7 @@ void flush_icache_mm(struct mm_struct *mm, bool local) * with flush_icache_deferred(). */ smp_mb(); - } else if (IS_ENABLED(CONFIG_RISCV_SBI) && - !riscv_use_ipi_for_rfence()) { + } else if (riscv_use_sbi_for_rfence()) { sbi_remote_fence_i(&others); } else { on_each_cpu_mask(&others, ipi_remote_fence_i, NULL, 1); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 8d12b26f5ac3..0373661bd1c4 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -78,10 +78,10 @@ static void __ipi_flush_tlb_all(void *info) void flush_tlb_all(void) { - if (riscv_use_ipi_for_rfence()) - on_each_cpu(__ipi_flush_tlb_all, NULL, 1); - else + if (riscv_use_sbi_for_rfence()) sbi_remote_sfence_vma_asid(NULL, 0, FLUSH_TLB_MAX_SIZE, FLUSH_TLB_NO_ASID); + else + on_each_cpu(__ipi_flush_tlb_all, NULL, 1); } struct flush_tlb_range_data { @@ -102,7 +102,6 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, unsigned long start, unsigned long size, unsigned long stride) { - struct flush_tlb_range_data ftd; bool broadcast; if (cpumask_empty(cmask)) @@ -118,20 +117,18 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, broadcast = true; } - if (broadcast) { - if (riscv_use_ipi_for_rfence()) { - ftd.asid = asid; - ftd.start = start; - ftd.size = size; - ftd.stride = stride; - on_each_cpu_mask(cmask, - __ipi_flush_tlb_range_asid, - &ftd, 1); - } else - sbi_remote_sfence_vma_asid(cmask, - start, size, asid); - } else { + if (!broadcast) { local_flush_tlb_range_asid(start, size, stride, asid); + } else if (riscv_use_sbi_for_rfence()) { + sbi_remote_sfence_vma_asid(cmask, start, size, asid); + } else { + struct flush_tlb_range_data ftd; + + ftd.asid = asid; + ftd.start = start; + ftd.size = size; + ftd.stride = stride; + on_each_cpu_mask(cmask, __ipi_flush_tlb_range_asid, &ftd, 1); } if (cmask != cpu_online_mask) diff --git a/drivers/clocksource/timer-clint.c b/drivers/clocksource/timer-clint.c index 09fd292eb83d..0bdd9d7ec545 100644 --- a/drivers/clocksource/timer-clint.c +++ b/drivers/clocksource/timer-clint.c @@ -251,7 +251,7 @@ static int __init clint_timer_init_dt(struct device_node *np) } irq_set_chained_handler(clint_ipi_irq, clint_ipi_interrupt); - riscv_ipi_set_virq_range(rc, BITS_PER_BYTE, true); + riscv_ipi_set_virq_range(rc, BITS_PER_BYTE); clint_clear_ipi(); #endif From patchwork Thu Feb 29 23:21:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FEC4C54798 for ; Thu, 29 Feb 2024 23:22:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D4306B009B; Thu, 29 Feb 2024 18:22:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 85E036B009C; Thu, 29 Feb 2024 18:22:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D6076B009D; Thu, 29 Feb 2024 18:22:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5DA2A6B009B for ; Thu, 29 Feb 2024 18:22:20 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 0EA3BA15CC for ; Thu, 29 Feb 2024 23:22:20 +0000 (UTC) X-FDA: 81846417240.20.8D71317 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf07.hostedemail.com (Postfix) with ESMTP id 664E840006 for ; Thu, 29 Feb 2024 23:22:18 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=TUnIFIhB; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf07.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709248938; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1/3iO4NfkALka4Buc8Img0QDYdUTnf5/4SNG0fcN5rQ=; b=SKGgge5U5sSIXu5gVRDK6RbvhFP/C7jHeStPar6ICNjE67Y5/5FZYa7j17JXIIGr2cg2q4 tG6Tu9aY1Mx+s27Cdwq6wEeJ2pSsvuaoJix4S+0nhh5/Go1rpBlY/U2OUleFhpWQ2ucAgh NXFgOT+1Il51V2twxw1rC77sSQHcIko= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=TUnIFIhB; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf07.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709248938; a=rsa-sha256; cv=none; b=Y8qFzPwS9/bQ/Ln/j9mT8mmbHBkBxpfZpzA3Feiuz4GbROLiITUJW6EjCsgpRtwwxX44Sw Wn7x7+ts7nOLtbb7NxKJA/vQ4CDFIKjPHlPMbUg/jH6xaAmSMs7eRHxshCPgxiIm6en4Fm QqTILjUomEBYr2WOSTRfhmPX+Wd5tUs= Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-6e46dcd8feaso903331b3a.2 for ; Thu, 29 Feb 2024 15:22:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248937; x=1709853737; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1/3iO4NfkALka4Buc8Img0QDYdUTnf5/4SNG0fcN5rQ=; b=TUnIFIhBW10F4zpw6+FCBFW82zSfHJO4zq/BBuhLIN1OlKrwONZeUqoPvqX8Kf682I eFwLgwjrk9SflZXI67b85FIc7s9E/vdz4IN7LYRvE7/HCoyhjCQ0FyA0D1NxJb296E6U rt/OYwi7x3A8sZpufpzEySko1Wj/pVlcXY/3V9nwIFJ4I7M5R6fip3DDmzUQTD+wfcGV zp1touQn/Jz1JDeBM8r/9nYY+hcorYmiIzrnzUIn0sgGuO7JJn485cHiFY1VeCZ+8XVg MkxcMs6H2ruWFDKIYK7O6ias3VJi00X/FCJsZvPD0tcK4f1iS6jY7mhz75YXyv/bYUVS 4Aww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248937; x=1709853737; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1/3iO4NfkALka4Buc8Img0QDYdUTnf5/4SNG0fcN5rQ=; b=itmdT3WHgiEK5ZbP2TT7jhkxi0FW7CYj4527VFCyK+XakEx6dQoP/1UmHbhpkmilHE Qol/m5Oe5+6FA/U9wlYYq3Yk+VsxkC4Y96PvNsQFH4J/DLOeFbdlb3V1AZmhcSF+xAqR 4vSDPdJXZ5PL0xg7LiBYNj/57jOIIsV/ZR4aAtai/yg32UGLdNXFRsT+O+0CCODFpgqr 5q9E9cOz/CQSmIqXTbNh9K2zJg2y5K0eZY5z6ghDwLZ4Wbt2L6jHkWJnH1cI5MnA9lfY xlZOqHsIG+1R+7tsBJe0HxdOEOIxooBDEqFMX/A7+f0D/cLp3MFRjyWcVvn3jntv8rP1 WxQg== X-Forwarded-Encrypted: i=1; AJvYcCW5JZu7YV2mwRpg3KcSnuuQGhHI7Y7uT0dleXk2Cr3tjj7dMp9d/EFJyh7KxnfLaivvw1De1ii0cbr1jgfXIRMfimg= X-Gm-Message-State: AOJu0YzKKJHq7H0H1D8tm8YW3Xm2INGMFr3/UgfomMRtWtcQQ6HX27GV JFdEh4+TvyWhVpOCFfFl6ZEM3M2Y4JBIiMFYc2SgA2zMzn/KM40pBKi7GnLljuw= X-Google-Smtp-Source: AGHT+IHEWuHXVZ/LoNCuQjq9ikFwqlpYM8ivEnjNJqMVh5lQ2hbDE0NnP51hssPDjMbEUOy86+sdSg== X-Received: by 2002:a05:6a00:14cf:b0:6e5:a996:5201 with SMTP id w15-20020a056a0014cf00b006e5a9965201mr172471pfu.10.1709248937282; Thu, 29 Feb 2024 15:22:17 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:16 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 04/13] riscv: mm: Broadcast kernel TLB flushes only when needed Date: Thu, 29 Feb 2024 15:21:45 -0800 Message-ID: <20240229232211.161961-5-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 664E840006 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: xmaf4ufxwwd48cr5exo1ymbjjr7qqfio X-HE-Tag: 1709248938-941850 X-HE-Meta: U2FsdGVkX19Zt2basboWfXrau6Gj1fctC6qGDmcMs9OWHKHKH/yGyvZMStQbRWg1R91JVZCL2548FG3MvoIoLAbb6eE+4KUw5An34daOzxrDVRCk/apsM8Y8lwP4TiK/0yLuSuybpMl5Lkn+rnAPm5Uqwf9G3pycn8Mlxm/kZQbyVB9wqnPshiUqY0o7Qbj9bFPOPWAREz8UE5Za5cidxP2lIousp4cXA427mMqZ+n6fhJ0EP5HErREU/sTtnx745K+m6DWLUZAt84Bm8Gd70rkeqYlEvxpjMoFdqB7AAgVK+YJVusD5bCYn+wxQPamNxDoLNKd0Pbd9TWi5quvkrynjLA0DglFjd7NQ1Bjl9eVWOj8zwZ9KXZ5inVxnEdMjn+DrX7uZzz/+tY5iM5HEc4FJG4gXQw8y9k0JeBHe8jHHz97Xo3P32n1pCQybsJVIGRtULlnGkcKHP/Xa1rk8IAJEyVAwgKTan2RK1kgR+XQR5YclNO8oHVOHilrLnJorWMJDczuuJOYkJIJflQMdjLOEA9ozpcBR8GBylc5EGL9iENPBcBV1ztIlPpGlAprFCuJwPb10vZmP46yx/uJGwLc/hJAQQgrK79VZjovOw/VnJ3KJH9zlDmknZhYh0T1Mg2zzD0qex4NQeH6tbtECNw3SWd6ZLQsBc3ZO50WLpFzV1nDXblNLkXxchxcUDHVRBCBAR+vafnxSp1WjAx6BTQ20FWWP3DGfneDS6fMdKMAUycCnZXgoqoFRtdWIFSzu1rrQk57uZV0+MFhlBFZYAw4UGemCGuPAyWMIyEZJmzNGl5OrfM1cjVVy+5Xz9u5DUqXha3Nd+SM2d33efsBFZlAoRIstPv07zUt5f7KFUk6yeep526d5hDqVJdn/HIe4wq8VfS5wQt9TbRks5T6ceeZqvYzDH4acXKZuz18HG0qvT8GPQvoa69xH/A2Osv6M7SIquz9FxhJusdPQUqp v+Q6fEZL WOEIUS9OPeYmvHBGNNimRXdlLZLLRFubrKe7OFR2klUezsCyADEZR6OaIQY0ZA62nrmwg/kWjVsc8oAEC+xDhoIPj6lI5wGfTuKdm7iwM+oNuTtQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: __flush_tlb_range() avoids broadcasting TLB flushes when an mm context is only active on the local CPU. Apply this same optimization to TLB flushes of kernel memory when only one CPU is online. This check can be constant-folded when SMP is disabled. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v4) Changes in v4: - New patch for v4 arch/riscv/mm/tlbflush.c | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 0373661bd1c4..8cdb082f00ca 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -102,22 +102,15 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, unsigned long start, unsigned long size, unsigned long stride) { - bool broadcast; + unsigned int cpu; if (cpumask_empty(cmask)) return; - if (cmask != cpu_online_mask) { - unsigned int cpuid; + cpu = get_cpu(); - cpuid = get_cpu(); - /* check if the tlbflush needs to be sent to other CPUs */ - broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids; - } else { - broadcast = true; - } - - if (!broadcast) { + /* Check if the TLB flush needs to be sent to other CPUs. */ + if (cpumask_any_but(cmask, cpu) >= nr_cpu_ids) { local_flush_tlb_range_asid(start, size, stride, asid); } else if (riscv_use_sbi_for_rfence()) { sbi_remote_sfence_vma_asid(cmask, start, size, asid); @@ -131,8 +124,7 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, on_each_cpu_mask(cmask, __ipi_flush_tlb_range_asid, &ftd, 1); } - if (cmask != cpu_online_mask) - put_cpu(); + put_cpu(); } static inline unsigned long get_mm_asid(struct mm_struct *mm) From patchwork Thu Feb 29 23:21:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577710 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECBF6C54798 for ; Thu, 29 Feb 2024 23:22:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 63E446B009C; Thu, 29 Feb 2024 18:22:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 551686B009D; Thu, 29 Feb 2024 18:22:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A38D6B009E; Thu, 29 Feb 2024 18:22:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2792D6B009C for ; Thu, 29 Feb 2024 18:22:21 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 05A2C16024C for ; Thu, 29 Feb 2024 23:22:21 +0000 (UTC) X-FDA: 81846417282.11.382C8C6 Received: from mail-oi1-f175.google.com (mail-oi1-f175.google.com [209.85.167.175]) by imf26.hostedemail.com (Postfix) with ESMTP id 50F40140007 for ; Thu, 29 Feb 2024 23:22:19 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="PTZUH/xG"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf26.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.167.175 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709248939; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9YO6O944kqWb4oNKtEMwGtpxpVk1nDXsHdzKgYw/NyE=; b=TD7HVZ5Vo7a2rkaYoX11GK9ou6QfuFnKcRqJck3VSI67z6Qa6JRUkSFuqUQi1rbDR8ZSXi XU8dHJm6N3vgzy/jNYG2wnmxLDZ6y22QmqXfPwaHsHn+5VuvL1yDySJlcFhzpqGojIf3Jj e/37NiSE6WW07IwTLKYiXviPAigOLXk= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="PTZUH/xG"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf26.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.167.175 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709248939; a=rsa-sha256; cv=none; b=0T6cyh6Myxtf/66LOiq2saSWT+LU7aYRxgYqkNoUME1CQ+KhV+61w2+G1VStrLRQugBFb2 VQzmGiJnR5UB3Gk9DM4NHC4TNFw16jEgFYLuSPIlewsZjQdWILkNjKANzujQyka9KbXXyP 7W05dp9yZ1bo57qzyrm3jt9oyCMsXDQ= Received: by mail-oi1-f175.google.com with SMTP id 5614622812f47-3bd4e6a7cb0so971696b6e.3 for ; Thu, 29 Feb 2024 15:22:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248938; x=1709853738; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9YO6O944kqWb4oNKtEMwGtpxpVk1nDXsHdzKgYw/NyE=; b=PTZUH/xGJcSbZV8vdA+e7QClZFoXrGD3HJxNC1GUJs/19erE8j4p8oRrzToIQ1cFX2 GG45d/wz1UJOW5w8+aC1enMmfR55pxecANDvn+JbrlZWTzumKF3dfH+c803C8C6V3bvi xOAwQRnn8N2v/LgC/cwAInE36p7T6H2xMOENzUFVGGdPTa1VYNDRXu0LDZlqAWfFFLKQ LOeaBC5n53+CFiZNK/mRtwMyATYhfXemGqKu0jq4CS1TW0RXt8t8jnFeKTjPTl1GeBS0 rFgZTS20l/PjvGg/73rQ4kZkKa2pHOOdjZieGv8i2X9Z9b9lzaCD83fbb3AjxqBVphxj 6Lxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248938; x=1709853738; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9YO6O944kqWb4oNKtEMwGtpxpVk1nDXsHdzKgYw/NyE=; b=F5kTGlqgBfgLxXHSkCySAL4wIGGusPbLJVslPBQa43Sjn7C8orUp6ka1dv2aNKBW0n 1C/nuhND06j+NhgBy7jSwJkpxD4PvXLKy61SyuVFhRgDzlKB/rvsqCfbzBCPo/CUjvA1 VbWUWrUhB4xe5VAy3TLndtN/+H3dGTcFpHScriRmeJGRucx2U0r5EH263atTlML1/saQ 7FoGheS++6mI9QlzZinvaagaCGIo3mwsrTCwFLsRXgZ4VUEie6lGTdwEiHeTPGTIHD9i oq9WToqQ4ruzE+iT0lEN0nkb1XP54E4wS/SbzyR99T2C1/PCor3TXxCt3L7unAfjJD+Z vIdA== X-Forwarded-Encrypted: i=1; AJvYcCUKfknPm7HBIv3g6F+PsmFXoVWHBKi9/RFQ4fL1dsSt/wyKtSYUXOXmkVik2z8QjwcCcJpVHHf42bbxpOBnq64klpY= X-Gm-Message-State: AOJu0YwS53+Rgp8KfUJbXQA+28UObK6tcbSLKtKoKf3IdEjcD1bMTI6I 5bjX1Vfoy9gJHSsfZlBnU2kZX+UchI7kY2EmQQ2M9UuPIom6OS7cEck21/lqoSQ= X-Google-Smtp-Source: AGHT+IF/+PdA1OOJxMn23/EE1kPZ70u92KMkSqj19wYtg5glHTCWIaPOP2OtxcWfToq1z4V1DP9b6Q== X-Received: by 2002:a05:6808:8f2:b0:3c1:c37b:1a71 with SMTP id d18-20020a05680808f200b003c1c37b1a71mr62346oic.57.1709248938472; Thu, 29 Feb 2024 15:22:18 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:17 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 05/13] riscv: Only send remote fences when some other CPU is online Date: Thu, 29 Feb 2024 15:21:46 -0800 Message-ID: <20240229232211.161961-6-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 50F40140007 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 1mj5i56bmajhn6qdcke7f4zujbzxmq8b X-HE-Tag: 1709248939-707819 X-HE-Meta: U2FsdGVkX1/nNSbZz05Rgi3+pDO9AU1+PsmnL8IRsLITRsFgkkr9p7k78oAwyf1AwSTnKAWEyH7BvM1OKf5twE81KidotlovR4IlZLyJ/rLHfF5deyo9aSzJ4Wzz6y45jsCSPY2kJ8Codz75qsVsTkvHJ/+5KPvZVIzM6KyFUiKtmQW2mESFe7UdTzlCtSgVhYP81qENXOirB09GlWMtCFZGB7g5Pzwg6WZbLX2SB807holMfMEtVAzAf1B5uqbSupsiX/TGN9DTwVGJgXN4rIUlEkoX83BhRwj/U767LuZYjtAHIpVi62NlvP9x32YzchWzfcAfuHHgJqukw8yRz0zhOpF7iWT4be8Inxy9gXx7qUSTqVe6bmTpEKytM6a5/JD81IsHgfIGK5pB90r0aOxyGnptz4Xr4J3yIBzJjO19WSIrZBWk4QhfZkWaTK6OcUCvqRL1OPvXq5gFS2H+3iz8wQY/QIb+NhDwALg2lPJIGgsGmxPqBDYJW+2Gji8PDUPE9N91fEixPp/wrd4EEmT6blnLbq4M91KcAiPAS/P/YK+paXz1/8uLjTto/aAiCwaTPiMiRA+jBNrKxVQIQ5pm3E7EmF/m+JoDfEgp2qjHIESmNIOdSg8oYxhlyooYWbx4JEEDMi5XyT2oDvLFeb9Lgu7nnupnQUHAcDkCGepzVxB0Ip4SC6K5UWI88Kq3YzYlHI6DStzVNNLFSMsknDf8sm9B6D5Rv7/4tcYKfz1zoGOHIV8K+HxAiUCLa9mrjI2KdykBvio6TuwRhj5pmRdyJgaztsrZaSw9c84HPUP/t2+vGVSObpjtQ2BJfcgh5Fv2qSgoKjTzZLb6UNOa3HH9b54ubDPJnPBJvLnMXBOl3isy0eD2VJ5BpuJzVVY8hh0URjXy/R8vD0oWTyuczlyXdfHDKk3UBYcT+pmGycujhdzcVkMCQ8BwneXdxfz8Cw2+wM1DiuJThGTG7A0 SCXQrsN2 2HYzE0DdaiXW4xu1ibha6w6JO61QVYKDruhbS8BbW28V07B1jjru9ufGAE/31HBIhES9YlsDITyDb2zOA7le5eMWCh8yCEzbFSa8eLEnGiVo+MMQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.060444, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If no other CPU is online, a local cache or TLB flush is sufficient. These checks can be constant-folded when SMP is disabled. Signed-off-by: Samuel Holland --- (no changes since v4) Changes in v4: - New patch for v4 arch/riscv/mm/cacheflush.c | 4 +++- arch/riscv/mm/tlbflush.c | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c index 47c485bc7df0..f7933ae88a55 100644 --- a/arch/riscv/mm/cacheflush.c +++ b/arch/riscv/mm/cacheflush.c @@ -21,7 +21,9 @@ void flush_icache_all(void) { local_flush_icache_all(); - if (riscv_use_sbi_for_rfence()) + if (num_online_cpus() < 2) + return; + else if (riscv_use_sbi_for_rfence()) sbi_remote_fence_i(NULL); else on_each_cpu(ipi_remote_fence_i, NULL, 1); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 8cdb082f00ca..69402c260a89 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -78,7 +78,9 @@ static void __ipi_flush_tlb_all(void *info) void flush_tlb_all(void) { - if (riscv_use_sbi_for_rfence()) + if (num_online_cpus() < 2) + local_flush_tlb_all(); + else if (riscv_use_sbi_for_rfence()) sbi_remote_sfence_vma_asid(NULL, 0, FLUSH_TLB_MAX_SIZE, FLUSH_TLB_NO_ASID); else on_each_cpu(__ipi_flush_tlb_all, NULL, 1); From patchwork Thu Feb 29 23:21:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B12DC54E49 for ; Thu, 29 Feb 2024 23:22:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D1B116B009D; Thu, 29 Feb 2024 18:22:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CCA606B009F; Thu, 29 Feb 2024 18:22:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF6CA6B00A0; Thu, 29 Feb 2024 18:22:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9C2BD6B009D for ; Thu, 29 Feb 2024 18:22:22 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 67BBEA0FFE for ; Thu, 29 Feb 2024 23:22:22 +0000 (UTC) X-FDA: 81846417324.30.9CB17EB Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) by imf28.hostedemail.com (Postfix) with ESMTP id 8E0FEC001F for ; Thu, 29 Feb 2024 23:22:20 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=LKW2OZRk; spf=pass (imf28.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709248940; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uLB9vlYe9JfwQMhDb14nNgv57tMyr8X6yVrB4e0zcQU=; b=FH8wN4mtRG6Tsl1L/A52wHLC+Eje7GFFE5h4Fr7qlgH7NIZ/Sgkb8MATfVJ9nyqso8gGNZ OwJZzGzGCNMg1YpMAN7Sd2VuQsMn5+IeDjvaTTzQWPdzu2erY4Bg3LW8iASA+HNvCFfxY5 rPJiHl5Nbi0cN+xgQoRNCP1In4KhgtE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709248940; a=rsa-sha256; cv=none; b=03rz7K6WOB2MczT0GizJzU9nk+2csc5aRAFmrGyMZZ3aG0yWbvWlpZEJgymaxI5fRYqjHW rE1Ydbq4FxTUJPLHcnAS/VIR/AMDHerG6JQkiJOifrIgHwCs8OHMqtrn/SnMH7X9ZQVagt keM8DcX0zasFb1B/d18WcRa+7i6KyKE= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=LKW2OZRk; spf=pass (imf28.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-6e09143c7bdso1080209b3a.3 for ; Thu, 29 Feb 2024 15:22:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248939; x=1709853739; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uLB9vlYe9JfwQMhDb14nNgv57tMyr8X6yVrB4e0zcQU=; b=LKW2OZRkis6HJlM5UJcSpn2IXN0HOz0Ml1DDsiY7HqBpzgK9fPj+LaUi+MuUKhkRPc /s1dDqU1sd6CRqntqN7nr2y5j2X69G3kkzmbsWOOIMdfrxiyONjlGS1P+4tpBbxZraQI HD1TJX3sr/iL7ZTY2cXAhjFx6VVWXcOINK7QugE3UxpJn9fRySKn7LYq/MUMIr3q0unF 1gC/uaEOVYhbXvoObxR/3EYliesWK+p+uhnnz4C/T4DsQwBTJ0hVJb7iyMbL6ty/YMxi xd3+k3Bwal9dl7DDqWWsTezqPJz4wBcAKqS+agYX4g25ogpj9WQ4GdoLWKvAkGRmLfK+ 2EUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248939; x=1709853739; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uLB9vlYe9JfwQMhDb14nNgv57tMyr8X6yVrB4e0zcQU=; b=Ad4DlFfEKwMD2BGrg8TU5BC1ldo5xMAcq6g5Dr1gbiQgKvgM9nGckeN6/mdIRGe4kr Iw5MPv7K4emYUZpJRHlR+Zf0cFBKLgtU4b9FASCeKxtYDROiCO3d+18yMrw+T6ToCoyI s7EDx3nmlM3FPjP7v59yQyLD8It2RQKQ6F7GB1e6D+2ffftv1tamkcVvkoLpCSuk2uvi Mxf9ETLCre+/SBdKgI1kV211FyIerCjr/963bNJ/1NVXSdIGfmu1cjwn+7UAArYEBqzv 7aWBbvCP8sizXykdj6E+n8xyF4yrSBq/McAM3OFoUvrSpuSVlSf1Wc7wEws0hY2BH5bK yh6g== X-Forwarded-Encrypted: i=1; AJvYcCXIVq9Em6uLuI7CGpIj3UJ6eicWHHCKWW2NHGgu1fyJHabUF2qctf3IxIe2WTJtRgnrqBuXrdx2ZJw+VWfKmqRXz04= X-Gm-Message-State: AOJu0YyGg1o6mP2JkH2CHYrpac2Uenc82vpWaSQg+k/1UDDwVA2Wl2CC 5tLL0I5jkKKYV4uTADfYM9fdT0Bhih8sRhOmkXP51jCMeP8THoBJuUnDkwfygng= X-Google-Smtp-Source: AGHT+IFNMqThgNRh63sPPm1+a+RZC7oBqAsJKeQJ9ncYhwlwuRqR+WMAL4ZrZQk20TebpNdIS4mveg== X-Received: by 2002:a05:6a00:2354:b0:6e5:80a4:2ff2 with SMTP id j20-20020a056a00235400b006e580a42ff2mr152068pfj.30.1709248939492; Thu, 29 Feb 2024 15:22:19 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:19 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 06/13] riscv: mm: Combine the SMP and UP TLB flush code Date: Thu, 29 Feb 2024 15:21:47 -0800 Message-ID: <20240229232211.161961-7-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8E0FEC001F X-Rspam-User: X-Stat-Signature: 6ro5wn6j1aq8jmhn3krd6fto96uzgtoz X-Rspamd-Server: rspam03 X-HE-Tag: 1709248940-6694 X-HE-Meta: U2FsdGVkX18eEbA9bQ8yb3VOQvpC3st/2InCpjwKpxRiRBlY8WLrHrf1A2sqGifxaD5YlbKducgNAYZKK12ZA89uW70gaeehFoT+BOFzWOku0HrB0/6RBRztZEY1bS8SiysO/MgHH8qKwbebc+/x9MVCjYkPK7WJeI2PcxGdyjx3S3SBVfC4dx2fqFFSLZHyPzOqONzdvqKPFpQhdlrCT1/GagZOavLhUaSTDVOsTiZTgSE//Hj87bYP6yTZldZpz7FkYUjLFJ+NHVSrF9iFlPiCfDT7BuKBz4MY3AyTCzBjvFoloJYBnPrQy2n6Ou4JUqDGZAZZ9iLhfg6126yuyy1JEMd3t0moMKH1ZVZydNYlrL4s9KHhr8W+ruWg6nYJM0dyi0QBDA5GqqzmDYSBI3U2hl6gH1UMEXUuyRi1eZ+nnC+dJtMK8xUaMDcqkSW1TuE2Htn8FmCoTTYTaRAyF8YPVt9JJGZltA8dyCG4uzm2u4IB2P+WJsh0UbAUydSIxGTplGm090wSME+XXrM87UVuQEIzaXm3uIn8yfi0e/jGMOdl32Dwc/6iLFH+S580e2T9gU5b4U2VMvHyATSfashA4RmTmaOSsk4YBH+u2/REvS7x+kq/an5VztEfIwukj1Ld10+HydikdQshJUHG1i3NiCh91MkYtSPbp53b3RR/4x1puD/kd7RZpkAs9wbMDAGU8k0PSk7BWAl6AkegaaK7gxpvboGSVg7q6xdJ95xtGFQ5v2Skc3FbVEcbPX4MdN1JyhJ/84cwjOT5+CHuTVO6madcvk1/baRnVmrI46W43i6jq0VfRrD+2BgWPYYpdTQqoHCSXDzNbtDIkpQqYzup1ZYoXxHwp4rsW4UNiVyRSOl5AIWB3ct4msghxRxlHTT6yQ0ZyCr8OLe993bXS4tr8H37YVVJA40rEGBy3jJ2ez1UEokHwF2PPgBWnhl6/chtYRnWscJvG+aXlW0 c5016so2 EhDWfJjSiXj073LesOUmE2kmNNjgZSzUyUHcSAvFd0A3NEfPhiDgIqv3kRO09GgfYAH14Cp6TyvuC9HXsUBV2kVsB4NCP8CcDf1qAKYV9/oJZwD4u9oV45HL7l0zU4YUBIjZu/LG6ODUJ0IRRbnItJHWSIZop/nuf2XM6dBuTC4MbGZjyHQiwL+Qcy2z7/EC8TShOPADbFyXLGVChIzfmWv1WnwZavu28HhEpXGKaeZZDTx0qfvFhtcycTA0HcbyzKnEvLXCMuhSo4z2jrXBh6AAZak4Ya3t9/djkAxywCdSxW1uB1NvmEuz21kwY2zmZy9uSNVzZSqdra70yAzqgBKE0pR4YCbe7eAoFzTs62Kc82yTx1u5WFEq146pYZn+GTTq3bPvVjyEJzr15HcmUdBaLbpYPThr3uEiP+QeZftywldLZd9RBhaMTcyml6J5trK3XFUk2O0ED6UI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In SMP configurations, all TLB flushing narrower than flush_tlb_all() goes through __flush_tlb_range(). Do the same in UP configurations. This allows UP configurations to take advantage of recent improvements to the code in tlbflush.c, such as support for huge pages and flushing multiple-page ranges. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v4) Changes in v4: - Merge the two copies of __flush_tlb_range() and rely on the compiler to optimize out the broadcast path (both clang and gcc do this) - Merge the two copies of flush_tlb_all() and rely on constant folding Changes in v2: - Move the SMP/UP merge earlier in the series to avoid build issues - Make a copy of __flush_tlb_range() instead of adding ifdefs inside - local_flush_tlb_all() is the only function used on !MMU (smpboot.c) arch/riscv/Kconfig | 2 +- arch/riscv/include/asm/tlbflush.h | 30 +++--------------------------- arch/riscv/mm/Makefile | 5 +---- 3 files changed, 5 insertions(+), 32 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 0bfcfec67ed5..de9b6f2279ff 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -60,7 +60,7 @@ config RISCV select ARCH_USE_MEMTEST select ARCH_USE_QUEUED_RWLOCKS select ARCH_USES_CFI_TRAPS if CFI_CLANG - select ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH if SMP && MMU + select ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH if MMU select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_GENERAL_HUGETLB if !RISCV_ISA_SVNAPOT diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 928f096dca21..4f86424b1ba5 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -27,12 +27,7 @@ static inline void local_flush_tlb_page(unsigned long addr) { ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); } -#else /* CONFIG_MMU */ -#define local_flush_tlb_all() do { } while (0) -#define local_flush_tlb_page(addr) do { } while (0) -#endif /* CONFIG_MMU */ -#if defined(CONFIG_SMP) && defined(CONFIG_MMU) void flush_tlb_all(void); void flush_tlb_mm(struct mm_struct *mm); void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, @@ -54,27 +49,8 @@ void arch_tlbbatch_add_pending(struct arch_tlbflush_unmap_batch *batch, unsigned long uaddr); void arch_flush_tlb_batched_pending(struct mm_struct *mm); void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); - -#else /* CONFIG_SMP && CONFIG_MMU */ - -#define flush_tlb_all() local_flush_tlb_all() -#define flush_tlb_page(vma, addr) local_flush_tlb_page(addr) - -static inline void flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) -{ - local_flush_tlb_all(); -} - -/* Flush a range of kernel pages */ -static inline void flush_tlb_kernel_range(unsigned long start, - unsigned long end) -{ - local_flush_tlb_all(); -} - -#define flush_tlb_mm(mm) flush_tlb_all() -#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() -#endif /* !CONFIG_SMP || !CONFIG_MMU */ +#else /* CONFIG_MMU */ +#define local_flush_tlb_all() do { } while (0) +#endif /* CONFIG_MMU */ #endif /* _ASM_RISCV_TLBFLUSH_H */ diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index 2c869f8026a8..cbe4d775ef56 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -13,14 +13,11 @@ endif KCOV_INSTRUMENT_init.o := n obj-y += init.o -obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o pgtable.o +obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o pgtable.o tlbflush.o obj-y += cacheflush.o obj-y += context.o obj-y += pmem.o -ifeq ($(CONFIG_MMU),y) -obj-$(CONFIG_SMP) += tlbflush.o -endif obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_KASAN) += kasan_init.o From patchwork Thu Feb 29 23:21:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577712 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E1ADC54798 for ; Thu, 29 Feb 2024 23:22:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 94E576B00A2; Thu, 29 Feb 2024 18:22:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8863F6B00A1; Thu, 29 Feb 2024 18:22:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68B196B00A2; Thu, 29 Feb 2024 18:22:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 54B856B009F for ; Thu, 29 Feb 2024 18:22:23 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 3485C1202A8 for ; Thu, 29 Feb 2024 23:22:23 +0000 (UTC) X-FDA: 81846417366.26.A2B085B Received: from mail-oa1-f49.google.com (mail-oa1-f49.google.com [209.85.160.49]) by imf17.hostedemail.com (Postfix) with ESMTP id 6F44440014 for ; Thu, 29 Feb 2024 23:22:21 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=i5HnTCVo; spf=pass (imf17.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.160.49 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709248941; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4jEC1X5CZdJa4ryFg9hWmhSsIovSPBi+ZbblNH/6mwk=; b=ssHpx3tq+gUhV1YD7FlN9N8sOfqT5vJovtrckJl+JVGvCyyqm+U+gHJhRg2ZNjTBA2Ob+w nkU8YicrQeDINXRoAE8HO5QfZ33NARpw7WBPcIk3oshPnhcCE0/XLJkwZiZnk9R5meHDOt setSsHCF1oSaMsvmO6CUGKG2mERU2vg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709248941; a=rsa-sha256; cv=none; b=6fEsdoKmdCxWIf0hd+AMuawKAepf4m6bz9eISHd+5Uo1kjM0ve4/wCKl3n1x+2TFSOQlZZ Uf7Xehrw8gn2E+e4lBcAA5PQ7OsY7vpPl3qEw94WsXBubX6Obc/miDe2Jnx6m9nNGT4FPD TaOVj+RqvzHZduT58WGwW+hLfEhcjS8= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=i5HnTCVo; spf=pass (imf17.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.160.49 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com Received: by mail-oa1-f49.google.com with SMTP id 586e51a60fabf-21f996af9bdso615127fac.2 for ; Thu, 29 Feb 2024 15:22:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248940; x=1709853740; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4jEC1X5CZdJa4ryFg9hWmhSsIovSPBi+ZbblNH/6mwk=; b=i5HnTCVou5yeicsyMtoB3sjEDcciIyI6/A/CsK585vuhUOctJZTBs43SZXSlj/Ma9u D8OGqv1d6AtqHO614sIM47TaNPAd7KXnyGkrLIiXUkjgJC0b/9Df6RWoC//5N5CAv863 CRtPVedO08YwZ05kj1kshKYHSYPozcLGUmMI81VsyT/tQAljaQT90UtfIk8cIAAFhhNS WGAIrYDusI9X5yp2dFPwRqwYBNCiHBvpHpby70uU/UuU9PUo6/nsrDCoVW7nrOOD9pQu K2fnDMsEqMYMQsQYaPogOiMYPHQf4OsPXO09FMDNhYRDsHJmQzD2UfFhmJ/ujdMkdJKa XLIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248940; x=1709853740; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4jEC1X5CZdJa4ryFg9hWmhSsIovSPBi+ZbblNH/6mwk=; b=kyTMvhNaDjw4ZRvQPFh+JPdHygipTqNgq8AH6a+5cQtTioGEBS+GJ6ARZeNtLFmW8n ppPRK5QYhjJO9nZkixQNxsbQ5iFDAj9WFpUr2itvHguyEhchbIjF9uv+Hy8iZojMq8f1 V888nX5huWNHcv/8q7KaukpFJW1Zc2QuSFbP2GI2GS/WZjhSeMDJVNOoGS0QbfYkufcx LzTz9jaO63ukDXykS4ilx32qk9Cwao4iithsR9abOHMTXEWy3EplvY1bbH56UH3Q00/z gMBW7tOvOHuAlcYDTiMn/hfdjIzAh4habJt9XbW64HmiBIphjwz6k/w7nbBwwhcRfi6O OeSg== X-Forwarded-Encrypted: i=1; AJvYcCVHFCBTEYW2cRy1dbV4eA0hZK3sChBEiEFEW3HSyYySQhb6aufQTDG0skLUfV6oHPyB/k6qW7H21rOkOvxHS18k7m8= X-Gm-Message-State: AOJu0Yyac39SNfYS0kRe2zyZ7nNipHb0m1Xn1XDMSHMbkGKng+oBUh62 KEWywMY/C4KbKM5wypwUEw4DeEWL/5S6r8QFRbY+gDsEL3AbvVP2GKARNGqyXZo= X-Google-Smtp-Source: AGHT+IGX4/HV5W58fetZh8Yw57gYr75gd/4aFWyaezvIp/BsV5LakI2yn+mW+sbVnuOSeLnU1moubA== X-Received: by 2002:a05:6870:ac22:b0:21e:dd7a:2d3e with SMTP id kw34-20020a056870ac2200b0021edd7a2d3emr4104469oab.22.1709248940592; Thu, 29 Feb 2024 15:22:20 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:20 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 07/13] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Date: Thu, 29 Feb 2024 15:21:48 -0800 Message-ID: <20240229232211.161961-8-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6F44440014 X-Rspam-User: X-Stat-Signature: 8zkdp5gd7xqirhz7tq3bj6wsyzh8cshx X-Rspamd-Server: rspam03 X-HE-Tag: 1709248941-754001 X-HE-Meta: U2FsdGVkX184KFET5FDYrNPJun6UixvdkiO38yZodmzTMv9zWWFlO9w/AbKov4Rh7Yjpkv2X84dSv5NZgRoIY44wy/lJmxU6vDLaL72SXDpctI8lHftPgGil89dMHEZNVqfWRPoLDdsi0BC+JnGHP6LMwz4YSiOX9WTXY+PevNusFowOs/xg+Zz1O5upnR/nWc1AI1OE+cNkKFEOssvpGEZZX+t+U5jbXkczaeJ23Ko4vubITtbCeA/oT7S9nV8tYAyTDdvO/b3SuR4hLc59oLa0gsvsD1iE0qqXpEmX0h54JKMxWvwhboX6A1AewD3bcZPKt4b7gAO5IgQ8usW9NRjpjOAWveT9u2CVKR85nwUAj8fDg2T/Hbzi2AkMx6mk0146PNptxNsc69hUU06urY11G8WerY8uHRRsDKuzagccH5QJ4esL8w95oioMpyNMeNJ6lES4DBPV+zfpQZTylSufFDRwXZta47sbwyaRpt0k2ZO0r0iU7kSIYKieHBTJl2ng3GEewMK7Md+B3h9LidQHoY1+3MVKt1B/Y9SQPyrX2goxwsofXOXcEHQmEfvP77rY9tg+fDWjrXeXJzlXl5Lp5b9HcnvgChCtX+bcoq5XoQRkMscMPEbUEQ9CcSLhOUarGv0KlLFkH4HpeN7mylMpxltDm3lkGdj2JsyWyv+6K92SXEyYGAPJxUlPt+QuZWIkyzPw7DqT0ctuQRtEpuy8kGQp/CZhlGjEueY/c8KxIISZ71afizbyhl2a933P6yJ9AMxNQf5Z3Mq6moGlrgR0eMYHqgKFw8bQMWrONtRXYuL9dwnzrHyuJBsdZB+tDY5YCR5wiNMXq/K7ITcBK9SkRQ3XtDAZDcyO4+2+BuxBS+5GJnuvHOu7amBZOx1C2Wg3I/gPYXB8X1A6eZeCc2tvkH9a8Ry14nmO6uB4Yz58OBSduY8FsJArNyQXgzoIXyA0uKkZh0QEzlhtVR0 A/tczNcY 0lyGWWKCX+ZiDLXgjFQNU+PbAhQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: commit 3f1e782998cd ("riscv: add ASID-based tlbflushing methods") added calls to the sfence.vma instruction with rs2 != x0. These single-ASID instruction variants are also affected by SiFive errata CIP-1200. Until now, the errata workaround was not needed for the single-ASID sfence.vma variants, because they were only used when the ASID allocator was enabled, and the affected SiFive platforms do not support multiple ASIDs. However, we are going to start using those sfence.vma variants regardless of ASID support, so now we need alternatives covering them. Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Rebase on Alexandre's "riscv: tlb flush improvements" series v5 arch/riscv/include/asm/errata_list.h | 12 +++++++++++- arch/riscv/include/asm/tlbflush.h | 19 ++++++++++++++++++- arch/riscv/mm/tlbflush.c | 23 ----------------------- 3 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index ea33288f8a25..6e2762101968 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -43,11 +43,21 @@ ALTERNATIVE(__stringify(RISCV_PTR do_page_fault), \ CONFIG_ERRATA_SIFIVE_CIP_453) #else /* !__ASSEMBLY__ */ -#define ALT_FLUSH_TLB_PAGE(x) \ +#define ALT_SFENCE_VMA_ASID(asid) \ +asm(ALTERNATIVE("sfence.vma x0, %0", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (asid) : "memory") + +#define ALT_SFENCE_VMA_ADDR(addr) \ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \ ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ : : "r" (addr) : "memory") +#define ALT_SFENCE_VMA_ADDR_ASID(addr, asid) \ +asm(ALTERNATIVE("sfence.vma %0, %1", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (addr), "r" (asid) : "memory") + /* * _val is marked as "will be overwritten", so need to set it to 0 * in the default case. diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 4f86424b1ba5..463b615d7728 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -22,10 +22,27 @@ static inline void local_flush_tlb_all(void) __asm__ __volatile__ ("sfence.vma" : : : "memory"); } +static inline void local_flush_tlb_all_asid(unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ASID(asid); + else + local_flush_tlb_all(); +} + /* Flush one page from local TLB */ static inline void local_flush_tlb_page(unsigned long addr) { - ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); + ALT_SFENCE_VMA_ADDR(addr); +} + +static inline void local_flush_tlb_page_asid(unsigned long addr, + unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ADDR_ASID(addr, asid); + else + local_flush_tlb_page(addr); } void flush_tlb_all(void); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 69402c260a89..365e0a0e4725 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -7,29 +7,6 @@ #include #include -static inline void local_flush_tlb_all_asid(unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma x0, %0" - : - : "r" (asid) - : "memory"); - else - local_flush_tlb_all(); -} - -static inline void local_flush_tlb_page_asid(unsigned long addr, - unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma %0, %1" - : - : "r" (addr), "r" (asid) - : "memory"); - else - local_flush_tlb_page(addr); -} - /* * Flush entire TLB if number of entries to be flushed is greater * than the threshold below. From patchwork Thu Feb 29 23:21:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577713 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FA84C54E49 for ; Thu, 29 Feb 2024 23:22:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C80FF6B009F; Thu, 29 Feb 2024 18:22:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C06606B00A1; Thu, 29 Feb 2024 18:22:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A80CF6B00A3; Thu, 29 Feb 2024 18:22:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 945776B009F for ; Thu, 29 Feb 2024 18:22:24 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 71EA4A1606 for ; Thu, 29 Feb 2024 23:22:24 +0000 (UTC) X-FDA: 81846417408.10.AD8012B Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by imf13.hostedemail.com (Postfix) with ESMTP id BDA8C2001B for ; Thu, 29 Feb 2024 23:22:22 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="Rc/zXJzV"; spf=pass (imf13.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709248942; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wHMbuRyYmPRZ8L34XsarFB0wBhmLHt6PApeEi+FQBPY=; b=RwMBvBVd7/l3PSJghCByimwURjYBieImiyKvqJBGdrHHMKwGrLscy673GOKZfS5GA5jf/J 6M3y8w90WWAMgJW4qOoUANMdndwiuWtiFS7ZlECwciBJS5WLruXUxuzVC4LAzgLW53WAoL gpfh6iivbXo1v1AZBPBoyE4WXahctTQ= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="Rc/zXJzV"; spf=pass (imf13.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709248942; a=rsa-sha256; cv=none; b=SomuLuU97ZKRbesK8N9V7vvQ25IqJNjDfbYkRku/SEuJZKU5m9ZZKccwZO6vHB5aGAeRm1 ZWwxuZqAj+UvcA9yTfwLQ/ZQLC+8bueChhD/vU9n3XgdN/B0ncAYybaxjVEnMwhsMbmaiX eKPXCnTCeEwfhJEM9Iafvn02OzCpya8= Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-6e58d259601so911849b3a.3 for ; Thu, 29 Feb 2024 15:22:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248942; x=1709853742; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wHMbuRyYmPRZ8L34XsarFB0wBhmLHt6PApeEi+FQBPY=; b=Rc/zXJzV8IHMtkYdMBdQaV4Oxngj23Qc6IBKHJIUdiJIYcgaKlBylVIaYboiovtY1c 1b+La/9TyLhkl4qEBx7PjUC4O/N2beooeCw2IGMKvQJCJc7jlchWKHpXePp7Y5917OSA FgXZ6/tj+acV4eNUVvlR7Njy9zDPxFeBMwKSEqYWaTjrzRH92rTipkz5daGF8u+qSdAA d+xREctrwA2RV6pJTtfrgvRMKWRRjWXO1ioDqDig1CdWYcdqgp5mIAQAdysH2tSBPzcy 2z8WRyPIh99ia3V+VQZ2E4T2BsI435WcThKvmhIx7BWijdS62xnYCq+jY624ag6Pjy05 rBaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248942; x=1709853742; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wHMbuRyYmPRZ8L34XsarFB0wBhmLHt6PApeEi+FQBPY=; b=di258RWcH1V48Mu0iuam5bOhw5AcxaE9mc2+ogbVINpbkufI754WUeUfqAGQ/z/uwI jo2TKRVL/KBiHG29nhrUj+cSc0pSWGhbkiM8glks9lBwBC8OGNwjWkp/GDuNAD024NV5 wbAhjFm2zgwgCOilgjL9mxgTkheWJBN14UmtasC2nlyFsPUJ0Sg+jyhQrDQCtZ+50/B0 ejYt0DF0P4X9CqYYLLqNnkxhuyHqIlVpH7QMtppBReVe+ECpC62gJXdFrOWnSySSvwVU gw666n0D2JhzBAXTR/udlyC2jzfZHfzQRByqlN/wRzCH9Ncl2x2tGNVRDJ70AXFOZbuK 6oZw== X-Forwarded-Encrypted: i=1; AJvYcCVyFlMnsumlkF29d3G7VUsZQ80x7WwhbZI06zofwwmnYvhntuTTeF5CO+Elx2dO/0o35zJHKllFckC67PXvUoLzLQI= X-Gm-Message-State: AOJu0YzwPUMTTpINoiqNbERL8yS1xDdclqxIod0Ghcb3H8WKS2zy7eXy wEZYdGIE+xiUSLcbysWUQToPYcbUIMdlcPlfnvFyNXFJk0gSKZgeqM1Hq7ukeEE= X-Google-Smtp-Source: AGHT+IE0KT3RuIOZoiwK4WAulPfaiFPRjhF12PREr/S+YlSI1LgFKc32EbVKWDUbhqJfm1349m8bOA== X-Received: by 2002:a05:6a00:1955:b0:6e4:eb29:357 with SMTP id s21-20020a056a00195500b006e4eb290357mr165166pfk.12.1709248941678; Thu, 29 Feb 2024 15:22:21 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:21 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 08/13] riscv: Avoid TLB flush loops when affected by SiFive CIP-1200 Date: Thu, 29 Feb 2024 15:21:49 -0800 Message-ID: <20240229232211.161961-9-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: BDA8C2001B X-Rspam-User: X-Stat-Signature: ar5c3d5edjnz9i1qqx53e3o5zqiypkfe X-Rspamd-Server: rspam01 X-HE-Tag: 1709248942-783585 X-HE-Meta: U2FsdGVkX18ELhFmfQls39lL9betIBAPR3koQzPrMJ+h6ZtGule4G61+mebWe9Cs9swpsNNztQgx9nI8P3XxOKonro+UBoU/weJQXjeBR5T2gZ3IIorNv7oPoiSDO8zdARl99geCxOzRzYTe02SpxuzXCKgLkaAdGdemQ2lcDAnmwXVm0fW+zpUIq9YsWwIwYqGzl+bHAxS2h4ncRQAT1lEjcsZd6nZcECWDwJyQ8HeAuAyaYPyECKjb4iPtu4wQuuEZmiEE74u+OEf6HHtKzn6nvFFDzuY0GDAr8IU4lVVnDqnY3FZz2nWkIS57qGw8AuwsSTeO60dZh4s6nt7OCnFsQAs9a+xkmfzXL8Njl3yhHjkeJ+Ga63dehRo2dIGXW991kUeFqd4tR+WVau8LaN/lYlqqnnra8G2IKyn++mHwg1VutKoDgNLnHDyCSTUIuRqgRa3qCza6mv4fHjzlQ+yyH2DcfOBYgKuEsU73cZkyqKuRouuJZsmZ7ScW7o2Q44fDssq+maxem7IbnTdLofkRxs/eFpdWCH1ZhwFZEAB5oITQgfjdj1NFzv3bmHFNLybIe7HDuNwOMdEnJYmamnFTzdNmrGxTiNpKvAJZrqspJYzyExZvBvoQyrkiblow44D+hud9bIrZlBtVetBBcOXXIpv+jiKRSJ9EYlMyqRW/GZ8DLQz65mSq3LKquUNJUcEUJkyUYAWEZt8Fe7yogIJfLr9qB6XTNn6y8Efv1aI+UYPzaP+OmOfwLo+RvI8RJJT4NFi4AOVMuUzUn9D1kghO5MiubGgl0HSaadQLZFk89JtJWHEICbzJlUJKbyLtxIcFDQvqjWUEqml0F0KpJj4wDuBPwH0f48tFRdi2yNR4kf5Vk0LuhWTjqr4vqnVDKTpMUbfKKw1Z3qxnA8w9r5AgfzsC0573bzk0oTo4exot3XWHozIQ3mLtyxh0h8yWsNpCW8pBiJ9Opneousk FwpDpm7E LhgnFVqvxulCcVdzY924JJDqx5r9CdRswHrbqvrxLJ4ruAjMZ537GdngOMe1JUuaMp597iZ11xhFn7FURwlx7s+ReukCjcS0A6YcraPQP42yqMn8I4yH1C7riFA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000052, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since implementations affected by SiFive errata CIP-1200 always use the global variant of the sfence.vma instruction, they only need to execute the instruction once. The range-based loop only hurts performance. Signed-off-by: Samuel Holland --- (no changes since v4) Changes in v4: - Only set tlb_flush_all_threshold when CONFIG_MMU=y. Changes in v3: - New patch for v3 arch/riscv/errata/sifive/errata.c | 5 +++++ arch/riscv/include/asm/tlbflush.h | 2 ++ arch/riscv/mm/tlbflush.c | 2 +- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c index 3d9a32d791f7..716cfedad3a2 100644 --- a/arch/riscv/errata/sifive/errata.c +++ b/arch/riscv/errata/sifive/errata.c @@ -42,6 +42,11 @@ static bool errata_cip_1200_check_func(unsigned long arch_id, unsigned long imp return false; if ((impid & 0xffffff) > 0x200630 || impid == 0x1200626) return false; + +#ifdef CONFIG_MMU + tlb_flush_all_threshold = 0; +#endif + return true; } diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 463b615d7728..8e329721375b 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -66,6 +66,8 @@ void arch_tlbbatch_add_pending(struct arch_tlbflush_unmap_batch *batch, unsigned long uaddr); void arch_flush_tlb_batched_pending(struct mm_struct *mm); void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); + +extern unsigned long tlb_flush_all_threshold; #else /* CONFIG_MMU */ #define local_flush_tlb_all() do { } while (0) #endif /* CONFIG_MMU */ diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 365e0a0e4725..22870f213188 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -11,7 +11,7 @@ * Flush entire TLB if number of entries to be flushed is greater * than the threshold below. */ -static unsigned long tlb_flush_all_threshold __read_mostly = 64; +unsigned long tlb_flush_all_threshold __read_mostly = 64; static void local_flush_tlb_range_threshold_asid(unsigned long start, unsigned long size, From patchwork Thu Feb 29 23:21:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577714 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48996C54798 for ; Thu, 29 Feb 2024 23:22:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C093E6B00A1; Thu, 29 Feb 2024 18:22:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B932D6B00A3; Thu, 29 Feb 2024 18:22:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E5136B00A4; Thu, 29 Feb 2024 18:22:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 89CA36B00A1 for ; Thu, 29 Feb 2024 18:22:25 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 67D54A0146 for ; Thu, 29 Feb 2024 23:22:25 +0000 (UTC) X-FDA: 81846417450.30.41616D0 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by imf28.hostedemail.com (Postfix) with ESMTP id BABD6C000C for ; Thu, 29 Feb 2024 23:22:23 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=DN+10HOT; spf=pass (imf28.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709248943; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lukLV3IzObY+fmX3TTuFURPx1t0s+LwQl+LTSkieoBQ=; b=DH0RV7s8Aa46XmDVdX4AlUwpHlA/r4V/nOwAk1iGx4PWBoZa8DAHaJl7SptzZRf9Y1E9e4 1rbAt23d7GJWzjWS5ZsmB/tI+R1n7QlgiNooUHGRs+4jJY1UIJVlZwLCG2AkZTBz6l9dmV gzeNQ9IyVHH42gbL15rO0jgb0UcQfnY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709248943; a=rsa-sha256; cv=none; b=aMpzudyXErpQ/tIkWFgmZ8whzwkwOjx7P+eHjRA3CH4XTgnR9rsrznQvzsaL+i+Hmt6foR +Vfq1qCN+m3BwixS+stYIBDHot3wx8fDvNR5PRgc3Ho10BZMlGi9XcLDAVzVAfcr6z3kMW ZIdt37rl55lJKJ0o2zwxiIMKUKHmsmM= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=DN+10HOT; spf=pass (imf28.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-6e55731af5cso1202649b3a.0 for ; Thu, 29 Feb 2024 15:22:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248943; x=1709853743; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lukLV3IzObY+fmX3TTuFURPx1t0s+LwQl+LTSkieoBQ=; b=DN+10HOTwGu2OuNbgU2ZpCm8FHUWGSrVSnBRLT359b7ust2Z4I/YpjvzDPd0Rul/Cl bZ3MY1+mVm88bC/X9sH3YgiXOl8xLh0+JP7N05UVDcAI7ReI/G6cOnTvH6rPZR+M0UHA 16/S/Nf0AW7FIqcjK6qBOFURZ3Z4CoiGpASsU6OJVGRdr5HsYt2a4GLEbbRLhD3NtEre VcLP4GlcFe0vKpeiYaKbwRRUSNLHQok48AYkO9jw0qtVhjLxA3x7klBzx5wSwQaKy0Eu Dfyg+CVoiFeseVdq3wU9qizEXdjPQqX39XEKN+NrImk8ekLaX48s3xB4B4i9qDf7WesL +RrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248943; x=1709853743; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lukLV3IzObY+fmX3TTuFURPx1t0s+LwQl+LTSkieoBQ=; b=fEDspJBC4t8rnsviCGO+bjDK9/WgiAQZFF2Sy03xx/OBMtS1ri9uDl+6iqRM5ahsd8 xEuZxeuQIoWNpjRkOJZPOokIE1V9xkj7KR+wYPoGoHmOGaJ4vggI1pqIF6AzO0plOXda VrroPxkHzXbi8mr9a+TAnhPXPjjegZ0BzQdoDCAbh768mKviANzOoEXaODz93q5eReIb fZUGAFrGA/1AZiDnBcIeaje0E0Qh7m7ziEuaAI/N3tCiAfzPt2Bf8T+WJ3+XAGBDHad/ aHovtwD1IeJb9xTy0Q2n05zJ4jGTn/N0eCExSR2EK9Qe9BMeXgyZrwgzCVAiCBNqa2g0 nAjg== X-Forwarded-Encrypted: i=1; AJvYcCVwH7/AQxvstraD8eU3UUwgA0bYSLoBN3aRuUgSB2Xxlju4BtwwFYiEe4z2rim7B64sSBTvnihVymv0M9SL1coRoIA= X-Gm-Message-State: AOJu0Yw7hQxC7cWcf5VcG0n5jAr94hsf/WfZfMacMbcwB9ibKjU4uEoc iYlHMXc2iRfPt+GZu1o9pUhPdNONHxv9UBKIoiQcev8BOwGQnYzd4A47ywH9WLg= X-Google-Smtp-Source: AGHT+IFn2rYTQCqOuHVpuGhcF7BC7asS4ROedWcgIAd4N2h6ux4X4gjzLV3woCw85UyqGNmjtKX//g== X-Received: by 2002:a05:6a00:1955:b0:6e5:5116:2cb5 with SMTP id s21-20020a056a00195500b006e551162cb5mr172054pfk.15.1709248942678; Thu, 29 Feb 2024 15:22:22 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:22 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 09/13] riscv: mm: Introduce cntx2asid/cntx2version helper macros Date: Thu, 29 Feb 2024 15:21:50 -0800 Message-ID: <20240229232211.161961-10-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: BABD6C000C X-Rspam-User: X-Stat-Signature: 6p6n8ycyd5w8187mptykmzukcq885m5r X-Rspamd-Server: rspam03 X-HE-Tag: 1709248943-813832 X-HE-Meta: U2FsdGVkX1+sBS0xpF7j+HfM/QovCD2FOSDInTko2o+Uon2uGUwxHbGLD0p79vsJ/nqwGW54OtS1mpq8qjJsHs8bAC7kSOR7VlMYJTwcIAk+ypulL0X7ovyfQXhbMjUr7AtQXc+OlvVFDz7qPET4OTsBaX4ksbV48wtWWru8jqCz34dNQFB+iVJd3cVYVxkXhpt1Tis9E30jrTbrwpwi3wh/pos1PjmUt54+tXBTHetlTCyHz5Rj2ejnCHICSYmqMjw39cLeSpul6Ib0v/L7MbnXPtcZST4dnO0zopg3AgrLSx2xdB9nL0LoNfshMdfHO60jXc6lAozE2j2C4Q6ORrBVeO2OGJdtDpdsgrpTLBVHnBSSlgvHTXC3+GehM57sh/AImbRDMYnTlKETvPaBr2/vfs6zcL1Zy/g3KSl8sba0ptBIZZmJ2Rc37bShpjH/AHvozpKqVRehJ8c2oqTqBYrNmoVl9eevagQFNPybWa52amx2LTmIS+KgUGciuBNC8k/vboMbOa0uMcem2jZuHzUi213oJwioxMDQAOlmO8B15ZwtKIe8bcO3FONvov7hTsxU6FwpS+fpNgqQLBj1K1pgV7r9w8UpcG3rhb85z2vcO+2Pb/JMH1QxH90Mvin440jkt8ydHemzIvR3LyIEYBjs71kSxv2wiLwUJc9mm5EF5GiVSg6xXTEqHFrHq1vgIRmqChJxwjPcBeObpMQkrXLrQKfsIG6uM8R27OvmL9JUH7Tb4SpLW4bcAloCGWs0R4D1pe6JdS8WQ/scw6MXjwY/quC+g5A650ajQDopc0T97NSNtvsI0skW0htTeejbYskOdAwXGpLQugS+W3HTnRwhHGSS6vJ/5HMUvyDG/s+yC1kYL8x5sOkTOUUmjR2H9NrNqPqX+awDKkgfUhnRjCrx/iNk2xc/MBtGWdIBJ2w0+Ev1dJyFyLBSxvNfSQsR6Z5WOIUcygCkKNHRHcx 9BJVqzBU mpjc8M4bRpgGIAqZ4HGdFcTvr6OKPAo4hanIKrEaKSAeYJOzxMSbXZ5WU7qUuQR69MZjzV5Zb9M+if3QRCCnHNzj5Cv2hewDqgFHadFXLFyJd4Klk0HwaPsY9jA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When using the ASID allocator, the MM context ID contains two values: the ASID in the lower bits, and the allocator version number in the remaining bits. Use macros to make this separation more obvious. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/include/asm/mmu.h | 3 +++ arch/riscv/mm/context.c | 12 ++++++------ arch/riscv/mm/tlbflush.c | 2 +- 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/arch/riscv/include/asm/mmu.h b/arch/riscv/include/asm/mmu.h index 355504b37f8e..a550fbf770be 100644 --- a/arch/riscv/include/asm/mmu.h +++ b/arch/riscv/include/asm/mmu.h @@ -26,6 +26,9 @@ typedef struct { #endif } mm_context_t; +#define cntx2asid(cntx) ((cntx) & asid_mask) +#define cntx2version(cntx) ((cntx) & ~asid_mask) + void __init create_pgd_mapping(pgd_t *pgdp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, pgprot_t prot); #endif /* __ASSEMBLY__ */ diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index ba8eb3944687..b562b3c44487 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -81,7 +81,7 @@ static void __flush_context(void) if (cntx == 0) cntx = per_cpu(reserved_context, i); - __set_bit(cntx & asid_mask, context_asid_map); + __set_bit(cntx2asid(cntx), context_asid_map); per_cpu(reserved_context, i) = cntx; } @@ -102,7 +102,7 @@ static unsigned long __new_context(struct mm_struct *mm) lockdep_assert_held(&context_lock); if (cntx != 0) { - unsigned long newcntx = ver | (cntx & asid_mask); + unsigned long newcntx = ver | cntx2asid(cntx); /* * If our current CONTEXT was active during a rollover, we @@ -115,7 +115,7 @@ static unsigned long __new_context(struct mm_struct *mm) * We had a valid CONTEXT in a previous life, so try to * re-use it if possible. */ - if (!__test_and_set_bit(cntx & asid_mask, context_asid_map)) + if (!__test_and_set_bit(cntx2asid(cntx), context_asid_map)) return newcntx; } @@ -168,7 +168,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) */ old_active_cntx = atomic_long_read(&per_cpu(active_context, cpu)); if (old_active_cntx && - ((cntx & ~asid_mask) == atomic_long_read(¤t_version)) && + (cntx2version(cntx) == atomic_long_read(¤t_version)) && atomic_long_cmpxchg_relaxed(&per_cpu(active_context, cpu), old_active_cntx, cntx)) goto switch_mm_fast; @@ -177,7 +177,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) /* Check that our ASID belongs to the current_version. */ cntx = atomic_long_read(&mm->context.id); - if ((cntx & ~asid_mask) != atomic_long_read(¤t_version)) { + if (cntx2version(cntx) != atomic_long_read(¤t_version)) { cntx = __new_context(mm); atomic_long_set(&mm->context.id, cntx); } @@ -191,7 +191,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) switch_mm_fast: csr_write(CSR_SATP, virt_to_pfn(mm->pgd) | - ((cntx & asid_mask) << SATP_ASID_SHIFT) | + (cntx2asid(cntx) << SATP_ASID_SHIFT) | satp_mode); if (need_flush_tlb) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 22870f213188..e194e14e5b2b 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -109,7 +109,7 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, static inline unsigned long get_mm_asid(struct mm_struct *mm) { return static_branch_unlikely(&use_asid_allocator) ? - atomic_long_read(&mm->context.id) & asid_mask : FLUSH_TLB_NO_ASID; + cntx2asid(atomic_long_read(&mm->context.id)) : FLUSH_TLB_NO_ASID; } void flush_tlb_mm(struct mm_struct *mm) From patchwork Thu Feb 29 23:21:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFF6AC54798 for ; Thu, 29 Feb 2024 23:22:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A85556B00A3; Thu, 29 Feb 2024 18:22:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A33396B00A4; Thu, 29 Feb 2024 18:22:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D3BD6B00A5; Thu, 29 Feb 2024 18:22:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 76D546B00A3 for ; Thu, 29 Feb 2024 18:22:26 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 59173A1447 for ; Thu, 29 Feb 2024 23:22:26 +0000 (UTC) X-FDA: 81846417492.22.4BF324A Received: from mail-oi1-f169.google.com (mail-oi1-f169.google.com [209.85.167.169]) by imf02.hostedemail.com (Postfix) with ESMTP id 8335F80017 for ; Thu, 29 Feb 2024 23:22:24 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=UCQdLiX2; spf=pass (imf02.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.167.169 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709248944; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=A+Ju3+5NsQ7zqLi78mVdBh82QtKzTZZzrfFOA+rFDFs=; b=or06n06OuxSnliozM3mFlgK8NBFy+dJio6TRvkBEYOGk8gUdzvkYadquy/gN6jG8pA17OF NYD4MPmgv1hgV2Td7vby6gE+lNrz6qHxBhU/02x9G9Lj/BCT1d5pyf+fuBVflUF6ioGDVo WqOGTqh52VFsHIYga47pFdqamCQ4lXc= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=UCQdLiX2; spf=pass (imf02.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.167.169 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709248944; a=rsa-sha256; cv=none; b=JE5w7ll67i/nJUiS6v7iYVJSugjLfBpRKPIUnCVaOfnJaJ5XjXlVdwpcn4hwKUHuQUfaK9 aw1aOkORYa102OxdytR1zWd1PLRqQ2/t4BwWFeT8BYvSThoPSJ5+88db5i7ruJKxlqg2AC vUnavGG6VQKv7EfU75LZyIGp7g2xTuU= Received: by mail-oi1-f169.google.com with SMTP id 5614622812f47-3c1c3ab19c3so734907b6e.1 for ; Thu, 29 Feb 2024 15:22:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248943; x=1709853743; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=A+Ju3+5NsQ7zqLi78mVdBh82QtKzTZZzrfFOA+rFDFs=; b=UCQdLiX2vZMXhsvYtH8+vXY+sM3gU+hDepp/nZT0xycKoGcv2yF6oMeZEnRFE0Gn4A oiESHu1oQJXJe9ha9bJ+9Fsoj/dbLPPu3DhGmsLzME85c22vwIlQe03vgtsAh/5UHfp8 aFunHWOlbfzzedOESNhr+MEMsWPBICBrbDbGes1vkIP1chhsrrGzT8Si9cagJVhgxrti cTl+Xn02qcY0E5LA5nCskO1AHHMDvtr9QSzI+vQmYIVhe1BA0FoICQC6tWuF+2w4x2YY 4Np866HKweWhylhMCGI44RmBYJd8Iuoggjo2crCXNbDyyzTxSGKsFR/7k9W2d1p+eaVE oKxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248943; x=1709853743; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=A+Ju3+5NsQ7zqLi78mVdBh82QtKzTZZzrfFOA+rFDFs=; b=j0gDL5+PtIVCt/96NcWoZlxtTaGKY2LojdvwJLxIJCr1sQa5ZVbzJdn3Kap/jQlvz6 SnQxIgUUz+rqvr7fO+Cyu65owC4kbtMEN5RdWuW4SwsrVr2GihVzznHlrAeSQRbvTZlM Xl83MSMCMGGz81tiYncFn/qJvYQOoz7ZNdngAQXUQJPod3BzPqpajVYZDygR5Dy7Cu9X luj8qODNACsyuZlNJlfHeNWR5OVUkqmotZAKTThYpYMMB7k+dp7pHkv0eIY2JI6ddQM8 A7k/Mhh6TM30HGxE+iCmTJeo6jcOB/PTekeY/xmaG+XRPN7R/38GyUPck+NfrEzPS/Rw KgIA== X-Forwarded-Encrypted: i=1; AJvYcCXV69hh+U7YSHhvGHvFfND8q01VhnRDAPK01vJTuZujbsjurd48P7M2476L8z9mhL4O9sZlUzL7W25qLjc5ouR4C2Y= X-Gm-Message-State: AOJu0YxJroozqPFQZt+Zp5Bs7jOXz/C1mgehA9Tbk4goga+NW4ASaOgX qIWAoYgmD4g9G+CCbH9JiPqqlUhMgSHa22BY73sYGtgiQ5aG1QdpChjYquAPWDE= X-Google-Smtp-Source: AGHT+IHzbXWkwDmlfZ/eL4jDsB3ccQ2hOxgMwKxCvl8MiEGc4S4tbeeGWY8zmXhmQ74ouvbz4gXZvw== X-Received: by 2002:a05:6808:1447:b0:3c1:b28b:76f0 with SMTP id x7-20020a056808144700b003c1b28b76f0mr91074oiv.9.1709248943714; Thu, 29 Feb 2024 15:22:23 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:23 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 10/13] riscv: mm: Use a fixed layout for the MM context ID Date: Thu, 29 Feb 2024 15:21:51 -0800 Message-ID: <20240229232211.161961-11-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8335F80017 X-Rspam-User: X-Stat-Signature: xdk5onamiund4y8kpuqcg7wqc79tzdao X-Rspamd-Server: rspam01 X-HE-Tag: 1709248944-488602 X-HE-Meta: U2FsdGVkX1+tzxXGPNe7e9krqUKlnfQS5naQY41RgpD9DWmRzwVwmDmh1kw0503TfXmV7j0uAYAfeu3PHjABwiTvHwEyiHGEktofGzYhC1E24g3eQbdWdE4glbZjflDJKW+F84fHEdnZpY/27xXaOe4023bLHs1/46WzFcdaYND1OMPIL5TWAYZjbaWd1N1r5QoblUroju/5Xufy3ZU45KUAjTKImhjZPaWsVxWtbVsirOYjqJAAbd6z/2y68B+uBzBJy6XjMikHp7McxLlzCjq/yc5UVBuV6yeMcxUMqZ5u0dr+MR1NFUtK7USe4r+K2NUlwh6lEGEdjmBwE9v+zhPcAzEYX40QvbrVkZ1tp37vYsqv8XcOW33/NY94AkPk+i4PdfAq7++WxYniSzTQgZQ5LfRJe6HdhInT3VZQTTE5kBkYmpkoEy3cHfkpc+k0bOBXblnARBCPcRWEGu/QEYjZMsAmy+DpiDx2mR3RA1+j8gh5uMNsj5PBCD2ch72wlK4thtLwpLGX5aAPCUxLmzicrLxtBZFsn4hjBCYSXKkkqs95yyuToFsWtSaJ3kbT5OVS3e0AUXqt2D4bPF5l5fYLvw86XwCcWJWA9InlCT6CPoPWA0I5fwhfzcBkh7DUhLyjHzrfPZxKAAY6PTM1YCc2EElrFYL73CkyXsFRJMe4XQ+sRM6CBuF94J4LysyZI10aGPfuT2h5YUkSj+mmUetRbQaEuKFa/2zVqj4PhDbUsc4mb3uD3cXzGYPtfXUw1Uih50BLHikDz/+4L9SR+bfTC8Tc/uEx5bnLaMxxmyH29kRMzQJ5DuB7fqqvSAVvXkfYHMAAai8NpSbhlykThS6jLBDRYkisvge/ZlMyHnrksrY3Tv9fZkLa+yzoP9B4p9uUfR1DvbzdwTkry6Gsw7K9PGse4lyzNX2CKWcxUtvcOX0lxClVXczzKmCdv+VmqWv26aOKjTombj7r4c1 cCOGhfA4 YIDhbUMoOXq1G+PqfdIczibF1qqmJkSQdLnYlCsagM43E5M3rRz50+qWHc8NAxqwRxwXYBANdRQK40fwx5tSVg3wNPZEfVABRQD9xbmMcJ3ngcZeEsYuu4vNMRYEHVgwXiIrtqPz0CXUwpnuazfijX9vRMSIYz8pFrPj1fTcuWDazJ8L7qQ8K84EK6Xqev0xUK2GuHVF2kFq9gVT9+rVg16LqRBMipCiLVedJ8HGdAAgwF6TcHZHfpSzTerX3OCBfdCYmBD/VEWmST/FtOKo2B/BKAei9VzeVQB1X1CVjt/lj0AAOxu2ul5hEwtWCFiGVKChfI/uaC/emuXXA0BVHszjHOg1Ery7uhBM4NnUdkcS8fta0RVQOsJHR5Iv5OxG+ljFVd2WlSU3nYW03FeRGmS2QhkqaqgACehGNWtAb/oOkYO9h9rRkIJJ3K9y4P5H8Gv149m1uIhOinpw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, the size of the ASID field in the MM context ID dynamically depends on the number of hardware-supported ASID bits. This requires reading a global variable to extract either field from the context ID. Instead, allocate the maximum possible number of bits to the ASID field, so the layout of the context ID is known at compile-time. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/include/asm/mmu.h | 4 ++-- arch/riscv/include/asm/tlbflush.h | 2 -- arch/riscv/mm/context.c | 6 ++---- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/arch/riscv/include/asm/mmu.h b/arch/riscv/include/asm/mmu.h index a550fbf770be..dc0273f7905f 100644 --- a/arch/riscv/include/asm/mmu.h +++ b/arch/riscv/include/asm/mmu.h @@ -26,8 +26,8 @@ typedef struct { #endif } mm_context_t; -#define cntx2asid(cntx) ((cntx) & asid_mask) -#define cntx2version(cntx) ((cntx) & ~asid_mask) +#define cntx2asid(cntx) ((cntx) & SATP_ASID_MASK) +#define cntx2version(cntx) ((cntx) & ~SATP_ASID_MASK) void __init create_pgd_mapping(pgd_t *pgdp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, pgprot_t prot); diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 8e329721375b..72e559934952 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -15,8 +15,6 @@ #define FLUSH_TLB_NO_ASID ((unsigned long)-1) #ifdef CONFIG_MMU -extern unsigned long asid_mask; - static inline void local_flush_tlb_all(void) { __asm__ __volatile__ ("sfence.vma" : : : "memory"); diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index b562b3c44487..5315af06cd4d 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -22,7 +22,6 @@ DEFINE_STATIC_KEY_FALSE(use_asid_allocator); static unsigned long asid_bits; static unsigned long num_asids; -unsigned long asid_mask; static atomic_long_t current_version; @@ -128,7 +127,7 @@ static unsigned long __new_context(struct mm_struct *mm) goto set_asid; /* We're out of ASIDs, so increment current_version */ - ver = atomic_long_add_return_relaxed(num_asids, ¤t_version); + ver = atomic_long_add_return_relaxed(BIT(SATP_ASID_BITS), ¤t_version); /* Flush everything */ __flush_context(); @@ -247,7 +246,6 @@ static int __init asids_init(void) /* Pre-compute ASID details */ if (asid_bits) { num_asids = 1 << asid_bits; - asid_mask = num_asids - 1; } /* @@ -255,7 +253,7 @@ static int __init asids_init(void) * at-least twice more than CPUs */ if (num_asids > (2 * num_possible_cpus())) { - atomic_long_set(¤t_version, num_asids); + atomic_long_set(¤t_version, BIT(SATP_ASID_BITS)); context_asid_map = bitmap_zalloc(num_asids, GFP_KERNEL); if (!context_asid_map) From patchwork Thu Feb 29 23:21:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577716 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C526C54798 for ; Thu, 29 Feb 2024 23:22:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 333EB6B00A4; Thu, 29 Feb 2024 18:22:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 21F13940007; Thu, 29 Feb 2024 18:22:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F3B776B00A6; Thu, 29 Feb 2024 18:22:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id DD63E6B00A4 for ; Thu, 29 Feb 2024 18:22:27 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A979CA150F for ; Thu, 29 Feb 2024 23:22:27 +0000 (UTC) X-FDA: 81846417534.17.64E27BF Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf13.hostedemail.com (Postfix) with ESMTP id D92B720023 for ; Thu, 29 Feb 2024 23:22:25 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=TtrUygqN; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf13.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709248945; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4wFLc5gcnBs285NRnicKVmmQVV53EnA/2rANcUM2uCA=; b=apzyWXXjs/KkrrVNs2LDCt+HJBFM64Tc+ltRFp7A9rzMRzDfL7xpF/5uJ2KBkbfQPAta+O GT955v74tF0d0YirnnAqFN0jWo+Z1elDMOK3ZlvPM/yim8Mgx3LHrYfqn4hCz9TIkN9EXU s1xEaHtUwb/e4yW2Kqku6TOHiCrhyF4= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=TtrUygqN; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf13.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709248946; a=rsa-sha256; cv=none; b=wUKFbDQnAaP2078UGBBM728n4eIa9mDF7DdsBTK5PUk+gxB11Yy57XeRGUuld14VN0Tac3 MaQBG/Xv6CZkusH+aEmbXjZJgsgbnyj4c6bO/FJmSwcYquY7a55pmnDJ84/fCZYETgANgi 7unKZI7pkusnNHXvimW3P4QbrvnvpaM= Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-6e4d48a5823so1324395b3a.1 for ; Thu, 29 Feb 2024 15:22:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248945; x=1709853745; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4wFLc5gcnBs285NRnicKVmmQVV53EnA/2rANcUM2uCA=; b=TtrUygqNWZ1rmdU0qJeXKlS0N5ZyazoMJEq9ukCYmHrSZpw6Gxmlbu/2kg2PAYuF7u MVpRGbp20jOQJhxKJQL5Ku423A0OoV9EiASqZbNz2VmRW3ITtEDBVx7i2L0LgVItwQJO woWQYSdN023p0k++v/sh9ebYLJN3ipZzgD6PYUjjFyFbwDTuH7DHspatP8GF15JjuK72 ADOoPX/p6wi6MqIKO+YRiTCX/hNX5hhmOyWDtLiRrWNJU2UpaUVryBtxno/vup34X69/ KX3tG++VCAVins3dP4NFXYaf+ZEFDfvgJZLJQYm0U2saEw+a+0koE2WSV4U53Gcmi5SG PJlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248945; x=1709853745; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4wFLc5gcnBs285NRnicKVmmQVV53EnA/2rANcUM2uCA=; b=PleEr2eR83NEzE+EApCD6crF2lCW69Pa+sVCeNldIckfEnF7dErw9oLvPwyRkLhPGf wCjZBBaoz0Fcrq70RR/nCXw1ZlnhUfn6NiaktUTn16wHVDCYwvYNREL5WVJ9nOjJn8et xcq2ry/pJYCEYMzx5V0h2sjmX/FGV4ZV95PZ6dgmpNW8xy4wWOh/tJ17F2QFAf6drPGS 76kkoJb6/t+6cDr2s5wkVaGyT497ZQXteo/CEDslI8Kt8bYJqPBArV4r7BSv8fHVK8wz tLrN6t+vhHwUFbNfulj//SQGKXXXGs6ni2o1NvnMvht+IN3ygbri5fAisxGnYAZF+MPD vJzw== X-Forwarded-Encrypted: i=1; AJvYcCWZ7JBMqlyNaiQurQPGFUYWlsnBagKi+rRSQ1MTOw5pdSDaw/qoiEBzQUTFrllgsv/bkrX6/qqy548XMhBp0y6QOW0= X-Gm-Message-State: AOJu0YzEGF8u7z31Qx9TgQexy1GiuPOvcRGmghzNuKmgYqvPeUGNxdnu kR0G3YgFGKlOSB1/00KZmn4BHppGyLAsO6qRCjnrwBzcZnXTEMZwU2kvDD6LY5E= X-Google-Smtp-Source: AGHT+IHQJB+ROn7vFDGIQNWMDCSQS4WpsKXvgO9ytwgYp+s+V0Upk9jrJAXqE3yMlwAjPHhZUdgCcg== X-Received: by 2002:a05:6a00:3d42:b0:6e4:eee0:5798 with SMTP id lp2-20020a056a003d4200b006e4eee05798mr179957pfb.14.1709248944849; Thu, 29 Feb 2024 15:22:24 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:24 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 11/13] riscv: mm: Make asid_bits a local variable Date: Thu, 29 Feb 2024 15:21:52 -0800 Message-ID: <20240229232211.161961-12-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: D92B720023 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: m7513s4ta95zemffr6nc8gw3npgxy9fp X-HE-Tag: 1709248945-562721 X-HE-Meta: U2FsdGVkX1+LNKzCvxq71lsQ+XTApnTb0p3NCD3TNm8rLmPX0yOCdu3DtsTyGOS150wuuLZTnVnjxoBGQmbxgeqHKU2cFb6EoQyz/cgh8V5tc1YWu6jkHVkCgXuwXGGlnCOQChakA9iX7MLJrk796QX6ZKk3OmU8wpA08GpLkRmSdL/0qx+x9C6ExsAcnFX7yKPbAc8D9w2bEyfxU90UH/3C1U045K7Bh2eUKY3yMYxF7Zs1kpQr6WGuyZ3XrcUdsrxXDzispGEFgNAc2X5CDoiJPeVhWQp1iaP1clnyjQZf3/cCLYmwdjO9NMvHPgVWEaH8CqKUJyCIcu7O2fWz8YWHgDXBMWYLuDq9e4omS5KNdhLmoj9nTC2eOJLNAE1RvS90N4DfyyV7/vaBISaVx6JbXOF3FDB1SB0yo76eYa8KLVNqR9rQOJEdp7/6Q35ft0bWpWGCSf07KU9rmldm8WdSJnqLeL+yCP0y0+8GaARwkLrP/ekT7MqDfyPCrjaxLtMMehyYpZl1DUA4yeiGRn6R2bNkztjwaeDU3vJKO1j3OzW2+1qm1V7fUBQQA0jlw5puEbgIPyC820vX9YaJ2CGNEbZAtwlA8rhjRCbUsvXkcxbwf/37zeVKBiFCqwk2qyHPCjaPZEKCSWYLwemmSInrsKG38w3DMe/1UDsy7tuzVZjSWCKk7Ix8HudNylORiTVp/L8jZ+/QW5JwnPLZyC0T4z8h9V3zVdghvy9QKSAvty5zgOwHKDWLV57Wtvra+QGRR6ybo13yRYXF5donWnw+2moaOGGRZNtJRkKlOuuylsBtypTQeXaNLddtFc98NAYd8k6hc+IgXthBIdCJeRd2rzZKEc6JiDIKT/m+P3OHI5TzHkdtfKlSWf97+070EgVXlUsBKWkjBpiHX4iCa+7/vvHOMw46wodrq6qTKvknoTivp9vSiqZNElgyFNcOf25Uqb0GV+u9IE+v7Fx DpyIplEn 87Z18SI7ZHRz0kLPK+jZv2F7SBE710kpOT8dLMugZicKr9EFkjD2tTwFpAj+gDBRKBatTEy7gfjq+9EJFkAHDj35m64wJpIOuAWhGktUJ4hFx2iPEpmZYnTFE6h93o70YH5QV6eUjvv+ND69EcJaL36JAH70t9cDaAPDP X-Bogosity: Ham, tests=bogofilter, spamicity=0.022934, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This variable is only used inside asids_init(). Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/mm/context.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 5315af06cd4d..0bf6d0070a14 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -20,7 +20,6 @@ DEFINE_STATIC_KEY_FALSE(use_asid_allocator); -static unsigned long asid_bits; static unsigned long num_asids; static atomic_long_t current_version; @@ -226,7 +225,7 @@ static inline void set_mm(struct mm_struct *prev, static int __init asids_init(void) { - unsigned long old; + unsigned long asid_bits, old; /* Figure-out number of ASID bits in HW */ old = csr_read(CSR_SATP); From patchwork Thu Feb 29 23:21:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577717 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0A11C54E49 for ; Thu, 29 Feb 2024 23:22:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9E66A940008; Thu, 29 Feb 2024 18:22:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 96EB1940007; Thu, 29 Feb 2024 18:22:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E848940008; Thu, 29 Feb 2024 18:22:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6597C940007 for ; Thu, 29 Feb 2024 18:22:28 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 49339A150F for ; Thu, 29 Feb 2024 23:22:28 +0000 (UTC) X-FDA: 81846417576.09.DCE2790 Received: from mail-oi1-f176.google.com (mail-oi1-f176.google.com [209.85.167.176]) by imf19.hostedemail.com (Postfix) with ESMTP id A05331A000B for ; Thu, 29 Feb 2024 23:22:26 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="eHG5bwA/"; spf=pass (imf19.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.167.176 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709248946; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hwpkv2OuO7YHUplKH98oFVfOa+CFbclxXNu0ysmduyU=; b=xKA9g+j/ocbn1C/5HG9gh/SWijsALu7iPj3AmeT0yjNIieO3Eyabt6FxjMLfknqvZmGjAj 4iLYuLyBTtZoVmAyX7d/bdwE99MmVfvSjlBj9eM99xYqmGW9TUilUhPAy+AFWMbOBllRgl LsbLDFtV9DYqVELVkDi2lIX/5CxaEZE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709248946; a=rsa-sha256; cv=none; b=hE5BHxrBtTOhaiRIpd0Gl/D+vvR01FnZhIXKLHM8bsf4Plo1xKFseT5NPhFWa53+EfkIrr UuMouVAN3l9mPZiQ2T3qCQUap1VqRlpwNwKgEvFLtPDINmbMyRy4cgUXen+6I5ae0Svufm uCjzmpaGXJVlI5YcSkG/1RHr8wPa1rQ= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="eHG5bwA/"; spf=pass (imf19.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.167.176 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com Received: by mail-oi1-f176.google.com with SMTP id 5614622812f47-3c19b79535aso860192b6e.0 for ; Thu, 29 Feb 2024 15:22:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248946; x=1709853746; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hwpkv2OuO7YHUplKH98oFVfOa+CFbclxXNu0ysmduyU=; b=eHG5bwA//EH8wWzWFmsKFQa7ELWJTngh7Qftet0lVFrGghHhW1kOLWmkUJz0XVI50E H8xpg4Hk6avyfCdqQZyMLPLHonhOsy+PIEusgMUK/cARUdDzt82WySsBKK6vOdzmIBDU nrdW/kv/d3AH5Dk8pds2zHY50VOdY9lhEQ60FFestma0eTM0ihEQtPrYGRWbuqo3B4ds 8bJBb1GE/Mxzq43FswtpY4l8SPnoKBj81aA6CAJF58yBgFVIZrkB4+OMoainyLrpD1Mv 0ghShYhSQ1SvLWpMHEpvYYNAqMZ1ecJJ6QEnFFo6v3/whT3UAe6Ga8KnNofgrOIEVR8K DMWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248946; x=1709853746; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hwpkv2OuO7YHUplKH98oFVfOa+CFbclxXNu0ysmduyU=; b=exh7V26OIfkm5ZkxSyNzipPD15kTETvOrvG2g1s/tfjd+lV2YPBHoJzj51DN2IVgcM GtmD8VuxjEwws8e7r2xdF95/5HVlaF3c/hFNhZSfpg9sMg3Pv+W6ju5O1xSK5UGhwCHF lk0g+gJiz0P/DtDPDyAj+wQKrEAZyBitBrLXmC3x7VSusKBDCNfeRhzDFQQe3xjUD1ts BGtMdO/N30mVzYY9lfuPgOzVKaG0CiodDQ7W18duNcVFAOxvfMHteGs40bY2YAL/Nhtj IP/+p+YtU1qsfxCriE3TtW1Wi9H/d/vJykLZIso3n7aZffVKZUqLvwO4z9xo9z2rY5dq zEcA== X-Forwarded-Encrypted: i=1; AJvYcCXnDcTIjI3Qu1vELa/1EC2wcMJn01Fwm8zdPmx93perhuW8VCBfcxgMvWh3gxrXCgOCovJA4N6H+vELXfQDFXn3tIo= X-Gm-Message-State: AOJu0Yx9HVOdaLkt+hZ9XVfUq2dj/3HPPSMr7ZUW5boNiv6dTjB0FndJ 74ly4vlAsoW6mnN/PhkakOTonJAz0Qar74owWbPptYhFbgYO5s4Tep/KMsrEguJB46zSUZvs8jv A X-Google-Smtp-Source: AGHT+IHmmt9lzZLUCIFAbm5GqyPJaePE9QFXJ4x/3NAo+fPZmowQvwtei5P759brqGkqtPXCOnb8SQ== X-Received: by 2002:a05:6808:17a7:b0:3c1:af49:222b with SMTP id bg39-20020a05680817a700b003c1af49222bmr47209oib.37.1709248945851; Thu, 29 Feb 2024 15:22:25 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:25 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 12/13] riscv: mm: Preserve global TLB entries when switching contexts Date: Thu, 29 Feb 2024 15:21:53 -0800 Message-ID: <20240229232211.161961-13-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: A05331A000B X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: x5561prkgzppysaq7rwk9p6eu71z7brh X-HE-Tag: 1709248946-281202 X-HE-Meta: U2FsdGVkX19RaDRzoXhYqswiEsxaFN9oKVj0MbiVA6vn1Yd0xgrgWxc4S7VRKZVROnVU9fWSJmMfI+j05hUAI+TCWHrIvBfga+CYCknzc8r1/E4Nfvp0deVdc2poCUmIHdGukzFZ7abXynmlT8jpaZCtwkUMXhVm7nalGaRVrf+V1zYdyMjm7XQo1YT8WrfSssfnvR/hwqs8pi7GS8pUk7gOZHztztoXesqwjguGG7Dwl6MtEpNvqBgWNRlt5q64jHK3HF7a0OAssn/TNOdD9pPKn2KhSjtDD5Y8B52EuKBoIUAuG/BRmx8F0idh4gVpSVh7CdBS9CjACivSpFZnD0/nWJh4hWPkdjmKVbTpndwNzT7fEEuvtEjRZ6U0HLOu7mkiXAVPjSGZ7X/gc4kM3g/02oYqtYkdl2XGwiQkrBgAcnOMYQmQu94bpW0zoZpjQekIdNWj604OUI9D98q6VdFMpXeAdudmfdT07K+ggIG/isubzqs4yl+rEEJ9Rg/LCRhiqpeTuYIAzUrKwYbTiN6tSRovWtWcYQqJ95GQnH+x3ZOoG1sEyRKcJI1O5JorSxDTyvyW14uAjT7vBBuaOmBtKsSBvREtT+tS6v5QvryaoY4L+fY1iP9QYwE3ACsZa1fIFy9RlEmiutUXfscAN3sosKrWSf3rSMY0Qwkd8Gn2IA95iMx3jBz4ihOKfDcy1cG5HA8jK2ohGraW2aZj6cQp0rgN2auXqlvgXVLAHn3N7bx5dNU6rJxZdYuwmy5P8ix1sdF1CQDJdwC83JaWLaXEYLiBwZiy/6hqAOKF6xyPvEYbLL1vPGpeLxRjJ8zGzxizHuZjMHFofAb/NG5HIDwkedxpZ+riIxomC3E5qqXqHOOFBls7+k3nW282kC2n8I9fKFGoDUnXeUIi6SIALdPH8B7a8TXukokM7LnBXlYhau8kXG+PEyU/7/qPZWoHUse3pNiMwc9AaqTviLg EyWOSXsX 858OEcHWOb/sdK+a4I5ViYlVMJJdorNZMa1MnoyFDXCBHZxKlw0Aj6KWNz5CRHNq7ndjBdDZGSSrsFS9qY1oasDKRS7MlzYvf6aNDYfBfbhumSqQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000004, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If the CPU does not support multiple ASIDs, all MM contexts use ASID 0. In this case, it is still beneficial to flush the TLB by ASID, as the single-ASID variant of the sfence.vma instruction preserves TLB entries for global (kernel) pages. This optimization is recommended by the RISC-V privileged specification: If the implementation does not provide ASIDs, or software chooses to always use ASID 0, then after every satp write, software should execute SFENCE.VMA with rs1=x0. In the common case that no global translations have been modified, rs2 should be set to a register other than x0 but which contains the value zero, so that global translations are not flushed. It is not possible to apply this optimization when using the ASID allocator, because that code must flush the TLB for all ASIDs at once when incrementing the version number. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/mm/context.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 0bf6d0070a14..60cb0b82240e 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -200,7 +200,7 @@ static void set_mm_noasid(struct mm_struct *mm) { /* Switch the page table and blindly nuke entire local TLB */ csr_write(CSR_SATP, virt_to_pfn(mm->pgd) | satp_mode); - local_flush_tlb_all(); + local_flush_tlb_all_asid(0); } static inline void set_mm(struct mm_struct *prev, From patchwork Thu Feb 29 23:21:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577718 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C3BFC54E49 for ; Thu, 29 Feb 2024 23:22:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2219A940009; Thu, 29 Feb 2024 18:22:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 180C2940007; Thu, 29 Feb 2024 18:22:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F17D8940009; Thu, 29 Feb 2024 18:22:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id DA69C940007 for ; Thu, 29 Feb 2024 18:22:29 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id BD1D11A0140 for ; Thu, 29 Feb 2024 23:22:29 +0000 (UTC) X-FDA: 81846417618.23.D24B442 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf27.hostedemail.com (Postfix) with ESMTP id DCDDE40020 for ; Thu, 29 Feb 2024 23:22:27 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=kS8t+75j; spf=pass (imf27.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709248947; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cJvv9umwOkXOwc0iOvLk+NQYD7Mm5kACvwQaK1YGZP0=; b=1u3oFHauuxjXQo5c6td7E3RIWTEAdcKbzbG4mitYf8jL6ZcT2TpT55EEpkAq6q/PCHT9ux 9d9zugNLI7qG5c2Gz/luB4gsevOU3/t8L5XJ1k8rHNhh13Wldkp2262NM38qHIzvAkoXMS SPMHlsFNBGRnBYVB07cL3QiWdsMor48= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709248947; a=rsa-sha256; cv=none; b=EfDwDbOIbHYrHTPq8B8AbGrTTHgTGqWgtL4T1hXuB590F+MEZ1dfkv+sze6uTCdTnnqtM+ YBSPUmA+jew5K5fuLWYwHeEkzozqZzo7ljL43VrmO8s297C4mQ4A2/3c6r+hLBB821ApYO rJyMKISVOwzncI4XZBUnEoEIvS8CXRQ= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=kS8t+75j; spf=pass (imf27.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-6e581ba25e1so1204950b3a.1 for ; Thu, 29 Feb 2024 15:22:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248947; x=1709853747; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cJvv9umwOkXOwc0iOvLk+NQYD7Mm5kACvwQaK1YGZP0=; b=kS8t+75je4RPkBs5zXOxP9gfTb4B3HKQuSdYR8QnSdUHBfUssoSlmaGN9DoDEt4n97 f10lX39RZdmcuLdSwP1OdE7py2APtCMEsvUV1BI558Mjx4FL6UpQZHM4/O7Ujp83csKL FTVgz/E6KhA+KjYNEt4JqyxHt3Tz7R7ZMESAM67AWEXXHSLh3xQ+gGO0feLuYjjTF6o9 fWGtKe5V5c86Fbreezzcuwb/L3/exI3FOpTncdVKI4L6ybcCyqdf7hCJSUH79XuU8K75 fau2JRmMzQtGz1ioas9aW6569Tf5miGsRYbGvuChg/EicNiSVls4ZcEVvR73ml+gGyDf vUaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248947; x=1709853747; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cJvv9umwOkXOwc0iOvLk+NQYD7Mm5kACvwQaK1YGZP0=; b=FuGZnNGxdQLkKMx1Vn9vCmqaDav9xMFXl6iH/fW/6nhvKUWqwJRaJNZhno0/JxEyf8 AOJKteNXtlkNLu6qsJeWob1hlL6uBxv5AakzdvXjVO9jwS3y1m/Iz8F2SXDMkXtaNPm7 Tknx+7gNPCcm4se2HKnvgDteXAEHbeTex6oEFA68ElzM4eqjzToTcN3dXk79ktuDKPMP DorXh6soBW3WeTiFTeA4ctRIolUBTTZn+IBFcJtiBwfep0cxrJRp6hmC1b28MrawKrGL O0VrUpNL9C/8wUYwYFxnUS1cBi3ZYcLYFwv1ENo1BSaG7kJO/8Sc4YIddlaxQEslyaBk sfHw== X-Forwarded-Encrypted: i=1; AJvYcCXMMW7uqdAuozNMM09OI172Rbm7KVw3ZwPqy0iC/zJjZvwD9/1r8gMNSENKqQytUOuBSZuk1OzjE+Cxrz7pLAqR3vA= X-Gm-Message-State: AOJu0YzZcJ03eTWQQjbxR4TrfzyZTJfkVoKGxmI9RhxlhTQRIrYjlD3m 0IYzkhT2SmTluu7Y4vCugDpMLw8GuwHb5zNNFQOIafI27FtCK/6NCKPZjwBLVrQ= X-Google-Smtp-Source: AGHT+IG000bMn11OdL3LH1zGovTy9Qg+UUgYE8CMrW4h6bNv2nzitQBp1dzsEHx4shrcmNE8zaAVpw== X-Received: by 2002:a05:6a00:27a3:b0:6e5:3dea:bf51 with SMTP id bd35-20020a056a0027a300b006e53deabf51mr562992pfb.7.1709248946831; Thu, 29 Feb 2024 15:22:26 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:26 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 13/13] riscv: mm: Always use an ASID to flush mm contexts Date: Thu, 29 Feb 2024 15:21:54 -0800 Message-ID: <20240229232211.161961-14-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Stat-Signature: a35zz3z1ppqyto7nxkpbxe1cn6potrkg X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: DCDDE40020 X-Rspam-User: X-HE-Tag: 1709248947-950160 X-HE-Meta: U2FsdGVkX1/jE03F0ExXoRONqr6Jk/K0OgEQ0S92rMsaCrB9zxQKcQ9EURIZHpI9cuZac7TsxVyjTvw7KsVYr0qZ8M6RSdmTPChMftIlXnAbuEYQilqXExOmn25O/rT0hxz2XyikScFZdp31Q2PRDa/7eDVR99wyRDEnwwirUALBlule8XPFKvWSzmI6pLzgYywa0AqzM27zd094H/QC2B0ubp+dUMrWIN+fLHRYx1bxzbm2zrviWREc9MsTq2+tDLhadL77IpNaZczVSEXII5Zqcw2J10H0jIc4ORGRQ4u4qpKbXiQ7O6EBUw5l4QzvqxjnoyadCmbOB5G6Ih2zKKFJYnDAbfOdr/MbGzyx22Y/5pqxFbXgKHeQXosO5afPGj43Z0VeYN8Fyvh5UAGVor/Dqq8U2keVYAKH97rVgUvVfUv+mG1Xr9QJ+9Ig4SvC9+NNNGAXTI2BayJfrXwqlGsm/klThsPnQGOPEsqlmaVJoB/Vz9Aw2Jt6+JsSzV8Nm6GJh8kNvHbbvdCwba9J2H1la4Z8zJ+ihan1NbGdmOfZ5/ZyfNAkBLPc5Ae1NFWn/oVr/zn6ldcz1HHU1KCj4bV5JapXivLJ7e5IvfHhSvn5uTAF2F4ZNXNDL4v+Y3Q6qoJ20Mw+9x6BFdb/AMcpNJuAc9TcxDxNRe/Fs0A4FNOQbi6Xz/mMGZuKmFocniUcxuM4jFrtYrCKF0rsGBb0e12W/6mVJ2Had8O2Us/y3T1naexUKt7qCEl6jwifz4xSjqESCnDaKquagQ7eXxGENBv/+hnxw/YmGeCQ6b9M9myI6FL2wkZ4xndWFUwF1LVi8pSNVw+f8ft3PQIIpiP98Q9vc835yDq5jX0KFRMoSLueuCEDh8y4mm+FkRi9MZrHSPofe2hRpGd0geY9W9egk0Ag2J51bOgKHeH9X69e7hebVdyuL3cPnghWUqcsUxh2NpSVGQ02+DkQrDHqL/H QVAQ0HHA ak1Z4n5h9r+w5DGhVlYPOS3yb4ccjHq0xXn7ZIm0M/FMfth7V+3Z79kl4QpeYV0Y7S5m/beiM3yEEUmkUdjsjNCjD/Vtuf/7kPtHzszsmHU1VU50iajQ7nZggyThx5sGQC2ZmyoMZhCFTqE9P9pbIWzSAJkX6O5JUOHEmmQ6Ieldabzi+GVjqZkDNedSLXw/HSyabu9uEJnwMZgvKqPj6HMgHcENEHJOhJHP/bAhU7xowHr3ZA9KKKFMk0ueXRQwTrQWgdCoRla/2y3xEenr9KY7VyhRCRpMbeAdqem3fwt69CjuboLUG6cbwRXpXpdC2iuYbBM+6tJEnewQpHDt31MlTbS10gbsOgijhoFiBgM/vd6vUQyiqLcsqHqK+LK1CtrbebjxdjVMgn4ruLVwdj4hWS/zGpaLbaE3QfPt85iRFd5LoT4iSo2rkCw5I8Xw13YYW3fXssl4aK5o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000234, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Even if multiple ASIDs are not supported, using the single-ASID variant of the sfence.vma instruction preserves TLB entries for global (kernel) pages. So it is always more efficient to use the single-ASID code path. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- Changes in v5: - Leave use_asid_allocator declared in asm/mmu_context.h Changes in v4: - There is now only one copy of __flush_tlb_range() Changes in v2: - Update both copies of __flush_tlb_range() arch/riscv/mm/tlbflush.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index e194e14e5b2b..5b473588a985 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -108,8 +108,7 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, static inline unsigned long get_mm_asid(struct mm_struct *mm) { - return static_branch_unlikely(&use_asid_allocator) ? - cntx2asid(atomic_long_read(&mm->context.id)) : FLUSH_TLB_NO_ASID; + return cntx2asid(atomic_long_read(&mm->context.id)); } void flush_tlb_mm(struct mm_struct *mm)