From patchwork Thu Jan 16 02:30:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rik van Riel X-Patchwork-Id: 13941174 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 114F2C02180 for ; Thu, 16 Jan 2025 03:05:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 950D36B0082; Wed, 15 Jan 2025 22:05:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 900F86B0085; Wed, 15 Jan 2025 22:05:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7EF736B0088; Wed, 15 Jan 2025 22:05:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6211D6B0082 for ; Wed, 15 Jan 2025 22:05:39 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 05546C0671 for ; Thu, 16 Jan 2025 03:05:38 +0000 (UTC) X-FDA: 83011824798.25.1BE45E0 Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) by imf22.hostedemail.com (Postfix) with ESMTP id 61109C0008 for ; Thu, 16 Jan 2025 03:05:37 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf22.hostedemail.com: domain of riel@shelob.surriel.com designates 96.67.55.147 as permitted sender) smtp.mailfrom=riel@shelob.surriel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736996737; a=rsa-sha256; cv=none; b=0AiTPzcUaZ3lyOcDtC5aFBf39fGN3qqk/QlJ3PfDS2vEQsPpmn+6n3LaiWhd4N01hsPgoG N2+jLGzJ5wgC9BnRQ5jnLpN4eekbpCzTmuTHYgNX3PaSJmulN/wW9mVcCuFPnqTLTjc8VX aIfXMVpGBuFV9BM9dTxIZ9h3KEyL+R8= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf22.hostedemail.com: domain of riel@shelob.surriel.com designates 96.67.55.147 as permitted sender) smtp.mailfrom=riel@shelob.surriel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736996737; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=orlGaKw9nqEENaNpZxcL9zaVEFoWFuuRr5656SZ4f60=; b=oT3TSlRxQ3vK5mebSsb2NpN8CCB0ASGCfh0dyZ+sQnafvLFnhKicqtiz+rc7IB6v5CZu2q 111Y1PhkO5JlOmm6w47GOAHeYOrf691tBuy0k8R2/l8E9l94lN6KSu2gL3atMFuIs/uTzj FacdxBbOb77lZttUM2mHJtIBBWNy1o4= Received: from fangorn.home.surriel.com ([10.0.13.7]) by shelob.surriel.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.97.1) (envelope-from ) id 1tYFfO-000000003nd-3VsE; Wed, 15 Jan 2025 21:31:30 -0500 From: Rik van Riel To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, bp@alien8.de, peterz@infradead.org, dave.hansen@linux.intel.com, zhengqi.arch@bytedance.com, nadav.amit@gmail.com, thomas.lendacky@amd.com, kernel-team@meta.com, linux-mm@kvack.org, akpm@linux-foundation.org, jannh@google.com, mhklinux@outlook.com, andrew.cooper3@citrix.com, Rik van Riel Subject: [PATCH v5 07/12] x86/tlb: use INVLPGB in flush_tlb_all Date: Wed, 15 Jan 2025 21:30:30 -0500 Message-ID: <20250116023127.1531583-8-riel@surriel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250116023127.1531583-1-riel@surriel.com> References: <20250116023127.1531583-1-riel@surriel.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 61109C0008 X-Rspamd-Server: rspam10 X-Stat-Signature: 4t8y3xg5akxzimr94dzrit3wfog7kaq8 X-HE-Tag: 1736996737-101135 X-HE-Meta: U2FsdGVkX1/ZV7tQczOCOAoCMX6WYx1F9+B+rMB5MPY6wXK9M2OFKyQ/4X9L53uQkIA6VhX73ywHiaACJNgYTl683TJSvpWt7ULqH++j4Akzkp5SlGERWqsWCd7wztbyBvOKDTZ7En6Tv6J3OFDmmSMyE900UwM2bEc6DM5kKaO5pCQLeu/yM8bd8wX2kARHunSq7Eps+cTcen5uB/7lMbELomoFbMWZjH4NUtuH3SyWELU0r2vu8pbKhoNDhJ2IM7V8swLJ+i4M1GZ0Hsj+v3fTgQJfqIC+yTB5FA1Myxfl6ytuHXtWT29NuT2t62JULNi8nDFfgA7HbeO7O7aQxkwiV+YgPdH9RbpDWR8IFPhZhPqrSAe8JaBsmsDWwtDTchKDD7BWlAbHcjng6yhSTQyc2MYXBeQIfyW6d8D9AC0sfHrFojN2s2HJ68c+8tWZDkTDA2nKRI1UDwwk1krUaELJxP3hgf/qbX7wOVdiVMqzo7YMMP96C/EWaVRBQy4eMALa3xW4XvNriVEbFwzMwWspO8GXBpJ0CKhaVZeZsYr7gpwzdo/e84enZfgKxVzOoz1/CRB4NsFe1xhdls5FSKriyRhLn2oKpc60kFC/jvCcRXZ2NxOaFqxzU7IWI8b1tBjF2ARxatEM+0oh0z5G+Miy1VYDZtHfBMOH9wXtsf4cHsCAx22GnBZ/DPTnPkfy7wjPx7jYT59IO5tgDEA4w0c3pIKdxCKeHvnnpgK6SWzAfCUbvtSjoWs/UWoiQJepVQI5KIod+gffvzymDwVVG0THwu98cWQzTK1ruuq3KVKBdeaK/X918GOvVZNtHr0mUQoEcksusTwloofixqK8umwqI9uLa6YN/WCpDd/O8XmY8ljuhqhuNJmkZsEeLMGZgVpNVl71S1U5FoX0puIG6OHzkRL1vioSLF+ZxoPi+oCllV+18yxvTPM9AngVBaEfPEJmngNkXIGk6sKOurN KeBUk7/+ hMM8wv14nVokU9myTS4urvCU8Br+TUQQV0s/gZQUIElmmqkULytC0BB5NXvhimqSoYjitL1ju79OZ6uug6jjMHJKQ3dCmLEjlhvESZi29lwHFoFcIYs1679HJBTtdyC89dqyo6A6Rrkcoobd4qP5QW6sV5JxNTEMr3sb1D5PqpOh+sakVBxxEqjjuNFFRzN8hY47R6jEgMXWd4jzQaVgE590k/dll3meGwI4edJ0kTFmNtt9E48Zp0YcomidpjZKMFUNyDIQ2aaE7nAIq56sfii9Wp6SPcgCL+dpppqWfkx6VrdpXORgs4gKaGkwVWIC9mHQw2uBgJvtw4m1u/FRBoN7hEAXZO3wuX7HKoNF16+Q+tqZqMv4F6HlKwMbo0JjQsRqmmoqRKL4CNor7i6AERwVH2XeSIB/0BwyXzZbs4YhzyXIFfvXCEzpUOQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The flush_tlb_all() function is not used a whole lot, but we might as well use broadcast TLB flushing there, too. Signed-off-by: Rik van Riel --- arch/x86/mm/tlb.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 0761dd224e84..49b3e90503a3 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -1065,6 +1065,19 @@ void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, } +static bool broadcast_flush_tlb_all(void) +{ + if (!IS_ENABLED(CONFIG_X86_BROADCAST_TLB_FLUSH)) + return false; + + if (!cpu_feature_enabled(X86_FEATURE_INVLPGB)) + return false; + + guard(preempt)(); + invlpgb_flush_all(); + return true; +} + static void do_flush_tlb_all(void *info) { count_vm_tlb_event(NR_TLB_REMOTE_FLUSH_RECEIVED); @@ -1073,6 +1086,8 @@ static void do_flush_tlb_all(void *info) void flush_tlb_all(void) { + if (broadcast_flush_tlb_all()) + return; count_vm_tlb_event(NR_TLB_REMOTE_FLUSH); on_each_cpu(do_flush_tlb_all, NULL, 1); }