From patchwork Fri Jan 10 18:40:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brendan Jackman X-Patchwork-Id: 13935233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BD5BE7719C for ; Fri, 10 Jan 2025 18:41:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 953FA6B00BB; Fri, 10 Jan 2025 13:40:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8D9356B00BD; Fri, 10 Jan 2025 13:40:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6DD376B00BF; Fri, 10 Jan 2025 13:40:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 46AE06B00BB for ; Fri, 10 Jan 2025 13:40:58 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 08A3880DD2 for ; Fri, 10 Jan 2025 18:40:58 +0000 (UTC) X-FDA: 82992408996.29.9AA1D67 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf11.hostedemail.com (Postfix) with ESMTP id 11EE640005 for ; Fri, 10 Jan 2025 18:40:55 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=P0eHEPKA; spf=pass (imf11.hostedemail.com: domain of 3tmmBZwgKCNE6xz79xAy3BB381.zB985AHK-997Ixz7.BE3@flex--jackmanb.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3tmmBZwgKCNE6xz79xAy3BB381.zB985AHK-997Ixz7.BE3@flex--jackmanb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736534456; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2/DwiwuishkWohq1Selw9AVX00i6WMTVrJEAuBvDTaE=; b=XI+xdl3r7RodmZ91CxQG4oB6b8CdMqSo9qHRxtWpwt8eH62OUSEBHFXhaa4UmzF9LiiKt2 kOLHjwwr+nBFewSSmLbHJll17AIpLn3w43XTDJLkVzY2ZS3Fnu9DDm6D5hxOf+SH/itOi9 zw777+xr6pFpUFnNoUEY1MNKVxV+hlk= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=P0eHEPKA; spf=pass (imf11.hostedemail.com: domain of 3tmmBZwgKCNE6xz79xAy3BB381.zB985AHK-997Ixz7.BE3@flex--jackmanb.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3tmmBZwgKCNE6xz79xAy3BB381.zB985AHK-997Ixz7.BE3@flex--jackmanb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736534456; a=rsa-sha256; cv=none; b=R8F0sC+HxtQqBRZ5W52hdj4v5vP2AaPdzF9SwN1sLQUIQMGeW4Ab0I7JOY3lYmh+s+4CiU rcs2mMNzRCCXnzwY17pEUKV2QXtpoPf2bm/h2ZzNNel43SAySdmKzxJI0tWgeTmKzG1aGA aYeQh9BAHHxjI1urph8sUATdpk//7+0= Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-436379713baso11445745e9.2 for ; Fri, 10 Jan 2025 10:40:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736534455; x=1737139255; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=2/DwiwuishkWohq1Selw9AVX00i6WMTVrJEAuBvDTaE=; b=P0eHEPKAP6G6b0C6XnWSXHAhXbUPA4PewuAbnVInejx3n9+nTPUoKOzaqiWZuyQqMt MnjczyN2i5GROJnBQuf9nS40r8DJIu/f1FSMh0Dds3e/AYJ48ktHPt0pKWXqOQaFgpi5 Fzowz45CnADaaBeOXjClM2/ausqCOV7L0tr6pvHtQjqdmtN/GRV+puJDQw9xEPyv25mQ mcF2C2XKYEUpDGuSGHH4n3XyznRHZJsbO8h220bT1vTDLqrKoYu36y577k9b50EQNLKD ya22ell4oVprQZ56rnzjPDCFfeHIRyrEaD2ref1ZEhY0DpUmA2XFcJi86EpREs+qDBmM zTdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736534455; x=1737139255; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2/DwiwuishkWohq1Selw9AVX00i6WMTVrJEAuBvDTaE=; b=WtrL0d0C7pu5UpJEX7Aa4BULTWrdeH2W8ObhZufi+LLUHg5mkoR3nHMauBRdlMQkL0 PRulGNgLAJ0ArlYOG9RCCZ2Nh/FxVeAYoZaMrwqzvnb4GkuxF4+3ttH5bkQg3QR6l8Z7 DUPqB2vS2ZSEnVsikBl6HKKcOU3HUbg0jSyVqomYwu2WuZJ1hvzF4yO9Erelr6xw2WeW fNNvx+y4K/l9FFCddHOqb+USBNa+1MfOGaaZvyLM4vczPf1JTx+5xDgby0QkKzzHhdH2 51QJf5aUnu8iji3KeHRa7HhuPeYiLF5oqQMRrU837suEo+IipE27RB7NAgK8mBWmp2YA dFIA== X-Forwarded-Encrypted: i=1; AJvYcCUAyUG1P0uQaO1oTvwTaDJJvDw4EPfKbEsyz1Tjkyc3Dj1Q/oyVHtY28nKdWOGm0UeL7VVofaTG3A==@kvack.org X-Gm-Message-State: AOJu0YzZFrdtv8Lr67bcVgoUd3n+ptNYmd3yCr70PAP7oqyBVFkKg7Is 8Ljv2cSX+ByoTX8mFo6ffhDrXGFKn3yFoFq5fhVx7Yuihl41ChuRr8Cx7HcdmIYNJst9TeKmo1x Sy8UKnLwXpA== X-Google-Smtp-Source: AGHT+IFG2gVbYjLEgd+rVsDr92xsiDXdhg3a2JRtZbNVmnh3EgFoUPLrUSfZq6jOhgEgZFCkNrLAplfHg1IQBQ== X-Received: from wmqa17.prod.google.com ([2002:a05:600c:3491:b0:434:fa72:f1bf]) (user=jackmanb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1d14:b0:436:5fc9:309d with SMTP id 5b1f17b1804b1-436e26f6d81mr55448175e9.30.1736534454514; Fri, 10 Jan 2025 10:40:54 -0800 (PST) Date: Fri, 10 Jan 2025 18:40:30 +0000 In-Reply-To: <20250110-asi-rfc-v2-v2-0-8419288bc805@google.com> Mime-Version: 1.0 References: <20250110-asi-rfc-v2-v2-0-8419288bc805@google.com> X-Mailer: b4 0.15-dev Message-ID: <20250110-asi-rfc-v2-v2-4-8419288bc805@google.com> Subject: [PATCH RFC v2 04/29] mm: asi: Add infrastructure for boot-time enablement From: Brendan Jackman To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Richard Henderson , Matt Turner , Vineet Gupta , Russell King , Catalin Marinas , Will Deacon , Guo Ren , Brian Cain , Huacai Chen , WANG Xuerui , Geert Uytterhoeven , Michal Simek , Thomas Bogendoerfer , Dinh Nguyen , Jonas Bonn , Stefan Kristiansson , Stafford Horne , "James E.J. Bottomley" , Helge Deller , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Naveen N Rao , Madhavan Srinivasan , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Yoshinori Sato , Rich Felker , John Paul Adrian Glaubitz , "David S. Miller" , Andreas Larsson , Richard Weinberger , Anton Ivanov , Johannes Berg , Chris Zankel , Max Filippov , Arnd Bergmann , Andrew Morton , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Uladzislau Rezki , Christoph Hellwig , Masami Hiramatsu , Mathieu Desnoyers , Mike Rapoport , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , Dennis Zhou , Tejun Heo , Christoph Lameter , Sean Christopherson , Paolo Bonzini , Ard Biesheuvel , Josh Poimboeuf , Pawan Gupta Cc: x86@kernel.org, linux-kernel@vger.kernel.org, linux-alpha@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-um@lists.infradead.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, kvm@vger.kernel.org, linux-efi@vger.kernel.org, Brendan Jackman , Junaid Shahid , Yosry Ahmed X-Rspamd-Queue-Id: 11EE640005 X-Rspamd-Server: rspam12 X-Stat-Signature: y3nd137hqfjfyuwf9dys7k5bfkk91t83 X-Rspam-User: X-HE-Tag: 1736534455-956592 X-HE-Meta: U2FsdGVkX19/kmwQRfzJtGdp6J8/YD0sLoEfAD3DsB7Neu8Et6qRhmcd/xikrlJ7Fam8Czc8o74s4q4yCmYpxGVA74ClsVtb5kVOeAgDNx9OnUrnOPZopShMdIkZoR0u6wL/mUII19ljp1Vy77C/o7RxgGSmtRn+aSTbU9BzVxqDH2OCmfdr/xGTce2cE6Ditwiz0DxHHvzLcqiPge15nLlwJAt8rAnOBNGgZjd0Pir5D6dJimYd7JQw/jDMLJIJly6gG4CPGxN5Fx/YyKjh1nw5EfgHMITYgQ+8X7O6WyiOwxJVXLIeGX77VV+qnXz/vAy0BPniMV6/8cDIiGj3EVfQhZ/jqADsvWZssfezoXxNLLl2Z/TlrLasvL6Sj+YA/U5O8hwZkWNqUyXJgsUo0irKM3+JPU9BlVzH5kW10w30RxE3PGOnKVazc6ayIVhjV59VijwDtGJnxa1fV+n949bD92y3nwPtZf4QL3A5y5kjCun0QnO2LhdTuFP+ocqYZPanmm3Mgdi7E1GvwzBJu8CDWODgO5fJwy7IXfERDzTb9k+b6xDbVUwrkHYBfqR+hQxg1FkLuiMpz67e92oZvTRAELmDN72x42JwWAAg0CAFrLAp+SZqQLMux4NXLnO2lF/+k2QphMkXi9Iqmk3wRxxcQnHvDAuXZdGj1llWgBymxN3eeceW2y9QXq9UUIZdKyFUQhfbeUfWCzp3frmzQcQLp4dBwjgtUzUU7/mILknfut3TAKYOLtJkydjYN4HzejhynoKnF/aim8MX52Oe0knVjQ+jsJxQ6cP6FHXYM7IgaaNG0ay/fkfmWrqDcgmmiQ4m7UTnbJgTu3rT7aVaoZXC4wRJ8F0YNnmjXICGh1WuVpax2tl37XZNj8VWhaBNj3EdJW9HHzeVnyxiymfEiIdtzvtxW0mDr1WsFn1LfIAjOMUWwi/ExZHUp3BucLxttc9o8LWlcuGcQN3FGv4 pfE1NCWv ck9OwoKKzcn+3lZbStzNRTyAObfttvpb6Y1Eb3AjsoWjUHFjiggue6ipuvuP+KJhaXtWZfWF6lOFUFCt86LoLLtQiw2fGXOIpKwTg/1P6rw2fKeiZ5hEfWNo49qS45sA3EQhORfLo1Yh/Bd+3Ky0g7TIM9o1YEe24pTxqVERQMV1uvcnUQc8YplJpPk2M+x4fKI5/Fv7+3XfqQLnfJpi5k4VoBSJdG7URB3pA/jaNd0K2BbwcHmRCVnGinKamevXxrUnB0vTKZkWEzwYVkajjUOv1G9kKKketmA8MRYKp2ysYTAsQSUFTfSfMexO0E+OtKbMhhg3CLr7DalkoDapB5XcTu4f8d14pLd0lp90xludGIwwd+PvDXON5DKQZQFmup3Mw2DbsCZ7SziT2XN+Umj6s49h00Sd1ygrC3m+dnQIrCa/40g+ICPN82e7EdeVqjMKKTae94nRaJvz4ScVcBxqQQyW6DokBcBKx2T75XngUiTx0y1Lb4x7L6G9me1LXgKeUJ8Jz819S4u8l59DtfebGRbsjHIQEX7CPWC1Eua0qiLzY8e+tYA+i1Qprs5Y0y1Fw X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a boot time parameter to control the newly added X86_FEATURE_ASI. "asi=on" or "asi=off" can be used in the kernel command line to enable or disable ASI at boot time. If not specified, ASI enablement depends on CONFIG_ADDRESS_SPACE_ISOLATION_DEFAULT_ON, which is off by default. asi_check_boottime_disable() is modeled after pti_check_boottime_disable(). The boot parameter is currently ignored until ASI is fully functional. Once we have a set of ASI features checked in that we have actually tested, we will stop ignoring the flag. But for now let's just add the infrastructure so we can implement the usage code. Ignoring checkpatch.pl CONFIG_DESCRIPTION because the _DEFAULT_ON Kconfig is trivial to explain. Checkpatch-args: --ignore CONFIG_DESCRIPTION Co-developed-by: Junaid Shahid Signed-off-by: Junaid Shahid Co-developed-by: Yosry Ahmed Signed-off-by: Yosry Ahmed Signed-off-by: Brendan Jackman --- arch/x86/Kconfig | 9 +++++ arch/x86/include/asm/asi.h | 19 ++++++++-- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/disabled-features.h | 8 ++++- arch/x86/mm/asi.c | 61 +++++++++++++++++++++++++++----- arch/x86/mm/init.c | 4 ++- include/asm-generic/asi.h | 4 +++ 7 files changed, 92 insertions(+), 14 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 5a50582eb210e9d1309856a737d32b76fa1bfc85..1fcb52cb8cd5084ac3cef04af61b7d1653162bdb 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -2533,6 +2533,15 @@ config MITIGATION_ADDRESS_SPACE_ISOLATION there are likely to be unhandled cases, in particular concerning TLB flushes. + +config ADDRESS_SPACE_ISOLATION_DEFAULT_ON + bool "Enable address space isolation by default" + default n + depends on MITIGATION_ADDRESS_SPACE_ISOLATION + help + If selected, ASI is enabled by default at boot if the asi=on or + asi=off are not specified. + config MITIGATION_RETPOLINE bool "Avoid speculative indirect branches in kernel" select OBJTOOL if HAVE_OBJTOOL diff --git a/arch/x86/include/asm/asi.h b/arch/x86/include/asm/asi.h index 7cc635b6653a3970ba9dbfdc9c828a470e27bd44..b9671ef2dd3278adceed18507fd260e21954d574 100644 --- a/arch/x86/include/asm/asi.h +++ b/arch/x86/include/asm/asi.h @@ -8,6 +8,7 @@ #include #include +#include #include #ifdef CONFIG_MITIGATION_ADDRESS_SPACE_ISOLATION @@ -66,6 +67,8 @@ * the N ASI classes. */ +#define static_asi_enabled() cpu_feature_enabled(X86_FEATURE_ASI) + /* * ASI uses a per-CPU tainting model to track what mitigation actions are * required on domain transitions. Taints exist along two dimensions: @@ -131,6 +134,8 @@ struct asi { DECLARE_PER_CPU_ALIGNED(struct asi *, curr_asi); +void asi_check_boottime_disable(void); + void asi_init_mm_state(struct mm_struct *mm); int asi_init_class(enum asi_class_id class_id, struct asi_taint_policy *taint_policy); @@ -155,7 +160,9 @@ void asi_exit(void); /* The target is the domain we'll enter when returning to process context. */ static __always_inline struct asi *asi_get_target(struct task_struct *p) { - return p->thread.asi_state.target; + return static_asi_enabled() + ? p->thread.asi_state.target + : NULL; } static __always_inline void asi_set_target(struct task_struct *p, @@ -166,7 +173,9 @@ static __always_inline void asi_set_target(struct task_struct *p, static __always_inline struct asi *asi_get_current(void) { - return this_cpu_read(curr_asi); + return static_asi_enabled() + ? this_cpu_read(curr_asi) + : NULL; } /* Are we currently in a restricted address space? */ @@ -175,7 +184,11 @@ static __always_inline bool asi_is_restricted(void) return (bool)asi_get_current(); } -/* If we exit/have exited, can we stay that way until the next asi_enter? */ +/* + * If we exit/have exited, can we stay that way until the next asi_enter? + * + * When ASI is disabled, this returns true. + */ static __always_inline bool asi_is_relaxed(void) { return !asi_get_target(current); diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 913fd3a7bac6506141de65f33b9ee61c615c7d7d..d6a808d10c3b8900d190ea01c66fc248863f05e2 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -474,6 +474,7 @@ #define X86_FEATURE_CLEAR_BHB_HW (21*32+ 3) /* BHI_DIS_S HW control enabled */ #define X86_FEATURE_CLEAR_BHB_LOOP_ON_VMEXIT (21*32+ 4) /* Clear branch history at vmexit using SW loop */ #define X86_FEATURE_FAST_CPPC (21*32 + 5) /* AMD Fast CPPC */ +#define X86_FEATURE_ASI (21*32+6) /* Kernel Address Space Isolation */ /* * BUG word(s) diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/asm/disabled-features.h index c492bdc97b0595ec77f89dc9b0cefe5e3e64be41..c7964ed4fef8b9441e1c0453da587787d8008d9d 100644 --- a/arch/x86/include/asm/disabled-features.h +++ b/arch/x86/include/asm/disabled-features.h @@ -50,6 +50,12 @@ # define DISABLE_PTI (1 << (X86_FEATURE_PTI & 31)) #endif +#ifdef CONFIG_MITIGATION_ADDRESS_SPACE_ISOLATION +# define DISABLE_ASI 0 +#else +# define DISABLE_ASI (1 << (X86_FEATURE_ASI & 31)) +#endif + #ifdef CONFIG_MITIGATION_RETPOLINE # define DISABLE_RETPOLINE 0 #else @@ -154,7 +160,7 @@ #define DISABLED_MASK17 0 #define DISABLED_MASK18 (DISABLE_IBT) #define DISABLED_MASK19 (DISABLE_SEV_SNP) -#define DISABLED_MASK20 0 +#define DISABLED_MASK20 (DISABLE_ASI) #define DISABLED_MASK21 0 #define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 22) diff --git a/arch/x86/mm/asi.c b/arch/x86/mm/asi.c index 105cd8b43eaf5c20acc80d4916b761559fb95d74..5baf563a078f5b3a6cd4b9f5e92baaf81b0774c4 100644 --- a/arch/x86/mm/asi.c +++ b/arch/x86/mm/asi.c @@ -4,6 +4,7 @@ #include #include +#include #include #include #include @@ -29,6 +30,9 @@ static inline bool asi_class_id_valid(enum asi_class_id class_id) static inline bool asi_class_initialized(enum asi_class_id class_id) { + if (!boot_cpu_has(X86_FEATURE_ASI)) + return 0; + if (WARN_ON(!asi_class_id_valid(class_id))) return false; @@ -51,6 +55,9 @@ EXPORT_SYMBOL_GPL(asi_init_class); void asi_uninit_class(enum asi_class_id class_id) { + if (!boot_cpu_has(X86_FEATURE_ASI)) + return; + if (!asi_class_initialized(class_id)) return; @@ -66,10 +73,36 @@ const char *asi_class_name(enum asi_class_id class_id) return asi_class_names[class_id]; } +void __init asi_check_boottime_disable(void) +{ + bool enabled = IS_ENABLED(CONFIG_MITIGATION_ADDRESS_SPACE_ISOLATION_DEFAULT_ON); + char arg[4]; + int ret; + + ret = cmdline_find_option(boot_command_line, "asi", arg, sizeof(arg)); + if (ret == 3 && !strncmp(arg, "off", 3)) { + enabled = false; + pr_info("ASI disabled through kernel command line.\n"); + } else if (ret == 2 && !strncmp(arg, "on", 2)) { + enabled = true; + pr_info("Ignoring asi=on param while ASI implementation is incomplete.\n"); + } else { + pr_info("ASI %s by default.\n", + enabled ? "enabled" : "disabled"); + } + + if (enabled) + pr_info("ASI enablement ignored due to incomplete implementation.\n"); +} + static void __asi_destroy(struct asi *asi) { - lockdep_assert_held(&asi->mm->asi_init_lock); + WARN_ON_ONCE(asi->ref_count <= 0); + if (--(asi->ref_count) > 0) + return; + free_pages((ulong)asi->pgd, PGD_ALLOCATION_ORDER); + memset(asi, 0, sizeof(struct asi)); } int asi_init(struct mm_struct *mm, enum asi_class_id class_id, struct asi **out_asi) @@ -79,6 +112,9 @@ int asi_init(struct mm_struct *mm, enum asi_class_id class_id, struct asi **out_ *out_asi = NULL; + if (!boot_cpu_has(X86_FEATURE_ASI)) + return 0; + if (WARN_ON(!asi_class_initialized(class_id))) return -EINVAL; @@ -122,7 +158,7 @@ void asi_destroy(struct asi *asi) { struct mm_struct *mm; - if (!asi) + if (!boot_cpu_has(X86_FEATURE_ASI) || !asi) return; if (WARN_ON(!asi_class_initialized(asi->class_id))) @@ -134,11 +170,7 @@ void asi_destroy(struct asi *asi) * to block concurrent asi_init calls. */ mutex_lock(&mm->asi_init_lock); - WARN_ON_ONCE(asi->ref_count <= 0); - if (--(asi->ref_count) == 0) { - free_pages((ulong)asi->pgd, PGD_ALLOCATION_ORDER); - memset(asi, 0, sizeof(struct asi)); - } + __asi_destroy(asi); mutex_unlock(&mm->asi_init_lock); } EXPORT_SYMBOL_GPL(asi_destroy); @@ -255,6 +287,9 @@ static noinstr void __asi_enter(void) noinstr void asi_enter(struct asi *asi) { + if (!static_asi_enabled()) + return; + VM_WARN_ON_ONCE(!asi); /* Should not have an asi_enter() without a prior asi_relax(). */ @@ -269,8 +304,10 @@ EXPORT_SYMBOL_GPL(asi_enter); noinstr void asi_relax(void) { - barrier(); - asi_set_target(current, NULL); + if (static_asi_enabled()) { + barrier(); + asi_set_target(current, NULL); + } } EXPORT_SYMBOL_GPL(asi_relax); @@ -279,6 +316,9 @@ noinstr void asi_exit(void) u64 unrestricted_cr3; struct asi *asi; + if (!static_asi_enabled()) + return; + preempt_disable_notrace(); VM_BUG_ON(this_cpu_read(cpu_tlbstate.loaded_mm) == @@ -310,6 +350,9 @@ EXPORT_SYMBOL_GPL(asi_exit); void asi_init_mm_state(struct mm_struct *mm) { + if (!boot_cpu_has(X86_FEATURE_ASI)) + return; + memset(mm->asi, 0, sizeof(mm->asi)); mutex_init(&mm->asi_init_lock); } diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index de4227ed5169ff84d0ce80b677caffc475198fa6..ded3a47f2a9c1f554824d4ad19f3b48bce271274 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -28,6 +28,7 @@ #include #include #include +#include /* * We need to define the tracepoints somewhere, and tlb.c @@ -251,7 +252,7 @@ static void __init probe_page_size_mask(void) __default_kernel_pte_mask = __supported_pte_mask; /* Except when with PTI where the kernel is mostly non-Global: */ if (cpu_feature_enabled(X86_FEATURE_PTI) || - IS_ENABLED(CONFIG_MITIGATION_ADDRESS_SPACE_ISOLATION)) + cpu_feature_enabled(X86_FEATURE_ASI)) __default_kernel_pte_mask &= ~_PAGE_GLOBAL; /* Enable 1 GB linear kernel mappings if available: */ @@ -754,6 +755,7 @@ void __init init_mem_mapping(void) unsigned long end; pti_check_boottime_disable(); + asi_check_boottime_disable(); probe_page_size_mask(); setup_pcid(); diff --git a/include/asm-generic/asi.h b/include/asm-generic/asi.h index 6b84202837605fa57e4a910318c8353b3f816f06..eedc961ee916a9e1da631ca489ea4a7bc9e6089f 100644 --- a/include/asm-generic/asi.h +++ b/include/asm-generic/asi.h @@ -65,6 +65,10 @@ static inline pgd_t *asi_pgd(struct asi *asi) { return NULL; } static inline void asi_handle_switch_mm(void) { } +#define static_asi_enabled() false + +static inline void asi_check_boottime_disable(void) { } + #endif /* !CONFIG_MITIGATION_ADDRESS_SPACE_ISOLATION */ #endif /* !_ASSEMBLY_ */