From patchwork Sat Jan 20 19:22:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: KarimAllah Ahmed X-Patchwork-Id: 10176675 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 79C5C6056F for ; Sat, 20 Jan 2018 19:24:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 633FE204C4 for ; Sat, 20 Jan 2018 19:24:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 55EB7205AF; Sat, 20 Jan 2018 19:24:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 96355204C4 for ; Sat, 20 Jan 2018 19:24:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932770AbeATTYO (ORCPT ); Sat, 20 Jan 2018 14:24:14 -0500 Received: from smtp-fw-9102.amazon.com ([207.171.184.29]:19581 "EHLO smtp-fw-9102.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932182AbeATTXw (ORCPT ); Sat, 20 Jan 2018 14:23:52 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.de; i=@amazon.de; q=dns/txt; s=amazon201209; t=1516476231; x=1548012231; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=EojizKRYknX/XwvkjVTewVXuI/D9x5y3A8wue1KELxw=; b=KSJpxI+iZDwHRjQ9uuPFYQPIvZqxgtnLFM0x2Sv394fznBX0yquyrMfk KsyQKzYZDs2lQ0lxyjDHASwbxdBHk+Ha7/V/d8Tz4FIAq1zBxZdaHPWBx tB7ldEsU4bbsMWgNBfyVYswODNPYwFEG32eS2NtJ9rfNSh4zmKjFb9KcJ k=; X-IronPort-AV: E=Sophos;i="5.46,387,1511827200"; d="scan'208";a="588403232" Received: from sea3-co-svc-lb6-vlan3.sea.amazon.com (HELO email-inbound-relay-1d-2c665b5d.us-east-1.amazon.com) ([10.47.22.38]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 20 Jan 2018 19:23:49 +0000 Received: from u54e1ad5160425a4b64ea.ant.amazon.com (iad1-ws-svc-lb91-vlan3.amazon.com [10.0.103.150]) by email-inbound-relay-1d-2c665b5d.us-east-1.amazon.com (8.14.7/8.14.7) with ESMTP id w0KJNXqq102399 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 20 Jan 2018 19:23:36 GMT Received: from u54e1ad5160425a4b64ea.ant.amazon.com (localhost [127.0.0.1]) by u54e1ad5160425a4b64ea.ant.amazon.com (8.15.2/8.15.2/Debian-3) with ESMTP id w0KJNWAD005269; Sat, 20 Jan 2018 20:23:32 +0100 Received: (from karahmed@localhost) by u54e1ad5160425a4b64ea.ant.amazon.com (8.15.2/8.15.2/Submit) id w0KJNUEG005265; Sat, 20 Jan 2018 20:23:30 +0100 From: KarimAllah Ahmed To: linux-kernel@vger.kernel.org Cc: KarimAllah Ahmed , Andi Kleen , Andrea Arcangeli , Andy Lutomirski , Arjan van de Ven , Ashok Raj , Asit Mallick , Borislav Petkov , Dan Williams , Dave Hansen , David Woodhouse , Greg Kroah-Hartman , "H . Peter Anvin" , Ingo Molnar , Janakarajan Natarajan , Joerg Roedel , Jun Nakajima , Laura Abbott , Linus Torvalds , Masami Hiramatsu , Paolo Bonzini , Peter Zijlstra , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , Thomas Gleixner , Tim Chen , Tom Lendacky , kvm@vger.kernel.org, x86@kernel.org Subject: [RFC 05/10] x86/speculation: Add basic IBRS support infrastructure Date: Sat, 20 Jan 2018 20:22:56 +0100 Message-Id: <1516476182-5153-6-git-send-email-karahmed@amazon.de> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1516476182-5153-1-git-send-email-karahmed@amazon.de> References: <1516476182-5153-1-git-send-email-karahmed@amazon.de> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: David Woodhouse Not functional yet; just add the handling for it in the Spectre v2 mitigation selection, and the X86_FEATURE_IBRS flag which will control the code to be added in later patches. Also take the #ifdef CONFIG_RETPOLINE from around the RSB-stuffing; IBRS mode will want that too. For now we are auto-selecting IBRS on Skylake. We will probably end up changing that but for now let's default to the safest option. XX: Do we want a microcode blacklist? [karahmed: simplify the switch block and get rid of all the magic] Signed-off-by: David Woodhouse Signed-off-by: KarimAllah Ahmed --- Documentation/admin-guide/kernel-parameters.txt | 1 + arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/nospec-branch.h | 2 - arch/x86/kernel/cpu/bugs.c | 108 +++++++++++++++--------- 4 files changed, 68 insertions(+), 44 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 8122b5f..e597650 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3932,6 +3932,7 @@ retpoline - replace indirect branches retpoline,generic - google's original retpoline retpoline,amd - AMD-specific minimal thunk + ibrs - Intel: Indirect Branch Restricted Speculation Not specifying this option is equivalent to spectre_v2=auto. diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 8ec9588..ae86ad9 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -211,6 +211,7 @@ #define X86_FEATURE_AMD_PRED_CMD ( 7*32+17) /* Prediction Command MSR (AMD) */ #define X86_FEATURE_MBA ( 7*32+18) /* Memory Bandwidth Allocation */ #define X86_FEATURE_RSB_CTXSW ( 7*32+19) /* Fill RSB on context switches */ +#define X86_FEATURE_IBRS ( 7*32+21) /* Use IBRS for Spectre v2 safety */ /* Virtualization flags: Linux defined, word 8 */ #define X86_FEATURE_TPR_SHADOW ( 8*32+ 0) /* Intel TPR Shadow */ diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h index c333c95..8759449 100644 --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -205,7 +205,6 @@ extern char __indirect_thunk_end[]; */ static inline void vmexit_fill_RSB(void) { -#ifdef CONFIG_RETPOLINE unsigned long loops; asm volatile (ANNOTATE_NOSPEC_ALTERNATIVE @@ -215,7 +214,6 @@ static inline void vmexit_fill_RSB(void) "910:" : "=r" (loops), ASM_CALL_CONSTRAINT : : "memory" ); -#endif } static inline void indirect_branch_prediction_barrier(void) diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index 96548ff..1d5e12f 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -79,6 +79,7 @@ enum spectre_v2_mitigation_cmd { SPECTRE_V2_CMD_RETPOLINE, SPECTRE_V2_CMD_RETPOLINE_GENERIC, SPECTRE_V2_CMD_RETPOLINE_AMD, + SPECTRE_V2_CMD_IBRS, }; static const char *spectre_v2_strings[] = { @@ -87,6 +88,7 @@ static const char *spectre_v2_strings[] = { [SPECTRE_V2_RETPOLINE_MINIMAL_AMD] = "Vulnerable: Minimal AMD ASM retpoline", [SPECTRE_V2_RETPOLINE_GENERIC] = "Mitigation: Full generic retpoline", [SPECTRE_V2_RETPOLINE_AMD] = "Mitigation: Full AMD retpoline", + [SPECTRE_V2_IBRS] = "Mitigation: Indirect Branch Restricted Speculation", }; #undef pr_fmt @@ -132,9 +134,17 @@ static enum spectre_v2_mitigation_cmd __init spectre_v2_parse_cmdline(void) spec2_print_if_secure("force enabled on command line."); return SPECTRE_V2_CMD_FORCE; } else if (match_option(arg, ret, "retpoline")) { + if (!IS_ENABLED(CONFIG_RETPOLINE)) { + pr_err("retpoline selected but not compiled in. Switching to AUTO select\n"); + return SPECTRE_V2_CMD_AUTO; + } spec2_print_if_insecure("retpoline selected on command line."); return SPECTRE_V2_CMD_RETPOLINE; } else if (match_option(arg, ret, "retpoline,amd")) { + if (!IS_ENABLED(CONFIG_RETPOLINE)) { + pr_err("retpoline,amd selected but not compiled in. Switching to AUTO select\n"); + return SPECTRE_V2_CMD_AUTO; + } if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD) { pr_err("retpoline,amd selected but CPU is not AMD. Switching to AUTO select\n"); return SPECTRE_V2_CMD_AUTO; @@ -142,8 +152,19 @@ static enum spectre_v2_mitigation_cmd __init spectre_v2_parse_cmdline(void) spec2_print_if_insecure("AMD retpoline selected on command line."); return SPECTRE_V2_CMD_RETPOLINE_AMD; } else if (match_option(arg, ret, "retpoline,generic")) { + if (!IS_ENABLED(CONFIG_RETPOLINE)) { + pr_err("retpoline,generic selected but not compiled in. Switching to AUTO select\n"); + return SPECTRE_V2_CMD_AUTO; + } spec2_print_if_insecure("generic retpoline selected on command line."); return SPECTRE_V2_CMD_RETPOLINE_GENERIC; + } else if (match_option(arg, ret, "ibrs")) { + if (!boot_cpu_has(X86_FEATURE_SPEC_CTRL)) { + pr_err("IBRS selected but no CPU support. Switching to AUTO select\n"); + return SPECTRE_V2_CMD_AUTO; + } + spec2_print_if_insecure("IBRS seleted on command line."); + return SPECTRE_V2_CMD_IBRS; } else if (match_option(arg, ret, "auto")) { return SPECTRE_V2_CMD_AUTO; } @@ -156,7 +177,7 @@ static enum spectre_v2_mitigation_cmd __init spectre_v2_parse_cmdline(void) return SPECTRE_V2_CMD_NONE; } -/* Check for Skylake-like CPUs (for RSB handling) */ +/* Check for Skylake-like CPUs (for RSB and IBRS handling) */ static bool __init is_skylake_era(void) { if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL && @@ -178,55 +199,58 @@ static void __init spectre_v2_select_mitigation(void) enum spectre_v2_mitigation_cmd cmd = spectre_v2_parse_cmdline(); enum spectre_v2_mitigation mode = SPECTRE_V2_NONE; - /* - * If the CPU is not affected and the command line mode is NONE or AUTO - * then nothing to do. - */ - if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V2) && - (cmd == SPECTRE_V2_CMD_NONE || cmd == SPECTRE_V2_CMD_AUTO)) - return; - switch (cmd) { case SPECTRE_V2_CMD_NONE: + if (boot_cpu_has_bug(X86_BUG_SPECTRE_V2)) + pr_err("kernel not compiled with retpoline; no mitigation available!"); return; - - case SPECTRE_V2_CMD_FORCE: - /* FALLTRHU */ - case SPECTRE_V2_CMD_AUTO: - goto retpoline_auto; - - case SPECTRE_V2_CMD_RETPOLINE_AMD: - if (IS_ENABLED(CONFIG_RETPOLINE)) - goto retpoline_amd; - break; - case SPECTRE_V2_CMD_RETPOLINE_GENERIC: - if (IS_ENABLED(CONFIG_RETPOLINE)) - goto retpoline_generic; + case SPECTRE_V2_CMD_IBRS: + mode = SPECTRE_V2_IBRS; + setup_force_cpu_cap(X86_FEATURE_IBRS); break; + case SPECTRE_V2_CMD_AUTO: + if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V2)) + return; + /* Fall through */ + case SPECTRE_V2_CMD_FORCE: + /* + * If we have IBRS support, and either Skylake or !RETPOLINE, + * then that's what we do. + */ + if (boot_cpu_has(X86_FEATURE_SPEC_CTRL) && + (is_skylake_era() || !retp_compiler())) { + mode = SPECTRE_V2_IBRS; + setup_force_cpu_cap(X86_FEATURE_IBRS); + break; + } + /* Fall through */ case SPECTRE_V2_CMD_RETPOLINE: - if (IS_ENABLED(CONFIG_RETPOLINE)) - goto retpoline_auto; - break; - } - pr_err("kernel not compiled with retpoline; no mitigation available!"); - return; + case SPECTRE_V2_CMD_RETPOLINE_AMD: + if (IS_ENABLED(CONFIG_RETPOLINE) && + boot_cpu_data.x86_vendor == X86_VENDOR_AMD) { + if (boot_cpu_has(X86_FEATURE_LFENCE_RDTSC)) { + mode = retp_compiler() ? SPECTRE_V2_RETPOLINE_AMD : + SPECTRE_V2_RETPOLINE_MINIMAL_AMD; + setup_force_cpu_cap(X86_FEATURE_RETPOLINE_AMD); + setup_force_cpu_cap(X86_FEATURE_RETPOLINE); + break; + } -retpoline_auto: - if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD) { - retpoline_amd: - if (!boot_cpu_has(X86_FEATURE_LFENCE_RDTSC)) { pr_err("LFENCE not serializing. Switching to generic retpoline\n"); - goto retpoline_generic; } - mode = retp_compiler() ? SPECTRE_V2_RETPOLINE_AMD : - SPECTRE_V2_RETPOLINE_MINIMAL_AMD; - setup_force_cpu_cap(X86_FEATURE_RETPOLINE_AMD); - setup_force_cpu_cap(X86_FEATURE_RETPOLINE); - } else { - retpoline_generic: - mode = retp_compiler() ? SPECTRE_V2_RETPOLINE_GENERIC : - SPECTRE_V2_RETPOLINE_MINIMAL; - setup_force_cpu_cap(X86_FEATURE_RETPOLINE); + /* Fall through */ + case SPECTRE_V2_CMD_RETPOLINE_GENERIC: + if (IS_ENABLED(CONFIG_RETPOLINE)) { + mode = retp_compiler() ? SPECTRE_V2_RETPOLINE_GENERIC : + SPECTRE_V2_RETPOLINE_MINIMAL; + setup_force_cpu_cap(X86_FEATURE_RETPOLINE); + break; + } + /* Fall through */ + default: + if (boot_cpu_has_bug(X86_BUG_SPECTRE_V2)) + pr_err("kernel not compiled with retpoline; no mitigation available!"); + return; } spectre_v2_enabled = mode;