From patchwork Thu Nov 24 00:04:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kim Phillips X-Patchwork-Id: 13054476 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C734C433FE for ; Thu, 24 Nov 2022 00:05:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230092AbiKXAFq (ORCPT ); Wed, 23 Nov 2022 19:05:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230076AbiKXAFd (ORCPT ); Wed, 23 Nov 2022 19:05:33 -0500 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2040.outbound.protection.outlook.com [40.107.236.40]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0ED4110CE98; Wed, 23 Nov 2022 16:05:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IfQ8E1wixQQ1LJIWxfNtteDm5dWgEyBB1rDsamtjFET4pJcjdzAx8cOr5ygwGUM+LYESYKsDyAoQAVz+rVjvwcSX+4triTc6a4bYtv1/DQ1fkf67mMTe4YmFy71R/qhRgACXBvaSfbXS/0Gk3fY+lmRSakfkOn76ImwnWUVG6Yt1wOrWRm2bhoz1QD9oLAVNF3PQTO+/5HpdhyJl/WOqEdvrla+gGhosY4TAv6TPkuJ9mxj/zg2OKzvkMwAP10n28sb6S9hYsvoqNXyaXVVYTPO9dIEgjG+KP2K8IJby89A6KlubCGHi+Zw42EgI/rjIs2WKNedT2oZh8TsSe+e9MA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=W20SzAEoQHbKu0wGpENiReJaqubY09OI+iAoYVje1AE=; b=lKPZdqwGpOj/uHnHuxxaXFXJYph68iaUobf4ClhG/H4rfgJMtbeAnl1SE+WqTlSxExJtQiOMFO+Io1sowt1CL95poUbIPsYHDxo7+uxDS9jYWIf3Lx4QLSSRzQkZT8imVC71Ga3roCs9Bh4DvBQbiA1XVzkl6Bgve2sm/tWufT3rIghXZAuUkbLyhrnBTsBsV0Krx5L1XobgwjMCJ+UhbJjjfQAoCRAFhhNYqasrodwNxREdvyZLSjQIC9oMEJqRdjWuVxsQtMi4EDrzhcqq9MRDVO1GP+gQW33gbbAJBHF/A/hECDHOD0Q8PXaqdwzXOeHCkKzFYj5BkjmKJTANLg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W20SzAEoQHbKu0wGpENiReJaqubY09OI+iAoYVje1AE=; b=IuJlLjqLx25RT+vMBOnkAa4d7Kkqhqk7Euu1heQoOhyveCvU7FBBJ+FpMMeDiSYpSvgA/ZKZ6PT34knRFH9v5XhT/DrlQePzFd8XNJglQichRIJMNk60Jw9TxupsoYqNp0Fs/g3xqd81fQHGUpoTh2qL7Atz8a9YA265eJW3hFQ= Received: from MW4PR03CA0221.namprd03.prod.outlook.com (2603:10b6:303:b9::16) by MW3PR12MB4412.namprd12.prod.outlook.com (2603:10b6:303:58::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.15; Thu, 24 Nov 2022 00:05:25 +0000 Received: from CO1NAM11FT073.eop-nam11.prod.protection.outlook.com (2603:10b6:303:b9:cafe::bf) by MW4PR03CA0221.outlook.office365.com (2603:10b6:303:b9::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.15 via Frontend Transport; Thu, 24 Nov 2022 00:05:20 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT073.mail.protection.outlook.com (10.13.174.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5834.8 via Frontend Transport; Thu, 24 Nov 2022 00:05:20 +0000 Received: from fritz.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Wed, 23 Nov 2022 18:05:18 -0600 From: Kim Phillips To: CC: Kim Phillips , Babu Moger , Borislav Petkov , Borislav Petkov , Boris Ostrovsky , Dave Hansen , "H. Peter Anvin" , Ingo Molnar , Joao Martins , Jonathan Corbet , "Konrad Rzeszutek Wilk" , Paolo Bonzini , Sean Christopherson , Thomas Gleixner , David Woodhouse , Greg Kroah-Hartman , Juergen Gross , Peter Zijlstra , Tony Luck , Tom Lendacky , , , Subject: [PATCH v2 1/2] x86/cpu, kvm: Use CPU capabilities for CPUID[0x80000021].EAX Date: Wed, 23 Nov 2022 18:04:48 -0600 Message-ID: <20221124000449.79014-2-kim.phillips@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221124000449.79014-1-kim.phillips@amd.com> References: <20221124000449.79014-1-kim.phillips@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT073:EE_|MW3PR12MB4412:EE_ X-MS-Office365-Filtering-Correlation-Id: 11bb414a-f36d-49e4-2151-08dacdaf936a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Mt9cdrBN8g0uB5Fawm+JpAMUfM6I/aUz3w/qB3pUvuRnn4d/vAyRGBPXXn2iiOMKhDwlFeLLAWO5yLAWNwld/XX+dl2Tt4gdxh9lfujrzeB0oKMpl2rhI1VHvDPZvHHOsh1ZqaKrnWyd1pZ2hBBNcPynWFy1ZZ4cYwEetgGENY0Wa2rKhddsjoQmRu+h/5Fjnjm9NyqSO2Afn3ltV4rHOtUryNsSGnYxq3iQOpHc4F8WOm1PW5W3RtvHE0Uh97aVxO0vsYssGy0dPp4RhX7xFaENmwqflyb/QR61++LD+n1cWTDJk6EeY0EQ+6RFrvdYQeCjEsAe/q8e6tQpO/yLKvE8HzvpxzAKW46IMYNGVu8Q0jbXFyoh+6M65AWtzgIycQ1Eir9nbq/rxJsFuHUL8WBlw+hy+VQqEpvmJObstl4PE/tUdYbaJNVaR8zreBCYimzDZekvnmHeq/oMBj7LNCaz+V9W6WZ2UjDK7+jy0KoJZUEfywGL/ZVuLb20jazkhyIuhU9l+m/wrtwFk4OO8GktCg2ul261BnWVtJyXSNdHXMhwOb36QsfTgY2MwVeycObWNsoDB/80KxkZ+e+/Xlgnm0TRHMwfr8P7UzaKERW2uiapXmiCkkvOnsLxhtCyVPJ10Q1e2zeTdZ3nlG3bbCtTO46bfkFfgDUZOTXcrxABNvXS8YFM35kGbEmuozfZuPv1prx4OAOnqdFIMEAjsr3rW2WDnRMDZLDosm3ncZw6e1VehE6fiTluU7G9tMx4LVSfqIl6Mg//BWrFrhbm4enBgfrYgJ24f9bWZoSRqnc5+CZzTGGH8He3652zDrWe X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(376002)(396003)(39860400002)(346002)(136003)(451199015)(46966006)(36840700001)(40470700004)(36756003)(86362001)(5660300002)(36860700001)(40480700001)(44832011)(2906002)(356005)(2616005)(81166007)(47076005)(426003)(186003)(40460700003)(26005)(83380400001)(82740400003)(336012)(16526019)(7416002)(1076003)(7696005)(82310400005)(70206006)(70586007)(54906003)(6666004)(966005)(41300700001)(478600001)(8676002)(316002)(6916009)(8936002)(4326008)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Nov 2022 00:05:20.0648 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 11bb414a-f36d-49e4-2151-08dacdaf936a X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT073.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4412 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The AMD Zen4 Automatic IBRS feature bit resides in the 0x80000021 leaf, for which there is already support for exposing Zen3 bits to the guest. Add AMD AutoIBRS feature bit support, including for the other bits, using scattered/synthetic bits. Add the corresponding word to KVM's feature machinery so that AutoIBRS gets advertized into the guest too. Co-developed-by: Babu Moger Co-developed-by: Borislav Petkov Signed-off-by: Kim Phillips --- v1: https://lore.kernel.org/lkml/20221104213651.141057-2-kim.phillips@amd.com/, and https://lore.kernel.org/lkml/20221104213651.141057-4-kim.phillips@amd.com/ v2: Addressed v1 comments: - Use synthetic/scattered bits instead of introducing new leaf [Boris] - Combine the rest of the leaf's bits being used [Paolo] Note: Bits not used by the host can be moved to kvm/cpuid.c if maintainers do not want them in cpufeatures.h. - Hoist bitsetting code to kvm_set_cpu_caps(), and use cpuid_entry_override() in __do_cpuid_func() [Paolo] arch/x86/include/asm/cpufeatures.h | 5 +++-- arch/x86/kernel/cpu/scattered.c | 4 ++++ arch/x86/kvm/cpuid.c | 35 +++++++++++++++++------------- arch/x86/kvm/reverse_cpuid.h | 24 +++++++++++++++----- 4 files changed, 45 insertions(+), 23 deletions(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 759b71cb4f9a..961eb49532b7 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -306,9 +306,10 @@ #define X86_FEATURE_RSB_VMEXIT_LITE (11*32+17) /* "" Fill RSB on VM exit when EIBRS is enabled */ #define X86_FEATURE_SGX_EDECCSSA (11*32+18) /* "" SGX EDECCSSA user leaf function */ #define X86_FEATURE_CALL_DEPTH (11*32+19) /* "" Call depth tracking for RSB stuffing */ - - #define X86_FEATURE_MSR_TSX_CTRL (11*32+20) /* "" MSR IA32_TSX_CTRL (Intel) implemented */ +#define X86_FEATURE_NO_NESTED_DATA_BP (11*32+21) /* "" AMD No Nested Data Breakpoints */ +#define X86_FEATURE_NULL_SEL_CLR_BASE (11*32+22) /* "" AMD Null Selector Clears Base */ +#define X86_FEATURE_AUTOIBRS (11*32+23) /* AMD Automatic IBRS */ /* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */ #define X86_FEATURE_AVX_VNNI (12*32+ 4) /* AVX VNNI instructions */ diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c index f53944fb8f7f..7ae7203cd410 100644 --- a/arch/x86/kernel/cpu/scattered.c +++ b/arch/x86/kernel/cpu/scattered.c @@ -45,6 +45,10 @@ static const struct cpuid_bit cpuid_bits[] = { { X86_FEATURE_CPB, CPUID_EDX, 9, 0x80000007, 0 }, { X86_FEATURE_PROC_FEEDBACK, CPUID_EDX, 11, 0x80000007, 0 }, { X86_FEATURE_MBA, CPUID_EBX, 6, 0x80000008, 0 }, + { X86_FEATURE_NO_NESTED_DATA_BP,CPUID_EAX, 0, 0x80000021, 0 }, + { X86_FEATURE_LFENCE_RDTSC, CPUID_EAX, 2, 0x80000021, 0 }, + { X86_FEATURE_NULL_SEL_CLR_BASE,CPUID_EAX, 6, 0x80000021, 0 }, + { X86_FEATURE_AUTOIBRS, CPUID_EAX, 8, 0x80000021, 0 }, { X86_FEATURE_PERFMON_V2, CPUID_EAX, 0, 0x80000022, 0 }, { X86_FEATURE_AMD_LBR_V2, CPUID_EAX, 1, 0x80000022, 0 }, { 0, 0, 0, 0, 0 } diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index c92c49a0b35b..61cd33a848cc 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -730,6 +730,25 @@ void kvm_set_cpu_caps(void) 0 /* SME */ | F(SEV) | 0 /* VM_PAGE_FLUSH */ | F(SEV_ES) | F(SME_COHERENT)); + /* + * Pass down these bits: + * EAX 0 NNDBP, Processor ignores nested data breakpoints + * EAX 2 LAS, LFENCE always serializing + * EAX 6 NSCB, Null selector clear base + * EAX 8 Automatic IBRS + * + * Other defined bits are for MSRs that KVM does not expose: + * EAX 3 SPCL, SMM page configuration lock + * EAX 13 PCMSR, Prefetch control MSR + */ + kvm_cpu_cap_init_scattered(CPUID_8000_0021_EAX, + SF(NO_NESTED_DATA_BP) | SF(LFENCE_RDTSC) | + SF(NULL_SEL_CLR_BASE) | SF(AUTOIBRS)); + if (static_cpu_has(X86_FEATURE_LFENCE_RDTSC)) + kvm_cpu_cap_set(X86_FEATURE_LFENCE_RDTSC); + if (!static_cpu_has_bug(X86_BUG_NULL_SEG)) + kvm_cpu_cap_set(X86_FEATURE_NULL_SEL_CLR_BASE); + kvm_cpu_cap_mask(CPUID_C000_0001_EDX, F(XSTORE) | F(XSTORE_EN) | F(XCRYPT) | F(XCRYPT_EN) | F(ACE2) | F(ACE2_EN) | F(PHE) | F(PHE_EN) | @@ -1211,21 +1230,7 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) break; case 0x80000021: entry->ebx = entry->ecx = entry->edx = 0; - /* - * Pass down these bits: - * EAX 0 NNDBP, Processor ignores nested data breakpoints - * EAX 2 LAS, LFENCE always serializing - * EAX 6 NSCB, Null selector clear base - * - * Other defined bits are for MSRs that KVM does not expose: - * EAX 3 SPCL, SMM page configuration lock - * EAX 13 PCMSR, Prefetch control MSR - */ - entry->eax &= BIT(0) | BIT(2) | BIT(6); - if (static_cpu_has(X86_FEATURE_LFENCE_RDTSC)) - entry->eax |= BIT(2); - if (!static_cpu_has_bug(X86_BUG_NULL_SEG)) - entry->eax |= BIT(6); + cpuid_entry_override(entry, CPUID_8000_0021_EAX); break; /*Add support for Centaur's CPUID instruction*/ case 0xC0000000: diff --git a/arch/x86/kvm/reverse_cpuid.h b/arch/x86/kvm/reverse_cpuid.h index 4e5b8444f161..0bf02c02bb0a 100644 --- a/arch/x86/kvm/reverse_cpuid.h +++ b/arch/x86/kvm/reverse_cpuid.h @@ -13,6 +13,7 @@ */ enum kvm_only_cpuid_leafs { CPUID_12_EAX = NCAPINTS, + CPUID_8000_0021_EAX, NR_KVM_CPU_CAPS, NKVMCAPINTS = NR_KVM_CPU_CAPS - NCAPINTS, @@ -25,6 +26,12 @@ enum kvm_only_cpuid_leafs { #define KVM_X86_FEATURE_SGX2 KVM_X86_FEATURE(CPUID_12_EAX, 1) #define KVM_X86_FEATURE_SGX_EDECCSSA KVM_X86_FEATURE(CPUID_12_EAX, 11) +/* AMD-defined Extended Feature 2 EAX, CPUID level 0x80000021 (EAX) */ +#define KVM_X86_FEATURE_NO_NESTED_DATA_BP KVM_X86_FEATURE(CPUID_8000_0021_EAX, 0) +#define KVM_X86_FEATURE_LFENCE_RDTSC KVM_X86_FEATURE(CPUID_8000_0021_EAX, 2) +#define KVM_X86_FEATURE_NULL_SEL_CLR_BASE KVM_X86_FEATURE(CPUID_8000_0021_EAX, 6) +#define KVM_X86_FEATURE_AUTOIBRS KVM_X86_FEATURE(CPUID_8000_0021_EAX, 8) + struct cpuid_reg { u32 function; u32 index; @@ -49,6 +56,7 @@ static const struct cpuid_reg reverse_cpuid[] = { [CPUID_7_1_EAX] = { 7, 1, CPUID_EAX}, [CPUID_12_EAX] = {0x00000012, 0, CPUID_EAX}, [CPUID_8000_001F_EAX] = {0x8000001f, 0, CPUID_EAX}, + [CPUID_8000_0021_EAX] = {0x80000021, 0, CPUID_EAX}, }; /* @@ -75,12 +83,16 @@ static __always_inline void reverse_cpuid_check(unsigned int x86_leaf) */ static __always_inline u32 __feature_translate(int x86_feature) { - if (x86_feature == X86_FEATURE_SGX1) - return KVM_X86_FEATURE_SGX1; - else if (x86_feature == X86_FEATURE_SGX2) - return KVM_X86_FEATURE_SGX2; - else if (x86_feature == X86_FEATURE_SGX_EDECCSSA) - return KVM_X86_FEATURE_SGX_EDECCSSA; + switch (x86_feature) { + case X86_FEATURE_SGX1: return KVM_X86_FEATURE_SGX1; + case X86_FEATURE_SGX2: return KVM_X86_FEATURE_SGX2; + case X86_FEATURE_SGX_EDECCSSA: return KVM_X86_FEATURE_SGX_EDECCSSA; + case X86_FEATURE_NO_NESTED_DATA_BP: return KVM_X86_FEATURE_NO_NESTED_DATA_BP; + case X86_FEATURE_LFENCE_RDTSC: return KVM_X86_FEATURE_LFENCE_RDTSC; + case X86_FEATURE_NULL_SEL_CLR_BASE: return KVM_X86_FEATURE_NULL_SEL_CLR_BASE; + case X86_FEATURE_AUTOIBRS: return KVM_X86_FEATURE_AUTOIBRS; + default: break; + } return x86_feature; } From patchwork Thu Nov 24 00:04:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kim Phillips X-Patchwork-Id: 13054477 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EC08C433FE for ; Thu, 24 Nov 2022 00:06:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230040AbiKXAGC (ORCPT ); Wed, 23 Nov 2022 19:06:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35544 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229965AbiKXAFn (ORCPT ); Wed, 23 Nov 2022 19:05:43 -0500 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2045.outbound.protection.outlook.com [40.107.212.45]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 40ADF10EA29; Wed, 23 Nov 2022 16:05:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZYAqrC56U85Sk/25TAP1s2h6CbC9oXKakTzePJnzdFg1NdCBK6G5GY4g6s5ndjp2GqXEzyr3HcPpXdsbDvSfehq1PKLls5KLWQSaaAB51lYjTYCw7h8USem8JXWP7ItIomfhnPPQQboMoe8abAiHBcWvUijJqMwkHd4qsPxkE3kLMNc9/Tew9E+LtxLTzIanO9eIbXxymyFlPTvbpO38yVEcoAnEb1yoPNxg1um1Ue6MBrLpJQUm6zXQGHKrQSuOs+x05CJASzXbcvQvAsXVACYJIcQYveoHAWWqAVcnv9JmCscIFgrV1lhYYu0uAfmelE+9c7VjBXn4kW5umZVRXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rEiENljY2DWeh3xsSsE+g4k7imsrnt3rwwOWJRnz7KI=; b=V4usQiQAoUE+kB7BHU8+uIrw5chST5hKqenL4Ib1cI1R8062r85UHT3uSR1N0h5WBcdNtumr9+QJAnldvNpG0p7cL8D1KgnwQf83fBoXk8whnpIwV+gNrbwBJOfNVABZ22Z1a5bKZqQFP93itBwZy2TZx9MT4T5S4+yxHYboe/tGKoJ7evagjKH57MhqcOjKyXKAlngu2rvSJEkEb+pE+/rurkUG/7R0I7oSN07ubCBt6daq8Kqtp+hwJqK4OhEeR1Y4ZyTqTYwVuCY1t/YsWGxCJcqMpdxx26FyqMm3D/RFMQTv61ta35VmG36Xh6JPruzOvPP7+b0ZWdyqEsJ0Jw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rEiENljY2DWeh3xsSsE+g4k7imsrnt3rwwOWJRnz7KI=; b=h538IsmcBxcN0XWD8OEyw+WEOW1tYiIIlewB89XKD3nZj7bFmOrvxW4tHR/sihTNx6eOddjkefN7FyZU2nIY9ntFtUAptpj/mVvhKDqQOIkKHqBrNer897ROH5bs2RDnHB96rjJ2xRfYKUtY5fyhEZkT/KVBAXxG8Tic5wZIp7w= Received: from MW4PR04CA0135.namprd04.prod.outlook.com (2603:10b6:303:84::20) by LV2PR12MB5799.namprd12.prod.outlook.com (2603:10b6:408:179::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.15; Thu, 24 Nov 2022 00:05:33 +0000 Received: from CO1NAM11FT071.eop-nam11.prod.protection.outlook.com (2603:10b6:303:84:cafe::93) by MW4PR04CA0135.outlook.office365.com (2603:10b6:303:84::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.18 via Frontend Transport; Thu, 24 Nov 2022 00:05:32 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT071.mail.protection.outlook.com (10.13.175.56) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5857.18 via Frontend Transport; Thu, 24 Nov 2022 00:05:32 +0000 Received: from fritz.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Wed, 23 Nov 2022 18:05:30 -0600 From: Kim Phillips To: CC: Kim Phillips , Babu Moger , Borislav Petkov , Borislav Petkov , Boris Ostrovsky , Dave Hansen , "H. Peter Anvin" , Ingo Molnar , Joao Martins , Jonathan Corbet , "Konrad Rzeszutek Wilk" , Paolo Bonzini , Sean Christopherson , Thomas Gleixner , David Woodhouse , Greg Kroah-Hartman , Juergen Gross , Peter Zijlstra , Tony Luck , Tom Lendacky , , , Subject: [PATCH v2 2/2] x86/cpu, kvm: Support AMD Automatic IBRS Date: Wed, 23 Nov 2022 18:04:49 -0600 Message-ID: <20221124000449.79014-3-kim.phillips@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221124000449.79014-1-kim.phillips@amd.com> References: <20221124000449.79014-1-kim.phillips@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT071:EE_|LV2PR12MB5799:EE_ X-MS-Office365-Filtering-Correlation-Id: 48245d85-916a-4f44-f47c-08dacdaf9ae7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: uFGfkzmg23z+kWXmRKAAMT+LzkT7U5PLPEUL64Fy397sIUI7/PXDLoAcx0eUdSaGhqbhmVlOOn9bxSHOauOGQ8WNeisiKSRthdp9YaHp+FsyP/a8k+d9p38vLbLrDo3NpfHbt3XaUtu60us512nJPnLyTOXEQzc6+r+V5IEsSGGKd/fpu1aOSGpzMtv5ztmSZtq8/FQTarVeQ18+hoktO46gIS8s+/tf5A2DhrLeCjkDWveyzEoMtYu2EvO14NYBR948+TEvzkQ/rUdLbQktxF5MZV7swnCn+nCTLzwj9MNpzogz6DEJ2NOb3ONDISrzJ7szXHOVMoeVr22xL8v+pjup7StYP67DozWjrBTmCDkupnr0MojH7Iu89xGU9kedu8oVaLe+5twhhMuVj6E2Wb94+vXiRpxTCls9aJEQojBx9qEZcopjgfCPvghmplqGgtGMtfzmIKP0vrc+hoJCTrfPjueRqeCtoxWpuD2XuYnIr02Istpn3uGMBTbR/Ul9C2egAGBDPEBW6P9mBXxtpETyY7dFkcphHfRt8SHjD73GxU66Pbavy6HfYnmCGsgx8+lzAoWBiW4MU6Xx+CT4m4zjI3TIdZKqMGenqKGzUjQ+kUoDatoQe1bd74OjwxXh4lGpt0Ogirccw3sFXL68Avb0tgGEhomfYI4R/G+5FcP5O2KqGVoO2DhfctsNTITqKKwQAhsl5YyYIYaviP+F+Dzu7IsBujhK16ag/Dko1MV0zRAxlztlsX2e8rHnWP3MY08vwg+xnXev3PHCZCP+Pg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(376002)(346002)(136003)(39860400002)(396003)(451199015)(40470700004)(36840700001)(46966006)(8676002)(6916009)(8936002)(82310400005)(40460700003)(36756003)(70206006)(4326008)(36860700001)(356005)(26005)(70586007)(478600001)(6666004)(316002)(5660300002)(7416002)(40480700001)(81166007)(82740400003)(86362001)(54906003)(966005)(44832011)(7696005)(83380400001)(41300700001)(426003)(186003)(336012)(47076005)(2616005)(2906002)(1076003)(16526019)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Nov 2022 00:05:32.6491 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 48245d85-916a-4f44-f47c-08dacdaf9ae7 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT071.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5799 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The AMD Zen4 core supports a new feature called Automatic IBRS. It is a "set-and-forget" feature that means that, like Intel's Enhanced IBRS, h/w manages its IBRS mitigation resources automatically across CPL transitions. The feature is advertised by CPUID_Fn80000021_EAX bit 8 and is enabled by setting MSR C000_0080 (EFER) bit 21. Enable Automatic IBRS by default if the CPU feature is present. It typically provides greater performance over the incumbent generic retpolines mitigation. Reuse the SPECTRE_V2_EIBRS spectre_v2_mitigation enum. AMD Automatic IBRS and Intel Enhanced IBRS have similar bugs.c enablement. Also allow for spectre_v2=autoibrs on the kernel command line. 'spectre_v2=autoibrs,retpoline' and 'autoibrs,lfence' are honoured but not required. AutoIBRS will also be enabled if the =eibrs[,{lfence,retpoline}] variants are specified. Signed-off-by: Kim Phillips --- v1: https://lore.kernel.org/lkml/20221104213651.141057-3-kim.phillips@amd.com/ v2: Address v1 comments: - Reuse SPECTRE_V2_EIBRS spectre_v2_mitigation enum [Boris, PeterZ, D.Hansen] - Change from Boris' diff: Moved setting X86_FEATURE_IBRS_ENHANCED to after BUG_EIBRS_PBRSB so PBRSB mitigations wouldn't be enabled. - Allow for users to specify "autoibrs,lfence/retpoline" instead of actively preventing the extra protections. AutoIBRS doesn't require the extra protection, but we allow it anyway. .../admin-guide/kernel-parameters.txt | 9 +++++--- arch/x86/include/asm/msr-index.h | 2 ++ arch/x86/kernel/cpu/bugs.c | 23 ++++++++++++------- arch/x86/kernel/cpu/common.c | 8 +++++++ arch/x86/kvm/svm/svm.c | 3 +++ arch/x86/kvm/x86.c | 3 +++ 6 files changed, 37 insertions(+), 11 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index a465d5242774..880016d06a8a 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -5698,9 +5698,12 @@ retpoline,generic - Retpolines retpoline,lfence - LFENCE; indirect branch retpoline,amd - alias for retpoline,lfence - eibrs - enhanced IBRS - eibrs,retpoline - enhanced IBRS + Retpolines - eibrs,lfence - enhanced IBRS + LFENCE + eibrs - Enhanced/Auto IBRS + autoibrs - Enhanced/Auto IBRS + eibrs,retpoline - Enhanced/Auto IBRS + Retpolines + autoibrs,retpoline- Enhanced/Auto IBRS + Retpolines + eibrs,lfence - Enhanced/Auto IBRS + LFENCE + autoibrs,lfence - Enhanced/Auto IBRS + LFENCE ibrs - use IBRS to protect kernel Not specifying this option is equivalent to diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 8519191c6409..88fdd75f6a2f 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -30,6 +30,7 @@ #define _EFER_SVME 12 /* Enable virtualization */ #define _EFER_LMSLE 13 /* Long Mode Segment Limit Enable */ #define _EFER_FFXSR 14 /* Enable Fast FXSAVE/FXRSTOR */ +#define _EFER_AUTOIBRS 21 /* Enable Automatic IBRS */ #define EFER_SCE (1<<_EFER_SCE) #define EFER_LME (1<<_EFER_LME) @@ -38,6 +39,7 @@ #define EFER_SVME (1<<_EFER_SVME) #define EFER_LMSLE (1<<_EFER_LMSLE) #define EFER_FFXSR (1<<_EFER_FFXSR) +#define EFER_AUTOIBRS (1<<_EFER_AUTOIBRS) /* Intel MSRs. Some also available on other CPUs */ diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index aa0819252c88..5f48dd4dbc48 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -1222,9 +1222,9 @@ static const char * const spectre_v2_strings[] = { [SPECTRE_V2_NONE] = "Vulnerable", [SPECTRE_V2_RETPOLINE] = "Mitigation: Retpolines", [SPECTRE_V2_LFENCE] = "Mitigation: LFENCE", - [SPECTRE_V2_EIBRS] = "Mitigation: Enhanced IBRS", - [SPECTRE_V2_EIBRS_LFENCE] = "Mitigation: Enhanced IBRS + LFENCE", - [SPECTRE_V2_EIBRS_RETPOLINE] = "Mitigation: Enhanced IBRS + Retpolines", + [SPECTRE_V2_EIBRS] = "Mitigation: Enhanced / Automatic IBRS", + [SPECTRE_V2_EIBRS_LFENCE] = "Mitigation: Enhanced / Automatic IBRS + LFENCE", + [SPECTRE_V2_EIBRS_RETPOLINE] = "Mitigation: Enhanced / Automatic IBRS + Retpolines", [SPECTRE_V2_IBRS] = "Mitigation: IBRS", }; @@ -1240,8 +1240,11 @@ static const struct { { "retpoline,lfence", SPECTRE_V2_CMD_RETPOLINE_LFENCE, false }, { "retpoline,generic", SPECTRE_V2_CMD_RETPOLINE_GENERIC, false }, { "eibrs", SPECTRE_V2_CMD_EIBRS, false }, + { "autoibrs", SPECTRE_V2_CMD_EIBRS, false }, { "eibrs,lfence", SPECTRE_V2_CMD_EIBRS_LFENCE, false }, + { "autoibrs,lfence", SPECTRE_V2_CMD_EIBRS_LFENCE, false }, { "eibrs,retpoline", SPECTRE_V2_CMD_EIBRS_RETPOLINE, false }, + { "autoibrs,retpoline", SPECTRE_V2_CMD_EIBRS_RETPOLINE, false }, { "auto", SPECTRE_V2_CMD_AUTO, false }, { "ibrs", SPECTRE_V2_CMD_IBRS, false }, }; @@ -1293,7 +1296,7 @@ static enum spectre_v2_mitigation_cmd __init spectre_v2_parse_cmdline(void) cmd == SPECTRE_V2_CMD_EIBRS_LFENCE || cmd == SPECTRE_V2_CMD_EIBRS_RETPOLINE) && !boot_cpu_has(X86_FEATURE_IBRS_ENHANCED)) { - pr_err("%s selected but CPU doesn't have eIBRS. Switching to AUTO select\n", + pr_err("%s selected but CPU doesn't have Enhanced or Automatic IBRS. Switching to AUTO select\n", mitigation_options[i].option); return SPECTRE_V2_CMD_AUTO; } @@ -1479,8 +1482,12 @@ static void __init spectre_v2_select_mitigation(void) pr_err(SPECTRE_V2_EIBRS_EBPF_MSG); if (spectre_v2_in_ibrs_mode(mode)) { - x86_spec_ctrl_base |= SPEC_CTRL_IBRS; - write_spec_ctrl_current(x86_spec_ctrl_base, true); + if (boot_cpu_has(X86_FEATURE_AUTOIBRS)) { + msr_set_bit(MSR_EFER, _EFER_AUTOIBRS); + } else { + x86_spec_ctrl_base |= SPEC_CTRL_IBRS; + write_spec_ctrl_current(x86_spec_ctrl_base, true); + } } switch (mode) { @@ -1564,8 +1571,8 @@ static void __init spectre_v2_select_mitigation(void) /* * Retpoline protects the kernel, but doesn't protect firmware. IBRS * and Enhanced IBRS protect firmware too, so enable IBRS around - * firmware calls only when IBRS / Enhanced IBRS aren't otherwise - * enabled. + * firmware calls only when IBRS / Enhanced / Automatic IBRS aren't + * otherwise enabled. * * Use "mode" to check Enhanced IBRS instead of boot_cpu_has(), because * the user might select retpoline on the kernel command line and if diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 73cc546e024d..6506024f97e9 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1406,6 +1406,14 @@ static void __init cpu_set_bug_bits(struct cpuinfo_x86 *c) !(ia32_cap & ARCH_CAP_PBRSB_NO)) setup_force_cpu_bug(X86_BUG_EIBRS_PBRSB); + /* + * AMD's AutoIBRS is equivalent to Intel's eIBRS - use the Intel flag only + * after IBRS_ENHANCED bugs such as BUG_EIBRS_PBRSB above have been + * determined. + */ + if (cpu_has(c, X86_FEATURE_AUTOIBRS)) + setup_force_cpu_cap(X86_FEATURE_IBRS_ENHANCED); + if (cpu_matches(cpu_vuln_whitelist, NO_MELTDOWN)) return; diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 4b6d2b050e57..3ac3d4cfce24 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4960,6 +4960,9 @@ static __init int svm_hardware_setup(void) tsc_aux_uret_slot = kvm_add_user_return_msr(MSR_TSC_AUX); + if (boot_cpu_has(X86_FEATURE_AUTOIBRS)) + kvm_enable_efer_bits(EFER_AUTOIBRS); + /* Check for pause filtering support */ if (!boot_cpu_has(X86_FEATURE_PAUSEFILTER)) { pause_filter_count = 0; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 490ec23c8450..db0f522fd597 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1682,6 +1682,9 @@ static int do_get_msr_feature(struct kvm_vcpu *vcpu, unsigned index, u64 *data) static bool __kvm_valid_efer(struct kvm_vcpu *vcpu, u64 efer) { + if (efer & EFER_AUTOIBRS && !guest_cpuid_has(vcpu, X86_FEATURE_AUTOIBRS)) + return false; + if (efer & EFER_FFXSR && !guest_cpuid_has(vcpu, X86_FEATURE_FXSR_OPT)) return false;