From patchwork Tue Aug 17 14:30:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 12441601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95DD2C4338F for ; Tue, 17 Aug 2021 14:30:58 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 649D360184 for ; Tue, 17 Aug 2021 14:30:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 649D360184 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.167908.306578 (Exim 4.92) (envelope-from ) id 1mG076-0006tY-4M; Tue, 17 Aug 2021 14:30:48 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 167908.306578; Tue, 17 Aug 2021 14:30:48 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mG076-0006tM-1M; Tue, 17 Aug 2021 14:30:48 +0000 Received: by outflank-mailman (input) for mailman id 167908; Tue, 17 Aug 2021 14:30:47 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mG075-0006rF-2I for xen-devel@lists.xenproject.org; Tue, 17 Aug 2021 14:30:47 +0000 Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 800b47b0-f7c7-4287-b1f9-2fd43673fe40; Tue, 17 Aug 2021 14:30:46 +0000 (UTC) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 800b47b0-f7c7-4287-b1f9-2fd43673fe40 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1629210646; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0F/eC+ZQL+tzTHQ9DEa02brpEmz+uk1auQTeRxAeWTQ=; b=ZaqyYAPSnT6AwBjOl8sPLtijE0cmcmL493E8Ttotu+Hz0rx45rPS/rFt hR0fYkRlCMXzicHu4wn50OucQC+kmJqTegSJP/f0t+RJULAJl5CA0qdvr OEla+K2TXk5KHzp73+OCGsvpnz6lkPQ0iyUzrfTntd8HKGq0EkWe3n5Ax Y=; Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: NQjNzKlI58qAtC/9jdjXSfwGuIaBX7uJ5+Q/iURErrjBghTYvOLeEPmteJqJ5sRSuAtffFDVWz 61mLjJ7l5tl3A8jka9xz6hEgP5j/Q44ipNsPSmxlqoFTDaGHshaY3aQlcRKeuR44V5ZD1Ce8gm D8iu2yDga1KbAOAUNIgdTymKwifSWQarJjBK0eRTDsjFes+0vtgxhWWUv3g10h3NrIi8RlZXWq 9Gsx/6DdHo7eueKv8sCR511zVTd2RQxzrf/PA9VD7+US1l9kpSd0as0GuTePC64iNQzoh4TA3O c8V6EYAcxTPrFi6+Jsz1Fcx8 X-SBRS: 5.1 X-MesageID: 50666693 X-Ironport-Server: esa6.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.83 X-Policy: $RELAYED IronPort-HdrOrdr: A9a23:leWhmKDRkB9JeeXlHemW55DYdb4zR+YMi2TC1yhKKCC9Ffbo7/ xG/c5rrCMc5wxhO03I9eruBEDEewK5yXcX2/h2AV7BZniFhILAFugLhuGOrwEIWReOkdK1vZ 0QCJSWY+eRMbEVt6jHCXGDYrMd/OU= X-IronPort-AV: E=Sophos;i="5.84,329,1620705600"; d="scan'208";a="50666693" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 1/3] x86/spec-ctrl: Split the "Hardware features" diagnostic line Date: Tue, 17 Aug 2021 15:30:04 +0100 Message-ID: <20210817143006.2821-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20210817143006.2821-1-andrew.cooper3@citrix.com> References: <20210817143006.2821-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Separate the read-only hints from the features requiring active actions on Xen's behalf. Also take the opportunity split the IBRS/IBPB and IBPB mess. More features with overlapping enumeration are on the way, and and it is not useful to split them like this. No practical change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu --- xen/arch/x86/spec_ctrl.c | 41 ++++++++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 17 deletions(-) diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index 739b7913ff86..9bf0fbf99813 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -317,23 +317,30 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps) printk("Speculative mitigation facilities:\n"); - /* Hardware features which pertain to speculative mitigations. */ - printk(" Hardware features:%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s\n", - (_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBRS/IBPB" : "", - (_7d0 & cpufeat_mask(X86_FEATURE_STIBP)) ? " STIBP" : "", - (_7d0 & cpufeat_mask(X86_FEATURE_L1D_FLUSH)) ? " L1D_FLUSH" : "", - (_7d0 & cpufeat_mask(X86_FEATURE_SSBD)) ? " SSBD" : "", - (_7d0 & cpufeat_mask(X86_FEATURE_MD_CLEAR)) ? " MD_CLEAR" : "", - (_7d0 & cpufeat_mask(X86_FEATURE_SRBDS_CTRL)) ? " SRBDS_CTRL" : "", - (e8b & cpufeat_mask(X86_FEATURE_IBPB)) ? " IBPB" : "", - (caps & ARCH_CAPS_IBRS_ALL) ? " IBRS_ALL" : "", - (caps & ARCH_CAPS_RDCL_NO) ? " RDCL_NO" : "", - (caps & ARCH_CAPS_RSBA) ? " RSBA" : "", - (caps & ARCH_CAPS_SKIP_L1DFL) ? " SKIP_L1DFL": "", - (caps & ARCH_CAPS_SSB_NO) ? " SSB_NO" : "", - (caps & ARCH_CAPS_MDS_NO) ? " MDS_NO" : "", - (caps & ARCH_CAPS_TSX_CTRL) ? " TSX_CTRL" : "", - (caps & ARCH_CAPS_TAA_NO) ? " TAA_NO" : ""); + /* + * Hardware read-only information, stating immunity to certain issues, or + * suggestions of which mitigation to use. + */ + printk(" Hardware hints:%s%s%s%s%s%s%s\n", + (caps & ARCH_CAPS_RDCL_NO) ? " RDCL_NO" : "", + (caps & ARCH_CAPS_IBRS_ALL) ? " IBRS_ALL" : "", + (caps & ARCH_CAPS_RSBA) ? " RSBA" : "", + (caps & ARCH_CAPS_SKIP_L1DFL) ? " SKIP_L1DFL" : "", + (caps & ARCH_CAPS_SSB_NO) ? " SSB_NO" : "", + (caps & ARCH_CAPS_MDS_NO) ? " MDS_NO" : "", + (caps & ARCH_CAPS_TAA_NO) ? " TAA_NO" : ""); + + /* Hardware features which need driving to mitigate issues. */ + printk(" Hardware features:%s%s%s%s%s%s%s%s\n", + (e8b & cpufeat_mask(X86_FEATURE_IBPB)) || + (_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBPB" : "", + (_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBRS" : "", + (_7d0 & cpufeat_mask(X86_FEATURE_STIBP)) ? " STIBP" : "", + (_7d0 & cpufeat_mask(X86_FEATURE_SSBD)) ? " SSBD" : "", + (_7d0 & cpufeat_mask(X86_FEATURE_L1D_FLUSH)) ? " L1D_FLUSH" : "", + (_7d0 & cpufeat_mask(X86_FEATURE_MD_CLEAR)) ? " MD_CLEAR" : "", + (_7d0 & cpufeat_mask(X86_FEATURE_SRBDS_CTRL)) ? " SRBDS_CTRL" : "", + (caps & ARCH_CAPS_TSX_CTRL) ? " TSX_CTRL" : ""); /* Compiled-in support which pertains to mitigations. */ if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) || IS_ENABLED(CONFIG_SHADOW_PAGING) ) From patchwork Tue Aug 17 14:30:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 12441597 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B655C4338F for ; Tue, 17 Aug 2021 14:30:44 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E36E160F39 for ; Tue, 17 Aug 2021 14:30:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E36E160F39 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.167904.306555 (Exim 4.92) (envelope-from ) id 1mG06r-0005wZ-Bu; Tue, 17 Aug 2021 14:30:33 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 167904.306555; Tue, 17 Aug 2021 14:30:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mG06r-0005wO-8i; Tue, 17 Aug 2021 14:30:33 +0000 Received: by outflank-mailman (input) for mailman id 167904; Tue, 17 Aug 2021 14:30:31 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mG06p-0005vQ-K5 for xen-devel@lists.xenproject.org; Tue, 17 Aug 2021 14:30:31 +0000 Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id abe11f68-ff67-11eb-a4c9-12813bfff9fa; Tue, 17 Aug 2021 14:30:30 +0000 (UTC) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: abe11f68-ff67-11eb-a4c9-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1629210630; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/FBvFqnB8wJbrimRiM/iWYtiWgIJODR5Qqcri5t8FTY=; b=QYuv1fDU3B6Xlv9DyfNStHFfbUs7D91IT0Bbe3SvitWGSLHwEOnU51fW nO/sAMxE3QTyuBBFjCwN2sPH7n05bdXav+rktS8hQlGyDaga6Kd94WvUx DZMKef4ziLVc77nqaXP8DyIvkkh3L8T2CV5ZTrysQArgA058tDQa6ZKXY E=; Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: lAxEz1MgR93f0CROO64TwK1rFH5kwgH/wWQ7ButvQg/DYnOntpQWlIRw+d1tVQFMQh0nBxgBcC EISkMlA+oM7gwDZg4RYtwt/4HRgcwiCddX4IJ86havyLwOuq15guf9h9GnGvjz9eNg6qqmM26k 3NtOn1OWVTxPY5tE5Q/QCN0u2vPos+iP8OWaMvPL0gv0DaFEd8bGZ2+K08frJ89SJ9pFASBZ1c 8Qu++GA8xOH8aXwAXfgmFgU3h620P4eSmybPSGYHNxvGiVjebES9QK2tCkAbcyAAH0R6+q1Bxn DRsMVBwmTDbinVdAiHKbGNNk X-SBRS: 5.1 X-MesageID: 51032081 X-Ironport-Server: esa1.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.83 X-Policy: $RELAYED IronPort-HdrOrdr: A9a23:74hY2qE2GeAaDVq/pLqELMeALOsnbusQ8zAXPiBKJCC9E/bo8v xG+c5w6faaslkssR0b9+xoW5PwI080l6QU3WB5B97LMDUO0FHCEGgI1/qA/9SPIUzDHu4279 YbT0B9YueAcGSTW6zBkXWF+9VL+qj5zEix792uq0uE1WtRGtldBwESMHf9LmRGADNoKLAeD5 Sm6s9Ot1ObCA8qhpTSPAhiYwDbzee77a7bXQ== X-IronPort-AV: E=Sophos;i="5.84,329,1620705600"; d="scan'208";a="51032081" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 2/3] x86/amd: Enumeration for speculative features/hints Date: Tue, 17 Aug 2021 15:30:05 +0100 Message-ID: <20210817143006.2821-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20210817143006.2821-1-andrew.cooper3@citrix.com> References: <20210817143006.2821-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 There is a step change in speculation protections between the Zen1 and Zen2 microarchitectures. Zen1 and older have no special support. Control bits in non-architectural MSRs are used to make lfence be dispatch-serialising (Spectre v1), and to disable Memory Disambiguation (Speculative Store Bypass). IBPB was retrofitted in a microcode update, and software methods are required for Spectre v2 protections. Because the bit controlling Memory Disambiguation is model specific, hypervisors are expected to expose a MSR_VIRT_SPEC_CTRL interface which abstracts the model specific details. Zen2 and later implement the MSR_SPEC_CTRL interface in hardware, and virtualise the interface for HVM guests to use. A number of hint bits are specified too to help guide OS software to the most efficient mitigation strategy. Zen3 introduced a new feature, Predictive Store Forwarding, along with a control to disable it in sensitive code. Add CPUID and VMCB details for all the new functionality. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu The current revision of the APM is buggy with the spec_ctrl VMCB field. It is currently described as a dword, but a correction is pending to change it to qword like every other MSR. --- tools/libs/light/libxl_cpuid.c | 10 ++++++++++ tools/misc/xen-cpuid.c | 8 +++++++- xen/arch/x86/hvm/svm/svm.c | 1 + xen/arch/x86/hvm/svm/vmcb.c | 1 + xen/include/asm-x86/cpufeature.h | 5 +++++ xen/include/asm-x86/hvm/svm/svm.h | 2 ++ xen/include/asm-x86/hvm/svm/vmcb.h | 4 +++- xen/include/asm-x86/msr-index.h | 3 +++ xen/include/public/arch-x86/cpufeatureset.h | 10 ++++++++++ 9 files changed, 42 insertions(+), 2 deletions(-) diff --git a/tools/libs/light/libxl_cpuid.c b/tools/libs/light/libxl_cpuid.c index b2c673841a45..5ed7a87180b8 100644 --- a/tools/libs/light/libxl_cpuid.c +++ b/tools/libs/light/libxl_cpuid.c @@ -274,8 +274,18 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list *cpuid, const char* str) {"rstr-fp-err-ptrs", 0x80000008, NA, CPUID_REG_EBX, 2, 1}, {"wbnoinvd", 0x80000008, NA, CPUID_REG_EBX, 9, 1}, {"ibpb", 0x80000008, NA, CPUID_REG_EBX, 12, 1}, + {"ibrs", 0x80000008, NA, CPUID_REG_EBX, 14, 1}, + {"amd-stibp", 0x80000008, NA, CPUID_REG_EBX, 15, 1}, + {"ibrs-always", 0x80000008, NA, CPUID_REG_EBX, 16, 1}, + {"stibp-always", 0x80000008, NA, CPUID_REG_EBX, 17, 1}, + {"ibrs-fast", 0x80000008, NA, CPUID_REG_EBX, 18, 1}, + {"ibrs-same-mode", 0x80000008, NA, CPUID_REG_EBX, 19, 1}, {"no-lmsl", 0x80000008, NA, CPUID_REG_EBX, 20, 1}, {"ppin", 0x80000008, NA, CPUID_REG_EBX, 23, 1}, + {"amd-ssbd", 0x80000008, NA, CPUID_REG_EBX, 24, 1}, + {"virt-ssbd", 0x80000008, NA, CPUID_REG_EBX, 25, 1}, + {"ssb-no", 0x80000008, NA, CPUID_REG_EBX, 26, 1}, + {"psfd", 0x80000008, NA, CPUID_REG_EBX, 28, 1}, {"nc", 0x80000008, NA, CPUID_REG_ECX, 0, 8}, {"apicidsize", 0x80000008, NA, CPUID_REG_ECX, 12, 4}, diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c index 735bcf8f0e60..713c1657f4df 100644 --- a/tools/misc/xen-cpuid.c +++ b/tools/misc/xen-cpuid.c @@ -148,12 +148,18 @@ static const char *const str_e8b[32] = [ 0] = "clzero", [ 2] = "rstr-fp-err-ptrs", - /* [ 8] */ [ 9] = "wbnoinvd", + /* [ 8] */ [ 9] = "wbnoinvd", [12] = "ibpb", + [14] = "ibrs", [15] = "amd-stibp", + [16] = "ibrs-always", [17] = "stibp-always", + [18] = "ibrs-fast", [19] = "ibrs-same-mode", [20] = "no-lmsl", /* [22] */ [23] = "ppin", + [24] = "amd-ssbd", [25] = "virt-ssbd", + [26] = "ssb-no", + [28] = "psfd", }; static const char *const str_7d0[32] = diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c index 642a64b747ae..8dc92c8b9f96 100644 --- a/xen/arch/x86/hvm/svm/svm.c +++ b/xen/arch/x86/hvm/svm/svm.c @@ -1659,6 +1659,7 @@ const struct hvm_function_table * __init start_svm(void) P(cpu_has_pause_thresh, "Pause-Intercept Filter Threshold"); P(cpu_has_tsc_ratio, "TSC Rate MSR"); P(cpu_has_svm_sss, "NPT Supervisor Shadow Stack"); + P(cpu_has_svm_spec_ctrl, "MSR_SPEC_CTRL virtualisation"); #undef P if ( !printed ) diff --git a/xen/arch/x86/hvm/svm/vmcb.c b/xen/arch/x86/hvm/svm/vmcb.c index 373d5d4af47e..55da9302e5d7 100644 --- a/xen/arch/x86/hvm/svm/vmcb.c +++ b/xen/arch/x86/hvm/svm/vmcb.c @@ -271,6 +271,7 @@ static void __init __maybe_unused build_assertions(void) BUILD_BUG_ON(offsetof(typeof(vmcb), rsp) != 0x5d8); BUILD_BUG_ON(offsetof(typeof(vmcb), rax) != 0x5f8); BUILD_BUG_ON(offsetof(typeof(vmcb), _g_pat) != 0x668); + BUILD_BUG_ON(offsetof(typeof(vmcb), spec_ctrl) != 0x6e0); /* Check struct segment_register against the VMCB segment layout. */ BUILD_BUG_ON(sizeof(vmcb.es) != 16); diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h index 5f6b83f71c21..d9604ff58a49 100644 --- a/xen/include/asm-x86/cpufeature.h +++ b/xen/include/asm-x86/cpufeature.h @@ -128,6 +128,11 @@ /* CPUID level 0x80000007.edx */ #define cpu_has_itsc boot_cpu_has(X86_FEATURE_ITSC) +/* CPUID level 0x80000008.ebx */ +#define cpu_has_amd_ssbd boot_cpu_has(X86_FEATURE_AMD_SSBD) +#define cpu_has_virt_ssbd boot_cpu_has(X86_FEATURE_VIRT_SSBD) +#define cpu_has_ssb_no boot_cpu_has(X86_FEATURE_SSB_NO) + /* CPUID level 0x00000007:0.edx */ #define cpu_has_avx512_4vnniw boot_cpu_has(X86_FEATURE_AVX512_4VNNIW) #define cpu_has_avx512_4fmaps boot_cpu_has(X86_FEATURE_AVX512_4FMAPS) diff --git a/xen/include/asm-x86/hvm/svm/svm.h b/xen/include/asm-x86/hvm/svm/svm.h index bee939156f4f..05e968502694 100644 --- a/xen/include/asm-x86/hvm/svm/svm.h +++ b/xen/include/asm-x86/hvm/svm/svm.h @@ -76,6 +76,7 @@ extern u32 svm_feature_flags; #define SVM_FEATURE_VLOADSAVE 15 /* virtual vmload/vmsave */ #define SVM_FEATURE_VGIF 16 /* Virtual GIF */ #define SVM_FEATURE_SSS 19 /* NPT Supervisor Shadow Stacks */ +#define SVM_FEATURE_SPEC_CTRL 20 /* MSR_SPEC_CTRL virtualisation */ #define cpu_has_svm_feature(f) (svm_feature_flags & (1u << (f))) #define cpu_has_svm_npt cpu_has_svm_feature(SVM_FEATURE_NPT) @@ -91,6 +92,7 @@ extern u32 svm_feature_flags; #define cpu_has_tsc_ratio cpu_has_svm_feature(SVM_FEATURE_TSCRATEMSR) #define cpu_has_svm_vloadsave cpu_has_svm_feature(SVM_FEATURE_VLOADSAVE) #define cpu_has_svm_sss cpu_has_svm_feature(SVM_FEATURE_SSS) +#define cpu_has_svm_spec_ctrl cpu_has_svm_feature(SVM_FEATURE_SPEC_CTRL) #define SVM_PAUSEFILTER_INIT 4000 #define SVM_PAUSETHRESH_INIT 1000 diff --git a/xen/include/asm-x86/hvm/svm/vmcb.h b/xen/include/asm-x86/hvm/svm/vmcb.h index 9e1e42f4941c..4fa2ddfb2ff2 100644 --- a/xen/include/asm-x86/hvm/svm/vmcb.h +++ b/xen/include/asm-x86/hvm/svm/vmcb.h @@ -521,7 +521,9 @@ struct vmcb_struct { u64 _lastbranchtoip; /* cleanbit 10 */ u64 _lastintfromip; /* cleanbit 10 */ u64 _lastinttoip; /* cleanbit 10 */ - u64 res17[301]; + u64 res17[9]; + u64 spec_ctrl; + u64 res18[291]; }; struct svm_domain { diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h index a14841055f0e..903923e5a58b 100644 --- a/xen/include/asm-x86/msr-index.h +++ b/xen/include/asm-x86/msr-index.h @@ -33,6 +33,7 @@ #define SPEC_CTRL_IBRS (_AC(1, ULL) << 0) #define SPEC_CTRL_STIBP (_AC(1, ULL) << 1) #define SPEC_CTRL_SSBD (_AC(1, ULL) << 2) +#define SPEC_CTRL_PSFD (_AC(1, ULL) << 7) #define MSR_PRED_CMD 0x00000049 #define PRED_CMD_IBPB (_AC(1, ULL) << 0) @@ -137,6 +138,8 @@ #define VM_CR_INIT_REDIRECTION (_AC(1, ULL) << 1) #define VM_CR_SVM_DISABLE (_AC(1, ULL) << 4) +#define MSR_VIRT_SPEC_CTRL 0xc001011f /* Layout matches MSR_SPEC_CTRL */ + /* * Legacy MSR constants in need of cleanup. No new MSRs below this comment. */ diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h index 380b51b1b3b8..f1b072cf44cc 100644 --- a/xen/include/public/arch-x86/cpufeatureset.h +++ b/xen/include/public/arch-x86/cpufeatureset.h @@ -255,8 +255,18 @@ XEN_CPUFEATURE(CLZERO, 8*32+ 0) /*A CLZERO instruction */ XEN_CPUFEATURE(RSTR_FP_ERR_PTRS, 8*32+ 2) /*A (F)X{SAVE,RSTOR} always saves/restores FPU Error pointers */ XEN_CPUFEATURE(WBNOINVD, 8*32+ 9) /* WBNOINVD instruction */ XEN_CPUFEATURE(IBPB, 8*32+12) /*A IBPB support only (no IBRS, used by AMD) */ +XEN_CPUFEATURE(IBRS, 8*32+14) /* MSR_SPEC_CTRL.IBRS */ +XEN_CPUFEATURE(AMD_STIBP, 8*32+15) /* MSR_SPEC_CTRL.STIBP */ +XEN_CPUFEATURE(IBRS_ALWAYS, 8*32+16) /* IBRS preferred always on */ +XEN_CPUFEATURE(STIBP_ALWAYS, 8*32+17) /* STIBP preferred always on */ +XEN_CPUFEATURE(IBRS_FAST, 8*32+18) /* IBRS preferred over software options */ +XEN_CPUFEATURE(IBRS_SAME_MODE, 8*32+19) /* IBRS provides same-mode protection */ XEN_CPUFEATURE(NO_LMSL, 8*32+20) /*S EFER.LMSLE no longer supported. */ XEN_CPUFEATURE(AMD_PPIN, 8*32+23) /* Protected Processor Inventory Number */ +XEN_CPUFEATURE(AMD_SSBD, 8*32+24) /* MSR_SPEC_CTRL.SSBD available */ +XEN_CPUFEATURE(VIRT_SSBD, 8*32+25) /* MSR_VIRT_SPEC_CTRL.SSBD */ +XEN_CPUFEATURE(SSB_NO, 8*32+26) /* Hardware not vulnerable to SSB */ +XEN_CPUFEATURE(PSFD, 8*32+28) /* MSR_SPEC_CTRL.PSFD */ /* Intel-defined CPU features, CPUID level 0x00000007:0.edx, word 9 */ XEN_CPUFEATURE(AVX512_4VNNIW, 9*32+ 2) /*A AVX512 Neural Network Instructions */ From patchwork Tue Aug 17 14:30:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 12441599 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F1A3C4320A for ; Tue, 17 Aug 2021 14:30:44 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4B08260F39 for ; Tue, 17 Aug 2021 14:30:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 4B08260F39 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.167905.306567 (Exim 4.92) (envelope-from ) id 1mG06s-0006Eu-SQ; Tue, 17 Aug 2021 14:30:34 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 167905.306567; Tue, 17 Aug 2021 14:30:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mG06s-0006Eh-Op; Tue, 17 Aug 2021 14:30:34 +0000 Received: by outflank-mailman (input) for mailman id 167905; Tue, 17 Aug 2021 14:30:33 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mG06r-0005xM-G9 for xen-devel@lists.xenproject.org; Tue, 17 Aug 2021 14:30:33 +0000 Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 0c2e31b0-1b19-4f7d-9dd6-62d4ef5f4e36; Tue, 17 Aug 2021 14:30:32 +0000 (UTC) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0c2e31b0-1b19-4f7d-9dd6-62d4ef5f4e36 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1629210632; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CwPEY11jKBjTwYDG78a/S8NdZCFVyFcp3aHlBMgwn68=; b=Fz/xyuPxdwDoM7BXlBqT7TXMardpdtEyK9LTYeD8EepBP3fPCtu9Q0QG AFNDrysYEvWC/DCtnrfcogBVb1xVH9IuzeBm8c8+6Wu2Bt591eanZYJpk 2eRehsyLDEmsGwApYzKnqjRBfnORCEkn3Ffk66P3opT+ZmKz9qEmS2FH/ w=; Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: +zfpUIN/3CMbW8KqFZl1IaUvrZUaK8HrnCY4fdc9eGQIZ5a2Oc/YkORdGE9m/NPAubn4TIkHr9 7cc3n0fL/NBh2Xdjgu/ZmZfGnSEby1nFDZhdpygShaZk78whZpxFiLCOHnTJ24Z7+tD+5qLCKn bS67DD8FruVh60HAswrJXXwXv3kPdJ8A2QNZusHU5dRqn6z+H4eEJdNc7CP5nIMFLFS61/0bD1 5ZowYb3sV1Mq/lnnM6/Hm4UKCbkkKbdAW/IBWYhLCfbjyYz11O1VyuWxfqi7q/4pBdQjB5DnGW bHkwObD747m87X3XoRPatBAd X-SBRS: 5.1 X-MesageID: 50642786 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.83 X-Policy: $RELAYED IronPort-HdrOrdr: A9a23:TkRFvKE8UCi5xR0ApLqFXpHXdLJyesId70hD6qkoc20zTiX5rb HsoB1p726xtN9xYgBmpTnuAtjifZqxz/JICMwqTNOftWrdySuVxeNZnOnfKlTbckWUnINgPO VbAs1D4bbLY2SS+Pyb3ODOKbcdKbe8n5xAzt2uqUuFBTsaEp2JaGxCe0Om+pkcfngFObMpUJ Wb/cZJvDymZDAeadm6HGAMW6zZq8TMj4+OW290Ozc3rA2FyS+u6KbnH3Gjr2Ajuh50sMYfGF L+4nzED2SYwoyG4w6Z03WW44VdmdPnxNcGDMuQitINIjGpjgqzfoxuV7CLoThw+YiUmSMXuc iJpw1lM9V46nvXcG3wqRzx2xP42DJr73P501eXjXbqvMS8TjMnDMhKg55fb3LimjgdlcA51L gO03OStpJRAx+FlCPh58LQXxUvjUawqWpKq59ss1VPFY8FLLNBp40W+01YVJ0aGjjh9YwhGO 5ySMnB+fdfazqhHjnkl3gqxMbpUmU4Hx+ATERHsNeSySJKkHdwyFZdzNADn20H6Ik2R/B/lq z52u0BrsACcidhBZgNSNvob6CMeyPwqVOlChPYHbyRfJt3cU4ki/XMkfsIDSGRCeM1JOhbou WBbLp8jx9HR6vDM7z04HR7yGGwfIzmZ0WU9ih33ekChlX/LICbRxG+dA== X-IronPort-AV: E=Sophos;i="5.84,329,1620705600"; d="scan'208";a="50642786" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , Wei Liu , =?utf-8?q?R?= =?utf-8?q?oger_Pau_Monn=C3=A9?= Subject: [PATCH 3/3] x86/amd: Use newer SSBD mechanisms if they exist Date: Tue, 17 Aug 2021 15:30:06 +0100 Message-ID: <20210817143006.2821-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20210817143006.2821-1-andrew.cooper3@citrix.com> References: <20210817143006.2821-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 The opencoded legacy Memory Disambiguation logic in init_amd() neglected Fam19h for the Zen3 microarchitecture. In practice, all Zen2 based system (AMD Fam17h Model >= 0x30 and Hygon Fam18h Model >= 0x4) have the architectural MSR_SPEC_CTRL and the SSBD bit within it. Implement the algorithm given in AMD's SSBD whitepaper, and leave a printk_once() behind in the case that no controls can be found. This now means that a user choosing `spec-ctrl=no-ssb` will actually turn off Memory Disambiguation on Fam19h/Zen3 systems. This still remains a single system-wide setting (for now), and is not context switched between vCPUs. As such, it doesn't interact with Intel's use of MSR_SPEC_CTRL and default_xen_spec_ctrl (yet). Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/cpu/amd.c | 69 +++++++++++++++++++++++++++++++++++------------- xen/arch/x86/cpu/cpu.h | 1 + xen/arch/x86/cpu/hygon.c | 10 +------ xen/arch/x86/spec_ctrl.c | 5 +++- 4 files changed, 57 insertions(+), 28 deletions(-) diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c index 2260eef3aab5..567565199373 100644 --- a/xen/arch/x86/cpu/amd.c +++ b/xen/arch/x86/cpu/amd.c @@ -681,6 +681,56 @@ void amd_init_lfence(struct cpuinfo_x86 *c) c->x86_capability); } +/* + * Refer to the AMD Speculative Store Bypass whitepaper: + * https://developer.amd.com/wp-content/resources/124441_AMD64_SpeculativeStoreBypassDisable_Whitepaper_final.pdf + */ +void amd_init_ssbd(const struct cpuinfo_x86 *c) +{ + int bit = -1; + + if (cpu_has_ssb_no) + return; + + if (cpu_has_amd_ssbd) { + wrmsrl(MSR_SPEC_CTRL, opt_ssbd ? SPEC_CTRL_SSBD : 0); + return; + } + + if (cpu_has_virt_ssbd) { + wrmsrl(MSR_VIRT_SPEC_CTRL, opt_ssbd ? SPEC_CTRL_SSBD : 0); + return; + } + + switch (c->x86) { + case 0x15: bit = 54; break; + case 0x16: bit = 33; break; + case 0x17: + case 0x18: bit = 10; break; + } + + if (bit >= 0) { + uint64_t val, mask = 1ull << bit; + + if (rdmsr_safe(MSR_AMD64_LS_CFG, val) || + ({ + val &= ~mask; + if ( opt_ssbd ) + val |= mask; + false; + }) || + wrmsr_safe(MSR_AMD64_LS_CFG, val) || + ({ + rdmsrl(MSR_AMD64_LS_CFG, val); + (val & mask) != (opt_ssbd * mask); + })) + bit = -1; + } + + if (bit < 0) + printk_once(XENLOG_ERR "No SSBD controls available\n"); +} + static void init_amd(struct cpuinfo_x86 *c) { u32 l, h; @@ -731,24 +781,7 @@ static void init_amd(struct cpuinfo_x86 *c) else /* Implicily "== 0x10 || >= 0x12" by being 64bit. */ amd_init_lfence(c); - /* - * If the user has explicitly chosen to disable Memory Disambiguation - * to mitigiate Speculative Store Bypass, poke the appropriate MSR. - */ - if (opt_ssbd) { - int bit = -1; - - switch (c->x86) { - case 0x15: bit = 54; break; - case 0x16: bit = 33; break; - case 0x17: bit = 10; break; - } - - if (bit >= 0 && !rdmsr_safe(MSR_AMD64_LS_CFG, value)) { - value |= 1ull << bit; - wrmsr_safe(MSR_AMD64_LS_CFG, value); - } - } + amd_init_ssbd(c); /* MFENCE stops RDTSC speculation */ if (!cpu_has_lfence_dispatch) diff --git a/xen/arch/x86/cpu/cpu.h b/xen/arch/x86/cpu/cpu.h index 1ac3b2867a04..1a5b3918b37e 100644 --- a/xen/arch/x86/cpu/cpu.h +++ b/xen/arch/x86/cpu/cpu.h @@ -21,3 +21,4 @@ extern bool detect_extended_topology(struct cpuinfo_x86 *c); void early_init_amd(struct cpuinfo_x86 *c); void amd_log_freq(const struct cpuinfo_x86 *c); void amd_init_lfence(struct cpuinfo_x86 *c); +void amd_init_ssbd(const struct cpuinfo_x86 *c); diff --git a/xen/arch/x86/cpu/hygon.c b/xen/arch/x86/cpu/hygon.c index 67e23c5df9e3..56792146739e 100644 --- a/xen/arch/x86/cpu/hygon.c +++ b/xen/arch/x86/cpu/hygon.c @@ -33,15 +33,7 @@ static void init_hygon(struct cpuinfo_x86 *c) unsigned long long value; amd_init_lfence(c); - - /* - * If the user has explicitly chosen to disable Memory Disambiguation - * to mitigiate Speculative Store Bypass, poke the appropriate MSR. - */ - if (opt_ssbd && !rdmsr_safe(MSR_AMD64_LS_CFG, value)) { - value |= 1ull << 10; - wrmsr_safe(MSR_AMD64_LS_CFG, value); - } + amd_init_ssbd(c); /* MFENCE stops RDTSC speculation */ if (!cpu_has_lfence_dispatch) diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index 9bf0fbf99813..0850afc09358 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -326,20 +326,23 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps) (caps & ARCH_CAPS_IBRS_ALL) ? " IBRS_ALL" : "", (caps & ARCH_CAPS_RSBA) ? " RSBA" : "", (caps & ARCH_CAPS_SKIP_L1DFL) ? " SKIP_L1DFL" : "", + (e8b & cpufeat_mask(X86_FEATURE_SSB_NO)) || (caps & ARCH_CAPS_SSB_NO) ? " SSB_NO" : "", (caps & ARCH_CAPS_MDS_NO) ? " MDS_NO" : "", (caps & ARCH_CAPS_TAA_NO) ? " TAA_NO" : ""); /* Hardware features which need driving to mitigate issues. */ - printk(" Hardware features:%s%s%s%s%s%s%s%s\n", + printk(" Hardware features:%s%s%s%s%s%s%s%s%s\n", (e8b & cpufeat_mask(X86_FEATURE_IBPB)) || (_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBPB" : "", (_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBRS" : "", (_7d0 & cpufeat_mask(X86_FEATURE_STIBP)) ? " STIBP" : "", + (e8b & cpufeat_mask(X86_FEATURE_AMD_SSBD)) || (_7d0 & cpufeat_mask(X86_FEATURE_SSBD)) ? " SSBD" : "", (_7d0 & cpufeat_mask(X86_FEATURE_L1D_FLUSH)) ? " L1D_FLUSH" : "", (_7d0 & cpufeat_mask(X86_FEATURE_MD_CLEAR)) ? " MD_CLEAR" : "", (_7d0 & cpufeat_mask(X86_FEATURE_SRBDS_CTRL)) ? " SRBDS_CTRL" : "", + (e8b & cpufeat_mask(X86_FEATURE_VIRT_SSBD)) ? " VIRT_SSBD" : "", (caps & ARCH_CAPS_TSX_CTRL) ? " TSX_CTRL" : ""); /* Compiled-in support which pertains to mitigations. */