From patchwork Wed Jan 24 02:41:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528417 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F02FBE6E; Wed, 24 Jan 2024 02:42:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064155; cv=none; b=T2zwGThiu+zmpVfGuIQbQArg/te3Vo9s9tuP6GJS51cc8hkS+HuZbtpX0D4+vIL15HoW8MVBzgwbTMzYcs2V57r3V97gnn0Wu3ureoxps6RnjamNudFp+ypaSTUzFK57J0pYftfwmxgUZfIedCoF0tqzc1IBjGHLrKjnLwidJog= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064155; c=relaxed/simple; bh=VvF0PBScwCsWmxl1ZnSXv9NBr/c+xJKpYRCNE4wuI0c=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NKn+N+TcACWYtpXDuxVJwMza4noRByafF8N6uBxUSWtapDYHWpNAgi5rMfzwgfBRliwxfESu1hziRUpRPHPn7b7YJWP8ldPlj5ChhBpNpScMcSw6yYQgcZepIDCEH/ZEHZQT5hyDEpKHLkO/WG24mPm8MjETY1bD7GjPuFOcfJg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=QKJ7czjw; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="QKJ7czjw" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064153; x=1737600153; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VvF0PBScwCsWmxl1ZnSXv9NBr/c+xJKpYRCNE4wuI0c=; b=QKJ7czjwLaBA5ABb6qSmRj2OSeRhFo/hR81D2BgiiMPZoWiSGj6ORfOb MsHy6vkL2sTEKXVxe3R/CEOjTE/oYh65ZaxYdSRfMTi4hCCGzXXHYfTw5 ir0mDebBqCBE5mN0z4Qq3/Iv1LTeY0/ONeSLwR97TdyqzoThN7RH1r8L8 PxpZ64B3doOhK37tFErsLdXdYNDhKMMk3GKJXMEjL5RhVocplLovmX92W 4rSNjC93g0K5Gw0yfGn/bfh3R7lN5NNpqq9r/zHG6fsaM2S08KVB16CYg R2joTnZ1nZeV9erfBauGvzEcJ+gMOoH1Ge6X6U4RL5Hs07CWzuz1oQkyr A==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586417" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586417" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:30 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825812" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:30 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 01/27] x86/fpu/xstate: Always preserve non-user xfeatures/flags in __state_perm Date: Tue, 23 Jan 2024 18:41:34 -0800 Message-Id: <20240124024200.102792-2-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson When granting userspace or a KVM guest access to an xfeature, preserve the entity's existing supervisor and software-defined permissions as tracked by __state_perm, i.e. use __state_perm to track *all* permissions even though all supported supervisor xfeatures are granted to all FPUs and FPU_GUEST_PERM_LOCKED disallows changing permissions. Effectively clobbering supervisor permissions results in inconsistent behavior, as xstate_get_group_perm() will report supervisor features for process that do NOT request access to dynamic user xfeatures, whereas any and all supervisor features will be absent from the set of permissions for any process that is granted access to one or more dynamic xfeatures (which right now means AMX). The inconsistency isn't problematic because fpu_xstate_prctl() already strips out everything except user xfeatures: case ARCH_GET_XCOMP_PERM: /* * Lockless snapshot as it can also change right after the * dropping the lock. */ permitted = xstate_get_host_group_perm(); permitted &= XFEATURE_MASK_USER_SUPPORTED; return put_user(permitted, uptr); case ARCH_GET_XCOMP_GUEST_PERM: permitted = xstate_get_guest_group_perm(); permitted &= XFEATURE_MASK_USER_SUPPORTED; return put_user(permitted, uptr); and similarly KVM doesn't apply the __state_perm to supervisor states (kvm_get_filtered_xcr0() incorporates xstate_get_guest_group_perm()): case 0xd: { u64 permitted_xcr0 = kvm_get_filtered_xcr0(); u64 permitted_xss = kvm_caps.supported_xss; But if KVM in particular were to ever change, dropping supervisor permissions would result in subtle bugs in KVM's reporting of supported CPUID settings. And the above behavior also means that having supervisor xfeatures in __state_perm is correctly handled by all users. Dropping supervisor permissions also creates another landmine for KVM. If more dynamic user xfeatures are ever added, requesting access to multiple xfeatures in separate ARCH_REQ_XCOMP_GUEST_PERM calls will result in the second invocation of __xstate_request_perm() computing the wrong ksize, as as the mask passed to xstate_calculate_size() would not contain *any* supervisor features. Commit 781c64bfcb73 ("x86/fpu/xstate: Handle supervisor states in XSTATE permissions") fudged around the size issue for userspace FPUs, but for reasons unknown skipped guest FPUs. Lack of a fix for KVM "works" only because KVM doesn't yet support virtualizing features that have supervisor xfeatures, i.e. as of today, KVM guest FPUs will never need the relevant xfeatures. Simply extending the hack-a-fix for guests would temporarily solve the ksize issue, but wouldn't address the inconsistency issue and would leave another lurking pitfall for KVM. KVM support for virtualizing CET will likely add CET_KERNEL as a guest-only xfeature, i.e. CET_KERNEL will not be set in xfeatures_mask_supervisor() and would again be dropped when granting access to dynamic xfeatures. Note, the existing clobbering behavior is rather subtle. The @permitted parameter to __xstate_request_perm() comes from: permitted = xstate_get_group_perm(guest); which is either fpu->guest_perm.__state_perm or fpu->perm.__state_perm, where __state_perm is initialized to: fpu->perm.__state_perm = fpu_kernel_cfg.default_features; and copied to the guest side of things: /* Same defaults for guests */ fpu->guest_perm = fpu->perm; fpu_kernel_cfg.default_features contains everything except the dynamic xfeatures, i.e. everything except XFEATURE_MASK_XTILE_DATA: fpu_kernel_cfg.default_features = fpu_kernel_cfg.max_features; fpu_kernel_cfg.default_features &= ~XFEATURE_MASK_USER_DYNAMIC; When __xstate_request_perm() restricts the local "mask" variable to compute the user state size: mask &= XFEATURE_MASK_USER_SUPPORTED; usize = xstate_calculate_size(mask, false); it subtly overwrites the target __state_perm with "mask" containing only user xfeatures: perm = guest ? &fpu->guest_perm : &fpu->perm; /* Pairs with the READ_ONCE() in xstate_get_group_perm() */ WRITE_ONCE(perm->__state_perm, mask); Cc: Maxim Levitsky Cc: Weijiang Yang Cc: Dave Hansen Cc: Paolo Bonzini Cc: Peter Zijlstra Cc: Chao Gao Cc: Rick Edgecombe Cc: John Allen Cc: kvm@vger.kernel.org Link: https://lore.kernel.org/all/ZTqgzZl-reO1m01I@google.com Signed-off-by: Sean Christopherson Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky Reviewed-by: Rick Edgecombe --- arch/x86/kernel/fpu/xstate.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index 117e74c44e75..07911532b108 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -1601,16 +1601,20 @@ static int __xstate_request_perm(u64 permitted, u64 requested, bool guest) if ((permitted & requested) == requested) return 0; - /* Calculate the resulting kernel state size */ + /* + * Calculate the resulting kernel state size. Note, @permitted also + * contains supervisor xfeatures even though supervisor are always + * permitted for kernel and guest FPUs, and never permitted for user + * FPUs. + */ mask = permitted | requested; - /* Take supervisor states into account on the host */ - if (!guest) - mask |= xfeatures_mask_supervisor(); ksize = xstate_calculate_size(mask, compacted); - /* Calculate the resulting user state size */ - mask &= XFEATURE_MASK_USER_SUPPORTED; - usize = xstate_calculate_size(mask, false); + /* + * Calculate the resulting user state size. Take care not to clobber + * the supervisor xfeatures in the new mask! + */ + usize = xstate_calculate_size(mask & XFEATURE_MASK_USER_SUPPORTED, false); if (!guest) { ret = validate_sigaltstack(usize); From patchwork Wed Jan 24 02:41:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528419 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BFA2C63AA; Wed, 24 Jan 2024 02:42:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064155; cv=none; b=Cntl0zVV7b24EdpO63ekpzm4fkDix2UfnG27lPU+/4ANJ+ep3SNoMIQzACZRffujqsmOgVkNlevOqQOqsUHuTyVsmsmm1uZmfHG7Fv6GA4/3CHA05lBDWPz/eOTTTTzjdnDDdqohSqgSRNEzUX1wlulfY7XRCwyuTYxfhf+MF3M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064155; c=relaxed/simple; bh=O+In4X9Kp5bd1+hICvCOfi4/QLtmFNAek1QhOW3ItLM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=TxvXNz5QW40sb2f3z4wTDyAfKjSbf6dNJvzMAxCR9mndOaaS3WQ/eV4rr0V7arbEbP3SGMUmMElScFavzYb55a/6Qldlekp9fwii3bIankAa6LD3tagrf0/dsJj7FFzkgaUDYko3gVV2upDZ64243NkGwbiAFhFDS02Q48ms/GI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=R2d4WT9Q; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="R2d4WT9Q" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064153; x=1737600153; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=O+In4X9Kp5bd1+hICvCOfi4/QLtmFNAek1QhOW3ItLM=; b=R2d4WT9QSXucDnqCFX6yTgxDpy48S/wywNI+GxfG69mu3j2fP4SWLtwH U9jfJR4/tA1qGruS9GatrKHUOGd3fTjxYTWk/8gbhehFuXH8149pK33t3 2xuAEEgWa2YGQGl/DTAabfSzgAz3d77GAdSckQTyp989xRpSG5Cl9eCPs C4fwGanQCgfQrUNeJZUFTCFB7DehrmPMC7G4O7/l8exUEpaSap5Z/cAiK l9GjiParh3WaeXS92sdnzd2n5p/F3NX7NboNXAgitl0jWcs7HLnFzJZ8I QwvEpIv5lrbaNUh2E6mEa0h72I+oPr8YHOV4E382kYAvfY6kLGVrvNl8D w==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586424" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586424" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:31 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825815" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:30 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 02/27] x86/fpu/xstate: Refine CET user xstate bit enabling Date: Tue, 23 Jan 2024 18:41:35 -0800 Message-Id: <20240124024200.102792-3-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Remove XFEATURE_CET_USER entry from dependency array as the entry doesn't reflect true dependency between CET features and the user xstate bit. Enable the bit in fpu_kernel_cfg.max_features when either SHSTK or IBT is available. Both user mode shadow stack and indirect branch tracking features depend on XFEATURE_CET_USER bit in XSS to automatically save/restore user mode xstate registers, i.e., IA32_U_CET and IA32_PL3_SSP whenever necessary. Note, the issue, i.e., CPUID only enumerates IBT but no SHSTK is resulted from CET KVM series which synthesizes guest CPUIDs based on userspace settings,in real world the case is rare. In other words, the existing dependency check is correct when only user mode SHSTK is available. Signed-off-by: Yang Weijiang Reviewed-by: Rick Edgecombe Tested-by: Rick Edgecombe --- arch/x86/kernel/fpu/xstate.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index 07911532b108..f6b98693da59 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -73,7 +73,6 @@ static unsigned short xsave_cpuid_features[] __initdata = { [XFEATURE_PT_UNIMPLEMENTED_SO_FAR] = X86_FEATURE_INTEL_PT, [XFEATURE_PKRU] = X86_FEATURE_OSPKE, [XFEATURE_PASID] = X86_FEATURE_ENQCMD, - [XFEATURE_CET_USER] = X86_FEATURE_SHSTK, [XFEATURE_XTILE_CFG] = X86_FEATURE_AMX_TILE, [XFEATURE_XTILE_DATA] = X86_FEATURE_AMX_TILE, }; @@ -798,6 +797,14 @@ void __init fpu__init_system_xstate(unsigned int legacy_size) fpu_kernel_cfg.max_features &= ~BIT_ULL(i); } + /* + * CET user mode xstate bit has been cleared by above sanity check. + * Now pick it up if either SHSTK or IBT is available. Either feature + * depends on the xstate bit to save/restore user mode states. + */ + if (boot_cpu_has(X86_FEATURE_SHSTK) || boot_cpu_has(X86_FEATURE_IBT)) + fpu_kernel_cfg.max_features |= BIT_ULL(XFEATURE_CET_USER); + if (!cpu_feature_enabled(X86_FEATURE_XFD)) fpu_kernel_cfg.max_features &= ~XFEATURE_MASK_USER_DYNAMIC; From patchwork Wed Jan 24 02:41:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528418 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0642B6FA5; Wed, 24 Jan 2024 02:42:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064155; cv=none; b=u+PVAMcIp4CrQb7DKJtqILwfmpX/ymeia2GhPpad8XVdmZmlCpeqVzs90l2MuPtpKuh5EHbQ2DypF3nTyOy76Ku7/4QhWW4zqP2ZWF5+mTMWBhQ/tLxIthaKy/tyERfKJ2nedJbDU39jKl9bNZMilJxgD+3Cylo7Nm1CRXSvzws= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064155; c=relaxed/simple; bh=zDQRJW1MEJIyRReHbn6CnswFHThkqwgVHPSva/vwncA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=g1C8/ezvzLQcpqMD3rgR/K9WQd3omhQAcs8TCsFO1gfvCm52H4K2nBJz5dr2/YbxMS0v28V9hkfFutgiknX/ACxRoHflYUmK/isV6866MwupshgH2XdW1MUWg2+TdpHNyLGw/XFlgZUQnLdSFVG4IJRPEJ11PRLjJJ2uUCeUWxQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ICedOzYC; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ICedOzYC" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064154; x=1737600154; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zDQRJW1MEJIyRReHbn6CnswFHThkqwgVHPSva/vwncA=; b=ICedOzYCNNbC2x+WVQf4N7Da+88kqau+bElRGXhVXSHtGVDd9ydKgFoo oCdOXPG8kNlbRu6hcWW8/cq5+lr4jtP3KfY/onDCeTErjLF0k7uT3h+I8 X/UC2eK8+LcQgvHMcpUTIvj8RQPUN3f+0l371dkToMtuCnDmOME6GSArm x2qgcK5qmJJkteLysv5scc/I6DVic3U0dwtmqlEceseH8DfgxBLR8GKWW w49QSKkqZer9PD8Chhlg4g1abwxRfw+97d2SWFiT5tWwHpTrLLFIAvONk fTCV+YJfsVz4Oqn4kaG3+BaWw5gzXeAg0Iu5VrX9iWWr6KDZK3oM+rH87 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586430" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586430" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:31 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825819" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:31 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 03/27] x86/fpu/xstate: Add CET supervisor mode state support Date: Tue, 23 Jan 2024 18:41:36 -0800 Message-Id: <20240124024200.102792-4-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add supervisor mode state support within FPU xstate management framework. Although supervisor shadow stack is not enabled/used today in kernel,KVM requires the support because when KVM advertises shadow stack feature to guest, architecturally it claims the support for both user and supervisor modes for guest OSes(Linux or non-Linux). CET supervisor states not only includes PL{0,1,2}_SSP but also IA32_S_CET MSR, but the latter is not xsave-managed. In virtualization world, guest IA32_S_CET is saved/stored into/from VM control structure. With supervisor xstate support, guest supervisor mode shadow stack state can be properly saved/restored when 1) guest/host FPU context is swapped 2) vCPU thread is sched out/in. The alternative is to enable it in KVM domain, but KVM maintainers NAKed the solution. The external discussion can be found at [*], it ended up with adding the support in kernel instead of KVM domain. Note, in KVM case, guest CET supervisor state i.e., IA32_PL{0,1,2}_MSRs, are preserved after VM-Exit until host/guest fpstates are swapped, but since host supervisor shadow stack is disabled, the preserved MSRs won't hurt host. [*]: https://lore.kernel.org/all/806e26c2-8d21-9cc9-a0b7-7787dd231729@intel.com/ Signed-off-by: Yang Weijiang Reviewed-by: Rick Edgecombe Reviewed-by: Maxim Levitsky --- arch/x86/include/asm/fpu/types.h | 14 ++++++++++++-- arch/x86/include/asm/fpu/xstate.h | 6 +++--- arch/x86/kernel/fpu/xstate.c | 6 +++++- 3 files changed, 20 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/fpu/types.h b/arch/x86/include/asm/fpu/types.h index eb810074f1e7..c6fd13a17205 100644 --- a/arch/x86/include/asm/fpu/types.h +++ b/arch/x86/include/asm/fpu/types.h @@ -116,7 +116,7 @@ enum xfeature { XFEATURE_PKRU, XFEATURE_PASID, XFEATURE_CET_USER, - XFEATURE_CET_KERNEL_UNUSED, + XFEATURE_CET_KERNEL, XFEATURE_RSRVD_COMP_13, XFEATURE_RSRVD_COMP_14, XFEATURE_LBR, @@ -139,7 +139,7 @@ enum xfeature { #define XFEATURE_MASK_PKRU (1 << XFEATURE_PKRU) #define XFEATURE_MASK_PASID (1 << XFEATURE_PASID) #define XFEATURE_MASK_CET_USER (1 << XFEATURE_CET_USER) -#define XFEATURE_MASK_CET_KERNEL (1 << XFEATURE_CET_KERNEL_UNUSED) +#define XFEATURE_MASK_CET_KERNEL (1 << XFEATURE_CET_KERNEL) #define XFEATURE_MASK_LBR (1 << XFEATURE_LBR) #define XFEATURE_MASK_XTILE_CFG (1 << XFEATURE_XTILE_CFG) #define XFEATURE_MASK_XTILE_DATA (1 << XFEATURE_XTILE_DATA) @@ -264,6 +264,16 @@ struct cet_user_state { u64 user_ssp; }; +/* + * State component 12 is Control-flow Enforcement supervisor states + */ +struct cet_supervisor_state { + /* supervisor ssp pointers */ + u64 pl0_ssp; + u64 pl1_ssp; + u64 pl2_ssp; +}; + /* * State component 15: Architectural LBR configuration state. * The size of Arch LBR state depends on the number of LBRs (lbr_depth). diff --git a/arch/x86/include/asm/fpu/xstate.h b/arch/x86/include/asm/fpu/xstate.h index d4427b88ee12..3b4a038d3c57 100644 --- a/arch/x86/include/asm/fpu/xstate.h +++ b/arch/x86/include/asm/fpu/xstate.h @@ -51,7 +51,8 @@ /* All currently supported supervisor features */ #define XFEATURE_MASK_SUPERVISOR_SUPPORTED (XFEATURE_MASK_PASID | \ - XFEATURE_MASK_CET_USER) + XFEATURE_MASK_CET_USER | \ + XFEATURE_MASK_CET_KERNEL) /* * A supervisor state component may not always contain valuable information, @@ -78,8 +79,7 @@ * Unsupported supervisor features. When a supervisor feature in this mask is * supported in the future, move it to the supported supervisor feature mask. */ -#define XFEATURE_MASK_SUPERVISOR_UNSUPPORTED (XFEATURE_MASK_PT | \ - XFEATURE_MASK_CET_KERNEL) +#define XFEATURE_MASK_SUPERVISOR_UNSUPPORTED (XFEATURE_MASK_PT) /* All supervisor states including supported and unsupported states. */ #define XFEATURE_MASK_SUPERVISOR_ALL (XFEATURE_MASK_SUPERVISOR_SUPPORTED | \ diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index f6b98693da59..03e166a87d61 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -51,7 +51,7 @@ static const char *xfeature_names[] = "Protection Keys User registers", "PASID state", "Control-flow User registers", - "Control-flow Kernel registers (unused)", + "Control-flow Kernel registers", "unknown xstate feature", "unknown xstate feature", "unknown xstate feature", @@ -73,6 +73,7 @@ static unsigned short xsave_cpuid_features[] __initdata = { [XFEATURE_PT_UNIMPLEMENTED_SO_FAR] = X86_FEATURE_INTEL_PT, [XFEATURE_PKRU] = X86_FEATURE_OSPKE, [XFEATURE_PASID] = X86_FEATURE_ENQCMD, + [XFEATURE_CET_KERNEL] = X86_FEATURE_SHSTK, [XFEATURE_XTILE_CFG] = X86_FEATURE_AMX_TILE, [XFEATURE_XTILE_DATA] = X86_FEATURE_AMX_TILE, }; @@ -277,6 +278,7 @@ static void __init print_xstate_features(void) print_xstate_feature(XFEATURE_MASK_PKRU); print_xstate_feature(XFEATURE_MASK_PASID); print_xstate_feature(XFEATURE_MASK_CET_USER); + print_xstate_feature(XFEATURE_MASK_CET_KERNEL); print_xstate_feature(XFEATURE_MASK_XTILE_CFG); print_xstate_feature(XFEATURE_MASK_XTILE_DATA); } @@ -346,6 +348,7 @@ static __init void os_xrstor_booting(struct xregs_state *xstate) XFEATURE_MASK_BNDCSR | \ XFEATURE_MASK_PASID | \ XFEATURE_MASK_CET_USER | \ + XFEATURE_MASK_CET_KERNEL | \ XFEATURE_MASK_XTILE) /* @@ -546,6 +549,7 @@ static bool __init check_xstate_against_struct(int nr) case XFEATURE_PASID: return XCHECK_SZ(sz, nr, struct ia32_pasid_state); case XFEATURE_XTILE_CFG: return XCHECK_SZ(sz, nr, struct xtile_cfg); case XFEATURE_CET_USER: return XCHECK_SZ(sz, nr, struct cet_user_state); + case XFEATURE_CET_KERNEL: return XCHECK_SZ(sz, nr, struct cet_supervisor_state); case XFEATURE_XTILE_DATA: check_xtile_data_against_struct(sz); return true; default: XSTATE_WARN_ON(1, "No structure for xstate: %d\n", nr); From patchwork Wed Jan 24 02:41:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528420 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 22B4DC120; Wed, 24 Jan 2024 02:42:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064157; cv=none; b=tEsbjKjpjOQRYQyFEg0Mpsk0lkOPbfxvCDz8o2lxeR+9X9hl/8K+zlm5TxBnnSfn9TYdp/RWphOB8Wq0Tk6WhFNRmzxPY8ouiXhN2t8tYl6lNk52IrPZpmEkZMrknUMqlYC55AAIfhN+pPSb8sSnAbbMQ9EyG2cwoSTLfvC1XJ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064157; c=relaxed/simple; bh=f4g1pickiFt8Es6a14IkTLu0NwpeGAkFFhCSd8aA94E=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=HEQYBVp0BTQiDn9FGBpL9VSYc4Yn0rhSPEQ+kN3x7DJFUPLYbymDRpzysfR9oRafglqQ5vxefpv8vVLNC9cPh8wvhiSrlhXkHRb3TD1FhnAVryeeOZGLEsbFNKVCf/ivI5E/sNwLt4SuP6yl9k0hOvfpjBrvOLpr4DcV0IUAXmU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=bgINgB9t; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="bgINgB9t" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064155; x=1737600155; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=f4g1pickiFt8Es6a14IkTLu0NwpeGAkFFhCSd8aA94E=; b=bgINgB9tjyZ6jqCn6LQb2JbCfPILbWuzZqCcAqr8B3Tfa2TtxuNoDUxX rE5l+tyXpjSlEjLUgbxXF8qLGjk1vAPj6uky0VUyuHlBoP6nQlTUrnDTC lkVe5iwuvmbUEQ3Y2tLi5jeVVK4zvAFS+oJrE6splqtkQa14K2efH9s7j dzJ0wPUvGI+igWIm543Ygzk6TRVGZbEqjLDhAL1vGXFHihjTVnWJVvi9c +7CqGSU/21/7+ysVQvWORbC/CVrbiVcbthlRGb840YIj+AWrGse8JTrGg /cNjSpLzZGKWrkjYVmHhh5YwGISIw/JxJVuPgwUWM6O8k8cbdOJXJYSoj Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586437" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586437" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825823" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:32 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 04/27] x86/fpu/xstate: Introduce XFEATURE_MASK_KERNEL_DYNAMIC xfeature set Date: Tue, 23 Jan 2024 18:41:37 -0800 Message-Id: <20240124024200.102792-5-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Define a new XFEATURE_MASK_KERNEL_DYNAMIC mask to specify the features that can be optionally enabled by kernel components. This is similar to XFEATURE_MASK_USER_DYNAMIC in that it contains optional xfeatures that can allows the FPU buffer to be dynamically sized. The difference is that the KERNEL variant contains supervisor features and will be enabled by kernel components that need them, and not directly by the user. Currently it's used by KVM to configure guest dedicated fpstate for calculating the xfeature and fpstate storage size etc. The kernel dynamic xfeatures now only contain XFEATURE_CET_KERNEL, which is supported by host as they're enabled in kernel XSS MSR setting but relevant CPU feature, i.e., supervisor shadow stack, is not enabled in host kernel therefore it can be omitted for normal fpstate by default. Remove the kernel dynamic feature from fpu_kernel_cfg.default_features so that the bits in xstate_bv and xcomp_bv are cleared and xsaves/xrstors can be optimized by HW for normal fpstate. Suggested-by: Dave Hansen Signed-off-by: Yang Weijiang Reviewed-by: Rick Edgecombe --- arch/x86/include/asm/fpu/xstate.h | 5 ++++- arch/x86/kernel/fpu/xstate.c | 1 + 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/fpu/xstate.h b/arch/x86/include/asm/fpu/xstate.h index 3b4a038d3c57..a212d3851429 100644 --- a/arch/x86/include/asm/fpu/xstate.h +++ b/arch/x86/include/asm/fpu/xstate.h @@ -46,9 +46,12 @@ #define XFEATURE_MASK_USER_RESTORE \ (XFEATURE_MASK_USER_SUPPORTED & ~XFEATURE_MASK_PKRU) -/* Features which are dynamically enabled for a process on request */ +/* Features which are dynamically enabled per userspace request */ #define XFEATURE_MASK_USER_DYNAMIC XFEATURE_MASK_XTILE_DATA +/* Features which are dynamically enabled per kernel side request */ +#define XFEATURE_MASK_KERNEL_DYNAMIC XFEATURE_MASK_CET_KERNEL + /* All currently supported supervisor features */ #define XFEATURE_MASK_SUPERVISOR_SUPPORTED (XFEATURE_MASK_PASID | \ XFEATURE_MASK_CET_USER | \ diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index 03e166a87d61..ca4b83c142eb 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -824,6 +824,7 @@ void __init fpu__init_system_xstate(unsigned int legacy_size) /* Clean out dynamic features from default */ fpu_kernel_cfg.default_features = fpu_kernel_cfg.max_features; fpu_kernel_cfg.default_features &= ~XFEATURE_MASK_USER_DYNAMIC; + fpu_kernel_cfg.default_features &= ~XFEATURE_MASK_KERNEL_DYNAMIC; fpu_user_cfg.default_features = fpu_user_cfg.max_features; fpu_user_cfg.default_features &= ~XFEATURE_MASK_USER_DYNAMIC; From patchwork Wed Jan 24 02:41:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528422 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CF5C8C2D5; Wed, 24 Jan 2024 02:42:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064157; cv=none; b=FbHUTt7u34ovzfzyIpSWen3vsDbbfqQrffIpnske7EvOIPqj4a06udxyhVKokR5jCqIdNEK9AVMRewhKBYKiWCwWBBMx+qZ1GCptFUoxDignDKAnvXI2TcEc2XCyOmCFfva4dGbT/o5Ex6cLx4OMYxh8/BQME10C52nzi/xG5+w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064157; c=relaxed/simple; bh=LwDKejYOhhJpC60ZUyFt7HkfISAfrU72v8OHnIWa074=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YqHK3tu8ZIVEiI/xZ1EWvw4BBnF48ei4o0fB1019I46Q5hxa/HOXwMwWxzSpMQjTzDzbzBP404+PDesI/H/bXQAydLVAVqg+/On5ZQGePPzXxrGan9uzJEGYir2pzAvwS6bvPPSpy2gY16opz/GpIN2ZZ/iZVsbGKKHH8Iu2I0c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=aPmaa4pr; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="aPmaa4pr" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064155; x=1737600155; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LwDKejYOhhJpC60ZUyFt7HkfISAfrU72v8OHnIWa074=; b=aPmaa4pr//DeHt5HnBC/MLc/F5c/53JwIZBUx0jVFkI/OA1L963eVeMT XYmMUUXYhrx79Nw2Jbu/YtwVCSVPIki+Oe3rUpDKpdm4dI7UV6Mn8gkDC CVp8B9skpBSXbkFk7waS8+uYhTV2XGrWF2kyfVq5yOqJe8DYH07esonnU 9twOftjGbcIM8gcIdav8C5gZuJSL2YIm9eHmkxq+zwN7wubGlJ8IuAZT/ N3maW2tpYjzvvdUfswe0lgFeB65sRgYfR6BZp0ea3Nk2Y9X5iT9XOSKfA A0bEnO2awk9Slj64OJi6dR8eV1zkyEtHvGcON6x0cTxh0PloSD/0ojc05 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586446" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586446" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:33 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825840" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:32 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 05/27] x86/fpu/xstate: Introduce fpu_guest_cfg for guest FPU configuration Date: Tue, 23 Jan 2024 18:41:38 -0800 Message-Id: <20240124024200.102792-6-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Define new fpu_guest_cfg to hold all guest FPU settings so that it can differ from generic kernel FPU settings, e.g., enabling CET supervisor xstate by default for guest fpstate while it's remained disabled in kernel FPU config. The kernel dynamic xfeatures are specifically used by guest fpstate now, add the mask for guest fpstate so that guest_perm.__state_permit == (fpu_kernel_cfg.default_xfeature | XFEATURE_MASK_KERNEL_DYNAMIC). And if guest fpstate is re-allocated to hold user dynamic xfeatures, the resulting permissions are consumed before calculate new guest fpstate. With new guest FPU config added, there're 3 categories of FPU configs in kernel, the usages and key fields are recapped as below. kernel FPU config: @fpu_kernel_cfg.max_features - all known and CPU supported user and supervisor features except independent kernel features @fpu_kernel_cfg.default_features - all known and CPU supported user and supervisor features except dynamic kernel features, independent kernel features and dynamic userspace features. @fpu_kernel_cfg.max_size - size of compacted buffer with 'fpu_kernel_cfg.max_features' @fpu_kernel_cfg.default_size - size of compacted buffer with 'fpu_kernel_cfg.default_features' user FPU config: @fpu_user_cfg.max_features - all known and CPU supported user features @fpu_user_cfg.default_features - all known and CPU supported user features except dynamic userspace features. @fpu_user_cfg.max_size - size of non-compacted buffer with 'fpu_user_cfg.max_features' @fpu_user_cfg.default_size - size of non-compacted buffer with 'fpu_user_cfg.default_features' guest FPU config: @fpu_guest_cfg.max_features - all known and CPU supported user and supervisor features except independent kernel features. @fpu_guest_cfg.default_features - all known and CPU supported user and supervisor features except independent kernel features and dynamic userspace features. @fpu_guest_cfg.max_size - size of compacted buffer with 'fpu_guest_cfg.max_features' @fpu_guest_cfg.default_size - size of compacted buffer with 'fpu_guest_cfg.default_features' Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky Reviewed-by: Rick Edgecombe --- arch/x86/include/asm/fpu/types.h | 2 +- arch/x86/kernel/fpu/core.c | 14 +++++++++++--- arch/x86/kernel/fpu/xstate.c | 10 ++++++++++ 3 files changed, 22 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/fpu/types.h b/arch/x86/include/asm/fpu/types.h index c6fd13a17205..306825ad6bc0 100644 --- a/arch/x86/include/asm/fpu/types.h +++ b/arch/x86/include/asm/fpu/types.h @@ -602,6 +602,6 @@ struct fpu_state_config { }; /* FPU state configuration information */ -extern struct fpu_state_config fpu_kernel_cfg, fpu_user_cfg; +extern struct fpu_state_config fpu_kernel_cfg, fpu_user_cfg, fpu_guest_cfg; #endif /* _ASM_X86_FPU_H */ diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index a21a4d0ecc34..516af626bf6a 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -33,9 +33,10 @@ DEFINE_STATIC_KEY_FALSE(__fpu_state_size_dynamic); DEFINE_PER_CPU(u64, xfd_state); #endif -/* The FPU state configuration data for kernel and user space */ +/* The FPU state configuration data for kernel, user space and guest. */ struct fpu_state_config fpu_kernel_cfg __ro_after_init; struct fpu_state_config fpu_user_cfg __ro_after_init; +struct fpu_state_config fpu_guest_cfg __ro_after_init; /* * Represents the initial FPU state. It's mostly (but not completely) zeroes, @@ -536,8 +537,15 @@ void fpstate_reset(struct fpu *fpu) fpu->perm.__state_perm = fpu_kernel_cfg.default_features; fpu->perm.__state_size = fpu_kernel_cfg.default_size; fpu->perm.__user_state_size = fpu_user_cfg.default_size; - /* Same defaults for guests */ - fpu->guest_perm = fpu->perm; + + /* Guest permission settings */ + fpu->guest_perm.__state_perm = fpu_guest_cfg.default_features; + fpu->guest_perm.__state_size = fpu_guest_cfg.default_size; + /* + * Set guest's __user_state_size to fpu_user_cfg.default_size so that + * existing uAPIs can still work. + */ + fpu->guest_perm.__user_state_size = fpu_user_cfg.default_size; } static inline void fpu_inherit_perms(struct fpu *dst_fpu) diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index ca4b83c142eb..9cbdc83d1eab 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -681,6 +681,7 @@ static int __init init_xstate_size(void) { /* Recompute the context size for enabled features: */ unsigned int user_size, kernel_size, kernel_default_size; + unsigned int guest_default_size; bool compacted = cpu_feature_enabled(X86_FEATURE_XCOMPACTED); /* Uncompacted user space size */ @@ -702,13 +703,18 @@ static int __init init_xstate_size(void) kernel_default_size = xstate_calculate_size(fpu_kernel_cfg.default_features, compacted); + guest_default_size = + xstate_calculate_size(fpu_guest_cfg.default_features, compacted); + if (!paranoid_xstate_size_valid(kernel_size)) return -EINVAL; fpu_kernel_cfg.max_size = kernel_size; fpu_user_cfg.max_size = user_size; + fpu_guest_cfg.max_size = kernel_size; fpu_kernel_cfg.default_size = kernel_default_size; + fpu_guest_cfg.default_size = guest_default_size; fpu_user_cfg.default_size = xstate_calculate_size(fpu_user_cfg.default_features, false); @@ -829,6 +835,10 @@ void __init fpu__init_system_xstate(unsigned int legacy_size) fpu_user_cfg.default_features = fpu_user_cfg.max_features; fpu_user_cfg.default_features &= ~XFEATURE_MASK_USER_DYNAMIC; + fpu_guest_cfg.max_features = fpu_kernel_cfg.max_features; + fpu_guest_cfg.default_features = fpu_guest_cfg.max_features; + fpu_guest_cfg.default_features &= ~XFEATURE_MASK_USER_DYNAMIC; + /* Store it for paranoia check at the end */ xfeatures = fpu_kernel_cfg.max_features; From patchwork Wed Jan 24 02:41:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528421 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DF67CC8D2; Wed, 24 Jan 2024 02:42:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064157; cv=none; b=N0d41SQwTboMOUz/6qBLZ7k5KZJtZrpFgiI6fzvTCKc8k7caahT69iZFAExy5f0SdmyLBZi7DBUem1fLwMlSU6ltiIPTNjTo8ImMFF5LxOV7CvOvgrfgMziLGJly2SEQeL6buAx9GCUI9+4/myD8FDreFB+6uW9AltmA+2TLHkk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064157; c=relaxed/simple; bh=LcUwY8V1632alJGE8IcrVFp1L0/aYLAVGYcE+EcRKn4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Lztu7vsJhJgHzj44r97PgP40ZAsx5dbW7NVMzrd7NbxcnSiex+bpSaVJjrbEF1B8+TDaw1k5FEXRrFXpz0PM3WdoH7yWbUY1+foh5577peaxH3bQycxt93B1n5LCDUswl5Fgv8MH1SZY9mrDnb49IPClYn7TYDlZiUzPOFsuGyw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=N9y/JW2v; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="N9y/JW2v" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064155; x=1737600155; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LcUwY8V1632alJGE8IcrVFp1L0/aYLAVGYcE+EcRKn4=; b=N9y/JW2vZyIMObBfAO6qEzKZGEZExj0aGO08Bj/gV17og8vFX52iKPiY Re8KQNH/la6NiUB9TbpOpbx9p8Xc+w2dDSyVtMfYeJkqc6AwizI1vI/u6 Sx6T5VkLuDnvtydSKZ8lTSMVoUP8hCm+w2OPcxLHjC6qsW/LhoiRIxwRi Dfy0jsB6Ih1YS2G6TKV9WmbBXsMCKBXJSlzYiYRoxxS9OY/4DQS+1VLac eQkdBEWoBG1hk4BP5UAV1biNa0dX0lUBhVeb/OQgGFodpR0VIGwOpLb83 edcdVWOwFiS7pcaevslZp+knJcP0de34M8DnTLckw4o4yJhe4UvDfuplP g==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586452" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586452" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:33 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825843" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:33 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 06/27] x86/fpu/xstate: Create guest fpstate with guest specific config Date: Tue, 23 Jan 2024 18:41:39 -0800 Message-Id: <20240124024200.102792-7-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Use fpu_guest_cfg to calculate guest fpstate settings, open code for __fpstate_reset() to avoid using kernel FPU config. Below configuration steps are currently enforced to get guest fpstate: 1) Kernel sets up guest FPU settings in fpu__init_system_xstate(). 2) User space sets vCPU thread group xstate permits via arch_prctl(). 3) User space creates guest fpstate via __fpu_alloc_init_guest_fpstate() for vcpu thread. 4) User space enables guest dynamic xfeatures and re-allocate guest fpstate. By adding kernel dynamic xfeatures in above #1 and #2, guest xstate area size is expanded to hold (fpu_kernel_cfg.default_features | kernel dynamic xfeatures | user dynamic xfeatures), then host xsaves/xrstors can operate for all guest xfeatures. The user_* fields remain unchanged for compatibility with KVM uAPIs. Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky Reviewed-by: Rick Edgecombe --- arch/x86/kernel/fpu/core.c | 44 ++++++++++++++++++++++++++++++-------- 1 file changed, 35 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index 516af626bf6a..6048352e1cc7 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -194,8 +194,6 @@ void fpu_reset_from_exception_fixup(void) } #if IS_ENABLED(CONFIG_KVM) -static void __fpstate_reset(struct fpstate *fpstate, u64 xfd); - static void fpu_init_guest_permissions(struct fpu_guest *gfpu) { struct fpu_state_perm *fpuperm; @@ -216,25 +214,53 @@ static void fpu_init_guest_permissions(struct fpu_guest *gfpu) gfpu->perm = perm & ~FPU_GUEST_PERM_LOCKED; } -bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu) +static struct fpstate *__fpu_alloc_init_guest_fpstate(struct fpu_guest *gfpu) { struct fpstate *fpstate; unsigned int size; - size = fpu_user_cfg.default_size + ALIGN(offsetof(struct fpstate, regs), 64); + /* + * fpu_guest_cfg.default_size is initialized to hold all enabled + * xfeatures except the user dynamic xfeatures. If the user dynamic + * xfeatures are enabled, the guest fpstate will be re-allocated to + * hold all guest enabled xfeatures, so omit user dynamic xfeatures + * here. + */ + size = fpu_guest_cfg.default_size + + ALIGN(offsetof(struct fpstate, regs), 64); + fpstate = vzalloc(size); if (!fpstate) - return false; + return NULL; + /* + * Initialize sizes and feature masks, use fpu_user_cfg.* + * for user_* settings for compatibility of exiting uAPIs. + */ + fpstate->size = fpu_guest_cfg.default_size; + fpstate->xfeatures = fpu_guest_cfg.default_features; + fpstate->user_size = fpu_user_cfg.default_size; + fpstate->user_xfeatures = fpu_user_cfg.default_features; + fpstate->xfd = 0; - /* Leave xfd to 0 (the reset value defined by spec) */ - __fpstate_reset(fpstate, 0); fpstate_init_user(fpstate); fpstate->is_valloc = true; fpstate->is_guest = true; gfpu->fpstate = fpstate; - gfpu->xfeatures = fpu_user_cfg.default_features; - gfpu->perm = fpu_user_cfg.default_features; + gfpu->xfeatures = fpu_guest_cfg.default_features; + gfpu->perm = fpu_guest_cfg.default_features; + + return fpstate; +} + +bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu) +{ + struct fpstate *fpstate; + + fpstate = __fpu_alloc_init_guest_fpstate(gfpu); + + if (!fpstate) + return false; /* * KVM sets the FP+SSE bits in the XSAVE header when copying FPU state From patchwork Wed Jan 24 02:41:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528425 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A96EBF9E6; Wed, 24 Jan 2024 02:42:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064160; cv=none; b=U8KesVja0I9ENV5BX3LQe+/yHVDmG92RawTpmfCPpRaLj9ysQTCmaq2OazLAfGOzfZMdZPVa0aPxKd8nv37gIrfQ1Gczv4mlaTADIQR/FC+d+5DMlOWpg0gzyZRn/J8HKPbBigu3W67M7gQEwIQKLD1danLp5jeVX/7OKyT5SN0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064160; c=relaxed/simple; bh=HiRiDRBJgwGeewv2BjjbZoB+AZJ6Z19DaJ2CBVn/WYI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bBcT5MQDUpTUW6pkijBjR9st6deiHfZyPIv/rgNt9qmrP12Dsr4stb1f8ECey83W1QC971qsrLiF7uQvCkjhlmcrPsbovFd5LHxcuBsQ/mnlrElxDDDm9cBChRNI4g3HBU3vBvjlYaWkQb8//QwaWXbumqCWLNgDftf2usHZo2o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Ve+5k0Cm; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Ve+5k0Cm" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064157; x=1737600157; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=HiRiDRBJgwGeewv2BjjbZoB+AZJ6Z19DaJ2CBVn/WYI=; b=Ve+5k0Cm2k/fU5ade3DeiTkvWOsTwWLj3qfneI6ZCV5eS8LG9OMM4Dgp ZkgGFti9OEarAzhMFsHm97xZWpVowaSUpgcpsidXrLYdz+1OQSZzCH0l0 3bFEKYkN40UJQwxzhZUBCQP/xCJVBUXIMpvcnhvh+t9ezmaQVTXSqWkJG 1ZPChaYXwCyDa0u13Y6L9Y8NhUdIwb9HvySF1PRVvflP9a7170QmMJzme 6t6tS0Cegju1VKbjSSa+zi0GA8rQud3EroRkGRhxhdv3/WvOG6o2Z4vP/ TK4iFPPxNCBmrrVqLhDNvxUaIex+Wc3Xtn+O2vVQSHD6zNO+OZ6qDV1d+ g==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586458" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586458" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825846" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:33 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 07/27] x86/fpu/xstate: Warn if kernel dynamic xfeatures detected in normal fpstate Date: Tue, 23 Jan 2024 18:41:40 -0800 Message-Id: <20240124024200.102792-8-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Kernel dynamic xfeatures now are __ONLY__ enabled for guest fpstate, i.e., never for normal kernel fpstate. The bits are added when guest FPU config is initialized. Guest fpstate is allocated with fpstate->is_guest set to %true. For normal fpstate, the bits should have been removed when initializes kernel FPU config settings, WARN_ONCE() if kernel detects normal fpstate xfeatures contains kernel dynamic xfeatures before executes xsaves. Signed-off-by: Yang Weijiang Reviewed-by: Rick Edgecombe Reviewed-by: Maxim Levitsky --- arch/x86/kernel/fpu/xstate.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/x86/kernel/fpu/xstate.h b/arch/x86/kernel/fpu/xstate.h index 3518fb26d06b..83ebf1e1cbb4 100644 --- a/arch/x86/kernel/fpu/xstate.h +++ b/arch/x86/kernel/fpu/xstate.h @@ -185,6 +185,9 @@ static inline void os_xsave(struct fpstate *fpstate) WARN_ON_FPU(!alternatives_patched); xfd_validate_state(fpstate, mask, false); + WARN_ON_FPU(!fpstate->is_guest && + (mask & XFEATURE_MASK_KERNEL_DYNAMIC)); + XSTATE_XSAVE(&fpstate->regs.xsave, lmask, hmask, err); /* We should never fault when copying to a kernel buffer: */ From patchwork Wed Jan 24 02:41:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528423 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C5B81FBEE; Wed, 24 Jan 2024 02:42:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064159; cv=none; b=mHHG2GE8bcsNXCdlVDiLnLI9rX9ZIJaDxPCP6vBUqP+3nPl+V1Mz3E8YVe6/98mT5So2M5osMBl2l403NZBFLgZ3I5cbHDjZ4Oz0ArX0fwzFB64vdzi/tu1bC9oK/++RWiyis3YglbF1qkaL/CVnN/5e6Arx2AiduSZZKEMWVRA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064159; c=relaxed/simple; bh=0U7ALq4ceBj9fsXzc9ZLpGtprVUwlTHjsrBo2twXG2w=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NUpt3dJKmLovI8//TpRfd7R97dfNdsu0R5h83QG3q77apjEspkQxzmEpxao0DiJi155gKmeJbCItI7+vkbZvC8jTg07HXRbZtiMODanZrA7kjGYTNUgRaBeTQmdkLgk6EoDJbJGElKN4nhL7toebCLpQ2DqDDOBaIfIHRQiXbGw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=QJaSdt1b; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="QJaSdt1b" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064157; x=1737600157; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0U7ALq4ceBj9fsXzc9ZLpGtprVUwlTHjsrBo2twXG2w=; b=QJaSdt1blPhz20acTALN3rW026MPMhpAVauAZtnANwgbXSkOG5inCb9Z zPZuyXkv31IOCFOloShKid7WXMq3gKgerHr4nHoIzMv0c2ozAsvuPRs9E r1yAA1tvbdkbG6hNzRKDXRfjzEcBJq/+DIZ2I0lkJNTMp3AzCvb9HaZ3N vIqRvscEUoca+bibrxL5bX3UZUs6qKkmSGflIuGAckdBz3dw96ocSNuyV g7blte0RZKClCN6WU7+0rMpzwS9sqzw/dcWZUrPC7SBMQ1qQkQZ6a+zJn 80cDHvqfVST9rQbXhwylhI8Vta8RK+hZHUFplMf48YvbXNgIl4CxFL+Sq g==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586464" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586464" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825849" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:34 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 08/27] KVM: x86: Rework cpuid_get_supported_xcr0() to operate on vCPU data Date: Tue, 23 Jan 2024 18:41:41 -0800 Message-Id: <20240124024200.102792-9-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson Rework and rename cpuid_get_supported_xcr0() to explicitly operate on vCPU state, i.e. on a vCPU's CPUID state, now that the only usage of the helper is to retrieve a vCPU's already-set CPUID. Prior to commit 275a87244ec8 ("KVM: x86: Don't adjust guest's CPUID.0x12.1 (allowed SGX enclave XFRM)"), KVM incorrectly fudged guest CPUID at runtime, which in turn necessitated massaging the incoming CPUID state for KVM_SET_CPUID{2} so as not to run afoul of kvm_cpuid_check_equal(). I.e. KVM also invoked cpuid_get_supported_xcr0() with the incoming CPUID state, and thus without an explicit vCPU object. Opportunistically move the helper below kvm_update_cpuid_runtime() to make it harder to repeat the mistake of querying supported XCR0 for runtime updates. No functional change intended. Signed-off-by: Sean Christopherson Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky --- arch/x86/kvm/cpuid.c | 33 ++++++++++++++++----------------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 294e5bd5f8a0..624954203b40 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -247,21 +247,6 @@ void kvm_update_pv_runtime(struct kvm_vcpu *vcpu) vcpu->arch.pv_cpuid.features = best->eax; } -/* - * Calculate guest's supported XCR0 taking into account guest CPUID data and - * KVM's supported XCR0 (comprised of host's XCR0 and KVM_SUPPORTED_XCR0). - */ -static u64 cpuid_get_supported_xcr0(struct kvm_cpuid_entry2 *entries, int nent) -{ - struct kvm_cpuid_entry2 *best; - - best = cpuid_entry2_find(entries, nent, 0xd, 0); - if (!best) - return 0; - - return (best->eax | ((u64)best->edx << 32)) & kvm_caps.supported_xcr0; -} - static void __kvm_update_cpuid_runtime(struct kvm_vcpu *vcpu, struct kvm_cpuid_entry2 *entries, int nent) { @@ -312,6 +297,21 @@ void kvm_update_cpuid_runtime(struct kvm_vcpu *vcpu) } EXPORT_SYMBOL_GPL(kvm_update_cpuid_runtime); +/* + * Calculate guest's supported XCR0 taking into account guest CPUID data and + * KVM's supported XCR0 (comprised of host's XCR0 and KVM_SUPPORTED_XCR0). + */ +static u64 vcpu_get_supported_xcr0(struct kvm_vcpu *vcpu) +{ + struct kvm_cpuid_entry2 *best; + + best = kvm_find_cpuid_entry_index(vcpu, 0xd, 0); + if (!best) + return 0; + + return (best->eax | ((u64)best->edx << 32)) & kvm_caps.supported_xcr0; +} + static bool kvm_cpuid_has_hyperv(struct kvm_cpuid_entry2 *entries, int nent) { #ifdef CONFIG_KVM_HYPERV @@ -361,8 +361,7 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) kvm_apic_set_version(vcpu); } - vcpu->arch.guest_supported_xcr0 = - cpuid_get_supported_xcr0(vcpu->arch.cpuid_entries, vcpu->arch.cpuid_nent); + vcpu->arch.guest_supported_xcr0 = vcpu_get_supported_xcr0(vcpu); kvm_update_pv_runtime(vcpu); From patchwork Wed Jan 24 02:41:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528424 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD768FBF6; Wed, 24 Jan 2024 02:42:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064160; cv=none; b=ZulyoLAK6r8j7KWeHG156gU3/9c+Yc6jTR2opmQZr4L1wnU4ehEBDSsLCnLFILTpc5UqKNbq5zceQppU4jXgEeNQD/uRNbeTPmBwgDZcvABO8AsBsf3VlpPflu1AcOWhHoloiPx4TC3lfReIzZr5oXz7ItVFfqaWy3ThVTBoC1A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064160; c=relaxed/simple; bh=mnzEVlzgWkMulqUCa41he1GtyrHqz8Magus+6evCOOA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nK/KNQvFX2GcYkTE/UpZdp0unPrZcxz1KtwvXQwq9mqaVS+0qRNJoVxHDI40ZaE/BVSJoOEul9SPsHA1aptacpZg8lh/uIT1n83qKTnoRtOeYlmXO8i1sffxwyxbGNvKxUAmiXMSSCSBqRlqoHwRoeFslNt3X7BovuuSJ+Mka1g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=i9OJaSVb; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="i9OJaSVb" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064158; x=1737600158; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mnzEVlzgWkMulqUCa41he1GtyrHqz8Magus+6evCOOA=; b=i9OJaSVbTEex5kpuog4ExnbY6MFevjv9eZl40a7FB+OAhgcPJtk/Hakq enrNaPJgHN5G4Q3SRVa5RybEzhyMz3CKX2dkvRtrr5yzAa7n8YygzE8Pz NcS1B3O4x0/EU1uCg0b2xIRGv6bqmY2FVs5syyDJkG8rFgvi3vYOLCTfP RE7+3jfjXqL/iCDjhzKnkJIwG1WfnrZKGN1M5WmoaPe2Cj9HCxSRGxPIX 1f79sfq9ZA9CJTpAhR/3gD+A4/OlzefuoL85AzVDuto0X7pLCfc3sOyAz UsV3VOhwxxGZU5p+pV+3Kp97cFPriCNmwgJGjcdI3VSdUUA3c88Lo9QSj Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586471" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586471" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825852" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:35 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 09/27] KVM: x86: Rename kvm_{g,s}et_msr() to menifest emulation operations Date: Tue, 23 Jan 2024 18:41:42 -0800 Message-Id: <20240124024200.102792-10-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Rename kvm_{g,s}et_msr() to kvm_emulate_msr_{read,write}() to make it more obvious that KVM uses these helpers to emulate guest behaviors, i.e., host_initiated == false in these helpers. Suggested-by: Sean Christopherson Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky Reviewed-by: Chao Gao --- arch/x86/include/asm/kvm_host.h | 4 ++-- arch/x86/kvm/smm.c | 4 ++-- arch/x86/kvm/vmx/nested.c | 13 +++++++------ arch/x86/kvm/x86.c | 10 +++++----- 4 files changed, 16 insertions(+), 15 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 7bc1daf68741..5c665165024c 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2013,8 +2013,8 @@ void kvm_prepare_emulation_failure_exit(struct kvm_vcpu *vcpu); void kvm_enable_efer_bits(u64); bool kvm_valid_efer(struct kvm_vcpu *vcpu, u64 efer); int __kvm_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data, bool host_initiated); -int kvm_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data); -int kvm_set_msr(struct kvm_vcpu *vcpu, u32 index, u64 data); +int kvm_emulate_msr_read(struct kvm_vcpu *vcpu, u32 index, u64 *data); +int kvm_emulate_msr_write(struct kvm_vcpu *vcpu, u32 index, u64 data); int kvm_emulate_rdmsr(struct kvm_vcpu *vcpu); int kvm_emulate_wrmsr(struct kvm_vcpu *vcpu); int kvm_emulate_as_nop(struct kvm_vcpu *vcpu); diff --git a/arch/x86/kvm/smm.c b/arch/x86/kvm/smm.c index dc3d95fdca7d..45c855389ea7 100644 --- a/arch/x86/kvm/smm.c +++ b/arch/x86/kvm/smm.c @@ -535,7 +535,7 @@ static int rsm_load_state_64(struct x86_emulate_ctxt *ctxt, vcpu->arch.smbase = smstate->smbase; - if (kvm_set_msr(vcpu, MSR_EFER, smstate->efer & ~EFER_LMA)) + if (kvm_emulate_msr_write(vcpu, MSR_EFER, smstate->efer & ~EFER_LMA)) return X86EMUL_UNHANDLEABLE; rsm_load_seg_64(vcpu, &smstate->tr, VCPU_SREG_TR); @@ -626,7 +626,7 @@ int emulator_leave_smm(struct x86_emulate_ctxt *ctxt) /* And finally go back to 32-bit mode. */ efer = 0; - kvm_set_msr(vcpu, MSR_EFER, efer); + kvm_emulate_msr_write(vcpu, MSR_EFER, efer); } #endif diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index db0ad1e6ec4b..b2e9853584b8 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -958,7 +958,7 @@ static u32 nested_vmx_load_msr(struct kvm_vcpu *vcpu, u64 gpa, u32 count) __func__, i, e.index, e.reserved); goto fail; } - if (kvm_set_msr(vcpu, e.index, e.value)) { + if (kvm_emulate_msr_write(vcpu, e.index, e.value)) { pr_debug_ratelimited( "%s cannot write MSR (%u, 0x%x, 0x%llx)\n", __func__, i, e.index, e.value); @@ -994,7 +994,7 @@ static bool nested_vmx_get_vmexit_msr_value(struct kvm_vcpu *vcpu, } } - if (kvm_get_msr(vcpu, msr_index, data)) { + if (kvm_emulate_msr_read(vcpu, msr_index, data)) { pr_debug_ratelimited("%s cannot read MSR (0x%x)\n", __func__, msr_index); return false; @@ -2686,7 +2686,7 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12, if ((vmcs12->vm_entry_controls & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL) && kvm_pmu_has_perf_global_ctrl(vcpu_to_pmu(vcpu)) && - WARN_ON_ONCE(kvm_set_msr(vcpu, MSR_CORE_PERF_GLOBAL_CTRL, + WARN_ON_ONCE(kvm_emulate_msr_write(vcpu, MSR_CORE_PERF_GLOBAL_CTRL, vmcs12->guest_ia32_perf_global_ctrl))) { *entry_failure_code = ENTRY_FAIL_DEFAULT; return -EINVAL; @@ -4568,8 +4568,9 @@ static void load_vmcs12_host_state(struct kvm_vcpu *vcpu, } if ((vmcs12->vm_exit_controls & VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL) && kvm_pmu_has_perf_global_ctrl(vcpu_to_pmu(vcpu))) - WARN_ON_ONCE(kvm_set_msr(vcpu, MSR_CORE_PERF_GLOBAL_CTRL, - vmcs12->host_ia32_perf_global_ctrl)); + WARN_ON_ONCE(kvm_emulate_msr_write(vcpu, + MSR_CORE_PERF_GLOBAL_CTRL, + vmcs12->host_ia32_perf_global_ctrl)); /* Set L1 segment info according to Intel SDM 27.5.2 Loading Host Segment and Descriptor-Table Registers */ @@ -4744,7 +4745,7 @@ static void nested_vmx_restore_host_state(struct kvm_vcpu *vcpu) goto vmabort; } - if (kvm_set_msr(vcpu, h.index, h.value)) { + if (kvm_emulate_msr_write(vcpu, h.index, h.value)) { pr_debug_ratelimited( "%s WRMSR failed (%u, 0x%x, 0x%llx)\n", __func__, j, h.index, h.value); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 27e23714e960..0e7dc3398293 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1976,17 +1976,17 @@ static int kvm_set_msr_with_filter(struct kvm_vcpu *vcpu, u32 index, u64 data) return kvm_set_msr_ignored_check(vcpu, index, data, false); } -int kvm_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data) +int kvm_emulate_msr_read(struct kvm_vcpu *vcpu, u32 index, u64 *data) { return kvm_get_msr_ignored_check(vcpu, index, data, false); } -EXPORT_SYMBOL_GPL(kvm_get_msr); +EXPORT_SYMBOL_GPL(kvm_emulate_msr_read); -int kvm_set_msr(struct kvm_vcpu *vcpu, u32 index, u64 data) +int kvm_emulate_msr_write(struct kvm_vcpu *vcpu, u32 index, u64 data) { return kvm_set_msr_ignored_check(vcpu, index, data, false); } -EXPORT_SYMBOL_GPL(kvm_set_msr); +EXPORT_SYMBOL_GPL(kvm_emulate_msr_write); static void complete_userspace_rdmsr(struct kvm_vcpu *vcpu) { @@ -8386,7 +8386,7 @@ static int emulator_set_msr_with_filter(struct x86_emulate_ctxt *ctxt, static int emulator_get_msr(struct x86_emulate_ctxt *ctxt, u32 msr_index, u64 *pdata) { - return kvm_get_msr(emul_to_vcpu(ctxt), msr_index, pdata); + return kvm_emulate_msr_read(emul_to_vcpu(ctxt), msr_index, pdata); } static int emulator_check_pmc(struct x86_emulate_ctxt *ctxt, From patchwork Wed Jan 24 02:41:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528426 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CAC46171BC; Wed, 24 Jan 2024 02:42:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064161; cv=none; b=AhAZ8YpcKlUrH8BU8kq4v7E01JpX10M8otlCPOTYD1UsiKH6ixnXFgWJ4riH9UgOHtlqIKe32Ffxn1Ed5jBJ0alY0m7NvLkhNSDeQTG34ybRwyrdC6iY2VGTr6BicxtKMcIQU+ed5K/HROCJ0NiXkFEl5wVPDY2QyIgge4qC/hM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064161; c=relaxed/simple; bh=cxpPX3Wn0VREKtBeuHc/MZkj33xyM0sqH6UvdE/wBRo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=T/bpQ2Oq51itYJDTgjul5LlKMC08SDBTmFcf9CWE1eJ02yX6DerK18H1yKipuwvIWoeFHhkudW2cmJ1nIvOflsvO6Xb0DpeOm3E6RspZZ3pyo6OapaERN2IuXJBPaKN73LOrCdrlhjbHfqX4nrKxBIAaFy1EWWYeIKkkJKMUjus= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=jzIb/tY7; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="jzIb/tY7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064159; x=1737600159; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cxpPX3Wn0VREKtBeuHc/MZkj33xyM0sqH6UvdE/wBRo=; b=jzIb/tY7JY9v72xklDRzq6stw171yM3YJ5nXSAYV9G0enQNgUIvFBKtP AyOtGxxbqekdBKOtVoxU4auKKNWpi/616w404wtiwnkaFanvy9I17BqGN yBgHnRRUY4dwGDGcTuBIr55A2x8jRzNUZ01NtoRfN2206o9x4rhDJdtv5 fKfhjK5F+ir3wm/OgoGzwK1fdTyR9N6tQVwaTLinq2aPCc4+Aqvs6x00e +uYeEhYiBPBVjWNd2H1ZNHCP66T8BBWF13ZEycWjJDHiU5mH0ScRepCWr KCSpmqplOj6PzcDzFXcI5KJYbTm1FXYgsPLemNj8dilsPiayaNFTkRX8/ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586478" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586478" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825855" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:35 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 10/27] KVM: x86: Refine xsave-managed guest register/MSR reset handling Date: Tue, 23 Jan 2024 18:41:43 -0800 Message-Id: <20240124024200.102792-11-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Tweak the code a bit to facilitate resetting more xstate components in the future, e.g., adding CET's xstate-managed MSRs. No functional change intended. Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky --- arch/x86/kvm/x86.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 0e7dc3398293..3671f4868d1b 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -12205,6 +12205,11 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu) static_branch_dec(&kvm_has_noapic_vcpu); } +static inline bool is_xstate_reset_needed(void) +{ + return kvm_cpu_cap_has(X86_FEATURE_MPX); +} + void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) { struct kvm_cpuid_entry2 *cpuid_0x1; @@ -12262,7 +12267,7 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) kvm_async_pf_hash_reset(vcpu); vcpu->arch.apf.halted = false; - if (vcpu->arch.guest_fpu.fpstate && kvm_mpx_supported()) { + if (vcpu->arch.guest_fpu.fpstate && is_xstate_reset_needed()) { struct fpstate *fpstate = vcpu->arch.guest_fpu.fpstate; /* @@ -12272,8 +12277,12 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) if (init_event) kvm_put_guest_fpu(vcpu); - fpstate_clear_xstate_component(fpstate, XFEATURE_BNDREGS); - fpstate_clear_xstate_component(fpstate, XFEATURE_BNDCSR); + if (kvm_cpu_cap_has(X86_FEATURE_MPX)) { + fpstate_clear_xstate_component(fpstate, + XFEATURE_BNDREGS); + fpstate_clear_xstate_component(fpstate, + XFEATURE_BNDCSR); + } if (init_event) kvm_load_guest_fpu(vcpu); From patchwork Wed Jan 24 02:41:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528428 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 562F117541; Wed, 24 Jan 2024 02:42:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064162; cv=none; b=jraaZdaxLTk7uqXxSSRZiL8Xi+GAP1r69mopqKEIClXI1UGiwJ+4ot/42iLzsSQbLA02f6d0CydoTvywmIWimeitaQ5dgx5h5+XMnmGeN72OsOpX/3PEdCXzjKl/hz/q81KpNlbBtk04p7jzDULvH6o99hYiw2vm1kgi3/4eGkY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064162; c=relaxed/simple; bh=deVC/2fUZfj8WeG3WjFFJ61TeWJaKY/1dV/6Kp3bPHY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pS69gigGbe53+CbCaZ9kGxV7J+/733Bd6Zv8nRPl25+n5z1od9vGSGSSf7WqleBFcVGxs1UJvdO1/oplqmH+cnW5WR45JYflkqrRD9iTUdnp8u5qE/km/O7/vdpTFej+xpVq1eonC8Qp+nelooLIn9ilDVyLgkl0F76zg1EpTSs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=hbARKvv8; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="hbARKvv8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064160; x=1737600160; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=deVC/2fUZfj8WeG3WjFFJ61TeWJaKY/1dV/6Kp3bPHY=; b=hbARKvv83UznF+VpyRQI3xoqkpb8fAV5VlVb1g7L1hgRsRDSoLNZGZb5 OpJNPUOIDx21TmWoRFe4i21UO41h7QhiI5tzIA0J30capf2vhAX5J1Al5 7fH735ACQVOGVjKXOmvunagP9s8f12ATQ/KnOj56Va4LYF7DgYw/MeOqo VZpqQ7yGUuRAdh0YXQYNlbHVr/YlLGXiKDjBZ1X//3p/WxY66wQUvHHuV 2Sr9L4zuaS1uH3uhwyJOTIiTvw953QFmlUOhX98GJv2I4JiDnWbO1gSXO mRQ2gdDFnIk+NxftlPKTiU0lMMOuk1/avHhawqry84S7ONPuzQteMO4QD Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586484" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586484" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:36 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825858" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:36 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 11/27] KVM: x86: Add kvm_msr_{read,write}() helpers Date: Tue, 23 Jan 2024 18:41:44 -0800 Message-Id: <20240124024200.102792-12-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Wrap __kvm_{get,set}_msr() into two new helpers for KVM usage and use the helpers to replace existing usage of the raw functions. kvm_msr_{read,write}() are KVM-internal helpers, i.e. used when KVM needs to get/set a MSR value for emulating CPU behavior, i.e., host_initiated == %true in the helpers. Suggested-by: Sean Christopherson Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky --- arch/x86/include/asm/kvm_host.h | 3 ++- arch/x86/kvm/cpuid.c | 2 +- arch/x86/kvm/x86.c | 16 +++++++++++++--- 3 files changed, 16 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 5c665165024c..40dd796ea085 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2012,9 +2012,10 @@ void kvm_prepare_emulation_failure_exit(struct kvm_vcpu *vcpu); void kvm_enable_efer_bits(u64); bool kvm_valid_efer(struct kvm_vcpu *vcpu, u64 efer); -int __kvm_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data, bool host_initiated); int kvm_emulate_msr_read(struct kvm_vcpu *vcpu, u32 index, u64 *data); int kvm_emulate_msr_write(struct kvm_vcpu *vcpu, u32 index, u64 data); +int kvm_msr_read(struct kvm_vcpu *vcpu, u32 index, u64 *data); +int kvm_msr_write(struct kvm_vcpu *vcpu, u32 index, u64 data); int kvm_emulate_rdmsr(struct kvm_vcpu *vcpu); int kvm_emulate_wrmsr(struct kvm_vcpu *vcpu); int kvm_emulate_as_nop(struct kvm_vcpu *vcpu); diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 624954203b40..acc360c76318 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -1548,7 +1548,7 @@ bool kvm_cpuid(struct kvm_vcpu *vcpu, u32 *eax, u32 *ebx, *edx = entry->edx; if (function == 7 && index == 0) { u64 data; - if (!__kvm_get_msr(vcpu, MSR_IA32_TSX_CTRL, &data, true) && + if (!kvm_msr_read(vcpu, MSR_IA32_TSX_CTRL, &data) && (data & TSX_CTRL_CPUID_CLEAR)) *ebx &= ~(F(RTM) | F(HLE)); } else if (function == 0x80000007) { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 3671f4868d1b..594c9e025f95 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1920,8 +1920,8 @@ static int kvm_set_msr_ignored_check(struct kvm_vcpu *vcpu, * Returns 0 on success, non-0 otherwise. * Assumes vcpu_load() was already called. */ -int __kvm_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data, - bool host_initiated) +static int __kvm_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data, + bool host_initiated) { struct msr_data msr; int ret; @@ -1947,6 +1947,16 @@ int __kvm_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data, return ret; } +int kvm_msr_write(struct kvm_vcpu *vcpu, u32 index, u64 data) +{ + return __kvm_set_msr(vcpu, index, data, true); +} + +int kvm_msr_read(struct kvm_vcpu *vcpu, u32 index, u64 *data) +{ + return __kvm_get_msr(vcpu, index, data, true); +} + static int kvm_get_msr_ignored_check(struct kvm_vcpu *vcpu, u32 index, u64 *data, bool host_initiated) { @@ -12296,7 +12306,7 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) MSR_IA32_MISC_ENABLE_BTS_UNAVAIL; __kvm_set_xcr(vcpu, 0, XFEATURE_MASK_FP); - __kvm_set_msr(vcpu, MSR_IA32_XSS, 0, true); + kvm_msr_write(vcpu, MSR_IA32_XSS, 0); } /* All GPRs except RDX (handled below) are zeroed on RESET/INIT. */ From patchwork Wed Jan 24 02:41:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528427 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B61D217559; Wed, 24 Jan 2024 02:42:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064162; cv=none; b=b9O4pxN/1nQRZwTBdgmpWO42MWKdD9xx3cOs0Lzcxl9/SIQgELKRWGr27yR3C53Ztfm55movb+VfmrA21/QxCCbxB1HZWJFWrarVszM7HROdsgfZT8xraX/E8xDkjNoYzP4TctvmzikofX/tJUpKRJzSMDPGHDHFP215mUDKm/0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064162; c=relaxed/simple; bh=T3X3fq7AuhPHM7WBbIb8WSrvU0kDaKc1Pze+7ySZgmo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=q1Z8AEW9byAoxQidpxzFvYo7bxyP1diw7MJW4SEWjuPAaOx70HLNP/6NBbTHa2JhgykSQ196ogK3mkTA9libnqLEdDRxR8qwFLblgcsMEKtb7hLUiZer8R9jZXGn/1Yx99mJX0v5MlUq0QLXXuJHLEqMA2tSOelMBAywmcTJZLE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Kg3FblHW; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Kg3FblHW" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064160; x=1737600160; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=T3X3fq7AuhPHM7WBbIb8WSrvU0kDaKc1Pze+7ySZgmo=; b=Kg3FblHWQpYERXvNPXmRkdkQ9Tw3fgEtYhtSc66+i/y3VJJoeWhBGyR0 c/MwoFWyHcCqMvRrlj9w+AfpSRkS5PqEQi/Od7Z1EH3TSkXPrXg70aISW nhOJUjy5iRLjlVESwR59shHCCLu3BTSouvuOo60G0RcMimfIYNiQ6fxZx MQZ5uc/1iRF12QPGIaofaVwVEkn2fqZhalj1sYOJQvFxJ4zRb79QGDfl2 4kyC70f6ncU0CmGG+4yljVJ1zNWnws96QmpvjDQ83Bod8BF7btenAlqKt EfpVPQT5tnrwNlNNSDJILMgKUa/klMRjM/lDPiYC9MlSjHPl8V73K6Avu g==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586493" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586493" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:37 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825861" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:36 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 12/27] KVM: x86: Report XSS as to-be-saved if there are supported features Date: Tue, 23 Jan 2024 18:41:45 -0800 Message-Id: <20240124024200.102792-13-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson Add MSR_IA32_XSS to list of MSRs reported to userspace if supported_xss is non-zero, i.e. KVM supports at least one XSS based feature. Before enabling CET virtualization series, guest IA32_MSR_XSS is guaranteed to be 0, i.e., XSAVES/XRSTORS is executed in non-root mode with XSS == 0, which equals to the effect of XSAVE/XRSTOR. Signed-off-by: Sean Christopherson Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky Reviewed-by: Chao Gao --- arch/x86/kvm/x86.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 594c9e025f95..b3a39886e418 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1464,6 +1464,7 @@ static const u32 msrs_to_save_base[] = { MSR_IA32_UMWAIT_CONTROL, MSR_IA32_XFD, MSR_IA32_XFD_ERR, + MSR_IA32_XSS, }; static const u32 msrs_to_save_pmu[] = { @@ -7374,6 +7375,10 @@ static void kvm_probe_msr_to_save(u32 msr_index) if (!(kvm_get_arch_capabilities() & ARCH_CAP_TSX_CTRL_MSR)) return; break; + case MSR_IA32_XSS: + if (!kvm_caps.supported_xss) + return; + break; default: break; } From patchwork Wed Jan 24 02:41:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528429 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 89AE517596; Wed, 24 Jan 2024 02:42:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064163; cv=none; b=R7ym0eeJz+q/sVsknf6sW8klrvIrz6KI5/PFrUcYPvST7tC9OsllLdDZQitkjJAuRmAuPI5XL1fJjN+mV4QSfErNRh8dStF7qZAc1tg4prDZyIdwjNNHoDJBvVfGAncXfdY44l/153AVzPTYbgIzCEUTMZ16GMVdiqIOnMBZTrA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064163; c=relaxed/simple; bh=Lh/HRtmyTmg9PjjoKlPhg0GrQura9c97fLsX2mTq5DU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EZgdM9AIEwsUL/QOYkc62kdWvSWOGoe7PFT52CzXcA7OhPoXPLGs3JU/Gx03MaR+tadBSfHWqagTe7G8lZASNxprSxOKsH6NDr+6HisRR3TlEAQtt8yElJUWsbHqTwfL+Xz4Ul8mhnygCae2AIJyMYBrfz0yCDVANU0ro5qrNug= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=AqnQCJNY; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="AqnQCJNY" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064161; x=1737600161; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Lh/HRtmyTmg9PjjoKlPhg0GrQura9c97fLsX2mTq5DU=; b=AqnQCJNYg6C/IVPfeJxmbeu7jttdbR2LtRpu+PoQ23DVqw1Rsc5MrEQO +Uwlf7SPM2BsgSNrKDsJPnfCSrYQwqraNFCr6x0QGmTwlfnSKaWgNlOm7 Xhl1bD4Bu69fl+zPet0Bon89n3lxuAEuHcri2ErSxKBpJSZD9utt0jNPU vhmU0tu198rLwG5VvdisE1+A1EcL7XDB3yHleV594GoWqsLUcoizIhJkL k2Lj23qGah4mboYLllMdh1PlVcd1bejWeyekJ5Mz17GYBduMsoHBAr7PZ v85hyDJr1ddnqw38FxypTVi7hxdfzZYuPdknEBh8t7s+acpGGTiGN2UD+ g==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586504" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586504" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:37 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825869" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:37 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com, Zhang Yi Z Subject: [PATCH v9 13/27] KVM: x86: Refresh CPUID on write to guest MSR_IA32_XSS Date: Tue, 23 Jan 2024 18:41:46 -0800 Message-Id: <20240124024200.102792-14-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Update CPUID.(EAX=0DH,ECX=1).EBX to reflect current required xstate size due to XSS MSR modification. CPUID(EAX=0DH,ECX=1).EBX reports the required storage size of all enabled xstate features in (XCR0 | IA32_XSS). The CPUID value can be used by guest before allocate sufficient xsave buffer. Note, KVM does not yet support any XSS based features, i.e. supported_xss is guaranteed to be zero at this time. Opportunistically modify XSS write access logic as: If XSAVES is not enabled in the guest CPUID, forbid setting IA32_XSS msr to anything but 0, even if the write is host initiated. Suggested-by: Sean Christopherson Co-developed-by: Zhang Yi Z Signed-off-by: Zhang Yi Z Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky --- arch/x86/include/asm/kvm_host.h | 3 ++- arch/x86/kvm/cpuid.c | 15 ++++++++++++++- arch/x86/kvm/x86.c | 16 ++++++++++++---- 3 files changed, 28 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 40dd796ea085..6efaaaa15945 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -772,7 +772,6 @@ struct kvm_vcpu_arch { bool at_instruction_boundary; bool tpr_access_reporting; bool xfd_no_write_intercept; - u64 ia32_xss; u64 microcode_version; u64 arch_capabilities; u64 perf_capabilities; @@ -828,6 +827,8 @@ struct kvm_vcpu_arch { u64 xcr0; u64 guest_supported_xcr0; + u64 guest_supported_xss; + u64 ia32_xss; struct kvm_pio_request pio; void *pio_data; diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index acc360c76318..3ab133530573 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -275,7 +275,8 @@ static void __kvm_update_cpuid_runtime(struct kvm_vcpu *vcpu, struct kvm_cpuid_e best = cpuid_entry2_find(entries, nent, 0xD, 1); if (best && (cpuid_entry_has(best, X86_FEATURE_XSAVES) || cpuid_entry_has(best, X86_FEATURE_XSAVEC))) - best->ebx = xstate_required_size(vcpu->arch.xcr0, true); + best->ebx = xstate_required_size(vcpu->arch.xcr0 | + vcpu->arch.ia32_xss, true); best = __kvm_find_kvm_cpuid_features(vcpu, entries, nent); if (kvm_hlt_in_guest(vcpu->kvm) && best && @@ -312,6 +313,17 @@ static u64 vcpu_get_supported_xcr0(struct kvm_vcpu *vcpu) return (best->eax | ((u64)best->edx << 32)) & kvm_caps.supported_xcr0; } +static u64 vcpu_get_supported_xss(struct kvm_vcpu *vcpu) +{ + struct kvm_cpuid_entry2 *best; + + best = kvm_find_cpuid_entry_index(vcpu, 0xd, 1); + if (!best) + return 0; + + return (best->ecx | ((u64)best->edx << 32)) & kvm_caps.supported_xss; +} + static bool kvm_cpuid_has_hyperv(struct kvm_cpuid_entry2 *entries, int nent) { #ifdef CONFIG_KVM_HYPERV @@ -362,6 +374,7 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) } vcpu->arch.guest_supported_xcr0 = vcpu_get_supported_xcr0(vcpu); + vcpu->arch.guest_supported_xss = vcpu_get_supported_xss(vcpu); kvm_update_pv_runtime(vcpu); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b3a39886e418..7b7a15aab3aa 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -3924,20 +3924,28 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) vcpu->arch.ia32_tsc_adjust_msr += adj; } break; - case MSR_IA32_XSS: - if (!msr_info->host_initiated && - !guest_cpuid_has(vcpu, X86_FEATURE_XSAVES)) + case MSR_IA32_XSS: { + /* + * If KVM reported support of XSS MSR, even guest CPUID doesn't + * support XSAVES, still allow userspace to set default value(0) + * to this MSR. + */ + if (!guest_cpuid_has(vcpu, X86_FEATURE_XSAVES) && + !(msr_info->host_initiated && data == 0)) return 1; /* * KVM supports exposing PT to the guest, but does not support * IA32_XSS[bit 8]. Guests have to use RDMSR/WRMSR rather than * XSAVES/XRSTORS to save/restore PT MSRs. */ - if (data & ~kvm_caps.supported_xss) + if (data & ~vcpu->arch.guest_supported_xss) return 1; + if (vcpu->arch.ia32_xss == data) + break; vcpu->arch.ia32_xss = data; kvm_update_cpuid_runtime(vcpu); break; + } case MSR_SMI_COUNT: if (!msr_info->host_initiated) return 1; From patchwork Wed Jan 24 02:41:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528430 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A4B3A1775A; Wed, 24 Jan 2024 02:42:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064164; cv=none; b=HlYrl/tfHcJf5MEO/yWIUoQjzvcJBGaScDplS27kIdxJcOtLGEO6haXL7gSsDWEaEKjou79MPrW7kswUYoim9fYOiGe2j5TbBeKNFBKDmHImc/ExquROEx6VsJu5HYKpfLj3Z/VlpE0pr5KJJs/CHDU/F4DVg5n52zdvJmJTHyw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064164; c=relaxed/simple; bh=0UE5IUCUA6slej5/EM6zfvBhsHMVYZs9Yfo7sj2LfOU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kPr4wQ+NSPNbk4GBuGfYmKCQSS6l0R9RuFujCBYdUw1O1L4GxtJrQ+GIHpHhLO4lL+JC+tPSIKwOttuPCW0Tahi9odavG6i8VE1fZt8+O4zo+VdBn6qB5joIBpn587NNkgHcQ/KsOO9PreBeULAJYLxfyDE0U21mVPFU5BIzpNU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=XRchG6Oe; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="XRchG6Oe" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064162; x=1737600162; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0UE5IUCUA6slej5/EM6zfvBhsHMVYZs9Yfo7sj2LfOU=; b=XRchG6OeN4qO7/YgppHEaV6KSn10R7WErRL81IN6tTF3RDJ0crXWGG3e O2ysLyzIBA94qrq28Tu4qczlwor6YnOT6wpMWJ1EjdwOZuW/4UT8m4NyM Zoc9vqmHdLko97K/bsD0BPq137kwp5viCs0xREyOX76ABGpzhVdG3Ip3S 4bM6FCinbB8jwvcpBRMzrQebq4Otg9q+kwNMzElflVddnuJ7naGGHTnHt eC9HiRrRzF/JEazvqHLaTy47ay5Wi/KmUe+972AKXpaGf86u7YT3uheIQ H/XiQ8EN3cyT/CcCsp5XrtLDtsop/g8QLZjY/ISnZ82H7OPGTEozLk+4U w==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586513" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586513" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:38 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825873" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:38 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 14/27] KVM: x86: Initialize kvm_caps.supported_xss Date: Tue, 23 Jan 2024 18:41:47 -0800 Message-Id: <20240124024200.102792-15-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Set original kvm_caps.supported_xss to (host_xss & KVM_SUPPORTED_XSS) if XSAVES is supported. host_xss contains the host supported xstate feature bits for thread FPU context switch, KVM_SUPPORTED_XSS includes all KVM enabled XSS feature bits, the resulting value represents the supervisor xstates that are available to guest and are backed by host FPU framework for swapping {guest,host} XSAVE-managed registers/MSRs. Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky Reviewed-by: Chao Gao --- arch/x86/kvm/x86.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 7b7a15aab3aa..f50c5a523b92 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -226,6 +226,8 @@ static struct kvm_user_return_msrs __percpu *user_return_msrs; | XFEATURE_MASK_BNDCSR | XFEATURE_MASK_AVX512 \ | XFEATURE_MASK_PKRU | XFEATURE_MASK_XTILE) +#define KVM_SUPPORTED_XSS 0 + u64 __read_mostly host_efer; EXPORT_SYMBOL_GPL(host_efer); @@ -9715,12 +9717,13 @@ static int __kvm_x86_vendor_init(struct kvm_x86_init_ops *ops) host_xcr0 = xgetbv(XCR_XFEATURE_ENABLED_MASK); kvm_caps.supported_xcr0 = host_xcr0 & KVM_SUPPORTED_XCR0; } + if (boot_cpu_has(X86_FEATURE_XSAVES)) { + rdmsrl(MSR_IA32_XSS, host_xss); + kvm_caps.supported_xss = host_xss & KVM_SUPPORTED_XSS; + } rdmsrl_safe(MSR_EFER, &host_efer); - if (boot_cpu_has(X86_FEATURE_XSAVES)) - rdmsrl(MSR_IA32_XSS, host_xss); - kvm_init_pmu_capability(ops->pmu_ops); if (boot_cpu_has(X86_FEATURE_ARCH_CAPABILITIES)) From patchwork Wed Jan 24 02:41:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528431 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CC3C01798E; Wed, 24 Jan 2024 02:42:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064164; cv=none; b=rXiejwmHfEhlIP29fBWrCi2GuG/xMxFb7ga26onJT9DNnSr1gbUK7MuXzcfTeZW3OFfasl9QmfkqvoIJXkO6LUw1A150AtpyYrSpsf13lwPr6EU46OJP9KDUxXJk2HOmVfOVf5nNTr4LsssrXQCyoicMGPhI0eCdlpMCPFjH4tM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064164; c=relaxed/simple; bh=LMZ9WqVy19AK6b6saLJ6FfZQNW3F536pCoX8lK5esR8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=hPuhgSlUBUg2Ab2RzIvPeJHV93NGyGZYMnNi1cH/wBXDzAZ+rTJQj0f45uiSkYRIPfPeg4AwS/U8vCxujceczJsj71j+GKSN9URJN+H0q9Y8vHUK6WaTGEBZAufXjdxDW2Fh6c3SJcH6zhvrmHHqBNTXo/udrC1WRWMfV/ZUTYs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=CPHRO5Wv; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="CPHRO5Wv" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064162; x=1737600162; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LMZ9WqVy19AK6b6saLJ6FfZQNW3F536pCoX8lK5esR8=; b=CPHRO5WvugX0UgPSLKGKQzhDYzP2c43ti5PqOlxTkNsEc2wQJxt64GPy 0s7NanC1PIWwhWB/U/uTRkaKfzVCTt1YXLZvkyPnKW7QEr/Hhb5GSrxFP BuNGxMAlYWQaSt90O/M2xBJeYuw6MY+8E9qCqouIh8QhF+qDURW8Oe8xQ d7gyVS5ApatinBdWugUeTJX2fRwB7uPwaUrbMBHEfIEMNQJryIVDRDz19 /oLTi/+qt+yEtjkzQT+R5EP9jyMmz2g86f29fths4JPVXiwRJzjWtfprJ YhB1GowUUmHIUe5VeakF1T266mrJMy8EhYF6I+4L53mi53AZbO3ia3AXT g==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586519" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586519" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825879" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:38 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 15/27] KVM: x86: Load guest FPU state when access XSAVE-managed MSRs Date: Tue, 23 Jan 2024 18:41:48 -0800 Message-Id: <20240124024200.102792-16-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson Load the guest's FPU state if userspace is accessing MSRs whose values are managed by XSAVES. Introduce two helpers, kvm_{get,set}_xstate_msr(), to facilitate access to such kind of MSRs. If MSRs supported in kvm_caps.supported_xss are passed through to guest, the guest MSRs are swapped with host's before vCPU exits to userspace and after it reenters kernel before next VM-entry. Because the modified code is also used for the KVM_GET_MSRS device ioctl(), explicitly check @vcpu is non-null before attempting to load guest state. The XSAVE-managed MSRs cannot be retrieved via the device ioctl() without loading guest FPU state (which doesn't exist). Note that guest_cpuid_has() is not queried as host userspace is allowed to access MSRs that have not been exposed to the guest, e.g. it might do KVM_SET_MSRS prior to KVM_SET_CPUID2. The two helpers are put here in order to manifest accessing xsave-managed MSRs requires special check and handling to guarantee the correctness of read/write to the MSRs. Signed-off-by: Sean Christopherson Co-developed-by: Yang Weijiang Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky --- arch/x86/kvm/x86.c | 35 ++++++++++++++++++++++++++++++++++- arch/x86/kvm/x86.h | 24 ++++++++++++++++++++++++ 2 files changed, 58 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index f50c5a523b92..bde780ae69bf 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -133,6 +133,9 @@ static int __set_sregs2(struct kvm_vcpu *vcpu, struct kvm_sregs2 *sregs2); static void __get_sregs2(struct kvm_vcpu *vcpu, struct kvm_sregs2 *sregs2); static DEFINE_MUTEX(vendor_module_lock); +static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu); +static void kvm_put_guest_fpu(struct kvm_vcpu *vcpu); + struct kvm_x86_ops kvm_x86_ops __read_mostly; #define KVM_X86_OP(func) \ @@ -4509,6 +4512,21 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) } EXPORT_SYMBOL_GPL(kvm_get_msr_common); +/* + * Returns true if the MSR in question is managed via XSTATE, i.e. is context + * switched with the rest of guest FPU state. + */ +static bool is_xstate_managed_msr(u32 index) +{ + switch (index) { + case MSR_IA32_U_CET: + case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + return true; + default: + return false; + } +} + /* * Read or write a bunch of msrs. All parameters are kernel addresses. * @@ -4519,11 +4537,26 @@ static int __msr_io(struct kvm_vcpu *vcpu, struct kvm_msrs *msrs, int (*do_msr)(struct kvm_vcpu *vcpu, unsigned index, u64 *data)) { + bool fpu_loaded = false; int i; - for (i = 0; i < msrs->nmsrs; ++i) + for (i = 0; i < msrs->nmsrs; ++i) { + /* + * If userspace is accessing one or more XSTATE-managed MSRs, + * temporarily load the guest's FPU state so that the guest's + * MSR value(s) is resident in hardware, i.e. so that KVM can + * get/set the MSR via RDMSR/WRMSR. + */ + if (vcpu && !fpu_loaded && kvm_caps.supported_xss && + is_xstate_managed_msr(entries[i].index)) { + kvm_load_guest_fpu(vcpu); + fpu_loaded = true; + } if (do_msr(vcpu, entries[i].index, &entries[i].data)) break; + } + if (fpu_loaded) + kvm_put_guest_fpu(vcpu); return i; } diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 2f7e19166658..9c19dfb5011d 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -543,4 +543,28 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size, unsigned int port, void *data, unsigned int count, int in); +/* + * Lock and/or reload guest FPU and access xstate MSRs. For accesses initiated + * by host, guest FPU is loaded in __msr_io(). For accesses initiated by guest, + * guest FPU should have been loaded already. + */ + +static inline void kvm_get_xstate_msr(struct kvm_vcpu *vcpu, + struct msr_data *msr_info) +{ + KVM_BUG_ON(!vcpu->arch.guest_fpu.fpstate->in_use, vcpu->kvm); + kvm_fpu_get(); + rdmsrl(msr_info->index, msr_info->data); + kvm_fpu_put(); +} + +static inline void kvm_set_xstate_msr(struct kvm_vcpu *vcpu, + struct msr_data *msr_info) +{ + KVM_BUG_ON(!vcpu->arch.guest_fpu.fpstate->in_use, vcpu->kvm); + kvm_fpu_get(); + wrmsrl(msr_info->index, msr_info->data); + kvm_fpu_put(); +} + #endif From patchwork Wed Jan 24 02:41:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528432 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 76F0C17BA3; Wed, 24 Jan 2024 02:42:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064165; cv=none; b=n7k27sgtNiUf0JXz9QcepDjvRMNpmoW8+2MGuZgTSLs75nvddF+qxlrEBeqPOPl/pQDuEvw0VvfRRLUSFLcVPSwBHdPW/eYf809LlND3ZPoKykIvYOjkdZgx9TH8Ir8jtArda2y/nXu7t6IiGLuOHt4lNGlbPT3dNvUk+l9bgNI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064165; c=relaxed/simple; bh=ky/X9OZtJn3bDUIMsiZj6Q/CKpBiasfn5wmoF4H7EIQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=XRsRUCZW0z6RfaiVDKVls62MmasF9gnz7Fzl22/tp6qlsQb69szWOJSLt7yXexHAJMRig4GSc+rPa6bPITgm3ZJoennBch12rvf94zqiMy8gN2kGrsjfa+mIWzfk/Ghvm+lLv5c4IpCk0P+TYcqkRL1ReaAhzUkLYjmcj103Xo0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=jt25OZLY; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="jt25OZLY" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064163; x=1737600163; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ky/X9OZtJn3bDUIMsiZj6Q/CKpBiasfn5wmoF4H7EIQ=; b=jt25OZLYS7LpZOu5zHqGSROrwMt/7ISLhFBLOB9hapQwFZFSrylUm+Gt b/jswxurHGf7uUEfbg6FPArDWW1jzBr3RkCHK5of4Bm8SEAxaPWmj6FLC 0uFHUpc/8uwIH6gLtCmvjHahBwXpaMa9g0OunD0RxO9bxgd0TYMQjd3lD o5Kw99Ci0kXhso6DQfNduh6jTmlZOmS3+NJoGY1WeWQ/1YHWyuZk9u6rW E6+4OsUfax8enofYiXXbrKGpMUUb7odN3Vk4tRW3YiCybc9vLi0wEQa2a 7bVqyvlD/XW0f3BQa/kdMzOQvVn7uK0gbPVGfAVkjCPbgECYUvtoUIrsr g==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586525" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586525" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825885" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:39 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 16/27] KVM: x86: Add fault checks for guest CR4.CET setting Date: Tue, 23 Jan 2024 18:41:49 -0800 Message-Id: <20240124024200.102792-17-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Check potential faults for CR4.CET setting per Intel SDM requirements. CET can be enabled if and only if CR0.WP == 1, i.e. setting CR4.CET == 1 faults if CR0.WP == 0 and setting CR0.WP == 0 fails if CR4.CET == 1. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Signed-off-by: Yang Weijiang Reviewed-by: Chao Gao Reviewed-by: Maxim Levitsky --- arch/x86/kvm/x86.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index bde780ae69bf..b418e4f5277b 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1006,6 +1006,9 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) (is_64_bit_mode(vcpu) || kvm_is_cr4_bit_set(vcpu, X86_CR4_PCIDE))) return 1; + if (!(cr0 & X86_CR0_WP) && kvm_is_cr4_bit_set(vcpu, X86_CR4_CET)) + return 1; + static_call(kvm_x86_set_cr0)(vcpu, cr0); kvm_post_set_cr0(vcpu, old_cr0, cr0); @@ -1217,6 +1220,9 @@ int kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) return 1; } + if ((cr4 & X86_CR4_CET) && !kvm_is_cr0_bit_set(vcpu, X86_CR0_WP)) + return 1; + static_call(kvm_x86_set_cr4)(vcpu, cr4); kvm_post_set_cr4(vcpu, old_cr4, cr4); From patchwork Wed Jan 24 02:41:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528433 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACAA718026; Wed, 24 Jan 2024 02:42:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064166; cv=none; b=s/THc3EdidnvTdX7pj4Tgtv5NqAgMTllrnAFafNdtpxSSLOYhvFASeue8RPGrD9np8gRl4pm/AftvxuJdyED7iTogXfST2ebbzk2uahSfLWfRYF0L9oF2B9Gmfkd+O8RNrioBaKgzzby0P344wa7N5blQ2K0lA78GYj2/Qd3PhU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064166; c=relaxed/simple; bh=4OgIxio6Sgg6qCqmMI3dmUyrNdmCfi/Fz+DUsVjuAWg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=acokwRt8UeXMgx8SUtZSlDd9bA7L8/BeX1o6MEbqQ7GwrmgmRj/NkN3lF3zKDxaWWWfMulS2MvJtI6l4AagKh3DNhs0OrvxYPT2MULHzPIQh7PGPgiPyjdAWh2Qh2N3eLw6UFc7DGPXfUbUw2uCqU1AogJVAYRJ7LVOgTiilH00= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Do3HmADP; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Do3HmADP" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064164; x=1737600164; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=4OgIxio6Sgg6qCqmMI3dmUyrNdmCfi/Fz+DUsVjuAWg=; b=Do3HmADPcfHmFvdJdeYnqTidAv/3Fl5FDqHA5gRVTW48wWVtNP64r5tz jJoTKkW8yECaE3925vlfgMzv2iGMr/p6Eecr2bnsyRlOtN4Z9J2hDKzQ5 19tvgMkTPlYS+M0tzZHu5jfgzcszeXJWjXLp8I+lVzqPzfihKP7qb2JDg dW580pBBSbtv2m9mCqRGpxOFTrF6YysQRrNTdTONk5AM0FFj8fKBwsgqY RPy7Pu2Hqk0iCOTkZhuRyKqbRsaL1XtjwwvTgA+tSCWMmtA6km+RLlHlP N7oGDwkINAdiWWOBPIT4y5NmfzhUjn1WLVDybYRFxlqqxl+qL5BWnu8lP Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586531" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586531" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825890" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:39 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 17/27] KVM: x86: Report KVM supported CET MSRs as to-be-saved Date: Tue, 23 Jan 2024 18:41:50 -0800 Message-Id: <20240124024200.102792-18-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add CET MSRs to the list of MSRs reported to userspace if the feature, i.e. IBT or SHSTK, associated with the MSRs is supported by KVM. SSP can only be read via RDSSP. Writing even requires destructive and potentially faulting operations such as SAVEPREVSSP/RSTORSSP or SETSSBSY/CLRSSBSY. Let the host use a pseudo-MSR that is just a wrapper for the GUEST_SSP field of the VMCS. Suggested-by: Chao Gao Signed-off-by: Yang Weijiang --- arch/x86/include/uapi/asm/kvm_para.h | 1 + arch/x86/kvm/vmx/vmx.c | 2 ++ arch/x86/kvm/x86.c | 18 ++++++++++++++++++ 3 files changed, 21 insertions(+) diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h index 6e64b27b2c1e..9864bbcf2470 100644 --- a/arch/x86/include/uapi/asm/kvm_para.h +++ b/arch/x86/include/uapi/asm/kvm_para.h @@ -58,6 +58,7 @@ #define MSR_KVM_ASYNC_PF_INT 0x4b564d06 #define MSR_KVM_ASYNC_PF_ACK 0x4b564d07 #define MSR_KVM_MIGRATION_CONTROL 0x4b564d08 +#define MSR_KVM_SSP 0x4b564d09 struct kvm_steal_time { __u64 steal; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index d21f55f323ea..b2f6bcf3bf9b 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7007,6 +7007,8 @@ static bool vmx_has_emulated_msr(struct kvm *kvm, u32 index) case MSR_AMD64_TSC_RATIO: /* This is AMD only. */ return false; + case MSR_KVM_SSP: + return kvm_cpu_cap_has(X86_FEATURE_SHSTK); default: return true; } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b418e4f5277b..a7368adad6b8 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1476,6 +1476,9 @@ static const u32 msrs_to_save_base[] = { MSR_IA32_XFD, MSR_IA32_XFD_ERR, MSR_IA32_XSS, + MSR_IA32_U_CET, MSR_IA32_S_CET, + MSR_IA32_PL0_SSP, MSR_IA32_PL1_SSP, MSR_IA32_PL2_SSP, + MSR_IA32_PL3_SSP, MSR_IA32_INT_SSP_TAB, }; static const u32 msrs_to_save_pmu[] = { @@ -1579,6 +1582,7 @@ static const u32 emulated_msrs_all[] = { MSR_K7_HWCR, MSR_KVM_POLL_CONTROL, + MSR_KVM_SSP, }; static u32 emulated_msrs[ARRAY_SIZE(emulated_msrs_all)]; @@ -7428,6 +7432,20 @@ static void kvm_probe_msr_to_save(u32 msr_index) if (!kvm_caps.supported_xss) return; break; + case MSR_IA32_U_CET: + case MSR_IA32_S_CET: + if (!kvm_cpu_cap_has(X86_FEATURE_SHSTK) && + !kvm_cpu_cap_has(X86_FEATURE_IBT)) + return; + break; + case MSR_IA32_INT_SSP_TAB: + if (!kvm_cpu_cap_has(X86_FEATURE_LM)) + return; + fallthrough; + case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + if (!kvm_cpu_cap_has(X86_FEATURE_SHSTK)) + return; + break; default: break; } From patchwork Wed Jan 24 02:41:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528434 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CBA5318036; Wed, 24 Jan 2024 02:42:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064166; cv=none; b=ntSkGpstBW37Q54dTSdxr4q56BHBBuy+f9/1bcOWCV1twVqLZYfLVdAV0FpPOUqTUcdboEPDdTH+RZzwFZtP9vOQyjYRtICjJHHspCKmvxFGGBy7dH0Y2DHPo18JkwfPlYYFCquVtBoZZiyDvuHKvzwIFaLIijd5wY4zYTSFRHg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064166; c=relaxed/simple; bh=Ky/BGy7YdiRQEiX3OiH3FB749KKoV/jpv2wQq4kCc/0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bSjaSz6u5s/noF9Nx/MsfL6pZFk7uOdBRc0M0Ztt2IS/J5wj4K+k9PgTUH66HV9RU6743PggBC/9Dp0UvRbrnkILrGsLpyXVeWdAAZuQIR+ziXexqaGQcdEJOTH++OTGwj2lVLXfDORNwBhRU36UL1g7C4X3LW0hfziNWI+rdR8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=CbDVM2+4; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="CbDVM2+4" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064164; x=1737600164; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Ky/BGy7YdiRQEiX3OiH3FB749KKoV/jpv2wQq4kCc/0=; b=CbDVM2+4HxyEX3niapZFJsSmH+tjZvljW8aLC/mh7+C9sP5iWdLyRoJL k+eLsDeSOtFlODZY6zk5LmHu7uYqQRqVqB97xKC7cvSWVfpoxA1f7A0yH d2AK97KCXlx/SykIyzpYxbA6LjrJNEwReN5J4kVBvwptsTbjANHqsgO0a 58MyltWSPHOjWsF2Mw09FHEc+d6duvemgVZ4i6uitir2hRrKWQibCHLCi e0gLiAtcXW/Xc4DCPAh0uiDYr5pIRyNZ8+8sg2W5Ra4upmaqxgju74mZf Yt0HIfRYWHteqqVy/dCOp7NVjzDApY8N09Ds0VJkFr/M076ACgW4r1uf2 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586537" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586537" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825894" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:40 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com, Zhang Yi Z Subject: [PATCH v9 18/27] KVM: VMX: Introduce CET VMCS fields and control bits Date: Tue, 23 Jan 2024 18:41:51 -0800 Message-Id: <20240124024200.102792-19-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Control-flow Enforcement Technology (CET) is a kind of CPU feature used to prevent Return/CALL/Jump-Oriented Programming (ROP/COP/JOP) attacks. It provides two sub-features(SHSTK,IBT) to defend against ROP/COP/JOP style control-flow subversion attacks. Shadow Stack (SHSTK): A shadow stack is a second stack used exclusively for control transfer operations. The shadow stack is separate from the data/normal stack and can be enabled individually in user and kernel mode. When shadow stack is enabled, CALL pushes the return address on both the data and shadow stack. RET pops the return address from both stacks and compares them. If the return addresses from the two stacks do not match, the processor generates a #CP. Indirect Branch Tracking (IBT): IBT introduces instruction(ENDBRANCH)to mark valid target addresses of indirect branches (CALL, JMP etc...). If an indirect branch is executed and the next instruction is _not_ an ENDBRANCH, the processor generates a #CP. These instruction behaves as a NOP on platforms that have no CET. Several new CET MSRs are defined to support CET: MSR_IA32_{U,S}_CET: CET settings for {user,supervisor} CET respectively. MSR_IA32_PL{0,1,2,3}_SSP: SHSTK pointer linear address for CPL{0,1,2,3}. MSR_IA32_INT_SSP_TAB: Linear address of SHSTK pointer table, whose entry is indexed by IST of interrupt gate desc. Two XSAVES state bits are introduced for CET: IA32_XSS:[bit 11]: Control saving/restoring user mode CET states IA32_XSS:[bit 12]: Control saving/restoring supervisor mode CET states. Six VMCS fields are introduced for CET: {HOST,GUEST}_S_CET: Stores CET settings for kernel mode. {HOST,GUEST}_SSP: Stores current active SSP. {HOST,GUEST}_INTR_SSP_TABLE: Stores current active MSR_IA32_INT_SSP_TAB. On Intel platforms, two additional bits are defined in VM_EXIT and VM_ENTRY control fields: If VM_EXIT_LOAD_CET_STATE = 1, host CET states are loaded from following VMCS fields at VM-Exit: HOST_S_CET HOST_SSP HOST_INTR_SSP_TABLE If VM_ENTRY_LOAD_CET_STATE = 1, guest CET states are loaded from following VMCS fields at VM-Entry: GUEST_S_CET GUEST_SSP GUEST_INTR_SSP_TABLE Co-developed-by: Zhang Yi Z Signed-off-by: Zhang Yi Z Signed-off-by: Yang Weijiang Reviewed-by: Chao Gao Reviewed-by: Maxim Levitsky --- arch/x86/include/asm/vmx.h | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index 0e73616b82f3..451fd4f4fedc 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -104,6 +104,7 @@ #define VM_EXIT_CLEAR_BNDCFGS 0x00800000 #define VM_EXIT_PT_CONCEAL_PIP 0x01000000 #define VM_EXIT_CLEAR_IA32_RTIT_CTL 0x02000000 +#define VM_EXIT_LOAD_CET_STATE 0x10000000 #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR 0x00036dff @@ -117,6 +118,7 @@ #define VM_ENTRY_LOAD_BNDCFGS 0x00010000 #define VM_ENTRY_PT_CONCEAL_PIP 0x00020000 #define VM_ENTRY_LOAD_IA32_RTIT_CTL 0x00040000 +#define VM_ENTRY_LOAD_CET_STATE 0x00100000 #define VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR 0x000011ff @@ -345,6 +347,9 @@ enum vmcs_field { GUEST_PENDING_DBG_EXCEPTIONS = 0x00006822, GUEST_SYSENTER_ESP = 0x00006824, GUEST_SYSENTER_EIP = 0x00006826, + GUEST_S_CET = 0x00006828, + GUEST_SSP = 0x0000682a, + GUEST_INTR_SSP_TABLE = 0x0000682c, HOST_CR0 = 0x00006c00, HOST_CR3 = 0x00006c02, HOST_CR4 = 0x00006c04, @@ -357,6 +362,9 @@ enum vmcs_field { HOST_IA32_SYSENTER_EIP = 0x00006c12, HOST_RSP = 0x00006c14, HOST_RIP = 0x00006c16, + HOST_S_CET = 0x00006c18, + HOST_SSP = 0x00006c1a, + HOST_INTR_SSP_TABLE = 0x00006c1c }; /* From patchwork Wed Jan 24 02:41:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528435 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44997182A7; Wed, 24 Jan 2024 02:42:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064166; cv=none; b=rlxPlHzrD4ZwAYcmtjFpqWwCnuCZUYo3wHPObpIprIvd4jvNcnwlqnCZ2h3DXth+h3Yb6efae4loGl7/xS17QRG/f5ASFFS/NAd78NXsum1sHmk4bawhd8dBgWYWzddfUD9vWC3JhgDc/+dflY+wQgh8k7D4hO0KFawC9Va9voA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064166; c=relaxed/simple; bh=F9opDfCZwYUSpXz1rhkoPf6Q6E9dmRJye3dTlxS+ZDs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IebQ9JGEWuStW2tHehNJ9SMqf4ljIO7KKxtAfzgnx0oZVkTT94kG4Edv95usw2iqHPKkAgz6el9JxsjGDA8Tzm+TkDhph0jBU4v5hjY1/VXogFrkN+4KNPQT4XWtEZQUcbO0gpdq0atd/Nku4Rl0J1TwEhTSsKPnPi+QMFABNR4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=J0+cL8v9; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="J0+cL8v9" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064165; x=1737600165; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=F9opDfCZwYUSpXz1rhkoPf6Q6E9dmRJye3dTlxS+ZDs=; b=J0+cL8v9JTr26Zgfc2bvZRgmcsD2zWIn5Q4bnRewOwbziZv56MFZNfIX Pi5swcMhU7NHMlKOdHWoA70aQo3bXNFQbdoML5S0Vgz71T8CjPdzvMYDq Pe3Qp8EKEjK5EVzabcQlkwKkuOakqC4tOuW4/wfYxeQrpKHMzl/k7eQqu U6nqRVDN4+UIVQaMUo/N2+dA7todLcLRWYLtA+d6WtWKwpOhVEGhpBt6d /R1XA1JHPS0QjTwOfHQixpG5SiK/fM9FcCaCX0AZ7hUvZrdJfraLJX7hf EdomgjASZyc6/6Qww4y+cLZpjVsK2RK4pmswKq+XMc6cZSU7ws6szQclt g==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586543" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586543" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825899" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:40 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 19/27] KVM: x86: Use KVM-governed feature framework to track "SHSTK/IBT enabled" Date: Tue, 23 Jan 2024 18:41:52 -0800 Message-Id: <20240124024200.102792-20-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Use the governed feature framework to track whether X86_FEATURE_SHSTK and X86_FEATURE_IBT features can be used by userspace and guest, i.e., the features can be used iff both KVM and guest CPUID can support them. TODO: remove this patch once Sean's refactor to "KVM-governed" framework is upstreamed. See the work here [*]. [*]: https://lore.kernel.org/all/20231110235528.1561679-1-seanjc@google.com/ Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky --- arch/x86/kvm/governed_features.h | 2 ++ arch/x86/kvm/vmx/vmx.c | 2 ++ 2 files changed, 4 insertions(+) diff --git a/arch/x86/kvm/governed_features.h b/arch/x86/kvm/governed_features.h index ad463b1ed4e4..daf0c0a3e29c 100644 --- a/arch/x86/kvm/governed_features.h +++ b/arch/x86/kvm/governed_features.h @@ -17,6 +17,8 @@ KVM_GOVERNED_X86_FEATURE(PFTHRESHOLD) KVM_GOVERNED_X86_FEATURE(VGIF) KVM_GOVERNED_X86_FEATURE(VNMI) KVM_GOVERNED_X86_FEATURE(LAM) +KVM_GOVERNED_X86_FEATURE(SHSTK) +KVM_GOVERNED_X86_FEATURE(IBT) #undef KVM_GOVERNED_X86_FEATURE #undef KVM_GOVERNED_FEATURE diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index b2f6bcf3bf9b..29a0fd3e83c5 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7764,6 +7764,8 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) kvm_governed_feature_check_and_set(vcpu, X86_FEATURE_VMX); kvm_governed_feature_check_and_set(vcpu, X86_FEATURE_LAM); + kvm_governed_feature_check_and_set(vcpu, X86_FEATURE_SHSTK); + kvm_governed_feature_check_and_set(vcpu, X86_FEATURE_IBT); vmx_setup_uret_msrs(vmx); From patchwork Wed Jan 24 02:41:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528436 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 93D5B18C1A; Wed, 24 Jan 2024 02:42:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064168; cv=none; b=Unb8MFYz9W7ojyNVNYxNDPfLz88rQU7jZ84hyNNNagh7js7Tl4H9GPRlGd25cTxlse3QCco1s0ygBEAPZnVpweDSYoSagrGYSMlHb+0xUFzEnEPTRBDsZQSdoKwePKl+TRbfPQS8LgAxtcMpd8lYHXEeCfd9JJqBitgwjCfUUAw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064168; c=relaxed/simple; bh=KjDe60Yb7E8XbbbfTdTf5HJMHBxDk3I1fydRBBz9T98=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tBW2TS9/fgLByaKwOBn3crNBN8KcRjObd7zhFtEezxa5aE4C0GFgpXL8I3JfZZ7D0yxqKTT4hJPO19AlCz8eUIkSHXqFMXrH6aeu96uzmv/9BXp0cE5q+hgJ3XnNLZ7aMoFFx6Xk8+vYozAY+93zdbXbNS4PvPgdDXNDqzwC9kQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=c/xpOLmZ; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="c/xpOLmZ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064166; x=1737600166; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KjDe60Yb7E8XbbbfTdTf5HJMHBxDk3I1fydRBBz9T98=; b=c/xpOLmZXPajLknn+aYPAzTYW1yGBD0mhw6M2YNGMks0WDivgAGQYjJv yDLgF8NGVJtRJsgE8RC2UuILpqka3hSCjvdkOIYNhfswJ/nN20jZR1WPL E1Ng6B2mc5U2k+iW85SkB5yc+C17RMT1JSJuzepaBqsBqs+W5l7B9BkiK f+cD4UX7WOCb+Zf6solvUJ/ZlDhN/PCB+tcj8dgZFZSIRC+mIQ6qnoFT3 /WT+IeoTaFx0eW2qPV5TZWH/i6/Mje2m8hLJCNIx7CUxHbE7o7OSVCa/o /LvaYUB8TvGydwEO/bxr81TurUOLMR1/S1UQOWD1v+2+aH4yprGmjljCq g==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586550" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586550" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825904" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:41 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 20/27] KVM: VMX: Emulate read and write to CET MSRs Date: Tue, 23 Jan 2024 18:41:53 -0800 Message-Id: <20240124024200.102792-21-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add emulation interface for CET MSR access. The emulation code is split into common part and vendor specific part. The former does common checks for MSRs, e.g., accessibility, data validity etc., then pass the operation to either XSAVE-managed MSRs via the helpers or CET VMCS fields. Suggested-by: Sean Christopherson Signed-off-by: Yang Weijiang --- arch/x86/kvm/vmx/vmx.c | 18 +++++++++ arch/x86/kvm/x86.c | 88 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 106 insertions(+) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 29a0fd3e83c5..064a5fe87948 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -2106,6 +2106,15 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) else msr_info->data = vmx->pt_desc.guest.addr_a[index / 2]; break; + case MSR_IA32_S_CET: + msr_info->data = vmcs_readl(GUEST_S_CET); + break; + case MSR_KVM_SSP: + msr_info->data = vmcs_readl(GUEST_SSP); + break; + case MSR_IA32_INT_SSP_TAB: + msr_info->data = vmcs_readl(GUEST_INTR_SSP_TABLE); + break; case MSR_IA32_DEBUGCTLMSR: msr_info->data = vmcs_read64(GUEST_IA32_DEBUGCTL); break; @@ -2415,6 +2424,15 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) else vmx->pt_desc.guest.addr_a[index / 2] = data; break; + case MSR_IA32_S_CET: + vmcs_writel(GUEST_S_CET, data); + break; + case MSR_KVM_SSP: + vmcs_writel(GUEST_SSP, data); + break; + case MSR_IA32_INT_SSP_TAB: + vmcs_writel(GUEST_INTR_SSP_TABLE, data); + break; case MSR_IA32_PERF_CAPABILITIES: if (data && !vcpu_to_pmu(vcpu)->version) return 1; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index a7368adad6b8..cf0f9e4474a4 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1850,6 +1850,36 @@ bool kvm_msr_allowed(struct kvm_vcpu *vcpu, u32 index, u32 type) } EXPORT_SYMBOL_GPL(kvm_msr_allowed); +#define CET_US_RESERVED_BITS GENMASK(9, 6) +#define CET_US_SHSTK_MASK_BITS GENMASK(1, 0) +#define CET_US_IBT_MASK_BITS (GENMASK_ULL(5, 2) | GENMASK_ULL(63, 10)) +#define CET_US_LEGACY_BITMAP_BASE(data) ((data) >> 12) + +static bool is_set_cet_msr_allowed(struct kvm_vcpu *vcpu, u32 index, u64 data, + bool host_initiated) +{ + bool msr_ctrl = index == MSR_IA32_S_CET || index == MSR_IA32_U_CET; + + if (guest_can_use(vcpu, X86_FEATURE_SHSTK)) + return true; + + if (msr_ctrl && guest_can_use(vcpu, X86_FEATURE_IBT)) + return true; + + /* + * If KVM supports the MSR, i.e. has enumerated the MSR existence to + * userspace, then userspace is allowed to write '0' irrespective of + * whether or not the MSR is exposed to the guest. + */ + if (!host_initiated || data) + return false; + + if (kvm_cpu_cap_has(X86_FEATURE_SHSTK)) + return true; + + return msr_ctrl && kvm_cpu_cap_has(X86_FEATURE_IBT); +} + /* * Write @data into the MSR specified by @index. Select MSR specific fault * checks are bypassed if @host_initiated is %true. @@ -1909,6 +1939,43 @@ static int __kvm_set_msr(struct kvm_vcpu *vcpu, u32 index, u64 data, data = (u32)data; break; + case MSR_IA32_U_CET: + case MSR_IA32_S_CET: + if (!is_set_cet_msr_allowed(vcpu, index, data, host_initiated)) + return 1; + if (data & CET_US_RESERVED_BITS) + return 1; + if (!guest_can_use(vcpu, X86_FEATURE_SHSTK) && + (data & CET_US_SHSTK_MASK_BITS)) + return 1; + if (!guest_can_use(vcpu, X86_FEATURE_IBT) && + (data & CET_US_IBT_MASK_BITS)) + return 1; + if (!IS_ALIGNED(CET_US_LEGACY_BITMAP_BASE(data), 4)) + return 1; + /* IBT can be suppressed iff the TRACKER isn't WAIT_ENDBR. */ + if ((data & CET_SUPPRESS) && (data & CET_WAIT_ENDBR)) + return 1; + break; + case MSR_IA32_INT_SSP_TAB: + if (!is_set_cet_msr_allowed(vcpu, index, data, host_initiated) || + !guest_cpuid_has(vcpu, X86_FEATURE_LM)) + return 1; + if (is_noncanonical_address(data, vcpu)) + return 1; + break; + case MSR_KVM_SSP: + if (!host_initiated) + return 1; + fallthrough; + case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + if (!is_set_cet_msr_allowed(vcpu, index, data, host_initiated)) + return 1; + if (is_noncanonical_address(data, vcpu)) + return 1; + if (!IS_ALIGNED(data, 4)) + return 1; + break; } msr.data = data; @@ -1952,6 +2019,19 @@ static int __kvm_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data, !guest_cpuid_has(vcpu, X86_FEATURE_RDPID)) return 1; break; + case MSR_IA32_INT_SSP_TAB: + if (!guest_can_use(vcpu, X86_FEATURE_SHSTK) || + !guest_cpuid_has(vcpu, X86_FEATURE_LM)) + return 1; + break; + case MSR_KVM_SSP: + if (!host_initiated) + return 1; + fallthrough; + case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + if (!guest_can_use(vcpu, X86_FEATURE_SHSTK)) + return 1; + break; } msr.index = index; @@ -4143,6 +4223,10 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) vcpu->arch.guest_fpu.xfd_err = data; break; #endif + case MSR_IA32_U_CET: + case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + kvm_set_xstate_msr(vcpu, msr_info); + break; default: if (kvm_pmu_is_valid_msr(vcpu, msr)) return kvm_pmu_set_msr(vcpu, msr_info); @@ -4502,6 +4586,10 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) msr_info->data = vcpu->arch.guest_fpu.xfd_err; break; #endif + case MSR_IA32_U_CET: + case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + kvm_get_xstate_msr(vcpu, msr_info); + break; default: if (kvm_pmu_is_valid_msr(vcpu, msr_info->index)) return kvm_pmu_get_msr(vcpu, msr_info); From patchwork Wed Jan 24 02:41:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528437 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D52B718E1E; Wed, 24 Jan 2024 02:42:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064168; cv=none; b=EgUyayNWxWgdk1JOfRUn0J79pc+00N9V9gOYnCjvPLInly/cLWCv9X3OqiOQrOLgtwveD/PHhq5b3JeBfShuiVwldOTMEnLqgON9e46KLfFX9+QePjQSd7WP9Z82Ux+r5EapKXu5QtaF4m868ofnzbDGlgUAdkeuDlrAOpqsByc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064168; c=relaxed/simple; bh=h8um0cUpGOSDzpshStMJ7pABEvMbvAz9TJHO3NVRyXM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=N7wwVHQbkDRfOobNLs2MEL6Kxvs5hg03q/4Z1attXQ+Fja/maePuui0wuKd6s/lAOV8ySLKN8DFX0lQzIYz/ClgL8UCnkCj4qUNVseRzZ22vB+BHdKi09+hJC2fMS/sK80LHA20UZTw2JHyjwOXU4srxxkNBtSROOeqNqWwRhbc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=RD+YoJYk; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="RD+YoJYk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064166; x=1737600166; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=h8um0cUpGOSDzpshStMJ7pABEvMbvAz9TJHO3NVRyXM=; b=RD+YoJYksDxfDmMEAVbos7KH/+l5xlDjF0Z+Rd323aJSSO4ok4bzjcev QXU3mpVb9vqRxnu+0YZNcAv2lahXIL4QzIb+UlvzGCt8ef4ATyUOxk3c+ emEst1PpeTr++i7cSwloMa3ye6jb74XQPrn5WEL8kn+O6sDSokwPB26Pe loA2hxRdYrLZsN3ghT+9mScAPPmXZN6xM4OSjvlxDrRX5ug5ZTkb+q25c 0Mluy8WpTxuwOtdM9FWfP+D2VRnmBnADJb+P+aFa6mjbpW8udwCxjxNa1 YkjHbTesc3R1ubnWH1fug1MeymsCN1hX8JuSkekQBmmmTlGf9fqOJN0qI w==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586556" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586556" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825909" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:42 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 21/27] KVM: x86: Save and reload SSP to/from SMRAM Date: Tue, 23 Jan 2024 18:41:54 -0800 Message-Id: <20240124024200.102792-22-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Save CET SSP to SMRAM on SMI and reload it on RSM. KVM emulates HW arch behavior when guest enters/leaves SMM mode,i.e., save registers to SMRAM at the entry of SMM and reload them at the exit to SMM. Per SDM, SSP is one of such registers on 64-bit Arch, and add the support for SSP. Note, on 32-bit Arch, SSP is not defined in SMRAM, so fail 32-bit CET guest launch. Suggested-by: Sean Christopherson Suggested-by: Chao Gao Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky --- arch/x86/kvm/cpuid.c | 11 +++++++++++ arch/x86/kvm/smm.c | 8 ++++++++ arch/x86/kvm/smm.h | 2 +- 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 3ab133530573..95233b0879a3 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -149,6 +149,17 @@ static int kvm_check_cpuid(struct kvm_vcpu *vcpu, if (vaddr_bits != 48 && vaddr_bits != 57 && vaddr_bits != 0) return -EINVAL; } + /* + * Prevent 32-bit guest launch if shadow stack is exposed as SSP + * state is not defined for 32-bit SMRAM. + */ + best = cpuid_entry2_find(entries, nent, 0x80000001, + KVM_CPUID_INDEX_NOT_SIGNIFICANT); + if (best && !(best->edx & F(LM))) { + best = cpuid_entry2_find(entries, nent, 0x7, 0); + if (best && (best->ecx & F(SHSTK))) + return -EINVAL; + } /* * Exposing dynamic xfeatures to the guest requires additional diff --git a/arch/x86/kvm/smm.c b/arch/x86/kvm/smm.c index 45c855389ea7..7aac9c54c353 100644 --- a/arch/x86/kvm/smm.c +++ b/arch/x86/kvm/smm.c @@ -275,6 +275,10 @@ static void enter_smm_save_state_64(struct kvm_vcpu *vcpu, enter_smm_save_seg_64(vcpu, &smram->gs, VCPU_SREG_GS); smram->int_shadow = static_call(kvm_x86_get_interrupt_shadow)(vcpu); + + if (guest_can_use(vcpu, X86_FEATURE_SHSTK)) + KVM_BUG_ON(kvm_msr_read(vcpu, MSR_KVM_SSP, &smram->ssp), + vcpu->kvm); } #endif @@ -564,6 +568,10 @@ static int rsm_load_state_64(struct x86_emulate_ctxt *ctxt, static_call(kvm_x86_set_interrupt_shadow)(vcpu, 0); ctxt->interruptibility = (u8)smstate->int_shadow; + if (guest_can_use(vcpu, X86_FEATURE_SHSTK)) + KVM_BUG_ON(kvm_msr_write(vcpu, MSR_KVM_SSP, smstate->ssp), + vcpu->kvm); + return X86EMUL_CONTINUE; } #endif diff --git a/arch/x86/kvm/smm.h b/arch/x86/kvm/smm.h index a1cf2ac5bd78..1e2a3e18207f 100644 --- a/arch/x86/kvm/smm.h +++ b/arch/x86/kvm/smm.h @@ -116,8 +116,8 @@ struct kvm_smram_state_64 { u32 smbase; u32 reserved4[5]; - /* ssp and svm_* fields below are not implemented by KVM */ u64 ssp; + /* svm_* fields below are not implemented by KVM */ u64 svm_guest_pat; u64 svm_host_efer; u64 svm_host_cr4; From patchwork Wed Jan 24 02:41:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528438 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F3C5B18E2E; Wed, 24 Jan 2024 02:42:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064169; cv=none; b=D/g2It0/SQptT0d6ws0iLOoo8m76UHjrQzrcxl42Kw02RLpLicCCGLGRvj8T7vW0IWq1ALdSSvgxA7JjuoKI3k+Bt4LRdLFsohS3CP7twh597WO9suwcOl9/+3NsRDiL+UfL119RL0II7rqSEt9cY3c9qHgoXWlwONmM723OkoA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064169; c=relaxed/simple; bh=aUDtDd9ZiQ5V5sav0MsmNz0UbTbFO26UdWqqrMbfWFc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NHUcwfqjoUyhjuYxdl5IRc3eOXgBEymTknIUjbzNDXGdwav25EOyUnqtCAMX4sBh1I7FQCVoexmyoR0CH8DBD8vLvm2pJZfOytYiD52cKQ4Ad1nbYpsZG7YobwUivHMqD/TAb66W1VaAxKZdkt2w+Nu1OpZXExlHu/wu3ixGnrQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=AQOESCbq; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="AQOESCbq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064167; x=1737600167; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=aUDtDd9ZiQ5V5sav0MsmNz0UbTbFO26UdWqqrMbfWFc=; b=AQOESCbqqW3TKW+tqFAsZVYe+3kjX5lpFFHTFJoqha3umZOjWFfwOsqH cHFQ9vtoqg1K+lebeGPbdbW/7tIkm0pY/YirBuCOq7Zxnu05UYZXSrM65 0c1kaKLzmwnKccmff84bt7B4AXeLlBw794qH902rrEsnE+cwxCSOGH/1M dJf6kqUh5qS/u4ESRy1YPNiwu6YeTKIGd7U84CoOceKjr1TZXJUtJtOui ARLMGTbKwJcL+itDsDdtX6qUTuVRRhE04vzdzATJH4Cua4fhwFLa3uCQM 24JANC5MjYNA5iSl2Wi/47/l8+194Nqd4z4xwMGhdH5AgPbH6COjo7lQG w==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586563" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586563" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825914" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:42 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 22/27] KVM: VMX: Set up interception for CET MSRs Date: Tue, 23 Jan 2024 18:41:55 -0800 Message-Id: <20240124024200.102792-23-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Enable/disable CET MSRs interception per associated feature configuration. Shadow Stack feature requires all CET MSRs passed through to guest to make it supported in user and supervisor mode while IBT feature only depends on MSR_IA32_{U,S}_CETS_CET to enable user and supervisor IBT. Note, this MSR design introduced an architectural limitation of SHSTK and IBT control for guest, i.e., when SHSTK is exposed, IBT is also available to guest from architectual perspective since IBT relies on subset of SHSTK relevant MSRs. Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky Reviewed-by: Chao Gao --- arch/x86/kvm/vmx/vmx.c | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 064a5fe87948..34e91dbbffed 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -692,6 +692,10 @@ static bool is_valid_passthrough_msr(u32 msr) case MSR_LBR_CORE_TO ... MSR_LBR_CORE_TO + 8: /* LBR MSRs. These are handled in vmx_update_intercept_for_lbr_msrs() */ return true; + case MSR_IA32_U_CET: + case MSR_IA32_S_CET: + case MSR_IA32_PL0_SSP ... MSR_IA32_INT_SSP_TAB: + return true; } r = possible_passthrough_msr_slot(msr) != -ENOENT; @@ -7767,6 +7771,41 @@ static void update_intel_pt_cfg(struct kvm_vcpu *vcpu) vmx->pt_desc.ctl_bitmask &= ~(0xfULL << (32 + i * 4)); } +static void vmx_update_intercept_for_cet_msr(struct kvm_vcpu *vcpu) +{ + bool incpt; + + if (kvm_cpu_cap_has(X86_FEATURE_SHSTK)) { + incpt = !guest_cpuid_has(vcpu, X86_FEATURE_SHSTK); + + vmx_set_intercept_for_msr(vcpu, MSR_IA32_U_CET, + MSR_TYPE_RW, incpt); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_S_CET, + MSR_TYPE_RW, incpt); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_PL0_SSP, + MSR_TYPE_RW, incpt); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_PL1_SSP, + MSR_TYPE_RW, incpt); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_PL2_SSP, + MSR_TYPE_RW, incpt); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_PL3_SSP, + MSR_TYPE_RW, incpt); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_INT_SSP_TAB, + MSR_TYPE_RW, incpt); + if (!incpt) + return; + } + + if (kvm_cpu_cap_has(X86_FEATURE_IBT)) { + incpt = !guest_cpuid_has(vcpu, X86_FEATURE_IBT); + + vmx_set_intercept_for_msr(vcpu, MSR_IA32_U_CET, + MSR_TYPE_RW, incpt); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_S_CET, + MSR_TYPE_RW, incpt); + } +} + static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -7845,6 +7884,8 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) /* Refresh #PF interception to account for MAXPHYADDR changes. */ vmx_update_exception_bitmap(vcpu); + + vmx_update_intercept_for_cet_msr(vcpu); } static u64 vmx_get_perf_capabilities(void) From patchwork Wed Jan 24 02:41:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528439 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDCBD1B5A0; Wed, 24 Jan 2024 02:42:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064170; cv=none; b=UfpFFHACyB3u8NZsvvOT+y0gYJa5dKjgQflEHyrMK8xFWWZgjCx7zuvc4vHssgiER3VgxcxkndCRaIA4m4N8WyXjoX7hCm8vR/7iO3TYsqzFZK1zfcnwxvXv+1pCic83jwsq2SMYfsZW9Iokl2q0W3UBWqk3Rn9Bg/X5Nu0FnaY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064170; c=relaxed/simple; bh=cqA+46P9Nu0FwZGZRW9VynovBYFc0Tg+zn4fsIptNWI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=q8cjc186EP2mpok6CKsfqrPl3aBDgYIyak0H2zjl34L5w3cojEwsDIjMDhQ4qqjsCDYwc018bxolplJAfIq5fEhPSg14hl2rCjCk2oHjrX3MrX+lKvJN09geQ9Y6DiAK3iuZuehOXQ22Bttv9l5V40t7FgUJVH1n3zDrQHlUH0s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=VvPI2eDt; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="VvPI2eDt" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064168; x=1737600168; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cqA+46P9Nu0FwZGZRW9VynovBYFc0Tg+zn4fsIptNWI=; b=VvPI2eDtwdL9H7REPO4CkUkoddW/L9teqGQjC701kAyUfH2pBuWLlWzi HHihqu3uO6x2NRgz/WI4ErsEbMRXqnakozyWNpIdmKJ/qrWgDnIxveKcb bWco01COp7xkPjSIJg7CuvDoYv+BO2QA19EUUN7drGZ31n9RCKmP0Y1gH 5mqjZ/UPjHsCtWPy0LOBETwST7U3AvBKnyxIzHfqJgPNCld56kMPknM18 63Gna38O7wTeTnetCztU53iLVMSznzff2Ena5WSYpz4nVTY6tC6Y9M0nt ICKY8rKWRVFfsHaz194eRO/bho+JfsH2IMwbKLjdrqjNkpgG+q7tzT/PW Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586570" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586570" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825918" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:43 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 23/27] KVM: VMX: Set host constant supervisor states to VMCS fields Date: Tue, 23 Jan 2024 18:41:56 -0800 Message-Id: <20240124024200.102792-24-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Save constant values to HOST_{S_CET,SSP,INTR_SSP_TABLE} field explicitly. Kernel IBT is supported and the setting in MSR_IA32_S_CET is static after post-boot(The exception is BIOS call case but vCPU thread never across it) and KVM doesn't need to refresh HOST_S_CET field before every VM-Enter/ VM-Exit sequence. Host supervisor shadow stack is not enabled now and SSP is not accessible to kernel mode, thus it's safe to set host IA32_INT_SSP_TAB/SSP VMCS field to 0s. When shadow stack is enabled for CPL3, SSP is reloaded from PL3_SSP before it exits to userspace. Check SDM Vol 2A/B Chapter 3/4 for SYSCALL/ SYSRET/SYSENTER SYSEXIT/RDSSP/CALL etc. Prevent KVM module loading if host supervisor shadow stack SHSTK_EN is set in MSR_IA32_S_CET as KVM cannot co-exit with it correctly. Suggested-by: Sean Christopherson Suggested-by: Chao Gao Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky Reviewed-by: Chao Gao --- arch/x86/kvm/vmx/capabilities.h | 4 ++++ arch/x86/kvm/vmx/vmx.c | 15 +++++++++++++++ arch/x86/kvm/x86.c | 14 ++++++++++++++ arch/x86/kvm/x86.h | 1 + 4 files changed, 34 insertions(+) diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilities.h index 41a4533f9989..ee8938818c8a 100644 --- a/arch/x86/kvm/vmx/capabilities.h +++ b/arch/x86/kvm/vmx/capabilities.h @@ -106,6 +106,10 @@ static inline bool cpu_has_load_perf_global_ctrl(void) return vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL; } +static inline bool cpu_has_load_cet_ctrl(void) +{ + return (vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_CET_STATE); +} static inline bool cpu_has_vmx_mpx(void) { return vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_BNDCFGS; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 34e91dbbffed..3993afbacd51 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -4371,6 +4371,21 @@ void vmx_set_constant_host_state(struct vcpu_vmx *vmx) if (cpu_has_load_ia32_efer()) vmcs_write64(HOST_IA32_EFER, host_efer); + + /* + * Supervisor shadow stack is not enabled on host side, i.e., + * host IA32_S_CET.SHSTK_EN bit is guaranteed to 0 now, per SDM + * description(RDSSP instruction), SSP is not readable in CPL0, + * so resetting the two registers to 0s at VM-Exit does no harm + * to kernel execution. When execution flow exits to userspace, + * SSP is reloaded from IA32_PL3_SSP. Check SDM Vol.2A/B Chapter + * 3 and 4 for details. + */ + if (cpu_has_load_cet_ctrl()) { + vmcs_writel(HOST_S_CET, host_s_cet); + vmcs_writel(HOST_SSP, 0); + vmcs_writel(HOST_INTR_SSP_TABLE, 0); + } } void set_cr4_guest_host_mask(struct vcpu_vmx *vmx) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index cf0f9e4474a4..9596763fae8d 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -114,6 +114,8 @@ static u64 __read_mostly efer_reserved_bits = ~((u64)EFER_SCE); #endif static u64 __read_mostly cr4_reserved_bits = CR4_RESERVED_BITS; +u64 __read_mostly host_s_cet; +EXPORT_SYMBOL_GPL(host_s_cet); #define KVM_EXIT_HYPERCALL_VALID_MASK (1 << KVM_HC_MAP_GPA_RANGE) @@ -9840,6 +9842,18 @@ static int __kvm_x86_vendor_init(struct kvm_x86_init_ops *ops) return -EIO; } + if (boot_cpu_has(X86_FEATURE_SHSTK)) { + rdmsrl(MSR_IA32_S_CET, host_s_cet); + /* + * Linux doesn't yet support supervisor shadow stacks (SSS), so + * KVM doesn't save/restore the associated MSRs, i.e. KVM may + * clobber the host values. Yell and refuse to load if SSS is + * unexpectedly enabled, e.g. to avoid crashing the host. + */ + if (WARN_ON_ONCE(host_s_cet & CET_SHSTK_EN)) + return -EIO; + } + x86_emulator_cache = kvm_alloc_emulator_cache(); if (!x86_emulator_cache) { pr_err("failed to allocate cache for x86 emulator\n"); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 9c19dfb5011d..656107e64c93 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -325,6 +325,7 @@ fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu); extern u64 host_xcr0; extern u64 host_xss; extern u64 host_arch_capabilities; +extern u64 host_s_cet; extern struct kvm_caps kvm_caps; From patchwork Wed Jan 24 02:41:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528440 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D8C9A1B5A4; Wed, 24 Jan 2024 02:42:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064171; cv=none; b=ae789kDBXPRMTqtLiiQidDtnZ469yF9WofXFfxTkJ0OmpgSScAMLdxSYfSR+4ABO/6iUQmVW8q0uQ/3T3rU9Tul4W6WD24mhJyyAhkQ4GS8I3OH30Rm+PXMKGb6b3Oo6fJi/n5V4nTfsYgIyBjy0PtqCDVx2P95FskXNkY25wvI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064171; c=relaxed/simple; bh=CEnMI5wtNp+rbmbsE5HoFVLRmNjiC+YpHuafjWHCY9Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=sHxGrtb8k3gRvJ2Fpwl59uVBiAm0D8YUrpkVJwwlFOrC7rxWmSrtNL2GPRc0Abh55jiC4PfLrv5QOkOUEOwNl3Um/rx+a4E3lvxtCKwWl1/1F2i3D6nc0jGxWVwhJoUaRbCM/lxnU4nCq9sm+PR4yPqim7lk01JhYtot1vnu5Xw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=YGekpiR3; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="YGekpiR3" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064168; x=1737600168; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CEnMI5wtNp+rbmbsE5HoFVLRmNjiC+YpHuafjWHCY9Q=; b=YGekpiR3K73efjmP+fBIajq1NWlu0W9tH7h/5I6YY3lbdILcuJV2D6Aj 4Y8qT0/WabN4tGluydweE0cq7Cg7PpJ0+qoNr5QHbc0LVohtDrGiOjWl6 QYEE4ej2Dp7dZmtOYagf7ouVlQVBtno+6cnaKYr5Wc4VclSw5fKkE4s26 XeaRIVCjPLvzoE9rUdJ6123+q0NwMK6KMvCaGQGUj7pjxkUUc/WK9TxS9 9QzixL2vx0mDVWy+rxqhm8iFR9nrcCPrJWb2RsdCClm9WvaM8Rnq6RcGx +Y4bQJIADWkDWemfkevKXQGyLOH7r1rY18MBYvy2kn/yaodwONrCM/pXP Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586578" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586578" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825925" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:43 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 24/27] KVM: x86: Enable CET virtualization for VMX and advertise to userspace Date: Tue, 23 Jan 2024 18:41:57 -0800 Message-Id: <20240124024200.102792-25-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Expose CET features to guest if KVM/host can support them, clear CPUID feature bits if KVM/host cannot support. Set CPUID feature bits so that CET features are available in guest CPUID. Add CR4.CET bit support in order to allow guest set CET master control bit. Disable KVM CET feature if unrestricted_guest is unsupported/disabled as KVM does not support emulating CET. The CET load-bits in VM_ENTRY/VM_EXIT control fields should be set to make guest CET xstates isolated from host's. On platforms with VMX_BASIC[bit56] == 0, inject #CP at VMX entry with error code will fail, and if VMX_BASIC[bit56] == 1, #CP injection with or without error code is allowed. Disable CET feature bits if the MSR bit is cleared so that nested VMM can inject #CP if and only if VMX_BASIC[bit56] == 1. Don't expose CET feature if either of {U,S}_CET xstate bits is cleared in host XSS or if XSAVES isn't supported. CET MSR contents after reset, power-up and INIT are set to 0s, clears the guest fpstate fields so that the guest MSRs are reset to 0s after the events. Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky --- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/include/asm/msr-index.h | 1 + arch/x86/kvm/cpuid.c | 25 ++++++++++++++++++++----- arch/x86/kvm/vmx/capabilities.h | 6 ++++++ arch/x86/kvm/vmx/vmx.c | 28 +++++++++++++++++++++++++++- arch/x86/kvm/vmx/vmx.h | 6 ++++-- arch/x86/kvm/x86.c | 31 +++++++++++++++++++++++++++++-- arch/x86/kvm/x86.h | 3 +++ 8 files changed, 91 insertions(+), 11 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 6efaaaa15945..161d0552be5f 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -134,7 +134,7 @@ | X86_CR4_OSXSAVE | X86_CR4_SMEP | X86_CR4_FSGSBASE \ | X86_CR4_OSXMMEXCPT | X86_CR4_LA57 | X86_CR4_VMXE \ | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP \ - | X86_CR4_LAM_SUP)) + | X86_CR4_LAM_SUP | X86_CR4_CET)) #define CR8_RESERVED_BITS (~(unsigned long)X86_CR8_TPR) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 1d51e1850ed0..233e00c01e62 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -1102,6 +1102,7 @@ #define VMX_BASIC_MEM_TYPE_MASK 0x003c000000000000LLU #define VMX_BASIC_MEM_TYPE_WB 6LLU #define VMX_BASIC_INOUT 0x0040000000000000LLU +#define VMX_BASIC_NO_HW_ERROR_CODE_CC 0x0100000000000000LLU /* Resctrl MSRs: */ /* - Intel: */ diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 95233b0879a3..fddc54991cd4 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -150,14 +150,14 @@ static int kvm_check_cpuid(struct kvm_vcpu *vcpu, return -EINVAL; } /* - * Prevent 32-bit guest launch if shadow stack is exposed as SSP - * state is not defined for 32-bit SMRAM. + * CET is not supported for 32-bit guest, prevent guest launch if + * shadow stack or IBT is enabled for 32-bit guest. */ best = cpuid_entry2_find(entries, nent, 0x80000001, KVM_CPUID_INDEX_NOT_SIGNIFICANT); if (best && !(best->edx & F(LM))) { best = cpuid_entry2_find(entries, nent, 0x7, 0); - if (best && (best->ecx & F(SHSTK))) + if (best && ((best->ecx & F(SHSTK)) || (best->edx & F(IBT)))) return -EINVAL; } @@ -665,7 +665,7 @@ void kvm_set_cpu_caps(void) F(AVX512_VPOPCNTDQ) | F(UMIP) | F(AVX512_VBMI2) | F(GFNI) | F(VAES) | F(VPCLMULQDQ) | F(AVX512_VNNI) | F(AVX512_BITALG) | F(CLDEMOTE) | F(MOVDIRI) | F(MOVDIR64B) | 0 /*WAITPKG*/ | - F(SGX_LC) | F(BUS_LOCK_DETECT) + F(SGX_LC) | F(BUS_LOCK_DETECT) | F(SHSTK) ); /* Set LA57 based on hardware capability. */ if (cpuid_ecx(7) & F(LA57)) @@ -683,7 +683,8 @@ void kvm_set_cpu_caps(void) F(SPEC_CTRL_SSBD) | F(ARCH_CAPABILITIES) | F(INTEL_STIBP) | F(MD_CLEAR) | F(AVX512_VP2INTERSECT) | F(FSRM) | F(SERIALIZE) | F(TSXLDTRK) | F(AVX512_FP16) | - F(AMX_TILE) | F(AMX_INT8) | F(AMX_BF16) | F(FLUSH_L1D) + F(AMX_TILE) | F(AMX_INT8) | F(AMX_BF16) | F(FLUSH_L1D) | + F(IBT) ); /* TSC_ADJUST and ARCH_CAPABILITIES are emulated in software. */ @@ -696,6 +697,20 @@ void kvm_set_cpu_caps(void) kvm_cpu_cap_set(X86_FEATURE_INTEL_STIBP); if (boot_cpu_has(X86_FEATURE_AMD_SSBD)) kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL_SSBD); + /* + * Don't use boot_cpu_has() to check availability of IBT because the + * feature bit is cleared in boot_cpu_data when ibt=off is applied + * in host cmdline. + * + * As currently there's no HW bug which requires disabling IBT feature + * while CPU can enumerate it, host cmdline option ibt=off is most + * likely due to administrative reason on host side, so KVM refers to + * CPU CPUID enumeration to enable the feature. In future if there's + * actually some bug clobbered ibt=off option, then enforce additional + * check here to disable the support in KVM. + */ + if (cpuid_edx(7) & F(IBT)) + kvm_cpu_cap_set(X86_FEATURE_IBT); kvm_cpu_cap_mask(CPUID_7_1_EAX, F(AVX_VNNI) | F(AVX512_BF16) | F(CMPCCXADD) | diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilities.h index ee8938818c8a..e12bc233d88b 100644 --- a/arch/x86/kvm/vmx/capabilities.h +++ b/arch/x86/kvm/vmx/capabilities.h @@ -79,6 +79,12 @@ static inline bool cpu_has_vmx_basic_inout(void) return (((u64)vmcs_config.basic_cap << 32) & VMX_BASIC_INOUT); } +static inline bool cpu_has_vmx_basic_no_hw_errcode(void) +{ + return ((u64)vmcs_config.basic_cap << 32) & + VMX_BASIC_NO_HW_ERROR_CODE_CC; +} + static inline bool cpu_has_virtual_nmis(void) { return vmcs_config.pin_based_exec_ctrl & PIN_BASED_VIRTUAL_NMIS && diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 3993afbacd51..ef7aca954228 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -2609,6 +2609,7 @@ static int setup_vmcs_config(struct vmcs_config *vmcs_conf, { VM_ENTRY_LOAD_IA32_EFER, VM_EXIT_LOAD_IA32_EFER }, { VM_ENTRY_LOAD_BNDCFGS, VM_EXIT_CLEAR_BNDCFGS }, { VM_ENTRY_LOAD_IA32_RTIT_CTL, VM_EXIT_CLEAR_IA32_RTIT_CTL }, + { VM_ENTRY_LOAD_CET_STATE, VM_EXIT_LOAD_CET_STATE }, }; memset(vmcs_conf, 0, sizeof(*vmcs_conf)); @@ -4934,6 +4935,14 @@ static void vmx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, 0); /* 22.2.1 */ + if (kvm_cpu_cap_has(X86_FEATURE_SHSTK)) + vmcs_writel(GUEST_SSP, 0); + if (kvm_cpu_cap_has(X86_FEATURE_SHSTK) || + kvm_cpu_cap_has(X86_FEATURE_IBT)) + vmcs_writel(GUEST_S_CET, 0); + if (kvm_cpu_cap_has(X86_FEATURE_SHSTK)) + vmcs_writel(GUEST_INTR_SSP_TABLE, 0); + kvm_make_request(KVM_REQ_APIC_PAGE_RELOAD, vcpu); vpid_sync_context(vmx->vpid); @@ -6353,6 +6362,12 @@ void dump_vmcs(struct kvm_vcpu *vcpu) if (vmcs_read32(VM_EXIT_MSR_STORE_COUNT) > 0) vmx_dump_msrs("guest autostore", &vmx->msr_autostore.guest); + if (vmentry_ctl & VM_ENTRY_LOAD_CET_STATE) { + pr_err("S_CET = 0x%016lx\n", vmcs_readl(GUEST_S_CET)); + pr_err("SSP = 0x%016lx\n", vmcs_readl(GUEST_SSP)); + pr_err("INTR SSP TABLE = 0x%016lx\n", + vmcs_readl(GUEST_INTR_SSP_TABLE)); + } pr_err("*** Host State ***\n"); pr_err("RIP = 0x%016lx RSP = 0x%016lx\n", vmcs_readl(HOST_RIP), vmcs_readl(HOST_RSP)); @@ -6430,6 +6445,12 @@ void dump_vmcs(struct kvm_vcpu *vcpu) if (secondary_exec_control & SECONDARY_EXEC_ENABLE_VPID) pr_err("Virtual processor ID = 0x%04x\n", vmcs_read16(VIRTUAL_PROCESSOR_ID)); + if (vmexit_ctl & VM_EXIT_LOAD_CET_STATE) { + pr_err("S_CET = 0x%016lx\n", vmcs_readl(HOST_S_CET)); + pr_err("SSP = 0x%016lx\n", vmcs_readl(HOST_SSP)); + pr_err("INTR SSP TABLE = 0x%016lx\n", + vmcs_readl(HOST_INTR_SSP_TABLE)); + } } /* @@ -7965,7 +7986,6 @@ static __init void vmx_set_cpu_caps(void) kvm_cpu_cap_set(X86_FEATURE_UMIP); /* CPUID 0xD.1 */ - kvm_caps.supported_xss = 0; if (!cpu_has_vmx_xsaves()) kvm_cpu_cap_clear(X86_FEATURE_XSAVES); @@ -7977,6 +7997,12 @@ static __init void vmx_set_cpu_caps(void) if (cpu_has_vmx_waitpkg()) kvm_cpu_cap_check_and_set(X86_FEATURE_WAITPKG); + + if (!cpu_has_load_cet_ctrl() || !enable_unrestricted_guest || + !cpu_has_vmx_basic_no_hw_errcode()) { + kvm_cpu_cap_clear(X86_FEATURE_SHSTK); + kvm_cpu_cap_clear(X86_FEATURE_IBT); + } } static void vmx_request_immediate_exit(struct kvm_vcpu *vcpu) diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index e3b0985bb74a..d0cad2624564 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -484,7 +484,8 @@ static inline u8 vmx_get_rvi(void) VM_ENTRY_LOAD_IA32_EFER | \ VM_ENTRY_LOAD_BNDCFGS | \ VM_ENTRY_PT_CONCEAL_PIP | \ - VM_ENTRY_LOAD_IA32_RTIT_CTL) + VM_ENTRY_LOAD_IA32_RTIT_CTL | \ + VM_ENTRY_LOAD_CET_STATE) #define __KVM_REQUIRED_VMX_VM_EXIT_CONTROLS \ (VM_EXIT_SAVE_DEBUG_CONTROLS | \ @@ -506,7 +507,8 @@ static inline u8 vmx_get_rvi(void) VM_EXIT_LOAD_IA32_EFER | \ VM_EXIT_CLEAR_BNDCFGS | \ VM_EXIT_PT_CONCEAL_PIP | \ - VM_EXIT_CLEAR_IA32_RTIT_CTL) + VM_EXIT_CLEAR_IA32_RTIT_CTL | \ + VM_EXIT_LOAD_CET_STATE) #define KVM_REQUIRED_VMX_PIN_BASED_VM_EXEC_CONTROL \ (PIN_BASED_EXT_INTR_MASK | \ diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 9596763fae8d..eb531823447a 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -231,7 +231,8 @@ static struct kvm_user_return_msrs __percpu *user_return_msrs; | XFEATURE_MASK_BNDCSR | XFEATURE_MASK_AVX512 \ | XFEATURE_MASK_PKRU | XFEATURE_MASK_XTILE) -#define KVM_SUPPORTED_XSS 0 +#define KVM_SUPPORTED_XSS (XFEATURE_MASK_CET_USER | \ + XFEATURE_MASK_CET_KERNEL) u64 __read_mostly host_efer; EXPORT_SYMBOL_GPL(host_efer); @@ -9921,6 +9922,20 @@ static int __kvm_x86_vendor_init(struct kvm_x86_init_ops *ops) if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES)) kvm_caps.supported_xss = 0; + if (!kvm_cpu_cap_has(X86_FEATURE_SHSTK) && + !kvm_cpu_cap_has(X86_FEATURE_IBT)) + kvm_caps.supported_xss &= ~(XFEATURE_MASK_CET_USER | + XFEATURE_MASK_CET_KERNEL); + + if ((kvm_caps.supported_xss & (XFEATURE_MASK_CET_USER | + XFEATURE_MASK_CET_KERNEL)) != + (XFEATURE_MASK_CET_USER | XFEATURE_MASK_CET_KERNEL)) { + kvm_cpu_cap_clear(X86_FEATURE_SHSTK); + kvm_cpu_cap_clear(X86_FEATURE_IBT); + kvm_caps.supported_xss &= ~(XFEATURE_MASK_CET_USER | + XFEATURE_MASK_CET_KERNEL); + } + #define __kvm_cpu_cap_has(UNUSED_, f) kvm_cpu_cap_has(f) cr4_reserved_bits = __cr4_reserved_bits(__kvm_cpu_cap_has, UNUSED_); #undef __kvm_cpu_cap_has @@ -12392,7 +12407,9 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu) static inline bool is_xstate_reset_needed(void) { - return kvm_cpu_cap_has(X86_FEATURE_MPX); + return kvm_cpu_cap_has(X86_FEATURE_MPX) || + kvm_cpu_cap_has(X86_FEATURE_SHSTK) || + kvm_cpu_cap_has(X86_FEATURE_IBT); } void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) @@ -12469,6 +12486,16 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) XFEATURE_BNDCSR); } + if (kvm_cpu_cap_has(X86_FEATURE_SHSTK)) { + fpstate_clear_xstate_component(fpstate, + XFEATURE_CET_USER); + fpstate_clear_xstate_component(fpstate, + XFEATURE_CET_KERNEL); + } else if (kvm_cpu_cap_has(X86_FEATURE_IBT)) { + fpstate_clear_xstate_component(fpstate, + XFEATURE_CET_USER); + } + if (init_event) kvm_load_guest_fpu(vcpu); } diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 656107e64c93..cc585051d24b 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -533,6 +533,9 @@ bool kvm_msr_allowed(struct kvm_vcpu *vcpu, u32 index, u32 type); __reserved_bits |= X86_CR4_PCIDE; \ if (!__cpu_has(__c, X86_FEATURE_LAM)) \ __reserved_bits |= X86_CR4_LAM_SUP; \ + if (!__cpu_has(__c, X86_FEATURE_SHSTK) && \ + !__cpu_has(__c, X86_FEATURE_IBT)) \ + __reserved_bits |= X86_CR4_CET; \ __reserved_bits; \ }) From patchwork Wed Jan 24 02:41:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528441 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E747028389; Wed, 24 Jan 2024 02:42:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064172; cv=none; b=QIvtOYp0Uwtet5CIdY10vBSIN69fYgnCdZ7Se8z8yl4Q0K1gC7WZURCiPGHgpyLklev5GBtp2FmJRW8s4IGel6O83cRwvkkhyo2Gh+Ysc9HwE4+KDx11M5x0xTgFYQSHnn2DLbQS3QMz0x8tGZRAOq8M32Voz+slrdYO/bIHWPE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064172; c=relaxed/simple; bh=VzRTyBG95NMIc7bDJOln3dLMG8SRjMb/e4tnQdiKqPY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NdnL99xVuNWjIT8cN8W7/6DsVt0jIjOYCtPM8r2tYJCST+PkcgrAuchYkf3q3nqNpLsOixaog9mdoo6bTbVTslfD7Wqg6ieJ4NjMwfhQuurmiXUQUVK9jtCJExziWf1AHwT+pTpFaeFugZ9AS2GIUOO4w71qqKdDwIFOSyJLaXk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=UxLr68fV; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="UxLr68fV" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064169; x=1737600169; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VzRTyBG95NMIc7bDJOln3dLMG8SRjMb/e4tnQdiKqPY=; b=UxLr68fVgWd7ARNLV1Xi16Nrqtj93xxsABiCNcWyqTB49wJYdzfrjxU0 qpAWW7TajgWbgfZXvERv+HdCfBivREAIlyeBHfbw6Hjcbq/09mCoS5PvC Y3W+gcKMYr51yh1impw4wETWgsiXKdLYeQ5GHenOe53YGHdRtDniwpxsr c4MSBhG7mQ4K11gng4RXZIe/RvKwcH8UkQWqgcrqMNKOssXOpyf3tPyNU ntUtBCj+QQcqj7zMlHsdwtK7kkR7U38BHy/G0MNKOgoP2Kwo3gu6BnS/q 8fq10TdLOTy7bafFrYjNEPzRFKBfOtKkB+Pxfa0o0n3Ey6FazdCKfrqCz w==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586588" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586588" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825929" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:44 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 25/27] KVM: nVMX: Introduce new VMX_BASIC bit for event error_code delivery to L1 Date: Tue, 23 Jan 2024 18:41:58 -0800 Message-Id: <20240124024200.102792-26-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Per SDM description(Vol.3D, Appendix A.1): "If bit 56 is read as 1, software can use VM entry to deliver a hardware exception with or without an error code, regardless of vector" Modify has_error_code check before inject events to nested guest. Only enforce the check when guest is in real mode, the exception is not hard exception and the platform doesn't enumerate bit56 in VMX_BASIC, in all other case ignore the check to make the logic consistent with SDM. Suggested-by: Chao Gao Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky Reviewed-by: Chao Gao --- arch/x86/kvm/vmx/nested.c | 27 ++++++++++++++++++--------- arch/x86/kvm/vmx/nested.h | 5 +++++ 2 files changed, 23 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index b2e9853584b8..468a7cf75035 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -1230,9 +1230,9 @@ static int vmx_restore_vmx_basic(struct vcpu_vmx *vmx, u64 data) { const u64 feature_and_reserved = /* feature (except bit 48; see below) */ - BIT_ULL(49) | BIT_ULL(54) | BIT_ULL(55) | + BIT_ULL(49) | BIT_ULL(54) | BIT_ULL(55) | BIT_ULL(56) | /* reserved */ - BIT_ULL(31) | GENMASK_ULL(47, 45) | GENMASK_ULL(63, 56); + BIT_ULL(31) | GENMASK_ULL(47, 45) | GENMASK_ULL(63, 57); u64 vmx_basic = vmcs_config.nested.basic; if (!is_bitwise_subset(vmx_basic, data, feature_and_reserved)) @@ -2865,7 +2865,6 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu, u8 vector = intr_info & INTR_INFO_VECTOR_MASK; u32 intr_type = intr_info & INTR_INFO_INTR_TYPE_MASK; bool has_error_code = intr_info & INTR_INFO_DELIVER_CODE_MASK; - bool should_have_error_code; bool urg = nested_cpu_has2(vmcs12, SECONDARY_EXEC_UNRESTRICTED_GUEST); bool prot_mode = !urg || vmcs12->guest_cr0 & X86_CR0_PE; @@ -2882,12 +2881,20 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu, CC(intr_type == INTR_TYPE_OTHER_EVENT && vector != 0)) return -EINVAL; - /* VM-entry interruption-info field: deliver error code */ - should_have_error_code = - intr_type == INTR_TYPE_HARD_EXCEPTION && prot_mode && - x86_exception_has_error_code(vector); - if (CC(has_error_code != should_have_error_code)) - return -EINVAL; + /* + * Cannot deliver error code in real mode or if the interrupt + * type is not hardware exception. For other cases, do the + * consistency check only if the vCPU doesn't enumerate + * VMX_BASIC_NO_HW_ERROR_CODE_CC. + */ + if (!prot_mode || intr_type != INTR_TYPE_HARD_EXCEPTION) { + if (CC(has_error_code)) + return -EINVAL; + } else if (!nested_cpu_has_no_hw_errcode_cc(vcpu)) { + if (CC(has_error_code != + x86_exception_has_error_code(vector))) + return -EINVAL; + } /* VM-entry exception error code */ if (CC(has_error_code && @@ -7011,6 +7018,8 @@ static void nested_vmx_setup_basic(struct nested_vmx_msrs *msrs) if (cpu_has_vmx_basic_inout()) msrs->basic |= VMX_BASIC_INOUT; + if (cpu_has_vmx_basic_no_hw_errcode()) + msrs->basic |= VMX_BASIC_NO_HW_ERROR_CODE_CC; } static void nested_vmx_setup_cr_fixed(struct nested_vmx_msrs *msrs) diff --git a/arch/x86/kvm/vmx/nested.h b/arch/x86/kvm/vmx/nested.h index cce4e2aa30fb..747061c2aeb9 100644 --- a/arch/x86/kvm/vmx/nested.h +++ b/arch/x86/kvm/vmx/nested.h @@ -285,6 +285,11 @@ static inline bool nested_cr4_valid(struct kvm_vcpu *vcpu, unsigned long val) __kvm_is_valid_cr4(vcpu, val); } +static inline bool nested_cpu_has_no_hw_errcode_cc(struct kvm_vcpu *vcpu) +{ + return to_vmx(vcpu)->nested.msrs.basic & VMX_BASIC_NO_HW_ERROR_CODE_CC; +} + /* No difference in the restrictions on guest and host CR4 in VMX operation. */ #define nested_guest_cr4_valid nested_cr4_valid #define nested_host_cr4_valid nested_cr4_valid From patchwork Wed Jan 24 02:41:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528442 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2A752BAFD; Wed, 24 Jan 2024 02:42:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064172; cv=none; b=aIaKU028MiFTEHcq+9H9w0VQ+gNDtAL1g0lyU28o10sGP+du5RxH/Ka8CKscz+1JXy8kROb+VAJwG17D8GCWsh1d1YYcQD4Cg0OlpsHWWdvuJBYEQvTJQ4yeGxIF3S0lYpNQ0/vJO6oOKWnGzapbtsCO+0CddNpxtgn8gH9BFFc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064172; c=relaxed/simple; bh=Y2iZ/863z0AfxrDmZWFmP7SSubY9D1G8VEP8bgNpmyw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=aYNgukmnOjKE8NzjFwPe6L7SXAY2rsXXl9TSL42Kz8SNfJwOKNhIpqC6YA0PNh+K48RYoxt4eq5QmHD/QEqPIslUwrdShQzVh+38lr3XOUYlDj2OLEfSf+Pw20G74gCxmXb8oIoIIxqdhp2JiqdWAz6vPB9r/jZj7S7gMVrEIks= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Vi1EJz+x; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Vi1EJz+x" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064171; x=1737600171; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Y2iZ/863z0AfxrDmZWFmP7SSubY9D1G8VEP8bgNpmyw=; b=Vi1EJz+xsxHpQdjn5QYRDXl0NzsrlFygl4LkwhuMhQjp+5nGioykIThl yZ1n8asx8YaEMwDWtUwwuv7Z1HxNeC4U6oqbu6hrTTismdxZ4Q8qxNBBJ P/XIGxQUkQVvNgn9YZUfJw8orl1haP6UyD+BOa7A1+QpXT+JT21YokxtM qf3Aij6Uv4mLi0q5V+f08pcR9stnN02NPF64L9fRaVzajze9torkYL3fh zSxYtKIBMQyy27UsZzydBIpDZA/nDkvlUyHRb+3UkYzDJFvuRkRxODVuK cARU/eGG8PA39FY/Ezim8DL14b1Cka/Lub3CU83DRl+k9TVnSNYiOutOW w==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586592" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586592" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825936" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:45 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 26/27] KVM: nVMX: Enable CET support for nested guest Date: Tue, 23 Jan 2024 18:41:59 -0800 Message-Id: <20240124024200.102792-27-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Set up CET MSRs, related VM_ENTRY/EXIT control bits and fixed CR4 setting to enable CET for nested VM. vmcs12 and vmcs02 needs to be synced when L2 exits to L1 or when L1 wants to resume L2, that way correct CET states can be observed by one another. Suggested-by: Chao Gao Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky --- arch/x86/kvm/vmx/nested.c | 57 +++++++++++++++++++++++++++++++++++++-- arch/x86/kvm/vmx/vmcs12.c | 6 +++++ arch/x86/kvm/vmx/vmcs12.h | 14 +++++++++- arch/x86/kvm/vmx/vmx.c | 2 ++ 4 files changed, 76 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 468a7cf75035..e330897a7e5e 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -691,6 +691,28 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, MSR_IA32_FLUSH_CMD, MSR_TYPE_W); + /* Pass CET MSRs to nested VM if L0 and L1 are set to pass-through. */ + nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, + MSR_IA32_U_CET, MSR_TYPE_RW); + + nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, + MSR_IA32_S_CET, MSR_TYPE_RW); + + nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, + MSR_IA32_PL0_SSP, MSR_TYPE_RW); + + nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, + MSR_IA32_PL1_SSP, MSR_TYPE_RW); + + nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, + MSR_IA32_PL2_SSP, MSR_TYPE_RW); + + nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, + MSR_IA32_PL3_SSP, MSR_TYPE_RW); + + nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, + MSR_IA32_INT_SSP_TAB, MSR_TYPE_RW); + kvm_vcpu_unmap(vcpu, &vmx->nested.msr_bitmap_map, false); vmx->nested.force_msr_bitmap_recalc = false; @@ -2506,6 +2528,17 @@ static void prepare_vmcs02_rare(struct vcpu_vmx *vmx, struct vmcs12 *vmcs12) if (kvm_mpx_supported() && vmx->nested.nested_run_pending && (vmcs12->vm_entry_controls & VM_ENTRY_LOAD_BNDCFGS)) vmcs_write64(GUEST_BNDCFGS, vmcs12->guest_bndcfgs); + + if (vmcs12->vm_entry_controls & VM_ENTRY_LOAD_CET_STATE) { + if (guest_can_use(&vmx->vcpu, X86_FEATURE_SHSTK)) { + vmcs_writel(GUEST_SSP, vmcs12->guest_ssp); + vmcs_writel(GUEST_INTR_SSP_TABLE, + vmcs12->guest_ssp_tbl); + } + if (guest_can_use(&vmx->vcpu, X86_FEATURE_SHSTK) || + guest_can_use(&vmx->vcpu, X86_FEATURE_IBT)) + vmcs_writel(GUEST_S_CET, vmcs12->guest_s_cet); + } } if (nested_cpu_has_xsaves(vmcs12)) @@ -4344,6 +4377,15 @@ static void sync_vmcs02_to_vmcs12_rare(struct kvm_vcpu *vcpu, vmcs12->guest_pending_dbg_exceptions = vmcs_readl(GUEST_PENDING_DBG_EXCEPTIONS); + if (guest_can_use(&vmx->vcpu, X86_FEATURE_SHSTK)) { + vmcs12->guest_ssp = vmcs_readl(GUEST_SSP); + vmcs12->guest_ssp_tbl = vmcs_readl(GUEST_INTR_SSP_TABLE); + } + if (guest_can_use(&vmx->vcpu, X86_FEATURE_SHSTK) || + guest_can_use(&vmx->vcpu, X86_FEATURE_IBT)) { + vmcs12->guest_s_cet = vmcs_readl(GUEST_S_CET); + } + vmx->nested.need_sync_vmcs02_to_vmcs12_rare = false; } @@ -4569,6 +4611,16 @@ static void load_vmcs12_host_state(struct kvm_vcpu *vcpu, if (vmcs12->vm_exit_controls & VM_EXIT_CLEAR_BNDCFGS) vmcs_write64(GUEST_BNDCFGS, 0); + if (vmcs12->vm_exit_controls & VM_EXIT_LOAD_CET_STATE) { + if (guest_can_use(vcpu, X86_FEATURE_SHSTK)) { + vmcs_writel(GUEST_SSP, vmcs12->host_ssp); + vmcs_writel(GUEST_INTR_SSP_TABLE, vmcs12->host_ssp_tbl); + } + if (guest_can_use(vcpu, X86_FEATURE_SHSTK) || + guest_can_use(vcpu, X86_FEATURE_IBT)) + vmcs_writel(GUEST_S_CET, vmcs12->host_s_cet); + } + if (vmcs12->vm_exit_controls & VM_EXIT_LOAD_IA32_PAT) { vmcs_write64(GUEST_IA32_PAT, vmcs12->host_ia32_pat); vcpu->arch.pat = vmcs12->host_ia32_pat; @@ -6840,7 +6892,7 @@ static void nested_vmx_setup_exit_ctls(struct vmcs_config *vmcs_conf, VM_EXIT_HOST_ADDR_SPACE_SIZE | #endif VM_EXIT_LOAD_IA32_PAT | VM_EXIT_SAVE_IA32_PAT | - VM_EXIT_CLEAR_BNDCFGS; + VM_EXIT_CLEAR_BNDCFGS | VM_EXIT_LOAD_CET_STATE; msrs->exit_ctls_high |= VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR | VM_EXIT_LOAD_IA32_EFER | VM_EXIT_SAVE_IA32_EFER | @@ -6862,7 +6914,8 @@ static void nested_vmx_setup_entry_ctls(struct vmcs_config *vmcs_conf, #ifdef CONFIG_X86_64 VM_ENTRY_IA32E_MODE | #endif - VM_ENTRY_LOAD_IA32_PAT | VM_ENTRY_LOAD_BNDCFGS; + VM_ENTRY_LOAD_IA32_PAT | VM_ENTRY_LOAD_BNDCFGS | + VM_ENTRY_LOAD_CET_STATE; msrs->entry_ctls_high |= (VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR | VM_ENTRY_LOAD_IA32_EFER | VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL); diff --git a/arch/x86/kvm/vmx/vmcs12.c b/arch/x86/kvm/vmx/vmcs12.c index 106a72c923ca..4233b5ca9461 100644 --- a/arch/x86/kvm/vmx/vmcs12.c +++ b/arch/x86/kvm/vmx/vmcs12.c @@ -139,6 +139,9 @@ const unsigned short vmcs12_field_offsets[] = { FIELD(GUEST_PENDING_DBG_EXCEPTIONS, guest_pending_dbg_exceptions), FIELD(GUEST_SYSENTER_ESP, guest_sysenter_esp), FIELD(GUEST_SYSENTER_EIP, guest_sysenter_eip), + FIELD(GUEST_S_CET, guest_s_cet), + FIELD(GUEST_SSP, guest_ssp), + FIELD(GUEST_INTR_SSP_TABLE, guest_ssp_tbl), FIELD(HOST_CR0, host_cr0), FIELD(HOST_CR3, host_cr3), FIELD(HOST_CR4, host_cr4), @@ -151,5 +154,8 @@ const unsigned short vmcs12_field_offsets[] = { FIELD(HOST_IA32_SYSENTER_EIP, host_ia32_sysenter_eip), FIELD(HOST_RSP, host_rsp), FIELD(HOST_RIP, host_rip), + FIELD(HOST_S_CET, host_s_cet), + FIELD(HOST_SSP, host_ssp), + FIELD(HOST_INTR_SSP_TABLE, host_ssp_tbl), }; const unsigned int nr_vmcs12_fields = ARRAY_SIZE(vmcs12_field_offsets); diff --git a/arch/x86/kvm/vmx/vmcs12.h b/arch/x86/kvm/vmx/vmcs12.h index 01936013428b..3884489e7f7e 100644 --- a/arch/x86/kvm/vmx/vmcs12.h +++ b/arch/x86/kvm/vmx/vmcs12.h @@ -117,7 +117,13 @@ struct __packed vmcs12 { natural_width host_ia32_sysenter_eip; natural_width host_rsp; natural_width host_rip; - natural_width paddingl[8]; /* room for future expansion */ + natural_width host_s_cet; + natural_width host_ssp; + natural_width host_ssp_tbl; + natural_width guest_s_cet; + natural_width guest_ssp; + natural_width guest_ssp_tbl; + natural_width paddingl[2]; /* room for future expansion */ u32 pin_based_vm_exec_control; u32 cpu_based_vm_exec_control; u32 exception_bitmap; @@ -292,6 +298,12 @@ static inline void vmx_check_vmcs12_offsets(void) CHECK_OFFSET(host_ia32_sysenter_eip, 656); CHECK_OFFSET(host_rsp, 664); CHECK_OFFSET(host_rip, 672); + CHECK_OFFSET(host_s_cet, 680); + CHECK_OFFSET(host_ssp, 688); + CHECK_OFFSET(host_ssp_tbl, 696); + CHECK_OFFSET(guest_s_cet, 704); + CHECK_OFFSET(guest_ssp, 712); + CHECK_OFFSET(guest_ssp_tbl, 720); CHECK_OFFSET(pin_based_vm_exec_control, 744); CHECK_OFFSET(cpu_based_vm_exec_control, 748); CHECK_OFFSET(exception_bitmap, 752); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index ef7aca954228..5e5ec8b0223b 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7731,6 +7731,8 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) cr4_fixed1_update(X86_CR4_PKE, ecx, feature_bit(PKU)); cr4_fixed1_update(X86_CR4_UMIP, ecx, feature_bit(UMIP)); cr4_fixed1_update(X86_CR4_LA57, ecx, feature_bit(LA57)); + cr4_fixed1_update(X86_CR4_CET, ecx, feature_bit(SHSTK)); + cr4_fixed1_update(X86_CR4_CET, edx, feature_bit(IBT)); entry = kvm_find_cpuid_entry_index(vcpu, 0x7, 1); cr4_fixed1_update(X86_CR4_LAM_SUP, eax, feature_bit(LAM)); From patchwork Wed Jan 24 02:42:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13528443 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A60842C69B; Wed, 24 Jan 2024 02:42:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.120 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064173; cv=none; b=APp/7gek0pGzFMcOUhZ5P67vqSzYDPYcb59y9BMBljM2AQclc0fz+wtq0aRmPSi9U/o/YEjumTxxmsZfkyhsh1vx9YMEQF7uPPXsDM8lQnuJ5BQ6SKTDPZd65Acet+fY37iwtzy49ip6eSOp0PUhAMaRH0Kw4c6cuCY1Q8ejC+g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706064173; c=relaxed/simple; bh=UwRtvVNhqC9a4CUwgibiSghyasJhQAcBvz9o6nhR8wU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=naZyBYLth+/LAvOk3HS71kHwHwT6eRbJua3cV7pLdpFx9wAlflt+PWt7Tkg+e2Lvz8mfgCQRIxv6xTKDhyKmDAYrNo9AdbYwY8LgZYbFU/S2wBbrvEekORl2YxRrl2anL+I5dYyI9BH/fJ8wC8n/K1UosPZ9K5z+CwRxSJe5EZ8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=CSXHSVNq; arc=none smtp.client-ip=192.55.52.120 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="CSXHSVNq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706064171; x=1737600171; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UwRtvVNhqC9a4CUwgibiSghyasJhQAcBvz9o6nhR8wU=; b=CSXHSVNq7Y5/fnoT+ZxQSv2cOsDsoqnXwePOoYu9WpBy7fi57rkZ34oE 0V6bN/J3Q8TlJVo1AHK7YWfkLN1Noyz3tetksL6TuqSJVEIUozZO8ZR2c o90teJ8/jf0VGzJc6SwVSi+vlHE6voEUDIPZ8B0FR8N4Fm3C2KK5CVO+N +3P0HNHzPHt6dxchYz0vsCx9pwUAoq5T8CI50Ejb9Q5JwK446HiRXjkEN 9+nI/Oz6DeYn/mnN87n7fKD6QDXeaE7RohILVm9KPvLVdv5XE8gaGgKQX xxaOBBK0QuSzDXVRhto/a8Udz6O/j9wpIQI7jiYkgF4R2gUxKY2V+yo7a w==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="400586599" X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="400586599" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,215,1701158400"; d="scan'208";a="1825939" Received: from 984fee00a5ca.jf.intel.com ([10.165.9.183]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 18:42:45 -0800 From: Yang Weijiang To: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org, yuan.yao@linux.intel.com Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, john.allen@amd.com, weijiang.yang@intel.com Subject: [PATCH v9 27/27] KVM: x86: Stop emulating for CET protected branch instructions Date: Tue, 23 Jan 2024 18:42:00 -0800 Message-Id: <20240124024200.102792-28-weijiang.yang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240124024200.102792-1-weijiang.yang@intel.com> References: <20240124024200.102792-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Don't emulate the branch instructions, e.g., CALL/RET/JMP etc., when CET is active in guest, return KVM_INTERNAL_ERROR_EMULATION to userspace to handle it. KVM doesn't emulate CPU behaviors to check CET protected stuffs while emulating guest instructions, instead it stops emulation on detecting the instructions in process are CET protected. By doing so, it can avoid generating bogus #CP in guest and preventing CET protected execution flow subversion from guest side. Suggested-by: Chao Gao Signed-off-by: Yang Weijiang --- arch/x86/kvm/emulate.c | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c index e223043ef5b2..ad15ce055a1d 100644 --- a/arch/x86/kvm/emulate.c +++ b/arch/x86/kvm/emulate.c @@ -178,6 +178,7 @@ #define IncSP ((u64)1 << 54) /* SP is incremented before ModRM calc */ #define TwoMemOp ((u64)1 << 55) /* Instruction has two memory operand */ #define IsBranch ((u64)1 << 56) /* Instruction is considered a branch. */ +#define IsProtected ((u64)1 << 57) /* Instruction is protected by CET. */ #define DstXacc (DstAccLo | SrcAccHi | SrcWrite) @@ -4098,9 +4099,9 @@ static const struct opcode group4[] = { static const struct opcode group5[] = { F(DstMem | SrcNone | Lock, em_inc), F(DstMem | SrcNone | Lock, em_dec), - I(SrcMem | NearBranch | IsBranch, em_call_near_abs), - I(SrcMemFAddr | ImplicitOps | IsBranch, em_call_far), - I(SrcMem | NearBranch | IsBranch, em_jmp_abs), + I(SrcMem | NearBranch | IsBranch | IsProtected, em_call_near_abs), + I(SrcMemFAddr | ImplicitOps | IsBranch | IsProtected, em_call_far), + I(SrcMem | NearBranch | IsBranch | IsProtected, em_jmp_abs), I(SrcMemFAddr | ImplicitOps | IsBranch, em_jmp_far), I(SrcMem | Stack | TwoMemOp, em_push), D(Undefined), }; @@ -4362,11 +4363,11 @@ static const struct opcode opcode_table[256] = { /* 0xC8 - 0xCF */ I(Stack | SrcImmU16 | Src2ImmByte | IsBranch, em_enter), I(Stack | IsBranch, em_leave), - I(ImplicitOps | SrcImmU16 | IsBranch, em_ret_far_imm), - I(ImplicitOps | IsBranch, em_ret_far), - D(ImplicitOps | IsBranch), DI(SrcImmByte | IsBranch, intn), + I(ImplicitOps | SrcImmU16 | IsBranch | IsProtected, em_ret_far_imm), + I(ImplicitOps | IsBranch | IsProtected, em_ret_far), + D(ImplicitOps | IsBranch), DI(SrcImmByte | IsBranch | IsProtected, intn), D(ImplicitOps | No64 | IsBranch), - II(ImplicitOps | IsBranch, em_iret, iret), + II(ImplicitOps | IsBranch | IsProtected, em_iret, iret), /* 0xD0 - 0xD7 */ G(Src2One | ByteOp, group2), G(Src2One, group2), G(Src2CL | ByteOp, group2), G(Src2CL, group2), @@ -4382,7 +4383,7 @@ static const struct opcode opcode_table[256] = { I2bvIP(SrcImmUByte | DstAcc, em_in, in, check_perm_in), I2bvIP(SrcAcc | DstImmUByte, em_out, out, check_perm_out), /* 0xE8 - 0xEF */ - I(SrcImm | NearBranch | IsBranch, em_call), + I(SrcImm | NearBranch | IsBranch | IsProtected, em_call), D(SrcImm | ImplicitOps | NearBranch | IsBranch), I(SrcImmFAddr | No64 | IsBranch, em_jmp_far), D(SrcImmByte | ImplicitOps | NearBranch | IsBranch), @@ -4401,7 +4402,7 @@ static const struct opcode opcode_table[256] = { static const struct opcode twobyte_table[256] = { /* 0x00 - 0x0F */ G(0, group6), GD(0, &group7), N, N, - N, I(ImplicitOps | EmulateOnUD | IsBranch, em_syscall), + N, I(ImplicitOps | EmulateOnUD | IsBranch | IsProtected, em_syscall), II(ImplicitOps | Priv, em_clts, clts), N, DI(ImplicitOps | Priv, invd), DI(ImplicitOps | Priv, wbinvd), N, N, N, D(ImplicitOps | ModRM | SrcMem | NoAccess), N, N, @@ -4432,8 +4433,8 @@ static const struct opcode twobyte_table[256] = { IIP(ImplicitOps, em_rdtsc, rdtsc, check_rdtsc), II(ImplicitOps | Priv, em_rdmsr, rdmsr), IIP(ImplicitOps, em_rdpmc, rdpmc, check_rdpmc), - I(ImplicitOps | EmulateOnUD | IsBranch, em_sysenter), - I(ImplicitOps | Priv | EmulateOnUD | IsBranch, em_sysexit), + I(ImplicitOps | EmulateOnUD | IsBranch | IsProtected, em_sysenter), + I(ImplicitOps | Priv | EmulateOnUD | IsBranch | IsProtected, em_sysexit), N, N, N, N, N, N, N, N, N, N, /* 0x40 - 0x4F */ @@ -4971,6 +4972,10 @@ int x86_decode_insn(struct x86_emulate_ctxt *ctxt, void *insn, int insn_len, int if (ctxt->d == 0) return EMULATION_FAILED; + if ((opcode.flags & IsProtected) && + (ctxt->ops->get_cr(ctxt, 4) & X86_CR4_CET)) + return EMULATION_FAILED; + ctxt->execute = opcode.u.execute; if (unlikely(emulation_type & EMULTYPE_TRAP_UD) &&