From patchwork Thu May 23 11:16:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13671640 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9416CC25B75 for ; Thu, 23 May 2024 11:17:03 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.728365.1133268 (Exim 4.92) (envelope-from ) id 1sA6RC-0006IY-G8; Thu, 23 May 2024 11:16:46 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 728365.1133268; Thu, 23 May 2024 11:16:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sA6RC-0006IR-BP; Thu, 23 May 2024 11:16:46 +0000 Received: by outflank-mailman (input) for mailman id 728365; Thu, 23 May 2024 11:16:45 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sA6RB-0006Hv-MS for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:16:45 +0000 Received: from mail-oi1-x230.google.com (mail-oi1-x230.google.com [2607:f8b0:4864:20::230]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ef20935b-18f5-11ef-b4bb-af5377834399; Thu, 23 May 2024 13:16:43 +0200 (CEST) Received: by mail-oi1-x230.google.com with SMTP id 5614622812f47-3c9b94951cfso2985814b6e.3 for ; Thu, 23 May 2024 04:16:43 -0700 (PDT) Received: from andrew-laptop.citrite.net ([217.156.233.157]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6a15f179cebsm142194236d6.3.2024.05.23.04.16.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 May 2024 04:16:38 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ef20935b-18f5-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1716463001; x=1717067801; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qvBv5IQ5vD8zkhmMyJHwoIHHhy8iEs5huUmFgDlUZVI=; b=rSrKe+EKldLoGvjAnhCSGhhdGbsSAqV1ug0KxVy3zGMMuN5+/t5jNCVUioLyatdXR6 OYwQ5rzBPaishsNbA5V/2EMhCMk3DBHSgIX1dxd/fBfp62zqa5ATzKf5ZDDckpFxVnx3 /a2BKa0q/gleXrpKx0v/cWdfdwSdKbcs71Xf4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716463001; x=1717067801; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qvBv5IQ5vD8zkhmMyJHwoIHHhy8iEs5huUmFgDlUZVI=; b=a8zFy3E1F55SBjROkQ96m5fBrqM/M5cR/W5xx/Iy8NbvVXwvi2VtZWiYvgWlEzzhNq sFbOKdUa+f2cXw9oJTktX/N9RMebVy3BoSUq2iBvlxtiKLz7IrvhV8IEGXfoDJhPPLBJ isT40byH4rFaurRV8NewnCdHplgy/T8/mtFs8n8GhNJac7IUj8lBmj2yw6asCLsiDh0A Ua0tU3QcHJPFVefyic9k4FZfThxuIhByEVzBh/inc5DPp6nfcUTlwGd2exOC5fT1LJms vGPv8UXIxbs+aRdsTN6L6AN5tsrQu88fwr//CTLa/xEG9x/pU7e3B4cRyioDSy1DKZMt WPgg== X-Gm-Message-State: AOJu0YzaRfjrq8fgYFBeP9UBcbE7AAus279Mgq7qMdRR6HEaxhGrPz3W XhgFtqZ6lBV6fkFyqfd2NYrB/Zv+hQSC+mTP2d77/Lr9XRfldQELZxmD8V/kjieJ2tgD/wTmfJx Ik0w= X-Google-Smtp-Source: AGHT+IFI+Z81iHkbwpld0dC+maE7dJsscshL4XYvKQRclSQAfFiiFIChxlLBh7SVyOT3zLXjBG48/A== X-Received: by 2002:a05:6870:818b:b0:23b:339d:6fbf with SMTP id 586e51a60fabf-24c68ada91bmr5712972fac.5.1716462999867; Thu, 23 May 2024 04:16:39 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH 1/7] x86/xstate: Fix initialisation of XSS cache Date: Thu, 23 May 2024 12:16:21 +0100 Message-Id: <20240523111627.28896-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240523111627.28896-1-andrew.cooper3@citrix.com> References: <20240523111627.28896-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 The clobbering of this_cpu(xcr0) and this_cpu(xss) to architecturally invalid values is to force the subsequent set_xcr0() and set_msr_xss() to reload the hardware register. While XCR0 is reloaded in xstate_init(), MSR_XSS isn't. This causes get_msr_xss() to return the invalid value, and logic of the form: old = get_msr_xss(); set_msr_xss(new); ... set_msr_xss(old); to try and restore the architecturally invalid value. The architecturally invalid value must be purged from the cache, meaning the hardware register must be written at least once. This in turn highlights that the invalid value must only be used in the case that the hardware register is available. Fixes: f7f4a523927f ("x86/xstate: reset cached register values on resume") Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné v3: * Split out of later patch --- xen/arch/x86/xstate.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c index 99cedb4f5e24..75788147966a 100644 --- a/xen/arch/x86/xstate.c +++ b/xen/arch/x86/xstate.c @@ -641,13 +641,6 @@ void xstate_init(struct cpuinfo_x86 *c) return; } - /* - * Zap the cached values to make set_xcr0() and set_msr_xss() really - * write it. - */ - this_cpu(xcr0) = 0; - this_cpu(xss) = ~0; - cpuid_count(XSTATE_CPUID, 0, &eax, &ebx, &ecx, &edx); feature_mask = (((u64)edx << 32) | eax) & XCNTXT_MASK; BUG_ON(!valid_xcr0(feature_mask)); @@ -657,8 +650,19 @@ void xstate_init(struct cpuinfo_x86 *c) * Set CR4_OSXSAVE and run "cpuid" to get xsave_cntxt_size. */ set_in_cr4(X86_CR4_OSXSAVE); + + /* + * Zap the cached values to make set_xcr0() and set_msr_xss() really write + * the hardware register. + */ + this_cpu(xcr0) = 0; if ( !set_xcr0(feature_mask) ) BUG(); + if ( cpu_has_xsaves ) + { + this_cpu(xss) = ~0; + set_msr_xss(0); + } if ( bsp ) { From patchwork Thu May 23 11:16:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13671641 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5D070C25B75 for ; Thu, 23 May 2024 11:17:06 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.728366.1133276 (Exim 4.92) (envelope-from ) id 1sA6RD-0006Xd-LJ; Thu, 23 May 2024 11:16:47 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 728366.1133276; Thu, 23 May 2024 11:16:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sA6RD-0006XT-Ih; Thu, 23 May 2024 11:16:47 +0000 Received: by outflank-mailman (input) for mailman id 728366; Thu, 23 May 2024 11:16:46 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sA6RC-0006Hv-Np for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:16:46 +0000 Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com [2607:f8b0:4864:20::82d]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id efcc8158-18f5-11ef-b4bb-af5377834399; Thu, 23 May 2024 13:16:44 +0200 (CEST) Received: by mail-qt1-x82d.google.com with SMTP id d75a77b69052e-43fa6d92b4eso5467811cf.1 for ; Thu, 23 May 2024 04:16:44 -0700 (PDT) Received: from andrew-laptop.citrite.net ([217.156.233.157]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6a15f179cebsm142194236d6.3.2024.05.23.04.16.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 May 2024 04:16:40 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: efcc8158-18f5-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1716463002; x=1717067802; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9lVBemlknG/SLrNwwy57y6cT2xHvDd19zUouoa8Th7Q=; b=iqXVxyiQOw5wrq2w+PQ4Yk93emKEY8leXPK8mDy01Erwpcg0Xpgp2mdZND+sULPKI3 060NcvmB8YLDAx+vqxNJ3+L74tl8U+b44GXyA9iSDWYt159iAO+zj2NqxzGdZlFdGDPu PCcPaBTQ1sXinPZh4mA76NT8tBaP1wFuhJu+4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716463002; x=1717067802; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9lVBemlknG/SLrNwwy57y6cT2xHvDd19zUouoa8Th7Q=; b=h6bCZlmgdAU8VixVMRbdQxMPwStMBygcHHR7PcVU49tnaOyXLmnDT5u5ZTCfaL84nQ f8CWlDZHA9QBQc7LFhZqPfCpMgc+1ey2PP+ag1VbfxObbziPkTwxBqtiTTG6RPKTm5AG ZyfE6+p19K9JzEA12c36hqTj8OjBD5WbqLNnyTRJTxvA3qaklZR94ZRNEaaLI6CD2U5i /d2ekiQ2KOCrnD25Y66rqAAf4BR8I2R9vTa+w7UUPhF7nX6BtZn64QX82ICkb8m8RYPx SZYUms6goQwb4eUyKsInjsNyvLpn657UeVRqHc0SY/i4tddXYqvdFyhLcdiY862T8Ca7 17+g== X-Gm-Message-State: AOJu0YwZ1C1JulkUTcwLWHx2JsIBsU45JFsUJoZ2DK2HnafVT2Xxirki 5WRo76IeoTAKUCh2gZq1MAiZDVLqbZc2kiTfXHVv4rUqHD8b82I4f74mAZdYQizXalrCbjTg52J B35E= X-Google-Smtp-Source: AGHT+IH8NEooV914qM081ZP9jeUslFGVNkKno/knvhMmAz3WHl1hGmGhcv0lu/itA9GoeLdPVcJ3Qw== X-Received: by 2002:a05:622a:164e:b0:43b:173e:ce1b with SMTP id d75a77b69052e-43fa74df0a0mr31155171cf.32.1716463001887; Thu, 23 May 2024 04:16:41 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH 2/7] x86/xstate: Cross-check dynamic XSTATE sizes at boot Date: Thu, 23 May 2024 12:16:22 +0100 Message-Id: <20240523111627.28896-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240523111627.28896-1-andrew.cooper3@citrix.com> References: <20240523111627.28896-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Right now, xstate_ctxt_size() performs a cross-check of size with CPUID in for every call. This is expensive, being used for domain create/migrate, as well as to service certain guest CPUID instructions. Instead, arrange to check the sizes once at boot. See the code comments for details. Right now, it just checks hardware against the algorithm expectations. Later patches will add further cross-checking. Introduce the missing X86_XCR0_* and X86_XSS_* constants, and a couple of missing CPUID bits. This is to maximise coverage in the sanity check, even if we don't expect to use/virtualise some of these features any time soon. Leave HDC and HWP alone for now. We don't have CPUID bits from them stored nicely. Only perform the cross-checks in debug builds. It's only developers or new hardware liable to trip these checks, and Xen at least tracks "maximum value ever seen in xcr0" for the lifetime of the VM, which we don't want to be tickling in the general case. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné v3: * New On Sapphire Rapids with the whole series inc diagnostics, we get this pattern: (XEN) *** check_new_xstate(, 0x00000003) (XEN) *** check_new_xstate(, 0x00000004) (XEN) *** check_new_xstate(, 0x000000e0) (XEN) *** check_new_xstate(, 0x00000200) (XEN) *** check_new_xstate(, 0x00060000) (XEN) *** check_new_xstate(, 0x00000100) (XEN) *** check_new_xstate(, 0x00000400) (XEN) *** check_new_xstate(, 0x00000800) (XEN) *** check_new_xstate(, 0x00001000) (XEN) *** check_new_xstate(, 0x00004000) (XEN) *** check_new_xstate(, 0x00008000) and on Genoa, this pattern: (XEN) *** check_new_xstate(, 0x00000003) (XEN) *** check_new_xstate(, 0x00000004) (XEN) *** check_new_xstate(, 0x000000e0) (XEN) *** check_new_xstate(, 0x00000200) (XEN) *** check_new_xstate(, 0x00000800) (XEN) *** check_new_xstate(, 0x00001000) --- xen/arch/x86/include/asm/x86-defns.h | 25 +++- xen/arch/x86/xstate.c | 150 ++++++++++++++++++++ xen/include/public/arch-x86/cpufeatureset.h | 3 + 3 files changed, 177 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/include/asm/x86-defns.h b/xen/arch/x86/include/asm/x86-defns.h index 48d7a3b7af45..d7602ab225c4 100644 --- a/xen/arch/x86/include/asm/x86-defns.h +++ b/xen/arch/x86/include/asm/x86-defns.h @@ -77,7 +77,7 @@ #define X86_CR4_PKS 0x01000000 /* Protection Key Supervisor */ /* - * XSTATE component flags in XCR0 + * XSTATE component flags in XCR0 | MSR_XSS */ #define X86_XCR0_FP_POS 0 #define X86_XCR0_FP (1ULL << X86_XCR0_FP_POS) @@ -95,11 +95,34 @@ #define X86_XCR0_ZMM (1ULL << X86_XCR0_ZMM_POS) #define X86_XCR0_HI_ZMM_POS 7 #define X86_XCR0_HI_ZMM (1ULL << X86_XCR0_HI_ZMM_POS) +#define X86_XSS_PROC_TRACE (_AC(1, ULL) << 8) #define X86_XCR0_PKRU_POS 9 #define X86_XCR0_PKRU (1ULL << X86_XCR0_PKRU_POS) +#define X86_XSS_PASID (_AC(1, ULL) << 10) +#define X86_XSS_CET_U (_AC(1, ULL) << 11) +#define X86_XSS_CET_S (_AC(1, ULL) << 12) +#define X86_XSS_HDC (_AC(1, ULL) << 13) +#define X86_XSS_UINTR (_AC(1, ULL) << 14) +#define X86_XSS_LBR (_AC(1, ULL) << 15) +#define X86_XSS_HWP (_AC(1, ULL) << 16) +#define X86_XCR0_TILE_CFG (_AC(1, ULL) << 17) +#define X86_XCR0_TILE_DATA (_AC(1, ULL) << 18) #define X86_XCR0_LWP_POS 62 #define X86_XCR0_LWP (1ULL << X86_XCR0_LWP_POS) +#define X86_XCR0_STATES \ + (X86_XCR0_FP | X86_XCR0_SSE | X86_XCR0_YMM | X86_XCR0_BNDREGS | \ + X86_XCR0_BNDCSR | X86_XCR0_OPMASK | X86_XCR0_ZMM | \ + X86_XCR0_HI_ZMM | X86_XCR0_PKRU | X86_XCR0_TILE_CFG | \ + X86_XCR0_TILE_DATA | \ + X86_XCR0_LWP) + +#define X86_XSS_STATES \ + (X86_XSS_PROC_TRACE | X86_XSS_PASID | X86_XSS_CET_U | \ + X86_XSS_CET_S | X86_XSS_HDC | X86_XSS_UINTR | X86_XSS_LBR | \ + X86_XSS_HWP | \ + 0) + /* * Debug status flags in DR6. * diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c index 75788147966a..33a5a89719ef 100644 --- a/xen/arch/x86/xstate.c +++ b/xen/arch/x86/xstate.c @@ -604,9 +604,156 @@ static bool valid_xcr0(uint64_t xcr0) if ( !(xcr0 & X86_XCR0_BNDREGS) != !(xcr0 & X86_XCR0_BNDCSR) ) return false; + /* TILE_CFG and TILE_DATA must be the same. */ + if ( !(xcr0 & X86_XCR0_TILE_CFG) != !(xcr0 & X86_XCR0_TILE_DATA) ) + return false; + return true; } +struct xcheck_state { + uint64_t states; + uint32_t uncomp_size; + uint32_t comp_size; +}; + +static void __init check_new_xstate(struct xcheck_state *s, uint64_t new) +{ + uint32_t hw_size; + + BUILD_BUG_ON(X86_XCR0_STATES & X86_XSS_STATES); + + BUG_ON(s->states & new); /* States only increase. */ + BUG_ON(!valid_xcr0(s->states | new)); /* Xen thinks it's a good value. */ + BUG_ON(new & ~(X86_XCR0_STATES | X86_XSS_STATES)); /* Known state. */ + BUG_ON((new & X86_XCR0_STATES) && + (new & X86_XSS_STATES)); /* User or supervisor, not both. */ + + s->states |= new; + if ( new & X86_XCR0_STATES ) + { + if ( !set_xcr0(s->states & X86_XCR0_STATES) ) + BUG(); + } + else + set_msr_xss(s->states & X86_XSS_STATES); + + /* + * Check the uncompressed size. Some XSTATEs are out-of-order and fill in + * prior holes in the state area, so we check that the size doesn't + * decrease. + */ + hw_size = cpuid_count_ebx(0xd, 0); + + if ( hw_size < s->uncomp_size ) + panic("XSTATE 0x%016"PRIx64", new bits {%63pbl}, uncompressed hw size %#x < prev size %#x\n", + s->states, &new, hw_size, s->uncomp_size); + + s->uncomp_size = hw_size; + + /* + * Check the compressed size, if available. All components strictly + * appear in index order. In principle there are no holes, but some + * components have their base address 64-byte aligned for efficiency + * reasons (e.g. AMX-TILE) and there are other components small enough to + * fit in the gap (e.g. PKRU) without increasing the overall length. + */ + hw_size = cpuid_count_ebx(0xd, 1); + + if ( cpu_has_xsavec ) + { + if ( hw_size < s->comp_size ) + panic("XSTATE 0x%016"PRIx64", new bits {%63pbl}, compressed hw size %#x < prev size %#x\n", + s->states, &new, hw_size, s->comp_size); + + s->comp_size = hw_size; + } + else + BUG_ON(hw_size); /* Compressed size reported, but no XSAVEC ? */ +} + +/* + * The {un,}compressed XSTATE sizes are reported by dynamic CPUID value, based + * on the current %XCR0 and MSR_XSS values. The exact layout is also feature + * and vendor specific. Cross-check Xen's understanding against real hardware + * on boot. + * + * Testing every combination is prohibitive, so we use a partial approach. + * Starting with nothing active, we add new XSTATEs and check that the CPUID + * dynamic values never decreases. + */ +static void __init noinline xstate_check_sizes(void) +{ + uint64_t old_xcr0 = get_xcr0(); + uint64_t old_xss = get_msr_xss(); + struct xcheck_state s = {}; + + /* + * User XSTATEs, increasing by index. + * + * Chronologically, Intel and AMD had identical layouts for AVX (YMM). + * AMD introduced LWP in Fam15h, following immediately on from YMM. Intel + * left an LWP-shaped hole when adding MPX (BND{CSR,REGS}) in Skylake. + * AMD removed LWP in Fam17h, putting PKRU in the same space, breaking + * layout compatibility with Intel and having a knock-on effect on all + * subsequent states. + */ + check_new_xstate(&s, X86_XCR0_SSE | X86_XCR0_FP); + + if ( cpu_has_avx ) + check_new_xstate(&s, X86_XCR0_YMM); + + if ( cpu_has_mpx ) + check_new_xstate(&s, X86_XCR0_BNDCSR | X86_XCR0_BNDREGS); + + if ( cpu_has_avx512f ) + check_new_xstate(&s, X86_XCR0_HI_ZMM | X86_XCR0_ZMM | X86_XCR0_OPMASK); + + if ( cpu_has_pku ) + check_new_xstate(&s, X86_XCR0_PKRU); + + if ( boot_cpu_has(X86_FEATURE_AMX_TILE) ) + check_new_xstate(&s, X86_XCR0_TILE_DATA | X86_XCR0_TILE_CFG); + + if ( boot_cpu_has(X86_FEATURE_LWP) ) + check_new_xstate(&s, X86_XCR0_LWP); + + /* + * Supervisor XSTATEs, increasing by index. + * + * Intel Broadwell in particular had Processor Trace but no XSAVES. There + * doesn't appear to have been a new enumeration when X86_XSS_PROC_TRACE + * was introduced in Skylake. + */ + if ( cpu_has_xsaves ) + { + if ( cpu_has_proc_trace ) + check_new_xstate(&s, X86_XSS_PROC_TRACE); + + if ( boot_cpu_has(X86_FEATURE_ENQCMD) ) + check_new_xstate(&s, X86_XSS_PASID); + + if ( boot_cpu_has(X86_FEATURE_CET_SS) || + boot_cpu_has(X86_FEATURE_CET_IBT) ) + { + check_new_xstate(&s, X86_XSS_CET_U); + check_new_xstate(&s, X86_XSS_CET_S); + } + + if ( boot_cpu_has(X86_FEATURE_UINTR) ) + check_new_xstate(&s, X86_XSS_UINTR); + + if ( boot_cpu_has(X86_FEATURE_ARCH_LBR) ) + check_new_xstate(&s, X86_XSS_LBR); + } + + /* Restore old state now the test is done. */ + if ( !set_xcr0(old_xcr0) ) + BUG(); + if ( cpu_has_xsaves ) + set_msr_xss(old_xss); +} + /* Collect the information of processor's extended state */ void xstate_init(struct cpuinfo_x86 *c) { @@ -683,6 +830,9 @@ void xstate_init(struct cpuinfo_x86 *c) if ( setup_xstate_features(bsp) && bsp ) BUG(); + + if ( IS_ENABLED(CONFIG_DEBUG) && bsp ) + xstate_check_sizes(); } int validate_xstate(const struct domain *d, uint64_t xcr0, uint64_t xcr0_accum, diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h index 6627453e3985..d9eba5e9a714 100644 --- a/xen/include/public/arch-x86/cpufeatureset.h +++ b/xen/include/public/arch-x86/cpufeatureset.h @@ -266,6 +266,7 @@ XEN_CPUFEATURE(IBPB_RET, 8*32+30) /*A IBPB clears RSB/RAS too. */ XEN_CPUFEATURE(AVX512_4VNNIW, 9*32+ 2) /*A AVX512 Neural Network Instructions */ XEN_CPUFEATURE(AVX512_4FMAPS, 9*32+ 3) /*A AVX512 Multiply Accumulation Single Precision */ XEN_CPUFEATURE(FSRM, 9*32+ 4) /*A Fast Short REP MOVS */ +XEN_CPUFEATURE(UINTR, 9*32+ 5) /* User-mode Interrupts */ XEN_CPUFEATURE(AVX512_VP2INTERSECT, 9*32+8) /*a VP2INTERSECT{D,Q} insns */ XEN_CPUFEATURE(SRBDS_CTRL, 9*32+ 9) /* MSR_MCU_OPT_CTRL and RNGDS_MITG_DIS. */ XEN_CPUFEATURE(MD_CLEAR, 9*32+10) /*!A| VERW clears microarchitectural buffers */ @@ -274,8 +275,10 @@ XEN_CPUFEATURE(TSX_FORCE_ABORT, 9*32+13) /* MSR_TSX_FORCE_ABORT.RTM_ABORT */ XEN_CPUFEATURE(SERIALIZE, 9*32+14) /*A SERIALIZE insn */ XEN_CPUFEATURE(HYBRID, 9*32+15) /* Heterogeneous platform */ XEN_CPUFEATURE(TSXLDTRK, 9*32+16) /*a TSX load tracking suspend/resume insns */ +XEN_CPUFEATURE(ARCH_LBR, 9*32+19) /* Architectural Last Branch Record */ XEN_CPUFEATURE(CET_IBT, 9*32+20) /* CET - Indirect Branch Tracking */ XEN_CPUFEATURE(AVX512_FP16, 9*32+23) /*A AVX512 FP16 instructions */ +XEN_CPUFEATURE(AMX_TILE, 9*32+24) /* AMX Tile architecture */ XEN_CPUFEATURE(IBRSB, 9*32+26) /*A IBRS and IBPB support (used by Intel) */ XEN_CPUFEATURE(STIBP, 9*32+27) /*A STIBP */ XEN_CPUFEATURE(L1D_FLUSH, 9*32+28) /*S MSR_FLUSH_CMD and L1D flush. */ From patchwork Thu May 23 11:16:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13671642 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E7BAFC25B75 for ; Thu, 23 May 2024 11:17:09 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.728367.1133286 (Exim 4.92) (envelope-from ) id 1sA6RI-0006rX-2v; Thu, 23 May 2024 11:16:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 728367.1133286; Thu, 23 May 2024 11:16:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sA6RH-0006rE-Vg; Thu, 23 May 2024 11:16:51 +0000 Received: by outflank-mailman (input) for mailman id 728367; Thu, 23 May 2024 11:16:50 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sA6RG-0006Hv-CY for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:16:50 +0000 Received: from mail-oi1-x233.google.com (mail-oi1-x233.google.com [2607:f8b0:4864:20::233]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f22261f1-18f5-11ef-b4bb-af5377834399; Thu, 23 May 2024 13:16:48 +0200 (CEST) Received: by mail-oi1-x233.google.com with SMTP id 5614622812f47-3cabac56b38so1546647b6e.3 for ; Thu, 23 May 2024 04:16:48 -0700 (PDT) Received: from andrew-laptop.citrite.net ([217.156.233.157]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6a15f179cebsm142194236d6.3.2024.05.23.04.16.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 May 2024 04:16:42 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f22261f1-18f5-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1716463006; x=1717067806; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3brSZl6/vV2dpGWkfgvqrJmcT56Fa2D0F6BOtDAqDOI=; b=LcahzV4fVqrQv+UWToDs4J/GQMy9S0YbbRRlhQpGlgUV/HhnnTGOMlN1Whrjju8QTW IgkiTIFaQpJBaA+1eFackREzUbuC162gDsO2ImJL0GEBGOlDVB/4/ok6oE5mrgJ5FOwh aSwq0Ed6u7iS/EFZ6r5dWe2BxDdhoBn6VNLzI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716463006; x=1717067806; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3brSZl6/vV2dpGWkfgvqrJmcT56Fa2D0F6BOtDAqDOI=; b=qYl9f9ft+clGR9+yRS9Sme1azyS731LsXSsTKkP9iTU9bXyq6YO+X5vqveVUH1Za/6 CqoyrScm6NLdo7sdTvAvkE2d6u36E42EhoX0qq92SC/NC5enAvqcKp6uujnZqFhei/lq gq6LWyauE2ZH3kkYyY6iDEOZPyBmwzrQmItu2OYkp4K5RCekfog/oF1bPwBA/b+nQZNa iIRefhgLS2ZxoMvacQuSBNymg95WD2mxJJ+ONIFuwo5bWgXy7mPruEXQVKgh9+waC9kV LGqQ2kV7xkWgOjc5xzMQxQTiymiMK1jR4HSxmExcnAHDeRFU5ocZc4F68twgVC3F+wVS NC2A== X-Gm-Message-State: AOJu0YxBnyQAmXYJ9z+dk3ozENG/tbBsHZYtNM9rNm8X8CHjz0xQvWuQ /5BdvNdiUC205LFaCWxKxUuWhc0BuEU4Cbqobq9PmUXqMbWyDqj36mQtYV+/8s98ASX0uwbI3Vs +QSo= X-Google-Smtp-Source: AGHT+IFpvJcWqnaHvo7hcFKKjX515sNn6S5bC6QmZNxfnQRU35FyPFQYvA+CWP5bK1L4FB1RMRQaEg== X-Received: by 2002:a54:4715:0:b0:3c9:6e96:88fb with SMTP id 5614622812f47-3cdb0c91a7bmr4613116b6e.4.1716463004868; Thu, 23 May 2024 04:16:44 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH 3/7] x86/boot: Collect the Raw CPU Policy earlier on boot Date: Thu, 23 May 2024 12:16:23 +0100 Message-Id: <20240523111627.28896-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240523111627.28896-1-andrew.cooper3@citrix.com> References: <20240523111627.28896-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 This is a tangle, but it's a small step in the right direction. xstate_init() is shortly going to want data from the Raw policy. calculate_raw_cpu_policy() is sufficiently separate from the other policies to be safe to do. No functional change. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné This is necessary for the forthcoming xstate_{un,}compressed_size() to perform boot-time sanity checks on state components which aren't fully enabled yet. I decided that doing this was better than extending the xstate_{offsets,sizes}[] logic that we're intending to retire in due course. v3: * New. --- xen/arch/x86/cpu-policy.c | 1 - xen/arch/x86/setup.c | 4 +++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c index b96f4ee55cc4..5b66f002df05 100644 --- a/xen/arch/x86/cpu-policy.c +++ b/xen/arch/x86/cpu-policy.c @@ -845,7 +845,6 @@ static void __init calculate_hvm_def_policy(void) void __init init_guest_cpu_policies(void) { - calculate_raw_cpu_policy(); calculate_host_policy(); if ( IS_ENABLED(CONFIG_PV) ) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index b50c9c84af6d..8850e5637a98 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1888,7 +1888,9 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p) tsx_init(); /* Needs microcode. May change HLE/RTM feature bits. */ - identify_cpu(&boot_cpu_data); + calculate_raw_cpu_policy(); /* Needs microcode. No other dependenices. */ + + identify_cpu(&boot_cpu_data); /* Needs microcode and raw policy. */ set_in_cr4(X86_CR4_OSFXSR | X86_CR4_OSXMMEXCPT); From patchwork Thu May 23 11:16:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13671643 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7949AC25B7C for ; Thu, 23 May 2024 11:17:12 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.728368.1133297 (Exim 4.92) (envelope-from ) id 1sA6RJ-00077g-D0; Thu, 23 May 2024 11:16:53 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 728368.1133297; Thu, 23 May 2024 11:16:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sA6RJ-00077W-8w; Thu, 23 May 2024 11:16:53 +0000 Received: by outflank-mailman (input) for mailman id 728368; Thu, 23 May 2024 11:16:51 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sA6RH-0006Hv-GC for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:16:51 +0000 Received: from mail-qv1-xf2d.google.com (mail-qv1-xf2d.google.com [2607:f8b0:4864:20::f2d]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f2c77944-18f5-11ef-b4bb-af5377834399; Thu, 23 May 2024 13:16:49 +0200 (CEST) Received: by mail-qv1-xf2d.google.com with SMTP id 6a1803df08f44-6a919a4bb83so23126766d6.1 for ; Thu, 23 May 2024 04:16:49 -0700 (PDT) Received: from andrew-laptop.citrite.net ([217.156.233.157]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6a15f179cebsm142194236d6.3.2024.05.23.04.16.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 May 2024 04:16:45 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f2c77944-18f5-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1716463007; x=1717067807; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Wwexe8CnptsNPO0v+knt+q1PXvcS9l92Rfmc3cBUT9k=; b=OUjOBJscARRhYbbgmXwvrGmTOrxewXq/TjEwLdeXNVFOFKzFDl5YAcumil08u5LVvR VT6JW/AZxBZyyJL6BJKosUCZM7mjrjXEU5hresS1GMATp0ffEjuw8y3K4EKLVt+3qQbG fw8U8RFiNpI4q8sqmoV27vuQl4jE6h3846hqU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716463007; x=1717067807; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Wwexe8CnptsNPO0v+knt+q1PXvcS9l92Rfmc3cBUT9k=; b=wTCaM5+MXCXcgG937V1ioB0TYrErFJ+ZPQWXbz0QyRFZNPaUwNtH/+ZBesnV5DvAFB bHgWlBAfCzq8eCUPeZgBqW+LYDOepJ/jktsUZjz23NxOpIC0pyPF+qOFhvpLwKkyzpPQ PeN3Zyk5/GhVO4j2LrfziYHS/v7ez92mj25FDy9N7cyAItsqNhv/p9KoH5Ez6Y78XVCl Cx0RADve1wuFJwIVK8B+fT4EhIbSFZNT8W8K5r+SfKtunjijkeHqtdrnOu4AYs9nW4+U WZDmRyV5TrB4P6rllnhNNpIuHB1V3kaA1aa12Hb+mgvjMdpcEGnvoHr8FybpNFFLff1k /p2A== X-Gm-Message-State: AOJu0Yw1xqyTpXXNO1dBPYsMd1jG+lIsX0H2gL7zQlSaVSaOVqogVffL OYaU5yqD/G50BDkk/XnRUdzSBFhLRmeq+xT+P5oi4lKZP3XCZHXcPXk3mxXSSEODiHBZ2V4Qp7n WZfE= X-Google-Smtp-Source: AGHT+IH0uVBwwjZDQEYD5tp1lGk2N/50w+jo3zrngapn0+tG/W/IPEGsiq58L5vBJimDo5vkz/U2BA== X-Received: by 2002:a05:6214:3c9e:b0:6aa:600e:23c with SMTP id 6a1803df08f44-6ab808f29e9mr55730046d6.43.1716463006986; Thu, 23 May 2024 04:16:46 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH 4/7] x86/xstate: Rework xstate_ctxt_size() as xstate_uncompressed_size() Date: Thu, 23 May 2024 12:16:24 +0100 Message-Id: <20240523111627.28896-5-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240523111627.28896-1-andrew.cooper3@citrix.com> References: <20240523111627.28896-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 We're soon going to need a compressed helper of the same form. The size of the uncompressed image depends on the single element with the largest offset + size. Sadly this isn't always the element with the largest index. Name the per-xstate-component cpu_policy struture, for legibility of the logic in xstate_uncompressed_size(). Cross-check with hardware during boot, and remove hw_uncompressed_size(). This means that the migration paths don't need to mess with XCR0 just to sanity check the buffer size. The users of hw_uncompressed_size() in xstate_init() can (and indeed need) to be replaced with CPUID instructions. They run with feature_mask in XCR0, and prior to setup_xstate_features() on the BSP. No practical change. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné v2: * Scan all features. LWP/APX_F are out-of-order. v3: * Rebase over boot time check. * Use the raw CPU policy. --- xen/arch/x86/domctl.c | 2 +- xen/arch/x86/hvm/hvm.c | 2 +- xen/arch/x86/include/asm/xstate.h | 2 +- xen/arch/x86/xstate.c | 78 +++++++++++++++++----------- xen/include/xen/lib/x86/cpu-policy.h | 2 +- 5 files changed, 51 insertions(+), 35 deletions(-) diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index 9a72d57333e9..c2f2016ed45a 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -833,7 +833,7 @@ long arch_do_domctl( uint32_t offset = 0; #define PV_XSAVE_HDR_SIZE (2 * sizeof(uint64_t)) -#define PV_XSAVE_SIZE(xcr0) (PV_XSAVE_HDR_SIZE + xstate_ctxt_size(xcr0)) +#define PV_XSAVE_SIZE(xcr0) (PV_XSAVE_HDR_SIZE + xstate_uncompressed_size(xcr0)) ret = -ESRCH; if ( (evc->vcpu >= d->max_vcpus) || diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index 2c66fe0f7a16..b84f4d2387d1 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -1190,7 +1190,7 @@ HVM_REGISTER_SAVE_RESTORE(CPU, hvm_save_cpu_ctxt, NULL, hvm_load_cpu_ctxt, 1, #define HVM_CPU_XSAVE_SIZE(xcr0) (offsetof(struct hvm_hw_cpu_xsave, \ save_area) + \ - xstate_ctxt_size(xcr0)) + xstate_uncompressed_size(xcr0)) static int cf_check hvm_save_cpu_xsave_states( struct vcpu *v, hvm_domain_context_t *h) diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/xstate.h index c08c267884f0..f5115199d4f9 100644 --- a/xen/arch/x86/include/asm/xstate.h +++ b/xen/arch/x86/include/asm/xstate.h @@ -107,7 +107,7 @@ void compress_xsave_states(struct vcpu *v, const void *src, unsigned int size); void xstate_free_save_area(struct vcpu *v); int xstate_alloc_save_area(struct vcpu *v); void xstate_init(struct cpuinfo_x86 *c); -unsigned int xstate_ctxt_size(u64 xcr0); +unsigned int xstate_uncompressed_size(uint64_t xcr0); static inline uint64_t xgetbv(unsigned int index) { diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c index 33a5a89719ef..1b3153600d9c 100644 --- a/xen/arch/x86/xstate.c +++ b/xen/arch/x86/xstate.c @@ -8,6 +8,8 @@ #include #include #include + +#include #include #include #include @@ -183,7 +185,7 @@ void expand_xsave_states(const struct vcpu *v, void *dest, unsigned int size) /* Check there is state to serialise (i.e. at least an XSAVE_HDR) */ BUG_ON(!v->arch.xcr0_accum); /* Check there is the correct room to decompress into. */ - BUG_ON(size != xstate_ctxt_size(v->arch.xcr0_accum)); + BUG_ON(size != xstate_uncompressed_size(v->arch.xcr0_accum)); if ( !(xstate->xsave_hdr.xcomp_bv & XSTATE_COMPACTION_ENABLED) ) { @@ -245,7 +247,7 @@ void compress_xsave_states(struct vcpu *v, const void *src, unsigned int size) u64 xstate_bv, valid; BUG_ON(!v->arch.xcr0_accum); - BUG_ON(size != xstate_ctxt_size(v->arch.xcr0_accum)); + BUG_ON(size != xstate_uncompressed_size(v->arch.xcr0_accum)); ASSERT(!xsave_area_compressed(src)); xstate_bv = ((const struct xsave_struct *)src)->xsave_hdr.xstate_bv; @@ -553,32 +555,6 @@ void xstate_free_save_area(struct vcpu *v) v->arch.xsave_area = NULL; } -static unsigned int hw_uncompressed_size(uint64_t xcr0) -{ - u64 act_xcr0 = get_xcr0(); - unsigned int size; - bool ok = set_xcr0(xcr0); - - ASSERT(ok); - size = cpuid_count_ebx(XSTATE_CPUID, 0); - ok = set_xcr0(act_xcr0); - ASSERT(ok); - - return size; -} - -/* Fastpath for common xstate size requests, avoiding reloads of xcr0. */ -unsigned int xstate_ctxt_size(u64 xcr0) -{ - if ( xcr0 == xfeature_mask ) - return xsave_cntxt_size; - - if ( xcr0 == 0 ) /* TODO: clean up paths passing 0 in here. */ - return 0; - - return hw_uncompressed_size(xcr0); -} - static bool valid_xcr0(uint64_t xcr0) { /* FP must be unconditionally set. */ @@ -611,6 +587,40 @@ static bool valid_xcr0(uint64_t xcr0) return true; } +unsigned int xstate_uncompressed_size(uint64_t xcr0) +{ + unsigned int size = XSTATE_AREA_MIN_SIZE, i; + + ASSERT((xcr0 & ~X86_XCR0_STATES) == 0); + + if ( xcr0 == xfeature_mask ) + return xsave_cntxt_size; + + if ( xcr0 == 0 ) /* TODO: clean up paths passing 0 in here. */ + return 0; + + if ( xcr0 <= (X86_XCR0_SSE | X86_XCR0_FP) ) + return size; + + /* + * For the non-legacy states, search all activate states and find the + * maximum offset+size. Some states (e.g. LWP, APX_F) are out-of-order + * with respect their index. + */ + xcr0 &= ~(X86_XCR0_SSE | X86_XCR0_FP); + for_each_set_bit ( i, &xcr0, 63 ) + { + const struct xstate_component *c = &raw_cpu_policy.xstate.comp[i]; + unsigned int s = c->offset + c->size; + + ASSERT(c->offset && c->size); + + size = max(size, s); + } + + return size; +} + struct xcheck_state { uint64_t states; uint32_t uncomp_size; @@ -619,7 +629,7 @@ struct xcheck_state { static void __init check_new_xstate(struct xcheck_state *s, uint64_t new) { - uint32_t hw_size; + uint32_t hw_size, xen_size; BUILD_BUG_ON(X86_XCR0_STATES & X86_XSS_STATES); @@ -651,6 +661,12 @@ static void __init check_new_xstate(struct xcheck_state *s, uint64_t new) s->uncomp_size = hw_size; + xen_size = xstate_uncompressed_size(s->states & X86_XCR0_STATES); + + if ( xen_size != hw_size ) + panic("XSTATE 0x%016"PRIx64", uncompressed hw size %#x != xen size %#x\n", + s->states, hw_size, xen_size); + /* * Check the compressed size, if available. All components strictly * appear in index order. In principle there are no holes, but some @@ -818,14 +834,14 @@ void xstate_init(struct cpuinfo_x86 *c) * xsave_cntxt_size is the max size required by enabled features. * We know FP/SSE and YMM about eax, and nothing about edx at present. */ - xsave_cntxt_size = hw_uncompressed_size(feature_mask); + xsave_cntxt_size = cpuid_count_ebx(0xd, 0); printk("xstate: size: %#x and states: %#"PRIx64"\n", xsave_cntxt_size, xfeature_mask); } else { BUG_ON(xfeature_mask != feature_mask); - BUG_ON(xsave_cntxt_size != hw_uncompressed_size(feature_mask)); + BUG_ON(xsave_cntxt_size != cpuid_count_ebx(0xd, 0)); } if ( setup_xstate_features(bsp) && bsp ) diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h index d5e447e9dc06..d26012c6da78 100644 --- a/xen/include/xen/lib/x86/cpu-policy.h +++ b/xen/include/xen/lib/x86/cpu-policy.h @@ -248,7 +248,7 @@ struct cpu_policy }; /* Per-component common state. Valid for i >= 2. */ - struct { + struct xstate_component { uint32_t size, offset; bool xss:1, align:1; uint32_t _res_d; From patchwork Thu May 23 11:16:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13671644 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D2E58C25B75 for ; Thu, 23 May 2024 11:17:13 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.728369.1133306 (Exim 4.92) (envelope-from ) id 1sA6RL-0007QQ-JS; Thu, 23 May 2024 11:16:55 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 728369.1133306; Thu, 23 May 2024 11:16:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sA6RL-0007QF-Gm; Thu, 23 May 2024 11:16:55 +0000 Received: by outflank-mailman (input) for mailman id 728369; Thu, 23 May 2024 11:16:54 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sA6RK-0006Hv-Hz for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:16:54 +0000 Received: from mail-qv1-xf2d.google.com (mail-qv1-xf2d.google.com [2607:f8b0:4864:20::f2d]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f4b33630-18f5-11ef-b4bb-af5377834399; Thu, 23 May 2024 13:16:52 +0200 (CEST) Received: by mail-qv1-xf2d.google.com with SMTP id 6a1803df08f44-69b4454e2f1so8701066d6.0 for ; Thu, 23 May 2024 04:16:52 -0700 (PDT) Received: from andrew-laptop.citrite.net ([217.156.233.157]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6a15f179cebsm142194236d6.3.2024.05.23.04.16.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 May 2024 04:16:48 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f4b33630-18f5-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1716463011; x=1717067811; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LRpbv1/wBcaMA0cTk1bCZyxrSAb4NeFKdZW8+eY/F+o=; b=pWUOmSIQAhLXWjJFvUvygq1VCgRvXfRv6ZyMBWU5AU3e8GPy5rSpSKYjo1caH+jl19 NqtgitM443GhHCMLS0X/1PQEoXSzyvQQ6LZTIQt6rLnxNQ1Hqhz+tkuqZDBfxBjsrJcx AcTX1sDuQ81tyxB8IUMNytv9fnoQQaqXkaMFQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716463011; x=1717067811; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LRpbv1/wBcaMA0cTk1bCZyxrSAb4NeFKdZW8+eY/F+o=; b=ESQDGgORVouPWnmPlHMS66g1kRN656Fpn7e/Sg9UTgM8QgTWqsXDxUZdgkQTeeiLan TMTjRK5ZKq0zCMuIR3UsmrI2wi75+XO812F+ePYChqOA1hLLPSr1vGHuyWGHbwmt1dfT 05JBujkJFq+ovJxwSj7+P09MQYDLIz7rc3T+Gid9RgaOa97clCi+P1ou7NSpQOs9+Slg FrHzGugwTPxqw9FafMhaXevoyYIhYHAv9maJcQ0p8aCPihEmUDJAOCnRZes6S5Kuy3UC zNmRpq1UjyNUcp1CEv0HShsFHxSJZElnQ7Gfrc3ZtZKTf1M08Db3XhTUiPOvLIyJMuYm 1UOQ== X-Gm-Message-State: AOJu0Yywg8xYhwr6DtPj7grBJRudEhn9v+QxUTMqPIAwE4uPEPSWUfnR tLbSeKOwAJddHhoJNOtGGJ1QKZOI0Q6ChnuVE/g7ATMkymnjLQT3vn65NbrYe5xkXOKQHbcIvUG yA0s= X-Google-Smtp-Source: AGHT+IEYLtrmNJ4sPHn6mziSkxTZt6d8QJUEhwLHRsESbQZLZsTWtZt7iKeF6iJXaGGTuiC6cwAxrg== X-Received: by 2002:a05:6214:5c49:b0:6ab:6f15:515e with SMTP id 6a1803df08f44-6ab7f3398f7mr48834776d6.13.1716463010789; Thu, 23 May 2024 04:16:50 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH 5/7] x86/cpu-policy: Simplify recalculate_xstate() Date: Thu, 23 May 2024 12:16:25 +0100 Message-Id: <20240523111627.28896-6-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240523111627.28896-1-andrew.cooper3@citrix.com> References: <20240523111627.28896-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Make use of xstate_uncompressed_size() helper rather than maintaining the running calculation while accumulating feature components. The rest of the CPUID data can come direct from the raw cpu policy. All per-component data form an ABI through the behaviour of the X{SAVE,RSTOR}* instructions. Use for_each_set_bit() rather than opencoding a slightly awkward version of it. Mask the attributes in ecx down based on the visible features. This isn't actually necessary for any components or attributes defined at the time of writing (up to AMX), but is added out of an abundance of caution. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné v2: * Tie ALIGN64 to xsavec rather than xsaves. v3: * Tweak commit message. --- xen/arch/x86/cpu-policy.c | 55 +++++++++++-------------------- xen/arch/x86/include/asm/xstate.h | 1 + 2 files changed, 21 insertions(+), 35 deletions(-) diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c index 5b66f002df05..304dc20cfab8 100644 --- a/xen/arch/x86/cpu-policy.c +++ b/xen/arch/x86/cpu-policy.c @@ -193,8 +193,7 @@ static void sanitise_featureset(uint32_t *fs) static void recalculate_xstate(struct cpu_policy *p) { uint64_t xstates = XSTATE_FP_SSE; - uint32_t xstate_size = XSTATE_AREA_MIN_SIZE; - unsigned int i, Da1 = p->xstate.Da1; + unsigned int i, ecx_mask = 0, Da1 = p->xstate.Da1; /* * The Da1 leaf is the only piece of information preserved in the common @@ -206,61 +205,47 @@ static void recalculate_xstate(struct cpu_policy *p) return; if ( p->basic.avx ) - { xstates |= X86_XCR0_YMM; - xstate_size = max(xstate_size, - xstate_offsets[X86_XCR0_YMM_POS] + - xstate_sizes[X86_XCR0_YMM_POS]); - } if ( p->feat.mpx ) - { xstates |= X86_XCR0_BNDREGS | X86_XCR0_BNDCSR; - xstate_size = max(xstate_size, - xstate_offsets[X86_XCR0_BNDCSR_POS] + - xstate_sizes[X86_XCR0_BNDCSR_POS]); - } if ( p->feat.avx512f ) - { xstates |= X86_XCR0_OPMASK | X86_XCR0_ZMM | X86_XCR0_HI_ZMM; - xstate_size = max(xstate_size, - xstate_offsets[X86_XCR0_HI_ZMM_POS] + - xstate_sizes[X86_XCR0_HI_ZMM_POS]); - } if ( p->feat.pku ) - { xstates |= X86_XCR0_PKRU; - xstate_size = max(xstate_size, - xstate_offsets[X86_XCR0_PKRU_POS] + - xstate_sizes[X86_XCR0_PKRU_POS]); - } - p->xstate.max_size = xstate_size; + /* Subleaf 0 */ + p->xstate.max_size = + xstate_uncompressed_size(xstates & ~XSTATE_XSAVES_ONLY); p->xstate.xcr0_low = xstates & ~XSTATE_XSAVES_ONLY; p->xstate.xcr0_high = (xstates & ~XSTATE_XSAVES_ONLY) >> 32; + /* Subleaf 1 */ p->xstate.Da1 = Da1; + if ( p->xstate.xsavec ) + ecx_mask |= XSTATE_ALIGN64; + if ( p->xstate.xsaves ) { + ecx_mask |= XSTATE_XSS; p->xstate.xss_low = xstates & XSTATE_XSAVES_ONLY; p->xstate.xss_high = (xstates & XSTATE_XSAVES_ONLY) >> 32; } - else - xstates &= ~XSTATE_XSAVES_ONLY; - for ( i = 2; i < min(63UL, ARRAY_SIZE(p->xstate.comp)); ++i ) + /* Subleafs 2+ */ + xstates &= ~XSTATE_FP_SSE; + BUILD_BUG_ON(ARRAY_SIZE(p->xstate.comp) < 63); + for_each_set_bit ( i, &xstates, 63 ) { - uint64_t curr_xstate = 1UL << i; - - if ( !(xstates & curr_xstate) ) - continue; - - p->xstate.comp[i].size = xstate_sizes[i]; - p->xstate.comp[i].offset = xstate_offsets[i]; - p->xstate.comp[i].xss = curr_xstate & XSTATE_XSAVES_ONLY; - p->xstate.comp[i].align = curr_xstate & xstate_align; + /* + * Pass through size (eax) and offset (ebx) directly. Visbility of + * attributes in ecx limited by visible features in Da1. + */ + p->xstate.raw[i].a = raw_cpu_policy.xstate.raw[i].a; + p->xstate.raw[i].b = raw_cpu_policy.xstate.raw[i].b; + p->xstate.raw[i].c = raw_cpu_policy.xstate.raw[i].c & ecx_mask; } } diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/xstate.h index f5115199d4f9..bfb66dd766b6 100644 --- a/xen/arch/x86/include/asm/xstate.h +++ b/xen/arch/x86/include/asm/xstate.h @@ -40,6 +40,7 @@ extern uint32_t mxcsr_mask; #define XSTATE_XSAVES_ONLY 0 #define XSTATE_COMPACTION_ENABLED (1ULL << 63) +#define XSTATE_XSS (1U << 0) #define XSTATE_ALIGN64 (1U << 1) extern u64 xfeature_mask; From patchwork Thu May 23 11:16:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13671645 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D30ADC25B7D for ; Thu, 23 May 2024 11:17:15 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.728370.1133317 (Exim 4.92) (envelope-from ) id 1sA6RN-0007lJ-VM; Thu, 23 May 2024 11:16:57 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 728370.1133317; Thu, 23 May 2024 11:16:57 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sA6RN-0007kz-QD; Thu, 23 May 2024 11:16:57 +0000 Received: by outflank-mailman (input) for mailman id 728370; Thu, 23 May 2024 11:16:57 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sA6RN-0006Hv-0K for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:16:57 +0000 Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com [2607:f8b0:4864:20::736]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f62566fc-18f5-11ef-b4bb-af5377834399; Thu, 23 May 2024 13:16:55 +0200 (CEST) Received: by mail-qk1-x736.google.com with SMTP id af79cd13be357-792c3741c71so77887785a.0 for ; Thu, 23 May 2024 04:16:55 -0700 (PDT) Received: from andrew-laptop.citrite.net ([217.156.233.157]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6a15f179cebsm142194236d6.3.2024.05.23.04.16.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 May 2024 04:16:51 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f62566fc-18f5-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1716463013; x=1717067813; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=us4603FaoecAqWNVKzEdaqnc883Hrl4rvqqHc2XtrfI=; b=bUd6IOtRE2wCKKwEStO+5AvFKg4I6nqhz8gwBIi5iW6t+OnMuUkNl0BWwllM45DlWW GyyglXfBfcSCdHJIJg3Zm/TgvZM8hy6IAUziNPEiF+4/sIpuOVIjp57rdkEf1Jg67Uf3 T94PxcTZHr3j5zzoWw8EIpvhCe0WATJvdlIxY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716463013; x=1717067813; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=us4603FaoecAqWNVKzEdaqnc883Hrl4rvqqHc2XtrfI=; b=eeB1tvtlrnKRsuuZ8hHYc3naJy75IIYXjc+cPUoTxD63fqqfpQ07d62QCcOPlAbP5H jqkQShHhvwjBufbG+vd4wjisOjBdws+ICSbrfeGsUqDQpu94/fT8mfevErU8AUBh2i1u hKLgYr1ppBoewaQx/lWG9Wl5aHLVRx+I98NEczdM2u9w5R/qk2+RBBv52Jb99qZojJIw pumt6ziU1FBSWmR1lhoCShqg0p3FnjZtv44U5UUb9XhILj9OXTbJwS83bbaevIKKWUEi pwbKyh49b9N6V0KflC3zWD3MqZ+/6xk55bd+NfSF9JQjlanCy1/xZ2Xac5QAJvLOYBTv UvKw== X-Gm-Message-State: AOJu0YwV43xrH0WsspzvjMptag/SZ05cZ1fYKbQaGrdm8A6OB8WhAiXm QRzJI4I/Q2xEUvxpIabZMtcDVEzV9zUioiCBwvXoNc4gSG341y8sNSpHG81+gxWga9l0ACWDsAS Q+1o= X-Google-Smtp-Source: AGHT+IFMd6G/LG3bqX6YqX/t3Cg+uvutJKi9sTJLns27rXhQaiIj1RFtnwEcd6kpIryKMzcdy+XPRw== X-Received: by 2002:a05:6214:2b8e:b0:6ab:932d:c24f with SMTP id 6a1803df08f44-6ab932dc50bmr12661656d6.52.1716463013119; Thu, 23 May 2024 04:16:53 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 6/7] x86/cpuid: Fix handling of XSAVE dynamic leaves Date: Thu, 23 May 2024 12:16:26 +0100 Message-Id: <20240523111627.28896-7-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240523111627.28896-1-andrew.cooper3@citrix.com> References: <20240523111627.28896-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 First, if XSAVE is available in hardware but not visible to the guest, the dynamic leaves shouldn't be filled in. Second, the comment concerning XSS state is wrong. VT-x doesn't manage host/guest state automatically, but there is provision for "host only" bits to be set, so the implications are still accurate. Introduce xstate_compressed_size() to mirror the uncompressed one. Cross check it at boot. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu v3: * Adjust commit message about !XSAVE guests * Rebase over boot time cross check * Use raw policy --- xen/arch/x86/cpuid.c | 24 ++++++++-------------- xen/arch/x86/include/asm/xstate.h | 1 + xen/arch/x86/xstate.c | 34 +++++++++++++++++++++++++++++++ 3 files changed, 43 insertions(+), 16 deletions(-) diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c index 7a38e032146a..a822e80c7ea7 100644 --- a/xen/arch/x86/cpuid.c +++ b/xen/arch/x86/cpuid.c @@ -330,23 +330,15 @@ void guest_cpuid(const struct vcpu *v, uint32_t leaf, case XSTATE_CPUID: switch ( subleaf ) { - case 1: - if ( !p->xstate.xsavec && !p->xstate.xsaves ) - break; - - /* - * TODO: Figure out what to do for XSS state. VT-x manages host - * vs guest MSR_XSS automatically, so as soon as we start - * supporting any XSS states, the wrong XSS will be in context. - */ - BUILD_BUG_ON(XSTATE_XSAVES_ONLY != 0); - fallthrough; case 0: - /* - * Read CPUID[0xD,0/1].EBX from hardware. They vary with enabled - * XSTATE, and appropriate XCR0|XSS are in context. - */ - res->b = cpuid_count_ebx(leaf, subleaf); + if ( p->basic.xsave ) + res->b = xstate_uncompressed_size(v->arch.xcr0); + break; + + case 1: + if ( p->xstate.xsavec ) + res->b = xstate_compressed_size(v->arch.xcr0 | + v->arch.msrs->xss.raw); break; } break; diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/xstate.h index bfb66dd766b6..da1d89d2f416 100644 --- a/xen/arch/x86/include/asm/xstate.h +++ b/xen/arch/x86/include/asm/xstate.h @@ -109,6 +109,7 @@ void xstate_free_save_area(struct vcpu *v); int xstate_alloc_save_area(struct vcpu *v); void xstate_init(struct cpuinfo_x86 *c); unsigned int xstate_uncompressed_size(uint64_t xcr0); +unsigned int xstate_compressed_size(uint64_t xstates); static inline uint64_t xgetbv(unsigned int index) { diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c index 1b3153600d9c..7b7f2dcaf651 100644 --- a/xen/arch/x86/xstate.c +++ b/xen/arch/x86/xstate.c @@ -621,6 +621,34 @@ unsigned int xstate_uncompressed_size(uint64_t xcr0) return size; } +unsigned int xstate_compressed_size(uint64_t xstates) +{ + unsigned int i, size = XSTATE_AREA_MIN_SIZE; + + if ( xstates == 0 ) /* TODO: clean up paths passing 0 in here. */ + return 0; + + if ( xstates <= (X86_XCR0_SSE | X86_XCR0_FP) ) + return size; + + /* + * For the compressed size, every component matters. Some componenets are + * rounded up to 64 first. + */ + xstates &= ~(X86_XCR0_SSE | X86_XCR0_FP); + for_each_set_bit ( i, &xstates, 63 ) + { + const struct xstate_component *c = &raw_cpu_policy.xstate.comp[i]; + + if ( c->align ) + size = ROUNDUP(size, 64); + + size += c->size; + } + + return size; +} + struct xcheck_state { uint64_t states; uint32_t uncomp_size; @@ -683,6 +711,12 @@ static void __init check_new_xstate(struct xcheck_state *s, uint64_t new) s->states, &new, hw_size, s->comp_size); s->comp_size = hw_size; + + xen_size = xstate_compressed_size(s->states); + + if ( xen_size != hw_size ) + panic("XSTATE 0x%016"PRIx64", compressed hw size %#x != xen size %#x\n", + s->states, hw_size, xen_size); } else BUG_ON(hw_size); /* Compressed size reported, but no XSAVEC ? */ From patchwork Thu May 23 11:16:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13671646 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 76030C25B79 for ; Thu, 23 May 2024 11:17:20 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.728373.1133327 (Exim 4.92) (envelope-from ) id 1sA6RS-0008Hr-9p; Thu, 23 May 2024 11:17:02 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 728373.1133327; Thu, 23 May 2024 11:17:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sA6RS-0008HF-5H; Thu, 23 May 2024 11:17:02 +0000 Received: by outflank-mailman (input) for mailman id 728373; Thu, 23 May 2024 11:17:00 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sA6RQ-0006Hv-Dp for xen-devel@lists.xenproject.org; Thu, 23 May 2024 11:17:00 +0000 Received: from mail-qv1-xf2f.google.com (mail-qv1-xf2f.google.com [2607:f8b0:4864:20::f2f]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f81b646d-18f5-11ef-b4bb-af5377834399; Thu, 23 May 2024 13:16:58 +0200 (CEST) Received: by mail-qv1-xf2f.google.com with SMTP id 6a1803df08f44-6ab86d61491so7106876d6.2 for ; Thu, 23 May 2024 04:16:58 -0700 (PDT) Received: from andrew-laptop.citrite.net ([217.156.233.157]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6a15f179cebsm142194236d6.3.2024.05.23.04.16.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 May 2024 04:16:54 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f81b646d-18f5-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1716463016; x=1717067816; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BRj9LcaEwZJZDPNXhHku6vQUZRLAf7w0mcjGEgwMX0k=; b=lmkq/B2qFVg/LhYiMqdxSiya/WB+LaHF4618OZcsAnio7t3FIjMOdkKiNwdaDskgIJ 9q+RwRugYlbf2JRHMGmnbRw8IFsMPNTtwE8+BhODLzc+3GT5YjhcrUADCJw5oLwmKg9o IaRMhwGlUswKcW+p125vtZhGD1NVEOUO0qWZ4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716463016; x=1717067816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BRj9LcaEwZJZDPNXhHku6vQUZRLAf7w0mcjGEgwMX0k=; b=DkeF4O+En75NZBKqMwsNswsWOLPvMhGzrzDGUWFB0I+jFx95GgqBT9sfyrCCNzfUXr +7iBkpFuB107kUr0PRc+CSWgrNfxEe9QU50JifvNDRBrEbcli0K74VRr8SilSo/9mszg Mvbt2UW/NRfifeiARd8HX79/aVlIlhyvx6FI3OrIDlGzjfllCE7ETNBWzHh9Pb5TD9mB C5aEvSAtVXDW0j1YkAZwfEJEN2QEHGFmBiBmxY+GcdSSXbn3sNuR0kGxEUGT7DmpmAVc uV086C9wbcThY3LnGdkqKP/39XB4Uf874ftRw1nw3HxlE/8sXZanDxvLxhTA4b9ivoF1 1HBw== X-Gm-Message-State: AOJu0YypQRlPXANNP7eW84r7JIZmNCgNUSEtNaAiFdQSRCoiLgnJflsV eB6nspqFm9g448n1Luk/zkMb/k+eFqzu2Aj6WQ5pfW0SM1QXbalW4p+VrZKtG35u8Ul8SY47W71 qSRA= X-Google-Smtp-Source: AGHT+IE0eWG609TbIYWGyd/qWXqqj6lCVcrCWA/lJ6+u/z7kOKwE4sZ5dOb3iisX8FOF4omHXz+8Qg== X-Received: by 2002:a05:6214:5987:b0:6ab:5b6d:f267 with SMTP id 6a1803df08f44-6ab80907622mr53889236d6.62.1716463015904; Thu, 23 May 2024 04:16:55 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH 7/7] x86/defns: Clean up X86_{XCR0,XSS}_* constants Date: Thu, 23 May 2024 12:16:27 +0100 Message-Id: <20240523111627.28896-8-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240523111627.28896-1-andrew.cooper3@citrix.com> References: <20240523111627.28896-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 With the exception of one case in read_bndcfgu() which can use ilog2(), the *_POS defines are unused. X86_XCR0_X87 is the name used by both the SDM and APM, rather than X86_XCR0_FP. No functional change. Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné v3: * New --- xen/arch/x86/i387.c | 2 +- xen/arch/x86/include/asm/x86-defns.h | 32 ++++++++++------------------ xen/arch/x86/include/asm/xstate.h | 4 ++-- xen/arch/x86/xstate.c | 18 ++++++++-------- 4 files changed, 23 insertions(+), 33 deletions(-) diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index 7a4297cc921e..fcdee10a6e69 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -369,7 +369,7 @@ void vcpu_setup_fpu(struct vcpu *v, struct xsave_struct *xsave_area, { v->arch.xsave_area->xsave_hdr.xstate_bv &= ~XSTATE_FP_SSE; if ( fcw_default != FCW_DEFAULT ) - v->arch.xsave_area->xsave_hdr.xstate_bv |= X86_XCR0_FP; + v->arch.xsave_area->xsave_hdr.xstate_bv |= X86_XCR0_X87; } } diff --git a/xen/arch/x86/include/asm/x86-defns.h b/xen/arch/x86/include/asm/x86-defns.h index d7602ab225c4..3bcdbaccd3aa 100644 --- a/xen/arch/x86/include/asm/x86-defns.h +++ b/xen/arch/x86/include/asm/x86-defns.h @@ -79,25 +79,16 @@ /* * XSTATE component flags in XCR0 | MSR_XSS */ -#define X86_XCR0_FP_POS 0 -#define X86_XCR0_FP (1ULL << X86_XCR0_FP_POS) -#define X86_XCR0_SSE_POS 1 -#define X86_XCR0_SSE (1ULL << X86_XCR0_SSE_POS) -#define X86_XCR0_YMM_POS 2 -#define X86_XCR0_YMM (1ULL << X86_XCR0_YMM_POS) -#define X86_XCR0_BNDREGS_POS 3 -#define X86_XCR0_BNDREGS (1ULL << X86_XCR0_BNDREGS_POS) -#define X86_XCR0_BNDCSR_POS 4 -#define X86_XCR0_BNDCSR (1ULL << X86_XCR0_BNDCSR_POS) -#define X86_XCR0_OPMASK_POS 5 -#define X86_XCR0_OPMASK (1ULL << X86_XCR0_OPMASK_POS) -#define X86_XCR0_ZMM_POS 6 -#define X86_XCR0_ZMM (1ULL << X86_XCR0_ZMM_POS) -#define X86_XCR0_HI_ZMM_POS 7 -#define X86_XCR0_HI_ZMM (1ULL << X86_XCR0_HI_ZMM_POS) +#define X86_XCR0_X87 (_AC(1, ULL) << 0) +#define X86_XCR0_SSE (_AC(1, ULL) << 1) +#define X86_XCR0_YMM (_AC(1, ULL) << 2) +#define X86_XCR0_BNDREGS (_AC(1, ULL) << 3) +#define X86_XCR0_BNDCSR (_AC(1, ULL) << 4) +#define X86_XCR0_OPMASK (_AC(1, ULL) << 5) +#define X86_XCR0_ZMM (_AC(1, ULL) << 6) +#define X86_XCR0_HI_ZMM (_AC(1, ULL) << 7) #define X86_XSS_PROC_TRACE (_AC(1, ULL) << 8) -#define X86_XCR0_PKRU_POS 9 -#define X86_XCR0_PKRU (1ULL << X86_XCR0_PKRU_POS) +#define X86_XCR0_PKRU (_AC(1, ULL) << 9) #define X86_XSS_PASID (_AC(1, ULL) << 10) #define X86_XSS_CET_U (_AC(1, ULL) << 11) #define X86_XSS_CET_S (_AC(1, ULL) << 12) @@ -107,11 +98,10 @@ #define X86_XSS_HWP (_AC(1, ULL) << 16) #define X86_XCR0_TILE_CFG (_AC(1, ULL) << 17) #define X86_XCR0_TILE_DATA (_AC(1, ULL) << 18) -#define X86_XCR0_LWP_POS 62 -#define X86_XCR0_LWP (1ULL << X86_XCR0_LWP_POS) +#define X86_XCR0_LWP (_AC(1, ULL) << 62) #define X86_XCR0_STATES \ - (X86_XCR0_FP | X86_XCR0_SSE | X86_XCR0_YMM | X86_XCR0_BNDREGS | \ + (X86_XCR0_X87 | X86_XCR0_SSE | X86_XCR0_YMM | X86_XCR0_BNDREGS | \ X86_XCR0_BNDCSR | X86_XCR0_OPMASK | X86_XCR0_ZMM | \ X86_XCR0_HI_ZMM | X86_XCR0_PKRU | X86_XCR0_TILE_CFG | \ X86_XCR0_TILE_DATA | \ diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/xstate.h index da1d89d2f416..f4a8e5f814a0 100644 --- a/xen/arch/x86/include/asm/xstate.h +++ b/xen/arch/x86/include/asm/xstate.h @@ -29,8 +29,8 @@ extern uint32_t mxcsr_mask; #define XSAVE_HDR_OFFSET FXSAVE_SIZE #define XSTATE_AREA_MIN_SIZE (FXSAVE_SIZE + XSAVE_HDR_SIZE) -#define XSTATE_FP_SSE (X86_XCR0_FP | X86_XCR0_SSE) -#define XCNTXT_MASK (X86_XCR0_FP | X86_XCR0_SSE | X86_XCR0_YMM | \ +#define XSTATE_FP_SSE (X86_XCR0_X87 | X86_XCR0_SSE) +#define XCNTXT_MASK (X86_XCR0_X87 | X86_XCR0_SSE | X86_XCR0_YMM | \ X86_XCR0_OPMASK | X86_XCR0_ZMM | X86_XCR0_HI_ZMM | \ XSTATE_NONLAZY) diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c index 7b7f2dcaf651..0ed2541665b3 100644 --- a/xen/arch/x86/xstate.c +++ b/xen/arch/x86/xstate.c @@ -313,7 +313,7 @@ void xsave(struct vcpu *v, uint64_t mask) "=m" (*ptr), \ "a" (lmask), "d" (hmask), "D" (ptr)) - if ( fip_width == 8 || !(mask & X86_XCR0_FP) ) + if ( fip_width == 8 || !(mask & X86_XCR0_X87) ) { XSAVE("0x48,"); } @@ -366,7 +366,7 @@ void xsave(struct vcpu *v, uint64_t mask) fip_width = 8; } #undef XSAVE - if ( mask & X86_XCR0_FP ) + if ( mask & X86_XCR0_X87 ) ptr->fpu_sse.x[FPU_WORD_SIZE_OFFSET] = fip_width; } @@ -558,7 +558,7 @@ void xstate_free_save_area(struct vcpu *v) static bool valid_xcr0(uint64_t xcr0) { /* FP must be unconditionally set. */ - if ( !(xcr0 & X86_XCR0_FP) ) + if ( !(xcr0 & X86_XCR0_X87) ) return false; /* YMM depends on SSE. */ @@ -599,7 +599,7 @@ unsigned int xstate_uncompressed_size(uint64_t xcr0) if ( xcr0 == 0 ) /* TODO: clean up paths passing 0 in here. */ return 0; - if ( xcr0 <= (X86_XCR0_SSE | X86_XCR0_FP) ) + if ( xcr0 <= (X86_XCR0_SSE | X86_XCR0_X87) ) return size; /* @@ -607,7 +607,7 @@ unsigned int xstate_uncompressed_size(uint64_t xcr0) * maximum offset+size. Some states (e.g. LWP, APX_F) are out-of-order * with respect their index. */ - xcr0 &= ~(X86_XCR0_SSE | X86_XCR0_FP); + xcr0 &= ~(X86_XCR0_SSE | X86_XCR0_X87); for_each_set_bit ( i, &xcr0, 63 ) { const struct xstate_component *c = &raw_cpu_policy.xstate.comp[i]; @@ -628,14 +628,14 @@ unsigned int xstate_compressed_size(uint64_t xstates) if ( xstates == 0 ) /* TODO: clean up paths passing 0 in here. */ return 0; - if ( xstates <= (X86_XCR0_SSE | X86_XCR0_FP) ) + if ( xstates <= (X86_XCR0_SSE | X86_XCR0_X87) ) return size; /* * For the compressed size, every component matters. Some componenets are * rounded up to 64 first. */ - xstates &= ~(X86_XCR0_SSE | X86_XCR0_FP); + xstates &= ~(X86_XCR0_SSE | X86_XCR0_X87); for_each_set_bit ( i, &xstates, 63 ) { const struct xstate_component *c = &raw_cpu_policy.xstate.comp[i]; @@ -748,7 +748,7 @@ static void __init noinline xstate_check_sizes(void) * layout compatibility with Intel and having a knock-on effect on all * subsequent states. */ - check_new_xstate(&s, X86_XCR0_SSE | X86_XCR0_FP); + check_new_xstate(&s, X86_XCR0_SSE | X86_XCR0_X87); if ( cpu_has_avx ) check_new_xstate(&s, X86_XCR0_YMM); @@ -1000,7 +1000,7 @@ uint64_t read_bndcfgu(void) : "=m" (*xstate) : "a" (X86_XCR0_BNDCSR), "d" (0), "D" (xstate) ); - bndcsr = (void *)xstate + xstate_offsets[X86_XCR0_BNDCSR_POS]; + bndcsr = (void *)xstate + xstate_offsets[ilog2(X86_XCR0_BNDCSR)]; } if ( cr0 & X86_CR0_TS )