From patchwork Thu Oct 1 20:38:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chang S. Bae" X-Patchwork-Id: 11812405 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 61C3E6CB for ; Thu, 1 Oct 2020 20:44:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4B60B2085B for ; Thu, 1 Oct 2020 20:44:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733256AbgJAUoJ (ORCPT ); Thu, 1 Oct 2020 16:44:09 -0400 Received: from mga11.intel.com ([192.55.52.93]:58726 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733064AbgJAUnQ (ORCPT ); Thu, 1 Oct 2020 16:43:16 -0400 IronPort-SDR: fza0v1cjHYKP6SqAW60YTJKuwKoYpQuNhb/YIARKPxMNCjZvZ97rfWamEVkEjmbGwfHsVRVIJi nw37Acklo+2Q== X-IronPort-AV: E=McAfee;i="6000,8403,9761"; a="160170701" X-IronPort-AV: E=Sophos;i="5.77,325,1596524400"; d="scan'208";a="160170701" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Oct 2020 13:42:51 -0700 IronPort-SDR: JT3cW7si4Isk6UuP3TJx5OsaCMJyGh1CNUi+kkrWKavd7ITA25Gy+gTjjqZWdjpg5JrNMIm9OC UKoMZJNULj8Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,325,1596524400"; d="scan'208";a="351297026" Received: from chang-linux-3.sc.intel.com ([172.25.66.175]) by FMSMGA003.fm.intel.com with ESMTP; 01 Oct 2020 13:42:51 -0700 From: "Chang S. Bae" To: tglx@linutronix.de, mingo@kernel.org, bp@suse.de, luto@kernel.org, x86@kernel.org Cc: len.brown@intel.com, dave.hansen@intel.com, jing2.liu@intel.com, ravi.v.shankar@intel.com, linux-kernel@vger.kernel.org, chang.seok.bae@intel.com, kvm@vger.kernel.org Subject: [RFC PATCH 01/22] x86/fpu/xstate: Modify area init helper prototypes to access all the possible areas Date: Thu, 1 Oct 2020 13:38:52 -0700 Message-Id: <20201001203913.9125-2-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201001203913.9125-1-chang.seok.bae@intel.com> References: <20201001203913.9125-1-chang.seok.bae@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The xstate infrastructure is not flexible to support dynamic areas in task->fpu. Change the fpstate_init() prototype to access task->fpu directly. It treats a null pointer as indicating init_fpstate, as this initial data does not belong to any task. For the compacted format, fpstate_init_xstate() now accepts the state component bitmap to configure XCOMP_BV. No functional change. Signed-off-by: Chang S. Bae Reviewed-by: Len Brown Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org --- arch/x86/include/asm/fpu/internal.h | 6 +++--- arch/x86/kernel/fpu/core.c | 14 +++++++++++--- arch/x86/kernel/fpu/init.c | 2 +- arch/x86/kernel/fpu/regset.c | 2 +- arch/x86/kernel/fpu/xstate.c | 3 +-- arch/x86/kvm/x86.c | 2 +- 6 files changed, 18 insertions(+), 11 deletions(-) diff --git a/arch/x86/include/asm/fpu/internal.h b/arch/x86/include/asm/fpu/internal.h index 0a460f2a3f90..c404fedf1a75 100644 --- a/arch/x86/include/asm/fpu/internal.h +++ b/arch/x86/include/asm/fpu/internal.h @@ -79,20 +79,20 @@ static __always_inline __pure bool use_fxsr(void) extern union fpregs_state init_fpstate; -extern void fpstate_init(union fpregs_state *state); +extern void fpstate_init(struct fpu *fpu); #ifdef CONFIG_MATH_EMULATION extern void fpstate_init_soft(struct swregs_state *soft); #else static inline void fpstate_init_soft(struct swregs_state *soft) {} #endif -static inline void fpstate_init_xstate(struct xregs_state *xsave) +static inline void fpstate_init_xstate(struct xregs_state *xsave, u64 xcomp_mask) { /* * XRSTORS requires these bits set in xcomp_bv, or it will * trigger #GP: */ - xsave->header.xcomp_bv = XCOMP_BV_COMPACTED_FORMAT | xfeatures_mask_all; + xsave->header.xcomp_bv = XCOMP_BV_COMPACTED_FORMAT | xcomp_mask; } static inline void fpstate_init_fxstate(struct fxregs_state *fx) diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index eb86a2b831b1..41d926c76615 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -191,8 +191,16 @@ static inline void fpstate_init_fstate(struct fregs_state *fp) fp->fos = 0xffff0000u; } -void fpstate_init(union fpregs_state *state) +/* If a null pointer is given, assume to take the initial FPU state, init_fpstate. */ +void fpstate_init(struct fpu *fpu) { + union fpregs_state *state; + + if (fpu) + state = &fpu->state; + else + state = &init_fpstate; + if (!static_cpu_has(X86_FEATURE_FPU)) { fpstate_init_soft(&state->soft); return; @@ -201,7 +209,7 @@ void fpstate_init(union fpregs_state *state) memset(state, 0, fpu_kernel_xstate_size); if (static_cpu_has(X86_FEATURE_XSAVES)) - fpstate_init_xstate(&state->xsave); + fpstate_init_xstate(&state->xsave, xfeatures_mask_all); if (static_cpu_has(X86_FEATURE_FXSR)) fpstate_init_fxstate(&state->fxsave); else @@ -261,7 +269,7 @@ static void fpu__initialize(struct fpu *fpu) WARN_ON_FPU(fpu != ¤t->thread.fpu); set_thread_flag(TIF_NEED_FPU_LOAD); - fpstate_init(&fpu->state); + fpstate_init(fpu); trace_x86_fpu_init_state(fpu); } diff --git a/arch/x86/kernel/fpu/init.c b/arch/x86/kernel/fpu/init.c index 61ddc3a5e5c2..4e89a2698cfb 100644 --- a/arch/x86/kernel/fpu/init.c +++ b/arch/x86/kernel/fpu/init.c @@ -125,7 +125,7 @@ static void __init fpu__init_system_generic(void) * Set up the legacy init FPU context. (xstate init might overwrite this * with a more modern format, if the CPU supports it.) */ - fpstate_init(&init_fpstate); + fpstate_init(NULL); fpu__init_system_mxcsr(); } diff --git a/arch/x86/kernel/fpu/regset.c b/arch/x86/kernel/fpu/regset.c index c413756ba89f..4c4d9059ff36 100644 --- a/arch/x86/kernel/fpu/regset.c +++ b/arch/x86/kernel/fpu/regset.c @@ -144,7 +144,7 @@ int xstateregs_set(struct task_struct *target, const struct user_regset *regset, * In case of failure, mark all states as init: */ if (ret) - fpstate_init(&fpu->state); + fpstate_init(fpu); return ret; } diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index 038e19c0019e..ee4946c60ab1 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -454,8 +454,7 @@ static void __init setup_init_fpu_buf(void) print_xstate_features(); if (boot_cpu_has(X86_FEATURE_XSAVES)) - init_fpstate.xsave.header.xcomp_bv = XCOMP_BV_COMPACTED_FORMAT | - xfeatures_mask_all; + fpstate_init_xstate(&init_fpstate.xsave, xfeatures_mask_all); /* * Init all the features state with header.xfeatures being 0x0 diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ce856e0ece84..9da8cb4b8589 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9448,7 +9448,7 @@ static int sync_regs(struct kvm_vcpu *vcpu) static void fx_init(struct kvm_vcpu *vcpu) { - fpstate_init(&vcpu->arch.guest_fpu->state); + fpstate_init(vcpu->arch.guest_fpu); if (boot_cpu_has(X86_FEATURE_XSAVES)) vcpu->arch.guest_fpu->state.xsave.header.xcomp_bv = host_xcr0 | XSTATE_COMPACTION_ENABLED; From patchwork Thu Oct 1 20:38:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chang S. Bae" X-Patchwork-Id: 11812403 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B549692C for ; Thu, 1 Oct 2020 20:44:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9AC5E2085B for ; Thu, 1 Oct 2020 20:44:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733264AbgJAUoK (ORCPT ); Thu, 1 Oct 2020 16:44:10 -0400 Received: from mga11.intel.com ([192.55.52.93]:58726 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729993AbgJAUm6 (ORCPT ); Thu, 1 Oct 2020 16:42:58 -0400 IronPort-SDR: OxPcjmLujwzBFihNYeTARm7LXUrUznOBIO85BlHeCtrQHBOBysFK7Oj8lweeglT2mahLPT3u+i ZRseESIXKSRg== X-IronPort-AV: E=McAfee;i="6000,8403,9761"; a="160170707" X-IronPort-AV: E=Sophos;i="5.77,325,1596524400"; d="scan'208";a="160170707" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Oct 2020 13:42:51 -0700 IronPort-SDR: SaNYuE0iXU5uewmHliv4GyDnOhLuLvHsQHdAPgbTpOPWkJXv2RfIZAd1y2gWMVMEzN7t6QMDFf s/ZbKFptNkYA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,325,1596524400"; d="scan'208";a="351297034" Received: from chang-linux-3.sc.intel.com ([172.25.66.175]) by FMSMGA003.fm.intel.com with ESMTP; 01 Oct 2020 13:42:51 -0700 From: "Chang S. Bae" To: tglx@linutronix.de, mingo@kernel.org, bp@suse.de, luto@kernel.org, x86@kernel.org Cc: len.brown@intel.com, dave.hansen@intel.com, jing2.liu@intel.com, ravi.v.shankar@intel.com, linux-kernel@vger.kernel.org, chang.seok.bae@intel.com, kvm@vger.kernel.org Subject: [RFC PATCH 03/22] x86/fpu/xstate: Modify address finder prototypes to access all the possible areas Date: Thu, 1 Oct 2020 13:38:54 -0700 Message-Id: <20201001203913.9125-4-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201001203913.9125-1-chang.seok.bae@intel.com> References: <20201001203913.9125-1-chang.seok.bae@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The xstate infrastructure is not flexible to support dynamic areas in task->fpu. Change the prototype of some address finding functions to access task->fpu directly. Make changes for both outer and inner helpers: get_xsave_addr() and __raw_xsave_addr(). No functional change. Signed-off-by: Chang S. Bae Reviewed-by: Len Brown Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org --- arch/x86/include/asm/fpu/internal.h | 2 +- arch/x86/include/asm/fpu/xstate.h | 2 +- arch/x86/include/asm/pgtable.h | 2 +- arch/x86/kernel/cpu/common.c | 2 +- arch/x86/kernel/fpu/xstate.c | 43 ++++++++++++++++++++--------- arch/x86/kvm/x86.c | 26 +++++++++++------ arch/x86/mm/pkeys.c | 2 +- 7 files changed, 52 insertions(+), 27 deletions(-) diff --git a/arch/x86/include/asm/fpu/internal.h b/arch/x86/include/asm/fpu/internal.h index c404fedf1a75..baca80e877a6 100644 --- a/arch/x86/include/asm/fpu/internal.h +++ b/arch/x86/include/asm/fpu/internal.h @@ -578,7 +578,7 @@ static inline void switch_fpu_finish(struct fpu *new_fpu) * return to userland e.g. for a copy_to_user() operation. */ if (current->mm) { - pk = get_xsave_addr(&new_fpu->state.xsave, XFEATURE_PKRU); + pk = get_xsave_addr(new_fpu, XFEATURE_PKRU); if (pk) pkru_val = pk->pkru; } diff --git a/arch/x86/include/asm/fpu/xstate.h b/arch/x86/include/asm/fpu/xstate.h index a315b055212f..3fbf45727ad6 100644 --- a/arch/x86/include/asm/fpu/xstate.h +++ b/arch/x86/include/asm/fpu/xstate.h @@ -100,7 +100,7 @@ extern u64 xstate_fx_sw_bytes[USER_XSTATE_FX_SW_WORDS]; extern void __init update_regset_xstate_info(unsigned int size, u64 xstate_mask); -void *get_xsave_addr(struct xregs_state *xsave, int xfeature_nr); +void *get_xsave_addr(struct fpu *fpu, int xfeature_nr); const void *get_xsave_field_ptr(int xfeature_nr); int using_compacted_format(void); int xfeature_size(int xfeature_nr); diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index b836138ce852..e24a8fb8f479 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -142,7 +142,7 @@ static inline void write_pkru(u32 pkru) if (!boot_cpu_has(X86_FEATURE_OSPKE)) return; - pk = get_xsave_addr(¤t->thread.fpu.state.xsave, XFEATURE_PKRU); + pk = get_xsave_addr(¤t->thread.fpu, XFEATURE_PKRU); /* * The PKRU value in xstate needs to be in sync with the value that is diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index d0363e15ec2e..183ee7f77065 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -478,7 +478,7 @@ static __always_inline void setup_pku(struct cpuinfo_x86 *c) return; cr4_set_bits(X86_CR4_PKE); - pk = get_xsave_addr(&init_fpstate.xsave, XFEATURE_PKRU); + pk = get_xsave_addr(NULL, XFEATURE_PKRU); if (pk) pk->pkru = init_pkru_value; /* diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index e3a9bddc39d9..bab22766b79b 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -891,15 +891,23 @@ void fpu__resume_cpu(void) * buffer the state is. Callers should ensure that the buffer * is valid. */ -static void *__raw_xsave_addr(struct xregs_state *xsave, int xfeature_nr) +static void *__raw_xsave_addr(struct fpu *fpu, int xfeature_nr) { + void *xsave; + if (!xfeature_enabled(xfeature_nr)) { WARN_ON_FPU(1); return NULL; } - return (void *)xsave + xstate_comp_offsets[xfeature_nr]; + if (fpu) + xsave = &fpu->state.xsave; + else + xsave = &init_fpstate.xsave; + + return xsave + xstate_comp_offsets[xfeature_nr]; } + /* * Given the xsave area and a state inside, this function returns the * address of the state. @@ -911,15 +919,18 @@ static void *__raw_xsave_addr(struct xregs_state *xsave, int xfeature_nr) * this will return NULL. * * Inputs: - * xstate: the thread's storage area for all FPU data + * fpu: the thread's FPU data to access all the FPU state storages. + (If a null pointer is given, assume the init_fpstate) * xfeature_nr: state which is defined in xsave.h (e.g. XFEATURE_FP, * XFEATURE_SSE, etc...) * Output: * address of the state in the xsave area, or NULL if the * field is not present in the xsave buffer. */ -void *get_xsave_addr(struct xregs_state *xsave, int xfeature_nr) +void *get_xsave_addr(struct fpu *fpu, int xfeature_nr) { + struct xregs_state *xsave; + /* * Do we even *have* xsave state? */ @@ -932,6 +943,12 @@ void *get_xsave_addr(struct xregs_state *xsave, int xfeature_nr) */ WARN_ONCE(!(xfeatures_mask_all & BIT_ULL(xfeature_nr)), "get of unsupported state"); + + if (fpu) + xsave = &fpu->state.xsave; + else + xsave = &init_fpstate.xsave; + /* * This assumes the last 'xsave*' instruction to * have requested that 'xfeature_nr' be saved. @@ -946,7 +963,7 @@ void *get_xsave_addr(struct xregs_state *xsave, int xfeature_nr) if (!(xsave->header.xfeatures & BIT_ULL(xfeature_nr))) return NULL; - return __raw_xsave_addr(xsave, xfeature_nr); + return __raw_xsave_addr(fpu, xfeature_nr); } EXPORT_SYMBOL_GPL(get_xsave_addr); @@ -977,7 +994,7 @@ const void *get_xsave_field_ptr(int xfeature_nr) */ fpu__save(fpu); - return get_xsave_addr(&fpu->state.xsave, xfeature_nr); + return get_xsave_addr(fpu, xfeature_nr); } #ifdef CONFIG_ARCH_HAS_PKEYS @@ -1112,7 +1129,7 @@ void copy_xstate_to_kernel(struct membuf to, struct fpu *fpu) * Copy only in-use xstates: */ if ((header.xfeatures >> i) & 1) { - void *src = __raw_xsave_addr(xsave, i); + void *src = __raw_xsave_addr(fpu, i); copy_part(&to, &last, xstate_offsets[i], xstate_sizes[i], src); @@ -1141,13 +1158,11 @@ int copy_kernel_to_xstate(struct fpu *fpu, const void *kbuf) if (validate_user_xstate_header(&hdr)) return -EINVAL; - xsave = &fpu->state.xsave; - for (i = 0; i < XFEATURE_MAX; i++) { u64 mask = ((u64)1 << i); if (hdr.xfeatures & mask) { - void *dst = __raw_xsave_addr(xsave, i); + void *dst = __raw_xsave_addr(fpu, i); offset = xstate_offsets[i]; size = xstate_sizes[i]; @@ -1156,6 +1171,8 @@ int copy_kernel_to_xstate(struct fpu *fpu, const void *kbuf) } } + xsave = &fpu->state.xsave; + if (xfeatures_mxcsr_quirk(hdr.xfeatures)) { offset = offsetof(struct fxregs_state, mxcsr); size = MXCSR_AND_FLAGS_SIZE; @@ -1198,13 +1215,11 @@ int copy_user_to_xstate(struct fpu *fpu, const void __user *ubuf) if (validate_user_xstate_header(&hdr)) return -EINVAL; - xsave = &fpu->state.xsave; - for (i = 0; i < XFEATURE_MAX; i++) { u64 mask = ((u64)1 << i); if (hdr.xfeatures & mask) { - void *dst = __raw_xsave_addr(xsave, i); + void *dst = __raw_xsave_addr(fpu, i); offset = xstate_offsets[i]; size = xstate_sizes[i]; @@ -1214,6 +1229,8 @@ int copy_user_to_xstate(struct fpu *fpu, const void __user *ubuf) } } + xsave = &fpu->state.xsave; + if (xfeatures_mxcsr_quirk(hdr.xfeatures)) { offset = offsetof(struct fxregs_state, mxcsr); size = MXCSR_AND_FLAGS_SIZE; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 9da8cb4b8589..c4b8d3705625 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4182,10 +4182,15 @@ static int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu, static void fill_xsave(u8 *dest, struct kvm_vcpu *vcpu) { - struct xregs_state *xsave = &vcpu->arch.guest_fpu->state.xsave; - u64 xstate_bv = xsave->header.xfeatures; + struct xregs_state *xsave; + struct fpu *guest_fpu; + u64 xstate_bv; u64 valid; + guest_fpu = vcpu->arch.guest_fpu; + xsave = &guest_fpu->state.xsave; + xstate_bv = xsave->header.xfeatures; + /* * Copy legacy XSAVE area, to avoid complications with CPUID * leaves 0 and 1 in the loop below. @@ -4204,7 +4209,7 @@ static void fill_xsave(u8 *dest, struct kvm_vcpu *vcpu) while (valid) { u64 xfeature_mask = valid & -valid; int xfeature_nr = fls64(xfeature_mask) - 1; - void *src = get_xsave_addr(xsave, xfeature_nr); + void *src = get_xsave_addr(guest_fpu, xfeature_nr); if (src) { u32 size, offset, ecx, edx; @@ -4224,10 +4229,14 @@ static void fill_xsave(u8 *dest, struct kvm_vcpu *vcpu) static void load_xsave(struct kvm_vcpu *vcpu, u8 *src) { - struct xregs_state *xsave = &vcpu->arch.guest_fpu->state.xsave; u64 xstate_bv = *(u64 *)(src + XSAVE_HDR_OFFSET); + struct xregs_state *xsave; + struct fpu *guest_fpu; u64 valid; + guest_fpu = vcpu->arch.guest_fpu; + xsave = &guest_fpu->state.xsave; + /* * Copy legacy XSAVE area, to avoid complications with CPUID * leaves 0 and 1 in the loop below. @@ -4247,7 +4256,7 @@ static void load_xsave(struct kvm_vcpu *vcpu, u8 *src) while (valid) { u64 xfeature_mask = valid & -valid; int xfeature_nr = fls64(xfeature_mask) - 1; - void *dest = get_xsave_addr(xsave, xfeature_nr); + void *dest = get_xsave_addr(guest_fpu, xfeature_nr); if (dest) { u32 size, offset, ecx, edx; @@ -9662,6 +9671,7 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) vcpu->arch.apf.halted = false; if (kvm_mpx_supported()) { + struct fpu *guest_fpu = vcpu->arch.guest_fpu; void *mpx_state_buffer; /* @@ -9670,12 +9680,10 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) */ if (init_event) kvm_put_guest_fpu(vcpu); - mpx_state_buffer = get_xsave_addr(&vcpu->arch.guest_fpu->state.xsave, - XFEATURE_BNDREGS); + mpx_state_buffer = get_xsave_addr(guest_fpu, XFEATURE_BNDREGS); if (mpx_state_buffer) memset(mpx_state_buffer, 0, sizeof(struct mpx_bndreg_state)); - mpx_state_buffer = get_xsave_addr(&vcpu->arch.guest_fpu->state.xsave, - XFEATURE_BNDCSR); + mpx_state_buffer = get_xsave_addr(guest_fpu, XFEATURE_BNDCSR); if (mpx_state_buffer) memset(mpx_state_buffer, 0, sizeof(struct mpx_bndcsr)); if (init_event) diff --git a/arch/x86/mm/pkeys.c b/arch/x86/mm/pkeys.c index 8873ed1438a9..772e8bc3d49d 100644 --- a/arch/x86/mm/pkeys.c +++ b/arch/x86/mm/pkeys.c @@ -177,7 +177,7 @@ static ssize_t init_pkru_write_file(struct file *file, return -EINVAL; WRITE_ONCE(init_pkru_value, new_init_pkru); - pk = get_xsave_addr(&init_fpstate.xsave, XFEATURE_PKRU); + pk = get_xsave_addr(NULL, XFEATURE_PKRU); if (!pk) return -EINVAL; pk->pkru = new_init_pkru; From patchwork Thu Oct 1 20:38:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chang S. Bae" X-Patchwork-Id: 11812395 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A7CD6CB for ; Thu, 1 Oct 2020 20:43:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4A61520872 for ; Thu, 1 Oct 2020 20:43:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727713AbgJAUmy (ORCPT ); Thu, 1 Oct 2020 16:42:54 -0400 Received: from mga11.intel.com ([192.55.52.93]:58716 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726515AbgJAUmw (ORCPT ); Thu, 1 Oct 2020 16:42:52 -0400 IronPort-SDR: WUjfmZ/pJ32mhbyS183zl6hnDH+X7/TLhPa0PPQWGfE9nUEVjeoxmpahyRGnl3Sm9uwlh4nsnb g3F7rVHVtjYg== X-IronPort-AV: E=McAfee;i="6000,8403,9761"; a="160170709" X-IronPort-AV: E=Sophos;i="5.77,325,1596524400"; d="scan'208";a="160170709" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Oct 2020 13:42:51 -0700 IronPort-SDR: gYo6blMVVytvDhYheP8ApAZAlpOmQ2A89VQg/UPDpHm+Kx/6i9XmpVsZdmaRY+5tVIlSt2d90y 8kVc6oIR+F7g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,325,1596524400"; d="scan'208";a="351297037" Received: from chang-linux-3.sc.intel.com ([172.25.66.175]) by FMSMGA003.fm.intel.com with ESMTP; 01 Oct 2020 13:42:51 -0700 From: "Chang S. Bae" To: tglx@linutronix.de, mingo@kernel.org, bp@suse.de, luto@kernel.org, x86@kernel.org Cc: len.brown@intel.com, dave.hansen@intel.com, jing2.liu@intel.com, ravi.v.shankar@intel.com, linux-kernel@vger.kernel.org, chang.seok.bae@intel.com, kvm@vger.kernel.org Subject: [RFC PATCH 04/22] x86/fpu/xstate: Modify save and restore helper prototypes to access all the possible areas Date: Thu, 1 Oct 2020 13:38:55 -0700 Message-Id: <20201001203913.9125-5-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201001203913.9125-1-chang.seok.bae@intel.com> References: <20201001203913.9125-1-chang.seok.bae@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The xstate infrastructure is not flexible to support dynamic areas in task->fpu. Make the xstate save and restore helpers to access task->fpu directly. No functional change. Signed-off-by: Chang S. Bae Reviewed-by: Len Brown Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org --- arch/x86/include/asm/fpu/internal.h | 9 ++++++--- arch/x86/kernel/fpu/core.c | 4 ++-- arch/x86/kernel/fpu/signal.c | 3 +-- arch/x86/kvm/x86.c | 2 +- 4 files changed, 10 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/fpu/internal.h b/arch/x86/include/asm/fpu/internal.h index baca80e877a6..6eec5209750f 100644 --- a/arch/x86/include/asm/fpu/internal.h +++ b/arch/x86/include/asm/fpu/internal.h @@ -396,8 +396,9 @@ static inline int copy_user_to_xregs(struct xregs_state __user *buf, u64 mask) * Restore xstate from kernel space xsave area, return an error code instead of * an exception. */ -static inline int copy_kernel_to_xregs_err(struct xregs_state *xstate, u64 mask) +static inline int copy_kernel_to_xregs_err(struct fpu *fpu, u64 mask) { + struct xregs_state *xstate = &fpu->state.xsave; u32 lmask = mask; u32 hmask = mask >> 32; int err; @@ -424,8 +425,10 @@ static inline void __copy_kernel_to_fpregs(union fpregs_state *fpstate, u64 mask } } -static inline void copy_kernel_to_fpregs(union fpregs_state *fpstate) +static inline void copy_kernel_to_fpregs(struct fpu *fpu) { + union fpregs_state *fpstate = &fpu->state; + /* * AMD K7/K8 CPUs don't save/restore FDP/FIP/FOP unless an exception is * pending. Clear the x87 state here by setting it to fixed values. @@ -510,7 +513,7 @@ static inline void __fpregs_load_activate(void) return; if (!fpregs_state_valid(fpu, cpu)) { - copy_kernel_to_fpregs(&fpu->state); + copy_kernel_to_fpregs(fpu); fpregs_activate(fpu); fpu->last_cpu = cpu; } diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index 41d926c76615..39ddb22c143b 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -172,7 +172,7 @@ void fpu__save(struct fpu *fpu) if (!test_thread_flag(TIF_NEED_FPU_LOAD)) { if (!copy_fpregs_to_fpstate(fpu)) { - copy_kernel_to_fpregs(&fpu->state); + copy_kernel_to_fpregs(fpu); } } @@ -248,7 +248,7 @@ int fpu__copy(struct task_struct *dst, struct task_struct *src) memcpy(&dst_fpu->state, &src_fpu->state, fpu_kernel_xstate_size); else if (!copy_fpregs_to_fpstate(dst_fpu)) - copy_kernel_to_fpregs(&dst_fpu->state); + copy_kernel_to_fpregs(dst_fpu); fpregs_unlock(); diff --git a/arch/x86/kernel/fpu/signal.c b/arch/x86/kernel/fpu/signal.c index adbf63114bc2..6f3bcc7dab80 100644 --- a/arch/x86/kernel/fpu/signal.c +++ b/arch/x86/kernel/fpu/signal.c @@ -427,8 +427,7 @@ static int __fpu__restore_sig(void __user *buf, void __user *buf_fx, int size) * Restore previously saved supervisor xstates along with * copied-in user xstates. */ - ret = copy_kernel_to_xregs_err(&fpu->state.xsave, - user_xfeatures | xfeatures_mask_supervisor()); + ret = copy_kernel_to_xregs_err(fpu, user_xfeatures | xfeatures_mask_supervisor()); } else if (use_fxsr()) { ret = __copy_from_user(&fpu->state.fxsave, buf_fx, state_size); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c4b8d3705625..192d52ff5b8c 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8877,7 +8877,7 @@ static void kvm_put_guest_fpu(struct kvm_vcpu *vcpu) kvm_save_current_fpu(vcpu->arch.guest_fpu); - copy_kernel_to_fpregs(&vcpu->arch.user_fpu->state); + copy_kernel_to_fpregs(vcpu->arch.user_fpu); fpregs_mark_activate(); fpregs_unlock(); From patchwork Thu Oct 1 20:38:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chang S. Bae" X-Patchwork-Id: 11812397 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 194E092C for ; Thu, 1 Oct 2020 20:43:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E355620872 for ; Thu, 1 Oct 2020 20:43:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727053AbgJAUmw (ORCPT ); Thu, 1 Oct 2020 16:42:52 -0400 Received: from mga11.intel.com ([192.55.52.93]:58716 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726487AbgJAUmw (ORCPT ); Thu, 1 Oct 2020 16:42:52 -0400 IronPort-SDR: bzJm7NXfutW+pKYPqPIIQDENBNJR/JGmR1gYq0GOmxVtmgYlkA9fQAGOfGhP6mHP7dBH/KjAns PTLg0W8K7i5g== X-IronPort-AV: E=McAfee;i="6000,8403,9761"; a="160170713" X-IronPort-AV: E=Sophos;i="5.77,325,1596524400"; d="scan'208";a="160170713" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Oct 2020 13:42:52 -0700 IronPort-SDR: XvABefGlMnfU14ZroFxEz2vm+mSLXNyejep8HVTiYSja4asGEQM6YGmNLk9JaYZuQ42KkCpfLu QpJVTpUTWWpg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,325,1596524400"; d="scan'208";a="351297044" Received: from chang-linux-3.sc.intel.com ([172.25.66.175]) by FMSMGA003.fm.intel.com with ESMTP; 01 Oct 2020 13:42:51 -0700 From: "Chang S. Bae" To: tglx@linutronix.de, mingo@kernel.org, bp@suse.de, luto@kernel.org, x86@kernel.org Cc: len.brown@intel.com, dave.hansen@intel.com, jing2.liu@intel.com, ravi.v.shankar@intel.com, linux-kernel@vger.kernel.org, chang.seok.bae@intel.com, kvm@vger.kernel.org Subject: [RFC PATCH 06/22] x86/fpu/xstate: Outline dynamic xstate area size in the task context Date: Thu, 1 Oct 2020 13:38:57 -0700 Message-Id: <20201001203913.9125-7-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201001203913.9125-1-chang.seok.bae@intel.com> References: <20201001203913.9125-1-chang.seok.bae@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The xstate area size in task->fpu used to be fixed at runtime. To accommodate dynamic user states, introduce variables for representing the maximum and default (as minimum) area sizes. do_extra_xstate_size_checks() is ready to calculate both sizes, which can be compared with CPUID. CPUID can immediately provide the maximum size. The code needs to rewrite XCR0 registers to get the default size that excludes the dynamic parts. It is not always straightforward especially when inter-dependency exists between state component bits. To make it simple, the code double-checks the maximum size only. No functional change as long as the kernel does not support the dynamic area. Signed-off-by: Chang S. Bae Reviewed-by: Len Brown Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org --- arch/x86/include/asm/processor.h | 10 ++----- arch/x86/kernel/fpu/core.c | 6 ++-- arch/x86/kernel/fpu/init.c | 33 ++++++++++++---------- arch/x86/kernel/fpu/signal.c | 2 +- arch/x86/kernel/fpu/xstate.c | 48 +++++++++++++++++++++----------- arch/x86/kernel/process.c | 6 ++++ arch/x86/kvm/x86.c | 2 +- 7 files changed, 65 insertions(+), 42 deletions(-) diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h index 97143d87994c..f5f83aa1b90f 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -477,7 +477,8 @@ DECLARE_PER_CPU_ALIGNED(struct stack_canary, stack_canary); DECLARE_PER_CPU(struct irq_stack *, softirq_stack_ptr); #endif /* X86_64 */ -extern unsigned int fpu_kernel_xstate_size; +extern unsigned int fpu_kernel_xstate_default_size; +extern unsigned int fpu_kernel_xstate_max_size; extern unsigned int fpu_user_xstate_size; struct perf_event; @@ -551,12 +552,7 @@ struct thread_struct { }; /* Whitelist the FPU state from the task_struct for hardened usercopy. */ -static inline void arch_thread_struct_whitelist(unsigned long *offset, - unsigned long *size) -{ - *offset = offsetof(struct thread_struct, fpu.state); - *size = fpu_kernel_xstate_size; -} +extern void arch_thread_struct_whitelist(unsigned long *offset, unsigned long *size); /* * Thread-synchronous status. diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index 39ddb22c143b..875620fdfe61 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -206,7 +206,7 @@ void fpstate_init(struct fpu *fpu) return; } - memset(state, 0, fpu_kernel_xstate_size); + memset(state, 0, fpu_kernel_xstate_default_size); if (static_cpu_has(X86_FEATURE_XSAVES)) fpstate_init_xstate(&state->xsave, xfeatures_mask_all); @@ -233,7 +233,7 @@ int fpu__copy(struct task_struct *dst, struct task_struct *src) * Don't let 'init optimized' areas of the XSAVE area * leak into the child task: */ - memset(&dst_fpu->state.xsave, 0, fpu_kernel_xstate_size); + memset(&dst_fpu->state.xsave, 0, fpu_kernel_xstate_default_size); /* * If the FPU registers are not current just memcpy() the state. @@ -245,7 +245,7 @@ int fpu__copy(struct task_struct *dst, struct task_struct *src) */ fpregs_lock(); if (test_thread_flag(TIF_NEED_FPU_LOAD)) - memcpy(&dst_fpu->state, &src_fpu->state, fpu_kernel_xstate_size); + memcpy(&dst_fpu->state, &src_fpu->state, fpu_kernel_xstate_default_size); else if (!copy_fpregs_to_fpstate(dst_fpu)) copy_kernel_to_fpregs(dst_fpu); diff --git a/arch/x86/kernel/fpu/init.c b/arch/x86/kernel/fpu/init.c index 4e89a2698cfb..ee6499075a89 100644 --- a/arch/x86/kernel/fpu/init.c +++ b/arch/x86/kernel/fpu/init.c @@ -131,13 +131,17 @@ static void __init fpu__init_system_generic(void) } /* - * Size of the FPU context state. All tasks in the system use the - * same context size, regardless of what portion they use. - * This is inherent to the XSAVE architecture which puts all state - * components into a single, continuous memory block: + * Size of the maximum FPU context state. It is inherent to the XSAVE architecture + * which puts all state components into a single, continuous memory block: */ -unsigned int fpu_kernel_xstate_size; -EXPORT_SYMBOL_GPL(fpu_kernel_xstate_size); +unsigned int fpu_kernel_xstate_max_size; + +/* + * Size of the initial FPU context state. All tasks in the system use this context + * size by default. + */ +unsigned int fpu_kernel_xstate_default_size; +EXPORT_SYMBOL_GPL(fpu_kernel_xstate_default_size); /* Get alignment of the TYPE. */ #define TYPE_ALIGN(TYPE) offsetof(struct { char x; TYPE test; }, test) @@ -167,9 +171,9 @@ static void __init fpu__init_task_struct_size(void) /* * Add back the dynamically-calculated register state - * size. + * size by default. */ - task_size += fpu_kernel_xstate_size; + task_size += fpu_kernel_xstate_default_size; /* * We dynamically size 'struct fpu', so we require that @@ -194,6 +198,7 @@ static void __init fpu__init_task_struct_size(void) static void __init fpu__init_system_xstate_size_legacy(void) { static int on_boot_cpu __initdata = 1; + unsigned int size; WARN_ON_FPU(!on_boot_cpu); on_boot_cpu = 0; @@ -204,17 +209,17 @@ static void __init fpu__init_system_xstate_size_legacy(void) */ if (!boot_cpu_has(X86_FEATURE_FPU)) { - fpu_kernel_xstate_size = sizeof(struct swregs_state); + size = sizeof(struct swregs_state); } else { if (boot_cpu_has(X86_FEATURE_FXSR)) - fpu_kernel_xstate_size = - sizeof(struct fxregs_state); + size = sizeof(struct fxregs_state); else - fpu_kernel_xstate_size = - sizeof(struct fregs_state); + size = sizeof(struct fregs_state); } - fpu_user_xstate_size = fpu_kernel_xstate_size; + fpu_kernel_xstate_default_size = size; + fpu_kernel_xstate_max_size = size; + fpu_user_xstate_size = size; } /* diff --git a/arch/x86/kernel/fpu/signal.c b/arch/x86/kernel/fpu/signal.c index 6f3bcc7dab80..4fcd2caa63d3 100644 --- a/arch/x86/kernel/fpu/signal.c +++ b/arch/x86/kernel/fpu/signal.c @@ -289,8 +289,8 @@ static int copy_user_to_fpregs_zeroing(void __user *buf, u64 xbv, int fx_only) static int __fpu__restore_sig(void __user *buf, void __user *buf_fx, int size) { + int state_size = fpu_kernel_xstate_default_size; struct user_i387_ia32_struct *envp = NULL; - int state_size = fpu_kernel_xstate_size; int ia32_fxstate = (buf != buf_fx); struct task_struct *tsk = current; struct fpu *fpu = &tsk->thread.fpu; diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index bf2b09bf9b38..6e0d8a9699ed 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -623,13 +623,20 @@ static void check_xstate_against_struct(int nr) */ static void do_extra_xstate_size_checks(void) { - int paranoid_xstate_size = FXSAVE_SIZE + XSAVE_HDR_SIZE; + int paranoid_min_size, paranoid_max_size; int i; + paranoid_min_size = FXSAVE_SIZE + XSAVE_HDR_SIZE; + paranoid_max_size = paranoid_min_size; + for (i = FIRST_EXTENDED_XFEATURE; i < XFEATURE_MAX; i++) { + bool dynamic; + if (!xfeature_enabled(i)) continue; + dynamic = (xfeatures_mask_user_dynamic & BIT_ULL(i)) ? true : false; + check_xstate_against_struct(i); /* * Supervisor state components can be managed only by @@ -639,23 +646,32 @@ static void do_extra_xstate_size_checks(void) XSTATE_WARN_ON(xfeature_is_supervisor(i)); /* Align from the end of the previous feature */ - if (xfeature_is_aligned(i)) - paranoid_xstate_size = ALIGN(paranoid_xstate_size, 64); + if (xfeature_is_aligned(i)) { + paranoid_max_size = ALIGN(paranoid_max_size, 64); + if (!dynamic) + paranoid_min_size = ALIGN(paranoid_min_size, 64); + } /* * The offset of a given state in the non-compacted * format is given to us in a CPUID leaf. We check * them for being ordered (increasing offsets) in * setup_xstate_features(). */ - if (!using_compacted_format()) - paranoid_xstate_size = xfeature_uncompacted_offset(i); + if (!using_compacted_format()) { + paranoid_max_size = xfeature_uncompacted_offset(i); + if (!dynamic) + paranoid_min_size = xfeature_uncompacted_offset(i); + } /* * The compacted-format offset always depends on where * the previous state ended. */ - paranoid_xstate_size += xfeature_size(i); + paranoid_max_size += xfeature_size(i); + if (!dynamic) + paranoid_min_size += xfeature_size(i); } - XSTATE_WARN_ON(paranoid_xstate_size != fpu_kernel_xstate_size); + XSTATE_WARN_ON(paranoid_max_size != fpu_kernel_xstate_max_size); + fpu_kernel_xstate_default_size = paranoid_min_size; } @@ -740,27 +756,27 @@ static bool is_supported_xstate_size(unsigned int test_xstate_size) static int __init init_xstate_size(void) { /* Recompute the context size for enabled features: */ - unsigned int possible_xstate_size; + unsigned int possible_max_xstate_size; unsigned int xsave_size; xsave_size = get_xsave_size(); if (boot_cpu_has(X86_FEATURE_XSAVES)) - possible_xstate_size = get_xsaves_size_no_dynamic(); + possible_max_xstate_size = get_xsaves_size_no_dynamic(); else - possible_xstate_size = xsave_size; - - /* Ensure we have the space to store all enabled: */ - if (!is_supported_xstate_size(possible_xstate_size)) - return -EINVAL; + possible_max_xstate_size = xsave_size; /* * The size is OK, we are definitely going to use xsave, * make it known to the world that we need more space. */ - fpu_kernel_xstate_size = possible_xstate_size; + fpu_kernel_xstate_max_size = possible_max_xstate_size; do_extra_xstate_size_checks(); + /* Ensure we have the default space: */ + if (!is_supported_xstate_size(fpu_kernel_xstate_default_size)) + return -EINVAL; + /* * User space is always in standard format. */ @@ -865,7 +881,7 @@ void __init fpu__init_system_xstate(void) pr_info("x86/fpu: Enabled xstate features 0x%llx, context size is %d bytes, using '%s' format.\n", xfeatures_mask_all, - fpu_kernel_xstate_size, + fpu_kernel_xstate_max_size, boot_cpu_has(X86_FEATURE_XSAVES) ? "compacted" : "standard"); return; diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index ba4593a913fa..43d38bd09fb1 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -96,6 +96,12 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) return fpu__copy(dst, src); } +void arch_thread_struct_whitelist(unsigned long *offset, unsigned long *size) +{ + *offset = offsetof(struct thread_struct, fpu.state); + *size = fpu_kernel_xstate_default_size; +} + /* * Free thread data structures etc.. */ diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 192d52ff5b8c..ecec6418ccca 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8848,7 +8848,7 @@ static void kvm_save_current_fpu(struct fpu *fpu) */ if (test_thread_flag(TIF_NEED_FPU_LOAD)) memcpy(&fpu->state, ¤t->thread.fpu.state, - fpu_kernel_xstate_size); + fpu_kernel_xstate_default_size); else copy_fpregs_to_fpstate(fpu); } From patchwork Thu Oct 1 20:39:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chang S. Bae" X-Patchwork-Id: 11812399 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 794D16CB for ; Thu, 1 Oct 2020 20:43:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 63F7320848 for ; Thu, 1 Oct 2020 20:43:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733050AbgJAUnR (ORCPT ); Thu, 1 Oct 2020 16:43:17 -0400 Received: from mga11.intel.com ([192.55.52.93]:58716 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733062AbgJAUnR (ORCPT ); Thu, 1 Oct 2020 16:43:17 -0400 IronPort-SDR: hljPix7sDZsE5TIcZIfzIQROxurmP2vO8X3hkZFYzynvzazLOc++YncwmAyuEY7kpZEpc9CJgL kCmuCj1Wasjw== X-IronPort-AV: E=McAfee;i="6000,8403,9761"; a="160170722" X-IronPort-AV: E=Sophos;i="5.77,325,1596524400"; d="scan'208";a="160170722" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Oct 2020 13:42:52 -0700 IronPort-SDR: s99G8RnmpQ7YAa58hSoFTv4NZhVrvX0Mpopu3DBKEKM/fu03wUi90frHoMGh4pjyuAKbw7fXwB tgBZbSjbV2aw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,325,1596524400"; d="scan'208";a="351297056" Received: from chang-linux-3.sc.intel.com ([172.25.66.175]) by FMSMGA003.fm.intel.com with ESMTP; 01 Oct 2020 13:42:52 -0700 From: "Chang S. Bae" To: tglx@linutronix.de, mingo@kernel.org, bp@suse.de, luto@kernel.org, x86@kernel.org Cc: len.brown@intel.com, dave.hansen@intel.com, jing2.liu@intel.com, ravi.v.shankar@intel.com, linux-kernel@vger.kernel.org, chang.seok.bae@intel.com, kvm@vger.kernel.org Subject: [RFC PATCH 10/22] x86/fpu/xstate: Update xstate save function for supporting dynamic user xstate Date: Thu, 1 Oct 2020 13:39:01 -0700 Message-Id: <20201001203913.9125-11-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201001203913.9125-1-chang.seok.bae@intel.com> References: <20201001203913.9125-1-chang.seok.bae@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org copy_xregs_to_kernel() used to save all user states in an invariably sufficient buffer. When the dynamic user state is enabled, it becomes conditional which state to be saved. fpu->state_mask can indicate which state components are reserved to be saved in XSAVE buffer. Use it as XSAVE's instruction mask to select states. KVM saves xstate in guest_fpu and user_fpu. With the change, the KVM code needs to ensure a valid fpu->state_mask before XSAVE. No functional change until the kernel supports dynamic user states. Signed-off-by: Chang S. Bae Reviewed-by: Len Brown Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org --- arch/x86/include/asm/fpu/internal.h | 3 +-- arch/x86/kernel/fpu/core.c | 2 +- arch/x86/kvm/x86.c | 11 ++++++++--- 3 files changed, 10 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/fpu/internal.h b/arch/x86/include/asm/fpu/internal.h index 2dfb3b6f58fc..3b03ead87a46 100644 --- a/arch/x86/include/asm/fpu/internal.h +++ b/arch/x86/include/asm/fpu/internal.h @@ -331,9 +331,8 @@ static inline void copy_kernel_to_xregs_booting(struct xregs_state *xstate) /* * Save processor xstate to xsave area. */ -static inline void copy_xregs_to_kernel(struct xregs_state *xstate) +static inline void copy_xregs_to_kernel(struct xregs_state *xstate, u64 mask) { - u64 mask = xfeatures_mask_all; u32 lmask = mask; u32 hmask = mask >> 32; int err; diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index dca4961fcc36..ece6428ba85b 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -99,7 +99,7 @@ int copy_fpregs_to_fpstate(struct fpu *fpu) if (likely(use_xsave())) { struct xregs_state *xsave = &xstate->xsave; - copy_xregs_to_kernel(xsave); + copy_xregs_to_kernel(xsave, fpu->state_mask); /* * AVX512 state is tracked here because its use is diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ecec6418ccca..a8b5f507083c 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8842,15 +8842,20 @@ static int complete_emulated_mmio(struct kvm_vcpu *vcpu) static void kvm_save_current_fpu(struct fpu *fpu) { + struct fpu *src_fpu = ¤t->thread.fpu; + /* * If the target FPU state is not resident in the CPU registers, just * memcpy() from current, else save CPU state directly to the target. */ - if (test_thread_flag(TIF_NEED_FPU_LOAD)) - memcpy(&fpu->state, ¤t->thread.fpu.state, + if (test_thread_flag(TIF_NEED_FPU_LOAD)) { + memcpy(&fpu->state, &src_fpu->state, fpu_kernel_xstate_default_size); - else + } else { + if (fpu->state_mask != src_fpu->state_mask) + fpu->state_mask = src_fpu->state_mask; copy_fpregs_to_fpstate(fpu); + } } /* Swap (qemu) user FPU context for the guest FPU context. */