From patchwork Thu Oct 14 23:09:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12559587 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41876C433F5 for ; Thu, 14 Oct 2021 23:09:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1D62F610F8 for ; Thu, 14 Oct 2021 23:09:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234380AbhJNXLq (ORCPT ); Thu, 14 Oct 2021 19:11:46 -0400 Received: from Galois.linutronix.de ([193.142.43.55]:45622 "EHLO galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234326AbhJNXLh (ORCPT ); Thu, 14 Oct 2021 19:11:37 -0400 Message-ID: <20211014230739.126107370@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1634252970; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=QVaekX9bXV3d6X1DWoFzjhPj7QWVsMVJXuCognxNjoU=; b=AMsMNFqGhXraF4N1pYoEZhBNpOaOIApNa2/55yFVeTvbPHdjXcxmUx6LjX4A/6+qrIVfnB E6tMC5OCIFE5oYAuoUfAsqIitCqrRLWX5A/1SsnUsKuZt4qvyCDiDEMxExjXJ59QLCbF6N f4J5y5Cpxcb5r84aOJy/ML0gzIgXKHom/GwdafmWoEjX8Fo8CHbdPhxra2AoisazvQTYEN sNPEqJF+WWtuHjxye7DEIm/iL818fZ279Bhd5JTsdDVazaCUurjuVSqx9bsmYrQDM0RMtG 5Ncm8X2IQZNb1Pts1RIRMN5YElgktzK4qxoqNJ2BpvmBGdB9Igj+A97hW81pWw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1634252970; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=QVaekX9bXV3d6X1DWoFzjhPj7QWVsMVJXuCognxNjoU=; b=ZVwuDUAKPdx/TicDk1bOeye6fC4eTCFnvXKRw9WG6Ut589rPXm2u2wl98mCHvfx+s0aTs5 eiBf+zGdgickiSAw== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, "Chang S. Bae" , Dave Hansen , Arjan van de Ven , kvm@vger.kernel.org, Paolo Bonzini Subject: [patch 1/8] x86/fpu: Provide struct fpu_config References: <20211014225711.615197738@linutronix.de> MIME-Version: 1.0 Date: Fri, 15 Oct 2021 01:09:29 +0200 (CEST) Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Provide a struct to store information about the maximum supported and the default feature set and buffer sizes for both user and kernel space. This allows quick retrieval of this information for the upcoming support for dynamically enabled features. Signed-off-by: Thomas Gleixner --- arch/x86/include/asm/fpu/types.h | 39 +++++++++++++++++++++++++++++++++++++++ arch/x86/kernel/fpu/core.c | 4 ++++ 2 files changed, 43 insertions(+) --- a/arch/x86/include/asm/fpu/types.h +++ b/arch/x86/include/asm/fpu/types.h @@ -378,4 +378,43 @@ struct fpu { */ }; +/* + * FPU state configuration data. Initialized at boot time. Read only after init. + */ +struct fpu_state_config { + /* + * @max_size: + * + * The maximum size of the register state buffer. Includes all + * supported features except independent managed features. + */ + unsigned int max_size; + /* + * @default_size: + * + * The default size of the register state buffer. Includes all + * supported features except independent managed features and + * features which have to be requested by user space before usage. + */ + unsigned int default_size; + /* + * @max_features: + * + * The maximum supported features bitmap. Does not include + * independent managed features. + */ + u64 max_features; + /* + * @default_features: + * + * The default supported features bitmap. Does not include + * independent managed features and features which have to + * be requested by user space before usage. + */ + u64 default_features; +}; + +/* FPU state configuration information */ +extern struct fpu_state_config fpu_kernel_cfg, fpu_user_cfg; + #endif /* _ASM_X86_FPU_H */ --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -25,6 +25,10 @@ #define CREATE_TRACE_POINTS #include +/* The FPU state configuration data for kernel and user space */ +struct fpu_state_config fpu_kernel_cfg __ro_after_init; +struct fpu_state_config fpu_user_cfg __ro_after_init; + /* * Represents the initial FPU state. It's mostly (but not completely) zeroes, * depending on the FPU hardware format: From patchwork Thu Oct 14 23:09:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12559589 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59B2BC433FE for ; Thu, 14 Oct 2021 23:09:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 431F7610F8 for ; Thu, 14 Oct 2021 23:09:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234400AbhJNXLs (ORCPT ); Thu, 14 Oct 2021 19:11:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45276 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234338AbhJNXLi (ORCPT ); Thu, 14 Oct 2021 19:11:38 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 50A7DC061753; Thu, 14 Oct 2021 16:09:33 -0700 (PDT) Message-ID: <20211014230739.184014242@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1634252971; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=3waKzyY1T7PuMxjCNrXEk0Bp26hUU6HXuRX+FyaOY24=; b=Qn1Pkzn87g4uwvEYpSI7nYS57SA4ObeHiCQZ5y0kMmkvM1C2ApRv6/cnfB+owMlLVDdbU3 sY9BiZ1KyIrNbdRapRfyistRccNgREG8jcqs2PuHWAg2w/GYlpw6z66a+aUUNosYCBM/Ll roXHhaUVbvXDeMMzpvTqGCFVpbC9806ErwFFKCnbnETVia5d+LKwoE0+7e9fmjfgEaMmzy xXmrRV2oZVPRZwpipRDHp2eeIp/J4eTdyQ5JOTytEDr/Kh46DZx/DaKWq3QJWnHroEAUjo zDwWRdBtbd5hv60ml4Vgq/xEAgjEPkxDrVLld1zIxG3MdF//LIWvuC4bsA1F9A== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1634252971; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=3waKzyY1T7PuMxjCNrXEk0Bp26hUU6HXuRX+FyaOY24=; b=NoV8jzCW56WvEkPIoOlHQ1aOs4l8psUBuTtBL3Xt6hVJJIQB0SmOEEfbwtfKAfU35G2VL3 oj0eGj8p8aLE97BQ== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, "Chang S. Bae" , Dave Hansen , Arjan van de Ven , kvm@vger.kernel.org, Paolo Bonzini Subject: [patch 2/8] x86/fpu: Cleanup fpu__init_system_xstate_size_legacy() References: <20211014225711.615197738@linutronix.de> MIME-Version: 1.0 Date: Fri, 15 Oct 2021 01:09:31 +0200 (CEST) Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Clean the function up before making changes. No functional change. Signed-off-by: Thomas Gleixner --- arch/x86/kernel/fpu/init.c | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) --- a/arch/x86/kernel/fpu/init.c +++ b/arch/x86/kernel/fpu/init.c @@ -199,17 +199,12 @@ static void __init fpu__init_system_xsta * Note that xstate sizes might be overwritten later during * fpu__init_system_xstate(). */ - - if (!boot_cpu_has(X86_FEATURE_FPU)) { + if (!cpu_feature_enabled(X86_FEATURE_FPU)) fpu_kernel_xstate_size = sizeof(struct swregs_state); - } else { - if (boot_cpu_has(X86_FEATURE_FXSR)) - fpu_kernel_xstate_size = - sizeof(struct fxregs_state); - else - fpu_kernel_xstate_size = - sizeof(struct fregs_state); - } + else if (cpu_feature_enabled(X86_FEATURE_FXSR)) + fpu_kernel_xstate_size = sizeof(struct fxregs_state); + else + fpu_kernel_xstate_size = sizeof(struct fregs_state); fpu_user_xstate_size = fpu_kernel_xstate_size; fpstate_reset(¤t->thread.fpu); From patchwork Thu Oct 14 23:09:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12559591 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88167C433EF for ; Thu, 14 Oct 2021 23:09:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 71803610F8 for ; Thu, 14 Oct 2021 23:09:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234363AbhJNXLu (ORCPT ); Thu, 14 Oct 2021 19:11:50 -0400 Received: from Galois.linutronix.de ([193.142.43.55]:45642 "EHLO galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234340AbhJNXLj (ORCPT ); Thu, 14 Oct 2021 19:11:39 -0400 Message-ID: <20211014230739.241223689@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1634252973; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=yzWB5E4wnJvvvoyzC0APWr55A2ww99SfT84mV6LalVo=; b=WQ4jnPPcqxlCjfyrdXzypN7icKZH8Zm40c6vJOA11bNjIrl+ltECGtYgadMGmcmPvcNoYm oLdqMWmY9tuVqMg0ZfsGGoZZHmV8LNqFftThUhMqE79Oy1P4uVmV8uGMTAJj3tMvteT6rG bjJKuHhitB8umdM77aZ+fMczmfZCQWA4ejcQ2l0qP09/cKGQqUvZnQl1q7VIzzNiB43A/w 2zZztOMgjSfUGGUgsG1/V519zfL11JyUp9OiCDmRkkBH4sPS6aEAm9KbFOX3xT922J7SRg wBI6NrHLTZpoGaaWmkIAg6HKZ37lEfFwFNhyx2SP6kllbzQH8hnuc0GzG88oSg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1634252973; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=yzWB5E4wnJvvvoyzC0APWr55A2ww99SfT84mV6LalVo=; b=DCy/2I8lYGn8n31xnIsai6T0H0N5iTnvpXMmwzcLKsEXE2F3IkSR3853+axXRpxg/54PS7 p0oBVAagHv4WyABg== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, "Chang S. Bae" , Dave Hansen , Arjan van de Ven , kvm@vger.kernel.org, Paolo Bonzini Subject: [patch 3/8] x86/fpu/xstate: Cleanup size calculations References: <20211014225711.615197738@linutronix.de> MIME-Version: 1.0 Date: Fri, 15 Oct 2021 01:09:32 +0200 (CEST) Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The size calculations are partially unreadable gunk. Clean them up. No functional change. Signed-off-by: Thomas Gleixner --- arch/x86/kernel/fpu/xstate.c | 82 ++++++++++++++++++++++++------------------- 1 file changed, 46 insertions(+), 36 deletions(-) --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -527,7 +527,7 @@ static void __init __xstate_dump_leaves( * that our software representation matches what the CPU * tells us about the state's size. */ -static void __init check_xstate_against_struct(int nr) +static bool __init check_xstate_against_struct(int nr) { /* * Ask the CPU for the size of the state. @@ -557,7 +557,9 @@ static void __init check_xstate_against_ ((nr >= XFEATURE_RSRVD_COMP_11) && (nr <= XFEATURE_LBR))) { WARN_ONCE(1, "no structure for xstate: %d\n", nr); XSTATE_WARN_ON(1); + return false; } + return true; } /* @@ -569,38 +571,44 @@ static void __init check_xstate_against_ * covered by these checks. Only the size of the buffer for task->fpu * is checked here. */ -static void __init do_extra_xstate_size_checks(void) +static bool __init paranoid_xstate_size_valid(unsigned int kernel_size) { - int paranoid_xstate_size = FXSAVE_SIZE + XSAVE_HDR_SIZE; + bool compacted = cpu_feature_enabled(X86_FEATURE_XSAVES); + unsigned int size = FXSAVE_SIZE + XSAVE_HDR_SIZE; int i; for_each_extended_xfeature(i, xfeatures_mask_all) { - check_xstate_against_struct(i); + if (!check_xstate_against_struct(i)) + return false; /* * Supervisor state components can be managed only by * XSAVES. */ - if (!cpu_feature_enabled(X86_FEATURE_XSAVES)) - XSTATE_WARN_ON(xfeature_is_supervisor(i)); + if (!compacted && xfeature_is_supervisor(i)) { + XSTATE_WARN_ON(1); + return false; + } /* Align from the end of the previous feature */ if (xfeature_is_aligned(i)) - paranoid_xstate_size = ALIGN(paranoid_xstate_size, 64); + size = ALIGN(size, 64); /* - * The offset of a given state in the non-compacted - * format is given to us in a CPUID leaf. We check - * them for being ordered (increasing offsets) in - * setup_xstate_features(). XSAVES uses compacted format. + * In compacted format the enabled features are packed, + * i.e. disabled features do not occupy space. + * + * In non-compacted format the offsets are fixed and + * disabled states still occupy space in the memory buffer. */ - if (!cpu_feature_enabled(X86_FEATURE_XSAVES)) - paranoid_xstate_size = xfeature_uncompacted_offset(i); + if (!compacted) + size = xfeature_uncompacted_offset(i); /* - * The compacted-format offset always depends on where - * the previous state ended. + * Add the feature size even for non-compacted format + * to make the end result correct */ - paranoid_xstate_size += xfeature_size(i); + size += xfeature_size(i); } - XSTATE_WARN_ON(paranoid_xstate_size != fpu_kernel_xstate_size); + XSTATE_WARN_ON(size != kernel_size); + return size == kernel_size; } /* @@ -653,7 +661,7 @@ static unsigned int __init get_xsaves_si return size; } -static unsigned int __init get_xsave_size(void) +static unsigned int __init get_xsave_size_user(void) { unsigned int eax, ebx, ecx, edx; /* @@ -684,31 +692,33 @@ static bool __init is_supported_xstate_s static int __init init_xstate_size(void) { /* Recompute the context size for enabled features: */ - unsigned int possible_xstate_size; - unsigned int xsave_size; + unsigned int user_size, kernel_size; - xsave_size = get_xsave_size(); + /* Uncompacted user space size */ + user_size = get_xsave_size_user(); - if (boot_cpu_has(X86_FEATURE_XSAVES)) - possible_xstate_size = get_xsaves_size_no_independent(); + /* + * XSAVES kernel size includes supervisor states and + * uses compacted format. + * + * XSAVE does not support supervisor states so + * kernel and user size is identical. + */ + if (cpu_feature_enabled(X86_FEATURE_XSAVES)) + kernel_size = get_xsaves_size_no_independent(); else - possible_xstate_size = xsave_size; + kernel_size = user_size; - /* Ensure we have the space to store all enabled: */ - if (!is_supported_xstate_size(possible_xstate_size)) + /* Ensure we have the space to store all enabled features. */ + if (!is_supported_xstate_size(kernel_size)) return -EINVAL; - /* - * The size is OK, we are definitely going to use xsave, - * make it known to the world that we need more space. - */ - fpu_kernel_xstate_size = possible_xstate_size; - do_extra_xstate_size_checks(); + if (!paranoid_xstate_size_valid(kernel_size)) + return -EINVAL; + + fpu_kernel_xstate_size = kernel_size; + fpu_user_xstate_size = user_size; - /* - * User space is always in standard format. - */ - fpu_user_xstate_size = xsave_size; return 0; } From patchwork Thu Oct 14 23:09:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12559601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3AD4C433F5 for ; Thu, 14 Oct 2021 23:10:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BA91C60F11 for ; Thu, 14 Oct 2021 23:10:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234427AbhJNXMD (ORCPT ); Thu, 14 Oct 2021 19:12:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45294 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234351AbhJNXLm (ORCPT ); Thu, 14 Oct 2021 19:11:42 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7EC86C061753; Thu, 14 Oct 2021 16:09:36 -0700 (PDT) Message-ID: <20211014230739.296830097@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1634252974; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=3R+u38TW/8EzeINdAnoi+SOFILANDSR/nFwamEbZoiw=; b=q1aT23Hp/4MnDLRU1zpKhugWb3GQkiTBwXRNT5BIgZHb58fCOjE78KxWvJjZPrPdzhDCG0 n/HtRwAWN/6TTszB/MuNFyQJEuZZ32YkSABhl39yY2wd2JXRNHZiMfmlMuj4ExsdZ9j5xA DWBUTNRgjPP2yf0deMNoPXdK8VIqzeMqSwn79B/0zsUQlVVvlAX4zDZP6261JaI+wCj3J8 QufAtcGKRL+h2k8EkGZ2D4PxMrDXAoQIFKJkUYGCEQ8tC/G+iRJP1OyAci2Tk0QfGBAJw9 VFW1bGLOiBhJ0286L7DqgP4pHCyY6KcG4Vx8oGOly8eRM5eC/Ga2eQsM8NotkQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1634252974; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=3R+u38TW/8EzeINdAnoi+SOFILANDSR/nFwamEbZoiw=; b=BlPY2MTGoM4lGxl0NEvxSYbTZKJ/B8I4Z+ptt4AiU/leIUAyD2buYfVrElnaYTWGi5hlvL 4uFw1sAuC2CaudDg== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, "Chang S. Bae" , Dave Hansen , Arjan van de Ven , kvm@vger.kernel.org, Paolo Bonzini Subject: [patch 4/8] x86/fpu: Move xstate size to fpu_*_cfg References: <20211014225711.615197738@linutronix.de> MIME-Version: 1.0 Date: Fri, 15 Oct 2021 01:09:34 +0200 (CEST) Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Use the new kernel and user space config storage to store and retrieve the XSTATE buffer sizes. The default and the maximum size are the same for now, but will change when support for dynamically enabled features is added. Signed-off-by: Thomas Gleixner --- arch/x86/kernel/fpu/core.c | 8 ++++---- arch/x86/kernel/fpu/init.c | 31 ++++++++++++++----------------- arch/x86/kernel/fpu/internal.h | 2 -- arch/x86/kernel/fpu/regset.c | 2 +- arch/x86/kernel/fpu/signal.c | 6 +++--- arch/x86/kernel/fpu/xstate.c | 32 ++++++++++++++++++-------------- arch/x86/kernel/fpu/xstate.h | 2 +- 7 files changed, 41 insertions(+), 42 deletions(-) --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -298,7 +298,7 @@ void fpu_sync_fpstate(struct fpu *fpu) static inline unsigned int init_fpstate_copy_size(void) { if (!use_xsave()) - return fpu_kernel_xstate_size; + return fpu_kernel_cfg.default_size; /* XSAVE(S) just needs the legacy and the xstate header part */ return sizeof(init_fpstate.regs.xsave); @@ -347,8 +347,8 @@ void fpstate_reset(struct fpu *fpu) fpu->fpstate = &fpu->__fpstate; /* Initialize sizes and feature masks */ - fpu->fpstate->size = fpu_kernel_xstate_size; - fpu->fpstate->user_size = fpu_user_xstate_size; + fpu->fpstate->size = fpu_kernel_cfg.default_size; + fpu->fpstate->user_size = fpu_user_cfg.default_size; fpu->fpstate->xfeatures = xfeatures_mask_all; fpu->fpstate->user_xfeatures = xfeatures_mask_uabi(); } @@ -420,7 +420,7 @@ int fpu_clone(struct task_struct *dst) void fpu_thread_struct_whitelist(unsigned long *offset, unsigned long *size) { *offset = offsetof(struct thread_struct, fpu.__fpstate.regs); - *size = fpu_kernel_xstate_size; + *size = fpu_kernel_cfg.default_size; } /* --- a/arch/x86/kernel/fpu/init.c +++ b/arch/x86/kernel/fpu/init.c @@ -133,14 +133,6 @@ static void __init fpu__init_system_gene fpu__init_system_mxcsr(); } -/* - * Size of the FPU context state. All tasks in the system use the - * same context size, regardless of what portion they use. - * This is inherent to the XSAVE architecture which puts all state - * components into a single, continuous memory block: - */ -unsigned int fpu_kernel_xstate_size __ro_after_init; - /* Get alignment of the TYPE. */ #define TYPE_ALIGN(TYPE) offsetof(struct { char x; TYPE test; }, test) @@ -171,7 +163,7 @@ static void __init fpu__init_task_struct * Add back the dynamically-calculated register state * size. */ - task_size += fpu_kernel_xstate_size; + task_size += fpu_kernel_cfg.default_size; /* * We dynamically size 'struct fpu', so we require that @@ -195,25 +187,30 @@ static void __init fpu__init_task_struct */ static void __init fpu__init_system_xstate_size_legacy(void) { + unsigned int size; + /* - * Note that xstate sizes might be overwritten later during - * fpu__init_system_xstate(). + * Note that the size configuration might be overwritten later + * during fpu__init_system_xstate(). */ if (!cpu_feature_enabled(X86_FEATURE_FPU)) - fpu_kernel_xstate_size = sizeof(struct swregs_state); + size = sizeof(struct swregs_state); else if (cpu_feature_enabled(X86_FEATURE_FXSR)) - fpu_kernel_xstate_size = sizeof(struct fxregs_state); + size = sizeof(struct fxregs_state); else - fpu_kernel_xstate_size = sizeof(struct fregs_state); + size = sizeof(struct fregs_state); - fpu_user_xstate_size = fpu_kernel_xstate_size; + fpu_kernel_cfg.max_size = size; + fpu_kernel_cfg.default_size = size; + fpu_user_cfg.max_size = size; + fpu_user_cfg.default_size = size; fpstate_reset(¤t->thread.fpu); } static void __init fpu__init_init_fpstate(void) { /* Bring init_fpstate size and features up to date */ - init_fpstate.size = fpu_kernel_xstate_size; + init_fpstate.size = fpu_kernel_cfg.max_size; init_fpstate.xfeatures = xfeatures_mask_all; } @@ -234,7 +231,7 @@ void __init fpu__init_system(struct cpui fpu__init_system_generic(); fpu__init_system_xstate_size_legacy(); - fpu__init_system_xstate(); + fpu__init_system_xstate(fpu_kernel_cfg.max_size); fpu__init_task_struct_size(); fpu__init_init_fpstate(); } --- a/arch/x86/kernel/fpu/internal.h +++ b/arch/x86/kernel/fpu/internal.h @@ -2,8 +2,6 @@ #ifndef __X86_KERNEL_FPU_INTERNAL_H #define __X86_KERNEL_FPU_INTERNAL_H -extern unsigned int fpu_kernel_xstate_size; -extern unsigned int fpu_user_xstate_size; extern struct fpstate init_fpstate; /* CPU feature check wrappers */ --- a/arch/x86/kernel/fpu/regset.c +++ b/arch/x86/kernel/fpu/regset.c @@ -153,7 +153,7 @@ int xstateregs_set(struct task_struct *t /* * A whole standard-format XSAVE buffer is needed: */ - if (pos != 0 || count != fpu_user_xstate_size) + if (pos != 0 || count != fpu_user_cfg.max_size) return -EFAULT; if (!kbuf) { --- a/arch/x86/kernel/fpu/signal.c +++ b/arch/x86/kernel/fpu/signal.c @@ -503,7 +503,7 @@ fpu__alloc_mathframe(unsigned long sp, i unsigned long __init fpu__get_fpstate_size(void) { - unsigned long ret = fpu_user_xstate_size; + unsigned long ret = fpu_user_cfg.max_size; if (use_xsave()) ret += FP_XSTATE_MAGIC2_SIZE; @@ -531,12 +531,12 @@ unsigned long __init fpu__get_fpstate_si */ void __init fpu__init_prepare_fx_sw_frame(void) { - int size = fpu_user_xstate_size + FP_XSTATE_MAGIC2_SIZE; + int size = fpu_user_cfg.default_size + FP_XSTATE_MAGIC2_SIZE; fx_sw_reserved.magic1 = FP_XSTATE_MAGIC1; fx_sw_reserved.extended_size = size; fx_sw_reserved.xfeatures = xfeatures_mask_uabi(); - fx_sw_reserved.xstate_size = fpu_user_xstate_size; + fx_sw_reserved.xstate_size = fpu_user_cfg.default_size; if (IS_ENABLED(CONFIG_IA32_EMULATION) || IS_ENABLED(CONFIG_X86_32)) { --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -78,13 +78,6 @@ static unsigned int xstate_supervisor_on { [ 0 ... XFEATURE_MAX - 1] = -1}; /* - * The XSAVE area of kernel can be in standard or compacted format; - * it is always in standard format for user mode. This is the user - * mode standard format size used for signal and ptrace frames. - */ -unsigned int fpu_user_xstate_size __ro_after_init; - -/* * Return whether the system supports a given xfeature. * * Also return the name of the (most advanced) feature that the caller requested: @@ -716,8 +709,11 @@ static int __init init_xstate_size(void) if (!paranoid_xstate_size_valid(kernel_size)) return -EINVAL; - fpu_kernel_xstate_size = kernel_size; - fpu_user_xstate_size = user_size; + /* Keep it the same for now */ + fpu_kernel_cfg.max_size = kernel_size; + fpu_kernel_cfg.default_size = kernel_size; + fpu_user_cfg.max_size = user_size; + fpu_user_cfg.default_size = user_size; return 0; } @@ -726,11 +722,18 @@ static int __init init_xstate_size(void) * We enabled the XSAVE hardware, but something went wrong and * we can not use it. Disable it. */ -static void __init fpu__init_disable_system_xstate(void) +static void __init fpu__init_disable_system_xstate(unsigned int legacy_size) { xfeatures_mask_all = 0; cr4_clear_bits(X86_CR4_OSXSAVE); setup_clear_cpu_cap(X86_FEATURE_XSAVE); + + /* Restore the legacy size.*/ + fpu_kernel_cfg.max_size = legacy_size; + fpu_kernel_cfg.default_size = legacy_size; + fpu_user_cfg.max_size = legacy_size; + fpu_user_cfg.default_size = legacy_size; + fpstate_reset(¤t->thread.fpu); } @@ -738,7 +741,7 @@ static void __init fpu__init_disable_sys * Enable and initialize the xsave feature. * Called once per system bootup. */ -void __init fpu__init_system_xstate(void) +void __init fpu__init_system_xstate(unsigned int legacy_size) { unsigned int eax, ebx, ecx, edx; u64 xfeatures; @@ -810,7 +813,8 @@ void __init fpu__init_system_xstate(void * Update info used for ptrace frames; use standard-format size and no * supervisor xstates: */ - update_regset_xstate_info(fpu_user_xstate_size, xfeatures_mask_uabi()); + update_regset_xstate_info(fpu_user_cfg.max_size, + xfeatures_mask_uabi()); fpu__init_prepare_fx_sw_frame(); setup_init_fpu_buf(); @@ -830,13 +834,13 @@ void __init fpu__init_system_xstate(void print_xstate_offset_size(); pr_info("x86/fpu: Enabled xstate features 0x%llx, context size is %d bytes, using '%s' format.\n", xfeatures_mask_all, - fpu_kernel_xstate_size, + fpu_kernel_cfg.max_size, boot_cpu_has(X86_FEATURE_XSAVES) ? "compacted" : "standard"); return; out_disable: /* something went wrong, try to boot without any XSAVE support */ - fpu__init_disable_system_xstate(); + fpu__init_disable_system_xstate(legacy_size); } /* --- a/arch/x86/kernel/fpu/xstate.h +++ b/arch/x86/kernel/fpu/xstate.h @@ -31,7 +31,7 @@ extern int copy_sigframe_from_user_to_xs extern void fpu__init_cpu_xstate(void); -extern void fpu__init_system_xstate(void); +extern void fpu__init_system_xstate(unsigned int legacy_size); extern void *get_xsave_addr(struct xregs_state *xsave, int xfeature_nr); From patchwork Thu Oct 14 23:09:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12559595 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74303C433EF for ; Thu, 14 Oct 2021 23:09:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 588E260F11 for ; Thu, 14 Oct 2021 23:09:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234442AbhJNXL5 (ORCPT ); Thu, 14 Oct 2021 19:11:57 -0400 Received: from Galois.linutronix.de ([193.142.43.55]:45662 "EHLO galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234357AbhJNXLn (ORCPT ); Thu, 14 Oct 2021 19:11:43 -0400 Message-ID: <20211014230739.352041752@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1634252976; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=Iupuk7aJ0zvBpAgWac2JQmQ4S38b/G2vWCFaT6UOkWs=; b=GBIHX5QLT5q4ZSBDrNFhv3L3jUaEJrLuzG393ymJB0U6MDuFNDhY+/m8gmwmDrlsJ/imYO qD+XVmuB6CtAMlSq/aJnuEmwMo/mIAfRwL7vucmOroVa5jelxDLPk3t3f1vvr2tMSihcMA ecqgki+RTJHsjXdLqNjBAZDaCmPNXgp1iBpyPOMoIeykloEZD1Kjmgp4t8/odIgUa5k87R 3ODAmZ+5NN1yqCzHtfTeidB8JhboFnnopOEhpkPoWCQXlK6YpdVda68cdGCcvJaW6n2lH8 6RSzdvLQmIydlLZlJFLe583rXvNh+vclV37ObMcEkCKDRBF0MQGIkXHmsLVEwA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1634252976; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=Iupuk7aJ0zvBpAgWac2JQmQ4S38b/G2vWCFaT6UOkWs=; b=yZBUB9qaf5yFhJoePqd3E0ETPk36pclNuuQ95b7XTrCDkLoitu9XcPAiJwYFX6jNHBK69A Ye2G3TN+CUUPkIDg== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, "Chang S. Bae" , Dave Hansen , Arjan van de Ven , kvm@vger.kernel.org, Paolo Bonzini Subject: [patch 5/8] x86/fpu: Move xstate feature masks to fpu_*_cfg References: <20211014225711.615197738@linutronix.de> MIME-Version: 1.0 Date: Fri, 15 Oct 2021 01:09:35 +0200 (CEST) Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Move the feature mask storage to the kernel and user config structs. Default and maximum feature set are the same for now. Signed-off-by: Thomas Gleixner --- arch/x86/include/asm/fpu/xstate.h | 10 ++---- arch/x86/kernel/fpu/core.c | 4 +- arch/x86/kernel/fpu/init.c | 2 - arch/x86/kernel/fpu/signal.c | 3 +- arch/x86/kernel/fpu/xstate.c | 57 +++++++++++++++++++------------------- 5 files changed, 38 insertions(+), 38 deletions(-) --- a/arch/x86/include/asm/fpu/xstate.h +++ b/arch/x86/include/asm/fpu/xstate.h @@ -78,11 +78,9 @@ XFEATURE_MASK_INDEPENDENT | \ XFEATURE_MASK_SUPERVISOR_UNSUPPORTED) -extern u64 xfeatures_mask_all; - static inline u64 xfeatures_mask_supervisor(void) { - return xfeatures_mask_all & XFEATURE_MASK_SUPERVISOR_SUPPORTED; + return fpu_kernel_cfg.max_features & XFEATURE_MASK_SUPERVISOR_SUPPORTED; } /* @@ -91,7 +89,7 @@ static inline u64 xfeatures_mask_supervi */ static inline u64 xfeatures_mask_uabi(void) { - return xfeatures_mask_all & XFEATURE_MASK_USER_SUPPORTED; + return fpu_kernel_cfg.max_features & XFEATURE_MASK_USER_SUPPORTED; } /* @@ -102,7 +100,7 @@ static inline u64 xfeatures_mask_uabi(vo */ static inline u64 xfeatures_mask_restore_user(void) { - return xfeatures_mask_all & XFEATURE_MASK_USER_RESTORE; + return fpu_kernel_cfg.max_features & XFEATURE_MASK_USER_RESTORE; } /* @@ -111,7 +109,7 @@ static inline u64 xfeatures_mask_restore */ static inline u64 xfeatures_mask_fpstate(void) { - return xfeatures_mask_all & \ + return fpu_kernel_cfg.max_features & \ (XFEATURE_MASK_USER_RESTORE | XFEATURE_MASK_SUPERVISOR_SUPPORTED); } --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -337,8 +337,8 @@ void fpstate_reset(struct fpu *fpu) /* Initialize sizes and feature masks */ fpu->fpstate->size = fpu_kernel_cfg.default_size; fpu->fpstate->user_size = fpu_user_cfg.default_size; - fpu->fpstate->xfeatures = xfeatures_mask_all; - fpu->fpstate->user_xfeatures = xfeatures_mask_uabi(); + fpu->fpstate->xfeatures = fpu_kernel_cfg.default_features; + fpu->fpstate->user_xfeatures = fpu_user_cfg.default_features; } #if IS_ENABLED(CONFIG_KVM) --- a/arch/x86/kernel/fpu/init.c +++ b/arch/x86/kernel/fpu/init.c @@ -211,7 +211,7 @@ static void __init fpu__init_init_fpstat { /* Bring init_fpstate size and features up to date */ init_fpstate.size = fpu_kernel_cfg.max_size; - init_fpstate.xfeatures = xfeatures_mask_all; + init_fpstate.xfeatures = fpu_kernel_cfg.max_features; } /* --- a/arch/x86/kernel/fpu/signal.c +++ b/arch/x86/kernel/fpu/signal.c @@ -409,7 +409,8 @@ static bool __fpu_restore_sig(void __use u64 mask = user_xfeatures | xfeatures_mask_supervisor(); fpregs->xsave.header.xfeatures &= mask; - success = !os_xrstor_safe(&fpregs->xsave, xfeatures_mask_all); + success = !os_xrstor_safe(&fpregs->xsave, + fpu_kernel_cfg.max_features); } else { success = !fxrstor_safe(&fpregs->fxsave); } --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -62,12 +62,6 @@ static short xsave_cpuid_features[] __in X86_FEATURE_ENQCMD, }; -/* - * This represents the full set of bits that should ever be set in a kernel - * XSAVE buffer, both supervisor and user xstates. - */ -u64 xfeatures_mask_all __ro_after_init; - static unsigned int xstate_offsets[XFEATURE_MAX] __ro_after_init = { [ 0 ... XFEATURE_MAX - 1] = -1}; static unsigned int xstate_sizes[XFEATURE_MAX] __ro_after_init = @@ -84,7 +78,7 @@ static unsigned int xstate_supervisor_on */ int cpu_has_xfeatures(u64 xfeatures_needed, const char **feature_name) { - u64 xfeatures_missing = xfeatures_needed & ~xfeatures_mask_all; + u64 xfeatures_missing = xfeatures_needed & ~fpu_kernel_cfg.max_features; if (unlikely(feature_name)) { long xfeature_idx, max_idx; @@ -134,7 +128,7 @@ static bool xfeature_is_supervisor(int x */ void fpu__init_cpu_xstate(void) { - if (!boot_cpu_has(X86_FEATURE_XSAVE) || !xfeatures_mask_all) + if (!boot_cpu_has(X86_FEATURE_XSAVE) || !fpu_kernel_cfg.max_features) return; cr4_set_bits(X86_CR4_OSXSAVE); @@ -144,7 +138,7 @@ void fpu__init_cpu_xstate(void) * managed by XSAVE{C, OPT, S} and XRSTOR{S}. Only XSAVE user * states can be set here. */ - xsetbv(XCR_XFEATURE_ENABLED_MASK, xfeatures_mask_uabi()); + xsetbv(XCR_XFEATURE_ENABLED_MASK, fpu_user_cfg.max_features); /* * MSR_IA32_XSS sets supervisor states managed by XSAVES. @@ -157,7 +151,7 @@ void fpu__init_cpu_xstate(void) static bool xfeature_enabled(enum xfeature xfeature) { - return xfeatures_mask_all & BIT_ULL(xfeature); + return fpu_kernel_cfg.max_features & BIT_ULL(xfeature); } /* @@ -183,7 +177,7 @@ static void __init setup_xstate_features xstate_sizes[XFEATURE_SSE] = sizeof_field(struct fxregs_state, xmm_space); - for_each_extended_xfeature(i, xfeatures_mask_all) { + for_each_extended_xfeature(i, fpu_kernel_cfg.max_features) { cpuid_count(XSTATE_CPUID, i, &eax, &ebx, &ecx, &edx); xstate_sizes[i] = eax; @@ -288,14 +282,14 @@ static void __init setup_xstate_comp_off xmm_space); if (!cpu_feature_enabled(X86_FEATURE_XSAVES)) { - for_each_extended_xfeature(i, xfeatures_mask_all) + for_each_extended_xfeature(i, fpu_kernel_cfg.max_features) xstate_comp_offsets[i] = xstate_offsets[i]; return; } next_offset = FXSAVE_SIZE + XSAVE_HDR_SIZE; - for_each_extended_xfeature(i, xfeatures_mask_all) { + for_each_extended_xfeature(i, fpu_kernel_cfg.max_features) { if (xfeature_is_aligned(i)) next_offset = ALIGN(next_offset, 64); @@ -319,7 +313,7 @@ static void __init setup_supervisor_only next_offset = FXSAVE_SIZE + XSAVE_HDR_SIZE; - for_each_extended_xfeature(i, xfeatures_mask_all) { + for_each_extended_xfeature(i, fpu_kernel_cfg.max_features) { if (!xfeature_is_supervisor(i)) continue; @@ -338,7 +332,7 @@ static void __init print_xstate_offset_s { int i; - for_each_extended_xfeature(i, xfeatures_mask_all) { + for_each_extended_xfeature(i, fpu_kernel_cfg.max_features) { pr_info("x86/fpu: xstate_offset[%d]: %4d, xstate_sizes[%d]: %4d\n", i, xstate_comp_offsets[i], i, xstate_sizes[i]); } @@ -401,7 +395,7 @@ static void __init setup_init_fpu_buf(vo setup_xstate_features(); print_xstate_features(); - xstate_init_xcomp_bv(&init_fpstate.regs.xsave, xfeatures_mask_all); + xstate_init_xcomp_bv(&init_fpstate.regs.xsave, fpu_kernel_cfg.max_features); /* * Init all the features state with header.xfeatures being 0x0 @@ -570,7 +564,7 @@ static bool __init paranoid_xstate_size_ unsigned int size = FXSAVE_SIZE + XSAVE_HDR_SIZE; int i; - for_each_extended_xfeature(i, xfeatures_mask_all) { + for_each_extended_xfeature(i, fpu_kernel_cfg.max_features) { if (!check_xstate_against_struct(i)) return false; /* @@ -724,7 +718,7 @@ static int __init init_xstate_size(void) */ static void __init fpu__init_disable_system_xstate(unsigned int legacy_size) { - xfeatures_mask_all = 0; + fpu_kernel_cfg.max_features = 0; cr4_clear_bits(X86_CR4_OSXSAVE); setup_clear_cpu_cap(X86_FEATURE_XSAVE); @@ -768,13 +762,13 @@ void __init fpu__init_system_xstate(unsi * Find user xstates supported by the processor. */ cpuid_count(XSTATE_CPUID, 0, &eax, &ebx, &ecx, &edx); - xfeatures_mask_all = eax + ((u64)edx << 32); + fpu_kernel_cfg.max_features = eax + ((u64)edx << 32); /* * Find supervisor xstates supported by the processor. */ cpuid_count(XSTATE_CPUID, 1, &eax, &ebx, &ecx, &edx); - xfeatures_mask_all |= ecx + ((u64)edx << 32); + fpu_kernel_cfg.max_features |= ecx + ((u64)edx << 32); if ((xfeatures_mask_uabi() & XFEATURE_MASK_FPSSE) != XFEATURE_MASK_FPSSE) { /* @@ -783,7 +777,7 @@ void __init fpu__init_system_xstate(unsi * booting without it. This is too early to BUG(). */ pr_err("x86/fpu: FP/SSE not present amongst the CPU's xstate features: 0x%llx.\n", - xfeatures_mask_all); + fpu_kernel_cfg.max_features); goto out_disable; } @@ -792,14 +786,21 @@ void __init fpu__init_system_xstate(unsi */ for (i = 0; i < ARRAY_SIZE(xsave_cpuid_features); i++) { if (!boot_cpu_has(xsave_cpuid_features[i])) - xfeatures_mask_all &= ~BIT_ULL(i); + fpu_kernel_cfg.max_features &= ~BIT_ULL(i); } - xfeatures_mask_all &= XFEATURE_MASK_USER_SUPPORTED | + fpu_kernel_cfg.max_features &= XFEATURE_MASK_USER_SUPPORTED | XFEATURE_MASK_SUPERVISOR_SUPPORTED; + fpu_user_cfg.max_features = fpu_kernel_cfg.max_features; + fpu_user_cfg.max_features &= XFEATURE_MASK_USER_SUPPORTED; + + /* Identical for now */ + fpu_kernel_cfg.default_features = fpu_kernel_cfg.max_features; + fpu_user_cfg.default_features = fpu_user_cfg.max_features; + /* Store it for paranoia check at the end */ - xfeatures = xfeatures_mask_all; + xfeatures = fpu_kernel_cfg.max_features; /* Enable xstate instructions to be able to continue with initialization: */ fpu__init_cpu_xstate(); @@ -825,15 +826,15 @@ void __init fpu__init_system_xstate(unsi * Paranoia check whether something in the setup modified the * xfeatures mask. */ - if (xfeatures != xfeatures_mask_all) { + if (xfeatures != fpu_kernel_cfg.max_features) { pr_err("x86/fpu: xfeatures modified from 0x%016llx to 0x%016llx during init, disabling XSAVE\n", - xfeatures, xfeatures_mask_all); + xfeatures, fpu_kernel_cfg.max_features); goto out_disable; } print_xstate_offset_size(); pr_info("x86/fpu: Enabled xstate features 0x%llx, context size is %d bytes, using '%s' format.\n", - xfeatures_mask_all, + fpu_kernel_cfg.max_features, fpu_kernel_cfg.max_size, boot_cpu_has(X86_FEATURE_XSAVES) ? "compacted" : "standard"); return; @@ -908,7 +909,7 @@ void *get_xsave_addr(struct xregs_state * We should not ever be requesting features that we * have not enabled. */ - WARN_ONCE(!(xfeatures_mask_all & BIT_ULL(xfeature_nr)), + WARN_ONCE(!(fpu_kernel_cfg.max_features & BIT_ULL(xfeature_nr)), "get of unsupported state"); /* * This assumes the last 'xsave*' instruction to From patchwork Thu Oct 14 23:09:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12559593 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7F85C433EF for ; Thu, 14 Oct 2021 23:09:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8E1DC60F11 for ; Thu, 14 Oct 2021 23:09:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234352AbhJNXLv (ORCPT ); Thu, 14 Oct 2021 19:11:51 -0400 Received: from Galois.linutronix.de ([193.142.43.55]:45678 "EHLO galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234364AbhJNXLo (ORCPT ); Thu, 14 Oct 2021 19:11:44 -0400 Message-ID: <20211014230739.408879849@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1634252978; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=bAQJJerKDNICmvVOHamcMxaejexvNO2x+7HSc41vSD8=; b=Aycrphoa7Oc2AP93wNyT62FD1jbbwnzzR/dQehyzuO/HUs97ohLxqdtxVeohGbtm0Pq1Hx 3K+k3EpKkTlqGGEAtQE4hl4tfy3vq98TWTPdvXfeWT4XHEngnW9jJgQ0WXdNefFO01A+fd T6tVY04Qqx9ffJYyhhOqRj109ZIEFzELtboBdgLRkJ1OJzTlAfdRTXbP3tKcF3h/Zrm/9s xJMvLjAUB2ZzNq3ay5ilWxmroAUxfehHddiZSatyg9pLuDCHyf9nBNDtQ52/rPK1FYUPtB J9YYDHP/OVVav4u4jr9DQhUalbeq9dnwpM2qSTLafB03nMi+WoZCRmiu1wgiiQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1634252978; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=bAQJJerKDNICmvVOHamcMxaejexvNO2x+7HSc41vSD8=; b=mGy95CLvebg78ULpga21g2NFrGZoxCTMpMaLvFVhwRcxmmFj89R/8+yqIhIrfXnt91+4Rd rLPWzV8LPgzscjAw== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, "Chang S. Bae" , Dave Hansen , Arjan van de Ven , kvm@vger.kernel.org, Paolo Bonzini Subject: [patch 6/8] x86/fpu: Mop up xfeatures_mask_uabi() References: <20211014225711.615197738@linutronix.de> MIME-Version: 1.0 Date: Fri, 15 Oct 2021 01:09:37 +0200 (CEST) Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Use the new fpu_user_cfg to retrieve the information instead of xfeatures_mask_uabi() which will be not longer correct when dynamically enabled features become available. Signed-off-by: Thomas Gleixner --- arch/x86/include/asm/fpu/xstate.h | 9 --------- arch/x86/kernel/fpu/core.c | 4 ++-- arch/x86/kernel/fpu/signal.c | 2 +- arch/x86/kernel/fpu/xstate.c | 6 +++--- 4 files changed, 6 insertions(+), 15 deletions(-) --- a/arch/x86/include/asm/fpu/xstate.h +++ b/arch/x86/include/asm/fpu/xstate.h @@ -84,15 +84,6 @@ static inline u64 xfeatures_mask_supervi } /* - * The xfeatures which are enabled in XCR0 and expected to be in ptrace - * buffers and signal frames. - */ -static inline u64 xfeatures_mask_uabi(void) -{ - return fpu_kernel_cfg.max_features & XFEATURE_MASK_USER_SUPPORTED; -} - -/* * The xfeatures which are restored by the kernel when returning to user * mode. This is not necessarily the same as xfeatures_mask_uabi() as the * kernel does not manage all XCR0 enabled features via xsave/xrstor as --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -237,7 +237,7 @@ int fpu_copy_kvm_uabi_to_fpstate(struct } /* Ensure that XCOMP_BV is set up for XSAVES */ - xstate_init_xcomp_bv(&kstate->regs.xsave, xfeatures_mask_uabi()); + xstate_init_xcomp_bv(&kstate->regs.xsave, fpu_user_cfg.max_features); return 0; } EXPORT_SYMBOL_GPL(fpu_copy_kvm_uabi_to_fpstate); @@ -333,7 +333,7 @@ void fpstate_init_user(struct fpstate *f return; } - xstate_init_xcomp_bv(&fpstate->regs.xsave, xfeatures_mask_uabi()); + xstate_init_xcomp_bv(&fpstate->regs.xsave, fpu_user_cfg.max_features); if (cpu_feature_enabled(X86_FEATURE_FXSR)) fpstate_init_fxstate(fpstate); --- a/arch/x86/kernel/fpu/signal.c +++ b/arch/x86/kernel/fpu/signal.c @@ -530,7 +530,7 @@ void __init fpu__init_prepare_fx_sw_fram fx_sw_reserved.magic1 = FP_XSTATE_MAGIC1; fx_sw_reserved.extended_size = size; - fx_sw_reserved.xfeatures = xfeatures_mask_uabi(); + fx_sw_reserved.xfeatures = fpu_user_cfg.default_features; fx_sw_reserved.xstate_size = fpu_user_cfg.default_size; if (IS_ENABLED(CONFIG_IA32_EMULATION) || --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -770,7 +770,7 @@ void __init fpu__init_system_xstate(unsi cpuid_count(XSTATE_CPUID, 1, &eax, &ebx, &ecx, &edx); fpu_kernel_cfg.max_features |= ecx + ((u64)edx << 32); - if ((xfeatures_mask_uabi() & XFEATURE_MASK_FPSSE) != XFEATURE_MASK_FPSSE) { + if ((fpu_kernel_cfg.max_features & XFEATURE_MASK_FPSSE) != XFEATURE_MASK_FPSSE) { /* * This indicates that something really unexpected happened * with the enumeration. Disable XSAVE and try to continue @@ -815,7 +815,7 @@ void __init fpu__init_system_xstate(unsi * supervisor xstates: */ update_regset_xstate_info(fpu_user_cfg.max_size, - xfeatures_mask_uabi()); + fpu_user_cfg.max_features); fpu__init_prepare_fx_sw_frame(); setup_init_fpu_buf(); @@ -853,7 +853,7 @@ void fpu__resume_cpu(void) * Restore XCR0 on xsave capable CPUs: */ if (cpu_feature_enabled(X86_FEATURE_XSAVE)) - xsetbv(XCR_XFEATURE_ENABLED_MASK, xfeatures_mask_uabi()); + xsetbv(XCR_XFEATURE_ENABLED_MASK, fpu_user_cfg.max_features); /* * Restore IA32_XSS. The same CPUID bit enumerates support From patchwork Thu Oct 14 23:09:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12559599 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBDDBC433EF for ; Thu, 14 Oct 2021 23:09:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C1F5561130 for ; Thu, 14 Oct 2021 23:09:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234340AbhJNXMB (ORCPT ); Thu, 14 Oct 2021 19:12:01 -0400 Received: from Galois.linutronix.de ([193.142.43.55]:45684 "EHLO galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234379AbhJNXLq (ORCPT ); Thu, 14 Oct 2021 19:11:46 -0400 Message-ID: <20211014230739.461348278@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1634252979; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=Rzw02zaGEt/Tb3o44nL3X5P6XPRDUNaCla4lCE+NSiE=; b=Qvuwjt6GZtJ6FQq5xgvEY3vmyt/PZlSGUadBl8HDVVDyDTxBFh2tVv0I0021oDT7B8QHEb JCT/CPMeo5ZloRBvdO0OjeSiSZ/v1kxlAt4u7TDHMXcgvV/jQJq4e3KUUAHJ7LL5AZQwMG egDknicIK2HklK/qM1Whykm0koFY4Qd0a4Pok7YQLOFKdJ61DnbnFBw9aPafIMT2zFz905 TmY7vKQRJtONvW4YTiEN5ILRBKco8Xpt7Lx6r68AWidqESqHK1vhpmlEoK239rX+vM1ThN edN4uBoAoinVDuMiU3t3IDecFX6k6edCsxyKfidnSo+eLcj7F+SEMRxV2YXMOA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1634252979; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=Rzw02zaGEt/Tb3o44nL3X5P6XPRDUNaCla4lCE+NSiE=; b=uuRjY1d8PUEOIOUW664gniOsULptlv5Pgh7bv5sWzttc51kokkUEakPb8XkjhwhJE8SQUw bvtNB56dLmflIRAA== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, "Chang S. Bae" , Dave Hansen , Arjan van de Ven , kvm@vger.kernel.org, Paolo Bonzini Subject: [patch 7/8] x86/fpu: Rework restore_regs_from_fpstate() References: <20211014225711.615197738@linutronix.de> MIME-Version: 1.0 Date: Fri, 15 Oct 2021 01:09:38 +0200 (CEST) Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org xfeatures_mask_fpstate() is not longer valid when dynamically enabled features come into play. Rework restore_regs_from_fpstate() so it takes a constant mask which will then be applied against the maximum feature set so that the restore operation brings all features which are not in the xsave buffer xfeature bitmap into init state. This ensures that if the previous task used a dynamically enabled feature that the task which restores has all unused components properly initialized. Cleanup the last user of xfeatures_mask_fpstate() as well and remove it. Signed-off-by: Thomas Gleixner --- arch/x86/include/asm/fpu/xstate.h | 27 +++++++-------------------- arch/x86/kernel/fpu/context.h | 6 +----- arch/x86/kernel/fpu/core.c | 17 ++++++++++++++--- arch/x86/kernel/fpu/xstate.c | 2 +- 4 files changed, 23 insertions(+), 29 deletions(-) --- a/arch/x86/include/asm/fpu/xstate.h +++ b/arch/x86/include/asm/fpu/xstate.h @@ -78,30 +78,17 @@ XFEATURE_MASK_INDEPENDENT | \ XFEATURE_MASK_SUPERVISOR_UNSUPPORTED) -static inline u64 xfeatures_mask_supervisor(void) -{ - return fpu_kernel_cfg.max_features & XFEATURE_MASK_SUPERVISOR_SUPPORTED; -} - /* - * The xfeatures which are restored by the kernel when returning to user - * mode. This is not necessarily the same as xfeatures_mask_uabi() as the - * kernel does not manage all XCR0 enabled features via xsave/xrstor as - * some of them have to be switched eagerly on context switch and exec(). + * The feature mask required to restore FPU state: + * - All user states which are not eagerly switched in switch_to()/exec() + * - The suporvisor states */ -static inline u64 xfeatures_mask_restore_user(void) -{ - return fpu_kernel_cfg.max_features & XFEATURE_MASK_USER_RESTORE; -} +#define XFEATURE_MASK_FPSTATE (XFEATURE_MASK_USER_RESTORE | \ + XFEATURE_MASK_SUPERVISOR_SUPPORTED) -/* - * Like xfeatures_mask_restore_user() but additionally restors the - * supported supervisor states. - */ -static inline u64 xfeatures_mask_fpstate(void) +static inline u64 xfeatures_mask_supervisor(void) { - return fpu_kernel_cfg.max_features & \ - (XFEATURE_MASK_USER_RESTORE | XFEATURE_MASK_SUPERVISOR_SUPPORTED); + return fpu_kernel_cfg.max_features & XFEATURE_MASK_SUPERVISOR_SUPPORTED; } static inline u64 xfeatures_mask_independent(void) --- a/arch/x86/kernel/fpu/context.h +++ b/arch/x86/kernel/fpu/context.h @@ -61,8 +61,6 @@ static inline void fpregs_restore_userre return; if (!fpregs_state_valid(fpu, cpu)) { - u64 mask; - /* * This restores _all_ xstate which has not been * established yet. @@ -72,9 +70,7 @@ static inline void fpregs_restore_userre * flush_thread(). So it is excluded because it might be * not up to date in current->thread.fpu.xsave state. */ - mask = xfeatures_mask_restore_user() | - xfeatures_mask_supervisor(); - restore_fpregs_from_fpstate(fpu->fpstate, mask); + restore_fpregs_from_fpstate(fpu->fpstate, XFEATURE_MASK_FPSTATE); fpregs_activate(fpu); fpu->last_cpu = cpu; --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -150,6 +150,17 @@ void restore_fpregs_from_fpstate(struct } if (use_xsave()) { + /* + * Restoring state always needs to modify all features + * which are in @mask even if the current task cannot use + * extended features. + * + * So fpstate->xfeatures cannot be used here, because then + * a feature for which the task has no permission but was + * used by the previous task would not go into init state. + */ + mask = fpu_kernel_cfg.max_features & mask; + os_xrstor(&fpstate->regs.xsave, mask); } else { if (use_fxsr()) @@ -161,7 +172,7 @@ void restore_fpregs_from_fpstate(struct void fpu_reset_from_exception_fixup(void) { - restore_fpregs_from_fpstate(&init_fpstate, xfeatures_mask_fpstate()); + restore_fpregs_from_fpstate(&init_fpstate, XFEATURE_MASK_FPSTATE); } #if IS_ENABLED(CONFIG_KVM) @@ -179,7 +190,7 @@ void fpu_swap_kvm_fpu(struct fpu *save, } if (rstor) { - restore_mask &= xfeatures_mask_fpstate(); + restore_mask &= XFEATURE_MASK_FPSTATE; restore_fpregs_from_fpstate(rstor->fpstate, restore_mask); } @@ -499,7 +510,7 @@ void fpu__clear_user_states(struct fpu * } /* Reset user states in registers. */ - restore_fpregs_from_init_fpstate(xfeatures_mask_restore_user()); + restore_fpregs_from_init_fpstate(XFEATURE_MASK_USER_RESTORE); /* * Now all FPU registers have their desired values. Inform the FPU --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -344,7 +344,7 @@ static void __init print_xstate_offset_s */ static __init void os_xrstor_booting(struct xregs_state *xstate) { - u64 mask = xfeatures_mask_fpstate(); + u64 mask = fpu_kernel_cfg.max_features & XFEATURE_MASK_FPSTATE; u32 lmask = mask; u32 hmask = mask >> 32; int err; From patchwork Thu Oct 14 23:09:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12559597 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56C83C433FE for ; Thu, 14 Oct 2021 23:09:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3953A61108 for ; Thu, 14 Oct 2021 23:09:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234392AbhJNXL7 (ORCPT ); Thu, 14 Oct 2021 19:11:59 -0400 Received: from Galois.linutronix.de ([193.142.43.55]:45642 "EHLO galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234384AbhJNXLr (ORCPT ); Thu, 14 Oct 2021 19:11:47 -0400 Message-ID: <20211014230739.514095101@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1634252981; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=zoeOGaSz/EfV2jZB8yk8D3VJZjEJF1rVrAGB3lz2cKU=; b=c63bRbuv/KiP8xcNrCe99L+qMtrMKmd+Ts7jDnfPUC/6Arc3UVnOPz7PwEXDIDyP6fwbPi 0bwf4Ky26/xjzSQPXCdaFuxP87L4DMMfRNf2y35v93mODP9x6BABRDwjAn/zT2dVVH9oqY hXGPDpId2sD6KWEe8XCieTAlvGrfxYiLH+ntQ0OKtoRSCvORVHPj7DU8r/GjxrtX2wM7HR lNeGwd0oUGgYPrKERPxzL+v1sz1JVJKu2aInSDECSDwzWylQHm8a++bQd4112Apk6Peqnc oWioqjLd8SUX5kiWn8XSL7nl34vd1fuHOVAnKfjKjrR2vHIfCtLSjPw49WaeGg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1634252981; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=zoeOGaSz/EfV2jZB8yk8D3VJZjEJF1rVrAGB3lz2cKU=; b=DLwQULNnOlJuTnDHSnYzUbR6bWaB5T5NndOSL3ea6yP7pfRQtGscz/EmwvDIyLm2yycAkd iC2CrzayVjVAgsDA== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, "Chang S. Bae" , Dave Hansen , Arjan van de Ven , kvm@vger.kernel.org, Paolo Bonzini Subject: [patch 8/8] x86/fpu/xstate: Move remaining xfeature helpers to core References: <20211014225711.615197738@linutronix.de> MIME-Version: 1.0 Date: Fri, 15 Oct 2021 01:09:40 +0200 (CEST) Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Now that everything is mopped up, move all the helpers and prototypes into the core header. They are not required by the outside. Signed-off-by: Thomas Gleixner --- arch/x86/include/asm/fpu/xstate.h | 13 ------------- arch/x86/kernel/fpu/xstate.h | 13 +++++++++++++ 2 files changed, 13 insertions(+), 13 deletions(-) --- a/arch/x86/include/asm/fpu/xstate.h +++ b/arch/x86/include/asm/fpu/xstate.h @@ -86,19 +86,6 @@ #define XFEATURE_MASK_FPSTATE (XFEATURE_MASK_USER_RESTORE | \ XFEATURE_MASK_SUPERVISOR_SUPPORTED) -static inline u64 xfeatures_mask_supervisor(void) -{ - return fpu_kernel_cfg.max_features & XFEATURE_MASK_SUPERVISOR_SUPPORTED; -} - -static inline u64 xfeatures_mask_independent(void) -{ - if (!boot_cpu_has(X86_FEATURE_ARCH_LBR)) - return XFEATURE_MASK_INDEPENDENT & ~XFEATURE_MASK_LBR; - - return XFEATURE_MASK_INDEPENDENT; -} - extern u64 xstate_fx_sw_bytes[USER_XSTATE_FX_SW_WORDS]; extern void __init update_regset_xstate_info(unsigned int size, --- a/arch/x86/kernel/fpu/xstate.h +++ b/arch/x86/kernel/fpu/xstate.h @@ -35,6 +35,19 @@ extern void fpu__init_system_xstate(unsi extern void *get_xsave_addr(struct xregs_state *xsave, int xfeature_nr); +static inline u64 xfeatures_mask_supervisor(void) +{ + return fpu_kernel_cfg.max_features & XFEATURE_MASK_SUPERVISOR_SUPPORTED; +} + +static inline u64 xfeatures_mask_independent(void) +{ + if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR)) + return XFEATURE_MASK_INDEPENDENT & ~XFEATURE_MASK_LBR; + + return XFEATURE_MASK_INDEPENDENT; +} + /* XSAVE/XRSTOR wrapper functions */ #ifdef CONFIG_X86_64