From patchwork Wed Jun 5 11:41:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13686727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A6E0AC25B76 for ; Wed, 5 Jun 2024 11:50:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wseIEfTyuev/m0ML5A5GwwEwvwK7NfsmdD83ezd9KuU=; b=rvwdfcrDKNF2iI q6nekCrRKZlwx7NdJ1egnGuXsIi6CzFyda72/6st+jEsd4gjOyKXXhW4wnxUzjdLbwc0mdXWn5bCB LO9GGMURBKQ5siwUEVI0r/vW/XWz7BVqsHd/RulbJ4Qmt74LWLJNEJcF9LDg7iFEI3lc6buWBrsGP g7uwPfhidWGxreR6QM9PZTQThO8xE/NYeN+1bb3/NvRM3+a4D87Pi3p/E9su1L+dVASMiwFQUM/RE u0MyAIyP3cqL6iIt0ntdYHG43vONO94qTtlYzPWBe/gZq/vaEMK/nX4ud9zAfZXfbSnXGVEzxYlA8 0598X0Cs6sjn+ki8Eh2Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEp9f-00000005mV9-2dqK; Wed, 05 Jun 2024 11:50:11 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEp9c-00000005mRr-0dIR for linux-arm-kernel@lists.infradead.org; Wed, 05 Jun 2024 11:50:09 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 748FE6187E; Wed, 5 Jun 2024 11:50:07 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 01F53C4AF09; Wed, 5 Jun 2024 11:50:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1717588207; bh=Aa17kOoXpX/HoXHXzofiE99MuhIDY6p0rI+xW1GTtrQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=rVCBHKWhHA237EB6RPwpSILVDLYPNE8KqNzQsXy07/6G4MO60f7LdDvSId8e3ewyB sCb7rF95UCszxwUW6VUZhAGczzdeUQymomZohA7uaO3jo3CBpZ/PBO5j2YlxbDbNed ezgjrg3H++OknfISav5F19mA+IklAr8wStlaRHGzBXnokhHbg9MQJwhma7tfWXom47 UMJ57499EtmomkSD0euqqYZ7Ry7kcDuQvkvf5u6dFrAzlVUT7d+G/06afLdT6J6ql9 1YiEiV/+kYSDprMf2qUIAU51c/U8VTnjU/fMFj3pR6LVCYepfxOkA3YZsQ7c9D5hLV FBkDxotBeVopg== From: Mark Brown Date: Wed, 05 Jun 2024 12:41:27 +0100 Subject: [PATCH 1/4] arm64/fpsimd: Introduce __bit_to_vl() helper MIME-Version: 1.0 Message-Id: <20240605-kvm-arm64-fix-pkvm-sve-vl-v1-1-680d6b43b4c1@kernel.org> References: <20240605-kvm-arm64-fix-pkvm-sve-vl-v1-0-680d6b43b4c1@kernel.org> In-Reply-To: <20240605-kvm-arm64-fix-pkvm-sve-vl-v1-0-680d6b43b4c1@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.14-dev-d4707 X-Developer-Signature: v=1; a=openpgp-sha256; l=3011; i=broonie@kernel.org; h=from:subject:message-id; bh=Aa17kOoXpX/HoXHXzofiE99MuhIDY6p0rI+xW1GTtrQ=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmYFDmUQYv4GiElk8lqZoB3sQBV0ApYQH/AfmNeizO 1sflt5KJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZmBQ5gAKCRAk1otyXVSH0HqICA CDvBoq6AwiNa+cnUxr9ztBR/u4A5iQIk71gQXuIzvTwZTDP9PwOfqkmq/W/xeYVHxnJNJc5NVRJ2p8 dhrOBC3DxAlnEFw/pLhSxU6XsARczaz3CnNE3X6ulv9PypAG6sOy+auofHZPMKS9dfvnVdskiZ60Hs 1NeDFb+OlRuqY5+2LCYtkDWWZDE7UyWHuedBoK+KJZ/SGJ5VpLWG7Z0WNihx4bR4NRjBdSwELH63eT 0wqlTPMOd3TxdzTSL8M5nyX3QUR4tEW0ApLJyDAxtMdUa/1xDV6Ada/mZjQnwNFKpbL+w/O+q85KMJ B7PhUoXwIlZzexuq4cAtwl1hQzM9lP X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240605_045008_362562_482F8F20 X-CRM114-Status: GOOD ( 13.54 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In all cases where we use the existing __bit_to_vq() helper we immediately convert the result into a VL. Provide and use __bit_to_vl() doing this directly. Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 4 ++++ arch/arm64/kernel/fpsimd.c | 12 ++++++------ 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index bc69ac368d73..51c21265b4fa 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -172,6 +172,10 @@ static inline unsigned int __bit_to_vq(unsigned int bit) return SVE_VQ_MAX - bit; } +static inline unsigned int __bit_to_vl(unsigned int bit) +{ + return sve_vl_from_vq(__bit_to_vq(bit)); +} struct vl_info { enum vec_type type; diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 82e8a6017382..22542fb81812 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -530,7 +530,7 @@ static unsigned int find_supported_vector_length(enum vec_type type, bit = find_next_bit(info->vq_map, SVE_VQ_MAX, __vq_to_bit(sve_vq_from_vl(vl))); - return sve_vl_from_vq(__bit_to_vq(bit)); + return __bit_to_vl(bit); } #if defined(CONFIG_ARM64_SVE) && defined(CONFIG_SYSCTL) @@ -1103,7 +1103,7 @@ int vec_verify_vq_map(enum vec_type type) * Mismatches above sve_max_virtualisable_vl are fine, since * no guest is allowed to configure ZCR_EL2.LEN to exceed this: */ - if (sve_vl_from_vq(__bit_to_vq(b)) <= info->max_virtualisable_vl) { + if (__bit_to_vl(b) <= info->max_virtualisable_vl) { pr_warn("%s: cpu%d: Unsupported vector length(s) present\n", info->name, smp_processor_id()); return -EINVAL; @@ -1169,7 +1169,7 @@ void __init sve_setup(void) set_bit(__vq_to_bit(SVE_VQ_MIN), info->vq_map); max_bit = find_first_bit(info->vq_map, SVE_VQ_MAX); - info->max_vl = sve_vl_from_vq(__bit_to_vq(max_bit)); + info->max_vl = __bit_to_vl(max_bit); /* * For the default VL, pick the maximum supported value <= 64. @@ -1188,7 +1188,7 @@ void __init sve_setup(void) /* No virtualisable VLs? This is architecturally forbidden. */ info->max_virtualisable_vl = SVE_VQ_MIN; else /* b + 1 < SVE_VQ_MAX */ - info->max_virtualisable_vl = sve_vl_from_vq(__bit_to_vq(b + 1)); + info->max_virtualisable_vl = __bit_to_vl(b + 1); if (info->max_virtualisable_vl > info->max_vl) info->max_virtualisable_vl = info->max_vl; @@ -1305,10 +1305,10 @@ void __init sme_setup(void) WARN_ON(bitmap_empty(info->vq_map, SVE_VQ_MAX)); min_bit = find_last_bit(info->vq_map, SVE_VQ_MAX); - info->min_vl = sve_vl_from_vq(__bit_to_vq(min_bit)); + info->min_vl = __bit_to_vl(min_bit); max_bit = find_first_bit(info->vq_map, SVE_VQ_MAX); - info->max_vl = sve_vl_from_vq(__bit_to_vq(max_bit)); + info->max_vl = __bit_to_vl(max_bit); WARN_ON(info->min_vl > info->max_vl); From patchwork Wed Jun 5 11:41:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13686728 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 093D3C27C53 for ; Wed, 5 Jun 2024 11:50:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2wgfzdJfZ1/E22Nu2rHFQ4T08GD8U7BJ6gOVgz1I5ow=; b=wWvEaxskaT8y5D KqPjDVkjdxNNA+NQV75IUDJXzk7T1w7T/hEUVfn6CUXgOzYIFM6rebwtSDy04KqZTvs7qLxGOqq+W BJXrkoMDMlRYSJKCsfCoGTe/GEfnQzhzRhWIs+bZa/0I2sq2fjpezcNIfYEeIOWikTD3QmRjcZ9tF CshXpizu653yBUfVZhdKF77gf//yw74K/wgdmsXUUlKEGAB466fPRl6Nc0DTaz0oDbtMQT3Z2nEPO J6aTMocPaBWenj02MTXcFZz0gznQFrxgH4woNjQUY601FWdBliK0o7oLUAXzzPCFvDPsPk79gYt9j 62+6Ck9Lwrr8JUpstBNg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEp9j-00000005mXX-0coA; Wed, 05 Jun 2024 11:50:15 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEp9e-00000005mUK-3pcu for linux-arm-kernel@lists.infradead.org; Wed, 05 Jun 2024 11:50:12 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 554696181B; Wed, 5 Jun 2024 11:50:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BB49BC4AF0A; Wed, 5 Jun 2024 11:50:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1717588210; bh=0HRuKE8VlXUba+j3EwHzj90KT4DQxSJsorHaMRo2+XQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=JTc6iI5Xt2gKkG6jzOLpgp9JosGSx32VAglx5WX3G2X2sRgBDpIM9ChF7WDq+GHBE cturpVQV9rI/nKk+lShBopKjb0v5c7jNQQg6dxIbtOwJOoFpwsTdJKK4zDXwSi2RYE s7RwqJHwqfFZRqhzLC73vkblm98Xz/h716R6kZBid7jMKLP4LjLOgwz2yd6O61Shq5 3b3ODe5E3QJwAZkDUEL/YgW7C45P8d6OGVZ27cDG0WLUZT+TpXAJvBFO2h5fBdZAyX V6SRCgojTr+9u+C9Rt2EuGDBTpp3Yk65p6TPM9ZhkGb9kf6+Jlb9o9U1IOKlhINLj8 8LAa8X/N9sTWg== From: Mark Brown Date: Wed, 05 Jun 2024 12:41:28 +0100 Subject: [PATCH 2/4] arm64/fpsimd: Discover maximum vector length implemented by any CPU MIME-Version: 1.0 Message-Id: <20240605-kvm-arm64-fix-pkvm-sve-vl-v1-2-680d6b43b4c1@kernel.org> References: <20240605-kvm-arm64-fix-pkvm-sve-vl-v1-0-680d6b43b4c1@kernel.org> In-Reply-To: <20240605-kvm-arm64-fix-pkvm-sve-vl-v1-0-680d6b43b4c1@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.14-dev-d4707 X-Developer-Signature: v=1; a=openpgp-sha256; l=4299; i=broonie@kernel.org; h=from:subject:message-id; bh=0HRuKE8VlXUba+j3EwHzj90KT4DQxSJsorHaMRo2+XQ=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmYFDniQKKB3lNFxU6WDccvOf/Ww+JlNUgz3udSCRy bnFnw/uJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZmBQ5wAKCRAk1otyXVSH0OqyB/ wNfv5HhFD0WNmHYdHM/Rb5yPlCnrAftBkUrzdEtnR5ro/4hHSlplNLYs57As2SqP7e8hwPe0TYSYFk DhvCSELsGfkDFn7PYz+os1aOPgFGHO0SVZ/pJuFLC/ZqI9argK5wg7VoQHZUP9yAy7N3ji7kC5KxLr 8iEa8CzZaIdaMX90p0o4RMO2D7kwv0A0k5VLLaQHZDEUGbbJARn+KBR+huwM3AFw0dvSYc+VkHCc23 po3tNZVqfEeB3xHfXKK2OMkY+NDfGRbSX24mznjzplDD1VlVAFO3O28BAe6S+zYU3o0BBPituikoEH Ae+vh9utlClmvraRzvOBzvbt361e0i X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240605_045011_090814_75937120 X-CRM114-Status: GOOD ( 18.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When discovering the vector lengths for SVE and SME we do not currently record the maximum VL supported on any individual CPU. This is expected to be the same for all CPUs but the architecture allows asymmetry, if we do encounter an asymmetric system then some CPUs may support VLs higher than the maximum Linux will use. Since the pKVM hypervisor needs to support saving and restoring anything the host can physically set it needs to know the maximum value any CPU could have, add support for enumerating it and validation for late CPUs. Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 13 +++++++++++++ arch/arm64/kernel/fpsimd.c | 26 +++++++++++++++++++++++++- 2 files changed, 38 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index 51c21265b4fa..cd19713c9deb 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -188,6 +188,9 @@ struct vl_info { int max_vl; int max_virtualisable_vl; + /* Maximum vector length observed on any CPU */ + int max_cpu_vl; + /* * Set of available vector lengths, * where length vq encoded as bit __vq_to_bit(vq): @@ -278,6 +281,11 @@ static inline int vec_max_virtualisable_vl(enum vec_type type) return vl_info[type].max_virtualisable_vl; } +static inline int vec_max_cpu_vl(enum vec_type type) +{ + return vl_info[type].max_cpu_vl; +} + static inline int sve_max_vl(void) { return vec_max_vl(ARM64_VEC_SVE); @@ -288,6 +296,11 @@ static inline int sve_max_virtualisable_vl(void) return vec_max_virtualisable_vl(ARM64_VEC_SVE); } +static inline int sve_max_cpu_vl(void) +{ + return vec_max_cpu_vl(ARM64_VEC_SVE); +} + /* Ensure vq >= SVE_VQ_MIN && vq <= SVE_VQ_MAX before calling this function */ static inline bool vq_available(enum vec_type type, unsigned int vq) { diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 22542fb81812..27f3593547f1 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -129,6 +129,7 @@ __ro_after_init struct vl_info vl_info[ARM64_VEC_MAX] = { .min_vl = SVE_VL_MIN, .max_vl = SVE_VL_MIN, .max_virtualisable_vl = SVE_VL_MIN, + .max_cpu_vl = SVE_VL_MIN, }, #endif #ifdef CONFIG_ARM64_SME @@ -1041,8 +1042,13 @@ static void vec_probe_vqs(struct vl_info *info, void __init vec_init_vq_map(enum vec_type type) { struct vl_info *info = &vl_info[type]; + unsigned long b; + vec_probe_vqs(info, info->vq_map); bitmap_copy(info->vq_partial_map, info->vq_map, SVE_VQ_MAX); + + b = find_first_bit(info->vq_map, SVE_VQ_MAX); + info->max_cpu_vl = __bit_to_vl(b); } /* @@ -1054,11 +1060,16 @@ void vec_update_vq_map(enum vec_type type) { struct vl_info *info = &vl_info[type]; DECLARE_BITMAP(tmp_map, SVE_VQ_MAX); + unsigned long b; vec_probe_vqs(info, tmp_map); bitmap_and(info->vq_map, info->vq_map, tmp_map, SVE_VQ_MAX); bitmap_or(info->vq_partial_map, info->vq_partial_map, tmp_map, SVE_VQ_MAX); + + b = find_first_bit(tmp_map, SVE_VQ_MAX); + if (__bit_to_vl(b) > info->max_cpu_vl) + info->max_cpu_vl = __bit_to_vl(b); } /* @@ -1069,9 +1080,10 @@ int vec_verify_vq_map(enum vec_type type) { struct vl_info *info = &vl_info[type]; DECLARE_BITMAP(tmp_map, SVE_VQ_MAX); - unsigned long b; + unsigned long b, max_vl; vec_probe_vqs(info, tmp_map); + max_vl = __bit_to_vl(find_first_bit(tmp_map, SVE_VQ_MAX)); bitmap_complement(tmp_map, tmp_map, SVE_VQ_MAX); if (bitmap_intersects(tmp_map, info->vq_map, SVE_VQ_MAX)) { @@ -1083,6 +1095,18 @@ int vec_verify_vq_map(enum vec_type type) if (!IS_ENABLED(CONFIG_KVM) || !is_hyp_mode_available()) return 0; + /* + * pKVM allocates and uses storage for host state based on the + * largest per-PE VL, reject new PEs with a larger maximum. + */ + if (is_protected_kvm_enabled()) { + if (max_vl > info->max_cpu_vl) { + pr_warn("%s: cpu%d: would increase maximum VL\n", + info->name, smp_processor_id()); + return -EINVAL; + } + } + /* * For KVM, it is necessary to ensure that this CPU doesn't * support any vector length that guests may have probed as From patchwork Wed Jun 5 11:41:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13686729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 362F6C25B76 for ; Wed, 5 Jun 2024 11:50:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Kv25Yy5z+hh6pgZ2n8/1Mpn62OxFy87ekBeVG5iFwkY=; b=J9oXYz3WP6+Epr OmfIG3/MU0KPu0gmdLBrCoFjICEM5fjLc44+I2ThQkHn4v6El2IIC6IY0xwifs+ZhRTi5kBaQtATu i/42HT2xiBz4q2AXewOfNtydcJ9xSnq2rJimWAkNyfhTxy/0eOX552rb8d+jUtBnnHgL+8G38ked1 531H5fdtDsqQyvywpyKFqj3FTTXpw2nIQfJttpTI5YNsRiD9u/uoS71sUDRilhWLfmccpNspg0sSw jeNIdKypL4s29M6oHy80F/HGf+n4gCAknvbDjxd9DcW0v1CTIgWQe/m+NjrDd+pSHqrAAIIqDIgzp IFnSMz8+HrH1M1xkATDA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEp9m-00000005mZe-1k9h; Wed, 05 Jun 2024 11:50:18 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEp9h-00000005mWD-3CkL for linux-arm-kernel@lists.infradead.org; Wed, 05 Jun 2024 11:50:15 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id F1700617BD; Wed, 5 Jun 2024 11:50:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7E766C3277B; Wed, 5 Jun 2024 11:50:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1717588212; bh=6c1J1xU5lj/uDWDKMELN0USooXuGEwyYBBdoSr9IN5k=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=kpzyHpVZUc2cTuibhG56v9GllJK173FJzG3r6TeFWkRGAYVtW2e9xDXplXbVGsKVa 92wrW67uFQBubZkx6H6Ix0dnnC+N7VpxYRe6jEBw3l9RfgnNwB/Hnd9GzC6c2sgIll WGAPzU4yawFn0cUCVfNeVABJv3VvMO4jOR0JvKBb0YSji8iU4U95cEac8CDc75BFUw ZtZuaZneMOfCi6EZeOtOgiioD7GIV0o6z6O8aQKn60PHUzuLxBtBOd461WGRjM2q7R Bwtx/ah62/GxMbMe/QMI96UFKb5KQ+WZ+I77Uxb9UrTkasGbhVnBK9ehWx3Rr0flP7 UHEo4IGhJH2fA== From: Mark Brown Date: Wed, 05 Jun 2024 12:41:29 +0100 Subject: [PATCH 3/4] KVM: arm64: Fix FFR offset calculation for pKVM host state save and restore MIME-Version: 1.0 Message-Id: <20240605-kvm-arm64-fix-pkvm-sve-vl-v1-3-680d6b43b4c1@kernel.org> References: <20240605-kvm-arm64-fix-pkvm-sve-vl-v1-0-680d6b43b4c1@kernel.org> In-Reply-To: <20240605-kvm-arm64-fix-pkvm-sve-vl-v1-0-680d6b43b4c1@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.14-dev-d4707 X-Developer-Signature: v=1; a=openpgp-sha256; l=2968; i=broonie@kernel.org; h=from:subject:message-id; bh=6c1J1xU5lj/uDWDKMELN0USooXuGEwyYBBdoSr9IN5k=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmYFDoS+tQMHD3gju9xsmaBv3OV04ouECGRPwHWpXn FW5Cn7OJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZmBQ6AAKCRAk1otyXVSH0JjbB/ 4wVrMIkRIgpQYuRW+a+2JfCt54xdzClYfx+fQf+SGGIjsHUnHvOEMSO2msR5qAC+tQ1JCEFvIliLWt TFjxCf4X8WrQtcqJvT+1ToflMa2Ib9zeEeP2OQ3ETsOdmbWUjZBhq9cBt1SxOkPSEOAGXRdiCPGNBO /4qBks58qe38BvrN6HbU0rIoYCOGjSGasUw/j0BCCPrE+kgPjpXgt8attTusyAjhwCMYtWYatoNF1Q /MvzeLeVT3eiyCiK5aQxv9YAA8NQynz8pe+SA3T1trQldGnoEtfmS1tlmaIEzoatn5rSJQ30TRmV1Z 6zhHdyz+542bvrleMcCzWoVL0dZjiI X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240605_045013_990110_5B8877BB X-CRM114-Status: GOOD ( 15.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When saving and restoring the SVE state for the host we configure the hardware for the maximum VL it supports, but when calculating offsets in memory we use the maximum usable VL for the host. Since these two values may not be the same this may result in data corruption. We can just read the current VL from the hardware with an instruction so do that instead of a saved value. Fixes: b5b9955617bc ("KVM: arm64: Eagerly restore host fpsimd/sve state in pKVM") Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_hyp.h | 1 + arch/arm64/kvm/hyp/fpsimd.S | 5 +++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 2 +- 4 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index b05bceca3385..7510383d78a6 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -113,6 +113,7 @@ void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs); void __sve_save_state(void *sve_pffr, u32 *fpsr, int save_ffr); void __sve_restore_state(void *sve_pffr, u32 *fpsr, int restore_ffr); +int __sve_get_vl(void); u64 __guest_enter(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S index e950875e31ce..d272dbf36da8 100644 --- a/arch/arm64/kvm/hyp/fpsimd.S +++ b/arch/arm64/kvm/hyp/fpsimd.S @@ -31,3 +31,8 @@ SYM_FUNC_START(__sve_save_state) sve_save 0, x1, x2, 3 ret SYM_FUNC_END(__sve_save_state) + +SYM_FUNC_START(__sve_get_vl) + _sve_rdvl 0, 1 + ret +SYM_FUNC_END(__sve_get_vl) diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 0c4de44534b7..06efcca765cc 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -327,7 +327,7 @@ static inline void __hyp_sve_save_host(void) sve_state->zcr_el1 = read_sysreg_el1(SYS_ZCR); write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); - __sve_save_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + __sve_save_state(sve_state->sve_regs + sve_ffr_offset(__sve_get_vl()), &sve_state->fpsr, true); } diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index f43d845f3c4e..bd8f671e848c 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -49,7 +49,7 @@ static void __hyp_sve_restore_host(void) * supported by the system (or limited at EL3). */ write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); - __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(__sve_get_vl()), &sve_state->fpsr, true); write_sysreg_el1(sve_state->zcr_el1, SYS_ZCR); From patchwork Wed Jun 5 11:41:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13686730 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C04C0C27C53 for ; Wed, 5 Jun 2024 11:50:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=fjsfPTGCTboU4WKfkV0z6cmcdSOmRMSn0niO3LkAPXM=; b=PrhGEN02wn7zGn xea2cSQvOdFYd3gQwu8Ci83ouQA0TCBWB+DBcrSzdkg7e3Zcy+I0Tbm3GymUQqDnOasxqodHrfK3G +zzW+CXpV3UPftAOaTLecPDscrJDYtpzwzl1C6a2vdmzd1vSEoO87NKjfGWrz91DpAMgrx+4i0eXJ 9YED+h2/kFWWSv1nZ8Qmus5mnQzQ2ftPcxzFhe4ItCjSbJStivaU9Egu/h+U+f0rSUNFJSvPOFNN8 4jQ40CIw9v1ch1W/kdchjf9YfEVRqPRKodZQsyZoIVtgx58PBlyZ4VxrJTQIUSGkQRTlQE6ZwwOu7 Q6kdwpSe0cZgd1G+I6Qw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEp9q-00000005mcA-1p7T; Wed, 05 Jun 2024 11:50:22 +0000 Received: from sin.source.kernel.org ([145.40.73.55]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEp9m-00000005mYs-0fzc for linux-arm-kernel@lists.infradead.org; Wed, 05 Jun 2024 11:50:20 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 4D9F6CE16FE; Wed, 5 Jun 2024 11:50:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 445BEC4AF08; Wed, 5 Jun 2024 11:50:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1717588215; bh=I+qiD9f7AIjkeTzE7i3sCyRtTk9/3BCUmNFuB2K/uis=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=LAlnpnZYLXMLUpTvG3MQVyE+K6rpEEFqw4vUwMUmPLD5xDfHwoSWshyluK3r9vLSi Pkm44HYBiFqaMrp7zVwDhsImoH0ogYo56Quk5PxLqo0wbmwHuUgNCNg4pTBVsNHNV6 nn76CiUxAiOa2D8tyHhG9HR3rLSNCvToPBwiTWCYN+BTdfHKk76Jslemkq3As8asPJ UN2UzbEgOlPfgJorsQQrZktTxjFQ7lOVcdpwMwscEcBOWIHo6mqfIBYX+yDsj7TI3O IiG4zT7SSzpv2LSo3tIRcShRBwZaLncdu2Mi1iZBO7YzThWu3m1spjTroRC0m8qO60 sBneQuG06drFg== From: Mark Brown Date: Wed, 05 Jun 2024 12:41:30 +0100 Subject: [PATCH 4/4] KVM: arm64: Avoid underallocating storage for host SVE state MIME-Version: 1.0 Message-Id: <20240605-kvm-arm64-fix-pkvm-sve-vl-v1-4-680d6b43b4c1@kernel.org> References: <20240605-kvm-arm64-fix-pkvm-sve-vl-v1-0-680d6b43b4c1@kernel.org> In-Reply-To: <20240605-kvm-arm64-fix-pkvm-sve-vl-v1-0-680d6b43b4c1@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.14-dev-d4707 X-Developer-Signature: v=1; a=openpgp-sha256; l=4775; i=broonie@kernel.org; h=from:subject:message-id; bh=I+qiD9f7AIjkeTzE7i3sCyRtTk9/3BCUmNFuB2K/uis=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmYFDpNLP6xsTXnP4x+K63OmWAJYiakPHrQkrNYpo1 ZXtEslWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZmBQ6QAKCRAk1otyXVSH0K9cB/ 49BkO45ApDn9aeWm8mUJ7CZsd+CKe1eEiMHrx3B2uFlNMFMa2p6dYhjR3kVwQJBgSKZNTsBSvfQW/b lTwH0VLX38XW3mnZDcisC9jPnwSe1OXSdaBnrR/DUTEgYr0AHHg3TGVcRFlGADgEBFxK1NUz1M5A/v EtS9jcxTuzkrKolSa7nRDIPjICi0nC1riWydKf5Gy4O9vVWybhQKBoG7cQx+4AmvwkISFmxvU+Kqz6 dE58oIqi3KKMKri4kd7H3KWfrX8FI8zcdV/J+NsVvhZ06HCMP1oFWVxun2qOoHHYxlPhxFgVeCcLSk K0XMNgb6L16jUyGCpHiZSmMb3F/z9m X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240605_045018_810793_76084355 X-CRM114-Status: GOOD ( 16.04 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We size the allocation for the host SVE state using the maximum VL shared by all CPUs in the host. As observed during review on an asymmetric system this may be less than the maximum VL supported on some of the CPUs. Since the pKVM hypervisor saves and restores the host state using the maximum VL for the current CPU this may lead to buffer overflows, fix this by changing pKVM to use the maximum VL for any CPU to size allocations and limit host configurations. Fixes: 66d5b53e20a6 ("KVM: arm64: Allocate memory mapped at hyp for host sve state in pKVM") Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/include/asm/kvm_hyp.h | 2 +- arch/arm64/include/asm/kvm_pkvm.h | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 4 ++-- arch/arm64/kvm/hyp/nvhe/pkvm.c | 2 +- arch/arm64/kvm/reset.c | 6 +++--- 6 files changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 36b8e97bf49e..a28fae10596f 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -76,7 +76,7 @@ static inline enum kvm_mode kvm_get_mode(void) { return KVM_MODE_NONE; }; DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); extern unsigned int __ro_after_init kvm_sve_max_vl; -extern unsigned int __ro_after_init kvm_host_sve_max_vl; +extern unsigned int __ro_after_init kvm_host_sve_max_cpu_vl; int __init kvm_arm_init_sve(void); u32 __attribute_const__ kvm_target_cpu(void); diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 7510383d78a6..47426df69875 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -144,6 +144,6 @@ extern u64 kvm_nvhe_sym(id_aa64smfr0_el1_sys_val); extern unsigned long kvm_nvhe_sym(__icache_flags); extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits); -extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_vl); +extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_cpu_vl); #endif /* __ARM64_KVM_HYP_H__ */ diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h index cd56acd9a842..6fc0cf42fca3 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -134,7 +134,7 @@ static inline size_t pkvm_host_sve_state_size(void) return 0; return size_add(sizeof(struct cpu_sve_state), - SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_vl))); + SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_cpu_vl))); } #endif /* __ARM64_KVM_PKVM_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index bd8f671e848c..d232775b72c9 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -90,8 +90,8 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) hyp_vcpu->vcpu.arch.ctxt = host_vcpu->arch.ctxt; hyp_vcpu->vcpu.arch.sve_state = kern_hyp_va(host_vcpu->arch.sve_state); - /* Limit guest vector length to the maximum supported by the host. */ - hyp_vcpu->vcpu.arch.sve_max_vl = min(host_vcpu->arch.sve_max_vl, kvm_host_sve_max_vl); + /* Limit guest vector length to the maximum supported by any CPU. */ + hyp_vcpu->vcpu.arch.sve_max_vl = min(host_vcpu->arch.sve_max_vl, kvm_host_sve_max_cpu_vl); hyp_vcpu->vcpu.arch.hw_mmu = host_vcpu->arch.hw_mmu; diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 95cf18574251..08e825de09d1 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -18,7 +18,7 @@ unsigned long __icache_flags; /* Used by kvm_get_vttbr(). */ unsigned int kvm_arm_vmid_bits; -unsigned int kvm_host_sve_max_vl; +unsigned int kvm_host_sve_max_cpu_vl; /* * Set trap register values based on features in ID_AA64PFR0. diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index 3fc8ca164dbe..59cccb477cf3 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -32,7 +32,7 @@ /* Maximum phys_shift supported for any VM on this host */ static u32 __ro_after_init kvm_ipa_limit; -unsigned int __ro_after_init kvm_host_sve_max_vl; +unsigned int __ro_after_init kvm_host_sve_max_cpu_vl; /* * ARMv8 Reset Values @@ -52,8 +52,8 @@ int __init kvm_arm_init_sve(void) { if (system_supports_sve()) { kvm_sve_max_vl = sve_max_virtualisable_vl(); - kvm_host_sve_max_vl = sve_max_vl(); - kvm_nvhe_sym(kvm_host_sve_max_vl) = kvm_host_sve_max_vl; + kvm_host_sve_max_cpu_vl = sve_max_cpu_vl(); + kvm_nvhe_sym(kvm_host_sve_max_cpu_vl) = kvm_host_sve_max_cpu_vl; /* * The get_sve_reg()/set_sve_reg() ioctl interface will need