From patchwork Fri Dec 20 16:46:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917104 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5640E21B8E0; Fri, 20 Dec 2024 16:50:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713458; cv=none; b=mss4Hkq+cp4phbTQHHrEJchhktVj350+XN+CXS0z/b3jnFM9skeP58xpsHsgKY0WYyXHHRrdPc7bqz2nMEP/Jf+KxPKkgyw7Mby/+1tsqejb8Y0DzymxA9eZ3gKKOurPHvE+bAGi4lZfdp0zqwioOuJHNHWYBe/LgMe/ZhiX/Yc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713458; c=relaxed/simple; bh=TrCGcoJZcMaq9NQXaz5QIPxcM7YhfKCIwJUJHa3hpoY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=aT89yI17UsMp6/zLNIL7fV9A9urfCU0L4Y5XlxMlZs6Ar4j+nGWhO85vOmSEEBjNRrDWEEwPZs6xnp7Xk5ePdxsd0FgAc/ema9W+USrkqa1sg8qa15gMr12Ns+cmArJ+O3FQFyzldByu2Hm7bIcMQcsIVBRBQDIG/kxT++qPeCI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lY0GcXKj; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lY0GcXKj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8449CC4CEDC; Fri, 20 Dec 2024 16:50:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713457; bh=TrCGcoJZcMaq9NQXaz5QIPxcM7YhfKCIwJUJHa3hpoY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=lY0GcXKjx5ustz3wJJVvDwWEYFi8aYA9kUIpUGgXqIzbsMD32EMGVjJiKRlaYZ/RA Zmm+CQtXyi0DhErPJNdx5p0Qdk6l++6MvPcV9lYmRNOCgvUONrepMHLwkPUQGPlZYS QT2cqs0oYH2HENIhTGDERPJFRdyR57oKpGnn+250g+GnAoR3T17PaTPGSXZ9zz9ZX1 QED1NVDfdpDli5hlfltgDvRCokxf6pUfXZsuCXvK9Uv3Iumf1bwFRJX89lhRHZrIZO dGBe+nM/afqlStmsu+o5lKEKjrVwMpc0/bhawrZk0bOF3OLu6+DjFQ638o2Zt+HEz1 qa3SLIdyux1+Q== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:26 +0000 Subject: [PATCH RFC v3 01/27] arm64/fpsimd: Update FA64 and ZT0 enables when loading SME state Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-1-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=5400; i=broonie@kernel.org; h=from:subject:message-id; bh=TrCGcoJZcMaq9NQXaz5QIPxcM7YhfKCIwJUJHa3hpoY=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBV7Q2cNbktgqMjPzjwEQdgceGjgwNGp30spIyG LQzQEDmJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgVQAKCRAk1otyXVSH0OtoB/ 4+5F12FDcBOuENPVMcOHBNTl8Je2KSQtp0eVdxl6YuybjZSByEi2ZxgVs2D1oHbBdYMCaSEJb6p3Ni JBmkEyhWsgL60rA9Xlqs3fQ/5sXix3J+IXXW3qALcWrfxrc37N7gXtpIUbFyaiK03W/4xdK8PZqoPZ eYkaIYjo8v8F6Q8nWRFNmfokZDZEg7VS2slCKAomIpzuH3t2XnMwR+SC1HNno25JOB7nc/OirmzSEo Vs2jpccl3x5ltbHgBObc62H0IVS21EvP+jql9unY6lSrYpbGFuDRUTAmhNRMbpV84DCg0nZL9EQ24x ZtVYFiL5r54EsGGYC5aUFgZej7qmQe X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Currently we enable EL0 and EL1 access to FA64 and ZT0 at boot and leave them enabled throughout the runtime of the system. When we add KVM support we will need to make this configuration dynamic, these features may be disabled for some KVM guests. Since the host kernel saves the floating point state for non-protected guests and we wish to avoid KVM having to reload the floating point state needlessly on guest reentry let's move the configuration of these enables to the floating point state reload. We provide a helper which does the configuration as part of a read/modify/write operation along with the configuration of the task VL, then update the floating point state load and SME access trap to use it. We also remove the setting of the enable bits from the CPU feature identification and resume paths. There will be a small overhead from setting the enables one at a time but this should be negligable in the context of the state load or access trap. Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 12 +++++++++++ arch/arm64/kernel/cpufeature.c | 2 -- arch/arm64/kernel/fpsimd.c | 44 ++++++++++++----------------------------- 3 files changed, 25 insertions(+), 33 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index f2a84efc361858d4deda99faf1967cc7cac386c1..95355892d47b3ec1c77a3ab19ccad0d7f9a8d621 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -429,6 +429,18 @@ static inline size_t sme_state_size(struct task_struct const *task) #endif /* ! CONFIG_ARM64_SME */ +#define sme_cond_update_smcr(vl, fa64, zt0, reg) \ + do { \ + u64 __old = read_sysreg_s((reg)); \ + u64 __new = vl; \ + if (fa64) \ + __new |= SMCR_ELx_FA64; \ + if (zt0) \ + __new |= SMCR_ELx_EZT0; \ + if (__old != __new) \ + write_sysreg_s(__new, (reg)); \ + } while (0) + /* For use by EFI runtime services calls only */ extern void __efi_fpsimd_begin(void); extern void __efi_fpsimd_end(void); diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 6ce71f444ed84f9056196bb21bbfac61c9687e30..ab669ef8244eea70d9e20024d5584d2e960389dc 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -2848,7 +2848,6 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .type = ARM64_CPUCAP_SYSTEM_FEATURE, .capability = ARM64_SME_FA64, .matches = has_cpuid_feature, - .cpu_enable = cpu_enable_fa64, ARM64_CPUID_FIELDS(ID_AA64SMFR0_EL1, FA64, IMP) }, { @@ -2856,7 +2855,6 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .type = ARM64_CPUCAP_SYSTEM_FEATURE, .capability = ARM64_SME2, .matches = has_cpuid_feature, - .cpu_enable = cpu_enable_sme2, ARM64_CPUID_FIELDS(ID_AA64PFR1_EL1, SME, SME2) }, #endif /* CONFIG_ARM64_SME */ diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 8c4c1a2186cc510a7826d15ec36225857c07ed71..92c085288ed2cbc4f51f49546c6abbde6ba891a3 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -399,9 +399,15 @@ static void task_fpsimd_load(void) if (system_supports_sme()) { unsigned long sme_vl = task_get_sme_vl(current); - /* Ensure VL is set up for restoring data */ + /* + * Ensure VL is set up for restoring data. KVM might + * disable subfeatures so we reset them each time. + */ if (test_thread_flag(TIF_SME)) - sme_set_vq(sve_vq_from_vl(sme_vl) - 1); + sme_cond_update_smcr(sve_vq_from_vl(sme_vl) - 1, + system_supports_fa64(), + system_supports_sme2(), + SYS_SMCR_EL1); write_sysreg_s(current->thread.svcr, SYS_SVCR); @@ -1267,26 +1273,6 @@ void cpu_enable_sme(const struct arm64_cpu_capabilities *__always_unused p) isb(); } -void cpu_enable_sme2(const struct arm64_cpu_capabilities *__always_unused p) -{ - /* This must be enabled after SME */ - BUILD_BUG_ON(ARM64_SME2 <= ARM64_SME); - - /* Allow use of ZT0 */ - write_sysreg_s(read_sysreg_s(SYS_SMCR_EL1) | SMCR_ELx_EZT0_MASK, - SYS_SMCR_EL1); -} - -void cpu_enable_fa64(const struct arm64_cpu_capabilities *__always_unused p) -{ - /* This must be enabled after SME */ - BUILD_BUG_ON(ARM64_SME_FA64 <= ARM64_SME); - - /* Allow use of FA64 */ - write_sysreg_s(read_sysreg_s(SYS_SMCR_EL1) | SMCR_ELx_FA64_MASK, - SYS_SMCR_EL1); -} - void __init sme_setup(void) { struct vl_info *info = &vl_info[ARM64_VEC_SME]; @@ -1330,17 +1316,9 @@ void __init sme_setup(void) void sme_suspend_exit(void) { - u64 smcr = 0; - if (!system_supports_sme()) return; - if (system_supports_fa64()) - smcr |= SMCR_ELx_FA64; - if (system_supports_sme2()) - smcr |= SMCR_ELx_EZT0; - - write_sysreg_s(smcr, SYS_SMCR_EL1); write_sysreg_s(0, SYS_SMPRI_EL1); } @@ -1457,7 +1435,11 @@ void do_sme_acc(unsigned long esr, struct pt_regs *regs) if (!test_thread_flag(TIF_FOREIGN_FPSTATE)) { unsigned long vq_minus_one = sve_vq_from_vl(task_get_sme_vl(current)) - 1; - sme_set_vq(vq_minus_one); + + sme_cond_update_smcr(vq_minus_one, + system_supports_fa64(), + system_supports_sme2(), + SYS_SMCR_EL1); fpsimd_bind_task_to_cpu(); } From patchwork Fri Dec 20 16:46:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917105 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C7C2521C9E1; Fri, 20 Dec 2024 16:51:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713461; cv=none; b=ZZiUcXrw1CkYVMi+LvKwMcWUaaDNIFlJ1z8YzUK/Am5jOHYGHZ8OTKrkOWpi0njmP0PXUSioo5GwRIVuwneKnWNGWlHALr9cfKjvK2Le26OBwofnseq/AtVlSIroDghu82c3Gp0VNd3l6TnS9xob6TFoNfT6ufmM7/zeoIXMNP4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713461; c=relaxed/simple; bh=tNHHLZ/GUKR3zyLQMr9SmLccDJKFOywaygU53a8ORrU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=FpeGKoRTgY4hKlj8LQsq4vOddUTlK79ckcDUHMR4m8Wt7A2Gtea9oJwuLG/4I1Zi67RjB3BqwfWPkLT1ovcqpB2MMK+l5kqEo28f82YytBsNolTdkMv4pZPPvuSq1vrB6eOpsxGFxGb4AouVboYzlFoRK3yBO5bYHK81Cq5AFs8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=U0fMS7+s; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="U0fMS7+s" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4A804C4CECD; Fri, 20 Dec 2024 16:50:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713461; bh=tNHHLZ/GUKR3zyLQMr9SmLccDJKFOywaygU53a8ORrU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=U0fMS7+sg4ObOs47tgUj0fJ6TvQ3m1eKtDUMQOgcNhgNYtgarH025ARgglxrkQ+G5 EVEq4J4DM2rO48CtxNBvZ6r6SCBrgs+K92FgefbzrWCLec/XnR4YbQSbMFN284x3Sz SpqLlZEUVk7WIks5QIbfSIkyEP+kW9votWz62CfS4pC8RYL36fyMrAFwiLV4wGf7iK Qb+RLAIeaC+M8hxAGz52UD00v5YPM+ZOeMcT9QMPzZWXnBtuYt3QuSC+QE7GmeP0Sw lmEwoRR4W0mWrOWy4qI6IlLHSKZ2pA7QxWNKknKhxZZnubS+/ZUNHIisQSGEunWj6I 305q5yriDAmBg== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:27 +0000 Subject: [PATCH RFC v3 02/27] arm64/fpsimd: Decide to save ZT0 and streaming mode FFR at bind time Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-2-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=3401; i=broonie@kernel.org; h=from:subject:message-id; bh=tNHHLZ/GUKR3zyLQMr9SmLccDJKFOywaygU53a8ORrU=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBWsX7ZqoOdB4b5mGScOlPYolUX6HFXVv3II/y+ il9/MEyJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgVgAKCRAk1otyXVSH0CzoB/ 0b3XUZC1SIgcyr9aZTGKpBhSfDnj+lLwovphIWHJbmHxOLdeTpwX7t2E+7WHr3+fSGnQtllpadq5Xg inrHu99pLcZyW/J5cjYKX2MWKGZFy8O5iiWhsK+TOCAoafRsrz+9oiVh9yS5NPBG40xobgGIJhlxom qcyODonMbX+yhcoEB/mpdde0lGAc2zTXucrUmjNMHhHIPxwxgdC81HZP/6Y2pzpoLIYIMeDcZ2F1Em nW1dvtLh//GnK0DEbO5Rbh22K+6mRJWMJtDV6WJI6/kT3eg9mJGc8GuCXNJg8IjSFTqzgWrtlganth MZbZToJdR+xo9jlKj0GNdCPPLKsfiI X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Some parts of the SME state are optional, enabled by additional features on top of the base FEAT_SME and controlled with enable bits in SMCR_ELx. We unconditionally enable these for the host but for KVM we will allow the feature set exposed to guests to be restricted by the VMM. These are the FFR register (FEAT_SME_FA64) and ZT0 (FEAT_SME2). We defer saving of guest floating point state for non-protected guests to the host kernel. We also want to avoid having to reconfigure the guest floating point state if nothing used the floating point state while running the host. If the guest was running with the optional features disabled then traps will be enabled for them so the host kernel will need to skip accessing that state when saving state for the guest. Support this by moving the decision about saving this state to the point where we bind floating point state to the CPU, adding a new variable to the cpu_fp_state which uses the enable bits in SMCR_ELx to flag which features are enabled. Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 1 + arch/arm64/kernel/fpsimd.c | 10 ++++++++-- arch/arm64/kvm/fpsimd.c | 1 + 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index 95355892d47b3ec1c77a3ab19ccad0d7f9a8d621..144cc805bfea112341b89c9c6028cf4b2a201c6c 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -88,6 +88,7 @@ struct cpu_fp_state { void *sme_state; u64 *svcr; u64 *fpmr; + u64 sme_features; unsigned int sve_vl; unsigned int sme_vl; enum fp_type *fp_type; diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 92c085288ed2cbc4f51f49546c6abbde6ba891a3..7c66ed6e43c34d1b5e1cc00595c12244d13d3d0d 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -478,12 +478,12 @@ static void fpsimd_save_user_state(void) if (*svcr & SVCR_ZA_MASK) sme_save_state(last->sme_state, - system_supports_sme2()); + last->sme_features & SMCR_ELx_EZT0); /* If we are in streaming mode override regular SVE. */ if (*svcr & SVCR_SM_MASK) { save_sve_regs = true; - save_ffr = system_supports_fa64(); + save_ffr = last->sme_features & SMCR_ELx_FA64; vl = last->sme_vl; } } @@ -1722,6 +1722,12 @@ static void fpsimd_bind_task_to_cpu(void) last->to_save = FP_STATE_CURRENT; current->thread.fpsimd_cpu = smp_processor_id(); + last->sme_features = 0; + if (system_supports_fa64()) + last->sme_features |= SMCR_ELx_FA64; + if (system_supports_sme2()) + last->sme_features |= SMCR_ELx_EZT0; + /* * Toggle SVE and SME trapping for userspace if needed, these * are serialsied by ret_to_user(). diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index ea5484ce1f3ba3121b6938bda15f7a8057d49051..09b65abaf9db60cc57dbc554ad2108a80c2dc46b 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -138,6 +138,7 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu) fp_state.svcr = &__vcpu_sys_reg(vcpu, SVCR); fp_state.fpmr = &__vcpu_sys_reg(vcpu, FPMR); fp_state.fp_type = &vcpu->arch.fp_type; + fp_state.sme_features = 0; if (vcpu_has_sve(vcpu)) fp_state.to_save = FP_STATE_SVE; From patchwork Fri Dec 20 16:46:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917106 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE85821A44F; Fri, 20 Dec 2024 16:51:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713466; cv=none; b=TZveCU0s4zt8sXRBiKsXf03XB/1vcKO+Y35XFAmicPmlnvHHF6n7rlz6OZ1x0BHON17CIluDDRCrbfxo2ycgKYD2F1HQyffVZysBPJhWqnzDhV5/u+UPRTmMBLm5oRGpHHg0CM7jNfd1/zhiXNX85ubtKVl5aW/cVN96AM8v1NE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713466; c=relaxed/simple; bh=a79dWEQuz4lN5+ZR85xtsyqrLMzIRNiH+ZgTGzqkyTk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=AwZaG3TZ7QUj5XueFSjJ+ZqqWvixnsad7DB2l6lSrcJNfjrZz4BjgMliHzp6zo+/4PT2TwTIGmWnAzcEeRXDYwPf/rwXZdV2a3Fs/eDVQhOIXg4724seFJ1YjRTh7QBEiSppQJO4cz3cUDnj83Nb+PEzJt+0kj6/fCcqVq0C7NY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=j+DXmh56; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="j+DXmh56" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 158D5C4CED3; Fri, 20 Dec 2024 16:51:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713465; bh=a79dWEQuz4lN5+ZR85xtsyqrLMzIRNiH+ZgTGzqkyTk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=j+DXmh56L3Btf6+Rvx9v/lj7WF1rd1Mf1XID8Z9kAZ3M+t5+t8gJOxAkz3uNTEm4/ elq75KEhDQt8euIshEsiS6gE1x+vWWacdzIGscAXsAGkxRGCO5D8HGFEjlDPQblLrk avtWc3gQBWXU/gf39yUAjPsExinlsxFtLMZ8C4A9brZOdR+hhw6IuT9UtrkzVkmnAU VBx4t8NVEilPraTk/+T6zvMOzRwe/MyqP1v/BLztsNE0itaP3FzTXTcVnC26TVeF9q l3Falxc5SslcFF1e/tx4KinHyAV3FBzGQ66QqAQqfmwWWjDrspDE54tZmuqc7ooynM q1b58utBo7FnQ== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:28 +0000 Subject: [PATCH RFC v3 03/27] arm64/fpsimd: Check enable bit for FA64 when saving EFI state Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-3-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=1524; i=broonie@kernel.org; h=from:subject:message-id; bh=a79dWEQuz4lN5+ZR85xtsyqrLMzIRNiH+ZgTGzqkyTk=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBW2Hy1qulZq7d1XQX6Oaip9UEsJD/p5OTWqqfR SoTdq3yJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgVgAKCRAk1otyXVSH0J1/B/ sEiIHvTFQypVacv8wFQ6IUikCeXXpGO8HJ3ETxw9BOn2LpiB1GMEA3dSd5oYMRAzbciGfQc6HXaKjb KUU3+CDXVTd0j1bVkY2+HVgLUrNYO4iKaS9nV0yV0+0+cU1i+A7F8m42z5GpiMMI+/+2LlWKNRo9wB uSxHSAbWxSkIJfKei5WM9jDeDU8yXHm2p2FwO+v+z2+R3KODeUZp1JsFjhvTpkCnMOYZwTmhN7ZZ2k v2hprIvZlNsDYT6zFrC31afAfa8iUKZqY8Lo/hX+zbxHRUIq7cDWuYXg2HFe5JFcrClWkb0rrwYV8d HgU+PccmqKURz+Egk40B3ty57FkAAx X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Currently when deciding if we need to save FFR when in streaming mode prior to EFI calls we check if FA64 is supported by the system. Since KVM guest support will mean that FA64 might be enabled and disabled at runtime switch to checking if traps for FA64 are enabled in SMCR_EL1 instead. Signed-off-by: Mark Brown --- arch/arm64/kernel/fpsimd.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 7c66ed6e43c34d1b5e1cc00595c12244d13d3d0d..a6f9a102fadb0547b4988cb5b0c239ca90a262a0 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1980,6 +1980,11 @@ static DEFINE_PER_CPU(bool, efi_sm_state); * either doing something wrong or you need to propose some refactoring. */ +static bool fa64_enabled(void) +{ + return read_sysreg_s(SYS_SMCR_EL1) & SMCR_ELx_FA64; +} + /* * __efi_fpsimd_begin(): prepare FPSIMD for making an EFI runtime services call */ @@ -2014,7 +2019,7 @@ void __efi_fpsimd_begin(void) * Unless we have FA64 FFR does not * exist in streaming mode. */ - if (!system_supports_fa64()) + if (!fa64_enabled()) ffr = !(svcr & SVCR_SM_MASK); } @@ -2065,7 +2070,7 @@ void __efi_fpsimd_end(void) * Unless we have FA64 FFR does not * exist in streaming mode. */ - if (!system_supports_fa64()) + if (!fa64_enabled()) ffr = false; } } From patchwork Fri Dec 20 16:46:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917107 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A42A021A931; Fri, 20 Dec 2024 16:51:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713470; cv=none; b=cf/72tAfg15eal5yyEQyB4zEz9DpzN0GRgjaCRNL0r0OvyiduUPHqfBBYSAHOzXQQu34sPcpl2dqHmrM1mLezITxgBwTy2OUkpm6mY8wlID4D/lbAENk61q7qs9Z+ZJeusqcZoNgLwS89wExN629k5H8agj8ewPUnzYLfEO2H2Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713470; c=relaxed/simple; bh=fk7img7pT6wgpuZQLS9waoWrhY76ao9cwX81iGcxSKQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Zh1oLFdICW7Ciwwl5f6f/4yBdLrxJ4zAN1AzwGGTYH3NmJwawcEasu8fF168NS4LE6xOoDu6nB91HM2nKGgl59QNv9yFNfI4FZL9+jVCtAfXDoZWHLV6mUbX7kNBueXkuJn8SsTOnhkMwT+3/XRUFZHyDSc6DjEc3nnjLggULBI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=r40FRzzA; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="r40FRzzA" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DE162C4CEDD; Fri, 20 Dec 2024 16:51:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713470; bh=fk7img7pT6wgpuZQLS9waoWrhY76ao9cwX81iGcxSKQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=r40FRzzAD3sThPfoOUv5MyiG47YJJ0vUQJfugJx/VJ2FdIH4Jq1EQjAqh5xLDpERG pRPOMBM0moFgk2Gk8JtejusZ97o1LMZON6Jz5p+Rs1HExLVpMmPpllE+R7lKLmspFp YL+9ifTPscFe0nulX2K2q9kBmzbsdsydoAyevBAGYd8T4MHQC+I5y4ndI3DxxKO8tP IhJCXqOmgFehQaX0nZQ7itPUxCjACzMYXxIPg4s+42L6byPdHRM4RXXGa8Jr3C0czi AVunGWujtHucm8uUfArMp9QaAyonFMc2Uc31N+IQbyI3biz4o+k1ntq7W8YKMHHe1Y ptJz6lfM1rerQ== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:29 +0000 Subject: [PATCH RFC v3 04/27] arm64/fpsimd: Determine maximum virtualisable SME vector length Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-4-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=2233; i=broonie@kernel.org; h=from:subject:message-id; bh=fk7img7pT6wgpuZQLS9waoWrhY76ao9cwX81iGcxSKQ=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBXHrVm0TzLRgpwCo4EdZG2bTDNm8vyq1D2fzBF 7WlvjpuJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgVwAKCRAk1otyXVSH0BO3B/ 0Qyd/4GV4pTYzevIebRa4EB3r1j8mUDGAUC466RIT9GNevbNxUj/nblW6f2YTc27rq9axg3XaNJxh7 oVessmtZ70Q7Dt7AmJeBZ6jU24Sfa2pqjH05G8Xi+d9VU8Jy/vRa9fmom8LcX2vmJLtJixGNPcIg0g TaS5sQ2wmPvYfON+whNW9XgsG3MK4cdpIc+pPu90FNe9bMScnZ4LDXQx+PUvmMaZ8LhZY7PnXXOmzL 8JNOilCBVafF3TGjr2asrEQvoLkUpeCRmlobv5NBg7g6dHGOAebedFOx+2B2r5cCRxZkgyT7g/Ncjc 6imY3E7ReG1krovkI0/39Mtgs69+gf X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB As with SVE we can only virtualise SME vector lengths that are supported by all CPUs in the system, implement similar checks to those for SVE. Since unlike SVE there are no specific vector lengths that are architecturally required the handling is subtly different, we report a system where this happens with a maximum vector length of -1. Signed-off-by: Mark Brown --- arch/arm64/kernel/fpsimd.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index a6f9a102fadb0547b4988cb5b0c239ca90a262a0..d976708d84854846fe38a35a19c60ff36f44030a 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1276,7 +1276,8 @@ void cpu_enable_sme(const struct arm64_cpu_capabilities *__always_unused p) void __init sme_setup(void) { struct vl_info *info = &vl_info[ARM64_VEC_SME]; - int min_bit, max_bit; + DECLARE_BITMAP(tmp_map, SVE_VQ_MAX); + int min_bit, max_bit, b; if (!system_supports_sme()) return; @@ -1306,12 +1307,32 @@ void __init sme_setup(void) */ set_sme_default_vl(find_supported_vector_length(ARM64_VEC_SME, 32)); + bitmap_andnot(tmp_map, info->vq_partial_map, info->vq_map, + SVE_VQ_MAX); + + b = find_last_bit(tmp_map, SVE_VQ_MAX); + if (b >= SVE_VQ_MAX) + /* All VLs virtualisable */ + info->max_virtualisable_vl = SVE_VQ_MAX; + else if (b == SVE_VQ_MAX - 1) + /* No virtualisable VLs */ + info->max_virtualisable_vl = -1; + else + info->max_virtualisable_vl = sve_vl_from_vq(__bit_to_vq(b + 1)); + + if (info->max_virtualisable_vl > info->max_vl) + info->max_virtualisable_vl = info->max_vl; + pr_info("SME: minimum available vector length %u bytes per vector\n", info->min_vl); pr_info("SME: maximum available vector length %u bytes per vector\n", info->max_vl); pr_info("SME: default vector length %u bytes per vector\n", get_sme_default_vl()); + + /* KVM decides whether to support mismatched systems. Just warn here: */ + if (info->max_virtualisable_vl < info->max_vl) + pr_warn("SME: unvirtualisable vector lengths present\n"); } void sme_suspend_exit(void) From patchwork Fri Dec 20 16:46:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917108 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E89921A931; Fri, 20 Dec 2024 16:51:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713475; cv=none; b=gZvCaNqGNRnY4bCln2q7ym5MmYLmVBcrDXj7YO1ReXsebw+LTNG9+rXR1KpoDuoCosoV+Cwn2dSCvEXc1Wk6a7mb+qvwToE+2e9OSZR//k4+EUwlJS7j3tdNz39MNPSRkb5GI8gvxoJuu+mV+0PwaTFHwypqEVypJpH0GF3KO2Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713475; c=relaxed/simple; bh=phaIeng1WiDU914WINDJrAI0+xgwAcLxe8d8S1huhjs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=EGKiG8xYEh8FsxYjKf3i0mGCUGMPki6p1ENKVTII04Lehs1yICht6bTUfpM9brZe5+66RUn+5S8cPpvv3L5o+Gy/JDZ/BsEGuMsWIP8PuqGlK8cLWmOobR8kXHS0HjvZaKhdge9YRCkz9jMm0Jyajn9JIEZrApd7WMkBThAGdqg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Y579vi1S; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Y579vi1S" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C29C9C4CECD; Fri, 20 Dec 2024 16:51:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713474; bh=phaIeng1WiDU914WINDJrAI0+xgwAcLxe8d8S1huhjs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Y579vi1Siyxesfo001+xeaEan2VSDLQZ9byAjEVwNhCPhi4w7RBfGWzT/7ITaLy0e rTP8ObslcZNaKQu1/iMRhRLBGCpbDrMkOwF0y1Hq/0QSIWjeIlGq376oxgH9ByHfGU nnyHmSQ5IcpzB/P7d3nyvVytEjnuqn4zap5/ILeIAlP2h6rSVIldsvW4QfADqJFEab mOXC4BBpmbCCepnGBv8ImV59Wh1q3vlvWfeWlfa3XxrSBB1JfsN70Mh+r7w/AmVgLT kqgHf43tUXcvD4nB3CWpAoF7+HkvYVGzMsvPCZ/PFacDNCum3yHMf4XUnGkNsIitWi LxH1Bc/xtXi9g== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:30 +0000 Subject: [PATCH RFC v3 05/27] KVM: arm64: Introduce non-UNDEF FGT control Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-5-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=2905; i=broonie@kernel.org; h=from:subject:message-id; bh=phaIeng1WiDU914WINDJrAI0+xgwAcLxe8d8S1huhjs=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBYrQCW+zfkM01sqaOmib9a+ea/2zSO7p87v75U yigA7fOJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgWAAKCRAk1otyXVSH0LtUB/ 9mLPR4Y7kByGjFjEVoh3AjCUjU6X4Sugn+ucmuPfz/9zXw1Ws2WqNmt9lVj4D3OAQAjNHAE2tFHxKK SDPcK7rQjxyKA7OgE64Um0DRYlGSFoFW0/Os0jgToloBTUMXiK3tzA5LU+raz9MfA+dLiA4PjcfXjn x3wcUxnX0aBklwYBtK0TGvrqKrWuzAG6w1mbFiLFbIQbQuDMWXWknV9uswR7ySE+pSawQdiNB0r4zT 03ubmpq7Bi/gc1NGs7KcBiIiCNljl2Am+IP6HW/IQcv+IAbUvNOrBTDqHx+Dq34IwOZ0j8V0aD5qqc x6RGwPTwPObRuJx6Z4pA2lxBzJ+5zG X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB We have support for determining a set of fine grained traps to enable for the guest which is tied to the support for injecting UNDEFs for undefined features. This means that we can't use the mechanism for system registers which should be present but need emulation, such as SMPRI_EL1 which should be accessible when SME is present but if SME priority support is absent SMPRI_EL1.Priority should be RAZ. Add an additional set of fine grained traps fgt, mirroring the existing fgu array. We use the same format where we always set the bit for the trap in the array as for FGU. This makes it clear what is being explicitly managed and keeps the code consistent. We do not convert the handling of ARM_WORKAROUND_AMPERE_ACO3_CPU_38 to this mechanism since this only enables a write trap and when implementing the existing UNDEF that we would share the read and write trap enablement (this being the overwhelmingly common case). Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 6 ++++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 7 ++++--- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 7ba742b9067e0216a156eebb3e5ea6bb69239a44..fca81ede6140c0ee7d03cb6ca8f5eead45b87033 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -288,6 +288,12 @@ struct kvm_arch { */ u64 fgu[__NR_FGT_GROUP_IDS__]; + /* + * Additional FGTs to enable for the guests, eg. for emulated + * registers, + */ + u64 fgt[__NR_FGT_GROUP_IDS__]; + /* * Stage 2 paging state for VMs with nested S2 using a virtual * VMID. diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 34f53707892dfe7bba41620e7adb65f1f8376018..247dfadcdb22e1ef96f92a9d86e66c9eefb44600 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -98,9 +98,9 @@ static inline void __activate_traps_fpsimd32(struct kvm_vcpu *vcpu) id; \ }) -#define compute_undef_clr_set(vcpu, kvm, reg, clr, set) \ +#define compute_trap_clr_set(vcpu, kvm, trap, reg, clr, set) \ do { \ - u64 hfg = kvm->arch.fgu[reg_to_fgt_group_id(reg)]; \ + u64 hfg = kvm->arch.trap[reg_to_fgt_group_id(reg)]; \ set |= hfg & __ ## reg ## _MASK; \ clr |= hfg & __ ## reg ## _nMASK; \ } while(0) @@ -113,7 +113,8 @@ static inline void __activate_traps_fpsimd32(struct kvm_vcpu *vcpu) if (vcpu_has_nv(vcpu) && !is_hyp_ctxt(vcpu)) \ compute_clr_set(vcpu, reg, c, s); \ \ - compute_undef_clr_set(vcpu, kvm, reg, c, s); \ + compute_trap_clr_set(vcpu, kvm, fgu, reg, c, s); \ + compute_trap_clr_set(vcpu, kvm, fgt, reg, c, s); \ \ s |= set; \ c |= clr; \ From patchwork Fri Dec 20 16:46:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917109 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 466FB21A45A; Fri, 20 Dec 2024 16:51:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713480; cv=none; b=ZcAXmgcyeVEnS5zaLldlp+AAXZeDfqFpflSkf3iaqRN6KyTxJmyWQJDaCMIO+ehsLGuKpH0JfC3CzPiEPIZL72/zodIH8+WaBl5FIKhx+8bRvY9vnq+6NRcVrAWnlaGVbk3eqRsNkmOIRhkVaO4YJ6n57bJcc9Gk84hJNx7hy7E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713480; c=relaxed/simple; bh=FQHYwXdmL3GR1X0fB9NNfgXhfHPcJEGCJwnB7whsybM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=iI6IwBOyxyKljMCItKDv8R0jDP77G9DrQJzwbeePDaGruqaRDurGfnG+rlkHhl0vAd/FyozorhgmKiwbv7d3hTavHQvO/16+C44tuitVkzWMXSaR58UhZB+9IZiYW2VyONxsmpFAgulXoIuOJbIkeec+Za9lS9g/wQjA6f/tNRM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=sPgjhNy0; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="sPgjhNy0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 82FB2C4CEDC; Fri, 20 Dec 2024 16:51:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713479; bh=FQHYwXdmL3GR1X0fB9NNfgXhfHPcJEGCJwnB7whsybM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=sPgjhNy0pQKxJmsHGssak/MuCZK0aWt8tI8thh7Ae1mYzo8bd/TG5eVWNdGdoCbat uhsduMcY4oz+ozc2revjhXx5Ldwd1tUs1mbdHzzqvAy//oLujy57t/PM1i9A3p0Nub mdeL6ThJaawK+c185FioTm68KJk0O1YgsDofyKeKjeHv1yFjsK5vsdDBVK/1OJCiZM ldjMcZeZNFIo7QxH02lr5TZehIbMzW4naWTGh4+nUmdwOv9LbldTXEl5+lhS3pke75 AlZHD8K0aqKZ45t2sii65DiV0vD8fj+BJuKdfndpTf5Cf+1tp8appwhRUbYMix1aeT qMpcvUEnLKZQg== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:31 +0000 Subject: [PATCH RFC v3 06/27] KVM: arm64: Pull ctxt_has_ helpers to start of sysreg-sr.h Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-6-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=2305; i=broonie@kernel.org; h=from:subject:message-id; bh=FQHYwXdmL3GR1X0fB9NNfgXhfHPcJEGCJwnB7whsybM=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBZgmK/htQnfZrdQRb9ftVqYA9k0EzKH9YaTlHT OJFL6e+JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgWQAKCRAk1otyXVSH0CzFB/ 9j6wPn0JHK8/QjGznInjFWbpmWtsO145mBHDy9gv2tvca9YZMjsAUZtnDXUvpcSrkpBx/jXpG6ytgq 7XDa+pa2H5NQ1gj/Rq39700Ks+VG2FD4l8mxJW+k67UAJrVWHrcFPmbruxwgKOFEDZEOmns7L+v6u3 qhfFaSKWmiAwdmNqo8rrfoUvLUsBj88wdGyI/UzD0SiEVx/HvDtIpqX9wyORNCGkOhumngUqYW3Sgw wANy0gt4totBOrUpWdnf8xnhpP2NjDOaMMU/HPlMODt07U/NElcodWuT/SQ5lL1MwpMxjcUuATF8wK +JaOWIYQjAWxxaIDZFs42qEFNYffZT X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Rather than add earlier prototypes of specific ctxt_has_ helpers let's just pull all their definitions to the top of sysreg-sr.h so they're all available to all the individual save/restore functions. Signed-off-by: Mark Brown --- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 32 ++++++++++++++---------------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h index a651c43ad679fcc5a13ab7a619e252d96fd46281..8c234d53acb2753c59aa37d7a66f856f2eb87882 100644 --- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h @@ -16,23 +16,6 @@ #include #include -static inline bool ctxt_has_s1poe(struct kvm_cpu_context *ctxt); - -static inline void __sysreg_save_common_state(struct kvm_cpu_context *ctxt) -{ - ctxt_sys_reg(ctxt, MDSCR_EL1) = read_sysreg(mdscr_el1); - - // POR_EL0 can affect uaccess, so must be saved/restored early. - if (ctxt_has_s1poe(ctxt)) - ctxt_sys_reg(ctxt, POR_EL0) = read_sysreg_s(SYS_POR_EL0); -} - -static inline void __sysreg_save_user_state(struct kvm_cpu_context *ctxt) -{ - ctxt_sys_reg(ctxt, TPIDR_EL0) = read_sysreg(tpidr_el0); - ctxt_sys_reg(ctxt, TPIDRRO_EL0) = read_sysreg(tpidrro_el0); -} - static inline struct kvm_vcpu *ctxt_to_vcpu(struct kvm_cpu_context *ctxt) { struct kvm_vcpu *vcpu = ctxt->__hyp_running_vcpu; @@ -83,6 +66,21 @@ static inline bool ctxt_has_s1poe(struct kvm_cpu_context *ctxt) return kvm_has_s1poe(kern_hyp_va(vcpu->kvm)); } +static inline void __sysreg_save_common_state(struct kvm_cpu_context *ctxt) +{ + ctxt_sys_reg(ctxt, MDSCR_EL1) = read_sysreg(mdscr_el1); + + // POR_EL0 can affect uaccess, so must be saved/restored early. + if (ctxt_has_s1poe(ctxt)) + ctxt_sys_reg(ctxt, POR_EL0) = read_sysreg_s(SYS_POR_EL0); +} + +static inline void __sysreg_save_user_state(struct kvm_cpu_context *ctxt) +{ + ctxt_sys_reg(ctxt, TPIDR_EL0) = read_sysreg(tpidr_el0); + ctxt_sys_reg(ctxt, TPIDRRO_EL0) = read_sysreg(tpidrro_el0); +} + static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt) { ctxt_sys_reg(ctxt, SCTLR_EL1) = read_sysreg_el1(SYS_SCTLR); From patchwork Fri Dec 20 16:46:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917110 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E08121B1A7; Fri, 20 Dec 2024 16:51:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713484; cv=none; b=mW3y8eYWls8YRmklBi8I1HEuDqaBzIV6jh5ucNixzQPtLkdks8/5YapbLi1zVJylV2XHva/DrmJ4RW8m/b8/VweTchi3R+sf28k5wCu7L4xrPji5IgnTkluVYf/MRfw5uZkGn5tY6B1CfFgHXKpVDrU/HShufzlIsy5lQzKUOm0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713484; c=relaxed/simple; bh=a+yncXwMMueTmbDqZnDIxIoJ5xsyf/PU5pYLC6ZbRbM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=PWlM12fL2EQ0A9XRbPK5VVZooKDKZLEWs0OQpeKxNNY4BjFxUOPUadzBD+JB/tv7P7ZkyQ4dKX1YIkKGCYOl9BG4nN8lW7mCUebRPiGsBST3U0sjfkDizkmKYMKKnUDYkBsc5MAXKbcu+7JC9OKmhsA8E69pwh40PRkhL9uenAA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=vLhYu+o1; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="vLhYu+o1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2D7A3C4CED3; Fri, 20 Dec 2024 16:51:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713484; bh=a+yncXwMMueTmbDqZnDIxIoJ5xsyf/PU5pYLC6ZbRbM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=vLhYu+o1l++cILw7iyIVCcLat3F3ew01Ad54HxoZ0pxbH6rThGz/L0017gDWjF0Te 2dsAfTr8PLX+dlbs/lw6y+VBYIILA/8P6jovxfMPegV5yFcjF14U1UvdVGD7jL8f73 xwpkqjohLRCroOVtSg0rhhqL5tuTq8e8tSFGANkEH/3xgEMf3iz3fPW5W5WyHfMb13 XoKFe3Aba1VLOG+VjFFcL7zslcQ+OGdwUErrh0dZc9tFWio/bUfXnnVo3X4TY6EzJW 2PXEDecDpmzbhMkIiwKCs+s4zXLdWAGtEDKNHbB1pVUEMKbgGGpL+adtC/HI4R7tF+ x48mrTY9irCgg== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:32 +0000 Subject: [PATCH RFC v3 07/27] KVM: arm64: Convert cpacr_clear_set() to a static inline Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-7-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=2083; i=broonie@kernel.org; h=from:subject:message-id; bh=a+yncXwMMueTmbDqZnDIxIoJ5xsyf/PU5pYLC6ZbRbM=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBZdrU2V+OphtPgF7NwJy6uVDv4tmY4K7qACS27 7872fXSJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgWQAKCRAk1otyXVSH0BvtB/ wI5sZMQ9hxIxROX5W9UZZrhMrY9fD+KkpsCZhFytntZ+8JR14prgNoUKPp3948eBLgtc15UJFySKGQ Ndru+92DG2ZY0Jfpg8pOhwNHdNFRf7i8iy3x6NzdfOWsfqsFgHVk4qGK9zhmwxyzl84dDIWHGhpD6e aRNiSuALRNHMgpvN3KbhOIO2wOCiR1eJz6gqILHINr0Cx6I7KcuiDQ4ryhz0W6rq+fPlwFLAM8lKW4 grkNv1hEE5V4XG9cdtv5p0H99/KNSRxF0mf2l92afCdiNeSWHOqx3stkBvfaEzA96MuM5jrGDz875+ KAGD/VaHBJfxPDBer/1e77U+E3qsCQ X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Currently cpacr_clear_set() is defined as a macro in order to allow it to include a number of build time asserts that the bits being set and cleared are appropriate. While this check is welcome it only works when the arguments are constant which starts to scale poorly as we add SME unless we do multiple updates of the system register. Convert the function to a static inline so that it can accept runtime variable arguments. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_emulate.h | 27 +++++++++------------------ 1 file changed, 9 insertions(+), 18 deletions(-) diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h index 2d91fb88298a263dcd73a4269318f8edf1379650..5f05da7f538d29d321c424233f21b8448d8b4628 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -592,24 +592,15 @@ static __always_inline void kvm_incr_pc(struct kvm_vcpu *vcpu) cptr; \ }) -#define cpacr_clear_set(clr, set) \ - do { \ - BUILD_BUG_ON((set) & CPTR_VHE_EL2_RES0); \ - BUILD_BUG_ON((clr) & CPACR_ELx_E0POE); \ - __build_check_all_or_none((clr), CPACR_ELx_FPEN); \ - __build_check_all_or_none((set), CPACR_ELx_FPEN); \ - __build_check_all_or_none((clr), CPACR_ELx_ZEN); \ - __build_check_all_or_none((set), CPACR_ELx_ZEN); \ - __build_check_all_or_none((clr), CPACR_ELx_SMEN); \ - __build_check_all_or_none((set), CPACR_ELx_SMEN); \ - \ - if (has_vhe() || has_hvhe()) \ - sysreg_clear_set(cpacr_el1, clr, set); \ - else \ - sysreg_clear_set(cptr_el2, \ - __cpacr_to_cptr_clr(clr, set), \ - __cpacr_to_cptr_set(clr, set));\ - } while (0) +static __always_inline void cpacr_clear_set(u64 clr, u64 set) +{ + if (has_vhe() || has_hvhe()) + sysreg_clear_set(cpacr_el1, clr, set); + else + sysreg_clear_set(cptr_el2, + __cpacr_to_cptr_clr(clr, set), + __cpacr_to_cptr_set(clr, set)); +} static __always_inline void kvm_write_cptr_el2(u64 val) { From patchwork Fri Dec 20 16:46:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917111 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 76BA721B1BC; Fri, 20 Dec 2024 16:51:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713489; cv=none; b=fUWJNLxw8Qy5wZSV9QUGRlL3PqMOVXRso+VtNYT5oE4g+gMx+aPrlxn1eYf/GthtpQ3Gm//BZfv7/crdBfFQ0GHUMUGDsSCgUI6lBdwR/LeX69DxQA2lAz3DEtzKeXPTCW35ZY1dCErJEGrKv3aG507GVrb7zdDuV6DbfvnZgQg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713489; c=relaxed/simple; bh=/i3TwaKgCBLSWDUFzgZJ5E4rHb3UDDDbQ/9/hT2MAGs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=QP3EUwUh/YIi/AddD6wIiHG1c19lCqL7axrTeEFVHQanh56xi6R8y6l58jmlJGbTiFJFkLP/Wwp+AJDVz0ItRYpJXuj9XOIGL5Qfcztgs8Eu48VDzKLaoprL8JlLaDUqkh9X+yNfyoqBBWR+6ME5/qZKqBc13aK/walxXWI0G6Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=crEj/i06; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="crEj/i06" Received: by smtp.kernel.org (Postfix) with ESMTPSA id ACAB8C4CEDC; Fri, 20 Dec 2024 16:51:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713488; bh=/i3TwaKgCBLSWDUFzgZJ5E4rHb3UDDDbQ/9/hT2MAGs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=crEj/i06gGmZk5M1LHwTVOq3Ku4eZAwf5Lzjlpp4UJMXlDax1g3QMdBxXpwDIF7FL 3f4Sw2E19x5pgh+/SUWeExgu5ZqBXoC0UQYa9MfQqEiEqAYj7i1uvEoPBmwRt3Zch/ yBRzbMe3fLfIJ0ZQU2nlMSoQ4Z4Ft0xsLUnbt10tgLxn8eAPoNfh4jbCJEWaXWYKkj 6+JriIQTN1rfxrzPFATGPza69g/KlQ6qL7xrR2ftbW4arHneM8tELY2c83conD3+sc Ne7bT4T5Z9rC0XG/I9u3BvNBBAG9H0C3RtojGEnJmqcjV8an0PT22I8GWfXd1YQeOo D+NPSWaiBOsYg== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:33 +0000 Subject: [PATCH RFC v3 08/27] KVM: arm64: Move SVE state access macros after feature test macros Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-8-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=2613; i=broonie@kernel.org; h=from:subject:message-id; bh=/i3TwaKgCBLSWDUFzgZJ5E4rHb3UDDDbQ/9/hT2MAGs=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBaSCMFrVKugleuCqcjESXruDC/s6CaD4yZ5F6Z p7wReKWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgWgAKCRAk1otyXVSH0LqlB/ 9idSTtrsfAftjwtJ51rl1MDfrczjOESiyC+PCFAePnmVi3KdagMZPCJQvdE7wrKQuchGOWvo9/CLq+ cN7EaMnLRHfgp8Z/hUnReRjkCeiHX6BBOj0iNBGSRirbpxgDO95XXMQAGYnQLEeGAXP1431tdMRmzf LAzBVbJtNVeUBBEHCRD0Yn5Wb8AKl7fWQYwAJHJikESsLd6fPuwCVKe+oekBg/R1YaUxiTNfVApgOM s4PZRY/HFMAqM77VIzYl/XflL9D6jQAwR2yKjwLDgt7OK5tGxJk41JJH0hl36jcAdH41yrYuMic6lU kq+tPZijmIyttMxtCnye3BE46zZ1AN X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB In preparation for SME support move the macros used to access SVE state after the feature test macros, we will need to test for SME subfeatures to determine the size of the SME state. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 46 +++++++++++++++++++-------------------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index fca81ede6140c0ee7d03cb6ca8f5eead45b87033..97b617606221e8c11fd2b55d9636848d8453209f 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -934,29 +934,6 @@ struct kvm_vcpu_arch { #define IN_WFI __vcpu_single_flag(sflags, BIT(7)) -/* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */ -#define vcpu_sve_pffr(vcpu) (kern_hyp_va((vcpu)->arch.sve_state) + \ - sve_ffr_offset((vcpu)->arch.sve_max_vl)) - -#define vcpu_sve_max_vq(vcpu) sve_vq_from_vl((vcpu)->arch.sve_max_vl) - -#define vcpu_sve_zcr_elx(vcpu) \ - (unlikely(is_hyp_ctxt(vcpu)) ? ZCR_EL2 : ZCR_EL1) - -#define vcpu_sve_state_size(vcpu) ({ \ - size_t __size_ret; \ - unsigned int __vcpu_vq; \ - \ - if (WARN_ON(!sve_vl_valid((vcpu)->arch.sve_max_vl))) { \ - __size_ret = 0; \ - } else { \ - __vcpu_vq = vcpu_sve_max_vq(vcpu); \ - __size_ret = SVE_SIG_REGS_SIZE(__vcpu_vq); \ - } \ - \ - __size_ret; \ -}) - #define KVM_GUESTDBG_VALID_MASK (KVM_GUESTDBG_ENABLE | \ KVM_GUESTDBG_USE_SW_BP | \ KVM_GUESTDBG_USE_HW | \ @@ -992,6 +969,29 @@ struct kvm_vcpu_arch { #define vcpu_gp_regs(v) (&(v)->arch.ctxt.regs) +/* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */ +#define vcpu_sve_pffr(vcpu) (kern_hyp_va((vcpu)->arch.sve_state) + \ + sve_ffr_offset((vcpu)->arch.sve_max_vl)) + +#define vcpu_sve_max_vq(vcpu) sve_vq_from_vl((vcpu)->arch.sve_max_vl) + +#define vcpu_sve_zcr_elx(vcpu) \ + (unlikely(is_hyp_ctxt(vcpu)) ? ZCR_EL2 : ZCR_EL1) + +#define vcpu_sve_state_size(vcpu) ({ \ + size_t __size_ret; \ + unsigned int __vcpu_vq; \ + \ + if (WARN_ON(!sve_vl_valid((vcpu)->arch.sve_max_vl))) { \ + __size_ret = 0; \ + } else { \ + __vcpu_vq = vcpu_sve_max_vq(vcpu); \ + __size_ret = SVE_SIG_REGS_SIZE(__vcpu_vq); \ + } \ + \ + __size_ret; \ +}) + /* * Only use __vcpu_sys_reg/ctxt_sys_reg if you know you want the * memory backed version of a register, and not the one most recently From patchwork Fri Dec 20 16:46:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917112 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3509D221468; Fri, 20 Dec 2024 16:51:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713492; cv=none; b=eRMDj+LLg9gOcl0kSluhg/TilK0dXsdGd1jDmv4TpqM1CMSQsWiUx8Ez5KqBDvQmvJq5PwHkFiT1ESTfC/l2Hrx1xXeTEPfX2zPl51lkv+4xAficRngbMKsmHKu5uBW9V1pGsMAHMA57w03kg9vhIN8ZQPCRHCZl/SkqI8pMnwI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713492; c=relaxed/simple; bh=2lAEiIWK9uzi24UbNru8zKCgLUdInhjJdDB12WbDPZ8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=XWMuzw3OTRVnV3fyS/H/j575tpaInN+KgMcKOTHRrQQF6YVXqy4hOgagyKg9HJLgM+B72VNyNB2m0U199OzLVbrvqW86xZjDEzIrSrLO7Xjg9hzR3YMz7UafHlf/Qjgos7qd2Vm7v18T9RFlBEfRTcIT3VbkXiyEceSfEi/S4DU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Q1dx1hu+; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Q1dx1hu+" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6DCE2C4CECD; Fri, 20 Dec 2024 16:51:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713491; bh=2lAEiIWK9uzi24UbNru8zKCgLUdInhjJdDB12WbDPZ8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Q1dx1hu+4QMc1mfXYgXW5GgGjr1mUzSxYFJgffUwtQmrpsnMXNfGgX8nb11rLLwX4 XrwGngKrIFIqsSMFXDHY+DeRYIkvUBxaj7lXGhbESDULbxOFuJP0m2annSQVOaftr6 /hnel/AIzS9qfRWSdb0e4LywnwIeogj15u6skZCozHsnbFB5upp0FdfsQid551iBVv ky5aO+ca1oyoNRVkh/T5eu+O8F05tYl87lh8vFqxq6K0e7UrRwyXM2ZgMSmsnPk+vo pmvCYRSDeBpeitZMZfu1C9xvwuRxmh4cksqhwFjs/SPBxZsVfkHIGZM4QyMOJKKLWx V7AuoSMAXwjfg== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:34 +0000 Subject: [PATCH RFC v3 09/27] KVM: arm64: Factor SVE guest exit handling out into a function Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-9-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=3607; i=broonie@kernel.org; h=from:subject:message-id; bh=2lAEiIWK9uzi24UbNru8zKCgLUdInhjJdDB12WbDPZ8=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBb/PNHOdXC2QqIWTml92xipzWpK0C3H1dFAHdq 34bDHjSJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgWwAKCRAk1otyXVSH0H5LB/ 47krYQyM/wfFiJzoZcaTezF83TBLhU5DbVnNlFyU3Egm70Ppq11143hsfCVTZd8BQ1p+1iWtkpMWVq 1BVFdSxpQEeZ8bEmA/oTGLpm+MY3fKmHWviByjr1N6GedW9hPQcxyxVvtZhzu2slXSjRD3RMMYM3PT p61lT89WeRN/AIS3wtwEgYzqigpqu8Hlrv3dUDKYf5HnxmOmqdMd/rfu4Scw1D2z9ZnnO7SFSYzXKU sBXgBW2jIfIRcfTw5ro3Xe4PuKHVUV5bdAl/5/QzwuPwFRSanXJRr4Wo+zAZtyt324t5mz8Sc8N1E/ 9/cg7CYQeavXMq0CfnAD5jwEHozx9w X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB The SVE portion of kvm_vcpu_put() is quite large, especially given the comments required. When we add similar handling for SME the function will get even larger, in order to keep things managable factor the SVE portion out of the main kvm_vcpu_put(). Signed-off-by: Mark Brown --- arch/arm64/kvm/fpsimd.c | 67 +++++++++++++++++++++++++++---------------------- 1 file changed, 37 insertions(+), 30 deletions(-) diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index 09b65abaf9db60cc57dbc554ad2108a80c2dc46b..3c2e0b96877ac5b4f3b9d8dfa38975f11b74b60d 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -151,6 +151,41 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu) } } +static void kvm_vcpu_put_sve(struct kvm_vcpu *vcpu) +{ + u64 zcr; + + if (!vcpu_has_sve(vcpu)) + return; + + zcr = read_sysreg_el1(SYS_ZCR); + + /* + * If the vCPU is in the hyp context then ZCR_EL1 is loaded + * with its vEL2 counterpart. + */ + __vcpu_sys_reg(vcpu, vcpu_sve_zcr_elx(vcpu)) = zcr; + + /* + * Restore the VL that was saved when bound to the CPU, which + * is the maximum VL for the guest. Because the layout of the + * data when saving the sve state depends on the VL, we need + * to use a consistent (i.e., the maximum) VL. Note that this + * means that at guest exit ZCR_EL1 is not necessarily the + * same as on guest entry. + * + * ZCR_EL2 holds the guest hypervisor's VL when running a + * nested guest, which could be smaller than the max for the + * vCPU. Similar to above, we first need to switch to a VL + * consistent with the layout of the vCPU's SVE state. KVM + * support for NV implies VHE, so using the ZCR_EL1 alias is + * safe. + */ + if (!has_vhe() || (vcpu_has_nv(vcpu) && !is_hyp_ctxt(vcpu))) + sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, + SYS_ZCR_EL1); +} + /* * Write back the vcpu FPSIMD regs if they are dirty, and invalidate the * cpu FPSIMD regs so that they can't be spuriously reused if this vcpu @@ -179,38 +214,10 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu) } if (guest_owns_fp_regs()) { - if (vcpu_has_sve(vcpu)) { - u64 zcr = read_sysreg_el1(SYS_ZCR); - - /* - * If the vCPU is in the hyp context then ZCR_EL1 is - * loaded with its vEL2 counterpart. - */ - __vcpu_sys_reg(vcpu, vcpu_sve_zcr_elx(vcpu)) = zcr; - - /* - * Restore the VL that was saved when bound to the CPU, - * which is the maximum VL for the guest. Because the - * layout of the data when saving the sve state depends - * on the VL, we need to use a consistent (i.e., the - * maximum) VL. - * Note that this means that at guest exit ZCR_EL1 is - * not necessarily the same as on guest entry. - * - * ZCR_EL2 holds the guest hypervisor's VL when running - * a nested guest, which could be smaller than the - * max for the vCPU. Similar to above, we first need to - * switch to a VL consistent with the layout of the - * vCPU's SVE state. KVM support for NV implies VHE, so - * using the ZCR_EL1 alias is safe. - */ - if (!has_vhe() || (vcpu_has_nv(vcpu) && !is_hyp_ctxt(vcpu))) - sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, - SYS_ZCR_EL1); - } + kvm_vcpu_put_sve(vcpu); /* - * Flush (save and invalidate) the fpsimd/sve state so that if + * Flush (save and invalidate) the FP state so that if * the host tries to use fpsimd/sve, it's not using stale data * from the guest. * From patchwork Fri Dec 20 16:46:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917113 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F29C5221462; Fri, 20 Dec 2024 16:51:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713496; cv=none; b=EFixN4Lm8NRd0jjcJaUR0drfhzE9s1B6WnJatyqT3uaIBLKNCKykgNOZrxxafSwvf1r8SjqrYjnjIyhyczSnR8OVfzMFVNWzC3MJwuWxIDGwnEALXamlFJEaoOAJw8XnFFuMyu2cnL+U/CrsR1Z/HEMJsl7PhoFmECK2LPeXmHI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713496; c=relaxed/simple; bh=3aaZhNirOZ+7l9BeX+c3sgo5KaaCuR6ys35EPFSwf5I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dsfZJZMpnRHdYTOt7r+KL9mnQbseDw2esYMXbXOHuYLsMo/xFmyjpFwi13IJJDePhcARLu/vjXTXWhdTgsG8LmbDmPzTo3omWrqjIBMJyoAe0Q+wTJbcleRTd9GXh9KSe/8xCKFva2Wjht5uJ2M18Rvm/bxCwWrbbpaqhmgDTNo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ivQ4iWm1; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ivQ4iWm1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3109EC4CEDC; Fri, 20 Dec 2024 16:51:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713495; bh=3aaZhNirOZ+7l9BeX+c3sgo5KaaCuR6ys35EPFSwf5I=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ivQ4iWm1QqcGDvdvFW2D7CizC4bu4+NLYAkUu1z6hEuoDkKIRPNgFdoOjGrPLZPx0 8iunglxHnzAf/Kw+EyWDi999khVY2SmWxFb8jn6V/WCUXOzthp0RDO0TjfSwCpBPx1 L/2qboNeQKxSTgBDgs9xsKF1MgOLis83ORHb6s1Osr0wy0iI+gkFhZnXnAZc6IGlMA 9CLcet2iXGVta8LAt5sse2AR6W67K1pGEafmWc71jWluWW6OPtkr4S3gseNAQI9cHz XN4+4r8upRTpK463qnJEv/IUwvzk/VqN6OvUDrNayOJuUcyn+AFmXPH8xOIB/OsExE o9Js31he67Nqg== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:35 +0000 Subject: [PATCH RFC v3 10/27] KVM: arm64: Rename SVE finalization constants to be more general Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-10-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=7533; i=broonie@kernel.org; h=from:subject:message-id; bh=3aaZhNirOZ+7l9BeX+c3sgo5KaaCuR6ys35EPFSwf5I=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBcHuq5Lk45QXE8KUQ/QhEs1N1N14WzcOZ4wGiW krf/jliJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgXAAKCRAk1otyXVSH0IF6B/ 4t9q0gL7iHDyrcIJDUPG9jcN0euzIJJrpuoBsjuqccr3yuWuHS+PskLa+15vbpMf/dMpx6VTNMpISo Hw6QNQq5HECzh66lBgfetGl9X28k6KB+mid8G6x/X5XrkYPLBL8fKzWu5kowdwGINS1WOiwmkybCIe ShCjwWUEz9BNx3ertYmduRBlhprlCpEsQ4HkTbtlJzXd+mnkjgSh3qpwND2fqFoOWOaLs8UO+62vIA /gHCB3DrMJtiqB1jvIvtFW1dmg4hlPEe0UHsNRglkTmz8yrW3HNQLVfv+aLRFFmtuNvldsTEX09MVN GVJBvuszAqq3rHyaa7GqsuPLCjBtkE X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Due to the overlap between SVE and SME vector length configuration created by streaming mode SVE we will finalize both at once. Rename the existing finalization to use _VEC (vector) for the naming to avoid confusion. Since this includes the userspace API we create an alias KVM_ARM_VCPU_VEC for the existing KVM_ARM_VCPU_SVE capability, existing code which does not enable SME will be unaffected and any SME only code will not need to use SVE constants. No functional change. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 6 ++++-- arch/arm64/include/uapi/asm/kvm.h | 6 ++++++ arch/arm64/kvm/guest.c | 10 +++++----- arch/arm64/kvm/hyp/nvhe/pkvm.c | 2 +- arch/arm64/kvm/reset.c | 20 ++++++++++---------- 5 files changed, 26 insertions(+), 18 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 97b617606221e8c11fd2b55d9636848d8453209f..f64ad573573cf000c4644f12f9e072a2fdfc3824 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -873,7 +873,7 @@ struct kvm_vcpu_arch { /* KVM_ARM_VCPU_INIT completed */ #define VCPU_INITIALIZED __vcpu_single_flag(cflags, BIT(0)) /* SVE config completed */ -#define VCPU_SVE_FINALIZED __vcpu_single_flag(cflags, BIT(1)) +#define VCPU_VEC_FINALIZED __vcpu_single_flag(cflags, BIT(1)) /* Exception pending */ #define PENDING_EXCEPTION __vcpu_single_flag(iflags, BIT(0)) @@ -948,6 +948,8 @@ struct kvm_vcpu_arch { #define vcpu_has_sve(vcpu) kvm_has_sve((vcpu)->kvm) #endif +#define vcpu_has_vec(vcpu) vcpu_has_sve(vcpu) + #ifdef CONFIG_ARM64_PTR_AUTH #define vcpu_has_ptrauth(vcpu) \ ((cpus_have_final_cap(ARM64_HAS_ADDRESS_AUTH) || \ @@ -1414,7 +1416,7 @@ struct kvm *kvm_arch_alloc_vm(void); int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature); bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu); -#define kvm_arm_vcpu_sve_finalized(vcpu) vcpu_get_flag(vcpu, VCPU_SVE_FINALIZED) +#define kvm_arm_vcpu_vec_finalized(vcpu) vcpu_get_flag(vcpu, VCPU_VEC_FINALIZED) #define kvm_has_mte(kvm) \ (system_supports_mte() && \ diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h index 66736ff04011e0fa9fcfb74154d5613bf4ee89f7..9d80d22af9d4e00204f5096fb7c8c2ee8c3646c1 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -109,6 +109,12 @@ struct kvm_regs { #define KVM_ARM_VCPU_PTRAUTH_GENERIC 6 /* VCPU uses generic authentication */ #define KVM_ARM_VCPU_HAS_EL2 7 /* Support nested virtualization */ +/* + * An alias for _SVE since we finalize VL configuration for both SVE and SME + * simultaneously. + */ +#define KVM_ARM_VCPU_VEC KVM_ARM_VCPU_SVE + struct kvm_vcpu_init { __u32 target; __u32 features[7]; diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index 12dad841f2a51276eee4d4da7400c1b2a5732ff8..62ff51d6e4584acc71205f5d4b1d2f3d2e2d2f88 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -342,7 +342,7 @@ static int set_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) if (!vcpu_has_sve(vcpu)) return -ENOENT; - if (kvm_arm_vcpu_sve_finalized(vcpu)) + if (kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; /* too late! */ if (WARN_ON(vcpu->arch.sve_state)) @@ -497,7 +497,7 @@ static int get_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) if (ret) return ret; - if (!kvm_arm_vcpu_sve_finalized(vcpu)) + if (!kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; if (copy_to_user(uptr, vcpu->arch.sve_state + region.koffset, @@ -523,7 +523,7 @@ static int set_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) if (ret) return ret; - if (!kvm_arm_vcpu_sve_finalized(vcpu)) + if (!kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; if (copy_from_user(vcpu->arch.sve_state + region.koffset, uptr, @@ -657,7 +657,7 @@ static unsigned long num_sve_regs(const struct kvm_vcpu *vcpu) return 0; /* Policed by KVM_GET_REG_LIST: */ - WARN_ON(!kvm_arm_vcpu_sve_finalized(vcpu)); + WARN_ON(!kvm_arm_vcpu_vec_finalized(vcpu)); return slices * (SVE_NUM_PREGS + SVE_NUM_ZREGS + 1 /* FFR */) + 1; /* KVM_REG_ARM64_SVE_VLS */ @@ -675,7 +675,7 @@ static int copy_sve_reg_indices(const struct kvm_vcpu *vcpu, return 0; /* Policed by KVM_GET_REG_LIST: */ - WARN_ON(!kvm_arm_vcpu_sve_finalized(vcpu)); + WARN_ON(!kvm_arm_vcpu_vec_finalized(vcpu)); /* * Enumerate this first, so that userspace can save/restore in diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 446a9114b0d3ee4323a9cd8d618d36035e85e4d0..0a4e1f5105592b23a0505bf7680c66e76b5c2a65 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -314,7 +314,7 @@ static void pkvm_vcpu_init_sve(struct pkvm_hyp_vcpu *hyp_vcpu, struct kvm_vcpu * struct kvm_vcpu *vcpu = &hyp_vcpu->vcpu; if (!vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) - vcpu_clear_flag(vcpu, VCPU_SVE_FINALIZED); + vcpu_clear_flag(vcpu, VCPU_VEC_FINALIZED); } static int init_pkvm_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu, diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index 803e11b0dc8f5eb74b07b0ad745b0c4f666713d5..ce726b1d4e8e90cfd4459a6cb9c67b8805424e22 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -92,7 +92,7 @@ static void kvm_vcpu_enable_sve(struct kvm_vcpu *vcpu) * Finalize vcpu's maximum SVE vector length, allocating * vcpu->arch.sve_state as necessary. */ -static int kvm_vcpu_finalize_sve(struct kvm_vcpu *vcpu) +static int kvm_vcpu_finalize_vec(struct kvm_vcpu *vcpu) { void *buf; unsigned int vl; @@ -122,21 +122,21 @@ static int kvm_vcpu_finalize_sve(struct kvm_vcpu *vcpu) } vcpu->arch.sve_state = buf; - vcpu_set_flag(vcpu, VCPU_SVE_FINALIZED); + vcpu_set_flag(vcpu, VCPU_VEC_FINALIZED); return 0; } int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature) { switch (feature) { - case KVM_ARM_VCPU_SVE: - if (!vcpu_has_sve(vcpu)) + case KVM_ARM_VCPU_VEC: + if (!vcpu_has_vec(vcpu)) return -EINVAL; - if (kvm_arm_vcpu_sve_finalized(vcpu)) + if (kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; - return kvm_vcpu_finalize_sve(vcpu); + return kvm_vcpu_finalize_vec(vcpu); } return -EINVAL; @@ -144,7 +144,7 @@ int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature) bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu) { - if (vcpu_has_sve(vcpu) && !kvm_arm_vcpu_sve_finalized(vcpu)) + if (vcpu_has_vec(vcpu) && !kvm_arm_vcpu_vec_finalized(vcpu)) return false; return true; @@ -161,7 +161,7 @@ void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu) kfree(vcpu->arch.ccsidr); } -static void kvm_vcpu_reset_sve(struct kvm_vcpu *vcpu) +static void kvm_vcpu_reset_vec(struct kvm_vcpu *vcpu) { if (vcpu_has_sve(vcpu)) memset(vcpu->arch.sve_state, 0, vcpu_sve_state_size(vcpu)); @@ -204,11 +204,11 @@ void kvm_reset_vcpu(struct kvm_vcpu *vcpu) if (loaded) kvm_arch_vcpu_put(vcpu); - if (!kvm_arm_vcpu_sve_finalized(vcpu)) { + if (!kvm_arm_vcpu_vec_finalized(vcpu)) { if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) kvm_vcpu_enable_sve(vcpu); } else { - kvm_vcpu_reset_sve(vcpu); + kvm_vcpu_reset_vec(vcpu); } if (vcpu_el1_is_32bit(vcpu)) From patchwork Fri Dec 20 16:46:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917114 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9CA96223C43; Fri, 20 Dec 2024 16:51:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713499; cv=none; b=Pkjp3oWLUwdZFYhlB6BPC2q1Qo0BnTaX4PtChOYr/IncG3JKg9EQtBFy/DbMC78R0c0WEvscnCsPiU4w2Vo7p3o+nAp9qi43/ag6EafoM6NLexQlvIb4SRTWbEQ0v/9MQ6fBMTG5eQ8yPpELpaYIHfXYvuKyHZcULSJ+Vka3hdM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713499; c=relaxed/simple; bh=7OCb0PhrPXhOe9yy7xz5wCwKFvJttcECxrueTTUU5Bg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ghLxwiRY9XydzeCzxadyk5wFOfb512FlGZDCWaZBksr7teCkzsSDpiriYtS2bHr0tI0AObJvBW3xFnp3ZbGBIP/W+Td3GCHQ420kJLCYXfUWYhZtKNeVz1NOPlKqjKYN+MM3+9lyFEiYfEfJF/SV5QfHYJ86eQaPCS7ifAg0oEY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YaL3Hkwi; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YaL3Hkwi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E640FC4CEDE; Fri, 20 Dec 2024 16:51:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713499; bh=7OCb0PhrPXhOe9yy7xz5wCwKFvJttcECxrueTTUU5Bg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=YaL3HkwitvdguEJtZx4peZ1nnd94NuDI3EO0yV7lLCtKtqCmnzmDyigmbwEf1rw7G rSxF63rZe5Ln56L6AQlL9DdFf3pRla/QIQLVKv/0oDJWkDjUQWSPU+qMF/J/rCbxJ8 4/a0CUYdUmRnsacXDXcu0viadw+w6VSfMNLbai+pAUnNh7uqZdUFC2eac3rj5W+jn9 YiUQSWsoW1SNLgJUBgGsvpyYvFWgqpndmbFg2uecCOV7keCtK9M0yLcrwpda6cucQt Glphl0JKT3v9Tz9XlRTGkvZ8gVAVMCFrwmxhYUj5kYH3d9rdM3UAoBASZIFJGPhYGb X+ncROwd55qGQ== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:36 +0000 Subject: [PATCH RFC v3 11/27] KVM: arm64: Document the KVM ABI for SME Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-11-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=13316; i=broonie@kernel.org; h=from:subject:message-id; bh=7OCb0PhrPXhOe9yy7xz5wCwKFvJttcECxrueTTUU5Bg=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBdRNgLlQWYZwBMDz7K3tsDc9TPCzzl8Tlf9+1u S2tdtiqJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgXQAKCRAk1otyXVSH0My0B/ 9OvwJoLIfGU3NdiiBhPEcW9W2PNUpf6MX7v6I+6kVucr/Rr4rLSan0l/BUU8rt5h3Wz7z73XgEV5Yn HrYIA6KpgALLDcuX6N8zMqfuUrq7Nqh0G66EAndhENfxcnIJkHVMRSBZCBDk5aWDKaG8ZEpHIsL2Z9 SKgc2D0WWbbyXPxkZ+Fvecz44pJOss5Wv8MXIXTVLlSA6mxtDcAK0wjjpsp8JcRYbqCpgjdkTD2r5K UPaePDsXJ4SEF2jkgDzvxUj7AKakFotr1xskAwiWkn+qYBc7JuW0Hb97qwTiDiPJRP9okX8wLVe0bK DMe4nGrJE9wcIkkrqg8Eu9IJBboDMH X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME, the Scalable Matrix Extension, is an arm64 extension which adds support for matrix operations, with core concepts patterned after SVE. SVE introduced some complication in the ABI since it adds new vector floating point registers with runtime configurable size, the size being controlled by a prameter called the vector length (VL). To provide control of this to VMMs we offer two phase configuration of SVE, SVE must first be enabled for the vCPU with KVM_ARM_VCPU_INIT(KVM_ARM_VCPU_SVE), after which vector length may then be configured but the configurably sized floating point registers are inaccessible until finalized with a call to KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_SVE) after which the configurably sized registers can be accessed. SME introduces an additional independent configurable vector length which as well as controlling the size of the new ZA register also provides an alternative view of the configurably sized SVE registers (known as streaming mode) with the guest able to switch between the two modes as it pleases. There is also a fixed sized register ZT0 introduced in SME2. As well as streaming mode the guest may enable and disable ZA and (where SME2 is available) ZT0 dynamically independently of streaming mode. These modes are controlled via the system register SVCR. We handle the configuration of the vector length for SME in a similar manner to SVE, requiring initialization and finalization of the feature with a pseudo register controlling the available SME vector lengths as for SVE. Further, if the guest has both SVE and SME then finalizing one prevents further configuration of the vector length for the other. Where both SVE and SME are configured for the guest we always present the SVE registers to userspace as having the larger of the configured maximum SVE and SME vector lengths, discarding extra data at load time and zero padding on read as required if the active vector length is lower. Note that this means that enabling or disabling streaming mode while the guest is stopped will not zero Zn or Pn as it will when the guest is running, but it does allow SVCR, Zn and Pn to be read and written in any order. Userspace access to ZA and (if configured) ZT0 is always available, they will be zeroed when the guest runs if disabled in SVCR and the value read will be zero if the guest stops with them disabled. This mirrors the behaviour of the architecture, enabling access causes ZA and ZT0 to be zeroed, while allowing access to SVCR, ZA and ZT0 to be performed in any order. If SME is enabled for a guest without SVE then the FPSIMD Vn registers must be accessed via the low 128 bits of the SVE Zn registers as is the case when SVE is enabled. This is not ideal but allows access to SVCR and the registers in any order without duplication or ambiguity about which values should take effect. This may be an issue for VMMs that are unaware of SME on systems that implement it without SVE if they let SME be enabled, the lack of access to Vn may surprise them, but it seems like an unusual implementation choice. For SME unware VMMs on systems with both SVE and SME support the SVE registers may be larger than expected, this should be less disruptive than on a system without SVE as they will simply ignore the high bits of the registers. Signed-off-by: Mark Brown --- Documentation/virt/kvm/api.rst | 117 +++++++++++++++++++++++++++++------------ 1 file changed, 82 insertions(+), 35 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 454c2aaa155e5b994ee1f68502d8fdf55cf6700a..1d9270f8492392d9142cc38abe5a25cf5ab5f7d6 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -406,7 +406,7 @@ Errors: instructions from device memory (arm64) ENOSYS data abort outside memslots with no syndrome info and KVM_CAP_ARM_NISV_TO_USER not enabled (arm64) - EPERM SVE feature set but not finalized (arm64) + EPERM SVE or SME feature set but not finalized (arm64) ======= ============================================================== This ioctl is used to run a guest virtual cpu. While there are no @@ -2580,12 +2580,12 @@ Specifically: 0x6020 0000 0010 00d5 FPCR 32 fp_regs.fpcr ======================= ========= ===== ======================================= -.. [1] These encodings are not accepted for SVE-enabled vcpus. See - :ref:`KVM_ARM_VCPU_INIT`. +.. [1] These encodings are not accepted for SVE enabled vcpus. See + :ref:`KVM_ARM_VCPU_INIT`. They are also not accepted when SME is + enabled without SVE and the vcpu is in streaming mode. The equivalent register content can be accessed via bits [127:0] of - the corresponding SVE Zn registers instead for vcpus that have SVE - enabled (see below). + the corresponding SVE Zn registers in these cases (see below). arm64 CCSIDR registers are demultiplexed by CSSELR value:: @@ -2616,24 +2616,34 @@ arm64 SVE registers have the following bit patterns:: 0x6050 0000 0015 060 FFR bits[256*slice + 255 : 256*slice] 0x6060 0000 0015 ffff KVM_REG_ARM64_SVE_VLS pseudo-register -Access to register IDs where 2048 * slice >= 128 * max_vq will fail with -ENOENT. max_vq is the vcpu's maximum supported vector length in 128-bit -quadwords: see [2]_ below. +arm64 SME registers have the following bit patterns: + + 0x6080 0000 0017 00 ZA.H[n] bits[2048*slice + 2047 : 2048*slice] + 0x60XX 0000 0017 0100 ZT0 + 0x6060 0000 0017 fffe KVM_REG_ARM64_SME_VLS pseudo-register + +Access to Z, P or ZA register IDs where 2048 * slice >= 128 * max_vq +will fail with ENOENT. max_vq is the vcpu's maximum supported vector +length in 128-bit quadwords: see [2]_ below. + +Access to the ZA and ZT0 registers is only available if SVCR.ZA is set +to 1. These registers are only accessible on vcpus for which SVE is enabled. See KVM_ARM_VCPU_INIT for details. -In addition, except for KVM_REG_ARM64_SVE_VLS, these registers are not -accessible until the vcpu's SVE configuration has been finalized -using KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_SVE). See KVM_ARM_VCPU_INIT -and KVM_ARM_VCPU_FINALIZE for more information about this procedure. +In addition, except for KVM_REG_ARM64_SVE_VLS and +KVM_REG_ARM64_SME_VLS, these registers are not accessible until the +vcpu's SVE and SME configuration has been finalized using +KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC). See KVM_ARM_VCPU_INIT and +KVM_ARM_VCPU_FINALIZE for more information about this procedure. -KVM_REG_ARM64_SVE_VLS is a pseudo-register that allows the set of vector -lengths supported by the vcpu to be discovered and configured by -userspace. When transferred to or from user memory via KVM_GET_ONE_REG -or KVM_SET_ONE_REG, the value of this register is of type -__u64[KVM_ARM64_SVE_VLS_WORDS], and encodes the set of vector lengths as -follows:: +KVM_REG_ARM64_SVE_VLS and KVM_ARM64_VCPU_SME_VLS are pseudo-registers +that allows the set of vector lengths supported by the vcpu to be +discovered and configured by userspace. When transferred to or from +user memory via KVM_GET_ONE_REG or KVM_SET_ONE_REG, the value of this +register is of type __u64[KVM_ARM64_SVE_VLS_WORDS], and encodes the +set of vector lengths as follows:: __u64 vector_lengths[KVM_ARM64_SVE_VLS_WORDS]; @@ -2645,19 +2655,25 @@ follows:: /* Vector length vq * 16 bytes not supported */ .. [2] The maximum value vq for which the above condition is true is - max_vq. This is the maximum vector length available to the guest on - this vcpu, and determines which register slices are visible through - this ioctl interface. + max_vq. This is the maximum vector length currently available to + the guest on this vcpu, and determines which register slices are + visible through this ioctl interface. + + If SME is supported then the max_vq used for the Z and P registers + then while SVCR.SM is 1 this vector length will be the maximum SME + vector length available for the guest, otherwise it will be the + maximum SVE vector length available. (See Documentation/arch/arm64/sve.rst for an explanation of the "vq" nomenclature.) -KVM_REG_ARM64_SVE_VLS is only accessible after KVM_ARM_VCPU_INIT. -KVM_ARM_VCPU_INIT initialises it to the best set of vector lengths that -the host supports. +KVM_REG_ARM64_SVE_VLS and KVM_REG_ARM_SME_VLS are only accessible +after KVM_ARM_VCPU_INIT. KVM_ARM_VCPU_INIT initialises them to the +best set of vector lengths that the host supports. -Userspace may subsequently modify it if desired until the vcpu's SVE -configuration is finalized using KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_SVE). +Userspace may subsequently modify these registers if desired until the +vcpu's SVE and SME configuration is finalized using +KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC). Apart from simply removing all vector lengths from the host set that exceed some value, support for arbitrarily chosen sets of vector lengths @@ -2665,8 +2681,8 @@ is hardware-dependent and may not be available. Attempting to configure an invalid set of vector lengths via KVM_SET_ONE_REG will fail with EINVAL. -After the vcpu's SVE configuration is finalized, further attempts to -write this register will fail with EPERM. +After the vcpu's SVE or SME configuration is finalized, further +attempts to write these registers will fail with EPERM. arm64 bitmap feature firmware pseudo-registers have the following bit pattern:: @@ -3449,6 +3465,7 @@ The initial values are defined as: - General Purpose registers, including PC and SP: set to 0 - FPSIMD/NEON registers: set to 0 - SVE registers: set to 0 + - SME registers: set to 0 - System registers: Reset to their architecturally defined values as for a warm reset to EL1 (resp. SVC) @@ -3491,7 +3508,7 @@ Possible features: - KVM_ARM_VCPU_SVE: Enables SVE for the CPU (arm64 only). Depends on KVM_CAP_ARM_SVE. - Requires KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_SVE): + Requires KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC): * After KVM_ARM_VCPU_INIT: @@ -3499,7 +3516,7 @@ Possible features: initial value of this pseudo-register indicates the best set of vector lengths possible for a vcpu on this host. - * Before KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_SVE): + * Before KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC}): - KVM_RUN and KVM_GET_REG_LIST are not available; @@ -3512,11 +3529,40 @@ Possible features: KVM_SET_ONE_REG, to modify the set of vector lengths available for the vcpu. - * After KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_SVE): + * After KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC): - the KVM_REG_ARM64_SVE_VLS pseudo-register is immutable, and can no longer be written using KVM_SET_ONE_REG. + - KVM_ARM_VCPU_SME: Enables SME for the CPU (arm64 only). + Depends on KVM_CAP_ARM_SME. + Requires KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC): + + * After KVM_ARM_VCPU_INIT: + + - KVM_REG_ARM64_SME_VLS may be read using KVM_GET_ONE_REG: the + initial value of this pseudo-register indicates the best set of + vector lengths possible for a vcpu on this host. + + * Before KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC}): + + - KVM_RUN and KVM_GET_REG_LIST are not available; + + - KVM_GET_ONE_REG and KVM_SET_ONE_REG cannot be used to access + the scalable architectural SVE registers + KVM_REG_ARM64_SVE_ZREG(), KVM_REG_ARM64_SVE_PREG() or + KVM_REG_ARM64_SVE_FFR, the matrix register + KVM_REG_ARM64_SME_ZA() or the LUT register KVM_REG_ARM64_ZT(); + + - KVM_REG_ARM64_SME_VLS may optionally be written using + KVM_SET_ONE_REG, to modify the set of vector lengths available + for the vcpu. + + * After KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_VEC): + + - the KVM_REG_ARM64_SME_VLS pseudo-register is immutable, and can + no longer be written using KVM_SET_ONE_REG. + 4.83 KVM_ARM_PREFERRED_TARGET ----------------------------- @@ -5086,11 +5132,12 @@ Errors: Recognised values for feature: - ===== =========================================== - arm64 KVM_ARM_VCPU_SVE (requires KVM_CAP_ARM_SVE) - ===== =========================================== + ===== ============================================================== + arm64 KVM_ARM_VCPU_VEC (requires KVM_CAP_ARM_SVE or KVM_CAP_ARM_SME) + arm64 KVM_ARM_VCPU_SVE (alias for KVM_ARM_VCPU_VEC) + ===== ============================================================== -Finalizes the configuration of the specified vcpu feature. +Finalizes the configuration of the specified vcpu features. The vcpu must already have been initialised, enabling the affected feature, by means of a successful :ref:`KVM_ARM_VCPU_INIT ` call with the From patchwork Fri Dec 20 16:46:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917115 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71F82225A32; Fri, 20 Dec 2024 16:51:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713503; cv=none; b=U3ij5z2KOVIY+Hsxk25VtCdcSICfd7WWdkkgugWwA2QjZf4ZeUKmQ9ZDP88PppNzqJiXjVfWAtErNzKUzj5A6LsoOdirQkFbCteoQNdwh9twiVN2Zzq2Q3l0LqSHKg6pYzRViavKCiEpJxUWxtKELBP+LMI8IoOisCQ6foCOfvI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713503; c=relaxed/simple; bh=h8tZV5l27wMi7QZ5NW8MxQzjInn8bIQ6N+gjfbDmOZc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=LhgjSqPbtN4LQ9DvXNCU7NorhBSNVExb1k9rtWnv2zRi+vUbbt3GZivdJrXeme0N6MLKhUj+lJHDem98h50bcYXPaWcbdGkP9cUTJdBj+fSGOcOrxC5l748wU52k6jkOFrJHh8BZwX2zpnVwiJhbKB3/mdxNYCsH2K8UMcCkg74= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dkAByGPQ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dkAByGPQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A56C0C4CECD; Fri, 20 Dec 2024 16:51:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713503; bh=h8tZV5l27wMi7QZ5NW8MxQzjInn8bIQ6N+gjfbDmOZc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=dkAByGPQ/h/zgIqhkzJAORHDMIhQJ3TUM9neW6eHt1GtXikZn+iCRia/Y8bCX+MZc 5wyGc+VLJtOfcT4OfttexHbBH7ITBNFBZz0KQoRessnzWckchu2FFCjawHwIKgCpVS jKdpNLMhuhnseRWg0iOckGMpYYdYc9Eja8Dfbq7OKAGZe2mrbzsjT8LV0SL2ZRlXRF U4LwyFFX985RvP+Saycum72bN+ZS/eueiV7qjwC3WMyF6E6dKNWTrZYPCPV2qmxcQM OJBavdKvXqCLr4VxkTxDhcaZHx94r+YVGrNOIRkfh0hPt7wMqWuqm/hBSthougxx5f 6KYxmggs3xcsQ== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:37 +0000 Subject: [PATCH RFC v3 12/27] KVM: arm64: Define internal features for SME Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-12-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=2935; i=broonie@kernel.org; h=from:subject:message-id; bh=h8tZV5l27wMi7QZ5NW8MxQzjInn8bIQ6N+gjfbDmOZc=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBd4w3/lE8UZpVmijwql4FlP/nsQSz593ePQ9Xc nGIMzxOJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgXQAKCRAk1otyXVSH0MdEB/ 9qMnB/e2qi9gqsYL5d2AWSroqgmWPN0FOQjIOFJOU3GIyj2480egTCTNt6+EkHDe1haabUHYho0egr PAFlfc0Lr4+phfys3RTH/dtQr/P++q1GlGCH9aCUDiKl9enB74nktMfgPryQnwNteNu0AwZFCr9K6F 8aliU8+zvqfIMYjkXWJyCDZTy7/TyN6+orQ9D4vPihMYCODSFMJ160xyaoZWsYMkg38v4LjPm7QrwX mfBR7ta2dupKszGKrM1s2SZKONb4tOKezSRbT2D8OSLWJ4ELho14oEAIM3LU5ZgXqX3bPVxan8v/v2 gM5HTvfMXhobBSg4P6LEkPzoOFYEg6 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB In order to simplify interdependencies in the rest of the series define the feature detection for SME and it's subfeatures. Due to the need for vector length configuration we define a flag for SME like for SVE. We also have two subfeatures which add architectural state, FA64 and SME2, which are configured via the normal ID register scheme. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 23 +++++++++++++++++++++-- arch/arm64/kvm/sys_regs.c | 2 +- 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index f64ad573573cf000c4644f12f9e072a2fdfc3824..022214e57e74404e8d590a5820a9e77160869b1b 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -39,7 +39,7 @@ #define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS -#define KVM_VCPU_MAX_FEATURES 7 +#define KVM_VCPU_MAX_FEATURES 9 #define KVM_VCPU_VALID_FEATURES (BIT(KVM_VCPU_MAX_FEATURES) - 1) #define KVM_REQ_SLEEP \ @@ -339,6 +339,8 @@ struct kvm_arch { #define KVM_ARCH_FLAG_FGU_INITIALIZED 8 /* SVE exposed to guest */ #define KVM_ARCH_FLAG_GUEST_HAS_SVE 9 + /* SME exposed to guest */ +#define KVM_ARCH_FLAG_GUEST_HAS_SME 10 unsigned long flags; /* VM-wide vCPU feature set */ @@ -948,7 +950,16 @@ struct kvm_vcpu_arch { #define vcpu_has_sve(vcpu) kvm_has_sve((vcpu)->kvm) #endif -#define vcpu_has_vec(vcpu) vcpu_has_sve(vcpu) +#define kvm_has_sme(kvm) \ + test_bit(KVM_ARCH_FLAG_GUEST_HAS_SME, &(kvm)->arch.flags) + +#ifdef __KVM_NVHE_HYPERVISOR__ +#define vcpu_has_sme(vcpu) kvm_has_sme(kern_hyp_va((vcpu)->kvm)) +#else +#define vcpu_has_sme(vcpu) kvm_has_sme((vcpu)->kvm) +#endif + +#define vcpu_has_vec(vcpu) (vcpu_has_sve(vcpu) || vcpu_has_sme(vcpu)) #ifdef CONFIG_ARM64_PTR_AUTH #define vcpu_has_ptrauth(vcpu) \ @@ -1542,4 +1553,12 @@ void kvm_set_vm_id_reg(struct kvm *kvm, u32 reg, u64 val); #define kvm_has_s1poe(k) \ (kvm_has_feat((k), ID_AA64MMFR3_EL1, S1POE, IMP)) +#define kvm_has_fa64(k) \ + (system_supports_sme() && \ + kvm_has_feat((k), ID_AA64SMFR0_EL1, FA64, IMP)) + +#define kvm_has_sme2(k) \ + (system_supports_sme() && \ + kvm_has_feat((k), ID_AA64PFR1_EL1, SME, SME2)) + #endif /* __ARM64_KVM_HOST_H__ */ diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 83c6b4a07ef56cf0ed9c8751ec80686f45dca6b2..1b16716a6d53525fbe694cc8d5d009d72b6ce416 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1727,7 +1727,7 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu, static unsigned int sme_visibility(const struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd) { - if (kvm_has_feat(vcpu->kvm, ID_AA64PFR1_EL1, SME, IMP)) + if (vcpu_has_sme(vcpu)) return 0; return REG_HIDDEN; From patchwork Fri Dec 20 16:46:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917116 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 217D421C17E; Fri, 20 Dec 2024 16:51:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713507; cv=none; b=sE5Sw2ECZKlc3OmjsGhutL/tlSssU+u6zZw42HYexLLbZ99ZuYMv2wsiaSJ1qLZ2k5nMRRGtDCULT7B5f5O//AJ4EztdZY2aYdzi7igtwxR2gx8ZTQiByT6mcy7B9hgjW09rPeb6zmfpfJ+reawbTPEfEU6XkmmePcc+juuGetA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713507; c=relaxed/simple; bh=UbPAO2RjCbS5UdMd6M1SYfFSDmJ+s49CndGfTDSD4Tg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=G/EwJfm759JGoxhvhhdbuawKeWVBP4UzyMSq6yt5I+sz2wCVz1Nk/n0/5SiXBRtxI6X4bhjtjdj3ApftcpfSxR36RmB1Clq7A2QdeyFsUijoR5U2BHFYDADnZHCAJva2sXCa+O+0pHjNSrEVDw1P+jVN1wgxVgMa77JOnqN58wY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qVIUMkZ2; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qVIUMkZ2" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 68F7FC4CEDE; Fri, 20 Dec 2024 16:51:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713506; bh=UbPAO2RjCbS5UdMd6M1SYfFSDmJ+s49CndGfTDSD4Tg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=qVIUMkZ2FTeYFFW6MzulXykk/p6vUYIuFCiTtX1PTCRXOJd5AFNunEsurBUu++y7J xfsfagZ2OLab8zRFcKJporZxAKrl63777HR6iOLKuiXGKWwnpNAU9pLEYtUpovTG4+ X7SEHmlsZ6A9Mab8h3eRsdN0/dsAhuNafPiYubis834JW540tzWX/GxRNF2TPmMn7k OBIKLj8KBNH4L8kL0n6rggloUFaWLMm/qUX8Q3xIOkaa50ji5CNihmRc92KfGSFoIl dc4iYAx+ME8kWPPEU0v5v3vAPP73OlwNdUv//bqer9/beEdYHly9wscjZNSBtXD+ST NSx6V+hPhEvUQ== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:38 +0000 Subject: [PATCH RFC v3 13/27] KVM: arm64: Rename sve_state_reg_region Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-13-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=2408; i=broonie@kernel.org; h=from:subject:message-id; bh=UbPAO2RjCbS5UdMd6M1SYfFSDmJ+s49CndGfTDSD4Tg=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBezixYsQFK0OuTzhIjDQ9L5vedbkx8KdXI+oJ+ MByXF6GJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgXgAKCRAk1otyXVSH0DFgB/ 4n1MpQ+jOywpaT+XfkmFTNXoUjjcWniJR6PZzwE3f/+HiNUo7qNhJmrjK4WsmqxJDxxFW9GI2MOtNe 1Oc4OSu2kyc25wQX2pJwzaP/jF+V2xx92SI9d+ipJJBr+IZtrKO52UBsPn+iMl4b1ksBSH14nh7b0U Kf35rjwc8MK8u18CbYpLcNvYHDJD4LFxEXEvRB9xKwnmXnYnlEng9XAyAr7h4lPlgnwA1gzEb307k3 GCa+eAw6IZIY46Hb3Ww8rWz6S6ofTp7BozSAkednJfD+05rJXn6vHelrW5zgwIXkBZfIYcfaI8qV6D sn02KtcFvcJndag9PqRThjvba09EZ7 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB As for SVE we will need to pull parts of dynamically sized registers out of a block of memory for SME so we will use a similar code pattern for this. Rename the current struct sve_state_reg_region in preparation for this. No functional change. Signed-off-by: Mark Brown --- arch/arm64/kvm/guest.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index 62ff51d6e4584acc71205f5d4b1d2f3d2e2d2f88..cde733417f25b5af4f5e996f91c2b962a4d361fd 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -404,9 +404,9 @@ static int set_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) */ #define vcpu_sve_slices(vcpu) 1 -/* Bounds of a single SVE register slice within vcpu->arch.sve_state */ -struct sve_state_reg_region { - unsigned int koffset; /* offset into sve_state in kernel memory */ +/* Bounds of a single register slice within vcpu->arch.s[mv]e_state */ +struct vec_state_reg_region { + unsigned int koffset; /* offset into s[mv]e_state in kernel memory */ unsigned int klen; /* length in kernel memory */ unsigned int upad; /* extra trailing padding in user memory */ }; @@ -415,7 +415,7 @@ struct sve_state_reg_region { * Validate SVE register ID and get sanitised bounds for user/kernel SVE * register copy */ -static int sve_reg_to_region(struct sve_state_reg_region *region, +static int sve_reg_to_region(struct vec_state_reg_region *region, struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) { @@ -485,7 +485,7 @@ static int sve_reg_to_region(struct sve_state_reg_region *region, static int get_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) { int ret; - struct sve_state_reg_region region; + struct vec_state_reg_region region; char __user *uptr = (char __user *)reg->addr; /* Handle the KVM_REG_ARM64_SVE_VLS pseudo-reg as a special case: */ @@ -511,7 +511,7 @@ static int get_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) static int set_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) { int ret; - struct sve_state_reg_region region; + struct vec_state_reg_region region; const char __user *uptr = (const char __user *)reg->addr; /* Handle the KVM_REG_ARM64_SVE_VLS pseudo-reg as a special case: */ From patchwork Fri Dec 20 16:46:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917117 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9813821C17E; Fri, 20 Dec 2024 16:51:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713510; cv=none; b=jach4/h9d/TACsqnPtE2LL58VkxgfHU3NFaP7QZQJb7BPJnqh2UX6TbbiixUsbBQ0912U+Mb6a7q/BcP2Fi+LjIJNNIqF6D72GYj0n7qtDF6NlzJ1pWyeAItWwpLL9pM00ZT+a18FrPbjvSpDvPWhQSAR19ngFU/5aLhkiqOSmw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713510; c=relaxed/simple; bh=YX6ec2Y9jRqUBq+hF4J3MaIT2MldDbLjE2bFPa/5Uc4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=GSqYGnvh6+drxtAJzW59YzZ8UugzAg4XsmsqtML7Mhu+LOwHEM+tNSsVIPSxX+ly56agUweVTjZGtZeNrVAoaMcwIMmYMPX0NjhL6rhrScG3OyHUcfHE7/eBxQJcIwcUh/Swwv55QoaJq3XliMwFOJBFzLDiu7O764d7IzLJXfA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=AWWTyFT8; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="AWWTyFT8" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2B451C4CED7; Fri, 20 Dec 2024 16:51:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713510; bh=YX6ec2Y9jRqUBq+hF4J3MaIT2MldDbLjE2bFPa/5Uc4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=AWWTyFT8xAvkubZ/rsOKV1ZC/mH0p9tipTonz114J98laU3xBnBPHi/bqhSJS1feV EF1nJr9CJIs2YYnvJ0k1hHeAd6E4NJ71tFnB1YZlodC9AL8jPUAG6yx+SxI81mZzHz XFCnFS+k74CsfchEXUuAdKtUhq/rEAQ4f3+1xPOYHr3MJDQYBW8fvH7Bu8UPQaCOqM YRjRWsmcLf3uCqm3Ejo00zr8t9Kb0f7mBXn6s+xBKYDcWCjsyLejk8Yxnz+o7MeFxM vVuEiJT+Bqlq6zAVjiuJp7bOSX6XnkAFNrKKtz3iOKNq7liEOk06G25XTpAl/f8P5N bHJ/zrB1BpUzw== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:39 +0000 Subject: [PATCH RFC v3 14/27] KVM: arm64: Store vector lengths in an array Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-14-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=11987; i=broonie@kernel.org; h=from:subject:message-id; bh=YX6ec2Y9jRqUBq+hF4J3MaIT2MldDbLjE2bFPa/5Uc4=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBfj9AI2R2o7U7XViGvhTSGqE5CJ94qvpxOkITB QWIN3k+JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgXwAKCRAk1otyXVSH0C/dB/ 4v1xu5gvUpuIA5A7ePvct103fedcOSPZOiua+g9panN54N2NicPZoyCVeaeONRti+JQyfdGIiHDFCS pY3PBttagbOpROPi9vRN5xOrlKDJLTUpWUR4AeWetcHh1l1QvxHEkUdXiQefieHDYKtIoKpouDLYet Y7MGAmAvdrO0sM9dV6L93jRk5aQxSl/bytrIu/z+YQXB7MNptzRZDZJh7RnvD/LjpfmV72RiEKGdb5 njP8wlmQyDpC5W81BVkFbSd89TjFOBW80mFIaz7ANMif7JPIwAPET5RPmIU5bsq6uYzNZlPcYCf076 sX9sBuoc+wts296vYN+JdzUNbcrO4b X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME adds a second vector length configured in a very similar way to the SVE vector length, in order to facilitate future code sharing for SME refactor our storage of vector lengths to use an array like the host does. We do not yet take much advantage of this so the intermediate code is not as clean as might be. No functional change. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 17 +++++++++++------ arch/arm64/include/asm/kvm_hyp.h | 2 +- arch/arm64/include/asm/kvm_pkvm.h | 2 +- arch/arm64/kvm/fpsimd.c | 2 +- arch/arm64/kvm/guest.c | 6 +++--- arch/arm64/kvm/hyp/include/hyp/switch.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 11 ++++++----- arch/arm64/kvm/hyp/nvhe/pkvm.c | 2 +- arch/arm64/kvm/reset.c | 22 +++++++++++----------- 9 files changed, 37 insertions(+), 31 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 022214e57e74404e8d590a5820a9e77160869b1b..63e1410146f76fd584374765c04b3ba14090afdc 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -74,8 +74,10 @@ enum kvm_mode kvm_get_mode(void); static inline enum kvm_mode kvm_get_mode(void) { return KVM_MODE_NONE; }; #endif -extern unsigned int __ro_after_init kvm_sve_max_vl; -extern unsigned int __ro_after_init kvm_host_sve_max_vl; +extern unsigned int __ro_after_init kvm_max_vl[ARM64_VEC_MAX]; +extern unsigned int __ro_after_init kvm_host_max_vl[ARM64_VEC_MAX]; +DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); + int __init kvm_arm_init_sve(void); u32 __attribute_const__ kvm_target_cpu(void); @@ -709,7 +711,7 @@ struct kvm_vcpu_arch { */ void *sve_state; enum fp_type fp_type; - unsigned int sve_max_vl; + unsigned int max_vl[ARM64_VEC_MAX]; /* Stage 2 paging state used by the hardware on next switch */ struct kvm_s2_mmu *hw_mmu; @@ -984,9 +986,12 @@ struct kvm_vcpu_arch { /* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */ #define vcpu_sve_pffr(vcpu) (kern_hyp_va((vcpu)->arch.sve_state) + \ - sve_ffr_offset((vcpu)->arch.sve_max_vl)) + sve_ffr_offset((vcpu)->arch.max_vl[ARM64_VEC_SVE])) + +#define vcpu_vec_max_vq(vcpu, type) sve_vq_from_vl((vcpu)->arch.max_vl[type]) + +#define vcpu_sve_max_vq(vcpu) vcpu_vec_max_vq(vcpu, ARM64_VEC_SVE) -#define vcpu_sve_max_vq(vcpu) sve_vq_from_vl((vcpu)->arch.sve_max_vl) #define vcpu_sve_zcr_elx(vcpu) \ (unlikely(is_hyp_ctxt(vcpu)) ? ZCR_EL2 : ZCR_EL1) @@ -995,7 +1000,7 @@ struct kvm_vcpu_arch { size_t __size_ret; \ unsigned int __vcpu_vq; \ \ - if (WARN_ON(!sve_vl_valid((vcpu)->arch.sve_max_vl))) { \ + if (WARN_ON(!sve_vl_valid((vcpu)->arch.max_vl[ARM64_VEC_SVE]))) { \ __size_ret = 0; \ } else { \ __vcpu_vq = vcpu_sve_max_vq(vcpu); \ diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index c838309e4ec47e395d78127a8ee6bad8390c4411..21943cb98542750a1b626a8de6bbc095d7770ccf 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -143,6 +143,6 @@ extern u64 kvm_nvhe_sym(id_aa64smfr0_el1_sys_val); extern unsigned long kvm_nvhe_sym(__icache_flags); extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits); -extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_vl); +extern unsigned int kvm_nvhe_sym(kvm_host_max_vl[ARM64_VEC_MAX]); #endif /* __ARM64_KVM_HYP_H__ */ diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h index 400f7cef1e81b29925ed00593d8198f8d2700025..e6021a2418529064dcd31b4a5301e4d6f6ac8acd 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -159,7 +159,7 @@ static inline size_t pkvm_host_sve_state_size(void) return 0; return size_add(sizeof(struct cpu_sve_state), - SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_vl))); + SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]))); } #endif /* __ARM64_KVM_PKVM_H__ */ diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index 3c2e0b96877ac5b4f3b9d8dfa38975f11b74b60d..51c844e25dfa460ecab5bb0dfc50c7680318aa20 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -133,7 +133,7 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu) */ fp_state.st = &vcpu->arch.ctxt.fp_regs; fp_state.sve_state = vcpu->arch.sve_state; - fp_state.sve_vl = vcpu->arch.sve_max_vl; + fp_state.sve_vl = vcpu->arch.max_vl[ARM64_VEC_SVE]; fp_state.sme_state = NULL; fp_state.svcr = &__vcpu_sys_reg(vcpu, SVCR); fp_state.fpmr = &__vcpu_sys_reg(vcpu, FPMR); diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index cde733417f25b5af4f5e996f91c2b962a4d361fd..5fda5dbc0c3c0ce3a20a732a68421376e54f23ca 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -318,7 +318,7 @@ static int get_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) if (!vcpu_has_sve(vcpu)) return -ENOENT; - if (WARN_ON(!sve_vl_valid(vcpu->arch.sve_max_vl))) + if (WARN_ON(!sve_vl_valid(vcpu->arch.max_vl[ARM64_VEC_SVE]))) return -EINVAL; memset(vqs, 0, sizeof(vqs)); @@ -356,7 +356,7 @@ static int set_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) if (vq_present(vqs, vq)) max_vq = vq; - if (max_vq > sve_vq_from_vl(kvm_sve_max_vl)) + if (max_vq > sve_vq_from_vl(kvm_max_vl[ARM64_VEC_SVE])) return -EINVAL; /* @@ -375,7 +375,7 @@ static int set_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) return -EINVAL; /* vcpu->arch.sve_state will be alloc'd by kvm_vcpu_finalize_sve() */ - vcpu->arch.sve_max_vl = sve_vl_from_vq(max_vq); + vcpu->arch.max_vl[ARM64_VEC_SVE] = sve_vl_from_vq(max_vq); return 0; } diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 247dfadcdb22e1ef96f92a9d86e66c9eefb44600..09a9a237d6dd22d4bb941714363675abdab1baa7 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -370,8 +370,8 @@ static inline void __hyp_sve_save_host(void) struct cpu_sve_state *sve_state = *host_data_ptr(sve_state); sve_state->zcr_el1 = read_sysreg_el1(SYS_ZCR); - write_sysreg_s(sve_vq_from_vl(kvm_host_sve_max_vl) - 1, SYS_ZCR_EL2); - __sve_save_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + write_sysreg_s(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]) - 1, SYS_ZCR_EL2); + __sve_save_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_max_vl[ARM64_VEC_SVE]), &sve_state->fpsr, true); } diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 6aa0b13d86e581a36ed529bcd932498045d2d6df..7468d8516ecaa1370861e51ad4f65adbc01a5d97 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -33,7 +33,7 @@ static void __hyp_sve_save_guest(struct kvm_vcpu *vcpu) */ sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2); __sve_save_state(vcpu_sve_pffr(vcpu), &vcpu->arch.ctxt.fp_regs.fpsr, true); - write_sysreg_s(sve_vq_from_vl(kvm_host_sve_max_vl) - 1, SYS_ZCR_EL2); + write_sysreg_s(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]) - 1, SYS_ZCR_EL2); } static void __hyp_sve_restore_host(void) @@ -49,8 +49,8 @@ static void __hyp_sve_restore_host(void) * that was discovered, if we wish to use larger VLs this will * need to be revisited. */ - write_sysreg_s(sve_vq_from_vl(kvm_host_sve_max_vl) - 1, SYS_ZCR_EL2); - __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + write_sysreg_s(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]) - 1, SYS_ZCR_EL2); + __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_max_vl[ARM64_VEC_SVE]), &sve_state->fpsr, true); write_sysreg_el1(sve_state->zcr_el1, SYS_ZCR); @@ -101,7 +101,8 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) hyp_vcpu->vcpu.arch.sve_state = kern_hyp_va(host_vcpu->arch.sve_state); /* Limit guest vector length to the maximum supported by the host. */ - hyp_vcpu->vcpu.arch.sve_max_vl = min(host_vcpu->arch.sve_max_vl, kvm_host_sve_max_vl); + hyp_vcpu->vcpu.arch.max_vl[ARM64_VEC_SVE] = min(host_vcpu->arch.max_vl[ARM64_VEC_SVE], + kvm_host_max_vl[ARM64_VEC_SVE]); hyp_vcpu->vcpu.arch.hw_mmu = host_vcpu->arch.hw_mmu; @@ -483,7 +484,7 @@ void handle_trap(struct kvm_cpu_context *host_ctxt) case ESR_ELx_EC_SVE: cpacr_clear_set(0, CPACR_ELx_ZEN); isb(); - sve_cond_update_zcr_vq(sve_vq_from_vl(kvm_host_sve_max_vl) - 1, + sve_cond_update_zcr_vq(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]) - 1, SYS_ZCR_EL2); break; case ESR_ELx_EC_IABT_LOW: diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 0a4e1f5105592b23a0505bf7680c66e76b5c2a65..fea01612ac47a8a2f42edb9f17490edbaa89d04c 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -20,7 +20,7 @@ unsigned long __icache_flags; /* Used by kvm_get_vttbr(). */ unsigned int kvm_arm_vmid_bits; -unsigned int kvm_host_sve_max_vl; +unsigned int kvm_host_max_vl[ARM64_VEC_MAX]; static void pkvm_vcpu_reset_hcr(struct kvm_vcpu *vcpu) { diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index ce726b1d4e8e90cfd4459a6cb9c67b8805424e22..3cb91dc6dc3dc5cc484900dbd9f4cdfedb3e2b4a 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -32,7 +32,7 @@ /* Maximum phys_shift supported for any VM on this host */ static u32 __ro_after_init kvm_ipa_limit; -unsigned int __ro_after_init kvm_host_sve_max_vl; +unsigned int __ro_after_init kvm_host_max_vl[ARM64_VEC_MAX]; /* * ARMv8 Reset Values @@ -46,14 +46,14 @@ unsigned int __ro_after_init kvm_host_sve_max_vl; #define VCPU_RESET_PSTATE_SVC (PSR_AA32_MODE_SVC | PSR_AA32_A_BIT | \ PSR_AA32_I_BIT | PSR_AA32_F_BIT) -unsigned int __ro_after_init kvm_sve_max_vl; +unsigned int __ro_after_init kvm_max_vl[ARM64_VEC_MAX]; int __init kvm_arm_init_sve(void) { if (system_supports_sve()) { - kvm_sve_max_vl = sve_max_virtualisable_vl(); - kvm_host_sve_max_vl = sve_max_vl(); - kvm_nvhe_sym(kvm_host_sve_max_vl) = kvm_host_sve_max_vl; + kvm_max_vl[ARM64_VEC_SVE] = sve_max_virtualisable_vl(); + kvm_host_max_vl[ARM64_VEC_SVE] = sve_max_vl(); + kvm_nvhe_sym(kvm_host_max_vl[ARM64_VEC_SVE]) = kvm_host_max_vl[ARM64_VEC_SVE]; /* * The get_sve_reg()/set_sve_reg() ioctl interface will need @@ -61,16 +61,16 @@ int __init kvm_arm_init_sve(void) * order to support vector lengths greater than * VL_ARCH_MAX: */ - if (WARN_ON(kvm_sve_max_vl > VL_ARCH_MAX)) - kvm_sve_max_vl = VL_ARCH_MAX; + if (WARN_ON(kvm_max_vl[ARM64_VEC_SVE] > VL_ARCH_MAX)) + kvm_max_vl[ARM64_VEC_SVE] = VL_ARCH_MAX; /* * Don't even try to make use of vector lengths that * aren't available on all CPUs, for now: */ - if (kvm_sve_max_vl < sve_max_vl()) + if (kvm_max_vl[ARM64_VEC_SVE] < sve_max_vl()) pr_warn("KVM: SVE vector length for guests limited to %u bytes\n", - kvm_sve_max_vl); + kvm_max_vl[ARM64_VEC_SVE]); } return 0; @@ -78,7 +78,7 @@ int __init kvm_arm_init_sve(void) static void kvm_vcpu_enable_sve(struct kvm_vcpu *vcpu) { - vcpu->arch.sve_max_vl = kvm_sve_max_vl; + vcpu->arch.max_vl[ARM64_VEC_SVE] = kvm_max_vl[ARM64_VEC_SVE]; /* * Userspace can still customize the vector lengths by writing @@ -99,7 +99,7 @@ static int kvm_vcpu_finalize_vec(struct kvm_vcpu *vcpu) size_t reg_sz; int ret; - vl = vcpu->arch.sve_max_vl; + vl = vcpu->arch.max_vl[ARM64_VEC_SVE]; /* * Responsibility for these properties is shared between From patchwork Fri Dec 20 16:46:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917118 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A458D227596; Fri, 20 Dec 2024 16:51:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713514; cv=none; b=Gw5HSiCHYmrtqCHStO+c6fg+X6bxmWVubXGXd7+1/BZ4hp3+mK8xL7acVEQBR1WHAYeNfJm+Dse5uazMkR0sf6kLwLucc2Hn3SXYiZxtNBoxphowg7mhy1+Ces2YzXK5gxUU2p0CjYT8aYqC6GICGW0qmQ9r6JZKufhro2qRgNQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713514; c=relaxed/simple; bh=w/k7ZJmMvp/OVIFpeNK2zrs4Wzgsc08+6x/VGr/ScII=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=D5SkGJ+71BY3xv5nJjBCWR2QQcxtgCigjGc8iNur3lOBra95Bchh5YnmfD9inesGwKASWzwgwG3ePa5h97U1wB0cdga5Y0Lrt6nCCk1tc1nze9TwtoCokjkeohIQuSC2lNb67XZkpGNirU+U9IcO+BYMa7qiphZl+aouX8V+CAc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=e0ktmsbb; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="e0ktmsbb" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E3DF6C4CEDC; Fri, 20 Dec 2024 16:51:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713514; bh=w/k7ZJmMvp/OVIFpeNK2zrs4Wzgsc08+6x/VGr/ScII=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=e0ktmsbbm5FDNDR5MoMwvtfc0qDUQTc5LrSKx3/e4Lby4aKsVXzANWMdQWduAexd0 C78/MID7uaV0muMf1zwIxfYh/qnijfkIGRRzgpgGTw7m3MozopCJdUcgvyfs5VK9eS 1t13Q8RC4VMeHwAQ3GkmxolxOB/2fMYqbi0BlLk7I+wwGTrffCnP3nKTih3WHKAudK A0hTh8qHiFmQKt6lIc5RnDP7LoFqA5TzksIOYcrvtHgCGv746B7dfTl/PajY/n1ZcA hyT6lrsDaWsZP2njfEJvOFvfUQWQmFxIAzhMg7eF+g72Zs11cFfjiY0Gqyn7E1XoV5 Qe6YtKRvKIbfA== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:40 +0000 Subject: [PATCH RFC v3 15/27] KVM: arm64: Implement SME vector length configuration Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-15-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=8603; i=broonie@kernel.org; h=from:subject:message-id; bh=w/k7ZJmMvp/OVIFpeNK2zrs4Wzgsc08+6x/VGr/ScII=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBg1Xq2Xwz+zqCI1mPkIHI+jJvVcfK2Hh+ImNTs gxxCBXmJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgYAAKCRAk1otyXVSH0LGsB/ 9OJ7HLuMHDveNVksjyf1bbDHlXmzYczSjuA7DA+BNGkYEvcDW7z/zSgSncCvSS6orxbw5Nid3LNtfb vrwO4GH4vjX3sREqAaERDeL1swVJQIuMFNhZQGYQ3ecW3777BiOjkxRbNdPxvK1Rf9JMUgnLKTHO17 WRdJzOR53ciMI7/wIzFahj6XvDordxiCU7rLNX1xPaF6Bt8sUmfg70aH0w3HOCFzF/YIIAB5XvZAsO T0pwtPr6ja2yX0XqiNMfKh0HbQcBE3tKOWcWIhElCNa0AorbKHsEf/7dXWMNE8Ki/nb8ENEKzk4GUD ceu1yZHolSdoqpmB67ChuN8It0Ercu X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME implements a vector length which architecturally looks very similar to that for SVE, configured in a very similar manner. This controls the vector length used for the ZA matrix register, and for the SVE vector and predicate registers when in streaming mode. The only substantial difference is that unlike SVE the architecture does not guarantee that any particular vector length will be implemented. Configuration for SME vector lengths is done using a virtual register as for SVE, hook up the implementation for the virtual register. Since we do not yet have support for any of the new SME registers stub register access functions are provided that only allow VL configuration. These will be extended as the SME specific registers, as for SVE. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 7 +++ arch/arm64/include/uapi/asm/kvm.h | 9 ++++ arch/arm64/kvm/guest.c | 94 +++++++++++++++++++++++++++++++-------- 3 files changed, 91 insertions(+), 19 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 63e1410146f76fd584374765c04b3ba14090afdc..02f620d95f7dd2cb2b29cc25e78e7ef404cfad4c 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -708,8 +708,15 @@ struct kvm_vcpu_arch { * low 128 bits of the SVE Z registers. When the core * floating point code saves the register state of a task it * records which view it saved in fp_type. + * + * If SME support is also present then it provides an + * alternative view of the SVE registers accessed as for the Z + * registers when PSTATE.SM is 1, plus an additional set of + * SME specific state in the matrix register ZA and LUT + * register ZT0. */ void *sve_state; + void *sme_state; enum fp_type fp_type; unsigned int max_vl[ARM64_VEC_MAX]; diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h index 9d80d22af9d4e00204f5096fb7c8c2ee8c3646c1..efb384cf9d503007f68aad9233ba949128c94b8b 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -356,6 +356,15 @@ struct kvm_arm_counter_offset { #define KVM_ARM64_SVE_VLS_WORDS \ ((KVM_ARM64_SVE_VQ_MAX - KVM_ARM64_SVE_VQ_MIN) / 64 + 1) +/* SME registers */ +#define KVM_REG_ARM64_SME (0x17 << KVM_REG_ARM_COPROC_SHIFT) + +/* Vector lengths pseudo-register: */ +#define KVM_REG_ARM64_SME_VLS (KVM_REG_ARM64 | KVM_REG_ARM64_SME | \ + KVM_REG_SIZE_U512 | 0xffff) +#define KVM_ARM64_SME_VLS_WORDS \ + ((KVM_ARM64_SVE_VQ_MAX - KVM_ARM64_SVE_VQ_MIN) / 64 + 1) + /* Bitmap feature firmware registers */ #define KVM_REG_ARM_FW_FEAT_BMAP (0x0016 << KVM_REG_ARM_COPROC_SHIFT) #define KVM_REG_ARM_FW_FEAT_BMAP_REG(r) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \ diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index 5fda5dbc0c3c0ce3a20a732a68421376e54f23ca..8820cd42a27ac05874ed52c572c1dd9b29a77a1b 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -310,22 +310,20 @@ static int set_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) #define vq_mask(vq) ((u64)1 << ((vq) - SVE_VQ_MIN) % 64) #define vq_present(vqs, vq) (!!((vqs)[vq_word(vq)] & vq_mask(vq))) -static int get_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +static int get_vec_vls(enum vec_type vec_type, struct kvm_vcpu *vcpu, + const struct kvm_one_reg *reg) { unsigned int max_vq, vq; u64 vqs[KVM_ARM64_SVE_VLS_WORDS]; - if (!vcpu_has_sve(vcpu)) - return -ENOENT; - - if (WARN_ON(!sve_vl_valid(vcpu->arch.max_vl[ARM64_VEC_SVE]))) + if (WARN_ON(!sve_vl_valid(vcpu->arch.max_vl[vec_type]))) return -EINVAL; memset(vqs, 0, sizeof(vqs)); - max_vq = vcpu_sve_max_vq(vcpu); + max_vq = vcpu_vec_max_vq(vcpu, vec_type); for (vq = SVE_VQ_MIN; vq <= max_vq; ++vq) - if (sve_vq_available(vq)) + if (vq_available(vec_type, vq)) vqs[vq_word(vq)] |= vq_mask(vq); if (copy_to_user((void __user *)reg->addr, vqs, sizeof(vqs))) @@ -334,18 +332,13 @@ static int get_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) return 0; } -static int set_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +static int set_vec_vls(enum vec_type vec_type, struct kvm_vcpu *vcpu, + const struct kvm_one_reg *reg) { unsigned int max_vq, vq; u64 vqs[KVM_ARM64_SVE_VLS_WORDS]; - if (!vcpu_has_sve(vcpu)) - return -ENOENT; - - if (kvm_arm_vcpu_vec_finalized(vcpu)) - return -EPERM; /* too late! */ - - if (WARN_ON(vcpu->arch.sve_state)) + if (WARN_ON(!sve_vl_valid(vcpu->arch.max_vl[vec_type]))) return -EINVAL; if (copy_from_user(vqs, (const void __user *)reg->addr, sizeof(vqs))) @@ -356,18 +349,18 @@ static int set_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) if (vq_present(vqs, vq)) max_vq = vq; - if (max_vq > sve_vq_from_vl(kvm_max_vl[ARM64_VEC_SVE])) + if (max_vq > sve_vq_from_vl(kvm_max_vl[vec_type])) return -EINVAL; /* * Vector lengths supported by the host can't currently be * hidden from the guest individually: instead we can only set a - * maximum via ZCR_EL2.LEN. So, make sure the available vector + * maximum via xCR_EL2.LEN. So, make sure the available vector * lengths match the set requested exactly up to the requested * maximum: */ for (vq = SVE_VQ_MIN; vq <= max_vq; ++vq) - if (vq_present(vqs, vq) != sve_vq_available(vq)) + if (vq_present(vqs, vq) != vq_available(vec_type, vq)) return -EINVAL; /* Can't run with no vector lengths at all: */ @@ -375,11 +368,33 @@ static int set_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) return -EINVAL; /* vcpu->arch.sve_state will be alloc'd by kvm_vcpu_finalize_sve() */ - vcpu->arch.max_vl[ARM64_VEC_SVE] = sve_vl_from_vq(max_vq); + vcpu->arch.max_vl[vec_type] = sve_vl_from_vq(max_vq); return 0; } +static int get_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +{ + if (!vcpu_has_sve(vcpu)) + return -ENOENT; + + return get_vec_vls(ARM64_VEC_SVE, vcpu, reg); +} + +static int set_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +{ + if (!vcpu_has_sve(vcpu)) + return -ENOENT; + + if (kvm_arm_vcpu_vec_finalized(vcpu)) + return -EPERM; /* too late! */ + + if (WARN_ON(vcpu->arch.sve_state)) + return -EINVAL; + + return set_vec_vls(ARM64_VEC_SVE, vcpu, reg); +} + #define SVE_REG_SLICE_SHIFT 0 #define SVE_REG_SLICE_BITS 5 #define SVE_REG_ID_SHIFT (SVE_REG_SLICE_SHIFT + SVE_REG_SLICE_BITS) @@ -533,6 +548,45 @@ static int set_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) return 0; } +static int get_sme_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +{ + if (!vcpu_has_sme(vcpu)) + return -ENOENT; + + return get_vec_vls(ARM64_VEC_SME, vcpu, reg); +} + +static int set_sme_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +{ + if (!vcpu_has_sme(vcpu)) + return -ENOENT; + + if (kvm_arm_vcpu_vec_finalized(vcpu)) + return -EPERM; /* too late! */ + + if (WARN_ON(vcpu->arch.sme_state)) + return -EINVAL; + + return set_vec_vls(ARM64_VEC_SME, vcpu, reg); +} + +static int get_sme_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +{ + /* Handle the KVM_REG_ARM64_SME_VLS pseudo-reg as a special case: */ + if (reg->id == KVM_REG_ARM64_SME_VLS) + return get_sme_vls(vcpu, reg); + + return -EINVAL; +} + +static int set_sme_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +{ + /* Handle the KVM_REG_ARM64_SME_VLS pseudo-reg as a special case: */ + if (reg->id == KVM_REG_ARM64_SME_VLS) + return set_sme_vls(vcpu, reg); + + return -EINVAL; +} int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs) { return -EINVAL; @@ -775,6 +829,7 @@ int kvm_arm_get_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) case KVM_REG_ARM_FW_FEAT_BMAP: return kvm_arm_get_fw_reg(vcpu, reg); case KVM_REG_ARM64_SVE: return get_sve_reg(vcpu, reg); + case KVM_REG_ARM64_SME: return get_sme_reg(vcpu, reg); } if (is_timer_reg(reg->id)) @@ -795,6 +850,7 @@ int kvm_arm_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) case KVM_REG_ARM_FW_FEAT_BMAP: return kvm_arm_set_fw_reg(vcpu, reg); case KVM_REG_ARM64_SVE: return set_sve_reg(vcpu, reg); + case KVM_REG_ARM64_SME: return set_sme_reg(vcpu, reg); } if (is_timer_reg(reg->id)) From patchwork Fri Dec 20 16:46:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917119 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1CD3521C198; Fri, 20 Dec 2024 16:51:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713518; cv=none; b=gPnVgDaBiSj1xw2TORU/iEhtVZDLC/C9DKuJIsWnZz5+rT16fB2syIv0dimp4it+lDxV0p0VQ5lCUuUido/4UKix/391XQzxx8E9Bk4YDABzeb2kwuM8yk/mDwp9rLK6NBT2fH8eO6Bq1qxH6t7fZiB34F4gUYhz6+VraU0aYd4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713518; c=relaxed/simple; bh=ziNSU1l2k6s22Zl2zTPsgpLBrD3+gSVFEY6O/oZT2Fs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ThdwLFwYcXApBsR2cbVvzr66DOj8MS7691Fkjemd0c9XSo8TAmkiEwEzuK79fDRfQAATv6NnQ7oeT20t835/ZbyBhIpj8WE0cC0VX87M9uFa0UqBTOcPIimRnwO4Zy9yhqya1H9jDainDtXa8k4egJaqDKcv962y+3/xy5xEHVQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lJV06mgz; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lJV06mgz" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A497AC4CED7; Fri, 20 Dec 2024 16:51:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713518; bh=ziNSU1l2k6s22Zl2zTPsgpLBrD3+gSVFEY6O/oZT2Fs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=lJV06mgzy5StSd1+UxdmtGNogKoICEOQbIwQrX4sftFKHzD/Xz4Fk3+HQFPmhaDSv YBYHGSHLmMRXDZZz3zncLv5JzR20izZubLUVEqHjOoonKGZqeop0jSrRp5OjB5XIxc P47WuASqoGzviaf15wBCQXq0n7BRquj5092/6C9Ii5YKSh1c3ZNXCZToNgkE923MJy RMgLsyuT9M8diUPzAVwypVx7fpr6VI6nocKdV/C7tuV4OoXzn4K3ekMxyeXFeToAv7 JRs7/+GVPggrZUJTvAntVz559nMYJjExfWTnAY9p7grNrliu5LMQtPxbt/wLf/0/bP rjln9eSHIcHuA== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:41 +0000 Subject: [PATCH RFC v3 16/27] KVM: arm64: Add definitions for SME control register Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-16-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=5625; i=broonie@kernel.org; h=from:subject:message-id; bh=ziNSU1l2k6s22Zl2zTPsgpLBrD3+gSVFEY6O/oZT2Fs=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBgO7cl0oEi7XeyJsA/z/OYmqthjqYTof8sIJnf bwSelFCJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgYAAKCRAk1otyXVSH0AtZB/ 0YJLUHxjPX6X2u7w+Y+Q0Dv142hnpW7NfvtMvucCl3io6nDs1dHsgnAB+6A5heum5WaQT5R2ffckUT jBbpf6Y4e4/BowX2p8DR4bDZJDJjFLCJDc1LEBfRgENZsr5B7rf5Jfh1GKKxTNp+zt1roTmuf1O3t7 QxudXhCPz58qeFMKT0WIRSeBQDB3cn37bQ6JI7lf4LW+E0xYAeKMceB8X2lvNlP2RaMJKuPLNvbWVR L+LXVHfGmFG5BZI7Yv28w3WBDH9MaFW/On/Xj9el+cRFQqtTwSoG+1OnCSUYVlzMoFe0RbG4nUyrVj XQzXQmxaX1dm4v/aKWILW5ifJjw0u4 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME is configured by the system registers SMCR_EL1 and SMCR_EL2, add definitions and userspace access for them. They will be context switched together with the rest of SME state. In systems with SME priority support there are additional registers SMPRI_EL1 and SMPRIMAP_EL2 managing the priorities however we do not currently have any support for SME priorities and mask that support out from guests. The intention is to revist this once we have physical implementations and can properly evaluate the practical impacts that they have. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 6 ++++++ arch/arm64/include/asm/vncr_mapping.h | 1 + arch/arm64/kvm/sys_regs.c | 37 ++++++++++++++++++++++++++++++++++- 3 files changed, 43 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 02f620d95f7dd2cb2b29cc25e78e7ef404cfad4c..8d6342dde02fd99cfd7d2bedeccf0581ad3504ee 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -484,6 +484,7 @@ enum vcpu_sysreg { CPTR_EL2, /* Architectural Feature Trap Register (EL2) */ HACR_EL2, /* Hypervisor Auxiliary Control Register */ ZCR_EL2, /* SVE Control Register (EL2) */ + SMCR_EL2, /* SME Control Register (EL2) */ TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */ TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */ TCR_EL2, /* Translation Control Register (EL2) */ @@ -521,6 +522,7 @@ enum vcpu_sysreg { VNCR(ACTLR_EL1),/* Auxiliary Control Register */ VNCR(CPACR_EL1),/* Coprocessor Access Control */ VNCR(ZCR_EL1), /* SVE Control */ + VNCR(SMCR_EL1), /* SME Control */ VNCR(TTBR0_EL1),/* Translation Table Base Register 0 */ VNCR(TTBR1_EL1),/* Translation Table Base Register 1 */ VNCR(TCR_EL1), /* Translation Control Register */ @@ -998,7 +1000,11 @@ struct kvm_vcpu_arch { #define vcpu_vec_max_vq(vcpu, type) sve_vq_from_vl((vcpu)->arch.max_vl[type]) #define vcpu_sve_max_vq(vcpu) vcpu_vec_max_vq(vcpu, ARM64_VEC_SVE) +#define vcpu_sme_max_vq(vcpu) vcpu_vec_max_vq(vcpu, ARM64_VEC_SME) +#define vcpu_max_vl(vcpu) max((vcpu)->arch.max_vl[ARM64_VEC_SVE], \ + (vcpu)->arch.max_vl[ARM64_VEC_SME]) +#define vcpu_max_vq(vcpu) sve_vq_from_vl(vcpu_max_vl(vcpu)) #define vcpu_sve_zcr_elx(vcpu) \ (unlikely(is_hyp_ctxt(vcpu)) ? ZCR_EL2 : ZCR_EL1) diff --git a/arch/arm64/include/asm/vncr_mapping.h b/arch/arm64/include/asm/vncr_mapping.h index 4f9bbd4d6c2671753124599475e5138bf6b9c749..74fc7400efbc7de6b8dd81a485f1e9d545baf7a9 100644 --- a/arch/arm64/include/asm/vncr_mapping.h +++ b/arch/arm64/include/asm/vncr_mapping.h @@ -42,6 +42,7 @@ #define VNCR_HDFGWTR_EL2 0x1D8 #define VNCR_ZCR_EL1 0x1E0 #define VNCR_HAFGRTR_EL2 0x1E8 +#define VNCR_SMCR_EL1 0x1F0 #define VNCR_TTBR0_EL1 0x200 #define VNCR_TTBR1_EL1 0x210 #define VNCR_FAR_EL1 0x220 diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 1b16716a6d53525fbe694cc8d5d009d72b6ce416..a9429d9d63b54b5b4d4fe365aa6af4d84a256539 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -142,6 +142,7 @@ static bool get_el2_to_el1_mapping(unsigned int reg, MAPPED_EL2_SYSREG(ELR_EL2, ELR_EL1, NULL ); MAPPED_EL2_SYSREG(SPSR_EL2, SPSR_EL1, NULL ); MAPPED_EL2_SYSREG(ZCR_EL2, ZCR_EL1, NULL ); + MAPPED_EL2_SYSREG(SMCR_EL2, SMCR_EL1, NULL ); MAPPED_EL2_SYSREG(CONTEXTIDR_EL2, CONTEXTIDR_EL1, NULL ); default: return false; @@ -2405,6 +2406,37 @@ static bool access_zcr_el2(struct kvm_vcpu *vcpu, return true; } +static unsigned int sme_el2_visibility(const struct kvm_vcpu *vcpu, + const struct sys_reg_desc *rd) +{ + return __el2_visibility(vcpu, rd, sme_visibility); +} + +static bool access_smcr_el2(struct kvm_vcpu *vcpu, + struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + unsigned int vq; + u64 smcr; + + if (guest_hyp_sve_traps_enabled(vcpu)) { + kvm_inject_nested_sve_trap(vcpu); + return true; + } + + if (!p->is_write) { + p->regval = vcpu_read_sys_reg(vcpu, SMCR_EL2); + return true; + } + + smcr = p->regval; + vq = SYS_FIELD_GET(SMCR_ELx, LEN, smcr) + 1; + vq = min(vq, vcpu_sme_max_vq(vcpu)); + vcpu_write_sys_reg(vcpu, SYS_FIELD_PREP(SMCR_ELx, LEN, vq - 1), + SMCR_EL2); + return true; +} + static unsigned int s1poe_visibility(const struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd) { @@ -2649,7 +2681,7 @@ static const struct sys_reg_desc sys_reg_descs[] = { { SYS_DESC(SYS_ZCR_EL1), NULL, reset_val, ZCR_EL1, 0, .visibility = sve_visibility }, { SYS_DESC(SYS_TRFCR_EL1), undef_access }, { SYS_DESC(SYS_SMPRI_EL1), undef_access }, - { SYS_DESC(SYS_SMCR_EL1), undef_access }, + { SYS_DESC(SYS_SMCR_EL1), NULL, reset_val, SMCR_EL1, 0, .visibility = sme_visibility }, { SYS_DESC(SYS_TTBR0_EL1), access_vm_reg, reset_unknown, TTBR0_EL1 }, { SYS_DESC(SYS_TTBR1_EL1), access_vm_reg, reset_unknown, TTBR1_EL1 }, { SYS_DESC(SYS_TCR_EL1), access_vm_reg, reset_val, TCR_EL1, 0 }, @@ -2995,6 +3027,9 @@ static const struct sys_reg_desc sys_reg_descs[] = { EL2_REG_VNCR(HCRX_EL2, reset_val, 0), + EL2_REG_FILTERED(SMCR_EL2, access_smcr_el2, reset_val, 0, + sme_el2_visibility), + EL2_REG(TTBR0_EL2, access_rw, reset_val, 0), EL2_REG(TTBR1_EL2, access_rw, reset_val, 0), EL2_REG(TCR_EL2, access_rw, reset_val, TCR_EL2_RES1), From patchwork Fri Dec 20 16:46:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917120 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D32C021C198; Fri, 20 Dec 2024 16:52:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713521; cv=none; b=Bc5mkP0MI07UB9OSX6WpwnyB8ZICmPHQ7FuDEN1fhf0p91gzbaVpUv7pzYP/qItzJyNhjpoUzGkSCsdWVaSHdgR93qp/98/dZq+VG7OpZGk2aqOwBnMab//5l4EuQNIRMCeWacwJMsRr9fEiZ1BYPJ4PQd6Wk/C0zqsKD8nVIFs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713521; c=relaxed/simple; bh=bf57/Mt8lkovk9yTZssGPdXZll/Sjexpxib+fRQYfOY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=YGlMB2tVD9+IsU0wpcsw69voVfrbfRfCrIfUGb7PUb0mORC+NaMM9whQbZI4uWRoNqsmVCcLIAmrHnsR5x8NHee/z3Juarf7Z1ulzd3VHg8/SEw43soNZQYWI/yzmNUiLcQFT4OREpirZar+0wpXtfiMfeKlCeG6fy6Kf09CsGU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kC/tLDSQ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kC/tLDSQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 69B4CC4CEDC; Fri, 20 Dec 2024 16:51:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713521; bh=bf57/Mt8lkovk9yTZssGPdXZll/Sjexpxib+fRQYfOY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=kC/tLDSQl2HZQDgFxku8z644+vm4uF7U5j/HxAXs/eEds3UqDpzlPZo71f8pFBFG2 u8YzAGod4rZDiBRBorz4VgkYlG6kV4f7Mw2U9iyC2yAjKXfa/i+fb6zGYeeR5qo8Nn aE0WtAD7f141UoQIV7BdKVwDx87aIpOiRNyeua9fRG4I4HwWJHmXSJ2qYilSHVajMl y7rsa2eo/78mwfKJ0jSQOVVvGKkcNZSXPJSq+H5RLVFaiUhgh/SUNydJcUFL/Pkj8p i4ct6ksCigh51yqeXQ3rpAHv3S4xwXyFqATtEJqTTJb/FQFxewUOzexp6C0UnTeY5M JKnI0Yf4vFssw== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:42 +0000 Subject: [PATCH RFC v3 17/27] KVM: arm64: Support TPIDR2_EL0 Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-17-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=4482; i=broonie@kernel.org; h=from:subject:message-id; bh=bf57/Mt8lkovk9yTZssGPdXZll/Sjexpxib+fRQYfOY=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBhz28kj9hkbHjMLTSudh2GXiccrVHMInwBUooh +mbZBVKJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgYQAKCRAk1otyXVSH0IkLB/ 4zrC9EMskzptln4mEPPt0kerJH86JtWGK4Qv5qziXvaYzaKI8ZB0/+Hb2bKehkg9bH6DymfV7VLzPQ HT/1mWCt96XTe2R3/pMS6HVbuFVKqAGhtDdoL0qa73iQkRAHgWQzcuBqe5LdEYnGo7qtVs3nJyDwV5 fSnOBMOGtljuA3aqtzQzJsJD1R1oSHZTKi0LWE2pfTrp5gQuVHAk68xg709FcTrIHjm6rw7wXMfI1M ZLtlOFVE6PuH+fio3zBOSyXBTAsjL4vwVDsPw3sd0ON/pN9HyKTU1cK88AWYA8m5U1PJQ5JKneu6gG KT1qvLysO4wyuioNxkNH5Te2VFLOMc X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME adds a new thread ID register, TPIDR2_EL0. This is used in userspace for delayed saving of the ZA state but in terms of the architecture is not really connected to SME other than being part of FEAT_SME. It has an independent fine grained trap and the runtime connection with the rest of SME is purely software defined. Expose the register as a system register if the guest supports SME, context switching it along with the other EL0 TPIDRs. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 15 +++++++++++++++ arch/arm64/kvm/sys_regs.c | 9 ++++++--- 3 files changed, 22 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 8d6342dde02fd99cfd7d2bedeccf0581ad3504ee..063b75eb4f3bc4fb425d2abc8118a950bccc2317 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -428,6 +428,7 @@ enum vcpu_sysreg { CSSELR_EL1, /* Cache Size Selection Register */ TPIDR_EL0, /* Thread ID, User R/W */ TPIDRRO_EL0, /* Thread ID, User R/O */ + TPIDR2_EL0, /* Thread ID, Register 2 */ TPIDR_EL1, /* Thread ID, Privileged */ CNTKCTL_EL1, /* Timer Control Register (EL1) */ PAR_EL1, /* Physical Address Register */ diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h index 8c234d53acb2753c59aa37d7a66f856f2eb87882..93d2b81e8d0678a16c88bda3549ee790db7f5bc2 100644 --- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h @@ -66,6 +66,17 @@ static inline bool ctxt_has_s1poe(struct kvm_cpu_context *ctxt) return kvm_has_s1poe(kern_hyp_va(vcpu->kvm)); } +static inline bool ctxt_has_sme(struct kvm_cpu_context *ctxt) +{ + struct kvm_vcpu *vcpu; + + if (!system_supports_sme()) + return false; + + vcpu = ctxt_to_vcpu(ctxt); + return kvm_has_sme(kern_hyp_va(vcpu->kvm)); +} + static inline void __sysreg_save_common_state(struct kvm_cpu_context *ctxt) { ctxt_sys_reg(ctxt, MDSCR_EL1) = read_sysreg(mdscr_el1); @@ -79,6 +90,8 @@ static inline void __sysreg_save_user_state(struct kvm_cpu_context *ctxt) { ctxt_sys_reg(ctxt, TPIDR_EL0) = read_sysreg(tpidr_el0); ctxt_sys_reg(ctxt, TPIDRRO_EL0) = read_sysreg(tpidrro_el0); + if (ctxt_has_sme(ctxt)) + ctxt_sys_reg(ctxt, TPIDR2_EL0) = read_sysreg_s(SYS_TPIDR2_EL0); } static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt) @@ -148,6 +161,8 @@ static inline void __sysreg_restore_user_state(struct kvm_cpu_context *ctxt) { write_sysreg(ctxt_sys_reg(ctxt, TPIDR_EL0), tpidr_el0); write_sysreg(ctxt_sys_reg(ctxt, TPIDRRO_EL0), tpidrro_el0); + if (ctxt_has_sme(ctxt)) + write_sysreg_s(ctxt_sys_reg(ctxt, TPIDR2_EL0), SYS_TPIDR2_EL0); } static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt, diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index a9429d9d63b54b5b4d4fe365aa6af4d84a256539..b5a38fc7a4a9ed4fce053018eb6ff353ae5c0d09 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -2855,7 +2855,8 @@ static const struct sys_reg_desc sys_reg_descs[] = { .visibility = s1poe_visibility }, { SYS_DESC(SYS_TPIDR_EL0), NULL, reset_unknown, TPIDR_EL0 }, { SYS_DESC(SYS_TPIDRRO_EL0), NULL, reset_unknown, TPIDRRO_EL0 }, - { SYS_DESC(SYS_TPIDR2_EL0), undef_access }, + { SYS_DESC(SYS_TPIDR2_EL0), NULL, reset_unknown, TPIDR2_EL0, + .visibility = sme_visibility}, { SYS_DESC(SYS_SCXTNUM_EL0), undef_access }, @@ -4959,8 +4960,7 @@ void kvm_calculate_traps(struct kvm_vcpu *vcpu) HFGxTR_EL2_nMAIR2_EL1 | HFGxTR_EL2_nS2POR_EL1 | HFGxTR_EL2_nACCDATA_EL1 | - HFGxTR_EL2_nSMPRI_EL1_MASK | - HFGxTR_EL2_nTPIDR2_EL0_MASK); + HFGxTR_EL2_nSMPRI_EL1_MASK); if (!kvm_has_feat(kvm, ID_AA64ISAR0_EL1, TLB, OS)) kvm->arch.fgu[HFGITR_GROUP] |= (HFGITR_EL2_TLBIRVAALE1OS| @@ -5007,6 +5007,9 @@ void kvm_calculate_traps(struct kvm_vcpu *vcpu) kvm->arch.fgu[HAFGRTR_GROUP] |= ~(HAFGRTR_EL2_RES0 | HAFGRTR_EL2_RES1); + if (!kvm_has_feat(kvm, ID_AA64PFR1_EL1, SME, IMP)) + kvm->arch.fgu[HFGxTR_GROUP] |= HFGxTR_EL2_nTPIDR2_EL0; + set_bit(KVM_ARCH_FLAG_FGU_INITIALIZED, &kvm->arch.flags); out: mutex_unlock(&kvm->arch.config_lock); From patchwork Fri Dec 20 16:46:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917121 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD6D121C198; Fri, 20 Dec 2024 16:52:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713526; cv=none; b=GeSRnn4Rkl4pNtoNmx9xeVcOzIXZUQMY+coYgUPE1hNqww9r2ygdueX5ndQsXQBVmjmPpgdWbaHxidPzEQgXu7ugombvK6bR765NTnj5nA1P2f4udtwt5xx+b47bxXGnv1KZiLsSZjIcs7yd5sEi+hec7QmKiwmGNpw4ecT7wNw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713526; c=relaxed/simple; bh=ugWnPjaP6zeZk+xhqbQnNKY7IW5UR6NCSNSJraf8UWI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=PgCoKGbnKnE9ZGSS2SrrKIo+3sV5J4MEa5x0R9ku8MXBUCFOIoOWPALd8uesHDZrtHNA1x5F7TybSTT4dIHZXF6OZ9vJuzW82djbdXefyMUzGxtxlIyVxV0U88yp6uU1yTUEEtmB3BtTYZldLw0mN73Nb98+ftJmHoFvY9vsjLQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QqkvOVSB; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QqkvOVSB" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2CB5EC4CECD; Fri, 20 Dec 2024 16:52:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713525; bh=ugWnPjaP6zeZk+xhqbQnNKY7IW5UR6NCSNSJraf8UWI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=QqkvOVSBClkTAoxPDQIOr6BOiamrjEBCoqgeF7rmEv8oue8TNeeOMeAGx7vHGSkui /oiM9Gl527gyYX24SsWvk/Ts4rytLCjqk8OepzPydPtl0COdSrzbav8OGHCPDeYmfU TN2a1aRPBrRIMPbvzhveKnzefTpk15rqA6qq61DiYlEE8DX1eSgImlCvUQjqFOTZU/ oo6fIhQ3HXrDPJoKzd2JJLy1EqFaNw5VR6bNN2R9ayQg6r2AJOrhuq3HVZC5ccmrUr thxHTbiF0i5kNXsWVm0eHgFAVU8f3LlXVBEXis20aDtvCIW+Q0xTplsdMZqdt2sBTZ Wqa1B84n4TZgw== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:43 +0000 Subject: [PATCH RFC v3 18/27] KVM: arm64: Support SMIDR_EL1 for guests Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-18-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=4343; i=broonie@kernel.org; h=from:subject:message-id; bh=ugWnPjaP6zeZk+xhqbQnNKY7IW5UR6NCSNSJraf8UWI=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBi306AU+25ymsIInB6kYrxfAyUEl8/NHvT1eLF XMFeG7CJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgYgAKCRAk1otyXVSH0BVOB/ 4qxP9v6MmfqF6r8myKyAk8gF2kl3QE96D+Za51PzoU1WnpI0pYByC6QnpHzdKbLeo/LBGvfvKcPYUJ ZzPhLV6RlKvCV0b7zuzT3ISPcdyOvaArKcyMNOMaPEVV1xalmWFM0u+efWWlp+Nb76XoCdonD+cFqh ZcnlV1Jpc5QnZBhf7Mn3t2qOZ7ckLA8xe5Sxh5TN/2hq0CV8qiufZ2AF32D8CEDU8ciXe5bdWipWXv EBedDKL4CaTlQGhum+UFkRLC0ZUj7Y6uYFOvRMacK+FTvkk5VH8MqUBO9evSNyeRQsSEmRHWy5uetL +IbVJho34Pi7AW3GFwVbg1DgjIXL6E X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME adds an identification register SMIDR_EL1 which provides a basic description of the SME implementation, describing the implementation in a manner similar to MIDR_EL1 for the PE as well as indicating support for priority management. Since we do not currently support SME priority control we mask out SMPS, indicating that priority management is not supported. We do the same for Affinity, indicating that there is no physical sharing, and unknown fields. As for MIDR_EL1 and REVIDR_EL1 we expose the implementer and revision information to guests with the raw value from the CPU we are running on, this may present issues for asymmetric systems or for migration as it does for the existing registers. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/kvm/sys_regs.c | 46 ++++++++++++++++++++++++++++++++++++--- 2 files changed, 44 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 063b75eb4f3bc4fb425d2abc8118a950bccc2317..a304b02efcadba5371edffe97e911bba0634ed62 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -472,6 +472,7 @@ enum vcpu_sysreg { /* FP/SIMD/SVE */ SVCR, FPMR, + SMIDR_EL1, /* Streaming Mode Identification Register */ /* 32bit specific registers. */ DACR32_EL2, /* Domain Access Control Register */ diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index b5a38fc7a4a9ed4fce053018eb6ff353ae5c0d09..416c855153ca532e4c6557d78599e9af0f913071 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -882,6 +882,39 @@ static u64 reset_mpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) return mpidr; } +static u64 reset_smidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) +{ + u64 smidr = 0; + + if (!system_supports_sme()) + return smidr; + + smidr = read_sysreg_s(SYS_SMIDR_EL1); + + /* + * Mask out any priority or affinity information, or fields we + * don't know about. + */ + smidr &= ~(SMIDR_EL1_SMPS_MASK | SMIDR_EL1_AFFINITY_MASK | + SMIDR_EL1_RES0); + + vcpu_write_sys_reg(vcpu, smidr, SMIDR_EL1); + + return smidr; +} + +static bool access_smidr(struct kvm_vcpu *vcpu, + struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + if (p->is_write) + return write_to_read_only(vcpu, p, r); + + p->regval = vcpu_read_sys_reg(vcpu, r->reg); + + return true; +} + static unsigned int pmu_visibility(const struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) { @@ -1576,7 +1609,9 @@ static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu, if (!kvm_has_mte(vcpu->kvm)) val &= ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_MTE); - val &= ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_SME); + if (!vcpu_has_sme(vcpu)) + val &= ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_SME); + val &= ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_RNDR_trap); val &= ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_NMI); val &= ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_MTE_frac); @@ -1676,6 +1711,10 @@ static unsigned int id_visibility(const struct kvm_vcpu *vcpu, if (!vcpu_has_sve(vcpu)) return REG_RAZ; break; + case SYS_ID_AA64SMFR0_EL1: + if (!vcpu_has_sme(vcpu)) + return REG_RAZ; + break; } return 0; @@ -2601,7 +2640,7 @@ static const struct sys_reg_desc sys_reg_descs[] = { ID_WRITABLE(ID_AA64PFR2_EL1, ID_AA64PFR2_EL1_FPMR), ID_UNALLOCATED(4,3), ID_WRITABLE(ID_AA64ZFR0_EL1, ~ID_AA64ZFR0_EL1_RES0), - ID_HIDDEN(ID_AA64SMFR0_EL1), + ID_WRITABLE(ID_AA64SMFR0_EL1, ~ID_AA64SMFR0_EL1_RES0), ID_UNALLOCATED(4,6), ID_WRITABLE(ID_AA64FPFR0_EL1, ~ID_AA64FPFR0_EL1_RES0), @@ -2799,7 +2838,8 @@ static const struct sys_reg_desc sys_reg_descs[] = { { SYS_DESC(SYS_CLIDR_EL1), access_clidr, reset_clidr, CLIDR_EL1, .set_user = set_clidr, .val = ~CLIDR_EL1_RES0 }, { SYS_DESC(SYS_CCSIDR2_EL1), undef_access }, - { SYS_DESC(SYS_SMIDR_EL1), undef_access }, + { SYS_DESC(SYS_SMIDR_EL1), .access = access_smidr, .reset = reset_smidr, + .reg = SMIDR_EL1, .visibility = sme_visibility }, { SYS_DESC(SYS_CSSELR_EL1), access_csselr, reset_unknown, CSSELR_EL1 }, ID_FILTERED(CTR_EL0, ctr_el0, CTR_EL0_DIC_MASK | From patchwork Fri Dec 20 16:46:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917122 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 55109229124; Fri, 20 Dec 2024 16:52:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713529; cv=none; b=cu7+glzM7SqnfAGJ/n8bxoYurnHExP9A5suhk7cJxvN+aIQWm5cWRv7jDS/RLvl9jsY7s5JzVPEdQGiJ9J+sbSMppUxoN1/+Kj5/xPuzB8bNIb1qBd9i8HPojpg6UssoeDEVkipjnbuQBtKh/7kjMEatbtni6PjM8fei6rrLmuU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713529; c=relaxed/simple; bh=nbNePGtF8r3EJAN415gAlX/P47OROWPqRW2uyywE6tA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=GHU8uWNcCSQF9cZRonl3KYOnFDhkVLZu9kEErlMR/4z+UXcHTrpiZsNtvFOPUVOYJNrPgnw4lBY7TWTQjORmtwDYOaU0srfXrr4BqH1BM+aOYFs/NchBdry5p6cycDYUqD24ewI1ID5K+fyIhywI7HJAv3r8ojlWKS3hXEPMcdE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qBB6xvvQ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qBB6xvvQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DE008C4CEDC; Fri, 20 Dec 2024 16:52:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713529; bh=nbNePGtF8r3EJAN415gAlX/P47OROWPqRW2uyywE6tA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=qBB6xvvQWiFPCg4R74HB3qD7FPcu+ghKOH4IUJUyo7T7hZxyeiUlOM/lk+aIdKRfa 3Uc19En22fkH6sR82F/qOc22AjEuFwWYazyr5yPIqAoLdkd+0GJevPiv49d+T0Mc1U E+yF+1tlTnmsM6aI3ilCMzob3O9NbbsnlTHTMPWl31rqd1Rablh5tEZup5w+ghRkgj eSP+EneQWh6oJILbsA2fWaNT+GGbMFa1pLBaFrHE7xhgPSx2pb5CA5btsMEPjvC5dI NxRnPAHagE81Z2dmA4d1Cis2UzApUBGaVHVQPWmYi7MCQSrX4SPb5TKzIFkxMRO+9G bJoeOtEUqIxxw== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:44 +0000 Subject: [PATCH RFC v3 19/27] KVM: arm64: Support SME priority registers Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-19-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=5580; i=broonie@kernel.org; h=from:subject:message-id; bh=nbNePGtF8r3EJAN415gAlX/P47OROWPqRW2uyywE6tA=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBjw1WaaWirHNRbsPCY7dfheJevhiP0Erb7GaMr DSXKCK2JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgYwAKCRAk1otyXVSH0MNaB/ 0V3OPkEINR3rr9MbskzhpJwVpv/Zi6DFRbb9aiFIE/jx5AD79tpsLbNtgVhEX9NTnUsLrW19EoT4Sy zMCLid+1c68k5H1N+CZB/N5n87OUyFckpBjFqzfrJq6oD3o5DviVG2vZQrJzv2mIpUwuYCzkyT8PhV c8Yg0Ix0Avxxhnyp/9zXQIx5DcbaeOlFovdcysq88hKfAEm4GbufXPZAKJOoz1XPZ3ks31yPL+mt0N g74nltEXTJaGBZGzdgqcYkoDTKiFoq/BakL6fOGYQIVVrz27pUVwNHGVXVCVnSFJ8SGN1Lwu37N1T5 Bws2MORMIkGTglmAilbds1FVkVzW3D X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME has optional support for configuring the relative priorities of PEs in systems where they share a single SME hardware block, known as a SMCU. Currently we do not have any support for this in Linux and will also hide it from KVM guests, pending experience with practical implementations. The interface for configuring priority support is via two new system registers, these registers are always available if SME is supported. The register SMPRI_EL1 allows control of SME execution priorities. Since we disable SME priority support for guests this register is RAZ, define it as such and enable fine grained traps for SMPRI_EL1 to ensure that guests can't write to it even if the hardware supports priorites. We could with some adjustment to the generic FGT code allow untrapped reads but since we don't currently advertise priority support to guests there should be no reason for frequent accesses. There is also an EL2 register SMPRIMAP_EL2 for virtualisation of priorities, this is also RAZ when priority configuration is not supported but has no specific traps available. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 2 ++ arch/arm64/include/asm/vncr_mapping.h | 1 + arch/arm64/kvm/sys_regs.c | 30 +++++++++++++++++++++++++----- 3 files changed, 28 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index a304b02efcadba5371edffe97e911bba0634ed62..f72024ffc89d955b3a0a20b6503ceb451abe824f 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -473,6 +473,7 @@ enum vcpu_sysreg { SVCR, FPMR, SMIDR_EL1, /* Streaming Mode Identification Register */ + SMPRI_EL1, /* Streaming Mode Priority Register */ /* 32bit specific registers. */ DACR32_EL2, /* Domain Access Control Register */ @@ -525,6 +526,7 @@ enum vcpu_sysreg { VNCR(CPACR_EL1),/* Coprocessor Access Control */ VNCR(ZCR_EL1), /* SVE Control */ VNCR(SMCR_EL1), /* SME Control */ + VNCR(SMPRIMAP_EL2), /* Streaming Mode Priority Mapping Register */ VNCR(TTBR0_EL1),/* Translation Table Base Register 0 */ VNCR(TTBR1_EL1),/* Translation Table Base Register 1 */ VNCR(TCR_EL1), /* Translation Control Register */ diff --git a/arch/arm64/include/asm/vncr_mapping.h b/arch/arm64/include/asm/vncr_mapping.h index 74fc7400efbc7de6b8dd81a485f1e9d545baf7a9..1685df741294b68e5ae4a4503258c3ee2667dda9 100644 --- a/arch/arm64/include/asm/vncr_mapping.h +++ b/arch/arm64/include/asm/vncr_mapping.h @@ -43,6 +43,7 @@ #define VNCR_ZCR_EL1 0x1E0 #define VNCR_HAFGRTR_EL2 0x1E8 #define VNCR_SMCR_EL1 0x1F0 +#define VNCR_SMPRIMAP_EL2 0x1F0 #define VNCR_TTBR0_EL1 0x200 #define VNCR_TTBR1_EL1 0x210 #define VNCR_FAR_EL1 0x220 diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 416c855153ca532e4c6557d78599e9af0f913071..c327b5544ad95c965a3d24cea059997eb57135f2 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1782,6 +1782,15 @@ static unsigned int fp8_visibility(const struct kvm_vcpu *vcpu, return REG_HIDDEN; } +static unsigned int sme_raz_visibility(const struct kvm_vcpu *vcpu, + const struct sys_reg_desc *rd) +{ + if (vcpu_has_sme(vcpu)) + return REG_RAZ; + + return REG_HIDDEN; +} + static u64 sanitise_id_aa64pfr0_el1(const struct kvm_vcpu *vcpu, u64 val) { if (!vcpu_has_sve(vcpu)) @@ -2719,7 +2728,14 @@ static const struct sys_reg_desc sys_reg_descs[] = { { SYS_DESC(SYS_ZCR_EL1), NULL, reset_val, ZCR_EL1, 0, .visibility = sve_visibility }, { SYS_DESC(SYS_TRFCR_EL1), undef_access }, - { SYS_DESC(SYS_SMPRI_EL1), undef_access }, + + /* + * SMPRI_EL1 is UNDEF when SME is disabled, the UNDEF is + * handled via FGU which is handled without consulting this + * table. + */ + { SYS_DESC(SYS_SMPRI_EL1), trap_raz_wi, .visibility = sme_raz_visibility }, + { SYS_DESC(SYS_SMCR_EL1), NULL, reset_val, SMCR_EL1, 0, .visibility = sme_visibility }, { SYS_DESC(SYS_TTBR0_EL1), access_vm_reg, reset_unknown, TTBR0_EL1 }, { SYS_DESC(SYS_TTBR1_EL1), access_vm_reg, reset_unknown, TTBR1_EL1 }, @@ -3068,6 +3084,8 @@ static const struct sys_reg_desc sys_reg_descs[] = { EL2_REG_VNCR(HCRX_EL2, reset_val, 0), + EL2_REG_FILTERED(SMPRIMAP_EL2, trap_raz_wi, reset_val, 0, + sme_el2_visibility), EL2_REG_FILTERED(SMCR_EL2, access_smcr_el2, reset_val, 0, sme_el2_visibility), @@ -4999,8 +5017,7 @@ void kvm_calculate_traps(struct kvm_vcpu *vcpu) kvm->arch.fgu[HFGxTR_GROUP] = (HFGxTR_EL2_nAMAIR2_EL1 | HFGxTR_EL2_nMAIR2_EL1 | HFGxTR_EL2_nS2POR_EL1 | - HFGxTR_EL2_nACCDATA_EL1 | - HFGxTR_EL2_nSMPRI_EL1_MASK); + HFGxTR_EL2_nACCDATA_EL1); if (!kvm_has_feat(kvm, ID_AA64ISAR0_EL1, TLB, OS)) kvm->arch.fgu[HFGITR_GROUP] |= (HFGITR_EL2_TLBIRVAALE1OS| @@ -5047,8 +5064,11 @@ void kvm_calculate_traps(struct kvm_vcpu *vcpu) kvm->arch.fgu[HAFGRTR_GROUP] |= ~(HAFGRTR_EL2_RES0 | HAFGRTR_EL2_RES1); - if (!kvm_has_feat(kvm, ID_AA64PFR1_EL1, SME, IMP)) - kvm->arch.fgu[HFGxTR_GROUP] |= HFGxTR_EL2_nTPIDR2_EL0; + if (kvm_has_feat(kvm, ID_AA64PFR1_EL1, SME, IMP)) + kvm->arch.fgt[HFGxTR_GROUP] |= HFGxTR_EL2_nSMPRI_EL1_MASK; + else + kvm->arch.fgu[HFGxTR_GROUP] |= (HFGxTR_EL2_nTPIDR2_EL0 | + HFGxTR_EL2_nSMPRI_EL1_MASK); set_bit(KVM_ARCH_FLAG_FGU_INITIALIZED, &kvm->arch.flags); out: From patchwork Fri Dec 20 16:46:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917123 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1D4F4229124; Fri, 20 Dec 2024 16:52:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713533; cv=none; b=i+IXQp+PrIzf5+LGaRNb0i6ifRY1B7XivmT3cOorjZ+8cphh9a8Md6hy89A8oGGp98jEp5ocWVgv9fI4+wJarPH07DipAO1WeHasYxiaSAxoKbfQaTHgpWxCJ3TB3AqEveoQPWSwFEXEQ8EYXI7Gg9JLKvrRwy/uSaqof36poTs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713533; c=relaxed/simple; bh=ENOuC1TJcLemryiEjkWj2obhMQPzOqxnrV0P2mql+Ks=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=kc3oX1L4VRcdJjTnwc7p+yxcmzpejXGkhcziA8kXk8ocilo5ey0tJV8KyJFOEH0sVy85cZLqi/8UKSdLUu/W2rAtgpQ673mlIXdi6c69nfhRqA3YbKMOA5Zad7J39pkSKf2UHZ2c75JHW2JdjKJJcTu37OmCucyTSKIs9d9pVL0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ssi8EUnV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ssi8EUnV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A3D48C4CED7; Fri, 20 Dec 2024 16:52:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713533; bh=ENOuC1TJcLemryiEjkWj2obhMQPzOqxnrV0P2mql+Ks=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ssi8EUnVVNwfhRrHMmu+SM0Q4NnT9gMhK3MqiAI8pA6M0bBsb959zketSgp2K4CN5 JlpNGd3mV8s19GLNEdJ0aEOxoR/MILHM6yw4kfFVMnCkgoIbKEgdUlkM4Ifwp6IdEm D3DW05Lwv9eapii3FYvUq84+RhtvBtmubKLBZoH1hYLZjI14TVC9BoNcfMYY2ng8Ae B6zIJL1JjJ1in59kolXbEgkojFnZZhH4Eih9/LGrLfA3CdV5pciy0hR0RX/7FBMASt YvWyq5ejgR4+RXyjYBys+gtLXfY8uW+wrm3etNfKMUzak/AT4qVMfvL7jaqXSxGGKD eA2epJRg+O6Hg== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:45 +0000 Subject: [PATCH RFC v3 20/27] KVM: arm64: Provide assembly for SME state restore Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-20-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=1651; i=broonie@kernel.org; h=from:subject:message-id; bh=ENOuC1TJcLemryiEjkWj2obhMQPzOqxnrV0P2mql+Ks=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBk3E6dm3e3qW64VbwkZ8BsgUXtHAv6JoenG6H9 ivnP0ZWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgZAAKCRAk1otyXVSH0BkOB/ 9pZK/XWcKBxIAbq8Kxz8iIvhPYuh/k36bbRLFCMarZ1Z5rPMkf8dntgyNKiUgIzPm6Uw+3qRbKIe5E Bm2brl/HNycaJBXjRtj5afSTrMoQDsvGkVtIs0I8duEJFnqCf43+XJ0apduhKJ6oaPiDBTJDnOcZTe ULQT9t7Hwlzm4FsjQuXTcdNoOnXcjEnlG8ng440HTikYzJLhmd6F5lBW/Ivx99Ke5XlM+MRrm3k9VZ ot6KxSdKkSj8wIFQr80TkHle48IdnsMkMSd2EBdOrTcrop+pK8XvZWc6CR5iq8aZcEi165we8JzTAp 5tBTTH/jNzQnZA5FYD5EIyS0Nhtz5x X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Provide a __sme_restore_state() for the hypervisor to allow it to restore ZA and ZT for guests. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_hyp.h | 2 ++ arch/arm64/kvm/hyp/fpsimd.S | 16 ++++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 21943cb98542750a1b626a8de6bbc095d7770ccf..5a1f8e4be18624efa6b887f09c36f0e8ad318c40 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -113,6 +113,8 @@ void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs); void __sve_save_state(void *sve_pffr, u32 *fpsr, int save_ffr); void __sve_restore_state(void *sve_pffr, u32 *fpsr, int restore_ffr); +int __sve_get_vl(void); +void __sme_restore_state(void const *state, bool restore_zt); u64 __guest_enter(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S index e950875e31cee4df58d041519b7584356463c91b..9e4bce86ef2e632a6071480c06a0b7d69bf48f3d 100644 --- a/arch/arm64/kvm/hyp/fpsimd.S +++ b/arch/arm64/kvm/hyp/fpsimd.S @@ -31,3 +31,19 @@ SYM_FUNC_START(__sve_save_state) sve_save 0, x1, x2, 3 ret SYM_FUNC_END(__sve_save_state) + +SYM_FUNC_START(__sve_get_vl) + _sve_rdvl 0, 1 + ret +SYM_FUNC_END(__sve_get_vl) + +SYM_FUNC_START(__sme_restore_state) + _sme_rdsvl 2, 1 // x2 = VL/8 + sme_load_za 0, x2, 12 // Leaves x0 pointing to end of ZA + + cbz x1, 1f + _ldr_zt 0 + +1: + ret +SYM_FUNC_END(__sme_restore_state) From patchwork Fri Dec 20 16:46:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917124 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 31B8D21C9F9; Fri, 20 Dec 2024 16:52:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713537; cv=none; b=O/++RPIiR95aFOKEdW697B2RBgTkCY58P0bEis0U8s4+rUHPjLAPWTgnlSJuQvzVOXw17+X6gMszs13xwtPiEDAyq2lpZJ/BE8aJa9aFe2Dgjp+Xv+NZRJpg9ey97SyHuYN3dd7VgoxMLAKi7ou+qDjkZ0lBxMxNAof0sv9SE0A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713537; c=relaxed/simple; bh=oT97B18Lcj3DIvzsI/I20ru+s+YwnRLB91G6FP1pjN8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=rP1OmGZFc3hinyda3dRHhulCiu+5+FuN+od3CJZg/1hWmPGcOSiKT0UwQZuVUD2ZvlxGG1XqHtgjnk5eskBVPMljgezcuDLbpHiqLTwKXWMaI6rGkns7+2VTqRJ/X7ugiVbKMDAquxAOy3qFWG7Oi6ieTbwhShObMlvIwFoS7B0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QVE4DBGz; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QVE4DBGz" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6BD80C4CECD; Fri, 20 Dec 2024 16:52:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713536; bh=oT97B18Lcj3DIvzsI/I20ru+s+YwnRLB91G6FP1pjN8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=QVE4DBGzEAvkDIs5//Rvftp0K1Zc/67QgtcWHuu8P2+XX48Tlc3XFzjlzxVfQC/mu aDmtV+3Wnx9JQ4eiuCNnp8B6KQzt5h1wRf+1PveezydN7kidzJkV7XrX97oZ49L5wG /aXkAlw57vV5L5Vp2IlLT+4pZezgmb/EPuQsevaWc/ESm3DuGrUqTk2HadEPCo+i7v wE7mEM/06l6vzck1ylzG67cRsJb7255G2LaOtMnEWxXN4VuK11N4jDiISGwn+vS8oY BTc25yL/SZa+8h797XC3YmVzINmAXkOe3NLOYmHYhNwGoFZ8F/NyRXuh2qlM4UyuOj 1TD5ppblofzUw== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:46 +0000 Subject: [PATCH RFC v3 21/27] KVM: arm64: Support Z and P registers in streaming mode Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-21-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=5443; i=broonie@kernel.org; h=from:subject:message-id; bh=oT97B18Lcj3DIvzsI/I20ru+s+YwnRLB91G6FP1pjN8=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBkr3a/U3MJcsXc1G/DdjHUNycFN6r4p1HSt5bz KwIQFnKJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgZAAKCRAk1otyXVSH0NReB/ 9UUiar4/GYJPm8d3uFYutrU3VU6WP4mLzUtuPaDEan5+DXOi9wLRyQowJbJcdmFW3wCLSoVAGrwX3+ QKNuQY0cKI/8Wftjq+dJAOkKqeiun+0r7WZIGqtt6AlnUJo3887OLk5ONaR3cxDo6wzVMJ3XoZZCU0 ZFjujmpHZtR0KPrQmSwfutmV86iCMvhlxWyW7VGb/EU/rOWuIyMg+rG9si7FDbhPx2Uxe6Wov9HA+K KupUKeWF3Vdk2eIS0bPiz6R7GHab24U0AAh+dwe4fpuoJvjN7axRXZTPCt6TWp4AYDVXGvO2X4zJ9+ WtyoyJhO0kDgu8qLaObJUI8uiO/0Xo X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME introduces a mode called streaming mode where the Z, P and optionally FFR registers can be accessed using the SVE instructions but with the SME vector length. Reflect this in the ABI for accessing the guest registers by making the vector length for the vcpu reflect the vector length that would be seen by the guest were it running, using the SME vector length when the guest is configured for streaming mode. Since SME may be present without SVE we also update the existing checks for access to the Z, P and V registers to check for either SVE or streaming mode. When not in streaming mode the guest floating point state may be accessed via the V registers. Any VMM that supports SME must be aware of the need to configure streaming mode prior to writing the floating point registers that this creates. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 9 +++++++-- arch/arm64/kvm/guest.c | 38 ++++++++++++++++++++++++++++++++++---- 2 files changed, 41 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index f72024ffc89d955b3a0a20b6503ceb451abe824f..7393672fa0ee9c4ac13adb48a973f94929f767ea 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1017,16 +1017,18 @@ struct kvm_vcpu_arch { size_t __size_ret; \ unsigned int __vcpu_vq; \ \ - if (WARN_ON(!sve_vl_valid((vcpu)->arch.max_vl[ARM64_VEC_SVE]))) { \ + if (WARN_ON(!sve_vl_valid(vcpu_max_vl(vcpu)))) { \ __size_ret = 0; \ } else { \ - __vcpu_vq = vcpu_sve_max_vq(vcpu); \ + __vcpu_vq = sve_vl_from_vq(vcpu_max_vl(vcpu)); \ __size_ret = SVE_SIG_REGS_SIZE(__vcpu_vq); \ } \ \ __size_ret; \ }) +#define vcpu_sme_state(vcpu) (kern_hyp_va((vcpu)->arch.sme_state)) + /* * Only use __vcpu_sys_reg/ctxt_sys_reg if you know you want the * memory backed version of a register, and not the one most recently @@ -1583,4 +1585,7 @@ void kvm_set_vm_id_reg(struct kvm *kvm, u32 reg, u64 val); (system_supports_sme() && \ kvm_has_feat((k), ID_AA64PFR1_EL1, SME, SME2)) +#define vcpu_in_streaming_mode(vcpu) \ + (__vcpu_sys_reg(vcpu, SVCR) & SVCR_SM_MASK) + #endif /* __ARM64_KVM_HOST_H__ */ diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index 8820cd42a27ac05874ed52c572c1dd9b29a77a1b..cf468ac93c9e75d642d7293e020d04c4267ffff4 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -73,6 +73,11 @@ static u64 core_reg_offset_from_id(u64 id) return id & ~(KVM_REG_ARCH_MASK | KVM_REG_SIZE_MASK | KVM_REG_ARM_CORE); } +static bool vcpu_has_sve_regs(const struct kvm_vcpu *vcpu) +{ + return vcpu_has_sve(vcpu) || vcpu_in_streaming_mode(vcpu); +} + static int core_reg_size_from_offset(const struct kvm_vcpu *vcpu, u64 off) { int size; @@ -110,9 +115,10 @@ static int core_reg_size_from_offset(const struct kvm_vcpu *vcpu, u64 off) /* * The KVM_REG_ARM64_SVE regs must be used instead of * KVM_REG_ARM_CORE for accessing the FPSIMD V-registers on - * SVE-enabled vcpus: + * SVE-enabled vcpus or when a SME enabled vcpu is in + * streaming mode: */ - if (vcpu_has_sve(vcpu) && core_reg_offset_is_vreg(off)) + if (vcpu_has_sve_regs(vcpu) && core_reg_offset_is_vreg(off)) return -EINVAL; return size; @@ -426,6 +432,24 @@ struct vec_state_reg_region { unsigned int upad; /* extra trailing padding in user memory */ }; +/* + * We represent the Z and P registers to userspace using either the + * SVE or SME vector length, depending on which features the guest has + * and if the guest is in streaming mode. + */ +static unsigned int vcpu_sve_cur_vq(struct kvm_vcpu *vcpu) +{ + unsigned int vq = 0; + + if (vcpu_has_sve(vcpu)) + vq = vcpu_sve_max_vq(vcpu); + + if (vcpu_in_streaming_mode(vcpu)) + vq = vcpu_sme_max_vq(vcpu); + + return vq; +} + /* * Validate SVE register ID and get sanitised bounds for user/kernel SVE * register copy @@ -466,7 +490,7 @@ static int sve_reg_to_region(struct vec_state_reg_region *region, if (!vcpu_has_sve(vcpu) || (reg->id & SVE_REG_SLICE_MASK) > 0) return -ENOENT; - vq = vcpu_sve_max_vq(vcpu); + vq = vcpu_sve_cur_vq(vcpu); reqoffset = SVE_SIG_ZREG_OFFSET(vq, reg_num) - SVE_SIG_REGS_OFFSET; @@ -476,7 +500,7 @@ static int sve_reg_to_region(struct vec_state_reg_region *region, if (!vcpu_has_sve(vcpu) || (reg->id & SVE_REG_SLICE_MASK) > 0) return -ENOENT; - vq = vcpu_sve_max_vq(vcpu); + vq = vcpu_sve_cur_vq(vcpu); reqoffset = SVE_SIG_PREG_OFFSET(vq, reg_num) - SVE_SIG_REGS_OFFSET; @@ -515,6 +539,9 @@ static int get_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) if (!kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; + if (!vcpu_has_sve_regs(vcpu)) + return -EBUSY; + if (copy_to_user(uptr, vcpu->arch.sve_state + region.koffset, region.klen) || clear_user(uptr + region.klen, region.upad)) @@ -541,6 +568,9 @@ static int set_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) if (!kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; + if (!vcpu_has_sve_regs(vcpu)) + return -EBUSY; + if (copy_from_user(vcpu->arch.sve_state + region.koffset, uptr, region.klen)) return -EFAULT; From patchwork Fri Dec 20 16:46:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917125 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 926EE229B02; Fri, 20 Dec 2024 16:52:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713540; cv=none; b=N+dmk0ys5WPr3if1/Ua9c8sedvXsDlAKPgmKY8bSqEjbzOOfDnjAP30elVijyyUT1ptXRq6haCy9DIsypLQnyrwpYsDmQAbAru+5e3p7sGyla6SuDowDilTgJSnb4uDRwrmDT2g75LiMZY3IH6e2wiy9eldkwjhBfyVgp28Ru3c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713540; c=relaxed/simple; bh=VTrfqmU0w3nAHvC2QpKS0cpx16L+oXuNWw9Ueg9bmLQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=kIGRtHtLMJIqL6Qpz4WlOEG39GWuMi30qhWWhcfxqoIZJWuHX+Ig38M5puE7/8VzUkB9RwEBdHsnVpeOAGUqw3qXSSRVhOK3AN8w7cPBcbzuMjEUmke8fdbY8S4c6wMzNIgGcYeIEjCHaFIgqtJNAImB/PJkGv1nUgYbtye/2rA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qxF4HzCY; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qxF4HzCY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2ABE1C4CEDE; Fri, 20 Dec 2024 16:52:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713540; bh=VTrfqmU0w3nAHvC2QpKS0cpx16L+oXuNWw9Ueg9bmLQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=qxF4HzCYE6OBWCA89xa34iOlhk+jCjymHi4GHS0mzK1Ef6ZcHGhwiqUkxMo1FaaeD BQhWwLWOU3JtNHlcBSQmFqvQpkhC+HrdIQ2liyFBF6b5cPQa02wWXPn6uEjXIsAjNQ nasq4ngzZKw6Jw3Zv5RBP+Ya0SIFjxD2ejvAWoVxe1pL9CxiOiKUMvQfQcmtDxRgSC kdtNfOWIKXSYm5RjGImEofDpdWEj0/gvaGTD6b8i0cQqQEW1bPo6IApKZZ33rz1EHY 5wreWHm38C1z5asNv69JiN6tzKxp+wjVblmtLiiKucGLw+eBrf01cyQEFWnn9q8tij ZXWkrMvJ54JCA== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:47 +0000 Subject: [PATCH RFC v3 22/27] KVM: arm64: Expose SME specific state to userspace Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-22-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=8295; i=broonie@kernel.org; h=from:subject:message-id; bh=VTrfqmU0w3nAHvC2QpKS0cpx16L+oXuNWw9Ueg9bmLQ=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBlHn387PCb2XL6zkz5FUtjqLQ07RLFXrxbc2nw N2vw43mJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgZQAKCRAk1otyXVSH0LBgB/ 0dCjYpNVQaZcv3rNvxGd3kWiiWjUY4Qt3BOWPGwzDnAfkqRoY2GPVc7f1q/5VP3lmHHZKapZE1KbLL 9t+hViZi17u2KbomWg3jmOJrBA37FMiS4CG05CSf9FzCXz3IVzW7NMoNzBRwaXTMS/xjlOOX39KbDo d0D5wvRXQBUdtaV9vYSeVZ0VSNqVA+QeoRiyAjr7YjR5FohZEtygaJEqjStpHIiiz4BR90vw4kVHKN NOAoivOKUrExRepDVvhnCY4TFxDseiwU+vUvNrOsJlTFXwROWgPO1kYD6OG79z+lYwWx1q2GsiPdNx CpQF6mBh1wsnHISZx7xYc5vYWIALnu X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME introduces two new registers, the ZA matrix register and the ZT0 LUT register. Both of these registers are only accessible when PSTATE.ZA is set and ZT0 is only present if SME2 is enabled for the guest. Provide support for configuring these from VMMs. The ZA matrix is a single SVL*SVL register which is available when PSTATE.ZA is set. We follow the pattern established by the architecture itself and expose this to userspace as a series of horizontal SVE vectors with the streaming mode vector length, using the format already established for the SVE vectors themselves. ZT0 is a single register with a refreshingly fixed size 512 bit register which is like ZA accessible only when PSTATE.ZA is set. Add support for it to the userspace API, as with ZA we allow the register to be read or written regardless of the state of PSTATE.ZA in order to simplify userspace usage. The value will be reset to 0 whenever PSTATE.ZA changes from 0 to 1, userspace can read stale values but these are not observable by the guest without manipulation of PSTATE.ZA by userspace. While there is currently only one ZT register the naming as ZT0 and the instruction encoding clearly leave room for future extensions adding more ZT registers. This encoding can readily support such an extension if one is introduced. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 19 +++++++ arch/arm64/include/uapi/asm/kvm.h | 17 ++++++ arch/arm64/kvm/guest.c | 114 +++++++++++++++++++++++++++++++++++++- 3 files changed, 148 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 7393672fa0ee9c4ac13adb48a973f94929f767ea..3e064520a86f25fb7b1185b3aca342f593f04994 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1029,6 +1029,22 @@ struct kvm_vcpu_arch { #define vcpu_sme_state(vcpu) (kern_hyp_va((vcpu)->arch.sme_state)) +#define vcpu_sme_state_size(vcpu) ({ \ + size_t __size_ret; \ + unsigned int __vcpu_vq; \ + \ + if (WARN_ON(!sve_vl_valid((vcpu)->arch.max_vl[ARM64_VEC_SME]))) { \ + __size_ret = 0; \ + } else { \ + __vcpu_vq = vcpu_sme_max_vq(vcpu); \ + __size_ret = ZA_SIG_REGS_SIZE(__vcpu_vq); \ + if (system_supports_sme2()) \ + __size_ret += ZT_SIG_REG_SIZE; \ + } \ + \ + __size_ret; \ +}) + /* * Only use __vcpu_sys_reg/ctxt_sys_reg if you know you want the * memory backed version of a register, and not the one most recently @@ -1588,4 +1604,7 @@ void kvm_set_vm_id_reg(struct kvm *kvm, u32 reg, u64 val); #define vcpu_in_streaming_mode(vcpu) \ (__vcpu_sys_reg(vcpu, SVCR) & SVCR_SM_MASK) +#define vcpu_za_enabled(vcpu) \ + (__vcpu_sys_reg(vcpu, SVCR) & SVCR_ZA_MASK) + #endif /* __ARM64_KVM_HOST_H__ */ diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h index efb384cf9d503007f68aad9233ba949128c94b8b..5092f39138cbf17d9e89191de23ab2ee9f3fa77d 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -359,6 +359,23 @@ struct kvm_arm_counter_offset { /* SME registers */ #define KVM_REG_ARM64_SME (0x17 << KVM_REG_ARM_COPROC_SHIFT) +#define KVM_ARM64_SME_VQ_MIN __SVE_VQ_MIN +#define KVM_ARM64_SME_VQ_MAX __SVE_VQ_MAX + +/* ZA and ZTn occupy blocks at the following offsets within this range: */ +#define KVM_REG_ARM64_SME_ZA_BASE 0 +#define KVM_REG_ARM64_SME_ZT_BASE 0x600 + +#define KVM_ARM64_SME_MAX_ZAHREG (__SVE_VQ_BYTES * KVM_ARM64_SME_VQ_MAX) + +#define KVM_REG_ARM64_SME_ZAHREG(n, i) \ + (KVM_REG_ARM64 | KVM_REG_ARM64_SME | KVM_REG_ARM64_SME_ZA_BASE | \ + KVM_REG_SIZE_U2048 | \ + (((n) & (KVM_ARM64_SME_MAX_ZAHREG - 1)) << 5) | \ + ((i) & (KVM_ARM64_SVE_MAX_SLICES - 1))) + +#define KVM_REG_ARM64_SME_ZTREG_SIZE (512 / 8) + /* Vector lengths pseudo-register: */ #define KVM_REG_ARM64_SME_VLS (KVM_REG_ARM64 | KVM_REG_ARM64_SME | \ KVM_REG_SIZE_U512 | 0xffff) diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index cf468ac93c9e75d642d7293e020d04c4267ffff4..ad32f0f539be9acd5ff78412b369d4134b30559f 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -600,23 +600,133 @@ static int set_sme_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) return set_vec_vls(ARM64_VEC_SME, vcpu, reg); } +/* + * Validate SVE register ID and get sanitised bounds for user/kernel SVE + * register copy + */ +static int sme_reg_to_region(struct vec_state_reg_region *region, + struct kvm_vcpu *vcpu, + const struct kvm_one_reg *reg) +{ + /* reg ID ranges for ZA.H[n] registers */ + unsigned int vq = vcpu_sme_max_vq(vcpu) - 1; + const u64 za_h_max = vq * __SVE_VQ_BYTES; + const u64 zah_id_min = KVM_REG_ARM64_SME_ZAHREG(0, 0); + const u64 zah_id_max = KVM_REG_ARM64_SME_ZAHREG(za_h_max - 1, + SVE_NUM_SLICES - 1); + unsigned int reg_num; + + unsigned int reqoffset, reqlen; /* User-requested offset and length */ + unsigned int maxlen; /* Maximum permitted length */ + + size_t sme_state_size; + + reg_num = (reg->id & SVE_REG_ID_MASK) >> SVE_REG_ID_SHIFT; + + if (reg->id >= zah_id_min && reg->id <= zah_id_max) { + if (!vcpu_has_sme(vcpu) || (reg->id & SVE_REG_SLICE_MASK) > 0) + return -ENOENT; + + /* ZA is exposed as SVE vectors ZA.H[n] */ + reqoffset = ZA_SIG_ZAV_OFFSET(vq, reg_num) - + ZA_SIG_REGS_OFFSET; + reqlen = KVM_SVE_ZREG_SIZE; + maxlen = SVE_SIG_ZREG_SIZE(vq); + } else if (reg->id == KVM_REG_ARM64_SME_ZT_BASE) { + /* ZA is exposed as SVE vectors ZA.H[n] */ + if (!kvm_has_feat(vcpu->kvm, ID_AA64PFR1_EL1, SME, SME2) || + (reg->id & SVE_REG_SLICE_MASK) > 0 || + reg_num > 0) + return -ENOENT; + + /* ZT0 is stored after ZA */ + reqlen = KVM_REG_ARM64_SME_ZTREG_SIZE; + maxlen = KVM_REG_ARM64_SME_ZTREG_SIZE; + } else { + return -EINVAL; + } + + sme_state_size = vcpu_sme_state_size(vcpu); + if (WARN_ON(!sme_state_size)) + return -EINVAL; + + region->koffset = array_index_nospec(reqoffset, sme_state_size); + region->klen = min(maxlen, reqlen); + region->upad = reqlen - region->klen; + + return 0; +} + +/* + * ZA is exposed as an array of horizontal vectors with the same + * format as SVE, mirroring the architecture's LDR ZA[Wv, offs], [Xn] + * instruction. + */ + static int get_sme_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) { + int ret; + struct vec_state_reg_region region; + char __user *uptr = (char __user *)reg->addr; + /* Handle the KVM_REG_ARM64_SME_VLS pseudo-reg as a special case: */ if (reg->id == KVM_REG_ARM64_SME_VLS) return get_sme_vls(vcpu, reg); - return -EINVAL; + /* Try to interpret reg ID as an architectural SME register... */ + ret = sme_reg_to_region(®ion, vcpu, reg); + if (ret) + return ret; + + if (!kvm_arm_vcpu_vec_finalized(vcpu)) + return -EPERM; + + /* + * None of the SME specific registers are accessible unless + * PSTATE.ZA is set. + */ + if (!vcpu_za_enabled(vcpu)) + return -EINVAL; + + if (copy_from_user(vcpu->arch.sme_state + region.koffset, uptr, + region.klen)) + return -EFAULT; + + return 0; } static int set_sme_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) { + int ret; + struct vec_state_reg_region region; + char __user *uptr = (char __user *)reg->addr; + /* Handle the KVM_REG_ARM64_SME_VLS pseudo-reg as a special case: */ if (reg->id == KVM_REG_ARM64_SME_VLS) return set_sme_vls(vcpu, reg); - return -EINVAL; + /* Try to interpret reg ID as an architectural SME register... */ + ret = sme_reg_to_region(®ion, vcpu, reg); + if (ret) + return ret; + + if (!kvm_arm_vcpu_vec_finalized(vcpu)) + return -EPERM; + + /* + * None of the SME specific registers are accessible unless + * PSTATE.ZA is set. + */ + if (!vcpu_za_enabled(vcpu)) + return -EINVAL; + + if (copy_from_user(vcpu->arch.sme_state + region.koffset, uptr, + region.klen)) + return -EFAULT; + + return 0; } + int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs) { return -EINVAL; From patchwork Fri Dec 20 16:46:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917126 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACA62229B33; Fri, 20 Dec 2024 16:52:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713544; cv=none; b=ECFrgosigGYEY+xmJJwwd+/XD6aYjaHBDL915UuNTJBNhuIXfvp79lEihOuWRaI9pKVxYf55rjPH5BKI3p5sYCCMDmGjQeO80UIXHD57lm+HD+Z4GX6mjviGwvqENUqwJYycQsyoGXuryIh2CF/zyPVpcL+v1WJh7jHoxqloDaM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713544; c=relaxed/simple; bh=as9CT/g9mNlDzCdZ9peiKL5S9mY67eyR2QFZtQSInNI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=HIQfKnvziNKCyJplZpInh6X8pH0TGE98gaWKEQxJbjKcG7QRc04k7Eelrac6ydCAYfm6UJdi5uF7F/LbUCeUDmDLmLNaWJjhwN6AaFYG/Ca+t2AMbtFG3h4nBwEeTTu4iaBOLSphxkya5vnqxrfv6DjrfATGB+7X4GAuyQhSbCU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qzRTli+z; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qzRTli+z" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E03C8C4CED7; Fri, 20 Dec 2024 16:52:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713544; bh=as9CT/g9mNlDzCdZ9peiKL5S9mY67eyR2QFZtQSInNI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=qzRTli+zDj0vEd6Vr7gHWaH4rdO1HaJf71+tvwwSX4tSmpbTozhLsNwLgESByiZf/ l6vFAw0oUN90YoZqL3ntPpRA65wvnwlN7ISEXu7+46nnEzQrlk+WD/84X5qqova993 uqmSlnAjstZtRuLhx3a4lqwhhcFodtw41LlWqGaiGYMBx+3tuq0FpbaY0TTFBCVDSU G5sxgGvkhLXjwB170VnP80crh5OEtzPIm2dfCN/Yu09CS9YenE559ybMMMgZL8vQHZ 4JeSJm90JUmEtGrzzukNxuN8fSRsFCrrFP4He04oZ0PBr3v9oguN38XsW9OUpyd9M7 CDFsGlFoLLbdg== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:48 +0000 Subject: [PATCH RFC v3 23/27] KVM: arm64: Context switch SME state for normal guests Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-23-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=13636; i=broonie@kernel.org; h=from:subject:message-id; bh=as9CT/g9mNlDzCdZ9peiKL5S9mY67eyR2QFZtQSInNI=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBmUnaId7VIz0BTgZ76cluVdFm7HqpNhTRSxK2J o9YXQZ+JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgZgAKCRAk1otyXVSH0LCYB/ wKhqcGuUS4wwdvXM3rFqtnty0GLrB7nhplHdYuqS3ICN4WKfq8eMOnojr1FVxrDHI+NvV1yGVC6RmT 2dH+OgZPEHB5IJmkfyiG3uu0PaX430NAHogQNNxe+DhHCphMtwvaVb3iSxhSL7G+zX8TdvXClMmz3t yb+V9Pnbu1RUicTK1qe+ojyZ76OdQmWr/624HIDI7U2/vUDNUA1Krh6hxsuVFP9Nt87Qn0nx3DDHNG SmQBUWPYABLASurZArNXuzwRakcnSTvHL5ydd5t6/V+kltNbE6a+nHwbrAKnJqHCAkYkxxCx+3Lyu0 /0RI84hNLL4IVcgcS/kqc82A0GFXM4 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB If the guest has SME state we need to context switch that state, provide support for that for normal guests. SME has three sets of registers, ZA, ZT (only present for SME2) and also streaming SVE which replaces the standard floating point registers when active. The first two are fairly straightforward, they are accessible only when PSTATE.ZA is set and we can reuse the assembly from the host to save and load them from a single contiguous buffer. When PSTATE.ZA is not set then these registers are inaccessible, if the guest enables PSTATE.ZA then all bits will be set to 0 and nothing is required on restore. Streaming mode is slightly more complicated, when enabled via PSTATE.SM it provides a version of the SVE registers using the SME vector length and may optionally omit the FFR register. SME may also be present without SVE. The register state is stored in sve_state as for non-streaming SVE mode, we make an initial selection of registers to update based on the guest SVE support and then override this when loading SVCR if streaming mode is enabled. Since in order to avoid duplication with SME we now restore the register state outside of the SVE specific restore function we need to move the restore of the effective VL for nested guests to a separate restore function run after loading the floating point register state, along with the similar handling required for SME. The selection of which vector length to use is handled by vcpu_sve_pffr(). Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 9 ++++ arch/arm64/include/asm/kvm_emulate.h | 6 +++ arch/arm64/include/asm/kvm_host.h | 3 ++ arch/arm64/kvm/fpsimd.c | 86 ++++++++++++++++++++---------- arch/arm64/kvm/hyp/include/hyp/switch.h | 93 ++++++++++++++++++++++++++++----- 5 files changed, 156 insertions(+), 41 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index 144cc805bfea112341b89c9c6028cf4b2a201c6c..f517b371e0132271a9bd693349a828e2b824ff07 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -442,6 +442,15 @@ static inline size_t sme_state_size(struct task_struct const *task) write_sysreg_s(__new, (reg)); \ } while (0) +#define sme_cond_update_smcr_vq(val, reg) \ + do { \ + u64 __smcr = read_sysreg_s((reg)); \ + u64 __new = __smcr & ~SMCR_ELx_LEN_MASK; \ + __new |= (val) & SMCR_ELx_LEN_MASK; \ + if (__smcr != __new) \ + write_sysreg_s(__new, (reg)); \ + } while (0) + /* For use by EFI runtime services calls only */ extern void __efi_fpsimd_begin(void); extern void __efi_fpsimd_end(void); diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h index 5f05da7f538d29d321c424233f21b8448d8b4628..c7f3d14c1d69d9b3f7c1c22ad0919c278d2140c1 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -688,4 +688,10 @@ static inline bool guest_hyp_sve_traps_enabled(const struct kvm_vcpu *vcpu) { return __guest_hyp_cptr_xen_trap_enabled(vcpu, ZEN); } + +static inline bool guest_hyp_sme_traps_enabled(const struct kvm_vcpu *vcpu) +{ + return __guest_hyp_cptr_xen_trap_enabled(vcpu, SMEN); +} + #endif /* __ARM64_KVM_EMULATE_H__ */ diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 3e064520a86f25fb7b1185b3aca342f593f04994..4fcb2c2603ae2bc51d6993f1f6a3f81f2689717c 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1013,6 +1013,9 @@ struct kvm_vcpu_arch { #define vcpu_sve_zcr_elx(vcpu) \ (unlikely(is_hyp_ctxt(vcpu)) ? ZCR_EL2 : ZCR_EL1) +#define vcpu_sme_smcr_elx(vcpu) \ + (unlikely(is_hyp_ctxt(vcpu)) ? SMCR_EL2 : SMCR_EL1) + #define vcpu_sve_state_size(vcpu) ({ \ size_t __size_ret; \ unsigned int __vcpu_vq; \ diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index 51c844e25dfa460ecab5bb0dfc50c7680318aa20..d2a47d7163374ea51157c4817dd13fa43bd2146a 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -127,19 +127,25 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu) WARN_ON_ONCE(!irqs_disabled()); if (guest_owns_fp_regs()) { - /* - * Currently we do not support SME guests so SVCR is - * always 0 and we just need a variable to point to. - */ fp_state.st = &vcpu->arch.ctxt.fp_regs; fp_state.sve_state = vcpu->arch.sve_state; fp_state.sve_vl = vcpu->arch.max_vl[ARM64_VEC_SVE]; - fp_state.sme_state = NULL; + fp_state.sme_vl = vcpu->arch.max_vl[ARM64_VEC_SME]; + fp_state.sme_state = vcpu->arch.sme_state; fp_state.svcr = &__vcpu_sys_reg(vcpu, SVCR); fp_state.fpmr = &__vcpu_sys_reg(vcpu, FPMR); fp_state.fp_type = &vcpu->arch.fp_type; + fp_state.sme_features = 0; + if (kvm_has_fa64(vcpu->kvm)) + fp_state.sme_features |= SMCR_ELx_FA64; + if (kvm_has_sme2(vcpu->kvm)) + fp_state.sme_features |= SMCR_ELx_EZT0; + /* + * For SME only hosts fpsimd_save() will override the + * state selection if we are in streaming mode. + */ if (vcpu_has_sve(vcpu)) fp_state.to_save = FP_STATE_SVE; else @@ -186,6 +192,32 @@ static void kvm_vcpu_put_sve(struct kvm_vcpu *vcpu) SYS_ZCR_EL1); } +static void kvm_vcpu_put_sme(struct kvm_vcpu *vcpu) +{ + u64 smcr; + + if (!vcpu_has_sme(vcpu)) + return; + + smcr = read_sysreg_el1(SYS_SMCR); + + /* + * If the vCPU is in the hyp context then SMCR_EL1 is loaded + * with its vEL2 counterpart. + */ + __vcpu_sys_reg(vcpu, vcpu_sme_smcr_elx(vcpu)) = smcr; + + /* + * As for SVE we always save the SME state for the guest using + * the maximum VL supported by the guest so if we are using + * nVHE or were in a nested guest we need to set the VL for + * the host to match. + */ + if (!has_vhe() || (vcpu_has_nv(vcpu) && !is_hyp_ctxt(vcpu))) + sme_cond_update_smcr_vq(vcpu_sme_max_vq(vcpu) - 1, + SYS_SMCR_EL1); +} + /* * Write back the vcpu FPSIMD regs if they are dirty, and invalidate the * cpu FPSIMD regs so that they can't be spuriously reused if this vcpu @@ -198,23 +230,9 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu) local_irq_save(flags); - /* - * If we have VHE then the Hyp code will reset CPACR_EL1 to - * the default value and we need to reenable SME. - */ - if (has_vhe() && system_supports_sme()) { - /* Also restore EL0 state seen on entry */ - if (vcpu_get_flag(vcpu, HOST_SME_ENABLED)) - sysreg_clear_set(CPACR_EL1, 0, CPACR_ELx_SMEN); - else - sysreg_clear_set(CPACR_EL1, - CPACR_EL1_SMEN_EL0EN, - CPACR_EL1_SMEN_EL1EN); - isb(); - } - if (guest_owns_fp_regs()) { kvm_vcpu_put_sve(vcpu); + kvm_vcpu_put_sme(vcpu); /* * Flush (save and invalidate) the FP state so that if @@ -227,18 +245,30 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu) * when needed. */ fpsimd_save_and_flush_cpu_state(); - } else if (has_vhe() && system_supports_sve()) { + } else if (has_vhe() && (system_supports_sve() || + system_supports_sme())) { /* - * The FPSIMD/SVE state in the CPU has not been touched, and we - * have SVE (and VHE): CPACR_EL1 (alias CPTR_EL2) has been - * reset by kvm_reset_cptr_el2() in the Hyp code, disabling SVE - * for EL0. To avoid spurious traps, restore the trap state - * seen by kvm_arch_vcpu_load_fp(): + * The FP state in the CPU has not been touched, and + * we have a vector extension (and VHE): CPACR_EL1 + * (alias CPTR_EL2) has been reset by + * kvm_reset_cptr_el2() in the Hyp code, disabling SVE + * for EL0. To avoid spurious traps, restore the trap + * state seen by kvm_arch_vcpu_load_fp(): */ + u64 clear = 0; + u64 set = 0; + if (vcpu_get_flag(vcpu, HOST_SVE_ENABLED)) - sysreg_clear_set(CPACR_EL1, 0, CPACR_EL1_ZEN_EL0EN); + set |= CPACR_EL1_ZEN_EL0EN; else - sysreg_clear_set(CPACR_EL1, CPACR_EL1_ZEN_EL0EN, 0); + clear |= CPACR_EL1_ZEN_EL0EN; + + if (vcpu_get_flag(vcpu, HOST_SME_ENABLED)) + set |= CPACR_EL1_SMEN_EL0EN; + else + clear |= CPACR_EL1_SMEN_EL0EN; + + sysreg_clear_set(CPACR_EL1, clear, set); } local_irq_restore(flags); diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 09a9a237d6dd22d4bb941714363675abdab1baa7..3aed023ccaf336320b5ca5acab82e30fb52fb63d 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -343,6 +343,37 @@ static bool kvm_hyp_handle_mops(struct kvm_vcpu *vcpu, u64 *exit_code) return true; } +static inline void __hyp_sme_restore_guest(struct kvm_vcpu *vcpu, + bool *restore_sve, + bool *restore_ffr) +{ + u64 old_smcr, new_smcr; + struct kvm *kvm = kern_hyp_va(vcpu->kvm); + bool has_fa64 = kvm_has_fa64(kvm); + bool has_sme2 = kvm_has_sme2(kvm); + + old_smcr = read_sysreg_s(SYS_SMCR_EL2); + new_smcr = vcpu_sme_max_vq(vcpu) - 1; + if (has_fa64) + new_smcr |= SMCR_ELx_FA64_MASK; + if (has_sme2) + new_smcr |= SMCR_ELx_EZT0_MASK; + if (old_smcr != new_smcr) + write_sysreg_s(new_smcr, SYS_SMCR_EL2); + + write_sysreg_el1(__vcpu_sys_reg(vcpu, SMCR_EL1), SYS_SMCR); + + write_sysreg_s(__vcpu_sys_reg(vcpu, SVCR), SYS_SVCR); + + if (vcpu_in_streaming_mode(vcpu)) { + *restore_sve = true; + *restore_ffr = has_fa64; + } + + if (vcpu_za_enabled(vcpu)) + __sme_restore_state(vcpu_sme_state(vcpu), has_sme2); +} + static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu) { /* @@ -350,19 +381,26 @@ static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu) * vCPU. Start off with the max VL so we can load the SVE state. */ sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2); - __sve_restore_state(vcpu_sve_pffr(vcpu), - &vcpu->arch.ctxt.fp_regs.fpsr, - true); + write_sysreg_el1(__vcpu_sys_reg(vcpu, vcpu_sve_zcr_elx(vcpu)), SYS_ZCR); +} + +static inline void __hyp_nv_restore_guest_vls(struct kvm_vcpu *vcpu) +{ /* - * The effective VL for a VM could differ from the max VL when running a - * nested guest, as the guest hypervisor could select a smaller VL. Slap - * that into hardware before wrapping up. + * The effective VL for a VM could differ from the max VL when + * running a nested guest, as the guest hypervisor could + * select a smaller VL. Slap that into hardware before + * wrapping up. */ - if (vcpu_has_nv(vcpu) && !is_hyp_ctxt(vcpu)) + if (!(vcpu_has_nv(vcpu) && !is_hyp_ctxt(vcpu))) + return; + + if (vcpu_has_sve(vcpu)) sve_cond_update_zcr_vq(__vcpu_sys_reg(vcpu, ZCR_EL2), SYS_ZCR_EL2); - write_sysreg_el1(__vcpu_sys_reg(vcpu, vcpu_sve_zcr_elx(vcpu)), SYS_ZCR); + if (vcpu_has_sme(vcpu)) + sme_cond_update_smcr_vq(__vcpu_sys_reg(vcpu, SMCR_EL2), SYS_SMCR_EL2); } static inline void __hyp_sve_save_host(void) @@ -386,14 +424,18 @@ static void kvm_hyp_save_fpsimd_host(struct kvm_vcpu *vcpu); */ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code) { - bool sve_guest; - u8 esr_ec; + u64 cpacr; + bool restore_sve, restore_ffr; + bool sve_guest, sme_guest; + u8 esr_ec, esr_iss; if (!system_supports_fpsimd()) return false; sve_guest = vcpu_has_sve(vcpu); + sme_guest = vcpu_has_sme(vcpu); esr_ec = kvm_vcpu_trap_get_class(vcpu); + esr_iss = ESR_ELx_ISS(kvm_vcpu_get_esr(vcpu)); /* Only handle traps the vCPU can support here: */ switch (esr_ec) { @@ -412,6 +454,15 @@ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code) if (guest_hyp_sve_traps_enabled(vcpu)) return false; break; + case ESR_ELx_EC_SME: + if (!sme_guest) + return false; + if (guest_hyp_sme_traps_enabled(vcpu)) + return false; + if (!kvm_has_sme2(vcpu->kvm) && + (esr_iss == ESR_ELx_SME_ISS_ZT_DISABLED)) + return false; + break; default: return false; } @@ -419,10 +470,12 @@ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code) /* Valid trap. Switch the context: */ /* First disable enough traps to allow us to update the registers */ + cpacr = CPACR_ELx_FPEN; if (sve_guest || (is_protected_kvm_enabled() && system_supports_sve())) - cpacr_clear_set(0, CPACR_ELx_FPEN | CPACR_ELx_ZEN); - else - cpacr_clear_set(0, CPACR_ELx_FPEN); + cpacr |= CPACR_ELx_ZEN; + if (sme_guest) + cpacr |= CPACR_ELx_SMEN; + cpacr_clear_set(0, cpacr); isb(); /* Write out the host state if it's in the registers */ @@ -430,8 +483,20 @@ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code) kvm_hyp_save_fpsimd_host(vcpu); /* Restore the guest state */ + + /* These may be overridden for a SME guest */ + restore_sve = sve_guest; + restore_ffr = sve_guest; + if (sve_guest) __hyp_sve_restore_guest(vcpu); + if (sme_guest) + __hyp_sme_restore_guest(vcpu, &restore_sve, &restore_ffr); + + if (restore_sve) + __sve_restore_state(vcpu_sve_pffr(vcpu), + &vcpu->arch.ctxt.fp_regs.fpsr, + restore_ffr); else __fpsimd_restore_state(&vcpu->arch.ctxt.fp_regs); @@ -442,6 +507,8 @@ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code) if (!(read_sysreg(hcr_el2) & HCR_RW)) write_sysreg(__vcpu_sys_reg(vcpu, FPEXC32_EL2), fpexc32_el2); + __hyp_nv_restore_guest_vls(vcpu); + *host_data_ptr(fp_owner) = FP_STATE_GUEST_OWNED; return true; From patchwork Fri Dec 20 16:46:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917127 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5BC6C22A1EA; Fri, 20 Dec 2024 16:52:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713548; cv=none; b=LJkVXiGX/qCLaA4t5SGStOMGJ/LolVaGETq3wiDAYEEcNTTt7wiDhm9QJujkIcdzzR5k1kTiaLRsrE+Nn+9WpYZsX38/OIzxBw496IK7GQTbvXXRHIyj2Bdqy8MEAQSAd+ARHvP3PBX6Id/bXQmcigzbjClDv2s2sZmrH3t5o1M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713548; c=relaxed/simple; bh=xfubkaGiJKFVfZEF1qDdQha6MtUZ0W2BsuPsXw245lc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=J6OEPMtwudS2pcu30+AKgPhpyuEcM2E/5ovMJCTccOOzz0UatokszeaLIe1Z2b9qcNtH0WjXO2Zlhs2RVcEzZOkrZuas6uKW3V154MtjW4kQrhJ/Bg7ShJJHmpJj0uc9CI6En60Ov7OxbyZyxchD1+k7VMYVTElHO2dCn08obDA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ebhH6sA4; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ebhH6sA4" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A8A04C4CEE2; Fri, 20 Dec 2024 16:52:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713548; bh=xfubkaGiJKFVfZEF1qDdQha6MtUZ0W2BsuPsXw245lc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ebhH6sA4HrUQCgHSXZ1JpeaWxKMuuwEqhTpWV1q63LMujLJHAjp/QD/3Bv2+6cjec PtMBLMOFzwDA3Jhd3llNd20nfxvc76X3a2q2X3SBNt6Z+YNXXQXHViZ0ws04/GZbrn Nc8ehOYEDGMKi6Wpu0jlVJddfo3kxAXBdAdbjH/N553zqEc01I6WBECv25ZpmdbmV9 qn7E10EVpWSjjA4ZtRzUjGQUbcVu6BXbJZRlXKt/ovfJX/Vx10kn11Piv9Wb+PDehg U+8tnilWdXUo9aksu7KfBWlcs7WVWL60AL7KOEU/7blUEFa7BQdgs++CYGTGQ4hPbT E6bTXy8Nb9v6A== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:49 +0000 Subject: [PATCH RFC v3 24/27] KVM: arm64: Handle SME exceptions Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-24-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=8144; i=broonie@kernel.org; h=from:subject:message-id; bh=xfubkaGiJKFVfZEF1qDdQha6MtUZ0W2BsuPsXw245lc=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBnh8VqpDxj8bGZCifT1jzEDQ3NgG6oWQFXihXN u/Bk0RqJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgZwAKCRAk1otyXVSH0IsEB/ 4xSMDne5g6rapiNEvGOPsW+plT92CVdursADh2x4yQ7vqI5Sm8FZWtedD6XdXhfqOK/6G60pmwzali Rs8AjjUhSu6SHh3+JAkyR2d71J+X+UapT7XFZ8mrSSqhXW8AyD8LIGmBBSWlDylQ7m3yhLZoBr1EK9 07pG9U/74XsUVbY/d5qK71kvRaUNOU9ft+22CtK67KqoM+kw9PyGLctvLqUWJN3TCn/X4n1jVY+CcI yXW6OlRqaSTRhFBAFqdlmU+ur9mFURwDa3U5G1QhgYNW7fKX6AHBX+t+hBmaaLooNfUbzWqoMeS374 3NwdStpFJazie5oRHwFLT+WQGRE7Hg X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB The access control for SME follows the same structure as for the base FP and SVE extensions, with control being via CPACR_ELx.SMEN and CPTR_EL2.TSM mirroring the equivalent FPSIMD and SVE controls in those registers.Add handling for these controls and exceptions mirroring the existing handling for FPSIMD and SVE. When the hardware is in streaming mode guest operations that are invalid in in streaming mode will generate SME exceptions. Since these exceptions may be routed to EL1 with no opportunity for the hypervisor to intercept them we already have code in kvm_arch_vcpu_load_fp() which ensures that we exit streaming mode before running the guest. This ensures that guests do not receive unexpected SME exceptions. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_emulate.h | 4 ++-- arch/arm64/kvm/handle_exit.c | 14 ++++++++++++++ arch/arm64/kvm/hyp/nvhe/hyp-main.c | 6 ++++++ arch/arm64/kvm/hyp/nvhe/switch.c | 11 ++++++----- arch/arm64/kvm/hyp/vhe/switch.c | 21 ++++++++++++++++----- 5 files changed, 44 insertions(+), 12 deletions(-) diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h index c7f3d14c1d69d9b3f7c1c22ad0919c278d2140c1..4c52945779a20604e18d96c78ff920abec9c4dfe 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -624,14 +624,14 @@ static __always_inline void __kvm_reset_cptr_el2(struct kvm *kvm) if (!kvm_has_sve(kvm) || !guest_owns_fp_regs()) val |= CPACR_ELx_ZEN; - if (cpus_have_final_cap(ARM64_SME)) + if (!kvm_has_sme(kvm) || !guest_owns_fp_regs()) val |= CPACR_ELx_SMEN; } else { val = CPTR_NVHE_EL2_RES1; if (kvm_has_sve(kvm) && guest_owns_fp_regs()) val |= CPTR_EL2_TZ; - if (!cpus_have_final_cap(ARM64_SME)) + if (kvm_has_sme(kvm) && guest_owns_fp_regs()) val |= CPTR_EL2_TSM; } diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index d7c2990e7c9ed671833d1011638adeb2c15efd06..48076d0e34038808a36caf2310e11519fd04dd82 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -224,6 +224,19 @@ static int handle_sve(struct kvm_vcpu *vcpu) return 1; } +/* + * Guest access to SME registers should be routed to this handler only + * when the system doesn't support SME. + */ +static int handle_sme(struct kvm_vcpu *vcpu) +{ + if (guest_hyp_sme_traps_enabled(vcpu)) + return kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu)); + + kvm_inject_undefined(vcpu); + return 1; +} + /* * Two possibilities to handle a trapping ptrauth instruction: * @@ -307,6 +320,7 @@ static exit_handle_fn arm_exit_handlers[] = { [ESR_ELx_EC_SVC64] = handle_svc, [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg, [ESR_ELx_EC_SVE] = handle_sve, + [ESR_ELx_EC_SME] = handle_sme, [ESR_ELx_EC_ERET] = kvm_handle_eret, [ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort, [ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort, diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 7468d8516ecaa1370861e51ad4f65adbc01a5d97..481ecd757e0eba021dad6f3b268bb5235f803553 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -487,6 +487,12 @@ void handle_trap(struct kvm_cpu_context *host_ctxt) sve_cond_update_zcr_vq(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]) - 1, SYS_ZCR_EL2); break; + case ESR_ELx_EC_SME: + cpacr_clear_set(0, CPACR_ELx_SMEN); + isb(); + sme_cond_update_smcr_vq(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SME]) - 1, + SYS_SMCR_EL2); + break; case ESR_ELx_EC_IABT_LOW: case ESR_ELx_EC_DABT_LOW: handle_host_mem_abort(host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index 0ebf84a9f9e2715793bcd08c494539be25b6870e..7d29585f1fa03ad6b0063a82dcfba4c5c0b1e4a5 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -46,15 +46,14 @@ static void __activate_cptr_traps(struct kvm_vcpu *vcpu) val |= CPACR_ELx_FPEN; if (vcpu_has_sve(vcpu)) val |= CPACR_ELx_ZEN; + if (vcpu_has_sme(vcpu)) + val |= CPACR_ELx_SMEN; } } else { val |= CPTR_EL2_TTA | CPTR_NVHE_EL2_RES1; - /* - * Always trap SME since it's not supported in KVM. - * TSM is RES1 if SME isn't implemented. - */ - val |= CPTR_EL2_TSM; + if (!vcpu_has_sme(vcpu) || !guest_owns_fp_regs()) + val |= CPTR_EL2_TSM; if (!vcpu_has_sve(vcpu) || !guest_owns_fp_regs()) val |= CPTR_EL2_TZ; @@ -225,6 +224,7 @@ static const exit_handler_fn hyp_exit_handlers[] = { [ESR_ELx_EC_CP15_32] = kvm_hyp_handle_cp15_32, [ESR_ELx_EC_SYS64] = kvm_hyp_handle_sysreg, [ESR_ELx_EC_SVE] = kvm_hyp_handle_fpsimd, + [ESR_ELx_EC_SME] = kvm_hyp_handle_fpsimd, [ESR_ELx_EC_FP_ASIMD] = kvm_hyp_handle_fpsimd, [ESR_ELx_EC_IABT_LOW] = kvm_hyp_handle_iabt_low, [ESR_ELx_EC_DABT_LOW] = kvm_hyp_handle_dabt_low, @@ -236,6 +236,7 @@ static const exit_handler_fn pvm_exit_handlers[] = { [0 ... ESR_ELx_EC_MAX] = NULL, [ESR_ELx_EC_SYS64] = kvm_handle_pvm_sys64, [ESR_ELx_EC_SVE] = kvm_handle_pvm_restricted, + [ESR_ELx_EC_SME] = kvm_handle_pvm_restricted, [ESR_ELx_EC_FP_ASIMD] = kvm_hyp_handle_fpsimd, [ESR_ELx_EC_IABT_LOW] = kvm_hyp_handle_iabt_low, [ESR_ELx_EC_DABT_LOW] = kvm_hyp_handle_dabt_low, diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 80581b1c399595fd64d0ccada498edac322480a6..b2ce97d47b2715d8d7c7f4f365dc9b39f93b0673 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -83,6 +83,8 @@ static void __activate_cptr_traps(struct kvm_vcpu *vcpu) val |= CPACR_ELx_FPEN; if (vcpu_has_sve(vcpu)) val |= CPACR_ELx_ZEN; + if (vcpu_has_sme(vcpu)) + val |= CPACR_ELx_SMEN; } else { __activate_traps_fpsimd32(vcpu); } @@ -126,6 +128,8 @@ static void __activate_cptr_traps(struct kvm_vcpu *vcpu) val &= ~CPACR_ELx_FPEN; if (!(SYS_FIELD_GET(CPACR_ELx, ZEN, cptr) & BIT(0))) val &= ~CPACR_ELx_ZEN; + if (!(SYS_FIELD_GET(CPACR_ELx, SMEN, cptr) & BIT(0))) + val &= ~CPACR_ELx_SMEN; if (kvm_has_feat(vcpu->kvm, ID_AA64MMFR3_EL1, S2POE, IMP)) val |= cptr & CPACR_ELx_E0POE; @@ -380,22 +384,28 @@ static bool kvm_hyp_handle_cpacr_el1(struct kvm_vcpu *vcpu, u64 *exit_code) return true; } -static bool kvm_hyp_handle_zcr_el2(struct kvm_vcpu *vcpu, u64 *exit_code) +static bool kvm_hyp_handle_vec_cr_el2(struct kvm_vcpu *vcpu, u64 *exit_code) { u32 sysreg = esr_sys64_to_sysreg(kvm_vcpu_get_esr(vcpu)); if (!vcpu_has_nv(vcpu)) return false; - if (sysreg != SYS_ZCR_EL2) + switch (sysreg) { + case SYS_ZCR_EL2: + case SYS_SMCR_EL2: + break; + default: return false; + } if (guest_owns_fp_regs()) return false; /* - * ZCR_EL2 traps are handled in the slow path, with the expectation - * that the guest's FP context has already been loaded onto the CPU. + * ZCR_EL2 and SMCR_EL2 traps are handled in the slow path, + * with the expectation that the guest's FP context has + * already been loaded onto the CPU. * * Load the guest's FP context and unconditionally forward to the * slow path for handling (i.e. return false). @@ -412,7 +422,7 @@ static bool kvm_hyp_handle_sysreg_vhe(struct kvm_vcpu *vcpu, u64 *exit_code) if (kvm_hyp_handle_cpacr_el1(vcpu, exit_code)) return true; - if (kvm_hyp_handle_zcr_el2(vcpu, exit_code)) + if (kvm_hyp_handle_vec_cr_el2(vcpu, exit_code)) return true; return kvm_hyp_handle_sysreg(vcpu, exit_code); @@ -422,6 +432,7 @@ static const exit_handler_fn hyp_exit_handlers[] = { [0 ... ESR_ELx_EC_MAX] = NULL, [ESR_ELx_EC_CP15_32] = kvm_hyp_handle_cp15_32, [ESR_ELx_EC_SYS64] = kvm_hyp_handle_sysreg_vhe, + [ESR_ELx_EC_SME] = kvm_hyp_handle_fpsimd, [ESR_ELx_EC_SVE] = kvm_hyp_handle_fpsimd, [ESR_ELx_EC_FP_ASIMD] = kvm_hyp_handle_fpsimd, [ESR_ELx_EC_IABT_LOW] = kvm_hyp_handle_iabt_low, From patchwork Fri Dec 20 16:46:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917128 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 166BF22A80D; Fri, 20 Dec 2024 16:52:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713552; cv=none; b=Qj6lPl8xhPrMXS4FMRvLjKqwDWo3INWrLQt5EeV3rXhkRg7I+FLURvq7sxys2rVtg6xl6AYjIXbnI3JRqEuVFy/2fr1vDRelttkNx1wESzHKtSe958XZJwWxi8PPyW8SAeyMYyfO2M/JrsowS8WAuDhBsbnKpi+3bY3H8N7ckYs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713552; c=relaxed/simple; bh=/CEgAI+Ra5jr/FzEmZ6S+G/3G+R1dOJRvC2cjEojBkk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=iq4e7fYOBqp3IX0n5+aW0cHayvQEwOXt/mJ0dS04QUbuACBKwY6Coizusp01P91PXR1XY0lkFSx2gyUiAtrmRj1c7hgub0YZ6mmWDdutzkjFPHH5P3OVoIETkdkHk8xWseGIq3YYVZHErf3BmDcY8DbkU8h/kw/HAvJGuF8w2SY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TsoLx8Sm; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TsoLx8Sm" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 69515C4CED7; Fri, 20 Dec 2024 16:52:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713551; bh=/CEgAI+Ra5jr/FzEmZ6S+G/3G+R1dOJRvC2cjEojBkk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=TsoLx8SmQO4xtYjm/mfDGy7tyEp/3160l4cVOp6huvslgMBB0aC4lI40reWDI53xy frhM6dwXsdKw1BBiZb+EMQorYkhFoSKqO7XriL2qHl9f3nrrFhuvFYWGORcz7P9iM9 NQaTvG0atYMDkVMtOwY3j6fJHXSDNZOS7bWDyl1M0h5aAK7zGlQILmqbkK130U5NSh MwnmJWUYoCWsaHOZ+b8/dHZk+yZQEAiIrJHX9u8uMWsdEDiUyjgRPwLwxMyweKCb89 hLORJ4q8nsnvAJkfwIyEnTdAh8WkP10qC09m93YKjxQ1AZf6+MoOsYuQU15JQnpv77 nPScbn4GLT6wA== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:50 +0000 Subject: [PATCH RFC v3 25/27] KVM: arm64: Provide interface for configuring and enabling SME for guests Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-25-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=9252; i=broonie@kernel.org; h=from:subject:message-id; bh=/CEgAI+Ra5jr/FzEmZ6S+G/3G+R1dOJRvC2cjEojBkk=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBnI3UPpUtSalk6C7gc4wxNtDJW/JWiZD5SE/+W uOfU9RuJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgZwAKCRAk1otyXVSH0EvvB/ 0WwiHINr9p1O74GhSy8y7OBflr2zca+6b12KdDCtVljhdhMBoONMxeqFvWZ82tODmp0zXjU9TPXjp9 XGQDwJ+4TQ5DbOcW9r5psC/T9uHHCu5hm2Qp5XicbNUlbPncBTRMvsYaoSl7FGC0tJ7sKPFcILIcMm JsI7QzLuG9P4xFdZpyL1AvVlVCyv2HrevZX2pOutzFHfYTIy7++FXexd/RQw4Xs/tSTt9n2+jSz6iB 0IW5WBbdbKXwOYFMZEIp+fGoLZiOLQgr6y8eDDB+of0BykST1U9vmZHG2IPrew9OIGf2pq4+spqSTo 2rsB2HJJeBZdxzHKYGL8aTIi6MJHDt X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Since SME requires configuration of a vector length in order to know the size of both the streaming mode SVE state and ZA array we implement a capability for it and require that it be enabled and finalized before the SME specific state can be accessed, similarly to SVE. Due to the overlap with sizing the SVE state we finalise both SVE and SME with a single finalization, preventing any further changes to the SVE and SME configuration once KVM_ARM_VCPU_VEC (an alias for _VCPU_SVE) has been finalised. This is not a thing of great elegance but it ensures that we never have a state where one of SVE or SME is finalised and the other not, avoiding complexity. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/include/uapi/asm/kvm.h | 1 + arch/arm64/kvm/arm.c | 10 ++++ arch/arm64/kvm/reset.c | 114 ++++++++++++++++++++++++++++++++------ include/uapi/linux/kvm.h | 1 + 5 files changed, 109 insertions(+), 18 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 4fcb2c2603ae2bc51d6993f1f6a3f81f2689717c..838f7ad8ec14b7549ebdfe11e991f664894cc4e7 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -79,6 +79,7 @@ extern unsigned int __ro_after_init kvm_host_max_vl[ARM64_VEC_MAX]; DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); int __init kvm_arm_init_sve(void); +int __init kvm_arm_init_sme(void); u32 __attribute_const__ kvm_target_cpu(void); void kvm_reset_vcpu(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h index 5092f39138cbf17d9e89191de23ab2ee9f3fa77d..4110ff85623b6fe354038b2dffd59a27d0d6d75b 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -108,6 +108,7 @@ struct kvm_regs { #define KVM_ARM_VCPU_PTRAUTH_ADDRESS 5 /* VCPU uses address authentication */ #define KVM_ARM_VCPU_PTRAUTH_GENERIC 6 /* VCPU uses generic authentication */ #define KVM_ARM_VCPU_HAS_EL2 7 /* Support nested virtualization */ +#define KVM_ARM_VCPU_SME 8 /* enable SME for this CPU */ /* * An alias for _SVE since we finalize VL configuration for both SVE and SME diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 8a3d02cf0a7a28e6d2a2fe2b6d63a9dbbe2d4916..c8f44e2c54b613b53087c581176315bc3f4bb2c8 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -377,6 +377,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_ARM_SVE: r = system_supports_sve(); break; + case KVM_CAP_ARM_SME: + r = system_supports_sme(); + break; case KVM_CAP_ARM_PTRAUTH_ADDRESS: case KVM_CAP_ARM_PTRAUTH_GENERIC: r = kvm_has_full_ptr_auth(); @@ -1378,6 +1381,9 @@ static unsigned long system_supported_vcpu_features(void) if (!system_supports_sve()) clear_bit(KVM_ARM_VCPU_SVE, &features); + if (!system_supports_sme()) + clear_bit(KVM_ARM_VCPU_SME, &features); + if (!kvm_has_full_ptr_auth()) { clear_bit(KVM_ARM_VCPU_PTRAUTH_ADDRESS, &features); clear_bit(KVM_ARM_VCPU_PTRAUTH_GENERIC, &features); @@ -2756,6 +2762,10 @@ static __init int kvm_arm_init(void) if (err) return err; + err = kvm_arm_init_sme(); + if (err) + return err; + err = kvm_arm_vmid_alloc_init(); if (err) { kvm_err("Failed to initialize VMID allocator.\n"); diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index 3cb91dc6dc3dc5cc484900dbd9f4cdfedb3e2b4a..78b5aa7976dbbfec35d83f5e373ef87636c2e3e4 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -76,6 +76,34 @@ int __init kvm_arm_init_sve(void) return 0; } +int __init kvm_arm_init_sme(void) +{ + if (system_supports_sme()) { + kvm_max_vl[ARM64_VEC_SME] = sme_max_virtualisable_vl(); + kvm_host_max_vl[ARM64_VEC_SME] = sme_max_vl(); + kvm_nvhe_sym(kvm_host_max_vl[ARM64_VEC_SME]) = kvm_host_max_vl[ARM64_VEC_SME]; + + /* + * The get_sve_reg()/set_sve_reg() ioctl interface will need + * to be extended with multiple register slice support in + * order to support vector lengths greater than + * VL_ARCH_MAX: + */ + if (WARN_ON(kvm_max_vl[ARM64_VEC_SME] > VL_ARCH_MAX)) + kvm_max_vl[ARM64_VEC_SME] = VL_ARCH_MAX; + + /* + * Don't even try to make use of vector lengths that + * aren't available on all CPUs, for now: + */ + if (kvm_max_vl[ARM64_VEC_SME] < sme_max_vl()) + pr_warn("KVM: SME vector length for guests limited to %u bytes\n", + kvm_max_vl[ARM64_VEC_SME]); + } + + return 0; +} + static void kvm_vcpu_enable_sve(struct kvm_vcpu *vcpu) { vcpu->arch.max_vl[ARM64_VEC_SVE] = kvm_max_vl[ARM64_VEC_SVE]; @@ -88,42 +116,84 @@ static void kvm_vcpu_enable_sve(struct kvm_vcpu *vcpu) set_bit(KVM_ARCH_FLAG_GUEST_HAS_SVE, &vcpu->kvm->arch.flags); } +static void kvm_vcpu_enable_sme(struct kvm_vcpu *vcpu) +{ + vcpu->arch.max_vl[ARM64_VEC_SME] = kvm_max_vl[ARM64_VEC_SME]; + + /* + * Userspace can still customize the vector lengths by writing + * KVM_REG_ARM64_SME_VLS. Allocation is deferred until + * kvm_arm_vcpu_finalize(), which freezes the configuration. + */ + set_bit(KVM_ARCH_FLAG_GUEST_HAS_SME, &vcpu->kvm->arch.flags); +} + /* - * Finalize vcpu's maximum SVE vector length, allocating - * vcpu->arch.sve_state as necessary. + * Finalize vcpu's maximum vector lengths, allocating + * vcpu->arch.sve_state and vcpu->arch.sme_state as necessary. */ static int kvm_vcpu_finalize_vec(struct kvm_vcpu *vcpu) { - void *buf; + void *sve_state, *sme_state; unsigned int vl; - size_t reg_sz; int ret; - vl = vcpu->arch.max_vl[ARM64_VEC_SVE]; - /* * Responsibility for these properties is shared between * kvm_arm_init_sve(), kvm_vcpu_enable_sve() and * set_sve_vls(). Double-check here just to be sure: */ - if (WARN_ON(!sve_vl_valid(vl) || vl > sve_max_virtualisable_vl() || - vl > VL_ARCH_MAX)) - return -EIO; + if (vcpu_has_sve(vcpu)) { + vl = vcpu->arch.max_vl[ARM64_VEC_SVE]; + if (WARN_ON(!sve_vl_valid(vl) || + vl > sve_max_virtualisable_vl() || + vl > VL_ARCH_MAX)) + return -EIO; + } - reg_sz = vcpu_sve_state_size(vcpu); - buf = kzalloc(reg_sz, GFP_KERNEL_ACCOUNT); - if (!buf) + /* Similarly for SME */ + if (vcpu_has_sme(vcpu)) { + vl = vcpu->arch.max_vl[ARM64_VEC_SME]; + if (WARN_ON(!sve_vl_valid(vl) || + vl > sme_max_virtualisable_vl() || + vl > VL_ARCH_MAX)) + return -EIO; + } + + sve_state = kzalloc(vcpu_sve_state_size(vcpu), GFP_KERNEL_ACCOUNT); + if (!sve_state) return -ENOMEM; - ret = kvm_share_hyp(buf, buf + reg_sz); - if (ret) { - kfree(buf); - return ret; + ret = kvm_share_hyp(sve_state, sve_state + vcpu_sve_state_size(vcpu)); + if (ret) + goto err_sve_alloc; + + if (vcpu_has_sme(vcpu)) { + sme_state = kzalloc(vcpu_sme_state_size(vcpu), + GFP_KERNEL_ACCOUNT); + if (!sme_state) { + ret = -ENOMEM; + goto err_sve_map; + } + + ret = kvm_share_hyp(sme_state, + sme_state + vcpu_sme_state_size(vcpu)); + if (ret) + goto err_sve_map; + } else { + sme_state = NULL; } - - vcpu->arch.sve_state = buf; + + vcpu->arch.sve_state = sve_state; + vcpu->arch.sme_state = sme_state; vcpu_set_flag(vcpu, VCPU_VEC_FINALIZED); return 0; + +err_sve_map: + kvm_unshare_hyp(sve_state, sve_state + vcpu_sve_state_size(vcpu)); +err_sve_alloc: + kfree(sve_state); + return ret; } int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature) @@ -153,11 +223,15 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu) void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu) { void *sve_state = vcpu->arch.sve_state; + void *sme_state = vcpu->arch.sme_state; kvm_unshare_hyp(vcpu, vcpu + 1); if (sve_state) kvm_unshare_hyp(sve_state, sve_state + vcpu_sve_state_size(vcpu)); kfree(sve_state); + if (sme_state) + kvm_unshare_hyp(sme_state, sme_state + vcpu_sme_state_size(vcpu)); + kfree(sme_state); kfree(vcpu->arch.ccsidr); } @@ -165,6 +239,8 @@ static void kvm_vcpu_reset_vec(struct kvm_vcpu *vcpu) { if (vcpu_has_sve(vcpu)) memset(vcpu->arch.sve_state, 0, vcpu_sve_state_size(vcpu)); + if (vcpu_has_sme(vcpu)) + memset(vcpu->arch.sme_state, 0, vcpu_sme_state_size(vcpu)); } /** @@ -207,6 +283,8 @@ void kvm_reset_vcpu(struct kvm_vcpu *vcpu) if (!kvm_arm_vcpu_vec_finalized(vcpu)) { if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) kvm_vcpu_enable_sve(vcpu); + if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_SME)) + kvm_vcpu_enable_sme(vcpu); } else { kvm_vcpu_reset_vec(vcpu); } diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 502ea63b5d2e7371f3f0d8cb5b9757ff693ee363..e478d9e5e8bff3beeddd39043a62337e0b9d688d 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -933,6 +933,7 @@ struct kvm_enable_cap { #define KVM_CAP_PRE_FAULT_MEMORY 236 #define KVM_CAP_X86_APIC_BUS_CYCLES_NS 237 #define KVM_CAP_X86_GUEST_MODE 238 +#define KVM_CAP_ARM_SME 239 struct kvm_irq_routing_irqchip { __u32 irqchip; From patchwork Fri Dec 20 16:46:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917129 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E9EB22ACE4; Fri, 20 Dec 2024 16:52:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713556; cv=none; b=gOhnoFS87spH+jFQtnK5wI/wB68vj//3pb9cwE+Xa0KVHseiHIisOvpEoO1Fd0EKgz5qXwMcifUivSZTOdxFwlYLkMzRMGVS9M4A6oBIghq0f17RwAWMCVVWsCvkPt0RKwrUyAa+WIMexLmGGE+BcwW/J7fvB/wEiP+eVMrPXMc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713556; c=relaxed/simple; bh=BMpX53mDkGtZeDpM3IZW9C72ZCWeUNb4UNWXTPztxR8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=e0nFU9oFQc+Pzs6RHUGDa9AOLAnVbgtVn9+7qGeXpqr4rCozt+yhoqcb0sPpRL/3rL7OvZsVwswOgwDk+NAUteo5w1EqdnTHbIevzlRwzDtubZ1blnSZZwBKy4npy3hrNwGFT8gmIVI6MIfrftzCi87E5PlQkwSv9q/4QoOOL1A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qcTRhuQb; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qcTRhuQb" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 28D7FC4CECD; Fri, 20 Dec 2024 16:52:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713555; bh=BMpX53mDkGtZeDpM3IZW9C72ZCWeUNb4UNWXTPztxR8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=qcTRhuQbVKn4fohTb31NbiWO0D8Ehb1pTIFbnDGyigSlNolct15uwnKPDW30WN9g/ InDJmbxVv7nCvXjD3DVPbkwyQ9PRtje3M2q26cwrpyEYiFkdAL3E1neCI1t7okf9Tr BSavZ33MJoo9/AZwG3vJ+BLiGJz/THPM/fml2PWgCCONWWZkpR4S9e9TAivK6ho8Dw 6IC+mRZdSgCMsES2beqius/z1vXlB9wqzHOaowUjac44j/h8ExH7g1S1sTRFiwzSZI Y5fDpsttSsuN/t/es/H1me2muvwdzJKU6gWpXdgTR47Gbqe4MarxHsZaEuN1kM4Bqc sbX9SYiWzk5QQ== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:51 +0000 Subject: [PATCH RFC v3 26/27] KVM: arm64: selftests: Add SME system registers to get-reg-list Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-26-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=1700; i=broonie@kernel.org; h=from:subject:message-id; bh=BMpX53mDkGtZeDpM3IZW9C72ZCWeUNb4UNWXTPztxR8=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBoEEPCkL2wDGSGuRsaO/ZJz1XX7TZ4bauI+Jah 6W6aufGJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgaAAKCRAk1otyXVSH0Nf/B/ 9CYyGIA/2XCRRw7P5W9LfDx3YHsUN+7rm9dYArODfp6zBm+X/B7PQtaGbBl6i/rCOqU/GOy5lM5/Kg BE+6jEgB3jtsFmK/vmMuF+xFfYh0FXCRRXK1scYdYQYYSYsuld9+hSuzzGsc0JZpJdUklIZpCAC+ez vXERWUzbLNJUOLAs9Vk4sGSVvzfl6VRGBSIQjA40ki/NIGBY2kLC0Mddl/6yWCCjKF1wVSqeckIcnJ TzgrBwUzP0ie5VKhvFF5ahucgFbB0fgju+RaDyyPyXXb32tQnPDQWj7n34wclFS5xc+RkwsE0NzVXq qLffnpbiwWYywwxNlTq3YrzCMKQUen X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME adds a number of new system registers, update get-reg-list to check for them based on the visibility of SME. Signed-off-by: Mark Brown --- tools/testing/selftests/kvm/aarch64/get-reg-list.c | 32 +++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/kvm/aarch64/get-reg-list.c b/tools/testing/selftests/kvm/aarch64/get-reg-list.c index d43fb3f49050ba3de950d19d56b45beefec9dbeb..3e9c19c4a0d658f349a7d476a90b877882815709 100644 --- a/tools/testing/selftests/kvm/aarch64/get-reg-list.c +++ b/tools/testing/selftests/kvm/aarch64/get-reg-list.c @@ -23,6 +23,18 @@ struct feature_id_reg { }; static struct feature_id_reg feat_id_regs[] = { + { + ARM64_SYS_REG(3, 0, 1, 2, 4), /* SMPRI_EL1 */ + ARM64_SYS_REG(3, 0, 0, 4, 1), /* ID_AA64PFR1_EL1 */ + 24, + 1 + }, + { + ARM64_SYS_REG(3, 0, 1, 2, 6), /* SMCR_EL1 */ + ARM64_SYS_REG(3, 0, 0, 4, 1), /* ID_AA64PFR1_EL1 */ + 24, + 1 + }, { ARM64_SYS_REG(3, 0, 2, 0, 3), /* TCR2_EL1 */ ARM64_SYS_REG(3, 0, 0, 7, 3), /* ID_AA64MMFR3_EL1 */ @@ -52,7 +64,25 @@ static struct feature_id_reg feat_id_regs[] = { ARM64_SYS_REG(3, 0, 0, 7, 3), /* ID_AA64MMFR3_EL1 */ 16, 1 - } + }, + { + ARM64_SYS_REG(3, 1, 0, 0, 6), /* SMIDR_EL1 */ + ARM64_SYS_REG(3, 0, 0, 4, 1), /* ID_AA64PFR1_EL1 */ + 24, + 1 + }, + { + ARM64_SYS_REG(3, 3, 4, 2, 2), /* SVCR */ + ARM64_SYS_REG(3, 0, 0, 4, 1), /* ID_AA64PFR1_EL1 */ + 24, + 1 + }, + { + ARM64_SYS_REG(3, 3, 13, 0, 5), /* TPIDR2_EL0 */ + ARM64_SYS_REG(3, 0, 0, 4, 1), /* ID_AA64PFR1_EL1 */ + 24, + 1 + }, }; bool filter_reg(__u64 reg) From patchwork Fri Dec 20 16:46:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13917130 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9FB4F21A458; Fri, 20 Dec 2024 16:52:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713559; cv=none; b=tGc3e2RNLFvIHdl6itBzI0HTrAHujoxmElEvRH+VOLOD08xJYkTfUc8uI2YJrWxznI4tMpHFwFRcez4GN+1sDr/9wmSOG60tLnaGVRI0wYdomwFWuELNbVM9QDJhwdCaku3FZQbFolePT6/KV6IshvqX1Dlc7KHfBPGX8PWnQmE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734713559; c=relaxed/simple; bh=6kn8YcyG3uuWVTS1B2ujfbi26Z2jCCYjwR1TrhqEy2Q=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Y+KEGdZ0Bbj2tqPf8lVIURmvNMWafx4LfwGLHrq3v+oKTxTcMWWU5MKtQ/QTrzLcymiRdWdIkmDzcDE5JxckAMEBv/B44PXatxTgjAxghBjlpvK2Syj6H/j2a+1bgEebUkhYJyQS6R8ed9a9MQIa6LoVUE+IbC0V9DXMulFiNLI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XqRjWyNk; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XqRjWyNk" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E167AC4CED3; Fri, 20 Dec 2024 16:52:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734713559; bh=6kn8YcyG3uuWVTS1B2ujfbi26Z2jCCYjwR1TrhqEy2Q=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=XqRjWyNk9kIYPWXzlnb9og0pSjDfTTTe45d7kqmxpS9DXBj2LHwD6ojWrjz8MNbwu XwIouMaWV2fBKQ+cu7e/1Pos7V7ZvsOS+pwchWq6wY6ryoWo4bqjn+zzo+6+HW5QZN Br8LuB95Av9Nr6aVlrX765GiIvBh7Av6TuI6y27s0lLm5YjoApuZHxeATGCeaYUxfN UeFtlTva/ah9guJwpvQtv6eV8+jMr38vXUgyxmJ4VIh1nk1XIH2v5ZDFpVrLCRve1C Q9ZC4lkKJQzEw7HocYHRJI5X3zYDQegf0ANXiVp3x3NdNhGreJj44Bl6mzMAFFwSSO IX1T3X2OKfhcg== From: Mark Brown Date: Fri, 20 Dec 2024 16:46:52 +0000 Subject: [PATCH RFC v3 27/27] KVM: arm64: selftests: Add SME to set_id_regs test Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241220-kvm-arm64-sme-v3-27-05b018c1ffeb@kernel.org> References: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> In-Reply-To: <20241220-kvm-arm64-sme-v3-0-05b018c1ffeb@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=3741; i=broonie@kernel.org; h=from:subject:message-id; bh=6kn8YcyG3uuWVTS1B2ujfbi26Z2jCCYjwR1TrhqEy2Q=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnZaBpXKAVT7Xj2TrzanVTCDxoQiBb4OID5YM+cEX6 jYIeZXKJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ2WgaQAKCRAk1otyXVSH0IeFB/ 9sKnmq2qh8uWU0H5kwqGp5XMMe1DPeJUtofdlK9oQiX2eJ6fapOjfe0UTwC2KukPy2FJNYDiHfAhfZ M66Y9CgKhl52tOGqK7TBZoHhJLSA5EoNKUHEsD2u0Ck6av673xKkWaWzP9VtfzkEn17ZUthldmVcB+ QIs/WZbMDD51k777cWEyWmHmboUneznPe9Cok2xzLQRTWk8PV6goqbZm4/RRKtyB98khGEBocmacrx yUJgOvL+aX0tooOaYFhnmA8oXyJNJruPuaaqs06xzOkyVPfMLbU0KKMkgvfRHaqPdjtsJM/2TxjOO/ Jfded9UGUS71PlDH+JqqMIFCtnSqYm X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Add coverage of the SME ID registers to set_id_regs, ID_AA64PFR1_EL1.SME becomes writable and we add ID_AA64SMFR_EL1 and it's subfields. Signed-off-by: Mark Brown --- tools/testing/selftests/kvm/aarch64/set_id_regs.c | 29 +++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kvm/aarch64/set_id_regs.c b/tools/testing/selftests/kvm/aarch64/set_id_regs.c index a79b7f18452d2ec336ae623b8aa5c9cf329b6b4e..ce20ca3bd2733002465ff690e3b99d23d1284de4 100644 --- a/tools/testing/selftests/kvm/aarch64/set_id_regs.c +++ b/tools/testing/selftests/kvm/aarch64/set_id_regs.c @@ -138,6 +138,7 @@ static const struct reg_ftr_bits ftr_id_aa64pfr0_el1[] = { static const struct reg_ftr_bits ftr_id_aa64pfr1_el1[] = { REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64PFR1_EL1, CSV2_frac, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64PFR1_EL1, SME, 0), REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64PFR1_EL1, SSBS, ID_AA64PFR1_EL1_SSBS_NI), REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64PFR1_EL1, BT, 0), REG_FTR_END, @@ -183,6 +184,28 @@ static const struct reg_ftr_bits ftr_id_aa64mmfr2_el1[] = { REG_FTR_END, }; +static const struct reg_ftr_bits ftr_id_aa64smfr0_el1[] = { + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, FA64, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, LUTv2, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, SMEver, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, I16I64, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F64F64, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, I16I32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, B16B16, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F16F16, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F8F16, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F8F32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, I8I32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F16F32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, B16F32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, BI32I32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F32F32, 0) +, REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, SF8FMA, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, SF8DP4, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, SF8DP2, 0), + REG_FTR_END, +}; + static const struct reg_ftr_bits ftr_id_aa64zfr0_el1[] = { REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64ZFR0_EL1, F64MM, 0), REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64ZFR0_EL1, F32MM, 0), @@ -213,6 +236,7 @@ static struct test_feature_reg test_regs[] = { TEST_REG(SYS_ID_AA64MMFR0_EL1, ftr_id_aa64mmfr0_el1), TEST_REG(SYS_ID_AA64MMFR1_EL1, ftr_id_aa64mmfr1_el1), TEST_REG(SYS_ID_AA64MMFR2_EL1, ftr_id_aa64mmfr2_el1), + TEST_REG(SYS_ID_AA64ZFR0_EL1, ftr_id_aa64smfr0_el1), TEST_REG(SYS_ID_AA64ZFR0_EL1, ftr_id_aa64zfr0_el1), }; @@ -229,6 +253,7 @@ static void guest_code(void) GUEST_REG_SYNC(SYS_ID_AA64MMFR0_EL1); GUEST_REG_SYNC(SYS_ID_AA64MMFR1_EL1); GUEST_REG_SYNC(SYS_ID_AA64MMFR2_EL1); + GUEST_REG_SYNC(SYS_ID_AA64SMFR0_EL1); GUEST_REG_SYNC(SYS_ID_AA64ZFR0_EL1); GUEST_REG_SYNC(SYS_CTR_EL0); @@ -676,8 +701,8 @@ int main(void) ARRAY_SIZE(ftr_id_aa64isar2_el1) + ARRAY_SIZE(ftr_id_aa64pfr0_el1) + ARRAY_SIZE(ftr_id_aa64pfr1_el1) + ARRAY_SIZE(ftr_id_aa64mmfr0_el1) + ARRAY_SIZE(ftr_id_aa64mmfr1_el1) + ARRAY_SIZE(ftr_id_aa64mmfr2_el1) + - ARRAY_SIZE(ftr_id_aa64zfr0_el1) - ARRAY_SIZE(test_regs) + 2 + - MPAM_IDREG_TEST; + ARRAY_SIZE(ftr_id_aa64zfr0_el1) + ARRAY_SIZE(ftr_id_aa64smfr0_el1) - ARRAY_SIZE(test_regs) + + 2 + MPAM_IDREG_TEST; ksft_set_plan(test_cnt);