From patchwork Sun Jan 12 16:50:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13936460 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF6801B2190; Sun, 12 Jan 2025 16:50:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736700637; cv=none; b=Yi+DF+7CxuulGInDREfpofcJ/F9ZJRCzmzsgLXkrEamRIrRmm0/K86tGbOrD/XileHggskgiNw7PBIjUrUFzp/NzacoxJqPotv11eG8C+8b976lEW2JimuHBGYT3RW/Ty6K/fKAUNdZilETG1G8CQyOQwJXEiQbf8QV0wHLx9Ss= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736700637; c=relaxed/simple; bh=Xy1v5LbX4zOHiM5PEsmTxnSA3nlKDyqAo9G46Zj857I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=HVewRM+8QkEcSaBvD5XMR3rGb8d4PPn/7Cjsw1EK6QO+Szw7EcKrlpl/+7fpwhD6M7f9QJMxNOBrtdenYcs/LDbnYAUIvhbi9pqewDprmo/1IUP8N5ob/a0NKG4T73D2tH/xJ+WtfCBHJH5Vdqvr78sjIDiKkeYBZnjPn19bK1g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=F7g3rPXb; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="F7g3rPXb" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5AE3BC4CEE0; Sun, 12 Jan 2025 16:50:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736700637; bh=Xy1v5LbX4zOHiM5PEsmTxnSA3nlKDyqAo9G46Zj857I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F7g3rPXbv5V8IkvqNAWh4Ce+TU+l/kHr9RKs8fqsDSotPjNkszahaBjQ+adIu4dWd +v72GVwc/ScMiyW7s9lj3RlkoT5o2E6oqw8gfYMLaRXS8SwgsrHPNLuoVIQovY1OPM RPJPZQBjryrM/DddIFi8PE10J7buWv/VKHbB+Bg4+0VGUNNbUVhYF+WveLzPX8iGRe 9cll7QnF4MoTfN7caQItJ4GbOm33V6d7cBuH5nzxfQox0OT/F+FAmm3S9PFsXK8eAg 3dIKjhq/eVWpc7DOVKEO2/s38yVeNx6o+gciy40VO1F3hw3xoOrVJijugB24ABnH3U eTYX5Z8lzOK/Q== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1tX1AZ-00BNnv-BP; Sun, 12 Jan 2025 16:50:35 +0000 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Joey Gouly , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Joey Gouly Subject: [PATCH 1/2] KVM: arm64: nv: Always evaluate HCR_EL2 using sanitising accessors Date: Sun, 12 Jan 2025 16:50:28 +0000 Message-Id: <20250112165029.1181056-2-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250112165029.1181056-1-maz@kernel.org> References: <20250112165029.1181056-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, joey.gouly@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, Joey.Gouly@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false A lot of the NV code depends on HCR_EL2.{E2H,TGE}, and we assume in places that at least HCR_EL2.E2H is invariant for a given guest. However, we make a point in *not* using the sanitising accessor that would enforce this, and are at the mercy of the guest doing stupid things. Clearly, that's not good. Rework the HCR_EL2 accessors to use __vcpu_sys_reg() instead, guaranteeing that the RESx settings get applied, specially when HCR_EL2.E2H is evaluated. This results in fewer accessors overall. Huge thanks to Joey who spent a long time tracking this bug down. Reported-by: Joey Gouly Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/kvm_emulate.h | 36 ++++++++++++---------------- arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 4 ++-- 2 files changed, 17 insertions(+), 23 deletions(-) diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h index 333c163987a90..fad4f28ed7e81 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -184,29 +184,30 @@ static inline bool vcpu_is_el2(const struct kvm_vcpu *vcpu) return vcpu_is_el2_ctxt(&vcpu->arch.ctxt); } -static inline bool __vcpu_el2_e2h_is_set(const struct kvm_cpu_context *ctxt) +static inline bool vcpu_el2_e2h_is_set(const struct kvm_vcpu *vcpu) { return (!cpus_have_final_cap(ARM64_HAS_HCR_NV1) || - (ctxt_sys_reg(ctxt, HCR_EL2) & HCR_E2H)); + (__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_E2H)); } -static inline bool vcpu_el2_e2h_is_set(const struct kvm_vcpu *vcpu) +static inline bool vcpu_el2_tge_is_set(const struct kvm_vcpu *vcpu) { - return __vcpu_el2_e2h_is_set(&vcpu->arch.ctxt); + return ctxt_sys_reg(&vcpu->arch.ctxt, HCR_EL2) & HCR_TGE; } -static inline bool __vcpu_el2_tge_is_set(const struct kvm_cpu_context *ctxt) +static inline bool is_hyp_ctxt(const struct kvm_vcpu *vcpu) { - return ctxt_sys_reg(ctxt, HCR_EL2) & HCR_TGE; -} + bool e2h, tge; + u64 hcr; -static inline bool vcpu_el2_tge_is_set(const struct kvm_vcpu *vcpu) -{ - return __vcpu_el2_tge_is_set(&vcpu->arch.ctxt); -} + if (!vcpu_has_nv(vcpu)) + return false; + + hcr = __vcpu_sys_reg(vcpu, HCR_EL2); + + e2h = (hcr & HCR_E2H); + tge = (hcr & HCR_TGE); -static inline bool __is_hyp_ctxt(const struct kvm_cpu_context *ctxt) -{ /* * We are in a hypervisor context if the vcpu mode is EL2 or * E2H and TGE bits are set. The latter means we are in the user space @@ -215,14 +216,7 @@ static inline bool __is_hyp_ctxt(const struct kvm_cpu_context *ctxt) * Note that the HCR_EL2.{E2H,TGE}={0,1} isn't really handled in the * rest of the KVM code, and will result in a misbehaving guest. */ - return vcpu_is_el2_ctxt(ctxt) || - (__vcpu_el2_e2h_is_set(ctxt) && __vcpu_el2_tge_is_set(ctxt)) || - __vcpu_el2_tge_is_set(ctxt); -} - -static inline bool is_hyp_ctxt(const struct kvm_vcpu *vcpu) -{ - return vcpu_has_nv(vcpu) && __is_hyp_ctxt(&vcpu->arch.ctxt); + return vcpu_is_el2(vcpu) || (e2h && tge) || tge; } static inline bool vcpu_is_host_el0(const struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c index 5f78a39053a79..90b018e06f2cb 100644 --- a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c +++ b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c @@ -216,7 +216,7 @@ void __vcpu_load_switch_sysregs(struct kvm_vcpu *vcpu) __sysreg32_restore_state(vcpu); __sysreg_restore_user_state(guest_ctxt); - if (unlikely(__is_hyp_ctxt(guest_ctxt))) { + if (unlikely(is_hyp_ctxt(vcpu))) { __sysreg_restore_vel2_state(vcpu); } else { if (vcpu_has_nv(vcpu)) { @@ -260,7 +260,7 @@ void __vcpu_put_switch_sysregs(struct kvm_vcpu *vcpu) host_ctxt = host_data_ptr(host_ctxt); - if (unlikely(__is_hyp_ctxt(guest_ctxt))) + if (unlikely(is_hyp_ctxt(vcpu))) __sysreg_save_vel2_state(vcpu); else __sysreg_save_el1_state(guest_ctxt); From patchwork Sun Jan 12 16:50:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13936462 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4D101BD9D2; Sun, 12 Jan 2025 16:50:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736700637; cv=none; b=LGpIZLWfJSWh00d1XnCHcErT5t+CNYbDfZfhGxbyG3ZO3pEaqTy9zsPWu8D0freedgv+DksYxvPc/I2b4rJi1jZP+XxQvZx831zlTY9/+llvuvwjpLUDe6JKSpN4fTxzU+pGyGEG5OzcpOyNxoTCRnAacD2sQOO5K62x9fnP5N0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736700637; c=relaxed/simple; bh=61n9bky/zprPROXms9U/GF9BXN2TtQW4JY28YmSGENA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=hc2vElTURyE7Elgm7Vj8CYJXQs/O5i2ihhQPZqCiPE5quA8Xn/Mh1mcojMvkkvqK7hNJYQTgwnHHubQXFQ691tBe3bJv4n944+AZt7SuNR09x3TspjKAQo920YEUZ0A95yYcyamE78QCN3lVK0k81p13LFUAJ1osIgke2tPPSuw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KTUCD+1E; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KTUCD+1E" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 70F22C4CEE4; Sun, 12 Jan 2025 16:50:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736700637; bh=61n9bky/zprPROXms9U/GF9BXN2TtQW4JY28YmSGENA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KTUCD+1Ed8vRzPOUlYqhhTYPw9OUDHILIJOYAQ7eEz/lUAH7/VsxPLqbOlOXasPDZ ZyOwC+RzXUdu4+yjb1/yemKtn+qZCoZeoDgSrHo8Un2gL89Sc19F0vSsMas3XBNUJU RAqaDZA8Pnr9gJVUCfCTYC6uUjqxZlAvEG8YK0gUL3neYMYc+3G6iTAylUdTmzhDbA hbPaLGDG8q8HxslvTFxPF6u1Jh+iHjo5t1ETZdBq4Zwf1Ixwxar3G7jcs6XaTl+wx+ F2+qaV5QBD3Tt1oejlb0qtD+1IIasF/oE4LPz3w+26Ftpz1vhkfUIAz3Ec/IrilTzR esT/WwbSwXQAw== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1tX1AZ-00BNnv-J7; Sun, 12 Jan 2025 16:50:35 +0000 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Joey Gouly , Suzuki K Poulose , Oliver Upton , Zenghui Yu Subject: [PATCH 2/2] KVM: arm64: nv: Apply RESx settings to sysreg reset values Date: Sun, 12 Jan 2025 16:50:29 +0000 Message-Id: <20250112165029.1181056-3-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250112165029.1181056-1-maz@kernel.org> References: <20250112165029.1181056-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, joey.gouly@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false While we have sanitisation in place for the guest sysregs, we lack that sanitisation out of reset. So some of the fields could be evaluated and not reflect their RESx status, which sounds like a very bad idea. Apply the RESx masks to the the sysreg file in two situations: - when going via a reset of the sysregs - after having computed the RESx masks Having this separate reset phase from the actual reset handling is a bit grotty, but we need to apply this after the ID registers are final. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/kvm_nested.h | 2 +- arch/arm64/kvm/nested.c | 9 +++++++-- arch/arm64/kvm/sys_regs.c | 5 ++++- 3 files changed, 12 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h index 4792a3f1f4841..e3cd89ed94924 100644 --- a/arch/arm64/include/asm/kvm_nested.h +++ b/arch/arm64/include/asm/kvm_nested.h @@ -187,7 +187,7 @@ static inline bool kvm_supported_tlbi_s1e2_op(struct kvm_vcpu *vpcu, u32 instr) return true; } -int kvm_init_nv_sysregs(struct kvm *kvm); +int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu); #ifdef CONFIG_ARM64_PTR_AUTH bool kvm_auth_eretax(struct kvm_vcpu *vcpu, u64 *elr); diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c index 071198e1ba264..169c548f72d1a 100644 --- a/arch/arm64/kvm/nested.c +++ b/arch/arm64/kvm/nested.c @@ -1568,14 +1568,15 @@ static __always_inline void set_sysreg_masks(struct kvm *kvm, int sr, u64 res0, kvm->arch.sysreg_masks->mask[i].res1 = res1; } -int kvm_init_nv_sysregs(struct kvm *kvm) +int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu) { + struct kvm *kvm = vcpu->kvm; u64 res0, res1; lockdep_assert_held(&kvm->arch.config_lock); if (kvm->arch.sysreg_masks) - return 0; + goto out; kvm->arch.sysreg_masks = kzalloc(sizeof(*(kvm->arch.sysreg_masks)), GFP_KERNEL_ACCOUNT); @@ -1906,6 +1907,10 @@ int kvm_init_nv_sysregs(struct kvm *kvm) /* VNCR_EL2 */ set_sysreg_masks(kvm, VNCR_EL2, VNCR_EL2_RES0, VNCR_EL2_RES1); +out: + for (enum vcpu_sysreg sr = __SANITISED_REG_START__; sr < NR_SYS_REGS; sr++) + (void)__vcpu_sys_reg(vcpu, sr); + return 0; } diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 4a09b6ef94bb9..18bb81291c7ce 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -4576,6 +4576,9 @@ void kvm_reset_sys_regs(struct kvm_vcpu *vcpu) reset_vcpu_ftr_id_reg(vcpu, r); else r->reset(vcpu, r); + + if (r->reg >= __SANITISED_REG_START__ && r->reg < NR_SYS_REGS) + (void)__vcpu_sys_reg(vcpu, r->reg); } set_bit(KVM_ARCH_FLAG_ID_REGS_INITIALIZED, &kvm->arch.flags); @@ -5179,7 +5182,7 @@ int kvm_finalize_sys_regs(struct kvm_vcpu *vcpu) } if (vcpu_has_nv(vcpu)) { - int ret = kvm_init_nv_sysregs(kvm); + int ret = kvm_init_nv_sysregs(vcpu); if (ret) return ret; }