From patchwork Tue Mar 25 18:48:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 14029387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EFE15C36008 for ; Tue, 25 Mar 2025 19:07:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=fjz6UFDOl2Fn4T+Oz+zTHliN4xMdGVj4WuqTH6JGW/Y=; b=DJ8KimZg92Q2Y6SXzdjGW8+Wri +A97rJL8GslxiGCX97LQeCFnAdqqq6Y/wzsDbifptzi+Bxx1DA0GixBw3kKtWPEorU3sJk/bM8Kuu bfTSVPW49rL8ipzyIzpa1LujMhkb245MO/ZohAG6jUcjKd4WwgCmEU8NCLfkRSSTGk9Lumet83ZrI I5tavNKakrgmLWd51ApCvavlI3000nAU8vyWXD0Lx2WwPChXblmsmT2DzRjE9MqR5Rda89UDZqvwD Tini7qi1vYDFyst3rei/E/AYimfsgCw6wWQASX9b3SqVUh/AnJ8eD6e2HtHDJxHN+cRZeLEwz2vjT cFbkIM+A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1tx9cK-00000006onr-0hpA; Tue, 25 Mar 2025 19:07:16 +0000 Received: from sea.source.kernel.org ([172.234.252.31]) by bombadil.infradead.org with esmtps (Exim 4.98.1 #2 (Red Hat Linux)) id 1tx9LU-00000006lXI-0BJ1 for linux-arm-kernel@lists.infradead.org; Tue, 25 Mar 2025 18:49:53 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sea.source.kernel.org (Postfix) with ESMTP id 738AE441D8; Tue, 25 Mar 2025 18:49:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 192D7C4CEE4; Tue, 25 Mar 2025 18:49:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1742928591; bh=ef6nRj6EfRL3ZnUM5oD8oPk3CDmzXlH+Nvd6FtZ+kCU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=srIyxUUuE7aMv5CYI/4M46JaXsO4+YCRzhr5oqZN1Yc3r21yDRhWPa3NUcA8Tkg6/ gz1yNBDfZHDjUjK0hCf3zDBSaRVMDJRzbG4UYRt6aptqqwEMB5EYg4p+C4Xhy9gk2T yjEVk1kH2UJNSRX9vyu7Ac/FQnMJlq1lCcuQXfhVrieJqV2q9hq2zfBPLzwvbJNSCP KmK3Vtm/OF4I+w1Pk5Cug65C431zhVThNoh1pVHqAUUECjPw7XD2zvj1+/BVPGzIJk ZJLrMQxGSHfdQ3XTAqC1npyerp9xAB+xofJ1JnM/GdOQ94BXd4HNkLcjJg8DuGn9Fn zc2VVr4rVwNsQ== From: Mark Brown Date: Tue, 25 Mar 2025 18:48:23 +0000 Subject: [PATCH 6.1 09/12] KVM: arm64: Refactor exit handlers MIME-Version: 1.0 Message-Id: <20250325-stable-sve-6-1-v1-9-83259d427d84@kernel.org> References: <20250325-stable-sve-6-1-v1-0-83259d427d84@kernel.org> In-Reply-To: <20250325-stable-sve-6-1-v1-0-83259d427d84@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , James Morse , Suzuki K Poulose , Oliver Upton , Oleg Nesterov Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvmarm@lists.cs.columbia.edu, Mark Brown , Mark Rutland , Fuad Tabba X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=7652; i=broonie@kernel.org; h=from:subject:message-id; bh=6l0OhEE52eAZ925BRo7VF06hXL0ZtmAxhOziFxHG1dg=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBn4vqtJHzl39i9O9Es0G8A45hqS7fdPmy3pPJvjpAK Qpmmv1OJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ+L6rQAKCRAk1otyXVSH0GXKB/ 960uIvGaJ1yxfGPcRrKdoyPiZdBkErnwVSffqQxWhZ74Kj1c4kuR771aiDMM+QLbZlXBPBNgvH+hwm 1JXg962kTzyFBG+EReVZVEndwjU2DYQqgCcQcDElnid4F8arg7LuoMeEjroSEu9kMOPdCaymZV9Fnl HXv+TLagUfJhhVivFDJ4EfFNf83V9ejZeCokEgDeIoKJSNXb31pSF8nWHpnFrNttV7x+5p7jMxZTgr /96ZDZIP5N66ntunq3M2LytdF9+w39dUeXnDotDpQWho1fxHu60a0pCw7ZIB70UciTqL/XxdbpKkud Szw4eGhAbxoz/vDqI6LtiWSpPXf092 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250325_114952_142798_16B201ED X-CRM114-Status: GOOD ( 23.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Mark Rutland [ Upstream commit 9b66195063c5a145843547b1d692bd189be85287 ] The hyp exit handling logic is largely shared between VHE and nVHE/hVHE, with common logic in arch/arm64/kvm/hyp/include/hyp/switch.h. The code in the header depends on function definitions provided by arch/arm64/kvm/hyp/vhe/switch.c and arch/arm64/kvm/hyp/nvhe/switch.c when they include the header. This is an unusual header dependency, and prevents the use of arch/arm64/kvm/hyp/include/hyp/switch.h in other files as this would result in compiler warnings regarding missing definitions, e.g. | In file included from arch/arm64/kvm/hyp/nvhe/hyp-main.c:8: | ./arch/arm64/kvm/hyp/include/hyp/switch.h:733:31: warning: 'kvm_get_exit_handler_array' used but never defined | 733 | static const exit_handler_fn *kvm_get_exit_handler_array(struct kvm_vcpu *vcpu); | | ^~~~~~~~~~~~~~~~~~~~~~~~~~ | ./arch/arm64/kvm/hyp/include/hyp/switch.h:735:13: warning: 'early_exit_filter' used but never defined | 735 | static void early_exit_filter(struct kvm_vcpu *vcpu, u64 *exit_code); | | ^~~~~~~~~~~~~~~~~ Refactor the logic such that the header doesn't depend on anything from the C files. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Reviewed-by: Mark Brown Tested-by: Mark Brown Acked-by: Will Deacon Cc: Catalin Marinas Cc: Fuad Tabba Cc: Marc Zyngier Cc: Oliver Upton Reviewed-by: Oliver Upton Link: https://lore.kernel.org/r/20250210195226.1215254-7-mark.rutland@arm.com Signed-off-by: Marc Zyngier Signed-off-by: Mark Brown --- arch/arm64/kvm/hyp/include/hyp/switch.h | 30 ++++++------------------------ arch/arm64/kvm/hyp/nvhe/switch.c | 27 +++++++++++++++------------ arch/arm64/kvm/hyp/vhe/switch.c | 8 +++----- 3 files changed, 24 insertions(+), 41 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 50e6f3fcc27cd35822246144c1e5f7761e316746..379adbb9d8f2002599c2c135f936efaacab760a3 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -398,23 +398,16 @@ static bool kvm_hyp_handle_dabt_low(struct kvm_vcpu *vcpu, u64 *exit_code) typedef bool (*exit_handler_fn)(struct kvm_vcpu *, u64 *); -static const exit_handler_fn *kvm_get_exit_handler_array(struct kvm_vcpu *vcpu); - -static void early_exit_filter(struct kvm_vcpu *vcpu, u64 *exit_code); - /* * Allow the hypervisor to handle the exit with an exit handler if it has one. * * Returns true if the hypervisor handled the exit, and control should go back * to the guest, or false if it hasn't. */ -static inline bool kvm_hyp_handle_exit(struct kvm_vcpu *vcpu, u64 *exit_code) +static inline bool kvm_hyp_handle_exit(struct kvm_vcpu *vcpu, u64 *exit_code, + const exit_handler_fn *handlers) { - const exit_handler_fn *handlers = kvm_get_exit_handler_array(vcpu); - exit_handler_fn fn; - - fn = handlers[kvm_vcpu_trap_get_class(vcpu)]; - + exit_handler_fn fn = handlers[kvm_vcpu_trap_get_class(vcpu)]; if (fn) return fn(vcpu, exit_code); @@ -444,20 +437,9 @@ static inline void synchronize_vcpu_pstate(struct kvm_vcpu *vcpu, u64 *exit_code * the guest, false when we should restore the host state and return to the * main run loop. */ -static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) +static inline bool __fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code, + const exit_handler_fn *handlers) { - /* - * Save PSTATE early so that we can evaluate the vcpu mode - * early on. - */ - synchronize_vcpu_pstate(vcpu, exit_code); - - /* - * Check whether we want to repaint the state one way or - * another. - */ - early_exit_filter(vcpu, exit_code); - if (ARM_EXCEPTION_CODE(*exit_code) != ARM_EXCEPTION_IRQ) vcpu->arch.fault.esr_el2 = read_sysreg_el2(SYS_ESR); @@ -487,7 +469,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) goto exit; /* Check if there's an exit handler and allow it to handle the exit. */ - if (kvm_hyp_handle_exit(vcpu, exit_code)) + if (kvm_hyp_handle_exit(vcpu, exit_code, handlers)) goto guest; exit: /* Return to the host kernel and handle the exit */ diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index 895fb32000762fea1fce3e99f1c7838c2f148880..844c466f1b1f26620582972038ac2bd81f876182 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -209,21 +209,22 @@ static const exit_handler_fn *kvm_get_exit_handler_array(struct kvm_vcpu *vcpu) return hyp_exit_handlers; } -/* - * Some guests (e.g., protected VMs) are not be allowed to run in AArch32. - * The ARMv8 architecture does not give the hypervisor a mechanism to prevent a - * guest from dropping to AArch32 EL0 if implemented by the CPU. If the - * hypervisor spots a guest in such a state ensure it is handled, and don't - * trust the host to spot or fix it. The check below is based on the one in - * kvm_arch_vcpu_ioctl_run(). - * - * Returns false if the guest ran in AArch32 when it shouldn't have, and - * thus should exit to the host, or true if a the guest run loop can continue. - */ -static void early_exit_filter(struct kvm_vcpu *vcpu, u64 *exit_code) +static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) { + const exit_handler_fn *handlers = kvm_get_exit_handler_array(vcpu); struct kvm *kvm = kern_hyp_va(vcpu->kvm); + synchronize_vcpu_pstate(vcpu, exit_code); + + /* + * Some guests (e.g., protected VMs) are not be allowed to run in + * AArch32. The ARMv8 architecture does not give the hypervisor a + * mechanism to prevent a guest from dropping to AArch32 EL0 if + * implemented by the CPU. If the hypervisor spots a guest in such a + * state ensure it is handled, and don't trust the host to spot or fix + * it. The check below is based on the one in + * kvm_arch_vcpu_ioctl_run(). + */ if (kvm_vm_is_protected(kvm) && vcpu_mode_is_32bit(vcpu)) { /* * As we have caught the guest red-handed, decide that it isn't @@ -236,6 +237,8 @@ static void early_exit_filter(struct kvm_vcpu *vcpu, u64 *exit_code) *exit_code &= BIT(ARM_EXIT_WITH_SERROR_BIT); *exit_code |= ARM_EXCEPTION_IL; } + + return __fixup_guest_exit(vcpu, exit_code, handlers); } /* Switch to the guest for legacy non-VHE systems */ diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 45ac4a59cc2ce724f9c34adffa5f7970c5964526..f24569ac26c22978fc991221a9f5435e64cf4db2 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -114,13 +114,11 @@ static const exit_handler_fn hyp_exit_handlers[] = { [ESR_ELx_EC_PAC] = kvm_hyp_handle_ptrauth, }; -static const exit_handler_fn *kvm_get_exit_handler_array(struct kvm_vcpu *vcpu) +static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) { - return hyp_exit_handlers; -} + synchronize_vcpu_pstate(vcpu, exit_code); -static void early_exit_filter(struct kvm_vcpu *vcpu, u64 *exit_code) -{ + return __fixup_guest_exit(vcpu, exit_code, hyp_exit_handlers); } /* Switch to the guest for VHE systems running in EL2 */