From patchwork Wed May 29 12:12:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678750 Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EEA2438F96 for ; Wed, 29 May 2024 12:14:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984874; cv=none; b=tbC4oB+gCaFPb0zMFuUQ6NvUWwGGDU5h+Zv8fa3Vkmztjfvfh1WokPza02iQ2yMwER6kTgA9lXD8ypcyeJwPWtJTag9tyWzM25vVA+JntR8yLd+fNxac4r+oATE1ySay1kP3zncasQcThz3B2yFq/bV5nPMA5wQLkZjRg6Ecs+E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984874; c=relaxed/simple; bh=+Do1Lj6aoKl60X1qo3qBYBuQpEpA40aboieYSjIPVEo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=VR1R4c+JoAUFz49LHVrEO/JEZrDnahsqkQH3u49wsJJfEuA1Kq/qt0/d9t/gVcR6rTyrqKkBYESqZ95eYLf1rjPaw9ZtNtG5NdKwNjRoyXbz17rM8RrnjnpBKgJfOp0Kf3DUxXx9ozkp6HuEoZTGTPMdCeIOTnIxtudXmUeL9LM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=iwo5Lu88; arc=none smtp.client-ip=209.85.218.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="iwo5Lu88" Received: by mail-ej1-f74.google.com with SMTP id a640c23a62f3a-a6349bc2650so122962466b.1 for ; Wed, 29 May 2024 05:14:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984871; x=1717589671; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=LiBCWky943GnQdDxaq98aS8ZLEQ71giQKbDHRUj5HqQ=; b=iwo5Lu88ytOI7F1MsorvKrNo8OXi9DJbjsQ0f0a54C982oYtHy6QZWMp7K72Agf++l SFRdilLXpoC71pHkQWifVRlGx5G0NRU6VVcKPeFQELXpJYz3zfrJQ1RUS1EHpp/mQfw/ fJW3g/xE/p9Z10GelN0LHnDlPINL5ZbQl+LJAP1pNn6JlqIHIAyS4gRNXVl6kaLsAsvH xIvjxV4g6kLX/oMUvugAghxSEkxIrtqJiqyFHt1YlrfORNuzr5LWOLtd1rNxO346mzuJ yU+b0ZRyZ1gK0IIwcbMSgupHubx0lRLUbMYPZWvbE8tYi3yaVV4xJLqQxWOw7OVw817M cV9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984871; x=1717589671; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=LiBCWky943GnQdDxaq98aS8ZLEQ71giQKbDHRUj5HqQ=; b=weumFINuCxDwn7IB2dFRbt+M6HQ0UO1lbfYAHzdwrHvijlq9QPWbaSemBb7jrUS59y +TsoLxNgk7R0CDzi8RhNGGDKb4hrBB9qzkgn+Fp9w0APy9Q0iGfIUrM9A+SF2yWS+jRs M63+Fj6BAsb1mb+Ky0k8CAQpqdbLcG9EokSsjFc3/mQZiG24HSnrTr6Vlm6xIYQvvPQP Bw7oHGbXIXQkITcQWIUlZqXCejpbYAav6GjN5QLvoGKZmnuX1BsfONm07YQ7iMAGB0NM TDF3XZSgNA+691GIIg8d9NhNZdki7HSpUsBDyaYSHHkfJ7Kil9yNsd/YqcbaNVWfrT6n ELig== X-Forwarded-Encrypted: i=1; AJvYcCXdG7ZJ8iuXTTYOfb0b+futnCGCrDrDocO/Z95NM7d/sxbzNjnPtTDfLJ74GTkhFTV2kpWwFKk3odTGQggct9JzQ1Ok X-Gm-Message-State: AOJu0YyfYvE5U/t/EpBqCMQClye21S3u8b4V3YYSQsg71S4/wUtRcKNe VYYn98ORzXtT2GsVaygH88QUmF+k0T4H9K0t1aHkvGxb1H4p2YO3oOMQ7MTeNyGMvHMnRF1Kxw= = X-Google-Smtp-Source: AGHT+IFP6n0siCbzTCYAQHYIrAphWxkqP2Ntf7Z84XOpYY4Wgk7fdgvFlpZ4wx2gsGuIGmJgr+coEywpTQ== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a17:906:3185:b0:a5a:1c6:b892 with SMTP id a640c23a62f3a-a642d6aba53mr229666b.6.1716984871216; Wed, 29 May 2024 05:14:31 -0700 (PDT) Date: Wed, 29 May 2024 13:12:07 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-2-ptosi@google.com> Subject: [PATCH v4 01/13] KVM: arm64: Fix clobbered ELR in sync abort/SError From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort When the hypervisor receives a SError or synchronous exception (EL2h) while running with the __kvm_hyp_vector and if ELR_EL2 doesn't point to an extable entry, it panics indirectly by overwriting ELR with the address of a panic handler in order for the asm routine it returns to to ERET into the handler. However, this clobbers ELR_EL2 for the handler itself. As a result, hyp_panic(), when retrieving what it believes to be the PC where the exception happened, actually ends up reading the address of the panic handler that called it! This results in an erroneous and confusing panic message where the source of any synchronous exception (e.g. BUG() or kCFI) appears to be __guest_exit_panic, making it hard to locate the actual BRK instruction. Therefore, store the original ELR_EL2 in the per-CPU kvm_hyp_ctxt and point the sysreg to a routine that first restores it to its previous value before running __guest_exit_panic. Fixes: 7db21530479f ("KVM: arm64: Restore hyp when panicking in guest context") Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kvm/hyp/entry.S | 8 ++++++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 5 +++-- 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 81496083c041..27de1dddb0ab 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -128,6 +128,7 @@ int main(void) DEFINE(VCPU_FAULT_DISR, offsetof(struct kvm_vcpu, arch.fault.disr_el1)); DEFINE(VCPU_HCR_EL2, offsetof(struct kvm_vcpu, arch.hcr_el2)); DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_cpu_context, regs)); + DEFINE(CPU_ELR_EL2, offsetof(struct kvm_cpu_context, sys_regs[ELR_EL2])); DEFINE(CPU_RGSR_EL1, offsetof(struct kvm_cpu_context, sys_regs[RGSR_EL1])); DEFINE(CPU_GCR_EL1, offsetof(struct kvm_cpu_context, sys_regs[GCR_EL1])); DEFINE(CPU_APIAKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APIAKEYLO_EL1])); diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index f3aa7738b477..4433a234aa9b 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,6 +83,14 @@ alternative_else_nop_endif eret sb +SYM_INNER_LABEL(__guest_exit_restore_elr_and_panic, SYM_L_GLOBAL) + // x2-x29,lr: vcpu regs + // vcpu x0-x1 on the stack + + adr_this_cpu x0, kvm_hyp_ctxt, x1 + ldr x0, [x0, #CPU_ELR_EL2] + msr elr_el2, x0 + SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) // x2-x29,lr: vcpu regs // vcpu x0-x1 on the stack diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index a92566f36022..ed9a63f1f7bf 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -689,7 +689,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __kvm_unexpected_el2_exception(void) { - extern char __guest_exit_panic[]; + extern char __guest_exit_restore_elr_and_panic[]; unsigned long addr, fixup; struct kvm_exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); @@ -711,7 +711,8 @@ static inline void __kvm_unexpected_el2_exception(void) } /* Trigger a panic after restoring the hyp context. */ - write_sysreg(__guest_exit_panic, elr_el2); + this_cpu_ptr(&kvm_hyp_ctxt)->sys_regs[ELR_EL2] = elr_el2; + write_sysreg(__guest_exit_restore_elr_and_panic, elr_el2); } #endif /* __ARM64_KVM_HYP_SWITCH_H__ */ From patchwork Wed May 29 12:12:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678751 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1BFD6180A96 for ; Wed, 29 May 2024 12:14:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984876; cv=none; b=mMqvDXhj95c0m942eJUCGCDABMRsORdFeMk3nJg0E7iWCmAmRA6a4Q50vWnXmD3HtkqBqlJ4xxewWbX9D5Kt+F/vMF8QF+9b//FSmiV2tDbaQcuS1ly2OXMK78UCYmNvgkWNyKCWKdNzVFvJeBevd9epkNwqssMy1OAnyyp6t3U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984876; c=relaxed/simple; bh=BRujChrM4VFfYrpRS71aJErzF3wII2UsmVJlHzt0Ypw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ed+88tbz7cPDgUIQpXirFe2CMhI4N7K/7L73ZSbP9i5WPkGxqQrsHTGxyN7hFUDDuq3ioJwQ5T8h8qL35INxG2PL0cSGzRqTZrEC+KD3w+extX0BmOOszf+lkj/KZ/WRlhwgRjMoHsg1k4mKdqpa6u/rvRzwgdhkt1DL9ZW6SeM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=cqzHle9b; arc=none smtp.client-ip=209.85.208.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="cqzHle9b" Received: by mail-ed1-f73.google.com with SMTP id 4fb4d7f45d1cf-579cced186eso891366a12.1 for ; Wed, 29 May 2024 05:14:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984873; x=1717589673; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=tLmL1So5x4FnLH0kyn2ty29vEdym4EM/KIgVNMwbuEQ=; b=cqzHle9bxHtRS5QZhxySFLYVqHaztfaxfqnVL9R01yujk+0qDSGKdJxZmzpJ5Zp8WA x4SP8mZxR4a1YSz9qm8vaSU/jcRGgSpggYIKxIVBRCWJ6w3hXMkDkEgRC8r5f9HGFKT8 z2PNFvQWm0muCZ9DQyALXAMh9PIPuiVioPzAm6XCTYcYyJslv6vk1r1Zx5U0MPRRDPoT fcZu5nESXPkGWdQ5jolYpXoGIYQRMHWQMeASdVXBAq3xvhf3DterDrpOQv6eSkRDl2qT VCcX+h09bDYZch5672EROt9gdWanesoSnWwwY5+grLxEcG3eRE7jfQ2MMAg9P21AQlkn d6bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984873; x=1717589673; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=tLmL1So5x4FnLH0kyn2ty29vEdym4EM/KIgVNMwbuEQ=; b=T29IF4U38lzP53ymR6ZttHiwSYQeutH/QNuBO5LTSY5/5a0VbSqrWCIaWFtzxGrN7V os+KcebzGtJLq9SU4Jy3gaGAYsj4tpmOVYjxREQT6S/CajT+r/t1fP6kydDGq66VYBMZ hbozKiQurWY2KjiMKwdFEaIXiCzDeIJMdVKxpKNqdt7YNGkSmtfU8I6DnSmsbDJiwX/M kkfL43sw560H+niN8EnyhCAMhA8CbCxT7o4SVYtwepDAXAQxrD0F1msAHQ7IOKnRGcPh 2TrX41x7Z7OKTbcDSnkDZ7+hIJ4J+33PsDPBpB+tqghz3+ZwT85banTo0YstWY94LtaE c5rw== X-Forwarded-Encrypted: i=1; AJvYcCXDzce/H1l+W9TK2RAxwSItoRAEujpXBaIxsO6Lg45stwYaqxU/a0Kq3NcmkrDTLhX8fY4wm9sBE8CEq2Mdr951M26g X-Gm-Message-State: AOJu0YzzqvJXQNjT8Yvcc4cQehL+7NU3nbcPGD1DRV+utPz79jicmtVq 2fr+NQ5or3tKiuxSR/RuIs0mrEyl5MYPBhPN3jGZ8c7WjMIvi3EgJUowjYqR9165Xkf01s8+/Q= = X-Google-Smtp-Source: AGHT+IFoWb2krclv9TkDbcARu8wLLSDlMlxLJgD2KUqfjrU7i3St1EL1/OV74yeBjy7fXa7WBSwaGQ6E2A== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6402:3223:b0:572:6e36:f0f7 with SMTP id 4fb4d7f45d1cf-578519bbf75mr18449a12.6.1716984873520; Wed, 29 May 2024 05:14:33 -0700 (PDT) Date: Wed, 29 May 2024 13:12:08 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-3-ptosi@google.com> Subject: [PATCH v4 02/13] KVM: arm64: Fix __pkvm_init_switch_pgd call ABI From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Fix the mismatch between the (incorrect) C signature, C call site, and asm implementation by aligning all three on an API passing the parameters (pgd and SP) separately, instead of as a bundled struct. Remove the now unnecessary memory accesses while the MMU is off from the asm, which simplifies the C caller (as it does not need to convert a VA struct pointer to PA) and makes the code slightly more robust by offsetting the struct fields from C and properly expressing the call to the C compiler (e.g. type checker and kCFI). Fixes: f320bc742bc2 ("KVM: arm64: Prepare the creation of s1 mappings at EL2") Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/kvm_hyp.h | 3 +-- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 17 +++++++++-------- arch/arm64/kvm/hyp/nvhe/setup.c | 4 ++-- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 3e80464f8953..58b5a2b14d88 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -123,8 +123,7 @@ void __noreturn __hyp_do_panic(struct kvm_cpu_context *host_ctxt, u64 spsr, #endif #ifdef __KVM_NVHE_HYPERVISOR__ -void __pkvm_init_switch_pgd(phys_addr_t phys, unsigned long size, - phys_addr_t pgd, void *sp, void *cont_fn); +void __pkvm_init_switch_pgd(phys_addr_t pgd, void *sp, void (*fn)(void)); int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, unsigned long *per_cpu_base, u32 hyp_va_bits); void __noreturn __host_enter(struct kvm_cpu_context *host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 2994878d68ea..d859c4de06b6 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -265,33 +265,34 @@ alternative_else_nop_endif SYM_CODE_END(__kvm_handle_stub_hvc) +/* + * void __pkvm_init_switch_pgd(phys_addr_t pgd, void *sp, void (*fn)(void)); + */ SYM_FUNC_START(__pkvm_init_switch_pgd) /* Turn the MMU off */ pre_disable_mmu_workaround - mrs x2, sctlr_el2 - bic x3, x2, #SCTLR_ELx_M + mrs x9, sctlr_el2 + bic x3, x9, #SCTLR_ELx_M msr sctlr_el2, x3 isb tlbi alle2 /* Install the new pgtables */ - ldr x3, [x0, #NVHE_INIT_PGD_PA] - phys_to_ttbr x4, x3 + phys_to_ttbr x4, x0 alternative_if ARM64_HAS_CNP orr x4, x4, #TTBR_CNP_BIT alternative_else_nop_endif msr ttbr0_el2, x4 /* Set the new stack pointer */ - ldr x0, [x0, #NVHE_INIT_STACK_HYP_VA] - mov sp, x0 + mov sp, x1 /* And turn the MMU back on! */ dsb nsh isb - set_sctlr_el2 x2 - ret x1 + set_sctlr_el2 x9 + ret x2 SYM_FUNC_END(__pkvm_init_switch_pgd) .popsection diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 859f22f754d3..1cbd2c78f7a1 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -316,7 +316,7 @@ int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, { struct kvm_nvhe_init_params *params; void *virt = hyp_phys_to_virt(phys); - void (*fn)(phys_addr_t params_pa, void *finalize_fn_va); + typeof(__pkvm_init_switch_pgd) *fn; int ret; BUG_ON(kvm_check_pvm_sysreg_table()); @@ -340,7 +340,7 @@ int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, /* Jump in the idmap page to switch to the new page-tables */ params = this_cpu_ptr(&kvm_init_params); fn = (typeof(fn))__hyp_pa(__pkvm_init_switch_pgd); - fn(__hyp_pa(params), __pkvm_init_finalise); + fn(params->pgd_pa, (void *)params->stack_hyp_va, __pkvm_init_finalise); unreachable(); } From patchwork Wed May 29 12:12:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678752 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B618C38F96 for ; Wed, 29 May 2024 12:14:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984878; cv=none; b=OAtJUj+praEx1fHH9KN4DUP/ZSEPjmgj1wQMxqIeCznbteKRpTlUNhc4H8pG6OoIrSFynwL9HNo4I54p7DMkhCMIM69J/UA7ZSeaNhLTU6hQG3H/DtylV/PxFQtYR7rK6aVVshyqIN6h2EJ8p+EXw3AlXLAKQokQZyCLKbgKIwg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984878; c=relaxed/simple; bh=n/Rt+Z7/h3tKGyTZQjuwoohj1Dyd6vhk6ULQqnGfhKE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=VFgu0Wt077YEG3pIEqHb+zWiIdPPyy+nDVDfWW8Ajx2PzHCDY3uLJx5D8ziB00EzvZyYxM+c5xuxErf+gxFCV0ouh7NtQYRZ7Kkvoo8OLhJyjKx8gadjpVDw8Pbh38diAcIB/aeHFY7IA6jaW6PFjjmSDb5lhALjg0otjB+byWA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=1dtoPQOG; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="1dtoPQOG" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-62a083e617aso36991057b3.2 for ; Wed, 29 May 2024 05:14:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984876; x=1717589676; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=j4Rf2lfhPQdW0ulVQ+ZkbOO+HrzrcUNTk97aD9mbPlw=; b=1dtoPQOGx0us4ztHO19Zc/WhMDdk2YIzWcWJNNjTU6a+GOYfkU6rngTd5TvbFC4joe PvM8pJl8wIn2Y/LPxSNOkGzcFktirJs+7k/i/mzAy/5x+MR51WPw0I3Dq897NcwxBCcP EDrifxhqKtqFGsidk/aLYazYoAxENQQF1tRFTuIjeAs7HThOtYwC/25vtm1vms+V6311 zn4rIL0WL2Jn0DIXuNp67oHu3NY74I5cGgvSxtkOR1+WKklrsYS8sOI5wasO6+G2ahBF HBK0gUgAhMcyRwXENKhUWCvMqookGlvGy9T5nwailwe+36AiekQkBAFD2hfG7nyOFlo6 e9gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984876; x=1717589676; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=j4Rf2lfhPQdW0ulVQ+ZkbOO+HrzrcUNTk97aD9mbPlw=; b=XxMflCtv4ydK8FICa8SlT3UtgXVyTQ75op5zQrWdOCTCCoSa/r+y8Age6AWBMAiwbO rxWFP0oS95D62Tigy3TjO7FGiM3F2K7sl+FRSlDwEqkz+5FIK18W0oVz9mzWXUEOVbxf rsLMXeb+/0dKTyvakWDu9mKg+Ws0csb6zLhSsNNs2JGcTU6cC7N+msFF/lGtQzVF12BB lE2dVIC6hfnofpQIcWJRH0q4VD5nD6aMxNOqAlPWXJlOpc9oTX5biQKARB4npVpdJHWu nhuLCkrYnvG/2QdJnWoq+xXIfNFGY6IMhRtdi7B9mryxUlT0tag54kMJLG2JkvYuASix pMRA== X-Forwarded-Encrypted: i=1; AJvYcCUOctBksDY6ucApl8wG8JQ6dHbMiWZEmDfafW0wLF2Fw+mloHOWMAGwQSpPJZKCWZj9d4i8/3wi9resnSjp0PjKJd03 X-Gm-Message-State: AOJu0Yyiphxnq3rDPc1o7GmrdGIDA0khYVlD4sIX+XhLpfo8lVvPm0oh giz9g7cRgEeImhQgqTtEWoiJUyEsIZNYyCg3HYfvrVyWgmPK5ce144v0HuSbRh3O+oQVJd31Ug= = X-Google-Smtp-Source: AGHT+IEkdZ2CPDm5cYVOhM4+XL1fH8AlLKI24Mx4NO3l+ssr5tA1I25II+Qs22X8PnruQU2dGz49CMMUxw== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6902:f0f:b0:de5:4ed6:d3f3 with SMTP id 3f1490d57ef6-df7721df7cbmr3593228276.6.1716984875835; Wed, 29 May 2024 05:14:35 -0700 (PDT) Date: Wed, 29 May 2024 13:12:09 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-4-ptosi@google.com> Subject: [PATCH v4 03/13] KVM: arm64: nVHE: Simplify __guest_exit_panic path From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort In invalid_host_el2_vect (i.e. EL2{t,h} handlers in nVHE guest context), remove the duplicate vCPU context check that __guest_exit_panic also performs, allowing an unconditional branch to it. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/hyp/nvhe/host.S | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 135cfb294ee5..71fb311b4c0e 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -197,18 +197,13 @@ SYM_FUNC_END(__host_hvc) sub x0, sp, x0 // x0'' = sp' - x0' = (sp + x0) - sp = x0 sub sp, sp, x0 // sp'' = sp' - x0 = (sp + x0) - x0 = sp - /* If a guest is loaded, panic out of it. */ - stp x0, x1, [sp, #-16]! - get_loaded_vcpu x0, x1 - cbnz x0, __guest_exit_panic - add sp, sp, #16 - /* * The panic may not be clean if the exception is taken before the host * context has been saved by __host_exit or after the hyp context has * been partially clobbered by __host_enter. */ - b hyp_panic + stp x0, x1, [sp, #-16]! + b __guest_exit_panic .L__hyp_sp_overflow\@: /* Switch to the overflow stack */ From patchwork Wed May 29 12:12:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678753 Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0295C180A8C for ; Wed, 29 May 2024 12:14:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984881; cv=none; b=SQnNE24ig9fqgVs9FPLTzt91bFZ5SdYwYH/wF82XijndxPsyxTcgal8Ob2DOxAQWz99iGBJrJ9W+1/gp0iRR3td8p8vMvNeWa8fLpvY/SJ3VnBUfLw/tDe1WyaTiRynBhHbDJXS4Qu60VfeKxc8qmjnkdTlScWSYKAJmkozEOdE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984881; c=relaxed/simple; bh=UZ7wcKpZc50gJ2LcQ6z4UFnFYSLyGvDPH6gSgnrtEM0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XaHc1Rsx1JJ5v07UNXabY3nYUQIrN8mr59uayWnSaMWtAGlzvlpHxzHIK4MatWE0SQu96ZLXKpI8JI7yH0k81yZUnUpaF5ACaHGWLZGCa8GG5/YsQQLkO6vUf8Hg13bWhHCCKYwyMKOU+UErtdVmGh0/XGfy1cj3pLxcgahqQOs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=IAcVHXwg; arc=none smtp.client-ip=209.85.218.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="IAcVHXwg" Received: by mail-ej1-f74.google.com with SMTP id a640c23a62f3a-a6265d3cc76so111540166b.0 for ; Wed, 29 May 2024 05:14:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984878; x=1717589678; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=QLEJ3O4kV4RVdclClpDnM/s5WQUclK7m/dX1iG4Slsk=; b=IAcVHXwgYpD4JB2BvwJyr0x9xBaXhtt8yxtKgrjY93W01BOa+4roKzEKEhqdz8Bsma /Bw92o8nRJUYl8K+0Og/S6ZFnof2ClCI+X1Hwt6lAXyDDSCc+S8RQ7e+Xm9Dq90AYKu7 NWzElOg6sPfFh1SHderhr/oLYMMPohNR0x1najaljOZMOMBRKaifp3MFqt4/HlOBZUC9 R3LGaq2ZHPWFZdRJ+oijMSZF+QxgWzj7v4IUD0wcKacv6wTBCeO39J06gvGHCYvr40BE rvzPxhXocF93XZMfwycblpmUYeOxj1CToiK2kH6BWEY4KBpy2JrLj0eefxgRtSIP9Fkc zclw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984878; x=1717589678; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=QLEJ3O4kV4RVdclClpDnM/s5WQUclK7m/dX1iG4Slsk=; b=U3mn3PAd5lyji5SuNTiaUIjZxipGAsM1CguRubpp1iO796voqNqeb5VQU72OWFNKKo NT6yQG8Sn/6whpNq/bQFPm4nObWRBkiXzATMhHYS+yhWIq3QPclIo0LBvjaQDouwXw5W mSoCzDfSkHP2kId/m/AmJVrznxRjNsFr8/w25hCB2GP6/3RuMPXju7XTKZV0wTwWK59C 0zWacuOHRlalG7JKwGog0m3xihVM5g9Y8lXnbGj9WDblMvFQBd1+1s4irgSYeD0oj4rv Ubihe3+wVgjYcd8HxBUbUXnMJA94J2OjaY1tpLiBMKou0ppGh2fDcZMkNRxXQ5+TJhfi tfOA== X-Forwarded-Encrypted: i=1; AJvYcCUFBHWRIKGTK98UICCzZIo5OGzRQiczhkGYFuJlZsjkNTXx2Aw0mG2OQ8tJ1NvSbopd2+jE0C3AGweIttYI1HIG7lpq X-Gm-Message-State: AOJu0YyzYAdwUnpCSq2e12kC7CkKwb7+VnkfStnUKNUh4DyirFNbFfmz 1eHJ+posZE9kE0/c6Xa9VAQyelmwvHCGLK4MzjgKrXfioMYEE+DnXzIhTHXp3PQ1JAUpcbvTCA= = X-Google-Smtp-Source: AGHT+IHcGsvak0IaIBaJRsxrOxRAaw1LTq+hg6b9qMn9Z3a3+Nx+yXco9WUVoWIw3Nxf0/WiUp16udA02A== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a17:906:1991:b0:a5c:e96f:6022 with SMTP id a640c23a62f3a-a62642e4314mr1531066b.3.1716984878101; Wed, 29 May 2024 05:14:38 -0700 (PDT) Date: Wed, 29 May 2024 13:12:10 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-5-ptosi@google.com> Subject: [PATCH v4 04/13] KVM: arm64: nVHE: Add EL2h sync exception handler From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Introduce a handler for EL2h synchronous exceptions distinct from handlers for other "invalid" exceptions when running with the nVHE host vector. This will allow a future patch to handle kCFI (synchronous) errors without affecting other classes of exceptions. Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/kvm/hyp/nvhe/host.S | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 71fb311b4c0e..bc0a73d9fcd0 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -183,7 +183,7 @@ SYM_FUNC_END(__host_hvc) .endif .endm -.macro invalid_host_el2_vect +.macro __host_el2_vect handler:req .align 7 /* @@ -203,7 +203,7 @@ SYM_FUNC_END(__host_hvc) * been partially clobbered by __host_enter. */ stp x0, x1, [sp, #-16]! - b __guest_exit_panic + b \handler .L__hyp_sp_overflow\@: /* Switch to the overflow stack */ @@ -213,6 +213,10 @@ SYM_FUNC_END(__host_hvc) ASM_BUG() .endm +.macro host_el2_sync_vect + __host_el2_vect __guest_exit_panic +.endm + .macro invalid_host_el1_vect .align 7 mov x0, xzr /* restore_host = false */ @@ -222,6 +226,10 @@ SYM_FUNC_END(__host_hvc) b __hyp_do_panic .endm +.macro invalid_host_el2_vect + __host_el2_vect __guest_exit_panic +.endm + /* * The host vector does not use an ESB instruction in order to avoid consuming * SErrors that should only be consumed by the host. Guest entry is deferred by @@ -239,7 +247,7 @@ SYM_CODE_START(__kvm_hyp_host_vector) invalid_host_el2_vect // FIQ EL2t invalid_host_el2_vect // Error EL2t - invalid_host_el2_vect // Synchronous EL2h + host_el2_sync_vect // Synchronous EL2h invalid_host_el2_vect // IRQ EL2h invalid_host_el2_vect // FIQ EL2h invalid_host_el2_vect // Error EL2h From patchwork Wed May 29 12:12:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678754 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B29138F96 for ; Wed, 29 May 2024 12:14:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984883; cv=none; b=CNWdq+yoUixsMCAlRvrhIyuueyWC3LTRd+net8fqNWfbszq3LrKIsS/GyXBpJpGtBH59j0XFUGfrjE7fYgZR3aMgl0aiawcXRnEOPSLS/MhWTKtz/J+c3T6rzoknemOT9Xe4TRWCOKSoUhLLKBRUFA2VK1flQMwkawS8WGVGEVw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984883; c=relaxed/simple; bh=r+91pKsz/3ZztQui5RURf5PWtsJpKkKOgayFB1UGtqs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=oBVUCUJIUpwTIflJ1Ju20+uZn+HH5JVnNHQeF/QeMvS17qJ91/BGwliWsKityCyEzlH8zQrE1Dk5D98aT08AiqrZ76WIYmcSMzezTSIJm3Sgy67DZ0A6zYX/BdCV1auLDRNZ4srYug72lw9SDNuL6CmgSx4jaSmh566s+DuEWDs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=uEylCkLe; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="uEylCkLe" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-df771b71994so3514312276.3 for ; Wed, 29 May 2024 05:14:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984880; x=1717589680; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=hHat2RDFHpMYw2ro36KxYpGkz0GVoh8rlrjhKV6ZyaM=; b=uEylCkLeNsLUPCY6sNnka1aS7T5DKoqRP6O6XoMLJabbCdK+EmlRDimFyEZ5mN0BtP 1M9dGEZMfbz2ioM33MTvQ1buwyMXDfHf64RigeBGYyMKV7w1W4i2VJMQ3DKKAfhH2XhT 08WkGAhL9TBL1R84IBJOQ5Lw4gRlbaI2N1klIk5S16lio0hQvQUb0BNc2Sf1wyhtAwCB HsdvZ6ZoSkPnUhJoTrxd6ig59O6pLR0P1nRYhCAY77I+bw7QkR5u7+geYDszoTJC/F48 r5pNkqm39AxiVFc529qSs1a0vUl0BM0305wexNfPnxETxmw0edhUBDMgdgYdYEdeaica YPtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984880; x=1717589680; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=hHat2RDFHpMYw2ro36KxYpGkz0GVoh8rlrjhKV6ZyaM=; b=V6u3dosjwZlPgJYvo3igUeD2p+Ihtjtkii6iNZN+49bylcpqaPIiWUA8bCqw5yQ3G6 PHDaz7mQTN8Hbol6SiDZODuLuDUer4P0WDb207F04RuHkdzIVDaRy1QiMy7v6TSL7co1 aUHUtqsOG59+WCe2Dt9d5Yq8aCg5ctiwmB/bpEm7bhKd2eztziLCPembqu25zQylGp6l 6cZa1hw4Pu4zox7tKUNg3MMct4EG0a0YA4QwES1v1AQ3VBGB4HEyynM+8aNOe1QKVG1d 8hGaRn+gYFw2kl3unNuIQ5Bwyq516FVw2+pC/dFCECrakCmSVvfKVEsA75eU19JBMUxw oKBg== X-Forwarded-Encrypted: i=1; AJvYcCVjGNaExj0m04wIo9iOr5fUjavbtGnClZcnqfJs5pVhCTU+JSD8EG3KR9FMor9UVDBBPxg5/MkfqHcq06YHj+x17Rtf X-Gm-Message-State: AOJu0YyW3bh4dkhtzCKtlkibcfLLvYf+qQm2E1qh3FOe3gAwy6xshPSm 82S+MAzMlpOu0yh7uYffUylHg7aQ3PywKsi+SGxCxryAY3m4X4GVJcAgtIRqpsgHB2PJ4kv/Ag= = X-Google-Smtp-Source: AGHT+IHH51bjxff72q1pHbdvaYO/pbW32t5gktZkV+rMdlZvjhO2kbKYW+Qzcjh0fNRydZa0oQRqxrS2dQ== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6902:1889:b0:dee:6802:dc49 with SMTP id 3f1490d57ef6-df77213607amr4000587276.1.1716984880636; Wed, 29 May 2024 05:14:40 -0700 (PDT) Date: Wed, 29 May 2024 13:12:11 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-6-ptosi@google.com> Subject: [PATCH v4 05/13] KVM: arm64: Rename __guest_exit_panic __hyp_panic From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Use a name that expresses the fact that the routine might not exit through the guest but will always (directly or indirectly) end up executing hyp_panic(). Use CPU_LR_OFFSET to clarify that the routine returns to hyp_panic(). Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/hyp/entry.S | 6 +++--- arch/arm64/kvm/hyp/hyp-entry.S | 2 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/host.S | 4 ++-- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index 4433a234aa9b..343851c17373 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,7 +83,7 @@ alternative_else_nop_endif eret sb -SYM_INNER_LABEL(__guest_exit_restore_elr_and_panic, SYM_L_GLOBAL) +SYM_INNER_LABEL(__hyp_restore_elr_and_panic, SYM_L_GLOBAL) // x2-x29,lr: vcpu regs // vcpu x0-x1 on the stack @@ -91,7 +91,7 @@ SYM_INNER_LABEL(__guest_exit_restore_elr_and_panic, SYM_L_GLOBAL) ldr x0, [x0, #CPU_ELR_EL2] msr elr_el2, x0 -SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) +SYM_INNER_LABEL(__hyp_panic, SYM_L_GLOBAL) // x2-x29,lr: vcpu regs // vcpu x0-x1 on the stack @@ -109,7 +109,7 @@ SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) // accurate if the guest had been completely restored. adr_this_cpu x0, kvm_hyp_ctxt, x1 adr_l x1, hyp_panic - str x1, [x0, #CPU_XREG_OFFSET(30)] + str x1, [x0, #CPU_LR_OFFSET] get_vcpu_ptr x1, x0 diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 03f97d71984c..7e65ef738ec9 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -122,7 +122,7 @@ el2_error: eret sb -.macro invalid_vector label, target = __guest_exit_panic +.macro invalid_vector label, target = __hyp_panic .align 2 SYM_CODE_START_LOCAL(\label) b \target diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index ed9a63f1f7bf..d9931abf14c2 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -689,7 +689,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __kvm_unexpected_el2_exception(void) { - extern char __guest_exit_restore_elr_and_panic[]; + extern char __hyp_restore_elr_and_panic[]; unsigned long addr, fixup; struct kvm_exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); @@ -712,7 +712,7 @@ static inline void __kvm_unexpected_el2_exception(void) /* Trigger a panic after restoring the hyp context. */ this_cpu_ptr(&kvm_hyp_ctxt)->sys_regs[ELR_EL2] = elr_el2; - write_sysreg(__guest_exit_restore_elr_and_panic, elr_el2); + write_sysreg(__hyp_restore_elr_and_panic, elr_el2); } #endif /* __ARM64_KVM_HYP_SWITCH_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index bc0a73d9fcd0..a7db40a51e4a 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -214,7 +214,7 @@ SYM_FUNC_END(__host_hvc) .endm .macro host_el2_sync_vect - __host_el2_vect __guest_exit_panic + __host_el2_vect __hyp_panic .endm .macro invalid_host_el1_vect @@ -227,7 +227,7 @@ SYM_FUNC_END(__host_hvc) .endm .macro invalid_host_el2_vect - __host_el2_vect __guest_exit_panic + __host_el2_vect __hyp_panic .endm /* From patchwork Wed May 29 12:12:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678755 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15AA6181B87 for ; Wed, 29 May 2024 12:14:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984885; cv=none; b=TJR6TZUyLR22P8VrA11S/9/MRz1cLRkfTiooaNKpPFFnGxYkuASTxDWfOygLBChh0AfPZEgoom1SUgVXLOtMVdSvoh7x7YpBUzcbgrBolJSTnSlvK8vEMt4dcAOvgfjP2VKp2puwmq44XCZNQwC84aCkv7M3tL7OqMomjf8JcII= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984885; c=relaxed/simple; bh=+cKcneugNDE7L/qpVl91ixp1M6xylMEZ521Xa4hyBQI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Vg+9FPmYeNp/xQBbeQNataBooBc8PuDFBUfTxOxRPSf9ALjYS6G7pmZLsdRtXhM0KYoZIpPyLS4Hylx6F4VYalYtQtPFQDd6AiLChnCSMsmYSGAawcNnCKeBXeGvrPCYIrVwoTpxV8SQW+KDuk+3hUDlYtIjRhvuNTCQLTlzfEE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=SxL5x3rk; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SxL5x3rk" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-62a50486746so27729687b3.3 for ; Wed, 29 May 2024 05:14:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984883; x=1717589683; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=jFr7qj90it7wD8ycil4shHE/w7boH3XXa9rQ6TceJ7g=; b=SxL5x3rkad37Ye+46JX/VG7oHFQS9sK2Z/zbLDApvqy+KRko4D+fOpcdGOYUyNswxN mxIbQi9dv5tvtjWfcmVYBOmybVQJVRLW1StIg4Y9zFKORLNsBecj9H1YJdLA8rUvu06T tKtJYRCHw90BrPrcEKnaWFOsV6SFp0VfsnSyCFEWCpk1lcDdVzRFEQkSNqHBMtlK636X Bd7tq9byjUPWiJRMAHd7YF1FLany+9aFEHjN/lGXBiENpFJ+V6OKVHJObwwsGY4PyfFo hXFxvSpkiIY1lX/vZpdlTLzqCYZHWAcKrX79ww/g43YmU1UnF7xM/LqOTgEoDCro9P9G rOYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984883; x=1717589683; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=jFr7qj90it7wD8ycil4shHE/w7boH3XXa9rQ6TceJ7g=; b=TfqbGYZi0VoGqlNlLiXfbs2CwHmZel15sERXRu53OrFoyIt4/l98qKRT4GCa0kv2ud /7vlqtzwnL6amfPRkN97bpm2hnQYwtpGSIORp1Xiq+p3z8H2TzEe/Znh83YkQExbQQph 2PioLui8a6Luzc2GBrL4m2iE0A4QyJMZvVEM1YRcQOodA/2uAiJCin+MM36HXdEYweKm y7/3z+APPluIC25DK1/horRi+3vHcEF/1C6c+LdloLPR8i6qOGfwvhkmYoCwaq6DL+Sh b8CDYGDLm/Vi8zZnWYMVVGh5iJion0OxUn5wPMa9q3eBPOj35p45cbHqAaL17Wtba1Sr ASRw== X-Forwarded-Encrypted: i=1; AJvYcCWbuHG4u0zdNKnJeXlny7vDfbtAuQB6LaM4iQopQGF3rN6Y7H2tg+UPfCRi5f/xfJF7HDBRQtHpypu1MpypH/qHg6y8 X-Gm-Message-State: AOJu0Yw/bo+9p+QoCc2nv8sjqybJk9wNxCkKEgL3pGn+/OUhPgsEAMVe BVwNtcTqYi3HrTumjFxZ6X2hst4nPzRB4J+Na57QVKtpT9xl7gUVhQCNulu6bdrSQ420RMqRpg= = X-Google-Smtp-Source: AGHT+IFkqfeHx8YzJAunywpRdidgNdjLeTtWgg2p4V4vZYTrR/3mutQzX2qxXb5fN2wPSUT3J1iB5pmTTg== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a81:4c56:0:b0:61a:d0d2:b31 with SMTP id 00721157ae682-62a08d9304emr35784307b3.3.1716984883084; Wed, 29 May 2024 05:14:43 -0700 (PDT) Date: Wed, 29 May 2024 13:12:12 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-7-ptosi@google.com> Subject: [PATCH v4 06/13] KVM: arm64: nVHE: gen-hyprel: Skip R_AARCH64_ABS32 From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Ignore R_AARCH64_ABS32 relocations, instead of panicking, when emitting the relocation table of the hypervisor. The toolchain might produce them when generating function calls with kCFI to represent the 32-bit type ID which can then be resolved across compilation units at link time. These are NOT actual 32-bit addresses and are therefore not needed in the final (runtime) relocation table (which is unlikely to use 32-bit absolute addresses for arm64 anyway). Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/kvm/hyp/nvhe/gen-hyprel.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c b/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c index 6bc88a756cb7..b63f4e1c1033 100644 --- a/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c +++ b/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c @@ -50,6 +50,9 @@ #ifndef R_AARCH64_ABS64 #define R_AARCH64_ABS64 257 #endif +#ifndef R_AARCH64_ABS32 +#define R_AARCH64_ABS32 258 +#endif #ifndef R_AARCH64_PREL64 #define R_AARCH64_PREL64 260 #endif @@ -383,6 +386,9 @@ static void emit_rela_section(Elf64_Shdr *sh_rela) case R_AARCH64_ABS64: emit_rela_abs64(rela, sh_orig_name); break; + /* Allow 32-bit absolute relocation, for kCFI type hashes. */ + case R_AARCH64_ABS32: + break; /* Allow position-relative data relocations. */ case R_AARCH64_PREL64: case R_AARCH64_PREL32: From patchwork Wed May 29 12:12:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678756 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47593181B95 for ; Wed, 29 May 2024 12:14:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984887; cv=none; b=HjzVY1jKan7zTggyhluu7yNbUSqH62lJ9YDQmPFAL14bgaPg/hs9Wkh3B7ZuG2O5rfLMFZ9kqZxIBcVDV4yCTOUNhTxojU5AGrKyUig4xyr+l8Z9Byaw+WgWzC3Ty1rVWZ1LUlngzn3AvtGNsRCK6mSDZSb981gjtUYoAldAS5Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984887; c=relaxed/simple; bh=oY+YSwPHFqVVY3fzP9ou/E6BfHn7ARC/H1gWBIAm1+g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ijrFEkoGCH0D2vyCxov2lt1hnE4t4mam0zyLjWI0zLvyA1ibYiGh+vNuYdK2m1+wSj/EJVm+02B+8bYQp93g74aDOUx1To2fP0Euh1N7bflZsSt25scgkAsRSkaouL4c48sgRsGl1Bew0LCXlMj3xRV/2Z2br2hNLiqKisOc1OA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=GXtuJn9H; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="GXtuJn9H" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-627f382fb97so32315897b3.3 for ; Wed, 29 May 2024 05:14:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984885; x=1717589685; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=8LApYKgBNKEpQOUHCgPJ8rjKQvWvAO0g6PQmvtilIlw=; b=GXtuJn9HxhEM67Djw72HIPG3Vbsx/FqU15ckzdvn/tfUQXLGh+HpqlFormN42z5psV d6lhQHI1iozKFommHGTa+ULyWO4pXBYGamNklv9PwYVn95syjy8l84FuBVLjgvJPUNFM ibTrNBYmFH8Pz2U7KvhLq+fZ0dmNt7/hWeZaZV32k4Wp26AYn5uD9O6TLchlwRYTLK9S IpRLiehtYYpmyPAt3AZ5kkKWzKoi1E1mnQqF0qPWKvg1218ZZGylTBu867lHSjvSxuF8 dMp55l2QpbVmF/w2Zib+EMBJPX2ioQ6UyGdQO9FBRTnNXW8gKeQvdZQI+2DgVAc/pgiQ zDuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984885; x=1717589685; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=8LApYKgBNKEpQOUHCgPJ8rjKQvWvAO0g6PQmvtilIlw=; b=gpdQoIONzHYTeUNJvPA4FU5IYibaVvcGcI3GHD19MQvr9xfw5jgEEKxqW+jACMFWYH UHewC37GTDSPuudZFGnV2gCtSVdBO+3Fnm+3HWaBJKfg8AjW/t5gbeR6cHa+3C7tav/z RkZXPe2pinEiVtr/JLlBNeXFZqdTWyWH+anj/nP1Pf6bIuHJ6C5s2uer5UWzUtVm6Pvc O6MPlU+PdzCye6i8tgwrSupHPWowbNf57YZYoLScis5PjMGuGf/yqzcqUqGTmsp/p9+o CFK+eBSkbyI5EbxdV4JaXBhbZGe8LqdjovzmrZPhdGx3bdPKTblys91dohW3DK0GjKDV D7lg== X-Forwarded-Encrypted: i=1; AJvYcCUeckmo9uRGmvmbiIO+aGD6aNdHI6CsHRkF1q1iXgNKNPS+YhkLpw5/r7Ue6H0+EREYV44aQtnJxzE37l/QkcVoNZFF X-Gm-Message-State: AOJu0Yzkob0AxQNxn7xiP9/B4ggMJVcFRvtdXPQPIcuC/VeqbpZkLBkI tQBC+0Rpg6Uzkh5QF80R3+naGIaVKEh8PvFoJDrwqjg0i6+Z9jilccJDlme7XQ8GvP+/S1btDA= = X-Google-Smtp-Source: AGHT+IHSjvv323m46MNf64EsuTBJ/plesPXi5fnjH9bpFqNhZzaJSI42rMp8TuTkGGyShh9t0w2mdhFYDg== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6902:1006:b0:df7:8c1b:430a with SMTP id 3f1490d57ef6-df78c1b6af8mr3075741276.3.1716984885370; Wed, 29 May 2024 05:14:45 -0700 (PDT) Date: Wed, 29 May 2024 13:12:13 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-8-ptosi@google.com> Subject: [PATCH v4 07/13] KVM: arm64: VHE: Mark __hyp_call_panic __noreturn From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Given that the sole purpose of __hyp_call_panic() is to call panic(), a __noreturn function, give it the __noreturn attribute, removing the need for its caller to use unreachable(). Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/kvm/hyp/vhe/switch.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index d7af5f46f22a..0550b9f6317f 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -384,7 +384,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) return ret; } -static void __hyp_call_panic(u64 spsr, u64 elr, u64 par) +static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) { struct kvm_cpu_context *host_ctxt; struct kvm_vcpu *vcpu; @@ -409,7 +409,6 @@ void __noreturn hyp_panic(void) u64 par = read_sysreg_par(); __hyp_call_panic(spsr, elr, par); - unreachable(); } asmlinkage void kvm_unexpected_el2_exception(void) From patchwork Wed May 29 12:12:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678757 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6CDF3181310 for ; Wed, 29 May 2024 12:14:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984891; cv=none; b=A5/pD64azlEorNaAT2uiq7oE3ITkSz/4pQx3qee2R5+HSapbJR2uhnKwXOcoqZsGaGOgO1SURh8vx5zrlwoskjA/lVCxKjw+x4Sk/1tOH+4hc+ZYiMQIuIf7F1HpjaBBd8cF2xkFN1HixELPMDnnuEbKZ7+Q/Vu3RflRKuhMmj4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984891; c=relaxed/simple; bh=9btKNfRjz6fK+DhlL9k76Ks1QkjES/uXicjLnT/2h3c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RmZ3qtF7AuEo4W+m1z8GAOSaBi+jzT1DauBQuc1cW4G6h8vAePV0G6AXQX59cpn21ogXCbVlWV4BSJ5TwUibp/DEfKAmUK1dcaRvVBUiwaPDk3YgcGvUe73PDmg/Ww90RrByaGdLDIBoGatRZzBkUb4DPe0DShLqS5dBkzfmEXM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=c/y9KcD5; arc=none smtp.client-ip=209.85.208.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="c/y9KcD5" Received: by mail-ed1-f73.google.com with SMTP id 4fb4d7f45d1cf-579e27b0404so1044606a12.1 for ; Wed, 29 May 2024 05:14:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984888; x=1717589688; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=Fl+f30rl7dVa/IVaUSQ5SKFqgJp4W7PmIf/N7fy8cOU=; b=c/y9KcD5CXWwMDkErOJv4lzMpuTdFRm7T6/pyLJvhwBmlry9kygZ/tF4Rbj8lJw0kA bL7JGgTZxZRRNVy3gt8oubSUdD6Dh8OFEwQV4gHcV+fC42GOt5j7+ql6R033mYaLtJsC 8xC4px09WlFMpK0a5oPpR8uy7tfBpna6uofywc4jBqrxYQqknQUmUeWSZPRcHpBgicaC A7uUg9RsrSA06TvQHIeDaQRITPIf8KBP910m2zs8oIkau7Tg7fYR3ih9jyntrr7SYm4m Fam6O2AZMMZ2yJxLIWbjS7wEKyNCii2XjHCy4vdgOq44gGKZS3UimnI24/DWEZtUKS9D KrOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984888; x=1717589688; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=Fl+f30rl7dVa/IVaUSQ5SKFqgJp4W7PmIf/N7fy8cOU=; b=Tl9oSgSyyV8On+7kpCWHryVH1G8VQ+c4zxTcBBPxbBxrBIQtqRdpadEYnvPAZxgwW0 IzKh9xvy/ekEMOxxk1KWN4gtp5ULy9RmwHHP1bKS5fSjLTfXNW92IM6zx7wCzv5QIAwI djyD4GPEA0iIs4/0vMUjadtaM5CYPagslUFREtLElEPnyg0NS8fSDhgtWvKmrLkidF7L xmitEeY1y8jlpbrTBQPPzTZwfRVse717uFPedqO4+ksyhIhXzB9co1zc+kWB7vYQNkvn 75OdAxOW0f0R9FWcN9Cb4GZN7braFBA/tRrKQRzVPoY8C6FgDwqhsFKMwBGEehRWwetv 2INQ== X-Forwarded-Encrypted: i=1; AJvYcCW81DZGnx15DxAP5TklwIRa0zgdN/piWcrE3m8zx0IloukN/JQPzw47zOxnIdAMajzL7nMgMBYD75c+f9yWTIIgC7jq X-Gm-Message-State: AOJu0Yy+NIKBuohu4fM+yfakf+P0S0rDwudSBZF91YiDBpIoHw06lbYa fGsmHVtKmCeZCxfqhkRWvpCgPoPKGy71VsdbBpRr65ik0rBYBbKZDHOKTDO2sqq2/c+R31Q8yw= = X-Google-Smtp-Source: AGHT+IEEHNDVH5oua3/ecoC74+hCc08TVB6uy8myiwmNDRyQ2CUe66zxXOENHLIB7cqS59dn8IOjkr3XHg== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6402:1954:b0:578:b249:2b06 with SMTP id 4fb4d7f45d1cf-578b24964d6mr14880a12.6.1716984887626; Wed, 29 May 2024 05:14:47 -0700 (PDT) Date: Wed, 29 May 2024 13:12:14 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-9-ptosi@google.com> Subject: [PATCH v4 08/13] arm64: Introduce esr_comment() & esr_is_cfi_brk() From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort As it is already used in two places, move esr_comment() to a header for re-use, with a clearer name. Introduce esr_is_cfi_brk() to detect kCFI BRK syndromes, currently used by early_brk64() but soon to be also used by hypervisor code. Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/include/asm/esr.h | 11 +++++++++++ arch/arm64/kernel/debug-monitors.c | 4 +--- arch/arm64/kernel/traps.c | 8 +++----- arch/arm64/kvm/handle_exit.c | 2 +- 4 files changed, 16 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index 7abf09df7033..77569d207ecf 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -379,6 +379,11 @@ #ifndef __ASSEMBLY__ #include +static inline unsigned long esr_brk_comment(unsigned long esr) +{ + return esr & ESR_ELx_BRK64_ISS_COMMENT_MASK; +} + static inline bool esr_is_data_abort(unsigned long esr) { const unsigned long ec = ESR_ELx_EC(esr); @@ -386,6 +391,12 @@ static inline bool esr_is_data_abort(unsigned long esr) return ec == ESR_ELx_EC_DABT_LOW || ec == ESR_ELx_EC_DABT_CUR; } +static inline bool esr_is_cfi_brk(unsigned long esr) +{ + return ESR_ELx_EC(esr) == ESR_ELx_EC_BRK64 && + (esr_brk_comment(esr) & ~CFI_BRK_IMM_MASK) == CFI_BRK_IMM_BASE; +} + static inline bool esr_fsc_is_translation_fault(unsigned long esr) { /* Translation fault, level -1 */ diff --git a/arch/arm64/kernel/debug-monitors.c b/arch/arm64/kernel/debug-monitors.c index 64f2ecbdfe5c..024a7b245056 100644 --- a/arch/arm64/kernel/debug-monitors.c +++ b/arch/arm64/kernel/debug-monitors.c @@ -312,9 +312,7 @@ static int call_break_hook(struct pt_regs *regs, unsigned long esr) * entirely not preemptible, and we can use rcu list safely here. */ list_for_each_entry_rcu(hook, list, node) { - unsigned long comment = esr & ESR_ELx_BRK64_ISS_COMMENT_MASK; - - if ((comment & ~hook->mask) == hook->imm) + if ((esr_brk_comment(esr) & ~hook->mask) == hook->imm) fn = hook->fn; } diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 215e6d7f2df8..9e22683aa921 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -1105,8 +1105,6 @@ static struct break_hook ubsan_break_hook = { }; #endif -#define esr_comment(esr) ((esr) & ESR_ELx_BRK64_ISS_COMMENT_MASK) - /* * Initial handler for AArch64 BRK exceptions * This handler only used until debug_traps_init(). @@ -1115,15 +1113,15 @@ int __init early_brk64(unsigned long addr, unsigned long esr, struct pt_regs *regs) { #ifdef CONFIG_CFI_CLANG - if ((esr_comment(esr) & ~CFI_BRK_IMM_MASK) == CFI_BRK_IMM_BASE) + if (esr_is_cfi_brk(esr)) return cfi_handler(regs, esr) != DBG_HOOK_HANDLED; #endif #ifdef CONFIG_KASAN_SW_TAGS - if ((esr_comment(esr) & ~KASAN_BRK_MASK) == KASAN_BRK_IMM) + if ((esr_brk_comment(esr) & ~KASAN_BRK_MASK) == KASAN_BRK_IMM) return kasan_handler(regs, esr) != DBG_HOOK_HANDLED; #endif #ifdef CONFIG_UBSAN_TRAP - if ((esr_comment(esr) & ~UBSAN_BRK_MASK) == UBSAN_BRK_IMM) + if ((esr_brk_comment(esr) & ~UBSAN_BRK_MASK) == UBSAN_BRK_IMM) return ubsan_handler(regs, esr) != DBG_HOOK_HANDLED; #endif return bug_handler(regs, esr) != DBG_HOOK_HANDLED; diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index b037f0a0e27e..d41447193e13 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -423,7 +423,7 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, if (mode != PSR_MODE_EL2t && mode != PSR_MODE_EL2h) { kvm_err("Invalid host exception to nVHE hyp!\n"); } else if (ESR_ELx_EC(esr) == ESR_ELx_EC_BRK64 && - (esr & ESR_ELx_BRK64_ISS_COMMENT_MASK) == BUG_BRK_IMM) { + esr_brk_comment(esr) == BUG_BRK_IMM) { const char *file = NULL; unsigned int line = 0; From patchwork Wed May 29 12:12:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678758 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F27FD181BA7 for ; Wed, 29 May 2024 12:14:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984893; cv=none; b=QTmmVU4YHums+zOZXPthKKb9WJkp3Gw0/8MxWJ2HEn505kQPAxrwN4PMjBWXihewDkkv7WD0doxtA9zTXOirozvQ6GHgq5NihS15CB6NzIrqZwOFSHujCbKs33zYvJ6cGp7XOSpIedZQmxVCRtx+vfO5FuO0kEFltpH/3ZuReLQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984893; c=relaxed/simple; bh=WMM5G0N4qiU3hKk5YjCYS3EFswJ4zM88IW8Mlp+DlzQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=QMTYOUVAR9R6q0VXKyw+ZY2IBubZIvd2QzTF6Tuo5CCMucG14Ar1oR7zmjPoRU8Jd0fkAo7NHQ+VJ0xUhfsCdnsL6rKSOfZ39rHnXqkPA617TvaMNjoVXwgfFKTR2hn1qUU2gIVypHMIdfLn0JDW1cXkx1WLlF14n26RtTEkZ/0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WScUpzsK; arc=none smtp.client-ip=209.85.208.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WScUpzsK" Received: by mail-ed1-f73.google.com with SMTP id 4fb4d7f45d1cf-57865139b7dso1201616a12.1 for ; Wed, 29 May 2024 05:14:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984890; x=1717589690; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=8CshqLHlWDJc+sAJS8Y64z59WBwvQ7VfO+S7UGCs2ew=; b=WScUpzsK7gne4n8EhCkDuN4DS2Jb45bmol3C97hzTuY8D8Gu0S569u1JpR4Cc7zXTC gW+v+hprtbPD1sYViAqq0J9snNxdNtl9Nn1pBp7Dz3xc7MABO+jZ8Rwpgq8vQNfn0hew 810Ro9s67GSxXYPS8XZmMX/zmw9lvQuYRRV4hhTCsCuVJmWR+/l07zWB2Yggnd+ir2pL 5f0zByXgXOwNiPmN7EyjrH7dRLyM1BKRdDfxMgVU7lLQVAmEc9CjDezUJ2igW2U7Ut9R hNKx5JN6AtnFjBpo+9cOTpSkkkLG3+qscSaBmuSIf0jQL5tvQvsKryuVy/CBIJgJn/H+ LGfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984890; x=1717589690; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=8CshqLHlWDJc+sAJS8Y64z59WBwvQ7VfO+S7UGCs2ew=; b=nejW46joFyiUo1G5sd/fO6dRKVpGfFjxQMYR9Ie1Jr1X49nVRhiUVxflLoi8xemISc Yz8K/Y4W3tXm2tJSdb95DyRD81lWB2EttE3wPR8Mse/JZBDT3E7aZ1ERbAnvwJAVoKF8 Q6JFirl8FnfhZ7k+fYwbcFkgr1+kn2qJP85qBthvwiuGffjyEWnlPW4EhUhyYUTzlFqT u2S0pvLz8SKFis0Hnl9NnCvxk/HCHskWErvigbF+p7myFoPtP2daLkkm9Ce+s5UgHQQ/ aZ8w5pt9XiImTyL9vDE0Of0lEVDicXxooL2RuFaYpF9s3FIrb2RI7jxxAinLBRUTNcxj BpGA== X-Forwarded-Encrypted: i=1; AJvYcCVO2LLLxJLxJxMt1bNR23SzUAnEMQCyLQERgd8GNAeDb31NolKJLZvLxfsiR9/cLQ7o4r7qVGAKvVx2eRnyaQZpDnXM X-Gm-Message-State: AOJu0YwOCw7j5Dbws97/Uqo9D5cbM0NYIORtr0hcE7PpgbyG3cuhNcj2 K+K5YyAh+ivCobj/iRjV+CtK1IohWYCLHqxs9rcEW4vgDKQ05eet7Wx7gddXS6Cx2z6NoSTBUQ= = X-Google-Smtp-Source: AGHT+IE8phGpuuq3k7TrjskcqmK2dPZ5kpLf85cdSWAkkY4MTdPq3mbXxQAEr7SdzcRt27FdzPre1IagDw== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6402:35b:b0:578:60a0:9995 with SMTP id 4fb4d7f45d1cf-57a041685b6mr2744a12.3.1716984890481; Wed, 29 May 2024 05:14:50 -0700 (PDT) Date: Wed, 29 May 2024 13:12:15 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-10-ptosi@google.com> Subject: [PATCH v4 09/13] KVM: arm64: Introduce print_nvhe_hyp_panic helper From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Add a helper to display a panic banner soon to also be used for kCFI failures, to ensure that we remain consistent. Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/kvm/handle_exit.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index d41447193e13..b3d6657a259d 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -411,6 +411,12 @@ void handle_exit_early(struct kvm_vcpu *vcpu, int exception_index) kvm_handle_guest_serror(vcpu, kvm_vcpu_get_esr(vcpu)); } +static void print_nvhe_hyp_panic(const char *name, u64 panic_addr) +{ + kvm_err("nVHE hyp %s at: [<%016llx>] %pB!\n", name, panic_addr, + (void *)(panic_addr + kaslr_offset())); +} + void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, u64 elr_virt, u64 elr_phys, u64 par, uintptr_t vcpu, @@ -439,11 +445,9 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, if (file) kvm_err("nVHE hyp BUG at: %s:%u!\n", file, line); else - kvm_err("nVHE hyp BUG at: [<%016llx>] %pB!\n", panic_addr, - (void *)(panic_addr + kaslr_offset())); + print_nvhe_hyp_panic("BUG", panic_addr); } else { - kvm_err("nVHE hyp panic at: [<%016llx>] %pB!\n", panic_addr, - (void *)(panic_addr + kaslr_offset())); + print_nvhe_hyp_panic("panic", panic_addr); } /* Dump the nVHE hypervisor backtrace */ From patchwork Wed May 29 12:12:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678759 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A947F18131C for ; Wed, 29 May 2024 12:14:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984895; cv=none; b=fQMPzy+DrkEQgTt6tCBL9zXeq9UGf1tVv+68nHnz14JvvH+UPlTQi4OyKxIks2S6kAC+ncKm7zgD28UfgYCR0EbXBk1GCxG85oMOI0vNzt3EBGnmKfCsbptdl/fue04BN1nhO3W5kmYfGPmot9Io4BKGV9fzcu9cNydP4H/fe0g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984895; c=relaxed/simple; bh=RAWRwxx2x29Z9C8sFEWaOgpe83fasv5IFOL7pJOwjNE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=JV904Z/WjnFYNFFIC5vey7b6dUPGaOtGxQn1dq2xurnovt51CpRbNlO4yLTOgki2CxeD3ZJbEGTCXi3e3JTo1bBNRl6VEVeQkLNtIROvHLPuCmn7Rzsak+kZpgXfsvXxFHVhQGA8trI2JbJbjJO0ZAAJp3hBmvfsfYHz6gHO3A8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=iBjdCQU4; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="iBjdCQU4" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-df796aaa57dso2772949276.1 for ; Wed, 29 May 2024 05:14:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984893; x=1717589693; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=4CCra/rqIFpxe8HMFYt7fMIj2cwQLIfhoDjzzJPzF3M=; b=iBjdCQU4EHK6ZI6z9MtEIgWGV2fEAYZN+WjKBq9bipr/Y57cLzHNDjhm1t5phMOEjT a4FDYful+ML5M1K/u1b6Lwzc9GOO206H8dmcRV6h73vowL4/lVV9qTgaQ0HQ2XOF6aVx aY7dPicsjgpcgdoq/yNQBLwWNgtQgW/sl0WCHV/Ka2pPt1c84EKrSocyHLveOkOrldfr UMscRJzeBEDyoLMK30Dz6CbcqcQqfDvdTtldPRK77EmMke4ywQEOTV3MAOPdA4ZekMLz wFrhpUmAXjz9B62sFlIyI7+IEr7JxghbfZRW2v9TTdbde4mOjVBTYJD8VWJwb5XOjZ/2 GTkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984893; x=1717589693; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=4CCra/rqIFpxe8HMFYt7fMIj2cwQLIfhoDjzzJPzF3M=; b=xNynDX2cYeNdnJAy/AWeLNp/2tx9RIMLhuEvBW08A9c0hk4K/uCMV/wqgMKcGC8RQp rNfmhdYQKuG/d8ydkcccGtkL7UG5z/S1e/qXqHL7tSVRJQ3a8MG6nObRJ2Dry58I01R0 4Rsze2bFrEUYVZiYuR9WprovzQ5dO/jYzLakBZ0oRUP5icY8SJQ7kY7t0QKGtC8tSV2o ePOo70ukj1/jbzlme3XzcQq5Eipy51DmxRKi78tQayL+1+BucYyKmfZRzvMKx4JO0POC hkqGwlDXrwp93Sdi8YO9YTULgRuRPhfTojSMRYwFsspfX7i14RCGCu408Wnx9nRlc3vo 4q4A== X-Forwarded-Encrypted: i=1; AJvYcCX/6urZIT0f0yT1/L0zKcWPSNenPgDCy+LZYQmErq3C+OYFumSGdGcJixuAnaoCWnEtBbh8s3sUXlze/2KLmitEZR80 X-Gm-Message-State: AOJu0YxvD+B/YqYb94MTT6CNALkJIranJoFY20hkOTjMwDYeS14o91C0 f4mCyjIr1V8LXyZFwWZ9Qu5ZYAQR/5ITSaVB8Hte1vD+IratkBVV/lGnzpzMZ4SHsPDJEXppWg= = X-Google-Smtp-Source: AGHT+IHKlfLzIehWc22oDX8uMfcDmLL5aEy3cA9blPvEjAUh7oYjXMJskW3TDfNlIiBODMYBSdvEYmQV4w== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6902:1248:b0:df7:6f84:f36f with SMTP id 3f1490d57ef6-df772172705mr1421198276.4.1716984892739; Wed, 29 May 2024 05:14:52 -0700 (PDT) Date: Wed, 29 May 2024 13:12:16 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-11-ptosi@google.com> Subject: [PATCH v4 10/13] KVM: arm64: nVHE: Support CONFIG_CFI_CLANG at EL2 From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort The compiler implements kCFI by adding type information (u32) above every function that might be indirectly called and, whenever a function pointer is called, injects a read-and-compare of that u32 against the value corresponding to the expected type. In case of a mismatch, a BRK instruction gets executed. When the hypervisor triggers such an exception in nVHE, it panics and triggers and exception return to EL1. Therefore, teach nvhe_hyp_panic_handler() to detect kCFI errors from the ESR and report them. If necessary, remind the user that EL2 kCFI is not affected by CONFIG_CFI_PERMISSIVE. Pass $(CC_FLAGS_CFI) to the compiler when building the nVHE hyp code. Use SYM_TYPED_FUNC_START() for __pkvm_init_switch_pgd, as nVHE can't call it directly and must use a PA function pointer from C (because it is part of the idmap page), which would trigger a kCFI failure if the type ID wasn't present. Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/kvm/handle_exit.c | 10 ++++++++++ arch/arm64/kvm/hyp/nvhe/Makefile | 6 +++--- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 6 +++++- 3 files changed, 18 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index b3d6657a259d..69b08ac7322d 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -417,6 +417,14 @@ static void print_nvhe_hyp_panic(const char *name, u64 panic_addr) (void *)(panic_addr + kaslr_offset())); } +static void kvm_nvhe_report_cfi_failure(u64 panic_addr) +{ + print_nvhe_hyp_panic("CFI failure", panic_addr); + + if (IS_ENABLED(CONFIG_CFI_PERMISSIVE)) + kvm_err(" (CONFIG_CFI_PERMISSIVE ignored for hyp failures)\n"); +} + void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, u64 elr_virt, u64 elr_phys, u64 par, uintptr_t vcpu, @@ -446,6 +454,8 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, kvm_err("nVHE hyp BUG at: %s:%u!\n", file, line); else print_nvhe_hyp_panic("BUG", panic_addr); + } else if (IS_ENABLED(CONFIG_CFI_CLANG) && esr_is_cfi_brk(esr)) { + kvm_nvhe_report_cfi_failure(panic_addr); } else { print_nvhe_hyp_panic("panic", panic_addr); } diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 50fa0ffb6b7e..782b34b004be 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -89,9 +89,9 @@ quiet_cmd_hyprel = HYPREL $@ quiet_cmd_hypcopy = HYPCOPY $@ cmd_hypcopy = $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ $< $@ -# Remove ftrace, Shadow Call Stack, and CFI CFLAGS. -# This is equivalent to the 'notrace', '__noscs', and '__nocfi' annotations. -KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI), $(KBUILD_CFLAGS)) +# Remove ftrace and Shadow Call Stack CFLAGS. +# This is equivalent to the 'notrace' and '__noscs' annotations. +KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS), $(KBUILD_CFLAGS)) # Starting from 13.0.0 llvm emits SHT_REL section '.llvm.call-graph-profile' # when profile optimization is applied. gen-hyprel does not support SHT_REL and # causes a build failure. Remove profile optimization flags. diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index d859c4de06b6..b1c8977e2812 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -5,6 +5,7 @@ */ #include +#include #include #include @@ -267,8 +268,11 @@ SYM_CODE_END(__kvm_handle_stub_hvc) /* * void __pkvm_init_switch_pgd(phys_addr_t pgd, void *sp, void (*fn)(void)); + * + * SYM_TYPED_FUNC_START() allows C to call this ID-mapped function indirectly + * using a physical pointer without triggering a kCFI failure. */ -SYM_FUNC_START(__pkvm_init_switch_pgd) +SYM_TYPED_FUNC_START(__pkvm_init_switch_pgd) /* Turn the MMU off */ pre_disable_mmu_workaround mrs x9, sctlr_el2 From patchwork Wed May 29 12:12:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678760 Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 94321181BBF for ; Wed, 29 May 2024 12:14:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984898; cv=none; b=LAxi3/GJd8Cd62tdLUNG8i/29f+ebilrDboFofitvy+c91QEt54ThdxWQ5nQkM3AKeRbeQ1S7Z2VcERPD6rPRPSfBIRPBe4NMymY8y82CbZh6iXujz1QjtIKSQ+fLvkDtc/hmK0dMLps967pSwd5cEMIgdFNaeSfgGa9FAvWopw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984898; c=relaxed/simple; bh=d7ll60CcSbiQvQOKCWGY1hz1EabEJsXYqy8qzz9ADYs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=rl4YPsRtBxR5oH7UHl8auffDS6UtdlVLg0Ajv09DmFzLxzDRjiY6PEJWieTA9DXvzlvqHqHqOdjgWJnOn0130bZ+baLPvl93wIZW1HNfy4mXNTIp7jJm58Fo1CFpMUEMB0T+DLeJOTFmj3WaTJ5tV59c8V8xKgcCbi7hm+Eh+dY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=mW69H95v; arc=none smtp.client-ip=209.85.218.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="mW69H95v" Received: by mail-ej1-f73.google.com with SMTP id a640c23a62f3a-a635b551919so57826566b.2 for ; Wed, 29 May 2024 05:14:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984895; x=1717589695; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=20VEPEn3Gb7BPD8Z30RwNAM2J68OYkR1rMftiYbdXdo=; b=mW69H95vs0wZz/mmOdEJKBpnPnSQw3XZmr0Fw7AbHwplzVE0TBHaskjRYIuhPT0OTW l9s7dsQRzfUDISSON1srrehF+FrqqBVXd7+U37zX4n4hPc7ZVxsgITQbB1vmVpI54fcR YptWSZJzFepUwyt74d5AU8P8aJilr3NsvNRaS7a7GOqd2a2dwjbb93EmNmweq3+Wv+K0 Zo2xVABWvyBqm2Je8lOt14+vaoNU7EDVq2DcM6VndKF4nPyylZUipXv6hq0irdnffHfH Jal1AfFeTe+BXU+VeIWrfHiYl4oyQqTKMSNgfY10kvwYmE0C5Z4sMuwUMj2KUGkVrqm8 TnGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984895; x=1717589695; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=20VEPEn3Gb7BPD8Z30RwNAM2J68OYkR1rMftiYbdXdo=; b=Gx0DN8gIK6zD52mjN5Ee7TK3vxCEgUfUGvdqrZiTqQeW5B2pxq3LxOFWvyZL7v/zCl bKjdkGiEgNHo9rt1zi9Vcvi5fVGbi6S2nEnGuhs2faHuqYnTONVrc446DH0Pc93cWPPE h2CkNsAX+wsA3LESV74mNkknpU59AhIKZj8I2t9cLRlVL0HAMYfCgXXoWVb7K9SKrSZQ 12MMPRGDJsDqTyVowYMzchjv27kP89ooFagSKwf6qTRIGr6zzH3LZYdq8/A6KvDbw+ZI 8aDK6tj0NS6HXdtfml7LqCHBmC63fVFSaKEO4GXfa6a4+z+l/OvwM92mNIKRSD+VTYMa fnuQ== X-Forwarded-Encrypted: i=1; AJvYcCUhRRI9t/YD4Bb5cBsB7/vI4lFy0KcSTh9xKDIo3UpanSubpZR2GEjk4clw8hgEhgeeKCZBBhzJxB4aANyC98w93Gol X-Gm-Message-State: AOJu0YwxSJOXhZitpO4HKLCwGjzl7V0mESPs5Bts+PwCSsV/L69qS5Pq oVFaDA7nHwfibc18anFO0hqBOgAHYD3Ryg3/+045oYNWV/las82Cjr0ClkVNn+YnBMs6s62yWQ= = X-Google-Smtp-Source: AGHT+IHG1MR1oKr0DJdz40/0WuzMnmz17jEaYp2Xsp5D6m+QC6YtrLkmBnRMnvH0qBf1kHCVYvlptcFOhA== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a17:906:5842:b0:a59:aa99:694c with SMTP id a640c23a62f3a-a62649db3c6mr1603366b.8.1716984895098; Wed, 29 May 2024 05:14:55 -0700 (PDT) Date: Wed, 29 May 2024 13:12:17 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-12-ptosi@google.com> Subject: [PATCH v4 11/13] KVM: arm64: Improve CONFIG_CFI_CLANG error message From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort For kCFI, the compiler encodes in the immediate of the BRK (which the CPU places in ESR_ELx) the indices of the two registers it used to hold (resp.) the function pointer and expected type. Therefore, the kCFI handler must be able to parse the contents of the register file at the point where the exception was triggered. To achieve this, introduce a new hypervisor panic path that first stores the CPU context in the per-CPU kvm_hyp_ctxt before calling (directly or indirectly) hyp_panic() and execute it from all EL2 synchronous exception handlers i.e. - call it directly in host_el2_sync_vect (__kvm_hyp_host_vector, EL2t&h) - call it directly in el2t_sync_invalid (__kvm_hyp_vector, EL2t) - set ELR_EL2 to it in el2_sync (__kvm_hyp_vector, EL2h), which ERETs Teach hyp_panic() to decode the kCFI ESR and extract the target and type from the saved CPU context. In VHE, use that information to panic() with a specialized error message. In nVHE, only report it if the host (EL1) has access to the saved CPU context i.e. iff CONFIG_NVHE_EL2_DEBUG=y, which aligns with the behavior of CONFIG_PROTECTED_NVHE_STACKTRACE. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/handle_exit.c | 30 +++++++++++++++++++++++-- arch/arm64/kvm/hyp/entry.S | 24 +++++++++++++++++++- arch/arm64/kvm/hyp/hyp-entry.S | 2 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/host.S | 2 +- arch/arm64/kvm/hyp/vhe/switch.c | 26 +++++++++++++++++++-- 6 files changed, 79 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 69b08ac7322d..2fac3be3db00 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -26,6 +26,8 @@ #define CREATE_TRACE_POINTS #include "trace_handle_exit.h" +DECLARE_KVM_NVHE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt); + typedef int (*exit_handle_fn)(struct kvm_vcpu *); static void kvm_handle_guest_serror(struct kvm_vcpu *vcpu, u64 esr) @@ -417,10 +419,34 @@ static void print_nvhe_hyp_panic(const char *name, u64 panic_addr) (void *)(panic_addr + kaslr_offset())); } -static void kvm_nvhe_report_cfi_failure(u64 panic_addr) +static void kvm_nvhe_report_cfi_target(struct user_pt_regs *regs, u64 esr, + u64 hyp_offset) +{ + u64 va_mask = GENMASK_ULL(vabits_actual - 1, 0); + u8 type_idx = FIELD_GET(CFI_BRK_IMM_TYPE, esr); + u8 target_idx = FIELD_GET(CFI_BRK_IMM_TARGET, esr); + u32 expected_type = (u32)regs->regs[type_idx]; + u64 target_addr = (regs->regs[target_idx] & va_mask) + hyp_offset; + + kvm_err(" (target: [<%016llx>] %ps, expected type: 0x%08x)\n", + target_addr, (void *)(target_addr + kaslr_offset()), + expected_type); +} + +static void kvm_nvhe_report_cfi_failure(u64 panic_addr, u64 esr, u64 hyp_offset) { + struct user_pt_regs *regs = NULL; + print_nvhe_hyp_panic("CFI failure", panic_addr); + if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG) || !is_protected_kvm_enabled()) + regs = &this_cpu_ptr_nvhe_sym(kvm_hyp_ctxt)->regs; + + if (regs) + kvm_nvhe_report_cfi_target(regs, esr, hyp_offset); + else + kvm_err(" (no target information: !CONFIG_NVHE_EL2_DEBUG)\n"); + if (IS_ENABLED(CONFIG_CFI_PERMISSIVE)) kvm_err(" (CONFIG_CFI_PERMISSIVE ignored for hyp failures)\n"); } @@ -455,7 +481,7 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, else print_nvhe_hyp_panic("BUG", panic_addr); } else if (IS_ENABLED(CONFIG_CFI_CLANG) && esr_is_cfi_brk(esr)) { - kvm_nvhe_report_cfi_failure(panic_addr); + kvm_nvhe_report_cfi_failure(panic_addr, esr, hyp_offset); } else { print_nvhe_hyp_panic("panic", panic_addr); } diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index 343851c17373..8965dbc75972 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,7 +83,7 @@ alternative_else_nop_endif eret sb -SYM_INNER_LABEL(__hyp_restore_elr_and_panic, SYM_L_GLOBAL) +SYM_INNER_LABEL(__hyp_restore_elr_save_context_and_panic, SYM_L_GLOBAL) // x2-x29,lr: vcpu regs // vcpu x0-x1 on the stack @@ -91,6 +91,28 @@ SYM_INNER_LABEL(__hyp_restore_elr_and_panic, SYM_L_GLOBAL) ldr x0, [x0, #CPU_ELR_EL2] msr elr_el2, x0 +SYM_INNER_LABEL(__hyp_save_context_and_panic, SYM_L_GLOBAL) + // x0-x29,lr: hyp regs + + stp x0, x1, [sp, #-16]! + + adr_this_cpu x0, kvm_hyp_ctxt, x1 + + stp x2, x3, [x0, #CPU_XREG_OFFSET(2)] + + ldp x2, x3, [sp], #16 + + stp x2, x3, [x0, #CPU_XREG_OFFSET(0)] + stp x4, x5, [x0, #CPU_XREG_OFFSET(4)] + stp x6, x7, [x0, #CPU_XREG_OFFSET(6)] + stp x8, x9, [x0, #CPU_XREG_OFFSET(8)] + stp x10, x11, [x0, #CPU_XREG_OFFSET(10)] + stp x12, x13, [x0, #CPU_XREG_OFFSET(12)] + stp x14, x15, [x0, #CPU_XREG_OFFSET(14)] + stp x16, x17, [x0, #CPU_XREG_OFFSET(16)] + + save_callee_saved_regs x0 + SYM_INNER_LABEL(__hyp_panic, SYM_L_GLOBAL) // x2-x29,lr: vcpu regs // vcpu x0-x1 on the stack diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 7e65ef738ec9..d0d90d598338 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -130,7 +130,7 @@ SYM_CODE_END(\label) .endm /* None of these should ever happen */ - invalid_vector el2t_sync_invalid + invalid_vector el2t_sync_invalid, __hyp_save_context_and_panic invalid_vector el2t_irq_invalid invalid_vector el2t_fiq_invalid invalid_vector el2t_error_invalid diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index d9931abf14c2..77783dbc1833 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -689,7 +689,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __kvm_unexpected_el2_exception(void) { - extern char __hyp_restore_elr_and_panic[]; + extern char __hyp_restore_elr_save_context_and_panic[]; unsigned long addr, fixup; struct kvm_exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); @@ -712,7 +712,7 @@ static inline void __kvm_unexpected_el2_exception(void) /* Trigger a panic after restoring the hyp context. */ this_cpu_ptr(&kvm_hyp_ctxt)->sys_regs[ELR_EL2] = elr_el2; - write_sysreg(__hyp_restore_elr_and_panic, elr_el2); + write_sysreg(__hyp_restore_elr_save_context_and_panic, elr_el2); } #endif /* __ARM64_KVM_HYP_SWITCH_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index a7db40a51e4a..9343160f5357 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -214,7 +214,7 @@ SYM_FUNC_END(__host_hvc) .endm .macro host_el2_sync_vect - __host_el2_vect __hyp_panic + __host_el2_vect __hyp_save_context_and_panic .endm .macro invalid_host_el1_vect diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 0550b9f6317f..6c64783c3e00 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -17,6 +17,7 @@ #include #include +#include #include #include #include @@ -384,7 +385,24 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) return ret; } -static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) +static void __noreturn __hyp_call_panic_for_cfi(u64 elr, u64 esr) +{ + struct user_pt_regs *regs = &this_cpu_ptr(&kvm_hyp_ctxt)->regs; + u8 type_idx = FIELD_GET(CFI_BRK_IMM_TYPE, esr); + u8 target_idx = FIELD_GET(CFI_BRK_IMM_TARGET, esr); + u32 expected_type = (u32)regs->regs[type_idx]; + u64 target = regs->regs[target_idx]; + + panic("VHE hyp CFI failure at: [<%016llx>] %pB (target: [<%016llx>] %ps, expected type: 0x%08x)\n" +#ifdef CONFIG_CFI_PERMISSIVE + " (CONFIG_CFI_PERMISSIVE ignored for hyp failures)\n" +#endif + , + elr, (void *)elr, target, (void *)target, expected_type); +} +NOKPROBE_SYMBOL(__hyp_call_panic_for_cfi); + +static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par, u64 esr) { struct kvm_cpu_context *host_ctxt; struct kvm_vcpu *vcpu; @@ -395,6 +413,9 @@ static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) __deactivate_traps(vcpu); sysreg_restore_host_state_vhe(host_ctxt); + if (IS_ENABLED(CONFIG_CFI_CLANG) && esr_is_cfi_brk(esr)) + __hyp_call_panic_for_cfi(elr, esr); + panic("HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n", spsr, elr, read_sysreg_el2(SYS_ESR), read_sysreg_el2(SYS_FAR), @@ -407,8 +428,9 @@ void __noreturn hyp_panic(void) u64 spsr = read_sysreg_el2(SYS_SPSR); u64 elr = read_sysreg_el2(SYS_ELR); u64 par = read_sysreg_par(); + u64 esr = read_sysreg_el2(SYS_ESR); - __hyp_call_panic(spsr, elr, par); + __hyp_call_panic(spsr, elr, par, esr); } asmlinkage void kvm_unexpected_el2_exception(void) From patchwork Wed May 29 12:12:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678761 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5C45B180A92 for ; Wed, 29 May 2024 12:14:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984900; cv=none; b=ETIO2JX3aX6pkZt7zZnYCKUjSS4RRyzI+nD4ZGJGaqiEuDWzA98avW0hdc/KM3H9Ukih8eewpX31EqJqjzXWocnICptTZMFEcJNAlmQcqOosMYgBO3lftg7zFJzNqr8uJ018m6ELfM0U7IVlJMaQTNeIfO97vrQOgSDj748JzuY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984900; c=relaxed/simple; bh=w/OwqY7DTBtD/BmP+AMgJ3tEunuTuRYCMp3ThRXQ3OU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Cq1FdQFYixM6OwR0jVR7n4XVfsBdXBy6vJHZouoe7kj8QzmnyvC5uw91bkBoWl2ShwUqEk9G6ItrTB88DMr9WR40VSh/U4/10ykqICn8sYLpT3Nba6IRHA7+intZCogjTNEPW+EaleLpNCsH2lXBzlXRZKotZhYwEGNLJXE4QTQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=NHSO0RzI; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NHSO0RzI" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-df78acd5bbbso3131293276.1 for ; Wed, 29 May 2024 05:14:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984897; x=1717589697; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=uRgrsfnu4k+NI/qYYv/tuJDICq7UcPQuz5rX48+teys=; b=NHSO0RzIMbMoowgkwXbrtlWFvqNImqpSD7G6cwgLpHdp488EvCWfORJCFZ6j4zCo85 YNL0HrcyTU4thAGDoUNRQr/mLsp9F9QrV9iMfHsMAkCytx8uJNR6HbkF2ji7Pa81HSVc KwVhcAz3esbVP5bsWX3CfzYK5LKO8A2ZNquQ7mdM9QheQsj5ZeVAWpnuLz1Zo8Pf273X hmOg1lV2rPyDXEtQWYczIH3bsz8OYyf3Uprtr+qV6roHJ49k7CxUUKb2B+HaWslhtZEB zXgE3MYXiYz1Px3/1yjUmBkqmet5Sc0TnCK9KYeLz4P4b11xHjQARgwsLFYjsGQoq+xt Ak0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984897; x=1717589697; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=uRgrsfnu4k+NI/qYYv/tuJDICq7UcPQuz5rX48+teys=; b=O0GqUqo8FwHND2uVJZNPdSrzKjZe5DsJ8EW8IE9wqnsS8vyPJz9LU5UQ9qaaZEXgQ+ Yxedd/2ka8iO8nVkTK9bWdAJTiPNKmRVk4106rGw1B8kleWI0tDmXf2vQXZEXP09btxT +kxDaQYvbyNS0EdGB5mMaxfnw9DZakXsGyPCsgu+sWP/1Hk/r6AsY7mNU5MMC/yMCsNp j12sHnxtM3qzya3cCtJyb/LRyU1FMVOExBMse6UweqIxcgfkZHOmlwF7pSez1tPErQFK mutmqp6p/3Ewxhvf70aydtrpYzvLXpGL/8CP8yzPDlorVGKD2f5MQYEMpfNjHGTSmC5f iGbw== X-Forwarded-Encrypted: i=1; AJvYcCX55KaJTR1nCNDhbv60vkcTBnnIEGeRm1N+EyYwHt5UR05xufwYoWpFBH+tHJClvaqgkE+Mx+Kw7hxvO2J6hSLUuvoV X-Gm-Message-State: AOJu0YwEKUR66znjiNWFuB4ngVW7y1eA5IFgMpGjsG5rpfqAvaRXQmrT BAE4K25hvEJMDQfYkF3JP+a7JTy43DlSHNmsj+Qau/0jKwr5fTJnxVGOfKbYWHa7v47RUFAjZA= = X-Google-Smtp-Source: AGHT+IFkQb5OxQ4pScFZIWRQEFsGIMu9g6HuNiBaUgrqAIayPXl97t9kWzQUkwXiZZdHCWHOqYzW82AA/Q== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6902:1207:b0:dee:7884:acc7 with SMTP id 3f1490d57ef6-df77214fd6dmr1189597276.1.1716984897529; Wed, 29 May 2024 05:14:57 -0700 (PDT) Date: Wed, 29 May 2024 13:12:18 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-13-ptosi@google.com> Subject: [PATCH v4 12/13] KVM: arm64: VHE: Add test module for hyp kCFI From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort In order to easily periodically (and potentially automatically) validate that the hypervisor kCFI feature doesn't bitrot, introduce a way to trigger hypervisor kCFI faults from userspace on test builds of KVM. Add hooks in the hypervisor code to call registered callbacks (intended to trigger kCFI faults either for the callback call itself of from within the callback function) when running with guest or host VBAR_EL2. As the calls are issued from the KVM_RUN ioctl handling path, userspace gains control over when the actual triggering of the fault happens without needing to modify the KVM uAPI. Export kernel functions to register these callbacks from modules and introduce a kernel module intended to contain any testing logic. By limiting the changes to the core kernel to a strict minimum, this architectural split allows tests to be updated (within the module) without the need to redeploy (or recompile) the kernel (hyp) under test. Use the module parameters as the uAPI for configuring the fault condition being tested (i.e. either at insertion or post-insertion using /sys/module/.../parameters), which naturally makes it impossible for userspace to test kCFI without the module (and, inversely, makes the module only - not KVM - responsible for exposing said uAPI). As kCFI is implemented with a caller-side check of a callee-side value, make the module support 4 tests based on the location of the caller and callee (built-in or in-module), for each of the 2 hypervisor contexts (host & guest), selected by userspace using the 'guest' or 'host' module parameter. For this purpose, export symbols which the module can use to configure the callbacks for in-kernel and module-to-built-in kCFI faulting calls. Define the module-to-kernel API to allow the module to detect that it was loaded on a kernel built with support for it but which is running without a hypervisor (-ENXIO) or with one that doesn't use the VHE CPU feature (-EOPNOTSUPP), which is currently the only mode for which KVM supports hypervisor kCFI. Allow kernel build configs to set CONFIG_HYP_CFI_TEST to only support the in-kernel hooks (=y) or also build the test module (=m). Use intermediate internal Kconfig flags (CONFIG_HYP_SUPPORTS_CFI_TEST and CONFIG_HYP_CFI_TEST_MODULE) to simplify the Makefiles and #ifdefs. As the symbols for callback registration are only exported to modules when CONFIG_HYP_CFI_TEST != n, it is impossible for the test module to be non-forcefully inserted on a kernel that doesn't support it. Note that this feature must NOT result in any noticeable change (behavioral or binary size) when HYP_CFI_TEST_MODULE = n. CONFIG_HYP_CFI_TEST is intentionally independent of CONFIG_CFI_CLANG, to avoid arbitrarily limiting the number of flag combinations that can be tested with the module. Also note that, as VHE aliases VBAR_EL1 to VBAR_EL2 for the host, testing hypervisor kCFI in VHE and in host context is equivalent to testing kCFI support of the kernel itself i.e. EL1 in non-VHE and/or in non-virtualized environments. For this reason, CONFIG_CFI_PERMISSIVE **will** prevent the test module from triggering a hyp panic (although a warning still gets printed) in that context. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/kvm_cfi.h | 36 ++++++++ arch/arm64/kvm/Kconfig | 22 +++++ arch/arm64/kvm/Makefile | 3 + arch/arm64/kvm/hyp/include/hyp/cfi.h | 47 ++++++++++ arch/arm64/kvm/hyp/vhe/Makefile | 1 + arch/arm64/kvm/hyp/vhe/cfi.c | 37 ++++++++ arch/arm64/kvm/hyp/vhe/switch.c | 7 ++ arch/arm64/kvm/hyp_cfi_test.c | 43 +++++++++ arch/arm64/kvm/hyp_cfi_test_module.c | 133 +++++++++++++++++++++++++++ 9 files changed, 329 insertions(+) create mode 100644 arch/arm64/include/asm/kvm_cfi.h create mode 100644 arch/arm64/kvm/hyp/include/hyp/cfi.h create mode 100644 arch/arm64/kvm/hyp/vhe/cfi.c create mode 100644 arch/arm64/kvm/hyp_cfi_test.c create mode 100644 arch/arm64/kvm/hyp_cfi_test_module.c diff --git a/arch/arm64/include/asm/kvm_cfi.h b/arch/arm64/include/asm/kvm_cfi.h new file mode 100644 index 000000000000..13cc7b19d838 --- /dev/null +++ b/arch/arm64/include/asm/kvm_cfi.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ + +#ifndef __ARM64_KVM_CFI_H__ +#define __ARM64_KVM_CFI_H__ + +#include +#include + +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + +int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)); +int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)); + +#else + +static inline int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)) +{ + return -EOPNOTSUPP; +} + +static inline int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) +{ + return -EOPNOTSUPP; +} + +#endif /* CONFIG_HYP_SUPPORTS_CFI_TEST */ + +/* Symbols which the host can register as hyp callbacks; see . */ +void hyp_trigger_builtin_cfi_fault(void); +void hyp_builtin_cfi_fault_target(int unused); + +#endif /* __ARM64_KVM_CFI_H__ */ diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 58f09370d17e..5daa8079a120 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -65,4 +65,26 @@ config PROTECTED_NVHE_STACKTRACE If unsure, or not using protected nVHE (pKVM), say N. +config HYP_CFI_TEST + tristate "KVM hypervisor kCFI test support" + depends on KVM + help + Say Y or M here to build KVM with test hooks to support intentionally + triggering hypervisor kCFI faults in guest or host context. + + Say M here to also build a module which registers callbacks triggering + faults and selected by userspace through its parameters. + + Note that this feature is currently only supported in VHE mode. + + If unsure, say N. + +config HYP_SUPPORTS_CFI_TEST + def_bool y + depends on HYP_CFI_TEST + +config HYP_CFI_TEST_MODULE + def_tristate m if HYP_CFI_TEST = m + depends on HYP_CFI_TEST + endif # VIRTUALIZATION diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index a6497228c5a8..303be42ad90a 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -22,6 +22,7 @@ kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \ vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \ vgic/vgic-its.o vgic/vgic-debug.o +kvm-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += hyp_cfi_test.o kvm-$(CONFIG_HW_PERF_EVENTS) += pmu-emul.o pmu.o kvm-$(CONFIG_ARM64_PTR_AUTH) += pauth.o @@ -40,3 +41,5 @@ $(obj)/hyp_constants.h: $(obj)/hyp-constants.s FORCE obj-kvm := $(addprefix $(obj)/, $(kvm-y)) $(obj-kvm): $(obj)/hyp_constants.h + +obj-$(CONFIG_HYP_CFI_TEST_MODULE) += hyp_cfi_test_module.o diff --git a/arch/arm64/kvm/hyp/include/hyp/cfi.h b/arch/arm64/kvm/hyp/include/hyp/cfi.h new file mode 100644 index 000000000000..c6536040bc06 --- /dev/null +++ b/arch/arm64/kvm/hyp/include/hyp/cfi.h @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ + +#ifndef __ARM64_KVM_HYP_CFI_H__ +#define __ARM64_KVM_HYP_CFI_H__ + +#include +#include + +#include + +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + +int __kvm_register_cfi_test_cb(void (*cb)(void), bool in_host_ctxt); + +extern void (*hyp_test_host_ctxt_cfi)(void); +extern void (*hyp_test_guest_ctxt_cfi)(void); + +/* Hypervisor callbacks for the host to register. */ +void hyp_trigger_builtin_cfi_fault(void); +void hyp_builtin_cfi_fault_target(int unused); + +#else + +static inline +int __kvm_register_cfi_test_cb(void (*cb)(void), bool in_host_ctxt) +{ + return -EOPNOTSUPP; +} + +#define hyp_test_host_ctxt_cfi ((void(*)(void))(NULL)) +#define hyp_test_guest_ctxt_cfi ((void(*)(void))(NULL)) + +static inline void hyp_trigger_builtin_cfi_fault(void) +{ +} + +static inline void hyp_builtin_cfi_fault_target(int __always_unused unused) +{ +} + +#endif /* CONFIG_HYP_SUPPORTS_CFI_TEST */ + +#endif /* __ARM64_KVM_HYP_CFI_H__ */ diff --git a/arch/arm64/kvm/hyp/vhe/Makefile b/arch/arm64/kvm/hyp/vhe/Makefile index 3b9e5464b5b3..19ca584cc21e 100644 --- a/arch/arm64/kvm/hyp/vhe/Makefile +++ b/arch/arm64/kvm/hyp/vhe/Makefile @@ -9,3 +9,4 @@ ccflags-y := -D__KVM_VHE_HYPERVISOR__ obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o +obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += cfi.o diff --git a/arch/arm64/kvm/hyp/vhe/cfi.c b/arch/arm64/kvm/hyp/vhe/cfi.c new file mode 100644 index 000000000000..5849f239e27f --- /dev/null +++ b/arch/arm64/kvm/hyp/vhe/cfi.c @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ +#include + +#include + +void (*hyp_test_host_ctxt_cfi)(void); +void (*hyp_test_guest_ctxt_cfi)(void); + +int __kvm_register_cfi_test_cb(void (*cb)(void), bool in_host_ctxt) +{ + if (in_host_ctxt) + hyp_test_host_ctxt_cfi = cb; + else + hyp_test_guest_ctxt_cfi = cb; + + return 0; +} + +void hyp_builtin_cfi_fault_target(int __always_unused unused) +{ +} + +void hyp_trigger_builtin_cfi_fault(void) +{ + /* Intentional UB cast & dereference, to trigger a kCFI fault. */ + void (*target)(void) = (void *)&hyp_builtin_cfi_fault_target; + + /* + * READ_ONCE() prevents this indirect call from being optimized out, + * forcing the compiler to generate the kCFI check before the branch. + */ + READ_ONCE(target)(); +} diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 6c64783c3e00..fe70220876b4 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -4,6 +4,7 @@ * Author: Marc Zyngier */ +#include #include #include @@ -311,6 +312,9 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) struct kvm_cpu_context *guest_ctxt; u64 exit_code; + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_host_ctxt_cfi)) + hyp_test_host_ctxt_cfi(); + host_ctxt = host_data_ptr(host_ctxt); guest_ctxt = &vcpu->arch.ctxt; @@ -329,6 +333,9 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) sysreg_restore_guest_state_vhe(guest_ctxt); __debug_switch_to_guest(vcpu); + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_guest_ctxt_cfi)) + hyp_test_guest_ctxt_cfi(); + do { /* Jump in the fire! */ exit_code = __guest_enter(vcpu); diff --git a/arch/arm64/kvm/hyp_cfi_test.c b/arch/arm64/kvm/hyp_cfi_test.c new file mode 100644 index 000000000000..da7b25ca1b1f --- /dev/null +++ b/arch/arm64/kvm/hyp_cfi_test.c @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ +#include +#include +#include +#include + +#include +#include +#include + +/* For calling directly into the VHE hypervisor; see . */ +int __kvm_register_cfi_test_cb(void (*)(void), bool); + +static int kvm_register_cfi_test_cb(void (*vhe_cb)(void), bool in_host_ctxt) +{ + if (!is_hyp_mode_available()) + return -ENXIO; + + if (is_hyp_nvhe()) + return -EOPNOTSUPP; + + return __kvm_register_cfi_test_cb(vhe_cb, in_host_ctxt); +} + +int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)) +{ + return kvm_register_cfi_test_cb(cb, true); +} +EXPORT_SYMBOL(kvm_cfi_test_register_host_ctxt_cb); + +int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) +{ + return kvm_register_cfi_test_cb(cb, false); +} +EXPORT_SYMBOL(kvm_cfi_test_register_guest_ctxt_cb); + +/* Hypervisor callbacks for the test module to register. */ +EXPORT_SYMBOL(hyp_trigger_builtin_cfi_fault); +EXPORT_SYMBOL(hyp_builtin_cfi_fault_target); diff --git a/arch/arm64/kvm/hyp_cfi_test_module.c b/arch/arm64/kvm/hyp_cfi_test_module.c new file mode 100644 index 000000000000..eeda4be4d3ef --- /dev/null +++ b/arch/arm64/kvm/hyp_cfi_test_module.c @@ -0,0 +1,133 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include + +#include +#include +#include +#include + +static int set_host_mode(const char *val, const struct kernel_param *kp); +static int set_guest_mode(const char *val, const struct kernel_param *kp); + +#define M_DESC \ + "\n\t0: none" \ + "\n\t1: built-in caller & built-in callee" \ + "\n\t2: built-in caller & module callee" \ + "\n\t3: module caller & built-in callee" \ + "\n\t4: module caller & module callee" + +static unsigned int host_mode; +module_param_call(host, set_host_mode, param_get_uint, &host_mode, 0644); +MODULE_PARM_DESC(host, + "Hypervisor kCFI fault test case in host context:" M_DESC); + +static unsigned int guest_mode; +module_param_call(guest, set_guest_mode, param_get_uint, &guest_mode, 0644); +MODULE_PARM_DESC(guest, + "Hypervisor kCFI fault test case in guest context:" M_DESC); + +static void trigger_module2module_cfi_fault(void); +static void trigger_module2builtin_cfi_fault(void); +static void hyp_cfi_module2module_test_target(int); +static void hyp_cfi_builtin2module_test_target(int); + +static int set_param_mode(const char *val, const struct kernel_param *kp, + int (*register_cb)(void (*)(void))) +{ + unsigned int *mode = kp->arg; + int err; + + err = param_set_uint(val, kp); + if (err) + return err; + + switch (*mode) { + case 0: + return register_cb(NULL); + case 1: + return register_cb(hyp_trigger_builtin_cfi_fault); + case 2: + return register_cb((void *)hyp_cfi_builtin2module_test_target); + case 3: + return register_cb(trigger_module2builtin_cfi_fault); + case 4: + return register_cb(trigger_module2module_cfi_fault); + default: + return -EINVAL; + } +} + +static int set_host_mode(const char *val, const struct kernel_param *kp) +{ + return set_param_mode(val, kp, kvm_cfi_test_register_host_ctxt_cb); +} + +static int set_guest_mode(const char *val, const struct kernel_param *kp) +{ + return set_param_mode(val, kp, kvm_cfi_test_register_guest_ctxt_cb); +} + +static void __exit exit_hyp_cfi_test(void) +{ + int err; + + err = kvm_cfi_test_register_host_ctxt_cb(NULL); + if (err) + pr_err("Failed to unregister host context trigger: %d\n", err); + + err = kvm_cfi_test_register_guest_ctxt_cb(NULL); + if (err) + pr_err("Failed to unregister guest context trigger: %d\n", err); +} +module_exit(exit_hyp_cfi_test); + +static void trigger_module2builtin_cfi_fault(void) +{ + /* Intentional UB cast & dereference, to trigger a kCFI fault. */ + void (*target)(void) = (void *)&hyp_builtin_cfi_fault_target; + + /* + * READ_ONCE() prevents this indirect call from being optimized out, + * forcing the compiler to generate the kCFI check before the branch. + */ + READ_ONCE(target)(); + + pr_err_ratelimited("%s: Survived a kCFI violation\n", __func__); +} + +static void trigger_module2module_cfi_fault(void) +{ + /* Intentional UB cast & dereference, to trigger a kCFI fault. */ + void (*target)(void) = (void *)&hyp_cfi_module2module_test_target; + + /* + * READ_ONCE() prevents this indirect call from being optimized out, + * forcing the compiler to generate the kCFI check before the branch. + */ + READ_ONCE(target)(); + + pr_err_ratelimited("%s: Survived a kCFI violation\n", __func__); +} + +/* Use different functions, for clearer symbols in kCFI panic reports. */ +static noinline +void hyp_cfi_module2module_test_target(int __always_unused unused) +{ +} + +static noinline +void hyp_cfi_builtin2module_test_target(int __always_unused unused) +{ +} + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Pierre-Clément Tosi "); +MODULE_DESCRIPTION("KVM hypervisor kCFI test module"); From patchwork Wed May 29 12:12:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678762 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 86357180A97 for ; Wed, 29 May 2024 12:15:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984902; cv=none; b=OvQEMWKO5OcA2UW28nvLZD5qeYlWx0addCJNCJy4u801kLBP8+zWPFEj/lMxQ1fnW0B9cpQOfElYIpKgwKwu3a5KChu7Ss91H6GGaa48EWD3JNoTo9JYfL5GBXrCoDIZUQjnDb+PJvbRT9WJY6ikW/yI8IuIVEvorsH2lEKz9+o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716984902; c=relaxed/simple; bh=aNx67NqhbVPDIwlRTUNci0pmePVybS/Mw7w05xuUt0A=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gjCdp578Ix+S/38oEPxJx6GXJdIscBo+N9Y7zlZlwdlJvnQ0lXDXN59q/CyroX/aIPm15aiomNP9Prvb9xGNun6Buwhwsmxjf3DjxAsq2POp3Cm+dbFNEJOZMm2fBGga0LfGpk/u24BryF8uqQNAA41tiBsqIooFzu92HVk+WWM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=zrAneZ4Q; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ptosi.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="zrAneZ4Q" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-627e7f0ca54so36733497b3.3 for ; Wed, 29 May 2024 05:15:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984900; x=1717589700; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=2WYr4Xqu6SHeaXEynQRTMZNyZEDRWLI9UmjyxKqMZ9M=; b=zrAneZ4Qk+554+Sw75n8kBcwekhg2WzBQ0VepyzjZF5oDRVQyR3mkKD6OxxSc8S/CS T8XBFENIXjWuH3DnOUGSEz9aYptgZY3Rac7NU8Cc7MYLvb2wYRbBeyC0m44JituWBDjk zRF45KOVpSLXvDCZpMO2jEU0kirRV6CJehFsihNpDPcw6wOJztwXgKlVPgm4ONep8YlH N+VIbCrcNxQBjlUGYxPhVNUjsECERB1SvzAkAoJRoPFHdfAG/jqOyH7LqUO4K7yTyOVo EQeX/2NoP8rIn1Tl1ST+gQVW0puwYuqYP9opPefsD/PSWO8vdnP4zep3jZcQCzEyJ9F8 MCdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984900; x=1717589700; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=2WYr4Xqu6SHeaXEynQRTMZNyZEDRWLI9UmjyxKqMZ9M=; b=UonOkdSV05mzaUu078um3xt8sDYLeXBE0g64RmnVxGIxCsnYJXa3ghpLMzrBTsY7mi A1fhMSPM1MDPOiAAF8bUiENG1xOI5+4NI+LhUsFwbSsmPmYXpSQBxCZrRsAbkS+wQslp XrkL5UCRxbXeW/3KiCy3qNwdWBUOUd3c4IXjKVYXELrUK62XCqei53W+aDi2doJBanRO 1nvNW56dX1UpT+nZvQWKFr8v//RdGoCGXRuEmvFXyMKz+VxnMSylJ/k1+xN5IsPIUHs0 x8ZpZ6c30fY+po1Q5cM3O8SSN1W2XSUSTFCsoUYD3Cp5HOQvXhYOqDqkJw4rlnpv39bL SBkw== X-Forwarded-Encrypted: i=1; AJvYcCUpotVSu6gvH+n9l52bEX7KaOf7G2hTXg/3jBUFCRN9cLGPTsDCx5cVavvaEAeYQv4Wlz7eF8nBaujSD7ykBbKE2YzJ X-Gm-Message-State: AOJu0Ywv+lZfX/2o6Qj6rHqu0xGA6A3zJo3YKUsF7RY+DKjQsw91zaaC SydPSeiAXyMF9BMr6suyaQZxERPdfUoqZ5xP+L+2Oz3f7Xnjy8S6w1DHHDHL3XT/ddzM2IRsnw= = X-Google-Smtp-Source: AGHT+IHHUoIUjFzO4GjkhNWRkyuptnIeiIsPtLJmFaYxcziwtscpK8PJLaAhdGcG5Z1QL/wRAezHUEYegw== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a25:ea14:0:b0:df7:8f43:f8a3 with SMTP id 3f1490d57ef6-df78f440daemr2830788276.0.1716984899780; Wed, 29 May 2024 05:14:59 -0700 (PDT) Date: Wed, 29 May 2024 13:12:19 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-14-ptosi@google.com> Subject: [PATCH v4 13/13] KVM: arm64: nVHE: Support test module for hyp kCFI From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Extend support for the kCFI test module to nVHE by replicating the hooks on the KVM_RUN handler path currently existing in VHE in the nVHE code, exporting the equivalent callback targets for triggering built-in hyp kCFI faults, and exposing a new CONFIG_HYP_CFI_TEST-only host HVC to implement callback registration. Update the test module to register the nVHE equivalent callback for test case '1' (i.e. both EL2 hyp caller and callee are built-in) and document that other cases are not supported outside of VHE, as they require EL2 symbols in the module, which is not currently supported for nVHE. Note that a kernel in protected mode that doesn't support HYP_CFI_TEST will prevent the module from registering nVHE callbacks both by not exporting the necessary symbols (similar to VHE) but also by rejecting the corresponding HVC, if the module tries to issue it directly. Also note that the test module will run in pKVM (with HYP_CFI_TEST) independently of other debug Kconfig flags but that not stacktrace will be printed without PROTECTED_NVHE_STACKTRACE. This allows testing kCFI under conditions closer to release builds, if desired. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/kvm_asm.h | 3 ++ arch/arm64/include/asm/kvm_cfi.h | 6 ++-- arch/arm64/kvm/Kconfig | 2 -- arch/arm64/kvm/hyp/{vhe => }/cfi.c | 0 arch/arm64/kvm/hyp/nvhe/Makefile | 1 + arch/arm64/kvm/hyp/nvhe/hyp-main.c | 19 ++++++++++++ arch/arm64/kvm/hyp/nvhe/switch.c | 7 +++++ arch/arm64/kvm/hyp/vhe/Makefile | 2 +- arch/arm64/kvm/hyp_cfi_test.c | 44 ++++++++++++++++++++++++---- arch/arm64/kvm/hyp_cfi_test_module.c | 24 ++++++++------- 10 files changed, 86 insertions(+), 22 deletions(-) rename arch/arm64/kvm/hyp/{vhe => }/cfi.c (100%) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index a6330460d9e5..791054492920 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -79,6 +79,9 @@ enum __kvm_host_smccc_func { __KVM_HOST_SMCCC_FUNC___pkvm_init_vm, __KVM_HOST_SMCCC_FUNC___pkvm_init_vcpu, __KVM_HOST_SMCCC_FUNC___pkvm_teardown_vm, +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + __KVM_HOST_SMCCC_FUNC___kvm_register_cfi_test_cb, +#endif }; #define DECLARE_KVM_VHE_SYM(sym) extern char sym[] diff --git a/arch/arm64/include/asm/kvm_cfi.h b/arch/arm64/include/asm/kvm_cfi.h index 13cc7b19d838..ed6422eebce5 100644 --- a/arch/arm64/include/asm/kvm_cfi.h +++ b/arch/arm64/include/asm/kvm_cfi.h @@ -12,8 +12,8 @@ #ifdef CONFIG_HYP_SUPPORTS_CFI_TEST -int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)); -int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)); +int kvm_cfi_test_register_host_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb); +int kvm_cfi_test_register_guest_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb); #else @@ -31,6 +31,8 @@ static inline int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) /* Symbols which the host can register as hyp callbacks; see . */ void hyp_trigger_builtin_cfi_fault(void); +DECLARE_KVM_NVHE_SYM(hyp_trigger_builtin_cfi_fault); void hyp_builtin_cfi_fault_target(int unused); +DECLARE_KVM_NVHE_SYM(hyp_builtin_cfi_fault_target); #endif /* __ARM64_KVM_CFI_H__ */ diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 5daa8079a120..715c85088c06 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -75,8 +75,6 @@ config HYP_CFI_TEST Say M here to also build a module which registers callbacks triggering faults and selected by userspace through its parameters. - Note that this feature is currently only supported in VHE mode. - If unsure, say N. config HYP_SUPPORTS_CFI_TEST diff --git a/arch/arm64/kvm/hyp/vhe/cfi.c b/arch/arm64/kvm/hyp/cfi.c similarity index 100% rename from arch/arm64/kvm/hyp/vhe/cfi.c rename to arch/arm64/kvm/hyp/cfi.c diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 782b34b004be..115aa8880260 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -25,6 +25,7 @@ hyp-obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o cache.o setup.o mm.o mem_protect.o sys_regs.o pkvm.o stacktrace.o ffa.o hyp-obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o ../pgtable.o +hyp-obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += ../cfi.o hyp-obj-$(CONFIG_LIST_HARDENED) += list_debug.o hyp-obj-y += $(lib-objs) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index d5c48dc98f67..39ed06fbb190 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -5,6 +5,7 @@ */ #include +#include #include #include @@ -13,6 +14,8 @@ #include #include +#include + #include #include #include @@ -301,6 +304,19 @@ static void handle___pkvm_teardown_vm(struct kvm_cpu_context *host_ctxt) cpu_reg(host_ctxt, 1) = __pkvm_teardown_vm(handle); } +#ifndef CONFIG_HYP_SUPPORTS_CFI_TEST +__always_unused +#endif +static void handle___kvm_register_cfi_test_cb(struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(phys_addr_t, cb_phys, host_ctxt, 1); + DECLARE_REG(bool, in_host_ctxt, host_ctxt, 2); + + void (*cb)(void) = cb_phys ? __hyp_va(cb_phys) : NULL; + + cpu_reg(host_ctxt, 1) = __kvm_register_cfi_test_cb(cb, in_host_ctxt); +} + typedef void (*hcall_t)(struct kvm_cpu_context *); #define HANDLE_FUNC(x) [__KVM_HOST_SMCCC_FUNC_##x] = (hcall_t)handle_##x @@ -333,6 +349,9 @@ static const hcall_t host_hcall[] = { HANDLE_FUNC(__pkvm_init_vm), HANDLE_FUNC(__pkvm_init_vcpu), HANDLE_FUNC(__pkvm_teardown_vm), +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + HANDLE_FUNC(__kvm_register_cfi_test_cb), +#endif }; static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index 6758cd905570..52d2fada9e19 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -4,6 +4,7 @@ * Author: Marc Zyngier */ +#include #include #include @@ -249,6 +250,9 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) bool pmu_switch_needed; u64 exit_code; + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_host_ctxt_cfi)) + hyp_test_host_ctxt_cfi(); + /* * Having IRQs masked via PMR when entering the guest means the GIC * will not signal the CPU of interrupts of lower priority, and the @@ -309,6 +313,9 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) __debug_switch_to_guest(vcpu); + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_guest_ctxt_cfi)) + hyp_test_guest_ctxt_cfi(); + do { /* Jump in the fire! */ exit_code = __guest_enter(vcpu); diff --git a/arch/arm64/kvm/hyp/vhe/Makefile b/arch/arm64/kvm/hyp/vhe/Makefile index 19ca584cc21e..951c8c00a685 100644 --- a/arch/arm64/kvm/hyp/vhe/Makefile +++ b/arch/arm64/kvm/hyp/vhe/Makefile @@ -9,4 +9,4 @@ ccflags-y := -D__KVM_VHE_HYPERVISOR__ obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o -obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += cfi.o +obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += ../cfi.o diff --git a/arch/arm64/kvm/hyp_cfi_test.c b/arch/arm64/kvm/hyp_cfi_test.c index da7b25ca1b1f..6a02b43c45f6 100644 --- a/arch/arm64/kvm/hyp_cfi_test.c +++ b/arch/arm64/kvm/hyp_cfi_test.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include @@ -15,29 +16,60 @@ /* For calling directly into the VHE hypervisor; see . */ int __kvm_register_cfi_test_cb(void (*)(void), bool); -static int kvm_register_cfi_test_cb(void (*vhe_cb)(void), bool in_host_ctxt) +static int kvm_register_nvhe_cfi_test_cb(void *cb, bool in_host_ctxt) +{ + extern void *kvm_nvhe_sym(hyp_test_host_ctxt_cfi); + extern void *kvm_nvhe_sym(hyp_test_guest_ctxt_cfi); + + if (is_protected_kvm_enabled()) { + phys_addr_t cb_phys = cb ? virt_to_phys(cb) : 0; + + /* Use HVC as only the hyp can modify its callback pointers. */ + return kvm_call_hyp_nvhe(__kvm_register_cfi_test_cb, cb_phys, + in_host_ctxt); + } + + /* + * In non-protected nVHE, the pKVM HVC is not available but the + * hyp callback pointers can be accessed and modified directly. + */ + if (cb) + cb = kern_hyp_va(kvm_ksym_ref(cb)); + + if (in_host_ctxt) + kvm_nvhe_sym(hyp_test_host_ctxt_cfi) = cb; + else + kvm_nvhe_sym(hyp_test_guest_ctxt_cfi) = cb; + + return 0; +} + +static int kvm_register_cfi_test_cb(void (*vhe_cb)(void), void *nvhe_cb, + bool in_host_ctxt) { if (!is_hyp_mode_available()) return -ENXIO; if (is_hyp_nvhe()) - return -EOPNOTSUPP; + return kvm_register_nvhe_cfi_test_cb(nvhe_cb, in_host_ctxt); return __kvm_register_cfi_test_cb(vhe_cb, in_host_ctxt); } -int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)) +int kvm_cfi_test_register_host_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb) { - return kvm_register_cfi_test_cb(cb, true); + return kvm_register_cfi_test_cb(vhe_cb, nvhe_cb, true); } EXPORT_SYMBOL(kvm_cfi_test_register_host_ctxt_cb); -int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) +int kvm_cfi_test_register_guest_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb) { - return kvm_register_cfi_test_cb(cb, false); + return kvm_register_cfi_test_cb(vhe_cb, nvhe_cb, false); } EXPORT_SYMBOL(kvm_cfi_test_register_guest_ctxt_cb); /* Hypervisor callbacks for the test module to register. */ EXPORT_SYMBOL(hyp_trigger_builtin_cfi_fault); +EXPORT_SYMBOL(kvm_nvhe_sym(hyp_trigger_builtin_cfi_fault)); EXPORT_SYMBOL(hyp_builtin_cfi_fault_target); +EXPORT_SYMBOL(kvm_nvhe_sym(hyp_builtin_cfi_fault_target)); diff --git a/arch/arm64/kvm/hyp_cfi_test_module.c b/arch/arm64/kvm/hyp_cfi_test_module.c index eeda4be4d3ef..63a5e99cb164 100644 --- a/arch/arm64/kvm/hyp_cfi_test_module.c +++ b/arch/arm64/kvm/hyp_cfi_test_module.c @@ -20,9 +20,9 @@ static int set_guest_mode(const char *val, const struct kernel_param *kp); #define M_DESC \ "\n\t0: none" \ "\n\t1: built-in caller & built-in callee" \ - "\n\t2: built-in caller & module callee" \ - "\n\t3: module caller & built-in callee" \ - "\n\t4: module caller & module callee" + "\n\t2: built-in caller & module callee (VHE only)" \ + "\n\t3: module caller & built-in callee (VHE only)" \ + "\n\t4: module caller & module callee (VHE only)" static unsigned int host_mode; module_param_call(host, set_host_mode, param_get_uint, &host_mode, 0644); @@ -40,7 +40,7 @@ static void hyp_cfi_module2module_test_target(int); static void hyp_cfi_builtin2module_test_target(int); static int set_param_mode(const char *val, const struct kernel_param *kp, - int (*register_cb)(void (*)(void))) + int (*register_cb)(void (*)(void), void *)) { unsigned int *mode = kp->arg; int err; @@ -51,15 +51,17 @@ static int set_param_mode(const char *val, const struct kernel_param *kp, switch (*mode) { case 0: - return register_cb(NULL); + return register_cb(NULL, NULL); case 1: - return register_cb(hyp_trigger_builtin_cfi_fault); + return register_cb(hyp_trigger_builtin_cfi_fault, + kvm_nvhe_sym(hyp_trigger_builtin_cfi_fault)); case 2: - return register_cb((void *)hyp_cfi_builtin2module_test_target); + return register_cb((void *)hyp_cfi_builtin2module_test_target, + NULL); case 3: - return register_cb(trigger_module2builtin_cfi_fault); + return register_cb(trigger_module2builtin_cfi_fault, NULL); case 4: - return register_cb(trigger_module2module_cfi_fault); + return register_cb(trigger_module2module_cfi_fault, NULL); default: return -EINVAL; } @@ -79,11 +81,11 @@ static void __exit exit_hyp_cfi_test(void) { int err; - err = kvm_cfi_test_register_host_ctxt_cb(NULL); + err = kvm_cfi_test_register_host_ctxt_cb(NULL, NULL); if (err) pr_err("Failed to unregister host context trigger: %d\n", err); - err = kvm_cfi_test_register_guest_ctxt_cb(NULL); + err = kvm_cfi_test_register_guest_ctxt_cb(NULL, NULL); if (err) pr_err("Failed to unregister guest context trigger: %d\n", err); }