From patchwork Wed May 29 12:12:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678764 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F2922C27C43 for ; Wed, 29 May 2024 12:14:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=31PTmgXzMZ/9sv6oLJdp847p3JTi7uUYk18x1S+jU0c=; b=EsFYtXqtxWf1Ipe7G+AFpTwEwY JOKcJMzvf9o7GfbrZbDG/uoR1pCyNCUZKuczIJKIaHkE323Zy/p3K5NvrPkkQTU+Ulbd0ACJ6A0Jx f2TjFW9yM7mNvkuEBg645gAj7aKMOkrcEUaCfACiNZ+gFmrZ7zBJpfMxw6FxO3j/q1V7HYXlcneju /yUqap7VZN4eUD6NEsiuk1vWFHOEkVW6z0BU9ag0lETw0VtiCMedSKfds3CgXUTqfCizETk9fSt+s FfeX6CjttLATK64uvdCIDL1hMHfHtHMpbi/RdP68y/gt0RRo1E2XosrVts1DXfuv0iA4D+lS6SVIg d185X51Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICS-0000000430R-2iLN; Wed, 29 May 2024 12:14:36 +0000 Received: from mail-ej1-x649.google.com ([2a00:1450:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICP-000000042z5-397P for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 12:14:35 +0000 Received: by mail-ej1-x649.google.com with SMTP id a640c23a62f3a-a592c35ac06so113291066b.0 for ; Wed, 29 May 2024 05:14:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984871; x=1717589671; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=LiBCWky943GnQdDxaq98aS8ZLEQ71giQKbDHRUj5HqQ=; b=EFmt1YR/EeIkgwKfHmPTPfvFlSeT/HuJUbydoSRcWtEAkNp80An5dE4VSA6cPq3WKw Ru4jOmB9bGjHOTYpWDvs9Vx+Rl3i56BuTem1Fhq2UJJSDAwsyd4zCSYLJQhtDeeYseWw NDZczYfsq9UQ++6WyMnbWcQo/QAqCzaweUQa/zDVhP8Z3+0Fys6atv5q655VQGCZnaw0 F4Pkr7lytlM+EQkXXrttLuG1eY468p7sAR30pVrb3+90xeGFeLeBj/k+c8z0pWpcckWC yS29Glo78R8a0ZGpDtVHrMfFp1P0jLS+QkfrXp5TVtjlGNqPjnu98fNU16scG4W7ihHH p4qA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984871; x=1717589671; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=LiBCWky943GnQdDxaq98aS8ZLEQ71giQKbDHRUj5HqQ=; b=TRuHsVsTR8bOXEB1ImoN3/zbeRiot70HifO5p0ITOMoUcO0a4wsivwPW44xkr5H+9Z GrcRJhaR59Fzrh+mSQqnaWnzOPIF96k2TEyDKFi3aZdaUS/TuPkQTO68QSkKOswoXj1T ZRDhVi/Qhi4vYpX/TQHPn1quh32qvZbNzrvxflTgDk7LhZfBYhuJfyI6hAPuoV5eqU0s cDViJJRG1l/RCt42jdVBxSqlticdn0LkUhlUZawtNBv6Z1/0UshtXs6RYAWQ3O54+IBq tmkQ1/fUOkunue2kCuTyk1BALzfH75FRtJs6a/30MBzhtr+ilIFSiybxe86jX6oxsbUo MqGg== X-Forwarded-Encrypted: i=1; AJvYcCXxVsmbWdSq6nbIutqrOSYRPse7ebzanJWpD4bXIejDxqbiljJwgK1EmCKGbn5ADd/LwELuIrvsw0ORkgaJQ/O6HajGX6Nqb8QJlTfRc2uMHxOpMHc= X-Gm-Message-State: AOJu0Yz4x34Uxg5a30UWWNLB+rP6dwsSHaP2huqYMVoNoCRspH0mtgyK HEuESUnCdlqGILO+yXTaOdkhrO4GyfgB6yVkq1ptQKi05FiCiASCKYNj5Ed+Q3LnOyif+Ck2RA= = X-Google-Smtp-Source: AGHT+IFP6n0siCbzTCYAQHYIrAphWxkqP2Ntf7Z84XOpYY4Wgk7fdgvFlpZ4wx2gsGuIGmJgr+coEywpTQ== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a17:906:3185:b0:a5a:1c6:b892 with SMTP id a640c23a62f3a-a642d6aba53mr229666b.6.1716984871216; Wed, 29 May 2024 05:14:31 -0700 (PDT) Date: Wed, 29 May 2024 13:12:07 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-2-ptosi@google.com> Subject: [PATCH v4 01/13] KVM: arm64: Fix clobbered ELR in sync abort/SError From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_051433_834480_805E1F0E X-CRM114-Status: GOOD ( 16.62 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When the hypervisor receives a SError or synchronous exception (EL2h) while running with the __kvm_hyp_vector and if ELR_EL2 doesn't point to an extable entry, it panics indirectly by overwriting ELR with the address of a panic handler in order for the asm routine it returns to to ERET into the handler. However, this clobbers ELR_EL2 for the handler itself. As a result, hyp_panic(), when retrieving what it believes to be the PC where the exception happened, actually ends up reading the address of the panic handler that called it! This results in an erroneous and confusing panic message where the source of any synchronous exception (e.g. BUG() or kCFI) appears to be __guest_exit_panic, making it hard to locate the actual BRK instruction. Therefore, store the original ELR_EL2 in the per-CPU kvm_hyp_ctxt and point the sysreg to a routine that first restores it to its previous value before running __guest_exit_panic. Fixes: 7db21530479f ("KVM: arm64: Restore hyp when panicking in guest context") Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kvm/hyp/entry.S | 8 ++++++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 5 +++-- 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 81496083c041..27de1dddb0ab 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -128,6 +128,7 @@ int main(void) DEFINE(VCPU_FAULT_DISR, offsetof(struct kvm_vcpu, arch.fault.disr_el1)); DEFINE(VCPU_HCR_EL2, offsetof(struct kvm_vcpu, arch.hcr_el2)); DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_cpu_context, regs)); + DEFINE(CPU_ELR_EL2, offsetof(struct kvm_cpu_context, sys_regs[ELR_EL2])); DEFINE(CPU_RGSR_EL1, offsetof(struct kvm_cpu_context, sys_regs[RGSR_EL1])); DEFINE(CPU_GCR_EL1, offsetof(struct kvm_cpu_context, sys_regs[GCR_EL1])); DEFINE(CPU_APIAKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APIAKEYLO_EL1])); diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index f3aa7738b477..4433a234aa9b 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,6 +83,14 @@ alternative_else_nop_endif eret sb +SYM_INNER_LABEL(__guest_exit_restore_elr_and_panic, SYM_L_GLOBAL) + // x2-x29,lr: vcpu regs + // vcpu x0-x1 on the stack + + adr_this_cpu x0, kvm_hyp_ctxt, x1 + ldr x0, [x0, #CPU_ELR_EL2] + msr elr_el2, x0 + SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) // x2-x29,lr: vcpu regs // vcpu x0-x1 on the stack diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index a92566f36022..ed9a63f1f7bf 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -689,7 +689,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __kvm_unexpected_el2_exception(void) { - extern char __guest_exit_panic[]; + extern char __guest_exit_restore_elr_and_panic[]; unsigned long addr, fixup; struct kvm_exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); @@ -711,7 +711,8 @@ static inline void __kvm_unexpected_el2_exception(void) } /* Trigger a panic after restoring the hyp context. */ - write_sysreg(__guest_exit_panic, elr_el2); + this_cpu_ptr(&kvm_hyp_ctxt)->sys_regs[ELR_EL2] = elr_el2; + write_sysreg(__guest_exit_restore_elr_and_panic, elr_el2); } #endif /* __ARM64_KVM_HYP_SWITCH_H__ */ From patchwork Wed May 29 12:12:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678765 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9D04AC27C43 for ; Wed, 29 May 2024 12:14:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ZeVEyarowWOAWYFaLR9rNrg/saH3fb20K0t4f2AGQ0Q=; b=pdDSFYwnqyIeZiuWtPvQ8h+txX Aq3zd9T15WdAdHYXyHjM1E8OvSIF0gOMz2hBJcd33jY7h02tpz0MzCuOoXqgQAmWK0O3bh5g4oSPY sYBF/YU/orPWkAAhvpZYXAwL7JBHQYvRQUGB4hTIg2mCpsbnjGGjBuZbvLBJWRtX5gZ65Hh/8NEfW O766pk2rh6hbL0tZkhTaJyi8bp2iqrpWkZFya183hUDUk8HzoykkeZIUqiSdxAv36zeK8pRs+xKKF 2/vTfcL/j3FxQnghGfeF3ovzfY0bYDHGb+H9X9y6GwsgYIent6rZ/MVRxceIZXlPnHrfEQM4WOn4K Z/yiFg0A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICf-0000000437D-1CxG; Wed, 29 May 2024 12:14:49 +0000 Received: from mail-ed1-x549.google.com ([2a00:1450:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICR-000000042za-3fZR for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 12:14:37 +0000 Received: by mail-ed1-x549.google.com with SMTP id 4fb4d7f45d1cf-5785ea015dbso863249a12.2 for ; Wed, 29 May 2024 05:14:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984873; x=1717589673; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=tLmL1So5x4FnLH0kyn2ty29vEdym4EM/KIgVNMwbuEQ=; b=Uf8K4d0E5rBFMdvECN+fLyYWR+wpp2fuK67vVbFYnjKT9yR/XKFiiU7yFaSOcMoomQ Osv7OQRPHYbwbeq2xrV0I0U8hWl55ik94wtiDR1VtMHtrTYeUAXGDMwgUVK+rt/XwGES FVBIrAXKUthPgbLAFRlhOMOkdHw4Eud06W/q3Gcdeyt7gYG1T6Bq6j5TQk3/eDHV6cDj o77dMEgqXTmyOGZqwhpwwFcWSkbqYVVTeHsC0FaMyIfBeRm3vzQG+hqej8pJ7A7olm// DLMLWeJ/sFlp953vAuZ7XtE5tQrIppxfLrZaXFxRm5Skj8QGP/H9v741hJJ3hpJbZJOH VWdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984873; x=1717589673; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=tLmL1So5x4FnLH0kyn2ty29vEdym4EM/KIgVNMwbuEQ=; b=QPT++5+lZITAlN+sn8ShcmG6CW7iqMk7oomJ1j4VLUASfVQb1H9lCBsUW4zc3Pfq4M +VYOKeja4vZKh2plX3AznTxyutPr9i5zRyaK6v9WQDq38Cg5TUr2nD1ywGt1QyQ3ZgDu 60Ei8uODA7SLga5KKOVjPKVC3+q6eZvSz2CU5zD7zgOYBkHUnw+FmIXq1NYbfMi1gLHr 6C1x7GFkbyjUCuZaS5mcDPd3YF9ocMXmZUiMneqDXx7vwhmjjehQj1gNvmq8LPo1Cv8u HAQmhQAmigE1DlkJQJbEWauZ9Np9HPp/usRuUFqj0v4WCWAdDZrT/cNEuGAgPrryb9K9 oguw== X-Forwarded-Encrypted: i=1; AJvYcCWDiKQSC4AkJqpDTw/GB9DePnkJGrBbGaghv1ZX3sLXCyqQVPEP33++2+piQg6gi6YePStwjxNyzPLT1+YV5kuGKs4peWlmHZWJJwkHkqQG4Tv9cvc= X-Gm-Message-State: AOJu0YwX3wA2Ip4V9NzQ9cnePn6w2AIWMNSX1WS0I8sG6lleIRJ59PQO agGzi3kO0VBOVbMd5Jw4/KtJvul5HMprl1iJ+VlCfDykN0a+202CyCVckQ6h0ZdUTzb9o99IZA= = X-Google-Smtp-Source: AGHT+IFoWb2krclv9TkDbcARu8wLLSDlMlxLJgD2KUqfjrU7i3St1EL1/OV74yeBjy7fXa7WBSwaGQ6E2A== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6402:3223:b0:572:6e36:f0f7 with SMTP id 4fb4d7f45d1cf-578519bbf75mr18449a12.6.1716984873520; Wed, 29 May 2024 05:14:33 -0700 (PDT) Date: Wed, 29 May 2024 13:12:08 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-3-ptosi@google.com> Subject: [PATCH v4 02/13] KVM: arm64: Fix __pkvm_init_switch_pgd call ABI From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_051435_941789_EB7EFE77 X-CRM114-Status: GOOD ( 14.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Fix the mismatch between the (incorrect) C signature, C call site, and asm implementation by aligning all three on an API passing the parameters (pgd and SP) separately, instead of as a bundled struct. Remove the now unnecessary memory accesses while the MMU is off from the asm, which simplifies the C caller (as it does not need to convert a VA struct pointer to PA) and makes the code slightly more robust by offsetting the struct fields from C and properly expressing the call to the C compiler (e.g. type checker and kCFI). Fixes: f320bc742bc2 ("KVM: arm64: Prepare the creation of s1 mappings at EL2") Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/kvm_hyp.h | 3 +-- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 17 +++++++++-------- arch/arm64/kvm/hyp/nvhe/setup.c | 4 ++-- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 3e80464f8953..58b5a2b14d88 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -123,8 +123,7 @@ void __noreturn __hyp_do_panic(struct kvm_cpu_context *host_ctxt, u64 spsr, #endif #ifdef __KVM_NVHE_HYPERVISOR__ -void __pkvm_init_switch_pgd(phys_addr_t phys, unsigned long size, - phys_addr_t pgd, void *sp, void *cont_fn); +void __pkvm_init_switch_pgd(phys_addr_t pgd, void *sp, void (*fn)(void)); int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, unsigned long *per_cpu_base, u32 hyp_va_bits); void __noreturn __host_enter(struct kvm_cpu_context *host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 2994878d68ea..d859c4de06b6 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -265,33 +265,34 @@ alternative_else_nop_endif SYM_CODE_END(__kvm_handle_stub_hvc) +/* + * void __pkvm_init_switch_pgd(phys_addr_t pgd, void *sp, void (*fn)(void)); + */ SYM_FUNC_START(__pkvm_init_switch_pgd) /* Turn the MMU off */ pre_disable_mmu_workaround - mrs x2, sctlr_el2 - bic x3, x2, #SCTLR_ELx_M + mrs x9, sctlr_el2 + bic x3, x9, #SCTLR_ELx_M msr sctlr_el2, x3 isb tlbi alle2 /* Install the new pgtables */ - ldr x3, [x0, #NVHE_INIT_PGD_PA] - phys_to_ttbr x4, x3 + phys_to_ttbr x4, x0 alternative_if ARM64_HAS_CNP orr x4, x4, #TTBR_CNP_BIT alternative_else_nop_endif msr ttbr0_el2, x4 /* Set the new stack pointer */ - ldr x0, [x0, #NVHE_INIT_STACK_HYP_VA] - mov sp, x0 + mov sp, x1 /* And turn the MMU back on! */ dsb nsh isb - set_sctlr_el2 x2 - ret x1 + set_sctlr_el2 x9 + ret x2 SYM_FUNC_END(__pkvm_init_switch_pgd) .popsection diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 859f22f754d3..1cbd2c78f7a1 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -316,7 +316,7 @@ int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, { struct kvm_nvhe_init_params *params; void *virt = hyp_phys_to_virt(phys); - void (*fn)(phys_addr_t params_pa, void *finalize_fn_va); + typeof(__pkvm_init_switch_pgd) *fn; int ret; BUG_ON(kvm_check_pvm_sysreg_table()); @@ -340,7 +340,7 @@ int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, /* Jump in the idmap page to switch to the new page-tables */ params = this_cpu_ptr(&kvm_init_params); fn = (typeof(fn))__hyp_pa(__pkvm_init_switch_pgd); - fn(__hyp_pa(params), __pkvm_init_finalise); + fn(params->pgd_pa, (void *)params->stack_hyp_va, __pkvm_init_finalise); unreachable(); } From patchwork Wed May 29 12:12:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678767 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 407A5C27C44 for ; Wed, 29 May 2024 12:15:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=x3WqSzLm4lWStKF0RRAArfy8FzoJV9RFpf/U3TVS5RU=; b=z2a9439Mzx5s/FwIrF5X2xpp3a nfJNt1xHNg/YPTOrR9cjNaJO9B7w2LXx3RFAAiu9qMA8dHK9HozeQGjjd4YVMVeb6MozZqoQO6xNv YOQAQXuePIN/og3siVcAMBd950sBdqnvbbDFPTEGnKtukCA+1AuaG5uljAZO0BCvKZQGwbevIlOAS b4uUPhAPTw5+gLbLbgB9V7fVH4IlJdESwoOkTDTEVPWA2erfFkomvOSraLrOZ8CIfdK2Tb01bvPUI nphCAKN+Mr3uB8aFOtDZBTfoUYt01e4SMdB2i+wJrK5iTbHHCMPEgGYkdbzHiqHJpHYyzkpC8mE/p pfnizv5g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICg-0000000438G-24HR; Wed, 29 May 2024 12:14:50 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICT-0000000430N-3UYp for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 12:14:41 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-df4e7854de8so3533345276.1 for ; Wed, 29 May 2024 05:14:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984876; x=1717589676; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=j4Rf2lfhPQdW0ulVQ+ZkbOO+HrzrcUNTk97aD9mbPlw=; b=Sfoad8mkMDeXP/sqI6MU+IZrNsSQaT8RlaIAR3+FXpT/b6o62X84GkDOZtxN/Hpxjg swlewbfhfCL5dc7G36wVlY0ttYC9sblUAJp3lSUySeOnhWVy/byiEKWhUOnaXGYyuI7K kwOrmDJAonymm1qZznFVncKuugRGX7IMd0dDOmWwQvZAX80rTUF/qJn/DkDx2zO/pRYX Vp/v2HX02RBQvRCWsKYhVaHoAn5mIRQbNpPehNNDmwBU8PRRUybV1Xwod7peNECK610Q /X1gBdXovQWm1lt+5KlaNiwXHrOUk18yPYQ4ryn/K+Vwn5k4nqC3h+bBeyeRBiso7AOD T9Cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984876; x=1717589676; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=j4Rf2lfhPQdW0ulVQ+ZkbOO+HrzrcUNTk97aD9mbPlw=; b=ONBbslonPZZE/RefGejJCa0NzsKsq1jfZgPLPF0EfGwYM/vvV3hxicaovhmq338Afu RMt3Hr0f1LbENPodk9QyhqCNieNSAxu0K4Ow8v/XFhCb++i9NzpaabhgeQQwWXl3s2is NcUQG9TxjM96kbsB0TtLMnkilt37dwG8DXaJrQ15FD5OoqqVjp6kgWMovad9ye8Rshqb 50IdlWkW68sbfIKyh08T30Tfj7EWlpz1Y7XPua2CbZTMNflSWJ1OeM87g2rRtUJPLOZO mhrAWKei3dGrxdj7vczZBtZndhvnYi/L08KsxiG2OHm9B0gFbUtfGc8HxfGFA7ZxsxxB g2BA== X-Forwarded-Encrypted: i=1; AJvYcCUAvyL9ytbCXECx0yGGKC2lJ82cCuMhWFbFIfc3wqcg0gliT5J0lK1hnE3VTuOMLq0/FGijv0rfK5mLyGkL3CVIQ8bUCIWkDRcuWLr0DiS17kJ1pHk= X-Gm-Message-State: AOJu0Yz6KN61C4h2TKDV3+0YooHsmZrxOAJnv2+6Q2uTmrrAuo5KDJWx XFGWIln866RCtB611jx2j6yipxJN1RQk2GIdVggrpZ3IogFln4HSQfrITivKpX3XSlBX4UzvGQ= = X-Google-Smtp-Source: AGHT+IEkdZ2CPDm5cYVOhM4+XL1fH8AlLKI24Mx4NO3l+ssr5tA1I25II+Qs22X8PnruQU2dGz49CMMUxw== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6902:f0f:b0:de5:4ed6:d3f3 with SMTP id 3f1490d57ef6-df7721df7cbmr3593228276.6.1716984875835; Wed, 29 May 2024 05:14:35 -0700 (PDT) Date: Wed, 29 May 2024 13:12:09 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-4-ptosi@google.com> Subject: [PATCH v4 03/13] KVM: arm64: nVHE: Simplify __guest_exit_panic path From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_051438_103618_AC16F483 X-CRM114-Status: GOOD ( 11.77 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In invalid_host_el2_vect (i.e. EL2{t,h} handlers in nVHE guest context), remove the duplicate vCPU context check that __guest_exit_panic also performs, allowing an unconditional branch to it. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/hyp/nvhe/host.S | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 135cfb294ee5..71fb311b4c0e 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -197,18 +197,13 @@ SYM_FUNC_END(__host_hvc) sub x0, sp, x0 // x0'' = sp' - x0' = (sp + x0) - sp = x0 sub sp, sp, x0 // sp'' = sp' - x0 = (sp + x0) - x0 = sp - /* If a guest is loaded, panic out of it. */ - stp x0, x1, [sp, #-16]! - get_loaded_vcpu x0, x1 - cbnz x0, __guest_exit_panic - add sp, sp, #16 - /* * The panic may not be clean if the exception is taken before the host * context has been saved by __host_exit or after the hyp context has * been partially clobbered by __host_enter. */ - b hyp_panic + stp x0, x1, [sp, #-16]! + b __guest_exit_panic .L__hyp_sp_overflow\@: /* Switch to the overflow stack */ From patchwork Wed May 29 12:12:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678766 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C7BDEC25B75 for ; Wed, 29 May 2024 12:15:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=euK5Axvj4cqQQKVkJaAXlHMFo3EiHxaRl73wDM27v88=; b=McPg6NJkOiEAzFqtoaTpkyT2hh 4yqmFJ1mbL7FgZWIppI5BfG6Aak5Qdz520pJ9gMRxH0lvrc2/eEP2rGwYXp8G8DkpS++wrqRaHLdU 4z6ud7VRuT738ppmg86bZLX08mca25XGDZGO7r+RQ8phFfVY7HoDcbuU4oh0Y9lnDLZn9Daj3VlDF yQmpKgYvTGbY6QpjdsePElSJEO5xICDBDZEYV9mFqPVvkogZifEqb0DefSFCmGVOW88vdxWqxW0X2 2onkHCrpieZzW/KcztncvgLakR+wIw992Z8oY7prZmqA9V2neIZ7KANTGgmJ8T3i7kR090IZcZL+I TuRXKUQw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICi-0000000439f-0AIv; Wed, 29 May 2024 12:14:52 +0000 Received: from mail-ej1-x64a.google.com ([2a00:1450:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICX-0000000431g-19i1 for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 12:14:42 +0000 Received: by mail-ej1-x64a.google.com with SMTP id a640c23a62f3a-a6265d3cc76so111540266b.0 for ; Wed, 29 May 2024 05:14:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984878; x=1717589678; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=QLEJ3O4kV4RVdclClpDnM/s5WQUclK7m/dX1iG4Slsk=; b=y1Vcz1p3iR7WDKNnZY/xsqXgsEhWpujOYPtPMDzQlaLvTqbDot2VJ7BtjJGFtweNFD jMiFDBpvWlm51TWjKBuAktbXie5HmiXkmGugFPjcpbZ3xgOB/dVnTWLBFYMIdrJYe8a5 ugsDQ4gZg9lL3W/1+Ldy4PMbuvBtvGxnKfq6H7NyBzHSFX7I37Zx4yytYYiZ9DaPauJl EfjGAhE4AkW6Xa1T0UhvE0ij4p4y+FU2tXL60MRHpzAqc/5gFucJCs8hEwU74fX+aVjg NSYNHfCm7mu/R8rdNBLkSvlby7K7U7QGrCwMXhAyJSbY9SHsAaJnxJP6ilCMrFmrOjbW 64Lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984878; x=1717589678; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=QLEJ3O4kV4RVdclClpDnM/s5WQUclK7m/dX1iG4Slsk=; b=cCXUs/j/TNIu05B/E4eG3Xekm0tfm9/nOGX++BkB2RVoZM4W8HoCLiUP8d92FmMuSK XVrOqOOcr7ApsDktTUr0wdoxP1Rh6JHh0Pg2TewlHfENGYcrMdti/CcHzoWObcb8vBrB i4/U+C33qE0EUfg9NORp7mxBJIw0VEHxJdHxFb2iW9MPFul+qiU3y+Si+9qt3JgZp4xi z4a+O2KWtVblOMF6uJD4Oh7FacSjmujKtCRO43c60s1wV9IKX2fWf0PXNiqnKKTcjIEn egvEGW/b6/Dtg97aMPDgr8KeJ6WjueoqnPNFXnHrh3Ba9uMgIDLnAuIlZuArhlNZaU3X 7YYA== X-Forwarded-Encrypted: i=1; AJvYcCX8/sBun11rkNj+647U8sRoS+y96aAbHQHa8btbWf1LMQvXnrA5aQ4WyCNwHLy3w4Xi0uN+Ly5BE9HcktRS6tki8jHM98SE2YQ1Nv5l2nloKOcvsYE= X-Gm-Message-State: AOJu0YzCE24SqfKAMnLsMcUHntFG8Ne88LPiPc8ejoNH7xgCvekodrWR T1zkPU4KJTP0XvnREMMccNF3C1jJO/Mzr3BjGvG4iCf1pyJQ98PiZDa4Nalo8fhLa+Fa0ax0Ww= = X-Google-Smtp-Source: AGHT+IHcGsvak0IaIBaJRsxrOxRAaw1LTq+hg6b9qMn9Z3a3+Nx+yXco9WUVoWIw3Nxf0/WiUp16udA02A== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a17:906:1991:b0:a5c:e96f:6022 with SMTP id a640c23a62f3a-a62642e4314mr1531066b.3.1716984878101; Wed, 29 May 2024 05:14:38 -0700 (PDT) Date: Wed, 29 May 2024 13:12:10 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-5-ptosi@google.com> Subject: [PATCH v4 04/13] KVM: arm64: nVHE: Add EL2h sync exception handler From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_051441_419359_7B87F11B X-CRM114-Status: GOOD ( 12.24 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce a handler for EL2h synchronous exceptions distinct from handlers for other "invalid" exceptions when running with the nVHE host vector. This will allow a future patch to handle kCFI (synchronous) errors without affecting other classes of exceptions. Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/kvm/hyp/nvhe/host.S | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 71fb311b4c0e..bc0a73d9fcd0 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -183,7 +183,7 @@ SYM_FUNC_END(__host_hvc) .endif .endm -.macro invalid_host_el2_vect +.macro __host_el2_vect handler:req .align 7 /* @@ -203,7 +203,7 @@ SYM_FUNC_END(__host_hvc) * been partially clobbered by __host_enter. */ stp x0, x1, [sp, #-16]! - b __guest_exit_panic + b \handler .L__hyp_sp_overflow\@: /* Switch to the overflow stack */ @@ -213,6 +213,10 @@ SYM_FUNC_END(__host_hvc) ASM_BUG() .endm +.macro host_el2_sync_vect + __host_el2_vect __guest_exit_panic +.endm + .macro invalid_host_el1_vect .align 7 mov x0, xzr /* restore_host = false */ @@ -222,6 +226,10 @@ SYM_FUNC_END(__host_hvc) b __hyp_do_panic .endm +.macro invalid_host_el2_vect + __host_el2_vect __guest_exit_panic +.endm + /* * The host vector does not use an ESB instruction in order to avoid consuming * SErrors that should only be consumed by the host. Guest entry is deferred by @@ -239,7 +247,7 @@ SYM_CODE_START(__kvm_hyp_host_vector) invalid_host_el2_vect // FIQ EL2t invalid_host_el2_vect // Error EL2t - invalid_host_el2_vect // Synchronous EL2h + host_el2_sync_vect // Synchronous EL2h invalid_host_el2_vect // IRQ EL2h invalid_host_el2_vect // FIQ EL2h invalid_host_el2_vect // Error EL2h From patchwork Wed May 29 12:12:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678769 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 29A3AC27C43 for ; Wed, 29 May 2024 12:15:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ph5AqstOrYzWqFNqZm0q5TQ7FWWgZRrvs3ZuowMPn2c=; b=BpBeU3cpBqLdY+WAj7hYnqCd/H 4Rz3H0HBtDnNfGIBjxozCPriFfc9VOiYJGQ+XJaHAgOKyoZKYRdfFS8g7Wba/PaT2CCv0xVNpgQRb FMPjl65f7Z33DMPiCrW58oTgMIvSZKz6IrzMDsdoqCsGld5egCp1tuit8dgcw9Pk9TY1z7cCQv7nk fbV8mbYFH3b18wugGI2CyJFh6Ljgw0r+UvqW0VgLVKlM0NkZ/6ual64VyHedFOV03E1+KmQTopSJm gTPwHmegbJDpWnJ1m8TZPbIyZ/GGQVoWd/lIbne6pXwFkj0w0C/McwvfceOb8fk9BdTU87pQOD+Tx pofaZwRw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICj-000000043Ar-2d7m; Wed, 29 May 2024 12:14:53 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICY-0000000432E-1uCq for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 12:14:43 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-629f8a71413so33566917b3.1 for ; Wed, 29 May 2024 05:14:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984880; x=1717589680; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=hHat2RDFHpMYw2ro36KxYpGkz0GVoh8rlrjhKV6ZyaM=; b=iICt4potdCbCjDBM2Z3EjJ89L5C5WciIpc65kIQYucJdi5OdjAZlSJgICraLi+SAoY UX1BMx+vS1T7ruzHsXHuoBknkRwbsZ8BhOE33tlgYPFjxPiGyQ5U7qGeWTWuO7psOOIK jAEjTX32Wo04EngrYqkwvE0XQWRqSM1Vokleyn7BU6TqaxprtRD/16kuDRJaDG9cGE5/ otoALYVjHGYQX3RoccryM2X2DsDrO0EAaVLaV3DZhUBHfZCiO5hbdNYFHp0GrTM7Rtv5 0eaDO2QTTHw9n5SheokiYa74v8D6LRkia6EImEU7De9LlH1fFkq8Zh1YpERgG5sLUcEJ zavw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984880; x=1717589680; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=hHat2RDFHpMYw2ro36KxYpGkz0GVoh8rlrjhKV6ZyaM=; b=XHX9yH8IT3IkQDZnNqNuA9dnnXvavXmB1U7k8ITNWd9PbEbm4caf8Y0VikNBbfDJHR TzDRfIrrS/RH9js5NI5D0Fxs7jhd2lGpC0J0d3EHh/gLFuo6ERXcoLTvY5sAgZmo3l5k sCffQppPp9p16PzmiBY+tHHVh5RVGneHmHGAOv71PEg4BXhj+w+apOr2H5x2Bp0Yxuh9 ljuTlPlii5AWifYu+sgKRFDvfxbKYJrBLuR7sH1VPyo1CD6RShVL59b8WYXcIC8BHTFH Rjhhz9ZTS8RLHbaqL4fyLFgnJhzWWipS+ejEH7oIA8qTkXII8kz2gN2NQ8U6XbmJnV2X d6lQ== X-Forwarded-Encrypted: i=1; AJvYcCWYDhMJ9MR18IzJdXwfF7+ML8tubQNpob70ir4GBEgsGBKbM/8dQ+BeDvhUHv18AfO19+uuN+VWMAk1t6z9x34LyVQAw9XULfajNZvOT4QFrDBBpJg= X-Gm-Message-State: AOJu0Ywm9ToYuM2++UI/M70TLMPYUk3Kc5y03iOR2d5rCbldbQKXH8x5 XQn4ab7e5cuv/QpPT9MBDlKEzaT7gmRRYEooJbKdHGCyzcTWzcU93A+OrU7nvoGAXvAuRPRh2w= = X-Google-Smtp-Source: AGHT+IHH51bjxff72q1pHbdvaYO/pbW32t5gktZkV+rMdlZvjhO2kbKYW+Qzcjh0fNRydZa0oQRqxrS2dQ== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6902:1889:b0:dee:6802:dc49 with SMTP id 3f1490d57ef6-df77213607amr4000587276.1.1716984880636; Wed, 29 May 2024 05:14:40 -0700 (PDT) Date: Wed, 29 May 2024 13:12:11 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-6-ptosi@google.com> Subject: [PATCH v4 05/13] KVM: arm64: Rename __guest_exit_panic __hyp_panic From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_051442_553314_6C9FE017 X-CRM114-Status: GOOD ( 13.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Use a name that expresses the fact that the routine might not exit through the guest but will always (directly or indirectly) end up executing hyp_panic(). Use CPU_LR_OFFSET to clarify that the routine returns to hyp_panic(). Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/hyp/entry.S | 6 +++--- arch/arm64/kvm/hyp/hyp-entry.S | 2 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/host.S | 4 ++-- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index 4433a234aa9b..343851c17373 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,7 +83,7 @@ alternative_else_nop_endif eret sb -SYM_INNER_LABEL(__guest_exit_restore_elr_and_panic, SYM_L_GLOBAL) +SYM_INNER_LABEL(__hyp_restore_elr_and_panic, SYM_L_GLOBAL) // x2-x29,lr: vcpu regs // vcpu x0-x1 on the stack @@ -91,7 +91,7 @@ SYM_INNER_LABEL(__guest_exit_restore_elr_and_panic, SYM_L_GLOBAL) ldr x0, [x0, #CPU_ELR_EL2] msr elr_el2, x0 -SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) +SYM_INNER_LABEL(__hyp_panic, SYM_L_GLOBAL) // x2-x29,lr: vcpu regs // vcpu x0-x1 on the stack @@ -109,7 +109,7 @@ SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) // accurate if the guest had been completely restored. adr_this_cpu x0, kvm_hyp_ctxt, x1 adr_l x1, hyp_panic - str x1, [x0, #CPU_XREG_OFFSET(30)] + str x1, [x0, #CPU_LR_OFFSET] get_vcpu_ptr x1, x0 diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 03f97d71984c..7e65ef738ec9 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -122,7 +122,7 @@ el2_error: eret sb -.macro invalid_vector label, target = __guest_exit_panic +.macro invalid_vector label, target = __hyp_panic .align 2 SYM_CODE_START_LOCAL(\label) b \target diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index ed9a63f1f7bf..d9931abf14c2 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -689,7 +689,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __kvm_unexpected_el2_exception(void) { - extern char __guest_exit_restore_elr_and_panic[]; + extern char __hyp_restore_elr_and_panic[]; unsigned long addr, fixup; struct kvm_exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); @@ -712,7 +712,7 @@ static inline void __kvm_unexpected_el2_exception(void) /* Trigger a panic after restoring the hyp context. */ this_cpu_ptr(&kvm_hyp_ctxt)->sys_regs[ELR_EL2] = elr_el2; - write_sysreg(__guest_exit_restore_elr_and_panic, elr_el2); + write_sysreg(__hyp_restore_elr_and_panic, elr_el2); } #endif /* __ARM64_KVM_HYP_SWITCH_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index bc0a73d9fcd0..a7db40a51e4a 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -214,7 +214,7 @@ SYM_FUNC_END(__host_hvc) .endm .macro host_el2_sync_vect - __host_el2_vect __guest_exit_panic + __host_el2_vect __hyp_panic .endm .macro invalid_host_el1_vect @@ -227,7 +227,7 @@ SYM_FUNC_END(__host_hvc) .endm .macro invalid_host_el2_vect - __host_el2_vect __guest_exit_panic + __host_el2_vect __hyp_panic .endm /* From patchwork Wed May 29 12:12:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678768 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 69724C25B75 for ; Wed, 29 May 2024 12:15:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=3Hnn/h50QkaDn6Sm0DCWhvO/cqAbhSIwfzuF2xoWV6w=; b=lQEbO2zA+yvi7OhzGSd3WpTpGM 3dFL0L63SGvhDTNsaGxKmqfLeLfA3Or+HnwXUE2hgjiNEiBiXjWhOtJJfwWFkov3xCr8VQs0djepa ZQHiyy2nQ1kqVOS2pJoYcgi1n16HmD4btDZmIVOCw9AXKDCTOrviyWWiMHJl2JJ/qcPPOLh/AfqZb 4hdYw0mXWqf2sUFyhF1vqRMkBEO5u0gwJF77yYtitR+N0qFgmPAVcEJaRlCNWIs8HEzu3fDmCmGqt FcIxMVcCmQSbOSXSC0syrBj0TD2RZpWwmX/m9kSj9iykr9FqcbPKuPW7Tz9Os+ynxOqlo419oVQKD ChhhKgEA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICm-000000043Do-41mp; Wed, 29 May 2024 12:14:56 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICb-0000000433R-0uMc for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 12:14:46 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-627e4afa326so31446257b3.2 for ; Wed, 29 May 2024 05:14:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984883; x=1717589683; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=jFr7qj90it7wD8ycil4shHE/w7boH3XXa9rQ6TceJ7g=; b=qIaVLv7nJ4NZEYEW4yiNhhmVe6llvJplXcjQkgDk+z5gYC7BbMQRa1gL0ZFj0td+/9 PFEuEYb5/TScUeHPKYwfdoNy0A56xjQIUPmy+8tljl4RTtBGEZq2Man8QGOtVpcOLqYi s9ib5UeZYdl41YYLt0UsaqRTJaFIRBqq5orJt+T7DvUJca43sg6A6r0TT9ekPIXSHYv1 rjSHrw/3BspRM0qL0LOHISIFvvewx9/0u8BwbiUZlHN5ii2c8UyZ2BKlvQ63BISEglMC j5YBinjzMd3ntq+MnFz3bhN41QPw9CfdXn/isc3aPc7flcFt3qqPLfJRqHk5MTJWSKZp Tw4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984883; x=1717589683; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=jFr7qj90it7wD8ycil4shHE/w7boH3XXa9rQ6TceJ7g=; b=fv1xqxFeEyNIT5Y9DdJwyygIqEsUdxOuAZBu4cvGrJdQlL0SdWJMWZG5RoxBtvceq6 bBbc7R5lMi/aUP3PlWMlIDCPJhK4WsyLgMzLV7WR2OcQcD2+qyoyBqQ90aPI9+iuILrB 9gZpCXbyrLroEga0KSLh9MMKm2soVlP3dWz9sAw/JvnHck+uhavnpPwU8zg6XXo9k/x3 tpm+ZJ8SrPEyjbDTRqL6yVeJcrZuMt2eroYJK13Y0SXiaOvK/SaR4R9+0vvVQTgO7aat nCWCaIrldTxcItSHgbQZW/B8UMyXq0dovpOLe24kDkbLNMtx9510gC33WDlCDHFoKPEc h8gw== X-Forwarded-Encrypted: i=1; AJvYcCXI3j8MIkP0rENbeWFJIju1eEpRLh6u+LdD9AIe5Qf3vnnYio06SsaUFzm/7T+xvOrQprWqOONQ5lHhdhT4pcjWmlVi9sMYjtejzOGt69pva/UBux8= X-Gm-Message-State: AOJu0YxD1zqk0k0HtIfHbFdiUDYFfDKzOkJKIlXAwprpOZ5KYCGJ96uD gZnTecYt15Ysl8iq9V44h/aKV4xpMZssczyi8JMwGNQEQkLF5iRLc0vC2FYFyrOJTxssuj583Q= = X-Google-Smtp-Source: AGHT+IFkqfeHx8YzJAunywpRdidgNdjLeTtWgg2p4V4vZYTrR/3mutQzX2qxXb5fN2wPSUT3J1iB5pmTTg== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a81:4c56:0:b0:61a:d0d2:b31 with SMTP id 00721157ae682-62a08d9304emr35784307b3.3.1716984883084; Wed, 29 May 2024 05:14:43 -0700 (PDT) Date: Wed, 29 May 2024 13:12:12 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-7-ptosi@google.com> Subject: [PATCH v4 06/13] KVM: arm64: nVHE: gen-hyprel: Skip R_AARCH64_ABS32 From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_051445_313249_997042A8 X-CRM114-Status: GOOD ( 11.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Ignore R_AARCH64_ABS32 relocations, instead of panicking, when emitting the relocation table of the hypervisor. The toolchain might produce them when generating function calls with kCFI to represent the 32-bit type ID which can then be resolved across compilation units at link time. These are NOT actual 32-bit addresses and are therefore not needed in the final (runtime) relocation table (which is unlikely to use 32-bit absolute addresses for arm64 anyway). Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/kvm/hyp/nvhe/gen-hyprel.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c b/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c index 6bc88a756cb7..b63f4e1c1033 100644 --- a/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c +++ b/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c @@ -50,6 +50,9 @@ #ifndef R_AARCH64_ABS64 #define R_AARCH64_ABS64 257 #endif +#ifndef R_AARCH64_ABS32 +#define R_AARCH64_ABS32 258 +#endif #ifndef R_AARCH64_PREL64 #define R_AARCH64_PREL64 260 #endif @@ -383,6 +386,9 @@ static void emit_rela_section(Elf64_Shdr *sh_rela) case R_AARCH64_ABS64: emit_rela_abs64(rela, sh_orig_name); break; + /* Allow 32-bit absolute relocation, for kCFI type hashes. */ + case R_AARCH64_ABS32: + break; /* Allow position-relative data relocations. */ case R_AARCH64_PREL64: case R_AARCH64_PREL32: From patchwork Wed May 29 12:12:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678770 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 21DF8C25B75 for ; Wed, 29 May 2024 12:15:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=IccAikI5i0XyYqwm/Pmr01fkSrmGjP33aL5Dmw+mk7I=; b=sF9R7Il6hlxy/RBPYNG8s0jxLJ 22TQNJswkhwte1fF02jP7CCMeo6phvLytgAGYTrBYVjB9yiApQwB0/HheLNDuKYz+iHk0ni69dPau YXxVnFrrMrQR5Rp4tCxhdSLFkXhdlXcuMb1kI54C8I+l2FRVm1nObRY1nYnziupHYza9hz62UxHgq zOS99fuyLatYcLFrzgkirxg0rMnzFjtc8j+JQSlvprH1GF5qYxV4SJzmJnm8NEKbuml5iIoR39Dfh bn+5DTME5QFqcIkCYGnf6MWr+kJEOQNjtj83mte0aork8d7F1BWItP1Yo3POKR6YpmMZyu9hMnN5u N6FYrUuA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICu-000000043Kv-4Bka; Wed, 29 May 2024 12:15:05 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICc-0000000434w-3E08 for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 12:14:48 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-df4d62ff39fso3715621276.2 for ; Wed, 29 May 2024 05:14:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984885; x=1717589685; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=8LApYKgBNKEpQOUHCgPJ8rjKQvWvAO0g6PQmvtilIlw=; b=C1/nMV7FdcmojTSfFmCABGjK9kygComVrYi0Ceg52NWCQahT8g389eBEbw8nCwFtNQ JxzYAbjnbLnsnWVMflo2q1jRurpg+4QgN4mUi/uFzP0sEv14XnQCzNZ6Ot0nTAn92spP jRbcogVw/VSkq9yHI/v8+szJs7cf7tolSRDlzeFBPKhLV2pBpQlkXxoehOSTv/XeGfG1 jtunIqrc6eCm0WRK/ej4VeGcyxIQ3ZTdq5v1ugoTvDMwjCL3vEwoPt4TB3S9n6CE+opQ NoggtMaaGLgEoEa4ZRIll26ZmI/hhg9E9T0RBtonZfaY8vWEIGrVp4jHLoS1Du2qu2A6 jGZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984885; x=1717589685; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=8LApYKgBNKEpQOUHCgPJ8rjKQvWvAO0g6PQmvtilIlw=; b=N0CsJnTKlgQaghYbhQkTuaiKCJKp/dKx1lWjdgvcSfCjwFX5jb3UBnYR7THNMpFlCx N4i1YCPUhllPTSeniSHBj9sP2JdecsQWhJ2dVKhSRMTR5OM8ru7zjvU/RXOY2KCs/Pma awdHLxU5d8MwZ0ovQ/hgNZZY7pOrwddstJ2HX+FqsCSXOhOx9vGAkupMf39MYNjBmaoP LF4gyCelxjz9jJXhuAxZVGiyy2JDkeRw7gVwYWc2+s41ht0fHJn7sE+If4kJrDPM1KJL FO4ka+LHFZgtpJLOpxFdGIWZ1QxvQ8XwGJBaiY0zlVlXZmzrr96ktg1O9cEuEpFA/U3P igvQ== X-Forwarded-Encrypted: i=1; AJvYcCUB1/xW2m507Is9KqnVRRIeYv3xX0rNFn8RIfwHS3gw7rfiEhFYB0HxlYUKHE/6TzGTjuy0YnGRhDS1rbXoiOPkdb4ZaSVSLQHuic+4PwTzHazk7+E= X-Gm-Message-State: AOJu0YxCTjbdqVfL9RrMeDvZMv6NosM4VFF7hf4/Ezbi0QMSo53FB0Te ixv1bEWbX4ZgETnFCOzrKwKuZ1MERRk1gIGdFmuV/NWHbgn3yXXvzLtSCFJkS/cNOhfwNTJc2w= = X-Google-Smtp-Source: AGHT+IHSjvv323m46MNf64EsuTBJ/plesPXi5fnjH9bpFqNhZzaJSI42rMp8TuTkGGyShh9t0w2mdhFYDg== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6902:1006:b0:df7:8c1b:430a with SMTP id 3f1490d57ef6-df78c1b6af8mr3075741276.3.1716984885370; Wed, 29 May 2024 05:14:45 -0700 (PDT) Date: Wed, 29 May 2024 13:12:13 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-8-ptosi@google.com> Subject: [PATCH v4 07/13] KVM: arm64: VHE: Mark __hyp_call_panic __noreturn From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_051446_994622_FB9D2D7F X-CRM114-Status: GOOD ( 10.13 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Given that the sole purpose of __hyp_call_panic() is to call panic(), a __noreturn function, give it the __noreturn attribute, removing the need for its caller to use unreachable(). Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/kvm/hyp/vhe/switch.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index d7af5f46f22a..0550b9f6317f 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -384,7 +384,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) return ret; } -static void __hyp_call_panic(u64 spsr, u64 elr, u64 par) +static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) { struct kvm_cpu_context *host_ctxt; struct kvm_vcpu *vcpu; @@ -409,7 +409,6 @@ void __noreturn hyp_panic(void) u64 par = read_sysreg_par(); __hyp_call_panic(spsr, elr, par); - unreachable(); } asmlinkage void kvm_unexpected_el2_exception(void) From patchwork Wed May 29 12:12:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678771 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BDAC6C25B75 for ; Wed, 29 May 2024 12:16:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=W3CrXRM+RcSoasvgbfoQOYvZAkf1rgMASn8NwrCswqY=; b=Rr+NcOfTJaAfXFEOZi6tb3Mbx3 IUJlApu1I0mdUX1EtQliKrlds2HfWi7RwO9YH2iHBXAIbwvfg4C15WwLuTZjqUs81VHKVPRyHuUzQ F6n0Gj4YPYd3SL2HzYYpxkI3jr9qY3w2gmYdIEK+MlwtJfAeMltVG/QrPZs1HTabyx1bZxihlNxjs 7LNvvpRfHfrn8/2qzOaa7cJljfD/+TFMg0ZFaUhJ4eROuyOfxdD7KbbsrN7CLqFGOLCjthLdchqiK vTUIIhFndKuof6FiKvzErIciBKBdvJ7hi5j2WGAYBDRKyoXyqF//WF6aLNtFzpYu7fgLFy6dVn8hQ UqSXtIQw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCIED-000000043w8-3yRi; Wed, 29 May 2024 12:16:25 +0000 Received: from mail-ed1-x54a.google.com ([2a00:1450:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICg-00000004371-2Gnz for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 12:14:52 +0000 Received: by mail-ed1-x54a.google.com with SMTP id 4fb4d7f45d1cf-579e27b0404so1044605a12.1 for ; Wed, 29 May 2024 05:14:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984888; x=1717589688; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=Fl+f30rl7dVa/IVaUSQ5SKFqgJp4W7PmIf/N7fy8cOU=; b=cy4fQ7DvyleLwCNwXpZOGMhrFa6Fm3OTmBqamHzalGdg4pi10Xdw11Ql8T8T2W7chD TaJVZP+C2SckW2TiPin30+H/N16fnveKNfB/vzUEZRiGbHHIA3h5MBvqqCf03mqNp/Xy nXk/3P0kLUIJF+8CLzDYPwQ6A4Q9TzqTQO3/BTw2FsMFAe3ByubFnfAi2bnBYFqN1QTN weKscl/ehdCrc0Krgb6LixI23Wb8EhcPxG7PDJqNNLwTIaQ2AYOXVbnvpWQu+tcZxyzk WYZ2jKGnteaXj/Hgczy4QraAHxcMECDUrqoMrSgjl5CoUEExNG9Zbsyg0pGiWX9B0f1A 9OoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984888; x=1717589688; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=Fl+f30rl7dVa/IVaUSQ5SKFqgJp4W7PmIf/N7fy8cOU=; b=icUCU5oZai46kNDv2I3JekGFYqiDBlS8L787pQ9pttntySP0A6bBtw8VytxEmGYap0 0k5OTq+MgR8zZXNPMQ3WEyWPJXw/RrCz/u1IZIqgSf+QFJfSrZXVkSM4Lfr8LvSKSO2A Q3VWfdYwxQhXIrEBtJtkdMW2W5JIxMTZRSIW0Tdv6S93tq/rz55t3z0P7XB4pRwiu5Hm Th3cAndQYFNoo3Tq7FJMBYlGa5lExWkdISjvHj94tpJSfJeiWS/NhV6v93n5eoIrDGrb FtHQXNKWgPVO4xMshQNpFSBYId6fhSu1l24KhXZv1RfdFLFjdLY9ANDUbZDrcONutihx ulyA== X-Forwarded-Encrypted: i=1; AJvYcCWaTALvAnQswJmOe+BT85kav59lChda63BsgWgp/DWPWN7i3TE6gW2L3SjYNOfGN7utRsTO9WeK+PRQs5PUcs0iEtx/my6+/kggwK93VwqcWjA0WLc= X-Gm-Message-State: AOJu0YwIOHL9hR/MU5NYLYxRVr/GHVdicEE6n86oXUU6BxgnszjvoRAn j790fUHBP9Z/nAWZIpvdrItYx3HNFsZizHFwptJ6qcNkepf+zvAa807kjAfZN0feYi/3hB4o+Q= = X-Google-Smtp-Source: AGHT+IEEHNDVH5oua3/ecoC74+hCc08TVB6uy8myiwmNDRyQ2CUe66zxXOENHLIB7cqS59dn8IOjkr3XHg== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6402:1954:b0:578:b249:2b06 with SMTP id 4fb4d7f45d1cf-578b24964d6mr14880a12.6.1716984887626; Wed, 29 May 2024 05:14:47 -0700 (PDT) Date: Wed, 29 May 2024 13:12:14 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-9-ptosi@google.com> Subject: [PATCH v4 08/13] arm64: Introduce esr_comment() & esr_is_cfi_brk() From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_051450_712488_36719C29 X-CRM114-Status: GOOD ( 15.03 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org As it is already used in two places, move esr_comment() to a header for re-use, with a clearer name. Introduce esr_is_cfi_brk() to detect kCFI BRK syndromes, currently used by early_brk64() but soon to be also used by hypervisor code. Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/include/asm/esr.h | 11 +++++++++++ arch/arm64/kernel/debug-monitors.c | 4 +--- arch/arm64/kernel/traps.c | 8 +++----- arch/arm64/kvm/handle_exit.c | 2 +- 4 files changed, 16 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index 7abf09df7033..77569d207ecf 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -379,6 +379,11 @@ #ifndef __ASSEMBLY__ #include +static inline unsigned long esr_brk_comment(unsigned long esr) +{ + return esr & ESR_ELx_BRK64_ISS_COMMENT_MASK; +} + static inline bool esr_is_data_abort(unsigned long esr) { const unsigned long ec = ESR_ELx_EC(esr); @@ -386,6 +391,12 @@ static inline bool esr_is_data_abort(unsigned long esr) return ec == ESR_ELx_EC_DABT_LOW || ec == ESR_ELx_EC_DABT_CUR; } +static inline bool esr_is_cfi_brk(unsigned long esr) +{ + return ESR_ELx_EC(esr) == ESR_ELx_EC_BRK64 && + (esr_brk_comment(esr) & ~CFI_BRK_IMM_MASK) == CFI_BRK_IMM_BASE; +} + static inline bool esr_fsc_is_translation_fault(unsigned long esr) { /* Translation fault, level -1 */ diff --git a/arch/arm64/kernel/debug-monitors.c b/arch/arm64/kernel/debug-monitors.c index 64f2ecbdfe5c..024a7b245056 100644 --- a/arch/arm64/kernel/debug-monitors.c +++ b/arch/arm64/kernel/debug-monitors.c @@ -312,9 +312,7 @@ static int call_break_hook(struct pt_regs *regs, unsigned long esr) * entirely not preemptible, and we can use rcu list safely here. */ list_for_each_entry_rcu(hook, list, node) { - unsigned long comment = esr & ESR_ELx_BRK64_ISS_COMMENT_MASK; - - if ((comment & ~hook->mask) == hook->imm) + if ((esr_brk_comment(esr) & ~hook->mask) == hook->imm) fn = hook->fn; } diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 215e6d7f2df8..9e22683aa921 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -1105,8 +1105,6 @@ static struct break_hook ubsan_break_hook = { }; #endif -#define esr_comment(esr) ((esr) & ESR_ELx_BRK64_ISS_COMMENT_MASK) - /* * Initial handler for AArch64 BRK exceptions * This handler only used until debug_traps_init(). @@ -1115,15 +1113,15 @@ int __init early_brk64(unsigned long addr, unsigned long esr, struct pt_regs *regs) { #ifdef CONFIG_CFI_CLANG - if ((esr_comment(esr) & ~CFI_BRK_IMM_MASK) == CFI_BRK_IMM_BASE) + if (esr_is_cfi_brk(esr)) return cfi_handler(regs, esr) != DBG_HOOK_HANDLED; #endif #ifdef CONFIG_KASAN_SW_TAGS - if ((esr_comment(esr) & ~KASAN_BRK_MASK) == KASAN_BRK_IMM) + if ((esr_brk_comment(esr) & ~KASAN_BRK_MASK) == KASAN_BRK_IMM) return kasan_handler(regs, esr) != DBG_HOOK_HANDLED; #endif #ifdef CONFIG_UBSAN_TRAP - if ((esr_comment(esr) & ~UBSAN_BRK_MASK) == UBSAN_BRK_IMM) + if ((esr_brk_comment(esr) & ~UBSAN_BRK_MASK) == UBSAN_BRK_IMM) return ubsan_handler(regs, esr) != DBG_HOOK_HANDLED; #endif return bug_handler(regs, esr) != DBG_HOOK_HANDLED; diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index b037f0a0e27e..d41447193e13 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -423,7 +423,7 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, if (mode != PSR_MODE_EL2t && mode != PSR_MODE_EL2h) { kvm_err("Invalid host exception to nVHE hyp!\n"); } else if (ESR_ELx_EC(esr) == ESR_ELx_EC_BRK64 && - (esr & ESR_ELx_BRK64_ISS_COMMENT_MASK) == BUG_BRK_IMM) { + esr_brk_comment(esr) == BUG_BRK_IMM) { const char *file = NULL; unsigned int line = 0; From patchwork Wed May 29 12:12:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678774 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2F880C27C4F for ; Wed, 29 May 2024 12:16:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=AgqjFLX2k7mfI4/c4hmUS9QPHQi7R9HyWYMSQyi9bZE=; b=bWRIIGMCYgELDpN8fMNviJ7zXo oYZ9tfhiW/65Hm2bb4mjCB7MY7Il/42Gtfs+ZOaqJyx54PR1UqMfc2NUPTgSttFReZ0Qp426mMfO6 N7xknUNzoIsIzqqy/V0XD7zF9844tcToYjYa3qkqcM9t7FfLdNSl4OceALPa2L9BtFWS+2f3bZd3X Sknv68VhEC7MHQAUHjjKCJaK51YALpkJWy1UVM2Pi0Jo5rQ0AOgeO6bKGjlh67sOr06lMfpdhwQ1c jV9APba46cvkFz8vqJOmpJDbog5NAhBam0x1l7DDyOnqdnOu5Luc/RbDfxNtX3PPZP4TjBA1lZPQa aAY6wfVg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCIEE-000000043wM-30G6; Wed, 29 May 2024 12:16:26 +0000 Received: from mail-ed1-x54a.google.com ([2a00:1450:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICi-0000000439I-16en for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 12:14:54 +0000 Received: by mail-ed1-x54a.google.com with SMTP id 4fb4d7f45d1cf-579c4641702so1176027a12.0 for ; Wed, 29 May 2024 05:14:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984890; x=1717589690; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=8CshqLHlWDJc+sAJS8Y64z59WBwvQ7VfO+S7UGCs2ew=; b=QBI69Wrt+j7ApndsA1G2MUitYYHvUAUzFU6EyAfeJJ9UM7fLpiF+/dJidxDe9LBRya 5Kin1/ak3xrSinuk4uwZQd7D6jqZYrgV3DfEPPfyA2wdhU9cW5AU1qFIFHYVY92o9kJ+ ZqzSi7lknRoblJtMRq3+OnZwbzsvaecW+lGXhbTT9obaUuLEM5OgEzi4mECLPBcyDuRw 3bUyyXcTVL+YQWlW4DhJQMLX36RYqocN0sczIfCAXZaaIgaaxgmYi18Fnn8DIlUIMcdy w6Wyer0Yws2iDBtJ2wTP27s0gt+C5F9t49Ke+g9+Pi7yWU7+YiJQVwyNqXstG1CXUWEF TzIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984890; x=1717589690; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=8CshqLHlWDJc+sAJS8Y64z59WBwvQ7VfO+S7UGCs2ew=; b=mwbdZEyWsASCEvkvTSdVE8M49VXBgNk3cuos7/tQnoWzjJ6MDNjYe+eQuAjXApkuxl JLxWAnkAg1HHxZpj5p9Y1fvfJhxCQk9ihIMf6ZHbheQoxQVdY6B0UxnsssvHHO3+xEeZ 7YMQDaNspP3fWydGI/Vu2lFXEetOaTKRY+6CR9HWFHxZga0hcIRPP1qdc/8RunDXDrVR 43Ju68OrTXfk4OCRLcu02XnphpAWjwO1FPhtoYq85TooT71wE4++mg8UF+Ej+I51SHyH EVhMxB3VZZhoECriu7hd240x9oKUIY0cTTi50sdh57xiVW1zFvu8CLApznyGLy12yPjm ZHJQ== X-Forwarded-Encrypted: i=1; AJvYcCWr4NCQexsxVYe1W1ftePFw0Gvfqw9QBpJZbLdEE0Fkef6WYikKLGI5UCzW/a3G6pZVFk1AKGDLoQnq1kdKH3b4VFx0S8XBq694FPZTn8Y19CxXVkY= X-Gm-Message-State: AOJu0YwdNSljqWpvsUGaJJo8CdwXLi59sb6BlRd4S+HUWpmt8bCEcUnI VrpEauMMXjjvz9mAx/zipYwjVj+mGRupeHVga1PGxEZit8heY+Wz0FvBmqnYXXuKuqg8hEjOgA= = X-Google-Smtp-Source: AGHT+IE8phGpuuq3k7TrjskcqmK2dPZ5kpLf85cdSWAkkY4MTdPq3mbXxQAEr7SdzcRt27FdzPre1IagDw== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6402:35b:b0:578:60a0:9995 with SMTP id 4fb4d7f45d1cf-57a041685b6mr2744a12.3.1716984890481; Wed, 29 May 2024 05:14:50 -0700 (PDT) Date: Wed, 29 May 2024 13:12:15 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-10-ptosi@google.com> Subject: [PATCH v4 09/13] KVM: arm64: Introduce print_nvhe_hyp_panic helper From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_051452_442389_57D2E092 X-CRM114-Status: GOOD ( 10.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a helper to display a panic banner soon to also be used for kCFI failures, to ensure that we remain consistent. Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/kvm/handle_exit.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index d41447193e13..b3d6657a259d 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -411,6 +411,12 @@ void handle_exit_early(struct kvm_vcpu *vcpu, int exception_index) kvm_handle_guest_serror(vcpu, kvm_vcpu_get_esr(vcpu)); } +static void print_nvhe_hyp_panic(const char *name, u64 panic_addr) +{ + kvm_err("nVHE hyp %s at: [<%016llx>] %pB!\n", name, panic_addr, + (void *)(panic_addr + kaslr_offset())); +} + void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, u64 elr_virt, u64 elr_phys, u64 par, uintptr_t vcpu, @@ -439,11 +445,9 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, if (file) kvm_err("nVHE hyp BUG at: %s:%u!\n", file, line); else - kvm_err("nVHE hyp BUG at: [<%016llx>] %pB!\n", panic_addr, - (void *)(panic_addr + kaslr_offset())); + print_nvhe_hyp_panic("BUG", panic_addr); } else { - kvm_err("nVHE hyp panic at: [<%016llx>] %pB!\n", panic_addr, - (void *)(panic_addr + kaslr_offset())); + print_nvhe_hyp_panic("panic", panic_addr); } /* Dump the nVHE hypervisor backtrace */ From patchwork Wed May 29 12:12:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1DE7EC27C43 for ; Wed, 29 May 2024 12:16:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=VUO7VVqg7iJamb4gKrQQNTQnaHZ6omTCTjdGIizKP+U=; b=ViaD1Qh0R7Eff2QSQMs8xLu4EO zD7xH3QxhR3Aixxl/SQaBr/NoJkSvXo6Oh+fgyXVj3BIov8KQSDGC4ah1+5wGGqo7zJxAT+Ktcvzf aBSasPs+E8FO2Kgyfiy1d7pHPOwKJ7oN8xcBo331E4eQ3KvtPp/qqM5AIo5gupSUBpqLDkXmKQ+WJ ScsC2V6XCtQD2IDvDdZdP/H8Icazm3EyknaOApzNtgazWBO6ZkTGKvx1aODYKqzapKTxGMZvfq9eA Yb35pX+WqkfQmyqk8UbtIuafqNsPpN2wfNcHfKFlLAi+WWES7BaRoDFgx9jiEP245Ul28c0xNSj++ YgID54EA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCIEI-000000043xu-08a5; Wed, 29 May 2024 12:16:30 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICk-000000043An-1Mpe for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 12:14:57 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-629638f1cb0so31898767b3.3 for ; Wed, 29 May 2024 05:14:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984893; x=1717589693; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=4CCra/rqIFpxe8HMFYt7fMIj2cwQLIfhoDjzzJPzF3M=; b=TCiEXcDifpfQRb7kxpqPoaGJrwoR47BvxGYt7sHwPm7EkFbOydMpm0tj2hrpyplSib gvZzxd2g+5tqIo3TY1bw8K85zjDzP4Fyik+TJXcuZJDcK11YMFnKJdf8uFfLR6GZ6ROE btBY/BW8Q13zUtaxdQFZGAHqarGI1Cr94JW9camqPsG78/xP/7+bM3Spl6E6FYMDpWC2 KaHXWzyWwCwhIZPBhMzLaWFX+nXNqj58TLvatECKzxchs8Tes0bukCiwZLTuSX5Y6v6t od0VSVACEucvKiAR+l+ksC844z/Cskd1AIff1T+TzOqhCyOiIx7FEuraRYW187OCz0Td eWsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984893; x=1717589693; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=4CCra/rqIFpxe8HMFYt7fMIj2cwQLIfhoDjzzJPzF3M=; b=Ktq1Z+/hAtj+AO2hOEeP+deZWh7nu3v9DS/2Xfzyhw44BhddLOeCI6C+66GlP+wpR+ SUjXLCiElKS1aK1XDnuWyN3Yb1bbF9q2XOwlXvLgOzVG/6uIG8iY5ODfAkK6AA3rn88A uXi6sLaWRQS8P936Iuyuj/GcjgqNP1lYiNVOWaiqRkYVTnLDDJHPvxk4gIezFztoJbpl ewZyj/RYyAP2UD4QqejjsQ10z3T3dXdBxgeIIs+TI8KKaSk/VSBOKrOH063YipuMHZp0 NUDTBFzxaTz1opzQ/KP+1xv8qmV6G9GW2niHo+K/iUmIR3rkl2B/iUIbD8yRCbtK2DDr sPfg== X-Forwarded-Encrypted: i=1; AJvYcCXDRIK4BxCNnXbW/Rb7NguJwtIgVCjsIVisFaGuj8UthItQtarVgBIaVocaeQWu4hhIyEiZCLwEiu6TIk7WewVXyuGjiLIvfPZUGznW0mQO4GyuS/c= X-Gm-Message-State: AOJu0YzcSFNmyXg1ZFVxqJWC6UsZJbWa0mtmQ0BE1uI9M6+W4t2IUcLV p3gY0NJwE4YhcC1N2XsSqmBalP+nPqqd+lGd5hFubZ5uECxZG/pStWWyfBss4wg4SYh9TFWrWg= = X-Google-Smtp-Source: AGHT+IHKlfLzIehWc22oDX8uMfcDmLL5aEy3cA9blPvEjAUh7oYjXMJskW3TDfNlIiBODMYBSdvEYmQV4w== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6902:1248:b0:df7:6f84:f36f with SMTP id 3f1490d57ef6-df772172705mr1421198276.4.1716984892739; Wed, 29 May 2024 05:14:52 -0700 (PDT) Date: Wed, 29 May 2024 13:12:16 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-11-ptosi@google.com> Subject: [PATCH v4 10/13] KVM: arm64: nVHE: Support CONFIG_CFI_CLANG at EL2 From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_051454_550313_70F36140 X-CRM114-Status: GOOD ( 17.39 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The compiler implements kCFI by adding type information (u32) above every function that might be indirectly called and, whenever a function pointer is called, injects a read-and-compare of that u32 against the value corresponding to the expected type. In case of a mismatch, a BRK instruction gets executed. When the hypervisor triggers such an exception in nVHE, it panics and triggers and exception return to EL1. Therefore, teach nvhe_hyp_panic_handler() to detect kCFI errors from the ESR and report them. If necessary, remind the user that EL2 kCFI is not affected by CONFIG_CFI_PERMISSIVE. Pass $(CC_FLAGS_CFI) to the compiler when building the nVHE hyp code. Use SYM_TYPED_FUNC_START() for __pkvm_init_switch_pgd, as nVHE can't call it directly and must use a PA function pointer from C (because it is part of the idmap page), which would trigger a kCFI failure if the type ID wasn't present. Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/kvm/handle_exit.c | 10 ++++++++++ arch/arm64/kvm/hyp/nvhe/Makefile | 6 +++--- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 6 +++++- 3 files changed, 18 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index b3d6657a259d..69b08ac7322d 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -417,6 +417,14 @@ static void print_nvhe_hyp_panic(const char *name, u64 panic_addr) (void *)(panic_addr + kaslr_offset())); } +static void kvm_nvhe_report_cfi_failure(u64 panic_addr) +{ + print_nvhe_hyp_panic("CFI failure", panic_addr); + + if (IS_ENABLED(CONFIG_CFI_PERMISSIVE)) + kvm_err(" (CONFIG_CFI_PERMISSIVE ignored for hyp failures)\n"); +} + void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, u64 elr_virt, u64 elr_phys, u64 par, uintptr_t vcpu, @@ -446,6 +454,8 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, kvm_err("nVHE hyp BUG at: %s:%u!\n", file, line); else print_nvhe_hyp_panic("BUG", panic_addr); + } else if (IS_ENABLED(CONFIG_CFI_CLANG) && esr_is_cfi_brk(esr)) { + kvm_nvhe_report_cfi_failure(panic_addr); } else { print_nvhe_hyp_panic("panic", panic_addr); } diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 50fa0ffb6b7e..782b34b004be 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -89,9 +89,9 @@ quiet_cmd_hyprel = HYPREL $@ quiet_cmd_hypcopy = HYPCOPY $@ cmd_hypcopy = $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ $< $@ -# Remove ftrace, Shadow Call Stack, and CFI CFLAGS. -# This is equivalent to the 'notrace', '__noscs', and '__nocfi' annotations. -KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI), $(KBUILD_CFLAGS)) +# Remove ftrace and Shadow Call Stack CFLAGS. +# This is equivalent to the 'notrace' and '__noscs' annotations. +KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS), $(KBUILD_CFLAGS)) # Starting from 13.0.0 llvm emits SHT_REL section '.llvm.call-graph-profile' # when profile optimization is applied. gen-hyprel does not support SHT_REL and # causes a build failure. Remove profile optimization flags. diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index d859c4de06b6..b1c8977e2812 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -5,6 +5,7 @@ */ #include +#include #include #include @@ -267,8 +268,11 @@ SYM_CODE_END(__kvm_handle_stub_hvc) /* * void __pkvm_init_switch_pgd(phys_addr_t pgd, void *sp, void (*fn)(void)); + * + * SYM_TYPED_FUNC_START() allows C to call this ID-mapped function indirectly + * using a physical pointer without triggering a kCFI failure. */ -SYM_FUNC_START(__pkvm_init_switch_pgd) +SYM_TYPED_FUNC_START(__pkvm_init_switch_pgd) /* Turn the MMU off */ pre_disable_mmu_workaround mrs x9, sctlr_el2 From patchwork Wed May 29 12:12:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678778 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 805D7C27C43 for ; Wed, 29 May 2024 12:16:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ahRpjDuqtgEqs1a7BY/CaUCA1JBUkGgJgyJACEE7eGM=; b=xhcXsS4d7CZv/TQ5DZv13l4hA9 Npby3kJHAI/DL4s2jS/Be0Tt4cBwYVWOdL2PDNmP1DDXH5pgCibMos+APG2/UCvEeQsLZPVzYapf8 FiUDYhci4bGFAQv0vkZQXcmYmKB0iBveFMGNIHRBdN72bLcLv5huBaWfKLEai5EGVXUAUy47qix6A NAUHYfVYvbtBQMOIz2/UZQV0eYCLvifz2jd9Vb6NErRTiOJ14QX8eNia/5vdPbYr6PVuew2Duk4xy ROetaCZq5PzavsOkAWMp3IjKwgRcxiFUJMQF/UxoH1QJLQWcMXjdx2ETtlHj9QOflEE7Tpo9UN1ve YKMAULhQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCIEI-000000043yh-3sin; Wed, 29 May 2024 12:16:30 +0000 Received: from mail-ej1-x649.google.com ([2a00:1450:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICn-000000043Cw-0zYT for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 12:15:06 +0000 Received: by mail-ej1-x649.google.com with SMTP id a640c23a62f3a-a6266ffe72eso119416266b.1 for ; Wed, 29 May 2024 05:14:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984895; x=1717589695; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=20VEPEn3Gb7BPD8Z30RwNAM2J68OYkR1rMftiYbdXdo=; b=YRaD1vHNInWwcEpJ6EPtRCQcIxUPT8XRvwpmwqfkQE7FAiLV/hEyQk5brwcMWYP+cg vV7XpH/sRYz1vSuGRrKraNpUGm0h0zlM71syMG1Mp0M0lfA1NXvAwBqz3e015kuivock z8gTtMsuwTQYWguJ6Kk1T7LmNoD6coo6cE7gYY0RxMtAxpBTbU/6BOLFgbW7p4ESL3cP D5e4FOm6oROrQ4wCNEKsmoNC1dqKgvr/h1t6hAl97lQTn9vySV0lclCf1142yw/0u0Ld ssocFqMx27CgY9lSS2d4JxGKRyqKJeZhaLLP0/7bjR6csfDAS4th5O8p/ygV7Ni1Gbjk xXqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984895; x=1717589695; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=20VEPEn3Gb7BPD8Z30RwNAM2J68OYkR1rMftiYbdXdo=; b=aWoWx18uWT2+hiEQtOEMuB1UJxw35cPljMBM5Te9dKK4ToJ9a3qLoFOxf+iRakcC3F dzS9dLZoo8v7Y5SWgWUNX4zo3y1YNp+kH49D+cKqehgIWiVbOwtwdl7TAN39sJB/Xqqp PYBGRErCix+rNdFpU8YUu7mfEHagu6QL1jWV2akGQKxYZCxvF3xH+xol/dxKdex/iozB jRzf7w55v2eviUvG2X//rhfkDRwYK4yU6r7bNrXY4ONfSBb3SsH8HLJbmMUNqC0qmL++ 2I2lyHyebw5wI2mfQheM5m7wnwFdua/OeQaF0DrtD8q8Z7edUmPigtxez5oyEOALvIg9 GviQ== X-Forwarded-Encrypted: i=1; AJvYcCWOyxkIojI89e19787pgc2FqheisgKNiT3wuhAOmZUgrZJADLfZYKEW05XBXglqVUI7zTBTRX+3Wb2ty4gC6rZjWVNgIZJ5MIgJPpxOV/sTqnoYVF8= X-Gm-Message-State: AOJu0Yz+qs5wJnBWBd5/txlA8Zm+YwxehVppw3je4uOwPNITiOxApljy bdhYC7h+OtgsixGseUzdSqVCyo4+HALuiyTMIsAuHm41nhb2/OApAJdC9eP1Qa048nh+oQCCLQ= = X-Google-Smtp-Source: AGHT+IHG1MR1oKr0DJdz40/0WuzMnmz17jEaYp2Xsp5D6m+QC6YtrLkmBnRMnvH0qBf1kHCVYvlptcFOhA== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a17:906:5842:b0:a59:aa99:694c with SMTP id a640c23a62f3a-a62649db3c6mr1603366b.8.1716984895098; Wed, 29 May 2024 05:14:55 -0700 (PDT) Date: Wed, 29 May 2024 13:12:17 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-12-ptosi@google.com> Subject: [PATCH v4 11/13] KVM: arm64: Improve CONFIG_CFI_CLANG error message From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_051457_833494_A1BD0075 X-CRM114-Status: GOOD ( 21.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org For kCFI, the compiler encodes in the immediate of the BRK (which the CPU places in ESR_ELx) the indices of the two registers it used to hold (resp.) the function pointer and expected type. Therefore, the kCFI handler must be able to parse the contents of the register file at the point where the exception was triggered. To achieve this, introduce a new hypervisor panic path that first stores the CPU context in the per-CPU kvm_hyp_ctxt before calling (directly or indirectly) hyp_panic() and execute it from all EL2 synchronous exception handlers i.e. - call it directly in host_el2_sync_vect (__kvm_hyp_host_vector, EL2t&h) - call it directly in el2t_sync_invalid (__kvm_hyp_vector, EL2t) - set ELR_EL2 to it in el2_sync (__kvm_hyp_vector, EL2h), which ERETs Teach hyp_panic() to decode the kCFI ESR and extract the target and type from the saved CPU context. In VHE, use that information to panic() with a specialized error message. In nVHE, only report it if the host (EL1) has access to the saved CPU context i.e. iff CONFIG_NVHE_EL2_DEBUG=y, which aligns with the behavior of CONFIG_PROTECTED_NVHE_STACKTRACE. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/handle_exit.c | 30 +++++++++++++++++++++++-- arch/arm64/kvm/hyp/entry.S | 24 +++++++++++++++++++- arch/arm64/kvm/hyp/hyp-entry.S | 2 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/host.S | 2 +- arch/arm64/kvm/hyp/vhe/switch.c | 26 +++++++++++++++++++-- 6 files changed, 79 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 69b08ac7322d..2fac3be3db00 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -26,6 +26,8 @@ #define CREATE_TRACE_POINTS #include "trace_handle_exit.h" +DECLARE_KVM_NVHE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt); + typedef int (*exit_handle_fn)(struct kvm_vcpu *); static void kvm_handle_guest_serror(struct kvm_vcpu *vcpu, u64 esr) @@ -417,10 +419,34 @@ static void print_nvhe_hyp_panic(const char *name, u64 panic_addr) (void *)(panic_addr + kaslr_offset())); } -static void kvm_nvhe_report_cfi_failure(u64 panic_addr) +static void kvm_nvhe_report_cfi_target(struct user_pt_regs *regs, u64 esr, + u64 hyp_offset) +{ + u64 va_mask = GENMASK_ULL(vabits_actual - 1, 0); + u8 type_idx = FIELD_GET(CFI_BRK_IMM_TYPE, esr); + u8 target_idx = FIELD_GET(CFI_BRK_IMM_TARGET, esr); + u32 expected_type = (u32)regs->regs[type_idx]; + u64 target_addr = (regs->regs[target_idx] & va_mask) + hyp_offset; + + kvm_err(" (target: [<%016llx>] %ps, expected type: 0x%08x)\n", + target_addr, (void *)(target_addr + kaslr_offset()), + expected_type); +} + +static void kvm_nvhe_report_cfi_failure(u64 panic_addr, u64 esr, u64 hyp_offset) { + struct user_pt_regs *regs = NULL; + print_nvhe_hyp_panic("CFI failure", panic_addr); + if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG) || !is_protected_kvm_enabled()) + regs = &this_cpu_ptr_nvhe_sym(kvm_hyp_ctxt)->regs; + + if (regs) + kvm_nvhe_report_cfi_target(regs, esr, hyp_offset); + else + kvm_err(" (no target information: !CONFIG_NVHE_EL2_DEBUG)\n"); + if (IS_ENABLED(CONFIG_CFI_PERMISSIVE)) kvm_err(" (CONFIG_CFI_PERMISSIVE ignored for hyp failures)\n"); } @@ -455,7 +481,7 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, else print_nvhe_hyp_panic("BUG", panic_addr); } else if (IS_ENABLED(CONFIG_CFI_CLANG) && esr_is_cfi_brk(esr)) { - kvm_nvhe_report_cfi_failure(panic_addr); + kvm_nvhe_report_cfi_failure(panic_addr, esr, hyp_offset); } else { print_nvhe_hyp_panic("panic", panic_addr); } diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index 343851c17373..8965dbc75972 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,7 +83,7 @@ alternative_else_nop_endif eret sb -SYM_INNER_LABEL(__hyp_restore_elr_and_panic, SYM_L_GLOBAL) +SYM_INNER_LABEL(__hyp_restore_elr_save_context_and_panic, SYM_L_GLOBAL) // x2-x29,lr: vcpu regs // vcpu x0-x1 on the stack @@ -91,6 +91,28 @@ SYM_INNER_LABEL(__hyp_restore_elr_and_panic, SYM_L_GLOBAL) ldr x0, [x0, #CPU_ELR_EL2] msr elr_el2, x0 +SYM_INNER_LABEL(__hyp_save_context_and_panic, SYM_L_GLOBAL) + // x0-x29,lr: hyp regs + + stp x0, x1, [sp, #-16]! + + adr_this_cpu x0, kvm_hyp_ctxt, x1 + + stp x2, x3, [x0, #CPU_XREG_OFFSET(2)] + + ldp x2, x3, [sp], #16 + + stp x2, x3, [x0, #CPU_XREG_OFFSET(0)] + stp x4, x5, [x0, #CPU_XREG_OFFSET(4)] + stp x6, x7, [x0, #CPU_XREG_OFFSET(6)] + stp x8, x9, [x0, #CPU_XREG_OFFSET(8)] + stp x10, x11, [x0, #CPU_XREG_OFFSET(10)] + stp x12, x13, [x0, #CPU_XREG_OFFSET(12)] + stp x14, x15, [x0, #CPU_XREG_OFFSET(14)] + stp x16, x17, [x0, #CPU_XREG_OFFSET(16)] + + save_callee_saved_regs x0 + SYM_INNER_LABEL(__hyp_panic, SYM_L_GLOBAL) // x2-x29,lr: vcpu regs // vcpu x0-x1 on the stack diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 7e65ef738ec9..d0d90d598338 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -130,7 +130,7 @@ SYM_CODE_END(\label) .endm /* None of these should ever happen */ - invalid_vector el2t_sync_invalid + invalid_vector el2t_sync_invalid, __hyp_save_context_and_panic invalid_vector el2t_irq_invalid invalid_vector el2t_fiq_invalid invalid_vector el2t_error_invalid diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index d9931abf14c2..77783dbc1833 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -689,7 +689,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __kvm_unexpected_el2_exception(void) { - extern char __hyp_restore_elr_and_panic[]; + extern char __hyp_restore_elr_save_context_and_panic[]; unsigned long addr, fixup; struct kvm_exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); @@ -712,7 +712,7 @@ static inline void __kvm_unexpected_el2_exception(void) /* Trigger a panic after restoring the hyp context. */ this_cpu_ptr(&kvm_hyp_ctxt)->sys_regs[ELR_EL2] = elr_el2; - write_sysreg(__hyp_restore_elr_and_panic, elr_el2); + write_sysreg(__hyp_restore_elr_save_context_and_panic, elr_el2); } #endif /* __ARM64_KVM_HYP_SWITCH_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index a7db40a51e4a..9343160f5357 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -214,7 +214,7 @@ SYM_FUNC_END(__host_hvc) .endm .macro host_el2_sync_vect - __host_el2_vect __hyp_panic + __host_el2_vect __hyp_save_context_and_panic .endm .macro invalid_host_el1_vect diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 0550b9f6317f..6c64783c3e00 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -17,6 +17,7 @@ #include #include +#include #include #include #include @@ -384,7 +385,24 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) return ret; } -static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) +static void __noreturn __hyp_call_panic_for_cfi(u64 elr, u64 esr) +{ + struct user_pt_regs *regs = &this_cpu_ptr(&kvm_hyp_ctxt)->regs; + u8 type_idx = FIELD_GET(CFI_BRK_IMM_TYPE, esr); + u8 target_idx = FIELD_GET(CFI_BRK_IMM_TARGET, esr); + u32 expected_type = (u32)regs->regs[type_idx]; + u64 target = regs->regs[target_idx]; + + panic("VHE hyp CFI failure at: [<%016llx>] %pB (target: [<%016llx>] %ps, expected type: 0x%08x)\n" +#ifdef CONFIG_CFI_PERMISSIVE + " (CONFIG_CFI_PERMISSIVE ignored for hyp failures)\n" +#endif + , + elr, (void *)elr, target, (void *)target, expected_type); +} +NOKPROBE_SYMBOL(__hyp_call_panic_for_cfi); + +static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par, u64 esr) { struct kvm_cpu_context *host_ctxt; struct kvm_vcpu *vcpu; @@ -395,6 +413,9 @@ static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) __deactivate_traps(vcpu); sysreg_restore_host_state_vhe(host_ctxt); + if (IS_ENABLED(CONFIG_CFI_CLANG) && esr_is_cfi_brk(esr)) + __hyp_call_panic_for_cfi(elr, esr); + panic("HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n", spsr, elr, read_sysreg_el2(SYS_ESR), read_sysreg_el2(SYS_FAR), @@ -407,8 +428,9 @@ void __noreturn hyp_panic(void) u64 spsr = read_sysreg_el2(SYS_SPSR); u64 elr = read_sysreg_el2(SYS_ELR); u64 par = read_sysreg_par(); + u64 esr = read_sysreg_el2(SYS_ESR); - __hyp_call_panic(spsr, elr, par); + __hyp_call_panic(spsr, elr, par, esr); } asmlinkage void kvm_unexpected_el2_exception(void) From patchwork Wed May 29 12:12:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678772 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 428E9C27C44 for ; Wed, 29 May 2024 12:16:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=gSQKCBoKxhqdmyD64RMmWKvJUYq22CZWOM/2M0g1X/M=; b=aUnJhf/hyjwDemZJeHyCwJ+GDv hW/RcMYJvf9rtjWtqoeEy39FejQcs611eEFyVWQmFhqHQYWiPS7JZAcTTZwzc3UOUGvWvrfmvOK4c 1muGa2qk+//cW8YjBDKsX6UQukS3gVcuGjpgZ4D8xC4dI886yhayCGUB2jxwPekyKnriYP0w4suuy fYXTgi9ngnBRkuB/WeWuKS7pGT7k80cq26pVVJP3n+jhpsPkYukRaltxP4SMmUP8RPy4Yde6URTW9 nQCfvXmBZqp4CKiQ37Kh4d+9p1WO/yyKDL8JkGwQDOOGGRAPMhtXBHI5fxoamL9r+KISnnrrxgsqJ sIr0uX3g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCIEF-000000043wt-29tp; Wed, 29 May 2024 12:16:27 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICq-000000043FD-49E8 for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 12:15:15 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-dfa47ef0589so938950276.0 for ; Wed, 29 May 2024 05:14:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984897; x=1717589697; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=uRgrsfnu4k+NI/qYYv/tuJDICq7UcPQuz5rX48+teys=; b=njCK/w8zf0WDHuHtFtQaSmIMK9RAjhFh4q1FzUc4hBn/34DxwCp/R5v/u4JDFibeIY aHmYg3IA05z42K951Ed6vo47wJvddaWvQgV5Qa9Eh4qJHIb1QwS5k2m0kmNNfafJMPnY q/WiGleMiZTmuRAJcOVrh7CH9N74vzJS1Ex/Wm26/2BujxhNDDdQuZp2R18rRdhmrGe6 ti1kFnQfGOItgONY0VFIZ9fEgf89jOZLajI9RNpmCDKq6CZLD7IEWitxgH6Ya8so3zCS DLiDNxqntlgcPDvBhjFUJWWzcnvEqvKlLsO1G9Dmyqd94L7gGlHk1qc82NQwr5fvxi9j /CLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984897; x=1717589697; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=uRgrsfnu4k+NI/qYYv/tuJDICq7UcPQuz5rX48+teys=; b=S8J6DQTL4nCdrwzeNZ/16NNGl71uUiUpgjkpJ76sOPfC0nhjaJIGHP9lzrrGSZVAHw n5d+AoIy/PpOgh/PqYuYekJRZpAEWYaralNbfjIeSVRRauo5aiq21joNUE4swC7je9MJ ZlvvCT1Sm4xmiWs+z13j3yKHEwKZTYK+YVuY8XM8fwbLrOIkm7HPdTmZrT+yUgRuHgng 3MycK7k24dMFvhhYvK2NcfA0DMYA+v7dqKZnPBe4Z6SfH5u5bpde8Ujt8itQBGeixWhY pMnSeGncw9ocpTlMTrIpUjLynUtrTwUd7/rlT1+U/GrsduFOWuAq80f5gRpNBBVgSs7T w0wg== X-Forwarded-Encrypted: i=1; AJvYcCVz4SOx0Mzuqasjuc8lsko8ipoGg8+M4mTY2H3iaOgwBIQRGio9cRNCQ53TzHd7vMRsfn08CL2chaCxoBAxXKwEP1mTDpv7TsEzGM6kfWBtSV/2WbY= X-Gm-Message-State: AOJu0Yw9v1xl4LTavdpLFAPLRkW9KdJq4HbBRMAXK7sVVX+ovpvAD+oJ ew9ClTBVG1mHgcrFotwaNfxelwTSvF53UsOQk+XjYjHmsH+dyuBYdIX20ZIxIrQoRVpFvjF4JQ= = X-Google-Smtp-Source: AGHT+IFkQb5OxQ4pScFZIWRQEFsGIMu9g6HuNiBaUgrqAIayPXl97t9kWzQUkwXiZZdHCWHOqYzW82AA/Q== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6902:1207:b0:dee:7884:acc7 with SMTP id 3f1490d57ef6-df77214fd6dmr1189597276.1.1716984897529; Wed, 29 May 2024 05:14:57 -0700 (PDT) Date: Wed, 29 May 2024 13:12:18 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-13-ptosi@google.com> Subject: [PATCH v4 12/13] KVM: arm64: VHE: Add test module for hyp kCFI From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_051501_769657_359A2860 X-CRM114-Status: GOOD ( 32.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In order to easily periodically (and potentially automatically) validate that the hypervisor kCFI feature doesn't bitrot, introduce a way to trigger hypervisor kCFI faults from userspace on test builds of KVM. Add hooks in the hypervisor code to call registered callbacks (intended to trigger kCFI faults either for the callback call itself of from within the callback function) when running with guest or host VBAR_EL2. As the calls are issued from the KVM_RUN ioctl handling path, userspace gains control over when the actual triggering of the fault happens without needing to modify the KVM uAPI. Export kernel functions to register these callbacks from modules and introduce a kernel module intended to contain any testing logic. By limiting the changes to the core kernel to a strict minimum, this architectural split allows tests to be updated (within the module) without the need to redeploy (or recompile) the kernel (hyp) under test. Use the module parameters as the uAPI for configuring the fault condition being tested (i.e. either at insertion or post-insertion using /sys/module/.../parameters), which naturally makes it impossible for userspace to test kCFI without the module (and, inversely, makes the module only - not KVM - responsible for exposing said uAPI). As kCFI is implemented with a caller-side check of a callee-side value, make the module support 4 tests based on the location of the caller and callee (built-in or in-module), for each of the 2 hypervisor contexts (host & guest), selected by userspace using the 'guest' or 'host' module parameter. For this purpose, export symbols which the module can use to configure the callbacks for in-kernel and module-to-built-in kCFI faulting calls. Define the module-to-kernel API to allow the module to detect that it was loaded on a kernel built with support for it but which is running without a hypervisor (-ENXIO) or with one that doesn't use the VHE CPU feature (-EOPNOTSUPP), which is currently the only mode for which KVM supports hypervisor kCFI. Allow kernel build configs to set CONFIG_HYP_CFI_TEST to only support the in-kernel hooks (=y) or also build the test module (=m). Use intermediate internal Kconfig flags (CONFIG_HYP_SUPPORTS_CFI_TEST and CONFIG_HYP_CFI_TEST_MODULE) to simplify the Makefiles and #ifdefs. As the symbols for callback registration are only exported to modules when CONFIG_HYP_CFI_TEST != n, it is impossible for the test module to be non-forcefully inserted on a kernel that doesn't support it. Note that this feature must NOT result in any noticeable change (behavioral or binary size) when HYP_CFI_TEST_MODULE = n. CONFIG_HYP_CFI_TEST is intentionally independent of CONFIG_CFI_CLANG, to avoid arbitrarily limiting the number of flag combinations that can be tested with the module. Also note that, as VHE aliases VBAR_EL1 to VBAR_EL2 for the host, testing hypervisor kCFI in VHE and in host context is equivalent to testing kCFI support of the kernel itself i.e. EL1 in non-VHE and/or in non-virtualized environments. For this reason, CONFIG_CFI_PERMISSIVE **will** prevent the test module from triggering a hyp panic (although a warning still gets printed) in that context. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/kvm_cfi.h | 36 ++++++++ arch/arm64/kvm/Kconfig | 22 +++++ arch/arm64/kvm/Makefile | 3 + arch/arm64/kvm/hyp/include/hyp/cfi.h | 47 ++++++++++ arch/arm64/kvm/hyp/vhe/Makefile | 1 + arch/arm64/kvm/hyp/vhe/cfi.c | 37 ++++++++ arch/arm64/kvm/hyp/vhe/switch.c | 7 ++ arch/arm64/kvm/hyp_cfi_test.c | 43 +++++++++ arch/arm64/kvm/hyp_cfi_test_module.c | 133 +++++++++++++++++++++++++++ 9 files changed, 329 insertions(+) create mode 100644 arch/arm64/include/asm/kvm_cfi.h create mode 100644 arch/arm64/kvm/hyp/include/hyp/cfi.h create mode 100644 arch/arm64/kvm/hyp/vhe/cfi.c create mode 100644 arch/arm64/kvm/hyp_cfi_test.c create mode 100644 arch/arm64/kvm/hyp_cfi_test_module.c diff --git a/arch/arm64/include/asm/kvm_cfi.h b/arch/arm64/include/asm/kvm_cfi.h new file mode 100644 index 000000000000..13cc7b19d838 --- /dev/null +++ b/arch/arm64/include/asm/kvm_cfi.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ + +#ifndef __ARM64_KVM_CFI_H__ +#define __ARM64_KVM_CFI_H__ + +#include +#include + +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + +int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)); +int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)); + +#else + +static inline int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)) +{ + return -EOPNOTSUPP; +} + +static inline int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) +{ + return -EOPNOTSUPP; +} + +#endif /* CONFIG_HYP_SUPPORTS_CFI_TEST */ + +/* Symbols which the host can register as hyp callbacks; see . */ +void hyp_trigger_builtin_cfi_fault(void); +void hyp_builtin_cfi_fault_target(int unused); + +#endif /* __ARM64_KVM_CFI_H__ */ diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 58f09370d17e..5daa8079a120 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -65,4 +65,26 @@ config PROTECTED_NVHE_STACKTRACE If unsure, or not using protected nVHE (pKVM), say N. +config HYP_CFI_TEST + tristate "KVM hypervisor kCFI test support" + depends on KVM + help + Say Y or M here to build KVM with test hooks to support intentionally + triggering hypervisor kCFI faults in guest or host context. + + Say M here to also build a module which registers callbacks triggering + faults and selected by userspace through its parameters. + + Note that this feature is currently only supported in VHE mode. + + If unsure, say N. + +config HYP_SUPPORTS_CFI_TEST + def_bool y + depends on HYP_CFI_TEST + +config HYP_CFI_TEST_MODULE + def_tristate m if HYP_CFI_TEST = m + depends on HYP_CFI_TEST + endif # VIRTUALIZATION diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index a6497228c5a8..303be42ad90a 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -22,6 +22,7 @@ kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \ vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \ vgic/vgic-its.o vgic/vgic-debug.o +kvm-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += hyp_cfi_test.o kvm-$(CONFIG_HW_PERF_EVENTS) += pmu-emul.o pmu.o kvm-$(CONFIG_ARM64_PTR_AUTH) += pauth.o @@ -40,3 +41,5 @@ $(obj)/hyp_constants.h: $(obj)/hyp-constants.s FORCE obj-kvm := $(addprefix $(obj)/, $(kvm-y)) $(obj-kvm): $(obj)/hyp_constants.h + +obj-$(CONFIG_HYP_CFI_TEST_MODULE) += hyp_cfi_test_module.o diff --git a/arch/arm64/kvm/hyp/include/hyp/cfi.h b/arch/arm64/kvm/hyp/include/hyp/cfi.h new file mode 100644 index 000000000000..c6536040bc06 --- /dev/null +++ b/arch/arm64/kvm/hyp/include/hyp/cfi.h @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ + +#ifndef __ARM64_KVM_HYP_CFI_H__ +#define __ARM64_KVM_HYP_CFI_H__ + +#include +#include + +#include + +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + +int __kvm_register_cfi_test_cb(void (*cb)(void), bool in_host_ctxt); + +extern void (*hyp_test_host_ctxt_cfi)(void); +extern void (*hyp_test_guest_ctxt_cfi)(void); + +/* Hypervisor callbacks for the host to register. */ +void hyp_trigger_builtin_cfi_fault(void); +void hyp_builtin_cfi_fault_target(int unused); + +#else + +static inline +int __kvm_register_cfi_test_cb(void (*cb)(void), bool in_host_ctxt) +{ + return -EOPNOTSUPP; +} + +#define hyp_test_host_ctxt_cfi ((void(*)(void))(NULL)) +#define hyp_test_guest_ctxt_cfi ((void(*)(void))(NULL)) + +static inline void hyp_trigger_builtin_cfi_fault(void) +{ +} + +static inline void hyp_builtin_cfi_fault_target(int __always_unused unused) +{ +} + +#endif /* CONFIG_HYP_SUPPORTS_CFI_TEST */ + +#endif /* __ARM64_KVM_HYP_CFI_H__ */ diff --git a/arch/arm64/kvm/hyp/vhe/Makefile b/arch/arm64/kvm/hyp/vhe/Makefile index 3b9e5464b5b3..19ca584cc21e 100644 --- a/arch/arm64/kvm/hyp/vhe/Makefile +++ b/arch/arm64/kvm/hyp/vhe/Makefile @@ -9,3 +9,4 @@ ccflags-y := -D__KVM_VHE_HYPERVISOR__ obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o +obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += cfi.o diff --git a/arch/arm64/kvm/hyp/vhe/cfi.c b/arch/arm64/kvm/hyp/vhe/cfi.c new file mode 100644 index 000000000000..5849f239e27f --- /dev/null +++ b/arch/arm64/kvm/hyp/vhe/cfi.c @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ +#include + +#include + +void (*hyp_test_host_ctxt_cfi)(void); +void (*hyp_test_guest_ctxt_cfi)(void); + +int __kvm_register_cfi_test_cb(void (*cb)(void), bool in_host_ctxt) +{ + if (in_host_ctxt) + hyp_test_host_ctxt_cfi = cb; + else + hyp_test_guest_ctxt_cfi = cb; + + return 0; +} + +void hyp_builtin_cfi_fault_target(int __always_unused unused) +{ +} + +void hyp_trigger_builtin_cfi_fault(void) +{ + /* Intentional UB cast & dereference, to trigger a kCFI fault. */ + void (*target)(void) = (void *)&hyp_builtin_cfi_fault_target; + + /* + * READ_ONCE() prevents this indirect call from being optimized out, + * forcing the compiler to generate the kCFI check before the branch. + */ + READ_ONCE(target)(); +} diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 6c64783c3e00..fe70220876b4 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -4,6 +4,7 @@ * Author: Marc Zyngier */ +#include #include #include @@ -311,6 +312,9 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) struct kvm_cpu_context *guest_ctxt; u64 exit_code; + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_host_ctxt_cfi)) + hyp_test_host_ctxt_cfi(); + host_ctxt = host_data_ptr(host_ctxt); guest_ctxt = &vcpu->arch.ctxt; @@ -329,6 +333,9 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) sysreg_restore_guest_state_vhe(guest_ctxt); __debug_switch_to_guest(vcpu); + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_guest_ctxt_cfi)) + hyp_test_guest_ctxt_cfi(); + do { /* Jump in the fire! */ exit_code = __guest_enter(vcpu); diff --git a/arch/arm64/kvm/hyp_cfi_test.c b/arch/arm64/kvm/hyp_cfi_test.c new file mode 100644 index 000000000000..da7b25ca1b1f --- /dev/null +++ b/arch/arm64/kvm/hyp_cfi_test.c @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ +#include +#include +#include +#include + +#include +#include +#include + +/* For calling directly into the VHE hypervisor; see . */ +int __kvm_register_cfi_test_cb(void (*)(void), bool); + +static int kvm_register_cfi_test_cb(void (*vhe_cb)(void), bool in_host_ctxt) +{ + if (!is_hyp_mode_available()) + return -ENXIO; + + if (is_hyp_nvhe()) + return -EOPNOTSUPP; + + return __kvm_register_cfi_test_cb(vhe_cb, in_host_ctxt); +} + +int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)) +{ + return kvm_register_cfi_test_cb(cb, true); +} +EXPORT_SYMBOL(kvm_cfi_test_register_host_ctxt_cb); + +int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) +{ + return kvm_register_cfi_test_cb(cb, false); +} +EXPORT_SYMBOL(kvm_cfi_test_register_guest_ctxt_cb); + +/* Hypervisor callbacks for the test module to register. */ +EXPORT_SYMBOL(hyp_trigger_builtin_cfi_fault); +EXPORT_SYMBOL(hyp_builtin_cfi_fault_target); diff --git a/arch/arm64/kvm/hyp_cfi_test_module.c b/arch/arm64/kvm/hyp_cfi_test_module.c new file mode 100644 index 000000000000..eeda4be4d3ef --- /dev/null +++ b/arch/arm64/kvm/hyp_cfi_test_module.c @@ -0,0 +1,133 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include + +#include +#include +#include +#include + +static int set_host_mode(const char *val, const struct kernel_param *kp); +static int set_guest_mode(const char *val, const struct kernel_param *kp); + +#define M_DESC \ + "\n\t0: none" \ + "\n\t1: built-in caller & built-in callee" \ + "\n\t2: built-in caller & module callee" \ + "\n\t3: module caller & built-in callee" \ + "\n\t4: module caller & module callee" + +static unsigned int host_mode; +module_param_call(host, set_host_mode, param_get_uint, &host_mode, 0644); +MODULE_PARM_DESC(host, + "Hypervisor kCFI fault test case in host context:" M_DESC); + +static unsigned int guest_mode; +module_param_call(guest, set_guest_mode, param_get_uint, &guest_mode, 0644); +MODULE_PARM_DESC(guest, + "Hypervisor kCFI fault test case in guest context:" M_DESC); + +static void trigger_module2module_cfi_fault(void); +static void trigger_module2builtin_cfi_fault(void); +static void hyp_cfi_module2module_test_target(int); +static void hyp_cfi_builtin2module_test_target(int); + +static int set_param_mode(const char *val, const struct kernel_param *kp, + int (*register_cb)(void (*)(void))) +{ + unsigned int *mode = kp->arg; + int err; + + err = param_set_uint(val, kp); + if (err) + return err; + + switch (*mode) { + case 0: + return register_cb(NULL); + case 1: + return register_cb(hyp_trigger_builtin_cfi_fault); + case 2: + return register_cb((void *)hyp_cfi_builtin2module_test_target); + case 3: + return register_cb(trigger_module2builtin_cfi_fault); + case 4: + return register_cb(trigger_module2module_cfi_fault); + default: + return -EINVAL; + } +} + +static int set_host_mode(const char *val, const struct kernel_param *kp) +{ + return set_param_mode(val, kp, kvm_cfi_test_register_host_ctxt_cb); +} + +static int set_guest_mode(const char *val, const struct kernel_param *kp) +{ + return set_param_mode(val, kp, kvm_cfi_test_register_guest_ctxt_cb); +} + +static void __exit exit_hyp_cfi_test(void) +{ + int err; + + err = kvm_cfi_test_register_host_ctxt_cb(NULL); + if (err) + pr_err("Failed to unregister host context trigger: %d\n", err); + + err = kvm_cfi_test_register_guest_ctxt_cb(NULL); + if (err) + pr_err("Failed to unregister guest context trigger: %d\n", err); +} +module_exit(exit_hyp_cfi_test); + +static void trigger_module2builtin_cfi_fault(void) +{ + /* Intentional UB cast & dereference, to trigger a kCFI fault. */ + void (*target)(void) = (void *)&hyp_builtin_cfi_fault_target; + + /* + * READ_ONCE() prevents this indirect call from being optimized out, + * forcing the compiler to generate the kCFI check before the branch. + */ + READ_ONCE(target)(); + + pr_err_ratelimited("%s: Survived a kCFI violation\n", __func__); +} + +static void trigger_module2module_cfi_fault(void) +{ + /* Intentional UB cast & dereference, to trigger a kCFI fault. */ + void (*target)(void) = (void *)&hyp_cfi_module2module_test_target; + + /* + * READ_ONCE() prevents this indirect call from being optimized out, + * forcing the compiler to generate the kCFI check before the branch. + */ + READ_ONCE(target)(); + + pr_err_ratelimited("%s: Survived a kCFI violation\n", __func__); +} + +/* Use different functions, for clearer symbols in kCFI panic reports. */ +static noinline +void hyp_cfi_module2module_test_target(int __always_unused unused) +{ +} + +static noinline +void hyp_cfi_builtin2module_test_target(int __always_unused unused) +{ +} + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Pierre-Clément Tosi "); +MODULE_DESCRIPTION("KVM hypervisor kCFI test module"); From patchwork Wed May 29 12:12:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13678775 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D23DCC25B75 for ; Wed, 29 May 2024 12:16:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=1QxrsiMv8U5sL7gc3HzmimalKwQEXEGCUFbabHOkCus=; b=ITqTrL+KD0ClLvleteFMBchh83 OiZpOyTVw2gv1LmwTxtSx0iYBUPBRe/gFdNYgRUw5QzP4C+Cz5IMP3D2MZPAAEQ+7DFoMCdaZCfJY A7HWhZBhW3CxJqcbCtDrzPG91imBsVVRTEUEmX9jhH/ReA+kntr2vAHp2TJvrRKH63ASLnyNT1zsb psAPkm7AeokU993jEWfpRTQLyAQ5V31hu8R4M+YpPXVolvo84PxhFGWufY68EIAwiPBPW1HfX9Y+O JMry3O1tk1STai3etngB0C90/os1DBZjxnVlTDiKJDeHMG9mXHwWTmJcr0dI3hcs0c3ft6Or1FVHk /q+YLTOA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCIEH-000000043xI-0BO5; Wed, 29 May 2024 12:16:29 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCICr-000000043GV-10Ln for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 12:15:16 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-df771b4eacbso3511095276.2 for ; Wed, 29 May 2024 05:15:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716984900; x=1717589700; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=2WYr4Xqu6SHeaXEynQRTMZNyZEDRWLI9UmjyxKqMZ9M=; b=lYGV9dA6+/EvdaC7zSmV16R0me81U9eCSEirHLjGdi6XK0u1inapEajpBx3O7d6mVV PUi+5SHRblnsIURLUF56p02FeYjAXaQ5a+KbBts9k8GEjOkZsHnjbm/W2xoxYw3fBRiU vR+hPfJ0dTtfqgqeNwXVC9jF7CqB8zE4/ikqZNwrLEdPP7DiRgBW4E16Yt7K2SBJanQ/ hNxsR7d45+UKr0KgXUoAXYyL4U0pLA0Ly1yxOM3cHG1mf3cmYikQvSB57j4XOp/SbXQU NM9ZV4d/s+xgBb8fTVKcDETqXil065+js2y8BUAE+j7RdAW7OcvDXtUUMUBDSyqoZmYs SVkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716984900; x=1717589700; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=2WYr4Xqu6SHeaXEynQRTMZNyZEDRWLI9UmjyxKqMZ9M=; b=mW+VS6LglcSow9V9E+tN7lDCp3AQBrciqkcgBy3BKfMo2fA5rdxF9QTO4TVHZbJWlw wBAL2uk76S8pNUgbfbhb9+SIKeENxtLledhjCCY4uQMHZAFQdmMTIrHmKeSkZwaPoW73 6k+u0gwv+LBRF4W6qsTAhxaZogNRDv0Nx7cz3kfV4+VZZ2uRwKjWkOrHecxYrEzAgjCU H1PhTT1mxgiG5Heyt2RkCUWpiYInVxC9TNxOFJNHDpHqFCDc4OpLXjCQaM31DPvM2sUR e2Wy2PKrQ/uTSuce6cLZoVwEnbUjm/v/Pi689MXKcAgO5VPLcADHA8veu7OmNSvZa5E7 DIEQ== X-Forwarded-Encrypted: i=1; AJvYcCUMnkdd74Q/ZAI/7tvEy5z+vO66b9Sgfv3Ldq8RylUY7d/t5hUi2FbwdvUPxi2Vf8F3LcAyDRS/9KCvgTah0l+ZvCiidx0d3VLWXpqDGUObaqRR378= X-Gm-Message-State: AOJu0YycoGDSm0yhdXFhCurMPYOpXVfhNbhY2MDH+F/VtgQ+wZrUFXJg lAfsxUT7xoRG9Qdl7xdipuSdg8QKMSUlhQiSslIMq1PAqpPKIAEs/UgLCtGGu68LT+6qFudAng= = X-Google-Smtp-Source: AGHT+IHHUoIUjFzO4GjkhNWRkyuptnIeiIsPtLJmFaYxcziwtscpK8PJLaAhdGcG5Z1QL/wRAezHUEYegw== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a25:ea14:0:b0:df7:8f43:f8a3 with SMTP id 3f1490d57ef6-df78f440daemr2830788276.0.1716984899780; Wed, 29 May 2024 05:14:59 -0700 (PDT) Date: Wed, 29 May 2024 13:12:19 +0100 In-Reply-To: <20240529121251.1993135-1-ptosi@google.com> Mime-Version: 1.0 References: <20240529121251.1993135-1-ptosi@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240529121251.1993135-14-ptosi@google.com> Subject: [PATCH v4 13/13] KVM: arm64: nVHE: Support test module for hyp kCFI From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_051501_995668_DB6D0C0A X-CRM114-Status: GOOD ( 26.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Extend support for the kCFI test module to nVHE by replicating the hooks on the KVM_RUN handler path currently existing in VHE in the nVHE code, exporting the equivalent callback targets for triggering built-in hyp kCFI faults, and exposing a new CONFIG_HYP_CFI_TEST-only host HVC to implement callback registration. Update the test module to register the nVHE equivalent callback for test case '1' (i.e. both EL2 hyp caller and callee are built-in) and document that other cases are not supported outside of VHE, as they require EL2 symbols in the module, which is not currently supported for nVHE. Note that a kernel in protected mode that doesn't support HYP_CFI_TEST will prevent the module from registering nVHE callbacks both by not exporting the necessary symbols (similar to VHE) but also by rejecting the corresponding HVC, if the module tries to issue it directly. Also note that the test module will run in pKVM (with HYP_CFI_TEST) independently of other debug Kconfig flags but that not stacktrace will be printed without PROTECTED_NVHE_STACKTRACE. This allows testing kCFI under conditions closer to release builds, if desired. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/kvm_asm.h | 3 ++ arch/arm64/include/asm/kvm_cfi.h | 6 ++-- arch/arm64/kvm/Kconfig | 2 -- arch/arm64/kvm/hyp/{vhe => }/cfi.c | 0 arch/arm64/kvm/hyp/nvhe/Makefile | 1 + arch/arm64/kvm/hyp/nvhe/hyp-main.c | 19 ++++++++++++ arch/arm64/kvm/hyp/nvhe/switch.c | 7 +++++ arch/arm64/kvm/hyp/vhe/Makefile | 2 +- arch/arm64/kvm/hyp_cfi_test.c | 44 ++++++++++++++++++++++++---- arch/arm64/kvm/hyp_cfi_test_module.c | 24 ++++++++------- 10 files changed, 86 insertions(+), 22 deletions(-) rename arch/arm64/kvm/hyp/{vhe => }/cfi.c (100%) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index a6330460d9e5..791054492920 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -79,6 +79,9 @@ enum __kvm_host_smccc_func { __KVM_HOST_SMCCC_FUNC___pkvm_init_vm, __KVM_HOST_SMCCC_FUNC___pkvm_init_vcpu, __KVM_HOST_SMCCC_FUNC___pkvm_teardown_vm, +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + __KVM_HOST_SMCCC_FUNC___kvm_register_cfi_test_cb, +#endif }; #define DECLARE_KVM_VHE_SYM(sym) extern char sym[] diff --git a/arch/arm64/include/asm/kvm_cfi.h b/arch/arm64/include/asm/kvm_cfi.h index 13cc7b19d838..ed6422eebce5 100644 --- a/arch/arm64/include/asm/kvm_cfi.h +++ b/arch/arm64/include/asm/kvm_cfi.h @@ -12,8 +12,8 @@ #ifdef CONFIG_HYP_SUPPORTS_CFI_TEST -int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)); -int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)); +int kvm_cfi_test_register_host_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb); +int kvm_cfi_test_register_guest_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb); #else @@ -31,6 +31,8 @@ static inline int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) /* Symbols which the host can register as hyp callbacks; see . */ void hyp_trigger_builtin_cfi_fault(void); +DECLARE_KVM_NVHE_SYM(hyp_trigger_builtin_cfi_fault); void hyp_builtin_cfi_fault_target(int unused); +DECLARE_KVM_NVHE_SYM(hyp_builtin_cfi_fault_target); #endif /* __ARM64_KVM_CFI_H__ */ diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 5daa8079a120..715c85088c06 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -75,8 +75,6 @@ config HYP_CFI_TEST Say M here to also build a module which registers callbacks triggering faults and selected by userspace through its parameters. - Note that this feature is currently only supported in VHE mode. - If unsure, say N. config HYP_SUPPORTS_CFI_TEST diff --git a/arch/arm64/kvm/hyp/vhe/cfi.c b/arch/arm64/kvm/hyp/cfi.c similarity index 100% rename from arch/arm64/kvm/hyp/vhe/cfi.c rename to arch/arm64/kvm/hyp/cfi.c diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 782b34b004be..115aa8880260 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -25,6 +25,7 @@ hyp-obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o cache.o setup.o mm.o mem_protect.o sys_regs.o pkvm.o stacktrace.o ffa.o hyp-obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o ../pgtable.o +hyp-obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += ../cfi.o hyp-obj-$(CONFIG_LIST_HARDENED) += list_debug.o hyp-obj-y += $(lib-objs) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index d5c48dc98f67..39ed06fbb190 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -5,6 +5,7 @@ */ #include +#include #include #include @@ -13,6 +14,8 @@ #include #include +#include + #include #include #include @@ -301,6 +304,19 @@ static void handle___pkvm_teardown_vm(struct kvm_cpu_context *host_ctxt) cpu_reg(host_ctxt, 1) = __pkvm_teardown_vm(handle); } +#ifndef CONFIG_HYP_SUPPORTS_CFI_TEST +__always_unused +#endif +static void handle___kvm_register_cfi_test_cb(struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(phys_addr_t, cb_phys, host_ctxt, 1); + DECLARE_REG(bool, in_host_ctxt, host_ctxt, 2); + + void (*cb)(void) = cb_phys ? __hyp_va(cb_phys) : NULL; + + cpu_reg(host_ctxt, 1) = __kvm_register_cfi_test_cb(cb, in_host_ctxt); +} + typedef void (*hcall_t)(struct kvm_cpu_context *); #define HANDLE_FUNC(x) [__KVM_HOST_SMCCC_FUNC_##x] = (hcall_t)handle_##x @@ -333,6 +349,9 @@ static const hcall_t host_hcall[] = { HANDLE_FUNC(__pkvm_init_vm), HANDLE_FUNC(__pkvm_init_vcpu), HANDLE_FUNC(__pkvm_teardown_vm), +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + HANDLE_FUNC(__kvm_register_cfi_test_cb), +#endif }; static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index 6758cd905570..52d2fada9e19 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -4,6 +4,7 @@ * Author: Marc Zyngier */ +#include #include #include @@ -249,6 +250,9 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) bool pmu_switch_needed; u64 exit_code; + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_host_ctxt_cfi)) + hyp_test_host_ctxt_cfi(); + /* * Having IRQs masked via PMR when entering the guest means the GIC * will not signal the CPU of interrupts of lower priority, and the @@ -309,6 +313,9 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) __debug_switch_to_guest(vcpu); + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_guest_ctxt_cfi)) + hyp_test_guest_ctxt_cfi(); + do { /* Jump in the fire! */ exit_code = __guest_enter(vcpu); diff --git a/arch/arm64/kvm/hyp/vhe/Makefile b/arch/arm64/kvm/hyp/vhe/Makefile index 19ca584cc21e..951c8c00a685 100644 --- a/arch/arm64/kvm/hyp/vhe/Makefile +++ b/arch/arm64/kvm/hyp/vhe/Makefile @@ -9,4 +9,4 @@ ccflags-y := -D__KVM_VHE_HYPERVISOR__ obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o -obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += cfi.o +obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += ../cfi.o diff --git a/arch/arm64/kvm/hyp_cfi_test.c b/arch/arm64/kvm/hyp_cfi_test.c index da7b25ca1b1f..6a02b43c45f6 100644 --- a/arch/arm64/kvm/hyp_cfi_test.c +++ b/arch/arm64/kvm/hyp_cfi_test.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include @@ -15,29 +16,60 @@ /* For calling directly into the VHE hypervisor; see . */ int __kvm_register_cfi_test_cb(void (*)(void), bool); -static int kvm_register_cfi_test_cb(void (*vhe_cb)(void), bool in_host_ctxt) +static int kvm_register_nvhe_cfi_test_cb(void *cb, bool in_host_ctxt) +{ + extern void *kvm_nvhe_sym(hyp_test_host_ctxt_cfi); + extern void *kvm_nvhe_sym(hyp_test_guest_ctxt_cfi); + + if (is_protected_kvm_enabled()) { + phys_addr_t cb_phys = cb ? virt_to_phys(cb) : 0; + + /* Use HVC as only the hyp can modify its callback pointers. */ + return kvm_call_hyp_nvhe(__kvm_register_cfi_test_cb, cb_phys, + in_host_ctxt); + } + + /* + * In non-protected nVHE, the pKVM HVC is not available but the + * hyp callback pointers can be accessed and modified directly. + */ + if (cb) + cb = kern_hyp_va(kvm_ksym_ref(cb)); + + if (in_host_ctxt) + kvm_nvhe_sym(hyp_test_host_ctxt_cfi) = cb; + else + kvm_nvhe_sym(hyp_test_guest_ctxt_cfi) = cb; + + return 0; +} + +static int kvm_register_cfi_test_cb(void (*vhe_cb)(void), void *nvhe_cb, + bool in_host_ctxt) { if (!is_hyp_mode_available()) return -ENXIO; if (is_hyp_nvhe()) - return -EOPNOTSUPP; + return kvm_register_nvhe_cfi_test_cb(nvhe_cb, in_host_ctxt); return __kvm_register_cfi_test_cb(vhe_cb, in_host_ctxt); } -int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)) +int kvm_cfi_test_register_host_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb) { - return kvm_register_cfi_test_cb(cb, true); + return kvm_register_cfi_test_cb(vhe_cb, nvhe_cb, true); } EXPORT_SYMBOL(kvm_cfi_test_register_host_ctxt_cb); -int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) +int kvm_cfi_test_register_guest_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb) { - return kvm_register_cfi_test_cb(cb, false); + return kvm_register_cfi_test_cb(vhe_cb, nvhe_cb, false); } EXPORT_SYMBOL(kvm_cfi_test_register_guest_ctxt_cb); /* Hypervisor callbacks for the test module to register. */ EXPORT_SYMBOL(hyp_trigger_builtin_cfi_fault); +EXPORT_SYMBOL(kvm_nvhe_sym(hyp_trigger_builtin_cfi_fault)); EXPORT_SYMBOL(hyp_builtin_cfi_fault_target); +EXPORT_SYMBOL(kvm_nvhe_sym(hyp_builtin_cfi_fault_target)); diff --git a/arch/arm64/kvm/hyp_cfi_test_module.c b/arch/arm64/kvm/hyp_cfi_test_module.c index eeda4be4d3ef..63a5e99cb164 100644 --- a/arch/arm64/kvm/hyp_cfi_test_module.c +++ b/arch/arm64/kvm/hyp_cfi_test_module.c @@ -20,9 +20,9 @@ static int set_guest_mode(const char *val, const struct kernel_param *kp); #define M_DESC \ "\n\t0: none" \ "\n\t1: built-in caller & built-in callee" \ - "\n\t2: built-in caller & module callee" \ - "\n\t3: module caller & built-in callee" \ - "\n\t4: module caller & module callee" + "\n\t2: built-in caller & module callee (VHE only)" \ + "\n\t3: module caller & built-in callee (VHE only)" \ + "\n\t4: module caller & module callee (VHE only)" static unsigned int host_mode; module_param_call(host, set_host_mode, param_get_uint, &host_mode, 0644); @@ -40,7 +40,7 @@ static void hyp_cfi_module2module_test_target(int); static void hyp_cfi_builtin2module_test_target(int); static int set_param_mode(const char *val, const struct kernel_param *kp, - int (*register_cb)(void (*)(void))) + int (*register_cb)(void (*)(void), void *)) { unsigned int *mode = kp->arg; int err; @@ -51,15 +51,17 @@ static int set_param_mode(const char *val, const struct kernel_param *kp, switch (*mode) { case 0: - return register_cb(NULL); + return register_cb(NULL, NULL); case 1: - return register_cb(hyp_trigger_builtin_cfi_fault); + return register_cb(hyp_trigger_builtin_cfi_fault, + kvm_nvhe_sym(hyp_trigger_builtin_cfi_fault)); case 2: - return register_cb((void *)hyp_cfi_builtin2module_test_target); + return register_cb((void *)hyp_cfi_builtin2module_test_target, + NULL); case 3: - return register_cb(trigger_module2builtin_cfi_fault); + return register_cb(trigger_module2builtin_cfi_fault, NULL); case 4: - return register_cb(trigger_module2module_cfi_fault); + return register_cb(trigger_module2module_cfi_fault, NULL); default: return -EINVAL; } @@ -79,11 +81,11 @@ static void __exit exit_hyp_cfi_test(void) { int err; - err = kvm_cfi_test_register_host_ctxt_cb(NULL); + err = kvm_cfi_test_register_host_ctxt_cb(NULL, NULL); if (err) pr_err("Failed to unregister host context trigger: %d\n", err); - err = kvm_cfi_test_register_guest_ctxt_cb(NULL); + err = kvm_cfi_test_register_guest_ctxt_cb(NULL, NULL); if (err) pr_err("Failed to unregister guest context trigger: %d\n", err); }