From patchwork Fri May 10 11:26:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13661430 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9B175C41513 for ; Fri, 10 May 2024 11:27:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=5ShZfldscU7LD9LfwZnGHmW2G+AT8nOKcUnV5zqrPLs=; b=qE+KvqD7YrpXOBVaTDHx3QFPPc mx4t3FMDvylQNzbr52X+0BdMcQmPVYGjJZTSMzKF196qP3T73FW1M6sytxVUq1zA11LTErWqEW2ID cHgMftZ2Bh31/ApLbjmuYShV+b3rcl1CKP7n/CzAInrwFQnpjYCJuBc1JJTEJziDKRhFe4S162ZQ/ x2+UdefFzMe92kQSK0pT4gU2htuBidOKgA8HzyIu42InZAeciyrMDJS1Lri+5ObjSNJGLxRKvsSmY VbRElwygtgGJqSyT5g0Xa6WlOMWVtd7AHfxXg0n3T+ih4ARh2WBGgI+idlCmnYm32QFfIMcTBhxqI 8VydKU0w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPR-00000005327-2ZqX; Fri, 10 May 2024 11:27:29 +0000 Received: from mail-ej1-x649.google.com ([2a00:1450:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPO-000000052yG-1aJh for linux-arm-kernel@lists.infradead.org; Fri, 10 May 2024 11:27:28 +0000 Received: by mail-ej1-x649.google.com with SMTP id a640c23a62f3a-a59aedbd9a9so144160066b.0 for ; Fri, 10 May 2024 04:27:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715340440; x=1715945240; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=IlOgeEx7Dj0f9uD0T276ThAzO+p8Gd1L0Dz8H/gEavQ=; b=RByrRSptPQ7Fa89oQBqNmfy3XJEuyqUl/2+Lv4t++T3yl6+quMGVI2iNIhCdQnS1Mx tRwkJlGJ4LgpxO9n4HcthUm+dppugEruFxhNeGCkafP3lQTOfiX60vcgxGBQW9GV99jY 14nZORg+kzskVS4zPyEYcqivhFMV6fbp8wAtDihYYl6wxMUZehi0ymlOjv8HCOFJB4Q1 QILgZxogfSp5+Xy60nlyDMou2rzCb7GgPvw8tZ4Uo37rrsmy16gxzOeZPTRkEl8LSneL nsdjbx//ha/pLkzPziW7/4qkzLKZ9pjPnGUGoFlE/i7LYaKJyP6/CGdV+GhZ4YQs1xta czCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715340440; x=1715945240; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=IlOgeEx7Dj0f9uD0T276ThAzO+p8Gd1L0Dz8H/gEavQ=; b=EvqVvmW6E66601210zsx6Pm+RQh6X2nlXd41cizvi1fvX01yRs0cHjun4pEAnPbceZ iGHpCRB2XvpFoFxIS/W7yffvL7GIqQxcLvZz2RnB3tosyLpnUQWLkUbBtRZckkyN4mxL mnECyIsAUWfHTQSut8UII77HU7vX9Qc+2bqt33MkzKOD56cjO1gIbhvzGOsYrmEvkG+O HdKuuBI7spG67aJ9cyQ6z4mjYxT8029X1IhGzTwokaiI3Ys4OV6rTLH1lfHxKrVR7Ta3 R4WqcfvhPjNmcNp+mguf1VljFr+yQP1IuPhGeHVghdM0gw2CTALSvcDfgz+AR8QwwDnu v12w== X-Forwarded-Encrypted: i=1; AJvYcCWWuNrKDaen8l7AdRo8YlqLYK4DSyW2CIkb6YmyzyWNcqSiBhX+M4chE+FRAsP7S+y7vWT4WmXhwq+7uVN3mVI+/Htzs2t0ypO3wvjOjhqETBn8Ufg= X-Gm-Message-State: AOJu0YwF/2dwVCZgHbF7lboXBuIRd78YqVozgwpFxpOl3AkH8+ndwcy+ NuvSn25UilbeC8fhtmyLtocNuOEfHk6pLWWjOdD+3Y6TG5Y1Lc4J5EDP+DGoME7Bayp2BPu9rw= = X-Google-Smtp-Source: AGHT+IGzC6wBegF/BY4InGWvNQC2wCX5XWro3nkGNqjvsssmipwEVTEODbeq170WB3/2uigSQJYtADatpQ== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a17:906:456:b0:a59:cfcb:4973 with SMTP id a640c23a62f3a-a5a2d553699mr225366b.3.1715340439853; Fri, 10 May 2024 04:27:19 -0700 (PDT) Date: Fri, 10 May 2024 12:26:30 +0100 In-Reply-To: <20240510112645.3625702-1-ptosi@google.com> Mime-Version: 1.0 References: <20240510112645.3625702-1-ptosi@google.com> X-Mailer: git-send-email 2.45.0.118.g7fe29c98d7-goog Message-ID: <20240510112645.3625702-2-ptosi@google.com> Subject: [PATCH v3 01/12] KVM: arm64: Fix clobbered ELR in sync abort/SError From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240510_042726_518488_7B8B57B3 X-CRM114-Status: GOOD ( 15.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When the hypervisor receives a SError or synchronous exception (EL2h) while running with the __kvm_hyp_vector and if ELR_EL2 doesn't point to an extable entry, it panics indirectly by overwriting ELR with the address of a panic handler in order for the asm routine it returns to to ERET into the handler. However, this clobbers ELR_EL2 for the handler itself. As a result, hyp_panic(), when retrieving what it believes to be the PC where the exception happened, actually ends up reading the address of the panic handler that called it! This results in an erroneous and confusing panic message where the source of any synchronous exception (e.g. BUG() or kCFI) appears to be __guest_exit_panic, making it hard to locate the actual BRK instruction. Therefore, store the original ELR_EL2 in the per-CPU kvm_hyp_ctxt and point the sysreg to a routine that first restores it to its previous value before running __guest_exit_panic. Fixes: 7db21530479f ("KVM: arm64: Restore hyp when panicking in guest context") Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kvm/hyp/entry.S | 9 +++++++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 5 +++-- 3 files changed, 13 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 81496083c041..27de1dddb0ab 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -128,6 +128,7 @@ int main(void) DEFINE(VCPU_FAULT_DISR, offsetof(struct kvm_vcpu, arch.fault.disr_el1)); DEFINE(VCPU_HCR_EL2, offsetof(struct kvm_vcpu, arch.hcr_el2)); DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_cpu_context, regs)); + DEFINE(CPU_ELR_EL2, offsetof(struct kvm_cpu_context, sys_regs[ELR_EL2])); DEFINE(CPU_RGSR_EL1, offsetof(struct kvm_cpu_context, sys_regs[RGSR_EL1])); DEFINE(CPU_GCR_EL1, offsetof(struct kvm_cpu_context, sys_regs[GCR_EL1])); DEFINE(CPU_APIAKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APIAKEYLO_EL1])); diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index f3aa7738b477..bcaaf1a11b4e 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,6 +83,15 @@ alternative_else_nop_endif eret sb +SYM_INNER_LABEL(__guest_exit_restore_elr_and_panic, SYM_L_GLOBAL) + // x0-x29,lr: hyp regs + + stp x0, x1, [sp, #-16]! + adr_this_cpu x0, kvm_hyp_ctxt, x1 + ldr x0, [x0, #CPU_ELR_EL2] + msr elr_el2, x0 + ldp x0, x1, [sp], #16 + SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) // x2-x29,lr: vcpu regs // vcpu x0-x1 on the stack diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index e3fcf8c4d5b4..19a7ca2c1277 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -753,7 +753,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __kvm_unexpected_el2_exception(void) { - extern char __guest_exit_panic[]; + extern char __guest_exit_restore_elr_and_panic[]; unsigned long addr, fixup; struct kvm_exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); @@ -775,7 +775,8 @@ static inline void __kvm_unexpected_el2_exception(void) } /* Trigger a panic after restoring the hyp context. */ - write_sysreg(__guest_exit_panic, elr_el2); + this_cpu_ptr(&kvm_hyp_ctxt)->sys_regs[ELR_EL2] = elr_el2; + write_sysreg(__guest_exit_restore_elr_and_panic, elr_el2); } #endif /* __ARM64_KVM_HYP_SWITCH_H__ */ From patchwork Fri May 10 11:26:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13661432 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E000BC25B74 for ; Fri, 10 May 2024 11:27:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=jaA4spfG247fgevz/PnawAE+ugJZGMHEmQu0HVVHbL0=; b=SbTDFCxfCFwkOhUnOwkcd/vmNw VXzPnJ/eYRylC0oAD5iri305Gg+ZLB2ckRdnWvxPXOs2z/wwAcvL494zXZBqy5cz9lVZjZAJs1OjC zpf8pNPyn3uLCiCK3OLf05PDUDMjtiaSmDnGmaKbqxaIEWUIQbrWqbtsJs2Rvh3tT8rn4ggaL4VZ9 PUSn8PM+lPWrFmKK8FX8yH+7I0nmtAseWVB57/nRnQVNEyil3+An1U1nBWnbCF8wSX6O4X+8NNs3S 5YNbyrIMOsJ/VudvA2mpFMEKb9Mf7RDW7pmL36bppdOlh95bN6JiCOYrUOiojiGnaaLJfcgQ+ir+9 6B7ZAQpg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPQ-0000000531j-3vIM; Fri, 10 May 2024 11:27:28 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPN-000000052yl-37Q5 for linux-arm-kernel@lists.infradead.org; Fri, 10 May 2024 11:27:27 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-de604ccb373so2952711276.2 for ; Fri, 10 May 2024 04:27:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715340443; x=1715945243; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=sbYsZ3117akP+PvJr9gYUoi/CIT+2oSEgeDr5jj+0nI=; b=xCXrIfK47JGgAZxZVFeneVtwC2lM5a+jTwTikIXsnSpLjMyQV2Be9KnxgAhfl6ZyHd 2y682PBtRWN7Yzas6GYUzGYexF0HfZRsPe5on5bTSDZ1pmZNlmDcd+zOW9E9K4l999/H ITQ+rL/eJch7euVEvb409TQurr+3XmNiyO5ioxPoAug8U6LMAvAjiJB6Yg8KvelVHpcG i78Z8olQHhb2hOrEuy35heXVv1uUQ9pAP6F8SqL/XqM6n6D86RgFcbL/us3gzEA+gjt3 gSDNqwDNHfE+/RsnvRwD2eEpPCT2HxZbQjeTxu9OdkEwRpae9I7xzddJb/hUFv/iIpA3 653g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715340443; x=1715945243; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=sbYsZ3117akP+PvJr9gYUoi/CIT+2oSEgeDr5jj+0nI=; b=WzbdycTk3CX8/E8BD6NIuOSs3106WPQSPQaGzoieSp+VKQ78UrnB+1YxlBlytPz44j X+x6xqZSsCgCIsF+UM/JNuEpOgm3s/l/ydEGzkQ6eX3VkWSUJFsoH3w89bOnRLSC5p6+ NC+UgP0wFanDTD4ClFlXdCmJiDW2UFDiizvQ07FupJqE9mfUptb4wh73cfJwSb33OlND GfxC+G2lvGUPdKQQhAwRRPD4XZcb4IU2/ccWX0CdhNc3OaiwthF7MrswyiuQopWakB6s BBVFcS/e65hX6z8APIfwtncMhdBcO6yHAUAI0xgicGNC0xhgb6H+goPuJts1wD53bYEb c6pQ== X-Forwarded-Encrypted: i=1; AJvYcCWvgwQYMeTg27XSFyrcX8xiPDgQrVPU3sgaLpN9Pb8UQjtEh5P5kmc/zwssmEyJ6bwC4GyFKKE1iSmxbIzsnTydMgVRct6GovSinviCbDIqx/uR1Ds= X-Gm-Message-State: AOJu0YyELrzvgpUx3nXjpYi3T2S6H28YDm5gLNYTSapVg4xNwELysgOW qMxXMfdFoXLyd1P86jaBvdB4mv3463lmCBaEXs/FqRJSL4xe4NDoogOhcaPoPJdl61+K12IBLA= = X-Google-Smtp-Source: AGHT+IFdj/b7v+ntFzmw+lLCR1C20+GpqYEhAi+SWxria6LS62T7NLsO4dOK8hYMQ4bb7TazgmL1bjHlFQ== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6902:100a:b0:de0:ecc6:4681 with SMTP id 3f1490d57ef6-dee4f30f764mr166230276.1.1715340442754; Fri, 10 May 2024 04:27:22 -0700 (PDT) Date: Fri, 10 May 2024 12:26:31 +0100 In-Reply-To: <20240510112645.3625702-1-ptosi@google.com> Mime-Version: 1.0 References: <20240510112645.3625702-1-ptosi@google.com> X-Mailer: git-send-email 2.45.0.118.g7fe29c98d7-goog Message-ID: <20240510112645.3625702-3-ptosi@google.com> Subject: [PATCH v3 02/12] KVM: arm64: Fix __pkvm_init_switch_pgd C signature From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240510_042725_842153_BB4644BE X-CRM114-Status: GOOD ( 11.65 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Update the function declaration to match the asm implementation. Fixes: f320bc742bc2 ("KVM: arm64: Prepare the creation of s1 mappings at EL2") Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/include/asm/kvm_hyp.h | 3 +-- arch/arm64/kvm/hyp/nvhe/setup.c | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 3e2a1ac0c9bb..96daf7cf6802 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -123,8 +123,7 @@ void __noreturn __hyp_do_panic(struct kvm_cpu_context *host_ctxt, u64 spsr, #endif #ifdef __KVM_NVHE_HYPERVISOR__ -void __pkvm_init_switch_pgd(phys_addr_t phys, unsigned long size, - phys_addr_t pgd, void *sp, void *cont_fn); +void __pkvm_init_switch_pgd(phys_addr_t params, void (*finalize_fn)(void)); int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, unsigned long *per_cpu_base, u32 hyp_va_bits); void __noreturn __host_enter(struct kvm_cpu_context *host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index bc58d1b515af..bcaeb0fafd2d 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -316,7 +316,7 @@ int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, { struct kvm_nvhe_init_params *params; void *virt = hyp_phys_to_virt(phys); - void (*fn)(phys_addr_t params_pa, void *finalize_fn_va); + typeof(__pkvm_init_switch_pgd) *fn; int ret; BUG_ON(kvm_check_pvm_sysreg_table()); From patchwork Fri May 10 11:26:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13661434 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 49DE3C25B5F for ; Fri, 10 May 2024 11:27:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=RZ3K0mrCZzDs9fI1bfJOqz/2snI9xPICImlrpNNCzBg=; b=IbX52ryxz6rzJkmubFojcjun4n nRYXNDMNOnlgz06p3gGrGLdpqecyJpL9ebXffnLPHGt3xoUEmStA/W7K2OqNWY0F0QQPI1kDAEENz PMXqIe4rbI2+rs5uEm1CI4aB8cE7Y8hkgKGKuvl1WkeGTw+hK0gyErdLnf8cfHBcTqkMnBmphTQAT iS4OGgKCz/ZT3hj+la5wJmFf6KEu88BdEguyk3gRTr65ik+Dnqgi/HxgCs6EZcYMNxl4eVUJ6a1Zo vqP4OoqdK/VFxedUF2APQRpbgHTuKJIA6t6Ep5r3kjr64Txi8e5jGm3ygUqN77mmbJdSGfBtmSNbt tgMZH9aA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPZ-00000005364-0hBE; Fri, 10 May 2024 11:27:37 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPQ-000000052zm-2oOJ for linux-arm-kernel@lists.infradead.org; Fri, 10 May 2024 11:27:29 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-dc693399655so3067850276.1 for ; Fri, 10 May 2024 04:27:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715340445; x=1715945245; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=1X74EzIbLgwkSZ3RCE9vimfqgbYyFErrsdDhyQP2Whk=; b=1nDz8HJrN6/q1oSu3Lxc7GTCsSdxnPd2Ky5l4BM9NVBi9TvLl5+Yzei39T9PAwC16C mmiDI1riG7mabzJTMJcoG5b3w8WtFMv6BMIWwNiQ+3mYBlqDht7Kpzd6yE+nks2g8die Uz7mQRAAItiqnDXO7PxesCz+Y8O0AIXC2RPOV1ykw2oVaxIUmjvTo8lq0Kq0q9+HxxjX 6PpoxpE0XuWvghBpzE+mtYvR+xxZTWBrIpt2LbPVnKGiaSVL3y0fJPudDb+DHUKD7Slz iySzWnb8ROyapBRL1Vmrjeump1GGLKbaiVpbK4iXvx2kyZYqjfeSwVU4bqfTOwj43ST1 fq+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715340445; x=1715945245; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=1X74EzIbLgwkSZ3RCE9vimfqgbYyFErrsdDhyQP2Whk=; b=to3dQOQdvNcP/ZiQ5EloeSY3uPgiQxYIgqkxhtIJyyOfoKkzZ/kBz6iks0x1Sv3TXn HXXkWmO8Y6r2G2b1leSxyjaXdA+wtFq0G3xStQRJ7H5tvQwCh0cX7KdXSRm9FlJ32Baf eXimIdE1IeYSbgWxtvabVAy1dSe4JQVmIePUOMqVr4D0+qRuvzQS0vvq0JZTRJ52mBnZ DqisxwfuR91xfes/vsOXxqee2yi16lvQ2WpojMIldROoaYeIb3uvo+tgOeefCxDpfzon Z5nReEPIsldwmgmz7uhe0+s0K/pKVMD0NlocNKs09I8Kqjn6JhonsQ48qefbTWPo1/z2 nicw== X-Forwarded-Encrypted: i=1; AJvYcCUKXusi6E4h9lDZ9TPrs8Vqq/rAhe/8PFjRoh4LpfZGoJMNPliQNe4/uNYHjwMrGmBCHHqv1cR6EQB37p0GDXnHNwxZeN6sI1C+AjfMOXQpdlCLi40= X-Gm-Message-State: AOJu0Yy88H/M9zawSmIuCtSYjzsErLcnDKINqnCN04QcNdjgJmR0PpYm R0WF9wT1DQ4XY+8Xrq0Cq4g1Gz+QfwT7ty3Baz8ex9I6/kIk7W6mssDHDoeoKffIEzBq3WxRfw= = X-Google-Smtp-Source: AGHT+IE2vI8V+31DiouhAYXxHwv2C3egPY5JlN+7fyVhlwS8yXv78dpY0cirHvo9+5p4Wx/GqO7X9Ryw0g== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6902:2b88:b0:dcd:3172:7265 with SMTP id 3f1490d57ef6-dee4f3036e5mr593281276.8.1715340445332; Fri, 10 May 2024 04:27:25 -0700 (PDT) Date: Fri, 10 May 2024 12:26:32 +0100 In-Reply-To: <20240510112645.3625702-1-ptosi@google.com> Mime-Version: 1.0 References: <20240510112645.3625702-1-ptosi@google.com> X-Mailer: git-send-email 2.45.0.118.g7fe29c98d7-goog Message-ID: <20240510112645.3625702-4-ptosi@google.com> Subject: [PATCH v3 03/12] KVM: arm64: Pass pointer to __pkvm_init_switch_pgd From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240510_042728_741129_D7965F1D X-CRM114-Status: GOOD ( 13.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Make the function take a VA pointer, instead of a phys_addr_t, to fully take advantage of the high-level C language and its type checker. Perform all accesses to the kvm_nvhe_init_params before disabling the MMU, removing the need to access it using physical addresses, which was the reason for taking a phys_addr_t. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/kvm_hyp.h | 3 ++- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 12 +++++++++--- arch/arm64/kvm/hyp/nvhe/setup.c | 4 +--- 3 files changed, 12 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 96daf7cf6802..c195e71d0746 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -123,7 +123,8 @@ void __noreturn __hyp_do_panic(struct kvm_cpu_context *host_ctxt, u64 spsr, #endif #ifdef __KVM_NVHE_HYPERVISOR__ -void __pkvm_init_switch_pgd(phys_addr_t params, void (*finalize_fn)(void)); +void __pkvm_init_switch_pgd(struct kvm_nvhe_init_params *params, + void (*finalize_fn)(void)); int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, unsigned long *per_cpu_base, u32 hyp_va_bits); void __noreturn __host_enter(struct kvm_cpu_context *host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 2994878d68ea..5a15737b4233 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -265,7 +265,15 @@ alternative_else_nop_endif SYM_CODE_END(__kvm_handle_stub_hvc) +/* + * void __pkvm_init_switch_pgd(struct kvm_nvhe_init_params *params, + * void (*finalize_fn)(void)); + */ SYM_FUNC_START(__pkvm_init_switch_pgd) + /* Load the inputs from the VA pointer before turning the MMU off */ + ldr x5, [x0, #NVHE_INIT_PGD_PA] + ldr x0, [x0, #NVHE_INIT_STACK_HYP_VA] + /* Turn the MMU off */ pre_disable_mmu_workaround mrs x2, sctlr_el2 @@ -276,15 +284,13 @@ SYM_FUNC_START(__pkvm_init_switch_pgd) tlbi alle2 /* Install the new pgtables */ - ldr x3, [x0, #NVHE_INIT_PGD_PA] - phys_to_ttbr x4, x3 + phys_to_ttbr x4, x5 alternative_if ARM64_HAS_CNP orr x4, x4, #TTBR_CNP_BIT alternative_else_nop_endif msr ttbr0_el2, x4 /* Set the new stack pointer */ - ldr x0, [x0, #NVHE_INIT_STACK_HYP_VA] mov sp, x0 /* And turn the MMU back on! */ diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index bcaeb0fafd2d..45b83f3ed012 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -314,7 +314,6 @@ void __noreturn __pkvm_init_finalise(void) int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, unsigned long *per_cpu_base, u32 hyp_va_bits) { - struct kvm_nvhe_init_params *params; void *virt = hyp_phys_to_virt(phys); typeof(__pkvm_init_switch_pgd) *fn; int ret; @@ -338,9 +337,8 @@ int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, update_nvhe_init_params(); /* Jump in the idmap page to switch to the new page-tables */ - params = this_cpu_ptr(&kvm_init_params); fn = (typeof(fn))__hyp_pa(__pkvm_init_switch_pgd); - fn(__hyp_pa(params), __pkvm_init_finalise); + fn(this_cpu_ptr(&kvm_init_params), __pkvm_init_finalise); unreachable(); } From patchwork Fri May 10 11:26:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13661433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 90053C25B5F for ; Fri, 10 May 2024 11:27:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=dyi4MUei8EgEURHfIeKLIWoznIraBiyF54mkHrMUPE0=; b=wuyIV8jIHEAAwb2J9iIa8IFaX1 ul3dvuTEj4fFj09QKljkSwlMrZSQYfNFtgIRM5hsCAyvemFcQfB8AmwPngQXV8JhslnuqYh/9/nyv sfSHx57gQWEFkdmMz8LtMqp8jRTQ8bqLKI8PQ5MqBctOvB2vxcxSL1ENZ/lXiuCDOXGrdcyFHHiI+ yEga9jQK05ctA/GdtUFBJaP4FevlhyTn8sj+5RPwWz84CpiyCtO1XusHjStfsCEfwQYALPXDK+K66 VGCQpIJwN/vAQ/aDnydDaPV0nCaSYngsix3MqH6qOe9McwET6PCF0RyNH4XdKFWncW2xMW/j964OF 2yqEuOsg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPa-0000000536p-0Vns; Fri, 10 May 2024 11:27:38 +0000 Received: from mail-ej1-x64a.google.com ([2a00:1450:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPV-0000000531J-1u5r for linux-arm-kernel@lists.infradead.org; Fri, 10 May 2024 11:27:35 +0000 Received: by mail-ej1-x64a.google.com with SMTP id a640c23a62f3a-a59c69844aaso112190466b.0 for ; Fri, 10 May 2024 04:27:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715340448; x=1715945248; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=gLg0BCgwrJPFXcSMjQCXQoWFMEc6ibsqpEm8oPH+eqc=; b=sW0CdvCvXMTzuqjJVHq3U1/Syv6L52hVamqfxtCUkfHKJvVnTYs7JmsAxoy6A5RRsP xJvYfaTNTsuAdf52uPHj5AzJdajO1j2PA5DtZS2CB8cz5k/EIxIu3TN6gdmb5ybm435S 6hcwNzcT7qdqKv4QxIp+Q2VrS3lIvRyHy9m5UL53MWqvqngkuDpdToPVoU4TkIQ57lhI 2dh+muUng+OzEWwWu8MHevfefUeYqGJzNqU6b0DNZAyzzB9gNuygORlEOa5uDEot2aBQ lcWd7uI1k0yNU194BR0sZreU32xe44wS8Gqpzu4j5Uyaf4TMhQ1cKI8HK2xnYNXAlXj8 qbOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715340448; x=1715945248; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=gLg0BCgwrJPFXcSMjQCXQoWFMEc6ibsqpEm8oPH+eqc=; b=TH4FUZN3T7gcB2ZU1Xmu01n5ZYkO0Vymc2n1UmvFKHlemg+gpfjjiBU9XTIJP3cgT/ v3IYyiFNKYTfrLAxRhrX3cWbno4+1LSYWPJymR3NRxdS7b2Puw3Pw/FCRpiUAWmxTDHb ggVdS1+/AuqF/OSvJhS2JsznuG2n8bWQW3MtcNIS74tVKwJyWSIcL3kzs9TJwX90MYMg QmtLrQJagj0cc8WiQKvbrXTW0T2PYniVD0TqY8mz9EpxzGTdX5uEwZ4QMsLHFuxzuEsi MHfWQA3tYHBr8vpvUp5moJq7t1qh+n9dd+IaRijcvs8+b1cW789DsfMMmKDvjkam/joC KCgA== X-Forwarded-Encrypted: i=1; AJvYcCUcoGzoUAjMUq56DvwjHRkCcK+FTy6LtPbjLjYT8i4Wjl6TBB9Hc+gCoO8zAaPU+8vI47LJOdb8q8rTXn5LvcTcMGXTR05w/La9XUpj7A8pXqB7a+c= X-Gm-Message-State: AOJu0YxYqyjgNLuXkIyADcvyQFgNhq6H2DwDhNtBeC1dLhc84sFeemj1 bfFMM/96Ul0JpYgD8F9CrPS91YIqiFSUGp9fkuLjM5rv1B4H8HT1WPbfZ/yqG6i/TyLS9iqwZA= = X-Google-Smtp-Source: AGHT+IFepWD3/v8BdAdYkb3xCeH3t+IvJIqzRXw8IRofPX/9PW2hDEA+udTBjtBXHF9E/IajiV/5ddxCaw== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a17:907:12c9:b0:a59:d5f7:e56c with SMTP id a640c23a62f3a-a5a2d58a6f4mr225866b.5.1715340447570; Fri, 10 May 2024 04:27:27 -0700 (PDT) Date: Fri, 10 May 2024 12:26:33 +0100 In-Reply-To: <20240510112645.3625702-1-ptosi@google.com> Mime-Version: 1.0 References: <20240510112645.3625702-1-ptosi@google.com> X-Mailer: git-send-email 2.45.0.118.g7fe29c98d7-goog Message-ID: <20240510112645.3625702-5-ptosi@google.com> Subject: [PATCH v3 04/12] KVM: arm64: nVHE: Remove __guest_exit_panic path From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240510_042733_634494_0CD4BCEB X-CRM114-Status: GOOD ( 16.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In invalid_host_el2_vect (i.e. EL2{t,h} handlers in nVHE guest context), remove the duplicate vCPU context check that __guest_exit_panic also performs, allowing an unconditional branch to it. Rename __guest_exit_panic to __hyp_panic to better reflect that it might not exit through the guest but will always (directly or indirectly) end up executing hyp_panic(). Fix its wrong (probably bitrotten) ABI doc to reflect the ABI expected by VHE and (now) nVHE. Use CPU_LR_OFFSET to clarify that the routine returns to hyp_panic(). Restore x0, x1 before calling hyp_panic when __hyp_panic is executed in host context (i.e. called from __kvm_hyp_vector). Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/hyp/entry.S | 14 +++++++++----- arch/arm64/kvm/hyp/hyp-entry.S | 2 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/host.S | 8 +------- 4 files changed, 13 insertions(+), 15 deletions(-) diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index bcaaf1a11b4e..6a1ce9d21e5b 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,7 +83,7 @@ alternative_else_nop_endif eret sb -SYM_INNER_LABEL(__guest_exit_restore_elr_and_panic, SYM_L_GLOBAL) +SYM_INNER_LABEL(__hyp_restore_elr_and_panic, SYM_L_GLOBAL) // x0-x29,lr: hyp regs stp x0, x1, [sp, #-16]! @@ -92,13 +92,15 @@ SYM_INNER_LABEL(__guest_exit_restore_elr_and_panic, SYM_L_GLOBAL) msr elr_el2, x0 ldp x0, x1, [sp], #16 -SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) - // x2-x29,lr: vcpu regs - // vcpu x0-x1 on the stack +SYM_INNER_LABEL(__hyp_panic, SYM_L_GLOBAL) + // x0-x29,lr: vcpu regs + + stp x0, x1, [sp, #-16]! // If the hyp context is loaded, go straight to hyp_panic get_loaded_vcpu x0, x1 cbnz x0, 1f + ldp x0, x1, [sp], #16 b hyp_panic 1: @@ -110,10 +112,12 @@ SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) // accurate if the guest had been completely restored. adr_this_cpu x0, kvm_hyp_ctxt, x1 adr_l x1, hyp_panic - str x1, [x0, #CPU_XREG_OFFSET(30)] + str x1, [x0, #CPU_LR_OFFSET] get_vcpu_ptr x1, x0 + // Keep x0-x1 on the stack for __guest_exit + SYM_INNER_LABEL(__guest_exit, SYM_L_GLOBAL) // x0: return code // x1: vcpu diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 03f97d71984c..7e65ef738ec9 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -122,7 +122,7 @@ el2_error: eret sb -.macro invalid_vector label, target = __guest_exit_panic +.macro invalid_vector label, target = __hyp_panic .align 2 SYM_CODE_START_LOCAL(\label) b \target diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 19a7ca2c1277..9387e3a0b680 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -753,7 +753,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __kvm_unexpected_el2_exception(void) { - extern char __guest_exit_restore_elr_and_panic[]; + extern char __hyp_restore_elr_and_panic[]; unsigned long addr, fixup; struct kvm_exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); @@ -776,7 +776,7 @@ static inline void __kvm_unexpected_el2_exception(void) /* Trigger a panic after restoring the hyp context. */ this_cpu_ptr(&kvm_hyp_ctxt)->sys_regs[ELR_EL2] = elr_el2; - write_sysreg(__guest_exit_restore_elr_and_panic, elr_el2); + write_sysreg(__hyp_restore_elr_and_panic, elr_el2); } #endif /* __ARM64_KVM_HYP_SWITCH_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 135cfb294ee5..7397b4f1838a 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -196,19 +196,13 @@ SYM_FUNC_END(__host_hvc) tbz x0, #PAGE_SHIFT, .L__hyp_sp_overflow\@ sub x0, sp, x0 // x0'' = sp' - x0' = (sp + x0) - sp = x0 sub sp, sp, x0 // sp'' = sp' - x0 = (sp + x0) - x0 = sp - /* If a guest is loaded, panic out of it. */ - stp x0, x1, [sp, #-16]! - get_loaded_vcpu x0, x1 - cbnz x0, __guest_exit_panic - add sp, sp, #16 - /* * The panic may not be clean if the exception is taken before the host * context has been saved by __host_exit or after the hyp context has * been partially clobbered by __host_enter. */ - b hyp_panic + b __hyp_panic .L__hyp_sp_overflow\@: /* Switch to the overflow stack */ From patchwork Fri May 10 11:26:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13661436 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C3437C25B74 for ; Fri, 10 May 2024 11:28:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Q2dnU3Fd16Di75PWtFYK9mATfWSF1Eo5Ck2GBz5b4oM=; b=eoU+bbpbzsOlT+R/37XNUWjbJX ssMHP1E3WkxWyVZMs1RjXrFd1hgoo/iB8afxMhpW4kH9Y2jJJuhQLqC5njkb/aUURScjbVRuZ/G04 ELtR2aDsL6Zx6BjVIvbuP8HxFdZ2525YZTjZlqR+pSm3HOQM9IQ5T8zKs/+S5KWWV/Wt+QKxrvshg X0LW/F8hss9negHB2e51NilJDpCnRo8ehfVOdzrJUPGPd0icqPASFH1PoNV2DYTJdwRqzh9WEH+YV 1z16sAoQ4fOZfSaLloYX5Icxty0JT9QI5ZEhOADSyRozJUlmT6BPg3BprpV97I0RBN596xiHTYQp1 Di4z9GxQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPh-000000053BO-0dAn; Fri, 10 May 2024 11:27:45 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPX-0000000532U-12yD for linux-arm-kernel@lists.infradead.org; Fri, 10 May 2024 11:27:36 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-61dfa4090c1so27381707b3.3 for ; Fri, 10 May 2024 04:27:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715340450; x=1715945250; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=qcEzQNh9dWuFIVjzg9t8TylWGTumh3MZGtqI9gQXfso=; b=rqliXSwBBAjls1HBFP/R3yiHpYmgkElOwg+//u2QHwex37L9ZBZbuzoGSetik1ufhD H4NLC+t3bfhj7iygTO3IgN/WAEoHUPl5gt58ccjE4x66cA/Od+drfJ0dOmp6uO0Wt0Nj dVEGl8bT9pzbnTEF0+rfMaYa6c73tc6EfA+C7ZlMjFVhrl9k8jBd/iBspa36n/0fF37H Bvo+vVlMJq+5k5voOwxiCJwR9ZovTrZy6mM9pKgZVrfRMLcJxQaQr9TSdYFlHTftniiG +Hjvrpeo0riNdAZhInfy3ElyS6dYRKPXF5f0zszpmrGCyY0a/OMdxBLJQ3SQETWuhSes +bcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715340450; x=1715945250; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=qcEzQNh9dWuFIVjzg9t8TylWGTumh3MZGtqI9gQXfso=; b=UZx3+Xi7XwJ9PGE2K/6KmkPyzyqvKFTDo1kt4naeF6bHpANnDnSNweR7ZjnAXTl4nk VRenMouGdx/lzQaCr4SOSi7uSidodyf/AA1dEN2oocd65Y5hfuKjdAZjw0vPDH4OvR28 0U8jh/GqQ3RAj3+af5xizFCG9LEz9rTl+pP3WbXf55ljXCueHcKIvx+9nzaZj7mVBW+Z GgPK7AfA0CaJWidUaSEokmjDw8TIbyWL25xW1nODKyegV40uBad3ieTLwyN9p5ieu92E SGk+gQgW0X3kHBmZZnVrPILYQ4RocJvm8FAvckInCjD/AzRk5Cd+e0FGUOlRBYlgZAcz lFjA== X-Forwarded-Encrypted: i=1; AJvYcCVSQBcgujxsrH9mOPQ4nvwiGy51PsMi62Vd6aS8vwnqfETnxzrMCa/DY/ds75rWZTaZFzuVHmtVwraRa11S914+ab0o4hjYXbEfjvGPtaPvzQlEWtY= X-Gm-Message-State: AOJu0Yw13Y1d2rzC33qt3zQEd3AhOyZIsv2WWZNj02tcVOph4ZizDDci X4K+HlsTa4rSQod7uX2wCFYli54IWwkqlXWjZprSfzWENzB92WKfkt50tENZYSkrZQCB+PS3og= = X-Google-Smtp-Source: AGHT+IF00DsgRNr4wMWPoZLxFG4BzAtNHmn3WH+94RNNB3FGneRo+0N3ZZ2Yl4THjXTnNjJION1G71wXVw== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:690c:4507:b0:61b:3b02:6901 with SMTP id 00721157ae682-622b001dd60mr5562717b3.9.1715340450084; Fri, 10 May 2024 04:27:30 -0700 (PDT) Date: Fri, 10 May 2024 12:26:34 +0100 In-Reply-To: <20240510112645.3625702-1-ptosi@google.com> Mime-Version: 1.0 References: <20240510112645.3625702-1-ptosi@google.com> X-Mailer: git-send-email 2.45.0.118.g7fe29c98d7-goog Message-ID: <20240510112645.3625702-6-ptosi@google.com> Subject: [PATCH v3 05/12] KVM: arm64: nVHE: Add EL2h sync exception handler From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240510_042735_414759_920460A9 X-CRM114-Status: GOOD ( 12.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce a handler for EL2h synchronous exceptions distinct from handlers for other "invalid" exceptions when running with the nVHE host vector. This will allow a future patch to handle kCFI (synchronous) errors without affecting other classes of exceptions. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/hyp/nvhe/host.S | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 7397b4f1838a..0613b6e35137 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -183,7 +183,7 @@ SYM_FUNC_END(__host_hvc) .endif .endm -.macro invalid_host_el2_vect +.macro __host_el2_vect handler:req .align 7 /* @@ -202,7 +202,7 @@ SYM_FUNC_END(__host_hvc) * context has been saved by __host_exit or after the hyp context has * been partially clobbered by __host_enter. */ - b __hyp_panic + b \handler .L__hyp_sp_overflow\@: /* Switch to the overflow stack */ @@ -212,6 +212,10 @@ SYM_FUNC_END(__host_hvc) ASM_BUG() .endm +.macro host_el2_sync_vect + __host_el2_vect __hyp_panic +.endm + .macro invalid_host_el1_vect .align 7 mov x0, xzr /* restore_host = false */ @@ -221,6 +225,10 @@ SYM_FUNC_END(__host_hvc) b __hyp_do_panic .endm +.macro invalid_host_el2_vect + __host_el2_vect __hyp_panic +.endm + /* * The host vector does not use an ESB instruction in order to avoid consuming * SErrors that should only be consumed by the host. Guest entry is deferred by @@ -238,7 +246,7 @@ SYM_CODE_START(__kvm_hyp_host_vector) invalid_host_el2_vect // FIQ EL2t invalid_host_el2_vect // Error EL2t - invalid_host_el2_vect // Synchronous EL2h + host_el2_sync_vect // Synchronous EL2h invalid_host_el2_vect // IRQ EL2h invalid_host_el2_vect // FIQ EL2h invalid_host_el2_vect // Error EL2h From patchwork Fri May 10 11:26:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13661435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8E91EC25B5F for ; Fri, 10 May 2024 11:28:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=hobhivw7M3SvUJzOg845lnW335elVN0YR2ROC1ACUlc=; b=kfgmLqZ9suH3hsVkJY3mbo2KeK E0L/50yoR7ZwLoJSoBCh1chFZ5Ssqz0M0HTsjiWkLdyfF64HduAXgScZ6zjaCOqZlw/Bs02hovZWq 5i+Cj8lZSnNNUAeLwoT41JYUhslvQIewz+y6lVBs8WYlRoCv47zYTZANignWR2yOmXMzojgeMvWsq Iy3yy/k2VUUJzpSLIZDnRtzs6tzOT8Zpb0ZPg8mD7DJyCV1Y1VjJovKaXyf6pTCbnMMrYhiV95Jkr GFYtTt19gaUeDArYyIs1/u4Phuxa+ys4j7qfI18cGMCu23uFu1H6ELDE25d0U0VA1qUoXQfEuJmIE 1UabPUJQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPj-000000053Cx-2YQP; Fri, 10 May 2024 11:27:47 +0000 Received: from mail-ej1-x64a.google.com ([2a00:1450:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPY-0000000533O-1AOA for linux-arm-kernel@lists.infradead.org; Fri, 10 May 2024 11:27:37 +0000 Received: by mail-ej1-x64a.google.com with SMTP id a640c23a62f3a-a59a1fe7396so121441366b.3 for ; Fri, 10 May 2024 04:27:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715340452; x=1715945252; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=49RDvmrkbnpDyrHr4vuzEjRKizGtD9K6IaD6JegXO3A=; b=XXIeomYM1jpsx8rZ6aZTKpxINyi/vntY6J9fi3w1j93ZLUrRegy/Y3JAWAshnctAP+ 5XHvyE9ersLl6NEox9LGxGEeb/WyxWdL/ZDiUuaXpLwHcrCDqCEaQ2yrWgmuULvcMFpD yz8bcxr8njiWdEb5v56CApsltNiEnBzhxVH7EiWukXZ2C8VUfEBSjUt6COV6VcPE+PtD vTC2GH/94PLYLBA7Ldo5mMIwFr6xsuDcnH+5tnEIjpob/Ougzxy5XSlUhDTwBYVBEkHr RAxh+E7r2BwcHeZUe6TL4KyDVQgjhAaVgKAbqd0QM4kINj0D4G85DJ1vwrQq18AqnxNX 0gng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715340452; x=1715945252; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=49RDvmrkbnpDyrHr4vuzEjRKizGtD9K6IaD6JegXO3A=; b=eurQomGPV6JnKZzkLRuW+g22YsA1iq5YyCo0tvCF4BhwDITMWycaTm818TKZ3SQoPt J+KbEHYB7YKWMMbhJXp9KOR5C5dDc0izwSO6kISvMRTkFx+IGX9XQzBTWmgz36qplW1l GR2nENyijq1TpH1sfLS625RWFexpuIVpIGB/ic239/X4Q6+PUQag+NLpZucai/BtX7Im TfSB7qjt7z3JBM0W5FgZszmqY+jZKqmNfZ6ofDVB2eyfA/9a9IPfqqiY2xGmwLhMvvVB ybY67f3Fqbs9TXqZEjMloERRdkuwNnWBuP5m1eD+5ACFUQ7hj94IJUlHqcnCut+oUAqR 3ETQ== X-Forwarded-Encrypted: i=1; AJvYcCUjinLoNInGlz5Z+d9yKmttOR67G9DJzeDE1D4JWhyGdv+RoQMIwoxKJqfqjU+vRAwI7UFp0KnJrVUYNl4UQB2ArptaXIv5xulmhFiqnPhcPeypr0s= X-Gm-Message-State: AOJu0YymQyPRNbCNgvZ6E+ThE+uwiqthLM4KXPjlFqF73Fl78bg8yWBN IVNZwMgMtQwSC5LQ39FAgnqSTAM2zddfzVi1v/bTxfuGCu7FHuzS5+I8Xsnvbk0vfU3GWp4K5Q= = X-Google-Smtp-Source: AGHT+IGbgNiv89bjQrlHWpgNH9EcVBcfDrRvtOjM5hynq/y8S7do68WKAWDs0MZ3rKCKgKDcmOOmZ9+0TQ== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a17:906:1188:b0:a59:c39c:f202 with SMTP id a640c23a62f3a-a5a2d5a6a3amr235366b.5.1715340452299; Fri, 10 May 2024 04:27:32 -0700 (PDT) Date: Fri, 10 May 2024 12:26:35 +0100 In-Reply-To: <20240510112645.3625702-1-ptosi@google.com> Mime-Version: 1.0 References: <20240510112645.3625702-1-ptosi@google.com> X-Mailer: git-send-email 2.45.0.118.g7fe29c98d7-goog Message-ID: <20240510112645.3625702-7-ptosi@google.com> Subject: [PATCH v3 06/12] KVM: arm64: nVHE: gen-hyprel: Skip R_AARCH64_ABS32 From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240510_042736_436352_0B495CE8 X-CRM114-Status: GOOD ( 10.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Ignore R_AARCH64_ABS32 relocations, instead of panicking, when emitting the relocation table of the hypervisor. The toolchain might produce them when generating function calls with kCFI, to allow type ID resolution across compilation units (between the call-site check and the callee's prefixed u32) at link time. They are therefore not needed in the final (runtime) relocation table. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/hyp/nvhe/gen-hyprel.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c b/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c index 6bc88a756cb7..b63f4e1c1033 100644 --- a/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c +++ b/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c @@ -50,6 +50,9 @@ #ifndef R_AARCH64_ABS64 #define R_AARCH64_ABS64 257 #endif +#ifndef R_AARCH64_ABS32 +#define R_AARCH64_ABS32 258 +#endif #ifndef R_AARCH64_PREL64 #define R_AARCH64_PREL64 260 #endif @@ -383,6 +386,9 @@ static void emit_rela_section(Elf64_Shdr *sh_rela) case R_AARCH64_ABS64: emit_rela_abs64(rela, sh_orig_name); break; + /* Allow 32-bit absolute relocation, for kCFI type hashes. */ + case R_AARCH64_ABS32: + break; /* Allow position-relative data relocations. */ case R_AARCH64_PREL64: case R_AARCH64_PREL32: From patchwork Fri May 10 11:26:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13661437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ADE0BC25B10 for ; Fri, 10 May 2024 11:28:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ilAmgBr3BuVniyUkM2XRpw73FRNJaBTfLJNVgvVPIxQ=; b=2etcb4sJD+gSavrBZXwslncwXz 9m1pdxrVVGki8Qk0hvgNdAE/PnRZMNd49YrSpsfFagspPU8vPzhbb+64pQkVkR1WfFj/sEh6P3xIU fU2thangr7eD1+ywUP2LoreYCk4M4raPgppl4du2ZkyyPj9eNGtVW3zc4mnBb9FBinJQd843pBjW4 Pbk2+BbGBAxRz+nCZWOqphDiFOdr9KLdd8kqFMcqH/OF1+6XaZt8SU7Q1fDaU9vhjtq+kq4zfYxRM yUr/fME7Btz//b4TA5PZ551VKxGFihdpmArm5+TWPKgBTdg7ES1uTngujREQ27RTnXU/Et7/y3ILm GfsoNsXA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPl-000000053Ex-3DZI; Fri, 10 May 2024 11:27:49 +0000 Received: from mail-ej1-x649.google.com ([2a00:1450:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPb-0000000534v-16La for linux-arm-kernel@lists.infradead.org; Fri, 10 May 2024 11:27:40 +0000 Received: by mail-ej1-x649.google.com with SMTP id a640c23a62f3a-a5a180153aeso122655866b.3 for ; Fri, 10 May 2024 04:27:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715340455; x=1715945255; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=8DKhkb6dSizyRVlRq/XnhdKj6+BlqFPiK9mn9/+w2yc=; b=mAhOqwzWTwZ4KrRw/jWgv/vylVZejb29V/ys8svUI+msIwsRLu2bva4MRzNxuLQTPW MrRFUK7sJKYdUTq9Ar8anmAlD8RaJHvl/92o8xTJiiqZ902QEwlXLBYKWXXSoabPFyLV 3MC+P6N4t0ZjlWGwTM7EI5PMwBOu0xugsKAtDKS3Ot8iquaZUp5mI694d0nEU3ximRpK Rlehwxn0c9MoVwQ4X1PMfkhiUvN2Ehm1O2JM50jHKAjOSixgtL0X4sItf5kncfQz6pdY yOJSg7CE5pGbbyxabGwzhzqYUR3ds7y/MjNE8ftVLmZIVAYhxffvauGGdX/iodtbTjVI ZW7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715340455; x=1715945255; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=8DKhkb6dSizyRVlRq/XnhdKj6+BlqFPiK9mn9/+w2yc=; b=WzSE6Lgelive7DzbeZxreQNkGpeCj1bnnoxqG1whqY+MEcc9a7uYcUyz3niRUdoEkM ifQN9d9EPz//3OoOvacXfWaW+SJOhAT7iyCNis0tyfAvwOIxwH/UryfAIQOwioBIaaub lYlJnjPQMp/l8lYHzyRvZxOp+tGttKtFvoGSd792Ok4HE/9Vol3QE1DASYr2KWIFEDND NxP4sd0A0RogPa3ymVSCzDeLxN7bB4rmiqVYNcKVrqlnM3Xxpyj/7ukUu+qUudpYgQ6l 3ctcZPgE0f5YlgM8zCmJ4a3EfixRs5zhEEtE2GRzmrSzrt+8YyoEKA29RAJrQpMPxdgG ErYg== X-Forwarded-Encrypted: i=1; AJvYcCUeIHvqqpITqwTldAgmNwUfhbcbj2q9nX+bNpodxMceoepSMuzDknVYFblCYxwZOaC7d5hwAYDastHHwemtr/qlgyKmmB3E0rrO28c5w1EfVDY/pHg= X-Gm-Message-State: AOJu0YwFpE1wBO6gmYy2rojCPQjeM0sSl0QzTq6dNy4NwMhH74m5xzMZ qbxnmeH/YZ+GFsM10FBJi0iNVpXn0moYxoBGRcZLvRhyFkktIRUKskgg3nGGSjFaVSEEw/BWEw= = X-Google-Smtp-Source: AGHT+IGkpwxqeSN6EavlDA+BXVJp1GAbnEIVQnql+hC6cCYj3jthHFCaBNvnkUk/0Xp8MBeg6nGyR3xjIA== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a17:907:d383:b0:a59:dcdc:a490 with SMTP id a640c23a62f3a-a5a2d65d604mr184166b.12.1715340454681; Fri, 10 May 2024 04:27:34 -0700 (PDT) Date: Fri, 10 May 2024 12:26:36 +0100 In-Reply-To: <20240510112645.3625702-1-ptosi@google.com> Mime-Version: 1.0 References: <20240510112645.3625702-1-ptosi@google.com> X-Mailer: git-send-email 2.45.0.118.g7fe29c98d7-goog Message-ID: <20240510112645.3625702-8-ptosi@google.com> Subject: [PATCH v3 07/12] KVM: arm64: VHE: Mark __hyp_call_panic __noreturn From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240510_042739_368436_855B6380 X-CRM114-Status: UNSURE ( 9.81 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Given that the sole purpose of __hyp_call_panic() is to call panic(), a __noreturn function, give it the __noreturn attribute, removing the need for its caller to use unreachable(). Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/kvm/hyp/vhe/switch.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 1581df6aec87..9db04a286398 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -301,7 +301,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) return ret; } -static void __hyp_call_panic(u64 spsr, u64 elr, u64 par) +static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) { struct kvm_cpu_context *host_ctxt; struct kvm_vcpu *vcpu; @@ -326,7 +326,6 @@ void __noreturn hyp_panic(void) u64 par = read_sysreg_par(); __hyp_call_panic(spsr, elr, par); - unreachable(); } asmlinkage void kvm_unexpected_el2_exception(void) From patchwork Fri May 10 11:26:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13661438 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 30447C25B10 for ; Fri, 10 May 2024 11:28:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=DlKA7Yk7bIdyq3cJfjD0Tz7p6O6rYENS/j+St490Dps=; b=INbgZSZtnrxvh1sP64rp9Nwawf 2k+37xzjiBrs0J3W+js+DwBz7FrAp/LiCpabP99jCVzYqaOSLXnb/CESb5fvGISKSR9teF7qpSyF6 9Yx1rWBGG/9l1v5g2qoRz0WN5mSm8cErH4xnnI8TvfqJ1m+i5DVbBB3eGXkUcod5BDN+XaQ2voq9t D3l9f94Y5kpILFW3V1fBt4DMlefPdBGGYLzWgJf8W0Fy5URt3taMg4IkJR7mUTQ8LwtODWSO7n9Yr FEuIOtUxsXq4Cl90d8n9EPFLl3WXLTeZCMpjLNsTd/DNMg5mqr4vv0801i7o7V99DOCQsf26CIeM7 MVkuYUjA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPs-000000053JM-1cKv; Fri, 10 May 2024 11:27:56 +0000 Received: from mail-ej1-x64a.google.com ([2a00:1450:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPc-0000000536e-3g1X for linux-arm-kernel@lists.infradead.org; Fri, 10 May 2024 11:27:42 +0000 Received: by mail-ej1-x64a.google.com with SMTP id a640c23a62f3a-a59a17f35c8so123010066b.0 for ; Fri, 10 May 2024 04:27:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715340457; x=1715945257; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=BYiDxgRL4/PJcTPYc8te2M3RBep5LmiQ+KSg/yVolMc=; b=c+tuxYwT2EJlzausBFAn8XTsIBSowG9elXSRlr661uSvnIJ0QBsnDD4bU0AkyIQRko +EAqWX/vxDYqQAvLzb35yg3w4+LH47ZcCUk0dprtgQaD2bxjOwkCTCiOVMgP1JoGhj4x vtHbl7/tfJ02dLHMbljmVTPBgNliY7Ra2jdiCux3G7BAAJ8y+uzBxZOWVpwcgy7OxBhD 4S81bQYdL5Sd/rzYf0BJAI0uDVCwALeGwVFlTuY5cVSxMKHuH1ZXF4QFbOC4qJG6TFKc 40eZtKNGixbj1xHPFF28rSzy8KOqDw6Zx8PqEivKDiaVyjLF7BaPUr9sJMMrWVUfxc8w JWAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715340457; x=1715945257; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=BYiDxgRL4/PJcTPYc8te2M3RBep5LmiQ+KSg/yVolMc=; b=GsLwKCcSccQgw6bVADTKxuhT/pF7N9MUF/JrcmzEYghpw/4RnaFcONpXsfsXONhwvY I2Xx0xoMxIOPWNjJgGAVgO61JYewqgBkrCZvm2Usy5c/dEcR8jb/RTUC2m70v8zOr3oz UfuW4IrZqSn9e814nH6WnKff8nP3ztQM2N4wPnKIb071tSzVC5nQuYdskxkhj9P1+MUw Q3jtjc06NOan6GjEumdcFAWjojgQcyedY3NwYPme4Qk8lyeHLjBxsGVSgoTfQ5Kvj9jo 2d745WImvxIsCMFgs3CNiNBB8SABRZ/8Qc+/eUlcaFtWG3cUFZFG+Z2A1aC3xem57mxD oyQA== X-Forwarded-Encrypted: i=1; AJvYcCUB0plfUAzN/+JjzhzZJzmCkAHrQ6A5YsUFk0kSE8syMJNU+AXYMNA0nBJ3bl5ua+VgJQtiJSzIIinVsHuNdvO1upLRpulmaImaOgjppvqg6IEwGjc= X-Gm-Message-State: AOJu0YytBJowzuSTki0fh+dNQnaz8GZjS5kN0mHGDwJom1HLknHoOXKj Bj8UALnAj6ptekpVINqWW2tpHEtyA85Ce0UaJYuMjXi2ip5+AT1oPwvRtEySCgcX7xM+p2tdgw= = X-Google-Smtp-Source: AGHT+IHFd5o0vQu25tuDgldMkm4aPBD8ZTZa88+e/50cMnpBaWWfrW1M4Ov32MuLEgdpxktrbzGRQJLKwg== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a17:906:27cc:b0:a54:c131:8128 with SMTP id a640c23a62f3a-a5a2d6796c7mr215866b.14.1715340457059; Fri, 10 May 2024 04:27:37 -0700 (PDT) Date: Fri, 10 May 2024 12:26:37 +0100 In-Reply-To: <20240510112645.3625702-1-ptosi@google.com> Mime-Version: 1.0 References: <20240510112645.3625702-1-ptosi@google.com> X-Mailer: git-send-email 2.45.0.118.g7fe29c98d7-goog Message-ID: <20240510112645.3625702-9-ptosi@google.com> Subject: [PATCH v3 08/12] arm64: Move esr_comment() to From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240510_042741_086462_69C13AB1 X-CRM114-Status: GOOD ( 14.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org As it is already defined twice and is about to be needed for kCFI error detection, move esr_comment() to a header for re-use, with a clearer name. Signed-off-by: Pierre-Clément Tosi Acked-by: Will Deacon --- arch/arm64/include/asm/esr.h | 5 +++++ arch/arm64/kernel/debug-monitors.c | 4 +--- arch/arm64/kernel/traps.c | 8 +++----- arch/arm64/kvm/handle_exit.c | 2 +- 4 files changed, 10 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index 81606bf7d5ac..2bcf216be376 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -379,6 +379,11 @@ #ifndef __ASSEMBLY__ #include +static inline unsigned long esr_brk_comment(unsigned long esr) +{ + return esr & ESR_ELx_BRK64_ISS_COMMENT_MASK; +} + static inline bool esr_is_data_abort(unsigned long esr) { const unsigned long ec = ESR_ELx_EC(esr); diff --git a/arch/arm64/kernel/debug-monitors.c b/arch/arm64/kernel/debug-monitors.c index 64f2ecbdfe5c..024a7b245056 100644 --- a/arch/arm64/kernel/debug-monitors.c +++ b/arch/arm64/kernel/debug-monitors.c @@ -312,9 +312,7 @@ static int call_break_hook(struct pt_regs *regs, unsigned long esr) * entirely not preemptible, and we can use rcu list safely here. */ list_for_each_entry_rcu(hook, list, node) { - unsigned long comment = esr & ESR_ELx_BRK64_ISS_COMMENT_MASK; - - if ((comment & ~hook->mask) == hook->imm) + if ((esr_brk_comment(esr) & ~hook->mask) == hook->imm) fn = hook->fn; } diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 215e6d7f2df8..2652247032ae 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -1105,8 +1105,6 @@ static struct break_hook ubsan_break_hook = { }; #endif -#define esr_comment(esr) ((esr) & ESR_ELx_BRK64_ISS_COMMENT_MASK) - /* * Initial handler for AArch64 BRK exceptions * This handler only used until debug_traps_init(). @@ -1115,15 +1113,15 @@ int __init early_brk64(unsigned long addr, unsigned long esr, struct pt_regs *regs) { #ifdef CONFIG_CFI_CLANG - if ((esr_comment(esr) & ~CFI_BRK_IMM_MASK) == CFI_BRK_IMM_BASE) + if ((esr_brk_comment(esr) & ~CFI_BRK_IMM_MASK) == CFI_BRK_IMM_BASE) return cfi_handler(regs, esr) != DBG_HOOK_HANDLED; #endif #ifdef CONFIG_KASAN_SW_TAGS - if ((esr_comment(esr) & ~KASAN_BRK_MASK) == KASAN_BRK_IMM) + if ((esr_brk_comment(esr) & ~KASAN_BRK_MASK) == KASAN_BRK_IMM) return kasan_handler(regs, esr) != DBG_HOOK_HANDLED; #endif #ifdef CONFIG_UBSAN_TRAP - if ((esr_comment(esr) & ~UBSAN_BRK_MASK) == UBSAN_BRK_IMM) + if ((esr_brk_comment(esr) & ~UBSAN_BRK_MASK) == UBSAN_BRK_IMM) return ubsan_handler(regs, esr) != DBG_HOOK_HANDLED; #endif return bug_handler(regs, esr) != DBG_HOOK_HANDLED; diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 617ae6dea5d5..0bcafb3179d6 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -395,7 +395,7 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, if (mode != PSR_MODE_EL2t && mode != PSR_MODE_EL2h) { kvm_err("Invalid host exception to nVHE hyp!\n"); } else if (ESR_ELx_EC(esr) == ESR_ELx_EC_BRK64 && - (esr & ESR_ELx_BRK64_ISS_COMMENT_MASK) == BUG_BRK_IMM) { + esr_brk_comment(esr) == BUG_BRK_IMM) { const char *file = NULL; unsigned int line = 0; From patchwork Fri May 10 11:26:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13661440 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4D261C25B5F for ; Fri, 10 May 2024 11:28:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=09zZGZCUyl8fRA5ddgoj6b/zr+gCJrwJotIo569+B4I=; b=i7mKdm9k/SJgQpkarXGf1PDPMf VaL55wQEJF+FIE/SPHLbXcIHML5siedPugTXMbNkc5l49pLyrqIo2QS/KeoITqFAzL+yCDvR5xC4W d9f/GMD3D48eosCuX8KtE/KzQFGU0uh/STllxUvoWErkEM+AB/HRBC7nrCEpZwn71Zo3+gz9IuKNb k6Fqhaku7p10BdYPi5vfp15pD1+OBDo0KbKKQsA9Yd2iEoWF0VisO/kT4KmzhNM8AbNdc5NAbBQe0 C4lqHN2EDk6EE4ALODlFpmNQZLA6tWAgirUm4aNH7K5mHOXbNeSfN4SkuWkIsjE36vPpM113Qi6hY RAXzrKgQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OQS-000000053h5-31bI; Fri, 10 May 2024 11:28:36 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPg-0000000538F-1AFP for linux-arm-kernel@lists.infradead.org; Fri, 10 May 2024 11:27:47 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-de60321ce6cso3404782276.1 for ; Fri, 10 May 2024 04:27:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715340459; x=1715945259; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=qeYMQHI62TLtzWSMqgKJxIUsn2T5TYiEl6TIFioPKSw=; b=Iu1EecPuUA8vm4K9b/+9d2Kg26suM6ARqRPBm25HY0lorcPxCvWU2DD/Wam3MThvPw mOast3dO6MDsVAGDe5NNepdApRJA+/sbEuneePyko4SVD6xRBVMb7+DCMwe/n1gbX8fC gZ0qfIm0GuKBZvUm344T+ZnYV7D0cAe68GHdCn9zpoo/07W6WHlUG7kObojsOYryQkea lwCEGGANDwKdC5d+Yx8LP16YAnrjUp8cuWgAn+WWP4H54HZob6KpC2BYt+w9sxwt3JdG nGttZbRFTJT8mZ9eN9697WW+4Og1McCN6iqA69j+IvlJrTN0yF3UbLgvh5V5WSLnfl2Z nSoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715340459; x=1715945259; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=qeYMQHI62TLtzWSMqgKJxIUsn2T5TYiEl6TIFioPKSw=; b=szKytB8Sp6vLJABlLW+VkGRFpuGplWgdCLiWYHFYLGM9GqNza+jwBzxXtgncKHjDH+ EJ6JKnQCCyxTLOICAFjnYX0kGzngp6W61x1G09KMamJl6JhMFth8xIJ78JLNgOLtFx/+ 19l46Hz/UmEzGPqnMjhJAPreMQE/8l7NKqWTV2nYCUCWdA3wm0GY+z7yB8j8dE1DWlli J73JVWVBnpshhiDB9ANJ7YzSeQ5ZYYYWL23cjR/YEDD2M6dU88ljbDZGPpiARbcX7L9R 92Weasl6yXBMJxNLq+I0MGRY3jogTPpm0mk9BC8WVXiQm6mj65eJSTrq2PNC1lQFSZKi Qwkg== X-Forwarded-Encrypted: i=1; AJvYcCU1W9ORm5/Luv1fggHOm3mtuPRv0dB3JBfDBF4mtt/+2dI6mlLaFKJznRxIOakIrJ1PWWNwdyJi2HhzBxynF4dRkSSlEOpNl7Kz3Lw601nfrzZbHmY= X-Gm-Message-State: AOJu0YyyZoKB6MooHWg+6yphHv/VXndlBVrA5SYVjFwnAn4YUsrV9V8W 1dc/qBUq98VYid0XjCAToxeZI5N2xjpj3c2rFB1hH0gERIUujSzyhRYHsnpiRMN8BrBSSTz8FQ= = X-Google-Smtp-Source: AGHT+IG2Nk6GDvDApxzf2fQrO3hpSPR8XKLyC1KYy5HQT81VyW90ugxbfQimtrj4DVkv2r3ubwFrbsoU3Q== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:6902:1082:b0:dcb:b9d7:2760 with SMTP id 3f1490d57ef6-dee4f38cb25mr612041276.13.1715340459676; Fri, 10 May 2024 04:27:39 -0700 (PDT) Date: Fri, 10 May 2024 12:26:38 +0100 In-Reply-To: <20240510112645.3625702-1-ptosi@google.com> Mime-Version: 1.0 References: <20240510112645.3625702-1-ptosi@google.com> X-Mailer: git-send-email 2.45.0.118.g7fe29c98d7-goog Message-ID: <20240510112645.3625702-10-ptosi@google.com> Subject: [PATCH v3 09/12] KVM: arm64: VHE: Add test module for hyp kCFI From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240510_042744_393009_846CE86C X-CRM114-Status: GOOD ( 32.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In order to easily periodically (and potentially automatically) validate that the hypervisor kCFI feature doesn't bitrot, introduce a way to trigger hypervisor kCFI faults from userspace on test builds of KVM. Add hooks in the hypervisor code to call registered callbacks (intended to trigger kCFI faults either for the callback call itself of from within the callback function) when running with guest or host VBAR_EL2. As the calls are issued from the KVM_RUN ioctl handling path, userspace gains control over when the actual triggering of the fault happens without needing to modify the KVM uAPI. Export kernel functions to register these callbacks from modules and introduce a kernel module intended to contain any testing logic. By limiting the changes to the core kernel to a strict minimum, this architectural split allows tests to be updated (within the module) without the need to redeploy (or recompile) the kernel (hyp) under test. Use the module parameters as the uAPI for configuring the fault condition being tested (i.e. either at insertion or post-insertion using /sys/module/.../parameters), which naturally makes it impossible for userspace to test kCFI without the module (and, inversely, makes the module only - not KVM - responsible for exposing said uAPI). As kCFI is implemented with a caller-side check of a callee-side value, make the module support 4 tests based on the location of the caller and callee (built-in or in-module), for each of the 2 hypervisor contexts (host & guest), selected by userspace using the 'guest' or 'host' module parameter. For this purpose, export symbols which the module can use to configure the callbacks for in-kernel and module-to-built-in kCFI faulting calls. Define the module-to-kernel API to allow the module to detect that it was loaded on a kernel built with support for it but which is running without a hypervisor (-ENXIO) or with one that doesn't use the VHE CPU feature (-EOPNOTSUPP), which is currently the only mode for which KVM supports hypervisor kCFI. Allow kernel build configs to set CONFIG_HYP_CFI_TEST to only support the in-kernel hooks (=y) or also build the test module (=m). Use intermediate internal Kconfig flags (CONFIG_HYP_SUPPORTS_CFI_TEST and CONFIG_HYP_CFI_TEST_MODULE) to simplify the Makefiles and #ifdefs. As the symbols for callback registration are only exported to modules when CONFIG_HYP_CFI_TEST != n, it is impossible for the test module to be non-forcefully inserted on a kernel that doesn't support it. Note that this feature must NOT result in any noticeable change (behavioral or binary size) when HYP_CFI_TEST_MODULE = n. CONFIG_HYP_CFI_TEST is intentionally independent of CONFIG_CFI_CLANG, to avoid arbitrarily limiting the number of flag combinations that can be tested with the module. Also note that, as VHE aliases VBAR_EL1 to VBAR_EL2 for the host, testing hypervisor kCFI in VHE and in host context is equivalent to testing kCFI support of the kernel itself i.e. EL1 in non-VHE and/or in non-virtualized environments. For this reason, CONFIG_CFI_PERMISSIVE **will** prevent the test module from triggering a hyp panic (although a warning still gets printed) in that context. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/kvm_cfi.h | 36 ++++++++ arch/arm64/kvm/Kconfig | 22 +++++ arch/arm64/kvm/Makefile | 3 + arch/arm64/kvm/hyp/include/hyp/cfi.h | 47 ++++++++++ arch/arm64/kvm/hyp/vhe/Makefile | 1 + arch/arm64/kvm/hyp/vhe/cfi.c | 37 ++++++++ arch/arm64/kvm/hyp/vhe/switch.c | 7 ++ arch/arm64/kvm/hyp_cfi_test.c | 43 +++++++++ arch/arm64/kvm/hyp_cfi_test_module.c | 133 +++++++++++++++++++++++++++ 9 files changed, 329 insertions(+) create mode 100644 arch/arm64/include/asm/kvm_cfi.h create mode 100644 arch/arm64/kvm/hyp/include/hyp/cfi.h create mode 100644 arch/arm64/kvm/hyp/vhe/cfi.c create mode 100644 arch/arm64/kvm/hyp_cfi_test.c create mode 100644 arch/arm64/kvm/hyp_cfi_test_module.c diff --git a/arch/arm64/include/asm/kvm_cfi.h b/arch/arm64/include/asm/kvm_cfi.h new file mode 100644 index 000000000000..13cc7b19d838 --- /dev/null +++ b/arch/arm64/include/asm/kvm_cfi.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ + +#ifndef __ARM64_KVM_CFI_H__ +#define __ARM64_KVM_CFI_H__ + +#include +#include + +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + +int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)); +int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)); + +#else + +static inline int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)) +{ + return -EOPNOTSUPP; +} + +static inline int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) +{ + return -EOPNOTSUPP; +} + +#endif /* CONFIG_HYP_SUPPORTS_CFI_TEST */ + +/* Symbols which the host can register as hyp callbacks; see . */ +void hyp_trigger_builtin_cfi_fault(void); +void hyp_builtin_cfi_fault_target(int unused); + +#endif /* __ARM64_KVM_CFI_H__ */ diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 58f09370d17e..5daa8079a120 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -65,4 +65,26 @@ config PROTECTED_NVHE_STACKTRACE If unsure, or not using protected nVHE (pKVM), say N. +config HYP_CFI_TEST + tristate "KVM hypervisor kCFI test support" + depends on KVM + help + Say Y or M here to build KVM with test hooks to support intentionally + triggering hypervisor kCFI faults in guest or host context. + + Say M here to also build a module which registers callbacks triggering + faults and selected by userspace through its parameters. + + Note that this feature is currently only supported in VHE mode. + + If unsure, say N. + +config HYP_SUPPORTS_CFI_TEST + def_bool y + depends on HYP_CFI_TEST + +config HYP_CFI_TEST_MODULE + def_tristate m if HYP_CFI_TEST = m + depends on HYP_CFI_TEST + endif # VIRTUALIZATION diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index c0c050e53157..d42540ae3ea7 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -22,6 +22,7 @@ kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \ vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \ vgic/vgic-its.o vgic/vgic-debug.o +kvm-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += hyp_cfi_test.o kvm-$(CONFIG_HW_PERF_EVENTS) += pmu-emul.o pmu.o always-y := hyp_constants.h hyp-constants.s @@ -39,3 +40,5 @@ $(obj)/hyp_constants.h: $(obj)/hyp-constants.s FORCE obj-kvm := $(addprefix $(obj)/, $(kvm-y)) $(obj-kvm): $(obj)/hyp_constants.h + +obj-$(CONFIG_HYP_CFI_TEST_MODULE) += hyp_cfi_test_module.o diff --git a/arch/arm64/kvm/hyp/include/hyp/cfi.h b/arch/arm64/kvm/hyp/include/hyp/cfi.h new file mode 100644 index 000000000000..c6536040bc06 --- /dev/null +++ b/arch/arm64/kvm/hyp/include/hyp/cfi.h @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ + +#ifndef __ARM64_KVM_HYP_CFI_H__ +#define __ARM64_KVM_HYP_CFI_H__ + +#include +#include + +#include + +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + +int __kvm_register_cfi_test_cb(void (*cb)(void), bool in_host_ctxt); + +extern void (*hyp_test_host_ctxt_cfi)(void); +extern void (*hyp_test_guest_ctxt_cfi)(void); + +/* Hypervisor callbacks for the host to register. */ +void hyp_trigger_builtin_cfi_fault(void); +void hyp_builtin_cfi_fault_target(int unused); + +#else + +static inline +int __kvm_register_cfi_test_cb(void (*cb)(void), bool in_host_ctxt) +{ + return -EOPNOTSUPP; +} + +#define hyp_test_host_ctxt_cfi ((void(*)(void))(NULL)) +#define hyp_test_guest_ctxt_cfi ((void(*)(void))(NULL)) + +static inline void hyp_trigger_builtin_cfi_fault(void) +{ +} + +static inline void hyp_builtin_cfi_fault_target(int __always_unused unused) +{ +} + +#endif /* CONFIG_HYP_SUPPORTS_CFI_TEST */ + +#endif /* __ARM64_KVM_HYP_CFI_H__ */ diff --git a/arch/arm64/kvm/hyp/vhe/Makefile b/arch/arm64/kvm/hyp/vhe/Makefile index 3b9e5464b5b3..19ca584cc21e 100644 --- a/arch/arm64/kvm/hyp/vhe/Makefile +++ b/arch/arm64/kvm/hyp/vhe/Makefile @@ -9,3 +9,4 @@ ccflags-y := -D__KVM_VHE_HYPERVISOR__ obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o +obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += cfi.o diff --git a/arch/arm64/kvm/hyp/vhe/cfi.c b/arch/arm64/kvm/hyp/vhe/cfi.c new file mode 100644 index 000000000000..5849f239e27f --- /dev/null +++ b/arch/arm64/kvm/hyp/vhe/cfi.c @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ +#include + +#include + +void (*hyp_test_host_ctxt_cfi)(void); +void (*hyp_test_guest_ctxt_cfi)(void); + +int __kvm_register_cfi_test_cb(void (*cb)(void), bool in_host_ctxt) +{ + if (in_host_ctxt) + hyp_test_host_ctxt_cfi = cb; + else + hyp_test_guest_ctxt_cfi = cb; + + return 0; +} + +void hyp_builtin_cfi_fault_target(int __always_unused unused) +{ +} + +void hyp_trigger_builtin_cfi_fault(void) +{ + /* Intentional UB cast & dereference, to trigger a kCFI fault. */ + void (*target)(void) = (void *)&hyp_builtin_cfi_fault_target; + + /* + * READ_ONCE() prevents this indirect call from being optimized out, + * forcing the compiler to generate the kCFI check before the branch. + */ + READ_ONCE(target)(); +} diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 9db04a286398..b3268933b093 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -4,6 +4,7 @@ * Author: Marc Zyngier */ +#include #include #include @@ -221,6 +222,9 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) struct kvm_cpu_context *guest_ctxt; u64 exit_code; + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_host_ctxt_cfi)) + hyp_test_host_ctxt_cfi(); + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; host_ctxt->__hyp_running_vcpu = vcpu; guest_ctxt = &vcpu->arch.ctxt; @@ -245,6 +249,9 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) else vcpu_clear_flag(vcpu, VCPU_HYP_CONTEXT); + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_guest_ctxt_cfi)) + hyp_test_guest_ctxt_cfi(); + do { /* Jump in the fire! */ exit_code = __guest_enter(vcpu); diff --git a/arch/arm64/kvm/hyp_cfi_test.c b/arch/arm64/kvm/hyp_cfi_test.c new file mode 100644 index 000000000000..da7b25ca1b1f --- /dev/null +++ b/arch/arm64/kvm/hyp_cfi_test.c @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ +#include +#include +#include +#include + +#include +#include +#include + +/* For calling directly into the VHE hypervisor; see . */ +int __kvm_register_cfi_test_cb(void (*)(void), bool); + +static int kvm_register_cfi_test_cb(void (*vhe_cb)(void), bool in_host_ctxt) +{ + if (!is_hyp_mode_available()) + return -ENXIO; + + if (is_hyp_nvhe()) + return -EOPNOTSUPP; + + return __kvm_register_cfi_test_cb(vhe_cb, in_host_ctxt); +} + +int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)) +{ + return kvm_register_cfi_test_cb(cb, true); +} +EXPORT_SYMBOL(kvm_cfi_test_register_host_ctxt_cb); + +int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) +{ + return kvm_register_cfi_test_cb(cb, false); +} +EXPORT_SYMBOL(kvm_cfi_test_register_guest_ctxt_cb); + +/* Hypervisor callbacks for the test module to register. */ +EXPORT_SYMBOL(hyp_trigger_builtin_cfi_fault); +EXPORT_SYMBOL(hyp_builtin_cfi_fault_target); diff --git a/arch/arm64/kvm/hyp_cfi_test_module.c b/arch/arm64/kvm/hyp_cfi_test_module.c new file mode 100644 index 000000000000..eeda4be4d3ef --- /dev/null +++ b/arch/arm64/kvm/hyp_cfi_test_module.c @@ -0,0 +1,133 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include + +#include +#include +#include +#include + +static int set_host_mode(const char *val, const struct kernel_param *kp); +static int set_guest_mode(const char *val, const struct kernel_param *kp); + +#define M_DESC \ + "\n\t0: none" \ + "\n\t1: built-in caller & built-in callee" \ + "\n\t2: built-in caller & module callee" \ + "\n\t3: module caller & built-in callee" \ + "\n\t4: module caller & module callee" + +static unsigned int host_mode; +module_param_call(host, set_host_mode, param_get_uint, &host_mode, 0644); +MODULE_PARM_DESC(host, + "Hypervisor kCFI fault test case in host context:" M_DESC); + +static unsigned int guest_mode; +module_param_call(guest, set_guest_mode, param_get_uint, &guest_mode, 0644); +MODULE_PARM_DESC(guest, + "Hypervisor kCFI fault test case in guest context:" M_DESC); + +static void trigger_module2module_cfi_fault(void); +static void trigger_module2builtin_cfi_fault(void); +static void hyp_cfi_module2module_test_target(int); +static void hyp_cfi_builtin2module_test_target(int); + +static int set_param_mode(const char *val, const struct kernel_param *kp, + int (*register_cb)(void (*)(void))) +{ + unsigned int *mode = kp->arg; + int err; + + err = param_set_uint(val, kp); + if (err) + return err; + + switch (*mode) { + case 0: + return register_cb(NULL); + case 1: + return register_cb(hyp_trigger_builtin_cfi_fault); + case 2: + return register_cb((void *)hyp_cfi_builtin2module_test_target); + case 3: + return register_cb(trigger_module2builtin_cfi_fault); + case 4: + return register_cb(trigger_module2module_cfi_fault); + default: + return -EINVAL; + } +} + +static int set_host_mode(const char *val, const struct kernel_param *kp) +{ + return set_param_mode(val, kp, kvm_cfi_test_register_host_ctxt_cb); +} + +static int set_guest_mode(const char *val, const struct kernel_param *kp) +{ + return set_param_mode(val, kp, kvm_cfi_test_register_guest_ctxt_cb); +} + +static void __exit exit_hyp_cfi_test(void) +{ + int err; + + err = kvm_cfi_test_register_host_ctxt_cb(NULL); + if (err) + pr_err("Failed to unregister host context trigger: %d\n", err); + + err = kvm_cfi_test_register_guest_ctxt_cb(NULL); + if (err) + pr_err("Failed to unregister guest context trigger: %d\n", err); +} +module_exit(exit_hyp_cfi_test); + +static void trigger_module2builtin_cfi_fault(void) +{ + /* Intentional UB cast & dereference, to trigger a kCFI fault. */ + void (*target)(void) = (void *)&hyp_builtin_cfi_fault_target; + + /* + * READ_ONCE() prevents this indirect call from being optimized out, + * forcing the compiler to generate the kCFI check before the branch. + */ + READ_ONCE(target)(); + + pr_err_ratelimited("%s: Survived a kCFI violation\n", __func__); +} + +static void trigger_module2module_cfi_fault(void) +{ + /* Intentional UB cast & dereference, to trigger a kCFI fault. */ + void (*target)(void) = (void *)&hyp_cfi_module2module_test_target; + + /* + * READ_ONCE() prevents this indirect call from being optimized out, + * forcing the compiler to generate the kCFI check before the branch. + */ + READ_ONCE(target)(); + + pr_err_ratelimited("%s: Survived a kCFI violation\n", __func__); +} + +/* Use different functions, for clearer symbols in kCFI panic reports. */ +static noinline +void hyp_cfi_module2module_test_target(int __always_unused unused) +{ +} + +static noinline +void hyp_cfi_builtin2module_test_target(int __always_unused unused) +{ +} + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Pierre-Clément Tosi "); +MODULE_DESCRIPTION("KVM hypervisor kCFI test module"); From patchwork Fri May 10 11:26:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13661439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 16C94C25B5F for ; Fri, 10 May 2024 11:28:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=G8omM//u3RAnoKNSlpmO7VOWDy/zxrY50+NiJpp3o+w=; b=a3aGRM32klEw46+uLK77mu1l2H YtxANjpPSg2ZzP0JuIjTXlM1wSzv3yplH0NZ+E2i8WTdyA9O3s9jTdlMCUdD+sqmyHjtI4ai1gLd5 IWC14lTbSmplxkFnxToGz/9Oz70AXzgSzk5xAKtlfG0xoJcNj/4uC1SKbEQwt8DMaRKxNLLzgXcXY BF3YNHa7CoM7XCTJWglBPZ4Y7EnAfuzg9r/qv+myWsIebr8pQOe68zoWWobw5S2mIldlEZ7Ih6Krq clm2PwVx5NsZwmWO4vyHoTny2Qvh96N6Xr+3l34neVOTDLFKYvQmk9pAzvqgsr/lYXZgtxqrSej21 a8nBXJQA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OQZ-000000053jp-3wfs; Fri, 10 May 2024 11:28:39 +0000 Received: from mail-ed1-x54a.google.com ([2a00:1450:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPj-0000000539X-2O8H for linux-arm-kernel@lists.infradead.org; Fri, 10 May 2024 11:27:49 +0000 Received: by mail-ed1-x54a.google.com with SMTP id 4fb4d7f45d1cf-572b3091d68so1216502a12.2 for ; Fri, 10 May 2024 04:27:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715340462; x=1715945262; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=kFhYs5lDqdu/+fs82ZoKuSCT3z4IC8Cn6RP491+YFBU=; b=fNp5AGQyiHbg8gdrg6925gvTolRCnqVMtIfs3Swqb+kTK39ieUU2UBTnFpImcmX++x XCCD9CXypSdF6A34od7fGHkKxG8a2xgHEsvvePkhSL+fNxsms0fV1AM81HVWRZd1W0T1 DeHgbPX+a39neG0vdgGhMkHczYDZt3G5KKM1AM20jvjkGpIJ67Kvtuy302w5BTdMwfOu 73154CUhsKSG+IKZ1C22CJFt044PUWEa6gUbk/xQisCLprt7PTindSXKEMFtZHqxYUlt qzo3XVeVAJxVCuuudEGNWMrpiRaIvqfD2Uj+j9lcDeUORSNSonXyTqw92hE5R8lJnO3U YdcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715340462; x=1715945262; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=kFhYs5lDqdu/+fs82ZoKuSCT3z4IC8Cn6RP491+YFBU=; b=WsSFYzZyN6DigXE1yDubiHCF/Vj4K8EQQnUELFEfZg7ZLysd6OsdD9eYQvFmHsbSeb CUcKEkHWbKPlxJhiDaKeKcJgleiUzrOVRqBZ5Ll5SnejdPT3KexlTa9pcXT7L2IpDNyh 8bc+M30081ko8dpwjO0EALH+AoLZyAT7D+roM1NGJ1+1qOTZ20TdK1RneyaXLnr5cjlx 7XaTyE4at6YnR9r2oZ+iImAUTxf8AWRtQ9WOvknrEoG/x5KFqmUXQPXZWldBOfJBmfC+ oXTUpeSkHgI3uUyuhf4XxWpBQUT7ctd2bfEBAIPnysSTPzhtvu+NK4V7LBdS19vGkqx3 nc6g== X-Forwarded-Encrypted: i=1; AJvYcCVugAuKo8ZLJipNYI4HqNFjZcUJraAtw1ScKibQTmevXl5wW1Y58gjDeIKQmXN+NHG9uz5CdzEZCedKY0dcdU/3O7Z/kKk3MuO/zhB70U86Gcg2TFc= X-Gm-Message-State: AOJu0YwbyihMIsBht0muqFAWjxFpmcLsRroEwLKAnRjsUvQijYPE2H7V 2oU2T/TMdflCGDfkqx2Zku/BvInVPDFx+l66bQHUyY8Jv+1VIlgZrmn4TwRtoKzAc4m1O8JEhg= = X-Google-Smtp-Source: AGHT+IGJhtD/wN9N41xGH4SqpBDhacAaIk1ZGwSPDqSUff1bOTvFtTBlhk3QdXGF6oIEd68ubcVUJM9mKg== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:aa7:d4d2:0:b0:572:3273:62d5 with SMTP id 4fb4d7f45d1cf-5734d6ecb9cmr2976a12.5.1715340461789; Fri, 10 May 2024 04:27:41 -0700 (PDT) Date: Fri, 10 May 2024 12:26:39 +0100 In-Reply-To: <20240510112645.3625702-1-ptosi@google.com> Mime-Version: 1.0 References: <20240510112645.3625702-1-ptosi@google.com> X-Mailer: git-send-email 2.45.0.118.g7fe29c98d7-goog Message-ID: <20240510112645.3625702-11-ptosi@google.com> Subject: [PATCH v3 10/12] KVM: arm64: nVHE: Support CONFIG_CFI_CLANG at EL2 From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240510_042747_911751_5D01B7F9 X-CRM114-Status: GOOD ( 17.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The compiler implements kCFI by adding type information (u32) above every function that might be indirectly called and, whenever a function pointer is called, injects a read-and-compare of that u32 against the value corresponding to the expected type. In case of a mismatch, a BRK instruction gets executed. When the hypervisor triggers such an exception in nVHE, it panics and triggers and exception return to EL1. Therefore, teach nvhe_hyp_panic_handler() to detect kCFI errors from the ESR and report them. If necessary, remind the user that EL2 kCFI is not affected by CONFIG_CFI_PERMISSIVE. Pass $(CC_FLAGS_CFI) to the compiler when building the nVHE hyp code. Use SYM_TYPED_FUNC_START() for __pkvm_init_switch_pgd, as nVHE can't call it directly and must use a PA function pointer from C (because it is part of the idmap page), which would trigger a kCFI failure if the type ID wasn't present. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/esr.h | 6 ++++++ arch/arm64/kvm/handle_exit.c | 11 +++++++++++ arch/arm64/kvm/hyp/nvhe/Makefile | 6 +++--- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 6 +++++- 4 files changed, 25 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index 2bcf216be376..9eb9e6aa70cf 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -391,6 +391,12 @@ static inline bool esr_is_data_abort(unsigned long esr) return ec == ESR_ELx_EC_DABT_LOW || ec == ESR_ELx_EC_DABT_CUR; } +static inline bool esr_is_cfi_brk(unsigned long esr) +{ + return ESR_ELx_EC(esr) == ESR_ELx_EC_BRK64 && + (esr_brk_comment(esr) & ~CFI_BRK_IMM_MASK) == CFI_BRK_IMM_BASE; +} + static inline bool esr_fsc_is_translation_fault(unsigned long esr) { /* Translation fault, level -1 */ diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 0bcafb3179d6..0db23a6304ce 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -383,6 +383,15 @@ void handle_exit_early(struct kvm_vcpu *vcpu, int exception_index) kvm_handle_guest_serror(vcpu, kvm_vcpu_get_esr(vcpu)); } +static void kvm_nvhe_report_cfi_failure(u64 panic_addr) +{ + kvm_err("nVHE hyp CFI failure at: [<%016llx>] %pB!\n", panic_addr, + (void *)(panic_addr + kaslr_offset())); + + if (IS_ENABLED(CONFIG_CFI_PERMISSIVE)) + kvm_err(" (CONFIG_CFI_PERMISSIVE ignored for hyp failures)\n"); +} + void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, u64 elr_virt, u64 elr_phys, u64 par, uintptr_t vcpu, @@ -413,6 +422,8 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, else kvm_err("nVHE hyp BUG at: [<%016llx>] %pB!\n", panic_addr, (void *)(panic_addr + kaslr_offset())); + } else if (IS_ENABLED(CONFIG_CFI_CLANG) && esr_is_cfi_brk(esr)) { + kvm_nvhe_report_cfi_failure(panic_addr); } else { kvm_err("nVHE hyp panic at: [<%016llx>] %pB!\n", panic_addr, (void *)(panic_addr + kaslr_offset())); diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 2250253a6429..2eb915d8943f 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -89,9 +89,9 @@ quiet_cmd_hyprel = HYPREL $@ quiet_cmd_hypcopy = HYPCOPY $@ cmd_hypcopy = $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ $< $@ -# Remove ftrace, Shadow Call Stack, and CFI CFLAGS. -# This is equivalent to the 'notrace', '__noscs', and '__nocfi' annotations. -KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI), $(KBUILD_CFLAGS)) +# Remove ftrace and Shadow Call Stack CFLAGS. +# This is equivalent to the 'notrace' and '__noscs' annotations. +KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS), $(KBUILD_CFLAGS)) # Starting from 13.0.0 llvm emits SHT_REL section '.llvm.call-graph-profile' # when profile optimization is applied. gen-hyprel does not support SHT_REL and # causes a build failure. Remove profile optimization flags. diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 5a15737b4233..33fb5732ab83 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -5,6 +5,7 @@ */ #include +#include #include #include @@ -268,8 +269,11 @@ SYM_CODE_END(__kvm_handle_stub_hvc) /* * void __pkvm_init_switch_pgd(struct kvm_nvhe_init_params *params, * void (*finalize_fn)(void)); + * + * SYM_TYPED_FUNC_START() allows C to call this ID-mapped function indirectly + * using a physical pointer without triggering a kCFI failure. */ -SYM_FUNC_START(__pkvm_init_switch_pgd) +SYM_TYPED_FUNC_START(__pkvm_init_switch_pgd) /* Load the inputs from the VA pointer before turning the MMU off */ ldr x5, [x0, #NVHE_INIT_PGD_PA] ldr x0, [x0, #NVHE_INIT_STACK_HYP_VA] From patchwork Fri May 10 11:26:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13661441 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D4EA0C25B10 for ; Fri, 10 May 2024 11:29:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=RwO5G1987qtlJSL1mXVSImsFb9R90l0PzxewsOdz1zA=; b=0P5ZR7htvUN2X0mdxkOpq9YwUs TOEjr0RCRYJKUyD37pCh6u9BlhvDbzH2sT2M2pgKlo0Vo5+ZFaF2gj1PkIS52xw9nhHzAyTEXswb6 vNJ2mxI834cTsmcpOBpXppL6P49zOLY6xl9mtMW+8vguaa1Cl8PVrsvEY7diXQrBNw1CC9eHFxuLh LpuvYelzOAjOb3/wf3fnCS4M2kEqIjtc2OxBeFXPzobsHRS4fhvdx4UbAX9al+tdHLfgpzLtny90n yDR7km0tYzvXzj0rT5HGaVyd2xetJ4GM8KijtAu1nPuWnaDBJTA66OGVjK6LTfj7YqXD4aFlqaBmP KXCQGSHA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OQc-000000053l8-1ilp; Fri, 10 May 2024 11:28:42 +0000 Received: from mail-ej1-x649.google.com ([2a00:1450:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPk-000000053BG-36UO for linux-arm-kernel@lists.infradead.org; Fri, 10 May 2024 11:27:52 +0000 Received: by mail-ej1-x649.google.com with SMTP id a640c23a62f3a-a59efa9f108so95572266b.2 for ; Fri, 10 May 2024 04:27:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715340464; x=1715945264; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=n1ymwJj0J6YIOfbRVTRhLBZ520J+M8M3fYhLYcu4SBQ=; b=cIqbxxzgN5KTiszRHi5PBBA4WK6Tqz8wWV4rXRZAf6/VU+W1A0iHZLOsMf/F7UJh5T UF5JY4eVTMFfjqZ+MAriI6Du3VP5ykFEOgkVPyEwXmj3VFGT3RsesR/7ywMpLXk2arh2 ghzzefj+hbueQi9cxFLqT7NNDJ3Z26G02R7UfDKnOQs5xqPq14xyeXPz7Cmt1ULG5lKa SHlEOPI8M2dr/9egSG4oBXipDbvyE083zv7IM1zkQ/J0nvEI1XLETX4xT1ELs8DREZhy 26yyh7fqOeTmb5g4xS5ZFhLD58sCBvt/nDTphJk4aNSud0uzKDEbSA3/WTfJl59LRz4s nXjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715340464; x=1715945264; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=n1ymwJj0J6YIOfbRVTRhLBZ520J+M8M3fYhLYcu4SBQ=; b=bO9xcBgUkD4dl22hMqmeg9GZgYlxztJ08NmB21FREmFBODuP1Tb2ORWzOhDYATNUG6 y0GB0bdXm43YBQBgN+LgqI8BYLSrNI9GfzwVz1auYDLQd1hGSgV0nkjkO2fqkwosaejk uZ1mEym2lKGVuz/Sxbzpi+ROzbw4JnOUMN9xTc55yk4UjSTbK3UI89FdOFl0Q+chIZ4a rTxokQagU5w9Z7AbacJWXDzt1n9g8u4FNgHLfPiN0Mp22/Rbtyyvf69NE/VO0+flYqSc BKw7E+mHlrLKQaw8e1kzVtDOJf+jIOXcdwMJ1+5JpCTc52gkwYJoOuNIdNxBdWCC46C2 d9Sw== X-Forwarded-Encrypted: i=1; AJvYcCWP159ptHuBn203hw7m1SbxLXbzmPRJ31FMjc+IYPRE4yhP3DuHP/yMyY4KR4FIdPICAXkAS66mBfVb1ZFeJ+8oPMfNAEGBIdhkNjpt6kcCkkx1JUY= X-Gm-Message-State: AOJu0Ywtx/tBanmWdgCUvqLGEzv+buH2i6vSTUpUXQEYwcs4yRxipR/i cqmaAXIWBu+bpjS7b61xJ7ihygRoqWcy5UgOhO7XkPp0jkw64v0NMSaCVr0/zU/6akHp7yn6YA= = X-Google-Smtp-Source: AGHT+IH0RrjjfZ++WGHYEq92ODwn6HK5Otkgd5vcXO+mNguBfs2v+NnqgUGPVhO4xg5oozparmcLaLbhaQ== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a17:907:940f:b0:a59:cafe:85b6 with SMTP id a640c23a62f3a-a5a2d53fafbmr224066b.2.1715340464190; Fri, 10 May 2024 04:27:44 -0700 (PDT) Date: Fri, 10 May 2024 12:26:40 +0100 In-Reply-To: <20240510112645.3625702-1-ptosi@google.com> Mime-Version: 1.0 References: <20240510112645.3625702-1-ptosi@google.com> X-Mailer: git-send-email 2.45.0.118.g7fe29c98d7-goog Message-ID: <20240510112645.3625702-12-ptosi@google.com> Subject: [PATCH v3 11/12] KVM: arm64: nVHE: Support test module for hyp kCFI From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240510_042749_052991_808C8CAD X-CRM114-Status: GOOD ( 26.65 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Extend support for the kCFI test module to nVHE by replicating the hooks on the KVM_RUN handler path currently existing in VHE in the nVHE code, exporting the equivalent callback targets for triggering built-in hyp kCFI faults, and exposing a new CONFIG_HYP_CFI_TEST-only host HVC to implement callback registration. Update the test module to register the nVHE equivalent callback for test case '1' (i.e. both EL2 hyp caller and callee are built-in) and document that other cases are not supported outside of VHE, as they require EL2 symbols in the module, which is not currently supported for nVHE. Note that a kernel in protected mode that doesn't support HYP_CFI_TEST will prevent the module from registering nVHE callbacks both by not exporting the necessary symbols (similar to VHE) but also by rejecting the corresponding HVC, if the module tries to issue it directly. Also note that the test module will run in pKVM (with HYP_CFI_TEST) independently of other debug Kconfig flags but that not stacktrace will be printed without PROTECTED_NVHE_STACKTRACE. This allows testing kCFI under conditions closer to release builds, if desired. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/kvm_asm.h | 3 ++ arch/arm64/include/asm/kvm_cfi.h | 6 ++-- arch/arm64/kvm/Kconfig | 2 -- arch/arm64/kvm/hyp/{vhe => }/cfi.c | 0 arch/arm64/kvm/hyp/nvhe/Makefile | 1 + arch/arm64/kvm/hyp/nvhe/hyp-main.c | 19 ++++++++++++ arch/arm64/kvm/hyp/nvhe/switch.c | 7 +++++ arch/arm64/kvm/hyp/vhe/Makefile | 2 +- arch/arm64/kvm/hyp_cfi_test.c | 44 ++++++++++++++++++++++++---- arch/arm64/kvm/hyp_cfi_test_module.c | 24 ++++++++------- 10 files changed, 86 insertions(+), 22 deletions(-) rename arch/arm64/kvm/hyp/{vhe => }/cfi.c (100%) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 24b5e6b23417..3256c91ff234 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -81,6 +81,9 @@ enum __kvm_host_smccc_func { __KVM_HOST_SMCCC_FUNC___pkvm_init_vm, __KVM_HOST_SMCCC_FUNC___pkvm_init_vcpu, __KVM_HOST_SMCCC_FUNC___pkvm_teardown_vm, +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + __KVM_HOST_SMCCC_FUNC___kvm_register_cfi_test_cb, +#endif }; #define DECLARE_KVM_VHE_SYM(sym) extern char sym[] diff --git a/arch/arm64/include/asm/kvm_cfi.h b/arch/arm64/include/asm/kvm_cfi.h index 13cc7b19d838..ed6422eebce5 100644 --- a/arch/arm64/include/asm/kvm_cfi.h +++ b/arch/arm64/include/asm/kvm_cfi.h @@ -12,8 +12,8 @@ #ifdef CONFIG_HYP_SUPPORTS_CFI_TEST -int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)); -int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)); +int kvm_cfi_test_register_host_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb); +int kvm_cfi_test_register_guest_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb); #else @@ -31,6 +31,8 @@ static inline int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) /* Symbols which the host can register as hyp callbacks; see . */ void hyp_trigger_builtin_cfi_fault(void); +DECLARE_KVM_NVHE_SYM(hyp_trigger_builtin_cfi_fault); void hyp_builtin_cfi_fault_target(int unused); +DECLARE_KVM_NVHE_SYM(hyp_builtin_cfi_fault_target); #endif /* __ARM64_KVM_CFI_H__ */ diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 5daa8079a120..715c85088c06 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -75,8 +75,6 @@ config HYP_CFI_TEST Say M here to also build a module which registers callbacks triggering faults and selected by userspace through its parameters. - Note that this feature is currently only supported in VHE mode. - If unsure, say N. config HYP_SUPPORTS_CFI_TEST diff --git a/arch/arm64/kvm/hyp/vhe/cfi.c b/arch/arm64/kvm/hyp/cfi.c similarity index 100% rename from arch/arm64/kvm/hyp/vhe/cfi.c rename to arch/arm64/kvm/hyp/cfi.c diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 2eb915d8943f..09039d351726 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -25,6 +25,7 @@ hyp-obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o cache.o setup.o mm.o mem_protect.o sys_regs.o pkvm.o stacktrace.o ffa.o hyp-obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o ../pgtable.o +hyp-obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += ../cfi.o hyp-obj-$(CONFIG_LIST_HARDENED) += list_debug.o hyp-obj-y += $(lib-objs) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 2385fd03ed87..431860e8a98d 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -5,6 +5,7 @@ */ #include +#include #include #include @@ -13,6 +14,8 @@ #include #include +#include + #include #include #include @@ -314,6 +317,19 @@ static void handle___pkvm_teardown_vm(struct kvm_cpu_context *host_ctxt) cpu_reg(host_ctxt, 1) = __pkvm_teardown_vm(handle); } +#ifndef CONFIG_HYP_SUPPORTS_CFI_TEST +__always_unused +#endif +static void handle___kvm_register_cfi_test_cb(struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(phys_addr_t, cb_phys, host_ctxt, 1); + DECLARE_REG(bool, in_host_ctxt, host_ctxt, 2); + + void (*cb)(void) = cb_phys ? __hyp_va(cb_phys) : NULL; + + cpu_reg(host_ctxt, 1) = __kvm_register_cfi_test_cb(cb, in_host_ctxt); +} + typedef void (*hcall_t)(struct kvm_cpu_context *); #define HANDLE_FUNC(x) [__KVM_HOST_SMCCC_FUNC_##x] = (hcall_t)handle_##x @@ -348,6 +364,9 @@ static const hcall_t host_hcall[] = { HANDLE_FUNC(__pkvm_init_vm), HANDLE_FUNC(__pkvm_init_vcpu), HANDLE_FUNC(__pkvm_teardown_vm), +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + HANDLE_FUNC(__kvm_register_cfi_test_cb), +#endif }; static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index c50f8459e4fc..160311bf367b 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -4,6 +4,7 @@ * Author: Marc Zyngier */ +#include #include #include @@ -253,6 +254,9 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) bool pmu_switch_needed; u64 exit_code; + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_host_ctxt_cfi)) + hyp_test_host_ctxt_cfi(); + /* * Having IRQs masked via PMR when entering the guest means the GIC * will not signal the CPU of interrupts of lower priority, and the @@ -313,6 +317,9 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) __debug_switch_to_guest(vcpu); + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_guest_ctxt_cfi)) + hyp_test_guest_ctxt_cfi(); + do { /* Jump in the fire! */ exit_code = __guest_enter(vcpu); diff --git a/arch/arm64/kvm/hyp/vhe/Makefile b/arch/arm64/kvm/hyp/vhe/Makefile index 19ca584cc21e..951c8c00a685 100644 --- a/arch/arm64/kvm/hyp/vhe/Makefile +++ b/arch/arm64/kvm/hyp/vhe/Makefile @@ -9,4 +9,4 @@ ccflags-y := -D__KVM_VHE_HYPERVISOR__ obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o -obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += cfi.o +obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += ../cfi.o diff --git a/arch/arm64/kvm/hyp_cfi_test.c b/arch/arm64/kvm/hyp_cfi_test.c index da7b25ca1b1f..6a02b43c45f6 100644 --- a/arch/arm64/kvm/hyp_cfi_test.c +++ b/arch/arm64/kvm/hyp_cfi_test.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include @@ -15,29 +16,60 @@ /* For calling directly into the VHE hypervisor; see . */ int __kvm_register_cfi_test_cb(void (*)(void), bool); -static int kvm_register_cfi_test_cb(void (*vhe_cb)(void), bool in_host_ctxt) +static int kvm_register_nvhe_cfi_test_cb(void *cb, bool in_host_ctxt) +{ + extern void *kvm_nvhe_sym(hyp_test_host_ctxt_cfi); + extern void *kvm_nvhe_sym(hyp_test_guest_ctxt_cfi); + + if (is_protected_kvm_enabled()) { + phys_addr_t cb_phys = cb ? virt_to_phys(cb) : 0; + + /* Use HVC as only the hyp can modify its callback pointers. */ + return kvm_call_hyp_nvhe(__kvm_register_cfi_test_cb, cb_phys, + in_host_ctxt); + } + + /* + * In non-protected nVHE, the pKVM HVC is not available but the + * hyp callback pointers can be accessed and modified directly. + */ + if (cb) + cb = kern_hyp_va(kvm_ksym_ref(cb)); + + if (in_host_ctxt) + kvm_nvhe_sym(hyp_test_host_ctxt_cfi) = cb; + else + kvm_nvhe_sym(hyp_test_guest_ctxt_cfi) = cb; + + return 0; +} + +static int kvm_register_cfi_test_cb(void (*vhe_cb)(void), void *nvhe_cb, + bool in_host_ctxt) { if (!is_hyp_mode_available()) return -ENXIO; if (is_hyp_nvhe()) - return -EOPNOTSUPP; + return kvm_register_nvhe_cfi_test_cb(nvhe_cb, in_host_ctxt); return __kvm_register_cfi_test_cb(vhe_cb, in_host_ctxt); } -int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)) +int kvm_cfi_test_register_host_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb) { - return kvm_register_cfi_test_cb(cb, true); + return kvm_register_cfi_test_cb(vhe_cb, nvhe_cb, true); } EXPORT_SYMBOL(kvm_cfi_test_register_host_ctxt_cb); -int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) +int kvm_cfi_test_register_guest_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb) { - return kvm_register_cfi_test_cb(cb, false); + return kvm_register_cfi_test_cb(vhe_cb, nvhe_cb, false); } EXPORT_SYMBOL(kvm_cfi_test_register_guest_ctxt_cb); /* Hypervisor callbacks for the test module to register. */ EXPORT_SYMBOL(hyp_trigger_builtin_cfi_fault); +EXPORT_SYMBOL(kvm_nvhe_sym(hyp_trigger_builtin_cfi_fault)); EXPORT_SYMBOL(hyp_builtin_cfi_fault_target); +EXPORT_SYMBOL(kvm_nvhe_sym(hyp_builtin_cfi_fault_target)); diff --git a/arch/arm64/kvm/hyp_cfi_test_module.c b/arch/arm64/kvm/hyp_cfi_test_module.c index eeda4be4d3ef..63a5e99cb164 100644 --- a/arch/arm64/kvm/hyp_cfi_test_module.c +++ b/arch/arm64/kvm/hyp_cfi_test_module.c @@ -20,9 +20,9 @@ static int set_guest_mode(const char *val, const struct kernel_param *kp); #define M_DESC \ "\n\t0: none" \ "\n\t1: built-in caller & built-in callee" \ - "\n\t2: built-in caller & module callee" \ - "\n\t3: module caller & built-in callee" \ - "\n\t4: module caller & module callee" + "\n\t2: built-in caller & module callee (VHE only)" \ + "\n\t3: module caller & built-in callee (VHE only)" \ + "\n\t4: module caller & module callee (VHE only)" static unsigned int host_mode; module_param_call(host, set_host_mode, param_get_uint, &host_mode, 0644); @@ -40,7 +40,7 @@ static void hyp_cfi_module2module_test_target(int); static void hyp_cfi_builtin2module_test_target(int); static int set_param_mode(const char *val, const struct kernel_param *kp, - int (*register_cb)(void (*)(void))) + int (*register_cb)(void (*)(void), void *)) { unsigned int *mode = kp->arg; int err; @@ -51,15 +51,17 @@ static int set_param_mode(const char *val, const struct kernel_param *kp, switch (*mode) { case 0: - return register_cb(NULL); + return register_cb(NULL, NULL); case 1: - return register_cb(hyp_trigger_builtin_cfi_fault); + return register_cb(hyp_trigger_builtin_cfi_fault, + kvm_nvhe_sym(hyp_trigger_builtin_cfi_fault)); case 2: - return register_cb((void *)hyp_cfi_builtin2module_test_target); + return register_cb((void *)hyp_cfi_builtin2module_test_target, + NULL); case 3: - return register_cb(trigger_module2builtin_cfi_fault); + return register_cb(trigger_module2builtin_cfi_fault, NULL); case 4: - return register_cb(trigger_module2module_cfi_fault); + return register_cb(trigger_module2module_cfi_fault, NULL); default: return -EINVAL; } @@ -79,11 +81,11 @@ static void __exit exit_hyp_cfi_test(void) { int err; - err = kvm_cfi_test_register_host_ctxt_cb(NULL); + err = kvm_cfi_test_register_host_ctxt_cb(NULL, NULL); if (err) pr_err("Failed to unregister host context trigger: %d\n", err); - err = kvm_cfi_test_register_guest_ctxt_cb(NULL); + err = kvm_cfi_test_register_guest_ctxt_cb(NULL, NULL); if (err) pr_err("Failed to unregister guest context trigger: %d\n", err); } From patchwork Fri May 10 11:26:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13661561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 10847C25B10 for ; Fri, 10 May 2024 12:33:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=BC9+xBkmVgZ90bsi6iCvZ6H3g6GL7klXI9V6nAZLPCg=; b=NAEktXfJOtTMIZmpyboAw5dI4C ThvW/P0spqAe3SG8SoezOlx4vy4u/s2XUNt69HVFWsxkTcMMMj+t+oLN+eneKnUV1r9fW7+p25D8o JDp13/2PNTx3HwhnYCnQMYHLineUyjKNKt6NpWH/k+NVuhfTzSg+JtdbqzNEb5l+W8qZBwFjMsthk owL3mfc/tjp2xdzO+pB7Vz6pB6jLoSHkB9egQ98w5ZxkSsW42iRia2YCSs5UYNsqOKb5NBobMEfuA WXUmRzNCxeAvEaoBJC/1i8CgtP2T7hY4TN7YIYp7S4yn4+dYekn/qmNI8ifgdQ5Hk4rypsHsPtGcR oQQzP9CQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5PQj-00000005HH1-0LBh; Fri, 10 May 2024 12:32:53 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5OPn-000000053Cm-1Svp for linux-arm-kernel@lists.infradead.org; Fri, 10 May 2024 11:27:55 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6207c483342so31802277b3.1 for ; Fri, 10 May 2024 04:27:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715340466; x=1715945266; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=5CCzdSxscfzeRlLJ5wL+3zQfVKF6xgrgjNCyKCDms4U=; b=xq66NLlQ/YWIroRh9FC5MbyyhhO7Pyg5iS/tCKes2UI/aDyrCrV4U9ufNis9PZZe8t fTen+WGRNkvJXQo6c/MhVmiQAGKH72ovfYlDKPDioFgO8NJHDuykUMtd03YODmK4ZnPN XcKCETUkoaf+92OYu+vTxcr8NAOsK6jHxck5BKssjcL1GGLXE6ljl8PzCRFaA4IiCTWt pNT+Qybl9Ql3wcojvTeAm1cgiG7xgNOLpztQwNaKVyr5P9eXaVw0mjmZE1pagd0LZsBg vIW3zKUqq2wtPPChX4fCmT4mwc+B//42+jYM/F6t9P+yWKZ9d2CWcdwaKRdSJth1IUZD fqNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715340466; x=1715945266; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=5CCzdSxscfzeRlLJ5wL+3zQfVKF6xgrgjNCyKCDms4U=; b=RKijm7P+wgu02/OP1kX6fg0UNsbLrzqJRsXbB+7y8HdJrfLywK0mecpMmT7xn4MGt9 UQEiBXpISwmv485crmfL7Fcke9P4LVK4qkAXQTpmE6HbLSt8D66Ypx7DRsF/RmgOKN7p TSiaJQmN7VB6KtmRGsZP4Gptv1tC6O8nk2kA6byNjOC8oBQSz5ISmtgpEi1hFHjhQAkO tMrkABs+EIg7KlNKj6WB2+loVB9qDvF0So3fGE66+8mtCBZ/WmNtySveNGcAXLrByQm+ Gz01XIgceHhUcdl5MKc4sdPkUwOqgbGeycwgztlQkEwBTKczN78HixdibnEC8eXWheYC H+2Q== X-Forwarded-Encrypted: i=1; AJvYcCWDMglpkf+6L0Acu3/Id6cq1+/h/4CquCUKYWIi15dAktQqXj+RnUaHDIi5fT9QkJzDUt9c6fCeEkDoIcvp1t3zn6TYLUCQIwEdlVnNb1TA/tNCsRE= X-Gm-Message-State: AOJu0YyVWHVcBcNUY4Yt0AfyNqoAFf0FjYHD/SO3s5RQzlAYctcdzYzz 0zLFDNWaV9p4CAcErIRRPQdJiJ3iqGoJqyHEddEWwkv7AGZf3RN2dkSLBvEj5Q8vBKNdOGHrDg= = X-Google-Smtp-Source: AGHT+IHdS3ei2ZFY3/jZi7UOWajuQFEcULFtMW4+dDg7tAnI+MMeD/3xdux6nZQkCMsm0kZ06aqSsHCV0w== X-Received: from ptosi.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:11ec]) (user=ptosi job=sendgmr) by 2002:a05:690c:c9d:b0:618:8e4b:f49d with SMTP id 00721157ae682-622aff868c9mr6454537b3.4.1715340466673; Fri, 10 May 2024 04:27:46 -0700 (PDT) Date: Fri, 10 May 2024 12:26:41 +0100 In-Reply-To: <20240510112645.3625702-1-ptosi@google.com> Mime-Version: 1.0 References: <20240510112645.3625702-1-ptosi@google.com> X-Mailer: git-send-email 2.45.0.118.g7fe29c98d7-goog Message-ID: <20240510112645.3625702-13-ptosi@google.com> Subject: [PATCH v3 12/12] KVM: arm64: Improve CONFIG_CFI_CLANG error message From: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: " =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= " , Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240510_042751_978323_2141976B X-CRM114-Status: GOOD ( 21.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org For kCFI, the compiler encodes in the immediate of the BRK (which the CPU places in ESR_ELx) the indices of the two registers it used to hold (resp.) the function pointer and expected type. Therefore, the kCFI handler must be able to parse the contents of the register file at the point where the exception was triggered. To achieve this, introduce a new hypervisor panic path that first stores the CPU context in the per-CPU kvm_hyp_ctxt before calling (directly or indirectly) hyp_panic() and execute it from all EL2 synchronous exception handlers i.e. - call it directly in host_el2_sync_vect (__kvm_hyp_host_vector, EL2t&h) - call it directly in el2t_sync_invalid (__kvm_hyp_vector, EL2t) - set ELR_EL2 to it in el2_sync (__kvm_hyp_vector, EL2h), which ERETs Teach hyp_panic() to decode the kCFI ESR and extract the target and type from the saved CPU context. In VHE, use that information to panic() with a specialized error message. In nVHE, only report it if the host (EL1) has access to the saved CPU context i.e. iff CONFIG_NVHE_EL2_DEBUG=y, which aligns with the behavior of CONFIG_PROTECTED_NVHE_STACKTRACE. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/handle_exit.c | 30 +++++++++++++++++++++++-- arch/arm64/kvm/hyp/entry.S | 24 +++++++++++++++++++- arch/arm64/kvm/hyp/hyp-entry.S | 2 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/host.S | 2 +- arch/arm64/kvm/hyp/vhe/switch.c | 26 +++++++++++++++++++-- 6 files changed, 79 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 0db23a6304ce..d76e41a07df1 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -26,6 +26,8 @@ #define CREATE_TRACE_POINTS #include "trace_handle_exit.h" +DECLARE_KVM_NVHE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt); + typedef int (*exit_handle_fn)(struct kvm_vcpu *); static void kvm_handle_guest_serror(struct kvm_vcpu *vcpu, u64 esr) @@ -383,11 +385,35 @@ void handle_exit_early(struct kvm_vcpu *vcpu, int exception_index) kvm_handle_guest_serror(vcpu, kvm_vcpu_get_esr(vcpu)); } -static void kvm_nvhe_report_cfi_failure(u64 panic_addr) +static void kvm_nvhe_report_cfi_target(struct user_pt_regs *regs, u64 esr, + u64 hyp_offset) +{ + u64 va_mask = GENMASK_ULL(vabits_actual - 1, 0); + u8 type_idx = FIELD_GET(CFI_BRK_IMM_TYPE, esr); + u8 target_idx = FIELD_GET(CFI_BRK_IMM_TARGET, esr); + u32 expected_type = (u32)regs->regs[type_idx]; + u64 target_addr = (regs->regs[target_idx] & va_mask) + hyp_offset; + + kvm_err(" (target: [<%016llx>] %ps, expected type: 0x%08x)\n", + target_addr, (void *)(target_addr + kaslr_offset()), + expected_type); +} + +static void kvm_nvhe_report_cfi_failure(u64 panic_addr, u64 esr, u64 hyp_offset) { + struct user_pt_regs *regs = NULL; + kvm_err("nVHE hyp CFI failure at: [<%016llx>] %pB!\n", panic_addr, (void *)(panic_addr + kaslr_offset())); + if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG) || !is_protected_kvm_enabled()) + regs = &this_cpu_ptr_nvhe_sym(kvm_hyp_ctxt)->regs; + + if (regs) + kvm_nvhe_report_cfi_target(regs, esr, hyp_offset); + else + kvm_err(" (no target information: !CONFIG_NVHE_EL2_DEBUG)\n"); + if (IS_ENABLED(CONFIG_CFI_PERMISSIVE)) kvm_err(" (CONFIG_CFI_PERMISSIVE ignored for hyp failures)\n"); } @@ -423,7 +449,7 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, kvm_err("nVHE hyp BUG at: [<%016llx>] %pB!\n", panic_addr, (void *)(panic_addr + kaslr_offset())); } else if (IS_ENABLED(CONFIG_CFI_CLANG) && esr_is_cfi_brk(esr)) { - kvm_nvhe_report_cfi_failure(panic_addr); + kvm_nvhe_report_cfi_failure(panic_addr, esr, hyp_offset); } else { kvm_err("nVHE hyp panic at: [<%016llx>] %pB!\n", panic_addr, (void *)(panic_addr + kaslr_offset())); diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index 6a1ce9d21e5b..8838b453b9be 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,7 +83,7 @@ alternative_else_nop_endif eret sb -SYM_INNER_LABEL(__hyp_restore_elr_and_panic, SYM_L_GLOBAL) +SYM_INNER_LABEL(__hyp_restore_elr_save_context_and_panic, SYM_L_GLOBAL) // x0-x29,lr: hyp regs stp x0, x1, [sp, #-16]! @@ -92,6 +92,28 @@ SYM_INNER_LABEL(__hyp_restore_elr_and_panic, SYM_L_GLOBAL) msr elr_el2, x0 ldp x0, x1, [sp], #16 +SYM_INNER_LABEL(__hyp_save_context_and_panic, SYM_L_GLOBAL) + // x0-x29,lr: hyp regs + + stp x0, x1, [sp, #-16]! + + adr_this_cpu x0, kvm_hyp_ctxt, x1 + + stp x2, x3, [x0, #CPU_XREG_OFFSET(2)] + + ldp x2, x3, [sp], #16 + + stp x2, x3, [x0, #CPU_XREG_OFFSET(0)] + stp x4, x5, [x0, #CPU_XREG_OFFSET(4)] + stp x6, x7, [x0, #CPU_XREG_OFFSET(6)] + stp x8, x9, [x0, #CPU_XREG_OFFSET(8)] + stp x10, x11, [x0, #CPU_XREG_OFFSET(10)] + stp x12, x13, [x0, #CPU_XREG_OFFSET(12)] + stp x14, x15, [x0, #CPU_XREG_OFFSET(14)] + stp x16, x17, [x0, #CPU_XREG_OFFSET(16)] + + save_callee_saved_regs x0 + SYM_INNER_LABEL(__hyp_panic, SYM_L_GLOBAL) // x0-x29,lr: vcpu regs diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 7e65ef738ec9..d0d90d598338 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -130,7 +130,7 @@ SYM_CODE_END(\label) .endm /* None of these should ever happen */ - invalid_vector el2t_sync_invalid + invalid_vector el2t_sync_invalid, __hyp_save_context_and_panic invalid_vector el2t_irq_invalid invalid_vector el2t_fiq_invalid invalid_vector el2t_error_invalid diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 9387e3a0b680..f3d8fbc7a77b 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -753,7 +753,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __kvm_unexpected_el2_exception(void) { - extern char __hyp_restore_elr_and_panic[]; + extern char __hyp_restore_elr_save_context_and_panic[]; unsigned long addr, fixup; struct kvm_exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); @@ -776,7 +776,7 @@ static inline void __kvm_unexpected_el2_exception(void) /* Trigger a panic after restoring the hyp context. */ this_cpu_ptr(&kvm_hyp_ctxt)->sys_regs[ELR_EL2] = elr_el2; - write_sysreg(__hyp_restore_elr_and_panic, elr_el2); + write_sysreg(__hyp_restore_elr_save_context_and_panic, elr_el2); } #endif /* __ARM64_KVM_HYP_SWITCH_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 0613b6e35137..ec3e4f5c28cc 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -213,7 +213,7 @@ SYM_FUNC_END(__host_hvc) .endm .macro host_el2_sync_vect - __host_el2_vect __hyp_panic + __host_el2_vect __hyp_save_context_and_panic .endm .macro invalid_host_el1_vect diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index b3268933b093..17df57580c77 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -18,6 +18,7 @@ #include #include +#include #include #include #include @@ -308,7 +309,24 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) return ret; } -static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) +static void __noreturn __hyp_call_panic_for_cfi(u64 elr, u64 esr) +{ + struct user_pt_regs *regs = &this_cpu_ptr(&kvm_hyp_ctxt)->regs; + u8 type_idx = FIELD_GET(CFI_BRK_IMM_TYPE, esr); + u8 target_idx = FIELD_GET(CFI_BRK_IMM_TARGET, esr); + u32 expected_type = (u32)regs->regs[type_idx]; + u64 target = regs->regs[target_idx]; + + panic("VHE hyp CFI failure at: [<%016llx>] %pB (target: [<%016llx>] %ps, expected type: 0x%08x)\n" +#ifdef CONFIG_CFI_PERMISSIVE + " (CONFIG_CFI_PERMISSIVE ignored for hyp failures)\n" +#endif + , + elr, (void *)elr, target, (void *)target, expected_type); +} +NOKPROBE_SYMBOL(__hyp_call_panic_for_cfi); + +static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par, u64 esr) { struct kvm_cpu_context *host_ctxt; struct kvm_vcpu *vcpu; @@ -319,6 +337,9 @@ static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) __deactivate_traps(vcpu); sysreg_restore_host_state_vhe(host_ctxt); + if (IS_ENABLED(CONFIG_CFI_CLANG) && esr_is_cfi_brk(esr)) + __hyp_call_panic_for_cfi(elr, esr); + panic("HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n", spsr, elr, read_sysreg_el2(SYS_ESR), read_sysreg_el2(SYS_FAR), @@ -331,8 +352,9 @@ void __noreturn hyp_panic(void) u64 spsr = read_sysreg_el2(SYS_SPSR); u64 elr = read_sysreg_el2(SYS_ELR); u64 par = read_sysreg_par(); + u64 esr = read_sysreg_el2(SYS_ESR); - __hyp_call_panic(spsr, elr, par); + __hyp_call_panic(spsr, elr, par, esr); } asmlinkage void kvm_unexpected_el2_exception(void)