From patchwork Thu Mar 14 20:23:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13592734 Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com [209.85.167.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C606976034 for ; Thu, 14 Mar 2024 20:23:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447810; cv=none; b=eVH+OiDbU6B56SWLqpdv7QF149ygZRCpMAEIsG3HjE+FjkcSeiKbaQpj7QLm3o3MRQZFOquS8Mkn2giegtLLvxN6mahdRyiOeOfBs7mjmdgwEJbVo80I10wONy95hmetYQwZ3SO3b/bD7VFEfBvCmhjrMWbw8ky4ZUDMlZ4VyIE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447810; c=relaxed/simple; bh=T77wjvoVt0NOl52aLD2eH3RmLxFBGnGRYUOiu2OSeco=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=speguQtESGiQ5McLLavGygYc8cXmvTaW8RYmDM4+qaKxij3cU549KFUMwvociY4O6VXdQ1n7fH3WXjC0+zVxp10JLsFoUHz51Bntw7aFjBSyEM/XTzNHhUJeJz6EAyuTAwAdaLZRhAuzn/XTxY3YOblhbB9530shuI9mTgcyaVc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ye7p2FvL; arc=none smtp.client-ip=209.85.167.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ye7p2FvL" Received: by mail-lf1-f45.google.com with SMTP id 2adb3069b0e04-513d23be0b6so1145244e87.0 for ; Thu, 14 Mar 2024 13:23:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710447807; x=1711052607; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=vZK7QuyaiNpIMjIwqrL5CzPHrd9TH34r0/BLtCyNFDY=; b=ye7p2FvLf62F4b8I7F7xajfM1HkqbvVGpbNGjb3lZe9Sw3jnn3V9Mqn9CDQR3yETPW SMGoJYfoLfM87+k/0NYjDev3VYKnoBFSAgMes9OuBLTTL2AAEoHeLcP6Iq4vq50xjwrz mqspGoYmmzA7XZb9lc1eD0SqoS9h2gdpAyfqYyPPKz/+459EsmpuDZQ6ML+EL1o5OdEk Uk/81hd/PmOUc4/YZXcDf25EnalRN0lPjRisq0gYKDwHCh7/c2dzkoR+hLm155ZJcOBN STqnS8+SQRxMgh1w5ixoXZkcA336KGHkGPhim4apldQHFrsUOH3yq3fHf0//golL+sy4 QCtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710447807; x=1711052607; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=vZK7QuyaiNpIMjIwqrL5CzPHrd9TH34r0/BLtCyNFDY=; b=UtsTRqqAKF9GzVkptsLd8lSyb5C94hf+0ZddNO7+ZcxnKdU599/TD18V01qehvy2dq Hwa4808GoaF1GPqj9sbIyRh5+fDmWGdYgi+Z8Ef7WQkTaWjeAc+6d5UnUgO2jr/IqgQL wLWUaBtCF7SpeUInRKqF9+v/nHTL4TzyD9Ey7NObRJn5shFgFeIwqdDxFZwU1coW/68n g9d23+RZc+VLsqCs+/sbS3INkDf/4JEL3AhhliOqn4JnMMEYJMPDdtIn+eVgLogW9/ih ZcybPKlYThoDlFod5+z2YuW7BanhUylN2i6NlIFJe/47l586qP3TpOlupgnf6VSC20LP ESjg== X-Forwarded-Encrypted: i=1; AJvYcCWxzzGTs2XJves9dDzILo2C8PsplvbEVY4jidYo5nlS6eZ6yo/eFcrztMw08s40mD46AZFGSLkKjrPDQ/xUQIiNQ32g X-Gm-Message-State: AOJu0Ywo18kMIaeUHE20tqGJ1+jU/VK5J7YoEQTBb5CO7r/AIjN2L2kY Yzq/2K0S6OC9XfKVT3IFFm/raDzx3QPMaqRDk/eqf+RPzItsVFw6CI2mVes7uw== X-Google-Smtp-Source: AGHT+IHQUp4F505tINC00/h603tRttrCrGmHbysW4OYqjcYsqgNLrProci8dq24A2bPmtjVgq53zeg== X-Received: by 2002:ac2:57c2:0:b0:513:d522:a647 with SMTP id k2-20020ac257c2000000b00513d522a647mr809318lfo.63.1710447806531; Thu, 14 Mar 2024 13:23:26 -0700 (PDT) Received: from google.com (64.227.90.34.bc.googleusercontent.com. [34.90.227.64]) by smtp.gmail.com with ESMTPSA id h11-20020a1709062dcb00b00a4662df0319sm1018769eji.65.2024.03.14.13.23.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Mar 2024 13:23:25 -0700 (PDT) Date: Thu, 14 Mar 2024 20:23:22 +0000 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Marc Zyngier , Andrew Scull Subject: [PATCH 01/10] KVM: arm64: Fix clobbered ELR in sync abort Message-ID: <0dfcc4c5c898941147723ba530c81ddc8399ef55.1710446682.git.ptosi@google.com> References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: When the hypervisor receives a SError or synchronous exception (EL2h) while running with the __kvm_hyp_vector and if ELR_EL2 doesn't point to an extable entry, it panics indirectly by overwriting ELR with the address of a panic handler in order for the asm routine it returns to to ERET into the handler. This is done (instead of a simple function call) to ensure that the panic handler runs with the SPSel that was in use when the exception was triggered, necessary to support features such as the shadow call stack. However, this clobbers ELR_EL2 for the handler itself. As a result, hyp_panic(), when retrieving what it believes to be the PC where the exception happened, actually ends up reading the address of the panic handler that called it! This results in an erroneous and confusing panic message where the source of any synchronous exception (e.g. BUG() or kCFI) appears to be __guest_exit_panic, making it hard to locate the actual BRK instruction. Therefore, store the original ELR_EL2 in a per-CPU struct and point the sysreg to a routine that first restores it to its previous value before running __guest_exit_panic. Fixes: 7db21530479f ("KVM: arm64: Restore hyp when panicking in guest context") Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kvm/hyp/entry.S | 9 +++++++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 6 ++++-- 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 5a7dbbe0ce63..e62353168a57 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -128,6 +128,7 @@ int main(void) DEFINE(VCPU_FAULT_DISR, offsetof(struct kvm_vcpu, arch.fault.disr_el1)); DEFINE(VCPU_HCR_EL2, offsetof(struct kvm_vcpu, arch.hcr_el2)); DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_cpu_context, regs)); + DEFINE(CPU_ELR_EL2, offsetof(struct kvm_cpu_context, sys_regs[ELR_EL2])); DEFINE(CPU_RGSR_EL1, offsetof(struct kvm_cpu_context, sys_regs[RGSR_EL1])); DEFINE(CPU_GCR_EL1, offsetof(struct kvm_cpu_context, sys_regs[GCR_EL1])); DEFINE(CPU_APIAKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APIAKEYLO_EL1])); diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index f3aa7738b477..9cdf46da3051 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,6 +83,15 @@ alternative_else_nop_endif eret sb +SYM_INNER_LABEL(__guest_exit_panic_with_restored_elr, SYM_L_GLOBAL) + // x0-x29,lr: hyp regs + + stp x0, x1, [sp, #-16]! + adr_this_cpu x0, kvm_hyp_ctxt, x1 + ldr x0, [x0, #CPU_ELR_EL2] + msr elr_el2, x0 + ldp x0, x1, [sp], #16 + SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) // x2-x29,lr: vcpu regs // vcpu x0-x1 on the stack diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index a038320cdb08..6a8dc8d3c193 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -747,7 +747,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __kvm_unexpected_el2_exception(void) { - extern char __guest_exit_panic[]; + extern char __guest_exit_panic_with_restored_elr[]; unsigned long addr, fixup; struct kvm_exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); @@ -769,7 +769,9 @@ static inline void __kvm_unexpected_el2_exception(void) } /* Trigger a panic after restoring the hyp context. */ - write_sysreg(__guest_exit_panic, elr_el2); + write_sysreg(__guest_exit_panic_with_restored_elr, elr_el2); + + this_cpu_ptr(&kvm_hyp_ctxt)->sys_regs[ELR_EL2] = elr_el2; } #endif /* __ARM64_KVM_HYP_SWITCH_H__ */ From patchwork Thu Mar 14 20:23:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13592735 Received: from mail-lj1-f174.google.com (mail-lj1-f174.google.com [209.85.208.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F274A71741 for ; Thu, 14 Mar 2024 20:23:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447825; cv=none; b=If6HXahBJgpVdWotO5JOqofQOzyKZb6NyKVGhDJozf8UHe2vn3jHvB+DZaL1TDqMrwf/Xf74vbBneiTnhFuRl2R83IhpykyCDg2Y3nG4tPWDiuD0qM4jBzuoMyNPQ20xwhMnEfOSwaqPsy8aUkZyVjRuXmBOMc2AuG3DxYUStGY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447825; c=relaxed/simple; bh=/g/HcoEQQBJkRAL4ZwbS8omMCk85N2nyfxKtjk668ZM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=MQ+3ENWdvbeIJwPoY9a/9HaXVO9YZO+FnTPt9QBZNMtbIQEztxfaffCWR1xsEsI7usgxXhmOPNj8GRRTIZTz/T7qlXcd/nvVYUXag1WpAnsjA3pCIRLbkrqPbKyTI2vzz1o61gcQx8d3+Ozfck9sw21m4KXOp9VlNKQdPq8WzDw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=1EkErgJt; arc=none smtp.client-ip=209.85.208.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="1EkErgJt" Received: by mail-lj1-f174.google.com with SMTP id 38308e7fff4ca-2d475b6609eso17445011fa.2 for ; Thu, 14 Mar 2024 13:23:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710447822; x=1711052622; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=bhfRLCv5E8LczXhWX76XPlZKS4T8I7qgV0XO4dVuEZg=; b=1EkErgJtjpSjxkJR61YW7Hh5oWZkQImO+q1fy+cHh3S305YVpA0saVmsU3Fpb1xSxi 2r4rm3Y3vrmutcNcEqq7ejeJCMQkQHjzZTD5LhR9OOUbnCBQEWznmjguTXdWIJc8w4+x Tf3cdgO4KZhqgqlgggREuoyMf7RCLbouuUHCJCCdeMTQ9Gz6YZhgN5Uvpgb0o323JEnv u02ZjtWJLfpQQ/6/SpvWQZd0DAWX8IcpZWLTfV7U87Eau6gr6xdLSDFGVIEEmmyAyHs3 1+G52IoDNyl2SuDs+CZZ6/U4YAwVl8FJHlk1gqErcJfc3rFVkSspopsXtxYxp+dD8Zcd f7xA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710447822; x=1711052622; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=bhfRLCv5E8LczXhWX76XPlZKS4T8I7qgV0XO4dVuEZg=; b=a/9f/ffoN6i4Do5gRfIi+OM2yEfrzZ6DPfuLVBM4Y5x7Mip+rUpRT1dsWZmRalp7tY p/VoOfGveMftMDpJqy7ZWNMu0HJCE7lFxtgX41zCxl58gYnfAxU2bOjJCrxv/5xvfNms seibibWnqDbEKKogk6JmY9mJtjRougwr4F6eJvkP4k+dx7x+zzJb0egFhaltOdJTS2vN VXeNnY4KQl9Yr/NyYaXxRQtxNvWYxwNKaWabXUT/7DCqpVTURqmcWzs04bUrBsjeiq8T BU630MzWZuUBPgvpyRkaAB2Q0pbbrniz9kVj71YoZQM3jNBSWEI7xzmtTf8rPrkWQ2OG doFw== X-Forwarded-Encrypted: i=1; AJvYcCUZGu6o3wZyrjSCoaDbTaIyx+sSB47iYktn+Gb+aqgMPTSwBDzycuw+dxlBj63eVZ3Uw88dVJiklK4soY9Po8ngSHaz X-Gm-Message-State: AOJu0YyMlq5dGqytUCybPRXkMGtlrlrKYynVjkNlOSqsAW/HGTseUM9U fUqJpg+eSGjns0UtN8tbIGuB+vjOPhAHDuPqb2ohp7a4dUPyHeznUVYIdXpACg== X-Google-Smtp-Source: AGHT+IFKN6Q76VRunakd62YijaomXD5YfYUy4KHbzgcm8HAUJxkFpfyEXSR19suk+AGp/vhNuREp1g== X-Received: by 2002:a05:6512:52f:b0:513:ba0d:c2fa with SMTP id o15-20020a056512052f00b00513ba0dc2famr1938984lfc.54.1710447821974; Thu, 14 Mar 2024 13:23:41 -0700 (PDT) Received: from google.com (64.227.90.34.bc.googleusercontent.com. [34.90.227.64]) by smtp.gmail.com with ESMTPSA id v15-20020a1709063bcf00b00a46454a7e24sm1007519ejf.71.2024.03.14.13.23.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Mar 2024 13:23:41 -0700 (PDT) Date: Thu, 14 Mar 2024 20:23:37 +0000 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Marc Zyngier , Will Deacon , Quentin Perret Subject: [PATCH 02/10] KVM: arm64: Fix __pkvm_init_switch_pgd C signature Message-ID: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Update the function declaration to match the asm implementation. Fixes: f320bc742bc2 ("KVM: arm64: Prepare the creation of s1 mappings at EL2") Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/kvm_hyp.h | 3 +-- arch/arm64/kvm/hyp/nvhe/setup.c | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 145ce73fc16c..7a0082496d4a 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -123,8 +123,7 @@ void __noreturn __hyp_do_panic(struct kvm_cpu_context *host_ctxt, u64 spsr, #endif #ifdef __KVM_NVHE_HYPERVISOR__ -void __pkvm_init_switch_pgd(phys_addr_t phys, unsigned long size, - phys_addr_t pgd, void *sp, void *cont_fn); +void __pkvm_init_switch_pgd(phys_addr_t params, void (*finalize_fn)(void)); int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, unsigned long *per_cpu_base, u32 hyp_va_bits); void __noreturn __host_enter(struct kvm_cpu_context *host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index bc58d1b515af..bcaeb0fafd2d 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -316,7 +316,7 @@ int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, { struct kvm_nvhe_init_params *params; void *virt = hyp_phys_to_virt(phys); - void (*fn)(phys_addr_t params_pa, void *finalize_fn_va); + typeof(__pkvm_init_switch_pgd) *fn; int ret; BUG_ON(kvm_check_pvm_sysreg_table()); From patchwork Thu Mar 14 20:23:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13592736 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1528676034 for ; Thu, 14 Mar 2024 20:23:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447840; cv=none; b=W21b2dOc642mCfppuPnwavogjWWeHnw1OCVpV4rDpR2LvPWv2Wzieu+pHJNIDb/Wr0vsJacX6yiDFPcwKRkfrEjICCrcrIf+jZNFAfbuC11WAPGzmSpeKSZXigukWIYbBAdy/7B++8mASiPVXgYG2goQTWKr/E2dzFd33pPAUFg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447840; c=relaxed/simple; bh=i2a49zzhhviQbMDBD+mz1nCXcS8PgLH45YTWytn4lgM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=D5YUGglFLy0AVXPMdZrfN25Nuvfqa6so83VFyRSpG1kCVthFl4CpnNiDLcdUdCVLb43Vo3jqBuiu9RBnjWBgEfe3yB7a/8KeP+OVyuZ6P1c30nLhK5XctAuQ0+YnXUEdkivKqFh7Rvt89ln0jqUauKfcED13AbUsMPnAizj7PRI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Fhx/l3mq; arc=none smtp.client-ip=209.85.218.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Fhx/l3mq" Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-a44665605f3so146600266b.2 for ; Thu, 14 Mar 2024 13:23:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710447837; x=1711052637; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=SuRgs8OupZSCiXLSHaIaVM3v2VB8UqXtWW7TcEnKPlc=; b=Fhx/l3mqPTC1azWtW4NxVXXVCrdNfr3TvAT8DBet1MvWUe9nt0hvtVbVfK73QaOT4u aoQyJ/F70wGrKKG6T2PWS2UtmZGVki/9s9rTMYTlbJpxnu8MgXolAyb8aHIAWq00CbLn uP8421yVvsIsZLd9DQRagmpKXsB8xy0Q312Cl/4QYsdwK2ugDvEIRdFN/PuNGMTkQmir Y04o39M9tOa5W7cxdoVba7B2NNmBy53N2wUva03dHp2ib7k9743f0hFeWKkmALAlt5nw n+rezRf+VR3O79a9R9WMcv75qMWRAZGVIhzXFrLYn+lSssXH8HyO8gz2HHPbLgiGrP8D X7vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710447837; x=1711052637; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=SuRgs8OupZSCiXLSHaIaVM3v2VB8UqXtWW7TcEnKPlc=; b=GnnAX6ufZHoUPRQWYr647O0BKABRW8+BmPINl9mt00EZ0ZgEObHBqyKFBOZoJhOHQZ RoA/15jTg3faHHgZ+9PW6mAphmCs0f0RDt6Fs8YxEg7WHsY/pJY5CB7uPLr0qiTd+zak 4nJc9z+VOxLwXXfcamjHOTufeyv6P7dViBwBERUguUXhiQ33TC8WV6snH4McxQ5+MKnw rJJ+MhsyEfblilMZ3gjUtQ+/fFK5k0CjA6fdx9vgXxgt8m79UOfWkcnVxcjUHhGW8GwJ ee8KbXoBWRBcpGgk9hRrUwLXlKjru1CEihYE7QHYnK3yPfa/Kpya7mx6/qRYRIpEFaJB FElw== X-Forwarded-Encrypted: i=1; AJvYcCXq4hI1xOo9++KYMqDBNFy0I/ukft/VZLkh4++wbRJlRilaGOeIcgPTCq8e55gZAvwpwszeMxVl4C7AYvmhMEY0jfEz X-Gm-Message-State: AOJu0Yx2ZylxmEv/yGMuLJCZbs0+kGr9JYSwzLWGzqCtz0T5WXYm0cZS uSKsQlQmbQm/G9sxGf2FehvqYnFaLkQhF8K4xOIdNrsauUoXe0Jdu+uhcflHPQ== X-Google-Smtp-Source: AGHT+IFsZiG44+ltcDXnRY8E+FNEpm//GsTqieZeJfeGLiwfbk3sWb8YsaRi2GbAi6yCl23eklsVOw== X-Received: by 2002:a17:907:968c:b0:a46:220c:a55 with SMTP id hd12-20020a170907968c00b00a46220c0a55mr107555ejc.73.1710447837274; Thu, 14 Mar 2024 13:23:57 -0700 (PDT) Received: from google.com (64.227.90.34.bc.googleusercontent.com. [34.90.227.64]) by smtp.gmail.com with ESMTPSA id r7-20020a1709067fc700b00a45fefe4fc5sm1007501ejs.135.2024.03.14.13.23.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Mar 2024 13:23:56 -0700 (PDT) Date: Thu, 14 Mar 2024 20:23:53 +0000 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Marc Zyngier , Will Deacon , Quentin Perret Subject: [PATCH 03/10] KVM: arm64: Pass pointer to __pkvm_init_switch_pgd Message-ID: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Make the function take a VA pointer, instead of a phys_addr_t, to fully take advantage of the high-level C language and its type checker. Perform all accesses to the kvm_nvhe_init_params before disabling the MMU, removing the need to access it using physical addresses, which was the reason for taking a phys_addr_t. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/kvm_hyp.h | 3 ++- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 8 +++++--- arch/arm64/kvm/hyp/nvhe/setup.c | 4 +--- 3 files changed, 8 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 7a0082496d4a..32fb866d1229 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -123,7 +123,8 @@ void __noreturn __hyp_do_panic(struct kvm_cpu_context *host_ctxt, u64 spsr, #endif #ifdef __KVM_NVHE_HYPERVISOR__ -void __pkvm_init_switch_pgd(phys_addr_t params, void (*finalize_fn)(void)); +void __pkvm_init_switch_pgd(struct kvm_nvhe_init_params *params, + void (*finalize_fn)(void)); int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, unsigned long *per_cpu_base, u32 hyp_va_bits); void __noreturn __host_enter(struct kvm_cpu_context *host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 2994878d68ea..8958dd761837 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -266,6 +266,10 @@ alternative_else_nop_endif SYM_CODE_END(__kvm_handle_stub_hvc) SYM_FUNC_START(__pkvm_init_switch_pgd) + /* Load the inputs from the VA pointer before turning the MMU off */ + ldr x5, [x0, #NVHE_INIT_PGD_PA] + ldr x0, [x0, #NVHE_INIT_STACK_HYP_VA] + /* Turn the MMU off */ pre_disable_mmu_workaround mrs x2, sctlr_el2 @@ -276,15 +280,13 @@ SYM_FUNC_START(__pkvm_init_switch_pgd) tlbi alle2 /* Install the new pgtables */ - ldr x3, [x0, #NVHE_INIT_PGD_PA] - phys_to_ttbr x4, x3 + phys_to_ttbr x4, x5 alternative_if ARM64_HAS_CNP orr x4, x4, #TTBR_CNP_BIT alternative_else_nop_endif msr ttbr0_el2, x4 /* Set the new stack pointer */ - ldr x0, [x0, #NVHE_INIT_STACK_HYP_VA] mov sp, x0 /* And turn the MMU back on! */ diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index bcaeb0fafd2d..45b83f3ed012 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -314,7 +314,6 @@ void __noreturn __pkvm_init_finalise(void) int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, unsigned long *per_cpu_base, u32 hyp_va_bits) { - struct kvm_nvhe_init_params *params; void *virt = hyp_phys_to_virt(phys); typeof(__pkvm_init_switch_pgd) *fn; int ret; @@ -338,9 +337,8 @@ int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, update_nvhe_init_params(); /* Jump in the idmap page to switch to the new page-tables */ - params = this_cpu_ptr(&kvm_init_params); fn = (typeof(fn))__hyp_pa(__pkvm_init_switch_pgd); - fn(__hyp_pa(params), __pkvm_init_finalise); + fn(this_cpu_ptr(&kvm_init_params), __pkvm_init_finalise); unreachable(); } From patchwork Thu Mar 14 20:24:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13592737 Received: from mail-ed1-f46.google.com (mail-ed1-f46.google.com [209.85.208.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 277E47317E for ; Thu, 14 Mar 2024 20:24:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447861; cv=none; b=ez002zvXq/fTbPdocNb6UZe28yTTs/BGx/0lBP3CqoC2swjTDvbzXiI6RLqAdkEPOdgL1I020eRAf15JJk3am9JASYcSnK6kkdwRlnDaVyRPhMyLWXRNe6blTbRBP8LEnxabPIKEVt+rsDx0lIqEqnYLdGsKFBn1h+lLUBhbQc4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447861; c=relaxed/simple; bh=VNW8QZydmWpg/n7NPOfyNLIqspWVeZ/mxJN8WQ/9sL8=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=szC6f2bRGjDVx185yLT2PN63A0IPKpiwH1wiu3yuLQ0QuBEkNAQxEMIBhan0KYViYEBy7ac0205R987xIsRLHJk2Nk8+mzLls/fwXz4wsCPrctsf1atZRyM7s4oE8R9wrk4jXxXXoQIN0POoX2DyBcEwdemeNpP/WxejBqGk1E4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Wf7HEDqD; arc=none smtp.client-ip=209.85.208.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Wf7HEDqD" Received: by mail-ed1-f46.google.com with SMTP id 4fb4d7f45d1cf-568aa282ccdso275464a12.3 for ; Thu, 14 Mar 2024 13:24:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710447858; x=1711052658; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=FGHXKr9Rrdhb7ltTqwpYqVe2UrsjZifyrXsCqgg5SVI=; b=Wf7HEDqD89CIX62zz0HCMTgJZYFd2w+Vg+gKUV4D4BW3W60gmNc4WQ/LyWEmbXKh0p walKocegqY1iBfhHEWOn6SINtNRau22ObMKk538O0nn7vemuVn3PDhswtw/MgdXUg7sQ bPPDH8GPbIeJT0JUi3dzDsYkBvDqbJFt+BPA/8hoSAAnrIsjmhCo6L1uHNYHAxHObQ+1 eT12ZbAkFMfULbcV7NVwPC3uQPmLDjfH+gOHiKJxgb8qtJ8TIQ2WFBnx9L2MsByVT/U2 Eu2i99h90YYTSLJy8qZcpN2Lm2eF2YnUTldgVt7jjjQ6lfhceA1h/n1nrIrg5Ii1Kj02 8Oyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710447858; x=1711052658; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FGHXKr9Rrdhb7ltTqwpYqVe2UrsjZifyrXsCqgg5SVI=; b=t2me3gwEu04uLPQmSNXXh+aksiVku118J3i1ooJ4tSuJgfpjNaGvIkyuOnVCVN2Trt xP1aFLXFhUftiA2hxsi7JjBkBjrDIquX1vHvNA0Ql/e/wmXIeY++l3dm6Ts9jtScobC/ IwoI0NcCsqcICD/gL2QuuVyiCePc+vuBglnSCjnJp3bqcrsF6QtaVY7qJS/K0Sufs6LM i1Mgzkfcv6f9phvKpz11GnhOo9QED8RXrmBAaGmcwfl9gXDQHdTHTpaZI/UbYfjhf5L8 ApQj5HK1NqK/OvADjXB28p94BZ33Q7ocJTeEMyLnD0h/9hi6UBF2BasmH4coF9ghN17v /2gA== X-Forwarded-Encrypted: i=1; AJvYcCV+D+qExaFI3S+TU/71VqrsJPam1mjwnefDBvkCM7aIipoEn33HI7N9RFMVE4YbNSR5vg9BVaZ8GjYmS6LppmA9uuio X-Gm-Message-State: AOJu0Yy/VH5AKYy1UgdPBogrsfMV60/Nx5dfH3nf8IUKNdQxS/jMEUBg e9FSYwBs3fCQ3wjMN6iRfkIAR8SD8vOy0hygAIGh33apE1qOE63v0TenPrjGjP0xj12o1h32Q0G sQMsf X-Google-Smtp-Source: AGHT+IGDSjzpxZf8AVYQE+b2VkhZATxpfpcqloyDmGyoEsgY/Kxfu5Lu+ucHeadFj0dWnxHl32HuWQ== X-Received: by 2002:a05:6402:160c:b0:568:335d:a95d with SMTP id f12-20020a056402160c00b00568335da95dmr1235050edv.18.1710447858444; Thu, 14 Mar 2024 13:24:18 -0700 (PDT) Received: from google.com (64.227.90.34.bc.googleusercontent.com. [34.90.227.64]) by smtp.gmail.com with ESMTPSA id m24-20020aa7d358000000b0056729e902f7sm1019134edr.56.2024.03.14.13.24.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Mar 2024 13:24:18 -0700 (PDT) Date: Thu, 14 Mar 2024 20:24:14 +0000 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Marc Zyngier , Andrew Scull Subject: [PATCH 04/10] KVM: arm64: nVHE: Simplify __guest_exit_panic path Message-ID: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Immediately jump to __guest_exit_panic when taking an invalid EL2 exception with the nVHE host vector table instead of first duplicating the vCPU context check that __guest_exit_panic will also perform. Fix the wrong (probably bitrotten) __guest_exit_panic ABI doc to reflect how it is used by VHE and (now) nVHE and rename the routine to __hyp_panic to better reflect that it might not exit through the guest but will always (directly or indirectly) end up executing hyp_panic(). Use CPU_LR_OFFSET to clarify that the routine returns to hyp_panic(). Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/hyp/entry.S | 14 +++++++++----- arch/arm64/kvm/hyp/hyp-entry.S | 2 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/host.S | 8 +------- 4 files changed, 13 insertions(+), 15 deletions(-) diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index 9cdf46da3051..ac8aa8571b2f 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,7 +83,7 @@ alternative_else_nop_endif eret sb -SYM_INNER_LABEL(__guest_exit_panic_with_restored_elr, SYM_L_GLOBAL) +SYM_INNER_LABEL(__hyp_panic_with_restored_elr, SYM_L_GLOBAL) // x0-x29,lr: hyp regs stp x0, x1, [sp, #-16]! @@ -92,13 +92,15 @@ SYM_INNER_LABEL(__guest_exit_panic_with_restored_elr, SYM_L_GLOBAL) msr elr_el2, x0 ldp x0, x1, [sp], #16 -SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) - // x2-x29,lr: vcpu regs - // vcpu x0-x1 on the stack +SYM_INNER_LABEL(__hyp_panic, SYM_L_GLOBAL) + // x0-x29,lr: vcpu regs + + stp x0, x1, [sp, #-16]! // If the hyp context is loaded, go straight to hyp_panic get_loaded_vcpu x0, x1 cbnz x0, 1f + ldp x0, x1, [sp], #16 b hyp_panic 1: @@ -110,10 +112,12 @@ SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) // accurate if the guest had been completely restored. adr_this_cpu x0, kvm_hyp_ctxt, x1 adr_l x1, hyp_panic - str x1, [x0, #CPU_XREG_OFFSET(30)] + str x1, [x0, #CPU_LR_OFFSET] get_vcpu_ptr x1, x0 + // Keep x0-x1 on the stack for __guest_exit + SYM_INNER_LABEL(__guest_exit, SYM_L_GLOBAL) // x0: return code // x1: vcpu diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 03f97d71984c..7e65ef738ec9 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -122,7 +122,7 @@ el2_error: eret sb -.macro invalid_vector label, target = __guest_exit_panic +.macro invalid_vector label, target = __hyp_panic .align 2 SYM_CODE_START_LOCAL(\label) b \target diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 6a8dc8d3c193..0dc721ced358 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -747,7 +747,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __kvm_unexpected_el2_exception(void) { - extern char __guest_exit_panic_with_restored_elr[]; + extern char __hyp_panic_with_restored_elr[]; unsigned long addr, fixup; struct kvm_exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); @@ -769,7 +769,7 @@ static inline void __kvm_unexpected_el2_exception(void) } /* Trigger a panic after restoring the hyp context. */ - write_sysreg(__guest_exit_panic_with_restored_elr, elr_el2); + write_sysreg(__hyp_panic_with_restored_elr, elr_el2); this_cpu_ptr(&kvm_hyp_ctxt)->sys_regs[ELR_EL2] = elr_el2; } diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 7693a6757cd7..27c989c4976d 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -196,19 +196,13 @@ SYM_FUNC_END(__host_hvc) tbz x0, #PAGE_SHIFT, .L__hyp_sp_overflow\@ sub x0, sp, x0 // x0'' = sp' - x0' = (sp + x0) - sp = x0 sub sp, sp, x0 // sp'' = sp' - x0 = (sp + x0) - x0 = sp - /* If a guest is loaded, panic out of it. */ - stp x0, x1, [sp, #-16]! - get_loaded_vcpu x0, x1 - cbnz x0, __guest_exit_panic - add sp, sp, #16 - /* * The panic may not be clean if the exception is taken before the host * context has been saved by __host_exit or after the hyp context has * been partially clobbered by __host_enter. */ - b hyp_panic + b __hyp_panic .L__hyp_sp_overflow\@: /* Switch to the overflow stack */ From patchwork Thu Mar 14 20:24:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13592738 Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 400667317E for ; Thu, 14 Mar 2024 20:24:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447878; cv=none; b=cYItYlP8QJcOKnvG2OCN03AIzi7anLCp33QLklU/dsAcUDnebxC0dPpRmuvB9cN0aFC+0ufSby9js3FASJ2ExHzUq8O1VJuOUoWc5EBZSAFOZbJjmocM9CGT96clcpvr9WdNAlYKbI3IZBu7c9N/KxezVIb3zZq4MgjGaShU8Go= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447878; c=relaxed/simple; bh=dGbffwfi9F8o5vKXOK784z8lOxK3B61CBLe05LF4uDg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=i/v1Omr1/vKvRUxhr8WPzvj34R5p57GqFNWWaMKzZSHGAvpRIvX4SgS36rUbnw4PQtgUT0BfGPqOUlbN+VSOdkbjajDmbww2/62wi1//cWoj7MeGaARkA/HzAxcRv6s002Qka4tZjBmd8+wsoftfez25YhFasTEZUgQSPUlnm7o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=HFM++K7D; arc=none smtp.client-ip=209.85.218.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="HFM++K7D" Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-a466f6318e9so156636066b.0 for ; Thu, 14 Mar 2024 13:24:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710447875; x=1711052675; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=KeVadU2hZeC83X7EyIxA1rm2PDaUR9fvRb0RDycrigU=; b=HFM++K7DS9n3auHGlSq0b/iJ4dzTd9XK+5FpDgDikFfs+DNzZr/T6LTlEcMWB5bmTk MmiIUieOyAnY3UPQFT7XzoHsKrHUQpNvbkkGGsB0SW3hc1kHzbGqKb2VUr94sUkHHziN K7nUaSCDLnJOWKTPV4XQUlVxtY/phbZHbM1MkPxfpZEnD+Uuj4nYROHKXcic+42z3Lwj svtHNdPfVx269uSjQg/+1DVKMWebjxU7NB6LgaSCPbhl0PfXsPwkyWkIZ2RL1azVNNF7 dgcZw+akvFuCzEYwoJfvws4qEPZH5BN0qdg+A+JdCw8trHxQ9jBpis16s4fOGi1U5++Y d+GQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710447875; x=1711052675; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KeVadU2hZeC83X7EyIxA1rm2PDaUR9fvRb0RDycrigU=; b=DQj57DKJuPN0Kjp5IncSawxzTUGMsUfyg1z3CrL0WMUIvVJXH1XUOB2e0z0Xr6So32 pyXiRijGM5800o/Qa/5LBTvpWtxX+pBQ/HO6TIY3v+hLU+/f0EQcFknXlh+hBU+Oflkz lNkVy/Hc0mSMe76DhEypy0/PsoNqj01wTR1FlI7pQuAzQDGVAcv107ToV+X2lve6Y5L+ JRbtxyD+FxfHvPFgIrMrhVtfaYorxvQ6pFe4nTO8QOqvZVhsQ/He6298Wm/0ZXVMKhPQ pgPQa14vKW+Uu4vmLmBT1tqLNN4stTupKdNJ9eRqLLMgNd5uDRqTirK7KX2N7RcXKGYv 59lw== X-Forwarded-Encrypted: i=1; AJvYcCUkC1mNLh2wONQe8OXDf/dQfXnXG40AerOxRm2wGoqvzCYkcCUYWuPaSAmmFKUud1qG7MkYojETkgfVkAlyLnaey7RE X-Gm-Message-State: AOJu0YzHdABug41mnj9/HlWQh1uYijrm/yWau2wPam5AHj9b2I2Bx8iZ VlrKkAN6kNO7GshlcyDmLGGyVvSGPkHG8yqVcFXiFYWT7P8W4uz/2TrU+EG9kA== X-Google-Smtp-Source: AGHT+IHucnNpHcGZETVpGOJfR1+fsfUsDKCT0Rc1DvLZfkdGAS5fojQQqY/5p5KP8nfAh80FDRV5yA== X-Received: by 2002:a17:906:bc56:b0:a46:5e38:29d7 with SMTP id s22-20020a170906bc5600b00a465e3829d7mr1861511ejv.10.1710447875391; Thu, 14 Mar 2024 13:24:35 -0700 (PDT) Received: from google.com (64.227.90.34.bc.googleusercontent.com. [34.90.227.64]) by smtp.gmail.com with ESMTPSA id vi2-20020a170907d40200b00a45c8b6e965sm1012429ejc.3.2024.03.14.13.24.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Mar 2024 13:24:35 -0700 (PDT) Date: Thu, 14 Mar 2024 20:24:31 +0000 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Marc Zyngier Subject: [PATCH 05/10] KVM: arm64: nVHE: Add EL2 sync exception handler Message-ID: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Introduce handlers for EL2{t,h} synchronous exceptions distinct from handlers for other "invalid" exceptions when running with the nVHE host vector. This will allow a future patch to handle CFI (synchronous) errors without affecting other classes of exceptions. Remove superfluous SP overflow check from the non-synchronous handlers. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/hyp/nvhe/host.S | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 27c989c4976d..1b9111c2b480 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -183,7 +183,7 @@ SYM_FUNC_END(__host_hvc) .endif .endm -.macro invalid_host_el2_vect +.macro host_el2_sync_vect .align 7 /* @@ -221,6 +221,11 @@ SYM_FUNC_END(__host_hvc) b __hyp_do_panic .endm +.macro invalid_host_el2_vect + .align 7 + b __hyp_panic +.endm + /* * The host vector does not use an ESB instruction in order to avoid consuming * SErrors that should only be consumed by the host. Guest entry is deferred by @@ -233,12 +238,12 @@ SYM_FUNC_END(__host_hvc) */ .align 11 SYM_CODE_START(__kvm_hyp_host_vector) - invalid_host_el2_vect // Synchronous EL2t + host_el2_sync_vect // Synchronous EL2t invalid_host_el2_vect // IRQ EL2t invalid_host_el2_vect // FIQ EL2t invalid_host_el2_vect // Error EL2t - invalid_host_el2_vect // Synchronous EL2h + host_el2_sync_vect // Synchronous EL2h invalid_host_el2_vect // IRQ EL2h invalid_host_el2_vect // FIQ EL2h invalid_host_el2_vect // Error EL2h From patchwork Thu Mar 14 20:24:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13592739 Received: from mail-lj1-f173.google.com (mail-lj1-f173.google.com [209.85.208.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C369D7317E for ; Thu, 14 Mar 2024 20:24:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447897; cv=none; b=bUCYcRqCMMfw4Ma8OZkDF66s/4tP1Tc4XS/7jXjGfxUJF5GFbxCaxfykBlJ7li+/+WqzZDJJnGhPDrLtD1/eiucNmYII+xJitRizoeVOBGAztwmwQoGfQUm4XAt+55UvpPejoz7Dw5q56vGjLR0HEOvRkD38iQ1vCd3dG96R1Jc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447897; c=relaxed/simple; bh=6/Yu0QDPKynolND3gmbYFuiviBU1OTdPkmZfwvNmj30=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=YYH0z3l2Ce1ObMJX4dXYnAYRtjvKrmUHIkKs+Ycgh6tgK3CZlmmum8O2m3OeyHvy64c41dBwL/4DVEQnALk5VmSKw30J4uGJSUDWenm4g3NCvHTgCETXW1E7H2hK+1z/WGCWbw1QUt+W6lrJAXPM2QJ9WxmsnHADm2TIzegkd8k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LOmi4FF0; arc=none smtp.client-ip=209.85.208.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LOmi4FF0" Received: by mail-lj1-f173.google.com with SMTP id 38308e7fff4ca-2d485886545so5425901fa.2 for ; Thu, 14 Mar 2024 13:24:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710447894; x=1711052694; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=xuCZTiMpeaXF4P5JXeYvn/6B04UPpdx+BA/DKeRB3B0=; b=LOmi4FF0sbhZNtmfvZo6/4fZ8u9zWqEUQBUdD9kXOTG/Mqh8jCGF+eJL290oFKVlbh 6OBQg1sq36Ye2eyWGZPxyxRfyts3bdocrmOJPwuB/EKj/R5Sclp6nzfSdkAQO6i9HhtC LQ1M13aHrfvy4eIZowIVS9oI3bBXOjtERKAjDoCp+8EYIOjUQV2mcLaOQyRL/ScAX+cD UPqG1TyZoC57nm/VIJlzKAOEnYiUVrC+L+KB2VuP6ja8preWdoE2e5OS9EPbeTFytsCt UwG/8fdyYImAr+Y+4Ae00TDn+swd3loJo+3YLYliN5+/qysQOeSK7FXv1ViUmorjn6Xt 0tIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710447894; x=1711052694; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xuCZTiMpeaXF4P5JXeYvn/6B04UPpdx+BA/DKeRB3B0=; b=KnIS/zjQVwkg5K1G079aSvlbwau6x46p6D3N6F5l/tgwa5Qxcc8qXj1+bEXXALfGXt MmlASJWaPzVy7UlZNiBIPIQVVFdtUIyXbRO5W7VSPLJI8XQ2IaX852ofwAMb+Me5fPH8 BgWZi8nS8gLYPgJL6BAuA5iarDHwHz6G5enu9Ut8v2uxnc+GXckuGzQyTK6BaL2KvG8p ia04a7DhPAUvwjDr8/aFBEEPxxT2skDeU7QYpTJAiku/9ETWF+ZIinu1iIS5lmmtraaY GFAGQuatZDCN5V+9bwT05XrUC9x3JDNj110GICKIKEwQhoY1WjCoEjxCLsHt33F+cN/0 QPbg== X-Forwarded-Encrypted: i=1; AJvYcCUIhWHCiWSc5YBswNt0hg2FZhTEzR6yGi63C/H+buii0CsFo46bGXv6yXxGGAe9kO82HMVltMIwlyBRc1r+8gfnoCvD X-Gm-Message-State: AOJu0YwMvvWrQ2qeJ+Mvf+rP34zwbYbY4idh3T0qyxb0hMCORsBxtAnj YyXFNTiQzAiLhLJT91025wy81T5Fk4yGx189hFX1QUrRIAMdF01zSiBJ8kO15g== X-Google-Smtp-Source: AGHT+IGdzgal9d7jECsheAve01ljHpLhiGK1mtn9v3Osy56JhBtqA4ZSO+j/CgBHoCEF6ku+b5RVYA== X-Received: by 2002:ac2:4ed1:0:b0:513:caa7:3773 with SMTP id p17-20020ac24ed1000000b00513caa73773mr867447lfr.63.1710447893822; Thu, 14 Mar 2024 13:24:53 -0700 (PDT) Received: from google.com (64.227.90.34.bc.googleusercontent.com. [34.90.227.64]) by smtp.gmail.com with ESMTPSA id jy5-20020a170907762500b00a4675490095sm746999ejc.42.2024.03.14.13.24.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Mar 2024 13:24:53 -0700 (PDT) Date: Thu, 14 Mar 2024 20:24:49 +0000 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Marc Zyngier , David Brazdil Subject: [PATCH 06/10] KVM: arm64: nVHE: gen-hyprel: Skip R_AARCH64_ABS32 Message-ID: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Ignore R_AARCH64_ABS32 relocations, instead of panicking, when emitting the relocation table of the hypervisor. The toolchain might produce them when generating function calls with KCFI, to allow type ID resolution across compilation units (between the call-site check and the callee's prefixed u32) at link time. They are therefore not needed in the final (runtime) relocation table. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/hyp/nvhe/gen-hyprel.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c b/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c index 6bc88a756cb7..7b046d97b301 100644 --- a/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c +++ b/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c @@ -50,6 +50,9 @@ #ifndef R_AARCH64_ABS64 #define R_AARCH64_ABS64 257 #endif +#ifndef R_AARCH64_ABS32 +#define R_AARCH64_ABS32 258 +#endif #ifndef R_AARCH64_PREL64 #define R_AARCH64_PREL64 260 #endif @@ -383,6 +386,9 @@ static void emit_rela_section(Elf64_Shdr *sh_rela) case R_AARCH64_ABS64: emit_rela_abs64(rela, sh_orig_name); break; + /* Allow 32-bit absolute relocation, for KCFI type hashes. */ + case R_AARCH64_ABS32: + break; /* Allow position-relative data relocations. */ case R_AARCH64_PREL64: case R_AARCH64_PREL32: From patchwork Thu Mar 14 20:25:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13592740 Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C00CE7317E for ; Thu, 14 Mar 2024 20:25:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447920; cv=none; b=c8uifNToz28XnPzT+BX96RD3hVsxOIRedzHo8AglxSAkhZDV8cr6Js8Umk95zrslFpj9uRGXC0gx9G6GU5HpymIlH6l9Jry2iR7U9C55sK2OITLxC7Tb7vtfNHvLLP7iKMjfnrAZvMgmkNVt408J0GZI0rVI+eezDX/lh8bcm/U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447920; c=relaxed/simple; bh=cyQ2uR1WtWqFRxYPGfl+1TVBx2dUPQERmNqiM8XpAQQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=dwq989IQAuKjy9fJ/F1gyTge2+b90alQxWZOO1e/jBEkNoKuxJQ37oakAL0fSpPM6Pe83ojnPaGcmnfmOFWxTKk1ALacerocV6AhKXXcPjr9DDfhl6xIy8kRk27rJOBPBehrRPa4Rq7tIHdk3aVbSnazm7ExUz6+J6jTOICKve8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lT0KS1Te; arc=none smtp.client-ip=209.85.218.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lT0KS1Te" Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-a46805cd977so30141866b.0 for ; Thu, 14 Mar 2024 13:25:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710447917; x=1711052717; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=pRLwYMgl/UDUGVtTi+ey3Tg5z5Sm2xHrvmETTv6ml0c=; b=lT0KS1Tew6aZcV0+b8RRWzM8NOTkXqfsDsqVsW1k+Yssg7RiL6ISTULPlQgAkukokJ u7WFWzHo4GUNHzXJNvMPK4yn46yU+yQO3dyPer6vIub+k6vBzo0lKHhhIz6BCOBzDHXk 4bLMg9t0hFISNSCYw7q9g9jzKMaflrk0/3+Y0QV6QbLO0NozQR0auzTmsXBMgDOwlL7R nCXFn0BpXqfJYEPrEGQYvhWgoaLptxGqXAbuvkoxFKe1oJUcZjrCUUmSCFLvAgfu6ZVh sp4txkBHCGjaexB9iLBxV5UzegORcJgNLSYoS6Ng4y2LUCmiKMMdKT4S7Ir1qCsvYBUG CcfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710447917; x=1711052717; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=pRLwYMgl/UDUGVtTi+ey3Tg5z5Sm2xHrvmETTv6ml0c=; b=UuYB9gn4O02FhAxixD1YC6iWdMt8hAeIFvMzeiM4z9gjKzcdghS/w1EczmoBHlgbjU ps+wu0TTJUFt2ZY3bnvOHj1KJNkRtAkep9KWTQqtxvzKxxgSHKbk9P2vxdDD8oDjlRaI Ny3I793vZjra2Pg1Vi4YO3z3DcsQMxW2wiGeBsFMokIou/nSQ2evBvGV3XbM/Ash58Ex 2gBFCtwp7URx9Uq2f6zvfIVoU00HLfA8mxj6HUDKHrivG92KQ/yPu1zxbeT6UxvQFNfw 7a5P9mPormMYq3HWQddW3OxyGySaqbHR6BtmjcQat7oh6N+QfWp4GsD7c54KbBnuAR0w zs/w== X-Forwarded-Encrypted: i=1; AJvYcCX1rJYXG0eUvVhvDyYgzxA7XbmGe7Mmu9uWLmd1njrCzBehw/fTwv4pRYiTH1aDrfIawTqL+VD6xiydewYKw8xVci5x X-Gm-Message-State: AOJu0YxEefVI+6HzDJIUq6GVPGpHMJd8QU2gP1asrjuXXjwRW4vPy0H5 RfsYn1lCoLRGDoIh6x6XKvc/B1fz/vTgK+xjOQYSpRF8hkDg0cSn83gewxJfJA== X-Google-Smtp-Source: AGHT+IHiMpX8oq69CsmwoWeCDboIKyc9/3ET5RDJwETpARBSad/uPBpvw4EMATrK/2Ae9kiiUviJAw== X-Received: by 2002:a17:906:260e:b0:a46:13a9:b7af with SMTP id h14-20020a170906260e00b00a4613a9b7afmr83924ejc.47.1710447917073; Thu, 14 Mar 2024 13:25:17 -0700 (PDT) Received: from google.com (64.227.90.34.bc.googleusercontent.com. [34.90.227.64]) by smtp.gmail.com with ESMTPSA id l5-20020a1709066b8500b00a4320e22b31sm1025367ejr.19.2024.03.14.13.25.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Mar 2024 13:25:16 -0700 (PDT) Date: Thu, 14 Mar 2024 20:25:12 +0000 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Marc Zyngier Subject: [PATCH 07/10] KVM: arm64: VHE: Mark __hyp_call_panic __noreturn Message-ID: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Given that the sole purpose of __hyp_call_panic() is to call panic(), a __noreturn function, give it the __noreturn attribute, removing the need for its caller to use unreachable(). Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/hyp/vhe/switch.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 1581df6aec87..9db04a286398 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -301,7 +301,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) return ret; } -static void __hyp_call_panic(u64 spsr, u64 elr, u64 par) +static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) { struct kvm_cpu_context *host_ctxt; struct kvm_vcpu *vcpu; @@ -326,7 +326,6 @@ void __noreturn hyp_panic(void) u64 par = read_sysreg_par(); __hyp_call_panic(spsr, elr, par); - unreachable(); } asmlinkage void kvm_unexpected_el2_exception(void) From patchwork Thu Mar 14 20:25:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13592741 Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1DC647317E for ; Thu, 14 Mar 2024 20:25:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447936; cv=none; b=e8ekpp3RAWMlP7mP/pwct4jfRfxDAQBnmbGc6xJXJjKf/l3Nlo+KOnNiYvs3JeEfI5yer3GmmFaxoLKHwLqAALbGsib2pEiWD9GMIFUyugNVUS+oAglqQSMZ0sexONjZa9YxQtB1zZQBNn6zuZ7IZaRiz+txyccxh+l+mCetdjc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447936; c=relaxed/simple; bh=lkeAZpZwnGU6VpgChdZkLs5vz+1Qca88xONJCmcdPt0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=UiwccHXpWyZOwCaLI/p/zkmP3kcaMzpIY4FwxCQbPkXyUT5GMizeM6AA4dtlEvWXmy0qi8Nrv5OLAmdl03yudzkvw1pU4Bu9NZxKm7Ox81aTN7nRfp1qug+AiY1jY9dF9dLcjm9QyLFqgdsc80l2TJzAWoedR+Ge1vxb7QUn+8U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=JFqPui9A; arc=none smtp.client-ip=209.85.218.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="JFqPui9A" Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-a45f257b81fso175900266b.0 for ; Thu, 14 Mar 2024 13:25:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710447933; x=1711052733; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=NdkPYJlECk/ZyZUB9HYwKBtygG9TYfy3Je51QsbOPR4=; b=JFqPui9AM7fDE+9U0Lg+U60IfLIN51+kob/F5QFM3bfmoRVLwiuJehtJMNALQDk9lP DNIpoFWlGH3iJ2fozigg4KfUpa4bwm213SBx8MbFQfjDPyGEztH9JZb0SKgstyqNpY0L OrOb2egGgSWEQTKn5D1nmKU2BC7X3ph7Yk+gtP4Ti0Pr91Lhuw/gJ5r3zXdF/jqfunxe 5Qfb7BI88g4YtjxpPuk1MRc5dRrUMlfrGusllQF1NnYwbe4V11k51GLD2WqXR4YbIxsE 4Yfv+hwdhz3Jn3MI6TgkzrSFZTv3mvesI+g1QvdZXIFoc5lQV9yo8+H91bT2OcP5oltG EAQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710447933; x=1711052733; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NdkPYJlECk/ZyZUB9HYwKBtygG9TYfy3Je51QsbOPR4=; b=r2g+o8nCAm2+nD+LFJHwsF8cz3MaxPOjcqK83pxkmOWerHWSAYKI2HZAWBCvdtq3lP 2bNe7voK8Zj2VrgrrjGZwEryVjpE+XBybon8+G3lKT1jbGRzfqy8A/VAiNApWjiErJow GM9PJbKNkellW/N8mPuedOcBmUiZVfzavGA25+KGH659HD5z8uv2Z3Py9NFDcOUa9GnD EY/LYMuTxHufsMIqTr1PT9WFeyLlekC31KhwRS/5dlREU3dvbjLTlgu48Y40Fr8lQwj0 fDzv7/9XmXr0z1toAdNQbfP7nSl9VIGNvP+YHzAPTCgZg8DT44OfXA3JBjaLJfIjCsEE ImgQ== X-Forwarded-Encrypted: i=1; AJvYcCVE6Tb1cVmqWjbemq8yIiZviFFVboSt0lh43yS4bnr8qLlbNJdY2kMJKyhq014GaKy4WE4w2EJGpARTX02kl6pJSShi X-Gm-Message-State: AOJu0YzIlqWmvvRwiMo+TJusV1OYevsy87T+d9iOfS6DJYXbfxhzYIgM YwhT6wOjLshKd2G24GsAMh8jnA0dO6Pjh1wK0sP1cNPlTfn3hjDwgFzeZGA8kg== X-Google-Smtp-Source: AGHT+IEAguWOtI6oTSg0qIM4uQ4GgBItzXtzQUetPHeB42cik8WVSDP20zLH80zcpirES87Iw6GvQg== X-Received: by 2002:a17:906:d798:b0:a46:7cb1:51da with SMTP id pj24-20020a170906d79800b00a467cb151damr653684ejb.52.1710447933392; Thu, 14 Mar 2024 13:25:33 -0700 (PDT) Received: from google.com (64.227.90.34.bc.googleusercontent.com. [34.90.227.64]) by smtp.gmail.com with ESMTPSA id l15-20020a1709065a8f00b00a3d2d81daafsm1024436ejq.172.2024.03.14.13.25.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Mar 2024 13:25:33 -0700 (PDT) Date: Thu, 14 Mar 2024 20:25:29 +0000 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Marc Zyngier , Catalin Marinas , Will Deacon Subject: [PATCH 08/10] arm64: Move esr_comment() to Message-ID: <6374e3f9d15663e0ea55fa4261ac42f3348ad809.1710446682.git.ptosi@google.com> References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: As it is already defined twice and is about to be needed for CFI error detection, move esr_comment() to a header so that it can be reused. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/esr.h | 5 +++++ arch/arm64/kernel/debug-monitors.c | 4 +--- arch/arm64/kernel/traps.c | 2 -- arch/arm64/kvm/handle_exit.c | 2 +- 4 files changed, 7 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index 353fe08546cf..b0c23e7d6595 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -385,6 +385,11 @@ #ifndef __ASSEMBLY__ #include +static inline unsigned long esr_comment(unsigned long esr) +{ + return esr & ESR_ELx_BRK64_ISS_COMMENT_MASK; +} + static inline bool esr_is_data_abort(unsigned long esr) { const unsigned long ec = ESR_ELx_EC(esr); diff --git a/arch/arm64/kernel/debug-monitors.c b/arch/arm64/kernel/debug-monitors.c index 64f2ecbdfe5c..647134ffa9b9 100644 --- a/arch/arm64/kernel/debug-monitors.c +++ b/arch/arm64/kernel/debug-monitors.c @@ -312,9 +312,7 @@ static int call_break_hook(struct pt_regs *regs, unsigned long esr) * entirely not preemptible, and we can use rcu list safely here. */ list_for_each_entry_rcu(hook, list, node) { - unsigned long comment = esr & ESR_ELx_BRK64_ISS_COMMENT_MASK; - - if ((comment & ~hook->mask) == hook->imm) + if ((esr_comment(esr) & ~hook->mask) == hook->imm) fn = hook->fn; } diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 215e6d7f2df8..56317ca48519 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -1105,8 +1105,6 @@ static struct break_hook ubsan_break_hook = { }; #endif -#define esr_comment(esr) ((esr) & ESR_ELx_BRK64_ISS_COMMENT_MASK) - /* * Initial handler for AArch64 BRK exceptions * This handler only used until debug_traps_init(). diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 617ae6dea5d5..ffa67ac6656c 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -395,7 +395,7 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, if (mode != PSR_MODE_EL2t && mode != PSR_MODE_EL2h) { kvm_err("Invalid host exception to nVHE hyp!\n"); } else if (ESR_ELx_EC(esr) == ESR_ELx_EC_BRK64 && - (esr & ESR_ELx_BRK64_ISS_COMMENT_MASK) == BUG_BRK_IMM) { + esr_comment(esr) == BUG_BRK_IMM) { const char *file = NULL; unsigned int line = 0; From patchwork Thu Mar 14 20:25:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13592742 Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D0A07605D for ; Thu, 14 Mar 2024 20:25:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447951; cv=none; b=Z04MmJorvcqkU5ELqrSgQwmiC/qe/TZABnR+zj1XgQI1XUFGTAelMI99VsRuh5ZdADINPNJ3K0g88f7yKjjdvjRb0zwL6iniajd9QOUySMWXeXcjes/h/6slCf6pK0sZ2vM/VHdxB3RvBa5nmNmOcntYpqukY9ljg3j6dFvnr6s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447951; c=relaxed/simple; bh=t+Xjs0gyzs0q7A934MpYYmsAbai7ZXro++u99AU3aL4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=RheAq8UdEWKZxe4fjyNPQySoN1Hnp04hxxf//IyLaM3hrLyH2OR2GHyHBfHza+1tuX73jVds8jkcYfhED4dXgHGsPfpBj0qHyGS9h9z5h+pKK36qJRX9KU+WOLY219P3rt1/Bsg6mORPSmq5FEeuNgz1zCXFRzcHXhgHMCA8Cwk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=KE3zsUxb; arc=none smtp.client-ip=209.85.218.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="KE3zsUxb" Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-a466a256726so246679866b.1 for ; Thu, 14 Mar 2024 13:25:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710447948; x=1711052748; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=uw5V0Ihfo7A0TVnpjJhXgB0vBa49XYRYilcTAeCdiBg=; b=KE3zsUxblMtO7xe/B65ihVXw8TjllMZnWTsu3OBy/fKCxoMEalWLaazmqdbdhClZ/R EJQgtxu0809DPzeBRCL+aBUEMq/f+FdgSedGGdPeX0qqxlnRpMkNweiBMfn1nA0fCvp2 +CpPXvOMMwI/HWtSF29AJxbDtbPwGEAUQ5bzQLF58AhhkbP6TLU340AUzo0WXFBpAglI lJ+5uJdY0SAn/R3ao+2FmlQ8VTxEgqwnsMm7+0OkP+ye3+Oc0I2zrz4iF2oasfJeK3XW 1euNYeKtp/AE5OIhg0I1TfWXHZDuQe6PI8080XfzkTtQ8ypMwgjBi0EHSevZIg8T9eKY BpBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710447948; x=1711052748; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uw5V0Ihfo7A0TVnpjJhXgB0vBa49XYRYilcTAeCdiBg=; b=WxkuZ1ZZ7zEm73vL/wPg6XK2bCqAXbJ5IIKMAjy653/P012uL/pTdw2P0SywV9qgaS 3S2gLN4F3O01snFa62mXyPAXshYjOctxWYqhj0qJQfV5dDOJOe4xJxu6UXFlz4ugs4m8 L3EokZ6tAsKbfmclweNo8NMSWteOo3G7TEenefA0TGUGh2GMG0xAxt81JHluYg+axbJw JXW2zUt1MbCUEMk+U3p/KbursxYXvpl8qiVtvBQj2PF2U7mEHy/UUUKQD/t4c43yqWMZ ek7FeNAA3gqN0WMk+9b9mMymtIxKZqxoU2wwELC/W4pcEJXCyvSWnTWY6uwaduoO9gkJ q7WA== X-Forwarded-Encrypted: i=1; AJvYcCVcHM5fbQYg8dKg9ieo/p37ORPOOBrBLwh//Hv0ODasittB6p+yf8mwRs10C/DKUmGL78+AsjnK6OvSOTWidzPXYEbV X-Gm-Message-State: AOJu0YxBxnLkfrKyRnlxkutIudg/febdx4zINMfc6WXGR0Wxgh/DquuL SkwGAN8nVfPCIhC2nxINA/yX1CTBM20Sf4OvxAqWY1awgJfjpXD6Ta918mTomA== X-Google-Smtp-Source: AGHT+IFSNYqhVQ94RVZXOtYFwx8fhGK4ErioR53gKmAPavaCWzXWnVTwxghHAngotsX6fUmwQ3r8tQ== X-Received: by 2002:a17:906:c359:b0:a44:806f:ad56 with SMTP id ci25-20020a170906c35900b00a44806fad56mr2381102ejb.11.1710447947559; Thu, 14 Mar 2024 13:25:47 -0700 (PDT) Received: from google.com (64.227.90.34.bc.googleusercontent.com. [34.90.227.64]) by smtp.gmail.com with ESMTPSA id l5-20020a1709066b8500b00a4320e22b31sm1025683ejr.19.2024.03.14.13.25.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Mar 2024 13:25:47 -0700 (PDT) Date: Thu, 14 Mar 2024 20:25:43 +0000 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Marc Zyngier , Will Deacon , Quentin Perret , Vincent Donnefort Subject: [PATCH 09/10] KVM: arm64: nVHE: Support CONFIG_CFI_CLANG at EL2 Message-ID: <87885c41627a033d9772dd368049e7f8f5fd4ef7.1710446682.git.ptosi@google.com> References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The compiler implements KCFI by adding type information (u32) above every function that might be indirectly called and, whenever a function pointer is called, injects a read-and-compare of that u32 against the value corresponding to the expected type. In case of a mismatch, a BRK instruction gets executed. When the hypervisor triggers such an exception, it panics. Therefore, teach hyp_panic() to detect KCFI errors from the ESR and report them. If necessary, remind the user that CONFIG_CFI_PERMISSIVE doesn't affect EL2 KCFI. Pass $(CC_FLAGS_CFI) to the compiler when building the nVHE hyp code. Use SYM_TYPED_FUNC_START() for __pkvm_init_switch_pgd, as nVHE can't call it directly and must use a PA function pointer from C (because it is part of the idmap page), which would trigger a KCFI failure if the type ID wasn't present. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/esr.h | 6 ++++++ arch/arm64/kvm/handle_exit.c | 11 +++++++++++ arch/arm64/kvm/hyp/nvhe/Makefile | 6 +++--- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 3 ++- 4 files changed, 22 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index b0c23e7d6595..281e352a4c94 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -397,6 +397,12 @@ static inline bool esr_is_data_abort(unsigned long esr) return ec == ESR_ELx_EC_DABT_LOW || ec == ESR_ELx_EC_DABT_CUR; } +static inline bool esr_is_cfi_brk(unsigned long esr) +{ + return ESR_ELx_EC(esr) == ESR_ELx_EC_BRK64 && + (esr_comment(esr) & ~CFI_BRK_IMM_MASK) == CFI_BRK_IMM_BASE; +} + static inline bool esr_fsc_is_translation_fault(unsigned long esr) { return (esr & ESR_ELx_FSC_TYPE) == ESR_ELx_FSC_FAULT; diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index ffa67ac6656c..9b6574e50b13 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -383,6 +383,15 @@ void handle_exit_early(struct kvm_vcpu *vcpu, int exception_index) kvm_handle_guest_serror(vcpu, kvm_vcpu_get_esr(vcpu)); } +static void kvm_nvhe_report_cfi_failure(u64 panic_addr) +{ + kvm_err("nVHE hyp CFI failure at: [<%016llx>] %pB!\n", panic_addr, + (void *)(panic_addr + kaslr_offset())); + + if (IS_ENABLED(CONFIG_CFI_PERMISSIVE)) + kvm_err(" (CONFIG_CFI_PERMISSIVE ignored for hyp failures)\n"); +} + void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, u64 elr_virt, u64 elr_phys, u64 par, uintptr_t vcpu, @@ -413,6 +422,8 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, else kvm_err("nVHE hyp BUG at: [<%016llx>] %pB!\n", panic_addr, (void *)(panic_addr + kaslr_offset())); + } else if (IS_ENABLED(CONFIG_CFI_CLANG) && esr_is_cfi_brk(esr)) { + kvm_nvhe_report_cfi_failure(panic_addr); } else { kvm_err("nVHE hyp panic at: [<%016llx>] %pB!\n", panic_addr, (void *)(panic_addr + kaslr_offset())); diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 2250253a6429..2eb915d8943f 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -89,9 +89,9 @@ quiet_cmd_hyprel = HYPREL $@ quiet_cmd_hypcopy = HYPCOPY $@ cmd_hypcopy = $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ $< $@ -# Remove ftrace, Shadow Call Stack, and CFI CFLAGS. -# This is equivalent to the 'notrace', '__noscs', and '__nocfi' annotations. -KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI), $(KBUILD_CFLAGS)) +# Remove ftrace and Shadow Call Stack CFLAGS. +# This is equivalent to the 'notrace' and '__noscs' annotations. +KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS), $(KBUILD_CFLAGS)) # Starting from 13.0.0 llvm emits SHT_REL section '.llvm.call-graph-profile' # when profile optimization is applied. gen-hyprel does not support SHT_REL and # causes a build failure. Remove profile optimization flags. diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 8958dd761837..ade73fdfaad9 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -5,6 +5,7 @@ */ #include +#include #include #include @@ -265,7 +266,7 @@ alternative_else_nop_endif SYM_CODE_END(__kvm_handle_stub_hvc) -SYM_FUNC_START(__pkvm_init_switch_pgd) +SYM_TYPED_FUNC_START(__pkvm_init_switch_pgd) /* Load the inputs from the VA pointer before turning the MMU off */ ldr x5, [x0, #NVHE_INIT_PGD_PA] ldr x0, [x0, #NVHE_INIT_STACK_HYP_VA] From patchwork Thu Mar 14 20:26:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13592743 Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com [209.85.167.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A36CC745C2 for ; Thu, 14 Mar 2024 20:26:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447969; cv=none; b=jnPUroy5B7tozGRMMkCaCmqvILIAWZ40n0TIPVhUxECveeo85RkZEJqGkHU12JJFF1mTgStHSjyAn/oV8M3ns9Pk/+Fgoa8pmK1qE+TMDu3U6ooRw4N2maKieG42PBon5uiM26vH5BQYMH7APWe6+QY4BhvPVFqje6VcJIUMBzI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710447969; c=relaxed/simple; bh=lVUq8Tjf9oK6tIzQo5+3/KLAKzgTNPmjYvQfI7c4rMg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=dF/ARLeodfUihowVrd2wXLeK5sN7lM83SqmKvRXabO3r96NmkGu/LR9ZCUpi7CwCLQ4XxbPeECtdOTmQDmrkUZeLohdi1ZDLG4RqY58P+JbllbQ0wzBFfJoOmdMvm41TtxZvMQ5dTqVzA7oVoNN1b08QfAFRVk48BiMagjkLn/0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=keRFmsk5; arc=none smtp.client-ip=209.85.167.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="keRFmsk5" Received: by mail-lf1-f50.google.com with SMTP id 2adb3069b0e04-51323dfce59so1359478e87.3 for ; Thu, 14 Mar 2024 13:26:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710447966; x=1711052766; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=1X42YuqTctT4vzgW20/hDSVUJpY4koOC9rkGtJkqjtA=; b=keRFmsk5jchuksyIny0fCVVWHFBp/YsEnxjhrCx2lo+gftfhRL0bZ0DINCdTf3eqY9 tbNt/RFBX2JzFlKH6ZsVgI0pz6at2GgSgTy2OJwxnfHb0Lp+uSiAIKfmrpFe4JHaKkOQ sA9gi9xMNuNfE5qL2J7TDmIYIDikFthkfMUFgL90Jmpm+/2rsA94ZMHaNnKjSOZZI38s 8v8dHVmPZzNT5XH0jQQ+NMP0/y1k5VAmURxsKat88IFe/r0f7kli28nwd6Ziwpab7gWB EpNT8heRQrHPZvZ42Rz1LpBugx7pkhixDMYW0Y/5NFAodUuXVfPV/5tY0cmhGE6jE9Jn V1+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710447966; x=1711052766; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1X42YuqTctT4vzgW20/hDSVUJpY4koOC9rkGtJkqjtA=; b=mYipLO7HXBLQ7T/g/jax3Y6Z3YmxZ+TKbZE6Su06my8nv/RkoChzU1OhG0qw1OlPof oWeEnJuE5XmJ+uLlv19q4uzRxnP/BAMpZs+XCj4lagh7iNYK9vIiHRK/D+WcXVMKWMAr K3Q4y3LyMONZkxzOeDqr3Dvt9LijOh+s7CCy+lq51c+kQgDp810B6scyZrjC6LC4JCx6 gF8gKw8egvvhfpFXTA9A8SOuYEz3jDOaXfLaQGrSrCN2zsBLZa4ZsMRpWHNjUBpN1I5+ vKhEmmVj3PNFhKofm1O5hwZkSEVXdTVHK4jcCpBHCUQGkcQOc6wHMJOHq2qJdbZE56c1 zHmA== X-Forwarded-Encrypted: i=1; AJvYcCWDJSWlE2jbxZoW3S6wl1kyl+cHRSLgvLWFYT++ZdUKhGyjWYShSRVlaGtKLBU3oHLUaLIOVfnJo/hxonxu+k4IH7CE X-Gm-Message-State: AOJu0YwesHXuK8TpDTs+6E+OoimTg8oQ25HazPtnjafXqzdT6hkfGeiO gAFbPmNZhWNHJ8OJN+KV+aZq+PgJ32zqiq5hBHNyY+2mqZXaRWNqYjHOrv3XqxkBwKqOMilteV9 FTGrM X-Google-Smtp-Source: AGHT+IHkzNmsE0KL1mSwwtaDDxJ2Z0EOfjOFXiDM9tdY0BYN+4IxOoQ2ahyHAy73nBFdMUHgJ8ASJQ== X-Received: by 2002:a19:e054:0:b0:512:a4ce:abaa with SMTP id g20-20020a19e054000000b00512a4ceabaamr900363lfj.48.1710447965560; Thu, 14 Mar 2024 13:26:05 -0700 (PDT) Received: from google.com (64.227.90.34.bc.googleusercontent.com. [34.90.227.64]) by smtp.gmail.com with ESMTPSA id s11-20020a170906454b00b00a44f07e55d2sm1005673ejq.41.2024.03.14.13.26.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Mar 2024 13:26:05 -0700 (PDT) Date: Thu, 14 Mar 2024 20:26:01 +0000 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Marc Zyngier , Vincent Donnefort Subject: [PATCH 10/10] KVM: arm64: Improve CONFIG_CFI_CLANG error message Message-ID: <6efd86cefda82a35b1e002a2fb48811e3b1960db.1710446682.git.ptosi@google.com> References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: For KCFI, the compiler encodes in the immediate of the BRK (which the CPU places in ESR_ELx) the indices of the two registers it used to hold (resp.) the function pointer and expected type. Therefore, the CFI handler must be able to parse the contents of the register file at the point where the exception was triggered. To achieve this, introduce a new hypervisor panic path that first stores the CPU context in the per-CPU kvm_hyp_ctxt before calling (directly or indirectly) hyp_panic() and execute it from all EL2 synchronous exception handlers i.e. - call it directly in host_el2_sync_vect (__kvm_hyp_host_vector, EL2t&h) - call it directly in el2t_sync_invalid (__kvm_hyp_vector, EL2t) - set ELR_EL2 to it in el2_sync (__kvm_hyp_vector, EL2h), which ERETs Teach hyp_panic() to decode the KCFI ESR and extract the target and type from the saved CPU context. In VHE, use that information to panic() with a specialized error message. In nVHE, only report it if the host (EL1) has access to the saved CPU context i.e. iff CONFIG_NVHE_EL2_DEBUG=y, which aligns with the behavior of CONFIG_PROTECTED_NVHE_STACKTRACE. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/handle_exit.c | 30 +++++++++++++++++++++++-- arch/arm64/kvm/hyp/entry.S | 24 +++++++++++++++++++- arch/arm64/kvm/hyp/hyp-entry.S | 2 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/host.S | 2 +- arch/arm64/kvm/hyp/vhe/switch.c | 26 +++++++++++++++++++-- 6 files changed, 79 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 9b6574e50b13..d343a5130943 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -26,6 +26,8 @@ #define CREATE_TRACE_POINTS #include "trace_handle_exit.h" +DECLARE_KVM_NVHE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt); + typedef int (*exit_handle_fn)(struct kvm_vcpu *); static void kvm_handle_guest_serror(struct kvm_vcpu *vcpu, u64 esr) @@ -383,11 +385,35 @@ void handle_exit_early(struct kvm_vcpu *vcpu, int exception_index) kvm_handle_guest_serror(vcpu, kvm_vcpu_get_esr(vcpu)); } -static void kvm_nvhe_report_cfi_failure(u64 panic_addr) +static void kvm_nvhe_report_cfi_target(struct user_pt_regs *regs, u64 esr, + u64 hyp_offset) +{ + u64 va_mask = GENMASK_ULL(vabits_actual - 1, 0); + u8 type_idx = FIELD_GET(CFI_BRK_IMM_TYPE, esr); + u8 target_idx = FIELD_GET(CFI_BRK_IMM_TARGET, esr); + u32 expected_type = (u32)regs->regs[type_idx]; + u64 target_addr = (regs->regs[target_idx] & va_mask) + hyp_offset; + + kvm_err(" (target: [<%016llx>] %ps, expected type: 0x%08x)\n", + target_addr, (void *)(target_addr + kaslr_offset()), + expected_type); +} + +static void kvm_nvhe_report_cfi_failure(u64 panic_addr, u64 esr, u64 hyp_offset) { + struct user_pt_regs *regs = NULL; + kvm_err("nVHE hyp CFI failure at: [<%016llx>] %pB!\n", panic_addr, (void *)(panic_addr + kaslr_offset())); + if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG) || !is_protected_kvm_enabled()) + regs = &this_cpu_ptr_nvhe_sym(kvm_hyp_ctxt)->regs; + + if (regs) + kvm_nvhe_report_cfi_target(regs, esr, hyp_offset); + else + kvm_err(" (no target information: !CONFIG_NVHE_EL2_DEBUG)\n"); + if (IS_ENABLED(CONFIG_CFI_PERMISSIVE)) kvm_err(" (CONFIG_CFI_PERMISSIVE ignored for hyp failures)\n"); } @@ -423,7 +449,7 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, kvm_err("nVHE hyp BUG at: [<%016llx>] %pB!\n", panic_addr, (void *)(panic_addr + kaslr_offset())); } else if (IS_ENABLED(CONFIG_CFI_CLANG) && esr_is_cfi_brk(esr)) { - kvm_nvhe_report_cfi_failure(panic_addr); + kvm_nvhe_report_cfi_failure(panic_addr, esr, hyp_offset); } else { kvm_err("nVHE hyp panic at: [<%016llx>] %pB!\n", panic_addr, (void *)(panic_addr + kaslr_offset())); diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index ac8aa8571b2f..eb6699d2bb7a 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,7 +83,7 @@ alternative_else_nop_endif eret sb -SYM_INNER_LABEL(__hyp_panic_with_restored_elr, SYM_L_GLOBAL) +SYM_INNER_LABEL(__hyp_panic_with_context_and_restored_elr, SYM_L_GLOBAL) // x0-x29,lr: hyp regs stp x0, x1, [sp, #-16]! @@ -92,6 +92,28 @@ SYM_INNER_LABEL(__hyp_panic_with_restored_elr, SYM_L_GLOBAL) msr elr_el2, x0 ldp x0, x1, [sp], #16 +SYM_INNER_LABEL(__hyp_panic_with_context, SYM_L_GLOBAL) + // x0-x29,lr: hyp regs + + stp x0, x1, [sp, #-16]! + + adr_this_cpu x0, kvm_hyp_ctxt, x1 + + stp x2, x3, [x0, #CPU_XREG_OFFSET(2)] + + ldp x2, x3, [sp], #16 + + stp x2, x3, [x0, #CPU_XREG_OFFSET(0)] + stp x4, x5, [x0, #CPU_XREG_OFFSET(4)] + stp x6, x7, [x0, #CPU_XREG_OFFSET(6)] + stp x8, x9, [x0, #CPU_XREG_OFFSET(8)] + stp x10, x11, [x0, #CPU_XREG_OFFSET(10)] + stp x12, x13, [x0, #CPU_XREG_OFFSET(12)] + stp x14, x15, [x0, #CPU_XREG_OFFSET(14)] + stp x16, x17, [x0, #CPU_XREG_OFFSET(16)] + + save_callee_saved_regs x0 + SYM_INNER_LABEL(__hyp_panic, SYM_L_GLOBAL) // x0-x29,lr: vcpu regs diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 7e65ef738ec9..6eedab7f9767 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -130,7 +130,7 @@ SYM_CODE_END(\label) .endm /* None of these should ever happen */ - invalid_vector el2t_sync_invalid + invalid_vector el2t_sync_invalid, __hyp_panic_with_context invalid_vector el2t_irq_invalid invalid_vector el2t_fiq_invalid invalid_vector el2t_error_invalid diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 0dc721ced358..6c4b3f9d538f 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -747,7 +747,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __kvm_unexpected_el2_exception(void) { - extern char __hyp_panic_with_restored_elr[]; + extern char __hyp_panic_with_context_and_restored_elr[]; unsigned long addr, fixup; struct kvm_exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); @@ -769,7 +769,7 @@ static inline void __kvm_unexpected_el2_exception(void) } /* Trigger a panic after restoring the hyp context. */ - write_sysreg(__hyp_panic_with_restored_elr, elr_el2); + write_sysreg(__hyp_panic_with_context_and_restored_elr, elr_el2); this_cpu_ptr(&kvm_hyp_ctxt)->sys_regs[ELR_EL2] = elr_el2; } diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 1b9111c2b480..8bb6fed5ba4e 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -202,7 +202,7 @@ SYM_FUNC_END(__host_hvc) * context has been saved by __host_exit or after the hyp context has * been partially clobbered by __host_enter. */ - b __hyp_panic + b __hyp_panic_with_context .L__hyp_sp_overflow\@: /* Switch to the overflow stack */ diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 9db04a286398..c733f5bdab59 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -17,6 +17,7 @@ #include #include +#include #include #include #include @@ -301,7 +302,24 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) return ret; } -static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) +static void __noreturn __hyp_call_panic_for_cfi(u64 elr, u64 esr) +{ + struct user_pt_regs *regs = &this_cpu_ptr(&kvm_hyp_ctxt)->regs; + u8 type_idx = FIELD_GET(CFI_BRK_IMM_TYPE, esr); + u8 target_idx = FIELD_GET(CFI_BRK_IMM_TARGET, esr); + u32 expected_type = (u32)regs->regs[type_idx]; + u64 target = regs->regs[target_idx]; + + panic("VHE hyp CFI failure at: [<%016llx>] %pB (target: [<%016llx>] %ps, expected type: 0x%08x)\n" +#ifdef CONFIG_CFI_PERMISSIVE + " (CONFIG_CFI_PERMISSIVE ignored for hyp failures)\n" +#endif + , + elr, (void *)elr, target, (void *)target, expected_type); +} +NOKPROBE_SYMBOL(__hyp_call_panic_for_cfi); + +static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par, u64 esr) { struct kvm_cpu_context *host_ctxt; struct kvm_vcpu *vcpu; @@ -312,6 +330,9 @@ static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) __deactivate_traps(vcpu); sysreg_restore_host_state_vhe(host_ctxt); + if (IS_ENABLED(CONFIG_CFI_CLANG) && esr_is_cfi_brk(esr)) + __hyp_call_panic_for_cfi(elr, esr); + panic("HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n", spsr, elr, read_sysreg_el2(SYS_ESR), read_sysreg_el2(SYS_FAR), @@ -324,8 +345,9 @@ void __noreturn hyp_panic(void) u64 spsr = read_sysreg_el2(SYS_SPSR); u64 elr = read_sysreg_el2(SYS_ELR); u64 par = read_sysreg_par(); + u64 esr = read_sysreg_el2(SYS_ESR); - __hyp_call_panic(spsr, elr, par); + __hyp_call_panic(spsr, elr, par, esr); } asmlinkage void kvm_unexpected_el2_exception(void)