From patchwork Fri May 26 14:33:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 277D6C7EE2C for ; Fri, 26 May 2023 14:35:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242855AbjEZOfx (ORCPT ); Fri, 26 May 2023 10:35:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58640 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237386AbjEZOfu (ORCPT ); Fri, 26 May 2023 10:35:50 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11BDDE5D for ; Fri, 26 May 2023 07:35:20 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 663A86505F for ; Fri, 26 May 2023 14:33:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BE7ADC4339B; Fri, 26 May 2023 14:33:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685111633; bh=6wNp/EjkCubmH4jSbimzQJEPlzEQpa08AFTUAXoa3iE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ChZ+h+1biz5GYn+nrKqEROEW40i5Jh2C2OLIBoOqR9Ht2+/jJGiO31dnOGDRO9oPP HJLb0oLrBtMy5ScWVFdsLlPD2cG1lwxr2j/MD1C54BvgG1yuFvNrbMv4oKT6C7zmNT J6gqRZUh67PNfO8ZL6sCYGEK56Va1KhTb8P7dY8VrD6rA44r8q/kgW7lkW/n7AfUMB 9Jvhb1Fy8AycoeC2bVlILM/uItqm3gBQ+IIjeRi4KQWRJywCaWBg2WCfEZTJ8J/Moo YqPq6hedRQ3U9DXxjBUFCdP5xU+uFiGICa2vn+A8I15I/os5NBCXb4aVtjVv9A31/A hFbDYZ286q9aQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVr-000aHS-K6; Fri, 26 May 2023 15:33:51 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 01/17] KVM: arm64: Drop is_kernel_in_hyp_mode() from __invalidate_icache_guest_page() Date: Fri, 26 May 2023 15:33:32 +0100 Message-Id: <20230526143348.4072074-2-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org It is pretty obvious that is_kernel_in_hyp_mode() doesn't make much sense in the hypervisor part of KVM, and should be reserved to the kernel side. However, mem_protect.c::invalidate_icache_guest_page() calls into __invalidate_icache_guest_page(), which uses is_kernel_in_hyp_mode(). Given that this is part of the pKVM side of the hypervisor, this helper can only return true. Nothing goes really bad, but __invalidate_icache_guest_page() could spell out what the actual check is: we cannot invalidate the cache if the i-cache is VPIPT and we're running at EL1. Drop the is_kernel_in_hyp_mode() check for an explicit check against CurrentEL being EL1 or not. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/kvm_mmu.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 27e63c111f78..c8113b931263 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -227,7 +227,8 @@ static inline void __invalidate_icache_guest_page(void *va, size_t size) if (icache_is_aliasing()) { /* any kind of VIPT cache */ icache_inval_all_pou(); - } else if (is_kernel_in_hyp_mode() || !icache_is_vpipt()) { + } else if (read_sysreg(CurrentEL) != CurrentEL_EL1 || + !icache_is_vpipt()) { /* PIPT or VPIPT at EL2 (see comment in __kvm_tlb_flush_vmid_ipa) */ icache_inval_pou((unsigned long)va, (unsigned long)va + size); } From patchwork Fri May 26 14:33:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257004 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37A59C77B73 for ; Fri, 26 May 2023 14:35:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237416AbjEZOf5 (ORCPT ); Fri, 26 May 2023 10:35:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58740 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230167AbjEZOfz (ORCPT ); Fri, 26 May 2023 10:35:55 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0034CE71 for ; Fri, 26 May 2023 07:35:25 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8C0B265063 for ; Fri, 26 May 2023 14:33:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E88DDC433A0; Fri, 26 May 2023 14:33:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685111634; bh=Xl35f5y+0lJ6cnk0BItkJo8xNvJzARnksbTzaqW58BI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=r+kfFMiHYsGjwJcfHCFkTO0wJg6ZQPhYa+JZGFxnzoZMzMFdWyI6yZKMuhfR+6NN8 ORgog4mBwfeUYwoV251/69MsgRgmZTmkzoD6dPIONsCpa3UN8hS3uAVnfZBxAkRebu 8VvyT8maCPStot229vCbIZf8rDXbCvFZEYV87dVAjmwEStlvyrFirdj4jGMDMt1eI8 yBzZZshEwW+1Sn3eF8AERXyoyKthwcab+IqNi5Qme02v66BM9iDEzEXMFyJJ6NeNh+ TOMzPvfDOm9bje2fuKPTTD58V6d5EMN/gl58QU7ayErPKQZRE++EsKXKsdI8SPOkeV G0kw1PPx9QlgQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVr-000aHS-Sc; Fri, 26 May 2023 15:33:51 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 02/17] arm64: Prevent the use of is_kernel_in_hyp_mode() in hypervisor code Date: Fri, 26 May 2023 15:33:33 +0100 Message-Id: <20230526143348.4072074-3-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Using is_kernel_in_hyp_mode() in hypervisor code is a pretty bad mistake. This helper only checks for CurrentEL being EL2, which is always true. Make the link fail if using the helper in hypervisor context by referencing a non-existent function. Whilst we're at it, flag the helper as __always_inline, which it really should be. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/virt.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 4eb601e7de50..91029709d133 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -110,8 +110,13 @@ static inline bool is_hyp_mode_mismatched(void) return __boot_cpu_mode[0] != __boot_cpu_mode[1]; } -static inline bool is_kernel_in_hyp_mode(void) +extern void gotcha_is_kernel_in_hyp_mode(void); + +static __always_inline bool is_kernel_in_hyp_mode(void) { +#if defined(__KVM_NVHE_HYPERVISOR__) || defined(__KVM_VHE_HYPERVISOR__) + gotcha_is_kernel_in_hyp_mode(); +#endif return read_sysreg(CurrentEL) == CurrentEL_EL2; } From patchwork Fri May 26 14:33:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257000 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7017AC77B73 for ; Fri, 26 May 2023 14:35:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230005AbjEZOfB (ORCPT ); Fri, 26 May 2023 10:35:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57848 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236409AbjEZOe7 (ORCPT ); Fri, 26 May 2023 10:34:59 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F5B210D4 for ; Fri, 26 May 2023 07:34:28 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id BF0E960DE6 for ; Fri, 26 May 2023 14:33:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 291A7C433A1; Fri, 26 May 2023 14:33:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685111634; bh=cs9s6WrxAIG0+qz4uZPPdbaqmFHbrYSMu2k9xtlONS8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZUTyRSDOCEqlMgdHRbZXBJdiXX3jUdTnZl3BC+7R7NlQ6n2387EJe6vsn3qAuEBB8 RIVh/gsl8yVYzgRiADM3IpbBMp8bs0eucG9hYPE03GGALR9rvHQ/UjftpVRRkPWcHA +bhLDYCwgE3dvpMHc3YOWHWjIRG/wA2VBUHnXivZ/W/WdPRhqDufGcrmf5KLWlYcdH 0+5fcgbRac70n/QM9D6P/OWkKf80G+nObd/8EBXLWjXWuYIQDn1Xn0yy3mzIZGVmRm g1YTlKY1cJWGgoJl3PkH5kg6YKNKym4vku+5ztVY45FAbCcO6jjcWe9v7VYBWhcQXh dI84/XqBGCDlQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVs-000aHS-3e; Fri, 26 May 2023 15:33:52 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 03/17] arm64: Turn kaslr_feature_override into a generic SW feature override Date: Fri, 26 May 2023 15:33:34 +0100 Message-Id: <20230526143348.4072074-4-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Disabling KASLR from the command line is implemented as a feature override. Repaint it slightly so that it can further be used as more generic infrastructure for SW override purposes. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/cpufeature.h | 4 ++++ arch/arm64/kernel/cpufeature.c | 2 ++ arch/arm64/kernel/idreg-override.c | 16 ++++++---------- arch/arm64/kernel/kaslr.c | 6 +++--- 4 files changed, 15 insertions(+), 13 deletions(-) diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index 6bf013fb110d..bc1009890180 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -15,6 +15,8 @@ #define MAX_CPU_FEATURES 128 #define cpu_feature(x) KERNEL_HWCAP_ ## x +#define ARM64_SW_FEATURE_OVERRIDE_NOKASLR 0 + #ifndef __ASSEMBLY__ #include @@ -925,6 +927,8 @@ extern struct arm64_ftr_override id_aa64smfr0_override; extern struct arm64_ftr_override id_aa64isar1_override; extern struct arm64_ftr_override id_aa64isar2_override; +extern struct arm64_ftr_override arm64_sw_feature_override; + u32 get_kvm_ipa_limit(void); void dump_cpu_features(void); diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 7d7128c65161..2d2b7bb5fa0c 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -664,6 +664,8 @@ struct arm64_ftr_override __ro_after_init id_aa64smfr0_override; struct arm64_ftr_override __ro_after_init id_aa64isar1_override; struct arm64_ftr_override __ro_after_init id_aa64isar2_override; +struct arm64_ftr_override arm64_sw_feature_override; + static const struct __ftr_reg_entry { u32 sys_id; struct arm64_ftr_reg *reg; diff --git a/arch/arm64/kernel/idreg-override.c b/arch/arm64/kernel/idreg-override.c index 370ab84fd06e..8c93b6198bf5 100644 --- a/arch/arm64/kernel/idreg-override.c +++ b/arch/arm64/kernel/idreg-override.c @@ -138,15 +138,11 @@ static const struct ftr_set_desc smfr0 __initconst = { }, }; -extern struct arm64_ftr_override kaslr_feature_override; - -static const struct ftr_set_desc kaslr __initconst = { - .name = "kaslr", -#ifdef CONFIG_RANDOMIZE_BASE - .override = &kaslr_feature_override, -#endif +static const struct ftr_set_desc sw_features __initconst = { + .name = "arm64_sw", + .override = &arm64_sw_feature_override, .fields = { - FIELD("disabled", 0, NULL), + FIELD("nokaslr", ARM64_SW_FEATURE_OVERRIDE_NOKASLR, NULL), {} }, }; @@ -158,7 +154,7 @@ static const struct ftr_set_desc * const regs[] __initconst = { &isar1, &isar2, &smfr0, - &kaslr, + &sw_features, }; static const struct { @@ -175,7 +171,7 @@ static const struct { "id_aa64isar1.api=0 id_aa64isar1.apa=0 " "id_aa64isar2.gpa3=0 id_aa64isar2.apa3=0" }, { "arm64.nomte", "id_aa64pfr1.mte=0" }, - { "nokaslr", "kaslr.disabled=1" }, + { "nokaslr", "arm64_sw.nokaslr=1" }, }; static int __init parse_nokaslr(char *unused) diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c index e7477f21a4c9..5d4ce7f5f157 100644 --- a/arch/arm64/kernel/kaslr.c +++ b/arch/arm64/kernel/kaslr.c @@ -23,8 +23,6 @@ u64 __ro_after_init module_alloc_base; u16 __initdata memstart_offset_seed; -struct arm64_ftr_override kaslr_feature_override __initdata; - static int __init kaslr_init(void) { u64 module_range; @@ -36,7 +34,9 @@ static int __init kaslr_init(void) */ module_alloc_base = (u64)_etext - MODULES_VSIZE; - if (kaslr_feature_override.val & kaslr_feature_override.mask & 0xf) { + if (cpuid_feature_extract_unsigned_field(arm64_sw_feature_override.val & + arm64_sw_feature_override.mask, + ARM64_SW_FEATURE_OVERRIDE_NOKASLR)) { pr_info("KASLR disabled on command line\n"); return 0; } From patchwork Fri May 26 14:33:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257002 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BEF2C7EE2C for ; Fri, 26 May 2023 14:35:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237366AbjEZOfD (ORCPT ); Fri, 26 May 2023 10:35:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57854 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230167AbjEZOfA (ORCPT ); Fri, 26 May 2023 10:35:00 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9663310D5 for ; Fri, 26 May 2023 07:34:28 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CD7D365068 for ; Fri, 26 May 2023 14:33:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 392A8C433A4; Fri, 26 May 2023 14:33:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685111634; bh=SGsj8ICM0QC50tgffybqC2QCsVSiUinMmvMKn79p3gQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZXVV2DSmC3ewutGb+m6tch6/0/KqebZxiVCgS9tz32xfawLh+mGHaPaHdISOeMbF4 O91l1xhbZC4+G6HKh6gFjzPasVwI+SH69npCZKuaciTYFBP+L4L5NIRO2S1SMYxFEM YjOj+48PDrEMXNV1rkXCbpJzrKClO5Yp2EkA4Rln5ETXQMMqnvUL040J6auxuDqd9O oh/FXGuIqkAMhDF0uQ710nPFx6vVwqntpDRdDfQ0dPPPFT+cZay2fhFOzQ/msg6zzk sz0LLm6MmZG1YeVW/Okho52GJHR4LBLFU57U3L4H7OvKC0JF2QYOENfn7wRzLJ2394 ua3Il5lUHnWBA== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVs-000aHS-CW; Fri, 26 May 2023 15:33:52 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 04/17] arm64: Add KVM_HVHE capability and has_hvhe() predicate Date: Fri, 26 May 2023 15:33:35 +0100 Message-Id: <20230526143348.4072074-5-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Expose a capability keying the hVHE feature as well as a new predicate testing it. Nothing is so far using it, and nothing is enabling it yet. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/cpufeature.h | 1 + arch/arm64/include/asm/virt.h | 8 ++++++++ arch/arm64/kernel/cpufeature.c | 15 +++++++++++++++ arch/arm64/tools/cpucaps | 1 + 4 files changed, 25 insertions(+) diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index bc1009890180..3d4b547ae312 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -16,6 +16,7 @@ #define cpu_feature(x) KERNEL_HWCAP_ ## x #define ARM64_SW_FEATURE_OVERRIDE_NOKASLR 0 +#define ARM64_SW_FEATURE_OVERRIDE_HVHE 4 #ifndef __ASSEMBLY__ diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 91029709d133..5f84a87a6a2d 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -145,6 +145,14 @@ static __always_inline bool is_protected_kvm_enabled(void) return cpus_have_final_cap(ARM64_KVM_PROTECTED_MODE); } +static __always_inline bool has_hvhe(void) +{ + if (is_vhe_hyp_code()) + return false; + + return cpus_have_final_cap(ARM64_KVM_HVHE); +} + static inline bool is_hyp_nvhe(void) { return is_hyp_mode_available() && !is_kernel_in_hyp_mode(); diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 2d2b7bb5fa0c..04ef60571b37 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1998,6 +1998,15 @@ static bool has_nested_virt_support(const struct arm64_cpu_capabilities *cap, return true; } +static bool hvhe_possible(const struct arm64_cpu_capabilities *entry, + int __unused) +{ + u64 val; + + val = arm64_sw_feature_override.val & arm64_sw_feature_override.mask; + return cpuid_feature_extract_unsigned_field(val, ARM64_SW_FEATURE_OVERRIDE_HVHE); +} + #ifdef CONFIG_ARM64_PAN static void cpu_enable_pan(const struct arm64_cpu_capabilities *__unused) { @@ -2643,6 +2652,12 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .cpu_enable = cpu_enable_dit, ARM64_CPUID_FIELDS(ID_AA64PFR0_EL1, DIT, IMP) }, + { + .desc = "VHE for hypervisor only", + .capability = ARM64_KVM_HVHE, + .type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE, + .matches = hvhe_possible, + }, {}, }; diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps index 40ba95472594..3c23a55d7c2f 100644 --- a/arch/arm64/tools/cpucaps +++ b/arch/arm64/tools/cpucaps @@ -47,6 +47,7 @@ HAS_TLB_RANGE HAS_VIRT_HOST_EXTN HAS_WFXT HW_DBM +KVM_HVHE KVM_PROTECTED_MODE MISMATCHED_CACHE_TYPE MTE From patchwork Fri May 26 14:33:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257008 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D963C77B73 for ; Fri, 26 May 2023 14:36:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243557AbjEZOg1 (ORCPT ); Fri, 26 May 2023 10:36:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243680AbjEZOgZ (ORCPT ); Fri, 26 May 2023 10:36:25 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A4A5E7A for ; Fri, 26 May 2023 07:36:05 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0280465072 for ; Fri, 26 May 2023 14:33:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 63449C433AC; Fri, 26 May 2023 14:33:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685111634; bh=Eb4P+yvkDgqtafWekJ5k0MGdRtAE8LRnoYczU5+CLgY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=U+jbSdpAnVTx8gdgO3M4sW3Ek2fLcruyosOEdOe7fjRZRa+ZxHEcQwXsjOm/iFbP2 TxCE651RXucpnzPekydl4mAPYIkT91kRayQoQZ9OCW89vQhrdH5epwUGW7qPMFxj1i VSH05iaXKPpL1CCReAmV1iE/2bchMxOYAdRpe2Ie4nQXHtccFzMEROGpEgtWnTt8uM 8STSXnRusIYeJLTcOIRN/tGwqkmHobacKE7k00aFUy3+osVAU/h2EoYBfC7ceMJEeZ t4kZHDjPDKbnQjBYCLab1EnGtLzA7pYtNv56bhnYkZg8bllwFH/VU94P3nhyGRa6j8 Qtk1IxmN3hQnQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVs-000aHS-Je; Fri, 26 May 2023 15:33:52 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 05/17] arm64: Don't enable VHE for the kernel if OVERRIDE_HVHE is set Date: Fri, 26 May 2023 15:33:36 +0100 Message-Id: <20230526143348.4072074-6-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org If the OVERRIDE_HVHE SW override is set (as a precursor of the KVM_HVHE capability), do not enable VHE for the kernel and drop to EL1 as if VHE was either disabled or unavailable. Further changes will enable VHE at EL2 only, with the kernel still running at EL1. Signed-off-by: Marc Zyngier --- arch/arm64/kernel/hyp-stub.S | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index 9439240c3fcf..5c71e1019545 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -82,7 +82,15 @@ SYM_CODE_START_LOCAL(__finalise_el2) tbnz x1, #0, 1f // Needs to be VHE capable, obviously - check_override id_aa64mmfr1 ID_AA64MMFR1_EL1_VH_SHIFT 2f 1f x1 x2 + check_override id_aa64mmfr1 ID_AA64MMFR1_EL1_VH_SHIFT 0f 1f x1 x2 + +0: // Check whether we only want the hypervisor to run VHE, not the kernel + adr_l x1, arm64_sw_feature_override + ldr x2, [x1, FTR_OVR_VAL_OFFSET] + ldr x1, [x1, FTR_OVR_MASK_OFFSET] + and x2, x2, x1 + ubfx x2, x2, #ARM64_SW_FEATURE_OVERRIDE_HVHE, #4 + cbz x2, 2f 1: mov_q x0, HVC_STUB_ERR eret From patchwork Fri May 26 14:33:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E269C7EE23 for ; Fri, 26 May 2023 14:36:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243828AbjEZOgb (ORCPT ); Fri, 26 May 2023 10:36:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59426 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243787AbjEZOg3 (ORCPT ); Fri, 26 May 2023 10:36:29 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7DA0D1BC for ; Fri, 26 May 2023 07:36:08 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 68AFB65076 for ; Fri, 26 May 2023 14:33:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CAB81C4339E; Fri, 26 May 2023 14:33:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685111634; bh=RKBJeUiRouPE1bzbetDl/6+P89/xc2hCkSFuNbmCwpA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mHyXUlZSJ19LG9+Qga4G5hNza1n7qiZN47MJMWKN91WBKo3UPO9BbFOA5ufOnSNEk 6ZgYTaYGPqLb9q4QexKImkjC9tJwM8Fin7eKXLwhi1y0UcAgF5vKHkOAdIbVaS/KQO wBXpfNd3wOhz4o9Agr/L2h/lpGmfs3PpmByrOlPwCpVNE1jqymGEEcwhfOxEPft9Vj ocBpoNtjrzLNsrmfoiUecZ6d4SgBpnE7CYyClr0iDzYz+ITNA2W4BFUOtaJ4+ijvpt hW0rkA5IH+196D7OHvxuf7OF68lSJcei59ykWUnGaxRctt9GUYBNjIk4dkzg6z53pU +U5tudgVGnuVQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVs-000aHS-Qk; Fri, 26 May 2023 15:33:52 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 06/17] arm64: Allow EL1 physical timer access when running VHE Date: Fri, 26 May 2023 15:33:37 +0100 Message-Id: <20230526143348.4072074-7-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org To initialise the timer access from EL2 when HCR_EL2.E2H is set, we must make use the CNTHCTL_EL2 formap used is appropriate. This amounts to shifting the timer/counter enable bits by 10 to the left. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/el2_setup.h | 5 +++++ arch/arm64/kvm/hyp/nvhe/hyp-init.S | 9 +++++++++ 2 files changed, 14 insertions(+) diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h index 037724b19c5c..225bf1f2514d 100644 --- a/arch/arm64/include/asm/el2_setup.h +++ b/arch/arm64/include/asm/el2_setup.h @@ -34,6 +34,11 @@ */ .macro __init_el2_timers mov x0, #3 // Enable EL1 physical timers + mrs x1, hcr_el2 + and x1, x1, #HCR_E2H + cbz x1, .LnVHE_\@ + lsl x0, x0, #10 +.LnVHE_\@: msr cnthctl_el2, x0 msr cntvoff_el2, xzr // Clear virtual offset .endm diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index a6d67c2bb5ae..f9ee10e29497 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -95,6 +95,15 @@ SYM_CODE_START_LOCAL(___kvm_hyp_init) ldr x1, [x0, #NVHE_INIT_HCR_EL2] msr hcr_el2, x1 + mov x2, #HCR_E2H + and x2, x1, x2 + cbz x2, 1f + + mrs x1, cnthctl_el2 + and x1, x1, #~(BIT(0) | BIT(1)) + orr x1, x1, #(BIT(10) | BIT(11)) + msr cnthctl_el2, x1 +1: ldr x1, [x0, #NVHE_INIT_VTTBR] msr vttbr_el2, x1 From patchwork Fri May 26 14:33:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3136DC7EE23 for ; Fri, 26 May 2023 14:36:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243227AbjEZOgW (ORCPT ); Fri, 26 May 2023 10:36:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59312 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237375AbjEZOgV (ORCPT ); Fri, 26 May 2023 10:36:21 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F4B2E49 for ; Fri, 26 May 2023 07:36:01 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6485D65073 for ; Fri, 26 May 2023 14:33:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 30ED7C433EF; Fri, 26 May 2023 14:33:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685111635; bh=PWi6LY8XpJQzudXyIWXRXdjw/fgswuqZz0wPMwkGK68=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bODysDYCEpcWSSgue4rUMETsDImAbhG746gjq9mdCFw+TnPmfJlvE+SmMOW26Npwf PKhPRlFvmbN4Usc5ugce7JvehcxxghF9VUnSx/ykwJRXrlQvBDeDkboR+uqCeZd12K m6eubfQ9WY3xho3CAP/j49PTx6cwLNS9e7IWuJ3KfxfIsm1xPZUXNyvt9+buaOeDV6 HtTk/JOmcmYKQYPM4VQY0EFTkC9F70aK7qugfrCSsrWTa1HtVvJVndawW6GXRFpHtg +3u8tqUwNbYPE2ybB0VZXd7pDiBKzjDJGAlbn0/dgLwgx6DxvVPmB+RWByQdcKN6Ia m0XA+gaMelF8Q== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVt-000aHS-1A; Fri, 26 May 2023 15:33:53 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 07/17] arm64: Use CPACR_EL1 format to set CPTR_EL2 when E2H is set Date: Fri, 26 May 2023 15:33:38 +0100 Message-Id: <20230526143348.4072074-8-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When HCR_EL2.E2H is set, the CPTR_EL2 register takes the CPACR_EL1 format. Yes, this is good fun. Hack the bits of startup code that assume E2H=0 while setting up CPTR_EL2 to make them grok the CPTR_EL1 format. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/el2_setup.h | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h index 225bf1f2514d..bba508ffa12d 100644 --- a/arch/arm64/include/asm/el2_setup.h +++ b/arch/arm64/include/asm/el2_setup.h @@ -129,8 +129,15 @@ .endm /* Coprocessor traps */ -.macro __init_el2_nvhe_cptr +.macro __init_el2_cptr + mrs x1, hcr_el2 + and x1, x1, #HCR_E2H + cbz x1, .LnVHE_\@ + mov x0, #(CPACR_EL1_FPEN_EL1EN | CPACR_EL1_FPEN_EL0EN) + b .Lset_cptr_\@ +.LnVHE_\@: mov x0, #0x33ff +.Lset_cptr_\@: msr cptr_el2, x0 // Disable copro. traps to EL2 .endm @@ -196,7 +203,7 @@ __init_el2_gicv3 __init_el2_hstr __init_el2_nvhe_idregs - __init_el2_nvhe_cptr + __init_el2_cptr __init_el2_fgt __init_el2_nvhe_prepare_eret .endm @@ -244,7 +251,17 @@ .Linit_sve_\@: /* SVE register access */ mrs x0, cptr_el2 // Disable SVE traps + mrs x1, hcr_el2 + and x1, x1, #HCR_E2H + cbz x1, .Lcptr_nvhe_\@ + + // VHE case + orr x0, x0, #(CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN) + b .Lset_cptr_\@ + +.Lcptr_nvhe_\@: // nVHE case bic x0, x0, #CPTR_EL2_TZ +.Lset_cptr_\@: msr cptr_el2, x0 isb mov x1, #ZCR_ELx_LEN_MASK // SVE: Enable full vector From patchwork Fri May 26 14:33:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257006 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5BA1C77B73 for ; Fri, 26 May 2023 14:36:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243672AbjEZOgY (ORCPT ); Fri, 26 May 2023 10:36:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59322 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237293AbjEZOgW (ORCPT ); Fri, 26 May 2023 10:36:22 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8BFB3E6D for ; Fri, 26 May 2023 07:36:01 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D169165042 for ; Fri, 26 May 2023 14:33:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3FB0BC4339C; Fri, 26 May 2023 14:33:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685111635; bh=UaDqzpeXMxTsA0h1VeMgRA4blxON31O7kP8OlVYUq/E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YM7MX8Og8fTJWm0KWs/I7dne4K44omL0VLbEekfs24gCmx9xA4Mk2SjugXCspyakv LLi80+GB4edTIHojuCkYRwVpKm0ToLME9mt3vj5p5xOytsOsbAadmZNt4sn72OqB6U llDKyzNhvxnzemQqU7FG8OTDdt8jqH1vnqz8+vQDLiKnxrasWtPap7dkD9kRG71+uj 0r6/EbZ2RVJJiLpjwlbrfCjhHVzEXiX1Q4h706Q5oJEdVG0i/V0rkuL2UGxZWYvjre /4IpGBV7jGvCjCFUWJellPhYwmgGr9haAyyCBejFGRStaHW3MTCL28FF1eBsKS6tH5 /PyI/fRSxuBQg== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVt-000aHS-AY; Fri, 26 May 2023 15:33:53 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 08/17] KVM: arm64: Remove alternatives from sysreg accessors in VHE hypervisor context Date: Fri, 26 May 2023 15:33:39 +0100 Message-Id: <20230526143348.4072074-9-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org In the VHE hypervisor code, we should be using the remapped VHE accessors, no ifs, no buts. No need to generate any alternative. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/kvm_hyp.h | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index bdd9cf546d95..fea04eb25cb4 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -16,6 +16,23 @@ DECLARE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt); DECLARE_PER_CPU(unsigned long, kvm_hyp_vector); DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); +/* + * Unified accessors for registers that have a different encoding + * between VHE and non-VHE. They must be specified without their "ELx" + * encoding, but with the SYS_ prefix, as defined in asm/sysreg.h. + */ + +#if defined(__KVM_VHE_HYPERVISOR__) + +#define read_sysreg_el0(r) read_sysreg_s(r##_EL02) +#define write_sysreg_el0(v,r) write_sysreg_s(v, r##_EL02) +#define read_sysreg_el1(r) read_sysreg_s(r##_EL12) +#define write_sysreg_el1(v,r) write_sysreg_s(v, r##_EL12) +#define read_sysreg_el2(r) read_sysreg_s(r##_EL1) +#define write_sysreg_el2(v,r) write_sysreg_s(v, r##_EL1) + +#else // !__KVM_VHE_HYPERVISOR__ + #define read_sysreg_elx(r,nvh,vh) \ ({ \ u64 reg; \ @@ -35,12 +52,6 @@ DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); : : "rZ" (__val)); \ } while (0) -/* - * Unified accessors for registers that have a different encoding - * between VHE and non-VHE. They must be specified without their "ELx" - * encoding, but with the SYS_ prefix, as defined in asm/sysreg.h. - */ - #define read_sysreg_el0(r) read_sysreg_elx(r, _EL0, _EL02) #define write_sysreg_el0(v,r) write_sysreg_elx(v, r, _EL0, _EL02) #define read_sysreg_el1(r) read_sysreg_elx(r, _EL1, _EL12) @@ -48,6 +59,8 @@ DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); #define read_sysreg_el2(r) read_sysreg_elx(r, _EL2, _EL1) #define write_sysreg_el2(v,r) write_sysreg_elx(v, r, _EL2, _EL1) +#endif // __KVM_VHE_HYPERVISOR__ + /* * Without an __arch_swab32(), we fall back to ___constant_swab32(), but the * static inline can allow the compiler to out-of-line this. KVM always wants From patchwork Fri May 26 14:33:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CABA2C7EE2C for ; Fri, 26 May 2023 14:36:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243560AbjEZOgZ (ORCPT ); Fri, 26 May 2023 10:36:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59358 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243670AbjEZOgY (ORCPT ); Fri, 26 May 2023 10:36:24 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5502AE5B for ; Fri, 26 May 2023 07:36:02 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id EAE1465069 for ; Fri, 26 May 2023 14:33:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 58048C433D2; Fri, 26 May 2023 14:33:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685111635; bh=kD0iyRxdqfv2pvJX3Hjwo3JKxYWjJAruZ+EvPdKsFjc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fz488rrt29CFnO8bWJvAl0I475QVEG2yXe+abKp+5FUkAG2vOKhQCs3bnxTeiHYc9 BFlmQZTVRQQ24sORmWnRCXISPD4YSCuITccWa8tCVWu1WRtNI2s2y3Z0Z8GQd5wvA9 nCOdVMaw2ieZF1Tkwl+iqG5WRV/OH3S990JPl+iUT0zDdNu7GLk0fWXsyOhMSdOi+H LiWWv+Nu2J4HdBpeAVUVpNkA68V8v6YKaagGnUprcttnZYTQB4LdHQSjDeuzAtBwyK Q2rcOYxbvpEb/GhotbF8z3Zqba1m34nqyo09br3sz8/TzQY0NjEToH65HuIQoyOgFE OdLMKdv3bOmDw== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVt-000aHS-IW; Fri, 26 May 2023 15:33:53 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 09/17] KVM: arm64: Key use of VHE instructions in nVHE code off ARM64_KVM_HVHE Date: Fri, 26 May 2023 15:33:40 +0100 Message-Id: <20230526143348.4072074-10-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org We can now start with the fun stuff: if we enable VHE *only* for the hypervisor, we need to generate the VHE instructions when accessing the system registers. For this, reporpose the alternative sequence to be keyed off ARM64_KVM_HVHE in the nVHE hypervisor code, and only there. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/kvm_hyp.h | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index fea04eb25cb4..b7238c72a04c 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -33,12 +33,18 @@ DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); #else // !__KVM_VHE_HYPERVISOR__ +#if defined(__KVM_NVHE_HYPERVISOR__) +#define VHE_ALT_KEY ARM64_KVM_HVHE +#else +#define VHE_ALT_KEY ARM64_HAS_VIRT_HOST_EXTN +#endif + #define read_sysreg_elx(r,nvh,vh) \ ({ \ u64 reg; \ - asm volatile(ALTERNATIVE(__mrs_s("%0", r##nvh), \ + asm volatile(ALTERNATIVE(__mrs_s("%0", r##nvh), \ __mrs_s("%0", r##vh), \ - ARM64_HAS_VIRT_HOST_EXTN) \ + VHE_ALT_KEY) \ : "=r" (reg)); \ reg; \ }) @@ -48,7 +54,7 @@ DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); u64 __val = (u64)(v); \ asm volatile(ALTERNATIVE(__msr_s(r##nvh, "%x0"), \ __msr_s(r##vh, "%x0"), \ - ARM64_HAS_VIRT_HOST_EXTN) \ + VHE_ALT_KEY) \ : : "rZ" (__val)); \ } while (0) From patchwork Fri May 26 14:33:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257021 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC9B6C7EE2C for ; Fri, 26 May 2023 14:47:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243573AbjEZOr3 (ORCPT ); Fri, 26 May 2023 10:47:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41660 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237462AbjEZOr2 (ORCPT ); Fri, 26 May 2023 10:47:28 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB80E10D9 for ; Fri, 26 May 2023 07:46:52 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8430F615D1 for ; Fri, 26 May 2023 14:46:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D3EFBC433D2; Fri, 26 May 2023 14:46:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685112409; bh=gX1HEG+z9KW3eMtUvCzJlWpVp+Aform3r5YduMPqVos=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ofsW8JVOTHJSD7OCUF4Swuj9FRlkinEIJ+gqQ5CWgJAG4CwBVcnnEp6B4AQfsqYEu YEVC+18Zr//6R8aaMN/NsojaGFz650zEI0Wn+pFJ1O/2wjG3tXqhlK4vI0ok0IMiev GdvnV93rRCmwRZB9eXdWcd+9xtiY5Hg2wsgeXLS9/Wg7rt47eEL0Y6ZCCRcVTDvJCN xBY/PKtgPSLKF9GRLnIVFyNXjhdQhcAD4MutgwNtF697DV1fpLxWz51zQFsKClapZ2 ZM6wcIhfChWJI3y+PDYhgUU1MHTTSsX0DfvOt4k6t5rQOJ9ShGkwhj411PY3vsQnrQ FQx/JkXp+9ffw== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVt-000aHS-QU; Fri, 26 May 2023 15:33:53 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 10/17] KVM: arm64: Force HCR_EL2.E2H when ARM64_KVM_HVHE is set Date: Fri, 26 May 2023 15:33:41 +0100 Message-Id: <20230526143348.4072074-11-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Obviously, in order to be able to use VHE whilst at EL2, we need to set HCR_EL2.E2H. Do so when ARM64_KVM_HVHE is set. Signed-off-by: Marc Zyngier --- arch/arm64/kvm/arm.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 14391826241c..c12276dd2cf4 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1676,6 +1676,8 @@ static void __init cpu_prepare_hyp_mode(int cpu, u32 hyp_va_bits) params->hcr_el2 = HCR_HOST_NVHE_PROTECTED_FLAGS; else params->hcr_el2 = HCR_HOST_NVHE_FLAGS; + if (cpus_have_final_cap(ARM64_KVM_HVHE)) + params->hcr_el2 |= HCR_E2H; params->vttbr = params->vtcr = 0; /* From patchwork Fri May 26 14:33:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42179C77B7A for ; Fri, 26 May 2023 14:48:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243828AbjEZOse (ORCPT ); Fri, 26 May 2023 10:48:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42804 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237463AbjEZOsc (ORCPT ); Fri, 26 May 2023 10:48:32 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4451EE42 for ; Fri, 26 May 2023 07:48:07 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 97859650A0 for ; Fri, 26 May 2023 14:47:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0AF54C433D2; Fri, 26 May 2023 14:47:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685112423; bh=XkWyzW+sSMEWSg65caOPf/zhuNHxIwdjYYA7HpLQriM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VipzasFZ7QnwjDLqr+Ww2In84+92fEp8yCoQBpVscbftPSrPN5IRxUyqRm0YjMik0 JfZEOYwZQeS20pBrYYGrK1ylsKOpG4kyCacs/oMOG9DO8XUCcm7h4e9g6zlLvkIop2 H3LUsKvsD0o0RL3LaGgVxzmeX2mGilYLVsfJtBKvFa/egPR0feNPhHDfdV2ttVZ0tY 7OCzqzaaI4l6RLmTMynqZq5z3FR5LZnipPxc1/bbPRtywbEJD/X+ZWm/ttaRkADelw p4dSNROj3Bpux2hdV6X8jhJwoUWasxwBU9cu6BjzyhcEx294MvU5DoGpLGKrJwLgZo 67K2WtyV53f6w== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVu-000aHS-3E; Fri, 26 May 2023 15:33:54 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 11/17] KVM: arm64: Disable TTBR1_EL2 when using ARM64_KVM_HVHE Date: Fri, 26 May 2023 15:33:42 +0100 Message-Id: <20230526143348.4072074-12-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When using hVHE, we end-up with two TTBRs at EL2. That's great, but we're not quite ready for this just yet. Disable TTBR1_EL2 by setting TCR_EL2.EPD1 so that we only translate via TTBR0_EL2. Signed-off-by: Marc Zyngier --- arch/arm64/kvm/arm.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index c12276dd2cf4..35b32cb6faa5 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1666,7 +1666,13 @@ static void __init cpu_prepare_hyp_mode(int cpu, u32 hyp_va_bits) params->mair_el2 = read_sysreg(mair_el1); - tcr = (read_sysreg(tcr_el1) & TCR_EL2_MASK) | TCR_EL2_RES1; + tcr = read_sysreg(tcr_el1); + if (cpus_have_final_cap(ARM64_KVM_HVHE)) { + tcr |= TCR_EPD1_MASK; + } else { + tcr &= TCR_EL2_MASK; + tcr |= TCR_EL2_RES1; + } tcr &= ~TCR_T0SZ_MASK; tcr |= TCR_T0SZ(hyp_va_bits); params->tcr_el2 = tcr; From patchwork Fri May 26 14:33:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257023 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12851C7EE23 for ; Fri, 26 May 2023 14:48:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243787AbjEZOsD (ORCPT ); Fri, 26 May 2023 10:48:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42200 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243829AbjEZOsB (ORCPT ); Fri, 26 May 2023 10:48:01 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3AAEC10C8 for ; Fri, 26 May 2023 07:47:32 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6509961038 for ; Fri, 26 May 2023 14:46:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CD4CBC433D2; Fri, 26 May 2023 14:46:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685112411; bh=GV2OlEBM0jd+RwK4yjooVI5TfGUg81MrP522jOEKVMw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VdiFf1TZzQXsm6cnv4aYxrK5OAVRAwuugwvFof5Vhy4ZOk4iMJdhPCkorB6/X/5AR E8CWZF2R5IwX45JV0YM9VaKnnUjC01Hi/9UUcbEkUsSeak+lBSoT5YIvftkWyO87kb GpNpSUIvA7+PAMUX/HRvPvhiYFYJgZKp6eu8InsHI0edkXhEd06P2+kOcBP8OJqdpb 6VA46f3hBoZfWrznAC3+d4sRg07MPmNr5yj0DnPvVbmdegfH1p4ss50Ac4vHc8fw88 LN/EdUi/oEVG8tVbC9CXNggSUFoivi0Yt2pw1/THe9aBenjQte/pxOwdENQCQnnlUo OqCRI0JC/6dSQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVu-000aHS-B9; Fri, 26 May 2023 15:33:54 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 12/17] KVM: arm64: Adjust EL2 stage-1 leaf AP bits when ARM64_KVM_HVHE is set Date: Fri, 26 May 2023 15:33:43 +0100 Message-Id: <20230526143348.4072074-13-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org El2 stage-1 page-table format is subtly (and annoyingly) different when HCR_EL2.E2H is set. Take the ARM64_KVM_HVHE configuration into account when setting the AP bits. Signed-off-by: Marc Zyngier --- arch/arm64/kvm/hyp/pgtable.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 3d61bd3e591d..3664cd64227b 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -21,8 +21,10 @@ #define KVM_PTE_LEAF_ATTR_LO_S1_ATTRIDX GENMASK(4, 2) #define KVM_PTE_LEAF_ATTR_LO_S1_AP GENMASK(7, 6) -#define KVM_PTE_LEAF_ATTR_LO_S1_AP_RO 3 -#define KVM_PTE_LEAF_ATTR_LO_S1_AP_RW 1 +#define KVM_PTE_LEAF_ATTR_LO_S1_AP_RO \ + ({ cpus_have_final_cap(ARM64_KVM_HVHE) ? 2 : 3; }) +#define KVM_PTE_LEAF_ATTR_LO_S1_AP_RW \ + ({ cpus_have_final_cap(ARM64_KVM_HVHE) ? 0 : 1; }) #define KVM_PTE_LEAF_ATTR_LO_S1_SH GENMASK(9, 8) #define KVM_PTE_LEAF_ATTR_LO_S1_SH_IS 3 #define KVM_PTE_LEAF_ATTR_LO_S1_AF BIT(10) From patchwork Fri May 26 14:33:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257027 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5599EC7EE23 for ; Fri, 26 May 2023 14:48:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243796AbjEZOsf (ORCPT ); Fri, 26 May 2023 10:48:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243723AbjEZOsd (ORCPT ); Fri, 26 May 2023 10:48:33 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8783E45 for ; Fri, 26 May 2023 07:48:07 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6A3FE65096 for ; Fri, 26 May 2023 14:47:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B6498C433D2; Fri, 26 May 2023 14:47:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685112420; bh=0IXbsbrm0m3kqxst+/4HomRh3sFyzCQF1ni7ZpdNCX0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aA4tSc8yfZRbSH/UoUppoWe8h9Mog71lGo53BdHJ/9pv5h0ut6xY5EK7t6+vSP5bN lkVNRarX8iWzdT15xr3EgzGnLZg3enTeZitkdu3DC5KVGR6cs5knfgPJq7T0ryGeL+ cIUe1eKqeAEu1eonQTEie4UoIVYn6AnayWRDhx5DccUPxjdMTlgJpJCBx1WlGtxX8A FLDA8VGi9/bOA+MY8lXfMyL4qMo7LeR7g0jccOZ3ystSosKtZtQlLq+aO1FqMimm4H QwY+5Mswz6aKGllXfFDeu3tLYbFept+DCWwsvSFqNybEJaHYhaV020H3V7EKfAYRH+ 1ZGBYni1jhyiA== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVu-000aHS-Kx; Fri, 26 May 2023 15:33:54 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 13/17] KVM: arm64: Rework CPTR_EL2 programming for HVHE configuration Date: Fri, 26 May 2023 15:33:44 +0100 Message-Id: <20230526143348.4072074-14-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Just like we repainted the early arm64 code, we need to update the CPTR_EL2 accesses that are taking place in the nVHE code when hVHE is used, making them look as if they were CPACR_EL1 accesses. Just like the VHE code. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/kvm_arm.h | 4 +--- arch/arm64/include/asm/kvm_emulate.h | 31 +++++++++++++++++++++++++ arch/arm64/kvm/arm.c | 2 +- arch/arm64/kvm/fpsimd.c | 4 ++-- arch/arm64/kvm/hyp/include/hyp/switch.h | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 6 ++++- arch/arm64/kvm/hyp/nvhe/pkvm.c | 24 ++++++++++++++----- arch/arm64/kvm/hyp/nvhe/switch.c | 28 ++++++++++++---------- arch/arm64/kvm/hyp/vhe/switch.c | 2 +- arch/arm64/kvm/sys_regs.c | 2 +- 10 files changed, 77 insertions(+), 28 deletions(-) diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index baef29fcbeee..e448f8f7fd7e 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -285,7 +285,6 @@ #define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT) #define CPTR_EL2_TZ (1 << 8) #define CPTR_NVHE_EL2_RES1 0x000032ff /* known RES1 bits in CPTR_EL2 (nVHE) */ -#define CPTR_EL2_DEFAULT CPTR_NVHE_EL2_RES1 #define CPTR_NVHE_EL2_RES0 (GENMASK(63, 32) | \ GENMASK(29, 21) | \ GENMASK(19, 14) | \ @@ -347,8 +346,7 @@ ECN(SOFTSTP_CUR), ECN(WATCHPT_LOW), ECN(WATCHPT_CUR), \ ECN(BKPT32), ECN(VECTOR32), ECN(BRK64), ECN(ERET) -#define CPACR_EL1_DEFAULT (CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN |\ - CPACR_EL1_ZEN_EL1EN) +#define CPACR_EL1_TTA (1 << 28) #define kvm_mode_names \ { PSR_MODE_EL0t, "EL0t" }, \ diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h index b31b32ecbe2d..4d82e622240d 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -570,4 +570,35 @@ static inline bool vcpu_has_feature(struct kvm_vcpu *vcpu, int feature) return test_bit(feature, vcpu->arch.features); } +static __always_inline u64 kvm_get_reset_cptr_el2(struct kvm_vcpu *vcpu) +{ + u64 val; + + if (has_vhe()) { + val = (CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN | + CPACR_EL1_ZEN_EL1EN); + } else if (has_hvhe()) { + val = (CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN); + } else { + val = CPTR_NVHE_EL2_RES1; + + if (vcpu_has_sve(vcpu) && + (vcpu->arch.fp_state == FP_STATE_GUEST_OWNED)) + val |= CPTR_EL2_TZ; + if (cpus_have_final_cap(ARM64_SME)) + val &= ~CPTR_EL2_TSM; + } + + return val; +} + +static __always_inline void kvm_reset_cptr_el2(struct kvm_vcpu *vcpu) +{ + u64 val = kvm_get_reset_cptr_el2(vcpu); + + if (has_vhe() || has_hvhe()) + write_sysreg(val, cpacr_el1); + else + write_sysreg(val, cptr_el2); +} #endif /* __ARM64_KVM_EMULATE_H__ */ diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 35b32cb6faa5..01c975a0acc9 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1240,7 +1240,7 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu, } vcpu_reset_hcr(vcpu); - vcpu->arch.cptr_el2 = CPTR_EL2_DEFAULT; + vcpu->arch.cptr_el2 = kvm_get_reset_cptr_el2(vcpu); /* * Handle the "start in power-off" case. diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index 1279949599b5..62b20ea024a0 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -172,7 +172,7 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu) /* * If we have VHE then the Hyp code will reset CPACR_EL1 to - * CPACR_EL1_DEFAULT and we need to reenable SME. + * the default value and we need to reenable SME. */ if (has_vhe() && system_supports_sme()) { /* Also restore EL0 state seen on entry */ @@ -202,7 +202,7 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu) /* * The FPSIMD/SVE state in the CPU has not been touched, and we * have SVE (and VHE): CPACR_EL1 (alias CPTR_EL2) has been - * reset to CPACR_EL1_DEFAULT by the Hyp code, disabling SVE + * reset by kvm_reset_cptr_el2() in the Hyp code, disabling SVE * for EL0. To avoid spurious traps, restore the trap state * seen by kvm_arch_vcpu_load_fp(): */ diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index c41166f1a1dd..961681f2aed7 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -184,7 +184,7 @@ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code) /* Valid trap. Switch the context: */ /* First disable enough traps to allow us to update the registers */ - if (has_vhe()) { + if (has_vhe() || has_hvhe()) { reg = CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN; if (sve_guest) reg |= CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN; diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 728e01d4536b..ce602f9e93eb 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -392,7 +392,11 @@ void handle_trap(struct kvm_cpu_context *host_ctxt) handle_host_smc(host_ctxt); break; case ESR_ELx_EC_SVE: - sysreg_clear_set(cptr_el2, CPTR_EL2_TZ, 0); + if (has_hvhe()) + sysreg_clear_set(cpacr_el1, 0, (CPACR_EL1_ZEN_EL1EN | + CPACR_EL1_ZEN_EL0EN)); + else + sysreg_clear_set(cptr_el2, CPTR_EL2_TZ, 0); isb(); sve_cond_update_zcr_vq(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); break; diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index a06ece14a6d8..5d5ee735a7d9 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -27,6 +27,7 @@ static void pvm_init_traps_aa64pfr0(struct kvm_vcpu *vcpu) u64 hcr_set = HCR_RW; u64 hcr_clear = 0; u64 cptr_set = 0; + u64 cptr_clear = 0; /* Protected KVM does not support AArch32 guests. */ BUILD_BUG_ON(FIELD_GET(ARM64_FEATURE_MASK(ID_AA64PFR0_EL1_EL0), @@ -57,12 +58,17 @@ static void pvm_init_traps_aa64pfr0(struct kvm_vcpu *vcpu) } /* Trap SVE */ - if (!FIELD_GET(ARM64_FEATURE_MASK(ID_AA64PFR0_EL1_SVE), feature_ids)) - cptr_set |= CPTR_EL2_TZ; + if (!FIELD_GET(ARM64_FEATURE_MASK(ID_AA64PFR0_EL1_SVE), feature_ids)) { + if (has_hvhe()) + cptr_clear |= CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN; + else + cptr_set |= CPTR_EL2_TZ; + } vcpu->arch.hcr_el2 |= hcr_set; vcpu->arch.hcr_el2 &= ~hcr_clear; vcpu->arch.cptr_el2 |= cptr_set; + vcpu->arch.cptr_el2 &= ~cptr_clear; } /* @@ -120,8 +126,12 @@ static void pvm_init_traps_aa64dfr0(struct kvm_vcpu *vcpu) mdcr_set |= MDCR_EL2_TTRF; /* Trap Trace */ - if (!FIELD_GET(ARM64_FEATURE_MASK(ID_AA64DFR0_EL1_TraceVer), feature_ids)) - cptr_set |= CPTR_EL2_TTA; + if (!FIELD_GET(ARM64_FEATURE_MASK(ID_AA64DFR0_EL1_TraceVer), feature_ids)) { + if (has_hvhe()) + cptr_set |= CPACR_EL1_TTA; + else + cptr_set |= CPTR_EL2_TTA; + } vcpu->arch.mdcr_el2 |= mdcr_set; vcpu->arch.mdcr_el2 &= ~mdcr_clear; @@ -176,8 +186,10 @@ static void pvm_init_trap_regs(struct kvm_vcpu *vcpu) /* Clear res0 and set res1 bits to trap potential new features. */ vcpu->arch.hcr_el2 &= ~(HCR_RES0); vcpu->arch.mdcr_el2 &= ~(MDCR_EL2_RES0); - vcpu->arch.cptr_el2 |= CPTR_NVHE_EL2_RES1; - vcpu->arch.cptr_el2 &= ~(CPTR_NVHE_EL2_RES0); + if (!has_hvhe()) { + vcpu->arch.cptr_el2 |= CPTR_NVHE_EL2_RES1; + vcpu->arch.cptr_el2 &= ~(CPTR_NVHE_EL2_RES0); + } } /* diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index 71fa16a0dc77..5fa0b1c9ee8d 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -44,13 +44,24 @@ static void __activate_traps(struct kvm_vcpu *vcpu) __activate_traps_common(vcpu); val = vcpu->arch.cptr_el2; - val |= CPTR_EL2_TTA | CPTR_EL2_TAM; + val |= CPTR_EL2_TAM; /* Same bit irrespective of E2H */ + val |= has_hvhe() ? CPACR_EL1_TTA : CPTR_EL2_TTA; + if (cpus_have_final_cap(ARM64_SME)) { + if (has_hvhe()) + val &= ~(CPACR_EL1_SMEN_EL1EN | CPACR_EL1_SMEN_EL0EN); + else + val |= CPTR_EL2_TSM; + } + if (!guest_owns_fp_regs(vcpu)) { - val |= CPTR_EL2_TFP | CPTR_EL2_TZ; + if (has_hvhe()) + val &= ~(CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN | + CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN); + else + val |= CPTR_EL2_TFP | CPTR_EL2_TZ; + __activate_traps_fpsimd32(vcpu); } - if (cpus_have_final_cap(ARM64_SME)) - val |= CPTR_EL2_TSM; write_sysreg(val, cptr_el2); write_sysreg(__this_cpu_read(kvm_hyp_vector), vbar_el2); @@ -73,7 +84,6 @@ static void __activate_traps(struct kvm_vcpu *vcpu) static void __deactivate_traps(struct kvm_vcpu *vcpu) { extern char __kvm_hyp_host_vector[]; - u64 cptr; ___deactivate_traps(vcpu); @@ -98,13 +108,7 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu) write_sysreg(this_cpu_ptr(&kvm_init_params)->hcr_el2, hcr_el2); - cptr = CPTR_EL2_DEFAULT; - if (vcpu_has_sve(vcpu) && (vcpu->arch.fp_state == FP_STATE_GUEST_OWNED)) - cptr |= CPTR_EL2_TZ; - if (cpus_have_final_cap(ARM64_SME)) - cptr &= ~CPTR_EL2_TSM; - - write_sysreg(cptr, cptr_el2); + kvm_reset_cptr_el2(vcpu); write_sysreg(__kvm_hyp_host_vector, vbar_el2); } diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 3d868e84c7a0..034777a23b74 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -84,7 +84,7 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu) */ asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT)); - write_sysreg(CPACR_EL1_DEFAULT, cpacr_el1); + kvm_reset_cptr_el2(vcpu); if (!arm64_kernel_unmapped_at_el0()) host_vectors = __this_cpu_read(this_cpu_vector); diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 71b12094d613..11ededa80737 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -2180,7 +2180,7 @@ static const struct sys_reg_desc sys_reg_descs[] = { EL2_REG(ACTLR_EL2, access_rw, reset_val, 0), EL2_REG(HCR_EL2, access_rw, reset_val, 0), EL2_REG(MDCR_EL2, access_rw, reset_val, 0), - EL2_REG(CPTR_EL2, access_rw, reset_val, CPTR_EL2_DEFAULT ), + EL2_REG(CPTR_EL2, access_rw, reset_val, CPTR_NVHE_EL2_RES1), EL2_REG(HSTR_EL2, access_rw, reset_val, 0), EL2_REG(HACR_EL2, access_rw, reset_val, 0), From patchwork Fri May 26 14:33:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD4E4C77B7A for ; Fri, 26 May 2023 14:48:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243840AbjEZOsG (ORCPT ); Fri, 26 May 2023 10:48:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42316 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243828AbjEZOsE (ORCPT ); Fri, 26 May 2023 10:48:04 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C943AE4A for ; Fri, 26 May 2023 07:47:37 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 713806509F for ; Fri, 26 May 2023 14:46:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CFA27C433D2; Fri, 26 May 2023 14:46:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685112418; bh=ogemMlMjuGNlm+bP0xOneJD8Naa/ebP3Sa/rriTGMmk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jG3GIrrHLgXfmXYxPxTZZQ5oTQxzil6Ps/JUFu57fNcGl3RiHl9esPgXhQ9LQ1BPa QdjsVq/NH4gWiV1m+iAju8PGGCMMrdBrfHJVDbOsnFy+HG0tBAoUp0mgD335WJZTHm 2uwCjk7UkXMN/m5ZzworAjrlqJOGzdA/0REDPma1SziluNUqU5Eqwkd5QowRDRvyCC iAF6Q3SIVB5kUAcsqydwyYeqW9YVa45WxclK31AvK/w247G9aSjQUzGIAMfkHX8e6Q DoJ/SSQ5iHTlp/0u0arDJ4vULOIclhqdSfUs1FEkZc62KZm53Agxls9kh67P19pHzn wnZ+jpiHWZz9Q== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVu-000aHS-TA; Fri, 26 May 2023 15:33:54 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 14/17] KVM: arm64: Program the timer traps with VHE layout in hVHE mode Date: Fri, 26 May 2023 15:33:45 +0100 Message-Id: <20230526143348.4072074-15-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Just like the rest of the timer code, we need to shift the enable bits around when HCR_EL2.E2H is set, which is the case in hVHE mode. Signed-off-by: Marc Zyngier --- arch/arm64/kvm/hyp/nvhe/timer-sr.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/timer-sr.c b/arch/arm64/kvm/hyp/nvhe/timer-sr.c index b185ac0dbd47..3aaab20ae5b4 100644 --- a/arch/arm64/kvm/hyp/nvhe/timer-sr.c +++ b/arch/arm64/kvm/hyp/nvhe/timer-sr.c @@ -17,21 +17,24 @@ void __kvm_timer_set_cntvoff(u64 cntvoff) } /* - * Should only be called on non-VHE systems. + * Should only be called on non-VHE or hVHE setups. * VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe(). */ void __timer_disable_traps(struct kvm_vcpu *vcpu) { - u64 val; + u64 val, shift = 0; + + if (has_hvhe()) + shift = 10; /* Allow physical timer/counter access for the host */ val = read_sysreg(cnthctl_el2); - val |= CNTHCTL_EL1PCTEN | CNTHCTL_EL1PCEN; + val |= (CNTHCTL_EL1PCTEN | CNTHCTL_EL1PCEN) << shift; write_sysreg(val, cnthctl_el2); } /* - * Should only be called on non-VHE systems. + * Should only be called on non-VHE or hVHE setups. * VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe(). */ void __timer_enable_traps(struct kvm_vcpu *vcpu) @@ -50,5 +53,10 @@ void __timer_enable_traps(struct kvm_vcpu *vcpu) else clr |= CNTHCTL_EL1PCTEN; + if (has_hvhe()) { + clr <<= 10; + set <<= 10; + } + sysreg_clear_set(cnthctl_el2, clr, set); } From patchwork Fri May 26 14:33:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257028 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB15BC77B7A for ; Fri, 26 May 2023 14:48:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244039AbjEZOsh (ORCPT ); Fri, 26 May 2023 10:48:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243845AbjEZOsd (ORCPT ); Fri, 26 May 2023 10:48:33 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9750610CA for ; Fri, 26 May 2023 07:48:09 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 704DA65099 for ; Fri, 26 May 2023 14:47:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D06D4C433EF; Fri, 26 May 2023 14:47:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685112425; bh=Svn5PaYfn4owwrOXlSAGosifn4fpgZpDHUwDbWmHyHs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QoFWvNitpzOUhLR4P2UpHZQnH1BHwSIPufZtTgALv4pBfiFJTJrL6dwCDX6VVx6td U084tbPg3rDxLh4TpguXPj9Q85VG8+hvHEFFO1cDlzxDeK1l0ZbYBlseyYgmq6DOCE +qjQ1Ow/x4u6MaN95h8XXyury51SmSCXhKXd5IoFvpcAKN2wI11/ZCpO5hRfkdFcBp oV2czpZQZy9yoZGF7eTcubsUUpa5WftF5dfcBSPyXm64I+zSeouY8ag14NuOjn0DVu OA3TDMg+Owvo9BdZ6ftXmQuRg4fDFNvAluPJl5qFr59ryCRTzLmGGXDEquXx/KrEnE rAoJmC/H0ae8Q== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVv-000aHS-5M; Fri, 26 May 2023 15:33:55 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 15/17] KVM: arm64: Force HCR_E2H in guest context when ARM64_KVM_HVHE is set Date: Fri, 26 May 2023 15:33:46 +0100 Message-Id: <20230526143348.4072074-16-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Also make sure HCR_EL2.E2H is set when switching HCR_EL2 in guest context. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/kvm_emulate.h | 2 +- arch/arm64/kvm/hyp/nvhe/pkvm.c | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h index 4d82e622240d..cf40d19a72f8 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -74,7 +74,7 @@ static __always_inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu) static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu) { vcpu->arch.hcr_el2 = HCR_GUEST_FLAGS; - if (is_kernel_in_hyp_mode()) + if (has_vhe() || has_hvhe()) vcpu->arch.hcr_el2 |= HCR_E2H; if (cpus_have_const_cap(ARM64_HAS_RAS_EXTN)) { /* route synchronous external abort exceptions to EL2 */ diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 5d5ee735a7d9..8033ef353a5d 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -44,6 +44,9 @@ static void pvm_init_traps_aa64pfr0(struct kvm_vcpu *vcpu) BUILD_BUG_ON(!FIELD_GET(ARM64_FEATURE_MASK(ID_AA64PFR0_EL1_AdvSIMD), PVM_ID_AA64PFR0_ALLOW)); + if (has_hvhe()) + hcr_set |= HCR_E2H; + /* Trap RAS unless all current versions are supported */ if (FIELD_GET(ARM64_FEATURE_MASK(ID_AA64PFR0_EL1_RAS), feature_ids) < ID_AA64PFR0_EL1_RAS_V1P1) { From patchwork Fri May 26 14:33:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257022 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C508C77B7A for ; Fri, 26 May 2023 14:47:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237397AbjEZOra (ORCPT ); Fri, 26 May 2023 10:47:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41680 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242856AbjEZOr3 (ORCPT ); Fri, 26 May 2023 10:47:29 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E5B51BF for ; Fri, 26 May 2023 07:46:55 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9380A61598 for ; Fri, 26 May 2023 14:46:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 04E29C433EF; Fri, 26 May 2023 14:46:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685112414; bh=tV1GC4lt1AhQsaOyflUGckOCSD9Gpyp+4HrIfwAcgZY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IrKnuSQoOoA2fFttxHevJkJO8t3u9PyQb5LuKKJLcdS24P5RyD4dfr7AmncftDV34 fxm7nNRhJm/j2kKm52PjB7agYffPQrcict2w0coOpBj4jVL65ylbo3EKhP6UL2N3Rf oT1/4oGyx2H5xBEBJ+ETMtHV0fAxdQfGK1G8GkkxGysmBQ6InUOMU0QQK6eN7jIyje Rkqxlqq1Fj01F7mzHcrwDenJ0kuQDwDE5t2bqfMnk9rxqfy16UpBNgVrn5Ri0vehJs 7RgwsZdIIHXo5SOX42QQEgQP60kUhx+/oYTVUFxb758FYBdDe6o6VUx8olCChCPAaO S4AkCZHV7ak2g== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVv-000aHS-DR; Fri, 26 May 2023 15:33:55 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 16/17] arm64: Allow arm64_sw.hvhe on command line Date: Fri, 26 May 2023 15:33:47 +0100 Message-Id: <20230526143348.4072074-17-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add the arm64_sw.hvhe=1 option to force the use of the hVHE mode in the hypervisor code only. This enables the hVHE mode of operation when using KVM on VHE hardware. Signed-off-by: Marc Zyngier --- arch/arm64/kernel/idreg-override.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/arch/arm64/kernel/idreg-override.c b/arch/arm64/kernel/idreg-override.c index 8c93b6198bf5..c553d30089e5 100644 --- a/arch/arm64/kernel/idreg-override.c +++ b/arch/arm64/kernel/idreg-override.c @@ -138,11 +138,22 @@ static const struct ftr_set_desc smfr0 __initconst = { }, }; +static bool __init hvhe_filter(u64 val) +{ + u64 mmfr1 = read_sysreg(id_aa64mmfr1_el1); + + return (val == 1 && + lower_32_bits(__boot_status) == BOOT_CPU_MODE_EL2 && + cpuid_feature_extract_unsigned_field(mmfr1, + ID_AA64MMFR1_EL1_VH_SHIFT)); +} + static const struct ftr_set_desc sw_features __initconst = { .name = "arm64_sw", .override = &arm64_sw_feature_override, .fields = { FIELD("nokaslr", ARM64_SW_FEATURE_OVERRIDE_NOKASLR, NULL), + FIELD("hvhe", ARM64_SW_FEATURE_OVERRIDE_HVHE, hvhe_filter), {} }, }; From patchwork Fri May 26 14:33:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13257025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39659C7EE2C for ; Fri, 26 May 2023 14:48:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243864AbjEZOsH (ORCPT ); Fri, 26 May 2023 10:48:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42346 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243829AbjEZOsF (ORCPT ); Fri, 26 May 2023 10:48:05 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E740B10D0 for ; Fri, 26 May 2023 07:47:38 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 468DF6509D for ; Fri, 26 May 2023 14:46:57 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 90FF4C433D2; Fri, 26 May 2023 14:46:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685112416; bh=hSQy5gLZCuoxx57GR79Suj/ypECkMBpuktFZ/OH29A4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LSV9HG2WMLidZhOqVkX14dvISCMGINHGi/ePjCT+tLL/OGuAOV+58zi3qReiz+uYJ fvO5sqUvQRz9X3iMrGu7+who8YVx7aQTZv7m4i86u3El2Iqsw0R31TfY7lZ7ADe4sM jhCP9PNwQUBuo3pChROyaDi4FaznZx/zE4BG/iudJHyov2V4l4OPwAPzmdK/puUYfy zoWkpbSC/P+3Bow0oxwWsyIn8+uz+O2uTnQYeXvzspHULPHb5QeuebkjZ2SKjya5By xU2fyV6LRiHdPQfaRTtZTmb8q7SX9FpzMF4CmDtdY5SYImGwT/cIJF9xgwgsynX38J xC2iVAtR+KTjQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q2YVv-000aHS-M2; Fri, 26 May 2023 15:33:55 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Quentin Perret , Will Deacon , Fuad Tabba Subject: [PATCH v2 17/17] KVM: arm64: Terrible timer hack for M1 with hVHE Date: Fri, 26 May 2023 15:33:48 +0100 Message-Id: <20230526143348.4072074-18-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526143348.4072074-1-maz@kernel.org> References: <20230526143348.4072074-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, qperret@google.com, will@kernel.org, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org As our M1 friend doesn't have a GIC, it relies on a special hack to deal with masking the guest timers, in the form of an IMPDEF system register. Unfortunately, this sysreg is EL2-only, which means that the kernel cannot mask the interrupts itself, but has to kindly ask EL2 to do it. Yes, this is terrible, but we should be used to it by now. Add a M1-specific hypercall to deal with this. No, I'm not seriously suggesting we merge this crap. Not-seriously-suggested-by: Marc Zyngier Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/arch_timer.h | 8 +++++ arch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/kernel/image-vars.h | 3 ++ arch/arm64/kvm/arch_timer.c | 5 +++ arch/arm64/kvm/hyp/nvhe/hyp-main.c | 11 +++++++ arch/arm64/kvm/hyp/nvhe/timer-sr.c | 9 ++++++ drivers/irqchip/irq-apple-aic.c | 50 +++++++++++++++++++++++++++-- 7 files changed, 85 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h index af1fafbe7e1d..3817e923f52c 100644 --- a/arch/arm64/include/asm/arch_timer.h +++ b/arch/arm64/include/asm/arch_timer.h @@ -232,4 +232,12 @@ static inline bool arch_timer_have_evtstrm_feature(void) { return cpu_have_named_feature(EVTSTRM); } + +#ifdef CONFIG_APPLE_AIC +#define SYS_IMP_APL_VM_TMR_FIQ_ENA_EL2 sys_reg(3, 5, 15, 1, 3) +DECLARE_STATIC_KEY_FALSE(aic_impdef_timer_control); +#endif + +void __aic_timer_fiq_clear_set(u64 clear, u64 set); + #endif diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 43c3bc0f9544..a9e1444fe1d4 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -75,6 +75,7 @@ enum __kvm_host_smccc_func { __KVM_HOST_SMCCC_FUNC___vgic_v3_write_vmcr, __KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs, __KVM_HOST_SMCCC_FUNC___vgic_v3_restore_aprs, + __KVM_HOST_SMCCC_FUNC___aic_timer_fiq_clear_set, __KVM_HOST_SMCCC_FUNC___pkvm_vcpu_init_traps, __KVM_HOST_SMCCC_FUNC___pkvm_init_vm, __KVM_HOST_SMCCC_FUNC___pkvm_init_vcpu, diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 35f3c7959513..3f40f7188acc 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -106,6 +106,9 @@ KVM_NVHE_ALIAS(__hyp_rodata_end); /* pKVM static key */ KVM_NVHE_ALIAS(kvm_protected_mode_initialized); +/* Hack for M1 timer control in hVHE mode */ +KVM_NVHE_ALIAS(aic_impdef_timer_control); + #endif /* CONFIG_KVM */ #ifdef CONFIG_EFI_ZBOOT diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c index 05b022be885b..370e820ecadb 100644 --- a/arch/arm64/kvm/arch_timer.c +++ b/arch/arm64/kvm/arch_timer.c @@ -1259,6 +1259,11 @@ static int timer_irq_set_vcpu_affinity(struct irq_data *d, void *vcpu) return 0; } +void __aic_timer_fiq_clear_set(u64 clear, u64 set) +{ + kvm_call_hyp_nvhe(__aic_timer_fiq_clear_set, clear, set); +} + static int timer_irq_set_irqchip_state(struct irq_data *d, enum irqchip_irq_state which, bool val) { diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index ce602f9e93eb..555294aacfc9 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -192,6 +192,16 @@ static void handle___vgic_v3_restore_aprs(struct kvm_cpu_context *host_ctxt) __vgic_v3_restore_aprs(kern_hyp_va(cpu_if)); } +#define SYS_IMP_APL_VM_TMR_FIQ_ENA_EL2 sys_reg(3, 5, 15, 1, 3) + +static void handle___aic_timer_fiq_clear_set(struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(u64, clear, host_ctxt, 1); + DECLARE_REG(u64, set, host_ctxt, 2); + + __aic_timer_fiq_clear_set(clear, set); +} + static void handle___pkvm_init(struct kvm_cpu_context *host_ctxt) { DECLARE_REG(phys_addr_t, phys, host_ctxt, 1); @@ -322,6 +332,7 @@ static const hcall_t host_hcall[] = { HANDLE_FUNC(__vgic_v3_write_vmcr), HANDLE_FUNC(__vgic_v3_save_aprs), HANDLE_FUNC(__vgic_v3_restore_aprs), + HANDLE_FUNC(__aic_timer_fiq_clear_set), HANDLE_FUNC(__pkvm_vcpu_init_traps), HANDLE_FUNC(__pkvm_init_vm), HANDLE_FUNC(__pkvm_init_vcpu), diff --git a/arch/arm64/kvm/hyp/nvhe/timer-sr.c b/arch/arm64/kvm/hyp/nvhe/timer-sr.c index 3aaab20ae5b4..9335dbee88e5 100644 --- a/arch/arm64/kvm/hyp/nvhe/timer-sr.c +++ b/arch/arm64/kvm/hyp/nvhe/timer-sr.c @@ -60,3 +60,12 @@ void __timer_enable_traps(struct kvm_vcpu *vcpu) sysreg_clear_set(cnthctl_el2, clr, set); } + + +void __aic_timer_fiq_clear_set(u64 clear, u64 set) +{ +#ifdef CONFIG_APPLE_AIC + if (has_hvhe() && static_branch_likely(&aic_impdef_timer_control)) + sysreg_clear_set_s(SYS_IMP_APL_VM_TMR_FIQ_ENA_EL2, clear, set); +#endif +} diff --git a/drivers/irqchip/irq-apple-aic.c b/drivers/irqchip/irq-apple-aic.c index 5c534d9fd2b0..acd47f475f36 100644 --- a/drivers/irqchip/irq-apple-aic.c +++ b/drivers/irqchip/irq-apple-aic.c @@ -180,7 +180,6 @@ #define IPI_SR_PENDING BIT(0) /* Guest timer FIQ enable register */ -#define SYS_IMP_APL_VM_TMR_FIQ_ENA_EL2 sys_reg(3, 5, 15, 1, 3) #define VM_TMR_FIQ_ENABLE_V BIT(0) #define VM_TMR_FIQ_ENABLE_P BIT(1) @@ -236,6 +235,8 @@ enum fiq_hwirq { static DEFINE_STATIC_KEY_TRUE(use_fast_ipi); +DEFINE_STATIC_KEY_FALSE(aic_impdef_timer_control); + struct aic_info { int version; @@ -458,6 +459,40 @@ static unsigned long aic_fiq_get_idx(struct irq_data *d) return AIC_HWIRQ_IRQ(irqd_to_hwirq(d)); } +void __weak __aic_timer_fiq_clear_set(u64 clear, u64 set) { } + +static bool aic_check_timer_enabled(int timer) +{ + if (IS_ENABLED(CONFIG_KVM) && + static_branch_unlikely(&aic_impdef_timer_control)) + return __this_cpu_read(aic_fiq_unmasked) & BIT(timer); + return true; +} + +static void aic_hvhe_timer_mask(int timer, bool mask) +{ + u64 clr, set, bit; + + if (!(IS_ENABLED(CONFIG_KVM) && + static_branch_unlikely(&aic_impdef_timer_control))) + return; + + if (timer == AIC_TMR_EL0_VIRT) + bit = VM_TMR_FIQ_ENABLE_V; + else + bit = VM_TMR_FIQ_ENABLE_P; + + if (mask) { + clr = bit; + set = 0; + } else { + clr = 0; + set = bit; + } + + __aic_timer_fiq_clear_set(clr, set); +} + static void aic_fiq_set_mask(struct irq_data *d) { /* Only the guest timers have real mask bits, unfortunately. */ @@ -470,6 +505,9 @@ static void aic_fiq_set_mask(struct irq_data *d) sysreg_clear_set_s(SYS_IMP_APL_VM_TMR_FIQ_ENA_EL2, VM_TMR_FIQ_ENABLE_V, 0); isb(); break; + case AIC_TMR_EL0_VIRT: + aic_hvhe_timer_mask(AIC_TMR_EL0_VIRT, true); + break; default: break; } @@ -486,6 +524,9 @@ static void aic_fiq_clear_mask(struct irq_data *d) sysreg_clear_set_s(SYS_IMP_APL_VM_TMR_FIQ_ENA_EL2, 0, VM_TMR_FIQ_ENABLE_V); isb(); break; + case AIC_TMR_EL0_VIRT: + aic_hvhe_timer_mask(AIC_TMR_EL0_VIRT, false); + break; default: break; } @@ -545,7 +586,8 @@ static void __exception_irq_entry aic_handle_fiq(struct pt_regs *regs) generic_handle_domain_irq(aic_irqc->hw_domain, AIC_FIQ_HWIRQ(AIC_TMR_EL0_PHYS)); - if (TIMER_FIRING(read_sysreg(cntv_ctl_el0))) + if (TIMER_FIRING(read_sysreg(cntv_ctl_el0)) && + aic_check_timer_enabled(AIC_TMR_EL0_VIRT)) generic_handle_domain_irq(aic_irqc->hw_domain, AIC_FIQ_HWIRQ(AIC_TMR_EL0_VIRT)); @@ -1041,6 +1083,10 @@ static int __init aic_of_ic_init(struct device_node *node, struct device_node *p if (static_branch_likely(&use_fast_ipi)) pr_info("Using Fast IPIs"); + /* Caps are not final at this stage :-/ */ + if (cpus_have_cap(ARM64_KVM_HVHE)) + static_branch_enable(&aic_impdef_timer_control); + cpuhp_setup_state(CPUHP_AP_IRQ_APPLE_AIC_STARTING, "irqchip/apple-aic/ipi:starting", aic_init_cpu, NULL);