From patchwork Thu Sep 23 11:22:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12512425 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFB24C433F5 for ; Thu, 23 Sep 2021 11:25:25 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8126061107 for ; Thu, 23 Sep 2021 11:25:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8126061107 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=grfs9RL245ofplHfiyVX+2b7oPV371e0moL+6cxd2Ro=; b=HTqKgpBpv8FMRM 3HlPtSsP+LnQ/JO7kVfyfskuyzfqCBK05NgI+POcEUbk4vt5xdrUQbPIpRfkiRFFZMEef8PdiGKWz TiV7tEikMJDYGqTKEp98Xp4H8FKiF6AAIC/h9HopI6Dhx8Dz+AZHo1MrJvRGSdRvko+PjklkylxVp z4kj0Ym3+uFii/OW8pmtm1XzLQgfpugT0QSnWMPPzPPuUaIJ4evwae1uzE5j6kQix/yDqWDdPMb68 +MG5jxUf+8q3B8h6FKBQ8wZbejB4bY2jx0/Q7jb1FbHg79AkURG66193I3FFaY1p2ishMrcWunMms F2V0q/Zv06NEjyLB/8gg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTMov-00B4pX-BT; Thu, 23 Sep 2021 11:23:17 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTMoi-00B4mK-MQ for linux-arm-kernel@lists.infradead.org; Thu, 23 Sep 2021 11:23:06 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 0B855611B0; Thu, 23 Sep 2021 11:23:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1632396184; bh=9V6D01rnhhu/XVUyi+hFbDZbhJ0tu8PwpuRqUVWmUbk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EUArm6thYy22n/hiDJOWbwy+k/vvSIYESSWeMzpw8oXHTN3jgmC3ELFh+Vix2NuX4 xzIJ/Tz+yhSa6aDPRLc0Cx7j26c0BSkTAxLwairyJOR5AcuR4zfji+YiMJs/sQRiMK kffRQRqRPcpL1GkIv4fR0/k9RPRAvSFS0Aa/F4UBsaq3WM36LoZmcwdYR/w4ulueR5 Opx0Zmr0K7nR+Gy56KtALZVbSB3frUtBUanV1lsMslv3QMW5ZRzCF/v1CpVMTzel1m 8tejstLQW7YsYlENUPXaHbWjL6zvbS6C+UDaexUdYnJQKPKFQ8sQ9z54H+8FLx7Pw+ 275YAW0PuT3Lg== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Quentin Perret , Catalin Marinas , Alexandru Elisei , Suzuki K Poulose , kvmarm@lists.cs.columbia.edu Subject: [PATCH 1/5] arm64: Prevent kexec and hibernation if is_protected_kvm_enabled() Date: Thu, 23 Sep 2021 12:22:52 +0100 Message-Id: <20210923112256.15767-2-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210923112256.15767-1-will@kernel.org> References: <20210923112256.15767-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210923_042304_814988_13FF29AC X-CRM114-Status: GOOD ( 12.11 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When pKVM is enabled, the hypervisor code at EL2 and its data structures are inaccessible to the host kernel and cannot be torn down or replaced as this would defeat the integrity properies which pKVM aims to provide. Furthermore, the ABI between the host and EL2 is flexible and private to whatever the current implementation of KVM requires and so booting a new kernel with an old EL2 component is very likely to end in disaster. In preparation for uninstalling the hyp stub calls which are relied upon to reset EL2, disable kexec and hibernation in the host when protected KVM is enabled. Cc: Marc Zyngier Cc: Quentin Perret Signed-off-by: Will Deacon --- arch/arm64/kernel/smp.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 6f6ff072acbd..44369b99a57e 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -1128,5 +1128,6 @@ bool cpus_are_stuck_in_kernel(void) { bool smp_spin_tables = (num_possible_cpus() > 1 && !have_cpu_die()); - return !!cpus_stuck_in_kernel || smp_spin_tables; + return !!cpus_stuck_in_kernel || smp_spin_tables || + is_protected_kvm_enabled(); } From patchwork Thu Sep 23 11:22:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12512427 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4ED18C433EF for ; Thu, 23 Sep 2021 11:25:41 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1C18C6109E for ; Thu, 23 Sep 2021 11:25:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1C18C6109E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xK84clqQCT06qYdTwkL15YrMWTua1P5Bu+bd9gtAiMY=; b=V4GiZIfLR/X4Rg ZGPDyt2u+X9BalER/hqroPJSaMVfg2sKaTqoHLv7sMKYyNpqqSGMx0D3LemRuYVD+XtFR3yH5xpCq hkRPrbDvE/uzKIdqaXMCRHXbpxJNnw75B2olylUPwQdrVNTyp4SG8udhco9oYKjAWTERoIy89zIbo ZxNfcy8D7hl/UDZvoRu28rgzbUxzMQWzCcXTosvQlhTuAvugmIjrqwCKrG55tARf/PNJ6wH0Oi+vL vpf8iRi68eUuf34xfrhmWuVBPFZ79uTnhZxrmGnXy2WhdwgbIcAaQIYKmu41LMaS8De07UQV5LB21 WCZK5pGCFPC3tKQ42f6g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTMp5-00B4r4-Sa; Thu, 23 Sep 2021 11:23:28 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTMok-00B4nA-Hm for linux-arm-kernel@lists.infradead.org; Thu, 23 Sep 2021 11:23:08 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id D220061241; Thu, 23 Sep 2021 11:23:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1632396186; bh=/zJUSJKXZe/aGPOpfIYreLSK8t3hSTo2hsTEZp3Ikqk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bnqY0KwUADqS8/Np/UzJ+g2TMa8/7N+uBeREmHHeqa3yqKJysd/0GDn8C6TFnvZPx QwWRM5eFl0tAhYQumLns7Ngq4pqviU/d880hB3Ez+gQcV1zMfrdsjLp9rVYB4MX5Qr gNYJjMk8fGZj8T5TFcslxQpPk/M4HduVW1NZGGWTWzPZZqSxzHRapu9QQcNXWgDIX2 wtGHKo+b/U3IWMIjFUCgPyML7godbRP6Ra0urH6lCrlAn4qNHag88YN9hMQhFIVVd1 uGTdQ/dvU0fsQhRHBfRhemQ2Qg2QcVN3qg2QyV2teMvBGgOn70H6QmuxqOfGvR6pim Q37kDgzlo4LIg== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Quentin Perret , Catalin Marinas , Alexandru Elisei , Suzuki K Poulose , kvmarm@lists.cs.columbia.edu Subject: [PATCH 2/5] KVM: arm64: Reject stub hypercalls after pKVM has been initialised Date: Thu, 23 Sep 2021 12:22:53 +0100 Message-Id: <20210923112256.15767-3-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210923112256.15767-1-will@kernel.org> References: <20210923112256.15767-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210923_042306_667179_411B8D6E X-CRM114-Status: GOOD ( 12.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The stub hypercalls provide mechanisms to reset and replace the EL2 code, so uninstall them once pKVM has been initialised in order to ensure the integrity of the hypervisor code. To ensure pKVM initialisation remains functional, split cpu_hyp_reinit() into two helper functions to separate usage of the stub from usage of pkvm hypercalls either side of __pkvm_init on the boot CPU. Cc: Marc Zyngier Cc: Quentin Perret Signed-off-by: Will Deacon Reviewed-by: Quentin Perret --- arch/arm64/kvm/arm.c | 31 +++++++++++++++++++++++-------- arch/arm64/kvm/hyp/nvhe/host.S | 26 +++++++++++++++++--------- 2 files changed, 40 insertions(+), 17 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index fe102cd2e518..9506cf88fa0e 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1579,25 +1579,33 @@ static void cpu_set_hyp_vector(void) kvm_call_hyp_nvhe(__pkvm_cpu_set_vector, data->slot); } -static void cpu_hyp_reinit(void) +static void cpu_hyp_init_context(void) { kvm_init_host_cpu_context(&this_cpu_ptr_hyp_sym(kvm_host_data)->host_ctxt); - cpu_hyp_reset(); - - if (is_kernel_in_hyp_mode()) - kvm_timer_init_vhe(); - else + if (!is_kernel_in_hyp_mode()) cpu_init_hyp_mode(); +} +static void cpu_hyp_init_features(void) +{ cpu_set_hyp_vector(); - kvm_arm_init_debug(); + if (is_kernel_in_hyp_mode()) + kvm_timer_init_vhe(); + if (vgic_present) kvm_vgic_init_cpu_hardware(); } +static void cpu_hyp_reinit(void) +{ + cpu_hyp_reset(); + cpu_hyp_init_context(); + cpu_hyp_init_features(); +} + static void _kvm_arch_hardware_enable(void *discard) { if (!__this_cpu_read(kvm_arm_hardware_enabled)) { @@ -1788,10 +1796,17 @@ static int do_pkvm_init(u32 hyp_va_bits) int ret; preempt_disable(); - hyp_install_host_vector(); + cpu_hyp_init_context(); ret = kvm_call_hyp_nvhe(__pkvm_init, hyp_mem_base, hyp_mem_size, num_possible_cpus(), kern_hyp_va(per_cpu_base), hyp_va_bits); + cpu_hyp_init_features(); + + /* + * The stub hypercalls are now disabled, so set our local flag to + * prevent a later re-init attempt in kvm_arch_hardware_enable(). + */ + __this_cpu_write(kvm_arm_hardware_enabled, 1); preempt_enable(); return ret; diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 4b652ffb591d..0c6116d34e18 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -110,17 +110,14 @@ SYM_FUNC_START(__hyp_do_panic) b __host_enter_for_panic SYM_FUNC_END(__hyp_do_panic) -.macro host_el1_sync_vect - .align 7 -.L__vect_start\@: - stp x0, x1, [sp, #-16]! - mrs x0, esr_el2 - lsr x0, x0, #ESR_ELx_EC_SHIFT - cmp x0, #ESR_ELx_EC_HVC64 - b.ne __host_exit - +SYM_FUNC_START(__host_hvc) ldp x0, x1, [sp] // Don't fixup the stack yet + /* No stub for you, sonny Jim */ +alternative_if ARM64_KVM_PROTECTED_MODE + b __host_exit +alternative_else_nop_endif + /* Check for a stub HVC call */ cmp x0, #HVC_STUB_HCALL_NR b.hs __host_exit @@ -137,6 +134,17 @@ SYM_FUNC_END(__hyp_do_panic) ldr x5, =__kvm_handle_stub_hvc hyp_pa x5, x6 br x5 +SYM_FUNC_END(__host_hvc) + +.macro host_el1_sync_vect + .align 7 +.L__vect_start\@: + stp x0, x1, [sp, #-16]! + mrs x0, esr_el2 + lsr x0, x0, #ESR_ELx_EC_SHIFT + cmp x0, #ESR_ELx_EC_HVC64 + b.eq __host_hvc + b __host_exit .L__vect_end\@: .if ((.L__vect_end\@ - .L__vect_start\@) > 0x80) .error "host_el1_sync_vect larger than vector entry" From patchwork Thu Sep 23 11:22:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12512429 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3BE9C433EF for ; Thu, 23 Sep 2021 11:25:47 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B909760EC0 for ; Thu, 23 Sep 2021 11:25:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B909760EC0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=jkf/vywvb4X4ksV0cu0OluTOT+Hww6CGqZI7U0n+rqU=; b=GTNaaT1nrU02dF cSNh3uscAAGhVV4PU2l63h4TVnNNzGZTxvPVDjbAFmcymwxcgInh1cJYfEC01WuyPQlKwAdKHIFyf mGSthegovd8xuvK9Y38X9KW/OSz6c8u4tmP/t5vQJxt6JDnNISAbqbW/XkntZyE6Q3+EFVaIRjte/ zH0TwM/yT5PaqRr3P5SkJp1J9xofSCKWA7AGyncBTScPBN6i2wBr70GJV41gw00eVGRHW6S0EmzWy 8y8im4U9M7Oy9J4gdtf/dyNoQle2Qv+uZ15X6m0oB6u4TKkgg2rcKQmhoYW6sbYA1gIFFG+2QoNZh Cmg93bf31vQRJSkyJYbg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTMpH-00B4u2-EJ; Thu, 23 Sep 2021 11:23:39 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTMom-00B4nl-Bl for linux-arm-kernel@lists.infradead.org; Thu, 23 Sep 2021 11:23:09 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id A39B560EC0; Thu, 23 Sep 2021 11:23:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1632396188; bh=zTSEoixQnsN3jbLczk6zfspLIXgI2IJ8ZyDChfPcm6s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fFSg2eCfq7DlcK2X8lz+o8qpNPlQZb3vzjvDAK+6e4fjl+ZAgtnud90Mb5L+VKdU2 ffXHb/OEgiKumuztfDLE8gILHTcBs/Dq1pLEF2JB8ARcrg3pH80oyIbHDCzv42lfIB DZ+7C7MGSrgy6BGo8wrhXY+fvBfy7USZgHc6rDSVzxIb3cAk5k0RZJDBTUmw3WzJpy VxKVMMz38G/WBcy+6GBmcK0asTZoojzV90e92cWk9q2iqQzBG/kqtSEBzqYEKbEAuu wSX7TXUtDrRVLRhCesU3nGN+XEJ2i64KpmUyxp403TqmGFtv7NF+ITAdesmWU6tBN9 TcvWAT1j/2n6Q== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Quentin Perret , Catalin Marinas , Alexandru Elisei , Suzuki K Poulose , kvmarm@lists.cs.columbia.edu Subject: [PATCH 3/5] KVM: arm64: Propagate errors from __pkvm_prot_finalize hypercall Date: Thu, 23 Sep 2021 12:22:54 +0100 Message-Id: <20210923112256.15767-4-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210923112256.15767-1-will@kernel.org> References: <20210923112256.15767-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210923_042308_460992_6372B80C X-CRM114-Status: GOOD ( 13.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If the __pkvm_prot_finalize hypercall returns an error, we WARN but fail to propagate the failure code back to kvm_arch_init(). Pass a pointer to a zero-initialised return variable so that failure to finalise the pKVM protections on a host CPU can be reported back to KVM. Cc: Marc Zyngier Cc: Quentin Perret Signed-off-by: Will Deacon --- arch/arm64/kvm/arm.c | 30 +++++++++++++++++++----------- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 9506cf88fa0e..13bbf35896cd 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1986,9 +1986,25 @@ static int init_hyp_mode(void) return err; } -static void _kvm_host_prot_finalize(void *discard) +static void _kvm_host_prot_finalize(void *arg) { - WARN_ON(kvm_call_hyp_nvhe(__pkvm_prot_finalize)); + int *err = arg; + + if (WARN_ON(kvm_call_hyp_nvhe(__pkvm_prot_finalize))) + WRITE_ONCE(*err, -EINVAL); +} + +static int pkvm_drop_host_privileges(void) +{ + int ret = 0; + + /* + * Flip the static key upfront as that may no longer be possible + * once the host stage 2 is installed. + */ + static_branch_enable(&kvm_protected_mode_initialized); + on_each_cpu(_kvm_host_prot_finalize, &ret, 1); + return ret; } static int finalize_hyp_mode(void) @@ -2002,15 +2018,7 @@ static int finalize_hyp_mode(void) * None of other sections should ever be introspected. */ kmemleak_free_part(__hyp_bss_start, __hyp_bss_end - __hyp_bss_start); - - /* - * Flip the static key upfront as that may no longer be possible - * once the host stage 2 is installed. - */ - static_branch_enable(&kvm_protected_mode_initialized); - on_each_cpu(_kvm_host_prot_finalize, NULL, 1); - - return 0; + return pkvm_drop_host_privileges(); } struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr) From patchwork Thu Sep 23 11:22:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12512433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C96CC433EF for ; Thu, 23 Sep 2021 11:26:17 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2CCBC60EC0 for ; Thu, 23 Sep 2021 11:26:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2CCBC60EC0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=0KNNBZr5+4dI1IlbCgkd84qRKWPoZKu/a2rBJLpYzbA=; b=fGnoqvoTJbcpEi BgWyg7OPY7ywmTKV6tNQy4lIoVQEYb7A6OeF2NvJqjEjlEB8HpqbspTHyNUPbHkgPJ1ftdHQ9EjnM 1LbjCjwF3+vNLEeTQjIqaKmD4PmWpHMn1M86LbYwyNTFoWAaNoDBDRfxDv7sdkVQ2OB8hse38AFjM IOOqq66eKZQP8KG5peZbz7iW2itb+PUA79lVvvv2uSEqmqhRbjFzcgF2x7JDM/eo6cvlEqIMl3fUF nr4TsukXZoF3sVL8o19CRTm61OL01uzmOfJhlVZ7F56fe5BTYs6ErYfdEf0TG4UcpWS5WL4bNLO4h pAt9qcXmZL7IShC2m7SA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTMpS-00B4yK-Sx; Thu, 23 Sep 2021 11:23:51 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTMoo-00B4oC-6H for linux-arm-kernel@lists.infradead.org; Thu, 23 Sep 2021 11:23:11 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 75BEF6121F; Thu, 23 Sep 2021 11:23:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1632396189; bh=9XTol94Ae+OoVJE6oULI43J/aM3k1N+7z3owBAICrso=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RXMy8DYZ5qgn6LaLVqLH062rIFiZAAAwxgsAQJg2dkYGFmW+c9v1zKYJNJEL8Ncl6 /O7Pls6/YD7T2VtlghGoupJg3Vx6DXNs0TTtJ4cYKdvL7SCYHnDliF7m8sDXKIyIKz cfrqah6slI/Ob10r5teEu3UdmaEf97SwgYO0U0+r9qn0OsC/Xe4zZZL45/l2/toZim mbNN+6Tm3zVI6qPrdZUmtUysR76X9+j1RrZXRZ73BdjjJSEv64raFc8syVCHiytAwW hxjA0/AzL29OtgeAXGFmg5yNThvwhDSxpdlmangTe8fuVsf5zcFu5OxTGh6H7sfryn zc82FT5+misCQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Quentin Perret , Catalin Marinas , Alexandru Elisei , Suzuki K Poulose , kvmarm@lists.cs.columbia.edu Subject: [PATCH 4/5] KVM: arm64: Prevent re-finalisation of pKVM for a given CPU Date: Thu, 23 Sep 2021 12:22:55 +0100 Message-Id: <20210923112256.15767-5-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210923112256.15767-1-will@kernel.org> References: <20210923112256.15767-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210923_042310_292819_A4ADA203 X-CRM114-Status: GOOD ( 10.96 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org __pkvm_prot_finalize() completes the deprivilege of the host when pKVM is in use by installing a stage-2 translation table for the calling CPU. Issuing the hypercall multiple times for a given CPU makes little sense, but in such a case just return early with -EPERM rather than go through the whole page-table dance again. Cc: Marc Zyngier Cc: Quentin Perret Signed-off-by: Will Deacon Reviewed-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index bacd493a4eac..cafe17e5fa8f 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -123,6 +123,9 @@ int __pkvm_prot_finalize(void) struct kvm_s2_mmu *mmu = &host_kvm.arch.mmu; struct kvm_nvhe_init_params *params = this_cpu_ptr(&kvm_init_params); + if (params->hcr_el2 & HCR_VM) + return -EPERM; + params->vttbr = kvm_get_vttbr(mmu); params->vtcr = host_kvm.arch.vtcr; params->hcr_el2 |= HCR_VM; From patchwork Thu Sep 23 11:22:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12512431 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47719C433F5 for ; Thu, 23 Sep 2021 11:26:14 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0AE1660EC0 for ; Thu, 23 Sep 2021 11:26:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0AE1660EC0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZecFl0pGNmO2fk6c7k1Ag+yotzEEgqg2EvRvLzjg38M=; b=VjeFzf0qMO9lY5 6BOoSohblV7gj5CRZ70DYKHGNuiq0eMExiHd+4SdU8zOaJByoGlCSnJX38nBQ2Vd5ViZYDnGGU7HU OrLkNUB8vp9ewanhev1OVAocIWbnnCsqTTVvBvz8QspknCb/YdP8WX45vMAEE7ucQjFMqvFtOw0bp h1DxXeUG47gYNuXy53eYYm+Sxl7oWUi3VIBuoHCUYV226oPwYjwMBI1FofwbqXEtil59ZrH/2nW4e mc2N1/ovUbUWeUSshfRe5tohJhKI5P4LzmwsnSbGYcIJHtZWsDQp6j/4G/b7eOV9xXT+kuotntQRW B8nuqJJNR8mc/RUCpUlA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTMpf-00B53q-SE; Thu, 23 Sep 2021 11:24:04 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTMop-00B4ok-Ui for linux-arm-kernel@lists.infradead.org; Thu, 23 Sep 2021 11:23:13 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 48B1B611C6; Thu, 23 Sep 2021 11:23:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1632396191; bh=DMbPTWIl8l89NmtqiofRXVDto4/SgvGfTeCv5+IxVA4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=T/OpiqDkB5iYJpHJqk/+B4abVTRPILrHLDdWJlryB/ClPzMU8CK+I3YHhaCoZzZn0 8qXgkRZ2gI27nc/uMjEU8Bzk5CTBt+jCt3FErFGARvy+26PJC6lkc+JdandGbF7FI1 55DSPiiTxlgiJEwhxcSDbvwCgvfHEDZPcWJzdzueB7FJzFwNmmBUV6oe1vX9xLefdT 0WII+ELVsQ4ViateeWNJznqiPcG11hwsB9LD07NHqDpQTYLBM74hhE9b6OJ0Zfm339 nyQdoeJhQjJHyLT42dNT+oZUGXZ7s2jo8T7VR+w7YaIJ/AZOHsuxZAG8qNjX4EW6rg o21VPAp7L6n3w== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Quentin Perret , Catalin Marinas , Alexandru Elisei , Suzuki K Poulose , kvmarm@lists.cs.columbia.edu Subject: [PATCH 5/5] KVM: arm64: Disable privileged hypercalls after pKVM finalisation Date: Thu, 23 Sep 2021 12:22:56 +0100 Message-Id: <20210923112256.15767-6-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210923112256.15767-1-will@kernel.org> References: <20210923112256.15767-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210923_042312_066066_77F391F3 X-CRM114-Status: GOOD ( 13.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org After pKVM has been 'finalised' using the __pkvm_prot_finalize hypercall, the calling CPU will have a Stage-2 translation enabled to prevent access to memory pages owned by EL2. Although this forms a significant part of the process to deprivilege the host kernel, we also need to ensure that the hypercall interface is reduced so that the EL2 code cannot, for example, be re-initialised using a new set of vectors. Re-order the hypercalls so that only a suffix remains available after finalisation of pKVM. Cc: Marc Zyngier Cc: Quentin Perret Signed-off-by: Will Deacon --- arch/arm64/include/asm/kvm_asm.h | 43 ++++++++++++++++-------------- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 26 +++++++++++------- 2 files changed, 39 insertions(+), 30 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index e86045ac43ba..68630fd382c5 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -43,27 +43,30 @@ #define KVM_HOST_SMCCC_FUNC(name) KVM_HOST_SMCCC_ID(__KVM_HOST_SMCCC_FUNC_##name) +/* Hypercalls available only prior to pKVM finalisation */ #define __KVM_HOST_SMCCC_FUNC___kvm_hyp_init 0 -#define __KVM_HOST_SMCCC_FUNC___kvm_vcpu_run 1 -#define __KVM_HOST_SMCCC_FUNC___kvm_flush_vm_context 2 -#define __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid_ipa 3 -#define __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid 4 -#define __KVM_HOST_SMCCC_FUNC___kvm_flush_cpu_context 5 -#define __KVM_HOST_SMCCC_FUNC___kvm_timer_set_cntvoff 6 -#define __KVM_HOST_SMCCC_FUNC___kvm_enable_ssbs 7 -#define __KVM_HOST_SMCCC_FUNC___vgic_v3_get_gic_config 8 -#define __KVM_HOST_SMCCC_FUNC___vgic_v3_read_vmcr 9 -#define __KVM_HOST_SMCCC_FUNC___vgic_v3_write_vmcr 10 -#define __KVM_HOST_SMCCC_FUNC___vgic_v3_init_lrs 11 -#define __KVM_HOST_SMCCC_FUNC___kvm_get_mdcr_el2 12 -#define __KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs 13 -#define __KVM_HOST_SMCCC_FUNC___vgic_v3_restore_aprs 14 -#define __KVM_HOST_SMCCC_FUNC___pkvm_init 15 -#define __KVM_HOST_SMCCC_FUNC___pkvm_host_share_hyp 16 -#define __KVM_HOST_SMCCC_FUNC___pkvm_create_private_mapping 17 -#define __KVM_HOST_SMCCC_FUNC___pkvm_cpu_set_vector 18 -#define __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize 19 -#define __KVM_HOST_SMCCC_FUNC___kvm_adjust_pc 20 +#define __KVM_HOST_SMCCC_FUNC___kvm_get_mdcr_el2 1 +#define __KVM_HOST_SMCCC_FUNC___pkvm_init 2 +#define __KVM_HOST_SMCCC_FUNC___pkvm_create_private_mapping 3 +#define __KVM_HOST_SMCCC_FUNC___pkvm_cpu_set_vector 4 +#define __KVM_HOST_SMCCC_FUNC___kvm_enable_ssbs 5 +#define __KVM_HOST_SMCCC_FUNC___vgic_v3_init_lrs 6 +#define __KVM_HOST_SMCCC_FUNC___vgic_v3_get_gic_config 7 +#define __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize 8 + +/* Hypercalls available after pKVM finalisation */ +#define __KVM_HOST_SMCCC_FUNC___pkvm_host_share_hyp 9 +#define __KVM_HOST_SMCCC_FUNC___kvm_adjust_pc 10 +#define __KVM_HOST_SMCCC_FUNC___kvm_vcpu_run 11 +#define __KVM_HOST_SMCCC_FUNC___kvm_flush_vm_context 12 +#define __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid_ipa 13 +#define __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid 14 +#define __KVM_HOST_SMCCC_FUNC___kvm_flush_cpu_context 15 +#define __KVM_HOST_SMCCC_FUNC___kvm_timer_set_cntvoff 16 +#define __KVM_HOST_SMCCC_FUNC___vgic_v3_read_vmcr 17 +#define __KVM_HOST_SMCCC_FUNC___vgic_v3_write_vmcr 18 +#define __KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs 19 +#define __KVM_HOST_SMCCC_FUNC___vgic_v3_restore_aprs 20 #ifndef __ASSEMBLY__ diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 2da6aa8da868..4120e34288e1 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -165,36 +165,42 @@ typedef void (*hcall_t)(struct kvm_cpu_context *); #define HANDLE_FUNC(x) [__KVM_HOST_SMCCC_FUNC_##x] = (hcall_t)handle_##x static const hcall_t host_hcall[] = { - HANDLE_FUNC(__kvm_vcpu_run), + /* ___kvm_hyp_init */ + HANDLE_FUNC(__kvm_get_mdcr_el2), + HANDLE_FUNC(__pkvm_init), + HANDLE_FUNC(__pkvm_create_private_mapping), + HANDLE_FUNC(__pkvm_cpu_set_vector), + HANDLE_FUNC(__kvm_enable_ssbs), + HANDLE_FUNC(__vgic_v3_init_lrs), + HANDLE_FUNC(__pkvm_prot_finalize), + + HANDLE_FUNC(__pkvm_host_share_hyp), HANDLE_FUNC(__kvm_adjust_pc), + HANDLE_FUNC(__kvm_vcpu_run), HANDLE_FUNC(__kvm_flush_vm_context), HANDLE_FUNC(__kvm_tlb_flush_vmid_ipa), HANDLE_FUNC(__kvm_tlb_flush_vmid), HANDLE_FUNC(__kvm_flush_cpu_context), HANDLE_FUNC(__kvm_timer_set_cntvoff), - HANDLE_FUNC(__kvm_enable_ssbs), HANDLE_FUNC(__vgic_v3_get_gic_config), HANDLE_FUNC(__vgic_v3_read_vmcr), HANDLE_FUNC(__vgic_v3_write_vmcr), - HANDLE_FUNC(__vgic_v3_init_lrs), - HANDLE_FUNC(__kvm_get_mdcr_el2), HANDLE_FUNC(__vgic_v3_save_aprs), HANDLE_FUNC(__vgic_v3_restore_aprs), - HANDLE_FUNC(__pkvm_init), - HANDLE_FUNC(__pkvm_cpu_set_vector), - HANDLE_FUNC(__pkvm_host_share_hyp), - HANDLE_FUNC(__pkvm_create_private_mapping), - HANDLE_FUNC(__pkvm_prot_finalize), }; static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) { DECLARE_REG(unsigned long, id, host_ctxt, 0); + unsigned long hcall_min = 0; hcall_t hfn; + if (static_branch_unlikely(&kvm_protected_mode_initialized)) + hcall_min = __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize; + id -= KVM_HOST_SMCCC_ID(0); - if (unlikely(id >= ARRAY_SIZE(host_hcall))) + if (unlikely(id < hcall_min || id >= ARRAY_SIZE(host_hcall))) goto inval; hfn = host_hcall[id];