From patchwork Tue Sep 15 10:46:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Scull X-Patchwork-Id: 11776163 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 764E8618 for ; Tue, 15 Sep 2020 11:04:54 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 066AE21D1B for ; Tue, 15 Sep 2020 11:04:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="VAIxMn0B"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="vkb8dCDx" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 066AE21D1B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=5/s7GAnzY60GkldoNZ0HhnT3wdjTkqT1oJE32a/yI2I=; b=VAIxMn0B8WNK23X1ZSLaSFw52 MObpxyqYH3El83ns5DhoMjD34Hk7eaV7zww0tMWxVmFsKm/69zh3sgyfrxT3mfnZzE9rCcq+dq/Pg Kcpec3OEPJCGRm6qESUdbDCkt1IO97YuaUPsqSLlELsmYUg3tbIMeSc+iCiMW+LK1oQ4Cg68KE13O V8H3+8q7CpizI5pjLQSl+9HYxrOGgBUFkYxfFxFmWlGgi1HWdD+Tc0qUYaRLhflqHIuV6TWzeMbN1 T3iGSd7DeXPss7Rdt4gK+daTKZdvA0dCtIgEO4OfZDJtNN3IWQDkGEVBpodJ7Lx1dboqxc82hyIHW G+BE4s0Ig==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kI8iP-0006zJ-I1; Tue, 15 Sep 2020 11:01:41 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kI8Uf-0000xe-Bp for linux-arm-kernel@lists.infradead.org; Tue, 15 Sep 2020 10:47:31 +0000 Received: by mail-wm1-x34a.google.com with SMTP id x81so750524wmg.8 for ; Tue, 15 Sep 2020 03:47:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=W4m4qc+EQLpHs67vf0vC0C6t2vUo/yMuKUjWuD1dzP4=; b=vkb8dCDxF0dj3JJEqsXL7UajQpnNr8wsg0c79WMVui0sS7+V2cFOaQb7mjaEb+C5l9 NJ99S1i2/tXzW6SMK974SHQd30erNE9dLRyEWl1xVKDHRQMSWhUEVFpUnLi+cmCEA+f1 Y4Rpd9GheGW+f7ZCvDQ5G3UNJQUiokmn7GsfZfN5De9MLRakU+A4DcntxFLgZb/s8vWp 2lfEi9YGRhbmluGMMh1yTDNGbUPyW6RmHxGphcteqx48R/El3MUga+V/YgzOo5NMBn70 OCUTSAfAsI5JCwb8rGmlg7NqjmDk0Q+H47cgSCuIbs8kSC5D5t6zWZ9rm0UfLLVGnoqc KkGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=W4m4qc+EQLpHs67vf0vC0C6t2vUo/yMuKUjWuD1dzP4=; b=sJQoIsnjROYlebVl2EECtwOTdWt+2f0f46ZX7wbCbHpTj7FUyBtuQYhk1+oR6PWEZT 8uDpHcFUhD9yq4Wd108hLTCosEcQVDJk63VKxNdxg+/+WmQFlcIlwewWDxZToBPKOL/U b6K9ZcvAlg4TmFKUqgUVI8tdN4nde0qHFv4XZGCw72gzFvEeDFtnzdzpusoL/LmbrB9A Ta0mDnjNbzltClClO8Mjhgd+ImDlA43CqU5fhcrsin7+IRkAKhBp9fxl0Iea0VBNTslZ /20UDNVEspnLNYd775jAWWMTyMUlDb7sI4dpXn+ydzheR/X+D/MohEP4ddfyg6VmtXl5 mlLA== X-Gm-Message-State: AOAM530IHPIaSkK9enVivG5ZE7AP6a9vYyR5/Q8KDuWn4RWB5BTcB8W+ jl91AO2Oa6+4ZKPjo3Y5EsJf1rZbLrw= X-Google-Smtp-Source: ABdhPJyRFghnAGEo8UmrwUDpt0SKWSxTfv9Lp1d+mBRB0auusfqJBwQR0IcpvKR0uOssEeWzoQpGZHfKrkQ= X-Received: from ascull.lon.corp.google.com ([2a00:79e0:d:109:4a0f:cfff:fe4a:6363]) (user=ascull job=sendgmr) by 2002:a1c:7f14:: with SMTP id a20mr4133460wmd.95.1600166837701; Tue, 15 Sep 2020 03:47:17 -0700 (PDT) Date: Tue, 15 Sep 2020 11:46:37 +0100 In-Reply-To: <20200915104643.2543892-1-ascull@google.com> Message-Id: <20200915104643.2543892-14-ascull@google.com> Mime-Version: 1.0 References: <20200915104643.2543892-1-ascull@google.com> X-Mailer: git-send-email 2.28.0.618.gf4bc123cb7-goog Subject: [PATCH v4 13/19] KVM: arm64: nVHE: Handle hyp panics From: Andrew Scull To: kvmarm@lists.cs.columbia.edu X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200915_064725_677995_AB94C4D7 X-CRM114-Status: GOOD ( 19.42 ) X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:34a listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arm-kernel@lists.infradead.org, kernel-team@android.com, suzuki.poulose@arm.com, maz@kernel.org, Sudeep Holla , james.morse@arm.com, Andrew Scull , catalin.marinas@arm.com, will@kernel.org, julien.thierry.kdev@gmail.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Restore the host context when panicking from hyp to give the best chance of the panic being clean. The host requires that registers be preserved such as x18 for the shadow callstack. If the panic is caused by an exception from EL1, the host context is still valid so the panic can return straight back to the host. If the panic comes from EL2 then it's most likely that the hyp context is active and the host context needs to be restored. There are windows before and after the host context is saved and restored that restoration is attempted incorrectly and the panic won't be clean. Signed-off-by: Andrew Scull --- arch/arm64/include/asm/kvm_hyp.h | 2 +- arch/arm64/kvm/hyp/nvhe/host.S | 78 ++++++++++++++++++++++++-------- arch/arm64/kvm/hyp/nvhe/switch.c | 18 ++------ 3 files changed, 64 insertions(+), 34 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 0b525e05e5bf..6b664de5ec1f 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -94,7 +94,7 @@ u64 __guest_enter(struct kvm_vcpu *vcpu); void __noreturn hyp_panic(void); #ifdef __KVM_NVHE_HYPERVISOR__ -void __noreturn __hyp_do_panic(unsigned long, ...); +void __noreturn __hyp_do_panic(bool restore_host, u64 spsr, u64 elr, u64 par); #endif #endif /* __ARM64_KVM_HYP_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index d26e41773dc4..ff9a0f547b9f 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -47,6 +47,9 @@ SYM_FUNC_START(__host_exit) ldp x2, x3, [x29, #CPU_XREG_OFFSET(2)] ldp x4, x5, [x29, #CPU_XREG_OFFSET(4)] ldp x6, x7, [x29, #CPU_XREG_OFFSET(6)] + + /* x0-7 are use for panic arguments */ +__host_enter_for_panic: ldp x8, x9, [x29, #CPU_XREG_OFFSET(8)] ldp x10, x11, [x29, #CPU_XREG_OFFSET(10)] ldp x12, x13, [x29, #CPU_XREG_OFFSET(12)] @@ -57,18 +60,38 @@ SYM_FUNC_START(__host_exit) restore_callee_saved_regs x29 /* Do not touch any register after this! */ +__host_enter_without_restoring: eret sb SYM_FUNC_END(__host_exit) +/* + * void __noreturn __hyp_do_panic(bool restore_host, u64 spsr, u64 elr, u64 par); + */ SYM_FUNC_START(__hyp_do_panic) + /* Load the format arguments into x1-7 */ + mov x6, x3 + get_vcpu_ptr x7, x3 + + mrs x3, esr_el2 + mrs x4, far_el2 + mrs x5, hpfar_el2 + + /* Prepare and exit to the host's panic funciton. */ mov lr, #(PSR_F_BIT | PSR_I_BIT | PSR_A_BIT | PSR_D_BIT |\ PSR_MODE_EL1h) msr spsr_el2, lr ldr lr, =panic msr elr_el2, lr - eret - sb + + /* + * Set the panic format string and enter the host, conditionally + * restoring the host context. + */ + cmp x0, xzr + ldr x0, =__hyp_panic_string + b.eq __host_enter_without_restoring + b __host_enter_for_panic SYM_FUNC_END(__hyp_do_panic) .macro host_el1_sync_vect @@ -79,7 +102,7 @@ SYM_FUNC_END(__hyp_do_panic) lsr x0, x0, #ESR_ELx_EC_SHIFT cmp x0, #ESR_ELx_EC_HVC64 ldp x0, x1, [sp], #16 - b.ne hyp_panic + b.ne __host_exit /* Check for a stub HVC call */ cmp x0, #HVC_STUB_HCALL_NR @@ -105,16 +128,31 @@ SYM_FUNC_END(__hyp_do_panic) .endif .endm -.macro invalid_host_vect +.macro invalid_host_el2_vect .align 7 /* If a guest is loaded, panic out of it. */ stp x0, x1, [sp, #-16]! get_loaded_vcpu x0, x1 cbnz x0, __guest_exit_panic add sp, sp, #16 + + /* + * The panic may not be clean if the exception is taken before the host + * context has been saved by __host_exit or after the hyp context has + * been partially clobbered by __host_enter. + */ b hyp_panic .endm +.macro invalid_host_el1_vect + .align 7 + mov x0, xzr /* restore_host = false */ + mrs x1, spsr_el2 + mrs x2, elr_el2 + mrs x3, par_el1 + b __hyp_do_panic +.endm + /* * The host vector does not use an ESB instruction in order to avoid consuming * SErrors that should only be consumed by the host. Guest entry is deferred by @@ -127,23 +165,23 @@ SYM_FUNC_END(__hyp_do_panic) */ .align 11 SYM_CODE_START(__kvm_hyp_host_vector) - invalid_host_vect // Synchronous EL2t - invalid_host_vect // IRQ EL2t - invalid_host_vect // FIQ EL2t - invalid_host_vect // Error EL2t + invalid_host_el2_vect // Synchronous EL2t + invalid_host_el2_vect // IRQ EL2t + invalid_host_el2_vect // FIQ EL2t + invalid_host_el2_vect // Error EL2t - invalid_host_vect // Synchronous EL2h - invalid_host_vect // IRQ EL2h - invalid_host_vect // FIQ EL2h - invalid_host_vect // Error EL2h + invalid_host_el2_vect // Synchronous EL2h + invalid_host_el2_vect // IRQ EL2h + invalid_host_el2_vect // FIQ EL2h + invalid_host_el2_vect // Error EL2h host_el1_sync_vect // Synchronous 64-bit EL1 - invalid_host_vect // IRQ 64-bit EL1 - invalid_host_vect // FIQ 64-bit EL1 - invalid_host_vect // Error 64-bit EL1 - - invalid_host_vect // Synchronous 32-bit EL1 - invalid_host_vect // IRQ 32-bit EL1 - invalid_host_vect // FIQ 32-bit EL1 - invalid_host_vect // Error 32-bit EL1 + invalid_host_el1_vect // IRQ 64-bit EL1 + invalid_host_el1_vect // FIQ 64-bit EL1 + invalid_host_el1_vect // Error 64-bit EL1 + + invalid_host_el1_vect // Synchronous 32-bit EL1 + invalid_host_el1_vect // IRQ 32-bit EL1 + invalid_host_el1_vect // FIQ 32-bit EL1 + invalid_host_el1_vect // Error 32-bit EL1 SYM_CODE_END(__kvm_hyp_host_vector) diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index c99945cda779..29febf9a93f2 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -242,6 +242,8 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) if (system_uses_irq_prio_masking()) gic_write_pmr(GIC_PRIO_IRQOFF); + host_ctxt->__hyp_running_vcpu = NULL; + return exit_code; } @@ -250,31 +252,21 @@ void __noreturn hyp_panic(void) u64 spsr = read_sysreg_el2(SYS_SPSR); u64 elr = read_sysreg_el2(SYS_ELR); u64 par = read_sysreg(par_el1); + bool restore_host = true; struct kvm_cpu_context *host_ctxt; struct kvm_vcpu *vcpu; - unsigned long str_va; host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; vcpu = host_ctxt->__hyp_running_vcpu; - if (read_sysreg(vttbr_el2)) { + if (vcpu) { __timer_disable_traps(vcpu); __deactivate_traps(vcpu); __load_host_stage2(); __sysreg_restore_state_nvhe(host_ctxt); } - /* - * Force the panic string to be loaded from the literal pool, - * making sure it is a kernel address and not a PC-relative - * reference. - */ - asm volatile("ldr %0, =%1" : "=r" (str_va) : "S" (__hyp_panic_string)); - - __hyp_do_panic(str_va, - spsr, elr, - read_sysreg(esr_el2), read_sysreg_el2(SYS_FAR), - read_sysreg(hpfar_el2), par, vcpu); + __hyp_do_panic(restore_host, spsr, elr, par); unreachable(); }