From patchwork Thu Mar 30 10:04:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13193759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0523CC6FD1D for ; Thu, 30 Mar 2023 10:05:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=mFpJlrf93BmzYsaEUz5eGx04oHAnqDcnrt1WSFGHuwE=; b=YL2EkDumAIcNjq oet4FU3Xj4roA5WYs0kQK7T9MG7Bes9SNvIC0xFOlWanX7dfpTlG+H3EfJwq627sLzStnUTDmHN+e ohNzEtwauZfW9RnojtSQ9Bacv73vDPsX/buBy2kESmvYxoqEl0YuidB7EDDMuGp2eFToPXC/clKnD BZwTilhPNaNty5bCblCOcXgJ7ltCbBV9azA3jZHeJdmgGgKk61Kx2C6O4vY9C3y28u23xMIdRyAaZ qOjd1PlZwbEJ6CHb1w8IJmtYq3PL7vvVXXcfBsn8bDC5AYH90ErxLO29/JDhF3nTZQ9jYtqp5xbRr stWYyN2cwhbX1VYXQ2/A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1php93-003OZd-0y; Thu, 30 Mar 2023 10:04:37 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1php8x-003OVG-1D for linux-arm-kernel@lists.infradead.org; Thu, 30 Mar 2023 10:04:35 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CF03861FCA; Thu, 30 Mar 2023 10:04:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 39AD2C433EF; Thu, 30 Mar 2023 10:04:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1680170670; bh=Wq/Jem/6t7SqUQQ2xM4SVVX7/2HHigaqn+8GvfAmjTg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tWTLm9rp1GzT20QSJSUEs6Mlcs1qUeTUHube/w5IqJO5xHllv+LZTtW6oNFipJYen rUXLHCWRYDRvaa4XiR2Pt55r0PjQFLMI/gf+kH8SRQ+rsOcEapptthraI9PLgtu47s JOW5MxM7Sx3Mu40ugenMizrHqeK8qxfVm0zkjvW7LS9avmkXD4pAYW8DWb7fhYp9Z7 jEuRKfuaWPZQeZ/+S827IRimGMzbNBX++SSZsPzGAAyF7YpGNm/tHHKbSyKU9mz8Yi BxJImwDZQRH0ZXtsmj1gn+wz9rOgb2DA7xriSYyCLD0hwcOgUbC2SjI53xYyGWBPvV Vt+pNVzTCOIjg== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1php8u-004K66-4t; Thu, 30 Mar 2023 11:04:28 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Will Deacon Subject: [PATCH 1/2] KVM: arm64: nvhe: Synchronise with page table walker on MMU update Date: Thu, 30 Mar 2023 11:04:18 +0100 Message-Id: <20230330100419.1436629-2-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230330100419.1436629-1-maz@kernel.org> References: <20230330100419.1436629-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, will@kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230330_030431_489660_E14C9A28 X-CRM114-Status: GOOD ( 20.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When taking an exception between the EL1&0 translation regime and the EL2 translation regime, the page table walker is allowed to complete the walks started from EL0 or EL1 while running at EL2. It means that altering the system registers that define the EL1&0 translation regime is fraught with danger *unless* we wait for the completion of such walk with a DSB (R_LFHQG and subsequent statements in the ARM ARM). We already did the right thing for other external agents (SPE, TRBE), but not the PTW. In the case of nVHE, this is a bit involved, as there are a number of situations where this can happen (such as switching between host and guest, invalidating TLBs...). Signed-off-by: Marc Zyngier --- arch/arm64/kvm/hyp/nvhe/debug-sr.c | 2 -- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++ arch/arm64/kvm/hyp/nvhe/switch.c | 18 ++++++++++++++++++ arch/arm64/kvm/hyp/nvhe/tlb.c | 7 +++++++ 4 files changed, 32 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/debug-sr.c b/arch/arm64/kvm/hyp/nvhe/debug-sr.c index 2673bde62fad..d756b939f296 100644 --- a/arch/arm64/kvm/hyp/nvhe/debug-sr.c +++ b/arch/arm64/kvm/hyp/nvhe/debug-sr.c @@ -37,7 +37,6 @@ static void __debug_save_spe(u64 *pmscr_el1) /* Now drain all buffered data to memory */ psb_csync(); - dsb(nsh); } static void __debug_restore_spe(u64 pmscr_el1) @@ -69,7 +68,6 @@ static void __debug_save_trace(u64 *trfcr_el1) isb(); /* Drain the trace buffer to memory */ tsb_csync(); - dsb(nsh); } static void __debug_restore_trace(u64 trfcr_el1) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 552653fa18be..2e9ec4a2a4a3 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -297,6 +297,13 @@ int __pkvm_prot_finalize(void) params->vttbr = kvm_get_vttbr(mmu); params->vtcr = host_mmu.arch.vtcr; params->hcr_el2 |= HCR_VM; + + /* + * The CMO below not only cleans the updated params to the + * PoC, but also provides the DSB that ensures ongoing + * page-table walks that have started before we trapped to EL2 + * have completed. + */ kvm_flush_dcache_to_poc(params, sizeof(*params)); write_sysreg(params->hcr_el2, hcr_el2); diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index c2cb46ca4fb6..71fa16a0dc77 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -272,6 +272,17 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) */ __debug_save_host_buffers_nvhe(vcpu); + /* + * We're about to restore some new MMU state. Make sure + * ongoing page-table walks that have started before we + * trapped to EL2 have completed. This also synchronises the + * above disabling of SPE and TRBE. + * + * See DDI0487I.a D8.1.5 "Out-of-context translation regimes", + * rule R_LFHQG and subsequent information statements. + */ + dsb(nsh); + __kvm_adjust_pc(vcpu); /* @@ -306,6 +317,13 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) __timer_disable_traps(vcpu); __hyp_vgic_save_state(vcpu); + /* + * Same thing as before the guest run: we're about to switch + * the MMU context, so let's make sure we don't have any + * ongoing EL1&0 translations. + */ + dsb(nsh); + __deactivate_traps(vcpu); __load_host_stage2(); diff --git a/arch/arm64/kvm/hyp/nvhe/tlb.c b/arch/arm64/kvm/hyp/nvhe/tlb.c index d296d617f589..15c3e782dbd8 100644 --- a/arch/arm64/kvm/hyp/nvhe/tlb.c +++ b/arch/arm64/kvm/hyp/nvhe/tlb.c @@ -129,6 +129,13 @@ void __kvm_flush_cpu_context(struct kvm_s2_mmu *mmu) { struct tlb_inv_context cxt; + /* + * We're about to restore some guest MMU state. Make sure + * ongoing page-table walks that have started before we + * trapped to EL2 have completed. + */ + dsb(nsh); + /* Switch to requested VMID */ __tlb_switch_to_guest(mmu, &cxt);