From patchwork Tue Feb 27 11:34:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 10244947 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B112C60208 for ; Tue, 27 Feb 2018 11:45:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A07832865F for ; Tue, 27 Feb 2018 11:45:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 94B0E286D9; Tue, 27 Feb 2018 11:45:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id F03972865F for ; Tue, 27 Feb 2018 11:45:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=YD5KVmj0uDVX0kwv8sMURM0QdcsySQdvMYi0wf783JU=; b=IJ2XQKNp6dw12rtkDipQdEpdLJ EPO2DU1dR7pYpEr4JdnJ+k6iSnwiUnk4Su9nHLgDtwt/ucj1nCP7quAqCBo0xBoYL4xosrNo0sIjo 7k+v09pa7TkKYWNtPDH9F6EqxnnLi8vz3SCI8ZitSgzKlWYCNOs5Vbx0VOLhRd/aQra6lSI8CBw60 cd0JpqrQv4YQy4M6OwVz28YkTCtYlJFVGqlb0XtaaUvcPZFFjxCETKONkjcmYG6yvFhGg6wF27DAm C6i3MmX4hn95w6rYUlNYrT/be8yKjBXS/AI4Rda7/aaidFxVpU+oc06yTlUkqFIOrksAtiY/HY7G4 O3OBbAFw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1eqdhG-00074j-4w; Tue, 27 Feb 2018 11:45:26 +0000 Received: from mail-wm0-f67.google.com ([74.125.82.67]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1eqdXl-0007i9-EB for linux-arm-kernel@lists.infradead.org; Tue, 27 Feb 2018 11:35:54 +0000 Received: by mail-wm0-f67.google.com with SMTP id q83so23503814wme.5 for ; Tue, 27 Feb 2018 03:35:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=christofferdall-dk.20150623.gappssmtp.com; s=20150623; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=tFbBejsasEvKyXNRcrfWxyk/9GapDmJ3lzhwZR91iUA=; b=hmt3Kx0wfYplilEaYNoRdhEP2sU8t9foJmbxECGuoyYxRwbJkHPfWCwvYN9jHDHvH0 ZKI8zh7CszZF6J2Ba9GaSDJakRwHMKLWpmuKNfnKRiJjRiBd3kKY25TqLnPavEXLJxUU +/1kjEJeS6pq78MvEhiPZwrKeYTfKSP9cfrZ/yXoaOE0VwVrAidgYCik04gs6YGRwduy gPCEszfa8jcL1KrEa6hDtP0TDol0xaBE7R1jp5S6P02yZLz+vLGU8gRAqZp2rSMGjdax 5kfKbD5gdA83cZ6YDRE8maSzya9uv+aoitkE+zTa8gmgU9fg0EbXibL3eMjN11bmvNvZ qfeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=tFbBejsasEvKyXNRcrfWxyk/9GapDmJ3lzhwZR91iUA=; b=fpc43dcPx/mIexsvvgeFUWBaUGEjfBNUoo3WhdeEK44boa849OUbKiss0cXgQe+74D qPZVWmgPCcArh0UFIlAZhVCm3ORpl6Va0RpMkqM6WKt7gDpJbzEMhjYj7Y8vREmacD+x +fzpw6IxIW+/K35E6cCVqkOJ2qrReVkcroUHaeM6wbVRhLtOSCkJnvH5htXwEYIkwUOt Fbvg9riSVsEyb7CyP3Sr6OwkG0O/0s3HF3QSAvDSUz/F2hMFjnXX5RwoQf3WwG1uNYhZ B7aJnwYEgp4B3ZAHMtfRvVD6yg5xg/KqHi7HEirTsITKths5bA9/kQ6lpj3zzl2MFx9t 0y2g== X-Gm-Message-State: APf1xPDzQ0WNGuq6vEIEpakjg1rDxapgoneJ/sr0nES16QwjLszp4TkS 8Xw1AhjdBVEfya/e0qkvdRQXkw== X-Google-Smtp-Source: AH8x227F9hEtpFpjHVB4Jxa+6xSqtpo1HAT6yXNcSkSdw4ZDRj1YnIhtaAB6/8ru9KO5fogkPHpHHg== X-Received: by 10.80.184.58 with SMTP id j55mr18737181ede.45.1519731325345; Tue, 27 Feb 2018 03:35:25 -0800 (PST) Received: from localhost.localdomain (x50d2404e.cust.hiper.dk. [80.210.64.78]) by smtp.gmail.com with ESMTPSA id m1sm9176786ede.39.2018.02.27.03.35.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 03:35:24 -0800 (PST) From: Christoffer Dall To: kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 38/40] KVM: arm/arm64: Handle VGICv3 save/restore from the main VGIC code on VHE Date: Tue, 27 Feb 2018 12:34:27 +0100 Message-Id: <20180227113429.637-39-cdall@kernel.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20180227113429.637-1-cdall@kernel.org> References: <20180227113429.637-1-cdall@kernel.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180227_033538_044690_50EDDD5C X-CRM114-Status: GOOD ( 19.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrew Jones , kvm@vger.kernel.org, Marc Zyngier , Tomasz Nowicki , Julien Grall , Yury Norov , Dave Martin , Shih-Wei Li MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Christoffer Dall Just like we can program the GICv2 hypervisor control interface directly from the core vgic code, we can do the same for the GICv3 hypervisor control interface on VHE systems. We do this by simply calling the save/restore functions when we have VHE and we can then get rid of the save/restore function calls from the VHE world switch function. One caveat is that we now write GICv3 system register state before the potential early exit path in the run loop, and because we sync back state in the early exit path, we have to ensure that we read a consistent GIC state from the sync path, even though we have never actually run the guest with the newly written GIC state. We solve this by inserting an ISB in the early exit path. Signed-off-by: Christoffer Dall --- Notes: Changes since v4: - Added can_access_vgic_from_kernel() primitive to make the save/restore flow from the main vgic code slightly easier to understand. - Also added a __hyp prefix to the non-VHE world-switch save/restore functions for GICv3 to avoid confusion with the save/restore functions in the main VGIC code. Changes since v2: - Added ISB in the early exit path in the run loop as explained in the commit message. arch/arm64/kvm/hyp/switch.c | 13 ++++++------- virt/kvm/arm/arm.c | 1 + virt/kvm/arm/vgic/vgic.c | 21 +++++++++++++++++++-- 3 files changed, 26 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index 31badf6e91e8..86abbee40d3f 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -192,13 +192,15 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu) write_sysreg(0, vttbr_el2); } -static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu) +/* Save VGICv3 state on non-VHE systems */ +static void __hyp_text __hyp_vgic_save_state(struct kvm_vcpu *vcpu) { if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif)) __vgic_v3_save_state(vcpu); } -static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu) +/* Restore VGICv3 state on non_VEH systems */ +static void __hyp_text __hyp_vgic_restore_state(struct kvm_vcpu *vcpu) { if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif)) __vgic_v3_restore_state(vcpu); @@ -400,8 +402,6 @@ int kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) __activate_traps(vcpu); __activate_vm(vcpu->kvm); - __vgic_restore_state(vcpu); - sysreg_restore_guest_state_vhe(guest_ctxt); __debug_switch_to_guest(vcpu); @@ -415,7 +415,6 @@ int kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) fp_enabled = fpsimd_enabled_vhe(); sysreg_save_guest_state_vhe(guest_ctxt); - __vgic_save_state(vcpu); __deactivate_traps(vcpu); @@ -451,7 +450,7 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu) __activate_traps(vcpu); __activate_vm(kern_hyp_va(vcpu->kvm)); - __vgic_restore_state(vcpu); + __hyp_vgic_restore_state(vcpu); __timer_enable_traps(vcpu); /* @@ -484,7 +483,7 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu) __sysreg_save_state_nvhe(guest_ctxt); __sysreg32_save_state(vcpu); __timer_disable_traps(vcpu); - __vgic_save_state(vcpu); + __hyp_vgic_save_state(vcpu); __deactivate_traps(vcpu); __deactivate_vm(vcpu); diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c index 09dbee56ed8f..dba629c5f8ac 100644 --- a/virt/kvm/arm/arm.c +++ b/virt/kvm/arm/arm.c @@ -717,6 +717,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run) if (ret <= 0 || need_new_vmid_gen(vcpu->kvm) || kvm_request_pending(vcpu)) { vcpu->mode = OUTSIDE_GUEST_MODE; + isb(); /* Ensure work in x_flush_hwstate is committed */ kvm_pmu_sync_hwstate(vcpu); if (static_branch_unlikely(&userspace_irqchip_in_use)) kvm_timer_sync_hwstate(vcpu); diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c index 12e2a28f437e..eaab4a616ecf 100644 --- a/virt/kvm/arm/vgic/vgic.c +++ b/virt/kvm/arm/vgic/vgic.c @@ -19,6 +19,7 @@ #include #include #include +#include #include "vgic.h" @@ -749,10 +750,22 @@ static void vgic_flush_lr_state(struct kvm_vcpu *vcpu) vgic_clear_lr(vcpu, count); } +static inline bool can_access_vgic_from_kernel(void) +{ + /* + * GICv2 can always be accessed from the kernel because it is + * memory-mapped, and VHE systems can access GICv3 EL2 system + * registers. + */ + return !static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif) || has_vhe(); +} + static inline void vgic_save_state(struct kvm_vcpu *vcpu) { if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif)) vgic_v2_save_state(vcpu); + else + __vgic_v3_save_state(vcpu); } /* Sync back the hardware VGIC state into our emulation after a guest's run. */ @@ -760,7 +773,8 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu) { struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; - vgic_save_state(vcpu); + if (can_access_vgic_from_kernel()) + vgic_save_state(vcpu); WARN_ON(vgic_v4_sync_hwstate(vcpu)); @@ -777,6 +791,8 @@ static inline void vgic_restore_state(struct kvm_vcpu *vcpu) { if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif)) vgic_v2_restore_state(vcpu); + else + __vgic_v3_restore_state(vcpu); } /* Flush our emulation state into the GIC hardware before entering the guest. */ @@ -803,7 +819,8 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu) spin_unlock(&vcpu->arch.vgic_cpu.ap_list_lock); out: - vgic_restore_state(vcpu); + if (can_access_vgic_from_kernel()) + vgic_restore_state(vcpu); } void kvm_vgic_load(struct kvm_vcpu *vcpu)