From patchwork Tue Feb 27 11:34:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 10245061 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5416660211 for ; Tue, 27 Feb 2018 12:41:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 417DE288A8 for ; Tue, 27 Feb 2018 12:41:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 35D20288AE; Tue, 27 Feb 2018 12:41:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8C7FD288A8 for ; Tue, 27 Feb 2018 12:41:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=kKDSYug84dMVhkwkBgRvRf4cYoMASf4ZjXjeDF8RJi4=; b=IknAgw/ds0JIBUAF/P8Q7t+9kF DXNSRjdS4Ds5WvQ/IpfLiccHiGWtVQ6ORF/rjnZxiv0ffuMdwbt9aoWW3d5E+uOpJGsXI81XX42Yj D2oXJAUPbNFok1d7W3veRde8nf0bwiIEUhPp5ErtpgVlh5S9JwqoX2LTu2Z5C31Tof9S4a6Q9DuXn +j0J4cfOwAbDk3nuq/EWhIYSjPPJKx1fpMf1kT8s84noCJBPI1rALDECEoiqfqNq5m1QQxI6P3rll IY2O8yAe1L1eQ8qBMomM3wgR/xSvaKh4hNGAxiqCcp3bQ6Em1tnBOZOgib0o87eHM+dwqcEP2BihP uzLieIiA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1eqeZf-0004kN-Oo; Tue, 27 Feb 2018 12:41:39 +0000 Received: from merlin.infradead.org ([2001:8b0:10b:1231::1]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1eqeZd-0004jB-93 for linux-arm-kernel@bombadil.infradead.org; Tue, 27 Feb 2018 12:41:37 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=UEegtOAnjFMgNLflelQRhTWmUBiru1VYiXXaWMKYloE=; b=uXlBuUQM+GrvoGhr3U2Lgjrn0 7oAm5QfEclr3AAOaWNaAlTPLBbdvoPo6ycGgg5Qdg8YAIUbV0Th/0vUTFVoFotQ3518MPB/IQ6uW8 oHb3eeLg963dbydmGEwPG1Jl6CEXt/bj/fhDNsJOpnVztBUZWAty59lh6dZwMEYSARyhA/Ykb1WSs RXvCNtEgmSa+8rX5fxWduZ6IkwHwtAChWkYneTWdyEJmw3Zlk/vJTv49JNLfjVySlxW7oyoJshWd6 sqPkxv3a//Ef6T5R7CcFwFBVNiXqCaK8CfNqa5nUoLBTm4eegPYUC4koQb+9NALMdydlIYx+tBW+g CnADfBliA==; Received: from mail-wm0-x243.google.com ([2a00:1450:400c:c09::243]) by merlin.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1eqdXY-0006KA-9J for linux-arm-kernel@lists.infradead.org; Tue, 27 Feb 2018 11:35:26 +0000 Received: by mail-wm0-x243.google.com with SMTP id q83so23502372wme.5 for ; Tue, 27 Feb 2018 03:35:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=christofferdall-dk.20150623.gappssmtp.com; s=20150623; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=UEegtOAnjFMgNLflelQRhTWmUBiru1VYiXXaWMKYloE=; b=UI+4xkpt2hnPyLKkwUk5S9uLaUW4lvQMM/lumrmzq7jja4VN1nWG19AWyWyfYt3Y70 VWo3CzoUXiskVJ8wGDtDLCF1sCGxuuLyBen7Ulg6Zn8MAzf+JBqFCEsXSSwqo4MgzCkD rHA/PLJvmM/ZXt9H8EoP3QeD8uqL7YjbtM4ZdqLeP41xSn3QkdbSNJnMtDnt/drXAwtc NglZsv8w6RPPStliai3i9NPALV5QvNSS0AbrbYfAl4WMx5gsAd1qcLVqvduf2CJ6dqL+ 0hrv5gqcBZNmx0fsT1VLkjQlMZFdSexidjUXRA4rmb//fHY8XDkVQxTrax0NvR+LhHgJ OPHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=UEegtOAnjFMgNLflelQRhTWmUBiru1VYiXXaWMKYloE=; b=K4Mx6QurtH338pcLj2OhpnEi7tpFjGgIMN3KRKK6mD+Gxb1Rgp28C4VGW9EkY/kv8y P7sqdCPCL00zYjGy0XHRpA5t3R+H0dMg3QNeFh+FqV6Omv6UttmlsFpswutf/kebKUoh 0HY3eE4Xw27hyrjBFFToKjs6ykYiipvHkfcXeLznVn33wAM9lrjKN8+5on/cwCeHat7I HuE8jKEpxnJtlDxFx3AfzPfSkRxrdDUKENO/iY1owYy671LE3BHFyT412IWa7ikQI/8a KG1LtFCpiUD0kFBOj/V+G3l2MKmADLPf/GbFZhSTTmfCrIuoM3zn7x/Wn+k4lVzJ49/Q 3qfQ== X-Gm-Message-State: APf1xPDqNGIccWNnT1Lhlu3Nfin2IekGODEojQguFeUHhT5eQ7iQP9+q 6MgK1I3jU5FEG2QFCNT2CMHIPg== X-Google-Smtp-Source: AH8x227KQvkiu1Eok6r2HgpufSn5r+x++E+cmNvPf2InIscnewTzy2rJAXk6+CS6hjhSD3d1EJ6iMw== X-Received: by 10.80.168.227 with SMTP id k90mr18395932edc.249.1519731311493; Tue, 27 Feb 2018 03:35:11 -0800 (PST) Received: from localhost.localdomain (x50d2404e.cust.hiper.dk. [80.210.64.78]) by smtp.gmail.com with ESMTPSA id m1sm9176786ede.39.2018.02.27.03.35.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 03:35:10 -0800 (PST) From: Christoffer Dall To: kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 28/40] KVM: arm64: Defer saving/restoring 64-bit sysregs to vcpu load/put on VHE Date: Tue, 27 Feb 2018 12:34:17 +0100 Message-Id: <20180227113429.637-29-cdall@kernel.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20180227113429.637-1-cdall@kernel.org> References: <20180227113429.637-1-cdall@kernel.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180227_063524_395268_275A2A16 X-CRM114-Status: GOOD ( 19.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrew Jones , kvm@vger.kernel.org, Marc Zyngier , Tomasz Nowicki , Julien Grall , Yury Norov , Dave Martin , Shih-Wei Li MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Christoffer Dall Some system registers do not affect the host kernel's execution and can therefore be loaded when we are about to run a VCPU and we don't have to restore the host state to the hardware before the time when we are actually about to return to userspace or schedule out the VCPU thread. The EL1 system registers and the userspace state registers only affecting EL0 execution do not need to be saved and restored on every switch between the VM and the host, because they don't affect the host kernel's execution. We mark all registers which are now deffered as such in the vcpu_{read,write}_sys_reg accessors in sys-regs.c to ensure the most up-to-date copy is always accessed. Note MPIDR_EL1 (controlled via VMPIDR_EL2) is accessed from other vcpu threads, for example via the GIC emulation, and therefore must be declared as immediate, which is fine as the guest cannot modify this value. The 32-bit sysregs can also be deferred but we do this in a separate patch as it requires a bit more infrastructure. Reviewed-by: Andrew Jones Signed-off-by: Christoffer Dall --- Notes: Changes since v4: - Added explanatory note about MPIDR_EL1 to vcpu_read_sys_reg and vcpu_write_sys_reg. Changes since v3: - Changed to switch-based sysreg approach arch/arm64/kvm/hyp/sysreg-sr.c | 39 ++++++++++++++++++++++++++------- arch/arm64/kvm/sys_regs.c | 49 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 80 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c index 906606dc4e2c..9c60b8062724 100644 --- a/arch/arm64/kvm/hyp/sysreg-sr.c +++ b/arch/arm64/kvm/hyp/sysreg-sr.c @@ -25,8 +25,12 @@ /* * Non-VHE: Both host and guest must save everything. * - * VHE: Host must save tpidr*_el0, mdscr_el1, sp_el0, - * and guest must save everything. + * VHE: Host and guest must save mdscr_el1 and sp_el0 (and the PC and pstate, + * which are handled as part of the el2 return state) on every switch. + * tpidr_el0 and tpidrro_el0 only need to be switched when going + * to host userspace or a different VCPU. EL1 registers only need to be + * switched when potentially going to run a different VCPU. The latter two + * classes are handled as part of kvm_arch_vcpu_load and kvm_arch_vcpu_put. */ static void __hyp_text __sysreg_save_common_state(struct kvm_cpu_context *ctxt) @@ -93,14 +97,11 @@ void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt) void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt) { __sysreg_save_common_state(ctxt); - __sysreg_save_user_state(ctxt); } void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt) { - __sysreg_save_el1_state(ctxt); __sysreg_save_common_state(ctxt); - __sysreg_save_user_state(ctxt); __sysreg_save_el2_return_state(ctxt); } @@ -169,14 +170,11 @@ void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt) void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt) { __sysreg_restore_common_state(ctxt); - __sysreg_restore_user_state(ctxt); } void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt) { - __sysreg_restore_el1_state(ctxt); __sysreg_restore_common_state(ctxt); - __sysreg_restore_user_state(ctxt); __sysreg_restore_el2_return_state(ctxt); } @@ -240,6 +238,18 @@ void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu) */ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu) { + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context; + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt; + + if (!has_vhe()) + return; + + __sysreg_save_user_state(host_ctxt); + + __sysreg_restore_user_state(guest_ctxt); + __sysreg_restore_el1_state(guest_ctxt); + + vcpu->arch.sysregs_loaded_on_cpu = true; } /** @@ -255,6 +265,19 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu) */ void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu) { + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context; + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt; + + if (!has_vhe()) + return; + + __sysreg_save_el1_state(guest_ctxt); + __sysreg_save_user_state(guest_ctxt); + + /* Restore host user state */ + __sysreg_restore_user_state(host_ctxt); + + vcpu->arch.sysregs_loaded_on_cpu = false; } void __hyp_text __kvm_set_tpidr_el2(u64 tpidr_el2) diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index c809f0d1a059..17eb7772e059 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -85,8 +85,33 @@ u64 vcpu_read_sys_reg(struct kvm_vcpu *vcpu, int reg) /* * System registers listed in the switch are not saved on every * exit from the guest but are only saved on vcpu_put. + * + * Note that MPIDR_EL1 for the guest is set by KVM via VMPIDR_EL2 but + * should never be listed below, because the guest cannot modify its + * own MPIDR_EL1 and MPIDR_EL1 is accessed for VCPU A from VCPU B's + * thread when emulating cross-VCPU communication. */ switch (reg) { + case CSSELR_EL1: return read_sysreg_s(SYS_CSSELR_EL1); + case SCTLR_EL1: return read_sysreg_s(sctlr_EL12); + case ACTLR_EL1: return read_sysreg_s(SYS_ACTLR_EL1); + case CPACR_EL1: return read_sysreg_s(cpacr_EL12); + case TTBR0_EL1: return read_sysreg_s(ttbr0_EL12); + case TTBR1_EL1: return read_sysreg_s(ttbr1_EL12); + case TCR_EL1: return read_sysreg_s(tcr_EL12); + case ESR_EL1: return read_sysreg_s(esr_EL12); + case AFSR0_EL1: return read_sysreg_s(afsr0_EL12); + case AFSR1_EL1: return read_sysreg_s(afsr1_EL12); + case FAR_EL1: return read_sysreg_s(far_EL12); + case MAIR_EL1: return read_sysreg_s(mair_EL12); + case VBAR_EL1: return read_sysreg_s(vbar_EL12); + case CONTEXTIDR_EL1: return read_sysreg_s(contextidr_EL12); + case TPIDR_EL0: return read_sysreg_s(SYS_TPIDR_EL0); + case TPIDRRO_EL0: return read_sysreg_s(SYS_TPIDRRO_EL0); + case TPIDR_EL1: return read_sysreg_s(SYS_TPIDR_EL1); + case AMAIR_EL1: return read_sysreg_s(amair_EL12); + case CNTKCTL_EL1: return read_sysreg_s(cntkctl_EL12); + case PAR_EL1: return read_sysreg_s(SYS_PAR_EL1); } immediate_read: @@ -101,8 +126,32 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg) /* * System registers listed in the switch are not restored on every * entry to the guest but are only restored on vcpu_load. + * + * Note that MPIDR_EL1 for the guest is set by KVM via VMPIDR_EL2 but + * should never be listed below, because the the MPIDR should only be + * set once, before running the VCPU, and never changed later. */ switch (reg) { + case CSSELR_EL1: write_sysreg_s(val, SYS_CSSELR_EL1); return; + case SCTLR_EL1: write_sysreg_s(val, sctlr_EL12); return; + case ACTLR_EL1: write_sysreg_s(val, SYS_ACTLR_EL1); return; + case CPACR_EL1: write_sysreg_s(val, cpacr_EL12); return; + case TTBR0_EL1: write_sysreg_s(val, ttbr0_EL12); return; + case TTBR1_EL1: write_sysreg_s(val, ttbr1_EL12); return; + case TCR_EL1: write_sysreg_s(val, tcr_EL12); return; + case ESR_EL1: write_sysreg_s(val, esr_EL12); return; + case AFSR0_EL1: write_sysreg_s(val, afsr0_EL12); return; + case AFSR1_EL1: write_sysreg_s(val, afsr1_EL12); return; + case FAR_EL1: write_sysreg_s(val, far_EL12); return; + case MAIR_EL1: write_sysreg_s(val, mair_EL12); return; + case VBAR_EL1: write_sysreg_s(val, vbar_EL12); return; + case CONTEXTIDR_EL1: write_sysreg_s(val, contextidr_EL12); return; + case TPIDR_EL0: write_sysreg_s(val, SYS_TPIDR_EL0); return; + case TPIDRRO_EL0: write_sysreg_s(val, SYS_TPIDRRO_EL0); return; + case TPIDR_EL1: write_sysreg_s(val, SYS_TPIDR_EL1); return; + case AMAIR_EL1: write_sysreg_s(val, amair_EL12); return; + case CNTKCTL_EL1: write_sysreg_s(val, cntkctl_EL12); return; + case PAR_EL1: write_sysreg_s(val, SYS_PAR_EL1); return; } immediate_write: