From patchwork Tue Feb 27 11:34:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 10245063 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id AA87C60211 for ; Tue, 27 Feb 2018 12:42:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 97FA52022C for ; Tue, 27 Feb 2018 12:42:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8B8B7219AC; Tue, 27 Feb 2018 12:42:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EA53D2022C for ; Tue, 27 Feb 2018 12:42:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=1d6JYfMIqPTR1Nyc+wcWMetEJHiLyuMubbvtKbCkNc8=; b=A5PgeX/jw7MG2k0b78WKxOYNuc 814WTBUuSCppTin6N7JBFYRdwT9vdl8eX14OEvSMOO3+4a4KSBbqDbnXVGaiblKqT7fiHH738vQSr Q5kB3dxF2ywZyPZRzShWrFa4ozibfY6qYZ4wfpInqIJRV88Oe9/2RjNWaIIV5/YninuXxm0MN0m9C jRqkhs4iZBbkZhvvBdK31vCpj8c+kKaLi0vUHwaWOFkliTU4ao8IfdQqreBOklZ1D+G/tzTWw6s5t Wj2lb3lYkgUAwS11fmIiKSOAVGsNifxfAW/j0hPujW2GoBCbseygoMM9qOnYX4gDoZt3NdubGYSVa E/dqHnHg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1eqeaO-0005Dz-99; Tue, 27 Feb 2018 12:42:24 +0000 Received: from merlin.infradead.org ([2001:8b0:10b:1231::1]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1eqeZj-0004jB-1Y for linux-arm-kernel@bombadil.infradead.org; Tue, 27 Feb 2018 12:41:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=V5lPU8WQTRwqnQPUPeBQOzb2RF19xSZbdhkb5mQki2k=; b=eCgX2sQ7OXkwTM7iqHvjVO1LF GjILoBg6t3+LiJDsBYEXKcWO/3PavhdU/nZlsrGv4abAhB4fAEZ3JaYorWh8vNhrjnWf/LFjMbmiR /UXY+roDRBzSy9suP0ctShn86CdPJk4lA0ZOO7ZMU3v7Hn0bk0cCedNuuBIRlOfITvw73Vo5iwTgF JdiVMFsGlaWTVJ8Ffg3I8E0D/nWyQ+GVyd22caLTXX7DgThXuKPLV2pQRaOB3+ESYpaH/4X4/4M7v 8fY+EifV3jwk+XBpvMhHMfYTSjaa1OZpxbGDgpMgngCANf2FseZ4wH5QsHcg+afR9VUNv10E1zaRe ROPMeXv1Q==; Received: from mail-wm0-x244.google.com ([2a00:1450:400c:c09::244]) by merlin.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1eqdXb-0006KB-0V for linux-arm-kernel@lists.infradead.org; Tue, 27 Feb 2018 11:35:27 +0000 Received: by mail-wm0-x244.google.com with SMTP id x7so20759365wmc.0 for ; Tue, 27 Feb 2018 03:35:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=christofferdall-dk.20150623.gappssmtp.com; s=20150623; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=V5lPU8WQTRwqnQPUPeBQOzb2RF19xSZbdhkb5mQki2k=; b=uecg9VJZ7DPwDbG2xne2otj193a27cDPDYPXrmPpNROWZpt/LXlawK3TgBwj0IBgWf q/MfXm/OmsSGxkye/+K1H/iepQ7D7itq7cnq7jYnKDdcrNhtTU+VZSj0algZ/nJDNmy2 7/O+MK4OezCSchYWBJsdfUOdNFDsa1Cc3WOn7Bury7tp7Z64QZKRifH/vWj7HtzQsa5V 8x3f1CA0/GMs3zpnAtQvhUQhXOb4K2x2h3fMVxh2k/FES3vIquElk5O0xjwHqzTd9huO tb69Rm0cjayl+F44HZ5HbnxcCAIpYAVztz3mngAglcHVdxhGYjlHXbQHjCUsJqyXfb74 enpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=V5lPU8WQTRwqnQPUPeBQOzb2RF19xSZbdhkb5mQki2k=; b=JRiNUCDgCFpmTXOXlHl6Fd76tpQWzB4aQmWjUlAukj8/CkTeBt9hfbUfcHbmL+7q5d KAy2J+3pSam7g84BRlUGYeNWoocMx5ejelAxKn+SN2lWuPfugmSyD1YWUi5WkL6M50vM 9qRxqvHP94SkYTYZv4pXPIl9OAh0L6wQ9eKBWOEyOvm06ooVMyXjMafWvdP5BsfIER6V LRFG3MD3AVM5KMY+mwQfDNkJ5hHqjKO+r3s0Kua7Jj6KrYakoyjhpHovl0znHToxWWgL 238JHmJbIOi3uWgjcYP63ocGXbWCvX5WU1h3yyiPcmbmTUfcKwBdbhoCaTYW3YqVuX8a CN0g== X-Gm-Message-State: APf1xPAkQYfG+26rANi8s/u2H11AzniGOcJZPIf6yXT9/eKg082dTKn7 utuDo++m4gw7Gk/0JclHsSoBeg== X-Google-Smtp-Source: AH8x2271Rb7SaYpQ8dBeJYaFLvinJs3V2mGmbDNffdMZgZXaQ1LawMnYREBTZyO2neZssWqPILjO6Q== X-Received: by 10.80.145.76 with SMTP id f12mr18786872eda.23.1519731314365; Tue, 27 Feb 2018 03:35:14 -0800 (PST) Received: from localhost.localdomain (x50d2404e.cust.hiper.dk. [80.210.64.78]) by smtp.gmail.com with ESMTPSA id m1sm9176786ede.39.2018.02.27.03.35.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 03:35:13 -0800 (PST) From: Christoffer Dall To: kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 30/40] KVM: arm64: Defer saving/restoring 32-bit sysregs to vcpu load/put Date: Tue, 27 Feb 2018 12:34:19 +0100 Message-Id: <20180227113429.637-31-cdall@kernel.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20180227113429.637-1-cdall@kernel.org> References: <20180227113429.637-1-cdall@kernel.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180227_063527_137258_F7A95978 X-CRM114-Status: GOOD ( 17.65 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrew Jones , kvm@vger.kernel.org, Marc Zyngier , Tomasz Nowicki , Julien Grall , Yury Norov , Dave Martin , Shih-Wei Li MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Christoffer Dall When running a 32-bit VM (EL1 in AArch32), the AArch32 system registers can be deferred to vcpu load/put on VHE systems because neither the host kernel nor host userspace uses these registers. Note that we can't save DBGVCR32_EL2 conditionally based on the state of the debug dirty flag on VHE after this change, because during vcpu_load() we haven't calculated a valid debug flag yet, and when we've restored the register during vcpu_load() we also have to save it during vcpu_put(). This means that we'll always restore/save the register for VHE on load/put, but luckily vcpu load/put are called rarely, so saving an extra register unconditionally shouldn't significantly hurt performance. We can also not defer saving FPEXC32_32 because this register only holds a guest-valid value for 32-bit guests during the exit path when the guest has used FPSIMD registers and restored the register in the early assembly handler from taking the EL2 fault, and therefore we have to check if fpsimd is enabled for the guest in the exit path and save the register then, for both VHE and non-VHE guests. Reviewed-by: Marc Zyngier Reviewed-by: Andrew Jones Signed-off-by: Christoffer Dall --- Notes: Changes since v4: - Moved logic to directly read/write 32-bit sysregs in the vcpu_{read,write}_sys_reg when sysregs are loaded to this patch. - Removed inline attribute from __fpsimd_save_fpexc32 Changes since v3: - Rework the FPEXC32 save/restore logic to no longer attempt to save/restore this register lazily. Changes since v2: - New patch (deferred register handling has been reworked) arch/arm64/kvm/hyp/switch.c | 17 +++++++++++------ arch/arm64/kvm/hyp/sysreg-sr.c | 15 ++++++++++----- arch/arm64/kvm/sys_regs.c | 6 ++++++ 3 files changed, 27 insertions(+), 11 deletions(-) diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index ec31e447dd7f..9b0380d3c9c3 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -47,6 +47,15 @@ bool __hyp_text __fpsimd_enabled(void) return __fpsimd_is_enabled()(); } +/* Save the 32-bit only FPSIMD system register state */ +static void __hyp_text __fpsimd_save_fpexc32(struct kvm_vcpu *vcpu) +{ + if (!vcpu_el1_is_32bit(vcpu)) + return; + + vcpu->arch.ctxt.sys_regs[FPEXC32_EL2] = read_sysreg(fpexc32_el2); +} + static void __hyp_text __activate_traps_vhe(void) { u64 val; @@ -380,11 +389,6 @@ int kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) __vgic_restore_state(vcpu); - /* - * We must restore the 32-bit state before the sysregs, thanks - * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72). - */ - __sysreg32_restore_state(vcpu); sysreg_restore_guest_state_vhe(guest_ctxt); __debug_switch_to_guest(vcpu); @@ -398,7 +402,6 @@ int kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) fp_enabled = __fpsimd_enabled(); sysreg_save_guest_state_vhe(guest_ctxt); - __sysreg32_save_state(vcpu); __vgic_save_state(vcpu); __deactivate_traps(vcpu); @@ -408,6 +411,7 @@ int kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) if (fp_enabled) { __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs); __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs); + __fpsimd_save_fpexc32(vcpu); } __debug_switch_to_host(vcpu); @@ -477,6 +481,7 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu) if (fp_enabled) { __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs); __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs); + __fpsimd_save_fpexc32(vcpu); } /* diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c index 9c60b8062724..aacba4636871 100644 --- a/arch/arm64/kvm/hyp/sysreg-sr.c +++ b/arch/arm64/kvm/hyp/sysreg-sr.c @@ -196,10 +196,7 @@ void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu) sysreg[DACR32_EL2] = read_sysreg(dacr32_el2); sysreg[IFSR32_EL2] = read_sysreg(ifsr32_el2); - if (__fpsimd_enabled()) - sysreg[FPEXC32_EL2] = read_sysreg(fpexc32_el2); - - if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) + if (has_vhe() || vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) sysreg[DBGVCR32_EL2] = read_sysreg(dbgvcr32_el2); } @@ -221,7 +218,7 @@ void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu) write_sysreg(sysreg[DACR32_EL2], dacr32_el2); write_sysreg(sysreg[IFSR32_EL2], ifsr32_el2); - if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) + if (has_vhe() || vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) write_sysreg(sysreg[DBGVCR32_EL2], dbgvcr32_el2); } @@ -246,6 +243,13 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu) __sysreg_save_user_state(host_ctxt); + /* + * Load guest EL1 and user state + * + * We must restore the 32-bit state before the sysregs, thanks + * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72). + */ + __sysreg32_restore_state(vcpu); __sysreg_restore_user_state(guest_ctxt); __sysreg_restore_el1_state(guest_ctxt); @@ -273,6 +277,7 @@ void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu) __sysreg_save_el1_state(guest_ctxt); __sysreg_save_user_state(guest_ctxt); + __sysreg32_save_state(vcpu); /* Restore host user state */ __sysreg_restore_user_state(host_ctxt); diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 17eb7772e059..806b0b126a64 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -112,6 +112,9 @@ u64 vcpu_read_sys_reg(struct kvm_vcpu *vcpu, int reg) case AMAIR_EL1: return read_sysreg_s(amair_EL12); case CNTKCTL_EL1: return read_sysreg_s(cntkctl_EL12); case PAR_EL1: return read_sysreg_s(SYS_PAR_EL1); + case DACR32_EL2: return read_sysreg_s(SYS_DACR32_EL2); + case IFSR32_EL2: return read_sysreg_s(SYS_IFSR32_EL2); + case DBGVCR32_EL2: return read_sysreg_s(SYS_DBGVCR32_EL2); } immediate_read: @@ -152,6 +155,9 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg) case AMAIR_EL1: write_sysreg_s(val, amair_EL12); return; case CNTKCTL_EL1: write_sysreg_s(val, cntkctl_EL12); return; case PAR_EL1: write_sysreg_s(val, SYS_PAR_EL1); return; + case DACR32_EL2: write_sysreg_s(val, SYS_DACR32_EL2); return; + case IFSR32_EL2: write_sysreg_s(val, SYS_IFSR32_EL2); return; + case DBGVCR32_EL2: write_sysreg_s(val, SYS_DBGVCR32_EL2); return; } immediate_write: