From patchwork Fri Sep 29 15:01:40 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: George Dunlap X-Patchwork-Id: 9978265 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 58B3160311 for ; Fri, 29 Sep 2017 15:04:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4B4B42766D for ; Fri, 29 Sep 2017 15:04:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 402C028448; Fri, 29 Sep 2017 15:04:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id F3A902766D for ; Fri, 29 Sep 2017 15:04:22 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dxwnu-0003is-HR; Fri, 29 Sep 2017 15:02:14 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dxwnt-0003gO-3b for xen-devel@lists.xenproject.org; Fri, 29 Sep 2017 15:02:13 +0000 Received: from [85.158.143.35] by server-1.bemta-6.messagelabs.com id 48/CB-03414-4706EC95; Fri, 29 Sep 2017 15:02:12 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprAIsWRWlGSWpSXmKPExsXitHSDvW5Rwrl Ig1/PzS2+b5nM5MDocfjDFZYAxijWzLyk/IoE1owV7evZCk5ZVjSte8TYwPhMt4uRk0NCwF/i XONuFhCbTUBPYt7xr0A2B4eIgIrE7b0GXYxcHMwC7xkleh7cBasRFvCV+Dd/NzOIzSKgKvFzw x4wm1fARmL7h29MEDPlJc49uA0W5xSwldjU3w1mCwHVPLt2lx3CVpVY/OAoO0SvoMTJmU/A5j MLSEgcfPGCeQIj7ywkqVlIUgsYmVYxqhenFpWlFuma6CUVZaZnlOQmZuboGhqY6eWmFhcnpqf mJCYV6yXn525iBIYOAxDsYOy+7H+IUZKDSUmUVyD2XKQQX1J+SmVGYnFGfFFpTmrxIUYZDg4l Cd6N8UA5waLU9NSKtMwcYBDDpCU4eJREePeApHmLCxJzizPTIVKnGHU5Om7e/cMkxJKXn5cqJ c7bA1IkAFKUUZoHNwIWUZcYZaWEeRmBjhLiKUgtys0sQZV/xSjOwagkzHsYZApPZl4J3KZXQE cwAR0xeeIZkCNKEhFSUg2MG17MY97I+F7y2D4hn7lvLf4uOLhn8dNbv6u0p8vcidnlN+WG2Gn ldTpHknc2pSRaPLS5PLs5Lf3dDcbg2611wdGXfYQWLvmQa/PjZ3ngISf1H/ZL/M5OYctocuD4 LNdU7BJ/uVyC8+QGqeiGf21p+6zZoyYeiU9doT5/56+bDGFbufkVXBmWK7EUZyQaajEXFScCA FlwLNSjAgAA X-Env-Sender: prvs=438a79e6a=George.Dunlap@citrix.com X-Msg-Ref: server-3.tower-21.messagelabs.com!1506697327!81818687!1 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n, received_headers: No Received headers X-StarScan-Received: X-StarScan-Version: 9.4.45; banners=-,-,- X-VirusChecked: Checked Received: (qmail 12961 invoked from network); 29 Sep 2017 15:02:10 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-3.tower-21.messagelabs.com with RC4-SHA encrypted SMTP; 29 Sep 2017 15:02:10 -0000 X-IronPort-AV: E=Sophos;i="5.42,452,1500940800"; d="scan'208";a="449785860" From: George Dunlap To: Date: Fri, 29 Sep 2017 16:01:40 +0100 Message-ID: <20170929150144.7602-5-george.dunlap@citrix.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20170929150144.7602-1-george.dunlap@citrix.com> References: <20170929150144.7602-1-george.dunlap@citrix.com> MIME-Version: 1.0 Cc: Sergey Dyasli , Kevin Tian , Jan Beulich , Andrew Cooper , George Dunlap , Jun Nakajima Subject: [Xen-devel] [PATCH 5/9] x86/vvmx: Make updating shadow EPTP value more efficient X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP At the moment, the shadow EPTP value is written unconditionally in ept_handle_violation(). Instead, write the value on vmentry to the guest; but only write it if the value needs updating. To detect this, add a flag to the nestedvcpu struct, stale_np2m, to indicate when such an action is necessary. Set it when the nested p2m changes or when the np2m is flushed by an IPI, and clear it when we write the new value. Since an IPI invalidating the p2m may happen between nvmx_switch_guest() and vmx_vmenter, but we can't perform the vmwrite with interrupts disabled, check the flag just before entering the guest and restart the vmentry if it's set. Signed-off-by: Sergey Dyasli Signed-off-by: George Dunlap --- v2 - Merge patches 6, 7, and 14 ("x86/np2m: add stale_np2m flag", "x86/vvmx: restart nested vmentry in case of stale_np2m", and "x86/vvmx: remove EPTP write from ept_handle_violation()") - Write to stale_np2m before the vmwrite to avoid a race CC: Andrew Cooper CC: Jan Beulich CC: Jun Nakajima CC: Kevin Tian --- xen/arch/x86/hvm/nestedhvm.c | 2 ++ xen/arch/x86/hvm/vmx/entry.S | 6 ++++++ xen/arch/x86/hvm/vmx/vmx.c | 14 +++++++------- xen/arch/x86/hvm/vmx/vvmx.c | 20 ++++++++++++++++++++ xen/arch/x86/mm/p2m.c | 10 ++++++++-- xen/include/asm-x86/hvm/vcpu.h | 1 + 6 files changed, 44 insertions(+), 9 deletions(-) diff --git a/xen/arch/x86/hvm/nestedhvm.c b/xen/arch/x86/hvm/nestedhvm.c index f2f7469d86..74a464d162 100644 --- a/xen/arch/x86/hvm/nestedhvm.c +++ b/xen/arch/x86/hvm/nestedhvm.c @@ -56,6 +56,7 @@ nestedhvm_vcpu_reset(struct vcpu *v) nv->nv_vvmcxaddr = INVALID_PADDR; nv->nv_flushp2m = 0; nv->nv_p2m = NULL; + nv->stale_np2m = false; hvm_asid_flush_vcpu_asid(&nv->nv_n2asid); @@ -107,6 +108,7 @@ nestedhvm_flushtlb_ipi(void *info) */ hvm_asid_flush_core(); vcpu_nestedhvm(v).nv_p2m = NULL; + vcpu_nestedhvm(v).stale_np2m = true; } void diff --git a/xen/arch/x86/hvm/vmx/entry.S b/xen/arch/x86/hvm/vmx/entry.S index 53eedc6363..9fb8f89220 100644 --- a/xen/arch/x86/hvm/vmx/entry.S +++ b/xen/arch/x86/hvm/vmx/entry.S @@ -79,6 +79,8 @@ UNLIKELY_END(realmode) mov %rsp,%rdi call vmx_vmenter_helper + cmp $0,%eax + jne .Lvmx_vmentry_restart mov VCPU_hvm_guest_cr2(%rbx),%rax pop %r15 @@ -117,6 +119,10 @@ ENTRY(vmx_asm_do_vmentry) GET_CURRENT(bx) jmp .Lvmx_do_vmentry +.Lvmx_vmentry_restart: + sti + jmp .Lvmx_do_vmentry + .Lvmx_goto_emulator: sti mov %rsp,%rdi diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index 9cfa9b6965..c9a4111267 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -3249,12 +3249,6 @@ static void ept_handle_violation(ept_qual_t q, paddr_t gpa) case 0: // Unhandled L1 EPT violation break; case 1: // This violation is handled completly - /*Current nested EPT maybe flushed by other vcpus, so need - * to re-set its shadow EPTP pointer. - */ - if ( nestedhvm_vcpu_in_guestmode(current) && - nestedhvm_paging_mode_hap(current ) ) - __vmwrite(EPT_POINTER, get_shadow_eptp(current)); return; case -1: // This vioaltion should be injected to L1 VMM vcpu_nestedhvm(current).nv_vmexit_pending = 1; @@ -4203,13 +4197,17 @@ static void lbr_fixup(void) bdw_erratum_bdf14_fixup(); } -void vmx_vmenter_helper(const struct cpu_user_regs *regs) +int vmx_vmenter_helper(const struct cpu_user_regs *regs) { struct vcpu *curr = current; u32 new_asid, old_asid; struct hvm_vcpu_asid *p_asid; bool_t need_flush; + /* Shadow EPTP can't be updated here because irqs are disabled */ + if ( nestedhvm_vcpu_in_guestmode(curr) && vcpu_nestedhvm(curr).stale_np2m ) + return 1; + if ( curr->domain->arch.hvm_domain.pi_ops.do_resume ) curr->domain->arch.hvm_domain.pi_ops.do_resume(curr); @@ -4270,6 +4268,8 @@ void vmx_vmenter_helper(const struct cpu_user_regs *regs) __vmwrite(GUEST_RIP, regs->rip); __vmwrite(GUEST_RSP, regs->rsp); __vmwrite(GUEST_RFLAGS, regs->rflags | X86_EFLAGS_MBS); + + return 0; } /* diff --git a/xen/arch/x86/hvm/vmx/vvmx.c b/xen/arch/x86/hvm/vmx/vvmx.c index 2f468e6ced..48e37158af 100644 --- a/xen/arch/x86/hvm/vmx/vvmx.c +++ b/xen/arch/x86/hvm/vmx/vvmx.c @@ -1405,12 +1405,32 @@ static void virtual_vmexit(struct cpu_user_regs *regs) vmsucceed(regs); } +static void nvmx_eptp_update(void) +{ + if ( !nestedhvm_vcpu_in_guestmode(current) || + vcpu_nestedhvm(current).nv_vmexit_pending || + !vcpu_nestedhvm(current).stale_np2m || + !nestedhvm_paging_mode_hap(current) ) + return; + + /* + * Interrupts are enabled here, so we need to clear stale_np2m + * before we do the vmwrite. If we do it in the other order, an + * and IPI comes in changing the shadow eptp after the vmwrite, + * we'll complete the vmenter with a stale eptp value. + */ + vcpu_nestedhvm(current).stale_np2m = false; + __vmwrite(EPT_POINTER, get_shadow_eptp(current)); +} + void nvmx_switch_guest(void) { struct vcpu *v = current; struct nestedvcpu *nvcpu = &vcpu_nestedhvm(v); struct cpu_user_regs *regs = guest_cpu_user_regs(); + nvmx_eptp_update(); + /* * A pending IO emulation may still be not finished. In this case, no * virtual vmswitch is allowed. Or else, the following IO emulation will diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c index aa3182dec6..fd48a3b9db 100644 --- a/xen/arch/x86/mm/p2m.c +++ b/xen/arch/x86/mm/p2m.c @@ -1812,6 +1812,12 @@ static void assign_np2m(struct vcpu *v, struct p2m_domain *p2m) cpumask_set_cpu(v->processor, p2m->dirty_cpumask); } +static void nvcpu_flush(struct vcpu *v) +{ + hvm_asid_flush_vcpu(v); + vcpu_nestedhvm(v).stale_np2m = true; +} + struct p2m_domain * p2m_get_nestedp2m_locked(struct vcpu *v) { @@ -1835,7 +1841,7 @@ p2m_get_nestedp2m_locked(struct vcpu *v) if ( p2m->np2m_base == np2m_base || p2m->np2m_base == P2M_BASE_EADDR ) { if ( p2m->np2m_base == P2M_BASE_EADDR ) - hvm_asid_flush_vcpu(v); + nvcpu_flush(v); p2m->np2m_base = np2m_base; assign_np2m(v, p2m); nestedp2m_unlock(d); @@ -1851,7 +1857,7 @@ p2m_get_nestedp2m_locked(struct vcpu *v) p2m_flush_table(p2m); p2m_lock(p2m); p2m->np2m_base = np2m_base; - hvm_asid_flush_vcpu(v); + nvcpu_flush(v); assign_np2m(v, p2m); nestedp2m_unlock(d); diff --git a/xen/include/asm-x86/hvm/vcpu.h b/xen/include/asm-x86/hvm/vcpu.h index 6c54773f1c..5cfa4b4aa4 100644 --- a/xen/include/asm-x86/hvm/vcpu.h +++ b/xen/include/asm-x86/hvm/vcpu.h @@ -115,6 +115,7 @@ struct nestedvcpu { bool_t nv_flushp2m; /* True, when p2m table must be flushed */ struct p2m_domain *nv_p2m; /* used p2m table for this vcpu */ + bool stale_np2m; /* True when p2m_base in VMCX02 is no longer valid */ struct hvm_vcpu_asid nv_n2asid;