From patchwork Mon Feb 27 01:45:45 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chao Gao X-Patchwork-Id: 9592841 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id F15036058A for ; Mon, 27 Feb 2017 08:50:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E66E528452 for ; Mon, 27 Feb 2017 08:50:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DB4BB28454; Mon, 27 Feb 2017 08:50:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=2.0 tests=BAYES_00, DATE_IN_PAST_06_12, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 79D5428458 for ; Mon, 27 Feb 2017 08:50:30 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ciGyl-0005x3-20; Mon, 27 Feb 2017 08:48:23 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ciGyi-0005vq-VS for xen-devel@lists.xen.org; Mon, 27 Feb 2017 08:48:21 +0000 Received: from [85.158.137.68] by server-4.bemta-3.messagelabs.com id 02/D7-03705-4D7E3B85; Mon, 27 Feb 2017 08:48:20 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrHLMWRWlGSWpSXmKPExsXS1taRonv5+eY Ig7/H9SyWfFzM4sDocXT3b6YAxijWzLyk/IoE1owpk98yFTySr9h69jJrA+NbyS5GTg4hgUqJ 7RM+M4HYEgK8EkeWzWCFsAMk9l+6xAZRUy6xuakLrIZNQFni4tdesLiIgLTEtc+XGbsYuTiYB Z4ySTQ96gRLCAuESnS/agFrYBFQlfjc8pEdxOYVcJJ4svIa1AIFiSkP3zOD2JwCzhK3t3SxQi xzkrg84RnjBEbeBYwMqxg1ilOLylKLdI3M9JKKMtMzSnITM3N0DQ2M9XJTi4sT01NzEpOK9ZL zczcxAsOhnoGBcQdjw16/Q4ySHExKoryLjDdGCPEl5adUZiQWZ8QXleakFh9ilOHgUJLgPf5s c4SQYFFqempFWmYOMDBh0hIcPEoivBtB0rzFBYm5xZnpEKlTjIpS4ryHQBICIImM0jy4Nlg0X GKUlRLmZWRgYBDiKUgtys0sQZV/xSjOwagkzLsFZApPZl4J3PRXQIuZgBbPBrmZt7gkESEl1c CYfXVf59Z83YrVLXWSAdozUl88Czi0TW1T1Ltnm08u0Xk6q/fvWt2rn1RPbz+tuZzBmXGH/NJ /Wxje31hW/9XvvX4mn6myuITuEaE5P468kVshcDvt6lnV2Kncvxtlj3q5vKzapGKs/Uq9b0nU xKW/eN5YC6eEmHyxSdqw+Jkeu4tHkv19jb1KSizFGYmGWsxFxYkAP3cvdIECAAA= X-Env-Sender: chao.gao@intel.com X-Msg-Ref: server-4.tower-31.messagelabs.com!1488185284!29477171!5 X-Originating-IP: [134.134.136.100] X-SpamReason: No, hits=0.8 required=7.0 tests=DATE_IN_PAST_06_12 X-StarScan-Received: X-StarScan-Version: 9.2.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 40291 invoked from network); 27 Feb 2017 08:48:19 -0000 Received: from mga07.intel.com (HELO mga07.intel.com) (134.134.136.100) by server-4.tower-31.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 27 Feb 2017 08:48:19 -0000 Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga105.jf.intel.com with ESMTP; 27 Feb 2017 00:48:18 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.35,213,1484035200"; d="scan'208"; a="1135354091" Received: from skl-2s3.sh.intel.com ([10.239.48.35]) by fmsmga002.fm.intel.com with ESMTP; 27 Feb 2017 00:48:15 -0800 From: Chao Gao To: xen-devel@lists.xen.org Date: Mon, 27 Feb 2017 09:45:45 +0800 Message-Id: <1488159949-15011-5-git-send-email-chao.gao@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1488159949-15011-1-git-send-email-chao.gao@intel.com> References: <1488159949-15011-1-git-send-email-chao.gao@intel.com> Cc: Kevin Tian , Feng Wu , Jun Nakajima , George Dunlap , Andrew Cooper , Dario Faggioli , Jan Beulich , Chao Gao Subject: [Xen-devel] [PATCH v9 4/8] VMX: Make sure PI is in proper state before install the hooks X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Feng Wu We may hit the last ASSERT() in vmx_vcpu_block in the current code, since vmx_vcpu_block() may get called before vmx_pi_switch_to() has been installed or executed. Here We use cmpxchg to update the NDST field, this can make sure we only update the NDST when vmx_pi_switch_to() has not been called. So the NDST is in a proper state in vmx_vcpu_block(). Suggested-by: Jan Beulich Signed-off-by: Feng Wu Signed-off-by: Chao Gao Reviewed-by: Jan Beulich Acked-by: Kevin Tian --- v6: - Comments changes - Define macro 'APIC_INVALID_DEST' for '0xffffffff' v5: - Use 0xffffffff as the invalid value for NDST field. v4: - This patch is previously called "Pause/Unpause the domain before/after assigning PI hooks" - Remove the pause/unpause method - Use cmpxchg to update NDST xen/arch/x86/hvm/vmx/vmcs.c | 13 +++++-------- xen/arch/x86/hvm/vmx/vmx.c | 27 ++++++++++++++++++++++++++- xen/include/asm-x86/hvm/vmx/vmx.h | 2 ++ 3 files changed, 33 insertions(+), 9 deletions(-) diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c index 0b77dbc..7905d3e 100644 --- a/xen/arch/x86/hvm/vmx/vmcs.c +++ b/xen/arch/x86/hvm/vmx/vmcs.c @@ -956,16 +956,13 @@ void virtual_vmcs_vmwrite(const struct vcpu *v, u32 vmcs_encoding, u64 val) */ static void pi_desc_init(struct vcpu *v) { - uint32_t dest; - v->arch.hvm_vmx.pi_desc.nv = posted_intr_vector; - dest = cpu_physical_id(v->processor); - - if ( x2apic_enabled ) - v->arch.hvm_vmx.pi_desc.ndst = dest; - else - v->arch.hvm_vmx.pi_desc.ndst = MASK_INSR(dest, PI_xAPIC_NDST_MASK); + /* + * Mark NDST as invalid, then we can use this invalid value as a + * marker to whether update NDST or not in vmx_pi_hooks_assign(). + */ + v->arch.hvm_vmx.pi_desc.ndst = APIC_INVALID_DEST; } static int construct_vmcs(struct vcpu *v) diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index a7a70e7..e03786b 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -246,14 +246,39 @@ static void vmx_pi_do_resume(struct vcpu *v) /* This function is called when pcidevs_lock is held */ void vmx_pi_hooks_assign(struct domain *d) { + struct vcpu *v; + if ( !iommu_intpost || !has_hvm_container_domain(d) ) return; ASSERT(!d->arch.hvm_domain.pi_ops.vcpu_block); - d->arch.hvm_domain.pi_ops.vcpu_block = vmx_vcpu_block; + /* + * We carefully handle the timing here: + * - Install the context switch first + * - Then set the NDST field + * - Install the block and resume hooks in the end + * + * This can make sure the PI (especially the NDST feild) is + * in proper state when we call vmx_vcpu_block(). + */ d->arch.hvm_domain.pi_ops.switch_from = vmx_pi_switch_from; d->arch.hvm_domain.pi_ops.switch_to = vmx_pi_switch_to; + + for_each_vcpu ( d, v ) + { + unsigned int dest = cpu_physical_id(v->processor); + struct pi_desc *pi_desc = &v->arch.hvm_vmx.pi_desc; + + /* + * We don't need to update NDST if vmx_pi_switch_to() + * has already got called. + */ + (void)cmpxchg(&pi_desc->ndst, APIC_INVALID_DEST, + x2apic_enabled ? dest : MASK_INSR(dest, PI_xAPIC_NDST_MASK)); + } + + d->arch.hvm_domain.pi_ops.vcpu_block = vmx_vcpu_block; d->arch.hvm_domain.pi_ops.do_resume = vmx_pi_do_resume; } diff --git a/xen/include/asm-x86/hvm/vmx/vmx.h b/xen/include/asm-x86/hvm/vmx/vmx.h index f4183d9..00e6f0d 100644 --- a/xen/include/asm-x86/hvm/vmx/vmx.h +++ b/xen/include/asm-x86/hvm/vmx/vmx.h @@ -601,6 +601,8 @@ void vmx_pi_per_cpu_init(unsigned int cpu); void vmx_pi_hooks_assign(struct domain *d); void vmx_pi_hooks_deassign(struct domain *d); +#define APIC_INVALID_DEST 0xffffffff + /* EPT violation qualifications definitions */ typedef union ept_qual { unsigned long raw;