From patchwork Wed Aug 31 03:56:31 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Feng" X-Patchwork-Id: 9306315 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 54B4A607D2 for ; Wed, 31 Aug 2016 04:28:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 41DBA28B87 for ; Wed, 31 Aug 2016 04:28:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3693828C14; Wed, 31 Aug 2016 04:28:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D1F9428B87 for ; Wed, 31 Aug 2016 04:28:09 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bex5j-000196-Nw; Wed, 31 Aug 2016 04:25:35 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bex5i-00018n-RC for xen-devel@lists.xen.org; Wed, 31 Aug 2016 04:25:34 +0000 Received: from [85.158.143.35] by server-3.bemta-6.messagelabs.com id B4/74-05661-E3C56C75; Wed, 31 Aug 2016 04:25:34 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrFLMWRWlGSWpSXmKPExsXS1tYhomsbcyz cYPsOPYslHxezODB6HN39mymAMYo1My8pvyKBNeN53z7GgnbpilcHe1kaGDeKdjFycAgJVErM n8vWxcjJISHAK3Fk2QxWCNtXYlXbbSYQW0igXmLZ+0uMIDabgKLEwYuHwGpEBKQlrn2+DBTn4 mAWWMAo0XjxKDNIQljARWLGw4vMIPNZBFQllk/SAAnzCjhI/FvxhRlivpzEht3/wWZyCjhK/D j+kA3iHAeJHfeMJzDyLmBkWMWoXpxaVJZapGupl1SUmZ5RkpuYmaNraGCml5taXJyYnpqTmFS sl5yfu4kRGAYMQLCD8e6mgEOMkhxMSqK84X+PhgvxJeWnVGYkFmfEF5XmpBYfYpTh4FCS4LWL PhYuJFiUmp5akZaZAwxImLQEB4+SCK8ESJq3uCAxtzgzHSJ1ilFRSpz3XRRQQgAkkVGaB9cGi 4JLjLJSwryMQIcI8RSkFuVmlqDKv2IU52BUEuY9BDKFJzOvBG76K6DFTECLC+4cBllckoiQkm pg9Lq6fsOLS31TCkztIlbcuc/B+fAN8xnO/Ve6VpoVG3yd+WGNdLff13VKxVsyeZWvr+diWrJ +94QJ6+7Ff+BTO9d9o6ItXi31zIUE87/c4SxzitpkDu9xFr6fyBJ/u2S6j/LH/MyL9RGeemsM RLZW+L3awR27b826/bmNn18edmFlu7Bo+7OZukosxRmJhlrMRcWJAOxeBGR9AgAA X-Env-Sender: feng.wu@intel.com X-Msg-Ref: server-10.tower-21.messagelabs.com!1472617522!30750590!7 X-Originating-IP: [134.134.136.20] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTM0LjEzNC4xMzYuMjAgPT4gMzU1MzU4\n X-StarScan-Received: X-StarScan-Version: 8.84; banners=-,-,- X-VirusChecked: Checked Received: (qmail 31930 invoked from network); 31 Aug 2016 04:25:33 -0000 Received: from mga02.intel.com (HELO mga02.intel.com) (134.134.136.20) by server-10.tower-21.messagelabs.com with DHE-RSA-CAMELLIA256-SHA encrypted SMTP; 31 Aug 2016 04:25:33 -0000 Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP; 30 Aug 2016 21:25:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.30,260,1470726000"; d="scan'208";a="2931046" Received: from feng-bdw-de-pi.bj.intel.com ([10.238.154.74]) by fmsmga005.fm.intel.com with ESMTP; 30 Aug 2016 21:25:31 -0700 From: Feng Wu To: xen-devel@lists.xen.org Date: Wed, 31 Aug 2016 11:56:31 +0800 Message-Id: <1472615791-8664-7-git-send-email-feng.wu@intel.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1472615791-8664-1-git-send-email-feng.wu@intel.com> References: <1472615791-8664-1-git-send-email-feng.wu@intel.com> Cc: kevin.tian@intel.com, Feng Wu , george.dunlap@eu.citrix.com, andrew.cooper3@citrix.com, dario.faggioli@citrix.com, jbeulich@suse.com Subject: [Xen-devel] [PATCH v3 6/6] VMX: Fixup PI descritpor when cpu is offline X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP When cpu is offline, we need to move all the vcpus in its blocking list to another online cpu, this patch handles it. Signed-off-by: Feng Wu --- xen/arch/x86/hvm/vmx/vmcs.c | 1 + xen/arch/x86/hvm/vmx/vmx.c | 54 +++++++++++++++++++++++++++++++++++++++ xen/include/asm-x86/hvm/vmx/vmx.h | 1 + 3 files changed, 56 insertions(+) diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c index 1bd875a..f554d4c 100644 --- a/xen/arch/x86/hvm/vmx/vmcs.c +++ b/xen/arch/x86/hvm/vmx/vmcs.c @@ -578,6 +578,7 @@ void vmx_cpu_dead(unsigned int cpu) vmx_free_vmcs(per_cpu(vmxon_region, cpu)); per_cpu(vmxon_region, cpu) = 0; nvmx_cpu_dead(cpu); + vmx_pi_desc_fixup(cpu); } int vmx_cpu_up(void) diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index 071c063..5f428b7 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -211,6 +211,60 @@ static void vmx_pi_blocking_cleanup(struct vcpu *v) vmx_pi_remove_vcpu_from_blocking_list(v); } +void vmx_pi_desc_fixup(int cpu) +{ + unsigned int new_cpu, dest; + unsigned long flags; + struct arch_vmx_struct *vmx, *tmp; + spinlock_t *new_lock, *old_lock = &per_cpu(vmx_pi_blocking, cpu).lock; + struct list_head *blocked_vcpus = &per_cpu(vmx_pi_blocking, cpu).list; + + if ( !iommu_intpost ) + return; + + spin_lock_irqsave(old_lock, flags); + + list_for_each_entry_safe(vmx, tmp, blocked_vcpus, pi_blocking.list) + { + /* + * We need to find an online cpu as the NDST of the PI descriptor, it + * doesn't matter whether it is within the cpupool of the domain or + * not. As long as it is online, the vCPU will be woken up once the + * notification event arrives. + */ +restart: + new_cpu = cpumask_any(&cpu_online_map); + new_lock = &per_cpu(vmx_pi_blocking, new_cpu).lock; + + spin_lock(new_lock); + + /* + * If the new_cpu is not online, that means it became offline between + * we got 'new_cpu' and acquiring its lock above, we need to find + * another online cpu instead. Such as, this fucntion is being called + * on 'new_cpu' at the same time. Can this happen?? + */ + if ( !cpu_online(new_cpu) ) + { + spin_unlock(new_lock); + goto restart; + } + + ASSERT(vmx->pi_blocking.lock == old_lock); + + dest = cpu_physical_id(new_cpu); + write_atomic(&vmx->pi_desc.ndst, + x2apic_enabled ? dest : MASK_INSR(dest, PI_xAPIC_NDST_MASK)); + + list_move(&vmx->pi_blocking.list, + &per_cpu(vmx_pi_blocking, new_cpu).list); + vmx->pi_blocking.lock = new_lock; + spin_unlock(new_lock); + } + + spin_unlock_irqrestore(old_lock, flags); +} + /* This function is called when pcidevs_lock is held */ void vmx_pi_hooks_assign(struct domain *d) { diff --git a/xen/include/asm-x86/hvm/vmx/vmx.h b/xen/include/asm-x86/hvm/vmx/vmx.h index 4cdd9b1..9783c70 100644 --- a/xen/include/asm-x86/hvm/vmx/vmx.h +++ b/xen/include/asm-x86/hvm/vmx/vmx.h @@ -569,6 +569,7 @@ void free_p2m_hap_data(struct p2m_domain *p2m); void p2m_init_hap_data(struct p2m_domain *p2m); void vmx_pi_per_cpu_init(unsigned int cpu); +void vmx_pi_desc_fixup(int cpu); void vmx_pi_hooks_assign(struct domain *d); void vmx_pi_hooks_deassign(struct domain *d);