From patchwork Thu Apr 6 00:30:21 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chao Gao X-Patchwork-Id: 9666239 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D4D6160364 for ; Thu, 6 Apr 2017 07:35:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C56D727F9F for ; Thu, 6 Apr 2017 07:35:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BA3C12818B; Thu, 6 Apr 2017 07:35:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=2.0 tests=BAYES_00, DATE_IN_PAST_06_12, DKIM_SIGNED, RCVD_IN_DNSWL_MED, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 462E927F9F for ; Thu, 6 Apr 2017 07:35:38 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cw1ur-0003Vs-3z; Thu, 06 Apr 2017 07:33:13 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cw1up-0003VS-AG for xen-devel@lists.xen.org; Thu, 06 Apr 2017 07:33:11 +0000 Received: from [85.158.143.35] by server-8.bemta-6.messagelabs.com id 47/0B-03648-63FE5E85; Thu, 06 Apr 2017 07:33:10 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrGLMWRWlGSWpSXmKPExsXS1tYhomv2/mm EwcLdghZLPi5mcWD0OLr7N1MAYxRrZl5SfkUCa8aUJffYC3ZpVbxfo9/AOFu+i5GLg0XgFpPE 232/mUEcIYHpjBJT1k9n7WLk5JAQ4JU4smwGkM0BZAdInD3lDRIWEiiXOD5jCVgJm4CyxMWvv WwgtoiAtMS1z5cZQcqZBfIlnvYwgoSFBWIl1k/fDlbOIqAq8erwQ2YQm1fASeLQtWNsEJsUJK Y8fM8M0sop4Cxx6002xCYniQ0PP7JNYORbwMiwilG9OLWoLLVI11AvqSgzPaMkNzEzR9fQwEw vN7W4ODE9NScxqVgvOT93EyMwPBiAYAfjzudOhxglOZiURHkVfJ5ECPEl5adUZiQWZ8QXleak Fh9ilOHgUJLgZXr3NEJIsCg1PbUiLTMHGKgwaQkOHiUR3nyQNG9xQWJucWY6ROoUo6KUOK8kS EIAJJFRmgfXBouOS4yyUsK8jECHCPEUpBblZpagyr9iFOdgVBLmLQeZwpOZVwI3/RXQYiagxT 63wBaXJCKkpBoYW92uflb6N7nB7ertf0YH5sbNmC10PfX0U+ZTjLK7T3KXXvljU1VrcXTS1h7 LL/vMZq889/G61AKpDWu+3P1zwCkr2PrJhje2hoea37091+5YeuSEvsU2hcDV53e39izWKz34 KU9k2u3qrtsan3eeWnm33zO1/eCFj4d3p2yL9phmY6lXc/830wYlluKMREMt5qLiRAD/A/jci QIAAA== X-Env-Sender: chao.gao@intel.com X-Msg-Ref: server-2.tower-21.messagelabs.com!1491463986!48563541!2 X-Originating-IP: [134.134.136.20] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTM0LjEzNC4xMzYuMjAgPT4gMzU1MzU4\n X-StarScan-Received: X-StarScan-Version: 9.4.12; banners=-,-,- X-VirusChecked: Checked Received: (qmail 6154 invoked from network); 6 Apr 2017 07:33:09 -0000 Received: from mga02.intel.com (HELO mga02.intel.com) (134.134.136.20) by server-2.tower-21.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 6 Apr 2017 07:33:09 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=intel.com; i=@intel.com; q=dns/txt; s=intel; t=1491463989; x=1522999989; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=XMUb6bQhLootDcEi9RQqgA93HZbX6379F7OhL5jJhRo=; b=gg81dCFbHT9frOzorGiFuQNOuPzJG3EIh1TLz8qY9UzfDmLT1yf7ziKA IbHQUlyDYEi08JAXkvQprx4RsKCo/g==; Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Apr 2017 00:33:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.37,283,1488873600"; d="scan'208"; a="1151803771" Received: from skl-2s3.sh.intel.com ([10.239.48.35]) by fmsmga002.fm.intel.com with ESMTP; 06 Apr 2017 00:33:07 -0700 From: Chao Gao To: xen-devel@lists.xen.org Date: Thu, 6 Apr 2017 08:30:21 +0800 Message-Id: <1491438627-10456-2-git-send-email-chao.gao@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1491438627-10456-1-git-send-email-chao.gao@intel.com> References: <1491438627-10456-1-git-send-email-chao.gao@intel.com> Cc: Andrew Cooper , Jan Beulich , Chao Gao Subject: [Xen-devel] [PATCH v12 1/7] passthrough: don't migrate pirq when it is delivered through VT-d PI X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP When a vCPU was migrated to another pCPU, pt irqs binded to this vCPU might also need migration as a optimization to reduce IPI between pCPUs. When VT-d PI is enabled, interrupt vector will be recorded to a main memory resident data-structure and a notification whose destination is decided by NDST is generated. NDST is properly adjusted during vCPU migration so pirq directly injected to guest needn't be migrated. This patch adds a indicator, @posted, to show whether the pt irq is delivered through VT-d PI. Also this patch fixes a bug that hvm_migrate_pirq() accesses pirq_dpci->gmsi.dest_vcpu_id without checking the pirq_dpci's type. Signed-off-by: Chao Gao --- v12: - fix a logic error in fixed delivery case. v11: - rename the indicator to 'posted' - move setting 'posted' field to event lock un-locked region. v10: - Newly added. xen/arch/x86/hvm/hvm.c | 3 +++ xen/drivers/passthrough/io.c | 63 +++++++++----------------------------------- xen/include/xen/hvm/irq.h | 1 + 3 files changed, 17 insertions(+), 50 deletions(-) diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index eba6e9d..d4c8967 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -438,6 +438,9 @@ static int hvm_migrate_pirq(struct domain *d, struct hvm_pirq_dpci *pirq_dpci, struct vcpu *v = arg; if ( (pirq_dpci->flags & HVM_IRQ_DPCI_MACH_MSI) && + (pirq_dpci->flags & HVM_IRQ_DPCI_GUEST_MSI) && + /* Needn't migrate pirq if this pirq is delivered to guest directly.*/ + (!pirq_dpci->gmsi.posted) && (pirq_dpci->gmsi.dest_vcpu_id == v->vcpu_id) ) { struct irq_desc *desc = diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c index 83e0961..4d19413 100644 --- a/xen/drivers/passthrough/io.c +++ b/xen/drivers/passthrough/io.c @@ -259,52 +259,6 @@ static struct vcpu *vector_hashing_dest(const struct domain *d, return dest; } -/* - * The purpose of this routine is to find the right destination vCPU for - * an interrupt which will be delivered by VT-d posted-interrupt. There - * are several cases as below: - * - * - For lowest-priority interrupts, use vector-hashing mechanism to find - * the destination. - * - Otherwise, for single destination interrupt, it is straightforward to - * find the destination vCPU and return true. - * - For multicast/broadcast vCPU, we cannot handle it via interrupt posting, - * so return NULL. - */ -static struct vcpu *pi_find_dest_vcpu(const struct domain *d, uint32_t dest_id, - bool_t dest_mode, uint8_t delivery_mode, - uint8_t gvec) -{ - unsigned int dest_vcpus = 0; - struct vcpu *v, *dest = NULL; - - switch ( delivery_mode ) - { - case dest_LowestPrio: - return vector_hashing_dest(d, dest_id, dest_mode, gvec); - case dest_Fixed: - for_each_vcpu ( d, v ) - { - if ( !vlapic_match_dest(vcpu_vlapic(v), NULL, APIC_DEST_NOSHORT, - dest_id, dest_mode) ) - continue; - - dest_vcpus++; - dest = v; - } - - /* For fixed mode, we only handle single-destination interrupts. */ - if ( dest_vcpus == 1 ) - return dest; - - break; - default: - break; - } - - return NULL; -} - int pt_irq_create_bind( struct domain *d, xen_domctl_bind_pt_irq_t *pt_irq_bind) { @@ -365,6 +319,7 @@ int pt_irq_create_bind( { uint8_t dest, dest_mode, delivery_mode; int dest_vcpu_id; + const struct vcpu *vcpu; if ( !(pirq_dpci->flags & HVM_IRQ_DPCI_MAPPED) ) { @@ -442,17 +397,25 @@ int pt_irq_create_bind( dest_vcpu_id = hvm_girq_dest_2_vcpu_id(d, dest, dest_mode); pirq_dpci->gmsi.dest_vcpu_id = dest_vcpu_id; spin_unlock(&d->event_lock); + + pirq_dpci->gmsi.posted = false; + vcpu = (dest_vcpu_id >= 0) ? d->vcpu[dest_vcpu_id] : NULL; + if ( iommu_intpost ) + { + if ( delivery_mode == dest_LowestPrio ) + vcpu = vector_hashing_dest(d, dest, dest_mode, + pirq_dpci->gmsi.gvec); + if ( vcpu ) + pirq_dpci->gmsi.posted = true; + } if ( dest_vcpu_id >= 0 ) hvm_migrate_pirqs(d->vcpu[dest_vcpu_id]); /* Use interrupt posting if it is supported. */ if ( iommu_intpost ) { - const struct vcpu *vcpu = pi_find_dest_vcpu(d, dest, dest_mode, - delivery_mode, pirq_dpci->gmsi.gvec); - if ( vcpu ) - pi_update_irte( vcpu, info, pirq_dpci->gmsi.gvec ); + pi_update_irte(vcpu, info, pirq_dpci->gmsi.gvec); else dprintk(XENLOG_G_INFO, "%pv: deliver interrupt in remapping mode,gvec:%02x\n", diff --git a/xen/include/xen/hvm/irq.h b/xen/include/xen/hvm/irq.h index f041252..671a6f2 100644 --- a/xen/include/xen/hvm/irq.h +++ b/xen/include/xen/hvm/irq.h @@ -63,6 +63,7 @@ struct hvm_gmsi_info { uint32_t gvec; uint32_t gflags; int dest_vcpu_id; /* -1 :multi-dest, non-negative: dest_vcpu_id */ + bool posted; /* directly deliver to guest via VT-d PI? */ }; struct hvm_girq_dpci_mapping {