From patchwork Wed Mar 29 05:11:50 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chao Gao X-Patchwork-Id: 9651451 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C4712602C8 for ; Wed, 29 Mar 2017 12:17:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B3E11283E1 for ; Wed, 29 Mar 2017 12:17:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A7CA128478; Wed, 29 Mar 2017 12:17:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=2.0 tests=BAYES_00, DATE_IN_PAST_06_12, DKIM_SIGNED, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D24B6283E1 for ; Wed, 29 Mar 2017 12:17:03 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ctCUn-0003xe-FE; Wed, 29 Mar 2017 12:14:37 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ctCUl-0003xH-M6 for xen-devel@lists.xen.org; Wed, 29 Mar 2017 12:14:35 +0000 Received: from [193.109.254.147] by server-7.bemta-6.messagelabs.com id 9D/4E-04817-B25ABD85; Wed, 29 Mar 2017 12:14:35 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrBLMWRWlGSWpSXmKPExsVywNwkQldr6e0 Igy1fNCyWfFzM4sDocXT3b6YAxijWzLyk/IoE1oymB4/ZCnZpVDx7JdrAeFW2i5GLg0XgFpPE u523mUEcIYFpjBKTO9YwdTFyckgI8EocWTaDFcL2k/g4ZwE7iC0kUC6xtvUlI4jNJqAscfFrL xuILSIgLXHt82WgOAcHs0C+xNMesBJhgViJ3s8P2EDCLAKqEluv1ICEeQUcJdaeesIGMV1BYs rD98wgNqeAk8T5XWeYITY5Sqxb8ZBlAiPfAkaGVYwaxalFZalFukYWeklFmekZJbmJmTm6hgZ mermpxcWJ6ak5iUnFesn5uZsYgSHCAAQ7GM+vDTzEKMnBpCTKe8LwdoQQX1J+SmVGYnFGfFFp TmrxIUYZDg4lCV6VJUA5waLU9NSKtMwcYLDCpCU4eJREeC0WA6V5iwsSc4sz0yFSpxgVpcR5f 4MkBEASGaV5cG2wCLnEKCslzMsIdIgQT0FqUW5mCar8K0ZxDkYlYd6/IFN4MvNK4Ka/AlrMBL RY3OYWyOKSRISUVAOjrrv11J2Lk5//kFxbuWnFl1JDe6ui3VbrBRZWb5/w/lXK18L2jqJHk94 9d18m0Pfs9PJ3d5SehjY8ZpLjXD55wnzHYAGj0ru3Nk26lK5y7n2Bz/7ESweEF3LFfAqs3Oiu su7jyeU+L1d0FvnKybJsftG1ztma9aDtglnffPIduGco5T27zZ8upMRSnJFoqMVcVJwIABT7j 2yLAgAA X-Env-Sender: chao.gao@intel.com X-Msg-Ref: server-9.tower-27.messagelabs.com!1490789671!94414712!2 X-Originating-IP: [192.55.52.88] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTkyLjU1LjUyLjg4ID0+IDM3NDcyNQ==\n X-StarScan-Received: X-StarScan-Version: 9.2.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 19857 invoked from network); 29 Mar 2017 12:14:34 -0000 Received: from mga01.intel.com (HELO mga01.intel.com) (192.55.52.88) by server-9.tower-27.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 29 Mar 2017 12:14:34 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=intel.com; i=@intel.com; q=dns/txt; s=intel; t=1490789674; x=1522325674; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=DBf+sfbgV4fuStgOzdXF1L+oYvmtYohfEv1nKoSkBZ4=; b=E7GS/6mtBvXYoEK/fcVNUGAf+nzM+FxiPJWUupkVUHFpP7Aw7sHXOMyK X+XXJIeNYLWK+tMIhc2qrUuwUSiKhA==; Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Mar 2017 05:14:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.36,241,1486454400"; d="scan'208";a="80556979" Received: from skl-2s3.sh.intel.com ([10.239.48.35]) by orsmga005.jf.intel.com with ESMTP; 29 Mar 2017 05:14:30 -0700 From: Chao Gao To: xen-devel@lists.xen.org Date: Wed, 29 Mar 2017 13:11:50 +0800 Message-Id: <1490764315-7162-2-git-send-email-chao.gao@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1490764315-7162-1-git-send-email-chao.gao@intel.com> References: <1490764315-7162-1-git-send-email-chao.gao@intel.com> Cc: Andrew Cooper , Jan Beulich , Chao Gao Subject: [Xen-devel] [PATCH v11 1/6] passthrough: don't migrate pirq when it is delivered through VT-d PI X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP When a vCPU migrated to another pCPU, pt irqs binded to this vCPU also needed migration. When VT-d PI is enabled, interrupt vector will be recorded to a main memory resident data-structure and a notification whose destination is decided by NDST is generated. NDST is properly adjusted during vCPU migration so pirq directly injected to guest needn't be migrated. This patch adds a indicator, @posted, to show whether the pt irq is delivered through VT-d PI. Signed-off-by: Chao Gao --- v11: - rename the indicator to 'posted' - move setting 'posted' field to event lock un-locked region. v10: - Newly added. xen/arch/x86/hvm/hvm.c | 3 +++ xen/drivers/passthrough/io.c | 62 +++++++++----------------------------------- xen/include/xen/hvm/irq.h | 1 + 3 files changed, 16 insertions(+), 50 deletions(-) diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index 0282986..2d8de16 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -438,6 +438,9 @@ static int hvm_migrate_pirq(struct domain *d, struct hvm_pirq_dpci *pirq_dpci, struct vcpu *v = arg; if ( (pirq_dpci->flags & HVM_IRQ_DPCI_MACH_MSI) && + (pirq_dpci->flags & HVM_IRQ_DPCI_GUEST_MSI) && + /* Needn't migrate pirq if this pirq is delivered to guest directly.*/ + (!pirq_dpci->gmsi.posted) && (pirq_dpci->gmsi.dest_vcpu_id == v->vcpu_id) ) { struct irq_desc *desc = diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c index 080183e..d53976c 100644 --- a/xen/drivers/passthrough/io.c +++ b/xen/drivers/passthrough/io.c @@ -259,52 +259,6 @@ static struct vcpu *vector_hashing_dest(const struct domain *d, return dest; } -/* - * The purpose of this routine is to find the right destination vCPU for - * an interrupt which will be delivered by VT-d posted-interrupt. There - * are several cases as below: - * - * - For lowest-priority interrupts, use vector-hashing mechanism to find - * the destination. - * - Otherwise, for single destination interrupt, it is straightforward to - * find the destination vCPU and return true. - * - For multicast/broadcast vCPU, we cannot handle it via interrupt posting, - * so return NULL. - */ -static struct vcpu *pi_find_dest_vcpu(const struct domain *d, uint32_t dest_id, - bool_t dest_mode, uint8_t delivery_mode, - uint8_t gvec) -{ - unsigned int dest_vcpus = 0; - struct vcpu *v, *dest = NULL; - - switch ( delivery_mode ) - { - case dest_LowestPrio: - return vector_hashing_dest(d, dest_id, dest_mode, gvec); - case dest_Fixed: - for_each_vcpu ( d, v ) - { - if ( !vlapic_match_dest(vcpu_vlapic(v), NULL, APIC_DEST_NOSHORT, - dest_id, dest_mode) ) - continue; - - dest_vcpus++; - dest = v; - } - - /* For fixed mode, we only handle single-destination interrupts. */ - if ( dest_vcpus == 1 ) - return dest; - - break; - default: - break; - } - - return NULL; -} - int pt_irq_create_bind( struct domain *d, xen_domctl_bind_pt_irq_t *pt_irq_bind) { @@ -365,6 +319,7 @@ int pt_irq_create_bind( { uint8_t dest, dest_mode, delivery_mode; int dest_vcpu_id; + const struct vcpu *vcpu; if ( !(pirq_dpci->flags & HVM_IRQ_DPCI_MAPPED) ) { @@ -442,17 +397,24 @@ int pt_irq_create_bind( dest_vcpu_id = hvm_girq_dest_2_vcpu_id(d, dest, dest_mode); pirq_dpci->gmsi.dest_vcpu_id = dest_vcpu_id; spin_unlock(&d->event_lock); + + pirq_dpci->gmsi.posted = false; + vcpu = (dest_vcpu_id >= 0) ? d->vcpu[dest_vcpu_id] : NULL; + if ( iommu_intpost && (delivery_mode == dest_LowestPrio) ) + { + vcpu = vector_hashing_dest(d, dest, dest_mode, + pirq_dpci->gmsi.gvec); + if ( vcpu ) + pirq_dpci->gmsi.posted = true; + } if ( dest_vcpu_id >= 0 ) hvm_migrate_pirqs(d->vcpu[dest_vcpu_id]); /* Use interrupt posting if it is supported. */ if ( iommu_intpost ) { - const struct vcpu *vcpu = pi_find_dest_vcpu(d, dest, dest_mode, - delivery_mode, pirq_dpci->gmsi.gvec); - if ( vcpu ) - pi_update_irte( vcpu, info, pirq_dpci->gmsi.gvec ); + pi_update_irte(vcpu, info, pirq_dpci->gmsi.gvec); else dprintk(XENLOG_G_INFO, "%pv: deliver interrupt in remapping mode,gvec:%02x\n", diff --git a/xen/include/xen/hvm/irq.h b/xen/include/xen/hvm/irq.h index d3f8623..566854a 100644 --- a/xen/include/xen/hvm/irq.h +++ b/xen/include/xen/hvm/irq.h @@ -63,6 +63,7 @@ struct hvm_gmsi_info { uint32_t gvec; uint32_t gflags; int dest_vcpu_id; /* -1 :multi-dest, non-negative: dest_vcpu_id */ + bool posted; /* directly deliver to guest via VT-d PI? */ }; struct hvm_girq_dpci_mapping {