From patchwork Wed Nov 13 15:59:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 11242217 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1D3EB1850 for ; Wed, 13 Nov 2019 16:01:28 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EACDF235FF for ; Wed, 13 Nov 2019 16:01:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="XQISJW2q" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EACDF235FF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iUv3q-0000Bk-S0; Wed, 13 Nov 2019 16:00:02 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iUv3p-0008As-QB for xen-devel@lists.xenproject.org; Wed, 13 Nov 2019 16:00:01 +0000 X-Inumbo-ID: a5abdafe-062e-11ea-9631-bc764e2007e4 Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id a5abdafe-062e-11ea-9631-bc764e2007e4; Wed, 13 Nov 2019 16:00:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1573660801; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bgHkkgbISmiDZbwR1pmIMN6eUQMUp30B6FtQLTgTfZ4=; b=XQISJW2qGS17H0WgdcE//FU6HAdbBwK53NaaemamQ7wUXiYf6C2tIjXa Fbd5o2+QG8ae8X6MFg/Ft4xCbecWbWbMza1i5mjshLojClvlHPCWfwq1A M0MpCNknWC1cgewplQ1m1L1HMyvApDSkfB5FfzAGo1GAeMLmZi+5vlwOd 4=; Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa6.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: kDh40hpe6wgfYzYyYYJhlqZDRKgRto9LQj3Fv3twmxiRQLSCnzdIZ4a9A4hY92swmdoFyu6dEJ BuK2dNt/j6WDi/XGF9JHmH2huznDeusunCXPVVPoc82TogLjK8ZhKJJ84rJ3syLQmgYW/QfEuK W/c7FhbHIgnlzOprXS/9e03TqGR8kqhLy0k87Gc0E/SMrqIC4WSmrqgEtG29mgblx+Wbuw8GzR GbyDtEKzzVm7NSmtBmN0+B9Y06F8kL2mD09gu3av2ewfP8GvG1pkDBBtV1WZ4XgxKocVKd/tjB 5Qk= X-SBRS: 2.7 X-MesageID: 8684200 X-Ironport-Server: esa6.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.68,300,1569297600"; d="scan'208";a="8684200" From: Roger Pau Monne To: Date: Wed, 13 Nov 2019 16:59:38 +0100 Message-ID: <20191113155940.81837-2-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191113155940.81837-1-roger.pau@citrix.com> References: <20191113155940.81837-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH for-4.13 v4 1/3] vmx: add ASSERT to prevent syncing PIR to IRR... X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Kevin Tian , Jun Nakajima , Wei Liu , Andrew Cooper , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" ... if the vCPU is different than the one currently running or if it's running on a different pCPU. No functional change intended. Suggested by: Andrew Cooper Signed-off-by: Roger Pau Monné --- Cc: Juergen Gross --- xen/arch/x86/hvm/vmx/vmx.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index a55ff37733..e5e674c373 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -2054,6 +2054,17 @@ static void vmx_sync_pir_to_irr(struct vcpu *v) unsigned int group, i; DECLARE_BITMAP(pending_intr, NR_VECTORS); + if ( v != current && v->is_running ) + { + /* + * Syncing PIR to IRR must not be done behind the back of the CPU, + * since the IRR is controlled by the hardware when the vCPU is + * executing. + */ + ASSERT_UNREACHABLE(); + return; + } + if ( !pi_test_and_clear_on(&v->arch.hvm.vmx.pi_desc) ) return; From patchwork Wed Nov 13 15:59:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 11242219 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D88FE13B1 for ; Wed, 13 Nov 2019 16:01:29 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A8DA7235FA for ; Wed, 13 Nov 2019 16:01:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="BoIres4U" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A8DA7235FA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iUv3u-0000eH-5I; Wed, 13 Nov 2019 16:00:06 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iUv3t-0000Th-8r for xen-devel@lists.xenproject.org; Wed, 13 Nov 2019 16:00:05 +0000 X-Inumbo-ID: a72de23c-062e-11ea-a237-12813bfff9fa Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id a72de23c-062e-11ea-a237-12813bfff9fa; Wed, 13 Nov 2019 16:00:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1573660803; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=X1oDbAmauwWtuFAWXlOr5DGa14RPLRDRHRqxVKGG0ms=; b=BoIres4U4JLaM2rP9m7RFajzjboOEYCjvL+m9CmIOcK7hnXgFCDZpFqh NS8/7sceW3qxkYgd51ClBusOQhaJItjEdT00oKnsAkj57YB/mfi4mT4/o 70EUu3s9axVRKD3qZhB2qmBPXJ0RdY6qiYc9vr/Hdxa+jf5K3t5xjSWSZ Y=; Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa4.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: Z2R8kXFlY7xamkobaWVrB6yuC6+F0jWe072S3VWtlWoUEuVZn4N0DWZlBeVncTWSTF8Crf46XR IEap50omgzJ1ce8Vnb5jMCWEz87Prh2tV4ZgbiCeY/UWwr0D2NjNl/fcWz+hE01f8uJmK2k8Vf K1VOGLUElFH3qYznXl9/g1Wcq3p4XNKD/NeKe3N3mqPiabC4VYhG/jo/5Rk+O470Ztj/ukKAHw m7PU4/rtK0HISvPH7wCPn3jrVyxutoMaOU8H9TmD7IaXRhO/8JrUl9HNLs51YDsZDioDAxTN8l WrQ= X-SBRS: 2.7 X-MesageID: 8804583 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.68,300,1569297600"; d="scan'208";a="8804583" From: Roger Pau Monne To: Date: Wed, 13 Nov 2019 16:59:39 +0100 Message-ID: <20191113155940.81837-3-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191113155940.81837-1-roger.pau@citrix.com> References: <20191113155940.81837-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH for-4.13 v4 2/3] x86/passthrough: fix migration of MSI when using posted interrupts X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Wei Liu , Andrew Cooper , Joe Jin , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" When using posted interrupts and the guest migrates MSI from vCPUs Xen needs to flush any pending PIRR vectors on the previous vCPU, or else those vectors could get wrongly injected at a later point when the MSI fields are already updated. The usage of a fixed vCPU in lowest priority mode when using VT-d posted interrupts is also removed, and as a result VT-d posted interrupts are not used together with lowest priority mode and multiple destinations. That forces vlapic_lowest_prio to be called in order to select the destination vCPU during interrupt dispatch. Note that PIRR is synced to IRR both in pt_irq_destroy_bind and pt_irq_create_bind when the interrupt delivery data is being updated. Reported-by: Joe Jin Signed-off-by: Roger Pau Monné Tested-by: Joe Jin --- Cc: Joe Jin Cc: Juergen Gross --- Changes since v3: - In multi-destination mode make sure all destination vCPUs have PIR synced to IRR by using a bitmap. - Drop the bogus selection of a fixed vCPU when using lowest priority mode. Changes since v2: - Also sync PIRR with IRR when using CPU posted interrupts. - Force the selection of a specific vCPU when using posted interrupts for multi-dest. - Change vmsi_deliver_pirq to honor dest_vcpu_id. Changes since v1: - Store the vcpu id also in multi-dest mode if the interrupt is bound to a vcpu for posted delivery. - s/#if/#ifdef/. --- xen/arch/x86/hvm/hvm.c | 31 ++++++++ xen/arch/x86/hvm/vlapic.c | 19 +++++ xen/arch/x86/hvm/vmsi.c | 23 ------ xen/drivers/passthrough/io.c | 118 ++++++++++++++----------------- xen/include/asm-x86/hvm/hvm.h | 5 +- xen/include/asm-x86/hvm/vlapic.h | 3 + 6 files changed, 110 insertions(+), 89 deletions(-) diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index 06a7b40107..0e3379fa6f 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -43,6 +43,7 @@ #include #include #include +#include #include #include #include @@ -5266,6 +5267,36 @@ void hvm_set_segment_register(struct vcpu *v, enum x86_segment seg, alternative_vcall(hvm_funcs.set_segment_register, v, seg, reg); } +int hvm_intr_get_dests(struct domain *d, uint8_t dest, uint8_t dest_mode, + uint8_t delivery_mode, unsigned long *vcpus) +{ + struct vcpu *v; + + switch ( delivery_mode ) + { + case dest_LowestPrio: + /* + * Get all the possible destinations, but note that lowest priority + * mode is only going to inject the interrupt to the vCPU running at + * the least privilege level. + * + * Fallthrough + */ + case dest_Fixed: + for_each_vcpu ( d, v ) + if ( vlapic_match_dest(vcpu_vlapic(v), NULL, 0, dest, dest_mode) ) + __set_bit(v->vcpu_id, vcpus); + break; + + default: + gprintk(XENLOG_WARNING, "unsupported interrupt delivery mode %u\n", + delivery_mode); + return -EINVAL; + } + + return 0; +} + /* * Local variables: * mode: C diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c index 9466258d6f..9d9c6d391a 100644 --- a/xen/arch/x86/hvm/vlapic.c +++ b/xen/arch/x86/hvm/vlapic.c @@ -112,6 +112,25 @@ static void sync_pir_to_irr(struct vcpu *v) alternative_vcall(hvm_funcs.sync_pir_to_irr, v); } +void domain_sync_vlapic_pir(struct domain *d, unsigned long *vcpus) +{ + unsigned int id; + + if ( !bitmap_weight(vcpus, d->max_vcpus) ) + return; + + for ( id = find_first_bit(vcpus, d->max_vcpus); + id < d->max_vcpus; + id = find_next_bit(vcpus, d->max_vcpus, id + 1) ) + { + if ( d->vcpu[id] != current ) + vcpu_pause(d->vcpu[id]); + sync_pir_to_irr(d->vcpu[id]); + if ( d->vcpu[id] != current ) + vcpu_unpause(d->vcpu[id]); + } +} + static int vlapic_find_highest_irr(struct vlapic *vlapic) { sync_pir_to_irr(vlapic_vcpu(vlapic)); diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c index 6597d9f719..66891d7d20 100644 --- a/xen/arch/x86/hvm/vmsi.c +++ b/xen/arch/x86/hvm/vmsi.c @@ -121,29 +121,6 @@ void vmsi_deliver_pirq(struct domain *d, const struct hvm_pirq_dpci *pirq_dpci) vmsi_deliver(d, vector, dest, dest_mode, delivery_mode, trig_mode); } -/* Return value, -1 : multi-dests, non-negative value: dest_vcpu_id */ -int hvm_girq_dest_2_vcpu_id(struct domain *d, uint8_t dest, uint8_t dest_mode) -{ - int dest_vcpu_id = -1, w = 0; - struct vcpu *v; - - if ( d->max_vcpus == 1 ) - return 0; - - for_each_vcpu ( d, v ) - { - if ( vlapic_match_dest(vcpu_vlapic(v), NULL, 0, dest, dest_mode) ) - { - w++; - dest_vcpu_id = v->vcpu_id; - } - } - if ( w > 1 ) - return -1; - - return dest_vcpu_id; -} - /* MSI-X mask bit hypervisor interception */ struct msixtbl_entry { diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c index b292e79382..5289e89bc1 100644 --- a/xen/drivers/passthrough/io.c +++ b/xen/drivers/passthrough/io.c @@ -219,62 +219,6 @@ void free_hvm_irq_dpci(struct hvm_irq_dpci *dpci) xfree(dpci); } -/* - * This routine handles lowest-priority interrupts using vector-hashing - * mechanism. As an example, modern Intel CPUs use this method to handle - * lowest-priority interrupts. - * - * Here is the details about the vector-hashing mechanism: - * 1. For lowest-priority interrupts, store all the possible destination - * vCPUs in an array. - * 2. Use "gvec % max number of destination vCPUs" to find the right - * destination vCPU in the array for the lowest-priority interrupt. - */ -static struct vcpu *vector_hashing_dest(const struct domain *d, - uint32_t dest_id, - bool dest_mode, - uint8_t gvec) - -{ - unsigned long *dest_vcpu_bitmap; - unsigned int dest_vcpus = 0; - struct vcpu *v, *dest = NULL; - unsigned int i; - - dest_vcpu_bitmap = xzalloc_array(unsigned long, - BITS_TO_LONGS(d->max_vcpus)); - if ( !dest_vcpu_bitmap ) - return NULL; - - for_each_vcpu ( d, v ) - { - if ( !vlapic_match_dest(vcpu_vlapic(v), NULL, APIC_DEST_NOSHORT, - dest_id, dest_mode) ) - continue; - - __set_bit(v->vcpu_id, dest_vcpu_bitmap); - dest_vcpus++; - } - - if ( dest_vcpus != 0 ) - { - unsigned int mod = gvec % dest_vcpus; - unsigned int idx = 0; - - for ( i = 0; i <= mod; i++ ) - { - idx = find_next_bit(dest_vcpu_bitmap, d->max_vcpus, idx) + 1; - BUG_ON(idx > d->max_vcpus); - } - - dest = d->vcpu[idx - 1]; - } - - xfree(dest_vcpu_bitmap); - - return dest; -} - int pt_irq_create_bind( struct domain *d, const struct xen_domctl_bind_pt_irq *pt_irq_bind) { @@ -345,6 +289,8 @@ int pt_irq_create_bind( const struct vcpu *vcpu; uint32_t gflags = pt_irq_bind->u.msi.gflags & ~XEN_DOMCTL_VMSI_X86_UNMASKED; + DECLARE_BITMAP(dest_vcpus, MAX_VIRT_CPUS) = { }; + DECLARE_BITMAP(prev_vcpus, MAX_VIRT_CPUS) = { }; if ( !(pirq_dpci->flags & HVM_IRQ_DPCI_MAPPED) ) { @@ -411,6 +357,24 @@ int pt_irq_create_bind( pirq_dpci->gmsi.gvec = pt_irq_bind->u.msi.gvec; pirq_dpci->gmsi.gflags = gflags; + if ( pirq_dpci->gmsi.dest_vcpu_id != -1 ) + __set_bit(pirq_dpci->gmsi.dest_vcpu_id, prev_vcpus); + else + { + /* + * If previous configuration has multiple possible + * destinations record them in order to sync the PIR to IRR + * afterwards. + */ + dest = MASK_EXTR(pirq_dpci->gmsi.gflags, + XEN_DOMCTL_VMSI_X86_DEST_ID_MASK); + dest_mode = pirq_dpci->gmsi.gflags & + XEN_DOMCTL_VMSI_X86_DM_MASK; + delivery_mode = MASK_EXTR(pirq_dpci->gmsi.gflags, + XEN_DOMCTL_VMSI_X86_DELIV_MASK); + hvm_intr_get_dests(d, dest, dest_mode, delivery_mode, + prev_vcpus); + } } } /* Calculate dest_vcpu_id for MSI-type pirq migration. */ @@ -420,20 +384,16 @@ int pt_irq_create_bind( delivery_mode = MASK_EXTR(pirq_dpci->gmsi.gflags, XEN_DOMCTL_VMSI_X86_DELIV_MASK); - dest_vcpu_id = hvm_girq_dest_2_vcpu_id(d, dest, dest_mode); + hvm_intr_get_dests(d, dest, dest_mode, delivery_mode, dest_vcpus); + dest_vcpu_id = bitmap_weight(dest_vcpus, d->max_vcpus) != 1 ? + -1 : find_first_bit(dest_vcpus, d->max_vcpus); pirq_dpci->gmsi.dest_vcpu_id = dest_vcpu_id; spin_unlock(&d->event_lock); pirq_dpci->gmsi.posted = false; vcpu = (dest_vcpu_id >= 0) ? d->vcpu[dest_vcpu_id] : NULL; - if ( iommu_intpost ) - { - if ( delivery_mode == dest_LowestPrio ) - vcpu = vector_hashing_dest(d, dest, dest_mode, - pirq_dpci->gmsi.gvec); - if ( vcpu ) - pirq_dpci->gmsi.posted = true; - } + if ( vcpu && iommu_intpost ) + pirq_dpci->gmsi.posted = true; if ( vcpu && is_iommu_enabled(d) ) hvm_migrate_pirq(pirq_dpci, vcpu); @@ -442,6 +402,9 @@ int pt_irq_create_bind( pi_update_irte(vcpu ? &vcpu->arch.hvm.vmx.pi_desc : NULL, info, pirq_dpci->gmsi.gvec); + if ( hvm_funcs.deliver_posted_intr ) + domain_sync_vlapic_pir(d, prev_vcpus); + if ( pt_irq_bind->u.msi.gflags & XEN_DOMCTL_VMSI_X86_UNMASKED ) { unsigned long flags; @@ -731,6 +694,31 @@ int pt_irq_destroy_bind( else if ( pirq_dpci && pirq_dpci->gmsi.posted ) pi_update_irte(NULL, pirq, 0); + if ( hvm_funcs.deliver_posted_intr ) + { + DECLARE_BITMAP(vcpus, MAX_VIRT_CPUS) = { }; + + if ( pirq_dpci->gmsi.dest_vcpu_id != -1 ) + __set_bit(pirq_dpci->gmsi.dest_vcpu_id, vcpus); + else + { + /* + * If previous configuration has multiple possible + * destinations record them in order to sync the PIR to IRR. + */ + uint8_t dest = MASK_EXTR(pirq_dpci->gmsi.gflags, + XEN_DOMCTL_VMSI_X86_DEST_ID_MASK); + uint8_t dest_mode = pirq_dpci->gmsi.gflags & + XEN_DOMCTL_VMSI_X86_DM_MASK; + uint8_t delivery_mode = MASK_EXTR(pirq_dpci->gmsi.gflags, + XEN_DOMCTL_VMSI_X86_DELIV_MASK); + + hvm_intr_get_dests(d, dest, dest_mode, delivery_mode, vcpus); + } + + domain_sync_vlapic_pir(d, vcpus); + } + if ( pirq_dpci && (pirq_dpci->flags & HVM_IRQ_DPCI_MAPPED) && list_empty(&pirq_dpci->digl_list) ) { diff --git a/xen/include/asm-x86/hvm/hvm.h b/xen/include/asm-x86/hvm/hvm.h index f86af09898..899665fed8 100644 --- a/xen/include/asm-x86/hvm/hvm.h +++ b/xen/include/asm-x86/hvm/hvm.h @@ -266,7 +266,6 @@ int vmsi_deliver( uint8_t delivery_mode, uint8_t trig_mode); struct hvm_pirq_dpci; void vmsi_deliver_pirq(struct domain *d, const struct hvm_pirq_dpci *); -int hvm_girq_dest_2_vcpu_id(struct domain *d, uint8_t dest, uint8_t dest_mode); enum hvm_intblk hvm_interrupt_blocked(struct vcpu *v, struct hvm_intack intack); @@ -336,6 +335,10 @@ unsigned long hvm_cr4_guest_valid_bits(const struct domain *d, bool restore); bool hvm_flush_vcpu_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v), void *ctxt); +/* Get all the possible destination vCPUs of an interrupt. */ +int hvm_intr_get_dests(struct domain *d, uint8_t dest, uint8_t dest_mode, + uint8_t delivery_mode, unsigned long *vcpus); + #ifdef CONFIG_HVM #define hvm_get_guest_tsc(v) hvm_get_guest_tsc_fixed(v, 0) diff --git a/xen/include/asm-x86/hvm/vlapic.h b/xen/include/asm-x86/hvm/vlapic.h index dde66b4f0f..2bc2ebadf0 100644 --- a/xen/include/asm-x86/hvm/vlapic.h +++ b/xen/include/asm-x86/hvm/vlapic.h @@ -150,4 +150,7 @@ bool_t vlapic_match_dest( const struct vlapic *target, const struct vlapic *source, int short_hand, uint32_t dest, bool_t dest_mode); +/* Sync the PIR to IRR of all vlapics in the vcpus bitmap. */ +void domain_sync_vlapic_pir(struct domain *d, unsigned long *vcpus); + #endif /* __ASM_X86_HVM_VLAPIC_H__ */ From patchwork Wed Nov 13 15:59:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 11242215 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0640B1709 for ; Wed, 13 Nov 2019 16:01:28 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D62F0235FA for ; Wed, 13 Nov 2019 16:01:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="ZKaY6pea" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D62F0235FA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iUv3v-0000ns-NV; Wed, 13 Nov 2019 16:00:07 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iUv3u-0000fG-Qv for xen-devel@lists.xenproject.org; Wed, 13 Nov 2019 16:00:06 +0000 X-Inumbo-ID: a89e677c-062e-11ea-9631-bc764e2007e4 Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id a89e677c-062e-11ea-9631-bc764e2007e4; Wed, 13 Nov 2019 16:00:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1573660806; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=W1q6MPBTvw8gTbLpGnDdKlKjlT/tkizJehDwxhmmQ7Q=; b=ZKaY6peaQmRVE/DHK/GzegBbAOz0j5ePWq95FEZLbhg+vXC3vWhjGXdG qgViu8I+Daf/iK/AH6R4DAofPEqlR42JrrtNawAUwCO0OEHftqhEAmXVr PKdrnwl/QGP833mN9WuQcurQhu/wir6kt3/EH2WuKWpgXRTfRx3ErrO5/ s=; Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: 3OjtuRnKjZQ4YN+O0aZjLSoAlcxCE+m8KxZoDXICOFvkoV+3lu+tZVLoUxNgShcEOkd47zHFHz 89MHJbBT4zVwOFHmhvBI9bpd7JqZyFSzp9cDANwtX0X7QyWxiewhGs24lvs2nRyo9MZIfAjgqc YN6tfjf8g/uGZUGfqRIOh/AAkkdAziTNLmSWg9HGBJyW2fEtNxNZNM9njUuyrqMmVS+GpkAb1z 9kf4MIRfBuN/odwYZf5kLmnaNRBkCG/HO4OuJqpHUH/UaO2ndqRd0SFBEoE2GELNPCArDBrlZz sXg= X-SBRS: 2.7 X-MesageID: 8629980 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.68,300,1569297600"; d="scan'208";a="8629980" From: Roger Pau Monne To: Date: Wed, 13 Nov 2019 16:59:40 +0100 Message-ID: <20191113155940.81837-4-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191113155940.81837-1-roger.pau@citrix.com> References: <20191113155940.81837-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH for-4.13 v4 3/3] x86/vioapic: sync PIR to IRR when modifying entries X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Andrew Cooper , Wei Liu , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" If posted interrutps are being used sync PIR to IRR when an unmasked vIO-APIC entry is modified. Do this in order to prevent vectors in the IRR being set after a change to a vIO-APIC entry has been performed. Signed-off-by: Roger Pau Monné --- Cc: Juergen Gross --- xen/arch/x86/hvm/vioapic.c | 46 +++++++++++++++++++++++++++++++++----- 1 file changed, 41 insertions(+), 5 deletions(-) diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c index 9aeef32a14..90e6d1c4e6 100644 --- a/xen/arch/x86/hvm/vioapic.c +++ b/xen/arch/x86/hvm/vioapic.c @@ -212,6 +212,44 @@ static int vioapic_hwdom_map_gsi(unsigned int gsi, unsigned int trig, return ret; } +static inline int pit_channel0_enabled(void) +{ + return pt_active(¤t->domain->arch.vpit.pt0); +} + +static void sync_vcpus_pir(struct domain *d, union vioapic_redir_entry *ent, + unsigned int irq) +{ + DECLARE_BITMAP(vcpus, MAX_VIRT_CPUS) = { }; + + switch ( ent->fields.delivery_mode ) + { + case dest_LowestPrio: + case dest_Fixed: +#ifdef IRQ0_SPECIAL_ROUTING + if ( (irq == hvm_isa_irq_to_gsi(0)) && pit_channel0_enabled() ) + { + __set_bit(0, vcpus); + break; + } +#endif + hvm_intr_get_dests(d, ent->fields.dest_id, ent->fields.dest_mode, + ent->fields.delivery_mode, vcpus); + break; + + case dest_NMI: + /* Nothing to do, NMIs are not signaled on the PIR. */ + break; + + default: + gdprintk(XENLOG_WARNING, "unsupported delivery mode %02u\n", + ent->fields.delivery_mode); + break; + } + + domain_sync_vlapic_pir(d, vcpus); +} + static void vioapic_write_redirent( struct hvm_vioapic *vioapic, unsigned int idx, int top_word, uint32_t val) @@ -235,6 +273,9 @@ static void vioapic_write_redirent( pent = &vioapic->redirtbl[idx]; ent = *pent; + if ( !ent.fields.mask && hvm_funcs.deliver_posted_intr ) + sync_vcpus_pir(d, pent, vioapic->base_gsi + idx); + if ( top_word ) { /* Contains only the dest_id. */ @@ -391,11 +432,6 @@ static void ioapic_inj_irq( vlapic_set_irq(target, vector, trig_mode); } -static inline int pit_channel0_enabled(void) -{ - return pt_active(¤t->domain->arch.vpit.pt0); -} - static void vioapic_deliver(struct hvm_vioapic *vioapic, unsigned int pin) { uint16_t dest = vioapic->redirtbl[pin].fields.dest_id;