From patchwork Tue Apr 26 10:51:44 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kiszka X-Patchwork-Id: 8937701 Return-Path: X-Original-To: patchwork-qemu-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 2C3799F441 for ; Tue, 26 Apr 2016 10:52:31 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 6711120123 for ; Tue, 26 Apr 2016 10:52:30 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 405C720115 for ; Tue, 26 Apr 2016 10:52:29 +0000 (UTC) Received: from localhost ([::1]:37427 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1av0bU-00083P-Kp for patchwork-qemu-devel@patchwork.kernel.org; Tue, 26 Apr 2016 06:52:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55828) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1av0b4-0007W9-Qg for qemu-devel@nongnu.org; Tue, 26 Apr 2016 06:52:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1av0b0-00007r-Jh for qemu-devel@nongnu.org; Tue, 26 Apr 2016 06:52:02 -0400 Received: from mout.web.de ([212.227.17.11]:54655) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1av0ay-00007Z-Kv for qemu-devel@nongnu.org; Tue, 26 Apr 2016 06:51:58 -0400 Received: from md1f2u6c.ww002.siemens.net ([95.157.58.223]) by smtp.web.de (mrweb101) with ESMTPSA (Nemesis) id 0LgpVS-1bX9Va3pX4-00oDrD; Tue, 26 Apr 2016 12:51:49 +0200 To: Peter Xu References: <1461055122-32378-1-git-send-email-peterx@redhat.com> <571DA823.1030003@web.de> <20160425071806.GF3261@pxdev.xzpeter.org> <571DC61C.9020006@web.de> <20160426073426.GD28545@pxdev.xzpeter.org> <571F1F7F.5050604@web.de> <571F23B2.3000902@web.de> <20160426103819.GE28545@pxdev.xzpeter.org> From: Jan Kiszka Message-ID: <571F4840.2060800@web.de> Date: Tue, 26 Apr 2016 12:51:44 +0200 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: <20160426103819.GE28545@pxdev.xzpeter.org> X-Provags-ID: V03:K0:DYVCJdtE0DNW+gV5UFQUtU5sBKi8wJ28VRkuJOm9j4SqKa7MuTm WSifV9dJj5M+oM2GMIikeIKOmUxno9Qkeqmt9IGykvIgAVibxXPL5YbA0Jxyyen5ElplxNN 73J6vwvHYqA2RCot8xFEkMS84iOJq7bbR2wQEu/6+kkoAViLtwwItsbKp4Z4jSXgH8CR6+G iHNkLX2uIBrVNUG9oAtNA== X-UI-Out-Filterresults: notjunk:1; V01:K0:4OXe7QCYZjc=:uufZiRK1Es51QZoikHMlAK UKSFamEDyfZpsTVLVf4c3mkpeVAlgN9TlQyhSUrKG3loQFvbO4yLGeChyRbixalqI2DPFvjmV V+uyvF/Kbdiw8spH5H5sdA+8356YdB96N15WYdr2LXPVdnp5uPryBTVUoiQV/bIdaWpClmj8z y8z6sHSXxqy/8IgtR1BSQY7IP7Wdson8/Sd9DLVecjAfS2yOOICTErw/kiBoj4VQlCKhqnttj U44jHVZTJTX2qq5DBeUQJIwK1bhLn61gqxbxlHaJTPR4Hs4mLs6c2rr1mjobeVHVod+hOCCWD qXPeV7lLC6OfoeBheZJqv1QQQOi2377nklB/qS1090MfQhvB4sp0m02rAR8AkAUXsDZMHsosC LuchhJlt0SBIEHegDcOaGbrWvag90AnWJtCqyjHlzJNL9+G+gW3SDz1koY16rC+LPo1cRKmKw UkAf/grwX+4ORFf3k4hBU0XtF5pZdGZuoKCRq/8giCgOrZ2Mdzd59vxqBiYVOxaA7RdPD/U2H Tci7Ii0VSxHuYDOQ60ikbdfR9/FRuJESM9HWPYtoZYXS9cNrFLf3tgOq9q04SOqIbItiU7hrg t66vVI90YC3Kgt8pJsv/8JsGP0QZogO7pUzaPVuzbT5dC82Wy3cOW0TfuxUlgB3aSf0D4c1FA K9+OK4RozRg3uxMfJV8KUgwHHYZl9i79ms3Bw6ucGe5vO44xBP+RjxfZXrKEmgFuRzvhdv8cR xgDPws39JpiqrrhatWD3wrHV3Gm6dv+xavO1IxfoD53EDmmexHvipIiqTR3ye7QRmH0lbCVeH SeAAtIR X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 212.227.17.11 Subject: Re: [Qemu-devel] [PATCH v4 00/16] IOMMU: Enable interrupt remapping for Intel IOMMU X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: ehabkost@redhat.com, mst@redhat.com, jasowang@redhat.com, rkrcmar@redhat.com, qemu-devel@nongnu.org, alex.williamson@redhat.com, wexu@redhat.com, pbonzini@redhat.com, marcel@redhat.com, imammedo@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 2016-04-26 12:38, Peter Xu wrote: >>>> Hi, Jan, >>>> >>>> The above issue should be caused by EOI missing of level-triggered >>>> interrupts. Before that, I was always using edge-triggered >>>> interrupts for test, so didn't encounter this one. Would you please >>>> help try below patch? It can be applied directly onto the series, >>>> and should solve the issue (it works on my test vm, and I'll take it >>>> in v5 as well if it also works for you): >>>> >>> >>> Works here as well. I even made EIM working with some hack, though >>> Jailhouse spits out strange warnings, despite it works fine (x2apic >>> mode, split irqchip). >> >> Corrections: the warnings are issued by qemu, not Jailhouse, e.g. >> >> qemu-system-x86_64: VT-d Failed to remap interrupt for gsi 22. >> >> I suspect that comes from the hand-over phase of Jailhouse, when it >> mutes all interrupts in the system while reconfiguring IR and IOAPIC. >> >> Please convert this error (in kvm_arch_fixup_msi_route) into a trace >> point. It shall not annoy the host. Also check if you have more of such >> guest-triggerable error messages. > > Okay. This should be the only one. I can use trace instead. > > Meanwhile, I still suppose we should not seen it even with > error_report().. Would this happen when boot e.g. generic kernels? No, this is - most probably, still need to validate in details - an effect due to the special case with Jailhouse that interrupt and VT-d management is handed over from a running OS to a hypervisor. Jan PS: Current quick hack for EIM support (lacks compat mode blocking at least): diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 1082ab5..709a92c 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -907,6 +907,7 @@ static void vtd_interrupt_remap_table_setup(IntelIOMMUState *s) value = vtd_get_quad_raw(s, DMAR_IRTA_REG); s->intr_size = 1UL << ((value & VTD_IRTA_SIZE_MASK) + 1); s->intr_root = value & VTD_IRTA_ADDR_MASK; + s->intr_eime = value & VTD_IRTA_EIME; QLIST_FOREACH(notifier, &s->iec_notifiers, list) { if (notifier->iec_notify) { @@ -2052,11 +2053,13 @@ static int vtd_remap_irq_get(IntelIOMMUState *iommu, uint16_t index, VTDIrq *irq irq->trigger_mode = irte.trigger_mode; irq->vector = irte.vector; irq->delivery_mode = irte.delivery_mode; - /* Not support EIM yet: please refer to vt-d 9.10 DST bits */ + irq->dest = irte.dest_id; + if (!iommu->intr_eime) { #define VTD_IR_APIC_DEST_MASK (0xff00ULL) #define VTD_IR_APIC_DEST_SHIFT (8) - irq->dest = (irte.dest_id & VTD_IR_APIC_DEST_MASK) >> \ - VTD_IR_APIC_DEST_SHIFT; + irq->dest = (irq->dest & VTD_IR_APIC_DEST_MASK) >> + VTD_IR_APIC_DEST_SHIFT; + } irq->dest_mode = irte.dest_mode; irq->redir_hint = irte.redir_hint; @@ -2316,7 +2319,7 @@ static void vtd_init(IntelIOMMUState *s) s->ecap = VTD_ECAP_QI | VTD_ECAP_IRO; if (ms->iommu_intr) { - s->ecap |= VTD_ECAP_IR; + s->ecap |= VTD_ECAP_IR | VTD_ECAP_EIM; } vtd_reset_context_cache(s); @@ -2370,10 +2373,9 @@ static void vtd_init(IntelIOMMUState *s) vtd_define_quad(s, DMAR_FRCD_REG_0_2, 0, 0, 0x8000000000000000ULL); /* - * Interrupt remapping registers, not support extended interrupt - * mode for now. + * Interrupt remapping registers. */ - vtd_define_quad(s, DMAR_IRTA_REG, 0, 0xfffffffffffff00fULL, 0); + vtd_define_quad(s, DMAR_IRTA_REG, 0, 0xfffffffffffff80fULL, 0); } /* Should not reset address_spaces when reset because devices will still use diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h index 10c20fe..72b0114 100644 --- a/hw/i386/intel_iommu_internal.h +++ b/hw/i386/intel_iommu_internal.h @@ -176,6 +176,7 @@ /* IRTA_REG */ #define VTD_IRTA_ADDR_MASK (VTD_HAW_MASK ^ 0xfffULL) +#define VTD_IRTA_EIME (1ULL << 11) #define VTD_IRTA_SIZE_MASK (0xfULL) /* ECAP_REG */ @@ -184,6 +185,7 @@ #define VTD_ECAP_QI (1ULL << 1) /* Interrupt Remapping support */ #define VTD_ECAP_IR (1ULL << 3) +#define VTD_ECAP_EIM (1ULL << 4) /* CAP_REG */ /* (offset >> 4) << 24 */ diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h index d7613af..c6c53c6 100644 --- a/include/hw/i386/intel_iommu.h +++ b/include/hw/i386/intel_iommu.h @@ -261,6 +261,7 @@ struct IntelIOMMUState { bool intr_enabled; /* Whether guest enabled IR */ dma_addr_t intr_root; /* Interrupt remapping table pointer */ uint32_t intr_size; /* Number of IR table entries */ + bool intr_eime; /* Extended interrupt mode enabled */ QLIST_HEAD(, VTD_IEC_Notifier) iec_notifiers; /* IEC notify list */ };