From patchwork Fri Jun 24 05:51:58 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quan Xu X-Patchwork-Id: 9196693 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 567B26088F for ; Fri, 24 Jun 2016 05:59:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 46763283EB for ; Fri, 24 Jun 2016 05:59:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3A8BC28495; Fri, 24 Jun 2016 05:59:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 78C5628434 for ; Fri, 24 Jun 2016 05:59:05 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bGK6n-00084Y-Fz; Fri, 24 Jun 2016 05:56:53 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bGK6l-00082o-Vx for xen-devel@lists.xen.org; Fri, 24 Jun 2016 05:56:52 +0000 Received: from [193.109.254.147] by server-14.bemta-14.messagelabs.com id 08/37-27600-3ABCC675; Fri, 24 Jun 2016 05:56:51 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrLLMWRWlGSWpSXmKPExsXS1tYhobvodE6 4wctOdYslHxezODB6HN39mymAMYo1My8pvyKBNWNmzy/WghWBFfd/H2VvYFxp18XIySEkUCmx 785sJhBbQoBX4siyGawQtr/Emkvf2SBqaiSmXegGq2ETUJGY0fyOHcQWEZCWuPb5MmMXIxcHs 8B8RomXj+6CJYQF3CQuLbrEAmKzCKhKrD/2ixHE5hVwlNizpZ8ZYoGCxLIva8FsTgEniVfXNj FDLHOU+PixkXUCI+8CRoZVjOrFqUVlqUW6FnpJRZnpGSW5iZk5uoaGJnq5qcXFiempOYlJxXr J+bmbGIHBwAAEOxg3TnI+xCjJwaQkyjttUU64EF9SfkplRmJxRnxRaU5q8SFGGQ4OJQlellNA OcGi1PTUirTMHGBYwqQlOHiURHjDQNK8xQWJucWZ6RCpU4yKUuK8R08CJQRAEhmleXBtsFi4x CgrJczLCHSIEE9BalFuZgmq/CtGcQ5GJWHeOyBTeDLzSuCmvwJazAS0+G5/NsjikkSElFQDY1 JYzbotm3e0Vzs+vlabd0s7mLHRRbndt3Wf3M2dlnq23/ifSJQKJTzXEv8gULqj9Z0W93Smrh1 LTYpU89jm2X7p2jjNecKvg80pt/boisfxrT+THFHqvbX6ss9xFsura+MjHxTUn9A68E2gkSt2 ea++beHTNW+Ub/DxHqzXmb1DjGdzU7C/EktxRqKhFnNRcSIAVjp5uoACAAA= X-Env-Sender: quan.xu@intel.com X-Msg-Ref: server-11.tower-27.messagelabs.com!1466747808!41110206!2 X-Originating-IP: [134.134.136.24] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTM0LjEzNC4xMzYuMjQgPT4gMzkwOTcx\n X-StarScan-Received: X-StarScan-Version: 8.46; banners=-,-,- X-VirusChecked: Checked Received: (qmail 8564 invoked from network); 24 Jun 2016 05:56:50 -0000 Received: from mga09.intel.com (HELO mga09.intel.com) (134.134.136.24) by server-11.tower-27.messagelabs.com with SMTP; 24 Jun 2016 05:56:50 -0000 Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga102.jf.intel.com with ESMTP; 23 Jun 2016 22:56:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.26,518,1459839600"; d="scan'208";a="127718736" Received: from xen-commits.sh.intel.com ([10.239.82.178]) by fmsmga004.fm.intel.com with ESMTP; 23 Jun 2016 22:56:50 -0700 From: "Xu, Quan" To: xen-devel@lists.xen.org Date: Fri, 24 Jun 2016 13:51:58 +0800 Message-Id: <1466747518-54402-7-git-send-email-quan.xu@intel.com> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1466747518-54402-1-git-send-email-quan.xu@intel.com> References: <1466747518-54402-1-git-send-email-quan.xu@intel.com> Cc: Kevin Tian , dario.faggioli@citrix.com, Feng Wu , Jan Beulich , Quan Xu Subject: [Xen-devel] [PATCH v12 6/6] vt-d: fix vt-d Device-TLB flush timeout issue X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Quan Xu If Device-TLB flush timed out, we hide the target ATS device immediately and crash the domain owning this ATS device. If impacted domain is hardware domain, just throw out a warning. By hiding the device, we make sure it can't be assigned to any domain any longer (see device_assigned). Signed-off-by: Quan Xu CC: Jan Beulich CC: Kevin Tian CC: Feng Wu --- v12: 1. a forward declaration struct pci_ats_dev*, instead of including ats.h. 2. eliminate the loop. 3. use the same logic for logging and crashing as I did in other series (despite I have moved the domain crash logic up to the generic IOMMU layer, I think I am better still leave it as is). 4. enhance dev_invalidate_sync() with ASSERT(). --- xen/drivers/passthrough/pci.c | 6 +-- xen/drivers/passthrough/vtd/extern.h | 5 ++- xen/drivers/passthrough/vtd/qinval.c | 75 +++++++++++++++++++++++++++++------ xen/drivers/passthrough/vtd/x86/ats.c | 9 +---- xen/include/xen/pci.h | 1 + 5 files changed, 71 insertions(+), 25 deletions(-) diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index 98936f55c..843dc88 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -419,7 +419,7 @@ static void free_pdev(struct pci_seg *pseg, struct pci_dev *pdev) xfree(pdev); } -static void _pci_hide_device(struct pci_dev *pdev) +void pci_hide_existing_device(struct pci_dev *pdev) { if ( pdev->domain ) return; @@ -436,7 +436,7 @@ int __init pci_hide_device(int bus, int devfn) pdev = alloc_pdev(get_pseg(0), bus, devfn); if ( pdev ) { - _pci_hide_device(pdev); + pci_hide_existing_device(pdev); rc = 0; } pcidevs_unlock(); @@ -466,7 +466,7 @@ int __init pci_ro_device(int seg, int bus, int devfn) } __set_bit(PCI_BDF2(bus, devfn), pseg->ro_map); - _pci_hide_device(pdev); + pci_hide_existing_device(pdev); return 0; } diff --git a/xen/drivers/passthrough/vtd/extern.h b/xen/drivers/passthrough/vtd/extern.h index 45357f2..efaff28 100644 --- a/xen/drivers/passthrough/vtd/extern.h +++ b/xen/drivers/passthrough/vtd/extern.h @@ -25,6 +25,7 @@ #define VTDPREFIX "[VT-D]" +struct pci_ats_dev; extern bool_t rwbf_quirk; void print_iommu_regs(struct acpi_drhd_unit *drhd); @@ -60,8 +61,8 @@ int dev_invalidate_iotlb(struct iommu *iommu, u16 did, u64 addr, unsigned int size_order, u64 type); int __must_check qinval_device_iotlb_sync(struct iommu *iommu, - u32 max_invs_pend, - u16 sid, u16 size, u64 addr); + struct pci_ats_dev *ats_dev, + u16 did, u16 size, u64 addr); unsigned int get_cache_line_size(void); void cacheline_flush(char *); diff --git a/xen/drivers/passthrough/vtd/qinval.c b/xen/drivers/passthrough/vtd/qinval.c index 4492b29..e4e2771 100644 --- a/xen/drivers/passthrough/vtd/qinval.c +++ b/xen/drivers/passthrough/vtd/qinval.c @@ -27,11 +27,11 @@ #include "dmar.h" #include "vtd.h" #include "extern.h" +#include "../ats.h" #define VTD_QI_TIMEOUT 1 -static int __must_check invalidate_sync(struct iommu *iommu, - bool_t flush_dev_iotlb); +static int __must_check invalidate_sync(struct iommu *iommu); static void print_qi_regs(struct iommu *iommu) { @@ -103,7 +103,7 @@ static int __must_check queue_invalidate_context_sync(struct iommu *iommu, unmap_vtd_domain_page(qinval_entries); - return invalidate_sync(iommu, 0); + return invalidate_sync(iommu); } static int __must_check queue_invalidate_iotlb_sync(struct iommu *iommu, @@ -140,7 +140,7 @@ static int __must_check queue_invalidate_iotlb_sync(struct iommu *iommu, qinval_update_qtail(iommu, index); spin_unlock_irqrestore(&iommu->register_lock, flags); - return invalidate_sync(iommu, 0); + return invalidate_sync(iommu); } static int __must_check queue_invalidate_wait(struct iommu *iommu, @@ -199,24 +199,73 @@ static int __must_check queue_invalidate_wait(struct iommu *iommu, return -EOPNOTSUPP; } -static int __must_check invalidate_sync(struct iommu *iommu, - bool_t flush_dev_iotlb) +static int __must_check invalidate_sync(struct iommu *iommu) { struct qi_ctrl *qi_ctrl = iommu_qi_ctrl(iommu); ASSERT(qi_ctrl->qinval_maddr); - return queue_invalidate_wait(iommu, 0, 1, 1, flush_dev_iotlb); + return queue_invalidate_wait(iommu, 0, 1, 1, 0); +} + +static void dev_invalidate_iotlb_timeout(struct iommu *iommu, u16 did, + struct pci_dev *pdev) +{ + struct domain *d = NULL; + + if ( test_bit(did, iommu->domid_bitmap) ) + d = rcu_lock_domain_by_id(iommu->domid_map[did]); + + /* + * In case the domain has been freed or the IOMMU domid bitmap is + * not valid, the device no longer belongs to this domain. + */ + if ( d == NULL ) + return; + + pcidevs_lock(); + ASSERT(pdev->domain); + list_del(&pdev->domain_list); + pdev->domain = NULL; + pci_hide_existing_device(pdev); + pcidevs_unlock(); + + if ( !d->is_shutting_down && printk_ratelimit() ) + printk(XENLOG_WARNING VTDPREFIX + " dom%d: ATS device %04x:%02x:%02x.%u flush failed\n", + d->domain_id, pdev->seg, pdev->bus, + PCI_SLOT(pdev->devfn), PCI_FUNC(pdev->devfn)); + + if ( !is_hardware_domain(d) ) + domain_crash(d); + + rcu_unlock_domain(d); +} + +static int __must_check dev_invalidate_sync(struct iommu *iommu, u16 did, + struct pci_dev *pdev) +{ + struct qi_ctrl *qi_ctrl = iommu_qi_ctrl(iommu); + int rc; + + ASSERT(qi_ctrl->qinval_maddr); + rc = queue_invalidate_wait(iommu, 0, 1, 1, 1); + + if ( rc == -ETIMEDOUT ) + dev_invalidate_iotlb_timeout(iommu, did, pdev); + + return rc; } int qinval_device_iotlb_sync(struct iommu *iommu, - u32 max_invs_pend, - u16 sid, u16 size, u64 addr) + struct pci_ats_dev *ats_dev, + u16 did, u16 size, u64 addr) { unsigned long flags; unsigned int index; u64 entry_base; struct qinval_entry *qinval_entry, *qinval_entries; + struct pci_dev *pdev = ats_dev->pci_dev; spin_lock_irqsave(&iommu->register_lock, flags); index = qinval_next_index(iommu); @@ -227,9 +276,9 @@ int qinval_device_iotlb_sync(struct iommu *iommu, qinval_entry->q.dev_iotlb_inv_dsc.lo.type = TYPE_INVAL_DEVICE_IOTLB; qinval_entry->q.dev_iotlb_inv_dsc.lo.res_1 = 0; - qinval_entry->q.dev_iotlb_inv_dsc.lo.max_invs_pend = max_invs_pend; + qinval_entry->q.dev_iotlb_inv_dsc.lo.max_invs_pend = ats_dev->ats_queue_depth; qinval_entry->q.dev_iotlb_inv_dsc.lo.res_2 = 0; - qinval_entry->q.dev_iotlb_inv_dsc.lo.sid = sid; + qinval_entry->q.dev_iotlb_inv_dsc.lo.sid = PCI_BDF2(pdev->bus, pdev->devfn); qinval_entry->q.dev_iotlb_inv_dsc.lo.res_3 = 0; qinval_entry->q.dev_iotlb_inv_dsc.hi.size = size; @@ -240,7 +289,7 @@ int qinval_device_iotlb_sync(struct iommu *iommu, qinval_update_qtail(iommu, index); spin_unlock_irqrestore(&iommu->register_lock, flags); - return invalidate_sync(iommu, 1); + return dev_invalidate_sync(iommu, did, pdev); } static int __must_check queue_invalidate_iec_sync(struct iommu *iommu, @@ -271,7 +320,7 @@ static int __must_check queue_invalidate_iec_sync(struct iommu *iommu, qinval_update_qtail(iommu, index); spin_unlock_irqrestore(&iommu->register_lock, flags); - ret = invalidate_sync(iommu, 0); + ret = invalidate_sync(iommu); /* * reading vt-d architecture register will ensure diff --git a/xen/drivers/passthrough/vtd/x86/ats.c b/xen/drivers/passthrough/vtd/x86/ats.c index a6c53ea..be2a155 100644 --- a/xen/drivers/passthrough/vtd/x86/ats.c +++ b/xen/drivers/passthrough/vtd/x86/ats.c @@ -121,15 +121,12 @@ int dev_invalidate_iotlb(struct iommu *iommu, u16 did, list_for_each_entry( pdev, &ats_devices, list ) { struct pci_dev *pci_dev = pdev->pci_dev; - u16 sid; bool_t sbit; int rc = 0; if ( !pci_dev ) continue; - sid = PCI_BDF2(pci_dev->bus, pci_dev->devfn); - /* Only invalidate devices that belong to this IOMMU */ if ( pdev->iommu != iommu ) continue; @@ -144,8 +141,7 @@ int dev_invalidate_iotlb(struct iommu *iommu, u16 did, /* invalidate all translations: sbit=1,bit_63=0,bit[62:12]=1 */ sbit = 1; addr = (~0UL << PAGE_SHIFT_4K) & 0x7FFFFFFFFFFFFFFF; - rc = qinval_device_iotlb_sync(iommu, pdev->ats_queue_depth, - sid, sbit, addr); + rc = qinval_device_iotlb_sync(iommu, pdev, did, sbit, addr); break; case DMA_TLB_PSI_FLUSH: if ( !device_in_domain(iommu, pdev, did) ) @@ -164,8 +160,7 @@ int dev_invalidate_iotlb(struct iommu *iommu, u16 did, addr |= (((u64)1 << (size_order - 1)) - 1) << PAGE_SHIFT_4K; } - rc = qinval_device_iotlb_sync(iommu, pdev->ats_queue_depth, - sid, sbit, addr); + rc = qinval_device_iotlb_sync(iommu, pdev, did, sbit, addr); break; default: dprintk(XENLOG_WARNING VTDPREFIX, "invalid vt-d flush type\n"); diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h index 6ed29dd..e4940cd 100644 --- a/xen/include/xen/pci.h +++ b/xen/include/xen/pci.h @@ -118,6 +118,7 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, int pci_remove_device(u16 seg, u8 bus, u8 devfn); int pci_ro_device(int seg, int bus, int devfn); int pci_hide_device(int bus, int devfn); +void pci_hide_existing_device(struct pci_dev *pdev); struct pci_dev *pci_get_pdev(int seg, int bus, int devfn); struct pci_dev *pci_get_real_pdev(int seg, int bus, int devfn); struct pci_dev *pci_get_pdev_by_domain(