From patchwork Wed Jun 1 09:05:21 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quan Xu X-Patchwork-Id: 9146851 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2CECD60761 for ; Wed, 1 Jun 2016 09:12:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1E36425819 for ; Wed, 1 Jun 2016 09:12:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 12F652679B; Wed, 1 Jun 2016 09:12:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6B2E425819 for ; Wed, 1 Jun 2016 09:11:59 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b829p-0001hv-B1; Wed, 01 Jun 2016 09:09:45 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b829n-0001hT-Qx for xen-devel@lists.xen.org; Wed, 01 Jun 2016 09:09:43 +0000 Received: from [193.109.254.147] by server-11.bemta-14.messagelabs.com id 39/DE-23272-656AE475; Wed, 01 Jun 2016 09:09:42 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrFLMWRWlGSWpSXmKPExsXS1tYhoRu2zC/ c4NEMCYslHxezODB6HN39mymAMYo1My8pvyKBNeNW9x2mgt+eFUfX3mFqYLxk2sXIwSEkUCnx /UJlFyMnh4QAr8SRZTNYQcISAv4SGw8mg4SFBGokFv9eywhiswmoSMxofscOYosISEtc+3wZL M4sUCVxv/s5mC0s4CXRdLydBcRmEVCV2P/mCBOIzSvgINHX948ZYpWCxLIva8FsTgFHiXmTZz NC7HKQOHJiOfsERt4FjAyrGNWLU4vKUot0jfSSijLTM0pyEzNzdA0NTfRyU4uLE9NTcxKTivW S83M3MQLDgAEIdjC2zHE+xCjJwaQkypua5xcuxJeUn1KZkVicEV9UmpNafIhRhoNDSYJXfSlQ TrAoNT21Ii0zBxiQMGkJDh4lEd4okDRvcUFibnFmOkTqFKOilDhvAEhCACSRUZoH1waLgkuMs lLCvIxAhwjxFKQW5WaWoMq/YhTnYFQS5o0BmcKTmVcCN/0V0GImoMXxGT4gi0sSEVJSDYwL5r 54Ix940i+1skElW0Qs4qPqDfnfH7m3Sd36ln2cyShC2WT9sT8BvkUsD+7Iff5R2x86b/7KwMi cjdELTq//rLOg3ohvlmLPPevWtI6E+nvnmV9Fu6dva+l9Y1476+BKh6u/3m3bYzXrlUqYTo7J HpmjP6b/WSiY5rRSXCU+vSCsI9yhbpESS3FGoqEWc1FxIgAmjcRefQIAAA== X-Env-Sender: quan.xu@intel.com X-Msg-Ref: server-15.tower-27.messagelabs.com!1464772181!45105252!1 X-Originating-IP: [134.134.136.24] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTM0LjEzNC4xMzYuMjQgPT4gMzkwOTcx\n X-StarScan-Received: X-StarScan-Version: 8.46; banners=-,-,- X-VirusChecked: Checked Received: (qmail 21474 invoked from network); 1 Jun 2016 09:09:41 -0000 Received: from mga09.intel.com (HELO mga09.intel.com) (134.134.136.24) by server-15.tower-27.messagelabs.com with SMTP; 1 Jun 2016 09:09:41 -0000 Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga102.jf.intel.com with ESMTP; 01 Jun 2016 02:09:40 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.26,400,1459839600"; d="scan'208";a="712116248" Received: from xen-commits.sh.intel.com ([10.239.82.178]) by FMSMGA003.fm.intel.com with ESMTP; 01 Jun 2016 02:09:40 -0700 From: "Xu, Quan" To: xen-devel@lists.xen.org Date: Wed, 1 Jun 2016 17:05:21 +0800 Message-Id: <1464771922-7794-3-git-send-email-quan.xu@intel.com> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1464771922-7794-1-git-send-email-quan.xu@intel.com> References: <1464771922-7794-1-git-send-email-quan.xu@intel.com> Cc: Quan Xu , kevin.tian@intel.com, feng.wu@intel.com, dario.faggioli@citrix.com, jbeulich@suse.com Subject: [Xen-devel] [Patch v11 2/3] vt-d: synchronize for Device-TLB flush one by one X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Quan Xu Today we do Device-TLB flush synchronization after issuing flush requests for all ATS devices belonging to a VM. Doing so however imposes a limitation, i.e. that we can not figure out which flush request is blocked in the flush queue list, based on VT-d spec. To prepare correct Device-TLB flush timeout handling in next patch, we change the behavior to synchronize for every Device-TLB flush request. So the Device-TLB flush interface is changed a little bit, by checking timeout within the function instead of outside of function. Accordingly we also do a similar change for flush interfaces of IOTLB/IEC/Context, i.e. moving synchronization into the function. Since there is no user of a non-synced interface, we just rename existing ones with _sync suffix. v11: 1. Add __must_check annotation to invalidate_sync(). 2. Remove the redundant invalidate_sync() in flush_iotlb_qi(). 3. Refactor invalidate_sync() to indicate whether we need to flush device IOTLB or not (I'll change it back in next patch, as I'll add a specific function - dev_invalidate_sync() for device IOTLB invalidation). Signed-off-by: Quan Xu Reviewed-by: Jan Beulich --- xen/drivers/passthrough/vtd/extern.h | 5 +-- xen/drivers/passthrough/vtd/qinval.c | 65 ++++++++++++++++++++--------------- xen/drivers/passthrough/vtd/x86/ats.c | 8 ++--- 3 files changed, 45 insertions(+), 33 deletions(-) diff --git a/xen/drivers/passthrough/vtd/extern.h b/xen/drivers/passthrough/vtd/extern.h index 6772839..45357f2 100644 --- a/xen/drivers/passthrough/vtd/extern.h +++ b/xen/drivers/passthrough/vtd/extern.h @@ -59,8 +59,9 @@ int ats_device(const struct pci_dev *, const struct acpi_drhd_unit *); int dev_invalidate_iotlb(struct iommu *iommu, u16 did, u64 addr, unsigned int size_order, u64 type); -int qinval_device_iotlb(struct iommu *iommu, - u32 max_invs_pend, u16 sid, u16 size, u64 addr); +int __must_check qinval_device_iotlb_sync(struct iommu *iommu, + u32 max_invs_pend, + u16 sid, u16 size, u64 addr); unsigned int get_cache_line_size(void); void cacheline_flush(char *); diff --git a/xen/drivers/passthrough/vtd/qinval.c b/xen/drivers/passthrough/vtd/qinval.c index 1a37565..9116588 100644 --- a/xen/drivers/passthrough/vtd/qinval.c +++ b/xen/drivers/passthrough/vtd/qinval.c @@ -30,6 +30,9 @@ #define IOMMU_QI_TIMEOUT MILLISECS(1) +static int __must_check invalidate_sync(struct iommu *iommu, + bool_t flush_dev_iotlb); + static void print_qi_regs(struct iommu *iommu) { u64 val; @@ -69,8 +72,10 @@ static void qinval_update_qtail(struct iommu *iommu, unsigned int index) dmar_writeq(iommu->reg, DMAR_IQT_REG, (val << QINVAL_INDEX_SHIFT)); } -static void queue_invalidate_context(struct iommu *iommu, - u16 did, u16 source_id, u8 function_mask, u8 granu) +static int __must_check queue_invalidate_context_sync(struct iommu *iommu, + u16 did, u16 source_id, + u8 function_mask, + u8 granu) { unsigned long flags; unsigned int index; @@ -97,10 +102,14 @@ static void queue_invalidate_context(struct iommu *iommu, spin_unlock_irqrestore(&iommu->register_lock, flags); unmap_vtd_domain_page(qinval_entries); + + return invalidate_sync(iommu, 0); } -static void queue_invalidate_iotlb(struct iommu *iommu, - u8 granu, u8 dr, u8 dw, u16 did, u8 am, u8 ih, u64 addr) +static int __must_check queue_invalidate_iotlb_sync(struct iommu *iommu, + u8 granu, u8 dr, u8 dw, + u16 did, u8 am, u8 ih, + u64 addr) { unsigned long flags; unsigned int index; @@ -130,6 +139,8 @@ static void queue_invalidate_iotlb(struct iommu *iommu, unmap_vtd_domain_page(qinval_entries); qinval_update_qtail(iommu, index); spin_unlock_irqrestore(&iommu->register_lock, flags); + + return invalidate_sync(iommu, 0); } static int __must_check queue_invalidate_wait(struct iommu *iommu, @@ -200,8 +211,9 @@ static int __must_check invalidate_sync(struct iommu *iommu, return 0; } -int qinval_device_iotlb(struct iommu *iommu, - u32 max_invs_pend, u16 sid, u16 size, u64 addr) +int qinval_device_iotlb_sync(struct iommu *iommu, + u32 max_invs_pend, + u16 sid, u16 size, u64 addr) { unsigned long flags; unsigned int index; @@ -230,15 +242,17 @@ int qinval_device_iotlb(struct iommu *iommu, qinval_update_qtail(iommu, index); spin_unlock_irqrestore(&iommu->register_lock, flags); - return 0; + return invalidate_sync(iommu, 1); } -static void queue_invalidate_iec(struct iommu *iommu, u8 granu, u8 im, u16 iidx) +static int __must_check queue_invalidate_iec_sync(struct iommu *iommu, + u8 granu, u8 im, u16 iidx) { unsigned long flags; unsigned int index; u64 entry_base; struct qinval_entry *qinval_entry, *qinval_entries; + int ret; spin_lock_irqsave(&iommu->register_lock, flags); index = qinval_next_index(iommu); @@ -258,14 +272,9 @@ static void queue_invalidate_iec(struct iommu *iommu, u8 granu, u8 im, u16 iidx) unmap_vtd_domain_page(qinval_entries); qinval_update_qtail(iommu, index); spin_unlock_irqrestore(&iommu->register_lock, flags); -} - -static int __iommu_flush_iec(struct iommu *iommu, u8 granu, u8 im, u16 iidx) -{ - int ret; - queue_invalidate_iec(iommu, granu, im, iidx); ret = invalidate_sync(iommu, 0); + /* * reading vt-d architecture register will ensure * draining happens in implementation independent way. @@ -277,12 +286,12 @@ static int __iommu_flush_iec(struct iommu *iommu, u8 granu, u8 im, u16 iidx) int iommu_flush_iec_global(struct iommu *iommu) { - return __iommu_flush_iec(iommu, IEC_GLOBAL_INVL, 0, 0); + return queue_invalidate_iec_sync(iommu, IEC_GLOBAL_INVL, 0, 0); } int iommu_flush_iec_index(struct iommu *iommu, u8 im, u16 iidx) { - return __iommu_flush_iec(iommu, IEC_INDEX_INVL, im, iidx); + return queue_invalidate_iec_sync(iommu, IEC_INDEX_INVL, im, iidx); } static int __must_check flush_context_qi(void *_iommu, u16 did, @@ -308,11 +317,9 @@ static int __must_check flush_context_qi(void *_iommu, u16 did, } if ( qi_ctrl->qinval_maddr != 0 ) - { - queue_invalidate_context(iommu, did, sid, fm, - type >> DMA_CCMD_INVL_GRANU_OFFSET); - ret = invalidate_sync(iommu, 0); - } + ret = queue_invalidate_context_sync(iommu, did, sid, fm, + type >> DMA_CCMD_INVL_GRANU_OFFSET); + return ret; } @@ -350,14 +357,18 @@ static int __must_check flush_iotlb_qi(void *_iommu, u16 did, u64 addr, if (cap_read_drain(iommu->cap)) dr = 1; /* Need to conside the ih bit later */ - queue_invalidate_iotlb(iommu, - type >> DMA_TLB_FLUSH_GRANU_OFFSET, dr, - dw, did, size_order, 0, addr); - if ( flush_dev_iotlb ) - ret = dev_invalidate_iotlb(iommu, did, addr, size_order, type); - rc = invalidate_sync(iommu, flush_dev_iotlb); + rc = queue_invalidate_iotlb_sync(iommu, + type >> DMA_TLB_FLUSH_GRANU_OFFSET, + dr, dw, did, size_order, 0, addr); if ( !ret ) ret = rc; + + if ( flush_dev_iotlb ) + { + rc = dev_invalidate_iotlb(iommu, did, addr, size_order, type); + if ( !ret ) + ret = rc; + } } return ret; diff --git a/xen/drivers/passthrough/vtd/x86/ats.c b/xen/drivers/passthrough/vtd/x86/ats.c index 334b9c1..dfa4d30 100644 --- a/xen/drivers/passthrough/vtd/x86/ats.c +++ b/xen/drivers/passthrough/vtd/x86/ats.c @@ -134,8 +134,8 @@ int dev_invalidate_iotlb(struct iommu *iommu, u16 did, /* invalidate all translations: sbit=1,bit_63=0,bit[62:12]=1 */ sbit = 1; addr = (~0UL << PAGE_SHIFT_4K) & 0x7FFFFFFFFFFFFFFF; - rc = qinval_device_iotlb(iommu, pdev->ats_queue_depth, - sid, sbit, addr); + rc = qinval_device_iotlb_sync(iommu, pdev->ats_queue_depth, + sid, sbit, addr); break; case DMA_TLB_PSI_FLUSH: if ( !device_in_domain(iommu, pdev, did) ) @@ -154,8 +154,8 @@ int dev_invalidate_iotlb(struct iommu *iommu, u16 did, addr |= (((u64)1 << (size_order - 1)) - 1) << PAGE_SHIFT_4K; } - rc = qinval_device_iotlb(iommu, pdev->ats_queue_depth, - sid, sbit, addr); + rc = qinval_device_iotlb_sync(iommu, pdev->ats_queue_depth, + sid, sbit, addr); break; default: dprintk(XENLOG_WARNING VTDPREFIX, "invalid vt-d flush type\n");