From patchwork Fri Apr 1 14:47:37 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quan Xu X-Patchwork-Id: 8725381 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id D6912C0553 for ; Fri, 1 Apr 2016 14:53:20 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id EC493203AE for ; Fri, 1 Apr 2016 14:53:19 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ECC9F20390 for ; Fri, 1 Apr 2016 14:53:18 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1am0PW-0004PD-J0; Fri, 01 Apr 2016 14:50:54 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1am0PV-0004Nz-7r for xen-devel@lists.xen.org; Fri, 01 Apr 2016 14:50:53 +0000 Received: from [85.158.139.211] by server-5.bemta-5.messagelabs.com id 2F/43-03597-CCA8EF65; Fri, 01 Apr 2016 14:50:52 +0000 X-Env-Sender: quan.xu@intel.com X-Msg-Ref: server-5.tower-206.messagelabs.com!1459522251!32492534!1 X-Originating-IP: [134.134.136.20] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTM0LjEzNC4xMzYuMjAgPT4gMzU1MzU4\n X-StarScan-Received: X-StarScan-Version: 8.11; banners=-,-,- X-VirusChecked: Checked Received: (qmail 55553 invoked from network); 1 Apr 2016 14:50:51 -0000 Received: from mga02.intel.com (HELO mga02.intel.com) (134.134.136.20) by server-5.tower-206.messagelabs.com with SMTP; 1 Apr 2016 14:50:51 -0000 Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP; 01 Apr 2016 07:50:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.24,427,1455004800"; d="scan'208";a="936308475" Received: from xen-commits.sh.intel.com ([10.239.82.178]) by fmsmga001.fm.intel.com with ESMTP; 01 Apr 2016 07:50:28 -0700 From: Quan Xu To: xen-devel@lists.xen.org Date: Fri, 1 Apr 2016 22:47:37 +0800 Message-Id: <1459522059-102365-2-git-send-email-quan.xu@intel.com> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1459522059-102365-1-git-send-email-quan.xu@intel.com> References: <1459522059-102365-1-git-send-email-quan.xu@intel.com> Cc: kevin.tian@intel.com, feng.wu@intel.com, jbeulich@suse.com, dario.faggioli@citrix.com, Quan Xu Subject: [Xen-devel] [PATCH v9 1/3] VT-d: add a command line parameter for Queued Invalidation X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The command line parameter 'vtd_qi_timeout' specifies the timeout of the VT-d Queued Invalidation in milliseconds. By default, the timeout is 1ms, which can be boot-time changed. Add a __must_check annotation. The followup patch titled 'VT-d IOTLB/Context/IEC flush issue' addresses the __mustcheck. That is the other callers of this routine (two or three levels up) ignore the return code. This patch does not address this but the other does. Signed-off-by: Quan Xu --- docs/misc/xen-command-line.markdown | 18 ++++++++++++++++++ xen/drivers/passthrough/vtd/qinval.c | 17 ++++++++++++----- 2 files changed, 30 insertions(+), 5 deletions(-) diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown index ca77e3b..5a7ed5d 100644 --- a/docs/misc/xen-command-line.markdown +++ b/docs/misc/xen-command-line.markdown @@ -1532,6 +1532,24 @@ Note that if **watchdog** option is also specified vpmu will be turned off. As the virtualisation is not 100% safe, don't use the vpmu flag on production systems (see http://xenbits.xen.org/xsa/advisory-163.html)! +### vtd\_qi\_timeout (VT-d) +> `= ` + +> Default: `1` + +Specify the timeout of the VT-d Queued Invalidation in milliseconds. +By default, the spin timeout is 1ms, which can be boot-time changed. + +In current code, VT-d Queued Invalidation includes Device-TLB, IOTLB, +Context and IEC flush. If Device-TLB flush timed out, we would hide +the target ATS device and crash the domain owning this ATS device. +If impacted domain is hardware domain, just throw out a warning (done +in queue\_invalidate\_wait). IOTLB, Context and IEC flush timeout are +still in TODO-list. + +When you see error 'Queue invalidate wait descriptor timed out', try +increasing the vtd\_qi\_timeout to 10ms or more. + ### watchdog > `= force | ` diff --git a/xen/drivers/passthrough/vtd/qinval.c b/xen/drivers/passthrough/vtd/qinval.c index b81b0bd..52ba2c2 100644 --- a/xen/drivers/passthrough/vtd/qinval.c +++ b/xen/drivers/passthrough/vtd/qinval.c @@ -28,6 +28,11 @@ #include "vtd.h" #include "extern.h" +static unsigned int __read_mostly vtd_qi_timeout = 1; +integer_param("vtd_qi_timeout", vtd_qi_timeout); + +#define IOMMU_QI_TIMEOUT (vtd_qi_timeout * MILLISECS(1)) + static void print_qi_regs(struct iommu *iommu) { u64 val; @@ -130,10 +135,10 @@ static void queue_invalidate_iotlb(struct iommu *iommu, spin_unlock_irqrestore(&iommu->register_lock, flags); } -static int queue_invalidate_wait(struct iommu *iommu, +static int __must_check queue_invalidate_wait(struct iommu *iommu, u8 iflag, u8 sw, u8 fn) { - s_time_t start_time; + s_time_t timeout; volatile u32 poll_slot = QINVAL_STAT_INIT; unsigned int index; unsigned long flags; @@ -164,13 +169,15 @@ static int queue_invalidate_wait(struct iommu *iommu, if ( sw ) { /* In case all wait descriptor writes to same addr with same data */ - start_time = NOW(); + timeout = NOW() + IOMMU_QI_TIMEOUT; while ( poll_slot != QINVAL_STAT_DONE ) { - if ( NOW() > (start_time + DMAR_OPERATION_TIMEOUT) ) + if ( NOW() > timeout ) { print_qi_regs(iommu); - panic("queue invalidate wait descriptor was not executed"); + printk(XENLOG_WARNING VTDPREFIX + "Queue invalidate wait descriptor timed out.\n"); + return -ETIMEDOUT; } cpu_relax(); }