From patchwork Mon Feb 27 19:54:35 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 9594001 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C039A60574 for ; Mon, 27 Feb 2017 20:19:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B2AAD27F85 for ; Mon, 27 Feb 2017 20:19:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A67CC281C3; Mon, 27 Feb 2017 20:19:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 21E012818B for ; Mon, 27 Feb 2017 20:19:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Ui0DFkLSFCfVMpjK4CBDQeLw/g4tkdccWLzx70yxjFU=; b=DNX9oDPhzVA6IpD4mw530U8Fxp ykNZL76WfuJuS/MqtlmcUm9ZRXchM8rj53XwNuIu1J/FcSlPDt3j1IihTFoUUO+qaXrSe5qaKSRyf z45lNxv4wfVeBeR95YbiQGl1VEGaFUS521DU2pqO8STJ4T1VIUo5lqeWaWJm9TL2aRWDu78b4PN4L 4XGRkjqKlRgJi3Pt3VjvDQoBubqmm0YaReBS3xN4ySklFvIaVI3XGfxftPqk9j5DDzkU+uJiNazme Z9UlsU7yvnqr2bZWJXx7peWBN7IVlSrZhd75rSfW3KGRIK2xVpxn3iJhJqnM7wsSeCd/GkmN2QAc8 CsHeTaKQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1ciRlc-0007o2-On; Mon, 27 Feb 2017 20:19:32 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1ciRSH-0008Oe-6a for linux-arm-kernel@lists.infradead.org; Mon, 27 Feb 2017 19:59:39 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 86728169E; Mon, 27 Feb 2017 11:59:13 -0800 (PST) Received: from e106794-lin.cambridge.arm.com (e106794-lin.cambridge.arm.com [10.1.210.60]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id DB3D53F3E1; Mon, 27 Feb 2017 11:59:10 -0800 (PST) From: Jean-Philippe Brucker To: Subject: [RFC PATCH 24/30] iommu: Specify PASID state when unbinding a task Date: Mon, 27 Feb 2017 19:54:35 +0000 Message-Id: <20170227195441.5170-25-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170227195441.5170-1-jean-philippe.brucker@arm.com> References: <20170227195441.5170-1-jean-philippe.brucker@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170227_115933_403370_A9EA58B4 X-CRM114-Status: GOOD ( 14.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lorenzo Pieralisi , Shanker Donthineni , kvm@vger.kernel.org, Catalin Marinas , Joerg Roedel , Sinan Kaya , Will Deacon , iommu@lists.linux-foundation.org, Harv Abdulhamid , Alex Williamson , linux-pci@vger.kernel.org, Bjorn Helgaas , Robin Murphy , David Woodhouse , linux-arm-kernel@lists.infradead.org, Nate Watterson MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Provide a way for device drivers to tell the IOMMU driver about the state of the PASID they are trying to decommission. When unbinding a task from a device, the IOMMU driver needs to know whether it can immediately reuse the PASID for another task, or if there is additional work to be done before the PASID is safe to re-use. One hard requirement when calling unbind is that the associated PASID is not present in any transaction downstream of the IOMMU anymore. In other words, any read, write, page requests referring to this PASID has finished. For PCIe, this means that the driver has successfully executed the device-specific stop request mechanism described in 6.20.1 (Managing PASID TLP Prefix Usage). In particular: * device doesn't issue any new request for this PASID, * all non-posted requests for this PASID have been completed, * all posted requests for this PASID (addressing host memory) have been flushed to the host. Address Translation Requests are non-posted, and PRI Page Requests (PPR) are posted. In addition with PRI, device must implement one of the following mechanism (ATS spec 4.1.2. - Managing PASID TLP Prefix Usage): A. Finish transmitting any PPR affecting this PASID and wait for their response. In this case, the IOMMU driver can safely reuse the PASID and must not wait for a Stop Marker. B. Finish transmitting any PPR affecting this PASID and send a Stop Marker. The driver must wait to receive a Stop Marker for this PASID before reusing it. This patch lets the driver communicate the current state of the PASID with either IOMMU_PASID_FLUSHED for case A, or IOMMU_PASID_CLEAN for case B. It is an important distinction because, if the IOMMU driver reassigns a PASID while the IOMMU still holds pending PPR targeting that PASID internally, the PPR will trigger a fault in the wrong address space. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/iommu.c | 8 ++++++++ include/linux/iommu.h | 18 +++++++++++++++++- 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 26c5f6528c69..eed52500d469 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -1517,6 +1517,14 @@ EXPORT_SYMBOL_GPL(iommu_bind_task); * @dev: device bound to the task * @pasid: identifier of the bond * @flags: state of the PASID and driver-specific flags + * + * The caller must informs the IOMMU driver whether the PASID is safe to reuse + * immediately or if it needs more invalidation steps, by setting flags to + * either IOMMU_PASID_FLUSHED, or IOMMU_PASID_CLEAN. + * + * Without one of these flags, the device driver must have provided an + * invalidate_pasid callback in iommu_svm_ops. Otherwise, iommu_unbind_task + * returns an error. */ int iommu_unbind_task(struct device *dev, int pasid, int flags) { diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 9554f45d4305..204943ef38b2 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -50,6 +50,21 @@ struct notifier_block; #define IOMMU_FAULT_READ 0x0 #define IOMMU_FAULT_WRITE 0x1 +/* + * State of a PASID in the system + * + * IOMMU_PASID_FLUSHED: the device does not generate any traffic for this PASID + * anymore, and all references to the PASID have been flushed; in other words, + * the IOMMU will not receive any transaction referring to this instance of + * the PASID anymore. + * + * IOMMU_PASID_CLEAN: in addition to IOMMU_PASID_FLUSHED, the PASID isn't + * present in the IOMMU either. For instance when using PRI, the device waited + * for all of its page requests to come back with a response. + */ +#define IOMMU_PASID_FLUSHED 0x1 +#define IOMMU_PASID_CLEAN 0x2 + typedef int (*iommu_fault_handler_t)(struct iommu_domain *, struct device *, unsigned long, int, void *); @@ -147,7 +162,8 @@ struct iommu_resv_region { /* * @handle_fault: report or handle a fault from the device (FIXME: imprecise) - * @invalidate_pasid: stop using a PASID. + * @invalidate_pasid: stop using a PASID. Returns one of IOMMU_PASID_FLUSHED or + * IOMMU_PASID_CLEAN when stopped successfully. 0 otherwise. */ struct iommu_svm_ops { int (*handle_fault)(struct device *dev, int pasid, u64 address,