From patchwork Tue May 10 21:07:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 12845534 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4437C433EF for ; Tue, 10 May 2022 21:03:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233711AbiEJVDY (ORCPT ); Tue, 10 May 2022 17:03:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58770 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233044AbiEJVDW (ORCPT ); Tue, 10 May 2022 17:03:22 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E3E6A23724D; Tue, 10 May 2022 14:03:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652216601; x=1683752601; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BWGbvczE9hHSwD7LKlNCfOE3pQLWXV9yP/c7WBdOTFs=; b=CIZiIGiTI4/6nT76bSsZD9us8vF/LyF4wGia7EWTMsojBpFFhP2eXbNJ lW7yC9d53Vtj9AJPh6ZOHC5n/5x5EP11SCcKyKuuvocQO1DOfp9t09U2W 1QpTU0zxVvb+mISmXKYRNRmQ4SBQhFrFfaADUZ2iE8EwSw0WTdcrpcazV 5yDTYUjhqFCDhWV37j8Z9IY1TwL07rFfbfnHg/1wgz08PB5JlPxWLooZP l+wcuGSfNQA1AJzN30QJMRsM9vT5Y/caxL02m9ReRIsknigrAGcYhH1lG 4lrxS3Ex4/ulACnMHUIYDzcQf+M1VZbAbjbvt5Jz1dITdJhoZ5hX5vLth w==; X-IronPort-AV: E=McAfee;i="6400,9594,10343"; a="332538550" X-IronPort-AV: E=Sophos;i="5.91,215,1647327600"; d="scan'208";a="332538550" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 14:03:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,215,1647327600"; d="scan'208";a="553017098" Received: from otc-wp-03.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.79]) by orsmga002.jf.intel.com with ESMTP; 10 May 2022 14:03:20 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , dmaengine@vger.kernel.org, Joerg Roedel , David Woodhouse , Jean-Philippe Brucker , "Lu Baolu" , Jason Gunthorpe , vkoul@kernel.org, robin.murphy@arm.com, will@kernel.org Cc: Yi Liu , Dave Jiang , "Tian, Kevin" , Raj Ashok , Eric Auger , Jacob Pan Subject: [PATCH v3 1/4] iommu/vt-d: Implement domain ops for attach_dev_pasid Date: Tue, 10 May 2022 14:07:01 -0700 Message-Id: <20220510210704.3539577-2-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510210704.3539577-1-jacob.jun.pan@linux.intel.com> References: <20220510210704.3539577-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org On VT-d platforms with scalable mode enabled, devices issue DMA requests with PASID need to attach PASIDs to given IOMMU domains. The attach operation involves the following: - Programming the PASID into the device's PASID table - Tracking device domain and the PASID relationship - Managing IOTLB and device TLB invalidations This patch add attach_dev_pasid functions to the default domain ops which is used by DMA and identity domain types. It could be extended to support other domain types whenever necessary. Signed-off-by: Lu Baolu Signed-off-by: Jacob Pan --- drivers/iommu/intel/iommu.c | 81 ++++++++++++++++++++++++++++++++++++- include/linux/intel-iommu.h | 1 + 2 files changed, 80 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index a51b96fa9b3a..5408418f4f4b 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1562,6 +1562,10 @@ static void __iommu_flush_dev_iotlb(struct device_domain_info *info, sid = info->bus << 8 | info->devfn; qdep = info->ats_qdep; + if (info->pasid) { + qi_flush_dev_iotlb_pasid(info->iommu, sid, info->pfsid, + info->pasid, qdep, addr, mask); + } qi_flush_dev_iotlb(info->iommu, sid, info->pfsid, qdep, addr, mask); } @@ -1591,6 +1595,7 @@ static void iommu_flush_iotlb_psi(struct intel_iommu *iommu, unsigned int mask = ilog2(aligned_pages); uint64_t addr = (uint64_t)pfn << VTD_PAGE_SHIFT; u16 did = domain->iommu_did[iommu->seq_id]; + struct iommu_domain *iommu_domain = &domain->domain; BUG_ON(pages == 0); @@ -1599,6 +1604,9 @@ static void iommu_flush_iotlb_psi(struct intel_iommu *iommu, if (domain_use_first_level(domain)) { qi_flush_piotlb(iommu, did, PASID_RID2PASID, addr, pages, ih); + /* flush additional kernel DMA PASIDs attached */ + if (iommu_domain->pasid) + qi_flush_piotlb(iommu, did, iommu_domain->pasid, addr, pages, ih); } else { unsigned long bitmask = aligned_pages - 1; @@ -4265,10 +4273,13 @@ static void __dmar_remove_one_dev_info(struct device_domain_info *info) domain = info->domain; if (info->dev && !dev_is_real_dma_subdevice(info->dev)) { - if (dev_is_pci(info->dev) && sm_supported(iommu)) + if (dev_is_pci(info->dev) && sm_supported(iommu)) { intel_pasid_tear_down_entry(iommu, info->dev, PASID_RID2PASID, false); - + if (info->pasid) + intel_pasid_tear_down_entry(iommu, info->dev, + info->pasid, false); + } iommu_disable_dev_iotlb(info); domain_context_clear(info); intel_pasid_free_table(info->dev); @@ -4912,6 +4923,70 @@ static void intel_iommu_iotlb_sync_map(struct iommu_domain *domain, } } +static int intel_iommu_attach_dev_pasid(struct iommu_domain *domain, + struct device *dev, + ioasid_t pasid) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct dmar_domain *dmar_domain = to_dmar_domain(domain); + struct intel_iommu *iommu = info->iommu; + unsigned long flags; + int ret = 0; + + if (!sm_supported(iommu) || !info) + return -ENODEV; + + spin_lock_irqsave(&device_domain_lock, flags); + /* + * If the same device already has a PASID attached, just return. + * DMA layer will return the PASID value to the caller. + */ + if (pasid != PASID_RID2PASID && info->pasid) { + if (info->pasid == pasid) + ret = 0; + else { + dev_warn(dev, "Cannot attach PASID %u, %u already attached\n", + pasid, info->pasid); + ret = -EBUSY; + } + goto out_unlock_domain; + } + + spin_lock(&iommu->lock); + if (hw_pass_through && domain_type_is_si(dmar_domain)) + ret = intel_pasid_setup_pass_through(iommu, dmar_domain, + dev, pasid); + else if (domain_use_first_level(dmar_domain)) + ret = domain_setup_first_level(iommu, dmar_domain, + dev, pasid); + else + ret = intel_pasid_setup_second_level(iommu, dmar_domain, + dev, pasid); + + spin_unlock(&iommu->lock); +out_unlock_domain: + spin_unlock_irqrestore(&device_domain_lock, flags); + if (!ret) + info->pasid = pasid; + + return ret; +} + +static void intel_iommu_detach_dev_pasid(struct iommu_domain *domain, + struct device *dev, + ioasid_t pasid) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct intel_iommu *iommu = info->iommu; + unsigned long flags; + + WARN_ON(info->pasid != pasid); + spin_lock_irqsave(&iommu->lock, flags); + intel_pasid_tear_down_entry(iommu, dev, pasid, false); + info->pasid = 0; + spin_unlock_irqrestore(&iommu->lock, flags); +} + const struct iommu_ops intel_iommu_ops = { .capable = intel_iommu_capable, .domain_alloc = intel_iommu_domain_alloc, @@ -4940,6 +5015,8 @@ const struct iommu_ops intel_iommu_ops = { .iova_to_phys = intel_iommu_iova_to_phys, .free = intel_iommu_domain_free, .enforce_cache_coherency = intel_iommu_enforce_cache_coherency, + .attach_dev_pasid = intel_iommu_attach_dev_pasid, + .detach_dev_pasid = intel_iommu_detach_dev_pasid, } }; diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index 5af24befc9f1..55845a8c4f4d 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -627,6 +627,7 @@ struct device_domain_info { struct intel_iommu *iommu; /* IOMMU used by this device */ struct dmar_domain *domain; /* pointer to domain */ struct pasid_table *pasid_table; /* pasid table */ + ioasid_t pasid; /* DMA request with PASID */ }; static inline void __iommu_flush_cache( From patchwork Tue May 10 21:07:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 12845533 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4082C433FE for ; Tue, 10 May 2022 21:03:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233504AbiEJVDX (ORCPT ); Tue, 10 May 2022 17:03:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58772 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232985AbiEJVDW (ORCPT ); Tue, 10 May 2022 17:03:22 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E7568291CF7; Tue, 10 May 2022 14:03:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652216601; x=1683752601; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yaL5eTR2TDGWQSRHXqCNVN+WN1qFxw5PS9ZayzMjEpA=; b=erKWnLfrixI+VXL62Ppm5x6KFDd81bJLwf4fsiUE5wSW5vk0kqA195xU +jzmHen4Pq9XYcq52v9fixoEG9jLL1wuoBaLXX3wlpKpmWbC1YavPtS1k xMrLdRpzoKIEugcT8/kbm/tkxx61yl42fq9WA0gzIqeFCYX7GeKfsWTyH H6ZScb74fpC2N9yxig6CRjnvSlwVZQl/aOx+IzUETL5hwR8uaC+1dNa1z I6nmv1CzfHwz0u6JzSK4giykqzcDB6O48/+jXbtvR8aKWNdGV51GYpTnV o++WF7FP/U9HcCn77zpoaZ+PeVDgAFqpCzahgicBOgJkUsXxDpYBcnV4E A==; X-IronPort-AV: E=McAfee;i="6400,9594,10343"; a="332538554" X-IronPort-AV: E=Sophos;i="5.91,215,1647327600"; d="scan'208";a="332538554" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 14:03:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,215,1647327600"; d="scan'208";a="553017115" Received: from otc-wp-03.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.79]) by orsmga002.jf.intel.com with ESMTP; 10 May 2022 14:03:21 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , dmaengine@vger.kernel.org, Joerg Roedel , David Woodhouse , Jean-Philippe Brucker , "Lu Baolu" , Jason Gunthorpe , vkoul@kernel.org, robin.murphy@arm.com, will@kernel.org Cc: Yi Liu , Dave Jiang , "Tian, Kevin" , Raj Ashok , Eric Auger , Jacob Pan Subject: [PATCH v3 2/4] iommu: Add PASID support for DMA mapping API users Date: Tue, 10 May 2022 14:07:02 -0700 Message-Id: <20220510210704.3539577-3-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510210704.3539577-1-jacob.jun.pan@linux.intel.com> References: <20220510210704.3539577-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org DMA mapping API is the de facto standard for in-kernel DMA. It operates on a per device/RID basis which is not PASID-aware. Some modern devices such as Intel Data Streaming Accelerator, PASID is required for certain work submissions. To allow such devices use DMA mapping API, we need the following functionalities: 1. Provide device a way to retrieve a PASID for work submission within the kernel 2. Enable the kernel PASID on the IOMMU for the device 3. Attach the kernel PASID to the device's default DMA domain, let it be IOVA or physical address in case of pass-through. This patch introduces a driver facing API that enables DMA API PASID usage. Once enabled, device drivers can continue to use DMA APIs as is. There is no difference in dma_handle between without PASID and with PASID. Signed-off-by: Jacob Pan --- drivers/iommu/dma-iommu.c | 107 ++++++++++++++++++++++++++++++++++++++ include/linux/dma-iommu.h | 3 ++ include/linux/iommu.h | 2 + 3 files changed, 112 insertions(+) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 1ca85d37eeab..5984f3129fa2 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -34,6 +34,8 @@ struct iommu_dma_msi_page { phys_addr_t phys; }; +static DECLARE_IOASID_SET(iommu_dma_pasid); + enum iommu_dma_cookie_type { IOMMU_DMA_IOVA_COOKIE, IOMMU_DMA_MSI_COOKIE, @@ -370,6 +372,111 @@ void iommu_put_dma_cookie(struct iommu_domain *domain) domain->iova_cookie = NULL; } +/** + * iommu_attach_dma_pasid --Attach a PASID for in-kernel DMA. Use the device's + * DMA domain. + * @dev: Device to be enabled + * @pasid: The returned kernel PASID to be used for DMA + * + * DMA request with PASID will be mapped the same way as the legacy DMA. + * If the device is in pass-through, PASID will also pass-through. If the + * device is in IOVA, the PASID will point to the same IOVA page table. + * + * @return err code or 0 on success + */ +int iommu_attach_dma_pasid(struct device *dev, ioasid_t *pasid) +{ + struct iommu_domain *dom; + ioasid_t id, max; + int ret = 0; + + dom = iommu_get_domain_for_dev(dev); + if (!dom || !dom->ops || !dom->ops->attach_dev_pasid) + return -ENODEV; + + /* Only support domain types that DMA API can be used */ + if (dom->type == IOMMU_DOMAIN_UNMANAGED || + dom->type == IOMMU_DOMAIN_BLOCKED) { + dev_warn(dev, "Invalid domain type %d", dom->type); + return -EPERM; + } + + id = dom->pasid; + if (!id) { + /* + * First device to use PASID in its DMA domain, allocate + * a single PASID per DMA domain is all we need, it is also + * good for performance when it comes down to IOTLB flush. + */ + max = 1U << dev->iommu->pasid_bits; + if (!max) + return -EINVAL; + + id = ioasid_alloc(&iommu_dma_pasid, 1, max, dev); + if (id == INVALID_IOASID) + return -ENOMEM; + + dom->pasid = id; + atomic_set(&dom->pasid_users, 1); + } + + ret = dom->ops->attach_dev_pasid(dom, dev, id); + if (!ret) { + *pasid = id; + atomic_inc(&dom->pasid_users); + return 0; + } + + if (atomic_dec_and_test(&dom->pasid_users)) { + ioasid_free(id); + dom->pasid = 0; + } + + return ret; +} +EXPORT_SYMBOL(iommu_attach_dma_pasid); + +/** + * iommu_detach_dma_pasid --Disable in-kernel DMA request with PASID + * @dev: Device's PASID DMA to be disabled + * + * It is the device driver's responsibility to ensure no more incoming DMA + * requests with the kernel PASID before calling this function. IOMMU driver + * ensures PASID cache, IOTLBs related to the kernel PASID are cleared and + * drained. + * + */ +void iommu_detach_dma_pasid(struct device *dev) +{ + struct iommu_domain *dom; + ioasid_t pasid; + + dom = iommu_get_domain_for_dev(dev); + if (!dom || !dom->ops || !dom->ops->detach_dev_pasid) { + dev_warn(dev, "No ops for detaching PASID %u", pasid); + return; + } + /* Only support DMA API managed domain type */ + if (dom->type == IOMMU_DOMAIN_UNMANAGED || + dom->type == IOMMU_DOMAIN_BLOCKED) { + dev_err(dev, "Invalid domain type %d to detach DMA PASID %u\n", + dom->type, pasid); + return; + } + + pasid = dom->pasid; + if (!pasid) { + dev_err(dev, "No DMA PASID attached\n"); + return; + } + dom->ops->detach_dev_pasid(dom, dev, pasid); + if (atomic_dec_and_test(&dom->pasid_users)) { + ioasid_free(pasid); + dom->pasid = 0; + } +} +EXPORT_SYMBOL(iommu_detach_dma_pasid); + /** * iommu_dma_get_resv_regions - Reserved region driver helper * @dev: Device from iommu_get_resv_regions() diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h index 24607dc3c2ac..538650b9cb75 100644 --- a/include/linux/dma-iommu.h +++ b/include/linux/dma-iommu.h @@ -18,6 +18,9 @@ int iommu_get_dma_cookie(struct iommu_domain *domain); int iommu_get_msi_cookie(struct iommu_domain *domain, dma_addr_t base); void iommu_put_dma_cookie(struct iommu_domain *domain); +int iommu_attach_dma_pasid(struct device *dev, ioasid_t *pasid); +void iommu_detach_dma_pasid(struct device *dev); + /* Setup call for arch DMA mapping code */ void iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 dma_limit); int iommu_dma_init_fq(struct iommu_domain *domain); diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 1164524814cb..281a87fdce77 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -105,6 +105,8 @@ struct iommu_domain { enum iommu_page_response_code (*iopf_handler)(struct iommu_fault *fault, void *data); void *fault_data; + ioasid_t pasid; /* Used for DMA requests with PASID */ + atomic_t pasid_users; }; static inline bool iommu_is_dma_domain(struct iommu_domain *domain) From patchwork Tue May 10 21:07:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 12845535 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD632C433F5 for ; Tue, 10 May 2022 21:03:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233812AbiEJVDZ (ORCPT ); Tue, 10 May 2022 17:03:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58854 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233607AbiEJVDY (ORCPT ); Tue, 10 May 2022 17:03:24 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 023A1291CFA; Tue, 10 May 2022 14:03:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652216603; x=1683752603; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Bzje02YdvCwfEdXd4VLSz7VQ2pQZoJC0EJfVFm1lN80=; b=CsWvEQ6a4SfKzUArItoLxGis2qGLykhN5FkJygBO2s3aDWIsZO0dih8a DiOp1hVREHHEl3eWeThBefzJSHc3ioCIQVw8js7eR/PQBWiZ7k2zGdptX xkCe8pwsD6Ru8ysYx4QJnonCQENeEdPL56e+X/q0UQdbkOOr14YFZ+AsX BkGz/XnN1mM/LHvZ+YjQhntIdjYcEk51s5fTciTt5lVFQCNTM97+LosNU OFnhZ5rSmjzKlfkbHLDJwxalY6mk2d9r6NOR18A0UKyzJlszdKkqa+gRo pL2YPGvrZbHjRvyQpQjjDo/OUs72dMnbxPJYPXjd6HnBnVwT+9RaI+nw3 w==; X-IronPort-AV: E=McAfee;i="6400,9594,10343"; a="332538558" X-IronPort-AV: E=Sophos;i="5.91,215,1647327600"; d="scan'208";a="332538558" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 14:03:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,215,1647327600"; d="scan'208";a="553017131" Received: from otc-wp-03.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.79]) by orsmga002.jf.intel.com with ESMTP; 10 May 2022 14:03:22 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , dmaengine@vger.kernel.org, Joerg Roedel , David Woodhouse , Jean-Philippe Brucker , "Lu Baolu" , Jason Gunthorpe , vkoul@kernel.org, robin.murphy@arm.com, will@kernel.org Cc: Yi Liu , Dave Jiang , "Tian, Kevin" , Raj Ashok , Eric Auger , Jacob Pan Subject: [PATCH v3 3/4] dmaengine: idxd: Use DMA API for in-kernel DMA with PASID Date: Tue, 10 May 2022 14:07:03 -0700 Message-Id: <20220510210704.3539577-4-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510210704.3539577-1-jacob.jun.pan@linux.intel.com> References: <20220510210704.3539577-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org The current in-kernel supervisor PASID support is based on the SVM/SVA machinery in SVA lib. The binding between a kernel PASID and kernel mapping has many flaws. See discussions in the link below. This patch enables in-kernel DMA by switching from SVA lib to the standard DMA mapping APIs. Since both DMA requests with and without PASIDs are mapped identically, there is no change to how DMA APIs are used after the kernel PASID is enabled. Link: https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.com/ Signed-off-by: Jacob Pan --- drivers/dma/idxd/idxd.h | 1 - drivers/dma/idxd/init.c | 34 +++++++++------------------------- drivers/dma/idxd/sysfs.c | 7 ------- 3 files changed, 9 insertions(+), 33 deletions(-) diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index ccbefd0be617..190b08bd7c08 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -277,7 +277,6 @@ struct idxd_device { struct idxd_wq **wqs; struct idxd_engine **engines; - struct iommu_sva *sva; unsigned int pasid; int num_groups; diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index e1b5d1e4a949..e2e1c0eae6d6 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include "../dmaengine.h" @@ -466,36 +467,22 @@ static struct idxd_device *idxd_alloc(struct pci_dev *pdev, struct idxd_driver_d static int idxd_enable_system_pasid(struct idxd_device *idxd) { - int flags; - unsigned int pasid; - struct iommu_sva *sva; + u32 pasid; + int ret; - flags = SVM_FLAG_SUPERVISOR_MODE; - - sva = iommu_sva_bind_device(&idxd->pdev->dev, NULL, &flags); - if (IS_ERR(sva)) { - dev_warn(&idxd->pdev->dev, - "iommu sva bind failed: %ld\n", PTR_ERR(sva)); - return PTR_ERR(sva); - } - - pasid = iommu_sva_get_pasid(sva); - if (pasid == IOMMU_PASID_INVALID) { - iommu_sva_unbind_device(sva); - return -ENODEV; + ret = iommu_attach_dma_pasid(&idxd->pdev->dev, &pasid); + if (ret) { + dev_err(&idxd->pdev->dev, "No DMA PASID %d\n", ret); + return ret; } - - idxd->sva = sva; idxd->pasid = pasid; - dev_dbg(&idxd->pdev->dev, "system pasid: %u\n", pasid); + return 0; } static void idxd_disable_system_pasid(struct idxd_device *idxd) { - - iommu_sva_unbind_device(idxd->sva); - idxd->sva = NULL; + iommu_detach_dma_pasid(&idxd->pdev->dev); } static int idxd_probe(struct idxd_device *idxd) @@ -527,10 +514,7 @@ static int idxd_probe(struct idxd_device *idxd) else set_bit(IDXD_FLAG_PASID_ENABLED, &idxd->flags); } - } else if (!sva) { - dev_warn(dev, "User forced SVA off via module param.\n"); } - idxd_read_caps(idxd); idxd_read_table_offsets(idxd); diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c index dfd549685c46..a48928973bd4 100644 --- a/drivers/dma/idxd/sysfs.c +++ b/drivers/dma/idxd/sysfs.c @@ -839,13 +839,6 @@ static ssize_t wq_name_store(struct device *dev, if (strlen(buf) > WQ_NAME_SIZE || strlen(buf) == 0) return -EINVAL; - /* - * This is temporarily placed here until we have SVM support for - * dmaengine. - */ - if (wq->type == IDXD_WQT_KERNEL && device_pasid_enabled(wq->idxd)) - return -EOPNOTSUPP; - memset(wq->name, 0, WQ_NAME_SIZE + 1); strncpy(wq->name, buf, WQ_NAME_SIZE); strreplace(wq->name, '\n', '\0'); From patchwork Tue May 10 21:07:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 12845536 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 458D5C4167B for ; Tue, 10 May 2022 21:03:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233872AbiEJVD0 (ORCPT ); Tue, 10 May 2022 17:03:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58916 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233765AbiEJVDY (ORCPT ); Tue, 10 May 2022 17:03:24 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 124FB23724D; Tue, 10 May 2022 14:03:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652216604; x=1683752604; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=MOu4j9oTaoLByHMcGaRvSM1IqcXLeJucVfuaFmAdiQ0=; b=Y3d0qofRyM0eNUyf1DYG7QB27PRtxPoAdpuDEhEd7UXOTM8DTjr1k0Qk +9aH+c0rZTzhSWj2WU4IR679R1hdZ31sDwFYYcABG3lq76DlZl5xXb311 MvxxdllVvL8eEh217OGD8H4SbNk/7e6MRAYLd7HY29I/nA+KDgQR3RLqo wMcGcmdSoTcq8BrNJkPQZ7KO9cQ3Z6iS79VJ9N09f9ATT3Fpt0h8uswOf YiEH8XilDcZBqglAastTlhk0IjtqN6U4W9Jn4MxUzxRfOwl3IJdK0gnx4 E7wGGRtXYn4KJMJDwLToa2UGR5ws8x5tEt0qwHwfP7NHtNVCFfAa+1b32 A==; X-IronPort-AV: E=McAfee;i="6400,9594,10343"; a="332538562" X-IronPort-AV: E=Sophos;i="5.91,215,1647327600"; d="scan'208";a="332538562" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 14:03:23 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,215,1647327600"; d="scan'208";a="553017155" Received: from otc-wp-03.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.79]) by orsmga002.jf.intel.com with ESMTP; 10 May 2022 14:03:23 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , dmaengine@vger.kernel.org, Joerg Roedel , David Woodhouse , Jean-Philippe Brucker , "Lu Baolu" , Jason Gunthorpe , vkoul@kernel.org, robin.murphy@arm.com, will@kernel.org Cc: Yi Liu , Dave Jiang , "Tian, Kevin" , Raj Ashok , Eric Auger , Jacob Pan Subject: [PATCH v3 4/4] iommu/vt-d: Delete unused SVM flag Date: Tue, 10 May 2022 14:07:04 -0700 Message-Id: <20220510210704.3539577-5-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510210704.3539577-1-jacob.jun.pan@linux.intel.com> References: <20220510210704.3539577-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org Supervisor PASID for SVA/SVM is no longer supported, delete the unused flag. Signed-off-by: Jacob Pan --- drivers/iommu/intel/svm.c | 2 +- include/linux/intel-svm.h | 13 ------------- 2 files changed, 1 insertion(+), 14 deletions(-) diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 38c33cde177e..98ec77415770 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -750,7 +750,7 @@ static irqreturn_t prq_event_thread(int irq, void *d) * to unbind the mm while any page faults are outstanding. */ svm = pasid_private_find(req->pasid); - if (IS_ERR_OR_NULL(svm) || (svm->flags & SVM_FLAG_SUPERVISOR_MODE)) + if (IS_ERR_OR_NULL(svm)) goto bad_req; } diff --git a/include/linux/intel-svm.h b/include/linux/intel-svm.h index b3b125b332aa..6835a665c195 100644 --- a/include/linux/intel-svm.h +++ b/include/linux/intel-svm.h @@ -13,17 +13,4 @@ #define PRQ_RING_MASK ((0x1000 << PRQ_ORDER) - 0x20) #define PRQ_DEPTH ((0x1000 << PRQ_ORDER) >> 5) -/* - * The SVM_FLAG_SUPERVISOR_MODE flag requests a PASID which can be used only - * for access to kernel addresses. No IOTLB flushes are automatically done - * for kernel mappings; it is valid only for access to the kernel's static - * 1:1 mapping of physical memory — not to vmalloc or even module mappings. - * A future API addition may permit the use of such ranges, by means of an - * explicit IOTLB flush call (akin to the DMA API's unmap method). - * - * It is unlikely that we will ever hook into flush_tlb_kernel_range() to - * do such IOTLB flushes automatically. - */ -#define SVM_FLAG_SUPERVISOR_MODE BIT(0) - #endif /* __INTEL_SVM_H__ */