From patchwork Thu Mar 2 00:59:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13156609 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 778A1C678D4 for ; Thu, 2 Mar 2023 00:56:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229689AbjCBA4U (ORCPT ); Wed, 1 Mar 2023 19:56:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38722 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229676AbjCBA4T (ORCPT ); Wed, 1 Mar 2023 19:56:19 -0500 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E853F5678D; Wed, 1 Mar 2023 16:56:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677718578; x=1709254578; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=iAEeG4ALX1eVVTpD7m5C0QNfU+j9krX/lMoruCC5oxY=; b=Q9GxJoeeOMGps0DiQCbTkOhZyHOcwRcfiiit/Eztqgxda9FqYWd3Z0sh Mw4hxSt9vafYYAgmZOqllFHdOB87VNZRMxNWBDt6cy5r2b3DRzBlJkHf5 NSK1ruEz8WOqMKsdZz1BE9cxoA/HeEfLfm8SaATncpXvd6zajIPj8iUpX rHsAx5ZBZk+Cg1qbUxBAPT0oZL2NAbqsdVvqVf4nDxcQVYpHhC3EJuJK6 0jWIJKiuSNKwuHt4lv6zkIdCMKMAXPQ1rshVVNY6ZzzEK9hbYyB0nyO6F +oz9EL6flttG5Z8X2y49xr10pvUte8VnuaAU2EvfAUVBg5/vy4+nb0kiF w==; X-IronPort-AV: E=McAfee;i="6500,9779,10636"; a="397141475" X-IronPort-AV: E=Sophos;i="5.98,226,1673942400"; d="scan'208";a="397141475" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Mar 2023 16:56:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10636"; a="624708990" X-IronPort-AV: E=Sophos;i="5.98,226,1673942400"; d="scan'208";a="624708990" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.106]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2023 16:56:11 -0800 From: Jacob Pan To: LKML , iommu@lists.linux.dev, Jason Gunthorpe , "Lu Baolu" , Joerg Roedel , dmaengine@vger.kernel.org, vkoul@kernel.org Cc: "Robin Murphy" , "Will Deacon" , David Woodhouse , Raj Ashok , "Tian, Kevin" , Yi Liu , "Yu, Fenghua" , Dave Jiang , Tony Luck , "Zanussi, Tom" , Jacob Pan Subject: [PATCH 1/4] iommu/vt-d: Implement set device pasid op for default domain Date: Wed, 1 Mar 2023 16:59:56 -0800 Message-Id: <20230302005959.2695267-2-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230302005959.2695267-1-jacob.jun.pan@linux.intel.com> References: <20230302005959.2695267-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org On VT-d platforms, legacy DMA requests without PASID use device’s default domain, where RID_PASID is always attached. Device drivers can then use the DMA API for all in-kernel DMA on the RID. Ideally, devices capable of using ENQCMDS can also transparently use the default domain, consequently DMA API. However, VT-d architecture dictates that the PASID used by ENQCMDS must be different from the RID_PASID value. To provide support for transparent use of DMA API with non-RID_PASID value, this patch implements the set_dev_pasid() function for the default domain. The idea is that device drivers wishing to use ENQCMDS to submit work on buffers mapped by DMA API will call iommu_attach_device_pasid() beforehand. Signed-off-by: Jacob Pan --- drivers/iommu/intel/iommu.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 10f657828d3a..a0cb3bc851ac 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4665,6 +4665,10 @@ static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) case IOMMU_DOMAIN_SVA: intel_svm_remove_dev_pasid(dev, pasid); break; + case IOMMU_DOMAIN_DMA: + case IOMMU_DOMAIN_DMA_FQ: + case IOMMU_DOMAIN_IDENTITY: + break; default: /* should never reach here */ WARN_ON(1); @@ -4675,6 +4679,33 @@ static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) intel_pasid_tear_down_entry(iommu, dev, pasid, false); } +static int intel_iommu_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct dmar_domain *dmar_domain = to_dmar_domain(domain); + struct intel_iommu *iommu = info->iommu; + int ret = 0; + + if (!sm_supported(iommu) || !info) + return -ENODEV; + + if (WARN_ON(pasid == PASID_RID2PASID)) + return -EINVAL; + + if (hw_pass_through && domain_type_is_si(dmar_domain)) + ret = intel_pasid_setup_pass_through(iommu, dmar_domain, + dev, pasid); + else if (dmar_domain->use_first_level) + ret = domain_setup_first_level(iommu, dmar_domain, + dev, pasid); + else + ret = intel_pasid_setup_second_level(iommu, dmar_domain, + dev, pasid); + + return ret; +} + const struct iommu_ops intel_iommu_ops = { .capable = intel_iommu_capable, .domain_alloc = intel_iommu_domain_alloc, @@ -4702,6 +4733,7 @@ const struct iommu_ops intel_iommu_ops = { .iova_to_phys = intel_iommu_iova_to_phys, .free = intel_iommu_domain_free, .enforce_cache_coherency = intel_iommu_enforce_cache_coherency, + .set_dev_pasid = intel_iommu_set_dev_pasid, } }; From patchwork Thu Mar 2 00:59:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13156610 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71A0CC7EE33 for ; Thu, 2 Mar 2023 00:56:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229702AbjCBA4V (ORCPT ); Wed, 1 Mar 2023 19:56:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38724 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229674AbjCBA4T (ORCPT ); Wed, 1 Mar 2023 19:56:19 -0500 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 23CEF56794; Wed, 1 Mar 2023 16:56:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677718578; x=1709254578; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PevryAzFzxdZ/XWUGr+C5kejkQ3Lxn/8y5vx2rg2OzA=; b=XdSCXEYRPJCLKmV212mr/d/iX8rjphKFg12Bfkw2n0+v9c4r5ouc6xmE zyAytE72FX7+J8tmohhYNs7snjNtEBGePvrqSP+jEyfMtZH7YWbqWVJJs 1vwh4Txiv8IlssM6OlzktmuGO9SFnyh/xHn7Efww3/eVKbIIVQxyXzzpu e7ecmVc9VfekkxSfG0KIyPt7qypS5O4q6vyZV9E1JORLIZctXj7tZUWdW QGihXTHzuq1KLpiNZcZ9Uq44G6j3jseW4IexP0Ch1oVzYsfxmEFD6TLP6 zeV+tL3zvCFYQFSxcDQ0Sw990qXEkrNKxIbhOTSFU/hL9ejkJShANXnmV A==; X-IronPort-AV: E=McAfee;i="6500,9779,10636"; a="397141482" X-IronPort-AV: E=Sophos;i="5.98,226,1673942400"; d="scan'208";a="397141482" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Mar 2023 16:56:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10636"; a="624708994" X-IronPort-AV: E=Sophos;i="5.98,226,1673942400"; d="scan'208";a="624708994" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.106]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2023 16:56:12 -0800 From: Jacob Pan To: LKML , iommu@lists.linux.dev, Jason Gunthorpe , "Lu Baolu" , Joerg Roedel , dmaengine@vger.kernel.org, vkoul@kernel.org Cc: "Robin Murphy" , "Will Deacon" , David Woodhouse , Raj Ashok , "Tian, Kevin" , Yi Liu , "Yu, Fenghua" , Dave Jiang , Tony Luck , "Zanussi, Tom" , Jacob Pan Subject: [PATCH 2/4] iommu/vt-d: Use non-privileged mode for all PASIDs Date: Wed, 1 Mar 2023 16:59:57 -0800 Message-Id: <20230302005959.2695267-3-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230302005959.2695267-1-jacob.jun.pan@linux.intel.com> References: <20230302005959.2695267-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org For in-kernel DMA, use non-privileged access for all PASIDs to be consistent with RID_PASID. There's no need to differentiate user and kernel for in-kernel DMA. Signed-off-by: Jacob Pan --- drivers/iommu/intel/iommu.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index a0cb3bc851ac..9e3c056e392d 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -2334,8 +2334,6 @@ static int domain_setup_first_level(struct intel_iommu *iommu, if (level != 4 && level != 5) return -EINVAL; - if (pasid != PASID_RID2PASID) - flags |= PASID_FLAG_SUPERVISOR_MODE; if (level == 5) flags |= PASID_FLAG_FL5LP; From patchwork Thu Mar 2 00:59:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13156611 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E83EC7EE37 for ; Thu, 2 Mar 2023 00:56:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229705AbjCBA4V (ORCPT ); Wed, 1 Mar 2023 19:56:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38762 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229445AbjCBA4U (ORCPT ); Wed, 1 Mar 2023 19:56:20 -0500 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DCC6756795; Wed, 1 Mar 2023 16:56:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677718579; x=1709254579; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=D0qyjUJXtQ22gjOJkGfAodz2FNGnPAkWpKsE9lPkA68=; b=AQVN+4DGHAo9OTn5Ten3wzSBwdcRt77FgA7e7uqs87TFQkgkoZoCe5dS b3+jE07AEVFK64mVpedOb1pwb6sMjH6IYO21xiP6wqIMmw10AqfDR7qfj BCaGyUvrbzF4Gl6vElDxV9DlG6DgKLGqPgCWgKRjS5NefsNNH0HsJA/4Y Fv42F0h/TX9YPDRORZmiOCgOQgyc4O9qSYlZkvJIp21idpvqDMZizvOhq InjNidBsxgB4lHo3+XMwZA1kbYzsQXB+PCta4sxIPdbjyXCGqty6PVkAs pHqJhPegl6rpwDMzmw6V6F5dlh9Ntuep/5uX3cU59FYCBYGOENfW6n4WY g==; X-IronPort-AV: E=McAfee;i="6500,9779,10636"; a="397141489" X-IronPort-AV: E=Sophos;i="5.98,226,1673942400"; d="scan'208";a="397141489" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Mar 2023 16:56:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10636"; a="624708997" X-IronPort-AV: E=Sophos;i="5.98,226,1673942400"; d="scan'208";a="624708997" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.106]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2023 16:56:12 -0800 From: Jacob Pan To: LKML , iommu@lists.linux.dev, Jason Gunthorpe , "Lu Baolu" , Joerg Roedel , dmaengine@vger.kernel.org, vkoul@kernel.org Cc: "Robin Murphy" , "Will Deacon" , David Woodhouse , Raj Ashok , "Tian, Kevin" , Yi Liu , "Yu, Fenghua" , Dave Jiang , Tony Luck , "Zanussi, Tom" , Jacob Pan Subject: [PATCH 3/4] iommu/sva: Support reservation of global PASIDs Date: Wed, 1 Mar 2023 16:59:58 -0800 Message-Id: <20230302005959.2695267-4-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230302005959.2695267-1-jacob.jun.pan@linux.intel.com> References: <20230302005959.2695267-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org Global PASID allocation is under IOMMU SVA code since it is the primary use case. However, some architecture such as VT-d, global PASIDs are necessary for its internal use of DMA API with PASID. This patch introduces SVA APIs to reserve and release global PASIDs. Link: https://lore.kernel.org/all/20230301235646.2692846-4-jacob.jun.pan@linux.intel.com/ Signed-off-by: Jacob Pan --- drivers/iommu/iommu-sva.c | 25 +++++++++++++++++++++++++ include/linux/iommu.h | 14 ++++++++++++++ 2 files changed, 39 insertions(+) diff --git a/drivers/iommu/iommu-sva.c b/drivers/iommu/iommu-sva.c index 8c92a145e15d..cfdeafde88a9 100644 --- a/drivers/iommu/iommu-sva.c +++ b/drivers/iommu/iommu-sva.c @@ -149,6 +149,31 @@ u32 iommu_sva_get_pasid(struct iommu_sva *handle) } EXPORT_SYMBOL_GPL(iommu_sva_get_pasid); +ioasid_t iommu_sva_reserve_pasid(ioasid_t min, ioasid_t max) +{ + int ret; + + if (min == IOMMU_PASID_INVALID || max == IOMMU_PASID_INVALID || + min == 0 || max < min) + return IOMMU_PASID_INVALID; + + ret = ida_alloc_range(&iommu_global_pasid_ida, min, max, GFP_KERNEL); + if (ret < 0) + return IOMMU_PASID_INVALID; + + return ret; +} +EXPORT_SYMBOL_GPL(iommu_sva_reserve_pasid); + +void iommu_sva_unreserve_pasid(ioasid_t pasid) +{ + if (!pasid_valid(pasid)) + return; + + ida_free(&iommu_global_pasid_ida, pasid); +} +EXPORT_SYMBOL_GPL(iommu_sva_unreserve_pasid); + /* * I/O page fault handler for SVA */ diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 39a97bd8f04a..8ba07eb03d32 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -1192,6 +1192,9 @@ struct iommu_sva *iommu_sva_bind_device(struct device *dev, struct mm_struct *mm); void iommu_sva_unbind_device(struct iommu_sva *handle); u32 iommu_sva_get_pasid(struct iommu_sva *handle); +ioasid_t iommu_sva_reserve_pasid(ioasid_t min, ioasid_t max); +void iommu_sva_unreserve_pasid(ioasid_t pasid); + #else static inline struct iommu_sva * iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) @@ -1207,6 +1210,17 @@ static inline u32 iommu_sva_get_pasid(struct iommu_sva *handle) { return IOMMU_PASID_INVALID; } + +static inline ioasid_t iommu_sva_reserve_pasid(ioasid_t min, ioasid_t max); +{ + return IOMMU_PASID_INVALID; +} + +static inline void iommu_sva_unreserve_pasid(ioasid_t pasid) +{ + +} + #endif /* CONFIG_IOMMU_SVA */ #endif /* __LINUX_IOMMU_H */ From patchwork Thu Mar 2 00:59:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Pan X-Patchwork-Id: 13156612 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0A82C678D4 for ; Thu, 2 Mar 2023 00:56:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229620AbjCBA4W (ORCPT ); Wed, 1 Mar 2023 19:56:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38772 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229694AbjCBA4V (ORCPT ); Wed, 1 Mar 2023 19:56:21 -0500 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE7665679A; Wed, 1 Mar 2023 16:56:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677718579; x=1709254579; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+YAVTI4OAG6BzTQR9RW8CVbN3exjRQYGr55LpRVX4ZM=; b=UGW941imMAqiVKtAyLNsIMA5RABdtsbgFXio859YGYg3YnOk3ehPrMQ2 lvMjlkOEO41mxyYLNly00zi1X8YGgmTOOkaczV4YebZD5bGwqC3MIBPyG 2CA55pofcY9BT5Ga4Z0/Wh65aW98W/g5KRfvkOoCfU1YIbdYpUlflUu03 d0yUfM6mszDJS7Gst8Ek8I7VQEYEC4FneTVZzqZOHnO5KRAVOqVRHeFg+ /ygQCrT4xwBmM/+6oqebtgJRc4yDRPdh29L7Mtmg5A52Azw1moJ4T0xz8 EtJCJ3qCojlboRkAVwlebJLRUSvfI2Zqq2Zl6YfEXpg9nnlCTBeo+kYKw A==; X-IronPort-AV: E=McAfee;i="6500,9779,10636"; a="397141503" X-IronPort-AV: E=Sophos;i="5.98,226,1673942400"; d="scan'208";a="397141503" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Mar 2023 16:56:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10636"; a="624709000" X-IronPort-AV: E=Sophos;i="5.98,226,1673942400"; d="scan'208";a="624709000" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.106]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2023 16:56:12 -0800 From: Jacob Pan To: LKML , iommu@lists.linux.dev, Jason Gunthorpe , "Lu Baolu" , Joerg Roedel , dmaengine@vger.kernel.org, vkoul@kernel.org Cc: "Robin Murphy" , "Will Deacon" , David Woodhouse , Raj Ashok , "Tian, Kevin" , Yi Liu , "Yu, Fenghua" , Dave Jiang , Tony Luck , "Zanussi, Tom" , Jacob Pan Subject: [PATCH 4/4] dmaengine/idxd: Re-enable kernel workqueue under DMA API Date: Wed, 1 Mar 2023 16:59:59 -0800 Message-Id: <20230302005959.2695267-5-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230302005959.2695267-1-jacob.jun.pan@linux.intel.com> References: <20230302005959.2695267-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org Kernel workqueues were disabled due to flawed use of kernel VA and SVA API. Now That we have the support for attaching PASID to the device's default domain and the ability to reserve global PASIDs from SVA APIs, we can re-enable the kernel work queues and use them under DMA API. We also use non-privileged access for in-kernel DMA to be consistent with the IOMMU settings. Consequently, interrupt for user privilege is enabled for work completion IRQs. Link:https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.com/ Signed-off-by: Jacob Pan Reviewed-by: Dave Jiang --- drivers/dma/idxd/device.c | 30 +++++-------------------- drivers/dma/idxd/init.c | 47 +++++++++++++++++++++++++++++++++++---- drivers/dma/idxd/sysfs.c | 7 ------ 3 files changed, 48 insertions(+), 36 deletions(-) diff --git a/drivers/dma/idxd/device.c b/drivers/dma/idxd/device.c index 125652a8bb29..96faf4d3445e 100644 --- a/drivers/dma/idxd/device.c +++ b/drivers/dma/idxd/device.c @@ -299,21 +299,6 @@ void idxd_wqs_unmap_portal(struct idxd_device *idxd) } } -static void __idxd_wq_set_priv_locked(struct idxd_wq *wq, int priv) -{ - struct idxd_device *idxd = wq->idxd; - union wqcfg wqcfg; - unsigned int offset; - - offset = WQCFG_OFFSET(idxd, wq->id, WQCFG_PRIVL_IDX); - spin_lock(&idxd->dev_lock); - wqcfg.bits[WQCFG_PRIVL_IDX] = ioread32(idxd->reg_base + offset); - wqcfg.priv = priv; - wq->wqcfg->bits[WQCFG_PRIVL_IDX] = wqcfg.bits[WQCFG_PRIVL_IDX]; - iowrite32(wqcfg.bits[WQCFG_PRIVL_IDX], idxd->reg_base + offset); - spin_unlock(&idxd->dev_lock); -} - static void __idxd_wq_set_pasid_locked(struct idxd_wq *wq, int pasid) { struct idxd_device *idxd = wq->idxd; @@ -1324,15 +1309,14 @@ int drv_enable_wq(struct idxd_wq *wq) } /* - * In the event that the WQ is configurable for pasid and priv bits. - * For kernel wq, the driver should setup the pasid, pasid_en, and priv bit. - * However, for non-kernel wq, the driver should only set the pasid_en bit for - * shared wq. A dedicated wq that is not 'kernel' type will configure pasid and + * In the event that the WQ is configurable for pasid, the driver + * should setup the pasid, pasid_en bit. This is true for both kernel + * and user shared workqueues. There is no need to setup priv bit in + * that in-kernel DMA will also do user privileged requests. + * A dedicated wq that is not 'kernel' type will configure pasid and * pasid_en later on so there is no need to setup. */ if (test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags)) { - int priv = 0; - if (wq_pasid_enabled(wq)) { if (is_idxd_wq_kernel(wq) || wq_shared(wq)) { u32 pasid = wq_dedicated(wq) ? idxd->pasid : 0; @@ -1340,10 +1324,6 @@ int drv_enable_wq(struct idxd_wq *wq) __idxd_wq_set_pasid_locked(wq, pasid); } } - - if (is_idxd_wq_kernel(wq)) - priv = 1; - __idxd_wq_set_priv_locked(wq, priv); } rc = 0; diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index f30eef701970..dadc908318aa 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -501,14 +501,52 @@ static struct idxd_device *idxd_alloc(struct pci_dev *pdev, struct idxd_driver_d static int idxd_enable_system_pasid(struct idxd_device *idxd) { - return -EOPNOTSUPP; + struct pci_dev *pdev = idxd->pdev; + struct device *dev = &pdev->dev; + struct iommu_domain *domain; + union gencfg_reg gencfg; + ioasid_t pasid; + int ret; + + domain = iommu_get_domain_for_dev(dev); + if (!domain || domain->type == IOMMU_DOMAIN_BLOCKED) + return -EPERM; + + pasid = iommu_sva_reserve_pasid(1, dev->iommu->max_pasids); + if (pasid == IOMMU_PASID_INVALID) + return -ENOSPC; + + ret = iommu_attach_device_pasid(domain, dev, pasid); + if (ret) { + dev_err(dev, "failed to attach device pasid %d, domain type %d", + pasid, domain->type); + iommu_sva_unreserve_pasid(pasid); + return ret; + } + + /* Since we set user privilege for kernel DMA, enable completion IRQ */ + gencfg.bits = ioread32(idxd->reg_base + IDXD_GENCFG_OFFSET); + gencfg.user_int_en = 1; + iowrite32(gencfg.bits, idxd->reg_base + IDXD_GENCFG_OFFSET); + idxd->pasid = pasid; + + return ret; } static void idxd_disable_system_pasid(struct idxd_device *idxd) { + struct pci_dev *pdev = idxd->pdev; + struct device *dev = &pdev->dev; + struct iommu_domain *domain; + + domain = iommu_get_domain_for_dev(dev); + if (!domain || domain->type == IOMMU_DOMAIN_BLOCKED) + return; - iommu_sva_unbind_device(idxd->sva); + iommu_detach_device_pasid(domain, dev, idxd->pasid); + iommu_sva_unreserve_pasid(idxd->pasid); idxd->sva = NULL; + idxd->pasid = IOMMU_PASID_INVALID; } static int idxd_probe(struct idxd_device *idxd) @@ -530,8 +568,9 @@ static int idxd_probe(struct idxd_device *idxd) } else { set_bit(IDXD_FLAG_USER_PASID_ENABLED, &idxd->flags); - if (idxd_enable_system_pasid(idxd)) - dev_warn(dev, "No in-kernel DMA with PASID.\n"); + rc = idxd_enable_system_pasid(idxd); + if (rc) + dev_warn(dev, "No in-kernel DMA with PASID. %d\n", rc); else set_bit(IDXD_FLAG_PASID_ENABLED, &idxd->flags); } diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c index 3229dfc78650..09f5c3f2a992 100644 --- a/drivers/dma/idxd/sysfs.c +++ b/drivers/dma/idxd/sysfs.c @@ -944,13 +944,6 @@ static ssize_t wq_name_store(struct device *dev, if (strlen(buf) > WQ_NAME_SIZE || strlen(buf) == 0) return -EINVAL; - /* - * This is temporarily placed here until we have SVM support for - * dmaengine. - */ - if (wq->type == IDXD_WQT_KERNEL && device_pasid_enabled(wq->idxd)) - return -EOPNOTSUPP; - input = kstrndup(buf, count, GFP_KERNEL); if (!input) return -ENOMEM;