From patchwork Fri Mar 24 12:02:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 13186688 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A6BDC76196 for ; Fri, 24 Mar 2023 12:02:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231666AbjCXMC2 (ORCPT ); Fri, 24 Mar 2023 08:02:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230092AbjCXMC1 (ORCPT ); Fri, 24 Mar 2023 08:02:27 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 924CE23DBA; Fri, 24 Mar 2023 05:02:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679659346; x=1711195346; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6qHH3kdmqICVZ9+zA4TiLrfUZ5cUI9ZxES/yWptHuC0=; b=DiFQlU9Vf64fnPa11KS8FDtZZJ2EKVuFQtZveMXYA8NmEoEwWx/LzNvJ MxdFy9hPhPRKC00u2JnxWeg0YrIjxN1a+XlqKBTi4df2CB2FrJZqjRRw8 +zWENGCQjlywANjBoq/koDEyqAnjLEfTpDV7YlyiupdjG4+qQLmDrTxxe bfY8XBZCpRwxqPFTaTYsQQXKUESvaonlIaa2eONd0YTV2u3j6fcPaeO6g XpeudK60LdyzcqZwsbmpOYsa4nhEpZwRYDVLdtSLWBbjFW5VAbI0vz9ba 2RJigUFL3RhCCy0t412CR88KfsIsXwv6FWZ16jyRuj8i59sa1YMR8hWVK A==; X-IronPort-AV: E=McAfee;i="6600,9927,10658"; a="323634139" X-IronPort-AV: E=Sophos;i="5.98,287,1673942400"; d="scan'208";a="323634139" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2023 05:02:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10658"; a="682674713" X-IronPort-AV: E=Sophos;i="5.98,287,1673942400"; d="scan'208";a="682674713" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 24 Mar 2023 05:02:23 -0700 From: Lu Baolu To: iommu@lists.linux.dev, dmaengine@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Kevin Tian , Fenghua Yu , Dave Jiang , Vinod Koul , Jacob Pan , linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v3 1/6] dmaengine: idxd: Add enable/disable device IOPF feature Date: Fri, 24 Mar 2023 20:02:29 +0800 Message-Id: <20230324120234.313643-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230324120234.313643-1-baolu.lu@linux.intel.com> References: <20230324120234.313643-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org The iommu subsystem requires IOMMU_DEV_FEAT_IOPF must be enabled before and disabled after IOMMU_DEV_FEAT_SVA, if device's I/O page faults rely on the IOMMU. Add explicit IOMMU_DEV_FEAT_IOPF enabling/disabling in this driver. At present, missing IOPF enabling/disabling doesn't cause any real issue, because the IOMMU driver places the IOPF enabling/disabling in the path of SVA feature handling. But this may change. Reviewed-by: Dave Jiang Reviewed-by: Fenghua Yu Reviewed-by: Kevin Tian Signed-off-by: Lu Baolu --- drivers/dma/idxd/init.c | 31 +++++++++++++++++++++++++------ 1 file changed, 25 insertions(+), 6 deletions(-) diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index 640d3048368e..09ef62aa0635 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -516,6 +516,27 @@ static void idxd_disable_system_pasid(struct idxd_device *idxd) idxd->sva = NULL; } +static int idxd_enable_sva(struct pci_dev *pdev) +{ + int ret; + + ret = iommu_dev_enable_feature(&pdev->dev, IOMMU_DEV_FEAT_IOPF); + if (ret) + return ret; + + ret = iommu_dev_enable_feature(&pdev->dev, IOMMU_DEV_FEAT_SVA); + if (ret) + iommu_dev_disable_feature(&pdev->dev, IOMMU_DEV_FEAT_IOPF); + + return ret; +} + +static void idxd_disable_sva(struct pci_dev *pdev) +{ + iommu_dev_disable_feature(&pdev->dev, IOMMU_DEV_FEAT_SVA); + iommu_dev_disable_feature(&pdev->dev, IOMMU_DEV_FEAT_IOPF); +} + static int idxd_probe(struct idxd_device *idxd) { struct pci_dev *pdev = idxd->pdev; @@ -530,7 +551,7 @@ static int idxd_probe(struct idxd_device *idxd) dev_dbg(dev, "IDXD reset complete\n"); if (IS_ENABLED(CONFIG_INTEL_IDXD_SVM) && sva) { - if (iommu_dev_enable_feature(dev, IOMMU_DEV_FEAT_SVA)) { + if (idxd_enable_sva(pdev)) { dev_warn(dev, "Unable to turn on user SVA feature.\n"); } else { set_bit(IDXD_FLAG_USER_PASID_ENABLED, &idxd->flags); @@ -578,21 +599,19 @@ static int idxd_probe(struct idxd_device *idxd) if (device_pasid_enabled(idxd)) idxd_disable_system_pasid(idxd); if (device_user_pasid_enabled(idxd)) - iommu_dev_disable_feature(dev, IOMMU_DEV_FEAT_SVA); + idxd_disable_sva(pdev); return rc; } static void idxd_cleanup(struct idxd_device *idxd) { - struct device *dev = &idxd->pdev->dev; - perfmon_pmu_remove(idxd); idxd_cleanup_interrupts(idxd); idxd_cleanup_internals(idxd); if (device_pasid_enabled(idxd)) idxd_disable_system_pasid(idxd); if (device_user_pasid_enabled(idxd)) - iommu_dev_disable_feature(dev, IOMMU_DEV_FEAT_SVA); + idxd_disable_sva(idxd->pdev); } static int idxd_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) @@ -710,7 +729,7 @@ static void idxd_remove(struct pci_dev *pdev) pci_free_irq_vectors(pdev); pci_iounmap(pdev, idxd->reg_base); if (device_user_pasid_enabled(idxd)) - iommu_dev_disable_feature(&pdev->dev, IOMMU_DEV_FEAT_SVA); + idxd_disable_sva(pdev); pci_disable_device(pdev); destroy_workqueue(idxd->wq); perfmon_pmu_remove(idxd); From patchwork Fri Mar 24 12:02:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 13186689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 336EFC76196 for ; Fri, 24 Mar 2023 12:02:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231703AbjCXMCc (ORCPT ); Fri, 24 Mar 2023 08:02:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42544 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231748AbjCXMCa (ORCPT ); Fri, 24 Mar 2023 08:02:30 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F2E923C68; Fri, 24 Mar 2023 05:02:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679659349; x=1711195349; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6HPFNPQDz5pHaQknWW/YnEHrxyK/c9dkAaXEif8vAkY=; b=gqITNorM+C+pnjn4SwX5m1sFUFHtTtPw42toi+mIMcZtcPRZ2Q1BEyZd qlu1R3dlBKIxJWhaGt3I8TTazSRwhs4HKKforPWRQt4k/vsIHCEPvK28P NJ9IB4lq19RiirBJWdFG0iYh4eoQxpEE+fvCq728IWECY2r+fu4dJddqz PDbPBD15iLk5SR3A+r5g7kHxaxFJpEBzRkNFhcfa1nG1OU2bW/8H+FGG8 j/mAj3JMmGw2qLlYViGJrO0rUPpErisotfF8PQAq3X9gOuZPyqk7dV9QM I6xB5ktNLHly0n1TKmJCGIKEPA05TlNyrlXJHZ3h8D5xkLYgV02DuscjH A==; X-IronPort-AV: E=McAfee;i="6600,9927,10658"; a="323634149" X-IronPort-AV: E=Sophos;i="5.98,287,1673942400"; d="scan'208";a="323634149" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2023 05:02:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10658"; a="682674743" X-IronPort-AV: E=Sophos;i="5.98,287,1673942400"; d="scan'208";a="682674743" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 24 Mar 2023 05:02:26 -0700 From: Lu Baolu To: iommu@lists.linux.dev, dmaengine@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Kevin Tian , Fenghua Yu , Dave Jiang , Vinod Koul , Jacob Pan , linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v3 2/6] iommu/vt-d: Allow SVA with device-specific IOPF Date: Fri, 24 Mar 2023 20:02:30 +0800 Message-Id: <20230324120234.313643-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230324120234.313643-1-baolu.lu@linux.intel.com> References: <20230324120234.313643-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org Currently enabling SVA requires IOPF support from the IOMMU and device PCI PRI. However, some devices can handle IOPF by itself without ever sending PCI page requests nor advertising PRI capability. Allow SVA support with IOPF handled either by IOMMU (PCI PRI) or device driver (device-specific IOPF). As long as IOPF could be handled, SVA should continue to work. Signed-off-by: Lu Baolu Reviewed-by: Kevin Tian --- drivers/iommu/intel/iommu.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 7c2f4bd33582..caf664448ee9 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4650,7 +4650,21 @@ static int intel_iommu_enable_sva(struct device *dev) if (!(iommu->flags & VTD_FLAG_SVM_CAPABLE)) return -ENODEV; - if (!info->pasid_enabled || !info->pri_enabled || !info->ats_enabled) + if (!info->pasid_enabled || !info->ats_enabled) + return -EINVAL; + + /* + * Devices having device-specific I/O fault handling should not + * support PCI/PRI. The IOMMU side has no means to check the + * capability of device-specific IOPF. Therefore, IOMMU can only + * default that if the device driver enables SVA on a non-PRI + * device, it will handle IOPF in its own way. + */ + if (!info->pri_supported) + return 0; + + /* Devices supporting PRI should have it enabled. */ + if (!info->pri_enabled) return -EINVAL; ret = iopf_queue_add_device(iommu->iopf_queue, dev); From patchwork Fri Mar 24 12:02:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 13186690 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04B5DC6FD20 for ; Fri, 24 Mar 2023 12:02:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231801AbjCXMCk (ORCPT ); Fri, 24 Mar 2023 08:02:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42552 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231655AbjCXMCd (ORCPT ); Fri, 24 Mar 2023 08:02:33 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44357241DE; Fri, 24 Mar 2023 05:02:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679659352; x=1711195352; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=HlO9DJUB1VeUc0i1Z73Y/pek9tXf+C6otlR5IrcBXs8=; b=mKWkI3tkbYguRG4hfd0Vxh5zmLMtQfSXqG6g7fubW7ZC4xaIlP3hB2jN oMzCC4gWH6G8xqSOONZqo3Ud4OvWKULPApYZl2V4D3iTosTHedbbRa9wO RuDuOHUNDia6if7vXC94K3TZMVP1w7WwUf7plKR6rkpY7WcsCOkW6iI+1 QGocNBywheSpZGlIu9Z1r0zSe6ouvqBS9HJcrMRbyeN8bxZeakWdUT6fc Vgi1hTfJAs1RhLfVJO4TbxfUKMfO8sZc5/tmwgEcFwBPBmCzFweQH/KDC ZVtzGC+MXUw4kPXdR9I0IboIkJLz0IeOTDuGPZw9uzhGHXkJYySZJLdr5 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10658"; a="323634155" X-IronPort-AV: E=Sophos;i="5.98,287,1673942400"; d="scan'208";a="323634155" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2023 05:02:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10658"; a="682674777" X-IronPort-AV: E=Sophos;i="5.98,287,1673942400"; d="scan'208";a="682674777" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 24 Mar 2023 05:02:29 -0700 From: Lu Baolu To: iommu@lists.linux.dev, dmaengine@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Kevin Tian , Fenghua Yu , Dave Jiang , Vinod Koul , Jacob Pan , linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v3 3/6] iommu/vt-d: Move iopf code from SVA to IOPF enabling path Date: Fri, 24 Mar 2023 20:02:31 +0800 Message-Id: <20230324120234.313643-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230324120234.313643-1-baolu.lu@linux.intel.com> References: <20230324120234.313643-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org Generally enabling IOMMU_DEV_FEAT_SVA requires IOMMU_DEV_FEAT_IOPF, but some devices manage I/O Page Faults themselves instead of relying on the IOMMU. Move IOPF related code from SVA to IOPF enabling path. For the device drivers that relies on the IOMMU for IOPF through PCI/PRI, IOMMU_DEV_FEAT_IOPF must be enabled before and disabled after IOMMU_DEV_FEAT_SVA. Reviewed-by: Kevin Tian Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.c | 32 ++++++++++++++++++-------------- 1 file changed, 18 insertions(+), 14 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index caf664448ee9..a6f07c74da2d 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4638,7 +4638,6 @@ static int intel_iommu_enable_sva(struct device *dev) { struct device_domain_info *info = dev_iommu_priv_get(dev); struct intel_iommu *iommu; - int ret; if (!info || dmar_disabled) return -EINVAL; @@ -4667,6 +4666,21 @@ static int intel_iommu_enable_sva(struct device *dev) if (!info->pri_enabled) return -EINVAL; + return 0; +} + +static int intel_iommu_enable_iopf(struct device *dev) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct intel_iommu *iommu; + int ret; + + if (!info || !info->ats_enabled || !info->pri_enabled) + return -ENODEV; + iommu = info->iommu; + if (!iommu) + return -EINVAL; + ret = iopf_queue_add_device(iommu->iopf_queue, dev); if (ret) return ret; @@ -4678,7 +4692,7 @@ static int intel_iommu_enable_sva(struct device *dev) return ret; } -static int intel_iommu_disable_sva(struct device *dev) +static int intel_iommu_disable_iopf(struct device *dev) { struct device_domain_info *info = dev_iommu_priv_get(dev); struct intel_iommu *iommu = info->iommu; @@ -4695,16 +4709,6 @@ static int intel_iommu_disable_sva(struct device *dev) return ret; } -static int intel_iommu_enable_iopf(struct device *dev) -{ - struct device_domain_info *info = dev_iommu_priv_get(dev); - - if (info && info->pri_supported) - return 0; - - return -ENODEV; -} - static int intel_iommu_dev_enable_feat(struct device *dev, enum iommu_dev_features feat) { @@ -4725,10 +4729,10 @@ intel_iommu_dev_disable_feat(struct device *dev, enum iommu_dev_features feat) { switch (feat) { case IOMMU_DEV_FEAT_IOPF: - return 0; + return intel_iommu_disable_iopf(dev); case IOMMU_DEV_FEAT_SVA: - return intel_iommu_disable_sva(dev); + return 0; default: return -ENODEV; From patchwork Fri Mar 24 12:02:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 13186691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56244C6FD20 for ; Fri, 24 Mar 2023 12:02:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231823AbjCXMCr (ORCPT ); Fri, 24 Mar 2023 08:02:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231792AbjCXMCm (ORCPT ); Fri, 24 Mar 2023 08:02:42 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4D6C7241DA; Fri, 24 Mar 2023 05:02:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679659355; x=1711195355; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=J8yNL0xsswqSZQ3dx1kUSPhIC1pxHFcfvF18SpMWQe8=; b=iEwNKOgjgV1jHggFkxUNPb36K7099jnco1mNGGWu5Dev1nys4vh3/aGz MhoowBbaUW72dKTqhh9D1NqGe5svBoA9MSEh/Ss6N+xLAOsiqTTqmwZ2H D3hKajKQ2ztElwm+THeM5m56GVv2ulRIRLM4JEbx7+5xpuUJ5nVG4urah 6meXjltwSHPxnhb8km19C19x8EAGZBpyP/0Om8C6n/ZzcvKYJZU1ueQ6E qFkAsLyWZdRhoI8EEUlXMJ/4ut5yUorAy909ExoUVCVQx4UUkdNv6SbKj iB/UEubMwxxXHwxNpCBOJqKXbmo84WbOzaJiO3eWq/0wklIqIQwl2uNKA A==; X-IronPort-AV: E=McAfee;i="6600,9927,10658"; a="323634165" X-IronPort-AV: E=Sophos;i="5.98,287,1673942400"; d="scan'208";a="323634165" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2023 05:02:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10658"; a="682674824" X-IronPort-AV: E=Sophos;i="5.98,287,1673942400"; d="scan'208";a="682674824" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 24 Mar 2023 05:02:32 -0700 From: Lu Baolu To: iommu@lists.linux.dev, dmaengine@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Kevin Tian , Fenghua Yu , Dave Jiang , Vinod Koul , Jacob Pan , linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v3 4/6] iommu/vt-d: Move pfsid and ats_qdep calculation to device probe path Date: Fri, 24 Mar 2023 20:02:32 +0800 Message-Id: <20230324120234.313643-5-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230324120234.313643-1-baolu.lu@linux.intel.com> References: <20230324120234.313643-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org They should be part of the per-device iommu private data initialization. Reviewed-by: Jacob Pan Reviewed-by: Kevin Tian Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.c | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index a6f07c74da2d..6d77b4072fdd 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1406,20 +1406,6 @@ static void iommu_enable_pci_caps(struct device_domain_info *info) return; pdev = to_pci_dev(info->dev); - /* For IOMMU that supports device IOTLB throttling (DIT), we assign - * PFSID to the invalidation desc of a VF such that IOMMU HW can gauge - * queue depth at PF level. If DIT is not set, PFSID will be treated as - * reserved, which should be set to 0. - */ - if (!ecap_dit(info->iommu->ecap)) - info->pfsid = 0; - else { - struct pci_dev *pf_pdev; - - /* pdev will be returned if device is not a vf */ - pf_pdev = pci_physfn(pdev); - info->pfsid = pci_dev_id(pf_pdev); - } /* The PCIe spec, in its wisdom, declares that the behaviour of the device if you enable PASID support after ATS support is @@ -1438,7 +1424,6 @@ static void iommu_enable_pci_caps(struct device_domain_info *info) !pci_enable_ats(pdev, VTD_PAGE_SHIFT)) { info->ats_enabled = 1; domain_update_iotlb(info->domain); - info->ats_qdep = pci_ats_queue_depth(pdev); } } @@ -4521,6 +4506,17 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev) dmar_ats_supported(pdev, iommu)) { info->ats_supported = 1; info->dtlb_extra_inval = dev_needs_extra_dtlb_flush(pdev); + + /* + * For IOMMU that supports device IOTLB throttling + * (DIT), we assign PFSID to the invalidation desc + * of a VF such that IOMMU HW can gauge queue depth + * at PF level. If DIT is not set, PFSID will be + * treated as reserved, which should be set to 0. + */ + if (ecap_dit(iommu->ecap)) + info->pfsid = pci_dev_id(pci_physfn(pdev)); + info->ats_qdep = pci_ats_queue_depth(pdev); } if (sm_supported(iommu)) { if (pasid_supported(iommu)) { From patchwork Fri Mar 24 12:02:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 13186692 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CAA2C6FD20 for ; Fri, 24 Mar 2023 12:02:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231819AbjCXMC4 (ORCPT ); Fri, 24 Mar 2023 08:02:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231840AbjCXMCr (ORCPT ); Fri, 24 Mar 2023 08:02:47 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96E5C244B6; Fri, 24 Mar 2023 05:02:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679659358; x=1711195358; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nDsWWHt3XXkQGYiQaHFSNmSDapZTcbGSD8doZIV3ewQ=; b=U8cC2O/xvTnOy8Be1KmMz5zPdVy3edAdm5TraWFz42koY+b4X5NaoFyu Q+oglOYYjPAO/2RgSL6dsiU4ysLshElpyuMysMEGa26dOjRVVBtgcID93 6iXtZS/Q/3L/dF3OdsLrdm86nRpgTF+/OIXt5tDxBHwFNDmyU+LA3drVf YNAYD1cHvsedMA4aONdRnT1YSsT6gDGuwlmYWS6ltKx19LArX870HaxgA F1l+ZM7/Xsmaaxw9sY0xb25dod3yxZa6i0zlkSP0N82+/ZbdoDCA/eZIk UnSNFv0C4Ts6rxZ258nyPYt7iBFkHLICX6lyOzop9fqFMah/3oR3/b0EP g==; X-IronPort-AV: E=McAfee;i="6600,9927,10658"; a="323634179" X-IronPort-AV: E=Sophos;i="5.98,287,1673942400"; d="scan'208";a="323634179" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2023 05:02:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10658"; a="682674883" X-IronPort-AV: E=Sophos;i="5.98,287,1673942400"; d="scan'208";a="682674883" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 24 Mar 2023 05:02:35 -0700 From: Lu Baolu To: iommu@lists.linux.dev, dmaengine@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Kevin Tian , Fenghua Yu , Dave Jiang , Vinod Koul , Jacob Pan , linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v3 5/6] iommu/vt-d: Move PRI handling to IOPF feature path Date: Fri, 24 Mar 2023 20:02:33 +0800 Message-Id: <20230324120234.313643-6-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230324120234.313643-1-baolu.lu@linux.intel.com> References: <20230324120234.313643-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org PRI is only used for IOPF. With this move, the PCI/PRI feature could be controlled by the device driver through iommu_dev_enable/disable_feature() interfaces. Reviewed-by: Jacob Pan Signed-off-by: Lu Baolu Reviewed-by: Kevin Tian --- drivers/iommu/intel/iommu.c | 53 ++++++++++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 12 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 6d77b4072fdd..cd3a3c4b5e64 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1415,11 +1415,6 @@ static void iommu_enable_pci_caps(struct device_domain_info *info) if (info->pasid_supported && !pci_enable_pasid(pdev, info->pasid_supported & ~1)) info->pasid_enabled = 1; - if (info->pri_supported && - (info->pasid_enabled ? pci_prg_resp_pasid_required(pdev) : 1) && - !pci_reset_pri(pdev) && !pci_enable_pri(pdev, PRQ_DEPTH)) - info->pri_enabled = 1; - if (info->ats_supported && pci_ats_page_aligned(pdev) && !pci_enable_ats(pdev, VTD_PAGE_SHIFT)) { info->ats_enabled = 1; @@ -1442,11 +1437,6 @@ static void iommu_disable_pci_caps(struct device_domain_info *info) domain_update_iotlb(info->domain); } - if (info->pri_enabled) { - pci_disable_pri(pdev); - info->pri_enabled = 0; - } - if (info->pasid_enabled) { pci_disable_pasid(pdev); info->pasid_enabled = 0; @@ -4667,23 +4657,48 @@ static int intel_iommu_enable_sva(struct device *dev) static int intel_iommu_enable_iopf(struct device *dev) { + struct pci_dev *pdev = dev_is_pci(dev) ? to_pci_dev(dev) : NULL; struct device_domain_info *info = dev_iommu_priv_get(dev); struct intel_iommu *iommu; int ret; - if (!info || !info->ats_enabled || !info->pri_enabled) + if (!pdev || !info || !info->ats_enabled || !info->pri_supported) return -ENODEV; + + if (info->pri_enabled) + return -EBUSY; + iommu = info->iommu; if (!iommu) return -EINVAL; + /* PASID is required in PRG Response Message. */ + if (info->pasid_enabled && !pci_prg_resp_pasid_required(pdev)) + return -EINVAL; + + ret = pci_reset_pri(pdev); + if (ret) + return ret; + ret = iopf_queue_add_device(iommu->iopf_queue, dev); if (ret) return ret; ret = iommu_register_device_fault_handler(dev, iommu_queue_iopf, dev); if (ret) - iopf_queue_remove_device(iommu->iopf_queue, dev); + goto iopf_remove_device; + + ret = pci_enable_pri(pdev, PRQ_DEPTH); + if (ret) + goto iopf_unregister_handler; + info->pri_enabled = 1; + + return 0; + +iopf_unregister_handler: + iommu_unregister_device_fault_handler(dev); +iopf_remove_device: + iopf_queue_remove_device(iommu->iopf_queue, dev); return ret; } @@ -4694,6 +4709,20 @@ static int intel_iommu_disable_iopf(struct device *dev) struct intel_iommu *iommu = info->iommu; int ret; + if (!info->pri_enabled) + return -EINVAL; + + /* + * PCIe spec states that by clearing PRI enable bit, the Page + * Request Interface will not issue new page requests, but has + * outstanding page requests that have been transmitted or are + * queued for transmission. This is supposed to be called after + * the device driver has stopped DMA, all PASIDs have been + * unbound and the outstanding PRQs have been drained. + */ + pci_disable_pri(to_pci_dev(dev)); + info->pri_enabled = 0; + ret = iommu_unregister_device_fault_handler(dev); if (ret) return ret; From patchwork Fri Mar 24 12:02:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 13186693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0473BC6FD20 for ; Fri, 24 Mar 2023 12:03:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231898AbjCXMDI (ORCPT ); Fri, 24 Mar 2023 08:03:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231833AbjCXMCv (ORCPT ); Fri, 24 Mar 2023 08:02:51 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A205265B8; Fri, 24 Mar 2023 05:02:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679659361; x=1711195361; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=M/Jk/NyOL6YOkh9WpMFp/FMLnkzRYwNbBzLyoNFzzAU=; b=Tp5a5DlI92vpGpnCNmXRTLEMyGYXWHELMjRzcubt+IXqopxokZmzqfSW KVSN/K4KyaboIXyMOiehaL3XB5f6R1ZWMjduEBj56Sv/oZXVUYU2goAvK 5VbV+QrGuVMk4L6DHat3uMMRt2WcZF2ZUdb0TLcB4k+AxCRd8QucJSKys 89HQLLUybgG+nAwV4980m3FlgoSsJcBkHx0+kbrPL2hWtRBdAYUFeKtkf kQzvBdqY6XWAu2splMDWXmziMo4J5r/lzKq+O0+b3GvVF5j4tqjQkSkLO 9dgZMZzQ+W1F1XjNd3hPwzDnhrb28DP2wakcng75YVM0MXbDF7p1Fwe8o g==; X-IronPort-AV: E=McAfee;i="6600,9927,10658"; a="323634194" X-IronPort-AV: E=Sophos;i="5.98,287,1673942400"; d="scan'208";a="323634194" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2023 05:02:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10658"; a="682674914" X-IronPort-AV: E=Sophos;i="5.98,287,1673942400"; d="scan'208";a="682674914" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 24 Mar 2023 05:02:38 -0700 From: Lu Baolu To: iommu@lists.linux.dev, dmaengine@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Kevin Tian , Fenghua Yu , Dave Jiang , Vinod Koul , Jacob Pan , linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v3 6/6] iommu/vt-d: Remove unnecessary checks in iopf disabling path Date: Fri, 24 Mar 2023 20:02:34 +0800 Message-Id: <20230324120234.313643-7-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230324120234.313643-1-baolu.lu@linux.intel.com> References: <20230324120234.313643-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org iommu_unregister_device_fault_handler() and iopf_queue_remove_device() are called after device has stopped issuing new page falut requests and all outstanding page requests have been drained. They should never fail. Trigger a warning if it happens unfortunately. Signed-off-by: Lu Baolu Reviewed-by: Kevin Tian --- drivers/iommu/intel/iommu.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index cd3a3c4b5e64..c771233d6f2a 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4707,7 +4707,6 @@ static int intel_iommu_disable_iopf(struct device *dev) { struct device_domain_info *info = dev_iommu_priv_get(dev); struct intel_iommu *iommu = info->iommu; - int ret; if (!info->pri_enabled) return -EINVAL; @@ -4723,15 +4722,15 @@ static int intel_iommu_disable_iopf(struct device *dev) pci_disable_pri(to_pci_dev(dev)); info->pri_enabled = 0; - ret = iommu_unregister_device_fault_handler(dev); - if (ret) - return ret; - - ret = iopf_queue_remove_device(iommu->iopf_queue, dev); - if (ret) - iommu_register_device_fault_handler(dev, iommu_queue_iopf, dev); + /* + * With PRI disabled and outstanding PRQs drained, unregistering + * fault handler and removing device from iopf queue should never + * fail. + */ + WARN_ON(iommu_unregister_device_fault_handler(dev)); + WARN_ON(iopf_queue_remove_device(iommu->iopf_queue, dev)); - return ret; + return 0; } static int