From patchwork Mon May 6 17:20:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kuppuswamy Sathyanarayanan X-Patchwork-Id: 10931639 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 078A81390 for ; Mon, 6 May 2019 17:22:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DEC1128893 for ; Mon, 6 May 2019 17:22:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D2CBF288B2; Mon, 6 May 2019 17:22:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5237128893 for ; Mon, 6 May 2019 17:22:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726473AbfEFRWR (ORCPT ); Mon, 6 May 2019 13:22:17 -0400 Received: from mga07.intel.com ([134.134.136.100]:31139 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726393AbfEFRWR (ORCPT ); Mon, 6 May 2019 13:22:17 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 May 2019 10:22:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,438,1549958400"; d="scan'208";a="230014471" Received: from skuppusw-desk.jf.intel.com ([10.54.74.33]) by orsmga001.jf.intel.com with ESMTP; 06 May 2019 10:22:15 -0700 From: sathyanarayanan.kuppuswamy@linux.intel.com To: bhelgaas@google.com Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, ashok.raj@intel.com, keith.busch@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com Subject: [PATCH v2 1/5] PCI/ATS: Add PRI support for PCIe VF devices Date: Mon, 6 May 2019 10:20:03 -0700 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Kuppuswamy Sathyanarayanan When IOMMU tries to enable PRI for VF device in iommu_enable_dev_iotlb(), it always fails because PRI support for PCIe VF device is currently broken in PCIE driver. Current implementation expects the given PCIe device (PF & VF) to implement PRI capability before enabling the PRI support. But this assumption is incorrect. As per PCIe spec r4.0, sec 9.3.7.11, all VFs associated with PF can only use the Page Request Interface (PRI) of the PF and not implement it. Hence we need to create exception for handling the PRI support for PCIe VF device. Since PRI is shared between PF/VF devices, following rules should apply. 1. Enable PRI in VF only if its already enabled in PF. 2. When enabling/disabling PRI for VF, instead of configuring the registers just increase/decrease the usage count (pri_ref_cnt) of PF. 3. Disable PRI in PF only if pr_ref_cnt is zero. Cc: Ashok Raj Cc: Keith Busch Suggested-by: Ashok Raj Signed-off-by: Kuppuswamy Sathyanarayanan --- drivers/pci/ats.c | 53 +++++++++++++++++++++++++++++++++++++++++++-- include/linux/pci.h | 1 + 2 files changed, 52 insertions(+), 2 deletions(-) diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c index 97c08146534a..5582e5d83a3f 100644 --- a/drivers/pci/ats.c +++ b/drivers/pci/ats.c @@ -181,12 +181,39 @@ int pci_enable_pri(struct pci_dev *pdev, u32 reqs) u16 control, status; u32 max_requests; int pos; + struct pci_dev *pf; if (WARN_ON(pdev->pri_enabled)) return -EBUSY; pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PRI); - if (!pos) + + if (pdev->is_virtfn) { + /* + * Per PCIe r4.0, sec 9.3.7.11, VF must not implement PRI + * Capability. + */ + if (pos) { + dev_err(&pdev->dev, "VF must not implement PRI"); + return -EINVAL; + } + + pf = pci_physfn(pdev); + + /* If VF config does not match with PF, return error */ + if (!pf->pri_enabled) + return -EINVAL; + + pdev->pri_reqs_alloc = pf->pri_reqs_alloc; + pdev->pri_enabled = 1; + + /* Increment PF PRI refcount */ + atomic_inc(&pf->pri_ref_cnt); + + return 0; + } + + if (pdev->is_physfn && !pos) return -EINVAL; pci_read_config_word(pdev, pos + PCI_PRI_STATUS, &status); @@ -202,7 +229,6 @@ int pci_enable_pri(struct pci_dev *pdev, u32 reqs) pci_write_config_word(pdev, pos + PCI_PRI_CTRL, control); pdev->pri_enabled = 1; - return 0; } EXPORT_SYMBOL_GPL(pci_enable_pri); @@ -217,10 +243,27 @@ void pci_disable_pri(struct pci_dev *pdev) { u16 control; int pos; + struct pci_dev *pf; if (WARN_ON(!pdev->pri_enabled)) return; + /* All VFs should be disabled before disabling PF */ + if (atomic_read(&pdev->pri_ref_cnt)) + return; + + if (pdev->is_virtfn) { + /* Since VF shares PRI with PF, use PF config. */ + pf = pci_physfn(pdev); + + /* Decrement PF PRI refcount */ + atomic_dec(&pf->pri_ref_cnt); + + pdev->pri_enabled = 0; + + return; + } + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PRI); if (!pos) return; @@ -246,6 +289,9 @@ void pci_restore_pri_state(struct pci_dev *pdev) if (!pdev->pri_enabled) return; + if (pdev->is_virtfn) + return; + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PRI); if (!pos) return; @@ -270,6 +316,9 @@ int pci_reset_pri(struct pci_dev *pdev) if (WARN_ON(pdev->pri_enabled)) return -EBUSY; + if (pdev->is_virtfn) + return 0; + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PRI); if (!pos) return -EINVAL; diff --git a/include/linux/pci.h b/include/linux/pci.h index 77448215ef5b..699c79c99a39 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -450,6 +450,7 @@ struct pci_dev { #endif #ifdef CONFIG_PCI_PRI u32 pri_reqs_alloc; /* Number of PRI requests allocated */ + atomic_t pri_ref_cnt; /* Number of VFs with PRI enabled */ #endif #ifdef CONFIG_PCI_PASID u16 pasid_features; From patchwork Mon May 6 17:20:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kuppuswamy Sathyanarayanan X-Patchwork-Id: 10931647 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 633D5924 for ; Mon, 6 May 2019 17:22:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 424A2285E3 for ; Mon, 6 May 2019 17:22:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 408D32887E; Mon, 6 May 2019 17:22:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9A82828898 for ; Mon, 6 May 2019 17:22:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726393AbfEFRW3 (ORCPT ); Mon, 6 May 2019 13:22:29 -0400 Received: from mga07.intel.com ([134.134.136.100]:31140 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726762AbfEFRWS (ORCPT ); Mon, 6 May 2019 13:22:18 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 May 2019 10:22:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,438,1549958400"; d="scan'208";a="230014472" Received: from skuppusw-desk.jf.intel.com ([10.54.74.33]) by orsmga001.jf.intel.com with ESMTP; 06 May 2019 10:22:15 -0700 From: sathyanarayanan.kuppuswamy@linux.intel.com To: bhelgaas@google.com Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, ashok.raj@intel.com, keith.busch@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com Subject: [PATCH v2 2/5] PCI/ATS: Add PASID support for PCIe VF devices Date: Mon, 6 May 2019 10:20:04 -0700 Message-Id: <078b169334b4996d03d8608f205942c061590681.1557162861.git.sathyanarayanan.kuppuswamy@linux.intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Kuppuswamy Sathyanarayanan When IOMMU tries to enable PASID for VF device in iommu_enable_dev_iotlb(), it always fails because PASID support for PCIe VF device is currently broken in PCIE driver. Current implementation expects the given PCIe device (PF & VF) to implement PASID capability before enabling the PASID support. But this assumption is incorrect. As per PCIe spec r4.0, sec 9.3.7.14, all VFs associated with PF can only use the PASID of the PF and not implement it. Since PASID is shared between PF/VF devices, following rules should apply. 1. Enable PASID in VF only if its already enabled in PF. 2. Enable PASID in VF only if the requested features matches with PF config, otherwise return error. 3. When enabling/disabling PASID for VF, instead of configuring the PF registers just increase/decrease the usage count (pasid_ref_cnt). 4. Disable PASID in PF (configuring the registers) only if pasid_ref_cnt is zero. 5. When reading PASID features/settings for VF, use registers of corresponding PF. Cc: Ashok Raj Cc: Keith Busch Suggested-by: Ashok Raj Signed-off-by: Kuppuswamy Sathyanarayanan --- drivers/pci/ats.c | 55 ++++++++++++++++++++++++++++++++++++++++++++- include/linux/pci.h | 1 + 2 files changed, 55 insertions(+), 1 deletion(-) diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c index 5582e5d83a3f..e7a904e347c3 100644 --- a/drivers/pci/ats.c +++ b/drivers/pci/ats.c @@ -345,6 +345,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features) { u16 control, supported; int pos; + struct pci_dev *pf; if (WARN_ON(pdev->pasid_enabled)) return -EBUSY; @@ -353,7 +354,33 @@ int pci_enable_pasid(struct pci_dev *pdev, int features) return -EINVAL; pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PASID); - if (!pos) + + if (pdev->is_virtfn) { + /* + * Per PCIe r4.0, sec 9.3.7.14, VF must not implement + * Process Address Space ID (PASID) Capability. + */ + if (pos) { + dev_err(&pdev->dev, "VF must not implement PASID\n"); + return -EINVAL + } + /* Since VF shares PASID with PF, use PF config */ + pf = pci_physfn(pdev); + + /* If VF config does not match with PF, return error */ + if (!pf->pasid_enabled || pf->pasid_features != features) + return -EINVAL; + + pdev->pasid_features = features; + pdev->pasid_enabled = 1; + + /* Increment PF PASID refcount */ + atomic_inc(&pf->pasid_ref_cnt); + + return 0; + } + + if (pdev->is_physfn && !pos) return -EINVAL; pci_read_config_word(pdev, pos + PCI_PASID_CAP, &supported); @@ -382,10 +409,27 @@ void pci_disable_pasid(struct pci_dev *pdev) { u16 control = 0; int pos; + struct pci_dev *pf; if (WARN_ON(!pdev->pasid_enabled)) return; + /* All VFs PASID should be disabled before disabling PF PASID*/ + if (atomic_read(&pdev->pasid_ref_cnt)) + return; + + if (pdev->is_virtfn) { + /* Since VF shares PASID with PF, use PF config. */ + pf = pci_physfn(pdev); + + /* Decrement PF PASID refcount */ + atomic_dec(&pf->pasid_ref_cnt); + + pdev->pasid_enabled = 0; + + return; + } + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PASID); if (!pos) return; @@ -408,6 +452,9 @@ void pci_restore_pasid_state(struct pci_dev *pdev) if (!pdev->pasid_enabled) return; + if (pdev->is_virtfn) + return; + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PASID); if (!pos) return; @@ -432,6 +479,9 @@ int pci_pasid_features(struct pci_dev *pdev) u16 supported; int pos; + if (pdev->is_virtfn) + pdev = pci_physfn(pdev); + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PASID); if (!pos) return -EINVAL; @@ -488,6 +538,9 @@ int pci_max_pasids(struct pci_dev *pdev) u16 supported; int pos; + if (pdev->is_virtfn) + pdev = pci_physfn(pdev); + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PASID); if (!pos) return -EINVAL; diff --git a/include/linux/pci.h b/include/linux/pci.h index 699c79c99a39..2a761ea63f8d 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -454,6 +454,7 @@ struct pci_dev { #endif #ifdef CONFIG_PCI_PASID u16 pasid_features; + atomic_t pasid_ref_cnt; /* Number of VFs with PASID enabled */ #endif #ifdef CONFIG_PCI_P2PDMA struct pci_p2pdma *p2pdma; From patchwork Mon May 6 17:20:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kuppuswamy Sathyanarayanan X-Patchwork-Id: 10931641 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7FCC61390 for ; Mon, 6 May 2019 17:22:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5B9272887A for ; Mon, 6 May 2019 17:22:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5998F288B8; Mon, 6 May 2019 17:22:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC4742887A for ; Mon, 6 May 2019 17:22:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726792AbfEFRWS (ORCPT ); Mon, 6 May 2019 13:22:18 -0400 Received: from mga07.intel.com ([134.134.136.100]:31140 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726747AbfEFRWR (ORCPT ); Mon, 6 May 2019 13:22:17 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 May 2019 10:22:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,438,1549958400"; d="scan'208";a="230014473" Received: from skuppusw-desk.jf.intel.com ([10.54.74.33]) by orsmga001.jf.intel.com with ESMTP; 06 May 2019 10:22:15 -0700 From: sathyanarayanan.kuppuswamy@linux.intel.com To: bhelgaas@google.com Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, ashok.raj@intel.com, keith.busch@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com Subject: [PATCH v2 3/5] PCI/ATS: Skip VF ATS initialization if PF does not implement it Date: Mon, 6 May 2019 10:20:05 -0700 Message-Id: <21d93b3312418c1e28aeec238ef855c72efeb96a.1557162861.git.sathyanarayanan.kuppuswamy@linux.intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Kuppuswamy Sathyanarayanan If PF does not implement ATS and VF implements/uses it, it might lead to runtime issues. Also, as per spec r4.0, sec 9.3.7.8, PF should implement ATS if VF implements it. So add additional check to confirm given device aligns with the spec. Cc: Ashok Raj Cc: Keith Busch Suggested-by: Ashok Raj Reviewed-by: Keith Busch Signed-off-by: Kuppuswamy Sathyanarayanan --- drivers/pci/ats.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c index e7a904e347c3..718e6f414680 100644 --- a/drivers/pci/ats.c +++ b/drivers/pci/ats.c @@ -19,6 +19,7 @@ void pci_ats_init(struct pci_dev *dev) { int pos; + struct pci_dev *pdev; if (pci_ats_disabled()) return; @@ -27,6 +28,17 @@ void pci_ats_init(struct pci_dev *dev) if (!pos) return; + /* + * Per PCIe r4.0, sec 9.3.7.8, if VF implements Address Translation + * Services (ATS) Extended Capability then corresponding PF should + * also implement it. + */ + if (dev->is_virtfn) { + pdev = pci_physfn(dev); + if (!pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_ATS)) + return; + } + dev->ats_cap = pos; } From patchwork Mon May 6 17:20:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kuppuswamy Sathyanarayanan X-Patchwork-Id: 10931649 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 49A7C924 for ; Mon, 6 May 2019 17:22:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 28E3D28893 for ; Mon, 6 May 2019 17:22:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2574628896; Mon, 6 May 2019 17:22:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A908428898 for ; Mon, 6 May 2019 17:22:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726710AbfEFRWl (ORCPT ); Mon, 6 May 2019 13:22:41 -0400 Received: from mga07.intel.com ([134.134.136.100]:31139 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726407AbfEFRWR (ORCPT ); Mon, 6 May 2019 13:22:17 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 May 2019 10:22:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,438,1549958400"; d="scan'208";a="230014475" Received: from skuppusw-desk.jf.intel.com ([10.54.74.33]) by orsmga001.jf.intel.com with ESMTP; 06 May 2019 10:22:15 -0700 From: sathyanarayanan.kuppuswamy@linux.intel.com To: bhelgaas@google.com Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, ashok.raj@intel.com, keith.busch@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com Subject: [PATCH v2 4/5] PCI/ATS: Disable PF/VF ATS service independently Date: Mon, 6 May 2019 10:20:06 -0700 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Kuppuswamy Sathyanarayanan Currently all VF's needs to be disable their ATS service before disabling the ATS service in corresponding PF device. But this logic is incorrect and does not align with the spec. Also it might lead to some power and performance impact in the system. As per PCIe spec r4.0, sec 9.3.7.8, ATS Capabilities in VFs and their associated PFs may be enabled/disabled independently. So remove this dependency logic in enable/disable code. Cc: Ashok Raj Cc: Keith Busch Suggested-by: Ashok Raj Signed-off-by: Kuppuswamy Sathyanarayanan --- drivers/pci/ats.c | 11 ----------- include/linux/pci.h | 1 - 2 files changed, 12 deletions(-) diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c index 718e6f414680..977f5a1ace40 100644 --- a/drivers/pci/ats.c +++ b/drivers/pci/ats.c @@ -72,8 +72,6 @@ int pci_enable_ats(struct pci_dev *dev, int ps) pdev = pci_physfn(dev); if (pdev->ats_stu != ps) return -EINVAL; - - atomic_inc(&pdev->ats_ref_cnt); /* count enabled VFs */ } else { dev->ats_stu = ps; ctrl |= PCI_ATS_CTRL_STU(dev->ats_stu - PCI_ATS_MIN_STU); @@ -91,20 +89,11 @@ EXPORT_SYMBOL_GPL(pci_enable_ats); */ void pci_disable_ats(struct pci_dev *dev) { - struct pci_dev *pdev; u16 ctrl; if (WARN_ON(!dev->ats_enabled)) return; - if (atomic_read(&dev->ats_ref_cnt)) - return; /* VFs still enabled */ - - if (dev->is_virtfn) { - pdev = pci_physfn(dev); - atomic_dec(&pdev->ats_ref_cnt); - } - pci_read_config_word(dev, dev->ats_cap + PCI_ATS_CTRL, &ctrl); ctrl &= ~PCI_ATS_CTRL_ENABLE; pci_write_config_word(dev, dev->ats_cap + PCI_ATS_CTRL, ctrl); diff --git a/include/linux/pci.h b/include/linux/pci.h index 2a761ea63f8d..c38cbf8fa03b 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -446,7 +446,6 @@ struct pci_dev { }; u16 ats_cap; /* ATS Capability offset */ u8 ats_stu; /* ATS Smallest Translation Unit */ - atomic_t ats_ref_cnt; /* Number of VFs with ATS enabled */ #endif #ifdef CONFIG_PCI_PRI u32 pri_reqs_alloc; /* Number of PRI requests allocated */ From patchwork Mon May 6 17:20:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kuppuswamy Sathyanarayanan X-Patchwork-Id: 10931645 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E85B5912 for ; Mon, 6 May 2019 17:22:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C9DC028879 for ; Mon, 6 May 2019 17:22:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B090F287EF; Mon, 6 May 2019 17:22:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 591CF28895 for ; Mon, 6 May 2019 17:22:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726954AbfEFRW3 (ORCPT ); Mon, 6 May 2019 13:22:29 -0400 Received: from mga07.intel.com ([134.134.136.100]:31139 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726710AbfEFRWR (ORCPT ); Mon, 6 May 2019 13:22:17 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 May 2019 10:22:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,438,1549958400"; d="scan'208";a="230014478" Received: from skuppusw-desk.jf.intel.com ([10.54.74.33]) by orsmga001.jf.intel.com with ESMTP; 06 May 2019 10:22:15 -0700 From: sathyanarayanan.kuppuswamy@linux.intel.com To: bhelgaas@google.com Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, ashok.raj@intel.com, keith.busch@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com Subject: [PATCH v2 5/5] PCI: Skip Enhanced Allocation (EA) initalization for VF device Date: Mon, 6 May 2019 10:20:07 -0700 Message-Id: <9370d51d14d9cd0ad4101db14e808fa3ab7efcf5.1557162861.git.sathyanarayanan.kuppuswamy@linux.intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Kuppuswamy Sathyanarayanan As per PCIe r4.0, sec 9.3.6, VF must not implement Enhanced Allocation Capability. So skip pci_ea_init() for virtual devices. Cc: Ashok Raj Cc: Keith Busch Suggested-by: Ashok Raj Signed-off-by: Kuppuswamy Sathyanarayanan --- drivers/pci/pci.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 766f5779db92..0ba3930e3b54 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -2976,6 +2976,13 @@ void pci_ea_init(struct pci_dev *dev) int offset; int i; + /* + * Per PCIe r4.0, sec 9.3.6, VF must not implement Enhanced + * Allocation Capability. + */ + if (dev->is_virtfn) + return; + /* find PCI EA capability in list */ ea = pci_find_capability(dev, PCI_CAP_ID_EA); if (!ea)