From patchwork Fri May 12 02:50:32 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Cheng, Collins" X-Patchwork-Id: 9723473 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 05C8B600CB for ; Fri, 12 May 2017 02:50:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F093E28753 for ; Fri, 12 May 2017 02:50:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E53A728769; Fri, 12 May 2017 02:50:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 063EA28753 for ; Fri, 12 May 2017 02:50:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752996AbdELCui (ORCPT ); Thu, 11 May 2017 22:50:38 -0400 Received: from mail-sn1nam02on0054.outbound.protection.outlook.com ([104.47.36.54]:41036 "EHLO NAM02-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750991AbdELCuh (ORCPT ); Thu, 11 May 2017 22:50:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=3kevvv5tMSEUlfkO5w9JUNZNaVVrhfDNo4sgl2U4Oek=; b=y1uTBshdOvAhLaflXt5EJqno6TDOM81Y9s31Zsxn/t2TMWPNVF5UtcOJ0lzRUmSlo+eAdqCgLnDis5vsLiKmW549JG8EyMya3F15t+t8GcvWlvF4WGHOgihD3y0f4U2a6Njd/ROm5DcH6wK4IzrC2yH4VagvPRPKxJ9//GtPSUA= Received: from BN6PR12MB1249.namprd12.prod.outlook.com (10.168.227.135) by BN6PR12MB1651.namprd12.prod.outlook.com (10.172.19.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1084.16; Fri, 12 May 2017 02:50:34 +0000 Received: from BN6PR12MB1249.namprd12.prod.outlook.com ([10.168.227.135]) by BN6PR12MB1249.namprd12.prod.outlook.com ([10.168.227.135]) with mapi id 15.01.1084.020; Fri, 12 May 2017 02:50:33 +0000 From: "Cheng, Collins" To: Bjorn Helgaas , "linux-pci@vger.kernel.org" , "linux-kernel@vger.kernel.org" CC: "Deucher, Alexander" , "Zytaruk, Kelly" Subject: [PATCH] PCI: Make SR-IOV capable GPU working on the SR-IOV incapable platform Thread-Topic: [PATCH] PCI: Make SR-IOV capable GPU working on the SR-IOV incapable platform Thread-Index: AdLKyoNwOp/LDY/DRWyB/8lDz/Qk1g== Date: Fri, 12 May 2017 02:50:32 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: google.com; dkim=none (message not signed) header.d=none; google.com; dmarc=none action=none header.from=amd.com; x-originating-ip: [116.228.147.241] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; BN6PR12MB1651; 7:X+/r2juZmpD8+Ua212t7EJ5vzhmIbTNNYs6EMRkO7hCYcuTdT1ZpXB447GbA4TLUEWSwZCWssBbwxyIfxr9GK3zzqs1rSiJjWUgVpkR9Ntxr2VRbI3g0/hFOxEEIsnkqYETyihDE8Q5FNIHS1d0u62N/To6NX8TwVaEl4pRo4WXkJZsY783INSUNSSI1R4pR53D8PR0JtigrV863hIM+5iSaqTPhur3kzpHcicFozHnkdj3VBr81rQJ/Z5Nn5onJugaVytUs6yeWFeoK0x8rMJDOg32JnHyPleNjNIdlYne+gne2SKkKJqHspROZReD1AQcSlPbVcWkGcSJ2v9QDAw==; 20:QInz4/4guSraC7DGPkVZEYpbWpgzDNtMKf06+Kyng3+Pj0pg179WLSHAk9NLuPpHRPbaie64S64QoCcIqkx38THGJkw20YK0jnmlRFUH/FWnXTKc+1s+qsHzMpSJcFvVL2SHmfo/581t5vU/hJQen44AgjOluVH8J4svw/xuHu0OnosHx1lQysvAZv1rYYgZZaXETd4UM0sIXoxT5LiNcfmGPvSwOg2KMfkZPr3h4ebifIfUpgSqZefoeMWFkcX3 x-forefront-antispam-report: SFV:SKI; SCL:-1SFV:NSPM; SFS:(10009020)(6009001)(39850400002)(39410400002)(39450400003)(39860400002)(39400400002)(39840400002)(6506006)(72206003)(478600001)(50986999)(54906002)(99286003)(9686003)(6436002)(189998001)(33656002)(81166006)(8676002)(77096006)(8936002)(122556002)(55016002)(66066001)(7696004)(2900100001)(3660700001)(86362001)(53936002)(7736002)(2906002)(3280700002)(2201001)(38730400002)(3846002)(102836003)(6116002)(5660300001)(25786009)(74316002)(4326008)(305945005)(2501003)(54356999); DIR:OUT; SFP:1101; SCL:1; SRVR:BN6PR12MB1651; H:BN6PR12MB1249.namprd12.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; x-ms-office365-filtering-correlation-id: 3232ab2c-d150-403b-c653-08d498e1a8af x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254075)(48565401081)(201703131423075)(201703031133081); SRVR:BN6PR12MB1651; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(767451399110); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(5005006)(8121501046)(93006095)(93001095)(10201501046)(3002001)(6055026)(6041248)(20161123562025)(20161123564025)(20161123560025)(20161123555025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148); SRVR:BN6PR12MB1651; BCL:0; PCL:0; RULEID:; SRVR:BN6PR12MB1651; x-forefront-prvs: 0305463112 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-originalarrivaltime: 12 May 2017 02:50:32.9586 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1651 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hi Helgaas, Some AMD GPUs have hardware support for graphics SR-IOV. If the SR-IOV capable GPU is plugged into the SR-IOV incapable platform. It would cause a problem on PCI resource allocation in current Linux kernel. Therefore in order to allow the PF (Physical Function) device of SR-IOV capable GPU to work on the SR-IOV incapable platform, it is required to verify conditions for initializing BAR resources on AMD SR-IOV capable GPUs. If the device is an AMD graphics device and it supports SR-IOV it will require a large amount of resources. Before calling sriov_init() must ensure that the system BIOS also supports SR-IOV and that system BIOS has been able to allocate enough resources. If the VF BARs are zero then the system BIOS does not support SR-IOV or it could not allocate the resources and this platform will not support AMD graphics SR-IOV. Therefore do not call sriov_init(). If the system BIOS does support SR-IOV then the VF BARs will be properly initialized to non-zero values. Below is the patch against to Kernel 4.8 & 4.9. Please review. I checked the drivers/pci/quirks.c, it looks the workarounds/fixes in quirks.c are for specific devices and one or more device ID are defined for the specific devices. However my patch is for all AMD SR-IOV capable GPUs, that includes all existing and future AMD server GPUs. So it doesn't seem like a good fit to put the fix in quirks.c. Signed-off-by: Collins Cheng --- drivers/pci/iov.c | 63 ++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 60 insertions(+), 3 deletions(-) diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c index e30f05c..e4f1405 100644 --- a/drivers/pci/iov.c +++ b/drivers/pci/iov.c @@ -523,6 +523,45 @@ static void sriov_restore_state(struct pci_dev *dev) msleep(100); } +/* + * pci_vf_bar_valid - check if VF BARs have resource allocated + * @dev: the PCI device + * @pos: register offset of SR-IOV capability in PCI config space + * Returns true any VF BAR has resource allocated, false + * if all VF BARs are empty. + */ +static bool pci_vf_bar_valid(struct pci_dev *dev, int pos) +{ + int i; + u32 bar_value; + u32 bar_size_mask = ~(PCI_BASE_ADDRESS_SPACE | + PCI_BASE_ADDRESS_MEM_TYPE_64 | + PCI_BASE_ADDRESS_MEM_PREFETCH); + + for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) { + pci_read_config_dword(dev, pos + PCI_SRIOV_BAR + i * 4, &bar_value); + if (bar_value & bar_size_mask) + return true; + } + + return false; +} + +/* + * is_amd_display_adapter - check if it is an AMD/ATI GPU device + * @dev: the PCI device + * + * Returns true if device is an AMD/ATI display adapter, + * otherwise return false. + */ + +static bool is_amd_display_adapter(struct pci_dev *dev) +{ + return (((dev->class >> 16) == PCI_BASE_CLASS_DISPLAY) && + (dev->vendor == PCI_VENDOR_ID_ATI || + dev->vendor == PCI_VENDOR_ID_AMD)); +} + /** * pci_iov_init - initialize the IOV capability * @dev: the PCI device @@ -537,9 +576,27 @@ int pci_iov_init(struct pci_dev *dev) return -ENODEV; pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_SRIOV); - if (pos) - return sriov_init(dev, pos); - + if (pos) { + /* + * If the device is an AMD graphics device and it supports + * SR-IOV it will require a large amount of resources. + * Before calling sriov_init() must ensure that the system + * BIOS also supports SR-IOV and that system BIOS has been + * able to allocate enough resources. + * If the VF BARs are zero then the system BIOS does not + * support SR-IOV or it could not allocate the resources + * and this platform will not support AMD graphics SR-IOV. + * Therefore do not call sriov_init(). + * If the system BIOS does support SR-IOV then the VF BARs + * will be properly initialized to non-zero values. + */ + if (is_amd_display_adapter(dev)) { + if (pci_vf_bar_valid(dev, pos)) + return sriov_init(dev, pos); + } else { + return sriov_init(dev, pos); + } + } return -ENODEV; }