From patchwork Mon Sep 21 13:47:12 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jay Cornwall X-Patchwork-Id: 7234151 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 10E60BEEC1 for ; Mon, 21 Sep 2015 21:23:11 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id E86EF20726 for ; Mon, 21 Sep 2015 21:23:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8399C206F4 for ; Mon, 21 Sep 2015 21:23:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757037AbbIUVXH (ORCPT ); Mon, 21 Sep 2015 17:23:07 -0400 Received: from mail-bn1bon0091.outbound.protection.outlook.com ([157.56.111.91]:24544 "EHLO na01-bn1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1757028AbbIUVXF (ORCPT ); Mon, 21 Sep 2015 17:23:05 -0400 Received: from CY1PR1201CA0004.namprd12.prod.outlook.com (10.169.17.142) by BLUPR12MB0417.namprd12.prod.outlook.com (10.162.91.30) with Microsoft SMTP Server (TLS) id 15.1.274.16; Mon, 21 Sep 2015 20:49:19 +0000 Received: from DM3NAM03FT003.eop-NAM03.prod.protection.outlook.com (2a01:111:f400:7e49::201) by CY1PR1201CA0004.outlook.office365.com (2a01:111:e400:5b9a::14) with Microsoft SMTP Server (TLS) id 15.1.274.16 via Frontend Transport; Mon, 21 Sep 2015 20:49:19 +0000 Authentication-Results: spf=fail (sender IP is 165.204.84.221) smtp.mailfrom=jcornwall.me; jcornwall.me; dkim=none (message not signed) header.d=none; jcornwall.me; dmarc=none action=none header.from=jcornwall.me; Received-SPF: Fail (protection.outlook.com: domain of jcornwall.me does not designate 165.204.84.221 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.221; helo=atltwp01.amd.com; Received: from atltwp01.amd.com (165.204.84.221) by DM3NAM03FT003.mail.protection.outlook.com (10.152.82.87) with Microsoft SMTP Server id 15.1.274.4 via Frontend Transport; Mon, 21 Sep 2015 20:49:19 +0000 X-WSS-ID: 0NV1OI6-07-14F-02 X-M-MSG: Received: from satlvexedge01.amd.com (satlvexedge01.amd.com [10.177.96.28]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by atltwp01.amd.com (Axway MailGate 5.3.1) with ESMTPS id 2E92DCAE7BC; Mon, 21 Sep 2015 16:49:17 -0400 (EDT) Received: from SATLEXDAG06.amd.com (10.181.40.13) by satlvexedge01.amd.com (10.177.96.28) with Microsoft SMTP Server (TLS) id 14.3.195.1; Mon, 21 Sep 2015 15:49:27 -0500 Received: from jcornwal-tn.amd.com (10.180.168.240) by satlexdag06.amd.com (10.181.40.13) with Microsoft SMTP Server id 14.3.195.1; Mon, 21 Sep 2015 16:49:17 -0400 From: Jay Cornwall To: CC: Jay Cornwall Subject: [PATCH v2] PCI: Add pci_enable_atomic_request Date: Mon, 21 Sep 2015 08:47:12 -0500 Message-ID: <1442843232-12936-1-git-send-email-jay@jcornwall.me> X-Mailer: git-send-email 1.9.1 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-Matching-Connectors: 130873421595285798; (3b476b41-6bd3-4b00-1a0b-08d2adb5efdc); () X-Forefront-Antispam-Report: CIP:165.204.84.221; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(2980300002)(1109001)(1110001)(339900001)(189002)(199003)(85426001)(92566002)(46102003)(189998001)(86362001)(5001860100001)(64706001)(77096005)(2351001)(105606002)(74482002)(48376002)(97736004)(50226001)(110136002)(4001540100001)(5001830100001)(36756003)(68736005)(19580395003)(50986999)(5007970100001)(106466001)(229853001)(5003940100001)(11100500001)(19580405001)(47776003)(50466002)(33646002)(62966003)(87936001)(77156002)(53416004); DIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR12MB0417; H:atltwp01.amd.com; FPR:; SPF:Fail; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BLUPR12MB0417; 2:parn3Ier15RGdCrtH0OW3iFY5gSSxTsgeLhh6Fa/zKuPq+LrTQRouBfr7rgWVztKWRHct8gh+dpJNVpyDXLi5NzkNdyDs65JZGW/oLilcg+1sCIe+z5/Vc1sr6fobNWIKA5z9fxWwK8bG+BrRch3x4o+AVkrJZkshocDOosghIA=; 3:lQp9ZqIrOhzfG+bfZTMacM4kQ1at/FOr0a1EONG6DPUgVvTWobxi0WgROeXDTb5K3QdWYx+s8Z4gpRWU07GLg03vrLt9YTI8zxYiIe41bWX9kMlyYy1B7g2FXo42ilcF9SgNKDJxUXldscPhRyfwQyIKfHoUaH0rGjoUnTxadpygqeC/2HqRP8kueFuSXbFcWCreQIu66dy+G9HXBa9MCkaA9BffbQKwg1kKq2IeDk0pLHm6DKgjyEec4xcWsOhM; 25:P3mQB4MpCPcam6wBPRz6aIQlNuY9qIwMhm/aQLFhB7zIDsVC9lBKCE4rXK+K+fDptO1GRTCMW6/1ASlmL7xYxn5VZpl5iRDa376NA341yGcHp7QAVu9y/ZuOVgmsaKCFIYkEcst9R4ZCV9oge+6Z81PNzRwuh5bMWY6ZgHauOZlhyZTytYWNcc2Ka+ROxSYOKcBmNkd6umCzurf+7VKpKdYJcGe4QedgyDlqCGjIBe0VTZeMNsZWBzSm78f3KalMLyDJWsrHw5E1iV/93cuegg== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BLUPR12MB0417; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(8121501046)(520078)(5005006)(3002001); SRVR:BLUPR12MB0417; BCL:0; PCL:0; RULEID:(400006); SRVR:BLUPR12MB0417; X-Microsoft-Exchange-Diagnostics: 1; BLUPR12MB0417; 4:eEtvi+cLpwMswa1YP0Fo+XPHgCJJp+wL7s419koJUj/njLWe+fXkuhSrCbchOvG7x+rQnqOIyjNUQGwVs4JNUQVq9hYy5ZQQhzwLi1fbu5+0sEsYNpobi+WVuyztE4cZy8Jzj5GmA4Jnd3qDOf2WgkM/kiLb523l53b7mBniDJnS0O4yodtl6ff/uLkxokPIocxsaJiK5DM8hGwnlD/9kn/BzHaD0EmnNRBy3h/RscpHpVvaBaa9QEK1SsLuBaCeZe4ywshTjPyttau1QWRbCpYDUIotLfhMr/FRMh0vOY/76pHXV2fi17opKrys62DuCqlEzkoTWqfYSJv2rVVZ3Q== X-Forefront-PRVS: 07063A0A30 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BLUPR12MB0417; 23:U0o2tjZV+blr6Vl1XjgYvm2dWAyvNgvH+sIdluVyW?= =?us-ascii?Q?D62xdOCJVUHH2DyP8lZXtBipnzsDla331BYolXm8IYJM+wVW8VPxc7d1TcrM?= =?us-ascii?Q?ZfGhV3dbDPLvuZsNIU63Ba55nUgHpn6qWkjvs4Wood0yJNU3JYvV8Wnxwepi?= =?us-ascii?Q?xbsM/QhPBWYEpKYOaVImKwZr47CF17sVAVDY2aTXV3qAvwDZB2/C9iVpjm8m?= =?us-ascii?Q?Dq/pIlIiVIUonp9xcBkkVO6PMBFnc8JzBV5Cn11FNnyUSaBgU8vMsB3brCr9?= =?us-ascii?Q?kUSCtEGGP1GNuET/aZWeEiZGYRaQzCMiAlo8brlk56eTGeofOk6YIPD6j92E?= =?us-ascii?Q?ukB7OcuJTT4h0E3GTpOfEsn9i7yvfWsUaO3dyLuWpmoXcKSqI3w5bCAJ9cn5?= =?us-ascii?Q?L3d5hGFXUntk6XBTClScnZhFO1IVJwDxqPwsPIw/8I1hXoawQc3bU2vwX/LB?= =?us-ascii?Q?hAvBf8pHl28NBbEQo5USEsypwtxVD8UYMt8S6Ear5FqKhovdoytObAzdFtvd?= =?us-ascii?Q?dIQC2RFS1bs9v5ghO0P+Dc73URjPFqYubpkMs2S+qdM9L+R2yEl2eItFP4RM?= =?us-ascii?Q?DyYAqcSrMJxnpUE9EuNvwSuVxJh3Gy4OE+6IxmnCBzDhsZbuVlSUqCwIDbXD?= =?us-ascii?Q?kXJ1pX9vlukeS+dEJZ9zaQa821VBk6PFNQukzDyKRDach+MEIVy8g7GdexOg?= =?us-ascii?Q?4koFNK4BS1BZNl9iV1BDOjNWxORNGctz3y3eaHFsq2HvmGriRjTBhHymErcO?= =?us-ascii?Q?yxysLnJI3iB3OrezaoyOVaO4O//Db+gsnfq5uPNvZtndNDyd6sB2yurT6cn2?= =?us-ascii?Q?Y/C4C4lcohAY9g4a2xsQVPolIP5fqvUZClOSBWhVNLrheYR6yz2youF6PYMb?= =?us-ascii?Q?Xok/BzfUuKN0PHH1Sho/hqCt7yb+aaQWyfkeByN+E53nLb0GTOfV27IzjqQm?= =?us-ascii?Q?QI8gILO1tf/uqBQs8ErohpDfKLvO7d/ADTEw8kxLQV8OD1mFXYuf9yxQduIV?= =?us-ascii?Q?UzcNkEvyc5YAVRMiY4EhWxnhQWC5evjcvy3Vx/UWpHjmpti8itpZcFhRnsvV?= =?us-ascii?Q?RyQzkFfu+oAZsO8FCDNjyKf7kYxBWTAIh+sNVHpJaenvE5jF52GqF2IqaJYn?= =?us-ascii?Q?x48+8rj/2E=3D?= X-Microsoft-Exchange-Diagnostics: 1; BLUPR12MB0417; 5:D7/563weBsC2UlZ3kj4PZIQDW91KX9QAn/OaWpjmhZFigzP5HtaH4qLqPfnZ54Ui7h3tD/xMDJG/Fv94uulvm/5Kb7axwWxgyGqE4ZG+x/8a2udI/kjVHeoer7OOv9ecr26vJLEwLSWO2a3YvPxiWZcu0FyTFnzy+/R/QY6DlmQ=; 24:FbTN6LgivkRy2o74ZMJU/cJPcT8K12tumpoFWbQg/xuFNBFUbqANPIL0UhtaqUlHk4/kt3phRTceFiJ/rc9KGDwAJh8P6PnCqzT9N4tyIGs= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2015 20:49:19.2316 (UTC) X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[165.204.84.221]; Helo=[atltwp01.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR12MB0417 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-5.4 required=5.0 tests=BAYES_00, DATE_IN_PAST_06_12, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The PCIe 3.0 AtomicOp (6.15) feature allows atomic transctions to be requested by, routed through and completed by PCIe components. Routing and completion do not require software support. Component support for each is detectable via the DEVCAP2 register. AtomicOp requests are permitted only if a component's DEVCTL2.ATOMICOP_REQUESTER_ENABLE field is set. This capability cannot be detected but is a no-op if set on a component with no support. These requests can only be serviced if the upstream components support AtomicOp completion and/or routing to a component which does. A concrete example is the AMD Fiji-class GPU, which is specified to support AtomicOp requests, routed through a PLX 8747 switch (advertising AtomicOp routing) to a Haswell host bridge (advertising AtomicOp completion support). When AtomicOp requests are disabled the GPU logs attempts to initiate requests to an MMIO register for debugging. Add pci_enable_atomic_request for per-device control over AtomicOp requests. Upstream bridges are checked for AtomicOp routing capability and the call fails if any lack this capability. The root port is checked for AtomicOp completion capabilities and the call fails if it does not support any. Routes to other PCIe components are not checked for AtomicOp routing and completion capabilities. Signed-off-by: Jay Cornwall --- drivers/pci/pci.c | 67 +++++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 1 + include/uapi/linux/pci_regs.h | 5 ++++ 3 files changed, 73 insertions(+) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 6a9a111..58819e4 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -2453,6 +2453,73 @@ bool pci_acs_path_enabled(struct pci_dev *start, } /** + * pci_enable_atomic_request - enable or disable AtomicOp requester + * @dev: the PCI device + * + * Return true if the device is capable of generating AtomicOp requests, + * all upstream bridges support AtomicOp routing, and the root port supports + * 32-bit, 64-bit and/or 128-bit AtomicOp completion. + */ +bool pci_enable_atomic_request(struct pci_dev *dev) +{ + struct pci_bus *bus = dev->bus; + + if (!pci_is_pcie(dev)) + return false; + + switch (pci_pcie_type(dev)) { + /* PCIe 3.0, 6.15 specifies that endpoints and root ports are permitted + * to implement AtomicOp requester capabilities. + */ + case PCI_EXP_TYPE_ENDPOINT: + case PCI_EXP_TYPE_LEG_END: + case PCI_EXP_TYPE_RC_END: + case PCI_EXP_TYPE_ROOT_PORT: + break; + default: + return false; + } + + while (bus->parent) { + struct pci_dev *bridge = bus->self; + u32 cap; + + pcie_capability_read_dword(bridge, PCI_EXP_DEVCAP2, &cap); + + switch (pci_pcie_type(bridge)) { + /* PCIe 3.0, 6.15 specifies that upstream, downstream and + * root ports may implement AtomicOp routing capabilities. + * AtomicOp routing via a root port is not considered here. + */ + case PCI_EXP_TYPE_UPSTREAM: + case PCI_EXP_TYPE_DOWNSTREAM: + if (!(cap & PCI_EXP_DEVCAP2_ATOMIC_ROUTING)) + return false; + break; + + /* PCIe 3.0, 6.15 specifies that root ports are permitted to + * implement AtomicOp completion capabilities. + */ + case PCI_EXP_TYPE_ROOT_PORT: + if (!(cap & (PCI_EXP_DEVCAP2_ATOMIC_COMP32 | + PCI_EXP_DEVCAP2_ATOMIC_COMP64 | + PCI_EXP_DEVCAP2_ATOMIC_COMP128))) + return false; + break; + } + + + bus = bus->parent; + } + + pcie_capability_set_word(dev, PCI_EXP_DEVCTL2, + PCI_EXP_DEVCTL2_ATOMICOP_REQ); + + return true; +} +EXPORT_SYMBOL(pci_enable_atomic_request); + +/** * pci_swizzle_interrupt_pin - swizzle INTx for device behind bridge * @dev: the PCI device * @pin: the INTx pin (1=INTA, 2=INTB, 3=INTC, 4=INTD) diff --git a/include/linux/pci.h b/include/linux/pci.h index e90eb22..755f061 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1801,6 +1801,7 @@ void pci_request_acs(void); bool pci_acs_enabled(struct pci_dev *pdev, u16 acs_flags); bool pci_acs_path_enabled(struct pci_dev *start, struct pci_dev *end, u16 acs_flags); +bool pci_enable_atomic_request(struct pci_dev *dev); #define PCI_VPD_LRDT 0x80 /* Large Resource Data Type */ #define PCI_VPD_LRDT_ID(x) ((x) | PCI_VPD_LRDT) diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h index 413417f..013c2bd 100644 --- a/include/uapi/linux/pci_regs.h +++ b/include/uapi/linux/pci_regs.h @@ -571,6 +571,10 @@ */ #define PCI_EXP_DEVCAP2 36 /* Device Capabilities 2 */ #define PCI_EXP_DEVCAP2_ARI 0x00000020 /* Alternative Routing-ID */ +#define PCI_EXP_DEVCAP2_ATOMIC_ROUTING 0x00000040 /* AtomicOp routing */ +#define PCI_EXP_DEVCAP2_ATOMIC_COMP32 0x00000080 /* 32b AtomicOp completion */ +#define PCI_EXP_DEVCAP2_ATOMIC_COMP64 0x00000100 /* 64b AtomicOp completion */ +#define PCI_EXP_DEVCAP2_ATOMIC_COMP128 0x00000200 /* 128b AtomicOp completion*/ #define PCI_EXP_DEVCAP2_LTR 0x00000800 /* Latency tolerance reporting */ #define PCI_EXP_DEVCAP2_OBFF_MASK 0x000c0000 /* OBFF support mechanism */ #define PCI_EXP_DEVCAP2_OBFF_MSG 0x00040000 /* New message signaling */ @@ -578,6 +582,7 @@ #define PCI_EXP_DEVCTL2 40 /* Device Control 2 */ #define PCI_EXP_DEVCTL2_COMP_TIMEOUT 0x000f /* Completion Timeout Value */ #define PCI_EXP_DEVCTL2_ARI 0x0020 /* Alternative Routing-ID */ +#define PCI_EXP_DEVCTL2_ATOMICOP_REQ 0x0040 /* Allow AtomicOp requests */ #define PCI_EXP_DEVCTL2_IDO_REQ_EN 0x0100 /* Allow IDO for requests */ #define PCI_EXP_DEVCTL2_IDO_CMP_EN 0x0200 /* Allow IDO for completions */ #define PCI_EXP_DEVCTL2_LTR_EN 0x0400 /* Enable LTR mechanism */