From patchwork Thu Apr 13 20:30:45 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jayachandran C X-Patchwork-Id: 9679987 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 51D8F60326 for ; Thu, 13 Apr 2017 20:31:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 454DB286C2 for ; Thu, 13 Apr 2017 20:31:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 38D9A286C4; Thu, 13 Apr 2017 20:31:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAD_ENC_HEADER,BAYES_00, DKIM_SIGNED,DKIM_VALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CE8CD286C2 for ; Thu, 13 Apr 2017 20:31:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=IIpShUPw2nlAKoM65EX/BtrUR1uDwSsrZLOHUX5Oz9U=; b=C/r7fJhAEO+Otm KXSsIje9NBb7kLwyUv8Xk8310hID4RBDeq5sl3cCIDsl8aYeHI59stIr+W9w2kNa4zEIBfpsjU4mK 0ZcHLWj9TUg2Qeo9O+6vyUp0+r/UUgJ7JkIobfeyUh0YvPa9JZWqRfKnC+HgK8bwglHdoYhiCm6lg SNLMbQl78HtXbIYW0d9ZnV2NKhuxmpiKWfXHAe0XoGqc/eyNGMIGb7+OicZa/DEuwJepnVJ6ohYwO NhNHhdfxiySQEkfcztiT/DNMBmucb9oE9pGZfPRHNY1cmr1u3Wk8bmX+fWXFDzolceARXmg71C1ec dtl0uVOTsritnVxLOdhg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1cylPD-0001p6-CZ; Thu, 13 Apr 2017 20:31:51 +0000 Received: from mail-cys01nam02on0087.outbound.protection.outlook.com ([104.47.37.87] helo=NAM02-CY1-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1cylOr-0001K3-BE for linux-arm-kernel@lists.infradead.org; Thu, 13 Apr 2017 20:31:33 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=3SOOGthQSJwQ78t2zbnt5aAKaGY/SgeVYoic6QWnRUQ=; b=CF5Kei5LSHbdbZm+2C8IQN148wsjfmLIiWJCdqdJtLQ2XCPlgzYru5ICiaiSB2miYAXdK8WZ+gXiGDPTWPuhj6m7DrjaRLKfExDZ3AfViPJ6Bj5Woo2/3ofetsDMNMWqXL+2WaaMbFEthDxnwyKY9ad4ff95zQCUxeOXy3UANEU= Authentication-Results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=caviumnetworks.com; Received: from vulcan.caveonetworks.com (50.233.148.156) by MWHPR07MB3008.namprd07.prod.outlook.com (10.172.96.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1019.17; Thu, 13 Apr 2017 20:31:03 +0000 From: Jayachandran C To: Bjorn Helgaas , linux-pci@vger.kernel.org Subject: [PATCH v5 2/2] PCI: quirks: Fix ThunderX2 dma alias handling Date: Thu, 13 Apr 2017 20:30:45 +0000 Message-Id: <1492115445-4069-3-git-send-email-jnair@caviumnetworks.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1492115445-4069-1-git-send-email-jnair@caviumnetworks.com> References: <1492115445-4069-1-git-send-email-jnair@caviumnetworks.com> MIME-Version: 1.0 X-Originating-IP: [50.233.148.156] X-ClientProxiedBy: CY4PR06CA0034.namprd06.prod.outlook.com (10.173.44.20) To MWHPR07MB3008.namprd07.prod.outlook.com (10.172.96.14) X-MS-Office365-Filtering-Correlation-Id: 2bbd30a8-50c8-4443-b9b2-08d482ac01c0 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201703131423075)(201703031133081); SRVR:MWHPR07MB3008; X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3008; 3:e9BMQDFjW08GyTROgCVUKrdIos/eldqojgUxp6Jj6gYZmvXTlaD7t930sBjHqod5iqUTPRGmygIOY9nSOXy2aKJ+hZYW46DSHa2sokdUWboxXkQo0DXq74/gjMDLE8scyCHpip0DfNPv16Y42FTHZVDfuGtRQx10qdSSV6bGNpToS69HysSN1qBxWS91iGHNzNEHKAhI4aYqR9PW7fM2LDaMgagDze5lm1eWFZw1yLn1om6wOBvikV705TWwH6JyZ4oUi1Nm+lCXXzPi/9wRPqRLd/pIlIX3N5tLjWLokyr/iipVB1NbD+QBXP2VGwX/kv3m0BIl3xgYH77AGF8U1Q==; 25:0po+3avxhFt8Pdri9zJqiuLfKzBJok7FfzQyH30MCgalOEvBGcQelyhbogk3hh+dxOMoq7pkkYzFaJyBA3YatfmeWHw6XeUzcKxYoVx5oOzOqH41CTYAKO0vrAUcmeLvgb3mW2rytMsGXB3pjQfGHnZP3FchRySLaGOwmN9DZipcqf9uMzWazIdVPWTmmc8/JCkFJQvWkrWrDLu6ragdYY7RovZ7Wb0wXDSUa6Lw2aIDhj44vp7SHqZc2zRuwUfkDnhiQFg5RP2NJbJNmUdGp1LtEprDL7WIXjLCTGnlHV7cwhIdtpPfp6wV9azZdHDnooLQ077yWXiQTKO6EY6SGanp6poKK4qFsTejFY37xnPpYdAs+HIykz9Kl86n2PiLi1LE6MAG8j+nZa7RL4FYo0epzq+hWXA/SChmrigBilenc9vzKF0IlXwdSdYCrP6h/vmOzhsOHREYJ92ndOrCrQ== X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3008; 31:ojb5RiB45OV4K9X2WJNRRcCxihCvUE2wmK69hZ/FLodxpNqvG3n55eXQOfpaEdWMS3pTKnae6J8Z/qQ/nlmXMsj0CR4VnOk05ytIq+sfNJ5wk7FeaPtp35Jqb6H1Ye4JVW3v7n8J0flR3sp+EXr6j08u9cTv61AzSKPispcN0WS9L8FcPdads2S1YM4XKzzPTLQjn8hMELhVqmtD5SfSsrP3ArskslLgp7582AIecew=; 20:Oyz+aiQTSr1TYFhC11tnXBMGpfrSUIqv+GVpL/EpIJoS5xxt7R/9BvIjczUaBE9AXiM60MLmzGCGsAUXjwWXOKM2EoJ+7+/i6D7tzTdRx7UcXcXBkyd0b+I26513CeEOorn9oQgYVdk/RbLLGjorCYwjB4uUFjt3KVuj+TPzngtjnOEKQWJfoxmqVkrOyl/JqSjA80dWD1q7f/b+0kTmcImGtunpYsDuEoGy8WNo4k54872zHk41McXEPu9Ozp8GRwwPTjluAqgkGP5Qe8XjqIj0tQC2OvRkicY5B/rxRGQSaerNm9zg/zwWL1pGwXousX+Nmk/Kppse1l+EaM12WsYDOOLhKv29TpohDU6/SEq5RzpDmjzTV68QZlzVqkKtpOl4SXnC1py2HgW2VCOel63QoyLU4yTfU4bT53+Ul3QXEM4XmJ/3rZ4VwnpoS3M1t4ZZlbPJdwbhgmpQD9w+FCwkf3yPfQSiRbj4+TiH9YFYNN7PYH15c7gFK5Mzdy4l5dzdS1IgPI3yauBfoYGwNZ4EQTaG/wJ2crSRMcIXXXIzkXh+qcsmkbk3UGd0bXwOhNjZAbapxGAseNbUguBMcuFIfLuIY1Rq6yXFS0gqB3I= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(180628864354917); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(93006095)(6041248)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(20161123560025)(20161123555025)(20161123564025)(6072148); SRVR:MWHPR07MB3008; BCL:0; PCL:0; RULEID:; SRVR:MWHPR07MB3008; X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3008; 4:tt/akqLLfiX2LvNl7jUx9+q4Oky2JfmfVpBn+2eLa/J0fggWNwShMuhfMk69JIj+4t/Lfhgh2ZeJPVnWEPSjVQ2ZzD+koNL28E8gt/afuFfxLlDC7SiNYLYCXOwJObTcppvMlEDvc2Vj0J8kP7DSazKpTrVsJtmAdQbyLgzWHIpWsUOzEJUgaz2FoBbs2cKXuzjcOrUz9DIpC8qDx2jJefFAzcVqkprH86yLRFHlnZMK3PeUpSMCnsHYL0DfKrhUTh/LMeKZnhX3nHJQNDPn50R45eycv+NYgDqB6R8LZDzlM2oQSmvxdMTPNjo1L7M1kRnm8qFpQWWOElura/p3ppgGg7SX46UyStwW3vTQzOaYIm/XCaLp0nZiL4iCmZJENHyl7/3NTdwBV2CFP7QzXx87V5ymllhg0J/jIh8QwjR+I0FO59QhP44yw9jFUi0gMV5kbfuxaOrzVDadPp49i4Nnzk6ZCbTd6jTa6PX8zozm98tqswgw86Z09bctnXnuP5ctbDLNKDbPPEYrvVYJ9ANvzPaAq4vKb3/Q56IFmXUE4pYlaewqGiSXpwHdpskqdryyH3G8Pt/LiFD1HER2N90QCN5NQMxH3L0mt0zFaqZq0r/mvA63kUeviOLi85uaoWAyv6aqHqKxA8v0pbjUdoxejTfZgzH4/3q0STLLQbMh8LF2Ay5dwFGAUPe/k1Q0xyp7CdbalCMGToQS5BG9LiUfIG/0s072nSUsjqptainVdR3Lvzm13RpIMXgh2Knp X-Forefront-PRVS: 02760F0D1C X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(39400400002)(39840400002)(39410400002)(39850400002)(39450400003)(4326008)(50226002)(36756003)(2950100002)(38730400002)(110136004)(25786009)(53416004)(42882006)(47776003)(42186005)(6512007)(50986999)(76176999)(54906002)(5003940100001)(6506006)(5660300001)(6486002)(6666003)(53936002)(7736002)(33646002)(305945005)(48376002)(81166006)(8676002)(189998001)(66066001)(2906002)(3846002)(6116002); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR07MB3008; H:vulcan.caveonetworks.com; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR07MB3008; 23:w/nBOCYo/EO/SEHk0OGVm6EcZ3NLrVOjhFcSh6Qu8?= =?us-ascii?Q?1ptqsISerWdvQZZAgitvzjrggoUYjoodJG5CvH85VXDyoN5QK3jL3HkzG5eQ?= =?us-ascii?Q?iUdPjzd8WbYNpfNWIx6vKgpHrbUb9GCZBQ/OQbgq7GRI0f7u8d2sJp/UuYO8?= =?us-ascii?Q?GxHgIa4djy/K3aDfYcxQi3Edm03iIU1uwJMd/CvV1kHXDmfvftpcHQU+0PtG?= =?us-ascii?Q?7GzrmQrefN2chpwMHIB4Lu5hcl0G0AbZvzXlW7mVhL5obfHiSiYSNdMcx8s0?= =?us-ascii?Q?EK49Xxa8rf7xnxthlQHu6QQNYz/KXrZcAaGkcyvIL6R6dd3skFVphz8Tiub7?= =?us-ascii?Q?X+iUt0a7C6/a4Ydg0RvFzjh9c1H0yiS5q4bs9CoNlfu0neI72/X6UDKYNpSh?= =?us-ascii?Q?6YmEhTb219u/47Rdc7+HeuQtijxwSIysf900uykj5kuJoZPbyBLSOdGLuVuH?= =?us-ascii?Q?+54vuptdZRu666yOkj3Oqot29XrwJQ4UxP79nyEs6nmFfA/5lubSHcYq44TR?= =?us-ascii?Q?Bk1gB3iQ8O15Y3N7uTx7pPhcUWXGVWf5BxDNIZT7CD2CK3McWwLruemqDDL5?= =?us-ascii?Q?PveBK9CS9wEA8eNtNgTCLT4Po17scSPu+pDPxVleR0DUDKf0IF/iQT5DG1Wr?= =?us-ascii?Q?Xd+3KJOuc90YxuI1aushwXIxjchOiyPb3R+Xw4cyTEsbR38aa1ErEbRc33Jf?= =?us-ascii?Q?vXA6+4cndqORjSJisKoRRYn0fNS2t1H5mNcQHDtW+bTbpF7PZPkPxuhYmOEA?= =?us-ascii?Q?FXaNcLt8MZj5YsgiVdt1npRSSpmixFcMVVMeuqR8TaXrSZq1tqck9cBenDGP?= =?us-ascii?Q?tA77VtEg9V31ePCV7axRSw83hQ4QR9e/TOBmuFai9jBwnIdWF8J7ADQ4CXGO?= =?us-ascii?Q?WJUk/2VSZqPDjnwDdV1oKJQLLxCeSXANOadulfbPKHHDhysaTKwBiFNT2yU7?= =?us-ascii?Q?EZpusqppdT7sBBapZ9r9jU7dBCQ2inmZzUQIIoFaby8lGQUvYQgfv7WihgTY?= =?us-ascii?Q?i1FcYbCeacnctF6OeEB2Ig2?= X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3008; 6:LOEs+OKMsVP1yG7Gv1mvOLt7ZcNWbHy+x8kddbDYlO9EWAAAMhYKw3f7oHTptilMXwG7pYiWhrrPMXQ7/K6DYGAKiROPxB0VgDkH0RnGElTlJlSXPbJtZTIylRX1cIJA2wnkBPo5ZWWyCMp7jAok9D4our04QXr7zMssA85ulzDYwZIMzP4tpVSF8mSkg3dRps3kuGrRKpRM/CkFQIdLXRc0QgtRSueEJb1z9IBlb/lLSg6jYXoOH16X4664VSlk2KBpyN5lg3CWuphfCVochS25z6awbegq4WlSeoStdKWa4Gzpt9hZZCiICSuC5rq1aPNnkW35Upw6WXNGXjjt5jaFR5qmYREN5yACR8Qd3HtVOCvi0Cf0c7xGbdmAcLVcEp+TYwpih9Rvzp6nSpJJouECzSJDIR6zx9hpDQ+zmKTEiehrP4cYj62Nf4yWvfBZeKeZjpYPHv/9MUcpOfX18Q==; 5:mZhKmoyN5ORMb82my7wfoGHezTvbAWOgjJw5Xv01fv7gTjAeq1BGUk1KXC58C5NMK35ED8t32gscRPcp7awG1yiZRcKYA/ubsiRLEFsACg44qIA4Fnh0GVX4L57/BLEXttdEOuandwIsH3izj4kIHw==; 24:ShK86LTXAay6WL/ZcdjFM8HyUT23c8VHROejcT5N2+ZcNxCUcEJfnfJR8sKmhuSLxl4GGOO0r7POnu3glIjjoyYaGkaJrLVl4IhOIKbOeSE= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3008; 7:8fHN8C2QyfzKT0Jv1GlKkWMR7E/puKCy1yIhwZySLmBS1g1L1FO1MfhYM8bu/iOpxCnprEEGPciGG+ebV1i4l1E6pQe6E4SSa8DcbAM3McFqmMDkTnPTvYTmM2DKERPeoHibbKyP3YWWuyNFU4f98E90SDFz9/pNvfVSM3rqGW7EfWAwCiUxqbz3IBYipaOVJEqZkulVzhZr50rbuTpFmI5RPEUEyHiCzuSDERwDdxPiJaU+5OL78uH4MvtBI5DL6Qv4qhdurGlEMojXS9K0sC8YYIrmpqi6y9codlNJVmFnCi7KCgzJ3zyEfQrm6LLcDi0stVC++hRcgKpCpvAtKg== X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2017 20:31:03.6251 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR07MB3008 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170413_133129_754980_329B4F87 X-CRM114-Status: UNSURE ( 8.29 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jon Masters , Joerg Roedel , iommu@lists.linux-foundation.org, Jayachandran C , Robin Murphy , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP On Cavium ThunderX2 arm64 SoCs (called Broadcom Vulcan earlier), the PCI topology is slightly unusual. For a multi-node system, it looks like: 00:00.0 [PCI] bridge to [bus 01-1e] 01:0a.0 [PCI-PCIe bridge, type 8] bridge to [bus 02-04] 02:00.0 [PCIe root port, type 4] bridge to [bus 03-04] (XLATE_ROOT) 03:00.0 PCIe Endpoint pci_for_each_dma_alias() assumes IOMMU translation is done at the root of the PCI hierarchy. It generates 03:00.0, 01:0a.0, and 00:00.0 as DMA aliases for 03:00.0 because buses 01 and 00 are non-PCIe buses that doesn't carry the Requester ID. Because the ThunderX2 IOMMU is at 02:00.0, the Requester IDs 01:0a.0 or 00:00.0 are never valid for the endpoint. This quirk stops alias generation at the XLATE_ROOT bridge so we won't generate 01:0a.0 or 00:00.0 The current IOMMU code only maps the last alias (this is a separate bug in itself). Prior to this quirk, we only created IOMMU mappings for the invalid Requester ID 00:00:0, which never matched any DMA transactions. With this quirk, we create IOMMU mappings for a valid Requester ID, which fixes devices with no aliases but leaves devices with aliases still broken. The last alias for the endpoint is also used by the ARM GICv3 MSI-X code. Without this quirk, the GIC Interrupt Translation Tables are setup with the invalid Requester ID, and the MSI-X generated by the device fails to be translated and routed. Signed-off-by: Jayachandran C Reviewed-by: Robin Murphy Acked-by: David Daney --- drivers/pci/quirks.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index 6736836..564a84a 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -3958,6 +3958,20 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x2260, quirk_mic_x200_dma_alias); DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x2264, quirk_mic_x200_dma_alias); /* + * The IOMMU and interrupt controller on Broadcom Vulcan/Cavium ThunderX2 are + * associated not at the root bus, but at a bridge below. This quirk flag + * will ensure that the aliases are identified correctly. + */ +static void quirk_bridge_cavm_thrx2_pcie_root(struct pci_dev *pdev) +{ + pdev->dev_flags |= PCI_DEV_FLAGS_BRIDGE_XLATE_ROOT; +} +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM, 0x9000, + quirk_bridge_cavm_thrx2_pcie_root); +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM, 0x9084, + quirk_bridge_cavm_thrx2_pcie_root); + +/* * Intersil/Techwell TW686[4589]-based video capture cards have an empty (zero) * class code. Fix it. */