From patchwork Wed Jul 5 01:00:03 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sinan Kaya X-Patchwork-Id: 9825731 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 84A8F60361 for ; Wed, 5 Jul 2017 01:00:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6598826E16 for ; Wed, 5 Jul 2017 01:00:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 59A7926E40; Wed, 5 Jul 2017 01:00:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 88C6D26B41 for ; Wed, 5 Jul 2017 01:00:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752265AbdGEBAH (ORCPT ); Tue, 4 Jul 2017 21:00:07 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:51878 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752212AbdGEBAG (ORCPT ); Tue, 4 Jul 2017 21:00:06 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id EE0FD60AD3; Wed, 5 Jul 2017 01:00:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1499216405; bh=HWXMYe218mud1Kfm0nuE2IyMO75hi+M2HnL3ubDhYHw=; h=Subject:To:Cc:References:From:Date:In-Reply-To:From; b=BezqmvN1CzViYaS4NfOxpDkK2zOD1tBi5MmE6GUy2rh+TSzSoyR29YpgGOQ7Qnwg1 DAzbEDMzRo7RZd1Vy8MWpC4jtFNLOeh+lHM86itUU4cNDGCNXJ5JZ9N60fcwWW7s5m zPvwbXEjfi+YnuD+CbDPKb/dQM9P1cNFZRLdSwQI= Received: from [192.168.1.79] (104-182-54-152.lightspeed.rlghnc.sbcglobal.net [104.182.54.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: okaya@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 6D88A6077C; Wed, 5 Jul 2017 01:00:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1499216405; bh=HWXMYe218mud1Kfm0nuE2IyMO75hi+M2HnL3ubDhYHw=; h=Subject:To:Cc:References:From:Date:In-Reply-To:From; b=BezqmvN1CzViYaS4NfOxpDkK2zOD1tBi5MmE6GUy2rh+TSzSoyR29YpgGOQ7Qnwg1 DAzbEDMzRo7RZd1Vy8MWpC4jtFNLOeh+lHM86itUU4cNDGCNXJ5JZ9N60fcwWW7s5m zPvwbXEjfi+YnuD+CbDPKb/dQM9P1cNFZRLdSwQI= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 6D88A6077C Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=okaya@codeaurora.org Subject: Re: Red Hat (Fedora) bug report 1467674 concerning your kernel functional performance enhancements causing PCI Express crashes, To: Wim ten Have Cc: Bjorn Helgaas , "linux-pci@vger.kernel.org" References: <20170704161352.1cdb2670.wim.ten.have@oracle.com> <2acc9a73-0c2c-31f3-fcdf-42289213860e@codeaurora.org> <20170704195916.43fbe6e4.wim.ten.have@oracle.com> <97b8c58e-7384-0519-5ccb-837fa5054d84@codeaurora.org> From: Sinan Kaya Message-ID: <17f06c42-a96f-d1da-38e5-95117626eb29@codeaurora.org> Date: Tue, 4 Jul 2017 21:00:03 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <97b8c58e-7384-0519-5ccb-837fa5054d84@codeaurora.org> Content-Language: en-US Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 7/4/2017 6:25 PM, Sinan Kaya wrote: > On 7/4/2017 1:59 PM, Wim ten Have wrote: >> On Tue, 4 Jul 2017 11:57:37 -0400 >> Sinan Kaya wrote: >> >>> Hi, >>> >>> On 7/4/2017 11:32 AM, Bjorn Helgaas wrote: >>>> [+cc linux-pci] >>>> >>>> Thanks very much for the detailed problem report, Wim! I'm taking the >>>> liberty to forward to the linux-pci list in case others trip over the >>>> same thing. >>>> >>> >>> So, the spec is lying :) and reality doesn't match theory. > > The PCI Express bridge you have is a Broadcom HT 2100 bridge which seems to support > PCI-Express V1.0 and 1.0a compliant only. > > http://www.hard-net.de/info_wissen/chipsatz/broadcom/HT-2100.pdf > > I can also see this in your lspci output. > > 00:08.0 PCI bridge: Broadcom HT2100 PCI-Express Bridge (rev a2) (prog-if 00 [Normal decode]) > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 19 > NUMA node: 0 > Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 > I/O behind bridge: 0000f000-00000fff [empty] > Memory behind bridge: efe00000-efefffff [size=1M] > Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff [empty] > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- BridgeCtl: Parity+ SERR+ NoISA+ VGA- MAbort- >Reset- FastB2B- > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > Capabilities: [a0] HyperTransport: MSI Mapping Enable+ Fixed- > Mapping Address Base: 00000000fee00000 > Capabilities: [b0] Express (v1) Root Port (Slot-), MSI 00 > > I'll post a patch to apply extended tags to systems with PCI express v2 and higher > bridges only. > Please give this patch a try. I can make the patch pretty and re-post if it works for you. You should be seeing messages like this during boot. [ 3.949621] pci 0003:01:00.0: clearing extended tags capability [ 3.959540] pci 0003:01:00.1: clearing extended tags capability [ 3.969454] pci 0003:01:00.2: clearing extended tags capability [ 3.979373] pci 0003:01:00.3: clearing extended tags capability [ 3.989290] pci 0003:01:00.4: clearing extended tags capability diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index dfc9a27..c67af22 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1663,21 +1663,58 @@ static void program_hpp_type2(struct pci_dev *dev, struct hpp_type2 *hpp) */ } -static void pci_configure_extended_tags(struct pci_dev *dev) +static bool pcie_bus_exttags_supported(struct pci_bus *bus) +{ + bool exttags_supported = true; + struct pci_dev *bridge; + int rc; + u16 flags; + + bridge = bus->self; + while (bridge) { + if (pci_is_pcie(bridge)) { + rc = pcie_capability_read_word(bridge, PCI_EXP_FLAGS, + &flags); + if (!rc && ((flags & PCI_EXP_FLAGS_VERS) < 2)) { + exttags_supported = false; + break; + } + } + if (!bridge->bus->parent) + break; + bridge = bridge->bus->parent->self; + } + + return exttags_supported; +} + +static int pcie_bus_configure_exttags(struct pci_dev *dev, void *data) { u32 dev_cap; int ret; + bool supported; if (!pci_is_pcie(dev)) - return; + return 0; ret = pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, &dev_cap); if (ret) - return; + return 0; - if (dev_cap & PCI_EXP_DEVCAP_EXT_TAG) - pcie_capability_set_word(dev, PCI_EXP_DEVCTL, - PCI_EXP_DEVCTL_EXT_TAG); + if (dev_cap & PCI_EXP_DEVCAP_EXT_TAG) { + supported = pcie_bus_exttags_supported(dev->bus); + + if (supported) { + dev_info(&dev->dev, "setting extended tags capability\n"); + pcie_capability_set_word(dev, PCI_EXP_DEVCTL, + PCI_EXP_DEVCTL_EXT_TAG); + } else { + dev_info(&dev->dev, "clearing extended tags capability\n"); + pcie_capability_clear_word(dev, PCI_EXP_DEVCTL, + PCI_EXP_DEVCTL_EXT_TAG); + } + } + return 0; } static void pci_configure_device(struct pci_dev *dev) @@ -1686,7 +1723,6 @@ static void pci_configure_device(struct pci_dev *dev) int ret; pci_configure_mps(dev); - pci_configure_extended_tags(dev); memset(&hpp, 0, sizeof(hpp)); ret = pci_get_hp_params(dev, &hpp); @@ -2231,6 +2267,8 @@ void pcie_bus_configure_settings(struct pci_bus *bus) pcie_bus_configure_set(bus->self, &smpss); pci_walk_bus(bus, pcie_bus_configure_set, &smpss); + + pci_walk_bus(bus, pcie_bus_configure_exttags, NULL); } EXPORT_SYMBOL_GPL(pcie_bus_configure_settings);