| Message ID | alpine.DEB.2.21.2201022100430.56863@angie.orcam.me.uk (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | Bjorn Helgaas |
| Headers | show |
| Series | [v2] pci: Work around ASMedia ASM2824 PCIe link training failures | expand |
On Sun, 2 Jan 2022 23:25:25 +0000 (GMT) "Maciej W. Rozycki" <macro@orcam.me.uk> wrote: > Attempt to handle cases with a downstream port of the ASMedia ASM2824 > PCIe switch where link training never completes and the link continues > switching between speeds indefinitely with the data link layer never > reaching the active state. > > It has been observed with a downstream port of the ASMedia ASM2824 Gen 3 > switch wired to the upstream port of the Pericom PI7C9X2G304 Gen 2 > switch, using a Delock Riser Card PCI Express x1 > 2 x PCIe x1 device, > P/N 41433, wired to a SiFive HiFive Unmatched board. In this setup the > switches are supposed to negotiate the link speed of preferably 5.0GT/s, > falling back to 2.5GT/s. It looks like I might have similar issue on EPYC system with this switch. In my case ASM2824 switch board connected to EPYC's root port (via SlimSAS 8i cable and on upstream site it negotiates expected 8GT/s x8) were supposed to provide 10x slots with PCIe x1 Gen3 links. However lspci shows that link is capped at 2.5GT/s and NVME plugged into one of the slots is downgraded to 2.5GT/s. I wonder if it could be fixed to get expected 8GT/s per lane. Downstream port in question: 07:00.0 PCI bridge: ASMedia Technology Inc. ASM2824 PCIe Gen3 Packet Switch (rev 01) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 35 IOMMU group: 20 Bus: primary=07, secondary=08, subordinate=08, sec-latency=0 I/O behind bridge: [disabled] [32-bit] Memory behind bridge: ee700000-ee7fffff [size=1M] [32-bit] Prefetchable memory behind bridge: [disabled] [64-bit] Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16+ MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+ Address: 00000000fee00000 Data: 0000 Capabilities: [80] Express (v2) Downstream Port (Slot+), MSI 00 DevCap: MaxPayload 512 bytes, PhantFunc 0 ExtTag+ RBE+ DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x2, ASPM L1, Exit Latency L1 <64us ClockPM+ Surprise+ LLActRep+ BwNot+ ASPMOptComp+ LnkCtl: ASPM Disabled; Disabled- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1 TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- Slot #0, PowerLimit 0W; Interlock- NoCompl- SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock- Changed: MRL- PresDet- LinkState- DevCap2: Completion Timeout: Not Supported, TimeoutDis- NROPrPrP- LTR+ 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- ARIFwd+ AtomicOpsCap: Routing- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled, ARIFwd- AtomicOpsCtl: EgressBlck- LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS- LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis+, Selectable De-emphasis: -3.5dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [c0] Subsystem: Device 0000:0000 Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 Capabilities: [140 v1] Power Budgeting <?> Capabilities: [160 v1] Latency Tolerance Reporting Max snoop latency: 0ns Max no snoop latency: 0ns Capabilities: [180 v1] Multicast McastCap: MaxGroups 64, ECRCRegen- McastCtl: NumGroups 1, Enable- McastBAR: IndexPos 0, BaseAddr 0000000000000000 McastReceiveVec: 0000000000000000 McastBlockAllVec: 0000000000000000 McastBlockUntransVec: 0000000000000000 McastOverlayBAR: OverlaySize 0 (disabled), BaseAddr 0000000000000000 Capabilities: [1c0 v1] Secondary PCI Express LnkCtl3: LnkEquIntrruptEn- PerformEqu- LaneErrStat: 0 Capabilities: [200 v1] Vendor Specific Information: ID=001a Rev=0 Len=000 <?> Kernel driver in use: pcieport NVME: 08:00.0 Non-Volatile memory controller: Silicon Motion, Inc. SM2263EN/SM2263XT (DRAM-less) NVMe SSD Controllers (rev 03) (prog-if 02 [NVM Express]) Subsystem: Biwin Storage Technology Co., Ltd. Device 2263 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 34 NUMA node: 0 IOMMU group: 20 Region 0: Memory at ee700000 (64-bit, non-prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] MSI: Enable- Count=1/8 Maskable+ 64bit+ Address: 0000000000000000 Data: 0000 Masking: 00000000 Pending: 00000000 Capabilities: [70] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0W DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- FLReset- MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr+ TransPend- LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <8us ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s (downgraded), Width x1 (downgraded) TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- TPHComp- ExtTPHComp- AtomicOpsCap: 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled, AtomicOpsCtl: ReqEn- LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS- LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [b0] MSI-X: Enable+ Count=16 Masked- Vector table: BAR=0 offset=00002000 PBA: BAR=0 offset=00002100 Capabilities: [100 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 Capabilities: [158 v1] Secondary PCI Express LnkCtl3: LnkEquIntrruptEn- PerformEqu- LaneErrStat: 0 Capabilities: [178 v1] Latency Tolerance Reporting Max snoop latency: 0ns Max no snoop latency: 0ns Capabilities: [180 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+ PortCommonModeRestoreTime=10us PortTPowerOnTime=10us L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- T_CommonMode=0us LTR1.2_Threshold=0ns L1SubCtl2: T_PwrOn=10us Kernel driver in use: nvme Kernel modules: nvme full lspci output: https://pastebin.mozilla.org/Kb8Fs5Mj
Index: linux-macro/drivers/pci/quirks.c =================================================================== --- linux-macro.orig/drivers/pci/quirks.c +++ linux-macro/drivers/pci/quirks.c @@ -12,6 +12,7 @@ * file, where their drivers can use them. */ +#include <linux/bug.h> #include <linux/types.h> #include <linux/kernel.h> #include <linux/export.h> @@ -5857,3 +5858,98 @@ static void nvidia_ion_ahci_fixup(struct pdev->dev_flags |= PCI_DEV_FLAGS_HAS_MSI_MASKING; } DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_NVIDIA, 0x0ab8, nvidia_ion_ahci_fixup); + +/* + * Retrain the link of a downstream PCIe port by hand if necessary. + * + * This is needed at least where a downstream port of the ASMedia ASM2824 + * Gen 3 switch is wired to the upstream port of the Pericom PI7C9X2G304 + * Gen 2 switch, and observed with the Delock Riser Card PCI Express x1 > + * 2 x PCIe x1 device, P/N 41433, plugged into the SiFive HiFive Unmatched + * board. + * + * In such a configuration the switches are supposed to negotiate the link + * speed of preferably 5.0GT/s, falling back to 2.5GT/s. However the link + * continues switching between the two speeds indefinitely and the data + * link layer never reaches the active state, with link training reported + * repeatedly active ~84% of the time. Forcing the target link speed to + * 2.5GT/s with the upstream ASM2824 device makes the two switches talk to + * each other correctly however. And more interestingly retraining with a + * higher target link speed afterwards lets the two successfully negotiate + * 5.0GT/s. + * + * With the ASM2824 we can rely on the otherwise optional Data Link Layer + * Link Active status bit and in the failed link training scenario it will + * be off along with the Link Bandwidth Management Status indicating that + * hardware has changed the link speed or width in an attempt to correct + * unreliable link operation. For a port that has been left unconnected + * both bits will be clear. So use this information to detect the problem + * rather than polling the Link Training bit and watching out for flips or + * at least the active status. + * + * Restrict the speed to 2.5GT/s then with the Target Link Speed field, + * request a retrain and wait 200ms for the data link to go up. If this + * turns out successful, then lift the restriction, letting the devices + * negotiate a higher speed. Also check for a 2.5GT/s speed restriction + * the firmware may have already arranged and lift it too with ports that + * already report their data link being up. + */ +static void pcie_downstream_link_retrain(struct pci_dev *dev) +{ + u16 lnksta, lnkctl2; + u8 pos; + + pos = pci_find_capability(dev, PCI_CAP_ID_EXP); + BUG_ON(!pos); + + pci_read_config_word(dev, pos + PCI_EXP_LNKCTL2, &lnkctl2); + pci_read_config_word(dev, pos + PCI_EXP_LNKSTA, &lnksta); + if ((lnksta & (PCI_EXP_LNKSTA_LBMS | PCI_EXP_LNKSTA_DLLLA)) == + PCI_EXP_LNKSTA_LBMS) { + unsigned long timeout; + u16 lnkctl; + + pci_info(dev, "broken device, retraining non-functional downstream link at 2.5GT/s...\n"); + + pci_read_config_word(dev, pos + PCI_EXP_LNKCTL, &lnkctl); + lnkctl |= PCI_EXP_LNKCTL_RL; + lnkctl2 &= ~PCI_EXP_LNKCTL2_TLS; + lnkctl2 |= PCI_EXP_LNKCTL2_TLS_2_5GT; + pci_write_config_word(dev, pos + PCI_EXP_LNKCTL2, lnkctl2); + pci_write_config_word(dev, pos + PCI_EXP_LNKCTL, lnkctl); + + timeout = jiffies + msecs_to_jiffies(200); + do { + pci_read_config_word(dev, pos + PCI_EXP_LNKSTA, + &lnksta); + if (lnksta & PCI_EXP_LNKSTA_DLLLA) + break; + usleep_range(10000, 20000); + } while (time_before(jiffies, timeout)); + + pci_info(dev, "retraining %s!\n", + lnksta & PCI_EXP_LNKSTA_DLLLA ? + "succeeded" : "failed"); + } + + if ((lnksta & PCI_EXP_LNKSTA_DLLLA) && + (lnkctl2 & PCI_EXP_LNKCTL2_TLS) == PCI_EXP_LNKCTL2_TLS_2_5GT) { + u32 lnkcap; + u16 lnkctl; + + pci_info(dev, "removing 2.5GT/s downstream link speed restriction\n"); + pci_read_config_word(dev, pos + PCI_EXP_LNKCTL, &lnkctl); + pci_read_config_dword(dev, pos + PCI_EXP_LNKCAP, &lnkcap); + lnkctl |= PCI_EXP_LNKCTL_RL; + lnkctl2 &= ~PCI_EXP_LNKCTL2_TLS; + lnkctl2 |= lnkcap & PCI_EXP_LNKCAP_SLS; + pci_write_config_word(dev, pos + PCI_EXP_LNKCTL2, lnkctl2); + pci_write_config_word(dev, pos + PCI_EXP_LNKCTL, lnkctl); + } +} +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ASMEDIA, + PCI_DEVICE_ID_ASMEDIA_ASM2824, + pcie_downstream_link_retrain); +DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDOR_ID_ASMEDIA, + PCI_DEVICE_ID_ASMEDIA_ASM2824, + pcie_downstream_link_retrain); Index: linux-macro/include/linux/pci_ids.h =================================================================== --- linux-macro.orig/include/linux/pci_ids.h +++ linux-macro/include/linux/pci_ids.h @@ -2562,6 +2562,7 @@ #define PCI_SUBDEVICE_ID_QEMU 0x1100 #define PCI_VENDOR_ID_ASMEDIA 0x1b21 +#define PCI_DEVICE_ID_ASMEDIA_ASM2824 0x2824 #define PCI_VENDOR_ID_REDHAT 0x1b36
Attempt to handle cases with a downstream port of the ASMedia ASM2824 PCIe switch where link training never completes and the link continues switching between speeds indefinitely with the data link layer never reaching the active state. It has been observed with a downstream port of the ASMedia ASM2824 Gen 3 switch wired to the upstream port of the Pericom PI7C9X2G304 Gen 2 switch, using a Delock Riser Card PCI Express x1 > 2 x PCIe x1 device, P/N 41433, wired to a SiFive HiFive Unmatched board. In this setup the switches are supposed to negotiate the link speed of preferably 5.0GT/s, falling back to 2.5GT/s. However the link continues oscillating between the two speeds, at the rate of 34-35 times per second, with link training reported repeatedly active ~84% of the time, e.g.: 02:03.0 PCI bridge [0604]: ASMedia Technology Inc. ASM2824 PCIe Gen3 Packet Switch [1b21:2824] (rev 01) (prog-if 00 [Normal decode]) [...] Bus: primary=02, secondary=05, subordinate=05, sec-latency=0 [...] Capabilities: [80] Express (v2) Downstream Port (Slot+), MSI 00 [...] LnkSta: Speed 5GT/s (downgraded), Width x1 (ok) TrErr- Train+ SlotClk+ DLActive- BWMgmt+ ABWMgmt- [...] LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis+, Selectable De-emphasis: -3.5dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB [...] Forcibly limiting the target link speed to 2.5GT/s with the upstream ASM2824 device makes the two switches communicate correctly however: 02:03.0 PCI bridge [0604]: ASMedia Technology Inc. ASM2824 PCIe Gen3 Packet Switch [1b21:2824] (rev 01) (prog-if 00 [Normal decode]) [...] Bus: primary=02, secondary=05, subordinate=09, sec-latency=0 [...] Capabilities: [80] Express (v2) Downstream Port (Slot+), MSI 00 [...] LnkSta: Speed 2.5GT/s (downgraded), Width x1 (ok) TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- [...] LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis+, Selectable De-emphasis: -3.5dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB [...] and then: 05:00.0 PCI bridge [0604]: Pericom Semiconductor PI7C9X2G304 EL/SL PCIe2 3-Port/4-Lane Packet Switch [12d8:2304] (rev 05) (prog-if 00 [Normal decode]) [...] Bus: primary=05, secondary=06, subordinate=09, sec-latency=0 [...] Capabilities: [c0] Express (v2) Upstream Port, MSI 00 [...] LnkSta: Speed 2.5GT/s (downgraded), Width x1 (downgraded) TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- [...] LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB [...] Removing the speed restriction afterwards makes the two devices switch to 5.0GT/s then. Make use of these observations then and detect the inability to train the link, by checking for the Data Link Layer Link Active status bit implemented by the ASM2824 being off while the Link Bandwidth Management Status indicating that hardware has changed the link speed or width in an attempt to correct unreliable link operation. Restrict the speed to 2.5GT/s then with the Target Link Speed field, request a retrain and wait 200ms for the data link to go up. If this turns out successful, then lift the restriction, letting the devices negotiate a higher speed. Also check for a 2.5GT/s speed restriction the firmware may have already arranged and lift it too with ports that already report their data link being up. Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk> --- Hi, Reposting as it has gone nowhere. Mechanically regenerated for a merge conflict only, no changes otherwise. This proposal includes support for a scenario where the firmware has restricted the speed for a port already, for access to any downstream boot devices if nothing else, such as with the U-Boot change proposed here: <https://lists.denx.de/pipermail/u-boot/2021-November/467852.html>. This is why the removal of the 2.5GT/s speed restriction is done regardless of whether it was us to put it in place in the first place. Credit goes to Stefan for a suggestion to check for any interaction with the de-emphasis level, which gave me incentive for further experimentation that has eventually made me discover that lifting the restriction makes the two devices negotiate 5GT/s. This is especially worthwhile given that the Delock part (<https://www.delock.com/produkt/41433/merkmale.html>), since discontinued, has a 5GT/s x1 upstream port and a pair of 2.5GT/s x1 downstream ports so keeping the upstream port at it's maximum speed avoids a bottleneck there. For the background story as to how I tried to get the manufacturers of the devices involved, unsuccessfully, except for the guys from SiFive who were at least sympathetic (thank you!), but this level of hardware debug was beyond their skills and/or resources, and then how I discovered how to persuade the two devices to talk to each other at all, at 2.5GT/s, see: <https://lists.denx.de/pipermail/u-boot/2021-November/467199.html>. One case that has been nurturing me though is the reverse scenario, that is where the Pericom PI7C9X2G304 switch is upstream while the ASMedia ASM2824 switch is downstream. Presumably the same situation will happen, so matching on the ASM2824 ID only would be a problem. Unfortunately the other device does not implement the Data Link Layer Link Active status bit, so a more complex approach, such as clearing and then checking for the Link Bandwidth Management Status having been set again might be an option. Unlike U-Boot we cannot do aggressive polling of the Link Training bit. Option hardware with M.2 slots is commercially available with the ASM2824 onboard, so a test environment can be in principle arranged, though I'm not sure if just for the sake of such an experiment I'm willing to spend money that will ultimately go to a manufacturer that cannot be bothered to take responsibility for their faults and at the very least respond to a problem report. And without verifying the actual problem exists I'm reluctant to try and implement a workaround. On the other hand the problem with the Unmatched board is real and this change addresses it, at least for me. NB the BUG_ON there is a safety valve really for an "impossible" case. The ASM2824 is a PCIe device and necessarily it does have the PCI Express capability. Poking at that unguarded though just didn't feel right to me, while doing any kind of full-fledged recovery seemed like an overkill. Questions, comments, concerns? Otherwise please apply. Maciej Changes from v1: - Regenerate for a merge conflict. --- drivers/pci/quirks.c | 96 ++++++++++++++++++++++++++++++++++++++++++++++++ include/linux/pci_ids.h | 1 2 files changed, 97 insertions(+) linux-pcie-asm2824-manual-retrain.diff