[v2] PCI: dwc: fix enumeration end when reaching root subordinate
diff mbox

Message ID 1516008968-26285-1-git-send-email-koen.vandeputte@ncentric.com
State New
Headers show

Commit Message

Koen Vandeputte Jan. 15, 2018, 9:36 a.m. UTC
The subordinate value indicates the highest bus number which can be
reached downstream though a certain device.

Commit a20c7f36bd3d ("PCI: Do not allocate more buses than available in
parent")
ensures that downstream devices cannot assign busnumbers higher than the
upstream device subordinate number, which was indeed illogical.

By default, dw_pcie_setup_rc() inits the Root Complex subordinate to a
value of 0x01.

Due to this combined with above commit, enumeration stops digging deeper
downstream as soon as bus num 0x01 has been assigned, which is always
the case for a bridge device.

This results in all devices behind a bridge bus to remain undetected, as
these would be connected to bus 0x02 or higher.

Fix this by initializing the RC to a subordinate value of 0xff, which is
not altering hardware behaviour in any way, but informs probing
function pci_scan_bridge() later on which reads this value back from
register.

Following nasty errors during boot are also fixed by this:

[    0.459145] pci_bus 0000:02: busn_res: can not insert [bus 02-ff]
under [bus 01] (conflicts with (null) [bus 01])
...
[    0.464515] pci_bus 0000:03: [bus 03] partially hidden behind bridge
0000:01 [bus 01]
...
[    0.464892] pci_bus 0000:04: [bus 04] partially hidden behind bridge
0000:01 [bus 01]
...
[    0.466488] pci_bus 0000:05: [bus 05] partially hidden behind bridge
0000:01 [bus 01]
[    0.466506] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to
05
[    0.466517] pci_bus 0000:02: busn_res: can not insert [bus 02-05]
under [bus 01] (conflicts with (null) [bus 01])
[    0.466534] pci_bus 0000:02: [bus 02-05] partially hidden behind
bridge 0000:01 [bus 01]

Fixes: a20c7f36bd3d ("PCI: Do not allocate more buses than available in
parent")
Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>
Tested-by: Niklas Cassel <niklas.cassel@axis.com>
Cc: Binghui Wang <wangbinghui@hisilicon.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jesper Nilsson <jesper.nilsson@axis.com>
Cc: Jianguo Sun <sunjianguo1@huawei.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Kishon Vijay Abraham I <kishon@ti.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Mika Westerberg <mika.westerberg@linux.intel.com>
Cc: Minghuan Lian <minghuan.Lian@freescale.com>
Cc: Mingkai Hu <mingkai.hu@freescale.com>
Cc: Murali Karicheri <m-karicheri2@ti.com>
Cc: Pratyush Anand <pratyush.anand@gmail.com>
Cc: Richard Zhu <hongxing.zhu@nxp.com>
Cc: Roy Zang <tie-fei.zang@freescale.com>
Cc: Shawn Guo <shawn.guo@linaro.org>
Cc: Stanimir Varbanov <svarbanov@mm-sol.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Xiaowei Song <songxiaowei@hisilicon.com>
Cc: Zhou Wang <wangzhou1@hisilicon.com>
---
 drivers/pci/dwc/pcie-designware-host.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Mason Jan. 15, 2018, 11:50 a.m. UTC | #1
On 15/01/2018 10:36, Koen Vandeputte wrote:

> The subordinate value indicates the highest bus number which can be
> reached downstream though a certain device.
> 
> Commit a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> parent")
> ensures that downstream devices cannot assign busnumbers higher than the
> upstream device subordinate number, which was indeed illogical.
> 
> By default, dw_pcie_setup_rc() inits the Root Complex subordinate to a
> value of 0x01.
> 
> Due to this combined with above commit, enumeration stops digging deeper
> downstream as soon as bus num 0x01 has been assigned, which is always
> the case for a bridge device.
> 
> This results in all devices behind a bridge bus to remain undetected, as
> these would be connected to bus 0x02 or higher.
> 
> Fix this by initializing the RC to a subordinate value of 0xff, which is
> not altering hardware behaviour in any way, but informs probing
> function pci_scan_bridge() later on which reads this value back from
> register.
> 
> Following nasty errors during boot are also fixed by this:
> 
> [    0.459145] pci_bus 0000:02: busn_res: can not insert [bus 02-ff]
> under [bus 01] (conflicts with (null) [bus 01])
> ...
> [    0.464515] pci_bus 0000:03: [bus 03] partially hidden behind bridge
> 0000:01 [bus 01]
> ...
> [    0.464892] pci_bus 0000:04: [bus 04] partially hidden behind bridge
> 0000:01 [bus 01]
> ...
> [    0.466488] pci_bus 0000:05: [bus 05] partially hidden behind bridge
> 0000:01 [bus 01]
> [    0.466506] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to
> 05
> [    0.466517] pci_bus 0000:02: busn_res: can not insert [bus 02-05]
> under [bus 01] (conflicts with (null) [bus 01])
> [    0.466534] pci_bus 0000:02: [bus 02-05] partially hidden behind
> bridge 0000:01 [bus 01]

FWIW, I find that this wrap-around looks ugly :-p

Regards.
Fabio Estevam Jan. 15, 2018, 7:47 p.m. UTC | #2
Hi Koen,

On Mon, Jan 15, 2018 at 7:36 AM, Koen Vandeputte
<koen.vandeputte@ncentric.com> wrote:
> The subordinate value indicates the highest bus number which can be
> reached downstream though a certain device.
>
> Commit a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> parent")
> ensures that downstream devices cannot assign busnumbers higher than the
> upstream device subordinate number, which was indeed illogical.
>
> By default, dw_pcie_setup_rc() inits the Root Complex subordinate to a
> value of 0x01.
>
> Due to this combined with above commit, enumeration stops digging deeper
> downstream as soon as bus num 0x01 has been assigned, which is always
> the case for a bridge device.
>
> This results in all devices behind a bridge bus to remain undetected, as
> these would be connected to bus 0x02 or higher.
>
> Fix this by initializing the RC to a subordinate value of 0xff, which is
> not altering hardware behaviour in any way, but informs probing
> function pci_scan_bridge() later on which reads this value back from
> register.
>
> Following nasty errors during boot are also fixed by this:
>
> [    0.459145] pci_bus 0000:02: busn_res: can not insert [bus 02-ff]
> under [bus 01] (conflicts with (null) [bus 01])
> ...
> [    0.464515] pci_bus 0000:03: [bus 03] partially hidden behind bridge
> 0000:01 [bus 01]
> ...
> [    0.464892] pci_bus 0000:04: [bus 04] partially hidden behind bridge
> 0000:01 [bus 01]
> ...
> [    0.466488] pci_bus 0000:05: [bus 05] partially hidden behind bridge
> 0000:01 [bus 01]
> [    0.466506] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to
> 05
> [    0.466517] pci_bus 0000:02: busn_res: can not insert [bus 02-05]
> under [bus 01] (conflicts with (null) [bus 01])
> [    0.466534] pci_bus 0000:02: [bus 02-05] partially hidden behind
> bridge 0000:01 [bus 01]
>
> Fixes: a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> parent")
> Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>
> Tested-by: Niklas Cassel <niklas.cassel@axis.com>
> Cc: Binghui Wang <wangbinghui@hisilicon.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Jesper Nilsson <jesper.nilsson@axis.com>
> Cc: Jianguo Sun <sunjianguo1@huawei.com>
> Cc: Jingoo Han <jingoohan1@gmail.com>
> Cc: Kishon Vijay Abraham I <kishon@ti.com>
> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Mika Westerberg <mika.westerberg@linux.intel.com>
> Cc: Minghuan Lian <minghuan.Lian@freescale.com>
> Cc: Mingkai Hu <mingkai.hu@freescale.com>
> Cc: Murali Karicheri <m-karicheri2@ti.com>
> Cc: Pratyush Anand <pratyush.anand@gmail.com>
> Cc: Richard Zhu <hongxing.zhu@nxp.com>
> Cc: Roy Zang <tie-fei.zang@freescale.com>
> Cc: Shawn Guo <shawn.guo@linaro.org>
> Cc: Stanimir Varbanov <svarbanov@mm-sol.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Xiaowei Song <songxiaowei@hisilicon.com>
> Cc: Zhou Wang <wangzhou1@hisilicon.com>

This fixes Intel Wifi card detection via bridge.

I adapted it to kernel 4.9 and it worked fine, so:

Tested-by: Fabio Estevam <fabio.estevam@nxp.com>

Kernel 4.9 does not contain a20c7f36bd3d ("PCI: Do not allocate more
buses than available in parent"), so it seems the commit log needs to
be reworded for the older kernels.

Thanks
Mika Westerberg Jan. 15, 2018, 7:52 p.m. UTC | #3
On Mon, Jan 15, 2018 at 10:36:08AM +0100, Koen Vandeputte wrote:
> The subordinate value indicates the highest bus number which can be
> reached downstream though a certain device.
> 
> Commit a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> parent")
> ensures that downstream devices cannot assign busnumbers higher than the
> upstream device subordinate number, which was indeed illogical.
> 
> By default, dw_pcie_setup_rc() inits the Root Complex subordinate to a
> value of 0x01.
> 
> Due to this combined with above commit, enumeration stops digging deeper
> downstream as soon as bus num 0x01 has been assigned, which is always
> the case for a bridge device.
> 
> This results in all devices behind a bridge bus to remain undetected, as
> these would be connected to bus 0x02 or higher.
> 
> Fix this by initializing the RC to a subordinate value of 0xff, which is
> not altering hardware behaviour in any way, but informs probing
> function pci_scan_bridge() later on which reads this value back from
> register.
> 
> Following nasty errors during boot are also fixed by this:
> 
> [    0.459145] pci_bus 0000:02: busn_res: can not insert [bus 02-ff]
> under [bus 01] (conflicts with (null) [bus 01])
> ...
> [    0.464515] pci_bus 0000:03: [bus 03] partially hidden behind bridge
> 0000:01 [bus 01]
> ...
> [    0.464892] pci_bus 0000:04: [bus 04] partially hidden behind bridge
> 0000:01 [bus 01]
> ...
> [    0.466488] pci_bus 0000:05: [bus 05] partially hidden behind bridge
> 0000:01 [bus 01]
> [    0.466506] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to
> 05
> [    0.466517] pci_bus 0000:02: busn_res: can not insert [bus 02-05]
> under [bus 01] (conflicts with (null) [bus 01])
> [    0.466534] pci_bus 0000:02: [bus 02-05] partially hidden behind
> bridge 0000:01 [bus 01]
> 
> Fixes: a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> parent")
> Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>
> Tested-by: Niklas Cassel <niklas.cassel@axis.com>
> Cc: Binghui Wang <wangbinghui@hisilicon.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Jesper Nilsson <jesper.nilsson@axis.com>
> Cc: Jianguo Sun <sunjianguo1@huawei.com>
> Cc: Jingoo Han <jingoohan1@gmail.com>
> Cc: Kishon Vijay Abraham I <kishon@ti.com>
> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Mika Westerberg <mika.westerberg@linux.intel.com>

Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com>
Lorenzo Pieralisi Feb. 5, 2018, 7:14 p.m. UTC | #4
On Mon, Jan 15, 2018 at 10:36:08AM +0100, Koen Vandeputte wrote:
> The subordinate value indicates the highest bus number which can be
> reached downstream though a certain device.
> 
> Commit a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> parent")
> ensures that downstream devices cannot assign busnumbers higher than the
> upstream device subordinate number, which was indeed illogical.
> 
> By default, dw_pcie_setup_rc() inits the Root Complex subordinate to a
> value of 0x01.
> 
> Due to this combined with above commit, enumeration stops digging deeper
> downstream as soon as bus num 0x01 has been assigned, which is always
> the case for a bridge device.
> 
> This results in all devices behind a bridge bus to remain undetected, as
> these would be connected to bus 0x02 or higher.
> 
> Fix this by initializing the RC to a subordinate value of 0xff, which is
> not altering hardware behaviour in any way, but informs probing
> function pci_scan_bridge() later on which reads this value back from
> register.
> 
> Following nasty errors during boot are also fixed by this:
> 
> [    0.459145] pci_bus 0000:02: busn_res: can not insert [bus 02-ff]
> under [bus 01] (conflicts with (null) [bus 01])
> ...
> [    0.464515] pci_bus 0000:03: [bus 03] partially hidden behind bridge
> 0000:01 [bus 01]
> ...
> [    0.464892] pci_bus 0000:04: [bus 04] partially hidden behind bridge
> 0000:01 [bus 01]
> ...
> [    0.466488] pci_bus 0000:05: [bus 05] partially hidden behind bridge
> 0000:01 [bus 01]
> [    0.466506] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to
> 05
> [    0.466517] pci_bus 0000:02: busn_res: can not insert [bus 02-05]
> under [bus 01] (conflicts with (null) [bus 01])
> [    0.466534] pci_bus 0000:02: [bus 02-05] partially hidden behind
> bridge 0000:01 [bus 01]
> 
> Fixes: a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> parent")
> Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>
> Tested-by: Niklas Cassel <niklas.cassel@axis.com>
> Cc: Binghui Wang <wangbinghui@hisilicon.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Jesper Nilsson <jesper.nilsson@axis.com>
> Cc: Jianguo Sun <sunjianguo1@huawei.com>
> Cc: Jingoo Han <jingoohan1@gmail.com>
> Cc: Kishon Vijay Abraham I <kishon@ti.com>
> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Mika Westerberg <mika.westerberg@linux.intel.com>
> Cc: Minghuan Lian <minghuan.Lian@freescale.com>
> Cc: Mingkai Hu <mingkai.hu@freescale.com>
> Cc: Murali Karicheri <m-karicheri2@ti.com>
> Cc: Pratyush Anand <pratyush.anand@gmail.com>
> Cc: Richard Zhu <hongxing.zhu@nxp.com>
> Cc: Roy Zang <tie-fei.zang@freescale.com>
> Cc: Shawn Guo <shawn.guo@linaro.org>
> Cc: Stanimir Varbanov <svarbanov@mm-sol.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Xiaowei Song <songxiaowei@hisilicon.com>
> Cc: Zhou Wang <wangzhou1@hisilicon.com>
> ---
>  drivers/pci/dwc/pcie-designware-host.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

I would appreciate some testing from dwc host maintainers so that
we can merge this patch, it is a bug fix that should be merged as
soon as possible, please help Koen test it and provide feedback
on the list.

Lorenzo

> diff --git a/drivers/pci/dwc/pcie-designware-host.c b/drivers/pci/dwc/pcie-designware-host.c
> index bf558df5b7b3..2b5470173196 100644
> --- a/drivers/pci/dwc/pcie-designware-host.c
> +++ b/drivers/pci/dwc/pcie-designware-host.c
> @@ -616,7 +616,7 @@ void dw_pcie_setup_rc(struct pcie_port *pp)
>  	/* setup bus numbers */
>  	val = dw_pcie_readl_dbi(pci, PCI_PRIMARY_BUS);
>  	val &= 0xff000000;
> -	val |= 0x00010100;
> +	val |= 0x00ff0100;
>  	dw_pcie_writel_dbi(pci, PCI_PRIMARY_BUS, val);
>  
>  	/* setup command register */
> -- 
> 2.7.4
>
Mika Westerberg Feb. 5, 2018, 8:06 p.m. UTC | #5
On Mon, Feb 05, 2018 at 07:14:21PM +0000, Lorenzo Pieralisi wrote:
> On Mon, Jan 15, 2018 at 10:36:08AM +0100, Koen Vandeputte wrote:
> > The subordinate value indicates the highest bus number which can be
> > reached downstream though a certain device.
> > 
> > Commit a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> > parent")
> > ensures that downstream devices cannot assign busnumbers higher than the
> > upstream device subordinate number, which was indeed illogical.
> > 
> > By default, dw_pcie_setup_rc() inits the Root Complex subordinate to a
> > value of 0x01.
> > 
> > Due to this combined with above commit, enumeration stops digging deeper
> > downstream as soon as bus num 0x01 has been assigned, which is always
> > the case for a bridge device.
> > 
> > This results in all devices behind a bridge bus to remain undetected, as
> > these would be connected to bus 0x02 or higher.
> > 
> > Fix this by initializing the RC to a subordinate value of 0xff, which is
> > not altering hardware behaviour in any way, but informs probing
> > function pci_scan_bridge() later on which reads this value back from
> > register.
> > 
> > Following nasty errors during boot are also fixed by this:
> > 
> > [    0.459145] pci_bus 0000:02: busn_res: can not insert [bus 02-ff]
> > under [bus 01] (conflicts with (null) [bus 01])
> > ...
> > [    0.464515] pci_bus 0000:03: [bus 03] partially hidden behind bridge
> > 0000:01 [bus 01]
> > ...
> > [    0.464892] pci_bus 0000:04: [bus 04] partially hidden behind bridge
> > 0000:01 [bus 01]
> > ...
> > [    0.466488] pci_bus 0000:05: [bus 05] partially hidden behind bridge
> > 0000:01 [bus 01]
> > [    0.466506] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to
> > 05
> > [    0.466517] pci_bus 0000:02: busn_res: can not insert [bus 02-05]
> > under [bus 01] (conflicts with (null) [bus 01])
> > [    0.466534] pci_bus 0000:02: [bus 02-05] partially hidden behind
> > bridge 0000:01 [bus 01]
> > 
> > Fixes: a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> > parent")
> > Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>
> > Tested-by: Niklas Cassel <niklas.cassel@axis.com>
> > Cc: Binghui Wang <wangbinghui@hisilicon.com>
> > Cc: Bjorn Helgaas <bhelgaas@google.com>
> > Cc: Jesper Nilsson <jesper.nilsson@axis.com>
> > Cc: Jianguo Sun <sunjianguo1@huawei.com>
> > Cc: Jingoo Han <jingoohan1@gmail.com>
> > Cc: Kishon Vijay Abraham I <kishon@ti.com>
> > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> > Cc: Lucas Stach <l.stach@pengutronix.de>
> > Cc: Mika Westerberg <mika.westerberg@linux.intel.com>
> > Cc: Minghuan Lian <minghuan.Lian@freescale.com>
> > Cc: Mingkai Hu <mingkai.hu@freescale.com>
> > Cc: Murali Karicheri <m-karicheri2@ti.com>
> > Cc: Pratyush Anand <pratyush.anand@gmail.com>
> > Cc: Richard Zhu <hongxing.zhu@nxp.com>
> > Cc: Roy Zang <tie-fei.zang@freescale.com>
> > Cc: Shawn Guo <shawn.guo@linaro.org>
> > Cc: Stanimir Varbanov <svarbanov@mm-sol.com>
> > Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> > Cc: Xiaowei Song <songxiaowei@hisilicon.com>
> > Cc: Zhou Wang <wangzhou1@hisilicon.com>
> > ---
> >  drivers/pci/dwc/pcie-designware-host.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> I would appreciate some testing from dwc host maintainers so that
> we can merge this patch, it is a bug fix that should be merged as
> soon as possible, please help Koen test it and provide feedback
> on the list.

BTW, this should have a stable tag so that it gets backported to stable
kernels. I guess whoever is applying the patch can add it so no need for
another revision.
Fabio Estevam Feb. 5, 2018, 10:12 p.m. UTC | #6
Hi Mika

On Mon, Feb 5, 2018 at 6:06 PM, Mika Westerberg
<mika.westerberg@linux.intel.com> wrote:

> BTW, this should have a stable tag so that it gets backported to stable
> kernels. I guess whoever is applying the patch can add it so no need for
> another revision.

I agree it needs to be backported to stable as I adapted this patch to
4.9 and it fixed the issue there.

However the Fixes lines mentions commit a20c7f36bd3d, which only
appears in 4.15, so that is confusing for backporting to older
kernels.
Sebastian Reichel Feb. 6, 2018, 10:44 a.m. UTC | #7
Hi,

This patch fixes network support for GE Healthcare B850v3, which has
two network cards attached to a PCIe switch and is

Tested-by: Sebastian Reichel <sebastian.reichel@collabora.co.uk>

-- Sebastian
Fabio Estevam Feb. 14, 2018, 3:41 p.m. UTC | #8
Hi Lucas and Richard,

On Mon, Feb 5, 2018 at 5:14 PM, Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
> On Mon, Jan 15, 2018 at 10:36:08AM +0100, Koen Vandeputte wrote:
>> The subordinate value indicates the highest bus number which can be
>> reached downstream though a certain device.
>>
>> Commit a20c7f36bd3d ("PCI: Do not allocate more buses than available in
>> parent")
>> ensures that downstream devices cannot assign busnumbers higher than the
>> upstream device subordinate number, which was indeed illogical.
>>
>> By default, dw_pcie_setup_rc() inits the Root Complex subordinate to a
>> value of 0x01.
>>
>> Due to this combined with above commit, enumeration stops digging deeper
>> downstream as soon as bus num 0x01 has been assigned, which is always
>> the case for a bridge device.
>>
>> This results in all devices behind a bridge bus to remain undetected, as
>> these would be connected to bus 0x02 or higher.
>>
>> Fix this by initializing the RC to a subordinate value of 0xff, which is
>> not altering hardware behaviour in any way, but informs probing
>> function pci_scan_bridge() later on which reads this value back from
>> register.
>>
>> Following nasty errors during boot are also fixed by this:
>>
>> [    0.459145] pci_bus 0000:02: busn_res: can not insert [bus 02-ff]
>> under [bus 01] (conflicts with (null) [bus 01])
>> ...
>> [    0.464515] pci_bus 0000:03: [bus 03] partially hidden behind bridge
>> 0000:01 [bus 01]
>> ...
>> [    0.464892] pci_bus 0000:04: [bus 04] partially hidden behind bridge
>> 0000:01 [bus 01]
>> ...
>> [    0.466488] pci_bus 0000:05: [bus 05] partially hidden behind bridge
>> 0000:01 [bus 01]
>> [    0.466506] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to
>> 05
>> [    0.466517] pci_bus 0000:02: busn_res: can not insert [bus 02-05]
>> under [bus 01] (conflicts with (null) [bus 01])
>> [    0.466534] pci_bus 0000:02: [bus 02-05] partially hidden behind
>> bridge 0000:01 [bus 01]
>>
>> Fixes: a20c7f36bd3d ("PCI: Do not allocate more buses than available in
>> parent")
>> Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>
>> Tested-by: Niklas Cassel <niklas.cassel@axis.com>
>> Cc: Binghui Wang <wangbinghui@hisilicon.com>
>> Cc: Bjorn Helgaas <bhelgaas@google.com>
>> Cc: Jesper Nilsson <jesper.nilsson@axis.com>
>> Cc: Jianguo Sun <sunjianguo1@huawei.com>
>> Cc: Jingoo Han <jingoohan1@gmail.com>
>> Cc: Kishon Vijay Abraham I <kishon@ti.com>
>> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
>> Cc: Lucas Stach <l.stach@pengutronix.de>
>> Cc: Mika Westerberg <mika.westerberg@linux.intel.com>
>> Cc: Minghuan Lian <minghuan.Lian@freescale.com>
>> Cc: Mingkai Hu <mingkai.hu@freescale.com>
>> Cc: Murali Karicheri <m-karicheri2@ti.com>
>> Cc: Pratyush Anand <pratyush.anand@gmail.com>
>> Cc: Richard Zhu <hongxing.zhu@nxp.com>
>> Cc: Roy Zang <tie-fei.zang@freescale.com>
>> Cc: Shawn Guo <shawn.guo@linaro.org>
>> Cc: Stanimir Varbanov <svarbanov@mm-sol.com>
>> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
>> Cc: Xiaowei Song <songxiaowei@hisilicon.com>
>> Cc: Zhou Wang <wangzhou1@hisilicon.com>
>> ---
>>  drivers/pci/dwc/pcie-designware-host.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> I would appreciate some testing from dwc host maintainers so that
> we can merge this patch, it is a bug fix that should be merged as
> soon as possible, please help Koen test it and provide feedback
> on the list.

Is it possible for you to test this patch?

Thanks
Lucas Stach Feb. 14, 2018, 3:49 p.m. UTC | #9
Am Mittwoch, den 14.02.2018, 13:41 -0200 schrieb Fabio Estevam:
> Hi Lucas and Richard,
> 
> On Mon, Feb 5, 2018 at 5:14 PM, Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
[...]
> > I would appreciate some testing from dwc host maintainers so that
> > we can merge this patch, it is a bug fix that should be merged as
> > soon as possible, please help Koen test it and provide feedback
> > on the list.
> 
> Is it possible for you to test this patch?

I did not test myself, but a Pengutronix colleague did and reported it
did fix the bus scanning issues, while not introducing any obvious
issues.

Acked-by: Lucas Stach <l.stach@pengutronix.de>

Regards,
Lucas
Lorenzo Pieralisi Feb. 20, 2018, 3:39 p.m. UTC | #10
On Mon, Jan 15, 2018 at 10:36:08AM +0100, Koen Vandeputte wrote:
> The subordinate value indicates the highest bus number which can be
> reached downstream though a certain device.
> 
> Commit a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> parent")
> ensures that downstream devices cannot assign busnumbers higher than the
> upstream device subordinate number, which was indeed illogical.
> 
> By default, dw_pcie_setup_rc() inits the Root Complex subordinate to a
> value of 0x01.
> 
> Due to this combined with above commit, enumeration stops digging deeper
> downstream as soon as bus num 0x01 has been assigned, which is always
> the case for a bridge device.
> 
> This results in all devices behind a bridge bus to remain undetected, as
> these would be connected to bus 0x02 or higher.
> 
> Fix this by initializing the RC to a subordinate value of 0xff, which is
> not altering hardware behaviour in any way, but informs probing
> function pci_scan_bridge() later on which reads this value back from
> register.
> 
> Following nasty errors during boot are also fixed by this:
> 
> [    0.459145] pci_bus 0000:02: busn_res: can not insert [bus 02-ff]
> under [bus 01] (conflicts with (null) [bus 01])
> ...
> [    0.464515] pci_bus 0000:03: [bus 03] partially hidden behind bridge
> 0000:01 [bus 01]
> ...
> [    0.464892] pci_bus 0000:04: [bus 04] partially hidden behind bridge
> 0000:01 [bus 01]
> ...
> [    0.466488] pci_bus 0000:05: [bus 05] partially hidden behind bridge
> 0000:01 [bus 01]
> [    0.466506] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to
> 05
> [    0.466517] pci_bus 0000:02: busn_res: can not insert [bus 02-05]
> under [bus 01] (conflicts with (null) [bus 01])
> [    0.466534] pci_bus 0000:02: [bus 02-05] partially hidden behind
> bridge 0000:01 [bus 01]
> 
> Fixes: a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> parent")
> Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>
> Tested-by: Niklas Cassel <niklas.cassel@axis.com>
> Cc: Binghui Wang <wangbinghui@hisilicon.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Jesper Nilsson <jesper.nilsson@axis.com>
> Cc: Jianguo Sun <sunjianguo1@huawei.com>
> Cc: Jingoo Han <jingoohan1@gmail.com>
> Cc: Kishon Vijay Abraham I <kishon@ti.com>
> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Mika Westerberg <mika.westerberg@linux.intel.com>
> Cc: Minghuan Lian <minghuan.Lian@freescale.com>
> Cc: Mingkai Hu <mingkai.hu@freescale.com>
> Cc: Murali Karicheri <m-karicheri2@ti.com>
> Cc: Pratyush Anand <pratyush.anand@gmail.com>
> Cc: Richard Zhu <hongxing.zhu@nxp.com>
> Cc: Roy Zang <tie-fei.zang@freescale.com>
> Cc: Shawn Guo <shawn.guo@linaro.org>
> Cc: Stanimir Varbanov <svarbanov@mm-sol.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Xiaowei Song <songxiaowei@hisilicon.com>
> Cc: Zhou Wang <wangzhou1@hisilicon.com>
> ---
>  drivers/pci/dwc/pcie-designware-host.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

I have applied it to pci/dwc for v4.17, with a tag to stable
for v4.15, we will have to send a backport to older stable
kernels.

Thanks,
Lorenzo

> diff --git a/drivers/pci/dwc/pcie-designware-host.c b/drivers/pci/dwc/pcie-designware-host.c
> index bf558df5b7b3..2b5470173196 100644
> --- a/drivers/pci/dwc/pcie-designware-host.c
> +++ b/drivers/pci/dwc/pcie-designware-host.c
> @@ -616,7 +616,7 @@ void dw_pcie_setup_rc(struct pcie_port *pp)
>  	/* setup bus numbers */
>  	val = dw_pcie_readl_dbi(pci, PCI_PRIMARY_BUS);
>  	val &= 0xff000000;
> -	val |= 0x00010100;
> +	val |= 0x00ff0100;
>  	dw_pcie_writel_dbi(pci, PCI_PRIMARY_BUS, val);
>  
>  	/* setup command register */
> -- 
> 2.7.4
>
Fabio Estevam March 2, 2018, 10:57 p.m. UTC | #11
Hi Lorenzo,

On Tue, Feb 20, 2018 at 12:39 PM, Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:

> I have applied it to pci/dwc for v4.17, with a tag to stable
> for v4.15, we will have to send a backport to older stable
> kernels.

Since this is a regression, why not apply it to 4.16-rc instead?
Lorenzo Pieralisi March 5, 2018, 9:49 a.m. UTC | #12
On Fri, Mar 02, 2018 at 07:57:16PM -0300, Fabio Estevam wrote:
> Hi Lorenzo,
> 
> On Tue, Feb 20, 2018 at 12:39 PM, Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
> 
> > I have applied it to pci/dwc for v4.17, with a tag to stable
> > for v4.15, we will have to send a backport to older stable
> > kernels.
> 
> Since this is a regression, why not apply it to 4.16-rc instead?

It is a balance of urgency and making sure it is extensively tested -
I'd prefer it to go via usual release cycle (and -next) and then it will
trickle into stable kernels, let me know if that's not OK.

I would understand your point if dwc maintainers were more proactive
in testing their respective controllers - all of them should be affected
by this fix but I have just heard from a few of them.

Thanks,
Lorenzo
Fabio Estevam March 5, 2018, 11:55 a.m. UTC | #13
Hi Lorenzo,

On Mon, Mar 5, 2018 at 6:49 AM, Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:

> It is a balance of urgency and making sure it is extensively tested -
> I'd prefer it to go via usual release cycle (and -next) and then it will
> trickle into stable kernels, let me know if that's not OK.
>
> I would understand your point if dwc maintainers were more proactive
> in testing their respective controllers - all of them should be affected
> by this fix but I have just heard from a few of them.

We got this patch tested by: Koen, myself, Sebastian and the folks at
Pengutronix.

Looks like a decent amount of testing IMHO.

In this case I would prefer that we could fix the regression into
4.16-rc cycle rather than waiting until 4.17.

Thanks
Lorenzo Pieralisi March 5, 2018, 12:24 p.m. UTC | #14
On Mon, Mar 05, 2018 at 08:55:43AM -0300, Fabio Estevam wrote:
> Hi Lorenzo,
> 
> On Mon, Mar 5, 2018 at 6:49 AM, Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
> 
> > It is a balance of urgency and making sure it is extensively tested -
> > I'd prefer it to go via usual release cycle (and -next) and then it will
> > trickle into stable kernels, let me know if that's not OK.
> >
> > I would understand your point if dwc maintainers were more proactive
> > in testing their respective controllers - all of them should be affected
> > by this fix but I have just heard from a few of them.
> 
> We got this patch tested by: Koen, myself, Sebastian and the folks at
> Pengutronix.
> 
> Looks like a decent amount of testing IMHO.

IIUC you all tested the same dwc host bridge variant (ie imx6) - I want
to understand if it works across dwc variants because this patch affects
them all.

> In this case I would prefer that we could fix the regression into
> 4.16-rc cycle rather than waiting until 4.17.

I will decide what to do shortly - I would really appreciate if other
dwc host bridge maintainers (that are CC'ed) can share the testing effort.

Thanks,
Lorenzo
Kishon Vijay Abraham I March 5, 2018, 1:08 p.m. UTC | #15
Hi Lorenzo,

On Monday 05 March 2018 05:54 PM, Lorenzo Pieralisi wrote:
> On Mon, Mar 05, 2018 at 08:55:43AM -0300, Fabio Estevam wrote:
>> Hi Lorenzo,
>>
>> On Mon, Mar 5, 2018 at 6:49 AM, Lorenzo Pieralisi
>> <lorenzo.pieralisi@arm.com> wrote:
>>
>>> It is a balance of urgency and making sure it is extensively tested -
>>> I'd prefer it to go via usual release cycle (and -next) and then it will
>>> trickle into stable kernels, let me know if that's not OK.
>>>
>>> I would understand your point if dwc maintainers were more proactive
>>> in testing their respective controllers - all of them should be affected
>>> by this fix but I have just heard from a few of them.
>>
>> We got this patch tested by: Koen, myself, Sebastian and the folks at
>> Pengutronix.
>>
>> Looks like a decent amount of testing IMHO.
> 
> IIUC you all tested the same dwc host bridge variant (ie imx6) - I want
> to understand if it works across dwc variants because this patch affects
> them all.
> 
>> In this case I would prefer that we could fix the regression into
>> 4.16-rc cycle rather than waiting until 4.17.
> 
> I will decide what to do shortly - I would really appreciate if other
> dwc host bridge maintainers (that are CC'ed) can share the testing effort.

For some reason I don't see the issues mentioned in this patch in dra7xx. The
root bus has a subordinate bus number as 01 but I'm able to read the
configuration space of the devices behind the bridge with bus number 2. I'll
have to take a closer look at what exactly happens.

root@dra7xx-evm:~# lspci -v
00:00.0 PCI bridge: Texas Instruments Multicore DSP+ARM KeyStone II SOC (rev
01) (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 255
        Memory at 20100000 (64-bit, non-prefetchable) [size=1M]
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [70] Express Root Port (Slot-), MSI 00
        Capabilities: [100] Advanced Error Reporting
lspci: Unable to load libkmod resources: error -12

01:00.0 PCI bridge: Pericom Semiconductor Device 2304 (rev 05) (prog-if 00
[Normal decode])
        Flags: fast devsel
        Bus: primary=01, secondary=02, subordinate=04, sec-latency=0
        Capabilities: [40] Power Management version 3
        Capabilities: [5c] Vital Product Data
        Capabilities: [64] Vendor Specific Information: Len=34 <?>
        Capabilities: [b0] Subsystem: Device 0000:0000
        Capabilities: [c0] Express Upstream Port, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Virtual Channel
        Capabilities: [20c] Power Budgeting <?>
        Capabilities: [230] Latency Tolerance Reporting

02:01.0 PCI bridge: Pericom Semiconductor Device 2304 (rev 05) (prog-if 00
[Normal decode])
        Flags: fast devsel
        Bus: primary=02, secondary=03, subordinate=03, sec-latency=0
        Capabilities: [40] Power Management version 3
        Capabilities: [4c] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [64] Vendor Specific Information: Len=34 <?>
        Capabilities: [b0] Subsystem: Device 0000:0000
        Capabilities: [c0] Express Downstream Port (Slot-), MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Virtual Channel
        Capabilities: [20c] Power Budgeting <?>
        Capabilities: [220] Access Control Services

02:02.0 PCI bridge: Pericom Semiconductor Device 2304 (rev 05) (prog-if 00
[Normal decode])
        Flags: fast devsel
        Bus: primary=02, secondary=04, subordinate=04, sec-latency=0
        Capabilities: [40] Power Management version 3
        Capabilities: [4c] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [64] Vendor Specific Information: Len=34 <?>
        Capabilities: [b0] Subsystem: Device 0000:0000
        Capabilities: [c0] Express Downstream Port (Slot-), MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Virtual Channel
        Capabilities: [20c] Power Budgeting <?>
        Capabilities: [220] Access Control Services

Thanks
Kishon
Shawn Guo March 6, 2018, 8:16 a.m. UTC | #16
On Mon, Mar 05, 2018 at 06:38:48PM +0530, Kishon Vijay Abraham I wrote:
> Hi Lorenzo,
> 
> On Monday 05 March 2018 05:54 PM, Lorenzo Pieralisi wrote:
> > On Mon, Mar 05, 2018 at 08:55:43AM -0300, Fabio Estevam wrote:
> >> Hi Lorenzo,
> >>
> >> On Mon, Mar 5, 2018 at 6:49 AM, Lorenzo Pieralisi
> >> <lorenzo.pieralisi@arm.com> wrote:
> >>
> >>> It is a balance of urgency and making sure it is extensively tested -
> >>> I'd prefer it to go via usual release cycle (and -next) and then it will
> >>> trickle into stable kernels, let me know if that's not OK.
> >>>
> >>> I would understand your point if dwc maintainers were more proactive
> >>> in testing their respective controllers - all of them should be affected
> >>> by this fix but I have just heard from a few of them.
> >>
> >> We got this patch tested by: Koen, myself, Sebastian and the folks at
> >> Pengutronix.
> >>
> >> Looks like a decent amount of testing IMHO.
> > 
> > IIUC you all tested the same dwc host bridge variant (ie imx6) - I want
> > to understand if it works across dwc variants because this patch affects
> > them all.
> > 
> >> In this case I would prefer that we could fix the regression into
> >> 4.16-rc cycle rather than waiting until 4.17.
> > 
> > I will decide what to do shortly - I would really appreciate if other
> > dwc host bridge maintainers (that are CC'ed) can share the testing effort.
> 
> For some reason I don't see the issues mentioned in this patch in dra7xx. The
> root bus has a subordinate bus number as 01 but I'm able to read the
> configuration space of the devices behind the bridge with bus number 2. I'll
> have to take a closer look at what exactly happens.

Just for record, I do not seem to see this issue on pcie-histb driver
as well.  Or did I miss anything?

# lspci -v
00:00.0 PCI bridge: Device 19e5:5610 (rev 01) (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 24
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        I/O behind bridge: 00001000-00001fff
        Memory behind bridge: 03800000-041fffff
        Prefetchable memory behind bridge: 03000000-037fffff
        [virtual] Expansion ROM at f4200000 [disabled] [size=64K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [70] Express Root Port (Slot-), MSI 00
        Capabilities: [100] Advanced Error Reporting
        Kernel driver in use: pcieport
lspci: Unable to load libkmod resources: error -12

01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
        Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
        Flags: bus master, fast devsel, latency 0, IRQ 25
        Memory at f4000000 (32-bit, non-prefetchable) [size=128K]
        Memory at f3800000 (32-bit, non-prefetchable) [size=4M]
        I/O ports at 1000 [disabled] [size=32]
        Memory at f4040000 (32-bit, non-prefetchable) [size=16K]
        [virtual] Expansion ROM at f3000000 [disabled] [size=4M]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable+ Count=1/1 Maskable+ 64bit+
        Capabilities: [70] MSI-X: Enable- Count=10 Masked-
        Capabilities: [a0] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Device Serial Number 90-e2-ba-ff-ff-18-2b-48
        Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
        Kernel driver in use: igb

01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
        Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
        Flags: bus master, fast devsel, latency 0, IRQ 26
        Memory at f4020000 (32-bit, non-prefetchable) [size=128K]
        Memory at f3c00000 (32-bit, non-prefetchable) [size=4M]
        I/O ports at 1020 [disabled] [size=32]
        Memory at f4084000 (32-bit, non-prefetchable) [size=16K]
        [virtual] Expansion ROM at f3400000 [disabled] [size=4M]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable+ Count=1/1 Maskable+ 64bit+
        Capabilities: [70] MSI-X: Enable- Count=10 Masked-
        Capabilities: [a0] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Device Serial Number 90-e2-ba-ff-ff-18-2b-48
        Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
        Kernel driver in use: igb

Shawn
Niklas Cassel March 6, 2018, 10:13 a.m. UTC | #17
On 06/03/18 09:16, Shawn Guo wrote:
> On Mon, Mar 05, 2018 at 06:38:48PM +0530, Kishon Vijay Abraham I wrote:
>> Hi Lorenzo,
>>
>> On Monday 05 March 2018 05:54 PM, Lorenzo Pieralisi wrote:
>>> On Mon, Mar 05, 2018 at 08:55:43AM -0300, Fabio Estevam wrote:
>>>> Hi Lorenzo,
>>>>
>>>> On Mon, Mar 5, 2018 at 6:49 AM, Lorenzo Pieralisi
>>>> <lorenzo.pieralisi@arm.com> wrote:
>>>>
>>>>> It is a balance of urgency and making sure it is extensively tested -
>>>>> I'd prefer it to go via usual release cycle (and -next) and then it will
>>>>> trickle into stable kernels, let me know if that's not OK.
>>>>>
>>>>> I would understand your point if dwc maintainers were more proactive
>>>>> in testing their respective controllers - all of them should be affected
>>>>> by this fix but I have just heard from a few of them.
>>>>
>>>> We got this patch tested by: Koen, myself, Sebastian and the folks at
>>>> Pengutronix.
>>>>
>>>> Looks like a decent amount of testing IMHO.
>>>
>>> IIUC you all tested the same dwc host bridge variant (ie imx6) - I want
>>> to understand if it works across dwc variants because this patch affects
>>> them all.
>>>
>>>> In this case I would prefer that we could fix the regression into
>>>> 4.16-rc cycle rather than waiting until 4.17.
>>>
>>> I will decide what to do shortly - I would really appreciate if other
>>> dwc host bridge maintainers (that are CC'ed) can share the testing effort.
>>
>> For some reason I don't see the issues mentioned in this patch in dra7xx. The
>> root bus has a subordinate bus number as 01 but I'm able to read the
>> configuration space of the devices behind the bridge with bus number 2. I'll
>> have to take a closer look at what exactly happens.
> 
> Just for record, I do not seem to see this issue on pcie-histb driver
> as well.  Or did I miss anything?

Shawn:

You don't seem to have a PCIe switch, so your setup isn't affected by the bug.


Kishon:

Please add an endpoint device behind your switch, and retest.
You will probably not see your endpoint device without this fix.

On ARTPEC-6, 03:00.0 is only shown with this fix.


On ARTPEC-6 without this fix:

# lspci -v
00:00.0 Class 0604: Device 1912:0024
	Flags: bus master, fast devsel, latency 0, IRQ 44
	Memory at c0100000 (32-bit, prefetchable) [size=1M]
	Memory at c0200000 (32-bit, prefetchable) [size=1M]
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: None
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [50] MSI: Enable+ Count=1/1 Maskable+ 64bit+
	Capabilities: [70] Express Root Port (Slot-), MSI 00
	Capabilities: [b0] MSI-X: Enable- Count=1 Masked-
	Capabilities: [100] Advanced Error Reporting
	Kernel driver in use: pcieport

01:00.0 Class 0604: Device 12d8:2304 (rev 05)
	Flags: bus master, fast devsel, latency 0
	Bus: primary=01, secondary=02, subordinate=04, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: None
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [5c] Vital Product Data
	Capabilities: [64] Vendor Specific Information: Len=34 <?>
	Capabilities: [b0] Subsystem: Device 0000:0000
	Capabilities: [c0] Express Upstream Port, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Virtual Channel
	Capabilities: [20c] Power Budgeting <?>
	Capabilities: [230] Latency Tolerance Reporting
	Kernel driver in use: pcieport

02:01.0 Class 0604: Device 12d8:2304 (rev 05)
	Flags: bus master, fast devsel, latency 0, IRQ 45
	Bus: primary=02, secondary=03, subordinate=03, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: None
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [4c] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Capabilities: [64] Vendor Specific Information: Len=34 <?>
	Capabilities: [b0] Subsystem: Device 0000:0000
	Capabilities: [c0] Express Downstream Port (Slot+), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Virtual Channel
	Capabilities: [20c] Power Budgeting <?>
	Capabilities: [220] Access Control Services
	Kernel driver in use: pcieport

02:02.0 Class 0604: Device 12d8:2304 (rev 05)
	Flags: bus master, fast devsel, latency 0, IRQ 46
	Bus: primary=02, secondary=04, subordinate=04, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: None
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [4c] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Capabilities: [64] Vendor Specific Information: Len=34 <?>
	Capabilities: [b0] Subsystem: Device 0000:0000
	Capabilities: [c0] Express Downstream Port (Slot+), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Virtual Channel
	Capabilities: [20c] Power Budgeting <?>
	Capabilities: [220] Access Control Services
	Kernel driver in use: pcieport


On ARTPEC-6 with this fix:

# lspci -v
00:00.0 Class 0604: Device 1912:0024
	Flags: bus master, fast devsel, latency 0, IRQ 44
	Memory at c0100000 (32-bit, prefetchable) [size=1M]
	Memory at c0200000 (32-bit, prefetchable) [size=1M]
	Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: c0300000-c03fffff [size=1M]
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [50] MSI: Enable+ Count=1/1 Maskable+ 64bit+
	Capabilities: [70] Express Root Port (Slot-), MSI 00
	Capabilities: [b0] MSI-X: Enable- Count=1 Masked-
	Capabilities: [100] Advanced Error Reporting
	Kernel driver in use: pcieport

01:00.0 Class 0604: Device 12d8:2304 (rev 05)
	Flags: bus master, fast devsel, latency 0
	Bus: primary=01, secondary=02, subordinate=04, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: c0300000-c03fffff [size=1M]
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [5c] Vital Product Data
	Capabilities: [64] Vendor Specific Information: Len=34 <?>
	Capabilities: [b0] Subsystem: Device 0000:0000
	Capabilities: [c0] Express Upstream Port, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Virtual Channel
	Capabilities: [20c] Power Budgeting <?>
	Capabilities: [230] Latency Tolerance Reporting
	Kernel driver in use: pcieport

02:01.0 Class 0604: Device 12d8:2304 (rev 05)
	Flags: bus master, fast devsel, latency 0, IRQ 45
	Bus: primary=02, secondary=03, subordinate=03, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: c0300000-c03fffff [size=1M]
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [4c] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Capabilities: [64] Vendor Specific Information: Len=34 <?>
	Capabilities: [b0] Subsystem: Device 0000:0000
	Capabilities: [c0] Express Downstream Port (Slot+), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Virtual Channel
	Capabilities: [20c] Power Budgeting <?>
	Capabilities: [220] Access Control Services
	Kernel driver in use: pcieport

02:02.0 Class 0604: Device 12d8:2304 (rev 05)
	Flags: bus master, fast devsel, latency 0, IRQ 46
	Bus: primary=02, secondary=04, subordinate=04, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: None
	Prefetchable memory behind bridge: None
	Capabilities: [40] Power Management version 3
	Capabilities: [4c] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Capabilities: [64] Vendor Specific Information: Len=34 <?>
	Capabilities: [b0] Subsystem: Device 0000:0000
	Capabilities: [c0] Express Downstream Port (Slot+), MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Virtual Channel
	Capabilities: [20c] Power Budgeting <?>
	Capabilities: [220] Access Control Services
	Kernel driver in use: pcieport

03:00.0 Class 0c03: Device 1912:0015 (rev 02) (prog-if 30)
	Flags: fast devsel
	Memory at c0300000 (64-bit, non-prefetchable) [size=8K]
	Capabilities: [50] Power Management version 3
	Capabilities: [70] MSI: Enable- Count=1/8 Maskable- 64bit+
	Capabilities: [90] MSI-X: Enable- Count=8 Masked-
	Capabilities: [a0] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [150] Latency Tolerance Reporting


Best regards,
Niklas
Lorenzo Pieralisi March 7, 2018, 5:13 p.m. UTC | #18
On Fri, Mar 02, 2018 at 07:57:16PM -0300, Fabio Estevam wrote:
> Hi Lorenzo,
> 
> On Tue, Feb 20, 2018 at 12:39 PM, Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
> 
> > I have applied it to pci/dwc for v4.17, with a tag to stable
> > for v4.15, we will have to send a backport to older stable
> > kernels.
> 
> Since this is a regression, why not apply it to 4.16-rc instead?

I asked Bjorn to send it for -rc5 and he kindly accepted, it is now
in his for-linus branch.

Thank you for bearing with me on this.

Lorenzo
Fabio Estevam March 7, 2018, 5:23 p.m. UTC | #19
Hi Lorenzo,

On Wed, Mar 7, 2018 at 2:13 PM, Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:

> I asked Bjorn to send it for -rc5 and he kindly accepted, it is now
> in his for-linus branch.

Excellent, thanks!
Fabio Estevam March 12, 2018, 1:31 p.m. UTC | #20
Hi Koen,

On Mon, Jan 15, 2018 at 7:36 AM, Koen Vandeputte
<koen.vandeputte@ncentric.com> wrote:
> The subordinate value indicates the highest bus number which can be
> reached downstream though a certain device.
>
> Commit a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> parent")
> ensures that downstream devices cannot assign busnumbers higher than the
> upstream device subordinate number, which was indeed illogical.
>
> By default, dw_pcie_setup_rc() inits the Root Complex subordinate to a
> value of 0x01.
>
> Due to this combined with above commit, enumeration stops digging deeper
> downstream as soon as bus num 0x01 has been assigned, which is always
> the case for a bridge device.
>
> This results in all devices behind a bridge bus to remain undetected, as
> these would be connected to bus 0x02 or higher.
>
> Fix this by initializing the RC to a subordinate value of 0xff, which is
> not altering hardware behaviour in any way, but informs probing
> function pci_scan_bridge() later on which reads this value back from
> register.
>
> Following nasty errors during boot are also fixed by this:
>
> [    0.459145] pci_bus 0000:02: busn_res: can not insert [bus 02-ff]
> under [bus 01] (conflicts with (null) [bus 01])
> ...
> [    0.464515] pci_bus 0000:03: [bus 03] partially hidden behind bridge
> 0000:01 [bus 01]
> ...
> [    0.464892] pci_bus 0000:04: [bus 04] partially hidden behind bridge
> 0000:01 [bus 01]
> ...
> [    0.466488] pci_bus 0000:05: [bus 05] partially hidden behind bridge
> 0000:01 [bus 01]
> [    0.466506] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to
> 05
> [    0.466517] pci_bus 0000:02: busn_res: can not insert [bus 02-05]
> under [bus 01] (conflicts with (null) [bus 01])
> [    0.466534] pci_bus 0000:02: [bus 02-05] partially hidden behind
> bridge 0000:01 [bus 01]
>
> Fixes: a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> parent")
> Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>

This commit is in Linus' tree now as commit fc110ebdd014dd1368.

Could you please prepare a version that could be backported to 4.9
stable kernel?

Thanks

Patch
diff mbox

diff --git a/drivers/pci/dwc/pcie-designware-host.c b/drivers/pci/dwc/pcie-designware-host.c
index bf558df5b7b3..2b5470173196 100644
--- a/drivers/pci/dwc/pcie-designware-host.c
+++ b/drivers/pci/dwc/pcie-designware-host.c
@@ -616,7 +616,7 @@  void dw_pcie_setup_rc(struct pcie_port *pp)
 	/* setup bus numbers */
 	val = dw_pcie_readl_dbi(pci, PCI_PRIMARY_BUS);
 	val &= 0xff000000;
-	val |= 0x00010100;
+	val |= 0x00ff0100;
 	dw_pcie_writel_dbi(pci, PCI_PRIMARY_BUS, val);
 
 	/* setup command register */