| Message ID | 20230809102257.25121-1-jason.chien@sifive.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | hw/pci-host: Allow extended config space access for Designware PCIe host | expand |
On Wed, Aug 09, 2023 at 10:22:50AM +0000, Jason Chien wrote: > In pcie_bus_realize(), a root bus is realized as a PCIe bus and a non-root > bus is realized as a PCIe bus if its parent bus is a PCIe bus. However, > the child bus "dw-pcie" is realized before the parent bus "pcie" which is > the root PCIe bus. Thus, the extended configuration space is not accessible > on "dw-pcie". The issue can be resolved by adding the > PCI_BUS_EXTENDED_CONFIG_SPACE flag to "pcie" before "dw-pcie" is realized. > > Signed-off-by: Jason Chien <jason.chien@sifive.com> I think we should fix the order of initialization rather than hack around it. > --- > hw/pci-host/designware.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/hw/pci-host/designware.c b/hw/pci-host/designware.c > index 9e183caa48..388d252ee2 100644 > --- a/hw/pci-host/designware.c > +++ b/hw/pci-host/designware.c > @@ -694,6 +694,7 @@ static void designware_pcie_host_realize(DeviceState *dev, Error **errp) > &s->pci.io, > 0, 4, > TYPE_PCIE_BUS); > + pci->bus->flags |= PCI_BUS_EXTENDED_CONFIG_SPACE; > > memory_region_init(&s->pci.address_space_root, > OBJECT(s), > -- > 2.17.1
As far as I know, the order issue is caused by nested device realization. In this case, realizing TYPE_DESIGNWARE_PCIE_HOST will also realize TYPE_DESIGNWARE_PCIE_ROOT(see designware_pcie_host_realize()). device_set_realized() is the function that realizing a device must go through, and this function first realizes the device by dc->realize() and then realizes the device's child bus by qbus_realize(). Whether there is any child bus of the device may depend on dc->realize(). The realization flow will be like a recursive call to device_set_realized(). More precisely, the flow in this case is: qdev_realize() --> ... --> FIRST device_set_realized() --> FIRST dc->realize() --> ... --> designware_pcie_host_realize() --> qdev_realize() --> ... --> SECOND device_set_realized() --> SECOND dc->realize() --> ... --> designware_pcie_root_realize() --> ...--> back to the SECOND device_set_realized() --> SECOND qbus_realize() the CHILD bus "dw-pcie" --> ... --> back to the FIRST device_set_realized() --> FIRST qbus_realize() the PARENT bus "pcie". I also found this patch <https://lists.gnu.org/archive/html/qemu-devel/2021-11/msg02162.html> that solves the same bus issue. Do you have any suggestions on the order of realization? Thanks! On Thu, Aug 10, 2023 at 5:24 AM Michael S. Tsirkin <mst@redhat.com> wrote: > On Wed, Aug 09, 2023 at 10:22:50AM +0000, Jason Chien wrote: > > In pcie_bus_realize(), a root bus is realized as a PCIe bus and a > non-root > > bus is realized as a PCIe bus if its parent bus is a PCIe bus. However, > > the child bus "dw-pcie" is realized before the parent bus "pcie" which is > > the root PCIe bus. Thus, the extended configuration space is not > accessible > > on "dw-pcie". The issue can be resolved by adding the > > PCI_BUS_EXTENDED_CONFIG_SPACE flag to "pcie" before "dw-pcie" is > realized. > > > > Signed-off-by: Jason Chien <jason.chien@sifive.com> > > I think we should fix the order of initialization rather than > hack around it. > > > --- > > hw/pci-host/designware.c | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/hw/pci-host/designware.c b/hw/pci-host/designware.c > > index 9e183caa48..388d252ee2 100644 > > --- a/hw/pci-host/designware.c > > +++ b/hw/pci-host/designware.c > > @@ -694,6 +694,7 @@ static void designware_pcie_host_realize(DeviceState > *dev, Error **errp) > > &s->pci.io, > > 0, 4, > > TYPE_PCIE_BUS); > > + pci->bus->flags |= PCI_BUS_EXTENDED_CONFIG_SPACE; > > > > memory_region_init(&s->pci.address_space_root, > > OBJECT(s), > > -- > > 2.17.1 > >
On Fri, Aug 11, 2023 at 01:22:08AM +0800, Jason Chien wrote: > As far as I know, the order issue is caused by nested device realization. In > this case, realizing TYPE_DESIGNWARE_PCIE_HOST will also > realize TYPE_DESIGNWARE_PCIE_ROOT(see designware_pcie_host_realize()). > device_set_realized() is the function that realizing a device must go through, > and this function first realizes the device by dc->realize() and then realizes > the device's child bus by qbus_realize(). Whether there is any child bus of the > device may depend on dc->realize(). The realization flow will be like a > recursive call to device_set_realized(). More precisely, the flow in this case > is: qdev_realize() --> ... --> FIRST device_set_realized() --> FIRST dc-> > realize() --> ... --> designware_pcie_host_realize() --> qdev_realize() --> ... > --> SECOND device_set_realized() --> SECOND dc->realize() --> ... --> > designware_pcie_root_realize() --> ...--> back to the SECOND > device_set_realized() --> SECOND qbus_realize() the CHILD bus "dw-pcie" --> ... > --> back to the FIRST device_set_realized() --> FIRST qbus_realize() the PARENT > bus "pcie". > > I also found this patch that solves the same bus issue. Which patch? > Do you have any suggestions on the order of realization? Thanks! I see. It's not easy to fix. Worth thinking about but I guess your patch is ok for now. > On Thu, Aug 10, 2023 at 5:24 AM Michael S. Tsirkin <mst@redhat.com> wrote: > > On Wed, Aug 09, 2023 at 10:22:50AM +0000, Jason Chien wrote: > > In pcie_bus_realize(), a root bus is realized as a PCIe bus and a > non-root > > bus is realized as a PCIe bus if its parent bus is a PCIe bus. However, > > the child bus "dw-pcie" is realized before the parent bus "pcie" which is > > the root PCIe bus. Thus, the extended configuration space is not > accessible > > on "dw-pcie". The issue can be resolved by adding the > > PCI_BUS_EXTENDED_CONFIG_SPACE flag to "pcie" before "dw-pcie" is > realized. > > > > Signed-off-by: Jason Chien <jason.chien@sifive.com> > > I think we should fix the order of initialization rather than > hack around it. > > > --- > > hw/pci-host/designware.c | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/hw/pci-host/designware.c b/hw/pci-host/designware.c > > index 9e183caa48..388d252ee2 100644 > > --- a/hw/pci-host/designware.c > > +++ b/hw/pci-host/designware.c > > @@ -694,6 +694,7 @@ static void designware_pcie_host_realize(DeviceState > *dev, Error **errp) > > &s->pci.io, > > 0, 4, > > TYPE_PCIE_BUS); > > + pci->bus->flags |= PCI_BUS_EXTENDED_CONFIG_SPACE; > > > > memory_region_init(&s->pci.address_space_root, > > OBJECT(s), > > -- > > 2.17.1 > >
the patch <https://lists.gnu.org/archive/html/qemu-devel/2021-11/msg02162.html> link: https://lists.gnu.org/archive/html/qemu-devel/2021-11/msg02162.html On Fri, Aug 11, 2023 at 1:44 AM Michael S. Tsirkin <mst@redhat.com> wrote: > On Fri, Aug 11, 2023 at 01:22:08AM +0800, Jason Chien wrote: > > As far as I know, the order issue is caused by nested device > realization. In > > this case, realizing TYPE_DESIGNWARE_PCIE_HOST will also > > realize TYPE_DESIGNWARE_PCIE_ROOT(see designware_pcie_host_realize()). > > device_set_realized() is the function that realizing a device must go > through, > > and this function first realizes the device by dc->realize() and then > realizes > > the device's child bus by qbus_realize(). Whether there is any child bus > of the > > device may depend on dc->realize(). The realization flow will be like a > > recursive call to device_set_realized(). More precisely, the flow in > this case > > is: qdev_realize() --> ... --> FIRST device_set_realized() --> FIRST dc-> > > realize() --> ... --> designware_pcie_host_realize() --> qdev_realize() > --> ... > > --> SECOND device_set_realized() --> SECOND dc->realize() --> ... --> > > designware_pcie_root_realize() --> ...--> back to the SECOND > > device_set_realized() --> SECOND qbus_realize() the CHILD bus "dw-pcie" > --> ... > > --> back to the FIRST device_set_realized() --> FIRST qbus_realize() the > PARENT > > bus "pcie". > > > > I also found this patch that solves the same bus issue. > > Which patch? > > > Do you have any suggestions on the order of realization? Thanks! > > > I see. It's not easy to fix. Worth thinking about but I guess your > patch is ok for now. > > > On Thu, Aug 10, 2023 at 5:24 AM Michael S. Tsirkin <mst@redhat.com> > wrote: > > > > On Wed, Aug 09, 2023 at 10:22:50AM +0000, Jason Chien wrote: > > > In pcie_bus_realize(), a root bus is realized as a PCIe bus and a > > non-root > > > bus is realized as a PCIe bus if its parent bus is a PCIe bus. > However, > > > the child bus "dw-pcie" is realized before the parent bus "pcie" > which is > > > the root PCIe bus. Thus, the extended configuration space is not > > accessible > > > on "dw-pcie". The issue can be resolved by adding the > > > PCI_BUS_EXTENDED_CONFIG_SPACE flag to "pcie" before "dw-pcie" is > > realized. > > > > > > Signed-off-by: Jason Chien <jason.chien@sifive.com> > > > > I think we should fix the order of initialization rather than > > hack around it. > > > > > --- > > > hw/pci-host/designware.c | 1 + > > > 1 file changed, 1 insertion(+) > > > > > > diff --git a/hw/pci-host/designware.c b/hw/pci-host/designware.c > > > index 9e183caa48..388d252ee2 100644 > > > --- a/hw/pci-host/designware.c > > > +++ b/hw/pci-host/designware.c > > > @@ -694,6 +694,7 @@ static void > designware_pcie_host_realize(DeviceState > > *dev, Error **errp) > > > &s->pci.io, > > > 0, 4, > > > TYPE_PCIE_BUS); > > > + pci->bus->flags |= PCI_BUS_EXTENDED_CONFIG_SPACE; > > > > > > memory_region_init(&s->pci.address_space_root, > > > OBJECT(s), > > > -- > > > 2.17.1 > > > > > >
On Wed, Aug 09, 2023 at 10:22:50AM +0000, Jason Chien wrote: > In pcie_bus_realize(), a root bus is realized as a PCIe bus and a non-root > bus is realized as a PCIe bus if its parent bus is a PCIe bus. However, > the child bus "dw-pcie" is realized before the parent bus "pcie" which is > the root PCIe bus. Thus, the extended configuration space is not accessible > on "dw-pcie". The issue can be resolved by adding the > PCI_BUS_EXTENDED_CONFIG_SPACE flag to "pcie" before "dw-pcie" is realized. > > Signed-off-by: Jason Chien <jason.chien@sifive.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> I'm not planning another pull before release, hopefully another maintainer can pick it up? Peter? > --- > hw/pci-host/designware.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/hw/pci-host/designware.c b/hw/pci-host/designware.c > index 9e183caa48..388d252ee2 100644 > --- a/hw/pci-host/designware.c > +++ b/hw/pci-host/designware.c > @@ -694,6 +694,7 @@ static void designware_pcie_host_realize(DeviceState *dev, Error **errp) > &s->pci.io, > 0, 4, > TYPE_PCIE_BUS); > + pci->bus->flags |= PCI_BUS_EXTENDED_CONFIG_SPACE; > > memory_region_init(&s->pci.address_space_root, > OBJECT(s), > -- > 2.17.1
Reviewed-by: Frank Chang <frank.chang@sifive.com> On Wed, Aug 9, 2023 at 6:23 PM Jason Chien <jason.chien@sifive.com> wrote: > In pcie_bus_realize(), a root bus is realized as a PCIe bus and a non-root > bus is realized as a PCIe bus if its parent bus is a PCIe bus. However, > the child bus "dw-pcie" is realized before the parent bus "pcie" which is > the root PCIe bus. Thus, the extended configuration space is not accessible > on "dw-pcie". The issue can be resolved by adding the > PCI_BUS_EXTENDED_CONFIG_SPACE flag to "pcie" before "dw-pcie" is realized. > > Signed-off-by: Jason Chien <jason.chien@sifive.com> > --- > hw/pci-host/designware.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/hw/pci-host/designware.c b/hw/pci-host/designware.c > index 9e183caa48..388d252ee2 100644 > --- a/hw/pci-host/designware.c > +++ b/hw/pci-host/designware.c > @@ -694,6 +694,7 @@ static void designware_pcie_host_realize(DeviceState > *dev, Error **errp) > &s->pci.io, > 0, 4, > TYPE_PCIE_BUS); > + pci->bus->flags |= PCI_BUS_EXTENDED_CONFIG_SPACE; > > memory_region_init(&s->pci.address_space_root, > OBJECT(s), > -- > 2.17.1 > > >
On Thu, 10 Aug 2023 at 18:51, Michael S. Tsirkin <mst@redhat.com> wrote: > > On Wed, Aug 09, 2023 at 10:22:50AM +0000, Jason Chien wrote: > > In pcie_bus_realize(), a root bus is realized as a PCIe bus and a non-root > > bus is realized as a PCIe bus if its parent bus is a PCIe bus. However, > > the child bus "dw-pcie" is realized before the parent bus "pcie" which is > > the root PCIe bus. Thus, the extended configuration space is not accessible > > on "dw-pcie". The issue can be resolved by adding the > > PCI_BUS_EXTENDED_CONFIG_SPACE flag to "pcie" before "dw-pcie" is realized. > > > > Signed-off-by: Jason Chien <jason.chien@sifive.com> > > > Acked-by: Michael S. Tsirkin <mst@redhat.com> > > I'm not planning another pull before release, hopefully > another maintainer can pick it up? Peter? At the moment I don't have anything intended for 8.1 either, so whoever of us does it it would be a 1-patch pullreq... -- PMM
On Fri, 11 Aug 2023 at 10:55, Peter Maydell <peter.maydell@linaro.org> wrote: > > On Thu, 10 Aug 2023 at 18:51, Michael S. Tsirkin <mst@redhat.com> wrote: > > > > On Wed, Aug 09, 2023 at 10:22:50AM +0000, Jason Chien wrote: > > > In pcie_bus_realize(), a root bus is realized as a PCIe bus and a non-root > > > bus is realized as a PCIe bus if its parent bus is a PCIe bus. However, > > > the child bus "dw-pcie" is realized before the parent bus "pcie" which is > > > the root PCIe bus. Thus, the extended configuration space is not accessible > > > on "dw-pcie". The issue can be resolved by adding the > > > PCI_BUS_EXTENDED_CONFIG_SPACE flag to "pcie" before "dw-pcie" is realized. > > > > > > Signed-off-by: Jason Chien <jason.chien@sifive.com> > > > > > > Acked-by: Michael S. Tsirkin <mst@redhat.com> > > > > I'm not planning another pull before release, hopefully > > another maintainer can pick it up? Peter? > > At the moment I don't have anything intended for 8.1 either, > so whoever of us does it it would be a 1-patch pullreq... Also, at this stage in the release cycle it's always worth asking: is this a regression, or was this bug also in 8.0? thanks -- PMM
This bug was also in 8.0. Jason On Fri, Aug 11, 2023 at 10:07 PM Peter Maydell <peter.maydell@linaro.org> wrote: > On Fri, 11 Aug 2023 at 10:55, Peter Maydell <peter.maydell@linaro.org> > wrote: > > > > On Thu, 10 Aug 2023 at 18:51, Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > On Wed, Aug 09, 2023 at 10:22:50AM +0000, Jason Chien wrote: > > > > In pcie_bus_realize(), a root bus is realized as a PCIe bus and a > non-root > > > > bus is realized as a PCIe bus if its parent bus is a PCIe bus. > However, > > > > the child bus "dw-pcie" is realized before the parent bus "pcie" > which is > > > > the root PCIe bus. Thus, the extended configuration space is not > accessible > > > > on "dw-pcie". The issue can be resolved by adding the > > > > PCI_BUS_EXTENDED_CONFIG_SPACE flag to "pcie" before "dw-pcie" is > realized. > > > > > > > > Signed-off-by: Jason Chien <jason.chien@sifive.com> > > > > > > > > > Acked-by: Michael S. Tsirkin <mst@redhat.com> > > > > > > I'm not planning another pull before release, hopefully > > > another maintainer can pick it up? Peter? > > > > At the moment I don't have anything intended for 8.1 either, > > so whoever of us does it it would be a 1-patch pullreq... > > Also, at this stage in the release cycle it's always worth > asking: is this a regression, or was this bug also in 8.0? > > thanks > -- PMM >
diff --git a/hw/pci-host/designware.c b/hw/pci-host/designware.c index 9e183caa48..388d252ee2 100644 --- a/hw/pci-host/designware.c +++ b/hw/pci-host/designware.c @@ -694,6 +694,7 @@ static void designware_pcie_host_realize(DeviceState *dev, Error **errp) &s->pci.io, 0, 4, TYPE_PCIE_BUS); + pci->bus->flags |= PCI_BUS_EXTENDED_CONFIG_SPACE; memory_region_init(&s->pci.address_space_root, OBJECT(s),
In pcie_bus_realize(), a root bus is realized as a PCIe bus and a non-root bus is realized as a PCIe bus if its parent bus is a PCIe bus. However, the child bus "dw-pcie" is realized before the parent bus "pcie" which is the root PCIe bus. Thus, the extended configuration space is not accessible on "dw-pcie". The issue can be resolved by adding the PCI_BUS_EXTENDED_CONFIG_SPACE flag to "pcie" before "dw-pcie" is realized. Signed-off-by: Jason Chien <jason.chien@sifive.com> --- hw/pci-host/designware.c | 1 + 1 file changed, 1 insertion(+)