| Message ID | 1420736772-11088-3-git-send-email-marc.zyngier@arm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Delegated to: | Bjorn Helgaas |
| Headers | show |
> static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent, > struct pci_dev *bridge, int busnr) > { > @@ -713,6 +727,7 @@ static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent, > bridge->subordinate = child; > > add_dev: > + pci_set_bus_msi_domain(child); > ret = device_register(&child->dev); > WARN_ON(ret < 0); > > @@ -1507,6 +1522,17 @@ static void pci_init_capabilities(struct pci_dev *dev) > pci_enable_acs(dev); > } > > +static void pci_set_msi_domain(struct pci_dev *dev) > +{ > + /* > + * If no domain has been set through the pcibios callback, > + * inherit the default from the bus device. > + */ > + if (!dev_get_msi_domain(&dev->dev)) > + dev_set_msi_domain(&dev->dev, > + dev_get_msi_domain(&dev->bus->dev)); > +} Hi Marc, now we have two ways to associate the pci_dev and msi_domain, right ? 1. associate pci_dev and msi_domain in pcibios_add_device() like x86. 2. Inherit msi_domain from pci_dev->bus. My question is if all pci devices inherit msi_domain from the pci_bus, so all pci devices under same pci host bridge have the same msi_domain assigned by weak pcibios_set_phb_msi_domain(). So why not save the pci host bridge specific msi_domain in pci_host_bridge. Then pci devices could inherit the msi_domain from its pci host bridge directly, no need to involve pci bus in the assignment. If I misunderstood, please let me know, :) Thanks! Yijing. > + > void pci_device_add(struct pci_dev *dev, struct pci_bus *bus) > { > int ret; > @@ -1547,6 +1573,9 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus) > ret = pcibios_add_device(dev); > WARN_ON(ret < 0); > > + /* Setup MSI irq domain */ > + pci_set_msi_domain(dev); > + > /* Notifier could use PCI capabilities */ > dev->match_driver = false; > ret = device_add(&dev->dev); > @@ -1937,6 +1966,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus, > b->bridge = get_device(&bridge->dev); > device_enable_async_suspend(b->bridge); > pci_set_bus_of_node(b); > + pci_set_bus_msi_domain(b); > > if (!parent) > set_dev_node(b->bridge, pcibus_to_node(b)); > diff --git a/include/linux/pci.h b/include/linux/pci.h > index 360a966..13c65ab 100644 > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -1640,6 +1640,7 @@ int pcibios_set_pcie_reset_state(struct pci_dev *dev, > int pcibios_add_device(struct pci_dev *dev); > void pcibios_release_device(struct pci_dev *dev); > void pcibios_penalize_isa_irq(int irq, int active); > +void pcibios_set_phb_msi_domain(struct pci_bus *bus); > > #ifdef CONFIG_HIBERNATE_CALLBACKS > extern struct dev_pm_ops pcibios_pm_ops; >
On 13/01/15 12:34, Yijing Wang wrote: >> static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent, >> struct pci_dev *bridge, int busnr) >> { >> @@ -713,6 +727,7 @@ static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent, >> bridge->subordinate = child; >> >> add_dev: >> + pci_set_bus_msi_domain(child); >> ret = device_register(&child->dev); >> WARN_ON(ret < 0); >> >> @@ -1507,6 +1522,17 @@ static void pci_init_capabilities(struct pci_dev *dev) >> pci_enable_acs(dev); >> } >> >> +static void pci_set_msi_domain(struct pci_dev *dev) >> +{ >> + /* >> + * If no domain has been set through the pcibios callback, >> + * inherit the default from the bus device. >> + */ >> + if (!dev_get_msi_domain(&dev->dev)) >> + dev_set_msi_domain(&dev->dev, >> + dev_get_msi_domain(&dev->bus->dev)); >> +} > > Hi Marc, now we have two ways to associate the pci_dev and msi_domain, right ? > > 1. associate pci_dev and msi_domain in pcibios_add_device() like x86. > > 2. Inherit msi_domain from pci_dev->bus. > > My question is if all pci devices inherit msi_domain from the pci_bus, > so all pci devices under same pci host bridge have the same msi_domain assigned by > weak pcibios_set_phb_msi_domain(). So why not save the pci host bridge specific > msi_domain in pci_host_bridge. Then pci devices could inherit the msi_domain from > its pci host bridge directly, no need to involve pci bus in the assignment. But then, you would end-up maintaining another msi_domain field inside the pci_host bridge structure. What do you gain by doing so? With this series, msi_domain has the nice property of always being tied to a device (and struct pci_bus always has a device). We always have phb->bus->dev.msi_domain within reach, and architecture code can decide to override it on a per-device basis. What else do you need? What am I missing from your proposal? Thanks, M.
>>> +static void pci_set_msi_domain(struct pci_dev *dev) >>> +{ >>> + /* >>> + * If no domain has been set through the pcibios callback, >>> + * inherit the default from the bus device. >>> + */ >>> + if (!dev_get_msi_domain(&dev->dev)) >>> + dev_set_msi_domain(&dev->dev, >>> + dev_get_msi_domain(&dev->bus->dev)); >>> +} >> >> Hi Marc, now we have two ways to associate the pci_dev and msi_domain, right ? >> >> 1. associate pci_dev and msi_domain in pcibios_add_device() like x86. >> >> 2. Inherit msi_domain from pci_dev->bus. >> >> My question is if all pci devices inherit msi_domain from the pci_bus, >> so all pci devices under same pci host bridge have the same msi_domain assigned by >> weak pcibios_set_phb_msi_domain(). So why not save the pci host bridge specific >> msi_domain in pci_host_bridge. Then pci devices could inherit the msi_domain from >> its pci host bridge directly, no need to involve pci bus in the assignment. > > But then, you would end-up maintaining another msi_domain field inside > the pci_host bridge structure. What do you gain by doing so? My original thought is holding msi_domain field inside the pci_host_bridge is more simple than every bus maintaining the msi_domain, but this proposal has a disadvantage that sometimes we must setup for every device. I checked x86 DMAR code, and found most DMAR would report PCIe root port device associating the msi_domain, not the EP device. So pcibios_add_device could only associate these bridge device msi_domain, and its children devices will propagate from their parent bus(get msi_domain from its bridge). So now I agree your idea, please forgive my nagging :) Thanks! Yijing. > > With this series, msi_domain has the nice property of always being tied > to a device (and struct pci_bus always has a device). We always have > phb->bus->dev.msi_domain within reach, and architecture code can decide > to override it on a per-device basis. > > What else do you need? What am I missing from your proposal? > > Thanks, > > M. >
On 2015/1/9 1:06, Marc Zyngier wrote: > In order to be able to populate the device msi_domain field, > add the necesary hooks to propagate the PHB msi_domain across > secondary busses to devices. > > So far, nobody populates the initial msi_domain. Acked-by: Yijing Wang <wangyijing@huawei.com> > > Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> > --- > drivers/pci/probe.c | 30 ++++++++++++++++++++++++++++++ > include/linux/pci.h | 1 + > 2 files changed, 31 insertions(+) > > diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c > index 23212f8..977c079 100644 > --- a/drivers/pci/probe.c > +++ b/drivers/pci/probe.c > @@ -660,6 +660,20 @@ static void pci_set_bus_speed(struct pci_bus *bus) > } > } > > +void __weak pcibios_set_phb_msi_domain(struct pci_bus *bus) > +{ > +} > + > +static void pci_set_bus_msi_domain(struct pci_bus *bus) > +{ > + struct pci_dev *bridge = bus->self; > + > + if (!bridge) > + pcibios_set_phb_msi_domain(bus); > + else > + dev_set_msi_domain(&bus->dev, dev_get_msi_domain(&bridge->dev)); > +} > + > static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent, > struct pci_dev *bridge, int busnr) > { > @@ -713,6 +727,7 @@ static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent, > bridge->subordinate = child; > > add_dev: > + pci_set_bus_msi_domain(child); > ret = device_register(&child->dev); > WARN_ON(ret < 0); > > @@ -1507,6 +1522,17 @@ static void pci_init_capabilities(struct pci_dev *dev) > pci_enable_acs(dev); > } > > +static void pci_set_msi_domain(struct pci_dev *dev) > +{ > + /* > + * If no domain has been set through the pcibios callback, > + * inherit the default from the bus device. > + */ > + if (!dev_get_msi_domain(&dev->dev)) > + dev_set_msi_domain(&dev->dev, > + dev_get_msi_domain(&dev->bus->dev)); > +} > + > void pci_device_add(struct pci_dev *dev, struct pci_bus *bus) > { > int ret; > @@ -1547,6 +1573,9 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus) > ret = pcibios_add_device(dev); > WARN_ON(ret < 0); > > + /* Setup MSI irq domain */ > + pci_set_msi_domain(dev); > + > /* Notifier could use PCI capabilities */ > dev->match_driver = false; > ret = device_add(&dev->dev); > @@ -1937,6 +1966,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus, > b->bridge = get_device(&bridge->dev); > device_enable_async_suspend(b->bridge); > pci_set_bus_of_node(b); > + pci_set_bus_msi_domain(b); > > if (!parent) > set_dev_node(b->bridge, pcibus_to_node(b)); > diff --git a/include/linux/pci.h b/include/linux/pci.h > index 360a966..13c65ab 100644 > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -1640,6 +1640,7 @@ int pcibios_set_pcie_reset_state(struct pci_dev *dev, > int pcibios_add_device(struct pci_dev *dev); > void pcibios_release_device(struct pci_dev *dev); > void pcibios_penalize_isa_irq(int irq, int active); > +void pcibios_set_phb_msi_domain(struct pci_bus *bus); > > #ifdef CONFIG_HIBERNATE_CALLBACKS > extern struct dev_pm_ops pcibios_pm_ops; >
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 23212f8..977c079 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -660,6 +660,20 @@ static void pci_set_bus_speed(struct pci_bus *bus) } } +void __weak pcibios_set_phb_msi_domain(struct pci_bus *bus) +{ +} + +static void pci_set_bus_msi_domain(struct pci_bus *bus) +{ + struct pci_dev *bridge = bus->self; + + if (!bridge) + pcibios_set_phb_msi_domain(bus); + else + dev_set_msi_domain(&bus->dev, dev_get_msi_domain(&bridge->dev)); +} + static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent, struct pci_dev *bridge, int busnr) { @@ -713,6 +727,7 @@ static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent, bridge->subordinate = child; add_dev: + pci_set_bus_msi_domain(child); ret = device_register(&child->dev); WARN_ON(ret < 0); @@ -1507,6 +1522,17 @@ static void pci_init_capabilities(struct pci_dev *dev) pci_enable_acs(dev); } +static void pci_set_msi_domain(struct pci_dev *dev) +{ + /* + * If no domain has been set through the pcibios callback, + * inherit the default from the bus device. + */ + if (!dev_get_msi_domain(&dev->dev)) + dev_set_msi_domain(&dev->dev, + dev_get_msi_domain(&dev->bus->dev)); +} + void pci_device_add(struct pci_dev *dev, struct pci_bus *bus) { int ret; @@ -1547,6 +1573,9 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus) ret = pcibios_add_device(dev); WARN_ON(ret < 0); + /* Setup MSI irq domain */ + pci_set_msi_domain(dev); + /* Notifier could use PCI capabilities */ dev->match_driver = false; ret = device_add(&dev->dev); @@ -1937,6 +1966,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus, b->bridge = get_device(&bridge->dev); device_enable_async_suspend(b->bridge); pci_set_bus_of_node(b); + pci_set_bus_msi_domain(b); if (!parent) set_dev_node(b->bridge, pcibus_to_node(b)); diff --git a/include/linux/pci.h b/include/linux/pci.h index 360a966..13c65ab 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1640,6 +1640,7 @@ int pcibios_set_pcie_reset_state(struct pci_dev *dev, int pcibios_add_device(struct pci_dev *dev); void pcibios_release_device(struct pci_dev *dev); void pcibios_penalize_isa_irq(int irq, int active); +void pcibios_set_phb_msi_domain(struct pci_bus *bus); #ifdef CONFIG_HIBERNATE_CALLBACKS extern struct dev_pm_ops pcibios_pm_ops;
In order to be able to populate the device msi_domain field, add the necesary hooks to propagate the PHB msi_domain across secondary busses to devices. So far, nobody populates the initial msi_domain. Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> --- drivers/pci/probe.c | 30 ++++++++++++++++++++++++++++++ include/linux/pci.h | 1 + 2 files changed, 31 insertions(+)