| Message ID | 20240923-pcie_ep_range-v2-1-78d2ea434d9f@nxp.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | PCI: ep: dwc/imx6: Add bus address support for PCI endpoint devices | expand |
On Mon, Sep 23, 2024 at 02:59:19PM -0400, Frank Li wrote: > Endpoint Root complex > ┌───────┐ ┌─────────┐ > ┌─────┐ │ EP │ │ │ ┌─────┐ > │ │ │ Ctrl │ │ │ │ CPU │ > │ DDR │ │ │ │ ┌────┐ │ └──┬──┘ > │ │◄──────┼─ATU ◄─┼────────┼─┤BarN│◄─┼─────────┘ > │ │ │ │ │ └────┘ │ Outbound Transfer > └─────┘ │ │ │ │ > │ │ │ │ > │ │ │ │ > │ │ │ │ Inbound Transfer > │ │ │ │ ┌──▼──┐ > ┌───────┐ │ │ │ ┌───────┼─────►│DDR │ > │ │ outbound Transfer* │ │ │ └─────┘ > ┌─────┐ │ Bus ┼─────►│ ATU ─┬────────┼─┘ │ > │ │ │ Fabric│Bus │ │ PCI Addr │ > │ CPU ├───►│ │Addr │ │ 0xA000_0000 │ > │ │CPU │ │0x8000_0000 │ │ │ > └─────┘Addr└───────┘ │ │ │ │ > 0x7000_0000 └───────┘ └─────────┘ > > Add `bus_addr_base` to configure the outbound window address for CPU write. > The bus fabric generally passes the same address to the PCIe EP controller, > but some bus fabrics convert the address before sending it to the PCIe EP > controller. > > Above diagram, CPU write data to outbound windows address 0x7000_0000, > Bus fabric convert it to 0x8000_0000. ATU should use bus address > 0x8000_0000 as input address and convert to PCI address 0xA000_0000. > > Previously, `cpu_addr_fixup()` was used to handle address conversion. Now, > the device tree provides this information, preferring a common method. > > bus@5f000000 { > compatible = "simple-bus"; > ranges = <0x5f000000 0x0 0x5f000000 0x21000000>, > <0x80000000 0x0 0x70000000 0x10000000>; > > pcie-ep@5f010000 { > reg = <0x5f010000 0x00010000>, > <0x80000000 0x10000000>; > reg-names = "dbi", "addr_space"; > ... > }; > ... > }; > > 'ranges' in bus@5f000000 descript how address convert from CPU address > to bus address. > > Use `of_property_read_reg()` to obtain the bus address and set it to the > ATU correctly, eliminating the need for vendor-specific cpu_addr_fixup(). > > Signed-off-by: Frank Li <Frank.Li@nxp.com> > --- > drivers/pci/controller/dwc/pcie-designware-ep.c | 12 +++++++++++- > drivers/pci/controller/dwc/pcie-designware.h | 1 + > 2 files changed, 12 insertions(+), 1 deletion(-) > > diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c > index 43ba5c6738df1..51eefdcb1b293 100644 > --- a/drivers/pci/controller/dwc/pcie-designware-ep.c > +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c > @@ -9,6 +9,7 @@ > #include <linux/align.h> > #include <linux/bitfield.h> > #include <linux/of.h> > +#include <linux/of_address.h> > #include <linux/platform_device.h> > > #include "pcie-designware.h" > @@ -294,7 +295,7 @@ static int dw_pcie_ep_map_addr(struct pci_epc *epc, u8 func_no, u8 vfunc_no, > > atu.func_no = func_no; > atu.type = PCIE_ATU_TYPE_MEM; > - atu.cpu_addr = addr; > + atu.cpu_addr = addr - ep->phys_base + ep->bus_addr_base; If you convert the address here, aren't he drivers with cpu_addr_fixup() will be broken? You should only update the address if the callback is not available. - Mani > atu.pci_addr = pci_addr; > atu.size = size; > ret = dw_pcie_ep_outbound_atu(ep, &atu); > @@ -861,6 +862,7 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) > struct device *dev = pci->dev; > struct platform_device *pdev = to_platform_device(dev); > struct device_node *np = dev->of_node; > + int index; > > INIT_LIST_HEAD(&ep->func_list); > > @@ -873,6 +875,14 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) > return -EINVAL; > > ep->phys_base = res->start; > + ep->bus_addr_base = ep->phys_base; > + > + index = of_property_match_string(np, "reg-names", "addr_space"); > + if (index < 0) > + return -EINVAL; > + > + of_property_read_reg(np, index, &ep->bus_addr_base, NULL); > + > ep->addr_size = resource_size(res); > > if (ep->ops->pre_init) > diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h > index 347ab74ac35aa..c189781524fb8 100644 > --- a/drivers/pci/controller/dwc/pcie-designware.h > +++ b/drivers/pci/controller/dwc/pcie-designware.h > @@ -410,6 +410,7 @@ struct dw_pcie_ep { > struct list_head func_list; > const struct dw_pcie_ep_ops *ops; > phys_addr_t phys_base; > + phys_addr_t bus_addr_base; > size_t addr_size; > size_t page_size; > u8 bar_to_atu[PCI_STD_NUM_BARS]; > > -- > 2.34.1 >
On Wed, Oct 16, 2024 at 11:38:49PM +0530, Manivannan Sadhasivam wrote: > On Mon, Sep 23, 2024 at 02:59:19PM -0400, Frank Li wrote: > > Endpoint Root complex > > ┌───────┐ ┌─────────┐ > > ┌─────┐ │ EP │ │ │ ┌─────┐ > > │ │ │ Ctrl │ │ │ │ CPU │ > > │ DDR │ │ │ │ ┌────┐ │ └──┬──┘ > > │ │◄──────┼─ATU ◄─┼────────┼─┤BarN│◄─┼─────────┘ > > │ │ │ │ │ └────┘ │ Outbound Transfer > > └─────┘ │ │ │ │ > > │ │ │ │ > > │ │ │ │ > > │ │ │ │ Inbound Transfer > > │ │ │ │ ┌──▼──┐ > > ┌───────┐ │ │ │ ┌───────┼─────►│DDR │ > > │ │ outbound Transfer* │ │ │ └─────┘ > > ┌─────┐ │ Bus ┼─────►│ ATU ─┬────────┼─┘ │ > > │ │ │ Fabric│Bus │ │ PCI Addr │ > > │ CPU ├───►│ │Addr │ │ 0xA000_0000 │ > > │ │CPU │ │0x8000_0000 │ │ │ > > └─────┘Addr└───────┘ │ │ │ │ > > 0x7000_0000 └───────┘ └─────────┘ > > > > Add `bus_addr_base` to configure the outbound window address for CPU write. > > The bus fabric generally passes the same address to the PCIe EP controller, > > but some bus fabrics convert the address before sending it to the PCIe EP > > controller. > > > > Above diagram, CPU write data to outbound windows address 0x7000_0000, > > Bus fabric convert it to 0x8000_0000. ATU should use bus address > > 0x8000_0000 as input address and convert to PCI address 0xA000_0000. > > > > Previously, `cpu_addr_fixup()` was used to handle address conversion. Now, > > the device tree provides this information, preferring a common method. > > > > bus@5f000000 { > > compatible = "simple-bus"; > > ranges = <0x5f000000 0x0 0x5f000000 0x21000000>, > > <0x80000000 0x0 0x70000000 0x10000000>; > > > > pcie-ep@5f010000 { > > reg = <0x5f010000 0x00010000>, > > <0x80000000 0x10000000>; > > reg-names = "dbi", "addr_space"; > > ... > > }; > > ... > > }; > > > > 'ranges' in bus@5f000000 descript how address convert from CPU address > > to bus address. > > > > Use `of_property_read_reg()` to obtain the bus address and set it to the > > ATU correctly, eliminating the need for vendor-specific cpu_addr_fixup(). > > > > Signed-off-by: Frank Li <Frank.Li@nxp.com> > > --- > > drivers/pci/controller/dwc/pcie-designware-ep.c | 12 +++++++++++- > > drivers/pci/controller/dwc/pcie-designware.h | 1 + > > 2 files changed, 12 insertions(+), 1 deletion(-) > > > > diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c > > index 43ba5c6738df1..51eefdcb1b293 100644 > > --- a/drivers/pci/controller/dwc/pcie-designware-ep.c > > +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c > > @@ -9,6 +9,7 @@ > > #include <linux/align.h> > > #include <linux/bitfield.h> > > #include <linux/of.h> > > +#include <linux/of_address.h> > > #include <linux/platform_device.h> > > > > #include "pcie-designware.h" > > @@ -294,7 +295,7 @@ static int dw_pcie_ep_map_addr(struct pci_epc *epc, u8 func_no, u8 vfunc_no, > > > > atu.func_no = func_no; > > atu.type = PCIE_ATU_TYPE_MEM; > > - atu.cpu_addr = addr; > > + atu.cpu_addr = addr - ep->phys_base + ep->bus_addr_base; > > If you convert the address here, aren't he drivers with cpu_addr_fixup() will be > broken? You should only update the address if the callback is not available. Supposed not, ep->phys_base and ep->bus_addr_base should be same when use cpu_addr_fixup(). but I warry about some old dts have not reflect hardware bus address translate correct. How about use below method: https://lore.kernel.org/imx/20241008-pci_fixup_addr-v4-2-25e5200657bc@nxp.com/ Involve a varible 'using_dtbus_info' to controller it. Frank > > - Mani > > > atu.pci_addr = pci_addr; > > atu.size = size; > > ret = dw_pcie_ep_outbound_atu(ep, &atu); > > @@ -861,6 +862,7 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) > > struct device *dev = pci->dev; > > struct platform_device *pdev = to_platform_device(dev); > > struct device_node *np = dev->of_node; > > + int index; > > > > INIT_LIST_HEAD(&ep->func_list); > > > > @@ -873,6 +875,14 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) > > return -EINVAL; > > > > ep->phys_base = res->start; > > + ep->bus_addr_base = ep->phys_base; > > + > > + index = of_property_match_string(np, "reg-names", "addr_space"); > > + if (index < 0) > > + return -EINVAL; > > + > > + of_property_read_reg(np, index, &ep->bus_addr_base, NULL); > > + > > ep->addr_size = resource_size(res); > > > > if (ep->ops->pre_init) > > diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h > > index 347ab74ac35aa..c189781524fb8 100644 > > --- a/drivers/pci/controller/dwc/pcie-designware.h > > +++ b/drivers/pci/controller/dwc/pcie-designware.h > > @@ -410,6 +410,7 @@ struct dw_pcie_ep { > > struct list_head func_list; > > const struct dw_pcie_ep_ops *ops; > > phys_addr_t phys_base; > > + phys_addr_t bus_addr_base; > > size_t addr_size; > > size_t page_size; > > u8 bar_to_atu[PCI_STD_NUM_BARS]; > > > > -- > > 2.34.1 > > > > -- > மணிவண்ணன் சதாசிவம்
diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c index 43ba5c6738df1..51eefdcb1b293 100644 --- a/drivers/pci/controller/dwc/pcie-designware-ep.c +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c @@ -9,6 +9,7 @@ #include <linux/align.h> #include <linux/bitfield.h> #include <linux/of.h> +#include <linux/of_address.h> #include <linux/platform_device.h> #include "pcie-designware.h" @@ -294,7 +295,7 @@ static int dw_pcie_ep_map_addr(struct pci_epc *epc, u8 func_no, u8 vfunc_no, atu.func_no = func_no; atu.type = PCIE_ATU_TYPE_MEM; - atu.cpu_addr = addr; + atu.cpu_addr = addr - ep->phys_base + ep->bus_addr_base; atu.pci_addr = pci_addr; atu.size = size; ret = dw_pcie_ep_outbound_atu(ep, &atu); @@ -861,6 +862,7 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) struct device *dev = pci->dev; struct platform_device *pdev = to_platform_device(dev); struct device_node *np = dev->of_node; + int index; INIT_LIST_HEAD(&ep->func_list); @@ -873,6 +875,14 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) return -EINVAL; ep->phys_base = res->start; + ep->bus_addr_base = ep->phys_base; + + index = of_property_match_string(np, "reg-names", "addr_space"); + if (index < 0) + return -EINVAL; + + of_property_read_reg(np, index, &ep->bus_addr_base, NULL); + ep->addr_size = resource_size(res); if (ep->ops->pre_init) diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h index 347ab74ac35aa..c189781524fb8 100644 --- a/drivers/pci/controller/dwc/pcie-designware.h +++ b/drivers/pci/controller/dwc/pcie-designware.h @@ -410,6 +410,7 @@ struct dw_pcie_ep { struct list_head func_list; const struct dw_pcie_ep_ops *ops; phys_addr_t phys_base; + phys_addr_t bus_addr_base; size_t addr_size; size_t page_size; u8 bar_to_atu[PCI_STD_NUM_BARS];
Endpoint Root complex ┌───────┐ ┌─────────┐ ┌─────┐ │ EP │ │ │ ┌─────┐ │ │ │ Ctrl │ │ │ │ CPU │ │ DDR │ │ │ │ ┌────┐ │ └──┬──┘ │ │◄──────┼─ATU ◄─┼────────┼─┤BarN│◄─┼─────────┘ │ │ │ │ │ └────┘ │ Outbound Transfer └─────┘ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ Inbound Transfer │ │ │ │ ┌──▼──┐ ┌───────┐ │ │ │ ┌───────┼─────►│DDR │ │ │ outbound Transfer* │ │ │ └─────┘ ┌─────┐ │ Bus ┼─────►│ ATU ─┬────────┼─┘ │ │ │ │ Fabric│Bus │ │ PCI Addr │ │ CPU ├───►│ │Addr │ │ 0xA000_0000 │ │ │CPU │ │0x8000_0000 │ │ │ └─────┘Addr└───────┘ │ │ │ │ 0x7000_0000 └───────┘ └─────────┘ Add `bus_addr_base` to configure the outbound window address for CPU write. The bus fabric generally passes the same address to the PCIe EP controller, but some bus fabrics convert the address before sending it to the PCIe EP controller. Above diagram, CPU write data to outbound windows address 0x7000_0000, Bus fabric convert it to 0x8000_0000. ATU should use bus address 0x8000_0000 as input address and convert to PCI address 0xA000_0000. Previously, `cpu_addr_fixup()` was used to handle address conversion. Now, the device tree provides this information, preferring a common method. bus@5f000000 { compatible = "simple-bus"; ranges = <0x5f000000 0x0 0x5f000000 0x21000000>, <0x80000000 0x0 0x70000000 0x10000000>; pcie-ep@5f010000 { reg = <0x5f010000 0x00010000>, <0x80000000 0x10000000>; reg-names = "dbi", "addr_space"; ... }; ... }; 'ranges' in bus@5f000000 descript how address convert from CPU address to bus address. Use `of_property_read_reg()` to obtain the bus address and set it to the ATU correctly, eliminating the need for vendor-specific cpu_addr_fixup(). Signed-off-by: Frank Li <Frank.Li@nxp.com> --- drivers/pci/controller/dwc/pcie-designware-ep.c | 12 +++++++++++- drivers/pci/controller/dwc/pcie-designware.h | 1 + 2 files changed, 12 insertions(+), 1 deletion(-)