| Message ID | 20240919-pcie_ep_range-v1-0-b3e9d62780b7@nxp.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | PCI-EP: Add 'ranges' support for PCI endpoint devices | expand |
On Thu, Sep 19, 2024 at 5:03 PM Frank Li <Frank.Li@nxp.com> wrote: > > The PCI bus device tree supports 'ranges' properties that indicate > how to convert PCI addresses to CPU addresses. Many PCI controllers > are dual-role controllers, supporting both Root Complex (RC) and > Endpoint (EP) modes. The EP side also needs similar information for > proper address translation. > > This commit introduces several changes to add 'ranges' support for > PCI endpoint devices: > > 1. **Modify of_address.c**: Add support for the new `device_type` > "pci-ep", enabling it to parse 'ranges' using the same functions > as for PCI devices. > > 2. **Update DesignWare PCIe EP driver**: Enhance the driver to > support 'ranges' when 'addr_space' is missing, maintaining > compatibility with existing drivers. > > 3. **Update binding documentation**: Modify the device tree bindings > to include 'ranges' support and make 'addr_space' an optional > entry in 'reg-names'. > > 4. **Add i.MX8QXP EP support**: Incorporate support for the > i.MX8QXP PCIe EP in the driver. > > i.MX8QXP PCIe dts is upstreaming. Below is pcie-ep part. > > pcieb_ep: pcie-ep@5f010000 { > compatible = "fsl,imx8q-pcie-ep"; > reg = <0x5f010000 0x00010000>; > reg-names = "dbi"; > #address-cells = <3>; > #size-cells = <2>; > device_type = "pci-ep"; > ranges = <0x82000000 0 0x80000000 0x70000000 0 0x10000000>; How does a PCI endpoint set PCI addresses? Those get assigned by the PCI host system. They can't be static in DT. If you need the PCI address, just read your BAR registers. In general, why do you need this when none of the other PCI endpoint drivers have needed this? Rob
On Sat, Sep 21, 2024 at 09:43:17AM -0500, Rob Herring wrote: > On Thu, Sep 19, 2024 at 5:03 PM Frank Li <Frank.Li@nxp.com> wrote: > > > > The PCI bus device tree supports 'ranges' properties that indicate > > how to convert PCI addresses to CPU addresses. Many PCI controllers > > are dual-role controllers, supporting both Root Complex (RC) and > > Endpoint (EP) modes. The EP side also needs similar information for > > proper address translation. > > > > This commit introduces several changes to add 'ranges' support for > > PCI endpoint devices: > > > > 1. **Modify of_address.c**: Add support for the new `device_type` > > "pci-ep", enabling it to parse 'ranges' using the same functions > > as for PCI devices. > > > > 2. **Update DesignWare PCIe EP driver**: Enhance the driver to > > support 'ranges' when 'addr_space' is missing, maintaining > > compatibility with existing drivers. > > > > 3. **Update binding documentation**: Modify the device tree bindings > > to include 'ranges' support and make 'addr_space' an optional > > entry in 'reg-names'. > > > > 4. **Add i.MX8QXP EP support**: Incorporate support for the > > i.MX8QXP PCIe EP in the driver. > > > > i.MX8QXP PCIe dts is upstreaming. Below is pcie-ep part. > > > > pcieb_ep: pcie-ep@5f010000 { > > compatible = "fsl,imx8q-pcie-ep"; > > reg = <0x5f010000 0x00010000>; > > reg-names = "dbi"; > > #address-cells = <3>; > > #size-cells = <2>; > > device_type = "pci-ep"; > > ranges = <0x82000000 0 0x80000000 0x70000000 0 0x10000000>; > > How does a PCI endpoint set PCI addresses? Those get assigned by the > PCI host system. They can't be static in DT. PCI address is set by other channel, such as RC write some place in bar0. It indicates EP side outbound windows mapping. See below detail. Endpoint Root complex ┌───────┐ ┌─────────┐ ┌─────┐ │ EP │ │ │ ┌─────┐ │ │ │ Ctrl │ │ │ │ CPU │ │ DDR │ │ │ │ ┌────┐ │ └──┬──┘ │ │◄──────┼─ATU ◄─┼────────┼─┤BarN│◄─┼─────────┘ │ │ │ │ │ └────┘ │ Outbound Transfer └─────┘ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ Inbound Transfer │ │ │ │ ┌──▼──┐ ┌───────┐ │ │ │ ┌───────┼─────►│DDR │ │ │ outbound Transfer* │ │ │ └─────┘ ┌─────┐ │ Bus ┼─────►│ ATU ─┬────────┼─┘ │ │ │ │ Fabric│Bus │ │ PCI Addr │ │ CPU ├───►│ │Addr │ │ 0xA000_0000 │ │ │CPU │ │0x8000_0000 │ │ │ └─────┘Addr└───────┘ │ │ │ │ 0x7000_0000 └───────┘ └─────────┘ This ranges descript above diagram Endpoint outbound Transfer*'s information. There are address space (previous use addr_space in reg-name) indicate such informaiton, such as [0x7000_00000, 0xB000_0000] as PCI EP outbound windows. when cpu write 0x7000_0000, data will write to EP ctrl, the ATU in EP ctrl convert to PCI address such 0xA000,0000, then write to RC's DDR> The PCI Addr 0xA000_0000 information was sent to EP driver by use other channel, such as RC write it some place in Bar0. The 'range' here indicated EP side's outbound windows information. Most system CPU address is the same as bus address. but in imx8q, it is difference. Bus fabric convert 0x7000_0000 to 0x8000_00000. So need range indicate such address convertion. > > If you need the PCI address, just read your BAR registers. > > In general, why do you need this when none of the other PCI endpoint > drivers have needed this? Most system, the address is the same. Some system convert is simple, just cut some high address bit, so their driver hardcode it. Maybe imx8QM have first one, they have more than one controller and address map is not such simple. We use customer dt property in downstream kernel, but I think common solution should be better, other drivers can remove their hardcode in future. And it will be more symmetry with PCI host side's property. Frank > > Rob
On Sat, Sep 21, 2024 at 03:18:50PM -0400, Frank Li wrote: > On Sat, Sep 21, 2024 at 09:43:17AM -0500, Rob Herring wrote: > > On Thu, Sep 19, 2024 at 5:03 PM Frank Li <Frank.Li@nxp.com> wrote: > > > > > > The PCI bus device tree supports 'ranges' properties that indicate > > > how to convert PCI addresses to CPU addresses. Many PCI controllers > > > are dual-role controllers, supporting both Root Complex (RC) and > > > Endpoint (EP) modes. The EP side also needs similar information for > > > proper address translation. > > > > > > This commit introduces several changes to add 'ranges' support for > > > PCI endpoint devices: > > > > > > 1. **Modify of_address.c**: Add support for the new `device_type` > > > "pci-ep", enabling it to parse 'ranges' using the same functions > > > as for PCI devices. > > > > > > 2. **Update DesignWare PCIe EP driver**: Enhance the driver to > > > support 'ranges' when 'addr_space' is missing, maintaining > > > compatibility with existing drivers. > > > > > > 3. **Update binding documentation**: Modify the device tree bindings > > > to include 'ranges' support and make 'addr_space' an optional > > > entry in 'reg-names'. > > > > > > 4. **Add i.MX8QXP EP support**: Incorporate support for the > > > i.MX8QXP PCIe EP in the driver. > > > > > > i.MX8QXP PCIe dts is upstreaming. Below is pcie-ep part. > > > > > > pcieb_ep: pcie-ep@5f010000 { > > > compatible = "fsl,imx8q-pcie-ep"; > > > reg = <0x5f010000 0x00010000>; > > > reg-names = "dbi"; > > > #address-cells = <3>; > > > #size-cells = <2>; > > > device_type = "pci-ep"; > > > ranges = <0x82000000 0 0x80000000 0x70000000 0 0x10000000>; > > > > How does a PCI endpoint set PCI addresses? Those get assigned by the > > PCI host system. They can't be static in DT. > > PCI address is set by other channel, such as RC write some place in bar0. > > It indicates EP side outbound windows mapping. See below detail. > > > Endpoint Root complex > ┌───────┐ ┌─────────┐ > ┌─────┐ │ EP │ │ │ ┌─────┐ > │ │ │ Ctrl │ │ │ │ CPU │ > │ DDR │ │ │ │ ┌────┐ │ └──┬──┘ > │ │◄──────┼─ATU ◄─┼────────┼─┤BarN│◄─┼─────────┘ > │ │ │ │ │ └────┘ │ Outbound Transfer > └─────┘ │ │ │ │ > │ │ │ │ > │ │ │ │ > │ │ │ │ Inbound Transfer > │ │ │ │ ┌──▼──┐ > ┌───────┐ │ │ │ ┌───────┼─────►│DDR │ > │ │ outbound Transfer* │ │ │ └─────┘ > ┌─────┐ │ Bus ┼─────►│ ATU ─┬────────┼─┘ │ > │ │ │ Fabric│Bus │ │ PCI Addr │ > │ CPU ├───►│ │Addr │ │ 0xA000_0000 │ > │ │CPU │ │0x8000_0000 │ │ │ > └─────┘Addr└───────┘ │ │ │ │ > 0x7000_0000 └───────┘ └─────────┘ > > > This ranges descript above diagram Endpoint outbound Transfer*'s > information. There are address space (previous use addr_space in reg-name) > indicate such informaiton, such as [0x7000_00000, 0xB000_0000] as PCI EP > outbound windows. when cpu write 0x7000_0000, data will write to EP ctrl, > the ATU in EP ctrl convert to PCI address such 0xA000,0000, then write > to RC's DDR> > > The PCI Addr 0xA000_0000 information was sent to EP driver by use other > channel, such as RC write it some place in Bar0. > > The 'range' here indicated EP side's outbound windows information. Most > system CPU address is the same as bus address. but in imx8q, it is > difference. Bus fabric convert 0x7000_0000 to 0x8000_00000. > > So need range indicate such address convertion. > > > > > If you need the PCI address, just read your BAR registers. > > > > In general, why do you need this when none of the other PCI endpoint > > drivers have needed this? > > Most system, the address is the same. Some system convert is simple, just > cut some high address bit, so their driver hardcode it. Maybe imx8QM have > first one, they have more than one controller and address map is not > such simple. > > We use customer dt property in downstream kernel, but I think common > solution should be better, other drivers can remove their hardcode in > future. And it will be more symmetry with PCI host side's property. I found a more simple the method, will post v2 soon. Frank > > Frank > > > > Rob
On Mon, Sep 23, 2024 at 12:14:27PM -0400, Frank Li wrote: > On Sat, Sep 21, 2024 at 03:18:50PM -0400, Frank Li wrote: > > On Sat, Sep 21, 2024 at 09:43:17AM -0500, Rob Herring wrote: > > > On Thu, Sep 19, 2024 at 5:03 PM Frank Li <Frank.Li@nxp.com> wrote: > > > > > > > > The PCI bus device tree supports 'ranges' properties that indicate > > > > how to convert PCI addresses to CPU addresses. Many PCI controllers > > > > are dual-role controllers, supporting both Root Complex (RC) and > > > > Endpoint (EP) modes. The EP side also needs similar information for > > > > proper address translation. > > > > > > > > This commit introduces several changes to add 'ranges' support for > > > > PCI endpoint devices: > > > > > > > > 1. **Modify of_address.c**: Add support for the new `device_type` > > > > "pci-ep", enabling it to parse 'ranges' using the same functions > > > > as for PCI devices. > > > > > > > > 2. **Update DesignWare PCIe EP driver**: Enhance the driver to > > > > support 'ranges' when 'addr_space' is missing, maintaining > > > > compatibility with existing drivers. > > > > > > > > 3. **Update binding documentation**: Modify the device tree bindings > > > > to include 'ranges' support and make 'addr_space' an optional > > > > entry in 'reg-names'. > > > > > > > > 4. **Add i.MX8QXP EP support**: Incorporate support for the > > > > i.MX8QXP PCIe EP in the driver. > > > > > > > > i.MX8QXP PCIe dts is upstreaming. Below is pcie-ep part. > > > > > > > > pcieb_ep: pcie-ep@5f010000 { > > > > compatible = "fsl,imx8q-pcie-ep"; > > > > reg = <0x5f010000 0x00010000>; > > > > reg-names = "dbi"; > > > > #address-cells = <3>; > > > > #size-cells = <2>; > > > > device_type = "pci-ep"; > > > > ranges = <0x82000000 0 0x80000000 0x70000000 0 0x10000000>; > > > > > > How does a PCI endpoint set PCI addresses? Those get assigned by the > > > PCI host system. They can't be static in DT. > > > > PCI address is set by other channel, such as RC write some place in bar0. > > > > It indicates EP side outbound windows mapping. See below detail. > > > > > > Endpoint Root complex > > ┌───────┐ ┌─────────┐ > > ┌─────┐ │ EP │ │ │ ┌─────┐ > > │ │ │ Ctrl │ │ │ │ CPU │ > > │ DDR │ │ │ │ ┌────┐ │ └──┬──┘ > > │ │◄──────┼─ATU ◄─┼────────┼─┤BarN│◄─┼─────────┘ > > │ │ │ │ │ └────┘ │ Outbound Transfer > > └─────┘ │ │ │ │ > > │ │ │ │ > > │ │ │ │ > > │ │ │ │ Inbound Transfer > > │ │ │ │ ┌──▼──┐ > > ┌───────┐ │ │ │ ┌───────┼─────►│DDR │ > > │ │ outbound Transfer* │ │ │ └─────┘ > > ┌─────┐ │ Bus ┼─────►│ ATU ─┬────────┼─┘ │ > > │ │ │ Fabric│Bus │ │ PCI Addr │ > > │ CPU ├───►│ │Addr │ │ 0xA000_0000 │ > > │ │CPU │ │0x8000_0000 │ │ │ > > └─────┘Addr└───────┘ │ │ │ │ > > 0x7000_0000 └───────┘ └─────────┘ > > > > > > This ranges descript above diagram Endpoint outbound Transfer*'s > > information. There are address space (previous use addr_space in reg-name) > > indicate such informaiton, such as [0x7000_00000, 0xB000_0000] as PCI EP > > outbound windows. when cpu write 0x7000_0000, data will write to EP ctrl, > > the ATU in EP ctrl convert to PCI address such 0xA000,0000, then write > > to RC's DDR> > > > > The PCI Addr 0xA000_0000 information was sent to EP driver by use other > > channel, such as RC write it some place in Bar0. > > > > The 'range' here indicated EP side's outbound windows information. Most > > system CPU address is the same as bus address. but in imx8q, it is > > difference. Bus fabric convert 0x7000_0000 to 0x8000_00000. > > > > So need range indicate such address convertion. > > > > > > > > If you need the PCI address, just read your BAR registers. > > > > > > In general, why do you need this when none of the other PCI endpoint > > > drivers have needed this? > > > > Most system, the address is the same. Some system convert is simple, just > > cut some high address bit, so their driver hardcode it. Maybe imx8QM have > > first one, they have more than one controller and address map is not > > such simple. > > > > We use customer dt property in downstream kernel, but I think common > > solution should be better, other drivers can remove their hardcode in > > future. And it will be more symmetry with PCI host side's property. > > I found a more simple the method, will post v2 soon. V2 post at https://lore.kernel.org/imx/20240923-pcie_ep_range-v2-0-78d2ea434d9f@nxp.com/T/#t , which more simple and reasonable. Frank > > Frank > > > > > Frank > > > > > > Rob
The PCI bus device tree supports 'ranges' properties that indicate how to convert PCI addresses to CPU addresses. Many PCI controllers are dual-role controllers, supporting both Root Complex (RC) and Endpoint (EP) modes. The EP side also needs similar information for proper address translation. This commit introduces several changes to add 'ranges' support for PCI endpoint devices: 1. **Modify of_address.c**: Add support for the new `device_type` "pci-ep", enabling it to parse 'ranges' using the same functions as for PCI devices. 2. **Update DesignWare PCIe EP driver**: Enhance the driver to support 'ranges' when 'addr_space' is missing, maintaining compatibility with existing drivers. 3. **Update binding documentation**: Modify the device tree bindings to include 'ranges' support and make 'addr_space' an optional entry in 'reg-names'. 4. **Add i.MX8QXP EP support**: Incorporate support for the i.MX8QXP PCIe EP in the driver. i.MX8QXP PCIe dts is upstreaming. Below is pcie-ep part. pcieb_ep: pcie-ep@5f010000 { compatible = "fsl,imx8q-pcie-ep"; reg = <0x5f010000 0x00010000>; reg-names = "dbi"; #address-cells = <3>; #size-cells = <2>; device_type = "pci-ep"; ranges = <0x82000000 0 0x80000000 0x70000000 0 0x10000000>; num-lanes = <1>; interrupts = <GIC_SPI 104 IRQ_TYPE_LEVEL_HIGH>; interrupt-names = "dma"; clocks = <&pcieb_lpcg IMX_LPCG_CLK_6>, <&pcieb_lpcg IMX_LPCG_CLK_4>, <&pcieb_lpcg IMX_LPCG_CLK_5>; clock-names = "dbi", "mstr", "slv"; power-domains = <&pd IMX_SC_R_PCIE_B>; fsl,max-link-speed = <3>; num-ib-windows = <6>; num-ob-windows = <6>; status = "disabled"; }; These changes improve PCIe EP support by allowing proper address translation using 'ranges', ensuring compatibility with devices that rely on this information. Signed-off-by: Frank Li <Frank.Li@nxp.com> --- Frank Li (9): dt-bindings: PCI: pci-ep: Document 'ranges' property of: address: Add argument 'name' for of_node_is_pcie() of: address: Add device type pci-ep dt-bindings: PCI: snps,dw-pcie-ep: 'addr_space' not required if 'ranges' present PCI: dwc: ep: Replace phys_base and addr_size with range PCI: dwc: ep: Use 'ranges' from DT if 'addr_space' is missing dt-bindings: PCI: fsl,imx6q-pcie-ep: Add compatible string fsl,imx8q-pcie-ep PCI: imx6: Pass correct sub mode when calling phy_set_mode_ext() PCI: imx6: Add i.MX8Q PCIe Endpoint (EP) support .../devicetree/bindings/pci/fsl,imx6q-pcie-ep.yaml | 42 +++++++++++++++++++++- Documentation/devicetree/bindings/pci/pci-ep.yaml | 30 ++++++++++++++++ .../bindings/pci/snps,dw-pcie-common.yaml | 4 +-- .../devicetree/bindings/pci/snps,dw-pcie-ep.yaml | 21 ++++++++--- drivers/of/address.c | 30 ++++++++++++---- drivers/pci/controller/dwc/pci-imx6.c | 24 ++++++++++++- drivers/pci/controller/dwc/pcie-artpec6.c | 2 +- drivers/pci/controller/dwc/pcie-designware-ep.c | 23 ++++++++---- drivers/pci/controller/dwc/pcie-designware.h | 4 +-- 9 files changed, 157 insertions(+), 23 deletions(-) --- base-commit: 909eac36208b70a22fd0d1c3097e3af98dca7599 change-id: 20240918-pcie_ep_range-4c5c5e300e19 Best regards, --- Frank Li <Frank.Li@nxp.com>