[v3,00/11] PCI: microchip: Partition address translations

Message

Daire McNamara Jan. 11, 2023, 12:53 p.m. UTC

From: Daire McNamara <[e-mail]>

Changes since v2:
- Replaced GENMASK(63,0) with GENMASK_ULL(63,0) to remove warning
- Added patch to avoid warning on cast of argument to devm_add_action_or_reset()
- Added patch to enable building driver as a module

Changes since v1:
- Removed unused variables causing compile warnings
- Removed incorrect Signed-off-by: tags
- Capitalised msi and msi-x
- Capitalised FIC and respelled busses to buses
- Capitalised all comments
- Renamed fabric inter connect to Fabric Interface Controller as per PolarFire SoC TRM

Microchip PolarFire SoC is a 64-bit device and has DDR starting at
0x80000000 and 0x1000000000. Its PCIe rootport is connected to the CPU
Coreplex via an FPGA fabric. The AXI connections between the Coreplex and
the fabric are 64-bit and the AXI connections between the fabric and the
rootport are 32-bit.  For the CPU CorePlex to act as an AXI-Master to the
PCIe devices and for the PCIe devices to act as bus masters to DDR at these
base addresses, the fabric can be customised to add/remove offsets for bits
38-32 in each direction. These offsets, if present, vary with each
customer's design.

To support this variety, the rootport driver must know how much address
translation (both inbound and outbound) is performed by a particular
customer design and how much address translation must be provided by the
rootport.

This patchset contains a parent/child dma-ranges scheme suggested by Rob
Herring. It creates an FPGA PCIe parent bus which wraps the PCIe rootport
and implements a parsing scheme where the root port identifies what address
translations are performed by the FPGA fabric parent bus, and what
address translations must be done by the rootport itself.

See https://lore.kernel.org/linux-pci/[e-mail]/
for the relevant previous patch submission discussion.

It also re-partitions the probe() and init() functions as suggested by
Bjorn Helgaas to make them more maintainable as the init() function had
become too large.

It also contains some minor fixes and clean-ups that are pre-requisites:
- to align register, offset, and mask names with the hardware documentation
  and to have the register definitions appear in the same order as in the
  hardware documentation;
- to harvest the MSI information from the hardware configuration register
  as these depend on the FPGA fabric design and can vary with different
  customer designs;
- to clean up interrupt initialisation to make it more maintainable;
- to fix SEC and DED interrupt handling.

I expect Conor will take the dts patch via the soc tree once the PCIe parts
of the series are accepted.

Conor Dooley (1):
  riscv: dts: microchip: add parent ranges and dma-ranges for IKRD
    v2022.09

Daire McNamara (10):
  PCI: microchip: Correct the DED and SEC interrupt bit offsets
  PCI: microchip: Remove cast warning for devm_add_action_or_reset() arg
  PCI: microchip: enable building this driver as a module
  PCI: microchip: Align register, offset, and mask names with hw docs
  PCI: microchip: Enable event handlers to access bridge and ctrl ptrs
  PCI: microchip: Clean up initialisation of interrupts
  PCI: microchip: Gather MSI information from hardware config registers
  PCI: microchip: Re-partition code between probe() and init()
  PCI: microchip: Partition outbound address translation
  PCI: microchip: Partition inbound address translation

 .../dts/microchip/mpfs-icicle-kit-fabric.dtsi |  62 +-
 drivers/pci/controller/Kconfig                |   2 +-
 drivers/pci/controller/pcie-microchip-host.c  | 688 +++++++++++++-----
 3 files changed, 533 insertions(+), 219 deletions(-)


base-commit: 3c1f24109dfc4fb1a3730ed237e50183c6bb26b3

Comments

Conor Dooley Jan. 20, 2023, 11:07 a.m. UTC | #1

Hey Bjorn, Lorenzo,

On Wed, Jan 11, 2023 at 12:53:12PM +0000, [e-mail] wrote:
> From: Daire McNamara <[e-mail]>

> I expect Conor will take the dts patch via the soc tree once the PCIe parts
> of the series are accepted.
> 
> Conor Dooley (1):
>   riscv: dts: microchip: add parent ranges and dma-ranges for IKRD
>     v2022.09

I don't want to take this patch just yet as it needs to be sequenced
with some other dts changes, so please don't wait to hear anything on
that patch from me in terms of the rest of the series!

Thanks,
Conor.

> Daire McNamara (10):
>   PCI: microchip: Correct the DED and SEC interrupt bit offsets
>   PCI: microchip: Remove cast warning for devm_add_action_or_reset() arg
>   PCI: microchip: enable building this driver as a module
>   PCI: microchip: Align register, offset, and mask names with hw docs
>   PCI: microchip: Enable event handlers to access bridge and ctrl ptrs
>   PCI: microchip: Clean up initialisation of interrupts
>   PCI: microchip: Gather MSI information from hardware config registers
>   PCI: microchip: Re-partition code between probe() and init()
>   PCI: microchip: Partition outbound address translation
>   PCI: microchip: Partition inbound address translation
> 
>  .../dts/microchip/mpfs-icicle-kit-fabric.dtsi |  62 +-
>  drivers/pci/controller/Kconfig                |   2 +-
>  drivers/pci/controller/pcie-microchip-host.c  | 688 +++++++++++++-----
>  3 files changed, 533 insertions(+), 219 deletions(-)

Daire McNamara Jan. 31, 2023, 5:03 p.m. UTC | #2

Hi all,

Just touching base here.  Can I take it that things are in-hand, and
this patchset is moving into the kernel or is there something I need to
do at my end?

best regards
daire

On Wed, 2023-01-11 at 12:53 +0000, [e-mail] wrote:
> From: Daire McNamara <[e-mail]>
> 
> Changes since v2:
> - Replaced GENMASK(63,0) with GENMASK_ULL(63,0) to remove warning
> - Added patch to avoid warning on cast of argument to
> devm_add_action_or_reset()
> - Added patch to enable building driver as a module
> 
> Changes since v1:
> - Removed unused variables causing compile warnings
> - Removed incorrect Signed-off-by: tags
> - Capitalised msi and msi-x
> - Capitalised FIC and respelled busses to buses
> - Capitalised all comments
> - Renamed fabric inter connect to Fabric Interface Controller as per
> PolarFire SoC TRM
> 
> Microchip PolarFire SoC is a 64-bit device and has DDR starting at
> 0x80000000 and 0x1000000000. Its PCIe rootport is connected to the
> CPU
> Coreplex via an FPGA fabric. The AXI connections between the Coreplex
> and
> the fabric are 64-bit and the AXI connections between the fabric and
> the
> rootport are 32-bit.  For the CPU CorePlex to act as an AXI-Master to
> the
> PCIe devices and for the PCIe devices to act as bus masters to DDR at
> these
> base addresses, the fabric can be customised to add/remove offsets
> for bits
> 38-32 in each direction. These offsets, if present, vary with each
> customer's design.
> 
> To support this variety, the rootport driver must know how much
> address
> translation (both inbound and outbound) is performed by a particular
> customer design and how much address translation must be provided by
> the
> rootport.
> 
> This patchset contains a parent/child dma-ranges scheme suggested by
> Rob
> Herring. It creates an FPGA PCIe parent bus which wraps the PCIe
> rootport
> and implements a parsing scheme where the root port identifies what
> address
> translations are performed by the FPGA fabric parent bus, and what
> address translations must be done by the rootport itself.
> 
> See 
> https://lore.kernel.org/linux-pci/[e-mail]/
> for the relevant previous patch submission discussion.
> 
> It also re-partitions the probe() and init() functions as suggested
> by
> Bjorn Helgaas to make them more maintainable as the init() function
> had
> become too large.
> 
> It also contains some minor fixes and clean-ups that are pre-
> requisites:
> - to align register, offset, and mask names with the hardware
> documentation
>   and to have the register definitions appear in the same order as in
> the
>   hardware documentation;
> - to harvest the MSI information from the hardware configuration
> register
>   as these depend on the FPGA fabric design and can vary with
> different
>   customer designs;
> - to clean up interrupt initialisation to make it more maintainable;
> - to fix SEC and DED interrupt handling.
> 
> I expect Conor will take the dts patch via the soc tree once the PCIe
> parts
> of the series are accepted.
> 
> Conor Dooley (1):
>   riscv: dts: microchip: add parent ranges and dma-ranges for IKRD
>     v2022.09
> 
> Daire McNamara (10):
>   PCI: microchip: Correct the DED and SEC interrupt bit offsets
>   PCI: microchip: Remove cast warning for devm_add_action_or_reset()
> arg
>   PCI: microchip: enable building this driver as a module
>   PCI: microchip: Align register, offset, and mask names with hw docs
>   PCI: microchip: Enable event handlers to access bridge and ctrl
> ptrs
>   PCI: microchip: Clean up initialisation of interrupts
>   PCI: microchip: Gather MSI information from hardware config
> registers
>   PCI: microchip: Re-partition code between probe() and init()
>   PCI: microchip: Partition outbound address translation
>   PCI: microchip: Partition inbound address translation
> 
>  .../dts/microchip/mpfs-icicle-kit-fabric.dtsi |  62 +-
>  drivers/pci/controller/Kconfig                |   2 +-
>  drivers/pci/controller/pcie-microchip-host.c  | 688 +++++++++++++---
> --
>  3 files changed, 533 insertions(+), 219 deletions(-)
> 
> 
> base-commit: 3c1f24109dfc4fb1a3730ed237e50183c6bb26b3

Lorenzo Pieralisi Feb. 2, 2023, 11:02 a.m. UTC | #3

On Tue, Jan 31, 2023 at 05:03:00PM +0000, [e-mail] wrote:
> Hi all,
> 
> Just touching base here.  Can I take it that things are in-hand, and
> this patchset is moving into the kernel or is there something I need to
> do at my end?

I will have a look shortly, sorry for the delay.

Thanks,
Lorenzo

> best regards
> daire
> 
> On Wed, 2023-01-11 at 12:53 +0000, [e-mail] wrote:
> > From: Daire McNamara <[e-mail]>
> > 
> > Changes since v2:
> > - Replaced GENMASK(63,0) with GENMASK_ULL(63,0) to remove warning
> > - Added patch to avoid warning on cast of argument to
> > devm_add_action_or_reset()
> > - Added patch to enable building driver as a module
> > 
> > Changes since v1:
> > - Removed unused variables causing compile warnings
> > - Removed incorrect Signed-off-by: tags
> > - Capitalised msi and msi-x
> > - Capitalised FIC and respelled busses to buses
> > - Capitalised all comments
> > - Renamed fabric inter connect to Fabric Interface Controller as per
> > PolarFire SoC TRM
> > 
> > Microchip PolarFire SoC is a 64-bit device and has DDR starting at
> > 0x80000000 and 0x1000000000. Its PCIe rootport is connected to the
> > CPU
> > Coreplex via an FPGA fabric. The AXI connections between the Coreplex
> > and
> > the fabric are 64-bit and the AXI connections between the fabric and
> > the
> > rootport are 32-bit.  For the CPU CorePlex to act as an AXI-Master to
> > the
> > PCIe devices and for the PCIe devices to act as bus masters to DDR at
> > these
> > base addresses, the fabric can be customised to add/remove offsets
> > for bits
> > 38-32 in each direction. These offsets, if present, vary with each
> > customer's design.
> > 
> > To support this variety, the rootport driver must know how much
> > address
> > translation (both inbound and outbound) is performed by a particular
> > customer design and how much address translation must be provided by
> > the
> > rootport.
> > 
> > This patchset contains a parent/child dma-ranges scheme suggested by
> > Rob
> > Herring. It creates an FPGA PCIe parent bus which wraps the PCIe
> > rootport
> > and implements a parsing scheme where the root port identifies what
> > address
> > translations are performed by the FPGA fabric parent bus, and what
> > address translations must be done by the rootport itself.
> > 
> > See 
> > https://lore.kernel.org/linux-pci/[e-mail]/
> > for the relevant previous patch submission discussion.
> > 
> > It also re-partitions the probe() and init() functions as suggested
> > by
> > Bjorn Helgaas to make them more maintainable as the init() function
> > had
> > become too large.
> > 
> > It also contains some minor fixes and clean-ups that are pre-
> > requisites:
> > - to align register, offset, and mask names with the hardware
> > documentation
> >   and to have the register definitions appear in the same order as in
> > the
> >   hardware documentation;
> > - to harvest the MSI information from the hardware configuration
> > register
> >   as these depend on the FPGA fabric design and can vary with
> > different
> >   customer designs;
> > - to clean up interrupt initialisation to make it more maintainable;
> > - to fix SEC and DED interrupt handling.
> > 
> > I expect Conor will take the dts patch via the soc tree once the PCIe
> > parts
> > of the series are accepted.
> > 
> > Conor Dooley (1):
> >   riscv: dts: microchip: add parent ranges and dma-ranges for IKRD
> >     v2022.09
> > 
> > Daire McNamara (10):
> >   PCI: microchip: Correct the DED and SEC interrupt bit offsets
> >   PCI: microchip: Remove cast warning for devm_add_action_or_reset()
> > arg
> >   PCI: microchip: enable building this driver as a module
> >   PCI: microchip: Align register, offset, and mask names with hw docs
> >   PCI: microchip: Enable event handlers to access bridge and ctrl
> > ptrs
> >   PCI: microchip: Clean up initialisation of interrupts
> >   PCI: microchip: Gather MSI information from hardware config
> > registers
> >   PCI: microchip: Re-partition code between probe() and init()
> >   PCI: microchip: Partition outbound address translation
> >   PCI: microchip: Partition inbound address translation
> > 
> >  .../dts/microchip/mpfs-icicle-kit-fabric.dtsi |  62 +-
> >  drivers/pci/controller/Kconfig                |   2 +-
> >  drivers/pci/controller/pcie-microchip-host.c  | 688 +++++++++++++---
> > --
> >  3 files changed, 533 insertions(+), 219 deletions(-)
> > 
> > 
> > base-commit: 3c1f24109dfc4fb1a3730ed237e50183c6bb26b3