mbox series

[v1,0/5] PolarFire SoC Icicle Reference Design PCIe ?support?/fixes

Message ID 20240610-vertical-frugally-a92a55427dd9@wendy (mailing list archive)
Headers show
Series PolarFire SoC Icicle Reference Design PCIe ?support?/fixes | expand

Message

Conor Dooley June 10, 2024, 11:09 a.m. UTC
Hey all,

Here's some patches that add an Erratum to enable non-coherent DMA
support for PolarFire SoC. By nature of being an FPGA, and due to the
PCIe root ports being only 32-bit capable, many bitstreams configure the
FPGA fabric such that peripherals in the fabric become non-coherent. The
PCIe root ports on PolarFire SoC are connected to the core-complex via
the fabric, and therefore can be (and regularly are) made DMA
non-coherent. The Icicle Kit Reference Design has been configuring the
PCIe root port in this manner since late 2022 and in a way unsupported
by mainline since earlier that year. Adding this non-coherent DMA
support makes PCIe functional on those FPGA designs. Daire did almost all
the work to figure out how to support these kinds of designs, and this
series depends on his patches to introduce the required dma-ranges
handling for the root port driver:
https://lore.kernel.org/linux-pci/20240531085333.2501399-1-daire.mcnamara@microchip.com/

The final patch depends on:
https://lore.kernel.org/linux-pci/20240527-slather-backfire-db4605ae7cd7@wendy/

I'm not sure if an Erratum is really the right way to go about doing
this, but I didn't want to make ARCH_MICROCHIP depend on NONPORTABLE.
An alternative would be to add a menu under drivers/soc like Renesas
does and allow it to be toggled as an option there instead.

Thanks,
Conor.

CC: Paul Walmsley <paul.walmsley@sifive.com>
CC: Palmer Dabbelt <palmer@dabbelt.com>
CC: Conor Dooley <conor.dooley@microchip.com>
CC: Daire McNamara <daire.mcnamara@microchip.com>
CC: Rob Herring <robh@kernel.org>
CC: Krzysztof Kozlowski <krzk+dt@kernel.org>
CC: Samuel Holland <samuel.holland@sifive.com>
CC: linux-riscv@lists.infradead.org
CC: devicetree@vger.kernel.org

Conor Dooley (5):
  cache: ccache: allow building for PolarFire
  cache: ccache: add mpfs to nonstandard cache ops list
  RISC-V: Add an MPFS erratum for PCIe
  riscv: dts: microchip: modify memory map & add dma-ranges for pcie on
    icicle
  riscv: dts: microchip: update pcie reg properties

 arch/riscv/Kconfig.errata                     | 19 +++++
 .../dts/microchip/mpfs-icicle-kit-fabric.dtsi | 77 ++++++++++++-------
 .../boot/dts/microchip/mpfs-icicle-kit.dts    | 44 +++++++++--
 .../dts/microchip/mpfs-m100pfs-fabric.dtsi    |  6 +-
 .../dts/microchip/mpfs-polarberry-fabric.dtsi |  6 +-
 drivers/cache/Kconfig                         |  2 +-
 drivers/cache/sifive_ccache.c                 |  2 +
 7 files changed, 116 insertions(+), 40 deletions(-)

Comments

Conor Dooley June 10, 2024, 11:16 a.m. UTC | #1
On Mon, Jun 10, 2024 at 12:09:12PM +0100, Conor Dooley wrote:
> Hey all,
> 
> Here's some patches that add an Erratum to enable non-coherent DMA
> support for PolarFire SoC...

My local mail client shows that I sent two copies of this series, but I
only see one on lore. I accidentally ran the wrong command, and didn't
set an stmp server in `git send-email`, so I think that only one copy
actually went out and my mail client is confused.
Rob Herring (Arm) June 10, 2024, 7:55 p.m. UTC | #2
On Mon, 10 Jun 2024 12:09:12 +0100, Conor Dooley wrote:
> Hey all,
> 
> Here's some patches that add an Erratum to enable non-coherent DMA
> support for PolarFire SoC. By nature of being an FPGA, and due to the
> PCIe root ports being only 32-bit capable, many bitstreams configure the
> FPGA fabric such that peripherals in the fabric become non-coherent. The
> PCIe root ports on PolarFire SoC are connected to the core-complex via
> the fabric, and therefore can be (and regularly are) made DMA
> non-coherent. The Icicle Kit Reference Design has been configuring the
> PCIe root port in this manner since late 2022 and in a way unsupported
> by mainline since earlier that year. Adding this non-coherent DMA
> support makes PCIe functional on those FPGA designs. Daire did almost all
> the work to figure out how to support these kinds of designs, and this
> series depends on his patches to introduce the required dma-ranges
> handling for the root port driver:
> https://lore.kernel.org/linux-pci/20240531085333.2501399-1-daire.mcnamara@microchip.com/
> 
> The final patch depends on:
> https://lore.kernel.org/linux-pci/20240527-slather-backfire-db4605ae7cd7@wendy/
> 
> I'm not sure if an Erratum is really the right way to go about doing
> this, but I didn't want to make ARCH_MICROCHIP depend on NONPORTABLE.
> An alternative would be to add a menu under drivers/soc like Renesas
> does and allow it to be toggled as an option there instead.
> 
> Thanks,
> Conor.
> 
> CC: Paul Walmsley <paul.walmsley@sifive.com>
> CC: Palmer Dabbelt <palmer@dabbelt.com>
> CC: Conor Dooley <conor.dooley@microchip.com>
> CC: Daire McNamara <daire.mcnamara@microchip.com>
> CC: Rob Herring <robh@kernel.org>
> CC: Krzysztof Kozlowski <krzk+dt@kernel.org>
> CC: Samuel Holland <samuel.holland@sifive.com>
> CC: linux-riscv@lists.infradead.org
> CC: devicetree@vger.kernel.org
> 
> Conor Dooley (5):
>   cache: ccache: allow building for PolarFire
>   cache: ccache: add mpfs to nonstandard cache ops list
>   RISC-V: Add an MPFS erratum for PCIe
>   riscv: dts: microchip: modify memory map & add dma-ranges for pcie on
>     icicle
>   riscv: dts: microchip: update pcie reg properties
> 
>  arch/riscv/Kconfig.errata                     | 19 +++++
>  .../dts/microchip/mpfs-icicle-kit-fabric.dtsi | 77 ++++++++++++-------
>  .../boot/dts/microchip/mpfs-icicle-kit.dts    | 44 +++++++++--
>  .../dts/microchip/mpfs-m100pfs-fabric.dtsi    |  6 +-
>  .../dts/microchip/mpfs-polarberry-fabric.dtsi |  6 +-
>  drivers/cache/Kconfig                         |  2 +-
>  drivers/cache/sifive_ccache.c                 |  2 +
>  7 files changed, 116 insertions(+), 40 deletions(-)
> 
> --
> 2.43.2
> 
> 
> 


My bot found new DTB warnings on the .dts files added or changed in this
series.

Some warnings may be from an existing SoC .dtsi. Or perhaps the warnings
are fixed by another series. Ultimately, it is up to the platform
maintainer whether these warnings are acceptable or not. No need to reply
unless the platform maintainer has comments.

If you already ran DT checks and didn't see these error(s), then
make sure dt-schema is up to date:

  pip3 install dtschema --upgrade


New warnings running 'make CHECK_DTBS=y microchip/mpfs-icicle-kit.dtb' for 20240610-vertical-frugally-a92a55427dd9@wendy:

arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dtb: pcie@3000000000: reg: [[48, 0, 0, 134217728], [0, 1124106240, 0, 8192], [0, 1124114432, 0, 8192]] is too long
	from schema $id: http://devicetree.org/schemas/pci/microchip,pcie-host.yaml#
arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dtb: pcie@3000000000: reg-names:1: 'apb' was expected
	from schema $id: http://devicetree.org/schemas/pci/microchip,pcie-host.yaml#
arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dtb: pcie@3000000000: reg-names: ['cfg', 'bridge', 'ctrl'] is too long
	from schema $id: http://devicetree.org/schemas/pci/microchip,pcie-host.yaml#
arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dtb: pcie@3000000000: Unevaluated properties are not allowed ('#address-cells', '#interrupt-cells', '#size-cells', 'bus-range', 'device_type', 'dma-noncoherent', 'interrupt-controller', 'interrupt-map', 'interrupt-map-mask', 'interrupt-parent', 'interrupts', 'msi-parent', 'reg', 'reg-names' were unexpected)
	from schema $id: http://devicetree.org/schemas/pci/microchip,pcie-host.yaml#
Conor Dooley June 10, 2024, 9:28 p.m. UTC | #3
On Mon, Jun 10, 2024 at 01:55:14PM -0600, Rob Herring (Arm) wrote:
> On Mon, 10 Jun 2024 12:09:12 +0100, Conor Dooley wrote:

> > The final patch depends on:
> > https://lore.kernel.org/linux-pci/20240527-slather-backfire-db4605ae7cd7@wendy/


> 
> New warnings running 'make CHECK_DTBS=y microchip/mpfs-icicle-kit.dtb' for 20240610-vertical-frugally-a92a55427dd9@wendy:
> 
> arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dtb: pcie@3000000000: reg: [[48, 0, 0, 134217728], [0, 1124106240, 0, 8192], [0, 1124114432, 0, 8192]] is too long
> 	from schema $id: http://devicetree.org/schemas/pci/microchip,pcie-host.yaml#
> arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dtb: pcie@3000000000: reg-names:1: 'apb' was expected
> 	from schema $id: http://devicetree.org/schemas/pci/microchip,pcie-host.yaml#
> arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dtb: pcie@3000000000: reg-names: ['cfg', 'bridge', 'ctrl'] is too long
> 	from schema $id: http://devicetree.org/schemas/pci/microchip,pcie-host.yaml#
> arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dtb: pcie@3000000000: Unevaluated properties are not allowed ('#address-cells', '#interrupt-cells', '#size-cells', 'bus-range', 'device_type', 'dma-noncoherent', 'interrupt-controller', 'interrupt-map', 'interrupt-map-mask', 'interrupt-parent', 'interrupts', 'msi-parent', 'reg', 'reg-names' were unexpected)
> 	from schema $id: http://devicetree.org/schemas/pci/microchip,pcie-host.yaml#

Yah, not a problem. Patches for that are on the pci list atm.
Palmer Dabbelt July 3, 2024, 8:09 p.m. UTC | #4
On Mon, 10 Jun 2024 04:09:12 PDT (-0700), Conor Dooley wrote:
> Hey all,
>
> Here's some patches that add an Erratum to enable non-coherent DMA
> support for PolarFire SoC. By nature of being an FPGA, and due to the
> PCIe root ports being only 32-bit capable, many bitstreams configure the
> FPGA fabric such that peripherals in the fabric become non-coherent. The
> PCIe root ports on PolarFire SoC are connected to the core-complex via
> the fabric, and therefore can be (and regularly are) made DMA
> non-coherent. The Icicle Kit Reference Design has been configuring the
> PCIe root port in this manner since late 2022 and in a way unsupported
> by mainline since earlier that year. Adding this non-coherent DMA
> support makes PCIe functional on those FPGA designs. Daire did almost all
> the work to figure out how to support these kinds of designs, and this
> series depends on his patches to introduce the required dma-ranges
> handling for the root port driver:
> https://lore.kernel.org/linux-pci/20240531085333.2501399-1-daire.mcnamara@microchip.com/
>
> The final patch depends on:
> https://lore.kernel.org/linux-pci/20240527-slather-backfire-db4605ae7cd7@wendy/
>
> I'm not sure if an Erratum is really the right way to go about doing
> this, but I didn't want to make ARCH_MICROCHIP depend on NONPORTABLE.
> An alternative would be to add a menu under drivers/soc like Renesas
> does and allow it to be toggled as an option there instead.

Acked-by: Palmer Dabbelt <palmer@rivosinc.com>

if you want to take this via some other tree.

I don't see any issue calling it an erratum, if it lets us keep building 
generic kernels for these systems that seems like a good enough agrument 
to me.

>
> Thanks,
> Conor.
>
> CC: Paul Walmsley <paul.walmsley@sifive.com>
> CC: Palmer Dabbelt <palmer@dabbelt.com>
> CC: Conor Dooley <conor.dooley@microchip.com>
> CC: Daire McNamara <daire.mcnamara@microchip.com>
> CC: Rob Herring <robh@kernel.org>
> CC: Krzysztof Kozlowski <krzk+dt@kernel.org>
> CC: Samuel Holland <samuel.holland@sifive.com>
> CC: linux-riscv@lists.infradead.org
> CC: devicetree@vger.kernel.org
>
> Conor Dooley (5):
>   cache: ccache: allow building for PolarFire
>   cache: ccache: add mpfs to nonstandard cache ops list
>   RISC-V: Add an MPFS erratum for PCIe
>   riscv: dts: microchip: modify memory map & add dma-ranges for pcie on
>     icicle
>   riscv: dts: microchip: update pcie reg properties
>
>  arch/riscv/Kconfig.errata                     | 19 +++++
>  .../dts/microchip/mpfs-icicle-kit-fabric.dtsi | 77 ++++++++++++-------
>  .../boot/dts/microchip/mpfs-icicle-kit.dts    | 44 +++++++++--
>  .../dts/microchip/mpfs-m100pfs-fabric.dtsi    |  6 +-
>  .../dts/microchip/mpfs-polarberry-fabric.dtsi |  6 +-
>  drivers/cache/Kconfig                         |  2 +-
>  drivers/cache/sifive_ccache.c                 |  2 +
>  7 files changed, 116 insertions(+), 40 deletions(-)
Conor Dooley July 3, 2024, 8:15 p.m. UTC | #5
On Wed, Jul 03, 2024 at 01:09:43PM -0700, Palmer Dabbelt wrote:
> On Mon, 10 Jun 2024 04:09:12 PDT (-0700), Conor Dooley wrote:
> > Hey all,
> > 
> > Here's some patches that add an Erratum to enable non-coherent DMA
> > support for PolarFire SoC. By nature of being an FPGA, and due to the
> > PCIe root ports being only 32-bit capable, many bitstreams configure the
> > FPGA fabric such that peripherals in the fabric become non-coherent. The
> > PCIe root ports on PolarFire SoC are connected to the core-complex via
> > the fabric, and therefore can be (and regularly are) made DMA
> > non-coherent. The Icicle Kit Reference Design has been configuring the
> > PCIe root port in this manner since late 2022 and in a way unsupported
> > by mainline since earlier that year. Adding this non-coherent DMA
> > support makes PCIe functional on those FPGA designs. Daire did almost all
> > the work to figure out how to support these kinds of designs, and this
> > series depends on his patches to introduce the required dma-ranges
> > handling for the root port driver:
> > https://lore.kernel.org/linux-pci/20240531085333.2501399-1-daire.mcnamara@microchip.com/
> > 
> > The final patch depends on:
> > https://lore.kernel.org/linux-pci/20240527-slather-backfire-db4605ae7cd7@wendy/
> > 
> > I'm not sure if an Erratum is really the right way to go about doing
> > this, but I didn't want to make ARCH_MICROCHIP depend on NONPORTABLE.
> > An alternative would be to add a menu under drivers/soc like Renesas
> > does and allow it to be toggled as an option there instead.
> 
> Acked-by: Palmer Dabbelt <palmer@rivosinc.com>
> 
> if you want to take this via some other tree.

Yeah, I was expecting to take it myself but hadn't gone fishing for an
ack because the PCI parts haven't landed yet & the copy in my k.org tree
had a KISMET issue reported against it.

> I don't see any issue calling it an erratum, if it lets us keep building
> generic kernels for these systems that seems like a good enough agrument to
> me.

Cool :)