| Message ID | 20250114-imx95_lut-v9-0-39f58dbed03a@nxp.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | PCI: add enabe(disable)_device() hook for bridge | expand |
On Tue, Jan 14, 2025 at 03:37:07PM -0500, Frank Li wrote: > Some system's IOMMU stream(master) ID bits(such as 6bits) less than > pci_device_id (16bit). It needs add hardware configuration to enable > pci_device_id to stream ID convert. > > https://lore.kernel.org/imx/20240622173849.GA1432357@bhelgaas/ > This ways use pcie bus notifier (like apple pci controller), when new PCIe > device added, bus notifier will call register specific callback to handle > look up table (LUT) configuration. > > https://lore.kernel.org/imx/20240429150842.GC1709920-robh@kernel.org/ > which parse dt's 'msi-map' and 'iommu-map' property to static config LUT > table (qcom use this way). This way is rejected by DT maintainer Rob. > > Above ways can resolve LUT take or stream id out of usage the problem. If > there are not enough stream id resource, not error return, EP hardware > still issue DMA to do transfer, which may transfer to wrong possition. > > Add enable(disable)_device() hook for bridge can return error when not > enough resource, and PCI device can't enabled. > > Basicallly this version can match Bjorn's requirement: > 1: simple, because it is rare that there are no LUT resource. > 2: EP driver probe failure when no LUT, but lspci can see such device. > > [ 2.164415] nvme nvme0: pci function 0000:01:00.0 > [ 2.169142] pci 0000:00:00.0: Error enabling bridge (-1), continuing > [ 2.175654] nvme 0000:01:00.0: probe with driver nvme failed with error -12 > > > lspci > 0000:00:00.0 PCI bridge: Philips Semiconductors Device 0000 > 0000:01:00.0 Non-Volatile memory controller: Micron Technology Inc 2100AI NVMe SSD [Nitro] (rev 03) > > To: Bjorn Helgaas <bhelgaas@google.com> > To: Richard Zhu <hongxing.zhu@nxp.com> > To: Lucas Stach <l.stach@pengutronix.de> > To: Lorenzo Pieralisi <lpieralisi@kernel.org> > To: Krzysztof Wilczyński <kw@linux.com> > To: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> > To: Rob Herring <robh@kernel.org> > To: Shawn Guo <shawnguo@kernel.org> > To: Sascha Hauer <s.hauer@pengutronix.de> > To: Pengutronix Kernel Team <kernel@pengutronix.de> > To: Fabio Estevam <festevam@gmail.com> > Cc: linux-pci@vger.kernel.org > Cc: linux-kernel@vger.kernel.org > Cc: linux-arm-kernel@lists.infradead.org > Cc: imx@lists.linux.dev > Cc: Frank.li@nxp.com \ > Cc: alyssa@rosenzweig.io \ > Cc: bpf@vger.kernel.org \ > Cc: broonie@kernel.org \ > Cc: jgg@ziepe.ca \ > Cc: joro@8bytes.org \ > Cc: l.stach@pengutronix.de \ > Cc: lgirdwood@gmail.com \ > Cc: maz@kernel.org \ > Cc: p.zabel@pengutronix.de \ > Cc: robin.murphy@arm.com \ > Cc: will@kernel.org \ > Cc: Robin Murphy <robin.murphy@arm.com> > Cc: Marc Zyngier <maz@kernel.org> > > Signed-off-by: Frank Li <Frank.Li@nxp.com> Applied to pci/controller/imx6 for v6.14, thanks! And thanks for your patience. > --- > Changes in v9: > - update commit message and comments > - Rob agree use API to parse iommu-map and msi-map. > https://lore.kernel.org/imx/20250113225905.GA3325507-robh@kernel.org/ > - Link to v8: https://lore.kernel.org/r/20241210-imx95_lut-v8-0-2e730b2e5fde@nxp.com > > Changes in v8: > - update comment message according to Lorenzo Pieralisi's suggestion. > - rework err target table > - improve err==0 && target ==NULL description, use 1:1 map RID to > stream ID. > - invalidate case -> unexisted case, never happen > - sid_i will not do mask, add comments said only MSI glue layer add > controller id. > - rework iommu map and msi map return value check logic according to > Lorenzo Pieralisi's suggestion > - Link to v7: https://lore.kernel.org/r/20241203-imx95_lut-v7-0-d0cd6293225e@nxp.com > > Changes in v7: > - Rebase v6.13-rc1 > - Update patch 2 according to mani's feedback > - Link to v6: https://lore.kernel.org/r/20241118-imx95_lut-v6-0-a2951ba13347@nxp.com > > Changes in v6: > - Bjorn give review tags at v4, but v5 have big change, drop Bjorn's review > tag. > - Add back Marc Zyngier't review and test tags > - Add mani's ack at first patch > - Mini change for patch 2 according to mani's feedback > - Link to v5: https://lore.kernel.org/r/20241104-imx95_lut-v5-0-feb972f3f13b@nxp.com > > Changes in v5: > - Add help function of pci_bridge_enable(disable)_device > - Because big change, removed Bjorn's review tags and have not > added > Marc Zyngier't review and test tags > - Fix pci-imx6.c according to Mani's feedback > - Link to v4: https://lore.kernel.org/r/20241101-imx95_lut-v4-0-0fdf9a2fe754@nxp.com > > Changes in v4: > - Add Bjorn Helgaas review tag for patch1 > - check 'target' value for patch2 > - detail see each patches > - Link to v3: https://lore.kernel.org/r/20241024-imx95_lut-v3-0-7509c9bbab86@nxp.com > > Changes in v3: > - disable_device when error happen > - use target for of_map_id > - Check if rid already in lut table when enable deviced > - Link to v2: https://lore.kernel.org/r/20240930-imx95_lut-v2-0-3b6467ba539a@nxp.com > > Changes in v2: > - see each patch > - Link to v1: https://lore.kernel.org/r/20240926-imx95_lut-v1-0-d0c62087dbab@nxp.com > > --- > Frank Li (2): > PCI: Add enable_device() and disable_device() callbacks for bridges > PCI: imx6: Add IOMMU and ITS MSI support for i.MX95 > > drivers/pci/controller/dwc/pci-imx6.c | 199 +++++++++++++++++++++++++++++++++- > drivers/pci/pci.c | 36 +++++- > include/linux/pci.h | 2 + > 3 files changed, 235 insertions(+), 2 deletions(-) > --- > base-commit: 40384c840ea1944d7c5a392e8975ed088ecf0b37 > change-id: 20240926-imx95_lut-1c68222e0944 > > Best regards, > --- > Frank Li <Frank.Li@nxp.com> >
On Tue, 14 Jan 2025 22:33:41 +0000, Bjorn Helgaas <helgaas@kernel.org> wrote: > > On Tue, Jan 14, 2025 at 03:37:07PM -0500, Frank Li wrote: > > Some system's IOMMU stream(master) ID bits(such as 6bits) less than > > pci_device_id (16bit). It needs add hardware configuration to enable > > pci_device_id to stream ID convert. > > > > https://lore.kernel.org/imx/20240622173849.GA1432357@bhelgaas/ > > This ways use pcie bus notifier (like apple pci controller), when new PCIe > > device added, bus notifier will call register specific callback to handle > > look up table (LUT) configuration. > > > > https://lore.kernel.org/imx/20240429150842.GC1709920-robh@kernel.org/ > > which parse dt's 'msi-map' and 'iommu-map' property to static config LUT > > table (qcom use this way). This way is rejected by DT maintainer Rob. > > > > Above ways can resolve LUT take or stream id out of usage the problem. If > > there are not enough stream id resource, not error return, EP hardware > > still issue DMA to do transfer, which may transfer to wrong possition. > > > > Add enable(disable)_device() hook for bridge can return error when not > > enough resource, and PCI device can't enabled. > > > > Basicallly this version can match Bjorn's requirement: > > 1: simple, because it is rare that there are no LUT resource. > > 2: EP driver probe failure when no LUT, but lspci can see such device. > > > > [ 2.164415] nvme nvme0: pci function 0000:01:00.0 > > [ 2.169142] pci 0000:00:00.0: Error enabling bridge (-1), continuing > > [ 2.175654] nvme 0000:01:00.0: probe with driver nvme failed with error -12 > > > > > lspci > > 0000:00:00.0 PCI bridge: Philips Semiconductors Device 0000 > > 0000:01:00.0 Non-Volatile memory controller: Micron Technology Inc 2100AI NVMe SSD [Nitro] (rev 03) > > > > To: Bjorn Helgaas <bhelgaas@google.com> > > To: Richard Zhu <hongxing.zhu@nxp.com> > > To: Lucas Stach <l.stach@pengutronix.de> > > To: Lorenzo Pieralisi <lpieralisi@kernel.org> > > To: Krzysztof Wilczyński <kw@linux.com> > > To: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> > > To: Rob Herring <robh@kernel.org> > > To: Shawn Guo <shawnguo@kernel.org> > > To: Sascha Hauer <s.hauer@pengutronix.de> > > To: Pengutronix Kernel Team <kernel@pengutronix.de> > > To: Fabio Estevam <festevam@gmail.com> > > Cc: linux-pci@vger.kernel.org > > Cc: linux-kernel@vger.kernel.org > > Cc: linux-arm-kernel@lists.infradead.org > > Cc: imx@lists.linux.dev > > Cc: Frank.li@nxp.com \ > > Cc: alyssa@rosenzweig.io \ > > Cc: bpf@vger.kernel.org \ > > Cc: broonie@kernel.org \ > > Cc: jgg@ziepe.ca \ > > Cc: joro@8bytes.org \ > > Cc: l.stach@pengutronix.de \ > > Cc: lgirdwood@gmail.com \ > > Cc: maz@kernel.org \ > > Cc: p.zabel@pengutronix.de \ > > Cc: robin.murphy@arm.com \ > > Cc: will@kernel.org \ > > Cc: Robin Murphy <robin.murphy@arm.com> > > Cc: Marc Zyngier <maz@kernel.org> > > > > Signed-off-by: Frank Li <Frank.Li@nxp.com> > > Applied to pci/controller/imx6 for v6.14, thanks! And thanks for your > patience. While you're at it, could you please consider [1], which builds on top of the same infrastructure to remove the Apple PCIe IOMMU hack? Thanks, M. [1] https://lore.kernel.org/linux-pci/20241204150145.800408-1-maz@kernel.org/