diff mbox series

[v8,24/26] PCI: dwc: Set coherent DMA-mask on MSI-address allocation

Message ID 20221219144658.26620-25-Sergey.Semin@baikalelectronics.ru (mailing list archive)
State Superseded
Headers show
Series dmaengine: dw-edma: Add RP/EP local DMA controllers support | expand

Commit Message

Serge Semin Dec. 19, 2022, 2:46 p.m. UTC
The MSI target address requires to be reserved within the lowest 4GB
memory in order to support the PCIe peripherals with no 64-bit MSI TLPs
support. Since the allocation is done from the DMA-coherent memory let's
modify the allocation procedure to setting the coherent DMA-mask only and
avoiding the streaming DMA-mask modification. Thus at least the streaming
DMA operations would work with no artificial limitations. It will be
specifically useful for the eDMA-capable controllers so the corresponding
DMA-engine clients would map the DMA buffers with no need in the SWIOTLB
intervention for the buffers allocated above the 4GB memory region.

While at it let's add a brief comment about the reason of having the MSI
target address allocated from the DMA-coherent memory limited with the 4GB
upper bound.

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>

---

Changelog v8:
- This is a new patch added on v8 stage of the series.
  (@Robin, @Christoph)
---
 drivers/pci/controller/dwc/pcie-designware-host.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

Comments

Robin Murphy Jan. 11, 2023, 1:39 p.m. UTC | #1
On 2022-12-19 14:46, Serge Semin wrote:
> The MSI target address requires to be reserved within the lowest 4GB
> memory in order to support the PCIe peripherals with no 64-bit MSI TLPs
> support. Since the allocation is done from the DMA-coherent memory let's
> modify the allocation procedure to setting the coherent DMA-mask only and
> avoiding the streaming DMA-mask modification. Thus at least the streaming
> DMA operations would work with no artificial limitations. It will be
> specifically useful for the eDMA-capable controllers so the corresponding
> DMA-engine clients would map the DMA buffers with no need in the SWIOTLB
> intervention for the buffers allocated above the 4GB memory region.
> 
> While at it let's add a brief comment about the reason of having the MSI
> target address allocated from the DMA-coherent memory limited with the 4GB
> upper bound.

Reviewed-by: Robin Murphy <robin.murphy@arm.com>

> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> 
> ---
> 
> Changelog v8:
> - This is a new patch added on v8 stage of the series.
>    (@Robin, @Christoph)
> ---
>   drivers/pci/controller/dwc/pcie-designware-host.c | 11 ++++++++++-
>   1 file changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> index 3ab6ae3712c4..e10608af39b4 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> @@ -366,7 +366,16 @@ static int dw_pcie_msi_host_init(struct dw_pcie_rp *pp)
>   						    dw_chained_msi_isr, pp);
>   	}
>   
> -	ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
> +	/*
> +	 * Even though the iMSI-RX Module supports 64-bit addresses some
> +	 * peripheral PCIe devices may lack the 64-bit messages support. In
> +	 * order not to miss MSI TLPs from those devices the MSI target address
> +	 * has to be reserved within the lowest 4GB.
> +	 * Note until there is a better alternative found the reservation is
> +	 * done by allocating from the artificially limited DMA-coherent
> +	 * memory.
> +	 */
> +	ret = dma_set_coherent_mask(dev, DMA_BIT_MASK(32));
>   	if (ret)
>   		dev_warn(dev, "Failed to set DMA mask to 32-bit. Devices with only 32-bit MSI support may not work properly\n");
>
Serge Semin Jan. 12, 2023, 6:46 p.m. UTC | #2
On Wed, Jan 11, 2023 at 01:39:35PM +0000, Robin Murphy wrote:
> On 2022-12-19 14:46, Serge Semin wrote:
> > The MSI target address requires to be reserved within the lowest 4GB
> > memory in order to support the PCIe peripherals with no 64-bit MSI TLPs
> > support. Since the allocation is done from the DMA-coherent memory let's
> > modify the allocation procedure to setting the coherent DMA-mask only and
> > avoiding the streaming DMA-mask modification. Thus at least the streaming
> > DMA operations would work with no artificial limitations. It will be
> > specifically useful for the eDMA-capable controllers so the corresponding
> > DMA-engine clients would map the DMA buffers with no need in the SWIOTLB
> > intervention for the buffers allocated above the 4GB memory region.
> > 
> > While at it let's add a brief comment about the reason of having the MSI
> > target address allocated from the DMA-coherent memory limited with the 4GB
> > upper bound.
> 
> Reviewed-by: Robin Murphy <robin.murphy@arm.com>

Great! Thanks.

-Serge(y)

> 
> > Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> > 
> > ---
> > 
> > Changelog v8:
> > - This is a new patch added on v8 stage of the series.
> >    (@Robin, @Christoph)
> > ---
> >   drivers/pci/controller/dwc/pcie-designware-host.c | 11 ++++++++++-
> >   1 file changed, 10 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> > index 3ab6ae3712c4..e10608af39b4 100644
> > --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> > +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> > @@ -366,7 +366,16 @@ static int dw_pcie_msi_host_init(struct dw_pcie_rp *pp)
> >   						    dw_chained_msi_isr, pp);
> >   	}
> > -	ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
> > +	/*
> > +	 * Even though the iMSI-RX Module supports 64-bit addresses some
> > +	 * peripheral PCIe devices may lack the 64-bit messages support. In
> > +	 * order not to miss MSI TLPs from those devices the MSI target address
> > +	 * has to be reserved within the lowest 4GB.
> > +	 * Note until there is a better alternative found the reservation is
> > +	 * done by allocating from the artificially limited DMA-coherent
> > +	 * memory.
> > +	 */
> > +	ret = dma_set_coherent_mask(dev, DMA_BIT_MASK(32));
> >   	if (ret)
> >   		dev_warn(dev, "Failed to set DMA mask to 32-bit. Devices with only 32-bit MSI support may not work properly\n");
diff mbox series

Patch

diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index 3ab6ae3712c4..e10608af39b4 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -366,7 +366,16 @@  static int dw_pcie_msi_host_init(struct dw_pcie_rp *pp)
 						    dw_chained_msi_isr, pp);
 	}
 
-	ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
+	/*
+	 * Even though the iMSI-RX Module supports 64-bit addresses some
+	 * peripheral PCIe devices may lack the 64-bit messages support. In
+	 * order not to miss MSI TLPs from those devices the MSI target address
+	 * has to be reserved within the lowest 4GB.
+	 * Note until there is a better alternative found the reservation is
+	 * done by allocating from the artificially limited DMA-coherent
+	 * memory.
+	 */
+	ret = dma_set_coherent_mask(dev, DMA_BIT_MASK(32));
 	if (ret)
 		dev_warn(dev, "Failed to set DMA mask to 32-bit. Devices with only 32-bit MSI support may not work properly\n");