| Message ID | 20231212031914.47339-1-imguzh@gmail.com (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | aaf244141ed7195a9a56e03c2367f4a9d0b727a8 |
| Delegated to: | Kalle Valo |
| Headers | show |
| Series | [v2] wifi: ath11k: fix IOMMU errors on buffer rings | expand |
Zhenghao Gu <imguzh@gmail.com> writes: > virt_to_phys doesn't work on systems with IOMMU enabled, > which have non-identity physical-to-IOVA mappings. Can you give an example of such system? Just curious where you are seeing this. > It leads to IO_PAGE_FAULTs like this: > [IO_PAGE_FAULT domain=0x0023 address=0x1cce00000 flags=0x0020] > and no link can be established. What do you mean with link in this context? Are you talking about 802.11 association? > This patch changes that to dma_map_single(), which works correctly. Good catch. And virt_to_phys() documentation even says this: * This function does not give bus mappings for DMA transfers. In * almost all conceivable cases a device driver should not be using * this function > Tested-on: QCN9074 hw1.0 PCI WLAN.HK.2.7.0.1-01744-QCAHKSWPL_SILICONZ-1 > Signed-off-by: Zhenghao Gu <imguzh@gmail.com> Jeff, are you ok with this? I did some cosmetics changes in the pending branch (removed unnecessary parenthesis, reverse xmas tree etc), please check: https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/commit/?h=pending&id=fefa43d63e1928fce6e8c2bb626900e9ce98ca69
On 1/9/2024 7:41 AM, Kalle Valo wrote: > Zhenghao Gu <imguzh@gmail.com> writes: > >> virt_to_phys doesn't work on systems with IOMMU enabled, >> which have non-identity physical-to-IOVA mappings. > > Can you give an example of such system? Just curious where you are > seeing this. > >> It leads to IO_PAGE_FAULTs like this: >> [IO_PAGE_FAULT domain=0x0023 address=0x1cce00000 flags=0x0020] >> and no link can be established. > > What do you mean with link in this context? Are you talking about 802.11 > association? > >> This patch changes that to dma_map_single(), which works correctly. > > Good catch. And virt_to_phys() documentation even says this: > > * This function does not give bus mappings for DMA transfers. In > * almost all conceivable cases a device driver should not be using > * this function > >> Tested-on: QCN9074 hw1.0 PCI WLAN.HK.2.7.0.1-01744-QCAHKSWPL_SILICONZ-1 >> Signed-off-by: Zhenghao Gu <imguzh@gmail.com> > > Jeff, are you ok with this? > > I did some cosmetics changes in the pending branch (removed unnecessary > parenthesis, reverse xmas tree etc), please check: > > https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/commit/?h=pending&id=fefa43d63e1928fce6e8c2bb626900e9ce98ca69 > LGTM, incorporates the v1 feedback from the engineering team Acked-by: Jeff Johnson <quic_jjohnson@quicinc.com>
On Tue, Jan 9, 2024 at 9:41 AM Kalle Valo <kvalo@kernel.org> wrote: > > Zhenghao Gu <imguzh@gmail.com> writes: > > > virt_to_phys doesn't work on systems with IOMMU enabled, > > which have non-identity physical-to-IOVA mappings. > > Can you give an example of such system? Just curious where you are > seeing this. I'm testing on an AMD system with IOMMU enabled in the BIOS and "iommu=nopt" in the kernel command line. I also noticed https://bugzilla.kernel.org/show_bug.cgi?id=217056 which may be the same issue. > > > It leads to IO_PAGE_FAULTs like this: > > [IO_PAGE_FAULT domain=0x0023 address=0x1cce00000 flags=0x0020] > > and no link can be established. > > What do you mean with link in this context? Are you talking about 802.11 > association? Yes. > > > This patch changes that to dma_map_single(), which works correctly. > > Good catch. And virt_to_phys() documentation even says this: > > * This function does not give bus mappings for DMA transfers. In > * almost all conceivable cases a device driver should not be using > * this function > > > Tested-on: QCN9074 hw1.0 PCI WLAN.HK.2.7.0.1-01744-QCAHKSWPL_SILICONZ-1 > > Signed-off-by: Zhenghao Gu <imguzh@gmail.com> > > Jeff, are you ok with this? > > I did some cosmetics changes in the pending branch (removed unnecessary > parenthesis, reverse xmas tree etc), please check: > > https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/commit/?h=pending&id=fefa43d63e1928fce6e8c2bb626900e9ce98ca69
Zhenghao Gu <imguzh@gmail.com> wrote: > virt_to_phys() doesn't work on systems with IOMMU enabled, which have > non-identity physical-to-IOVA mappings. It leads to IO_PAGE_FAULTs like this: > > [IO_PAGE_FAULT domain=0x0023 address=0x1cce00000 flags=0x0020] > > And no association to the AP can be established. > > This patch changes that to dma_map_single(), which works correctly. Even > virt_to_phys() documentation says device drivers should not use it: > > This function does not give bus mappings for DMA transfers. In > almost all conceivable cases a device driver should not be using > this function > > Tested-on: QCN9074 hw1.0 PCI WLAN.HK.2.7.0.1-01744-QCAHKSWPL_SILICONZ-1 > > Signed-off-by: Zhenghao Gu <imguzh@gmail.com> > Acked-by: Jeff Johnson <quic_jjohnson@quicinc.com> > Signed-off-by: Kalle Valo <quic_kvalo@quicinc.com> Patch applied to ath-next branch of ath.git, thanks. aaf244141ed7 wifi: ath11k: fix IOMMU errors on buffer rings
diff --git a/drivers/net/wireless/ath/ath11k/dp.c b/drivers/net/wireless/ath/ath11k/dp.c index d070bcb3f..c74230e4c 100644 --- a/drivers/net/wireless/ath/ath11k/dp.c +++ b/drivers/net/wireless/ath/ath11k/dp.c @@ -104,11 +104,14 @@ void ath11k_dp_srng_cleanup(struct ath11k_base *ab, struct dp_srng *ring) if (!ring->vaddr_unaligned) return; - if (ring->cached) + if (ring->cached) { + dma_unmap_single(ab->dev, ring->paddr_unaligned, ring->size, + DMA_FROM_DEVICE); kfree(ring->vaddr_unaligned); - else + } else { dma_free_coherent(ab->dev, ring->size, ring->vaddr_unaligned, ring->paddr_unaligned); + } ring->vaddr_unaligned = NULL; } @@ -249,7 +252,17 @@ int ath11k_dp_srng_setup(struct ath11k_base *ab, struct dp_srng *ring, if (cached) { ring->vaddr_unaligned = kzalloc(ring->size, GFP_KERNEL); - ring->paddr_unaligned = virt_to_phys(ring->vaddr_unaligned); + if (!ring->vaddr_unaligned) + return -ENOMEM; + + ring->paddr_unaligned = + dma_map_single(ab->dev, ring->vaddr_unaligned, + ring->size, DMA_FROM_DEVICE); + if (dma_mapping_error(ab->dev, ring->paddr_unaligned)) { + kfree(ring->vaddr_unaligned); + ring->vaddr_unaligned = NULL; + return -ENOMEM; + } } } diff --git a/drivers/net/wireless/ath/ath11k/hal.c b/drivers/net/wireless/ath/ath11k/hal.c index 0a99aa7dd..7e4a8c820 100644 --- a/drivers/net/wireless/ath/ath11k/hal.c +++ b/drivers/net/wireless/ath/ath11k/hal.c @@ -628,15 +628,31 @@ u32 *ath11k_hal_srng_dst_peek(struct ath11k_base *ab, struct hal_srng *srng) return NULL; } +static u32 *ath11k_hal_srng_dst_peek_with_dma(struct ath11k_base *ab, + struct hal_srng *srng, dma_addr_t *paddr) +{ + lockdep_assert_held(&srng->lock); + + if (srng->u.dst_ring.tp != srng->u.dst_ring.cached_hp) { + *paddr = (srng->ring_base_paddr + + sizeof(*srng->ring_base_vaddr) * srng->u.dst_ring.tp); + return (srng->ring_base_vaddr + srng->u.dst_ring.tp); + } + + return NULL; +} + static void ath11k_hal_srng_prefetch_desc(struct ath11k_base *ab, struct hal_srng *srng) { u32 *desc; + dma_addr_t desc_paddr; + /* prefetch only if desc is available */ - desc = ath11k_hal_srng_dst_peek(ab, srng); + desc = ath11k_hal_srng_dst_peek_with_dma(ab, srng, &desc_paddr); if (likely(desc)) { - dma_sync_single_for_cpu(ab->dev, virt_to_phys(desc), + dma_sync_single_for_cpu(ab->dev, desc_paddr, (srng->entry_size * sizeof(u32)), DMA_FROM_DEVICE); prefetch(desc);
virt_to_phys doesn't work on systems with IOMMU enabled, which have non-identity physical-to-IOVA mappings. It leads to IO_PAGE_FAULTs like this: [IO_PAGE_FAULT domain=0x0023 address=0x1cce00000 flags=0x0020] and no link can be established. This patch changes that to dma_map_single(), which works correctly. Tested-on: QCN9074 hw1.0 PCI WLAN.HK.2.7.0.1-01744-QCAHKSWPL_SILICONZ-1 Signed-off-by: Zhenghao Gu <imguzh@gmail.com> --- Changes in v2: - ath11k_hal_srng_dst_peek_with_dma() is made static --- drivers/net/wireless/ath/ath11k/dp.c | 19 ++++++++++++++++--- drivers/net/wireless/ath/ath11k/hal.c | 20 ++++++++++++++++++-- 2 files changed, 34 insertions(+), 5 deletions(-)