[0/9] Implement multi-GPU DMA mappings for KFD

Message

Felix Kuehling April 14, 2021, 6:46 a.m. UTC

This patch series fixes DMA-mappings of system memory (GTT and userptr)
for KFD running on multi-GPU systems with IOMMU enabled. One SG-BO per
GPU is needed to maintain the DMA mappings of each BO.

I ran into some reservation issues when unmapping or freeing DMA-buf
imports. There are a few FIXME comments in this patch series where I'm
hoping for some expert advice. Patches 8 and 9 are some related fixes
in TTM and amdgpu_ttm. I'm pretty sure patch 9 is not the right way to
do this.

Felix Kuehling (9):
  drm/amdgpu: Rename kfd_bo_va_list to kfd_mem_attachment
  drm/amdgpu: Keep a bo-reference per-attachment
  drm/amdgpu: Simplify AQL queue mapping
  drm/amdgpu: Add multi-GPU DMA mapping helpers
  drm/amdgpu: DMA map/unmap when updating GPU mappings
  drm/amdgpu: Move kfd_mem_attach outside reservation
  drm/amdgpu: Add DMA mapping of GTT BOs
  drm/ttm: Don't count pages in SG BOs against pages_limit
  drm/amdgpu: Lock the attached dmabuf in unpopulate

 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h    |  18 +-
 .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  | 535 ++++++++++++------
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  13 +
 drivers/gpu/drm/ttm/ttm_tt.c                  |  27 +-
 4 files changed, 420 insertions(+), 173 deletions(-)

Comments

Felix Kuehling April 14, 2021, 6:50 a.m. UTC | #1

Sorry for the spam. I mis-spelled the amd-gfx list on the To: line of 
this patch series. Please ignore this and see the patch series I sent a 
minute later.

Regards,
   Felix

On 2021-04-14 2:46 a.m., Felix Kuehling wrote:
> This patch series fixes DMA-mappings of system memory (GTT and userptr)
> for KFD running on multi-GPU systems with IOMMU enabled. One SG-BO per
> GPU is needed to maintain the DMA mappings of each BO.
>
> I ran into some reservation issues when unmapping or freeing DMA-buf
> imports. There are a few FIXME comments in this patch series where I'm
> hoping for some expert advice. Patches 8 and 9 are some related fixes
> in TTM and amdgpu_ttm. I'm pretty sure patch 9 is not the right way to
> do this.
>
> Felix Kuehling (9):
>    drm/amdgpu: Rename kfd_bo_va_list to kfd_mem_attachment
>    drm/amdgpu: Keep a bo-reference per-attachment
>    drm/amdgpu: Simplify AQL queue mapping
>    drm/amdgpu: Add multi-GPU DMA mapping helpers
>    drm/amdgpu: DMA map/unmap when updating GPU mappings
>    drm/amdgpu: Move kfd_mem_attach outside reservation
>    drm/amdgpu: Add DMA mapping of GTT BOs
>    drm/ttm: Don't count pages in SG BOs against pages_limit
>    drm/amdgpu: Lock the attached dmabuf in unpopulate
>
>   drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h    |  18 +-
>   .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  | 535 ++++++++++++------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  13 +
>   drivers/gpu/drm/ttm/ttm_tt.c                  |  27 +-
>   4 files changed, 420 insertions(+), 173 deletions(-)
>