[RFC,0/5] Add option to disable implicit sync for userspace submits.

Message

Bas Nieuwenhuizen June 1, 2022, 12:40 a.m. UTC

This adds a context option to use DMA_RESV_USAGE_BOOKKEEP for userspace submissions,
based on Christians TTM work.

Disabling implicit sync is something we've wanted in radv for a while for resolving
some corner cases. A more immediate thing that would be solved here is avoiding a
bunch of implicit sync on GPU map/unmap operations as well, which helps with stutter
around sparse maps/unmaps.

I have experimental userspace in radv, but it isn't 100% ready yet. There are still
issues with some games that I'm looking at, but in the meantime I'm looking for early
feedback on the idea.

Besides the debugging an open question is whether it is worth adding the option to
wait on additional explicit syncobj in the VM map/unmap operations. My current radv
code waits on the wait syncobj in userspace on a thread before doing the operation
which results in some corner cases because we can't provide binary syncobj at
submission time (impacting the usual sync file exports). However adding these fences
adds the risk of head of line blocking because all VM operations get executed on the
same ring, so all later operations get blocked by waiting on the fences as well, which
can cause head of line blocking.

I'm looking to get more implementation experience with different games to see if we
need this, but if we need it it would be a somewhat separate addition to the UAPI.

Bas Nieuwenhuizen (5):
  drm/ttm: Refactor num_shared into usage.
  drm/amdgpu: Add separate mode for syncing DMA_RESV_USAGE_BOOKKEEP.
  drm/amdgpu: Allow explicit sync for VM ops.
  drm/amdgpu: Refactor amdgpu_vm_get_pd_bo.
  drm/amdgpu: Add option to disable implicit sync for a context.

 .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  | 21 ++++++------
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c        | 19 ++++++-----
 drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c       |  4 +--
 drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c       | 32 +++++++++++++++++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.h       |  1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c       | 10 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c    | 11 ++++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.h    |  3 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c      | 11 +++++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_sync.h      |  4 +--
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c       |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vkms.c      |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c        |  7 ++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h        |  3 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c    |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c   |  2 +-
 drivers/gpu/drm/amd/amdkfd/kfd_svm.c          |  2 +-
 .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c |  2 +-
 drivers/gpu/drm/qxl/qxl_release.c             |  2 +-
 drivers/gpu/drm/radeon/radeon_cs.c            |  5 +--
 drivers/gpu/drm/radeon/radeon_gem.c           |  2 +-
 drivers/gpu/drm/radeon/radeon_vm.c            |  4 +--
 drivers/gpu/drm/ttm/ttm_execbuf_util.c        |  5 ++-
 drivers/gpu/drm/vmwgfx/vmwgfx_resource.c      | 10 +++---
 drivers/gpu/drm/vmwgfx/vmwgfx_validation.c    |  2 +-
 include/drm/ttm/ttm_execbuf_util.h            |  3 +-
 include/uapi/drm/amdgpu_drm.h                 |  3 ++
 28 files changed, 112 insertions(+), 63 deletions(-)