[v3,00/10] Preemption support for A7XX

Message

Antonino Maniscalco Sept. 5, 2024, 2:51 p.m. UTC

This series implements preemption for A7XX targets, which allows the GPU to
switch to an higher priority ring when work is pushed to it, reducing latency
for high priority submissions.

This series enables L1 preemption with skip_save_restore which requires
the following userspace patches to function:

https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30544

A flag is added to `msm_submitqueue_create` to only allow submissions
from compatible userspace to be preempted, therefore maintaining
compatibility.

Preemption is currently only enabled by default on A750, it can be
enabled on other targets through the `enable_preemption` module
parameter. This is because more testing is required on other targets.

For testing on other HW it is sufficient to set that parameter to a
value of 1, then using the branch of mesa linked above, `TU_DEBUG=hiprio`
allows to run any application as high priority therefore preempting
submissions from other applications.

The `msm_gpu_preemption_trigger` and `msm_gpu_preemption_irq` traces
added in this series can be used to observe preemption's behavior as
well as measuring preemption latency.

Some commits from this series are based on a previous series to enable
preemption on A6XX targets:

https://lkml.kernel.org/[e-mail]

Signed-off-by: Antonino Maniscalco <[e-mail]>
---
Changes in v3:
- Added documentation about preemption
- Use quirks to determine which target supports preemption
- Add a module parameter to force disabling or enabling preemption
- Clear postamble when profiling
- Define A6XX_CP_CONTEXT_SWITCH_CNTL_LEVEL fields in a6xx.xml
- Make preemption records MAP_PRIV
- Removed user ctx record (NON_PRIV) and patch 2/9 as it's not needed
  anymore
- Link to v2: https://lore.kernel.org/r/[e-mail]

Changes in v2:
- Added preept_record_size for X185 in PATCH 3/7
- Added patches to reset perf counters
- Dropped unused defines
- Dropped unused variable (fixes warning)
- Only enable preemption on a750
- Reject MSM_SUBMITQUEUE_ALLOW_PREEMPT for unsupported targets
- Added Akhil's Reviewed-By tags to patches 1/9,2/9,3/9
- Added Neil's Tested-By tags
- Added explanation for UAPI changes in commit message
- Link to v1: https://lore.kernel.org/r/[e-mail]

---
Antonino Maniscalco (10):
      drm/msm: Fix bv_fence being used as bv_rptr
      drm/msm: Add a `preempt_record_size` field
      drm/msm: Add CONTEXT_SWITCH_CNTL bitfields
      drm/msm/A6xx: Implement preemption for A7XX targets
      drm/msm/A6xx: Sync relevant adreno_pm4.xml changes
      drm/msm/A6xx: Use posamble to reset counters on preemption
      drm/msm/A6xx: Add traces for preemption
      drm/msm/A6XX: Add a flag to allow preemption to submitqueue_create
      drm/msm/A6xx: Enable preemption for A750
      Documentation: document adreno preemption

 Documentation/gpu/msm-preemption.rst               |  98 +++++
 drivers/gpu/drm/msm/Makefile                       |   1 +
 drivers/gpu/drm/msm/adreno/a6xx_catalog.c          |   7 +-
 drivers/gpu/drm/msm/adreno/a6xx_gpu.c              | 331 +++++++++++++++-
 drivers/gpu/drm/msm/adreno/a6xx_gpu.h              | 166 ++++++++
 drivers/gpu/drm/msm/adreno/a6xx_preempt.c          | 430 +++++++++++++++++++++
 drivers/gpu/drm/msm/adreno/adreno_gpu.h            |   9 +-
 drivers/gpu/drm/msm/msm_drv.c                      |   4 +
 drivers/gpu/drm/msm/msm_gpu_trace.h                |  28 ++
 drivers/gpu/drm/msm/msm_ringbuffer.h               |   8 +
 drivers/gpu/drm/msm/msm_submitqueue.c              |   3 +
 drivers/gpu/drm/msm/registers/adreno/a6xx.xml      |   7 +-
 .../gpu/drm/msm/registers/adreno/adreno_pm4.xml    |  39 +-
 include/uapi/drm/msm_drm.h                         |   5 +-
 14 files changed, 1094 insertions(+), 42 deletions(-)
---
base-commit: 7c626ce4bae1ac14f60076d00eafe71af30450ba
change-id: 20240815-preemption-a750-t-fcee9a844b39

Best regards,

Comments

Akhil P Oommen Sept. 6, 2024, 7:58 p.m. UTC | #1

On Thu, Sep 05, 2024 at 04:51:18PM +0200, Antonino Maniscalco wrote:
> This series implements preemption for A7XX targets, which allows the GPU to
> switch to an higher priority ring when work is pushed to it, reducing latency
> for high priority submissions.
> 
> This series enables L1 preemption with skip_save_restore which requires
> the following userspace patches to function:
> 
> https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30544
> 
> A flag is added to `msm_submitqueue_create` to only allow submissions
> from compatible userspace to be preempted, therefore maintaining
> compatibility.
> 
> Preemption is currently only enabled by default on A750, it can be
> enabled on other targets through the `enable_preemption` module
> parameter. This is because more testing is required on other targets.
> 
> For testing on other HW it is sufficient to set that parameter to a
> value of 1, then using the branch of mesa linked above, `TU_DEBUG=hiprio`
> allows to run any application as high priority therefore preempting
> submissions from other applications.
> 
> The `msm_gpu_preemption_trigger` and `msm_gpu_preemption_irq` traces
> added in this series can be used to observe preemption's behavior as
> well as measuring preemption latency.
> 
> Some commits from this series are based on a previous series to enable
> preemption on A6XX targets:
> 
> https://lkml.kernel.org/[e-mail]
> 
> Signed-off-by: Antonino Maniscalco <[e-mail]>

Antonino, can you please test this once with per-process pt disabled to
ensure that is not broken? It is handy sometimes while debugging.
We just need to remove "adreno-smmu" compatible string from gpu smmu
node in DT.

-Akhil.

> ---
> Changes in v3:
> - Added documentation about preemption
> - Use quirks to determine which target supports preemption
> - Add a module parameter to force disabling or enabling preemption
> - Clear postamble when profiling
> - Define A6XX_CP_CONTEXT_SWITCH_CNTL_LEVEL fields in a6xx.xml
> - Make preemption records MAP_PRIV
> - Removed user ctx record (NON_PRIV) and patch 2/9 as it's not needed
>   anymore
> - Link to v2: https://lore.kernel.org/r/[e-mail]
> 
> Changes in v2:
> - Added preept_record_size for X185 in PATCH 3/7
> - Added patches to reset perf counters
> - Dropped unused defines
> - Dropped unused variable (fixes warning)
> - Only enable preemption on a750
> - Reject MSM_SUBMITQUEUE_ALLOW_PREEMPT for unsupported targets
> - Added Akhil's Reviewed-By tags to patches 1/9,2/9,3/9
> - Added Neil's Tested-By tags
> - Added explanation for UAPI changes in commit message
> - Link to v1: https://lore.kernel.org/r/[e-mail]
> 
> ---
> Antonino Maniscalco (10):
>       drm/msm: Fix bv_fence being used as bv_rptr
>       drm/msm: Add a `preempt_record_size` field
>       drm/msm: Add CONTEXT_SWITCH_CNTL bitfields
>       drm/msm/A6xx: Implement preemption for A7XX targets
>       drm/msm/A6xx: Sync relevant adreno_pm4.xml changes
>       drm/msm/A6xx: Use posamble to reset counters on preemption
>       drm/msm/A6xx: Add traces for preemption
>       drm/msm/A6XX: Add a flag to allow preemption to submitqueue_create
>       drm/msm/A6xx: Enable preemption for A750
>       Documentation: document adreno preemption
> 
>  Documentation/gpu/msm-preemption.rst               |  98 +++++
>  drivers/gpu/drm/msm/Makefile                       |   1 +
>  drivers/gpu/drm/msm/adreno/a6xx_catalog.c          |   7 +-
>  drivers/gpu/drm/msm/adreno/a6xx_gpu.c              | 331 +++++++++++++++-
>  drivers/gpu/drm/msm/adreno/a6xx_gpu.h              | 166 ++++++++
>  drivers/gpu/drm/msm/adreno/a6xx_preempt.c          | 430 +++++++++++++++++++++
>  drivers/gpu/drm/msm/adreno/adreno_gpu.h            |   9 +-
>  drivers/gpu/drm/msm/msm_drv.c                      |   4 +
>  drivers/gpu/drm/msm/msm_gpu_trace.h                |  28 ++
>  drivers/gpu/drm/msm/msm_ringbuffer.h               |   8 +
>  drivers/gpu/drm/msm/msm_submitqueue.c              |   3 +
>  drivers/gpu/drm/msm/registers/adreno/a6xx.xml      |   7 +-
>  .../gpu/drm/msm/registers/adreno/adreno_pm4.xml    |  39 +-
>  include/uapi/drm/msm_drm.h                         |   5 +-
>  14 files changed, 1094 insertions(+), 42 deletions(-)
> ---
> base-commit: 7c626ce4bae1ac14f60076d00eafe71af30450ba
> change-id: 20240815-preemption-a750-t-fcee9a844b39
> 
> Best regards,
> -- 
> Antonino Maniscalco <[e-mail]>
>

Rob Clark Sept. 9, 2024, 3:36 p.m. UTC | #2

On Fri, Sep 6, 2024 at 12:59 PM Akhil P Oommen <[e-mail]> wrote:
>
> On Thu, Sep 05, 2024 at 04:51:18PM +0200, Antonino Maniscalco wrote:
> > This series implements preemption for A7XX targets, which allows the GPU to
> > switch to an higher priority ring when work is pushed to it, reducing latency
> > for high priority submissions.
> >
> > This series enables L1 preemption with skip_save_restore which requires
> > the following userspace patches to function:
> >
> > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30544
> >
> > A flag is added to `msm_submitqueue_create` to only allow submissions
> > from compatible userspace to be preempted, therefore maintaining
> > compatibility.
> >
> > Preemption is currently only enabled by default on A750, it can be
> > enabled on other targets through the `enable_preemption` module
> > parameter. This is because more testing is required on other targets.
> >
> > For testing on other HW it is sufficient to set that parameter to a
> > value of 1, then using the branch of mesa linked above, `TU_DEBUG=hiprio`
> > allows to run any application as high priority therefore preempting
> > submissions from other applications.
> >
> > The `msm_gpu_preemption_trigger` and `msm_gpu_preemption_irq` traces
> > added in this series can be used to observe preemption's behavior as
> > well as measuring preemption latency.
> >
> > Some commits from this series are based on a previous series to enable
> > preemption on A6XX targets:
> >
> > https://lkml.kernel.org/[e-mail]
> >
> > Signed-off-by: Antonino Maniscalco <[e-mail]>
>
> Antonino, can you please test this once with per-process pt disabled to
> ensure that is not broken? It is handy sometimes while debugging.
> We just need to remove "adreno-smmu" compatible string from gpu smmu
> node in DT.

fwiw, I'd be ok supporting preemption on devices that have per-process
pgtables.  (And maybe we should be tainting the kernel if per-process
pgtables are disabled on a6xx+)

BR,
-R

> -Akhil.
>
> > ---
> > Changes in v3:
> > - Added documentation about preemption
> > - Use quirks to determine which target supports preemption
> > - Add a module parameter to force disabling or enabling preemption
> > - Clear postamble when profiling
> > - Define A6XX_CP_CONTEXT_SWITCH_CNTL_LEVEL fields in a6xx.xml
> > - Make preemption records MAP_PRIV
> > - Removed user ctx record (NON_PRIV) and patch 2/9 as it's not needed
> >   anymore
> > - Link to v2: https://lore.kernel.org/r/[e-mail]
> >
> > Changes in v2:
> > - Added preept_record_size for X185 in PATCH 3/7
> > - Added patches to reset perf counters
> > - Dropped unused defines
> > - Dropped unused variable (fixes warning)
> > - Only enable preemption on a750
> > - Reject MSM_SUBMITQUEUE_ALLOW_PREEMPT for unsupported targets
> > - Added Akhil's Reviewed-By tags to patches 1/9,2/9,3/9
> > - Added Neil's Tested-By tags
> > - Added explanation for UAPI changes in commit message
> > - Link to v1: https://lore.kernel.org/r/[e-mail]
> >
> > ---
> > Antonino Maniscalco (10):
> >       drm/msm: Fix bv_fence being used as bv_rptr
> >       drm/msm: Add a `preempt_record_size` field
> >       drm/msm: Add CONTEXT_SWITCH_CNTL bitfields
> >       drm/msm/A6xx: Implement preemption for A7XX targets
> >       drm/msm/A6xx: Sync relevant adreno_pm4.xml changes
> >       drm/msm/A6xx: Use posamble to reset counters on preemption
> >       drm/msm/A6xx: Add traces for preemption
> >       drm/msm/A6XX: Add a flag to allow preemption to submitqueue_create
> >       drm/msm/A6xx: Enable preemption for A750
> >       Documentation: document adreno preemption
> >
> >  Documentation/gpu/msm-preemption.rst               |  98 +++++
> >  drivers/gpu/drm/msm/Makefile                       |   1 +
> >  drivers/gpu/drm/msm/adreno/a6xx_catalog.c          |   7 +-
> >  drivers/gpu/drm/msm/adreno/a6xx_gpu.c              | 331 +++++++++++++++-
> >  drivers/gpu/drm/msm/adreno/a6xx_gpu.h              | 166 ++++++++
> >  drivers/gpu/drm/msm/adreno/a6xx_preempt.c          | 430 +++++++++++++++++++++
> >  drivers/gpu/drm/msm/adreno/adreno_gpu.h            |   9 +-
> >  drivers/gpu/drm/msm/msm_drv.c                      |   4 +
> >  drivers/gpu/drm/msm/msm_gpu_trace.h                |  28 ++
> >  drivers/gpu/drm/msm/msm_ringbuffer.h               |   8 +
> >  drivers/gpu/drm/msm/msm_submitqueue.c              |   3 +
> >  drivers/gpu/drm/msm/registers/adreno/a6xx.xml      |   7 +-
> >  .../gpu/drm/msm/registers/adreno/adreno_pm4.xml    |  39 +-
> >  include/uapi/drm/msm_drm.h                         |   5 +-
> >  14 files changed, 1094 insertions(+), 42 deletions(-)
> > ---
> > base-commit: 7c626ce4bae1ac14f60076d00eafe71af30450ba
> > change-id: 20240815-preemption-a750-t-fcee9a844b39
> >
> > Best regards,
> > --
> > Antonino Maniscalco <[e-mail]>
> >

Antonino Maniscalco Sept. 12, 2024, 1:49 p.m. UTC | #3

On 9/6/24 9:58 PM, Akhil P Oommen wrote:
> On Thu, Sep 05, 2024 at 04:51:18PM +0200, Antonino Maniscalco wrote:
>> This series implements preemption for A7XX targets, which allows the GPU to
>> switch to an higher priority ring when work is pushed to it, reducing latency
>> for high priority submissions.
>>
>> This series enables L1 preemption with skip_save_restore which requires
>> the following userspace patches to function:
>>
>> https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30544
>>
>> A flag is added to `msm_submitqueue_create` to only allow submissions
>> from compatible userspace to be preempted, therefore maintaining
>> compatibility.
>>
>> Preemption is currently only enabled by default on A750, it can be
>> enabled on other targets through the `enable_preemption` module
>> parameter. This is because more testing is required on other targets.
>>
>> For testing on other HW it is sufficient to set that parameter to a
>> value of 1, then using the branch of mesa linked above, `TU_DEBUG=hiprio`
>> allows to run any application as high priority therefore preempting
>> submissions from other applications.
>>
>> The `msm_gpu_preemption_trigger` and `msm_gpu_preemption_irq` traces
>> added in this series can be used to observe preemption's behavior as
>> well as measuring preemption latency.
>>
>> Some commits from this series are based on a previous series to enable
>> preemption on A6XX targets:
>>
>> https://lkml.kernel.org/[e-mail]
>>
>> Signed-off-by: Antonino Maniscalco <[e-mail]>
> 
> Antonino, can you please test this once with per-process pt disabled to
> ensure that is not broken? It is handy sometimes while debugging.
> We just need to remove "adreno-smmu" compatible string from gpu smmu
> node in DT.
> 

Removing that from the DT causes a crash inside 
`msm_iommu_pagetable_create` as it seems `create_private_address_space` 
is assigned unconditionally in a6xx_gpu.c . I tested it with the 
following change:

diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c
index 9e5a83b885f0..4111f5fd9721 100644
--- a/drivers/gpu/drm/msm/msm_gpu.c
+++ b/drivers/gpu/drm/msm/msm_gpu.c
@@ -832,11 +832,11 @@ msm_gpu_create_private_address_space(struct 
msm_gpu *gpu, struct task_struct *ta
          * If the target doesn't support private address spaces then return
          * the global one
          */
-       if (gpu->funcs->create_private_address_space) {
-               aspace = gpu->funcs->create_private_address_space(gpu);
-               if (!IS_ERR(aspace))
-                       aspace->pid = get_pid(task_pid(task));
-       }
+       /* if (gpu->funcs->create_private_address_space) { */
+       /*      aspace = gpu->funcs->create_private_address_space(gpu); */
+       /*      if (!IS_ERR(aspace)) */
+       /*              aspace->pid = get_pid(task_pid(task)); */
+       /* } */

         if (IS_ERR_OR_NULL(aspace))
                 aspace = msm_gem_address_space_get(gpu->aspace);

and it appears to work.

> -Akhil.
> 
>> ---
>> Changes in v3:
>> - Added documentation about preemption
>> - Use quirks to determine which target supports preemption
>> - Add a module parameter to force disabling or enabling preemption
>> - Clear postamble when profiling
>> - Define A6XX_CP_CONTEXT_SWITCH_CNTL_LEVEL fields in a6xx.xml
>> - Make preemption records MAP_PRIV
>> - Removed user ctx record (NON_PRIV) and patch 2/9 as it's not needed
>>    anymore
>> - Link to v2: https://lore.kernel.org/r/[e-mail]
>>
>> Changes in v2:
>> - Added preept_record_size for X185 in PATCH 3/7
>> - Added patches to reset perf counters
>> - Dropped unused defines
>> - Dropped unused variable (fixes warning)
>> - Only enable preemption on a750
>> - Reject MSM_SUBMITQUEUE_ALLOW_PREEMPT for unsupported targets
>> - Added Akhil's Reviewed-By tags to patches 1/9,2/9,3/9
>> - Added Neil's Tested-By tags
>> - Added explanation for UAPI changes in commit message
>> - Link to v1: https://lore.kernel.org/r/[e-mail]
>>
>> ---
>> Antonino Maniscalco (10):
>>        drm/msm: Fix bv_fence being used as bv_rptr
>>        drm/msm: Add a `preempt_record_size` field
>>        drm/msm: Add CONTEXT_SWITCH_CNTL bitfields
>>        drm/msm/A6xx: Implement preemption for A7XX targets
>>        drm/msm/A6xx: Sync relevant adreno_pm4.xml changes
>>        drm/msm/A6xx: Use posamble to reset counters on preemption
>>        drm/msm/A6xx: Add traces for preemption
>>        drm/msm/A6XX: Add a flag to allow preemption to submitqueue_create
>>        drm/msm/A6xx: Enable preemption for A750
>>        Documentation: document adreno preemption
>>
>>   Documentation/gpu/msm-preemption.rst               |  98 +++++
>>   drivers/gpu/drm/msm/Makefile                       |   1 +
>>   drivers/gpu/drm/msm/adreno/a6xx_catalog.c          |   7 +-
>>   drivers/gpu/drm/msm/adreno/a6xx_gpu.c              | 331 +++++++++++++++-
>>   drivers/gpu/drm/msm/adreno/a6xx_gpu.h              | 166 ++++++++
>>   drivers/gpu/drm/msm/adreno/a6xx_preempt.c          | 430 +++++++++++++++++++++
>>   drivers/gpu/drm/msm/adreno/adreno_gpu.h            |   9 +-
>>   drivers/gpu/drm/msm/msm_drv.c                      |   4 +
>>   drivers/gpu/drm/msm/msm_gpu_trace.h                |  28 ++
>>   drivers/gpu/drm/msm/msm_ringbuffer.h               |   8 +
>>   drivers/gpu/drm/msm/msm_submitqueue.c              |   3 +
>>   drivers/gpu/drm/msm/registers/adreno/a6xx.xml      |   7 +-
>>   .../gpu/drm/msm/registers/adreno/adreno_pm4.xml    |  39 +-
>>   include/uapi/drm/msm_drm.h                         |   5 +-
>>   14 files changed, 1094 insertions(+), 42 deletions(-)
>> ---
>> base-commit: 7c626ce4bae1ac14f60076d00eafe71af30450ba
>> change-id: 20240815-preemption-a750-t-fcee9a844b39
>>
>> Best regards,
>> -- 
>> Antonino Maniscalco <[e-mail]>
>>

Best regards,