| Message ID | 20240711100038.268803-2-vladimir.lypak@gmail.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | fixes for Adreno A5Xx preemption | expand |
On Thu, Jul 11, 2024 at 3:02 AM Vladimir Lypak <vladimir.lypak@gmail.com> wrote: > > Fine grain preemption (switching from/to points within submits) > requires extra handling in command stream of those submits, especially > when rendering with tiling (using GMEM). However this handling is > missing at this point in mesa (and always was). For this reason we get > random GPU faults and hangs if more than one priority level is used > because local preemption is enabled prior to executing command stream > from submit. > With that said it was ahead of time to enable local preemption by > default considering the fact that even on downstream kernel it is only > enabled if requested via UAPI. > > Fixes: a7a4c19c36de ("drm/msm/a5xx: fix setting of the CP_PREEMPT_ENABLE_LOCAL register") > Signed-off-by: Vladimir Lypak <vladimir.lypak@gmail.com> > --- > drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 8 ++++++-- > 1 file changed, 6 insertions(+), 2 deletions(-) > > diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c > index c0b5373e90d7..6c80d3003966 100644 > --- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c > +++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c > @@ -150,9 +150,13 @@ static void a5xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) > OUT_PKT7(ring, CP_SET_PROTECTED_MODE, 1); > OUT_RING(ring, 1); > > - /* Enable local preemption for finegrain preemption */ > + /* > + * Disable local preemption by default because it requires > + * user-space to be aware of it and provide additional handling > + * to restore rendering state or do various flushes on switch. > + */ > OUT_PKT7(ring, CP_PREEMPT_ENABLE_LOCAL, 1); > - OUT_RING(ring, 0x1); > + OUT_RING(ring, 0x0); From a quick look at the a530 pfp fw, it looks like CP_PREEMPT_ENABLE_LOCAL is allowed in IB1/IB2 (ie. not restricted to kernel RB). So we should just disable it in the kernel, and let userspace send a CP_PREEMPT_ENABLE_LOCAL to enable local preemption. BR, -R > /* Allow CP_CONTEXT_SWITCH_YIELD packets in the IB2 */ > OUT_PKT7(ring, CP_YIELD_ENABLE, 1); > -- > 2.45.2 >
On Mon, Jul 15, 2024 at 02:00:10PM -0700, Rob Clark wrote: > On Thu, Jul 11, 2024 at 3:02 AM Vladimir Lypak <vladimir.lypak@gmail.com> wrote: > > > > Fine grain preemption (switching from/to points within submits) > > requires extra handling in command stream of those submits, especially > > when rendering with tiling (using GMEM). However this handling is > > missing at this point in mesa (and always was). For this reason we get > > random GPU faults and hangs if more than one priority level is used > > because local preemption is enabled prior to executing command stream > > from submit. > > With that said it was ahead of time to enable local preemption by > > default considering the fact that even on downstream kernel it is only > > enabled if requested via UAPI. > > > > Fixes: a7a4c19c36de ("drm/msm/a5xx: fix setting of the CP_PREEMPT_ENABLE_LOCAL register") > > Signed-off-by: Vladimir Lypak <vladimir.lypak@gmail.com> > > --- > > drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 8 ++++++-- > > 1 file changed, 6 insertions(+), 2 deletions(-) > > > > diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c > > index c0b5373e90d7..6c80d3003966 100644 > > --- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c > > +++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c > > @@ -150,9 +150,13 @@ static void a5xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) > > OUT_PKT7(ring, CP_SET_PROTECTED_MODE, 1); > > OUT_RING(ring, 1); > > > > - /* Enable local preemption for finegrain preemption */ > > + /* > > + * Disable local preemption by default because it requires > > + * user-space to be aware of it and provide additional handling > > + * to restore rendering state or do various flushes on switch. > > + */ > > OUT_PKT7(ring, CP_PREEMPT_ENABLE_LOCAL, 1); > > - OUT_RING(ring, 0x1); > > + OUT_RING(ring, 0x0); > > From a quick look at the a530 pfp fw, it looks like > CP_PREEMPT_ENABLE_LOCAL is allowed in IB1/IB2 (ie. not restricted to > kernel RB). So we should just disable it in the kernel, and let > userspace send a CP_PREEMPT_ENABLE_LOCAL to enable local preemption. Ack. AFAIU about a5x preemption, this should work. -Akhil > > BR, > -R > > > /* Allow CP_CONTEXT_SWITCH_YIELD packets in the IB2 */ > > OUT_PKT7(ring, CP_YIELD_ENABLE, 1); > > -- > > 2.45.2 > >
diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c index c0b5373e90d7..6c80d3003966 100644 --- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c @@ -150,9 +150,13 @@ static void a5xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) OUT_PKT7(ring, CP_SET_PROTECTED_MODE, 1); OUT_RING(ring, 1); - /* Enable local preemption for finegrain preemption */ + /* + * Disable local preemption by default because it requires + * user-space to be aware of it and provide additional handling + * to restore rendering state or do various flushes on switch. + */ OUT_PKT7(ring, CP_PREEMPT_ENABLE_LOCAL, 1); - OUT_RING(ring, 0x1); + OUT_RING(ring, 0x0); /* Allow CP_CONTEXT_SWITCH_YIELD packets in the IB2 */ OUT_PKT7(ring, CP_YIELD_ENABLE, 1);
Fine grain preemption (switching from/to points within submits) requires extra handling in command stream of those submits, especially when rendering with tiling (using GMEM). However this handling is missing at this point in mesa (and always was). For this reason we get random GPU faults and hangs if more than one priority level is used because local preemption is enabled prior to executing command stream from submit. With that said it was ahead of time to enable local preemption by default considering the fact that even on downstream kernel it is only enabled if requested via UAPI. Fixes: a7a4c19c36de ("drm/msm/a5xx: fix setting of the CP_PREEMPT_ENABLE_LOCAL register") Signed-off-by: Vladimir Lypak <vladimir.lypak@gmail.com> --- drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)