| Message ID | 20200304153144.10675-1-tvrtko.ursulin@linux.intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | drm/i915/tgl: WaDisableGPGPUMidThreadPreemption | expand |
On Wed, Mar 04, 2020 at 03:31:44PM +0000, Tvrtko Ursulin wrote: > From: Tvrtko Ursulin <tvrtko.ursulin@intel.com> > > Enable FtrPerCtxtPreemptionGranularityControl bit and select thread- > group as the default preemption level. > > v2: > * Remove register whitelisting (Rafael, Tony). > > Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> > Cc: Michał Winiarski <michal.winiarski@intel.com> > Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> > Cc: piotr.zdunowski@intel.com > Cc: michal.mrozek@intel.com > Cc: Tony Ye <tony.ye@intel.com> > Cc: Rafael Antognolli <rafael.antognolli@intel.com> Thanks for CC'ing me. I also saw a reply from Jason yesterday, but I don't see it in the list now (though my mail client a mess lately). But he asked whether it's possible for Media and OpenCL drivers to also disable mid-thread preemption through the INTERFACE_DESCRIPTOR_DATA, instead of from the kernel side, so we could try to experiment with it in the future. Also, do you have an idea of how broken it is? Or is it just not tested because no driver is currently implementing it? And do you know if the windows 3D drivers implement it at all? I see code in the driver that seems to me that it's only disabled in certain cases... To summarize, I think we should either: 1) Disable mid-thread preemption from the kernel and not whitelist the register (just like you do in this patch); or 2) Not do anything at all from the kernel, and let userspace disable it if needed. I think 2) is better, if it's not an issue to the other userspace drivers (OpenCL and Media). -- Rafael > --- > drivers/gpu/drm/i915/gt/intel_workarounds.c | 9 +++++++-- > 1 file changed, 7 insertions(+), 2 deletions(-) > > diff --git a/drivers/gpu/drm/i915/gt/intel_workarounds.c b/drivers/gpu/drm/i915/gt/intel_workarounds.c > index cb7d85c42f13..7be71a1a5719 100644 > --- a/drivers/gpu/drm/i915/gt/intel_workarounds.c > +++ b/drivers/gpu/drm/i915/gt/intel_workarounds.c > @@ -601,6 +601,11 @@ static void tgl_ctx_workarounds_init(struct intel_engine_cs *engine, > */ > wa_add(wal, FF_MODE2, FF_MODE2_TDS_TIMER_MASK, > FF_MODE2_TDS_TIMER_128, 0); > + > + /* WaDisableGPGPUMidThreadPreemption:tgl */ > + WA_SET_FIELD_MASKED(GEN8_CS_CHICKEN1, > + GEN9_PREEMPT_GPGPU_LEVEL_MASK, > + GEN9_PREEMPT_GPGPU_THREAD_GROUP_LEVEL); > } > > static void > @@ -1475,8 +1480,8 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal) > PSDUNIT_CLKGATE_DIS); > } > > - if (IS_GEN_RANGE(i915, 9, 11)) { > - /* FtrPerCtxtPreemptionGranularityControl:skl,bxt,kbl,cfl,cnl,icl */ > + if (IS_GEN_RANGE(i915, 9, 12)) { > + /* FtrPerCtxtPreemptionGranularityControl:skl,bxt,kbl,cfl,cnl,icl,tgl */ > wa_masked_en(wal, > GEN7_FF_SLICE_CS_CHICKEN1, > GEN9_FFSC_PERCTX_PREEMPT_CTRL); > -- > 2.20.1 >
On 04/03/2020 16:02, Rafael Antognolli wrote: > On Wed, Mar 04, 2020 at 03:31:44PM +0000, Tvrtko Ursulin wrote: >> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com> >> >> Enable FtrPerCtxtPreemptionGranularityControl bit and select thread- >> group as the default preemption level. >> >> v2: >> * Remove register whitelisting (Rafael, Tony). >> >> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> >> Cc: Michał Winiarski <michal.winiarski@intel.com> >> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> >> Cc: piotr.zdunowski@intel.com >> Cc: michal.mrozek@intel.com >> Cc: Tony Ye <tony.ye@intel.com> >> Cc: Rafael Antognolli <rafael.antognolli@intel.com> > > Thanks for CC'ing me. I also saw a reply from Jason yesterday, but I > don't see it in the list now (though my mail client a mess lately). I saw nothing from Jason, but there was an email from you asking about interface descriptors and whitelisting which is why I copied you. > But he asked whether it's possible for Media and OpenCL drivers to > also disable mid-thread preemption through the > INTERFACE_DESCRIPTOR_DATA, instead of from the kernel side, so we could > try to experiment with it in the future. > > Also, do you have an idea of how broken it is? Or is it just not tested > because no driver is currently implementing it? And do you know if the > windows 3D drivers implement it at all? I see code in the driver that > seems to me that it's only disabled in certain cases... > > To summarize, I think we should either: > 1) Disable mid-thread preemption from the kernel and not whitelist > the register (just like you do in this patch); or > 2) Not do anything at all from the kernel, and let userspace disable > it if needed. > > I think 2) is better, if it's not an issue to the other userspace > drivers (OpenCL and Media). I know it is somewhat broken like in https://gitlab.freedesktop.org/drm/intel/issues/1293. And I know OpenCL and Media would prefer i915 to handle it, but that's always the case. :) OpenCL and Media folks are on the thread so can comment if they are okay with handling this themselves. Indeed a blanket ban in i915 means no one can try it out later without further kernel changes. Regards, Tvrtko
On Wed, Mar 04, 2020 at 04:24:13PM +0000, Tvrtko Ursulin wrote: > > On 04/03/2020 16:02, Rafael Antognolli wrote: > > On Wed, Mar 04, 2020 at 03:31:44PM +0000, Tvrtko Ursulin wrote: > > > From: Tvrtko Ursulin <tvrtko.ursulin@intel.com> > > > > > > Enable FtrPerCtxtPreemptionGranularityControl bit and select thread- > > > group as the default preemption level. > > > > > > v2: > > > * Remove register whitelisting (Rafael, Tony). > > > > > > Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> > > > Cc: Michał Winiarski <michal.winiarski@intel.com> > > > Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> > > > Cc: piotr.zdunowski@intel.com > > > Cc: michal.mrozek@intel.com > > > Cc: Tony Ye <tony.ye@intel.com> > > > Cc: Rafael Antognolli <rafael.antognolli@intel.com> > > > > Thanks for CC'ing me. I also saw a reply from Jason yesterday, but I > > don't see it in the list now (though my mail client a mess lately). > > I saw nothing from Jason, but there was an email from you asking about > interface descriptors and whitelisting which is why I copied you. > > > But he asked whether it's possible for Media and OpenCL drivers to > > also disable mid-thread preemption through the > > INTERFACE_DESCRIPTOR_DATA, instead of from the kernel side, so we could > > try to experiment with it in the future. > > > > Also, do you have an idea of how broken it is? Or is it just not tested > > because no driver is currently implementing it? And do you know if the > > windows 3D drivers implement it at all? I see code in the driver that > > seems to me that it's only disabled in certain cases... > > > > To summarize, I think we should either: > > 1) Disable mid-thread preemption from the kernel and not whitelist > > the register (just like you do in this patch); or > > 2) Not do anything at all from the kernel, and let userspace disable > > it if needed. > > > > I think 2) is better, if it's not an issue to the other userspace > > drivers (OpenCL and Media). > > I know it is somewhat broken like in > https://gitlab.freedesktop.org/drm/intel/issues/1293. > > And I know OpenCL and Media would prefer i915 to handle it, but that's > always the case. :) OpenCL and Media folks are on the thread so can comment > if they are okay with handling this themselves. > > Indeed a blanket ban in i915 means no one can try it out later without > further kernel changes. Well, based on your comment from the previous patch: "General thinking is, since MTP is considered not validated / broken / dangerous, i915 should default it off. But yes, whitelisting or not on top is open." Maybe we should simply ban it and be done. So this patch is: Acked-by: Rafael Antognolli <rafael.antognolli@intel.com> > Regards, > > Tvrtko
On Wed, Mar 4, 2020 at 11:01 AM Rafael Antognolli <rafael.antognolli@intel.com> wrote: > > On Wed, Mar 04, 2020 at 04:24:13PM +0000, Tvrtko Ursulin wrote: > > > > On 04/03/2020 16:02, Rafael Antognolli wrote: > > > On Wed, Mar 04, 2020 at 03:31:44PM +0000, Tvrtko Ursulin wrote: > > > > From: Tvrtko Ursulin <tvrtko.ursulin@intel.com> > > > > > > > > Enable FtrPerCtxtPreemptionGranularityControl bit and select thread- > > > > group as the default preemption level. > > > > > > > > v2: > > > > * Remove register whitelisting (Rafael, Tony). > > > > > > > > Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> > > > > Cc: Michał Winiarski <michal.winiarski@intel.com> > > > > Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> > > > > Cc: piotr.zdunowski@intel.com > > > > Cc: michal.mrozek@intel.com > > > > Cc: Tony Ye <tony.ye@intel.com> > > > > Cc: Rafael Antognolli <rafael.antognolli@intel.com> > > > > > > Thanks for CC'ing me. I also saw a reply from Jason yesterday, but I > > > don't see it in the list now (though my mail client a mess lately). > > > > I saw nothing from Jason, but there was an email from you asking about > > interface descriptors and whitelisting which is why I copied you. > > > > > But he asked whether it's possible for Media and OpenCL drivers to > > > also disable mid-thread preemption through the > > > INTERFACE_DESCRIPTOR_DATA, instead of from the kernel side, so we could > > > try to experiment with it in the future. > > > > > > Also, do you have an idea of how broken it is? Or is it just not tested > > > because no driver is currently implementing it? And do you know if the > > > windows 3D drivers implement it at all? I see code in the driver that > > > seems to me that it's only disabled in certain cases... > > > > > > To summarize, I think we should either: > > > 1) Disable mid-thread preemption from the kernel and not whitelist > > > the register (just like you do in this patch); or > > > 2) Not do anything at all from the kernel, and let userspace disable > > > it if needed. > > > > > > I think 2) is better, if it's not an issue to the other userspace > > > drivers (OpenCL and Media). > > > > I know it is somewhat broken like in > > https://gitlab.freedesktop.org/drm/intel/issues/1293. > > > > And I know OpenCL and Media would prefer i915 to handle it, but that's > > always the case. :) OpenCL and Media folks are on the thread so can comment > > if they are okay with handling this themselves. > > > > Indeed a blanket ban in i915 means no one can try it out later without > > further kernel changes. > > Well, based on your comment from the previous patch: > > "General thinking is, since MTP is considered not validated / broken / > dangerous, i915 should default it off. But yes, whitelisting or not on > top is open." > > Maybe we should simply ban it and be done. So this patch is: > > Acked-by: Rafael Antognolli <rafael.antognolli@intel.com> Agreed. If we think that it's broken or is likely to take additional kernel work to enable it properly, we shouldn't allow userspace to turn it on until we know the kernel is in good shape. Just ban it outright and we can figure out white-listing later if and when we get it properly working. Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Quoting Tvrtko Ursulin (2020-03-04 15:31:44) > From: Tvrtko Ursulin <tvrtko.ursulin@intel.com> > > Enable FtrPerCtxtPreemptionGranularityControl bit and select thread- > group as the default preemption level. > > v2: > * Remove register whitelisting (Rafael, Tony). > > Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> > Cc: Michał Winiarski <michal.winiarski@intel.com> > Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> > Cc: piotr.zdunowski@intel.com > Cc: michal.mrozek@intel.com > Cc: Tony Ye <tony.ye@intel.com> > Cc: Rafael Antognolli <rafael.antognolli@intel.com> /me mutters So do we need to disable force-preemption on rcs if we can't do fine-grained preemption? Acked-by: Chris Wilson <chris@chris-wilson.co.uk> -Chris
On 3/5/2020 1:01 AM, Rafael Antognolli wrote: > On Wed, Mar 04, 2020 at 04:24:13PM +0000, Tvrtko Ursulin wrote: >> >> On 04/03/2020 16:02, Rafael Antognolli wrote: >>> On Wed, Mar 04, 2020 at 03:31:44PM +0000, Tvrtko Ursulin wrote: >>>> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com> >>>> >>>> Enable FtrPerCtxtPreemptionGranularityControl bit and select thread- >>>> group as the default preemption level. >>>> >>>> v2: >>>> * Remove register whitelisting (Rafael, Tony). >>>> >>>> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> >>>> Cc: Michał Winiarski <michal.winiarski@intel.com> >>>> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> >>>> Cc: piotr.zdunowski@intel.com >>>> Cc: michal.mrozek@intel.com >>>> Cc: Tony Ye <tony.ye@intel.com> >>>> Cc: Rafael Antognolli <rafael.antognolli@intel.com> >>> >>> Thanks for CC'ing me. I also saw a reply from Jason yesterday, but I >>> don't see it in the list now (though my mail client a mess lately). >> >> I saw nothing from Jason, but there was an email from you asking about >> interface descriptors and whitelisting which is why I copied you. >> >>> But he asked whether it's possible for Media and OpenCL drivers to >>> also disable mid-thread preemption through the >>> INTERFACE_DESCRIPTOR_DATA, instead of from the kernel side, so we could >>> try to experiment with it in the future. >>> >>> Also, do you have an idea of how broken it is? Or is it just not tested >>> because no driver is currently implementing it? And do you know if the >>> windows 3D drivers implement it at all? I see code in the driver that >>> seems to me that it's only disabled in certain cases... >>> >>> To summarize, I think we should either: >>> 1) Disable mid-thread preemption from the kernel and not whitelist >>> the register (just like you do in this patch); or >>> 2) Not do anything at all from the kernel, and let userspace disable >>> it if needed. >>> >>> I think 2) is better, if it's not an issue to the other userspace >>> drivers (OpenCL and Media). >> >> I know it is somewhat broken like in >> https://gitlab.freedesktop.org/drm/intel/issues/1293. >> >> And I know OpenCL and Media would prefer i915 to handle it, but that's >> always the case. :) OpenCL and Media folks are on the thread so can comment >> if they are okay with handling this themselves. >> >> Indeed a blanket ban in i915 means no one can try it out later without >> further kernel changes. > > Well, based on your comment from the previous patch: > > "General thinking is, since MTP is considered not validated / broken / > dangerous, i915 should default it off. But yes, whitelisting or not on > top is open." > > Maybe we should simply ban it and be done. So this patch is: > > Acked-by: Rafael Antognolli <rafael.antognolli@intel.com> Acked-by: Tony Ye <tony.ye@intel.com> > >> Regards, >> >> Tvrtko
On 04/03/2020 19:11, Patchwork wrote: > == Series Details == > > Series: drm/i915/tgl: WaDisableGPGPUMidThreadPreemption > URL : https://patchwork.freedesktop.org/series/74274/ > State : success > > == Summary == > > CI Bug Log - changes from CI_DRM_8066 -> Patchwork_16821 > ==================================================== > > Summary > ------- > > **SUCCESS** > > No regressions found. > > External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16821/index.html > > Known issues > ------------ > > Here are the changes found in Patchwork_16821 that come from known issues: > > ### IGT changes ### > > #### Issues hit #### > > * igt@gem_exec_suspend@basic-s4-devices: > - fi-tgl-y: [PASS][1] -> [FAIL][2] ([CI#94]) > [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8066/fi-tgl-y/igt@gem_exec_suspend@basic-s4-devices.html > [2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16821/fi-tgl-y/igt@gem_exec_suspend@basic-s4-devices.html > > * igt@i915_selftest@live@gem_contexts: > - fi-cml-s: [PASS][3] -> [DMESG-FAIL][4] ([i915#877]) > [3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8066/fi-cml-s/igt@i915_selftest@live@gem_contexts.html > [4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16821/fi-cml-s/igt@i915_selftest@live@gem_contexts.html > > * igt@i915_selftest@live@hugepages: > - fi-tgl-y: [PASS][5] -> [INCOMPLETE][6] ([CI#94]) +1 similar issue > [5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8066/fi-tgl-y/igt@i915_selftest@live@hugepages.html > [6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16821/fi-tgl-y/igt@i915_selftest@live@hugepages.html > > * igt@prime_vgem@basic-fence-flip: > - fi-tgl-y: [PASS][7] -> [DMESG-WARN][8] ([CI#94] / [i915#402]) +1 similar issue > [7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8066/fi-tgl-y/igt@prime_vgem@basic-fence-flip.html > [8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16821/fi-tgl-y/igt@prime_vgem@basic-fence-flip.html > > > #### Possible fixes #### > > * igt@i915_selftest@live@workarounds: > - fi-icl-guc: [DMESG-FAIL][9] ([i915#922]) -> [PASS][10] > [9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8066/fi-icl-guc/igt@i915_selftest@live@workarounds.html > [10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16821/fi-icl-guc/igt@i915_selftest@live@workarounds.html > > * igt@prime_self_import@basic-llseek-bad: > - fi-tgl-y: [DMESG-WARN][11] ([CI#94] / [i915#402]) -> [PASS][12] +1 similar issue > [11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8066/fi-tgl-y/igt@prime_self_import@basic-llseek-bad.html > [12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16821/fi-tgl-y/igt@prime_self_import@basic-llseek-bad.html > > > [CI#94]: https://gitlab.freedesktop.org/gfx-ci/i915-infra/issues/94 > [i915#402]: https://gitlab.freedesktop.org/drm/intel/issues/402 > [i915#877]: https://gitlab.freedesktop.org/drm/intel/issues/877 > [i915#922]: https://gitlab.freedesktop.org/drm/intel/issues/922 > > > Participating hosts (45 -> 42) > ------------------------------ > > Additional (7): fi-kbl-7560u fi-bwr-2160 fi-kbl-7500u fi-cfl-8109u fi-skl-lmem fi-blb-e6850 fi-kbl-r > Missing (10): fi-kbl-soraka fi-icl-1065g7 fi-bdw-gvtdvm fi-byt-squawks fi-bsw-cyan fi-ctg-p8600 fi-bsw-kefka fi-bdw-samus fi-byt-clapper fi-skl-6600u > > > Build changes > ------------- > > * CI: CI-20190529 -> None > * Linux: CI_DRM_8066 -> Patchwork_16821 > > CI-20190529: 20190529 > CI_DRM_8066: 9e1454777a07902c85ce3febcc9648837a2224fd @ git://anongit.freedesktop.org/gfx-ci/linux > IGT_5491: d52794b426ae16630cc1e0354ae435ec98f6174b @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools > Patchwork_16821: 770b14fbde2879037069c8ea3aa5d48a1d283fd4 @ git://anongit.freedesktop.org/gfx-ci/linux > > > == Linux commits == > > 770b14fbde28 drm/i915/tgl: WaDisableGPGPUMidThreadPreemption And pushed, thanks for the reviews! Regards, Tvrtko
>>But he asked whether it's possible for Media and OpenCL drivers to also disable mid-thread preemption through the INTERFACE_DESCRIPTOR_DATA, instead of from the >>kernel side, so we could try to experiment with it in the future.
Interface Descriptor setting only switches the preemption from mid thread to thread group.
It doesn't allow to disable it completely and there are cases where this is required (i.e. VME).
For that we need mmio whitelist.
Michal
On 09/03/2020 17:02, Mrozek, Michal wrote: >>> But he asked whether it's possible for Media and OpenCL drivers to also disable mid-thread preemption through the INTERFACE_DESCRIPTOR_DATA, instead of from the >>kernel side, so we could try to experiment with it in the future. > > Interface Descriptor setting only switches the preemption from mid thread to thread group. > It doesn't allow to disable it completely and there are cases where this is required (i.e. VME). > For that we need mmio whitelist. With "disable it completely" you mean disable preemption completely - go lower than thread-group in granularity? Regards, Tvrtko
>>With "disable it completely" you mean disable preemption completely - go lower than thread-group in granularity?
Yes, disable it completely.
Michal
diff --git a/drivers/gpu/drm/i915/gt/intel_workarounds.c b/drivers/gpu/drm/i915/gt/intel_workarounds.c index cb7d85c42f13..7be71a1a5719 100644 --- a/drivers/gpu/drm/i915/gt/intel_workarounds.c +++ b/drivers/gpu/drm/i915/gt/intel_workarounds.c @@ -601,6 +601,11 @@ static void tgl_ctx_workarounds_init(struct intel_engine_cs *engine, */ wa_add(wal, FF_MODE2, FF_MODE2_TDS_TIMER_MASK, FF_MODE2_TDS_TIMER_128, 0); + + /* WaDisableGPGPUMidThreadPreemption:tgl */ + WA_SET_FIELD_MASKED(GEN8_CS_CHICKEN1, + GEN9_PREEMPT_GPGPU_LEVEL_MASK, + GEN9_PREEMPT_GPGPU_THREAD_GROUP_LEVEL); } static void @@ -1475,8 +1480,8 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal) PSDUNIT_CLKGATE_DIS); } - if (IS_GEN_RANGE(i915, 9, 11)) { - /* FtrPerCtxtPreemptionGranularityControl:skl,bxt,kbl,cfl,cnl,icl */ + if (IS_GEN_RANGE(i915, 9, 12)) { + /* FtrPerCtxtPreemptionGranularityControl:skl,bxt,kbl,cfl,cnl,icl,tgl */ wa_masked_en(wal, GEN7_FF_SLICE_CS_CHICKEN1, GEN9_FFSC_PERCTX_PREEMPT_CTRL);