| Message ID | 20230208105130.3233420-1-andrzej.hajda@intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v4] drm/i915: add guard page to ggtt->error_capture | expand |
On 08/02/2023 10:51, Andrzej Hajda wrote: > Write-combining memory allows speculative reads by CPU. > ggtt->error_capture is WC mapped to CPU, so CPU/MMU can try > to prefetch memory beyond the error_capture, ie it tries > to read memory pointed by next PTE in GGTT. > If this PTE points to invalid address DMAR errors will occur. > This behaviour was observed on ADL, RPL, DG2 platforms. Note that DG2 doesn't use this path for error capture, since it lacks mappable aperture. Do you know if CI sees any DMAR errors related to error capture on DG2/DG1? > To avoid it, guard scratch page should be added after error_capture. > The patch fixes the most annoying issue with error capture but > since WC reads are used also in other places there is a risk similar > problem can affect them as well. > > Signed-off-by: Andrzej Hajda <andrzej.hajda@intel.com> > Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> > --- > This patch tries to diminish plague of DMAR read errors present > in CI for ADL*, RPL*, DG2 platforms, see for example [1] (grep DMAR). > CI is usually tolerant for these errors, so the scale of the problem > is not really visible. > To show it I have counted lines containing DMAR read errors in dmesgs > produced by CI for all three versions of the patch, but in contrast to v2 > I have grepped only for lines containing "PTE Read access". > Below stats for kernel w/o patch vs patched one. > v1: 210 vs 0 > v2: 201 vs 0 > v3: 214 vs 0 > Apparently the patch fixes all common PTE read errors. > > In previous version there were different numbers due to less exact grepping, > "grep DMAR" catched write errors and "DMAR: DRHD: handling fault status reg" > lines, anyway the actual number of errors is much bigger - DMAR errors > are rate-limited. > > [1]: http://gfx-ci.igk.intel.com/tree/drm-tip/CI_DRM_12678/bat-adln-1/dmesg0.txt > > Changelog: > v2: > - modified commit message (I hope the diagnosis is correct), > - added bug checks to ensure scratch is initialized on gen3 platforms. > CI produces strange stacktrace for it suggesting scratch[0] is NULL, > to be removed after resolving the issue with gen3 platforms. > v3: > - removed bug checks, replaced with gen check. > v4: > - change code for scratch page insertion to support all platforms, > - add info in commit message there could be more similar issues > > Regards > Andrzej > --- > drivers/gpu/drm/i915/gt/intel_ggtt.c | 31 ++++++++++++++++++++++++---- > 1 file changed, 27 insertions(+), 4 deletions(-) > > diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c > index 842e69c7b21e49..6566d2066f1f8b 100644 > --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c > +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c > @@ -503,6 +503,21 @@ static void cleanup_init_ggtt(struct i915_ggtt *ggtt) > mutex_destroy(&ggtt->error_mutex); > } > > +static void > +ggtt_insert_scratch_pages(struct i915_ggtt *ggtt, u64 offset, u64 length) > +{ > + struct i915_address_space *vm = &ggtt->vm; > + > + if (GRAPHICS_VER(ggtt->vm.i915) < 8) > + return vm->clear_range(vm, offset, length); > + /* clear_range since gen8 is nop */ > + while (length > 0) { > + vm->insert_page(vm, px_dma(vm->scratch[0]), offset, I915_CACHE_NONE, 0); > + offset += I915_GTT_PAGE_SIZE; > + length -= I915_GTT_PAGE_SIZE; > + } > +} > + > static int init_ggtt(struct i915_ggtt *ggtt) > { > /* > @@ -551,8 +566,12 @@ static int init_ggtt(struct i915_ggtt *ggtt) > * paths, and we trust that 0 will remain reserved. However, > * the only likely reason for failure to insert is a driver > * bug, which we expect to cause other failures... > + * > + * Since CPU can perform speculative reads on error capture > + * (write-combining allows it) add scratch page after error > + * capture to avoid DMAR errors. > */ > - ggtt->error_capture.size = I915_GTT_PAGE_SIZE; > + ggtt->error_capture.size = 2 * I915_GTT_PAGE_SIZE; > ggtt->error_capture.color = I915_COLOR_UNEVICTABLE; > if (drm_mm_reserve_node(&ggtt->vm.mm, &ggtt->error_capture)) > drm_mm_insert_node_in_range(&ggtt->vm.mm, > @@ -562,11 +581,15 @@ static int init_ggtt(struct i915_ggtt *ggtt) > 0, ggtt->mappable_end, > DRM_MM_INSERT_LOW); > } > - if (drm_mm_node_allocated(&ggtt->error_capture)) > + if (drm_mm_node_allocated(&ggtt->error_capture)) { > + u64 start = ggtt->error_capture.start; > + u64 size = ggtt->error_capture.size; > + > + ggtt_insert_scratch_pages(ggtt, start, size); > drm_dbg(&ggtt->vm.i915->drm, > "Reserved GGTT:[%llx, %llx] for use by error capture\n", > - ggtt->error_capture.start, > - ggtt->error_capture.start + ggtt->error_capture.size); > + start, start + size); > + } > > /* > * The upper portion of the GuC address space has a sizeable hole
On 08.02.2023 12:03, Matthew Auld wrote: > On 08/02/2023 10:51, Andrzej Hajda wrote: >> Write-combining memory allows speculative reads by CPU. >> ggtt->error_capture is WC mapped to CPU, so CPU/MMU can try >> to prefetch memory beyond the error_capture, ie it tries >> to read memory pointed by next PTE in GGTT. >> If this PTE points to invalid address DMAR errors will occur. >> This behaviour was observed on ADL, RPL, DG2 platforms. > > Note that DG2 doesn't use this path for error capture, since it lacks > mappable aperture. Do you know if CI sees any DMAR errors related to > error capture on DG2/DG1? I have not tested personally but CI confirms it [1] (grep for DMAR:), but only on bat-dg2-11. I am not sure why only this one, but my patch silences DMAR errors on it. [1]: http://gfx-ci.igk.intel.com/tree/drm-tip/CI_DRM_12680/bat-dg2-11/dmesg0.txt Regards Andrzej > >> To avoid it, guard scratch page should be added after error_capture. >> The patch fixes the most annoying issue with error capture but >> since WC reads are used also in other places there is a risk similar >> problem can affect them as well. >> >> Signed-off-by: Andrzej Hajda <andrzej.hajda@intel.com> >> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> >> --- >> This patch tries to diminish plague of DMAR read errors present >> in CI for ADL*, RPL*, DG2 platforms, see for example [1] (grep DMAR). >> CI is usually tolerant for these errors, so the scale of the problem >> is not really visible. >> To show it I have counted lines containing DMAR read errors in dmesgs >> produced by CI for all three versions of the patch, but in contrast >> to v2 >> I have grepped only for lines containing "PTE Read access". >> Below stats for kernel w/o patch vs patched one. >> v1: 210 vs 0 >> v2: 201 vs 0 >> v3: 214 vs 0 >> Apparently the patch fixes all common PTE read errors. >> >> In previous version there were different numbers due to less exact >> grepping, >> "grep DMAR" catched write errors and "DMAR: DRHD: handling fault >> status reg" >> lines, anyway the actual number of errors is much bigger - DMAR errors >> are rate-limited. >> >> [1]: >> http://gfx-ci.igk.intel.com/tree/drm-tip/CI_DRM_12678/bat-adln-1/dmesg0.txt >> >> Changelog: >> v2: >> - modified commit message (I hope the diagnosis is correct), >> - added bug checks to ensure scratch is initialized on gen3 >> platforms. >> CI produces strange stacktrace for it suggesting scratch[0] is >> NULL, >> to be removed after resolving the issue with gen3 platforms. >> v3: >> - removed bug checks, replaced with gen check. >> v4: >> - change code for scratch page insertion to support all platforms, >> - add info in commit message there could be more similar issues >> >> Regards >> Andrzej >> --- >> drivers/gpu/drm/i915/gt/intel_ggtt.c | 31 ++++++++++++++++++++++++---- >> 1 file changed, 27 insertions(+), 4 deletions(-) >> >> diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c >> b/drivers/gpu/drm/i915/gt/intel_ggtt.c >> index 842e69c7b21e49..6566d2066f1f8b 100644 >> --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c >> +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c >> @@ -503,6 +503,21 @@ static void cleanup_init_ggtt(struct i915_ggtt >> *ggtt) >> mutex_destroy(&ggtt->error_mutex); >> } >> +static void >> +ggtt_insert_scratch_pages(struct i915_ggtt *ggtt, u64 offset, u64 >> length) >> +{ >> + struct i915_address_space *vm = &ggtt->vm; >> + >> + if (GRAPHICS_VER(ggtt->vm.i915) < 8) >> + return vm->clear_range(vm, offset, length); >> + /* clear_range since gen8 is nop */ >> + while (length > 0) { >> + vm->insert_page(vm, px_dma(vm->scratch[0]), offset, >> I915_CACHE_NONE, 0); >> + offset += I915_GTT_PAGE_SIZE; >> + length -= I915_GTT_PAGE_SIZE; >> + } >> +} >> + >> static int init_ggtt(struct i915_ggtt *ggtt) >> { >> /* >> @@ -551,8 +566,12 @@ static int init_ggtt(struct i915_ggtt *ggtt) >> * paths, and we trust that 0 will remain reserved. However, >> * the only likely reason for failure to insert is a driver >> * bug, which we expect to cause other failures... >> + * >> + * Since CPU can perform speculative reads on error capture >> + * (write-combining allows it) add scratch page after error >> + * capture to avoid DMAR errors. >> */ >> - ggtt->error_capture.size = I915_GTT_PAGE_SIZE; >> + ggtt->error_capture.size = 2 * I915_GTT_PAGE_SIZE; >> ggtt->error_capture.color = I915_COLOR_UNEVICTABLE; >> if (drm_mm_reserve_node(&ggtt->vm.mm, &ggtt->error_capture)) >> drm_mm_insert_node_in_range(&ggtt->vm.mm, >> @@ -562,11 +581,15 @@ static int init_ggtt(struct i915_ggtt *ggtt) >> 0, ggtt->mappable_end, >> DRM_MM_INSERT_LOW); >> } >> - if (drm_mm_node_allocated(&ggtt->error_capture)) >> + if (drm_mm_node_allocated(&ggtt->error_capture)) { >> + u64 start = ggtt->error_capture.start; >> + u64 size = ggtt->error_capture.size; >> + >> + ggtt_insert_scratch_pages(ggtt, start, size); >> drm_dbg(&ggtt->vm.i915->drm, >> "Reserved GGTT:[%llx, %llx] for use by error capture\n", >> - ggtt->error_capture.start, >> - ggtt->error_capture.start + ggtt->error_capture.size); >> + start, start + size); >> + } >> /* >> * The upper portion of the GuC address space has a sizeable hole
On 08.02.2023 12:17, Andrzej Hajda wrote: > > > On 08.02.2023 12:03, Matthew Auld wrote: >> On 08/02/2023 10:51, Andrzej Hajda wrote: >>> Write-combining memory allows speculative reads by CPU. >>> ggtt->error_capture is WC mapped to CPU, so CPU/MMU can try >>> to prefetch memory beyond the error_capture, ie it tries >>> to read memory pointed by next PTE in GGTT. >>> If this PTE points to invalid address DMAR errors will occur. >>> This behaviour was observed on ADL, RPL, DG2 platforms. >> >> Note that DG2 doesn't use this path for error capture, since it lacks >> mappable aperture. Do you know if CI sees any DMAR errors related to >> error capture on DG2/DG1? > > I have not tested personally but CI confirms it [1] (grep for DMAR:), > but only on bat-dg2-11. > I am not sure why only this one, but my patch silences DMAR errors on it. > [1]: > http://gfx-ci.igk.intel.com/tree/drm-tip/CI_DRM_12680/bat-dg2-11/dmesg0.txt OK, bat-dg2-11 has DG2 and ADL-P :) > > Regards > Andrzej > > >> >>> To avoid it, guard scratch page should be added after error_capture. >>> The patch fixes the most annoying issue with error capture but >>> since WC reads are used also in other places there is a risk similar >>> problem can affect them as well. >>> >>> Signed-off-by: Andrzej Hajda <andrzej.hajda@intel.com> >>> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> >>> --- >>> This patch tries to diminish plague of DMAR read errors present >>> in CI for ADL*, RPL*, DG2 platforms, see for example [1] (grep DMAR). >>> CI is usually tolerant for these errors, so the scale of the problem >>> is not really visible. >>> To show it I have counted lines containing DMAR read errors in dmesgs >>> produced by CI for all three versions of the patch, but in contrast >>> to v2 >>> I have grepped only for lines containing "PTE Read access". >>> Below stats for kernel w/o patch vs patched one. >>> v1: 210 vs 0 >>> v2: 201 vs 0 >>> v3: 214 vs 0 >>> Apparently the patch fixes all common PTE read errors. >>> >>> In previous version there were different numbers due to less exact >>> grepping, >>> "grep DMAR" catched write errors and "DMAR: DRHD: handling fault >>> status reg" >>> lines, anyway the actual number of errors is much bigger - DMAR errors >>> are rate-limited. >>> >>> [1]: >>> http://gfx-ci.igk.intel.com/tree/drm-tip/CI_DRM_12678/bat-adln-1/dmesg0.txt >>> >>> Changelog: >>> v2: >>> - modified commit message (I hope the diagnosis is correct), >>> - added bug checks to ensure scratch is initialized on gen3 >>> platforms. >>> CI produces strange stacktrace for it suggesting scratch[0] >>> is NULL, >>> to be removed after resolving the issue with gen3 platforms. >>> v3: >>> - removed bug checks, replaced with gen check. >>> v4: >>> - change code for scratch page insertion to support all platforms, >>> - add info in commit message there could be more similar issues >>> >>> Regards >>> Andrzej >>> --- >>> drivers/gpu/drm/i915/gt/intel_ggtt.c | 31 >>> ++++++++++++++++++++++++---- >>> 1 file changed, 27 insertions(+), 4 deletions(-) >>> >>> diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c >>> b/drivers/gpu/drm/i915/gt/intel_ggtt.c >>> index 842e69c7b21e49..6566d2066f1f8b 100644 >>> --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c >>> +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c >>> @@ -503,6 +503,21 @@ static void cleanup_init_ggtt(struct i915_ggtt >>> *ggtt) >>> mutex_destroy(&ggtt->error_mutex); >>> } >>> +static void >>> +ggtt_insert_scratch_pages(struct i915_ggtt *ggtt, u64 offset, u64 >>> length) >>> +{ >>> + struct i915_address_space *vm = &ggtt->vm; >>> + >>> + if (GRAPHICS_VER(ggtt->vm.i915) < 8) >>> + return vm->clear_range(vm, offset, length); >>> + /* clear_range since gen8 is nop */ >>> + while (length > 0) { >>> + vm->insert_page(vm, px_dma(vm->scratch[0]), offset, >>> I915_CACHE_NONE, 0); >>> + offset += I915_GTT_PAGE_SIZE; >>> + length -= I915_GTT_PAGE_SIZE; >>> + } >>> +} >>> + >>> static int init_ggtt(struct i915_ggtt *ggtt) >>> { >>> /* >>> @@ -551,8 +566,12 @@ static int init_ggtt(struct i915_ggtt *ggtt) >>> * paths, and we trust that 0 will remain reserved. However, >>> * the only likely reason for failure to insert is a driver >>> * bug, which we expect to cause other failures... >>> + * >>> + * Since CPU can perform speculative reads on error capture >>> + * (write-combining allows it) add scratch page after error >>> + * capture to avoid DMAR errors. >>> */ >>> - ggtt->error_capture.size = I915_GTT_PAGE_SIZE; >>> + ggtt->error_capture.size = 2 * I915_GTT_PAGE_SIZE; >>> ggtt->error_capture.color = I915_COLOR_UNEVICTABLE; >>> if (drm_mm_reserve_node(&ggtt->vm.mm, &ggtt->error_capture)) >>> drm_mm_insert_node_in_range(&ggtt->vm.mm, >>> @@ -562,11 +581,15 @@ static int init_ggtt(struct i915_ggtt *ggtt) >>> 0, ggtt->mappable_end, >>> DRM_MM_INSERT_LOW); >>> } >>> - if (drm_mm_node_allocated(&ggtt->error_capture)) >>> + if (drm_mm_node_allocated(&ggtt->error_capture)) { >>> + u64 start = ggtt->error_capture.start; >>> + u64 size = ggtt->error_capture.size; >>> + >>> + ggtt_insert_scratch_pages(ggtt, start, size); >>> drm_dbg(&ggtt->vm.i915->drm, >>> "Reserved GGTT:[%llx, %llx] for use by error capture\n", >>> - ggtt->error_capture.start, >>> - ggtt->error_capture.start + ggtt->error_capture.size); >>> + start, start + size); >>> + } >>> /* >>> * The upper portion of the GuC address space has a sizeable hole >
On 08/02/2023 11:29, Andrzej Hajda wrote: > > > On 08.02.2023 12:17, Andrzej Hajda wrote: >> >> >> On 08.02.2023 12:03, Matthew Auld wrote: >>> On 08/02/2023 10:51, Andrzej Hajda wrote: >>>> Write-combining memory allows speculative reads by CPU. >>>> ggtt->error_capture is WC mapped to CPU, so CPU/MMU can try >>>> to prefetch memory beyond the error_capture, ie it tries >>>> to read memory pointed by next PTE in GGTT. >>>> If this PTE points to invalid address DMAR errors will occur. >>>> This behaviour was observed on ADL, RPL, DG2 platforms. >>> >>> Note that DG2 doesn't use this path for error capture, since it lacks >>> mappable aperture. Do you know if CI sees any DMAR errors related to >>> error capture on DG2/DG1? >> >> I have not tested personally but CI confirms it [1] (grep for DMAR:), >> but only on bat-dg2-11. >> I am not sure why only this one, but my patch silences DMAR errors on it. >> [1]: >> http://gfx-ci.igk.intel.com/tree/drm-tip/CI_DRM_12680/bat-dg2-11/dmesg0.txt > > OK, bat-dg2-11 has DG2 and ADL-P :) Yeah, it looks like it's running all the tests twice, once for DG2 and then again on the ADL, for that CI host. And DMAR errors only appear on ADL. I totally didn't realise that CI was doing that. Also seems potentially wasteful, since we already have dedicated ADL hosts... > >> >> Regards >> Andrzej >> >> >>> >>>> To avoid it, guard scratch page should be added after error_capture. >>>> The patch fixes the most annoying issue with error capture but >>>> since WC reads are used also in other places there is a risk similar >>>> problem can affect them as well. >>>> >>>> Signed-off-by: Andrzej Hajda <andrzej.hajda@intel.com> >>>> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> >>>> --- >>>> This patch tries to diminish plague of DMAR read errors present >>>> in CI for ADL*, RPL*, DG2 platforms, see for example [1] (grep DMAR). >>>> CI is usually tolerant for these errors, so the scale of the problem >>>> is not really visible. >>>> To show it I have counted lines containing DMAR read errors in dmesgs >>>> produced by CI for all three versions of the patch, but in contrast >>>> to v2 >>>> I have grepped only for lines containing "PTE Read access". >>>> Below stats for kernel w/o patch vs patched one. >>>> v1: 210 vs 0 >>>> v2: 201 vs 0 >>>> v3: 214 vs 0 >>>> Apparently the patch fixes all common PTE read errors. >>>> >>>> In previous version there were different numbers due to less exact >>>> grepping, >>>> "grep DMAR" catched write errors and "DMAR: DRHD: handling fault >>>> status reg" >>>> lines, anyway the actual number of errors is much bigger - DMAR errors >>>> are rate-limited. >>>> >>>> [1]: >>>> http://gfx-ci.igk.intel.com/tree/drm-tip/CI_DRM_12678/bat-adln-1/dmesg0.txt >>>> >>>> Changelog: >>>> v2: >>>> - modified commit message (I hope the diagnosis is correct), >>>> - added bug checks to ensure scratch is initialized on gen3 >>>> platforms. >>>> CI produces strange stacktrace for it suggesting scratch[0] >>>> is NULL, >>>> to be removed after resolving the issue with gen3 platforms. >>>> v3: >>>> - removed bug checks, replaced with gen check. >>>> v4: >>>> - change code for scratch page insertion to support all platforms, >>>> - add info in commit message there could be more similar issues >>>> >>>> Regards >>>> Andrzej >>>> --- >>>> drivers/gpu/drm/i915/gt/intel_ggtt.c | 31 >>>> ++++++++++++++++++++++++---- >>>> 1 file changed, 27 insertions(+), 4 deletions(-) >>>> >>>> diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c >>>> b/drivers/gpu/drm/i915/gt/intel_ggtt.c >>>> index 842e69c7b21e49..6566d2066f1f8b 100644 >>>> --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c >>>> +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c >>>> @@ -503,6 +503,21 @@ static void cleanup_init_ggtt(struct i915_ggtt >>>> *ggtt) >>>> mutex_destroy(&ggtt->error_mutex); >>>> } >>>> +static void >>>> +ggtt_insert_scratch_pages(struct i915_ggtt *ggtt, u64 offset, u64 >>>> length) >>>> +{ >>>> + struct i915_address_space *vm = &ggtt->vm; >>>> + >>>> + if (GRAPHICS_VER(ggtt->vm.i915) < 8) >>>> + return vm->clear_range(vm, offset, length); >>>> + /* clear_range since gen8 is nop */ >>>> + while (length > 0) { >>>> + vm->insert_page(vm, px_dma(vm->scratch[0]), offset, >>>> I915_CACHE_NONE, 0); >>>> + offset += I915_GTT_PAGE_SIZE; >>>> + length -= I915_GTT_PAGE_SIZE; >>>> + } >>>> +} >>>> + >>>> static int init_ggtt(struct i915_ggtt *ggtt) >>>> { >>>> /* >>>> @@ -551,8 +566,12 @@ static int init_ggtt(struct i915_ggtt *ggtt) >>>> * paths, and we trust that 0 will remain reserved. However, >>>> * the only likely reason for failure to insert is a driver >>>> * bug, which we expect to cause other failures... >>>> + * >>>> + * Since CPU can perform speculative reads on error capture >>>> + * (write-combining allows it) add scratch page after error >>>> + * capture to avoid DMAR errors. >>>> */ >>>> - ggtt->error_capture.size = I915_GTT_PAGE_SIZE; >>>> + ggtt->error_capture.size = 2 * I915_GTT_PAGE_SIZE; >>>> ggtt->error_capture.color = I915_COLOR_UNEVICTABLE; >>>> if (drm_mm_reserve_node(&ggtt->vm.mm, &ggtt->error_capture)) >>>> drm_mm_insert_node_in_range(&ggtt->vm.mm, >>>> @@ -562,11 +581,15 @@ static int init_ggtt(struct i915_ggtt *ggtt) >>>> 0, ggtt->mappable_end, >>>> DRM_MM_INSERT_LOW); >>>> } >>>> - if (drm_mm_node_allocated(&ggtt->error_capture)) >>>> + if (drm_mm_node_allocated(&ggtt->error_capture)) { >>>> + u64 start = ggtt->error_capture.start; >>>> + u64 size = ggtt->error_capture.size; >>>> + >>>> + ggtt_insert_scratch_pages(ggtt, start, size); >>>> drm_dbg(&ggtt->vm.i915->drm, >>>> "Reserved GGTT:[%llx, %llx] for use by error capture\n", >>>> - ggtt->error_capture.start, >>>> - ggtt->error_capture.start + ggtt->error_capture.size); >>>> + start, start + size); >>>> + } >>>> /* >>>> * The upper portion of the GuC address space has a sizeable hole >> >
On 08.02.2023 20:52, Patchwork wrote: > *Patch Details* > *Series:* drm/i915: add guard page to ggtt->error_capture (rev6) > *URL:* https://patchwork.freedesktop.org/series/113560/ > <https://patchwork.freedesktop.org/series/113560/> > *State:* success > *Details:* > https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113560v6/index.html > <https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113560v6/index.html> > > > CI Bug Log - changes from CI_DRM_12715 -> Patchwork_113560v6 > > > Summary > > *SUCCESS* > > No regressions found. > > External URL: > https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113560v6/index.html > > > Participating hosts (36 -> 35) > > Missing (1): fi-snb-2520m > > > Possible new issues > > Here are the unknown changes that may have been introduced in > Patchwork_113560v6: > > > IGT changes > > > Suppressed > > The following results come from untrusted machines, tests, or statuses. > They do not affect the overall result. > > * igt@i915_selftest@live@reset: > o {bat-rpls-2}: PASS > <https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12715/bat-rpls-2/igt@i915_selftest@live@reset.html> -> ABORT <https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113560v6/bat-rpls-2/igt@i915_selftest@live@reset.html> This 'issue' and below 'possible fixes' are due to know issue with "timed out waiting for forcewake ack request", not related. Regards Andrzej > > > Known issues > > Here are the changes found in Patchwork_113560v6 that come from known > issues: > > > IGT changes > > > Possible fixes > > * > > igt@i915_selftest@live@gt_pm: > > o {bat-rpls-2}: DMESG-FAIL > <https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12715/bat-rpls-2/igt@i915_selftest@live@gt_pm.html> (i915#4258 <https://gitlab.freedesktop.org/drm/intel/issues/4258>) -> PASS <https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113560v6/bat-rpls-2/igt@i915_selftest@live@gt_pm.html> > * > > igt@i915_selftest@live@reset: > > o {bat-rpls-1}: ABORT > <https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12715/bat-rpls-1/igt@i915_selftest@live@reset.html> (i915#4983 <https://gitlab.freedesktop.org/drm/intel/issues/4983>) -> PASS <https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113560v6/bat-rpls-1/igt@i915_selftest@live@reset.html> > > {name}: This element is suppressed. This means it is ignored when computing > the status of the difference (SUCCESS, WARNING, or FAILURE). > > > Build changes > > * Linux: CI_DRM_12715 -> Patchwork_113560v6 > > CI-20190529: 20190529 > CI_DRM_12715: 07d1c54bcdf3fddd723f31d605b4e8c49328affa @ > git://anongit.freedesktop.org/gfx-ci/linux > IGT_7153: f47f859f13376958a2bd199423b1f0ff53dddbe0 @ > https://gitlab.freedesktop.org/drm/igt-gpu-tools.git > Patchwork_113560v6: 07d1c54bcdf3fddd723f31d605b4e8c49328affa @ > git://anongit.freedesktop.org/gfx-ci/linux > > > Linux commits > > 3b4ea4d26bfb drm/i915: add guard page to ggtt->error_capture >
On 08.02.2023 19:39, Patchwork wrote: > *Patch Details* > *Series:* drm/i915: add guard page to ggtt->error_capture (rev5) > *URL:* https://patchwork.freedesktop.org/series/113560/ > <https://patchwork.freedesktop.org/series/113560/> > *State:* success > *Details:* > https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113560v5/index.html > <https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113560v5/index.html> > > > CI Bug Log - changes from CI_DRM_12713 -> Patchwork_113560v5 > > > Summary > > *SUCCESS* > > No regressions found. > > External URL: > https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113560v5/index.html > > > Participating hosts (37 -> 35) > > Missing (2): fi-blb-e6850 fi-snb-2520m > > > Known issues > > Here are the changes found in Patchwork_113560v5 that come from known > issues: > > > IGT changes > > > Issues hit > > * igt@kms_cursor_legacy@basic-busy-flip-before-cursor@atomic-transitions: > o fi-bsw-n3050: PASS > <https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12713/fi-bsw-n3050/igt@kms_cursor_legacy@basic-busy-flip-before-cursor@atomic-transitions.html> -> FAIL <https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113560v5/fi-bsw-n3050/igt@kms_cursor_legacy@basic-busy-flip-before-cursor@atomic-transitions.html> (i915#6298 <https://gitlab.freedesktop.org/drm/intel/issues/6298>) Very common error: http://gfx-ci.igk.intel.com/cibuglog-ng/results/all?query_key=a35c73578e5625e7c07ee8d0cbc3a8bd1645592c Not related Regards Andrzej > > > Possible fixes > > * > > igt@i915_selftest@live@hangcheck: > > o > > {bat-adlm-1}: ABORT > <https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12713/bat-adlm-1/igt@i915_selftest@live@hangcheck.html> -> PASS <https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113560v5/bat-adlm-1/igt@i915_selftest@live@hangcheck.html> > > o > > fi-skl-guc: DMESG-WARN > <https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12713/fi-skl-guc/igt@i915_selftest@live@hangcheck.html> -> PASS <https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113560v5/fi-skl-guc/igt@i915_selftest@live@hangcheck.html> > > * > > igt@i915_selftest@live@requests: > > o {bat-rpls-1}: ABORT > <https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12713/bat-rpls-1/igt@i915_selftest@live@requests.html> (i915#7911 <https://gitlab.freedesktop.org/drm/intel/issues/7911>) -> PASS <https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113560v5/bat-rpls-1/igt@i915_selftest@live@requests.html> > > {name}: This element is suppressed. This means it is ignored when computing > the status of the difference (SUCCESS, WARNING, or FAILURE). > > > Build changes > > * Linux: CI_DRM_12713 -> Patchwork_113560v5 > > CI-20190529: 20190529 > CI_DRM_12713: 5180055794b438ce688a6804afb0af08e626ebab @ > git://anongit.freedesktop.org/gfx-ci/linux > IGT_7153: f47f859f13376958a2bd199423b1f0ff53dddbe0 @ > https://gitlab.freedesktop.org/drm/igt-gpu-tools.git > Patchwork_113560v5: 5180055794b438ce688a6804afb0af08e626ebab @ > git://anongit.freedesktop.org/gfx-ci/linux > > > Linux commits > > 13f608422386 drm/i915: add guard page to ggtt->error_capture >
Hi all, Gently ping on the patch. CI pollution is quite high: $ grep 'PTE Read access' CI/drm-tip/CI_DRM_12768/*/dmesg* | wc -l 308 Regards Andrzej On 08.02.2023 11:51, Andrzej Hajda wrote: > Write-combining memory allows speculative reads by CPU. > ggtt->error_capture is WC mapped to CPU, so CPU/MMU can try > to prefetch memory beyond the error_capture, ie it tries > to read memory pointed by next PTE in GGTT. > If this PTE points to invalid address DMAR errors will occur. > This behaviour was observed on ADL, RPL, DG2 platforms. > To avoid it, guard scratch page should be added after error_capture. > The patch fixes the most annoying issue with error capture but > since WC reads are used also in other places there is a risk similar > problem can affect them as well. > > Signed-off-by: Andrzej Hajda <andrzej.hajda@intel.com> > Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> > --- > This patch tries to diminish plague of DMAR read errors present > in CI for ADL*, RPL*, DG2 platforms, see for example [1] (grep DMAR). > CI is usually tolerant for these errors, so the scale of the problem > is not really visible. > To show it I have counted lines containing DMAR read errors in dmesgs > produced by CI for all three versions of the patch, but in contrast to v2 > I have grepped only for lines containing "PTE Read access". > Below stats for kernel w/o patch vs patched one. > v1: 210 vs 0 > v2: 201 vs 0 > v3: 214 vs 0 > Apparently the patch fixes all common PTE read errors. > > In previous version there were different numbers due to less exact grepping, > "grep DMAR" catched write errors and "DMAR: DRHD: handling fault status reg" > lines, anyway the actual number of errors is much bigger - DMAR errors > are rate-limited. > > [1]: http://gfx-ci.igk.intel.com/tree/drm-tip/CI_DRM_12678/bat-adln-1/dmesg0.txt > > Changelog: > v2: > - modified commit message (I hope the diagnosis is correct), > - added bug checks to ensure scratch is initialized on gen3 platforms. > CI produces strange stacktrace for it suggesting scratch[0] is NULL, > to be removed after resolving the issue with gen3 platforms. > v3: > - removed bug checks, replaced with gen check. > v4: > - change code for scratch page insertion to support all platforms, > - add info in commit message there could be more similar issues > > Regards > Andrzej > --- > drivers/gpu/drm/i915/gt/intel_ggtt.c | 31 ++++++++++++++++++++++++---- > 1 file changed, 27 insertions(+), 4 deletions(-) > > diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c > index 842e69c7b21e49..6566d2066f1f8b 100644 > --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c > +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c > @@ -503,6 +503,21 @@ static void cleanup_init_ggtt(struct i915_ggtt *ggtt) > mutex_destroy(&ggtt->error_mutex); > } > > +static void > +ggtt_insert_scratch_pages(struct i915_ggtt *ggtt, u64 offset, u64 length) > +{ > + struct i915_address_space *vm = &ggtt->vm; > + > + if (GRAPHICS_VER(ggtt->vm.i915) < 8) > + return vm->clear_range(vm, offset, length); > + /* clear_range since gen8 is nop */ > + while (length > 0) { > + vm->insert_page(vm, px_dma(vm->scratch[0]), offset, I915_CACHE_NONE, 0); > + offset += I915_GTT_PAGE_SIZE; > + length -= I915_GTT_PAGE_SIZE; > + } > +} > + > static int init_ggtt(struct i915_ggtt *ggtt) > { > /* > @@ -551,8 +566,12 @@ static int init_ggtt(struct i915_ggtt *ggtt) > * paths, and we trust that 0 will remain reserved. However, > * the only likely reason for failure to insert is a driver > * bug, which we expect to cause other failures... > + * > + * Since CPU can perform speculative reads on error capture > + * (write-combining allows it) add scratch page after error > + * capture to avoid DMAR errors. > */ > - ggtt->error_capture.size = I915_GTT_PAGE_SIZE; > + ggtt->error_capture.size = 2 * I915_GTT_PAGE_SIZE; > ggtt->error_capture.color = I915_COLOR_UNEVICTABLE; > if (drm_mm_reserve_node(&ggtt->vm.mm, &ggtt->error_capture)) > drm_mm_insert_node_in_range(&ggtt->vm.mm, > @@ -562,11 +581,15 @@ static int init_ggtt(struct i915_ggtt *ggtt) > 0, ggtt->mappable_end, > DRM_MM_INSERT_LOW); > } > - if (drm_mm_node_allocated(&ggtt->error_capture)) > + if (drm_mm_node_allocated(&ggtt->error_capture)) { > + u64 start = ggtt->error_capture.start; > + u64 size = ggtt->error_capture.size; > + > + ggtt_insert_scratch_pages(ggtt, start, size); > drm_dbg(&ggtt->vm.i915->drm, > "Reserved GGTT:[%llx, %llx] for use by error capture\n", > - ggtt->error_capture.start, > - ggtt->error_capture.start + ggtt->error_capture.size); > + start, start + size); > + } > > /* > * The upper portion of the GuC address space has a sizeable hole
On Wed, Feb 22, 2023 at 01:05:16PM +0100, Andrzej Hajda wrote: > Hi all, > > Gently ping on the patch. CI pollution is quite high: > $ grep 'PTE Read access' CI/drm-tip/CI_DRM_12768/*/dmesg* | wc -l > 308 > > Regards > Andrzej > > On 08.02.2023 11:51, Andrzej Hajda wrote: > > Write-combining memory allows speculative reads by CPU. > > ggtt->error_capture is WC mapped to CPU, so CPU/MMU can try > > to prefetch memory beyond the error_capture, ie it tries > > to read memory pointed by next PTE in GGTT. > > If this PTE points to invalid address DMAR errors will occur. > > This behaviour was observed on ADL, RPL, DG2 platforms. > > To avoid it, guard scratch page should be added after error_capture. > > The patch fixes the most annoying issue with error capture but > > since WC reads are used also in other places there is a risk similar > > problem can affect them as well. > > > > Signed-off-by: Andrzej Hajda <andrzej.hajda@intel.com> > > Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> > > --- > > This patch tries to diminish plague of DMAR read errors present > > in CI for ADL*, RPL*, DG2 platforms, see for example [1] (grep DMAR). > > CI is usually tolerant for these errors, so the scale of the problem > > is not really visible. is this info not relevant to the commit msg? > > To show it I have counted lines containing DMAR read errors in dmesgs > > produced by CI for all three versions of the patch, but in contrast to v2 > > I have grepped only for lines containing "PTE Read access". > > Below stats for kernel w/o patch vs patched one. > > v1: 210 vs 0 > > v2: 201 vs 0 > > v3: 214 vs 0 > > Apparently the patch fixes all common PTE read errors. > > > > In previous version there were different numbers due to less exact grepping, > > "grep DMAR" catched write errors and "DMAR: DRHD: handling fault status reg" > > lines, anyway the actual number of errors is much bigger - DMAR errors > > are rate-limited. > > > > [1]: http://gfx-ci.igk.intel.com/tree/drm-tip/CI_DRM_12678/bat-adln-1/dmesg0.txt > > > > Changelog: > > v2: > > - modified commit message (I hope the diagnosis is correct), > > - added bug checks to ensure scratch is initialized on gen3 platforms. > > CI produces strange stacktrace for it suggesting scratch[0] is NULL, > > to be removed after resolving the issue with gen3 platforms. > > v3: > > - removed bug checks, replaced with gen check. > > v4: > > - change code for scratch page insertion to support all platforms, > > - add info in commit message there could be more similar issues in general in i915 we keep the history in the commit msg itself as well... But with this CI history behind I would like Tvrtko or Joonas to take a look and merge to drm-intel-gt-next. > > > > Regards > > Andrzej > > --- > > drivers/gpu/drm/i915/gt/intel_ggtt.c | 31 ++++++++++++++++++++++++---- > > 1 file changed, 27 insertions(+), 4 deletions(-) > > > > diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c > > index 842e69c7b21e49..6566d2066f1f8b 100644 > > --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c > > +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c > > @@ -503,6 +503,21 @@ static void cleanup_init_ggtt(struct i915_ggtt *ggtt) > > mutex_destroy(&ggtt->error_mutex); > > } > > +static void > > +ggtt_insert_scratch_pages(struct i915_ggtt *ggtt, u64 offset, u64 length) > > +{ > > + struct i915_address_space *vm = &ggtt->vm; > > + > > + if (GRAPHICS_VER(ggtt->vm.i915) < 8) > > + return vm->clear_range(vm, offset, length); > > + /* clear_range since gen8 is nop */ > > + while (length > 0) { > > + vm->insert_page(vm, px_dma(vm->scratch[0]), offset, I915_CACHE_NONE, 0); > > + offset += I915_GTT_PAGE_SIZE; > > + length -= I915_GTT_PAGE_SIZE; > > + } > > +} > > + > > static int init_ggtt(struct i915_ggtt *ggtt) > > { > > /* > > @@ -551,8 +566,12 @@ static int init_ggtt(struct i915_ggtt *ggtt) > > * paths, and we trust that 0 will remain reserved. However, > > * the only likely reason for failure to insert is a driver > > * bug, which we expect to cause other failures... > > + * > > + * Since CPU can perform speculative reads on error capture > > + * (write-combining allows it) add scratch page after error > > + * capture to avoid DMAR errors. > > */ > > - ggtt->error_capture.size = I915_GTT_PAGE_SIZE; > > + ggtt->error_capture.size = 2 * I915_GTT_PAGE_SIZE; > > ggtt->error_capture.color = I915_COLOR_UNEVICTABLE; > > if (drm_mm_reserve_node(&ggtt->vm.mm, &ggtt->error_capture)) > > drm_mm_insert_node_in_range(&ggtt->vm.mm, > > @@ -562,11 +581,15 @@ static int init_ggtt(struct i915_ggtt *ggtt) > > 0, ggtt->mappable_end, > > DRM_MM_INSERT_LOW); > > } > > - if (drm_mm_node_allocated(&ggtt->error_capture)) > > + if (drm_mm_node_allocated(&ggtt->error_capture)) { > > + u64 start = ggtt->error_capture.start; > > + u64 size = ggtt->error_capture.size; > > + > > + ggtt_insert_scratch_pages(ggtt, start, size); > > drm_dbg(&ggtt->vm.i915->drm, > > "Reserved GGTT:[%llx, %llx] for use by error capture\n", > > - ggtt->error_capture.start, > > - ggtt->error_capture.start + ggtt->error_capture.size); > > + start, start + size); > > + } > > /* > > * The upper portion of the GuC address space has a sizeable hole >
Hi all, As suggested by Rodrigo added CC: Tvrtko and Joonas, see below. On 22.02.2023 21:48, Rodrigo Vivi wrote: > > On Wed, Feb 22, 2023 at 01:05:16PM +0100, Andrzej Hajda wrote: >> Hi all, >> >> Gently ping on the patch. CI pollution is quite high: >> $ grep 'PTE Read access' CI/drm-tip/CI_DRM_12768/*/dmesg* | wc -l >> 308 >> >> Regards >> Andrzej >> >> On 08.02.2023 11:51, Andrzej Hajda wrote: >>> Write-combining memory allows speculative reads by CPU. >>> ggtt->error_capture is WC mapped to CPU, so CPU/MMU can try >>> to prefetch memory beyond the error_capture, ie it tries >>> to read memory pointed by next PTE in GGTT. >>> If this PTE points to invalid address DMAR errors will occur. >>> This behaviour was observed on ADL, RPL, DG2 platforms. >>> To avoid it, guard scratch page should be added after error_capture. >>> The patch fixes the most annoying issue with error capture but >>> since WC reads are used also in other places there is a risk similar >>> problem can affect them as well. >>> >>> Signed-off-by: Andrzej Hajda <andrzej.hajda@intel.com> >>> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> >>> --- >>> This patch tries to diminish plague of DMAR read errors present >>> in CI for ADL*, RPL*, DG2 platforms, see for example [1] (grep DMAR). >>> CI is usually tolerant for these errors, so the scale of the problem >>> is not really visible. > > is this info not relevant to the commit msg? It is, but also seems slightly redundant for me, except the alarming tone :) > >>> To show it I have counted lines containing DMAR read errors in dmesgs >>> produced by CI for all three versions of the patch, but in contrast to v2 >>> I have grepped only for lines containing "PTE Read access". >>> Below stats for kernel w/o patch vs patched one. >>> v1: 210 vs 0 >>> v2: 201 vs 0 >>> v3: 214 vs 0 >>> Apparently the patch fixes all common PTE read errors. >>> >>> In previous version there were different numbers due to less exact grepping, >>> "grep DMAR" catched write errors and "DMAR: DRHD: handling fault status reg" >>> lines, anyway the actual number of errors is much bigger - DMAR errors >>> are rate-limited. >>> >>> [1]: http://gfx-ci.igk.intel.com/tree/drm-tip/CI_DRM_12678/bat-adln-1/dmesg0.txt >>> >>> Changelog: >>> v2: >>> - modified commit message (I hope the diagnosis is correct), >>> - added bug checks to ensure scratch is initialized on gen3 platforms. >>> CI produces strange stacktrace for it suggesting scratch[0] is NULL, >>> to be removed after resolving the issue with gen3 platforms. >>> v3: >>> - removed bug checks, replaced with gen check. >>> v4: >>> - change code for scratch page insertion to support all platforms, >>> - add info in commit message there could be more similar issues > > in general in i915 we keep the history in the commit msg itself as well... > > But with this CI history behind I would like Tvrtko or Joonas to take a look > and merge to drm-intel-gt-next. Tvrtko, Joonas any comments? Regards Andrzej > >>> >>> Regards >>> Andrzej >>> --- >>> drivers/gpu/drm/i915/gt/intel_ggtt.c | 31 ++++++++++++++++++++++++---- >>> 1 file changed, 27 insertions(+), 4 deletions(-) >>> >>> diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c >>> index 842e69c7b21e49..6566d2066f1f8b 100644 >>> --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c >>> +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c >>> @@ -503,6 +503,21 @@ static void cleanup_init_ggtt(struct i915_ggtt *ggtt) >>> mutex_destroy(&ggtt->error_mutex); >>> } >>> +static void >>> +ggtt_insert_scratch_pages(struct i915_ggtt *ggtt, u64 offset, u64 length) >>> +{ >>> + struct i915_address_space *vm = &ggtt->vm; >>> + >>> + if (GRAPHICS_VER(ggtt->vm.i915) < 8) >>> + return vm->clear_range(vm, offset, length); >>> + /* clear_range since gen8 is nop */ >>> + while (length > 0) { >>> + vm->insert_page(vm, px_dma(vm->scratch[0]), offset, I915_CACHE_NONE, 0); >>> + offset += I915_GTT_PAGE_SIZE; >>> + length -= I915_GTT_PAGE_SIZE; >>> + } >>> +} >>> + >>> static int init_ggtt(struct i915_ggtt *ggtt) >>> { >>> /* >>> @@ -551,8 +566,12 @@ static int init_ggtt(struct i915_ggtt *ggtt) >>> * paths, and we trust that 0 will remain reserved. However, >>> * the only likely reason for failure to insert is a driver >>> * bug, which we expect to cause other failures... >>> + * >>> + * Since CPU can perform speculative reads on error capture >>> + * (write-combining allows it) add scratch page after error >>> + * capture to avoid DMAR errors. >>> */ >>> - ggtt->error_capture.size = I915_GTT_PAGE_SIZE; >>> + ggtt->error_capture.size = 2 * I915_GTT_PAGE_SIZE; >>> ggtt->error_capture.color = I915_COLOR_UNEVICTABLE; >>> if (drm_mm_reserve_node(&ggtt->vm.mm, &ggtt->error_capture)) >>> drm_mm_insert_node_in_range(&ggtt->vm.mm, >>> @@ -562,11 +581,15 @@ static int init_ggtt(struct i915_ggtt *ggtt) >>> 0, ggtt->mappable_end, >>> DRM_MM_INSERT_LOW); >>> } >>> - if (drm_mm_node_allocated(&ggtt->error_capture)) >>> + if (drm_mm_node_allocated(&ggtt->error_capture)) { >>> + u64 start = ggtt->error_capture.start; >>> + u64 size = ggtt->error_capture.size; >>> + >>> + ggtt_insert_scratch_pages(ggtt, start, size); >>> drm_dbg(&ggtt->vm.i915->drm, >>> "Reserved GGTT:[%llx, %llx] for use by error capture\n", >>> - ggtt->error_capture.start, >>> - ggtt->error_capture.start + ggtt->error_capture.size); >>> + start, start + size); >>> + } >>> /* >>> * The upper portion of the GuC address space has a sizeable hole >>
On 08/02/2023 10:51, Andrzej Hajda wrote: > Write-combining memory allows speculative reads by CPU. > ggtt->error_capture is WC mapped to CPU, so CPU/MMU can try > to prefetch memory beyond the error_capture, ie it tries > to read memory pointed by next PTE in GGTT. > If this PTE points to invalid address DMAR errors will occur. > This behaviour was observed on ADL, RPL, DG2 platforms. > To avoid it, guard scratch page should be added after error_capture. > The patch fixes the most annoying issue with error capture but > since WC reads are used also in other places there is a risk similar > problem can affect them as well. > > Signed-off-by: Andrzej Hajda <andrzej.hajda@intel.com> > Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> Research tells the explanation is plausible so: Acked-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> On patch details below. What about "simiar risk in other places" - are there any plans to asses the potential impact elsewhere? > --- > This patch tries to diminish plague of DMAR read errors present > in CI for ADL*, RPL*, DG2 platforms, see for example [1] (grep DMAR). > CI is usually tolerant for these errors, so the scale of the problem > is not really visible. > To show it I have counted lines containing DMAR read errors in dmesgs > produced by CI for all three versions of the patch, but in contrast to v2 > I have grepped only for lines containing "PTE Read access". > Below stats for kernel w/o patch vs patched one. > v1: 210 vs 0 > v2: 201 vs 0 > v3: 214 vs 0 > Apparently the patch fixes all common PTE read errors. > > In previous version there were different numbers due to less exact grepping, > "grep DMAR" catched write errors and "DMAR: DRHD: handling fault status reg" > lines, anyway the actual number of errors is much bigger - DMAR errors > are rate-limited. > > [1]: http://gfx-ci.igk.intel.com/tree/drm-tip/CI_DRM_12678/bat-adln-1/dmesg0.txt > > Changelog: > v2: > - modified commit message (I hope the diagnosis is correct), > - added bug checks to ensure scratch is initialized on gen3 platforms. > CI produces strange stacktrace for it suggesting scratch[0] is NULL, > to be removed after resolving the issue with gen3 platforms. > v3: > - removed bug checks, replaced with gen check. > v4: > - change code for scratch page insertion to support all platforms, > - add info in commit message there could be more similar issues > > Regards > Andrzej > --- > drivers/gpu/drm/i915/gt/intel_ggtt.c | 31 ++++++++++++++++++++++++---- > 1 file changed, 27 insertions(+), 4 deletions(-) > > diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c > index 842e69c7b21e49..6566d2066f1f8b 100644 > --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c > +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c > @@ -503,6 +503,21 @@ static void cleanup_init_ggtt(struct i915_ggtt *ggtt) > mutex_destroy(&ggtt->error_mutex); > } > > +static void > +ggtt_insert_scratch_pages(struct i915_ggtt *ggtt, u64 offset, u64 length) > +{ > + struct i915_address_space *vm = &ggtt->vm; > + > + if (GRAPHICS_VER(ggtt->vm.i915) < 8) > + return vm->clear_range(vm, offset, length); > + /* clear_range since gen8 is nop */ It would also work to simply drop the <gen8 case and just do the loop below, right? If so would that be clearer? > + while (length > 0) { Maybe add a GEM_BUG_ON if length is not aligned? Granted it may be a huge stretch of imagination.. Regards, Tvrtko > + vm->insert_page(vm, px_dma(vm->scratch[0]), offset, I915_CACHE_NONE, 0); > + offset += I915_GTT_PAGE_SIZE; > + length -= I915_GTT_PAGE_SIZE; > + } > +} > + > static int init_ggtt(struct i915_ggtt *ggtt) > { > /* > @@ -551,8 +566,12 @@ static int init_ggtt(struct i915_ggtt *ggtt) > * paths, and we trust that 0 will remain reserved. However, > * the only likely reason for failure to insert is a driver > * bug, which we expect to cause other failures... > + * > + * Since CPU can perform speculative reads on error capture > + * (write-combining allows it) add scratch page after error > + * capture to avoid DMAR errors. > */ > - ggtt->error_capture.size = I915_GTT_PAGE_SIZE; > + ggtt->error_capture.size = 2 * I915_GTT_PAGE_SIZE; > ggtt->error_capture.color = I915_COLOR_UNEVICTABLE; > if (drm_mm_reserve_node(&ggtt->vm.mm, &ggtt->error_capture)) > drm_mm_insert_node_in_range(&ggtt->vm.mm, > @@ -562,11 +581,15 @@ static int init_ggtt(struct i915_ggtt *ggtt) > 0, ggtt->mappable_end, > DRM_MM_INSERT_LOW); > } > - if (drm_mm_node_allocated(&ggtt->error_capture)) > + if (drm_mm_node_allocated(&ggtt->error_capture)) { > + u64 start = ggtt->error_capture.start; > + u64 size = ggtt->error_capture.size; > + > + ggtt_insert_scratch_pages(ggtt, start, size); > drm_dbg(&ggtt->vm.i915->drm, > "Reserved GGTT:[%llx, %llx] for use by error capture\n", > - ggtt->error_capture.start, > - ggtt->error_capture.start + ggtt->error_capture.size); > + start, start + size); > + } > > /* > * The upper portion of the GuC address space has a sizeable hole
On 02.03.2023 11:43, Tvrtko Ursulin wrote: > > On 08/02/2023 10:51, Andrzej Hajda wrote: >> Write-combining memory allows speculative reads by CPU. >> ggtt->error_capture is WC mapped to CPU, so CPU/MMU can try >> to prefetch memory beyond the error_capture, ie it tries >> to read memory pointed by next PTE in GGTT. >> If this PTE points to invalid address DMAR errors will occur. >> This behaviour was observed on ADL, RPL, DG2 platforms. >> To avoid it, guard scratch page should be added after error_capture. >> The patch fixes the most annoying issue with error capture but >> since WC reads are used also in other places there is a risk similar >> problem can affect them as well. >> >> Signed-off-by: Andrzej Hajda <andrzej.hajda@intel.com> >> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> > > Research tells the explanation is plausible so: > > Acked-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> > Thanks for looking at it. > On patch details below. > > What about "simiar risk in other places" - are there any plans to > asses the potential impact elsewhere? Yes, merging this patch is the 1st step, as error_capture is responsible for most (maybe even all) regular DMAR read errors. After removing this noise it will be much easier to spot other DMAR read errors. There are also multiple regular DMAR write errors (less frequent, but still), which I hope to debug if time permits. Regards Andrzej > >> --- >> This patch tries to diminish plague of DMAR read errors present >> in CI for ADL*, RPL*, DG2 platforms, see for example [1] (grep DMAR). >> CI is usually tolerant for these errors, so the scale of the problem >> is not really visible. >> To show it I have counted lines containing DMAR read errors in dmesgs >> produced by CI for all three versions of the patch, but in contrast >> to v2 >> I have grepped only for lines containing "PTE Read access". >> Below stats for kernel w/o patch vs patched one. >> v1: 210 vs 0 >> v2: 201 vs 0 >> v3: 214 vs 0 >> Apparently the patch fixes all common PTE read errors. >> >> In previous version there were different numbers due to less exact >> grepping, >> "grep DMAR" catched write errors and "DMAR: DRHD: handling fault >> status reg" >> lines, anyway the actual number of errors is much bigger - DMAR errors >> are rate-limited. >> >> [1]: >> http://gfx-ci.igk.intel.com/tree/drm-tip/CI_DRM_12678/bat-adln-1/dmesg0.txt >> >> Changelog: >> v2: >> - modified commit message (I hope the diagnosis is correct), >> - added bug checks to ensure scratch is initialized on gen3 >> platforms. >> CI produces strange stacktrace for it suggesting scratch[0] is >> NULL, >> to be removed after resolving the issue with gen3 platforms. >> v3: >> - removed bug checks, replaced with gen check. >> v4: >> - change code for scratch page insertion to support all platforms, >> - add info in commit message there could be more similar issues >> >> Regards >> Andrzej >> --- >> drivers/gpu/drm/i915/gt/intel_ggtt.c | 31 ++++++++++++++++++++++++---- >> 1 file changed, 27 insertions(+), 4 deletions(-) >> >> diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c >> b/drivers/gpu/drm/i915/gt/intel_ggtt.c >> index 842e69c7b21e49..6566d2066f1f8b 100644 >> --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c >> +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c >> @@ -503,6 +503,21 @@ static void cleanup_init_ggtt(struct i915_ggtt >> *ggtt) >> mutex_destroy(&ggtt->error_mutex); >> } >> +static void >> +ggtt_insert_scratch_pages(struct i915_ggtt *ggtt, u64 offset, u64 >> length) >> +{ >> + struct i915_address_space *vm = &ggtt->vm; >> + >> + if (GRAPHICS_VER(ggtt->vm.i915) < 8) >> + return vm->clear_range(vm, offset, length); >> + /* clear_range since gen8 is nop */ > > It would also work to simply drop the <gen8 case and just do the loop > below, right? If so would that be clearer? > >> + while (length > 0) { > > Maybe add a GEM_BUG_ON if length is not aligned? Granted it may be a > huge stretch of imagination.. > > Regards, > > Tvrtko > >> + vm->insert_page(vm, px_dma(vm->scratch[0]), offset, >> I915_CACHE_NONE, 0); >> + offset += I915_GTT_PAGE_SIZE; >> + length -= I915_GTT_PAGE_SIZE; >> + } >> +} >> + >> static int init_ggtt(struct i915_ggtt *ggtt) >> { >> /* >> @@ -551,8 +566,12 @@ static int init_ggtt(struct i915_ggtt *ggtt) >> * paths, and we trust that 0 will remain reserved. However, >> * the only likely reason for failure to insert is a driver >> * bug, which we expect to cause other failures... >> + * >> + * Since CPU can perform speculative reads on error capture >> + * (write-combining allows it) add scratch page after error >> + * capture to avoid DMAR errors. >> */ >> - ggtt->error_capture.size = I915_GTT_PAGE_SIZE; >> + ggtt->error_capture.size = 2 * I915_GTT_PAGE_SIZE; >> ggtt->error_capture.color = I915_COLOR_UNEVICTABLE; >> if (drm_mm_reserve_node(&ggtt->vm.mm, &ggtt->error_capture)) >> drm_mm_insert_node_in_range(&ggtt->vm.mm, >> @@ -562,11 +581,15 @@ static int init_ggtt(struct i915_ggtt *ggtt) >> 0, ggtt->mappable_end, >> DRM_MM_INSERT_LOW); >> } >> - if (drm_mm_node_allocated(&ggtt->error_capture)) >> + if (drm_mm_node_allocated(&ggtt->error_capture)) { >> + u64 start = ggtt->error_capture.start; >> + u64 size = ggtt->error_capture.size; >> + >> + ggtt_insert_scratch_pages(ggtt, start, size); >> drm_dbg(&ggtt->vm.i915->drm, >> "Reserved GGTT:[%llx, %llx] for use by error capture\n", >> - ggtt->error_capture.start, >> - ggtt->error_capture.start + ggtt->error_capture.size); >> + start, start + size); >> + } >> /* >> * The upper portion of the GuC address space has a sizeable hole
On 02/03/2023 11:00, Andrzej Hajda wrote: > On 02.03.2023 11:43, Tvrtko Ursulin wrote: >> >> On 08/02/2023 10:51, Andrzej Hajda wrote: >>> Write-combining memory allows speculative reads by CPU. >>> ggtt->error_capture is WC mapped to CPU, so CPU/MMU can try >>> to prefetch memory beyond the error_capture, ie it tries >>> to read memory pointed by next PTE in GGTT. >>> If this PTE points to invalid address DMAR errors will occur. >>> This behaviour was observed on ADL, RPL, DG2 platforms. >>> To avoid it, guard scratch page should be added after error_capture. >>> The patch fixes the most annoying issue with error capture but >>> since WC reads are used also in other places there is a risk similar >>> problem can affect them as well. >>> >>> Signed-off-by: Andrzej Hajda <andrzej.hajda@intel.com> >>> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> >> >> Research tells the explanation is plausible so: >> >> Acked-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> >> > > Thanks for looking at it. > >> On patch details below. >> >> What about "simiar risk in other places" - are there any plans to >> asses the potential impact elsewhere? > > Yes, merging this patch is the 1st step, as error_capture is responsible > for most (maybe even all) regular > DMAR read errors. After removing this noise it will be much easier to > spot other DMAR read errors. > There are also multiple regular DMAR write errors (less frequent, but > still), which I hope to debug if time permits. > > Regards > Andrzej > >> >>> --- >>> This patch tries to diminish plague of DMAR read errors present >>> in CI for ADL*, RPL*, DG2 platforms, see for example [1] (grep DMAR). >>> CI is usually tolerant for these errors, so the scale of the problem >>> is not really visible. >>> To show it I have counted lines containing DMAR read errors in dmesgs >>> produced by CI for all three versions of the patch, but in contrast >>> to v2 >>> I have grepped only for lines containing "PTE Read access". >>> Below stats for kernel w/o patch vs patched one. >>> v1: 210 vs 0 >>> v2: 201 vs 0 >>> v3: 214 vs 0 >>> Apparently the patch fixes all common PTE read errors. >>> >>> In previous version there were different numbers due to less exact >>> grepping, >>> "grep DMAR" catched write errors and "DMAR: DRHD: handling fault >>> status reg" >>> lines, anyway the actual number of errors is much bigger - DMAR errors >>> are rate-limited. >>> >>> [1]: >>> http://gfx-ci.igk.intel.com/tree/drm-tip/CI_DRM_12678/bat-adln-1/dmesg0.txt >>> >>> Changelog: >>> v2: >>> - modified commit message (I hope the diagnosis is correct), >>> - added bug checks to ensure scratch is initialized on gen3 >>> platforms. >>> CI produces strange stacktrace for it suggesting scratch[0] is >>> NULL, >>> to be removed after resolving the issue with gen3 platforms. >>> v3: >>> - removed bug checks, replaced with gen check. >>> v4: >>> - change code for scratch page insertion to support all platforms, >>> - add info in commit message there could be more similar issues >>> >>> Regards >>> Andrzej >>> --- >>> drivers/gpu/drm/i915/gt/intel_ggtt.c | 31 ++++++++++++++++++++++++---- >>> 1 file changed, 27 insertions(+), 4 deletions(-) >>> >>> diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c >>> b/drivers/gpu/drm/i915/gt/intel_ggtt.c >>> index 842e69c7b21e49..6566d2066f1f8b 100644 >>> --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c >>> +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c >>> @@ -503,6 +503,21 @@ static void cleanup_init_ggtt(struct i915_ggtt >>> *ggtt) >>> mutex_destroy(&ggtt->error_mutex); >>> } >>> +static void >>> +ggtt_insert_scratch_pages(struct i915_ggtt *ggtt, u64 offset, u64 >>> length) >>> +{ >>> + struct i915_address_space *vm = &ggtt->vm; >>> + >>> + if (GRAPHICS_VER(ggtt->vm.i915) < 8) >>> + return vm->clear_range(vm, offset, length); >>> + /* clear_range since gen8 is nop */ >> >> It would also work to simply drop the <gen8 case and just do the loop >> below, right? If so would that be clearer? Alternatively, if you want to keep the dual path here, perhaps it would be better to replace the gen check with "clear_range !=/== nop_clear_range". That way maybe more of the platform knowledge remains at a central location. Another thing - are there any suspend-resume considerations (i915_ggtt_resume_vm)? Maybe this padding needs to be restored too. Regards, Tvrtko >> >>> + while (length > 0) { >> >> Maybe add a GEM_BUG_ON if length is not aligned? Granted it may be a >> huge stretch of imagination.. >> >> Regards, >> >> Tvrtko >> >>> + vm->insert_page(vm, px_dma(vm->scratch[0]), offset, >>> I915_CACHE_NONE, 0); >>> + offset += I915_GTT_PAGE_SIZE; >>> + length -= I915_GTT_PAGE_SIZE; >>> + } >>> +} >>> + >>> static int init_ggtt(struct i915_ggtt *ggtt) >>> { >>> /* >>> @@ -551,8 +566,12 @@ static int init_ggtt(struct i915_ggtt *ggtt) >>> * paths, and we trust that 0 will remain reserved. However, >>> * the only likely reason for failure to insert is a driver >>> * bug, which we expect to cause other failures... >>> + * >>> + * Since CPU can perform speculative reads on error capture >>> + * (write-combining allows it) add scratch page after error >>> + * capture to avoid DMAR errors. >>> */ >>> - ggtt->error_capture.size = I915_GTT_PAGE_SIZE; >>> + ggtt->error_capture.size = 2 * I915_GTT_PAGE_SIZE; >>> ggtt->error_capture.color = I915_COLOR_UNEVICTABLE; >>> if (drm_mm_reserve_node(&ggtt->vm.mm, &ggtt->error_capture)) >>> drm_mm_insert_node_in_range(&ggtt->vm.mm, >>> @@ -562,11 +581,15 @@ static int init_ggtt(struct i915_ggtt *ggtt) >>> 0, ggtt->mappable_end, >>> DRM_MM_INSERT_LOW); >>> } >>> - if (drm_mm_node_allocated(&ggtt->error_capture)) >>> + if (drm_mm_node_allocated(&ggtt->error_capture)) { >>> + u64 start = ggtt->error_capture.start; >>> + u64 size = ggtt->error_capture.size; >>> + >>> + ggtt_insert_scratch_pages(ggtt, start, size); >>> drm_dbg(&ggtt->vm.i915->drm, >>> "Reserved GGTT:[%llx, %llx] for use by error capture\n", >>> - ggtt->error_capture.start, >>> - ggtt->error_capture.start + ggtt->error_capture.size); >>> + start, start + size); >>> + } >>> /* >>> * The upper portion of the GuC address space has a sizeable hole >
On 03.03.2023 13:01, Tvrtko Ursulin wrote: > > On 02/03/2023 11:00, Andrzej Hajda wrote: >> On 02.03.2023 11:43, Tvrtko Ursulin wrote: >>> >>> On 08/02/2023 10:51, Andrzej Hajda wrote: >>>> Write-combining memory allows speculative reads by CPU. >>>> ggtt->error_capture is WC mapped to CPU, so CPU/MMU can try >>>> to prefetch memory beyond the error_capture, ie it tries >>>> to read memory pointed by next PTE in GGTT. >>>> If this PTE points to invalid address DMAR errors will occur. >>>> This behaviour was observed on ADL, RPL, DG2 platforms. >>>> To avoid it, guard scratch page should be added after error_capture. >>>> The patch fixes the most annoying issue with error capture but >>>> since WC reads are used also in other places there is a risk similar >>>> problem can affect them as well. >>>> >>>> Signed-off-by: Andrzej Hajda <andrzej.hajda@intel.com> >>>> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> >>> >>> Research tells the explanation is plausible so: >>> >>> Acked-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> >>> >> >> Thanks for looking at it. >> >>> On patch details below. >>> >>> What about "simiar risk in other places" - are there any plans to >>> asses the potential impact elsewhere? >> >> Yes, merging this patch is the 1st step, as error_capture is >> responsible for most (maybe even all) regular >> DMAR read errors. After removing this noise it will be much easier to >> spot other DMAR read errors. >> There are also multiple regular DMAR write errors (less frequent, but >> still), which I hope to debug if time permits. >> >> Regards >> Andrzej >> >>> >>>> --- >>>> This patch tries to diminish plague of DMAR read errors present >>>> in CI for ADL*, RPL*, DG2 platforms, see for example [1] (grep DMAR). >>>> CI is usually tolerant for these errors, so the scale of the problem >>>> is not really visible. >>>> To show it I have counted lines containing DMAR read errors in dmesgs >>>> produced by CI for all three versions of the patch, but in contrast >>>> to v2 >>>> I have grepped only for lines containing "PTE Read access". >>>> Below stats for kernel w/o patch vs patched one. >>>> v1: 210 vs 0 >>>> v2: 201 vs 0 >>>> v3: 214 vs 0 >>>> Apparently the patch fixes all common PTE read errors. >>>> >>>> In previous version there were different numbers due to less exact >>>> grepping, >>>> "grep DMAR" catched write errors and "DMAR: DRHD: handling fault >>>> status reg" >>>> lines, anyway the actual number of errors is much bigger - DMAR errors >>>> are rate-limited. >>>> >>>> [1]: >>>> http://gfx-ci.igk.intel.com/tree/drm-tip/CI_DRM_12678/bat-adln-1/dmesg0.txt >>>> >>>> Changelog: >>>> v2: >>>> - modified commit message (I hope the diagnosis is correct), >>>> - added bug checks to ensure scratch is initialized on gen3 >>>> platforms. >>>> CI produces strange stacktrace for it suggesting scratch[0] >>>> is NULL, >>>> to be removed after resolving the issue with gen3 platforms. >>>> v3: >>>> - removed bug checks, replaced with gen check. >>>> v4: >>>> - change code for scratch page insertion to support all >>>> platforms, >>>> - add info in commit message there could be more similar issues >>>> >>>> Regards >>>> Andrzej >>>> --- >>>> drivers/gpu/drm/i915/gt/intel_ggtt.c | 31 >>>> ++++++++++++++++++++++++---- >>>> 1 file changed, 27 insertions(+), 4 deletions(-) >>>> >>>> diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c >>>> b/drivers/gpu/drm/i915/gt/intel_ggtt.c >>>> index 842e69c7b21e49..6566d2066f1f8b 100644 >>>> --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c >>>> +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c >>>> @@ -503,6 +503,21 @@ static void cleanup_init_ggtt(struct i915_ggtt >>>> *ggtt) >>>> mutex_destroy(&ggtt->error_mutex); >>>> } >>>> +static void >>>> +ggtt_insert_scratch_pages(struct i915_ggtt *ggtt, u64 offset, u64 >>>> length) >>>> +{ >>>> + struct i915_address_space *vm = &ggtt->vm; >>>> + >>>> + if (GRAPHICS_VER(ggtt->vm.i915) < 8) >>>> + return vm->clear_range(vm, offset, length); >>>> + /* clear_range since gen8 is nop */ >>> >>> It would also work to simply drop the <gen8 case and just do the >>> loop below, right? If so would that be clearer? > Apparently I have missed this comment, loop does not work for gen3, vm->scratch[0] is not initialized, it uses some code from drivers/char/agp/intel-gtt.c for clearing range. > Alternatively, if you want to keep the dual path here, perhaps it > would be better to replace the gen check with "clear_range !=/== > nop_clear_range". That way maybe more of the platform knowledge > remains at a central location. There is vm->error_range in internal branch which does the same as clear_range, which 'when upstreamed (?)' should simplify this code. On the other side unstaticising nop_clear_range would allow to drop mock_clear_range, dpt_clear_range. > > Another thing - are there any suspend-resume considerations > (i915_ggtt_resume_vm)? Maybe this padding needs to be restored too. Didn't think about it, I will add code there too. Regards Andrzej > > Regards, > > Tvrtko > >>> >>>> + while (length > 0) { >>> >>> Maybe add a GEM_BUG_ON if length is not aligned? Granted it may be a >>> huge stretch of imagination.. >>> >>> Regards, >>> >>> Tvrtko >>> >>>> + vm->insert_page(vm, px_dma(vm->scratch[0]), offset, >>>> I915_CACHE_NONE, 0); >>>> + offset += I915_GTT_PAGE_SIZE; >>>> + length -= I915_GTT_PAGE_SIZE; >>>> + } >>>> +} >>>> + >>>> static int init_ggtt(struct i915_ggtt *ggtt) >>>> { >>>> /* >>>> @@ -551,8 +566,12 @@ static int init_ggtt(struct i915_ggtt *ggtt) >>>> * paths, and we trust that 0 will remain reserved. However, >>>> * the only likely reason for failure to insert is a driver >>>> * bug, which we expect to cause other failures... >>>> + * >>>> + * Since CPU can perform speculative reads on error capture >>>> + * (write-combining allows it) add scratch page after error >>>> + * capture to avoid DMAR errors. >>>> */ >>>> - ggtt->error_capture.size = I915_GTT_PAGE_SIZE; >>>> + ggtt->error_capture.size = 2 * I915_GTT_PAGE_SIZE; >>>> ggtt->error_capture.color = I915_COLOR_UNEVICTABLE; >>>> if (drm_mm_reserve_node(&ggtt->vm.mm, &ggtt->error_capture)) >>>> drm_mm_insert_node_in_range(&ggtt->vm.mm, >>>> @@ -562,11 +581,15 @@ static int init_ggtt(struct i915_ggtt *ggtt) >>>> 0, ggtt->mappable_end, >>>> DRM_MM_INSERT_LOW); >>>> } >>>> - if (drm_mm_node_allocated(&ggtt->error_capture)) >>>> + if (drm_mm_node_allocated(&ggtt->error_capture)) { >>>> + u64 start = ggtt->error_capture.start; >>>> + u64 size = ggtt->error_capture.size; >>>> + >>>> + ggtt_insert_scratch_pages(ggtt, start, size); >>>> drm_dbg(&ggtt->vm.i915->drm, >>>> "Reserved GGTT:[%llx, %llx] for use by error capture\n", >>>> - ggtt->error_capture.start, >>>> - ggtt->error_capture.start + ggtt->error_capture.size); >>>> + start, start + size); >>>> + } >>>> /* >>>> * The upper portion of the GuC address space has a sizeable >>>> hole >>
diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c index 842e69c7b21e49..6566d2066f1f8b 100644 --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c @@ -503,6 +503,21 @@ static void cleanup_init_ggtt(struct i915_ggtt *ggtt) mutex_destroy(&ggtt->error_mutex); } +static void +ggtt_insert_scratch_pages(struct i915_ggtt *ggtt, u64 offset, u64 length) +{ + struct i915_address_space *vm = &ggtt->vm; + + if (GRAPHICS_VER(ggtt->vm.i915) < 8) + return vm->clear_range(vm, offset, length); + /* clear_range since gen8 is nop */ + while (length > 0) { + vm->insert_page(vm, px_dma(vm->scratch[0]), offset, I915_CACHE_NONE, 0); + offset += I915_GTT_PAGE_SIZE; + length -= I915_GTT_PAGE_SIZE; + } +} + static int init_ggtt(struct i915_ggtt *ggtt) { /* @@ -551,8 +566,12 @@ static int init_ggtt(struct i915_ggtt *ggtt) * paths, and we trust that 0 will remain reserved. However, * the only likely reason for failure to insert is a driver * bug, which we expect to cause other failures... + * + * Since CPU can perform speculative reads on error capture + * (write-combining allows it) add scratch page after error + * capture to avoid DMAR errors. */ - ggtt->error_capture.size = I915_GTT_PAGE_SIZE; + ggtt->error_capture.size = 2 * I915_GTT_PAGE_SIZE; ggtt->error_capture.color = I915_COLOR_UNEVICTABLE; if (drm_mm_reserve_node(&ggtt->vm.mm, &ggtt->error_capture)) drm_mm_insert_node_in_range(&ggtt->vm.mm, @@ -562,11 +581,15 @@ static int init_ggtt(struct i915_ggtt *ggtt) 0, ggtt->mappable_end, DRM_MM_INSERT_LOW); } - if (drm_mm_node_allocated(&ggtt->error_capture)) + if (drm_mm_node_allocated(&ggtt->error_capture)) { + u64 start = ggtt->error_capture.start; + u64 size = ggtt->error_capture.size; + + ggtt_insert_scratch_pages(ggtt, start, size); drm_dbg(&ggtt->vm.i915->drm, "Reserved GGTT:[%llx, %llx] for use by error capture\n", - ggtt->error_capture.start, - ggtt->error_capture.start + ggtt->error_capture.size); + start, start + size); + } /* * The upper portion of the GuC address space has a sizeable hole