drm/i915: cleanup lrc engine state when lrc is freed

Message ID	1420724407-11767-1-git-send-email-mika.kuoppala@intel.com (mailing list archive)
State	New, archived
Headers	show Return-Path: <intel-gfx-bounces@lists.freedesktop.org> From: Mika Kuoppala <mika.kuoppala@linux.intel.com> To: intel-gfx@lists.freedesktop.org Date: Thu, 8 Jan 2015 15:40:07 +0200 Message-Id: <1420724407-11767-1-git-send-email-mika.kuoppala@intel.com> Subject: [Intel-gfx] [PATCH] drm/i915: cleanup lrc engine state when lrc is freed Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" <intel-gfx-bounces@lists.freedesktop.org>

Message ID

1420724407-11767-1-git-send-email-mika.kuoppala@intel.com (mailing list archive)

State

New, archived

Headers

From: Mika Kuoppala <mika.kuoppala@linux.intel.com>
To: intel-gfx@lists.freedesktop.org
Date: Thu,  8 Jan 2015 15:40:07 +0200
Message-Id: <1420724407-11767-1-git-send-email-mika.kuoppala@intel.com>
Subject: [Intel-gfx] [PATCH] drm/i915: cleanup lrc engine state when lrc is
	freed
Precedence: list
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
Errors-To: intel-gfx-bounces@lists.freedesktop.org
Sender: "Intel-gfx" <intel-gfx-bounces@lists.freedesktop.org>

Commit Message

Mika Kuoppala Jan. 8, 2015, 1:40 p.m. UTC

i915_gem_validate_context() will check the engine->state to see if it can
submit into a ringbuffer. But when we are releasing the context we leave the
engine state to a non null value. Thus after a successful hang recovery
we might mistakenly submit to a non initialized ringbuffer resulting in:

[ 1991.356418] ------------[ cut here ]------------
[ 1991.359192] WARNING: CPU: 1 PID: 2335 at lib/iomap.c:43 bad_io_access+0x3d/0x40()
[ 1991.361966] Bad IO access at port 0x24 (outl(val,port))
[ 1991.364750] Modules linked in: snd_hda_codec_hdmi i915 x86_pkg_temp_thermal coretemp kvm_intel kvm snd_hda_intel snd_hda_controller snd_hda_codec crct10dif_pclmul crc32_pclmul ghash_clmulni_intel snd_hwdep snd_pcm aesni_intel aes_x86_64 glue_helper lrw i2c_algo_bit gf128mul ablk_helper drm_kms_helper cryptd snd_seq_midi snd_seq_midi_event serio_raw drm snd_rawmidi snd_seq snd_seq_device snd_timer video snd soundcore mei_me lpc_ich bnep mac_hid acpi_pad mei rfcomm bluetooth parport_pc ppdev lp parport nls_iso8859_1 e1000e ptp ahci libahci pps_core sdhci_acpi sdhci
[ 1991.370827] CPU: 1 PID: 2335 Comm: gem_ringfill Tainted: G        W      3.19.0-rc3+ #50
[ 1991.373838] Hardware name: Intel Corporation Broadwell Client platform/SawTooth Peak, BIOS BDW-E1R1.86C.0092.R00.1408311942 08/31/2014
[ 1991.376902]  ffffffff81aa1a46 ffff88014910fac8 ffffffff8173dbcf 0000000000000001
[ 1991.379978]  ffff88014910fb18 ffff88014910fb08 ffffffff8107007a ffff88014910fb28
[ 1991.383037]  ffff880147209940 ffff8800aafa8718 ffff8800aafa0000 ffff8800aafa1918
[ 1991.386094] Call Trace:
[ 1991.389140]  [<ffffffff8173dbcf>] dump_stack+0x45/0x57
[ 1991.392207]  [<ffffffff8107007a>] warn_slowpath_common+0x8a/0xc0
[ 1991.395268]  [<ffffffff810700f6>] warn_slowpath_fmt+0x46/0x50
[ 1991.398330]  [<ffffffffa053290c>] ? intel_logical_ring_begin+0x3c/0x240 [i915]
[ 1991.401395]  [<ffffffff813985bd>] bad_io_access+0x3d/0x40
[ 1991.404462]  [<ffffffff81398763>] iowrite32+0x33/0x40
[ 1991.407529]  [<ffffffffa0533585>] gen8_init_rcs_context+0xd5/0x170 [i915]
[ 1991.410605]  [<ffffffffa0533d17>] intel_lr_context_deferred_create+0x657/0x8e0 [i915]
[ 1991.413668]  [<ffffffffa050eff1>] i915_gem_do_execbuffer.isra.22+0xed1/0xf60 [i915]
[ 1991.416736]  [<ffffffff811c0125>] ? __kmalloc+0x55/0x1b0
[ 1991.419801]  [<ffffffffa051029c>] ? i915_gem_execbuffer2+0x6c/0x2c0 [i915]
[ 1991.422772]  [<ffffffffa05102e1>] i915_gem_execbuffer2+0xb1/0x2c0 [i915]
[ 1991.425632]  [<ffffffffa01b8ab4>] drm_ioctl+0x1a4/0x630 [drm]
[ 1991.428454]  [<ffffffff811258bc>] ? acct_account_cputime+0x1c/0x20
[ 1991.431255]  [<ffffffff811ee378>] do_vfs_ioctl+0x2f8/0x510
[ 1991.434009]  [<ffffffff8109f834>] ? vtime_account_user+0x54/0x60
[ 1991.436778]  [<ffffffff811ee611>] SyS_ioctl+0x81/0xa0
[ 1991.439553]  [<ffffffff81745cb4>] ? int_check_syscall_exit_work+0x34/0x3d
[ 1991.442306]  [<ffffffff81745a2d>] system_call_fastpath+0x16/0x1b

Fix this by setting all the engine fields properly when lrc is freed.

Cc: Thomas Daniel <thomas.daniel@intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@intel.com>
---
 drivers/gpu/drm/i915/intel_lrc.c | 4 ++++
 1 file changed, 4 insertions(+)

Comments

Shuang He Jan. 8, 2015, 6:22 p.m. UTC | #1

Tested-By: PRC QA PRTS (Patch Regression Test System Contact: shuang.he@intel.com)
-------------------------------------Summary-------------------------------------
Platform          Delta          drm-intel-nightly          Series Applied
PNV                 -32              363/364              331/364
ILK                 -38              364/366              326/366
SNB              +4-64              443/450              383/450
IVB                 -44              496/498              452/498
BYT                 -8              288/289              280/289
HSW              +18-66              542/564              494/564
BDW                 -33              415/417              382/417
-------------------------------------Detailed-------------------------------------
Platform  Test                                drm-intel-nightly          Series Applied
 PNV  igt_gem_concurrent_blit_gpu-bcs-early-read-interruptible      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpu-bcs-gpu-read-after-write      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpu-bcs-gpu-read-after-write-interruptible      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpu-bcs-overwrite-source      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpu-bcs-overwrite-source-interruptible      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpu-rcs-early-read      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpu-rcs-early-read-interruptible      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpu-rcs-gpu-read-after-write      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpu-rcs-gpu-read-after-write-interruptible      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpu-rcs-overwrite-source      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpu-rcs-overwrite-source-interruptible      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpuX-bcs-early-read      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpuX-bcs-early-read-interruptible      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpuX-bcs-gpu-read-after-write      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpuX-bcs-gpu-read-after-write-interruptible      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpuX-bcs-overwrite-source      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpuX-bcs-overwrite-source-interruptible      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpuX-rcs-early-read      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpuX-rcs-early-read-interruptible      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpuX-rcs-gpu-read-after-write      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpuX-rcs-gpu-read-after-write-interruptible      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpuX-rcs-overwrite-source      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_concurrent_blit_gpuX-rcs-overwrite-source-interruptible      NSPT(4, M23M25)PASS(1, M25)      NSPT(1, M23)
 PNV  igt_gem_pread_after_blit_interruptible      NRUN(1, M23)PASS(1, M25)      NRUN(1, M23)
 PNV  igt_gem_pread_after_blit_interruptible-display      NRUN(1, M23)PASS(1, M25)      NRUN(1, M23)
 PNV  igt_gem_pread_after_blit_interruptible-snoop      NRUN(1, M23)PASS(1, M25)      NRUN(1, M23)
 PNV  igt_gem_pread_after_blit_interruptible-uncached      NRUN(1, M23)PASS(1, M25)      NRUN(1, M23)
 PNV  igt_gem_pread_after_blit_normal      NRUN(1, M23)PASS(1, M25)      NRUN(1, M23)
 PNV  igt_gem_pread_after_blit_normal-display      NRUN(1, M23)PASS(1, M25)      NRUN(1, M23)
 PNV  igt_gem_pread_after_blit_normal-snoop      NRUN(1, M23)PASS(1, M25)      NRUN(1, M23)
 PNV  igt_gem_pread_after_blit_normal-uncached      NRUN(1, M23)PASS(1, M25)      NRUN(1, M23)
*PNV  igt_gen3_mixed_blits      PASS(2, M25M23)      CRASH(1, M23)
 ILK  igt_gem_concurrent_blit_gpu-bcs-early-read      NSPT(4, M26M37)PASS(1, M37)      NSPT(1, M26)
 ILK  igt_gem_concurrent_blit_gpu-bcs-early-read-interruptible      NSPT(4, M26M37)PASS(1, M37)      NSPT(1, M26)
 ILK  igt_gem_concurrent_blit_gpu-bcs-gpu-read-after-write      NSPT(4, M26M37)PASS(1, M37)      NSPT(1, M26)
 ILK  igt_gem_concurrent_blit_gpu-bcs-gpu-read-after-write-interruptible      NSPT(4, M26M37)PASS(1, M37)      NSPT(1, M26)
 ILK  igt_gem_concurrent_blit_gpu-bcs-overwrite-source      NSPT(4, M26M37)PASS(1, M37)      NSPT(1, M26)
 ILK  igt_gem_concurrent_blit_gpu-bcs-overwrite-source-interruptible      NSPT(4, M26M37)PASS(1, M37)      NSPT(1, M26)
 ILK  igt_gem_concurrent_blit_gpuX-bcs-early-read      NSPT(4, M26M37)PASS(1, M37)      NSPT(1, M26)
 ILK  igt_gem_concurrent_blit_gpuX-bcs-early-read-interruptible      NSPT(4, M26M37)PASS(1, M37)      NSPT(1, M26)
 ILK  igt_gem_concurrent_blit_gpuX-bcs-gpu-read-after-write      NSPT(4, M26M37)PASS(1, M37)      NSPT(1, M26)
 ILK  igt_gem_concurrent_blit_gpuX-bcs-gpu-read-after-write-interruptible      NSPT(4, M26M37)PASS(1, M37)      NSPT(1, M26)
 ILK  igt_gem_concurrent_blit_gpuX-bcs-overwrite-source      NSPT(4, M26M37)PASS(1, M37)      NSPT(1, M26)
 ILK  igt_gem_concurrent_blit_gpuX-bcs-overwrite-source-interruptible      NSPT(4, M26M37)PASS(1, M37)      NSPT(1, M26)
 ILK  igt_gem_pread_after_blit_interruptible      NRUN(1, M26)PASS(1, M37)      NRUN(1, M26)
 ILK  igt_gem_pread_after_blit_interruptible-display      NRUN(1, M26)PASS(1, M37)      NRUN(1, M26)
 ILK  igt_gem_pread_after_blit_interruptible-snoop      NRUN(1, M26)PASS(1, M37)      NRUN(1, M26)
 ILK  igt_gem_pread_after_blit_interruptible-uncached      NRUN(1, M26)PASS(1, M37)      NRUN(1, M26)
 ILK  igt_gem_pread_after_blit_normal      NRUN(1, M26)PASS(1, M37)      NRUN(1, M26)
 ILK  igt_gem_pread_after_blit_normal-display      NRUN(1, M26)PASS(1, M37)      NRUN(1, M26)
 ILK  igt_gem_pread_after_blit_normal-snoop      NRUN(1, M26)PASS(1, M37)      NRUN(1, M26)
 ILK  igt_gem_pread_after_blit_normal-uncached      NRUN(1, M26)PASS(1, M37)      NRUN(1, M26)
 ILK  igt_kms_flip_nonexisting-fb      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_rcs-flip-vs-panning-interruptible      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_rcs-wf_vblank-vs-dpms-interruptible      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_render_direct-render      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_bcs-flip-vs-modeset-interruptible      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_blocking-absolute-wf_vblank-interruptible      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_busy-flip-interruptible      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_flip-vs-dpms-interruptible      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_flip-vs-panning      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_flip-vs-rmfb-interruptible      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_plain-flip-fb-recreate-interruptible      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_plain-flip-ts-check-interruptible      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_rcs-flip-vs-dpms      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_rcs-flip-vs-modeset      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_rcs-flip-vs-panning      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_vblank-vs-hang      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_wf_vblank-ts-check      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 ILK  igt_kms_flip_wf_vblank-vs-modeset-interruptible      DMESG_WARN(4, M26)PASS(1, M37)      DMESG_WARN(1, M26)
 SNB  igt_gem_concurrent_blit_gpu-bcs-early-read      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-bcs-early-read-forked      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-bcs-early-read-interruptible      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-bcs-gpu-read-after-write      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-bcs-gpu-read-after-write-forked      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-bcs-gpu-read-after-write-interruptible      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-bcs-overwrite-source      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-bcs-overwrite-source-forked      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-bcs-overwrite-source-interruptible      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-rcs-early-read      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-rcs-early-read-interruptible      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-rcs-gpu-read-after-write      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-rcs-gpu-read-after-write-forked      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-rcs-gpu-read-after-write-interruptible      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-rcs-overwrite-source      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-rcs-overwrite-source-forked      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpu-rcs-overwrite-source-interruptible      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-bcs-early-read      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-bcs-early-read-forked      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-bcs-early-read-interruptible      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-bcs-gpu-read-after-write      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-bcs-gpu-read-after-write-interruptible      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-bcs-overwrite-source      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-bcs-overwrite-source-forked      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-bcs-overwrite-source-interruptible      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-rcs-early-read      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-rcs-early-read-forked      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-rcs-early-read-interruptible      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-rcs-gpu-read-after-write      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-rcs-gpu-read-after-write-forked      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-rcs-gpu-read-after-write-interruptible      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-rcs-overwrite-source      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-rcs-overwrite-source-forked      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_concurrent_blit_gpuX-rcs-overwrite-source-interruptible      NSPT(4, M22M35)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_gem_pread_after_blit_interruptible      NRUN(1, M22)PASS(1, M35)      NRUN(1, M22)
 SNB  igt_gem_pread_after_blit_interruptible-display      NRUN(1, M22)PASS(1, M35)      NRUN(1, M22)
 SNB  igt_gem_pread_after_blit_interruptible-snoop      NRUN(1, M22)PASS(1, M35)      NRUN(1, M22)
 SNB  igt_gem_pread_after_blit_interruptible-uncached      NRUN(1, M22)PASS(1, M35)      NRUN(1, M22)
 SNB  igt_gem_pread_after_blit_normal      NRUN(1, M22)PASS(1, M35)      NRUN(1, M22)
 SNB  igt_gem_pread_after_blit_normal-display      NRUN(1, M22)PASS(1, M35)      NRUN(1, M22)
 SNB  igt_gem_pread_after_blit_normal-snoop      NRUN(1, M22)PASS(1, M35)      NRUN(1, M22)
 SNB  igt_gem_pread_after_blit_normal-uncached      NRUN(1, M22)PASS(1, M35)      NRUN(1, M22)
 SNB  igt_kms_cursor_crc_cursor-size-change      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_kms_flip_dpms-vs-vblank-race      DMESG_WARN(3, M35M22)PASS(4, M35M22)      PASS(1, M22)
 SNB  igt_kms_flip_dpms-vs-vblank-race-interruptible      DMESG_WARN(2, M35M22)PASS(5, M35M22)      PASS(1, M22)
 SNB  igt_kms_flip_event_leak      NSPT(3, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_kms_flip_modeset-vs-vblank-race      DMESG_WARN(4, M35M22)PASS(4, M35M22)      PASS(1, M22)
 SNB  igt_kms_mmio_vs_cs_flip_setcrtc_vs_cs_flip      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_kms_mmio_vs_cs_flip_setplane_vs_cs_flip      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_kms_plane_plane-position-hole-pipe-B-plane-1      DMESG_WARN(1, M35)PASS(8, M35M22)      PASS(1, M22)
 SNB  igt_kms_rotation_crc_primary-rotation      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_kms_rotation_crc_sprite-rotation      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_cursor      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_cursor-dpms      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_dpms-mode-unset-non-lpsp      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_dpms-non-lpsp      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_drm-resources-equal      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_fences      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_fences-dpms      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_gem-execbuf      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_gem-mmap-cpu      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_gem-mmap-gtt      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_gem-pread      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_i2c      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_modeset-non-lpsp      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_modeset-non-lpsp-stress-no-wait      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_pci-d3-state      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 SNB  igt_pm_rpm_rte      NSPT(4, M35M22)PASS(1, M35)      NSPT(1, M22)
 IVB  igt_gem_concurrent_blit_gpu-bcs-early-read      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-bcs-early-read-forked      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-bcs-early-read-interruptible      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-bcs-gpu-read-after-write      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-bcs-gpu-read-after-write-forked      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-bcs-gpu-read-after-write-interruptible      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-bcs-overwrite-source      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-bcs-overwrite-source-forked      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-bcs-overwrite-source-interruptible      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-rcs-early-read      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-rcs-early-read-forked      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-rcs-early-read-interruptible      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-rcs-gpu-read-after-write      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-rcs-gpu-read-after-write-forked      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-rcs-gpu-read-after-write-interruptible      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-rcs-overwrite-source      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-rcs-overwrite-source-forked      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpu-rcs-overwrite-source-interruptible      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-bcs-early-read      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-bcs-early-read-forked      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-bcs-early-read-interruptible      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-bcs-gpu-read-after-write      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-bcs-gpu-read-after-write-forked      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-bcs-gpu-read-after-write-interruptible      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-bcs-overwrite-source      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-bcs-overwrite-source-forked      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-bcs-overwrite-source-interruptible      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-rcs-early-read      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-rcs-early-read-forked      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-rcs-early-read-interruptible      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-rcs-gpu-read-after-write      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-rcs-gpu-read-after-write-forked      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-rcs-gpu-read-after-write-interruptible      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-rcs-overwrite-source      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-rcs-overwrite-source-forked      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_concurrent_blit_gpuX-rcs-overwrite-source-interruptible      NSPT(4, M21M34)PASS(1, M34)      NSPT(1, M21)
 IVB  igt_gem_pread_after_blit_interruptible      NRUN(1, M21)PASS(1, M34)      NRUN(1, M21)
 IVB  igt_gem_pread_after_blit_interruptible-display      NRUN(1, M21)PASS(1, M34)      NRUN(1, M21)
 IVB  igt_gem_pread_after_blit_interruptible-snoop      NRUN(1, M21)PASS(1, M34)      NRUN(1, M21)
 IVB  igt_gem_pread_after_blit_interruptible-uncached      NRUN(1, M21)PASS(1, M34)      NRUN(1, M21)
 IVB  igt_gem_pread_after_blit_normal      NRUN(1, M21)PASS(1, M34)      NRUN(1, M21)
 IVB  igt_gem_pread_after_blit_normal-display      NRUN(1, M21)PASS(1, M34)      NRUN(1, M21)
 IVB  igt_gem_pread_after_blit_normal-snoop      NRUN(1, M21)PASS(1, M34)      NRUN(1, M21)
 IVB  igt_gem_pread_after_blit_normal-uncached      NRUN(1, M21)PASS(1, M34)      NRUN(1, M21)
 BYT  igt_gem_pread_after_blit_interruptible      NRUN(1, M51)PASS(1, M48)      NRUN(1, M51)
 BYT  igt_gem_pread_after_blit_interruptible-display      NRUN(1, M51)PASS(1, M48)      NRUN(1, M51)
 BYT  igt_gem_pread_after_blit_interruptible-snoop      NRUN(1, M51)PASS(1, M48)      NRUN(1, M51)
 BYT  igt_gem_pread_after_blit_interruptible-uncached      NRUN(1, M51)PASS(1, M48)      NRUN(1, M51)
 BYT  igt_gem_pread_after_blit_normal      NRUN(1, M51)PASS(1, M48)      NRUN(1, M51)
 BYT  igt_gem_pread_after_blit_normal-display      NRUN(1, M51)PASS(1, M48)      NRUN(1, M51)
 BYT  igt_gem_pread_after_blit_normal-snoop      NRUN(1, M51)PASS(1, M48)      NRUN(1, M51)
 BYT  igt_gem_pread_after_blit_normal-uncached      NRUN(1, M51)PASS(1, M48)      NRUN(1, M51)
 HSW  igt_gem_concurrent_blit_gpu-bcs-early-read      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-bcs-early-read-forked      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-bcs-early-read-interruptible      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-bcs-gpu-read-after-write      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-bcs-gpu-read-after-write-forked      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-bcs-gpu-read-after-write-interruptible      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-bcs-overwrite-source      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-bcs-overwrite-source-forked      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-bcs-overwrite-source-interruptible      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-rcs-early-read      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-rcs-early-read-forked      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-rcs-early-read-interruptible      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-rcs-gpu-read-after-write      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-rcs-gpu-read-after-write-forked      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-rcs-gpu-read-after-write-interruptible      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-rcs-overwrite-source      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-rcs-overwrite-source-forked      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpu-rcs-overwrite-source-interruptible      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-bcs-early-read      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-bcs-early-read-forked      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-bcs-early-read-interruptible      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-bcs-gpu-read-after-write      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-bcs-gpu-read-after-write-forked      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-bcs-gpu-read-after-write-interruptible      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-bcs-overwrite-source      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-bcs-overwrite-source-forked      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-bcs-overwrite-source-interruptible      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-rcs-early-read      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-rcs-early-read-forked      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-rcs-early-read-interruptible      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-rcs-gpu-read-after-write      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-rcs-gpu-read-after-write-forked      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-rcs-gpu-read-after-write-interruptible      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-rcs-overwrite-source      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-rcs-overwrite-source-forked      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_concurrent_blit_gpuX-rcs-overwrite-source-interruptible      NSPT(4, M19M20)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_gem_pread_after_blit_interruptible      NRUN(1, M19)PASS(1, M40)      NRUN(1, M19)
 HSW  igt_gem_pread_after_blit_interruptible-display      NRUN(1, M19)PASS(1, M40)      NRUN(1, M19)
 HSW  igt_gem_pread_after_blit_interruptible-snoop      NRUN(1, M19)PASS(1, M40)      NRUN(1, M19)
 HSW  igt_gem_pread_after_blit_interruptible-uncached      NRUN(1, M19)PASS(1, M40)      NRUN(1, M19)
 HSW  igt_gem_pread_after_blit_normal      NRUN(1, M19)PASS(1, M40)      NRUN(1, M19)
 HSW  igt_gem_pread_after_blit_normal-display      NRUN(1, M19)PASS(1, M40)      NRUN(1, M19)
 HSW  igt_gem_pread_after_blit_normal-snoop      NRUN(1, M19)PASS(1, M40)      NRUN(1, M19)
 HSW  igt_gem_pread_after_blit_normal-uncached      NRUN(1, M19)PASS(1, M40)      NRUN(1, M19)
 HSW  igt_kms_cursor_crc_cursor-size-change      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_kms_fence_pin_leak      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_kms_flip_dpms-vs-vblank-race      DMESG_WARN(1, M40)PASS(4, M19M20)      PASS(1, M19)
*HSW  igt_kms_flip_dpms-vs-vblank-race-interruptible      DMESG_WARN(2, M40)PASS(4, M19M20)      NSPT(1, M19)
 HSW  igt_kms_flip_event_leak      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_kms_flip_flip-vs-dpms-off-vs-modeset      DMESG_WARN(1, M40)PASS(4, M19M20)      PASS(1, M19)
 HSW  igt_kms_flip_flip-vs-dpms-off-vs-modeset-interruptible      DMESG_WARN(2, M40M19)PASS(4, M19M20)      PASS(1, M19)
 HSW  igt_kms_flip_modeset-vs-vblank-race      DMESG_WARN(1, M40)PASS(4, M19M20)      PASS(1, M19)
 HSW  igt_kms_flip_modeset-vs-vblank-race-interruptible      DMESG_WARN(1, M40)PASS(4, M19M20)      PASS(1, M19)
 HSW  igt_kms_flip_single-buffer-flip-vs-dpms-off-vs-modeset-interruptible      DMESG_WARN(2, M40)PASS(5, M19M20)      PASS(1, M19)
 HSW  igt_kms_mmio_vs_cs_flip_setcrtc_vs_cs_flip      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_kms_mmio_vs_cs_flip_setplane_vs_cs_flip      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_kms_pipe_crc_basic_read-crc-pipe-B      TIMEOUT(1, M40)PASS(2, M19)      PASS(1, M19)
 HSW  igt_kms_pipe_crc_basic_read-crc-pipe-B-frame-sequence      TIMEOUT(1, M40)PASS(2, M19)      PASS(1, M19)
 HSW  igt_kms_pipe_crc_basic_read-crc-pipe-C      TIMEOUT(1, M40)PASS(2, M19)      PASS(1, M19)
 HSW  igt_kms_pipe_crc_basic_read-crc-pipe-C-frame-sequence      TIMEOUT(1, M40)PASS(2, M19)      PASS(1, M19)
 HSW  igt_kms_plane_plane-panning-bottom-right-pipe-A-plane-1      TIMEOUT(1, M40)PASS(2, M19)      PASS(1, M19)
 HSW  igt_kms_plane_plane-panning-bottom-right-pipe-A-plane-2      TIMEOUT(1, M40)PASS(2, M19)      PASS(1, M19)
 HSW  igt_kms_plane_plane-panning-bottom-right-pipe-B-plane-1      TIMEOUT(1, M40)PASS(3, M19M20)      PASS(1, M19)
 HSW  igt_kms_plane_plane-panning-bottom-right-pipe-B-plane-2      TIMEOUT(1, M40)PASS(3, M19M20)      PASS(1, M19)
 HSW  igt_kms_plane_plane-panning-bottom-right-pipe-C-plane-1      TIMEOUT(3, M40)PASS(6, M19M40M20)      PASS(1, M19)
 HSW  igt_kms_setmode_invalid-clone-exclusive-crtc      DMESG_WARN(1, M40)PASS(2, M19)      PASS(1, M19)
 HSW  igt_pm_lpsp_non-edp      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_pm_rpm_cursor      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_pm_rpm_cursor-dpms      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_pm_rpm_dpms-mode-unset-non-lpsp      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_pm_rpm_dpms-non-lpsp      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_pm_rpm_drm-resources-equal      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_pm_rpm_fences      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_pm_rpm_fences-dpms      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_pm_rpm_gem-execbuf      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_pm_rpm_gem-mmap-cpu      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_pm_rpm_gem-mmap-gtt      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_pm_rpm_gem-pread      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_pm_rpm_i2c      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_pm_rpm_modeset-non-lpsp      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_pm_rpm_modeset-non-lpsp-stress-no-wait      NSPT(3, M19)DMESG_WARN(1, M40)PASS(5, M40M20)      NSPT(1, M19)
 HSW  igt_pm_rpm_pci-d3-state      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
 HSW  igt_pm_rpm_rte      NSPT(2, M19)PASS(1, M40)      NSPT(1, M19)
*HSW  igt_gem_concurrent_blit_gtt-bcs-early-read-forked      PASS(2, M40M19)      DMESG_WARN(1, M19)
 HSW  igt_kms_flip_flip-vs-rmfb      DMESG_WARN(1, M40)PASS(3, M19)      PASS(1, M19)
 HSW  igt_kms_flip_flip-vs-rmfb-interruptible      DMESG_WARN(1, M40)PASS(2, M19)      PASS(1, M19)
 BDW  igt_gem_concurrent_blit_gpu-bcs-early-read      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpu-bcs-early-read-interruptible      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpu-bcs-gpu-read-after-write      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpu-bcs-gpu-read-after-write-interruptible      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpu-bcs-overwrite-source      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpu-bcs-overwrite-source-interruptible      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpu-rcs-early-read      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpu-rcs-early-read-interruptible      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpu-rcs-gpu-read-after-write      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpu-rcs-gpu-read-after-write-interruptible      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpu-rcs-overwrite-source      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpu-rcs-overwrite-source-interruptible      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpuX-bcs-early-read      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpuX-bcs-early-read-interruptible      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpuX-bcs-gpu-read-after-write      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpuX-bcs-gpu-read-after-write-interruptible      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpuX-bcs-overwrite-source      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpuX-bcs-overwrite-source-interruptible      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpuX-rcs-early-read      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpuX-rcs-early-read-interruptible      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpuX-rcs-gpu-read-after-write      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpuX-rcs-gpu-read-after-write-interruptible      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpuX-rcs-overwrite-source      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_concurrent_blit_gpuX-rcs-overwrite-source-interruptible      NSPT(4, M28M30)PASS(1, M30)      NSPT(1, M28)
 BDW  igt_gem_pread_after_blit_interruptible      NRUN(1, M28)PASS(1, M30)      NRUN(1, M28)
 BDW  igt_gem_pread_after_blit_interruptible-display      NRUN(1, M28)PASS(1, M30)      NRUN(1, M28)
 BDW  igt_gem_pread_after_blit_interruptible-snoop      NRUN(1, M28)PASS(1, M30)      NRUN(1, M28)
 BDW  igt_gem_pread_after_blit_interruptible-uncached      NRUN(1, M28)PASS(1, M30)      NRUN(1, M28)
 BDW  igt_gem_pread_after_blit_normal      NRUN(1, M28)PASS(1, M30)      NRUN(1, M28)
 BDW  igt_gem_pread_after_blit_normal-display      NRUN(1, M28)PASS(1, M30)      NRUN(1, M28)
 BDW  igt_gem_pread_after_blit_normal-snoop      NRUN(1, M28)PASS(1, M30)      NRUN(1, M28)
 BDW  igt_gem_pread_after_blit_normal-uncached      NRUN(1, M28)PASS(1, M30)      NRUN(1, M28)
*BDW  igt_gem_multi_bsd_sync_loop      PASS(5, M30M28)      DMESG_WARN(1, M28)
Note: You need to pay more attention to line start with '*'

Dave Gordon Jan. 9, 2015, 6:47 p.m. UTC | #2

On 08/01/15 13:40, Mika Kuoppala wrote:
> i915_gem_validate_context() will check the engine->state to see if it can
> submit into a ringbuffer. But when we are releasing the context we leave the
> engine state to a non null value. Thus after a successful hang recovery
> we might mistakenly submit to a non initialized ringbuffer resulting in:
> 
> [ 1991.356418] ------------[ cut here ]------------
> [ 1991.359192] WARNING: CPU: 1 PID: 2335 at lib/iomap.c:43 bad_io_access+0x3d/0x40()
> [ 1991.361966] Bad IO access at port 0x24 (outl(val,port))
> [ 1991.364750] Modules linked in: snd_hda_codec_hdmi i915 x86_pkg_temp_thermal coretemp kvm_intel kvm snd_hda_intel snd_hda_controller snd_hda_codec crct10dif_pclmul crc32_pclmul ghash_clmulni_intel snd_hwdep snd_pcm aesni_intel aes_x86_64 glue_helper lrw i2c_algo_bit gf128mul ablk_helper drm_kms_helper cryptd snd_seq_midi snd_seq_midi_event serio_raw drm snd_rawmidi snd_seq snd_seq_device snd_timer video snd soundcore mei_me lpc_ich bnep mac_hid acpi_pad mei rfcomm bluetooth parport_pc ppdev lp parport nls_iso8859_1 e1000e ptp ahci libahci pps_core sdhci_acpi sdhci
> [ 1991.370827] CPU: 1 PID: 2335 Comm: gem_ringfill Tainted: G        W      3.19.0-rc3+ #50
> [ 1991.373838] Hardware name: Intel Corporation Broadwell Client platform/SawTooth Peak, BIOS BDW-E1R1.86C.0092.R00.1408311942 08/31/2014
> [ 1991.376902]  ffffffff81aa1a46 ffff88014910fac8 ffffffff8173dbcf 0000000000000001
> [ 1991.379978]  ffff88014910fb18 ffff88014910fb08 ffffffff8107007a ffff88014910fb28
> [ 1991.383037]  ffff880147209940 ffff8800aafa8718 ffff8800aafa0000 ffff8800aafa1918
> [ 1991.386094] Call Trace:
> [ 1991.389140]  [<ffffffff8173dbcf>] dump_stack+0x45/0x57
> [ 1991.392207]  [<ffffffff8107007a>] warn_slowpath_common+0x8a/0xc0
> [ 1991.395268]  [<ffffffff810700f6>] warn_slowpath_fmt+0x46/0x50
> [ 1991.398330]  [<ffffffffa053290c>] ? intel_logical_ring_begin+0x3c/0x240 [i915]
> [ 1991.401395]  [<ffffffff813985bd>] bad_io_access+0x3d/0x40
> [ 1991.404462]  [<ffffffff81398763>] iowrite32+0x33/0x40
> [ 1991.407529]  [<ffffffffa0533585>] gen8_init_rcs_context+0xd5/0x170 [i915]
> [ 1991.410605]  [<ffffffffa0533d17>] intel_lr_context_deferred_create+0x657/0x8e0 [i915]
> [ 1991.413668]  [<ffffffffa050eff1>] i915_gem_do_execbuffer.isra.22+0xed1/0xf60 [i915]
> [ 1991.416736]  [<ffffffff811c0125>] ? __kmalloc+0x55/0x1b0
> [ 1991.419801]  [<ffffffffa051029c>] ? i915_gem_execbuffer2+0x6c/0x2c0 [i915]
> [ 1991.422772]  [<ffffffffa05102e1>] i915_gem_execbuffer2+0xb1/0x2c0 [i915]
> [ 1991.425632]  [<ffffffffa01b8ab4>] drm_ioctl+0x1a4/0x630 [drm]
> [ 1991.428454]  [<ffffffff811258bc>] ? acct_account_cputime+0x1c/0x20
> [ 1991.431255]  [<ffffffff811ee378>] do_vfs_ioctl+0x2f8/0x510
> [ 1991.434009]  [<ffffffff8109f834>] ? vtime_account_user+0x54/0x60
> [ 1991.436778]  [<ffffffff811ee611>] SyS_ioctl+0x81/0xa0
> [ 1991.439553]  [<ffffffff81745cb4>] ? int_check_syscall_exit_work+0x34/0x3d
> [ 1991.442306]  [<ffffffff81745a2d>] system_call_fastpath+0x16/0x1b
> 
> Fix this by setting all the engine fields properly when lrc is freed.
> 
> Cc: Thomas Daniel <thomas.daniel@intel.com>
> Signed-off-by: Mika Kuoppala <mika.kuoppala@intel.com>
> ---
>  drivers/gpu/drm/i915/intel_lrc.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
> index 7670a0f..32684d9 100644
> --- a/drivers/gpu/drm/i915/intel_lrc.c
> +++ b/drivers/gpu/drm/i915/intel_lrc.c
> @@ -1777,6 +1777,10 @@ void intel_lr_context_free(struct intel_context *ctx)
>  			intel_destroy_ringbuffer_obj(ringbuf);
>  			kfree(ringbuf);
>  			drm_gem_object_unreference(&ctx_obj->base);
> +			WARN_ON(ctx->engine[i].unpin_count != 0);
> +			ctx->engine[i].unpin_count = 0;
> +			ctx->engine[i].ringbuf = NULL;
> +			ctx->engine[i].state = NULL;
>  		}
>  	}
>  }

Hi,

I don't quite see how this can fix the problem illustrated by the stack
trace above.  AFAICS intel_lr_context_free() is called /only/ from
i915_gem_context_free(), which should mean that the refcount on the
intel_context object is already zero, and that it will be freed on
return. So the contents of ctx->engine[] should be irrelevant ...

void i915_gem_context_free(struct kref *ctx_ref)
{
    struct intel_context *ctx = container_of(ctx_ref,
                                             typeof(*ctx), ref);

    trace_i915_context_free(ctx);

    if (i915.enable_execlists)
        intel_lr_context_free(ctx);

    i915_ppgtt_put(ctx->ppgtt);

    if (ctx->legacy_hw_ctx.rcs_state)
        drm_gem_object_unreference(&ctx->legacy_hw_ctx.rcs_state->base);

    list_del(&ctx->link);
    kfree(ctx);
}

... unless something is trying to reuse the context while it is still in
the process of being deleted :(

In addition, the stack trace above implies that ctx->engine[].state WAS
NULL when i915_gem_validate_context() was called, otherwise it would not
have called intel_lr_context_deferred_create()

    if (i915.enable_execlists && !ctx->engine[ring->id].state) {
        int ret = intel_lr_context_deferred_create(ctx, ring);
        if (ret) {
             DRM_DEBUG("Could not create LRC %u: %d\n", ctx_id, ret);
             return ERR_PTR(ret);
        }
    }

and likewise that function would not have called gen8_init_rcs_context()
unless this was a new context:

    if (ctx == ring->default_context)
        lrc_setup_hardware_status_page(ring, ctx_obj);
    else if (ring->id == RCS && !ctx->rcs_initialized) {
        if (ring->init_context) {
            ret = ring->init_context(ring, ctx);
            if (ret) {
                 DRM_ERROR("ring init context: %d\n", ret);
                 ctx->engine[ring->id].ringbuf = NULL;
                 ctx->engine[ring->id].state = NULL;
                 goto error;
            }
        }

        ctx->rcs_initialized = true;
    }

Note that rcs_initialized is never cleared, even with your change, so
that in a use-after-free situation we wouldn't end up in this path. So I
think the mystery is how this context ended up in an inconsistent state:
has it been partially freed and then reused, or has some part of the new
context allocation path failed but not been unwound correctly?

And if setting to NULL a pointer that's inside a structure that's in the
process of being freed actually makes a difference, doesn't that mean
there's a use-after-free issue somewhere?

.Dave.

Ville Syrjälä Jan. 12, 2015, 4:24 p.m. UTC | #3

On Fri, Jan 09, 2015 at 06:47:41PM +0000, Dave Gordon wrote:
> On 08/01/15 13:40, Mika Kuoppala wrote:
> > i915_gem_validate_context() will check the engine->state to see if it can
> > submit into a ringbuffer. But when we are releasing the context we leave the
> > engine state to a non null value. Thus after a successful hang recovery
> > we might mistakenly submit to a non initialized ringbuffer resulting in:
> > 
> > [ 1991.356418] ------------[ cut here ]------------
> > [ 1991.359192] WARNING: CPU: 1 PID: 2335 at lib/iomap.c:43 bad_io_access+0x3d/0x40()
> > [ 1991.361966] Bad IO access at port 0x24 (outl(val,port))
> > [ 1991.364750] Modules linked in: snd_hda_codec_hdmi i915 x86_pkg_temp_thermal coretemp kvm_intel kvm snd_hda_intel snd_hda_controller snd_hda_codec crct10dif_pclmul crc32_pclmul ghash_clmulni_intel snd_hwdep snd_pcm aesni_intel aes_x86_64 glue_helper lrw i2c_algo_bit gf128mul ablk_helper drm_kms_helper cryptd snd_seq_midi snd_seq_midi_event serio_raw drm snd_rawmidi snd_seq snd_seq_device snd_timer video snd soundcore mei_me lpc_ich bnep mac_hid acpi_pad mei rfcomm bluetooth parport_pc ppdev lp parport nls_iso8859_1 e1000e ptp ahci libahci pps_core sdhci_acpi sdhci
> > [ 1991.370827] CPU: 1 PID: 2335 Comm: gem_ringfill Tainted: G        W      3.19.0-rc3+ #50
> > [ 1991.373838] Hardware name: Intel Corporation Broadwell Client platform/SawTooth Peak, BIOS BDW-E1R1.86C.0092.R00.1408311942 08/31/2014
> > [ 1991.376902]  ffffffff81aa1a46 ffff88014910fac8 ffffffff8173dbcf 0000000000000001
> > [ 1991.379978]  ffff88014910fb18 ffff88014910fb08 ffffffff8107007a ffff88014910fb28
> > [ 1991.383037]  ffff880147209940 ffff8800aafa8718 ffff8800aafa0000 ffff8800aafa1918
> > [ 1991.386094] Call Trace:
> > [ 1991.389140]  [<ffffffff8173dbcf>] dump_stack+0x45/0x57
> > [ 1991.392207]  [<ffffffff8107007a>] warn_slowpath_common+0x8a/0xc0
> > [ 1991.395268]  [<ffffffff810700f6>] warn_slowpath_fmt+0x46/0x50
> > [ 1991.398330]  [<ffffffffa053290c>] ? intel_logical_ring_begin+0x3c/0x240 [i915]
> > [ 1991.401395]  [<ffffffff813985bd>] bad_io_access+0x3d/0x40
> > [ 1991.404462]  [<ffffffff81398763>] iowrite32+0x33/0x40
> > [ 1991.407529]  [<ffffffffa0533585>] gen8_init_rcs_context+0xd5/0x170 [i915]
> > [ 1991.410605]  [<ffffffffa0533d17>] intel_lr_context_deferred_create+0x657/0x8e0 [i915]
> > [ 1991.413668]  [<ffffffffa050eff1>] i915_gem_do_execbuffer.isra.22+0xed1/0xf60 [i915]
> > [ 1991.416736]  [<ffffffff811c0125>] ? __kmalloc+0x55/0x1b0
> > [ 1991.419801]  [<ffffffffa051029c>] ? i915_gem_execbuffer2+0x6c/0x2c0 [i915]
> > [ 1991.422772]  [<ffffffffa05102e1>] i915_gem_execbuffer2+0xb1/0x2c0 [i915]
> > [ 1991.425632]  [<ffffffffa01b8ab4>] drm_ioctl+0x1a4/0x630 [drm]
> > [ 1991.428454]  [<ffffffff811258bc>] ? acct_account_cputime+0x1c/0x20
> > [ 1991.431255]  [<ffffffff811ee378>] do_vfs_ioctl+0x2f8/0x510
> > [ 1991.434009]  [<ffffffff8109f834>] ? vtime_account_user+0x54/0x60
> > [ 1991.436778]  [<ffffffff811ee611>] SyS_ioctl+0x81/0xa0
> > [ 1991.439553]  [<ffffffff81745cb4>] ? int_check_syscall_exit_work+0x34/0x3d
> > [ 1991.442306]  [<ffffffff81745a2d>] system_call_fastpath+0x16/0x1b
> > 
> > Fix this by setting all the engine fields properly when lrc is freed.
> > 
> > Cc: Thomas Daniel <thomas.daniel@intel.com>
> > Signed-off-by: Mika Kuoppala <mika.kuoppala@intel.com>
> > ---
> >  drivers/gpu/drm/i915/intel_lrc.c | 4 ++++
> >  1 file changed, 4 insertions(+)
> > 
> > diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
> > index 7670a0f..32684d9 100644
> > --- a/drivers/gpu/drm/i915/intel_lrc.c
> > +++ b/drivers/gpu/drm/i915/intel_lrc.c
> > @@ -1777,6 +1777,10 @@ void intel_lr_context_free(struct intel_context *ctx)
> >  			intel_destroy_ringbuffer_obj(ringbuf);
> >  			kfree(ringbuf);
> >  			drm_gem_object_unreference(&ctx_obj->base);
> > +			WARN_ON(ctx->engine[i].unpin_count != 0);
> > +			ctx->engine[i].unpin_count = 0;
> > +			ctx->engine[i].ringbuf = NULL;
> > +			ctx->engine[i].state = NULL;
> >  		}
> >  	}
> >  }
> 
> Hi,
> 
> I don't quite see how this can fix the problem illustrated by the stack
> trace above.  AFAICS intel_lr_context_free() is called /only/ from
> i915_gem_context_free(), which should mean that the refcount on the
> intel_context object is already zero, and that it will be freed on
> return. So the contents of ctx->engine[] should be irrelevant ...

I've now managed to hit this fairly reliably with
'gem_ringfill --r render' + ctrl-c straight after boot.

To me it looks like some kind of bigger mess with olr. If olr was already there,
logical_ring_alloc_request() will do absolutely nothing, which means the ringbuf
won't even be pinned/mapped and ringbug->virtual_start will remain NULL. And that
leads to the w/a init to call iowrite() with some close to 0 iomem addressses.

I'm not sure how this is supposed to actually work. To me it looks olr should never
be allowed to be != NULL when creating the context/ringbuf to avoid this problem. And
yet somehow the presence of olr doesn't always trigger the 'bad IO access' warns. In
fact I can't seem to trigger it more than once until I reboot the machine. I tried to
add a warning [1] to catch all cases when olr is already there, and it will trigger in
most cases when I hit ctrl-c while running gem_ringfill. And yet the 'Bad IO access'
doesn't follow it except once after boot.

[1]
@@ -1906,6 +1926,8 @@ int intel_lr_context_deferred_create(struct intel_context *ctx,
 
        }
 
+       WARN_ON(ring->outstanding_lazy_request);
+
        ret = populate_lr_context(ctx, ctx_obj, ring, ringbuf);
        if (ret) {
                DRM_DEBUG_DRIVER("Failed to populate LRC: %d\n", ret);



> 
> void i915_gem_context_free(struct kref *ctx_ref)
> {
>     struct intel_context *ctx = container_of(ctx_ref,
>                                              typeof(*ctx), ref);
> 
>     trace_i915_context_free(ctx);
> 
>     if (i915.enable_execlists)
>         intel_lr_context_free(ctx);
> 
>     i915_ppgtt_put(ctx->ppgtt);
> 
>     if (ctx->legacy_hw_ctx.rcs_state)
>         drm_gem_object_unreference(&ctx->legacy_hw_ctx.rcs_state->base);
> 
>     list_del(&ctx->link);
>     kfree(ctx);
> }
> 
> ... unless something is trying to reuse the context while it is still in
> the process of being deleted :(
> 
> In addition, the stack trace above implies that ctx->engine[].state WAS
> NULL when i915_gem_validate_context() was called, otherwise it would not
> have called intel_lr_context_deferred_create()
> 
>     if (i915.enable_execlists && !ctx->engine[ring->id].state) {
>         int ret = intel_lr_context_deferred_create(ctx, ring);
>         if (ret) {
>              DRM_DEBUG("Could not create LRC %u: %d\n", ctx_id, ret);
>              return ERR_PTR(ret);
>         }
>     }
> 
> and likewise that function would not have called gen8_init_rcs_context()
> unless this was a new context:
> 
>     if (ctx == ring->default_context)
>         lrc_setup_hardware_status_page(ring, ctx_obj);
>     else if (ring->id == RCS && !ctx->rcs_initialized) {
>         if (ring->init_context) {
>             ret = ring->init_context(ring, ctx);
>             if (ret) {
>                  DRM_ERROR("ring init context: %d\n", ret);
>                  ctx->engine[ring->id].ringbuf = NULL;
>                  ctx->engine[ring->id].state = NULL;
>                  goto error;
>             }
>         }
> 
>         ctx->rcs_initialized = true;
>     }
> 
> Note that rcs_initialized is never cleared, even with your change, so
> that in a use-after-free situation we wouldn't end up in this path. So I
> think the mystery is how this context ended up in an inconsistent state:
> has it been partially freed and then reused, or has some part of the new
> context allocation path failed but not been unwound correctly?
> 
> And if setting to NULL a pointer that's inside a structure that's in the
> process of being freed actually makes a difference, doesn't that mean
> there's a use-after-free issue somewhere?
> 
> .Dave.
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Mika Kuoppala Jan. 20, 2015, 4:04 p.m. UTC | #4

Ville Syrjälä <ville.syrjala@linux.intel.com> writes:

> On Fri, Jan 09, 2015 at 06:47:41PM +0000, Dave Gordon wrote:
>> On 08/01/15 13:40, Mika Kuoppala wrote:
>> > i915_gem_validate_context() will check the engine->state to see if it can
>> > submit into a ringbuffer. But when we are releasing the context we leave the
>> > engine state to a non null value. Thus after a successful hang recovery
>> > we might mistakenly submit to a non initialized ringbuffer resulting in:
>> > 
>> > [ 1991.356418] ------------[ cut here ]------------
>> > [ 1991.359192] WARNING: CPU: 1 PID: 2335 at lib/iomap.c:43 bad_io_access+0x3d/0x40()
>> > [ 1991.361966] Bad IO access at port 0x24 (outl(val,port))
>> > [ 1991.364750] Modules linked in: snd_hda_codec_hdmi i915 x86_pkg_temp_thermal coretemp kvm_intel kvm snd_hda_intel snd_hda_controller snd_hda_codec crct10dif_pclmul crc32_pclmul ghash_clmulni_intel snd_hwdep snd_pcm aesni_intel aes_x86_64 glue_helper lrw i2c_algo_bit gf128mul ablk_helper drm_kms_helper cryptd snd_seq_midi snd_seq_midi_event serio_raw drm snd_rawmidi snd_seq snd_seq_device snd_timer video snd soundcore mei_me lpc_ich bnep mac_hid acpi_pad mei rfcomm bluetooth parport_pc ppdev lp parport nls_iso8859_1 e1000e ptp ahci libahci pps_core sdhci_acpi sdhci
>> > [ 1991.370827] CPU: 1 PID: 2335 Comm: gem_ringfill Tainted: G        W      3.19.0-rc3+ #50
>> > [ 1991.373838] Hardware name: Intel Corporation Broadwell Client platform/SawTooth Peak, BIOS BDW-E1R1.86C.0092.R00.1408311942 08/31/2014
>> > [ 1991.376902]  ffffffff81aa1a46 ffff88014910fac8 ffffffff8173dbcf 0000000000000001
>> > [ 1991.379978]  ffff88014910fb18 ffff88014910fb08 ffffffff8107007a ffff88014910fb28
>> > [ 1991.383037]  ffff880147209940 ffff8800aafa8718 ffff8800aafa0000 ffff8800aafa1918
>> > [ 1991.386094] Call Trace:
>> > [ 1991.389140]  [<ffffffff8173dbcf>] dump_stack+0x45/0x57
>> > [ 1991.392207]  [<ffffffff8107007a>] warn_slowpath_common+0x8a/0xc0
>> > [ 1991.395268]  [<ffffffff810700f6>] warn_slowpath_fmt+0x46/0x50
>> > [ 1991.398330]  [<ffffffffa053290c>] ? intel_logical_ring_begin+0x3c/0x240 [i915]
>> > [ 1991.401395]  [<ffffffff813985bd>] bad_io_access+0x3d/0x40
>> > [ 1991.404462]  [<ffffffff81398763>] iowrite32+0x33/0x40
>> > [ 1991.407529]  [<ffffffffa0533585>] gen8_init_rcs_context+0xd5/0x170 [i915]
>> > [ 1991.410605]  [<ffffffffa0533d17>] intel_lr_context_deferred_create+0x657/0x8e0 [i915]
>> > [ 1991.413668]  [<ffffffffa050eff1>] i915_gem_do_execbuffer.isra.22+0xed1/0xf60 [i915]
>> > [ 1991.416736]  [<ffffffff811c0125>] ? __kmalloc+0x55/0x1b0
>> > [ 1991.419801]  [<ffffffffa051029c>] ? i915_gem_execbuffer2+0x6c/0x2c0 [i915]
>> > [ 1991.422772]  [<ffffffffa05102e1>] i915_gem_execbuffer2+0xb1/0x2c0 [i915]
>> > [ 1991.425632]  [<ffffffffa01b8ab4>] drm_ioctl+0x1a4/0x630 [drm]
>> > [ 1991.428454]  [<ffffffff811258bc>] ? acct_account_cputime+0x1c/0x20
>> > [ 1991.431255]  [<ffffffff811ee378>] do_vfs_ioctl+0x2f8/0x510
>> > [ 1991.434009]  [<ffffffff8109f834>] ? vtime_account_user+0x54/0x60
>> > [ 1991.436778]  [<ffffffff811ee611>] SyS_ioctl+0x81/0xa0
>> > [ 1991.439553]  [<ffffffff81745cb4>] ? int_check_syscall_exit_work+0x34/0x3d
>> > [ 1991.442306]  [<ffffffff81745a2d>] system_call_fastpath+0x16/0x1b
>> > 
>> > Fix this by setting all the engine fields properly when lrc is freed.
>> > 
>> > Cc: Thomas Daniel <thomas.daniel@intel.com>
>> > Signed-off-by: Mika Kuoppala <mika.kuoppala@intel.com>
>> > ---
>> >  drivers/gpu/drm/i915/intel_lrc.c | 4 ++++
>> >  1 file changed, 4 insertions(+)
>> > 
>> > diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
>> > index 7670a0f..32684d9 100644
>> > --- a/drivers/gpu/drm/i915/intel_lrc.c
>> > +++ b/drivers/gpu/drm/i915/intel_lrc.c
>> > @@ -1777,6 +1777,10 @@ void intel_lr_context_free(struct intel_context *ctx)
>> >  			intel_destroy_ringbuffer_obj(ringbuf);
>> >  			kfree(ringbuf);
>> >  			drm_gem_object_unreference(&ctx_obj->base);
>> > +			WARN_ON(ctx->engine[i].unpin_count != 0);
>> > +			ctx->engine[i].unpin_count = 0;
>> > +			ctx->engine[i].ringbuf = NULL;
>> > +			ctx->engine[i].state = NULL;
>> >  		}
>> >  	}
>> >  }
>> 
>> Hi,
>> 
>> I don't quite see how this can fix the problem illustrated by the stack
>> trace above.  AFAICS intel_lr_context_free() is called /only/ from
>> i915_gem_context_free(), which should mean that the refcount on the
>> intel_context object is already zero, and that it will be freed on
>> return. So the contents of ctx->engine[] should be irrelevant ...

Agreed that my patch does not address the problem, it is foobar.

But I tried to follow the tracks Ville uncovered and apparently
the problem is that we get the request allocation and the
actual add_request boundaries wrong in init.

The result is that we end up having ring->outstanding_lazy_request
with wrong ctx pointer (from previous emits)

And thus, the code thinks that the context is initialized, when in fact
it is the old context in olr that is fooling us.

-Mika


> I've now managed to hit this fairly reliably with
> 'gem_ringfill --r render' + ctrl-c straight after boot.
>
> To me it looks like some kind of bigger mess with olr. If olr was already there,
> logical_ring_alloc_request() will do absolutely nothing, which means the ringbuf
> won't even be pinned/mapped and ringbug->virtual_start will remain NULL. And that
> leads to the w/a init to call iowrite() with some close to 0 iomem addressses.
>
> I'm not sure how this is supposed to actually work. To me it looks olr should never
> be allowed to be != NULL when creating the context/ringbuf to avoid this problem. And
> yet somehow the presence of olr doesn't always trigger the 'bad IO access' warns. In
> fact I can't seem to trigger it more than once until I reboot the machine. I tried to
> add a warning [1] to catch all cases when olr is already there, and it will trigger in
> most cases when I hit ctrl-c while running gem_ringfill. And yet the 'Bad IO access'
> doesn't follow it except once after boot.
>
> [1]
> @@ -1906,6 +1926,8 @@ int intel_lr_context_deferred_create(struct intel_context *ctx,
>  
>         }
>  
> +       WARN_ON(ring->outstanding_lazy_request);
> +
>         ret = populate_lr_context(ctx, ctx_obj, ring, ringbuf);
>         if (ret) {
>                 DRM_DEBUG_DRIVER("Failed to populate LRC: %d\n", ret);
>
>
>
>> 
>> void i915_gem_context_free(struct kref *ctx_ref)
>> {
>>     struct intel_context *ctx = container_of(ctx_ref,
>>                                              typeof(*ctx), ref);
>> 
>>     trace_i915_context_free(ctx);
>> 
>>     if (i915.enable_execlists)
>>         intel_lr_context_free(ctx);
>> 
>>     i915_ppgtt_put(ctx->ppgtt);
>> 
>>     if (ctx->legacy_hw_ctx.rcs_state)
>>         drm_gem_object_unreference(&ctx->legacy_hw_ctx.rcs_state->base);
>> 
>>     list_del(&ctx->link);
>>     kfree(ctx);
>> }
>> 
>> ... unless something is trying to reuse the context while it is still in
>> the process of being deleted :(
>> 
>> In addition, the stack trace above implies that ctx->engine[].state WAS
>> NULL when i915_gem_validate_context() was called, otherwise it would not
>> have called intel_lr_context_deferred_create()
>> 
>>     if (i915.enable_execlists && !ctx->engine[ring->id].state) {
>>         int ret = intel_lr_context_deferred_create(ctx, ring);
>>         if (ret) {
>>              DRM_DEBUG("Could not create LRC %u: %d\n", ctx_id, ret);
>>              return ERR_PTR(ret);
>>         }
>>     }
>> 
>> and likewise that function would not have called gen8_init_rcs_context()
>> unless this was a new context:
>> 
>>     if (ctx == ring->default_context)
>>         lrc_setup_hardware_status_page(ring, ctx_obj);
>>     else if (ring->id == RCS && !ctx->rcs_initialized) {
>>         if (ring->init_context) {
>>             ret = ring->init_context(ring, ctx);
>>             if (ret) {
>>                  DRM_ERROR("ring init context: %d\n", ret);
>>                  ctx->engine[ring->id].ringbuf = NULL;
>>                  ctx->engine[ring->id].state = NULL;
>>                  goto error;
>>             }
>>         }
>> 
>>         ctx->rcs_initialized = true;
>>     }
>> 
>> Note that rcs_initialized is never cleared, even with your change, so
>> that in a use-after-free situation we wouldn't end up in this path. So I
>> think the mystery is how this context ended up in an inconsistent state:
>> has it been partially freed and then reused, or has some part of the new
>> context allocation path failed but not been unwound correctly?
>> 
>> And if setting to NULL a pointer that's inside a structure that's in the
>> process of being freed actually makes a difference, doesn't that mean
>> there's a use-after-free issue somewhere?
>> 
>> .Dave.
>> _______________________________________________
>> Intel-gfx mailing list
>> Intel-gfx@lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/intel-gfx
>
> -- 
> Ville Syrjälä
> Intel OTC

diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 7670a0f..32684d9 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -1777,6 +1777,10 @@  void intel_lr_context_free(struct intel_context *ctx)
 			intel_destroy_ringbuffer_obj(ringbuf);
 			kfree(ringbuf);
 			drm_gem_object_unreference(&ctx_obj->base);
+			WARN_ON(ctx->engine[i].unpin_count != 0);
+			ctx->engine[i].unpin_count = 0;
+			ctx->engine[i].ringbuf = NULL;
+			ctx->engine[i].state = NULL;
 		}
 	}
 }