From patchwork Mon May 18 18:17:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Janusz Krzysztofik X-Patchwork-Id: 11556075 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 20535913 for ; Mon, 18 May 2020 18:17:55 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 094F720671 for ; Mon, 18 May 2020 18:17:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 094F720671 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B5E346E46E; Mon, 18 May 2020 18:17:52 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9787D894DE for ; Mon, 18 May 2020 18:17:49 +0000 (UTC) IronPort-SDR: 945moTbH7JYQSKxOXVaVQKJn6x5VV4jKauna12O3b13iwdaalOMH85aYYiuBHtSfcvUf8cQC/1 mWseXuD9ATaA== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 May 2020 11:17:48 -0700 IronPort-SDR: UyuoAih0tLcpmAKxZK9ULuujxKaKi8QeR1c9YsPTleYo7tMiZ4XpMs4OQ9hngwR5zD9mQGp9rJ opwH866Gk7nA== X-IronPort-AV: E=Sophos;i="5.73,407,1583222400"; d="scan'208";a="439318285" Received: from jkrzyszt-desk.ger.corp.intel.com (HELO jkrzyszt-desk.igk.intel.com) ([172.22.244.18]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 May 2020 11:17:47 -0700 From: Janusz Krzysztofik To: intel-gfx@lists.freedesktop.org Date: Mon, 18 May 2020 20:17:17 +0200 Message-Id: <20200518181720.14625-2-janusz.krzysztofik@linux.intel.com> X-Mailer: git-send-email 2.21.1 In-Reply-To: <20200518181720.14625-1-janusz.krzysztofik@linux.intel.com> References: <20200518181720.14625-1-janusz.krzysztofik@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [RFC PATCH 1/4] drm/i915: Drop user contexts on driver remove X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Contexts associated with open device file descriptors together with their assigned address spaces are now closed on device file close. On address space closure its associated DMA mappings are revoked. If the device is removed while being open, subsequent attempts to revoke those mappings while closing the device file descriptor may may be judged by intel-iommu code as a bug and result in kernel panic. Since user contexts become useless after the device is no longer available, drop them on device removal. <4> [36.900985] ------------[ cut here ]------------ <2> [36.901005] kernel BUG at drivers/iommu/intel-iommu.c:3717! <4> [36.901105] invalid opcode: 0000 [#1] PREEMPT SMP NOPTI <4> [36.901117] CPU: 0 PID: 39 Comm: kworker/u8:1 Tainted: G U W 5.7.0-rc5-CI-CI_DRM_8485+ #1 <4> [36.901133] Hardware name: Intel Corporation Elkhart Lake Embedded Platform/ElkhartLake LPDDR4x T3 CRB, BIOS EHLSFWI1.R00.1484.A00.1911290833 11/29/2019 <4> [36.901250] Workqueue: i915 __i915_vm_release [i915] <4> [36.901264] RIP: 0010:intel_unmap+0x1f5/0x230 <4> [36.901274] Code: 01 e8 9f bc a9 ff 85 c0 74 09 80 3d df 60 09 01 00 74 19 65 ff 0d 13 12 97 7e 0f 85 fc fe ff ff e8 82 b0 95 ff e9 f2 fe ff ff <0f> 0b e8 d4 bd a9 ff 85 c0 75 de 48 c7 c2 10 84 2c 82 be 54 00 00 <4> [36.901302] RSP: 0018:ffffc900001ebdc0 EFLAGS: 00010246 <4> [36.901313] RAX: 0000000000000000 RBX: ffff8882561dd000 RCX: 0000000000000000 <4> [36.901324] RDX: 0000000000001000 RSI: 00000000ffd9c000 RDI: ffff888274c94000 <4> [36.901336] RBP: ffff888274c940b0 R08: 0000000000000000 R09: 0000000000000001 <4> [36.901348] R10: 000000000a25d812 R11: 00000000112af2d4 R12: ffff888252c70200 <4> [36.901360] R13: 00000000ffd9c000 R14: 0000000000001000 R15: ffff8882561dd010 <4> [36.901372] FS: 0000000000000000(0000) GS:ffff888278000000(0000) knlGS:0000000000000000 <4> [36.901386] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [36.901396] CR2: 00007f06def54950 CR3: 0000000255844000 CR4: 0000000000340ef0 <4> [36.901408] Call Trace: <4> [36.901418] ? process_one_work+0x1de/0x600 <4> [36.901494] cleanup_page_dma+0x37/0x70 [i915] <4> [36.901573] free_pd+0x9/0x20 [i915] <4> [36.901644] gen8_ppgtt_cleanup+0x59/0xc0 [i915] <4> [36.901721] __i915_vm_release+0x14/0x30 [i915] <4> [36.901733] process_one_work+0x268/0x600 <4> [36.901744] ? __schedule+0x307/0x8d0 <4> [36.901756] worker_thread+0x37/0x380 <4> [36.901766] ? process_one_work+0x600/0x600 <4> [36.901775] kthread+0x140/0x160 <4> [36.901783] ? kthread_park+0x80/0x80 <4> [36.901792] ret_from_fork+0x24/0x50 <4> [36.901804] Modules linked in: mei_hdcp i915 x86_pkg_temp_thermal coretemp crct10dif_pclmul crc32_pclmul ghash_clmulni_intel ax88179_178a usbnet mii mei_me mei prime_numbers intel_lpss_pci <4> [36.901857] ---[ end trace 52d1b4d81f8d1ea7 ]--- Signed-off-by: Janusz Krzysztofik --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 38 +++++++++++++++++++++ drivers/gpu/drm/i915/gem/i915_gem_context.h | 1 + drivers/gpu/drm/i915/i915_gem.c | 2 ++ 3 files changed, 41 insertions(+) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index 900ea8b7fc8f..0096a69fbfd3 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -927,6 +927,44 @@ void i915_gem_driver_release__contexts(struct drm_i915_private *i915) rcu_barrier(); /* and flush the left over RCU frees */ } +void i915_gem_driver_remove__contexts(struct drm_i915_private *i915) +{ + struct i915_gem_context *ctx, *cn; + + list_for_each_entry_safe(ctx, cn, &i915->gem.contexts.list, link) { + struct drm_i915_file_private *file_priv = ctx->file_priv; + struct i915_gem_context *entry; + unsigned long int id; + + if (i915_gem_context_is_closed(ctx) || IS_ERR(file_priv)) + continue; + + xa_for_each(&file_priv->context_xa, id, entry) { + struct i915_address_space *vm; + unsigned long int idx; + + if (entry != ctx) + continue; + + xa_erase(&file_priv->context_xa, id); + + if (id) + break; + + xa_for_each(&file_priv->vm_xa, idx, vm) { + xa_erase(&file_priv->vm_xa, idx); + i915_vm_put(vm); + } + + break; + } + + context_close(ctx); + } + + i915_gem_driver_release__contexts(i915); +} + static int gem_context_register(struct i915_gem_context *ctx, struct drm_i915_file_private *fpriv, u32 *id) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.h b/drivers/gpu/drm/i915/gem/i915_gem_context.h index 3702b2fb27ab..62808bea9239 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.h @@ -110,6 +110,7 @@ i915_gem_context_clear_user_engines(struct i915_gem_context *ctx) /* i915_gem_context.c */ void i915_gem_init__contexts(struct drm_i915_private *i915); +void i915_gem_driver_remove__contexts(struct drm_i915_private *i915); void i915_gem_driver_release__contexts(struct drm_i915_private *i915); int i915_gem_context_open(struct drm_i915_private *i915, diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 0cbcb9f54e7d..87d3c4f5b6c6 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -1189,6 +1189,8 @@ void i915_gem_driver_remove(struct drm_i915_private *dev_priv) /* Flush any outstanding unpin_work. */ i915_gem_drain_workqueue(dev_priv); + i915_gem_driver_remove__contexts(dev_priv); + i915_gem_drain_freed_objects(dev_priv); } From patchwork Mon May 18 18:17:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Janusz Krzysztofik X-Patchwork-Id: 11556071 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B9418138A for ; Mon, 18 May 2020 18:17:52 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A279E20671 for ; Mon, 18 May 2020 18:17:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A279E20671 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 17BA4896C4; Mon, 18 May 2020 18:17:52 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by gabe.freedesktop.org (Postfix) with ESMTPS id 501C8896D8 for ; Mon, 18 May 2020 18:17:50 +0000 (UTC) IronPort-SDR: LAlI+ycwTSsIIwo/cAuP5FldrT8KRY457cAd06XfT7dWQryC4uKuB/EtWrZ4cBNmp+eZchj6DQ tpIs+6Hx44xQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 May 2020 11:17:50 -0700 IronPort-SDR: SaEhro75nk5XdA/nbmwPNpSkb/G2U4AZ/Jzz3LvizJRLQiNK4Oa9nGpjUU2YLo1Thh6dfTl5iW tVZ4u+iXVbzQ== X-IronPort-AV: E=Sophos;i="5.73,407,1583222400"; d="scan'208";a="439318288" Received: from jkrzyszt-desk.ger.corp.intel.com (HELO jkrzyszt-desk.igk.intel.com) ([172.22.244.18]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 May 2020 11:17:49 -0700 From: Janusz Krzysztofik To: intel-gfx@lists.freedesktop.org Date: Mon, 18 May 2020 20:17:18 +0200 Message-Id: <20200518181720.14625-3-janusz.krzysztofik@linux.intel.com> X-Mailer: git-send-email 2.21.1 In-Reply-To: <20200518181720.14625-1-janusz.krzysztofik@linux.intel.com> References: <20200518181720.14625-1-janusz.krzysztofik@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [RFC PATCH 2/4] drm/i915: Release GT resources on driver remove X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" GT scratch page is now released and its DMA mappings revoked on driver release. If a device is removed while its file descriptor is still open, the driver is not released until last device file descriptor closure. In that case intel-iommu code may judge late DMA unmapping as a bug and kernel panic may occur. Since DMA mapped address space may be no longer usable after device removal, release GT resources including scratch page as well as a reference to its address space on driver remove. Implement that by just calling intel_gt_driver_release() on GT remove as that function has been already made safe to be called again on driver release even if already called before, e.g. on GEM initialization failure. <4> [39.201062] ------------[ cut here ]------------ <2> [39.201074] kernel BUG at drivers/iommu/intel-iommu.c:3717! <4> [39.201154] invalid opcode: 0000 [#1] PREEMPT SMP NOPTI <4> [39.201162] CPU: 6 PID: 7 Comm: kworker/u16:0 Tainted: G U W 5.7.0-rc5-CI-CI_DRM_8485+ #1 <4> [39.201172] Hardware name: Intel Corporation Ice Lake Client Platform/IceLake U DDR4 SODIMM PD RVP, BIOS ICLSFWR1.R00.3175.A00.1904261428 04/26/2019 <4> [39.201243] Workqueue: i915 __i915_gem_free_work [i915] <4> [39.201252] RIP: 0010:intel_unmap+0x1f5/0x230 <4> [39.201260] Code: 01 e8 9f bc a9 ff 85 c0 74 09 80 3d df 60 09 01 00 74 19 65 ff 0d 13 12 97 7e 0f 85 fc fe ff ff e8 82 b0 95 ff e9 f2 fe ff ff <0f> 0b e8 d4 bd a9 ff 85 c0 75 de 48 c7 c2 10 84 2c 82 be 54 00 00 <4> [39.201278] RSP: 0018:ffffc900000dbc98 EFLAGS: 00010246 <4> [39.201285] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffea0021d30000 <4> [39.201293] RDX: 000000000005f000 RSI: 00000000fed00000 RDI: ffff888889eec000 <4> [39.201301] RBP: ffff888889eec0b0 R08: 0000000000000000 R09: 00000000fffffffe <4> [39.201309] R10: 00000000458139fc R11: 00000000f6c6d8b2 R12: 0000000000000025 <4> [39.201318] R13: 00000000fed00000 R14: 000000000005f000 R15: 0000000000000025 <4> [39.201326] FS: 0000000000000000(0000) GS:ffff888890100000(0000) knlGS:0000000000000000 <4> [39.201335] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [39.201342] CR2: 0000560f1308e148 CR3: 0000000881972002 CR4: 0000000000760ee0 <4> [39.201350] PKRU: 55555554 <4> [39.201355] Call Trace: <4> [39.201361] intel_unmap_sg+0x7b/0x180 <4> [39.201412] shmem_put_pages+0x43/0x250 [i915] <4> [39.201472] ? __i915_gem_object_unset_pages.part.12+0x11b/0x1d0 [i915] <4> [39.201531] ? __i915_gem_object_unset_pages.part.12+0x133/0x1d0 [i915] <4> [39.201590] __i915_gem_object_put_pages+0x81/0xc0 [i915] <4> [39.201646] __i915_gem_free_objects.isra.21+0x1a7/0x4b0 [i915] <4> [39.201658] process_one_work+0x268/0x600 <4> [39.201666] ? __schedule+0x307/0x8d0 <4> [39.201675] worker_thread+0x1d0/0x380 <4> [39.201682] ? process_one_work+0x600/0x600 <4> [39.201689] kthread+0x140/0x160 <4> [39.201695] ? kthread_park+0x80/0x80 <4> [39.201703] ret_from_fork+0x24/0x50 <4> [39.201712] Modules linked in: snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic i915 mei_hdcp x86_pkg_temp_thermal coretemp crct10dif_pclmul crc32_pclmul snd_hda_intel snd_intel_dspcfg snd_hda_codec e1000e ax88179_178a usbnet snd_hwdep mii snd_hda_core ghash_clmulni_intel snd_pcm ptp pps_core mei_me mei intel_lpss_pci prime_numbers <4> [39.201764] ---[ end trace f3ec1bae3de04509 ]--- Signed-off-by: Janusz Krzysztofik Reviewed-by: Michał Winiarski --- drivers/gpu/drm/i915/gt/intel_gt.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c index f069551e412f..5771e80e85a6 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -599,6 +599,8 @@ void intel_gt_driver_remove(struct intel_gt *gt) intel_uc_driver_remove(>->uc); intel_engines_release(gt); + + intel_gt_driver_release(gt); } void intel_gt_driver_unregister(struct intel_gt *gt) From patchwork Mon May 18 18:17:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Janusz Krzysztofik X-Patchwork-Id: 11556073 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1829C913 for ; Mon, 18 May 2020 18:17:54 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F142D20826 for ; Mon, 18 May 2020 18:17:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F142D20826 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 78DF56E46C; Mon, 18 May 2020 18:17:52 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by gabe.freedesktop.org (Postfix) with ESMTPS id 661A9896C4 for ; Mon, 18 May 2020 18:17:51 +0000 (UTC) IronPort-SDR: V4hY0+QRdMNQhta42Ou91IgwadYLg0PNHZOOubjDynlHEOO+CN7WCUSclSAqZDoLJXVMV6g3Ir tErKwGseszBg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 May 2020 11:17:51 -0700 IronPort-SDR: hlx2+AR8mR+0LcrjCjSKRfChQMoElfi0vUe3QG6ieTzbKWRKKoUKRHXIxVb2ttj9Xekt6Ck6dw NQSq2obDfnEw== X-IronPort-AV: E=Sophos;i="5.73,407,1583222400"; d="scan'208";a="439318291" Received: from jkrzyszt-desk.ger.corp.intel.com (HELO jkrzyszt-desk.igk.intel.com) ([172.22.244.18]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 May 2020 11:17:50 -0700 From: Janusz Krzysztofik To: intel-gfx@lists.freedesktop.org Date: Mon, 18 May 2020 20:17:19 +0200 Message-Id: <20200518181720.14625-4-janusz.krzysztofik@linux.intel.com> X-Mailer: git-send-email 2.21.1 In-Reply-To: <20200518181720.14625-1-janusz.krzysztofik@linux.intel.com> References: <20200518181720.14625-1-janusz.krzysztofik@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [RFC PATCH 3/4] drm/i915: Move GGTT cleanup from driver_release to _remove X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" GGTT including its scratch page is not cleaned up until driver release. Since DMA mappings still exist before scratch page cleanup, unmapping them on last device close after the driver has been already removed may be judged by intel-iommu code as a bug and result in kernel panic. Since we abort requests and block user access to hardware on device removal, there seems not much sense in still keeping GGTT. Clean it up on driver remove. However, skip GGTT address space cleanup as its mutext may still be needed if there are objects to be freed. That cleanup is always called on address space release after all. [ 81.290284] ------------[ cut here ]------------ [ 81.291602] kernel BUG at drivers/iommu/intel-iommu.c:3591! [ 81.293558] invalid opcode: 0000 [#1] PREEMPT SMP [ 81.294695] CPU: 0 PID: 207 Comm: core_hotunplug Tainted: G U 5.4.17 #2 [ 81.296579] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [ 81.297959] RIP: 0010:intel_unmap+0x200/0x230 [ 81.299015] Code: 00 e8 e4 45 c5 ff 85 c0 74 09 80 3d 2b 84 c0 00 00 74 19 65 ff 0d 78 9a b2 7e 0f 85 fa fe ff ff e8 95 57 b1 ff e9 f0 fe ff ff <0f> 0b e8 19 4c c5 ff 85 c0 75 de 48 c7 c2 48 d2 e1 81 be 57 00 00 [ 81.303425] RSP: 0018:ffffc9000013fda0 EFLAGS: 00010246 [ 81.304683] RAX: 0000000000000000 RBX: ffff8882228dd0b0 RCX: 0000000000000000 [ 81.306384] RDX: 0000000000001000 RSI: 00000000af801000 RDI: ffff8882228dd0b0 [ 81.308086] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 [ 81.309788] R10: 0000000000000000 R11: 0000000000000000 R12: 00000000af801000 [ 81.311489] R13: ffff888223a00000 R14: 0000000000001000 R15: ffff888223a0a2e8 [ 81.313191] FS: 00007f5408e3c940(0000) GS:ffff888228600000(0000) knlGS:0000000000000000 [ 81.315116] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 81.316495] CR2: 0000000001fc0048 CR3: 000000022464a000 CR4: 00000000000006b0 [ 81.318196] Call Trace: [ 81.318967] cleanup_scratch_page+0x44/0x80 [i915] [ 81.320281] i915_ggtt_driver_release+0x15b/0x220 [i915] [ 81.321717] i915_driver_release+0x33/0x90 [i915] [ 81.322856] drm_release+0xbc/0xd0 [ 81.323691] __fput+0xcd/0x260 [ 81.324447] task_work_run+0x90/0xc0 [ 81.325323] do_syscall_64+0x3da/0x560 [ 81.326240] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 81.327457] RIP: 0033:0x7f54096ecba3 [ 81.328332] Code: 00 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb bb 0f 1f 80 00 00 00 00 64 8b 04 25 18 00 00 00 85 c0 75 14 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 45 c3 0f 1f 40 00 48 83 ec 18 89 7c 24 0c e8 [ 81.332741] RSP: 002b:00007ffcc5165698 EFLAGS: 00000246 ORIG_RAX: 0000000000000003 [ 81.334546] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007f54096ecba3 [ 81.336247] RDX: 00000000005cc5d0 RSI: 0000000000000005 RDI: 0000000000000004 [ 81.337949] RBP: 0000000000000003 R08: 00000000005b8014 R09: 0000000000000004 [ 81.339650] R10: 00000000005cc650 R11: 0000000000000246 R12: 00000000004022f0 [ 81.341352] R13: 00007ffcc5165850 R14: 0000000000000000 R15: 0000000000000000 [ 81.343059] Modules linked in: i915 mfd_core intel_gtt prime_numbers [ 81.345015] ---[ end trace 010aae55e56f8998 ]--- Signed-off-by: Janusz Krzysztofik drm/i915: Defer GGTT vm address space fini to vm release Signed-off-by: Janusz Krzysztofik --- drivers/gpu/drm/i915/gt/intel_ggtt.c | 13 +++++++++---- drivers/gpu/drm/i915/gt/intel_gtt.h | 1 + drivers/gpu/drm/i915/i915_drv.c | 2 ++ 3 files changed, 12 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c index 66165b10256e..ff2b4f74149a 100644 --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c @@ -701,7 +701,6 @@ static void ggtt_cleanup_hw(struct i915_ggtt *ggtt) ggtt->vm.cleanup(&ggtt->vm); mutex_unlock(&ggtt->vm.mutex); - i915_address_space_fini(&ggtt->vm); arch_phys_wc_del(ggtt->mtrr); @@ -709,6 +708,15 @@ static void ggtt_cleanup_hw(struct i915_ggtt *ggtt) io_mapping_fini(&ggtt->iomap); } +void i915_ggtt_driver_remove(struct drm_i915_private *i915) +{ + struct i915_ggtt *ggtt = &i915->ggtt; + + fini_aliasing_ppgtt(ggtt); + + ggtt_cleanup_hw(ggtt); +} + /** * i915_ggtt_driver_release - Clean up GGTT hardware initialization * @i915: i915 device @@ -718,10 +726,7 @@ void i915_ggtt_driver_release(struct drm_i915_private *i915) struct i915_ggtt *ggtt = &i915->ggtt; struct pagevec *pvec; - fini_aliasing_ppgtt(ggtt); - intel_ggtt_fini_fences(ggtt); - ggtt_cleanup_hw(ggtt); pvec = &i915->mm.wc_stash.pvec; if (pvec->nr) { diff --git a/drivers/gpu/drm/i915/gt/intel_gtt.h b/drivers/gpu/drm/i915/gt/intel_gtt.h index d93ebdf3fa0e..f140ce5c171a 100644 --- a/drivers/gpu/drm/i915/gt/intel_gtt.h +++ b/drivers/gpu/drm/i915/gt/intel_gtt.h @@ -501,6 +501,7 @@ int i915_ggtt_enable_hw(struct drm_i915_private *i915); void i915_ggtt_enable_guc(struct i915_ggtt *ggtt); void i915_ggtt_disable_guc(struct i915_ggtt *ggtt); int i915_init_ggtt(struct drm_i915_private *i915); +void i915_ggtt_driver_remove(struct drm_i915_private *i915); void i915_ggtt_driver_release(struct drm_i915_private *i915); static inline bool i915_ggtt_has_aperture(const struct i915_ggtt *ggtt) diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c index 34ee12f3f02d..e4d9f0f6f183 100644 --- a/drivers/gpu/drm/i915/i915_drv.c +++ b/drivers/gpu/drm/i915/i915_drv.c @@ -752,6 +752,7 @@ static int i915_driver_hw_probe(struct drm_i915_private *dev_priv) err_mem_regions: intel_memory_regions_driver_release(dev_priv); err_ggtt: + i915_ggtt_driver_remove(dev_priv); i915_ggtt_driver_release(dev_priv); err_perf: i915_perf_fini(dev_priv); @@ -768,6 +769,7 @@ static void i915_driver_hw_remove(struct drm_i915_private *dev_priv) i915_perf_fini(dev_priv); + i915_ggtt_driver_remove(dev_priv); if (pdev->msi_enabled) pci_disable_msi(pdev); From patchwork Mon May 18 18:17:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Janusz Krzysztofik X-Patchwork-Id: 11556077 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 19D87138A for ; Mon, 18 May 2020 18:17:56 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 02C1320671 for ; Mon, 18 May 2020 18:17:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 02C1320671 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 490476E471; Mon, 18 May 2020 18:17:53 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7D1736E46D for ; Mon, 18 May 2020 18:17:52 +0000 (UTC) IronPort-SDR: 24VEfL/uwmM0yXSn/j3nsqibj9sucD/coS/XLKGCaolSg0BJSnqA1BuOuxSE0d8SYsV8c72S4l m9QHzV1QJ0+Q== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 May 2020 11:17:52 -0700 IronPort-SDR: aRNJ1t059QILTwclHsulRYT+CMyUDPgH9AFvQX2rVgOP6CHDlyMJ/1iqKD7QivCM11b8hpCN1z tTufsFzeF65g== X-IronPort-AV: E=Sophos;i="5.73,407,1583222400"; d="scan'208";a="439318297" Received: from jkrzyszt-desk.ger.corp.intel.com (HELO jkrzyszt-desk.igk.intel.com) ([172.22.244.18]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 May 2020 11:17:51 -0700 From: Janusz Krzysztofik To: intel-gfx@lists.freedesktop.org Date: Mon, 18 May 2020 20:17:20 +0200 Message-Id: <20200518181720.14625-5-janusz.krzysztofik@linux.intel.com> X-Mailer: git-send-email 2.21.1 In-Reply-To: <20200518181720.14625-1-janusz.krzysztofik@linux.intel.com> References: <20200518181720.14625-1-janusz.krzysztofik@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [RFC PATCH 4/4] drm/i915: Move UC firmware cleanup from driver_release to _remove X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" UC firmware is stored in a GEM object. Clean it up on driver remove to avoid intel-iommu triggered kernel panic on late DMA unmapping or even an RPM related warning on object late removal in no IOMMU setups. <4> [93.335282] ------------[ cut here ]------------ <4> [93.335515] pm_runtime_get_sync() failed: -13 <4> [93.336056] WARNING: CPU: 6 PID: 200 at drivers/gpu/drm/i915/intel_runtime_pm.c:361 __intel_runtime_pm_get+0x4d/0x60 [i915] <4> [93.336104] Modules linked in: snd_hda_codec_hdmi mei_hdcp i915 x86_pkg_temp_thermal coretemp crct10dif_pclmul crc32_pclmul ghash_clmulni_intel snd_hda_intel cdc_ether snd_intel_dspcfg usbnet snd_hda_codec mii snd_hwdep snd_hda_core e1000e snd_pcm ptp pps_core mei_me mei intel_lpss_pci prime_numbers <4> [93.336268] CPU: 6 PID: 200 Comm: kworker/u16:3 Tainted: G U 5.7.0-rc4-CI-Trybot_6405+ #1 <4> [93.336289] Hardware name: Intel Corporation Tiger Lake Client Platform/TigerLake Y LPDDR4x T4 Crb, BIOS TGLSFWI1.R00.2457.A16.1912270059 12/27/2019 <4> [93.336811] Workqueue: i915 __i915_gem_free_work [i915] <4> [93.337296] RIP: 0010:__intel_runtime_pm_get+0x4d/0x60 [i915] <4> [93.337332] Code: ff ff 48 89 df 5b 5d e9 a1 fa ff ff 80 3d 4b 7a 2e 00 00 75 e1 89 c6 48 c7 c7 a8 2d 40 a0 c6 05 39 7a 2e 00 01 e8 53 fc e9 e0 <0f> 0b eb c8 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 41 57 41 <4> [93.337357] RSP: 0018:ffffc9000144bdd8 EFLAGS: 00010282 <4> [93.337384] RAX: 0000000000000000 RBX: ffff88838ee5bc40 RCX: 0000000000000001 <4> [93.337409] RDX: 0000000080000001 RSI: ffff88839d264928 RDI: 00000000ffffffff <4> [93.337440] RBP: 0000000000000001 R08: ffff88839d264928 R09: 0000000000000000 <4> [93.337467] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88838ee5bc40 <4> [93.337493] R13: 0000000000000006 R14: ffffffff82769a30 R15: ffff88839376bab0 <4> [93.337515] FS: 0000000000000000(0000) GS:ffff8883a4100000(0000) knlGS:0000000000000000 <4> [93.337542] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [93.337563] CR2: 000055bc19b16ff8 CR3: 00000003a11c4005 CR4: 0000000000760ee0 <4> [93.337583] PKRU: 55555554 <4> [93.337605] Call Trace: <4> [93.338148] i915_gem_object_release_mmap+0x23/0x70 [i915] <4> [93.338665] __i915_gem_free_objects.isra.21+0x10a/0x4b0 [i915] <4> [93.338741] process_one_work+0x268/0x600 <4> [93.338785] ? __schedule+0x307/0x8d0 <4> [93.338878] worker_thread+0x37/0x380 <4> [93.338929] ? process_one_work+0x600/0x600 <4> [93.338963] kthread+0x140/0x160 <4> [93.339006] ? kthread_park+0x80/0x80 <4> [93.339057] ret_from_fork+0x24/0x50 <4> [93.339181] irq event stamp: 204220 <4> [93.339219] hardirqs last enabled at (204219): [] console_unlock+0x4cd/0x5a0 <4> [93.339250] hardirqs last disabled at (204220): [] trace_hardirqs_off_thunk+0x1a/0x1c <4> [93.339277] softirqs last enabled at (204208): [] __do_softirq+0x395/0x49e <4> [93.339307] softirqs last disabled at (204197): [] irq_exit+0xba/0xc0 <4> [93.339330] ---[ end trace f066187622b8c484 ]--- Signed-off-by: Janusz Krzysztofik Reviewed-by: Michał Winiarski --- drivers/gpu/drm/i915/i915_gem.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 87d3c4f5b6c6..f9d37c7e6d6f 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -1191,6 +1191,8 @@ void i915_gem_driver_remove(struct drm_i915_private *dev_priv) i915_gem_driver_remove__contexts(dev_priv); + intel_uc_cleanup_firmwares(&dev_priv->gt.uc); + i915_gem_drain_freed_objects(dev_priv); } @@ -1202,7 +1204,6 @@ void i915_gem_driver_release(struct drm_i915_private *dev_priv) intel_wa_list_free(&dev_priv->gt_wa_list); - intel_uc_cleanup_firmwares(&dev_priv->gt.uc); i915_gem_cleanup_userptr(dev_priv); i915_gem_drain_freed_objects(dev_priv);