From patchwork Wed Sep 14 22:04:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Roper X-Patchwork-Id: 12976580 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 736A7C6FA82 for ; Wed, 14 Sep 2022 22:04:51 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id CFE1D10EA0E; Wed, 14 Sep 2022 22:04:44 +0000 (UTC) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by gabe.freedesktop.org (Postfix) with ESMTPS id 21B1510EA0E; Wed, 14 Sep 2022 22:04:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663193081; x=1694729081; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DeNnr9C26u2Vin05I4XJHX9ymoaMOzdoJW7jAwyYXBg=; b=dGIem6zMf4UmO7qCsijEYmTaxz1qN8l9sM1LZY5naN3ViHZNV1siJghf 7ThHg1TwpUbuC0aGBMIo40vb2V33PATt7/zycU83ubsnLqLsnRe2Juzco egc7yeyHqspeL8uU43/H3FXA2sZVQ1kbZsLhEHTEfbbNg2W55DMpnCjp6 KtFMLKxx9I1BaoErDLoNtQfXR8l4sWTJFKdsjBvIjAbVUtNEUEJfCf02y m8VxK02121fWlW5HKDt+mhjSfCNAnGGPiW+ISYemnL/9v9fS4FbIeDy2g HT4rBRRGHIt7G0one11U9Kq1PYgHtEdTyOOS7DcPV5Kf7+VfAqB2eBiiT A==; X-IronPort-AV: E=McAfee;i="6500,9779,10470"; a="278280793" X-IronPort-AV: E=Sophos;i="5.93,315,1654585200"; d="scan'208";a="278280793" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2022 15:04:40 -0700 X-IronPort-AV: E=Sophos;i="5.93,315,1654585200"; d="scan'208";a="647570924" Received: from mdroper-desk1.fm.intel.com ([10.1.27.134]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2022 15:04:40 -0700 From: Matt Roper To: intel-gfx@lists.freedesktop.org Subject: [PATCH 1/4] drm/i915/gt: Cleanup partial engine discovery failures Date: Wed, 14 Sep 2022 15:04:24 -0700 Message-Id: <20220914220427.3091448-2-matthew.d.roper@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220914220427.3091448-1-matthew.d.roper@intel.com> References: <20220914220427.3091448-1-matthew.d.roper@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Janusz Krzysztofik , Chris Wilson , dri-devel@lists.freedesktop.org, Chris Wilson Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Chris Wilson If we abort driver initialisation in the middle of gt/engine discovery, some engines will be fully setup and some not. Those incompletely setup engines only have 'engine->release == NULL' and so will leak any of the common objects allocated. Signed-off-by: Chris Wilson Cc: Janusz Krzysztofik Signed-off-by: Matt Roper --- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c index 1f7188129cd1..bff12b4ec314 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c @@ -1196,6 +1196,12 @@ void intel_engine_destroy_pinned_context(struct intel_context *ce) intel_context_put(ce); } +static void destroy_pinned_context(struct intel_context *ce) +{ + if (ce) + intel_engine_destroy_pinned_context(ce); +} + static struct intel_context * create_kernel_context(struct intel_engine_cs *engine) { @@ -1274,8 +1280,13 @@ int intel_engines_init(struct intel_gt *gt) return err; err = setup(engine); - if (err) + if (err) { + intel_engine_cleanup_common(engine); return err; + } + + /* The backend should now be responsible for cleanup */ + GEM_BUG_ON(engine->release == NULL); err = engine_init_common(engine); if (err) @@ -1307,8 +1318,7 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine) if (engine->default_state) fput(engine->default_state); - if (engine->kernel_context) - intel_engine_destroy_pinned_context(engine->kernel_context); + destroy_pinned_context(engine->kernel_context); GEM_BUG_ON(!llist_empty(&engine->barrier_tasks)); cleanup_status_page(engine); From patchwork Wed Sep 14 22:04:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Roper X-Patchwork-Id: 12976583 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D1000C6FA86 for ; Wed, 14 Sep 2022 22:05:21 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8D7FC10EA18; Wed, 14 Sep 2022 22:04:58 +0000 (UTC) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by gabe.freedesktop.org (Postfix) with ESMTPS id 432E010EA0F; Wed, 14 Sep 2022 22:04:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663193081; x=1694729081; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=axgtZyE1s4LmXVoQfhNr54tm5HXJ0zb4TZOgzZjpqQE=; b=NGMbzlDVdQghefieAXl4VrIRpfi4vwuDpK6YJHTkM9wPKXYTbFLCGrs6 nN+x9qBg7fqL1s40t58RSN7Y+orH7WdEGQqQkLjAWIqpWocPyG1XHFCoG 9eFXCJIBt7xzkR/ggCD+P9OStHMRbG6Qj7S/Y31ld2aD5/HseFs7UYJwg Sr2XsRDTyfW0g0v0MZj1Pk7XP+Ju6Yy6LL8xY2TQ6wQBxlWusdqa4ZRmO h9ofND8TDeGnbji3xR7wCEs33ZDlB8I8JNFX0jSyatRS1+a/swXLP7oPp wuhNjg4ffxCEBGmSqLKWRiJk7s8SycQBltH9eTxyEbrhwRpo39rFOG9M4 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10470"; a="278280794" X-IronPort-AV: E=Sophos;i="5.93,315,1654585200"; d="scan'208";a="278280794" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2022 15:04:40 -0700 X-IronPort-AV: E=Sophos;i="5.93,315,1654585200"; d="scan'208";a="647570928" Received: from mdroper-desk1.fm.intel.com ([10.1.27.134]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2022 15:04:40 -0700 From: Matt Roper To: intel-gfx@lists.freedesktop.org Subject: [PATCH 2/4] drm/i915: Make GEM resume all engines Date: Wed, 14 Sep 2022 15:04:25 -0700 Message-Id: <20220914220427.3091448-3-matthew.d.roper@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220914220427.3091448-1-matthew.d.roper@intel.com> References: <20220914220427.3091448-1-matthew.d.roper@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andi Shyti , dri-devel@lists.freedesktop.org, Tvrtko Ursulin Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Tvrtko Ursulin Walk all GTs from i915_gem_resume when resuming engines. Cc: Andi Shyti Signed-off-by: Tvrtko Ursulin Signed-off-by: Matt Roper Reviewed-by: Andi Shyti --- drivers/gpu/drm/i915/gem/i915_gem_pm.c | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pm.c b/drivers/gpu/drm/i915/gem/i915_gem_pm.c index 3428f735e786..2c80cc8362b6 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_pm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_pm.c @@ -212,7 +212,8 @@ int i915_gem_freeze_late(struct drm_i915_private *i915) void i915_gem_resume(struct drm_i915_private *i915) { - int ret; + struct intel_gt *gt; + int ret, i, j; GEM_TRACE("%s\n", dev_name(i915->drm.dev)); @@ -224,8 +225,25 @@ void i915_gem_resume(struct drm_i915_private *i915) * guarantee that the context image is complete. So let's just reset * it and start again. */ - intel_gt_resume(to_gt(i915)); + for_each_gt(gt, i915, i) + if (intel_gt_resume(gt)) + goto err_wedged; ret = lmem_restore(i915, I915_TTM_BACKUP_ALLOW_GPU); GEM_WARN_ON(ret); + + return; + +err_wedged: + for_each_gt(gt, i915, j) { + if (!intel_gt_is_wedged(gt)) { + dev_err(i915->drm.dev, + "Failed to re-initialize GPU[%u], declaring it wedged!\n", + j); + intel_gt_set_wedged(gt); + } + + if (j == i) + break; + } } From patchwork Wed Sep 14 22:04:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Roper X-Patchwork-Id: 12976581 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 11750ECAAD3 for ; Wed, 14 Sep 2022 22:05:07 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3FD6410EA10; Wed, 14 Sep 2022 22:04:46 +0000 (UTC) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6524010EA10; Wed, 14 Sep 2022 22:04:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663193081; x=1694729081; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1xrco0Aflh85IeWHAaJgIUxZJlQeD1IPivU8FCPK/Vk=; b=Bs1Q+2II73wCG1XqWZczX1tDvI5f0MqdGM7q9yg2jYDnfWW+QjFvVyV2 n4FAHY8yKowMAQYb+jV8nxeXu2xBT1EgGbToQBxKGplL/i+nuyP46kgXs vxuqZEfU7eP1OEjVB2XVITZeIbYXz4j+jXS94+8nOJPkzq6cau/Q1/bYQ V1oLwq07xYJ7q/uD2mMTJPnBcaS5Py/oq0F11jcuJv3X/WIXIIaWlWIn3 ZWozeCDVWdcVlhOiOYNcsNgxVB4W6k+CfI8BpRSyabM7ItFVfRqkpI/4U bDZTdo6zqBETqyJHjBo+ExX6cVpLKhVXTrNWUbb+u/B4qbzVdYYuzyI+/ g==; X-IronPort-AV: E=McAfee;i="6500,9779,10470"; a="278280795" X-IronPort-AV: E=Sophos;i="5.93,315,1654585200"; d="scan'208";a="278280795" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2022 15:04:40 -0700 X-IronPort-AV: E=Sophos;i="5.93,315,1654585200"; d="scan'208";a="647570931" Received: from mdroper-desk1.fm.intel.com ([10.1.27.134]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2022 15:04:40 -0700 From: Matt Roper To: intel-gfx@lists.freedesktop.org Subject: [PATCH 3/4] drm/i915: Make GEM suspend all GTs Date: Wed, 14 Sep 2022 15:04:26 -0700 Message-Id: <20220914220427.3091448-4-matthew.d.roper@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220914220427.3091448-1-matthew.d.roper@intel.com> References: <20220914220427.3091448-1-matthew.d.roper@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: dri-devel@lists.freedesktop.org, Tvrtko Ursulin Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Tvrtko Ursulin Walk all GTs when suspending. Signed-off-by: Tvrtko Ursulin Signed-off-by: Matt Roper Reviewed-by: Andi Shyti --- drivers/gpu/drm/i915/gem/i915_gem_pm.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pm.c b/drivers/gpu/drm/i915/gem/i915_gem_pm.c index 2c80cc8362b6..e5bfb6be9f7a 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_pm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_pm.c @@ -22,6 +22,9 @@ void i915_gem_suspend(struct drm_i915_private *i915) { + struct intel_gt *gt; + unsigned int i; + GEM_TRACE("%s\n", dev_name(i915->drm.dev)); intel_wakeref_auto(&to_gt(i915)->userfault_wakeref, 0); @@ -36,7 +39,8 @@ void i915_gem_suspend(struct drm_i915_private *i915) * state. Fortunately, the kernel_context is disposable and we do * not rely on its state. */ - intel_gt_suspend_prepare(to_gt(i915)); + for_each_gt(gt, i915, i) + intel_gt_suspend_prepare(gt); i915_gem_drain_freed_objects(i915); } @@ -131,7 +135,9 @@ void i915_gem_suspend_late(struct drm_i915_private *i915) &i915->mm.purge_list, NULL }, **phase; + struct intel_gt *gt; unsigned long flags; + unsigned int i; bool flush = false; /* @@ -154,7 +160,8 @@ void i915_gem_suspend_late(struct drm_i915_private *i915) * machine in an unusable condition. */ - intel_gt_suspend_late(to_gt(i915)); + for_each_gt(gt, i915, i) + intel_gt_suspend_late(gt); spin_lock_irqsave(&i915->mm.obj_lock, flags); for (phase = phases; *phase; phase++) { From patchwork Wed Sep 14 22:04:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Roper X-Patchwork-Id: 12976584 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E3F98C6FA82 for ; Wed, 14 Sep 2022 22:05:25 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 71ED710EA19; Wed, 14 Sep 2022 22:05:01 +0000 (UTC) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8656810EA11; Wed, 14 Sep 2022 22:04:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663193081; x=1694729081; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=C6rKbJuKmelDtan36nemGdxkVXK1LzvLsAkuzHs4Kvw=; b=UmIk1EkjgfeZH0Ct8v5/2+l5cOQ6giGg3YD2cUp7s6EoZMaRSSvjTR7a pE1BGVuqIadgpbohuD+AZXEfzmBJTl1rbc7z/4YVchrWMIyctPkHj9yjM 7v8be6e3a9rF7eDxEWNaTH06X8RAGUdq+dVkxA2e5UMkRq290Th4hka/c g/URAvjm3OA+msvacHsWj2f/8tUfe62EAxemkWboq43JyGzrYOHgMVWaP 16uVleoCoEF1iQCX0CaiUu6JyGqA7ZMD5to84jUSlj7eDhDJO0f3Yf6fT ivcSQiIvivqcnog8wvT9Qi5zcZDG87caZOsr2p/p8B8FcGOLiqkv2wlFX g==; X-IronPort-AV: E=McAfee;i="6500,9779,10470"; a="278280797" X-IronPort-AV: E=Sophos;i="5.93,315,1654585200"; d="scan'208";a="278280797" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2022 15:04:41 -0700 X-IronPort-AV: E=Sophos;i="5.93,315,1654585200"; d="scan'208";a="647570934" Received: from mdroper-desk1.fm.intel.com ([10.1.27.134]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2022 15:04:40 -0700 From: Matt Roper To: intel-gfx@lists.freedesktop.org Subject: [PATCH 4/4] drm/i915: Handle all GTs on driver (un)load paths Date: Wed, 14 Sep 2022 15:04:27 -0700 Message-Id: <20220914220427.3091448-5-matthew.d.roper@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220914220427.3091448-1-matthew.d.roper@intel.com> References: <20220914220427.3091448-1-matthew.d.roper@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniele Ceraolo Spurio , dri-devel@lists.freedesktop.org, Tvrtko Ursulin Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Tvrtko Ursulin This, along with the changes already landed in commit 1c66a12ab431 ("drm/i915: Handle each GT on init/release and suspend/resume") makes engines from all GTs actually known to the driver. To accomplish this we need to sprinkle a lot of for_each_gt calls around but is otherwise pretty un-eventuful. Cc: Daniele Ceraolo Spurio Signed-off-by: Tvrtko Ursulin Signed-off-by: Matt Roper --- drivers/gpu/drm/i915/i915_driver.c | 3 +- drivers/gpu/drm/i915/i915_gem.c | 46 ++++++++++++++++++++++-------- 2 files changed, 36 insertions(+), 13 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c index c459eb362c47..9d1fc2477f80 100644 --- a/drivers/gpu/drm/i915/i915_driver.c +++ b/drivers/gpu/drm/i915/i915_driver.c @@ -1661,7 +1661,8 @@ static int intel_runtime_suspend(struct device *kdev) intel_runtime_pm_enable_interrupts(dev_priv); - intel_gt_runtime_resume(to_gt(dev_priv)); + for_each_gt(gt, dev_priv, i) + intel_gt_runtime_resume(gt); enable_rpm_wakeref_asserts(rpm); diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index f18cc6270b2b..0bf71082f21a 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -1128,6 +1128,8 @@ void i915_gem_drain_workqueue(struct drm_i915_private *i915) int i915_gem_init(struct drm_i915_private *dev_priv) { + struct intel_gt *gt; + unsigned int i; int ret; /* We need to fallback to 4K pages if host doesn't support huge gtt. */ @@ -1158,9 +1160,11 @@ int i915_gem_init(struct drm_i915_private *dev_priv) */ intel_init_clock_gating(dev_priv); - ret = intel_gt_init(to_gt(dev_priv)); - if (ret) - goto err_unlock; + for_each_gt(gt, dev_priv, i) { + ret = intel_gt_init(gt); + if (ret) + goto err_unlock; + } return 0; @@ -1173,8 +1177,15 @@ int i915_gem_init(struct drm_i915_private *dev_priv) err_unlock: i915_gem_drain_workqueue(dev_priv); - if (ret != -EIO) - intel_uc_cleanup_firmwares(&to_gt(dev_priv)->uc); + if (ret != -EIO) { + for_each_gt(gt, dev_priv, i) { + intel_gt_driver_remove(gt); + intel_gt_driver_release(gt); + } + + for_each_gt(gt, dev_priv, i) + intel_uc_cleanup_firmwares(>->uc); + } if (ret == -EIO) { /* @@ -1182,10 +1193,12 @@ int i915_gem_init(struct drm_i915_private *dev_priv) * as wedged. But we only want to do this when the GPU is angry, * for all other failure, such as an allocation failure, bail. */ - if (!intel_gt_is_wedged(to_gt(dev_priv))) { - i915_probe_error(dev_priv, - "Failed to initialize GPU, declaring it wedged!\n"); - intel_gt_set_wedged(to_gt(dev_priv)); + for_each_gt(gt, dev_priv, i) { + if (!intel_gt_is_wedged(gt)) { + i915_probe_error(dev_priv, + "Failed to initialize GPU, declaring it wedged!\n"); + intel_gt_set_wedged(gt); + } } /* Minimal basic recovery for KMS */ @@ -1213,10 +1226,14 @@ void i915_gem_driver_unregister(struct drm_i915_private *i915) void i915_gem_driver_remove(struct drm_i915_private *dev_priv) { + struct intel_gt *gt; + unsigned int i; + intel_wakeref_auto_fini(&to_gt(dev_priv)->userfault_wakeref); i915_gem_suspend_late(dev_priv); - intel_gt_driver_remove(to_gt(dev_priv)); + for_each_gt(gt, dev_priv, i) + intel_gt_driver_remove(gt); dev_priv->uabi_engines = RB_ROOT; /* Flush any outstanding unpin_work. */ @@ -1227,9 +1244,14 @@ void i915_gem_driver_remove(struct drm_i915_private *dev_priv) void i915_gem_driver_release(struct drm_i915_private *dev_priv) { - intel_gt_driver_release(to_gt(dev_priv)); + struct intel_gt *gt; + unsigned int i; + + for_each_gt(gt, dev_priv, i) + intel_gt_driver_release(gt); - intel_uc_cleanup_firmwares(&to_gt(dev_priv)->uc); + for_each_gt(gt, dev_priv, i) + intel_uc_cleanup_firmwares(>->uc); i915_gem_drain_freed_objects(dev_priv);