From patchwork Thu Sep 15 23:26:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Roper X-Patchwork-Id: 12977861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 212CCC32771 for ; Thu, 15 Sep 2022 23:27:27 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 5D31B10EC14; Thu, 15 Sep 2022 23:27:06 +0000 (UTC) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9060C10EC12; Thu, 15 Sep 2022 23:27:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663284420; x=1694820420; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=32adMwr+xoi5uUOPdlifc4/erY9Xxj+lmU5EL0uD9Rg=; b=KsOuuellYuDWGJgoH8lMNJMeTPQP64Q574u+GeSC3/Djf1e93h8hhRnv P3XpKq28LZVxZCLFxXb1MKlKJK3Oa5MCsT80EhnaOgHjDMrB8OWT29MqG 5w6zH7gSk9H4jfr+0xk38EHlMHHYSnE/XPebrNNjvbbrCneGlGq+NxiqS OOtC1We8vLmkrxNojyNnq+pOMj7+i1J8W+gP5zACkuUtR95krpcHPm9mL E0MiVvjzRLpHknRKVaeTJzSQsg2Bo1hCRS/kAW6KFSs+YuSb6PMp+/7RX 3r29IAX/QNhJUDHAgi+2btZcCsnURacJuP5sp15CYBaI6zU44Qm5+fqPF w==; X-IronPort-AV: E=McAfee;i="6500,9779,10471"; a="299689108" X-IronPort-AV: E=Sophos;i="5.93,319,1654585200"; d="scan'208";a="299689108" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2022 16:26:59 -0700 X-IronPort-AV: E=Sophos;i="5.93,319,1654585200"; d="scan'208";a="685923299" Received: from mdroper-desk1.fm.intel.com ([10.1.27.134]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2022 16:26:59 -0700 From: Matt Roper To: intel-gfx@lists.freedesktop.org Subject: [PATCH v2 1/4] drm/i915/gt: Cleanup partial engine discovery failures Date: Thu, 15 Sep 2022 16:26:51 -0700 Message-Id: <20220915232654.3283095-2-matthew.d.roper@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220915232654.3283095-1-matthew.d.roper@intel.com> References: <20220915232654.3283095-1-matthew.d.roper@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Janusz Krzysztofik , Chris Wilson , dri-devel@lists.freedesktop.org, Chris Wilson Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Chris Wilson If we abort driver initialisation in the middle of gt/engine discovery, some engines will be fully setup and some not. Those incompletely setup engines only have 'engine->release == NULL' and so will leak any of the common objects allocated. v2: - Drop the destroy_pinned_context() helper for now. It's not really worth it with just a single callsite at the moment. (Janusz) Signed-off-by: Chris Wilson Cc: Janusz Krzysztofik Signed-off-by: Matt Roper Reviewed-by: Janusz Krzysztofik --- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c index 1f7188129cd1..2ddcad497fa3 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c @@ -1274,8 +1274,13 @@ int intel_engines_init(struct intel_gt *gt) return err; err = setup(engine); - if (err) + if (err) { + intel_engine_cleanup_common(engine); return err; + } + + /* The backend should now be responsible for cleanup */ + GEM_BUG_ON(engine->release == NULL); err = engine_init_common(engine); if (err) From patchwork Thu Sep 15 23:26:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Roper X-Patchwork-Id: 12977863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C7F6EC6FA8B for ; Thu, 15 Sep 2022 23:27:41 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C0E9510EC1B; Thu, 15 Sep 2022 23:27:11 +0000 (UTC) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by gabe.freedesktop.org (Postfix) with ESMTPS id C12C510EC14; Thu, 15 Sep 2022 23:27:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663284420; x=1694820420; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gX+8Yo6Hwg9hXCeGfYgrFgNMD+o/F/NbZDxyewWeGCw=; b=fdzhR7ZZaH3Lzwo2W59f3QVPapVi5rwiLsmtV1masiNpFkHY41FYyTT2 TFh5Ygalu1dknSUpyKdphydS9J5G/51D1mifxg/PVr852k2uk/32pXMzJ g6XNgpJAH8Z96dblgRvVr5VYOJetsxrJlftPKxyaupmMcVYlkLOy9P7Ta FSr7bltnC9IaM8aweqF1XyjU583l3rcRUlOn09ddky0FYd9tRFFtOXwjY cnBb6lJVUQJKpUYgokBpepZgyVkyMO0oE0fIwB9Z2YaMZXpICQTUkQCqI ziRj1b6xEqVNMk3htQNNkq/k1FIhiM6POz6pzozGcpavIc8Jhcy35RO4E g==; X-IronPort-AV: E=McAfee;i="6500,9779,10471"; a="299689109" X-IronPort-AV: E=Sophos;i="5.93,319,1654585200"; d="scan'208";a="299689109" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2022 16:26:59 -0700 X-IronPort-AV: E=Sophos;i="5.93,319,1654585200"; d="scan'208";a="685923302" Received: from mdroper-desk1.fm.intel.com ([10.1.27.134]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2022 16:26:59 -0700 From: Matt Roper To: intel-gfx@lists.freedesktop.org Subject: [PATCH v2 2/4] drm/i915: Make GEM resume all engines Date: Thu, 15 Sep 2022 16:26:52 -0700 Message-Id: <20220915232654.3283095-3-matthew.d.roper@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220915232654.3283095-1-matthew.d.roper@intel.com> References: <20220915232654.3283095-1-matthew.d.roper@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andi Shyti , Andi Shyti , dri-devel@lists.freedesktop.org, Tvrtko Ursulin Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Tvrtko Ursulin Walk all GTs from i915_gem_resume when resuming engines. Cc: Andi Shyti Signed-off-by: Tvrtko Ursulin Signed-off-by: Matt Roper Reviewed-by: Andi Shyti --- drivers/gpu/drm/i915/gem/i915_gem_pm.c | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pm.c b/drivers/gpu/drm/i915/gem/i915_gem_pm.c index 3428f735e786..2c80cc8362b6 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_pm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_pm.c @@ -212,7 +212,8 @@ int i915_gem_freeze_late(struct drm_i915_private *i915) void i915_gem_resume(struct drm_i915_private *i915) { - int ret; + struct intel_gt *gt; + int ret, i, j; GEM_TRACE("%s\n", dev_name(i915->drm.dev)); @@ -224,8 +225,25 @@ void i915_gem_resume(struct drm_i915_private *i915) * guarantee that the context image is complete. So let's just reset * it and start again. */ - intel_gt_resume(to_gt(i915)); + for_each_gt(gt, i915, i) + if (intel_gt_resume(gt)) + goto err_wedged; ret = lmem_restore(i915, I915_TTM_BACKUP_ALLOW_GPU); GEM_WARN_ON(ret); + + return; + +err_wedged: + for_each_gt(gt, i915, j) { + if (!intel_gt_is_wedged(gt)) { + dev_err(i915->drm.dev, + "Failed to re-initialize GPU[%u], declaring it wedged!\n", + j); + intel_gt_set_wedged(gt); + } + + if (j == i) + break; + } } From patchwork Thu Sep 15 23:26:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Roper X-Patchwork-Id: 12977864 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ACEDEC32771 for ; Thu, 15 Sep 2022 23:27:46 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id ACB4610EC19; Thu, 15 Sep 2022 23:27:09 +0000 (UTC) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by gabe.freedesktop.org (Postfix) with ESMTPS id F189310EC16; Thu, 15 Sep 2022 23:27:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663284421; x=1694820421; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PYtD+E/vSPis7wOIecLUjsBiB0SZeZOJgiCiWGSMUo4=; b=DK3wpy7eDRPHG0jjiv2kW2wZigxoOVvADEUisoTBCQkIIULf3gNcjgcm /OJ2AwkWNRm6g7NAuHfFdI4GGf2hsoXrNlCxKwMuDlYE5rVGe/riXE2mt xC2kC1c1aSTtXlIWNu5hDOxKHg497BMEvr82Z6pjOuekzQoPyky2sui32 hvNDbsfRCnNFRz4LOjCwcFUgRaxZXsIrUM/XWyIb+pJHlz7adXQp+/TSd AsfC3qKV4g3ET9ux7WIVEKYj8PjKl2itOelnMN/7gSAve0g/XCib6xzrq jpQc3Z3DMyRUEhvdnyRv+ewV+ud/EV1nPr64YGYOixBAdqnC+Tb0BhVP6 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10471"; a="299689110" X-IronPort-AV: E=Sophos;i="5.93,319,1654585200"; d="scan'208";a="299689110" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2022 16:27:00 -0700 X-IronPort-AV: E=Sophos;i="5.93,319,1654585200"; d="scan'208";a="685923305" Received: from mdroper-desk1.fm.intel.com ([10.1.27.134]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2022 16:26:59 -0700 From: Matt Roper To: intel-gfx@lists.freedesktop.org Subject: [PATCH v2 3/4] drm/i915: Make GEM suspend all GTs Date: Thu, 15 Sep 2022 16:26:53 -0700 Message-Id: <20220915232654.3283095-4-matthew.d.roper@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220915232654.3283095-1-matthew.d.roper@intel.com> References: <20220915232654.3283095-1-matthew.d.roper@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andi Shyti , dri-devel@lists.freedesktop.org, Tvrtko Ursulin Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Tvrtko Ursulin Walk all GTs when suspending. Signed-off-by: Tvrtko Ursulin Signed-off-by: Matt Roper Reviewed-by: Andi Shyti --- drivers/gpu/drm/i915/gem/i915_gem_pm.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pm.c b/drivers/gpu/drm/i915/gem/i915_gem_pm.c index 2c80cc8362b6..e5bfb6be9f7a 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_pm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_pm.c @@ -22,6 +22,9 @@ void i915_gem_suspend(struct drm_i915_private *i915) { + struct intel_gt *gt; + unsigned int i; + GEM_TRACE("%s\n", dev_name(i915->drm.dev)); intel_wakeref_auto(&to_gt(i915)->userfault_wakeref, 0); @@ -36,7 +39,8 @@ void i915_gem_suspend(struct drm_i915_private *i915) * state. Fortunately, the kernel_context is disposable and we do * not rely on its state. */ - intel_gt_suspend_prepare(to_gt(i915)); + for_each_gt(gt, i915, i) + intel_gt_suspend_prepare(gt); i915_gem_drain_freed_objects(i915); } @@ -131,7 +135,9 @@ void i915_gem_suspend_late(struct drm_i915_private *i915) &i915->mm.purge_list, NULL }, **phase; + struct intel_gt *gt; unsigned long flags; + unsigned int i; bool flush = false; /* @@ -154,7 +160,8 @@ void i915_gem_suspend_late(struct drm_i915_private *i915) * machine in an unusable condition. */ - intel_gt_suspend_late(to_gt(i915)); + for_each_gt(gt, i915, i) + intel_gt_suspend_late(gt); spin_lock_irqsave(&i915->mm.obj_lock, flags); for (phase = phases; *phase; phase++) { From patchwork Thu Sep 15 23:26:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Roper X-Patchwork-Id: 12977865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EB802ECAAA1 for ; Thu, 15 Sep 2022 23:27:52 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3DA2B10EC1A; Thu, 15 Sep 2022 23:27:19 +0000 (UTC) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by gabe.freedesktop.org (Postfix) with ESMTPS id 2F13410EC17; Thu, 15 Sep 2022 23:27:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663284421; x=1694820421; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tFlfD40hLtY3YkEKxAQn2mbJw+x4pC16bMSk3bOwU8s=; b=ahmnJ1A/xXHpWkX4zPKL04FvkENmExnG4KDc9GZf2ZbrmywxgB+H+sa0 m2xVQ5iqaQGMODucUJdKyLwNPqR3K7ryJUWgrNuK5T/dyJ87/QAchYO0+ pL4uf65fYfQPCIoaCfqF/03rjuRYX9x/0V/cHTHS2020qU0cGxAnS1pdk 9ODgdrdHQmaRIfkHZv6obQG2WswC8Da6QFFHi3WjHkMb6bxam/wboDi3u sVx7P48g6XT75Q1EQW8As3K9k0SPJWDS+G7wxBleTROItGguteJynEkBe Ljye9wMFPVOPKAr/VzdMO914wHDBKRQAf8m7kjyVoFOxfdplIzMtRO5Y6 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10471"; a="299689111" X-IronPort-AV: E=Sophos;i="5.93,319,1654585200"; d="scan'208";a="299689111" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2022 16:27:00 -0700 X-IronPort-AV: E=Sophos;i="5.93,319,1654585200"; d="scan'208";a="685923308" Received: from mdroper-desk1.fm.intel.com ([10.1.27.134]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2022 16:26:59 -0700 From: Matt Roper To: intel-gfx@lists.freedesktop.org Subject: [PATCH v2 4/4] drm/i915: Handle all GTs on driver (un)load paths Date: Thu, 15 Sep 2022 16:26:54 -0700 Message-Id: <20220915232654.3283095-5-matthew.d.roper@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220915232654.3283095-1-matthew.d.roper@intel.com> References: <20220915232654.3283095-1-matthew.d.roper@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniele Ceraolo Spurio , dri-devel@lists.freedesktop.org, Tvrtko Ursulin Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Tvrtko Ursulin This, along with the changes already landed in commit 1c66a12ab431 ("drm/i915: Handle each GT on init/release and suspend/resume") makes engines from all GTs actually known to the driver. To accomplish this we need to sprinkle a lot of for_each_gt calls around but is otherwise pretty un-eventuful. v2: - Consolidate adjacent GT loops in a couple places. (Daniele) Cc: Daniele Ceraolo Spurio Signed-off-by: Tvrtko Ursulin Signed-off-by: Matt Roper Reviewed-by: Andi Shyti --- drivers/gpu/drm/i915/i915_driver.c | 3 ++- drivers/gpu/drm/i915/i915_gem.c | 43 +++++++++++++++++++++--------- 2 files changed, 33 insertions(+), 13 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c index c459eb362c47..9d1fc2477f80 100644 --- a/drivers/gpu/drm/i915/i915_driver.c +++ b/drivers/gpu/drm/i915/i915_driver.c @@ -1661,7 +1661,8 @@ static int intel_runtime_suspend(struct device *kdev) intel_runtime_pm_enable_interrupts(dev_priv); - intel_gt_runtime_resume(to_gt(dev_priv)); + for_each_gt(gt, dev_priv, i) + intel_gt_runtime_resume(gt); enable_rpm_wakeref_asserts(rpm); diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index f18cc6270b2b..e936e67f3411 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -1128,6 +1128,8 @@ void i915_gem_drain_workqueue(struct drm_i915_private *i915) int i915_gem_init(struct drm_i915_private *dev_priv) { + struct intel_gt *gt; + unsigned int i; int ret; /* We need to fallback to 4K pages if host doesn't support huge gtt. */ @@ -1158,9 +1160,11 @@ int i915_gem_init(struct drm_i915_private *dev_priv) */ intel_init_clock_gating(dev_priv); - ret = intel_gt_init(to_gt(dev_priv)); - if (ret) - goto err_unlock; + for_each_gt(gt, dev_priv, i) { + ret = intel_gt_init(gt); + if (ret) + goto err_unlock; + } return 0; @@ -1173,8 +1177,13 @@ int i915_gem_init(struct drm_i915_private *dev_priv) err_unlock: i915_gem_drain_workqueue(dev_priv); - if (ret != -EIO) - intel_uc_cleanup_firmwares(&to_gt(dev_priv)->uc); + if (ret != -EIO) { + for_each_gt(gt, dev_priv, i) { + intel_gt_driver_remove(gt); + intel_gt_driver_release(gt); + intel_uc_cleanup_firmwares(>->uc); + } + } if (ret == -EIO) { /* @@ -1182,10 +1191,12 @@ int i915_gem_init(struct drm_i915_private *dev_priv) * as wedged. But we only want to do this when the GPU is angry, * for all other failure, such as an allocation failure, bail. */ - if (!intel_gt_is_wedged(to_gt(dev_priv))) { - i915_probe_error(dev_priv, - "Failed to initialize GPU, declaring it wedged!\n"); - intel_gt_set_wedged(to_gt(dev_priv)); + for_each_gt(gt, dev_priv, i) { + if (!intel_gt_is_wedged(gt)) { + i915_probe_error(dev_priv, + "Failed to initialize GPU, declaring it wedged!\n"); + intel_gt_set_wedged(gt); + } } /* Minimal basic recovery for KMS */ @@ -1213,10 +1224,14 @@ void i915_gem_driver_unregister(struct drm_i915_private *i915) void i915_gem_driver_remove(struct drm_i915_private *dev_priv) { + struct intel_gt *gt; + unsigned int i; + intel_wakeref_auto_fini(&to_gt(dev_priv)->userfault_wakeref); i915_gem_suspend_late(dev_priv); - intel_gt_driver_remove(to_gt(dev_priv)); + for_each_gt(gt, dev_priv, i) + intel_gt_driver_remove(gt); dev_priv->uabi_engines = RB_ROOT; /* Flush any outstanding unpin_work. */ @@ -1227,9 +1242,13 @@ void i915_gem_driver_remove(struct drm_i915_private *dev_priv) void i915_gem_driver_release(struct drm_i915_private *dev_priv) { - intel_gt_driver_release(to_gt(dev_priv)); + struct intel_gt *gt; + unsigned int i; - intel_uc_cleanup_firmwares(&to_gt(dev_priv)->uc); + for_each_gt(gt, dev_priv, i) { + intel_gt_driver_release(gt); + intel_uc_cleanup_firmwares(>->uc); + } i915_gem_drain_freed_objects(dev_priv);