From patchwork Wed Jun 29 15:25:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mauro Carvalho Chehab X-Patchwork-Id: 12900307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3E566C43334 for ; Wed, 29 Jun 2022 15:25:38 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 455CA10E416; Wed, 29 Jun 2022 15:25:33 +0000 (UTC) Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id C689510E3CE; Wed, 29 Jun 2022 15:25:31 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 28A7260B27; Wed, 29 Jun 2022 15:25:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 82781C341C8; Wed, 29 Jun 2022 15:25:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656516330; bh=pAnLsLhlU8wTTndeJDMYPSkKHinalsEo7X4amJuUpQM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=D8I9uMenNpLPkAOu4OU252AHAeLuS1ULOqBJj/jRtjYKlO2k3vsq15dD5295i4zLl sPMEt65sFl2uGdqm1TnQQIFC2H588xJBDlhDOFE8wRGaeq0/aXycMKqbTiyPUluyDx 5FyXeCdLf7ILdzHAIDXyAUaNk9plMiv5NReIo6ol13jHek2Dg/TYSlLrcax91M6pEM ws5vKSoQvOHqukpn4KHEEW+VUs01VZsKgpN04DlgNYGcvR9yT1sIj1wf2q2k2xCmVe ab/uJXDhqd+NY0ENfp8NP71Tct7dS7BS/OtFze54j7CaoaRgRPQwfQFLaUmkQWiVKt 0LwheY57ZH1nQ== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1o6ZZH-005hFI-Nx; Wed, 29 Jun 2022 16:25:27 +0100 From: Mauro Carvalho Chehab To: Subject: [PATCH v2 1/3] drm/i915/gt: Ignore TLB invalidations on idle engines Date: Wed, 29 Jun 2022 16:25:24 +0100 Message-Id: <5882cc9bf8d069151101b2469347ebe22bcbf4b8.1656516220.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fei Yang , David Airlie , dri-devel@lists.freedesktop.org, Daniele Ceraolo Spurio , Jason Ekstrand , Matthew Brost , Chris Wilson , Matthew Auld , Andi Shyti , =?utf-8?q?Thomas_Hellstr=C3=B6m?= , Lucas De Marchi , intel-gfx@lists.freedesktop.org, Oak Zeng , Rodrigo Vivi , Mauro Carvalho Chehab , Tvrtko Ursulin , linux-kernel@vger.kernel.org, stable@vger.kernel.org, John Harrison Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Chris Wilson As an extension of the current skip TLB invalidations, check if the device is powered down prior to any engine activity, as, on such cases, all the TLBs were already invalidated, so an explicit TLB invalidation is not needed. This becomes more significant with GuC, as it can only do so when the connection to the GuC is awake. Cc: stable@vger.kernel.org Signed-off-by: Chris Wilson Cc: Fei Yang Reviewed-by: Andi Shyti Acked-by: Thomas Hellström Signed-off-by: Mauro Carvalho Chehab --- See [PATCH v2 0/3] at: https://lore.kernel.org/all/cover.1656516220.git.mchehab@kernel.org/ drivers/gpu/drm/i915/gem/i915_gem_pages.c | 10 +++++---- drivers/gpu/drm/i915/gt/intel_gt.c | 26 +++++++++++++++++------ drivers/gpu/drm/i915/gt/intel_gt_pm.h | 3 +++ 3 files changed, 28 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pages.c b/drivers/gpu/drm/i915/gem/i915_gem_pages.c index 97c820eee115..6835279943df 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_pages.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_pages.c @@ -6,14 +6,15 @@ #include +#include "gt/intel_gt.h" +#include "gt/intel_gt_pm.h" + #include "i915_drv.h" #include "i915_gem_object.h" #include "i915_scatterlist.h" #include "i915_gem_lmem.h" #include "i915_gem_mman.h" -#include "gt/intel_gt.h" - void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj, struct sg_table *pages, unsigned int sg_page_sizes) @@ -217,10 +218,11 @@ __i915_gem_object_unset_pages(struct drm_i915_gem_object *obj) if (test_and_clear_bit(I915_BO_WAS_BOUND_BIT, &obj->flags)) { struct drm_i915_private *i915 = to_i915(obj->base.dev); + struct intel_gt *gt = to_gt(i915); intel_wakeref_t wakeref; - with_intel_runtime_pm_if_active(&i915->runtime_pm, wakeref) - intel_gt_invalidate_tlbs(to_gt(i915)); + with_intel_gt_pm_if_awake(gt, wakeref) + intel_gt_invalidate_tlbs(gt); } return pages; diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c index 8da3314bb6bf..30c60cd960e8 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -12,6 +12,7 @@ #include "i915_drv.h" #include "intel_context.h" +#include "intel_engine_pm.h" #include "intel_engine_regs.h" #include "intel_ggtt_gmch.h" #include "intel_gt.h" @@ -924,6 +925,7 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) struct drm_i915_private *i915 = gt->i915; struct intel_uncore *uncore = gt->uncore; struct intel_engine_cs *engine; + intel_engine_mask_t awake, tmp; enum intel_engine_id id; const i915_reg_t *regs; unsigned int num = 0; @@ -947,12 +949,27 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) GEM_TRACE("\n"); - assert_rpm_wakelock_held(&i915->runtime_pm); - mutex_lock(>->tlb_invalidate_lock); intel_uncore_forcewake_get(uncore, FORCEWAKE_ALL); + awake = 0; for_each_engine(engine, gt, id) { + struct reg_and_bit rb; + + if (!intel_engine_pm_is_awake(engine)) + continue; + + rb = get_reg_and_bit(engine, regs == gen8_regs, regs, num); + if (!i915_mmio_reg_offset(rb.reg)) + continue; + + intel_uncore_write_fw(uncore, rb.reg, rb.bit); + awake |= engine->mask; + } + + for_each_engine_masked(engine, gt, awake, tmp) { + struct reg_and_bit rb; + /* * HW architecture suggest typical invalidation time at 40us, * with pessimistic cases up to 100us and a recommendation to @@ -960,13 +977,8 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) */ const unsigned int timeout_us = 100; const unsigned int timeout_ms = 4; - struct reg_and_bit rb; rb = get_reg_and_bit(engine, regs == gen8_regs, regs, num); - if (!i915_mmio_reg_offset(rb.reg)) - continue; - - intel_uncore_write_fw(uncore, rb.reg, rb.bit); if (__intel_wait_for_register_fw(uncore, rb.reg, rb.bit, 0, timeout_us, timeout_ms, diff --git a/drivers/gpu/drm/i915/gt/intel_gt_pm.h b/drivers/gpu/drm/i915/gt/intel_gt_pm.h index bc898df7a48c..a334787a4939 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_pm.h +++ b/drivers/gpu/drm/i915/gt/intel_gt_pm.h @@ -55,6 +55,9 @@ static inline void intel_gt_pm_might_put(struct intel_gt *gt) for (tmp = 1, intel_gt_pm_get(gt); tmp; \ intel_gt_pm_put(gt), tmp = 0) +#define with_intel_gt_pm_if_awake(gt, wf) \ + for (wf = intel_gt_pm_get_if_awake(gt); wf; intel_gt_pm_put_async(gt), wf = 0) + static inline int intel_gt_pm_wait_for_idle(struct intel_gt *gt) { return intel_wakeref_wait_for_idle(>->wakeref); From patchwork Wed Jun 29 15:25:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mauro Carvalho Chehab X-Patchwork-Id: 12900306 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9360AC43334 for ; Wed, 29 Jun 2022 15:25:33 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C9F6610E389; Wed, 29 Jun 2022 15:25:32 +0000 (UTC) Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id A491010E3AB; Wed, 29 Jun 2022 15:25:31 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2742E60F54; Wed, 29 Jun 2022 15:25:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 859FDC385A5; Wed, 29 Jun 2022 15:25:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656516330; bh=vA3TDfCYgbDYTy0RpqmuVlzT3hHqXf/ueg6gMP+LLeA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OkWAWn/AWasmoB43WZRoped54kCAWZVtaOLrk99sjBLJ9IREfbBoDMcvhEGPg3QLe SzFj0S99e/mRJHJ2eKCcnY7uTa1D8zuMYeHZYlGRUsERU9m8heYM5PGe83DBsCWyOX DWtE8RVIxotFEHt+LTvaauEa6nToESZpoiDr6JDtvlWffsbG13vL5bWw6CsQWt/k5v bYKNpXn4WmfzCqaXQt4J9a6MqHWDYwIjoV+3MOChOfosE0wnf0A76FsueOuK45nyyx 5nwaOJho8Q4k1v5CbExOVnAZiAaWxx+OzQjS91J1n8sMy8meiPNAm33eHElznhncrg TO/lw41m5bA0Q== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1o6ZZH-005hFL-Ol; Wed, 29 Jun 2022 16:25:27 +0100 From: Mauro Carvalho Chehab To: Subject: [PATCH v2 2/3] drm/i915/gt: Serialize GRDOM access between multiple engine resets Date: Wed, 29 Jun 2022 16:25:25 +0100 Message-Id: <147cf88cd20e8ce75c7849ed5e4e913563657cdf.1656516220.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: David Airlie , dri-devel@lists.freedesktop.org, Chris Wilson , Matthew Brost , Mika Kuoppala , Chris Wilson , =?utf-8?q?Thomas_Hellstr=C3=B6m?= , Andi Shyti , intel-gfx@lists.freedesktop.org, Lucas De Marchi , Rodrigo Vivi , Mauro Carvalho Chehab , Tvrtko Ursulin , linux-kernel@vger.kernel.org, stable@vger.kernel.org, Bruce Chang , Tejas Upadhyay , Umesh Nerlige Ramappa , John Harrison Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Chris Wilson Don't allow two engines to be reset in parallel, as they would both try to select a reset bit (and send requests to common registers) and wait on that register, at the same time. Serialize control of the reset requests/acks using the uncore->lock, which will also ensure that no other GT state changes at the same time as the actual reset. Cc: stable@vger.kernel.org Reported-by: Mika Kuoppala Signed-off-by: Chris Wilson Cc: Mika Kuoppala Reviewed-by: Andi Shyti Acked-by: Thomas Hellström Signed-off-by: Mauro Carvalho Chehab --- See [PATCH v2 0/3] at: https://lore.kernel.org/all/cover.1656516220.git.mchehab@kernel.org/ drivers/gpu/drm/i915/gt/intel_reset.c | 37 ++++++++++++++++++++------- 1 file changed, 28 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_reset.c b/drivers/gpu/drm/i915/gt/intel_reset.c index a5338c3fde7a..c68d36fb5bbd 100644 --- a/drivers/gpu/drm/i915/gt/intel_reset.c +++ b/drivers/gpu/drm/i915/gt/intel_reset.c @@ -300,9 +300,9 @@ static int gen6_hw_domain_reset(struct intel_gt *gt, u32 hw_domain_mask) return err; } -static int gen6_reset_engines(struct intel_gt *gt, - intel_engine_mask_t engine_mask, - unsigned int retry) +static int __gen6_reset_engines(struct intel_gt *gt, + intel_engine_mask_t engine_mask, + unsigned int retry) { struct intel_engine_cs *engine; u32 hw_mask; @@ -321,6 +321,20 @@ static int gen6_reset_engines(struct intel_gt *gt, return gen6_hw_domain_reset(gt, hw_mask); } +static int gen6_reset_engines(struct intel_gt *gt, + intel_engine_mask_t engine_mask, + unsigned int retry) +{ + unsigned long flags; + int ret; + + spin_lock_irqsave(>->uncore->lock, flags); + ret = __gen6_reset_engines(gt, engine_mask, retry); + spin_unlock_irqrestore(>->uncore->lock, flags); + + return ret; +} + static struct intel_engine_cs *find_sfc_paired_vecs_engine(struct intel_engine_cs *engine) { int vecs_id; @@ -487,9 +501,9 @@ static void gen11_unlock_sfc(struct intel_engine_cs *engine) rmw_clear_fw(uncore, sfc_lock.lock_reg, sfc_lock.lock_bit); } -static int gen11_reset_engines(struct intel_gt *gt, - intel_engine_mask_t engine_mask, - unsigned int retry) +static int __gen11_reset_engines(struct intel_gt *gt, + intel_engine_mask_t engine_mask, + unsigned int retry) { struct intel_engine_cs *engine; intel_engine_mask_t tmp; @@ -583,8 +597,11 @@ static int gen8_reset_engines(struct intel_gt *gt, struct intel_engine_cs *engine; const bool reset_non_ready = retry >= 1; intel_engine_mask_t tmp; + unsigned long flags; int ret; + spin_lock_irqsave(>->uncore->lock, flags); + for_each_engine_masked(engine, gt, engine_mask, tmp) { ret = gen8_engine_reset_prepare(engine); if (ret && !reset_non_ready) @@ -612,17 +629,19 @@ static int gen8_reset_engines(struct intel_gt *gt, * This is best effort, so ignore any error from the initial reset. */ if (IS_DG2(gt->i915) && engine_mask == ALL_ENGINES) - gen11_reset_engines(gt, gt->info.engine_mask, 0); + __gen11_reset_engines(gt, gt->info.engine_mask, 0); if (GRAPHICS_VER(gt->i915) >= 11) - ret = gen11_reset_engines(gt, engine_mask, retry); + ret = __gen11_reset_engines(gt, engine_mask, retry); else - ret = gen6_reset_engines(gt, engine_mask, retry); + ret = __gen6_reset_engines(gt, engine_mask, retry); skip_reset: for_each_engine_masked(engine, gt, engine_mask, tmp) gen8_engine_reset_cancel(engine); + spin_unlock_irqrestore(>->uncore->lock, flags); + return ret; } From patchwork Wed Jun 29 15:25:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mauro Carvalho Chehab X-Patchwork-Id: 12900309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 769B1C433EF for ; Wed, 29 Jun 2022 15:25:43 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8875A10E342; Wed, 29 Jun 2022 15:25:34 +0000 (UTC) Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0CD0910E3AB; Wed, 29 Jun 2022 15:25:33 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id BC5A1B824BC; Wed, 29 Jun 2022 15:25:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7F761C34114; Wed, 29 Jun 2022 15:25:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656516330; bh=WfCqVL/HkSy9hkFb7D4vO+nAC41+jUaURlFZDbgG3v4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IRx39RV40TrGV8qKo3powlOOuJ+tBL7T9QXd+/oSz3zRcIFKlEshJoKdCH4iOYjv1 0iTPGzr+8Absf0vnUJCvFw8jTiCy3t2Z6490sH6TOO8Jw48hv6p+9nJOpailhc4EjM YixY6a7BVJO5imxUkRYh5MHuv1mictKpOYg8OFMxNf5SpeX/smRY0X7Vh5lF18pM8h OUASz1u7dWfJ2OvuRKS80bNkX/+HtGEodiw5gk+cxXCXG4+4FimqlraqpPaoHtcPSz /HCuNIZ5KYFYCKo7wCDK4Qi7YgZaKujpcK3Jnt0mxRsinZi5h2WBxq8ZldFQ2swy9P BbkmMNBJS+zag== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1o6ZZH-005hFO-PX; Wed, 29 Jun 2022 16:25:27 +0100 From: Mauro Carvalho Chehab To: Subject: [PATCH v2 3/3] drm/i915/gt: Serialize TLB invalidates with GT resets Date: Wed, 29 Jun 2022 16:25:26 +0100 Message-Id: X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Tvrtko Ursulin , Andi Shyti , Andi Shyti , =?utf-8?q?Thomas_Hellstr=C3=B6m?= , David Airlie , dri-devel@lists.freedesktop.org, Lucas De Marchi , linux-kernel@vger.kernel.org, Chris Wilson , Daniele Ceraolo Spurio , Rodrigo Vivi , Dave Airlie , stable@vger.kernel.org, Mauro Carvalho Chehab , intel-gfx@lists.freedesktop.org, John Harrison Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Chris Wilson Avoid trying to invalidate the TLB in the middle of performing an engine reset, as this may result in the reset timing out. Currently, the TLB invalidate is only serialised by its own mutex, forgoing the uncore lock, but we can take the uncore->lock as well to serialise the mmio access, thereby serialising with the GDRST. Tested on a NUC5i7RYB, BIOS RYBDWi35.86A.0380.2019.0517.1530 with i915 selftest/hangcheck. Cc: stable@vger.kernel.org Fixes: 7938d61591d3 ("drm/i915: Flush TLBs before releasing backing store") Reported-by: Mauro Carvalho Chehab Tested-by: Mauro Carvalho Chehab Reviewed-by: Mauro Carvalho Chehab Signed-off-by: Chris Wilson Cc: Tvrtko Ursulin Acked-by: Thomas Hellström Reviewed-by: Andi Shyti Signed-off-by: Mauro Carvalho Chehab --- See [PATCH v2 0/3] at: https://lore.kernel.org/all/cover.1656516220.git.mchehab@kernel.org/ drivers/gpu/drm/i915/gt/intel_gt.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c index 30c60cd960e8..7e57a90b4095 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -952,6 +952,8 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) mutex_lock(>->tlb_invalidate_lock); intel_uncore_forcewake_get(uncore, FORCEWAKE_ALL); + spin_lock_irq(&uncore->lock); /* seralise invalidate with GT reset */ + awake = 0; for_each_engine(engine, gt, id) { struct reg_and_bit rb; @@ -967,6 +969,8 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) awake |= engine->mask; } + spin_unlock_irq(&uncore->lock); + for_each_engine_masked(engine, gt, awake, tmp) { struct reg_and_bit rb;