From patchwork Fri Nov 30 11:31:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 10706161 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 19A7B14D6 for ; Fri, 30 Nov 2018 11:32:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 085202FF87 for ; Fri, 30 Nov 2018 11:32:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F08882FF88; Fri, 30 Nov 2018 11:32:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 305292FF8E for ; Fri, 30 Nov 2018 11:32:16 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B1D356E5F8; Fri, 30 Nov 2018 11:32:14 +0000 (UTC) X-Original-To: Intel-gfx@lists.freedesktop.org Delivered-To: Intel-gfx@lists.freedesktop.org Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by gabe.freedesktop.org (Postfix) with ESMTPS id CA18C6E5F7; Fri, 30 Nov 2018 11:32:12 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 Nov 2018 03:32:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,298,1539673200"; d="scan'208";a="96250391" Received: from bmolloy-mobl.ger.corp.intel.com (HELO localhost.localdomain) ([10.252.10.213]) by orsmga006.jf.intel.com with ESMTP; 30 Nov 2018 03:32:10 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Date: Fri, 30 Nov 2018 11:31:55 +0000 Message-Id: <20181130113201.13007-3-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181130113201.13007-1-tvrtko.ursulin@linux.intel.com> References: <20181130113201.13007-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 2/8] drm/i915: Introduce per-engine workarounds X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-gfx@lists.freedesktop.org, Rodrigo Vivi Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP From: Tvrtko Ursulin We stopped re-applying the GT workarounds after engine reset since commit 59b449d5c82a ("drm/i915: Split out functions for different kinds of workarounds"). Issue with this is that some of the GT workarounds live in the MMIO space which gets lost during engine resets. So far the registers in 0x2xxx and 0xbxxx address range have been identified to be affected. This losing of applied workarounds has obvious negative effects and can even lead to hard system hangs (see the linked Bugzilla). Rather than just restoring this re-application, because we have also observed that it is not safe to just re-write all GT workarounds after engine resets (GPU might be live and weird hardware states can happen), we introduce a new class of per-engine workarounds and move only the affected GT workarounds over. Using the framework introduced in the previous patch, we therefore after engine reset, re-apply only the workarounds living in the affected MMIO address ranges. Signed-off-by: Tvrtko Ursulin Bugzilla: https://bugzilla.freedesktop.org/show_bug.cgi?id=107945 Fixes: 59b449d5c82a ("drm/i915: Split out functions for different kinds of workarounds") Cc: Mika Kuoppala Cc: Ville Syrjälä Cc: Chris Wilson Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Rodrigo Vivi Cc: intel-gfx@lists.freedesktop.org --- drivers/gpu/drm/i915/intel_engine_cs.c | 2 + drivers/gpu/drm/i915/intel_lrc.c | 4 + drivers/gpu/drm/i915/intel_ringbuffer.h | 2 + drivers/gpu/drm/i915/intel_workarounds.c | 249 +++++++++++++---------- drivers/gpu/drm/i915/intel_workarounds.h | 3 + 5 files changed, 148 insertions(+), 112 deletions(-) diff --git a/drivers/gpu/drm/i915/intel_engine_cs.c b/drivers/gpu/drm/i915/intel_engine_cs.c index 759c0fd58f8c..ef5d202e9d45 100644 --- a/drivers/gpu/drm/i915/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/intel_engine_cs.c @@ -723,6 +723,8 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine) __intel_context_unpin(i915->kernel_context, engine); i915_timeline_fini(&engine->timeline); + + intel_wa_list_free(&engine->wa_list); } u64 intel_engine_get_active_head(const struct intel_engine_cs *engine) diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c index 11f4e6148557..dfafc3f710d6 100644 --- a/drivers/gpu/drm/i915/intel_lrc.c +++ b/drivers/gpu/drm/i915/intel_lrc.c @@ -1617,6 +1617,8 @@ static bool unexpected_starting_state(struct intel_engine_cs *engine) static int gen8_init_common_ring(struct intel_engine_cs *engine) { + intel_engine_workarounds_apply(engine); + intel_mocs_init_engine(engine); intel_engine_reset_breadcrumbs(engine); @@ -2314,6 +2316,8 @@ int logical_render_ring_init(struct intel_engine_cs *engine) ret); } + intel_engine_workarounds_init(engine); + return 0; err_cleanup_common: diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h index 8a2270b209b0..c5ff3d31cab7 100644 --- a/drivers/gpu/drm/i915/intel_ringbuffer.h +++ b/drivers/gpu/drm/i915/intel_ringbuffer.h @@ -15,6 +15,7 @@ #include "i915_selftest.h" #include "i915_timeline.h" #include "intel_gpu_commands.h" +#include "intel_workarounds.h" struct drm_printer; struct i915_sched_attr; @@ -451,6 +452,7 @@ struct intel_engine_cs { struct intel_hw_status_page status_page; struct i915_ctx_workarounds wa_ctx; + struct i915_wa_list wa_list; struct i915_vma *scratch; u32 irq_keep_mask; /* always keep these interrupts */ diff --git a/drivers/gpu/drm/i915/intel_workarounds.c b/drivers/gpu/drm/i915/intel_workarounds.c index ff20ebf9e040..be63a2af3481 100644 --- a/drivers/gpu/drm/i915/intel_workarounds.c +++ b/drivers/gpu/drm/i915/intel_workarounds.c @@ -653,17 +653,6 @@ static void gen9_gt_workarounds_init(struct drm_i915_private *dev_priv) { struct i915_wa_list *wal = &dev_priv->gt_wa_list; - /* WaContextSwitchWithConcurrentTLBInvalidate:skl,bxt,kbl,glk,cfl */ - wa_masked_en(wal, - GEN9_CSFE_CHICKEN1_RCS, - GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE); - - - /* WaEnableLbsSlaRetryTimerDecrement:skl,bxt,kbl,glk,cfl */ - wa_write_or(wal, - BDW_SCRATCH1, - GEN9_LBS_SLA_RETRY_TIMER_DECREMENT_ENABLE); - /* WaDisableKillLogic:bxt,skl,kbl */ if (!IS_COFFEELAKE(dev_priv)) wa_write_or(wal, @@ -685,24 +674,6 @@ static void gen9_gt_workarounds_init(struct drm_i915_private *dev_priv) wa_write_or(wal, GAM_ECOCHK, BDW_DISABLE_HDC_INVALIDATION); - - /* WaProgramL3SqcReg1DefaultForPerf:bxt,glk */ - if (IS_GEN9_LP(dev_priv)) - wa_write_masked_or(wal, - GEN8_L3SQCREG1, - L3_PRIO_CREDITS_MASK, - L3_GENERAL_PRIO_CREDITS(62) | - L3_HIGH_PRIO_CREDITS(2)); - - /* WaOCLCoherentLineFlush:skl,bxt,kbl,cfl */ - wa_write_or(wal, - GEN8_L3SQCREG4, - GEN8_LQSC_FLUSH_COHERENT_LINES); - - /* WaEnablePreemptionGranularityControlByUMD:skl,bxt,kbl,cfl,[cnl] */ - wa_masked_en(wal, - GEN7_FF_SLICE_CS_CHICKEN1, - GEN9_FFSC_PERCTX_PREEMPT_CTRL); } static void skl_gt_workarounds_init(struct drm_i915_private *dev_priv) @@ -711,11 +682,6 @@ static void skl_gt_workarounds_init(struct drm_i915_private *dev_priv) gen9_gt_workarounds_init(dev_priv); - /* WaEnableGapsTsvCreditFix:skl */ - wa_write_or(wal, - GEN8_GARBCNTL, - GEN9_GAPS_TSV_CREDIT_DISABLE); - /* WaDisableGafsUnitClkGating:skl */ wa_write_or(wal, GEN7_UCGCTL4, @@ -734,11 +700,6 @@ static void bxt_gt_workarounds_init(struct drm_i915_private *dev_priv) gen9_gt_workarounds_init(dev_priv); - /* WaDisablePooledEuLoadBalancingFix:bxt */ - wa_masked_en(wal, - FF_SLICE_CS_CHICKEN2, - GEN9_POOLED_EU_LOAD_BALANCING_FIX_DISABLE); - /* WaInPlaceDecompressionHang:bxt */ wa_write_or(wal, GEN9_GAMT_ECO_REG_RW_IA, @@ -751,11 +712,6 @@ static void kbl_gt_workarounds_init(struct drm_i915_private *dev_priv) gen9_gt_workarounds_init(dev_priv); - /* WaEnableGapsTsvCreditFix:kbl */ - wa_write_or(wal, - GEN8_GARBCNTL, - GEN9_GAPS_TSV_CREDIT_DISABLE); - /* WaDisableDynamicCreditSharing:kbl */ if (IS_KBL_REVID(dev_priv, 0, KBL_REVID_B0)) wa_write_or(wal, @@ -771,21 +727,6 @@ static void kbl_gt_workarounds_init(struct drm_i915_private *dev_priv) wa_write_or(wal, GEN9_GAMT_ECO_REG_RW_IA, GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS); - - /* WaKBLVECSSemaphoreWaitPoll:kbl */ - if (IS_KBL_REVID(dev_priv, KBL_REVID_A0, KBL_REVID_E0)) { - struct intel_engine_cs *engine; - unsigned int tmp; - - for_each_engine(engine, dev_priv, tmp) { - if (engine->id == RCS) - continue; - - wa_write(wal, - RING_SEMA_WAIT_POLL(engine->mmio_base), - 1); - } - } } static void glk_gt_workarounds_init(struct drm_i915_private *dev_priv) @@ -799,11 +740,6 @@ static void cfl_gt_workarounds_init(struct drm_i915_private *dev_priv) gen9_gt_workarounds_init(dev_priv); - /* WaEnableGapsTsvCreditFix:cfl */ - wa_write_or(wal, - GEN8_GARBCNTL, - GEN9_GAPS_TSV_CREDIT_DISABLE); - /* WaDisableGafsUnitClkGating:cfl */ wa_write_or(wal, GEN7_UCGCTL4, @@ -894,11 +830,6 @@ static void cnl_gt_workarounds_init(struct drm_i915_private *dev_priv) wa_write_or(wal, GEN9_GAMT_ECO_REG_RW_IA, GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS); - - /* WaEnablePreemptionGranularityControlByUMD:cnl */ - wa_masked_en(wal, - GEN7_FF_SLICE_CS_CHICKEN1, - GEN9_FFSC_PERCTX_PREEMPT_CTRL); } static void icl_gt_workarounds_init(struct drm_i915_private *dev_priv) @@ -907,53 +838,17 @@ static void icl_gt_workarounds_init(struct drm_i915_private *dev_priv) wa_init_mcr(dev_priv); - /* This is not an Wa. Enable for better image quality */ - wa_masked_en(wal, - _3D_CHICKEN3, - _3D_CHICKEN3_AA_LINE_QUALITY_FIX_ENABLE); - /* WaInPlaceDecompressionHang:icl */ wa_write_or(wal, GEN9_GAMT_ECO_REG_RW_IA, GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS); - /* WaPipelineFlushCoherentLines:icl */ - wa_write_or(wal, - GEN8_L3SQCREG4, - GEN8_LQSC_FLUSH_COHERENT_LINES); - - /* Wa_1405543622:icl - * Formerly known as WaGAPZPriorityScheme - */ - wa_write_or(wal, - GEN8_GARBCNTL, - GEN11_ARBITRATION_PRIO_ORDER_MASK); - - /* Wa_1604223664:icl - * Formerly known as WaL3BankAddressHashing - */ - wa_write_masked_or(wal, - GEN8_GARBCNTL, - GEN11_HASH_CTRL_EXCL_MASK, - GEN11_HASH_CTRL_EXCL_BIT0); - wa_write_masked_or(wal, - GEN11_GLBLINVL, - GEN11_BANK_HASH_ADDR_EXCL_MASK, - GEN11_BANK_HASH_ADDR_EXCL_BIT0); - /* WaModifyGamTlbPartitioning:icl */ wa_write_masked_or(wal, GEN11_GACB_PERF_CTRL, GEN11_HASH_CTRL_MASK, GEN11_HASH_CTRL_BIT0 | GEN11_HASH_CTRL_BIT4); - /* Wa_1405733216:icl - * Formerly known as WaDisableCleanEvicts - */ - wa_write_or(wal, - GEN8_L3SQCREG4, - GEN11_LQSC_CLEAN_EVICT_DISABLE); - /* Wa_1405766107:icl * Formerly known as WaCL2SFHalfMaxAlloc */ @@ -986,13 +881,6 @@ static void icl_gt_workarounds_init(struct drm_i915_private *dev_priv) INF_UNIT_LEVEL_CLKGATE, CGPSF_CLKGATE_DIS); - /* WaForwardProgressSoftReset:icl */ - wa_write_or(wal, - GEN10_SCRATCH_LNCF2, - PMFLUSHDONE_LNICRSDROP | - PMFLUSH_GAPL3UNBLOCK | - PMFLUSHDONE_LNEBLK); - /* Wa_1406463099:icl * Formerly known as WaGamTlbPendError */ @@ -1242,6 +1130,143 @@ void intel_whitelist_workarounds_apply(struct intel_engine_cs *engine) whitelist_apply(engine, whitelist_build(engine, &w)); } +static void rcs_engine_wa_init(struct intel_engine_cs *engine) +{ + struct drm_i915_private *i915 = engine->i915; + struct i915_wa_list *wal = &engine->wa_list; + + if (IS_GEN9(i915)) { + /* WaContextSwitchWithConcurrentTLBInvalidate:skl,bxt,kbl,glk,cfl */ + wa_masked_en(wal, + GEN9_CSFE_CHICKEN1_RCS, + GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE); + + /* WaEnableLbsSlaRetryTimerDecrement:skl,bxt,kbl,glk,cfl */ + wa_write_or(wal, + BDW_SCRATCH1, + GEN9_LBS_SLA_RETRY_TIMER_DECREMENT_ENABLE); + + /* WaProgramL3SqcReg1DefaultForPerf:bxt,glk */ + if (IS_GEN9_LP(i915)) + wa_write_masked_or(wal, + GEN8_L3SQCREG1, + L3_PRIO_CREDITS_MASK, + L3_GENERAL_PRIO_CREDITS(62) | + L3_HIGH_PRIO_CREDITS(2)); + + /* WaOCLCoherentLineFlush:skl,bxt,kbl,cfl */ + wa_write_or(wal, + GEN8_L3SQCREG4, + GEN8_LQSC_FLUSH_COHERENT_LINES); + } + + if (IS_BROXTON(i915)) { + /* WaDisablePooledEuLoadBalancingFix:bxt */ + wa_masked_en(wal, + FF_SLICE_CS_CHICKEN2, + GEN9_POOLED_EU_LOAD_BALANCING_FIX_DISABLE); + } + + if (IS_SKYLAKE(i915) || IS_KABYLAKE(i915) || IS_COFFEELAKE(i915)) { + /* WaEnableGapsTsvCreditFix:skl,kbl,cfl */ + wa_write_or(wal, + GEN8_GARBCNTL, + GEN9_GAPS_TSV_CREDIT_DISABLE); + } + + if (IS_GEN9(i915) || IS_CANNONLAKE(i915)) { + /* WaEnablePreemptionGranularityControlByUMD:skl,bxt,kbl,cfl,cnl */ + wa_masked_en(wal, + GEN7_FF_SLICE_CS_CHICKEN1, + GEN9_FFSC_PERCTX_PREEMPT_CTRL); + } + + if (IS_ICELAKE(i915)) { + /* This is not an Wa. Enable for better image quality */ + wa_masked_en(wal, + _3D_CHICKEN3, + _3D_CHICKEN3_AA_LINE_QUALITY_FIX_ENABLE); + + /* WaPipelineFlushCoherentLines:icl */ + wa_write_or(wal, + GEN8_L3SQCREG4, + GEN8_LQSC_FLUSH_COHERENT_LINES); + + /* + * Wa_1405543622:icl + * Formerly known as WaGAPZPriorityScheme + */ + wa_write_or(wal, + GEN8_GARBCNTL, + GEN11_ARBITRATION_PRIO_ORDER_MASK); + + /* + * Wa_1604223664:icl + * Formerly known as WaL3BankAddressHashing + */ + wa_write_masked_or(wal, + GEN8_GARBCNTL, + GEN11_HASH_CTRL_EXCL_MASK, + GEN11_HASH_CTRL_EXCL_BIT0); + wa_write_masked_or(wal, + GEN11_GLBLINVL, + GEN11_BANK_HASH_ADDR_EXCL_MASK, + GEN11_BANK_HASH_ADDR_EXCL_BIT0); + + /* + * Wa_1405733216:icl + * Formerly known as WaDisableCleanEvicts + */ + wa_write_or(wal, + GEN8_L3SQCREG4, + GEN11_LQSC_CLEAN_EVICT_DISABLE); + + /* WaForwardProgressSoftReset:icl */ + wa_write_or(wal, + GEN10_SCRATCH_LNCF2, + PMFLUSHDONE_LNICRSDROP | + PMFLUSH_GAPL3UNBLOCK | + PMFLUSHDONE_LNEBLK); + } +} + +static void xcs_engine_wa_init(struct intel_engine_cs *engine) +{ + struct drm_i915_private *i915 = engine->i915; + struct i915_wa_list *wal = &engine->wa_list; + + if (IS_KABYLAKE(i915)) { + /* WaKBLVECSSemaphoreWaitPoll:kbl */ + if (IS_KBL_REVID(i915, KBL_REVID_A0, KBL_REVID_E0)) { + wa_write(wal, + RING_SEMA_WAIT_POLL(engine->mmio_base), + 1); + } + } +} + +void intel_engine_workarounds_init(struct intel_engine_cs *engine) +{ + struct i915_wa_list *wal = &engine->wa_list; + + if (GEM_WARN_ON(INTEL_GEN(engine->i915) < 8)) + return; + + wa_init_start(wal, engine->name); + + if (engine->id == RCS) + rcs_engine_wa_init(engine); + else + xcs_engine_wa_init(engine); + + wa_init_finish(wal); +} + +void intel_engine_workarounds_apply(struct intel_engine_cs *engine) +{ + wa_list_apply(engine->i915, &engine->wa_list); +} + #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST) #include "selftests/intel_workarounds.c" #endif diff --git a/drivers/gpu/drm/i915/intel_workarounds.h b/drivers/gpu/drm/i915/intel_workarounds.h index 64aed9cf0200..2998767d51ca 100644 --- a/drivers/gpu/drm/i915/intel_workarounds.h +++ b/drivers/gpu/drm/i915/intel_workarounds.h @@ -36,4 +36,7 @@ void intel_gt_workarounds_apply(struct drm_i915_private *dev_priv); void intel_whitelist_workarounds_apply(struct intel_engine_cs *engine); +void intel_engine_workarounds_init(struct intel_engine_cs *engine); +void intel_engine_workarounds_apply(struct intel_engine_cs *engine); + #endif