From patchwork Fri Oct 7 19:11:20 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: akash.goel@intel.com X-Patchwork-Id: 9367407 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D3A66600C8 for ; Fri, 7 Oct 2016 18:53:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B5D6D297ED for ; Fri, 7 Oct 2016 18:53:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A86D8297F1; Fri, 7 Oct 2016 18:53:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3752D297ED for ; Fri, 7 Oct 2016 18:53:44 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 213C46EC2E; Fri, 7 Oct 2016 18:53:43 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by gabe.freedesktop.org (Postfix) with ESMTPS id 98E936EC2E for ; Fri, 7 Oct 2016 18:53:41 +0000 (UTC) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga101.jf.intel.com with ESMTP; 07 Oct 2016 11:53:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.31,456,1473145200"; d="scan'208";a="887810931" Received: from akashgoe-desktop.iind.intel.com ([10.223.82.36]) by orsmga003.jf.intel.com with ESMTP; 07 Oct 2016 11:53:39 -0700 From: akash.goel@intel.com To: intel-gfx@lists.freedesktop.org Date: Sat, 8 Oct 2016 00:41:20 +0530 Message-Id: <1475867480-30828-1-git-send-email-akash.goel@intel.com> X-Mailer: git-send-email 1.9.2 In-Reply-To: References: Cc: Akash Goel Subject: [Intel-gfx] [PATCH v10] drm/i915: Allocate intel_engine_cs structure only for the enabled engines X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP From: Akash Goel With the possibility of addition of many more number of rings in future, the drm_i915_private structure could bloat as an array, of type intel_engine_cs, is embedded inside it. struct intel_engine_cs engine[I915_NUM_ENGINES]; Though this is still fine as generally there is only a single instance of drm_i915_private structure used, but not all of the possible rings would be enabled or active on most of the platforms. Some memory can be saved by allocating intel_engine_cs structure only for the enabled/active engines. Currently the engine/ring ID is kept static and dev_priv->engine[] is simply indexed using the enums defined in intel_engine_id. To save memory and continue using the static engine/ring IDs, 'engine' is defined as an array of pointers. struct intel_engine_cs *engine[I915_NUM_ENGINES]; dev_priv->engine[engine_ID] will be NULL for disabled engine instances. There is a text size reduction of 928 bytes, from 1028200 to 1027272, for i915.o file (but for i915.ko file text size remain same as 1193131 bytes). v2: - Remove the engine iterator field added in drm_i915_private structure, instead pass a local iterator variable to the for_each_engine** macros. (Chris) - Do away with intel_engine_initialized() and instead directly use the NULL pointer check on engine pointer. (Chris) v3: - Remove for_each_engine_id() macro, as the updated macro for_each_engine() can be used in place of it. (Chris) - Protect the access to Render engine Fault register with a NULL check, as engine specific init is done later in Driver load sequence. v4: - Use !!dev_priv->engine[VCS] style for the engine check in getparam. (Chris) - Kill the superfluous init_engine_lists(). v5: - Cleanup the intel_engines_init() & intel_engines_setup(), with respect to allocation of intel_engine_cs structure. (Chris) v6: - Rebase. v7: - Optimize the for_each_engine_masked() macro. (Chris) - Change the type of 'iter' local variable to enum intel_engine_id. (Chris) - Rebase. v8: Rebase. v9: Rebase. v10: - For index calculation use engine ID instead of pointer based arithmetic in intel_engine_sync_index() as engine pointers are not contiguous now (Chris) - For appropriateness, rename local enum variable 'iter' to 'id'. (Joonas) - Use for_each_engine macro for cleanup in intel_engines_init() and remove check for NULL engine pointer in cleanup() routines. (Joonas) Cc: Chris Wilson Signed-off-by: Akash Goel Reviewed-by: Joonas Lahtinen --- drivers/gpu/drm/i915/i915_cmd_parser.c | 3 +- drivers/gpu/drm/i915/i915_debugfs.c | 57 ++++++++++++++++++------------ drivers/gpu/drm/i915/i915_drv.c | 11 +++--- drivers/gpu/drm/i915/i915_drv.h | 22 ++++-------- drivers/gpu/drm/i915/i915_gem.c | 29 +++++++-------- drivers/gpu/drm/i915/i915_gem_context.c | 15 ++++---- drivers/gpu/drm/i915/i915_gem_evict.c | 3 +- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 6 ++-- drivers/gpu/drm/i915/i915_gem_gtt.c | 14 +++++--- drivers/gpu/drm/i915/i915_gem_request.c | 5 +-- drivers/gpu/drm/i915/i915_gpu_error.c | 30 ++++++++-------- drivers/gpu/drm/i915/i915_guc_submission.c | 12 ++++--- drivers/gpu/drm/i915/i915_irq.c | 44 ++++++++++++----------- drivers/gpu/drm/i915/intel_breadcrumbs.c | 6 ++-- drivers/gpu/drm/i915/intel_display.c | 8 ++--- drivers/gpu/drm/i915/intel_engine_cs.c | 26 ++++++++++---- drivers/gpu/drm/i915/intel_guc_loader.c | 6 ++-- drivers/gpu/drm/i915/intel_lrc.c | 11 +++--- drivers/gpu/drm/i915/intel_overlay.c | 2 +- drivers/gpu/drm/i915/intel_pm.c | 17 +++++---- drivers/gpu/drm/i915/intel_ringbuffer.c | 15 ++++---- drivers/gpu/drm/i915/intel_ringbuffer.h | 8 +---- 22 files changed, 193 insertions(+), 157 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_cmd_parser.c b/drivers/gpu/drm/i915/i915_cmd_parser.c index 70980f8..f191d7b 100644 --- a/drivers/gpu/drm/i915/i915_cmd_parser.c +++ b/drivers/gpu/drm/i915/i915_cmd_parser.c @@ -1308,10 +1308,11 @@ int intel_engine_cmd_parser(struct intel_engine_cs *engine, int i915_cmd_parser_get_version(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; bool active = false; /* If the command parser is not enabled, report 0 - unsupported */ - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { if (intel_engine_needs_cmd_parser(engine)) { active = true; break; diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c index 20689f1..ba4f832 100644 --- a/drivers/gpu/drm/i915/i915_debugfs.c +++ b/drivers/gpu/drm/i915/i915_debugfs.c @@ -150,7 +150,7 @@ describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj) obj->base.size / 1024, obj->base.read_domains, obj->base.write_domain); - for_each_engine_id(engine, dev_priv, id) + for_each_engine(engine, dev_priv, id) seq_printf(m, "%x ", i915_gem_active_get_seqno(&obj->last_read[id], &obj->base.dev->struct_mutex)); @@ -332,11 +332,12 @@ static void print_batch_pool_stats(struct seq_file *m, struct drm_i915_gem_object *obj; struct file_stats stats; struct intel_engine_cs *engine; + enum intel_engine_id id; int j; memset(&stats, 0, sizeof(stats)); - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { for (j = 0; j < ARRAY_SIZE(engine->batch_pool.cache_list); j++) { list_for_each_entry(obj, &engine->batch_pool.cache_list[j], @@ -605,6 +606,7 @@ static int i915_gem_batch_pool_info(struct seq_file *m, void *data) struct drm_device *dev = &dev_priv->drm; struct drm_i915_gem_object *obj; struct intel_engine_cs *engine; + enum intel_engine_id id; int total = 0; int ret, j; @@ -612,7 +614,7 @@ static int i915_gem_batch_pool_info(struct seq_file *m, void *data) if (ret) return ret; - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { for (j = 0; j < ARRAY_SIZE(engine->batch_pool.cache_list); j++) { int count; @@ -664,8 +666,9 @@ static int i915_gem_request_info(struct seq_file *m, void *data) { struct drm_i915_private *dev_priv = node_to_i915(m->private); struct drm_device *dev = &dev_priv->drm; - struct intel_engine_cs *engine; struct drm_i915_gem_request *req; + struct intel_engine_cs *engine; + enum intel_engine_id id; int ret, any; ret = mutex_lock_interruptible(&dev->struct_mutex); @@ -673,7 +676,7 @@ static int i915_gem_request_info(struct seq_file *m, void *data) return ret; any = 0; - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { int count; count = 0; @@ -719,8 +722,9 @@ static int i915_gem_seqno_info(struct seq_file *m, void *data) { struct drm_i915_private *dev_priv = node_to_i915(m->private); struct intel_engine_cs *engine; + enum intel_engine_id id; - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) i915_ring_seqno_info(m, engine); return 0; @@ -731,6 +735,7 @@ static int i915_interrupt_info(struct seq_file *m, void *data) { struct drm_i915_private *dev_priv = node_to_i915(m->private); struct intel_engine_cs *engine; + enum intel_engine_id id; int i, pipe; intel_runtime_pm_get(dev_priv); @@ -899,7 +904,7 @@ static int i915_interrupt_info(struct seq_file *m, void *data) seq_printf(m, "Graphics Interrupt mask: %08x\n", I915_READ(GTIMR)); } - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { if (INTEL_GEN(dev_priv) >= 6) { seq_printf(m, "Graphics Interrupt mask (%s): %08x\n", @@ -947,7 +952,7 @@ static int i915_hws_info(struct seq_file *m, void *data) const u32 *hws; int i; - engine = &dev_priv->engine[(uintptr_t)node->info_ent->data]; + engine = dev_priv->engine[(uintptr_t)node->info_ent->data]; hws = engine->status_page.page_addr; if (hws == NULL) return 0; @@ -1334,7 +1339,7 @@ static int i915_hangcheck_info(struct seq_file *m, void *unused) intel_runtime_pm_get(dev_priv); - for_each_engine_id(engine, dev_priv, id) { + for_each_engine(engine, dev_priv, id) { acthd[id] = intel_engine_get_active_head(engine); seqno[id] = intel_engine_get_seqno(engine); } @@ -1350,7 +1355,7 @@ static int i915_hangcheck_info(struct seq_file *m, void *unused) } else seq_printf(m, "Hangcheck inactive\n"); - for_each_engine_id(engine, dev_priv, id) { + for_each_engine(engine, dev_priv, id) { struct intel_breadcrumbs *b = &engine->breadcrumbs; struct rb_node *rb; @@ -1949,6 +1954,7 @@ static int i915_context_status(struct seq_file *m, void *unused) struct drm_device *dev = &dev_priv->drm; struct intel_engine_cs *engine; struct i915_gem_context *ctx; + enum intel_engine_id id; int ret; ret = mutex_lock_interruptible(&dev->struct_mutex); @@ -1975,7 +1981,7 @@ static int i915_context_status(struct seq_file *m, void *unused) seq_putc(m, ctx->remap_slice ? 'R' : 'r'); seq_putc(m, '\n'); - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { struct intel_context *ce = &ctx->engine[engine->id]; seq_printf(m, "%s: ", engine->name); @@ -2042,6 +2048,7 @@ static int i915_dump_lrc(struct seq_file *m, void *unused) struct drm_device *dev = &dev_priv->drm; struct intel_engine_cs *engine; struct i915_gem_context *ctx; + enum intel_engine_id id; int ret; if (!i915.enable_execlists) { @@ -2054,7 +2061,7 @@ static int i915_dump_lrc(struct seq_file *m, void *unused) return ret; list_for_each_entry(ctx, &dev_priv->context_list, link) - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) i915_dump_lrc_obj(m, ctx, engine); mutex_unlock(&dev->struct_mutex); @@ -2163,14 +2170,15 @@ static int per_file_ctx(int id, void *ptr, void *data) static void gen8_ppgtt_info(struct seq_file *m, struct drm_i915_private *dev_priv) { - struct intel_engine_cs *engine; struct i915_hw_ppgtt *ppgtt = dev_priv->mm.aliasing_ppgtt; + struct intel_engine_cs *engine; + enum intel_engine_id id; int i; if (!ppgtt) return; - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { seq_printf(m, "%s\n", engine->name); for (i = 0; i < 4; i++) { u64 pdp = I915_READ(GEN8_RING_PDP_UDW(engine, i)); @@ -2185,11 +2193,12 @@ static void gen6_ppgtt_info(struct seq_file *m, struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; if (IS_GEN6(dev_priv)) seq_printf(m, "GFX_MODE: 0x%08x\n", I915_READ(GFX_MODE)); - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { seq_printf(m, "%s\n", engine->name); if (IS_GEN7(dev_priv)) seq_printf(m, "GFX_MODE: 0x%08x\n", @@ -2258,9 +2267,10 @@ out_unlock: static int count_irq_waiters(struct drm_i915_private *i915) { struct intel_engine_cs *engine; + enum intel_engine_id id; int count = 0; - for_each_engine(engine, i915) + for_each_engine(engine, i915, id) count += intel_engine_has_waiter(engine); return count; @@ -2423,7 +2433,7 @@ static void i915_guc_client_info(struct seq_file *m, seq_printf(m, "\tFailed doorbell: %u\n", client->b_fail); seq_printf(m, "\tLast submission result: %d\n", client->retcode); - for_each_engine_id(engine, dev_priv, id) { + for_each_engine(engine, dev_priv, id) { u64 submissions = client->submissions[id]; tot += submissions; seq_printf(m, "\tSubmissions: %llu %s\n", @@ -2466,7 +2476,7 @@ static int i915_guc_info(struct seq_file *m, void *data) seq_printf(m, "GuC last action error code: %d\n", guc.action_err); seq_printf(m, "\nGuC submissions:\n"); - for_each_engine_id(engine, dev_priv, id) { + for_each_engine(engine, dev_priv, id) { u64 submissions = guc.submissions[id]; total += submissions; seq_printf(m, "\t%-24s: %10llu, last seqno 0x%08x\n", @@ -3087,8 +3097,9 @@ static int i915_engine_info(struct seq_file *m, void *unused) { struct drm_i915_private *dev_priv = node_to_i915(m->private); struct intel_engine_cs *engine; + enum intel_engine_id id; - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { struct intel_breadcrumbs *b = &engine->breadcrumbs; struct drm_i915_gem_request *rq; struct rb_node *rb; @@ -3236,7 +3247,7 @@ static int i915_semaphore_status(struct seq_file *m, void *unused) page = i915_gem_object_get_page(dev_priv->semaphore->obj, 0); seqno = (uint64_t *)kmap_atomic(page); - for_each_engine_id(engine, dev_priv, id) { + for_each_engine(engine, dev_priv, id) { uint64_t offset; seq_printf(m, "%s\n", engine->name); @@ -3261,7 +3272,7 @@ static int i915_semaphore_status(struct seq_file *m, void *unused) kunmap_atomic(seqno); } else { seq_puts(m, " Last signal:"); - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) for (j = 0; j < num_rings; j++) seq_printf(m, "0x%08x\n", I915_READ(engine->semaphore.mbox.signal[j])); @@ -3269,7 +3280,7 @@ static int i915_semaphore_status(struct seq_file *m, void *unused) } seq_puts(m, "\nSync seqno:\n"); - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { for (j = 0; j < num_rings; j++) seq_printf(m, " 0x%08x ", engine->semaphore.sync_seqno[j]); @@ -3325,7 +3336,7 @@ static int i915_wa_registers(struct seq_file *m, void *unused) intel_runtime_pm_get(dev_priv); seq_printf(m, "Workarounds applied: %d\n", workarounds->count); - for_each_engine_id(engine, dev_priv, id) + for_each_engine(engine, dev_priv, id) seq_printf(m, "HW whitelist count for %s: %d\n", engine->name, workarounds->hw_whitelist_count[id]); for (i = 0; i < workarounds->count; ++i) { diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c index 89d3222..b6b8dea 100644 --- a/drivers/gpu/drm/i915/i915_drv.c +++ b/drivers/gpu/drm/i915/i915_drv.c @@ -255,16 +255,16 @@ static int i915_getparam(struct drm_device *dev, void *data, value = dev_priv->overlay ? 1 : 0; break; case I915_PARAM_HAS_BSD: - value = intel_engine_initialized(&dev_priv->engine[VCS]); + value = !!dev_priv->engine[VCS]; break; case I915_PARAM_HAS_BLT: - value = intel_engine_initialized(&dev_priv->engine[BCS]); + value = !!dev_priv->engine[BCS]; break; case I915_PARAM_HAS_VEBOX: - value = intel_engine_initialized(&dev_priv->engine[VECS]); + value = !!dev_priv->engine[VECS]; break; case I915_PARAM_HAS_BSD2: - value = intel_engine_initialized(&dev_priv->engine[VCS2]); + value = !!dev_priv->engine[VCS2]; break; case I915_PARAM_HAS_EXEC_CONSTANTS: value = INTEL_GEN(dev_priv) >= 4; @@ -1731,10 +1731,11 @@ int i915_resume_switcheroo(struct drm_device *dev) static void disable_engines_irq(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; /* Ensure irq handler finishes, and not run again. */ disable_irq(dev_priv->drm.irq); - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) tasklet_kill(&engine->irq_tasklet); } diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h index a219a35..76d1167 100644 --- a/drivers/gpu/drm/i915/i915_drv.h +++ b/drivers/gpu/drm/i915/i915_drv.h @@ -1804,7 +1804,7 @@ struct drm_i915_private { struct pci_dev *bridge_dev; struct i915_gem_context *kernel_context; - struct intel_engine_cs engine[I915_NUM_ENGINES]; + struct intel_engine_cs *engine[I915_NUM_ENGINES]; struct i915_vma *semaphore; u32 next_seqno; @@ -2121,19 +2121,11 @@ static inline struct drm_i915_private *guc_to_i915(struct intel_guc *guc) } /* Simple iterator over all initialised engines */ -#define for_each_engine(engine__, dev_priv__) \ - for ((engine__) = &(dev_priv__)->engine[0]; \ - (engine__) < &(dev_priv__)->engine[I915_NUM_ENGINES]; \ - (engine__)++) \ - for_each_if (intel_engine_initialized(engine__)) - -/* Iterator with engine_id */ -#define for_each_engine_id(engine__, dev_priv__, id__) \ - for ((engine__) = &(dev_priv__)->engine[0], (id__) = 0; \ - (engine__) < &(dev_priv__)->engine[I915_NUM_ENGINES]; \ - (engine__)++) \ - for_each_if (((id__) = (engine__)->id, \ - intel_engine_initialized(engine__))) +#define for_each_engine(engine__, dev_priv__, id__) \ + for ((id__) = 0; \ + (id__) < I915_NUM_ENGINES; \ + (id__)++) \ + for_each_if ((engine__) = (dev_priv__)->engine[(id__)]) #define __mask_next_bit(mask) ({ \ int __idx = ffs(mask) - 1; \ @@ -2144,7 +2136,7 @@ static inline struct drm_i915_private *guc_to_i915(struct intel_guc *guc) /* Iterator over subset of engines selected by mask */ #define for_each_engine_masked(engine__, dev_priv__, mask__, tmp__) \ for (tmp__ = mask__ & INTEL_INFO(dev_priv__)->ring_mask; \ - tmp__ ? (engine__ = &(dev_priv__)->engine[__mask_next_bit(tmp__)]), 1 : 0; ) + tmp__ ? (engine__ = (dev_priv__)->engine[__mask_next_bit(tmp__)]), 1 : 0; ) enum hdmi_force_audio { HDMI_AUDIO_OFF_DVI = -2, /* no aux data for HDMI-DVI converter */ diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index a89a889..e6f0fbf 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -2622,10 +2622,11 @@ static void i915_gem_reset_engine(struct intel_engine_cs *engine) void i915_gem_reset(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; i915_gem_retire_requests(dev_priv); - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) i915_gem_reset_engine(engine); i915_gem_restore_fences(&dev_priv->drm); @@ -2673,12 +2674,13 @@ static void i915_gem_cleanup_engine(struct intel_engine_cs *engine) void i915_gem_set_wedged(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; lockdep_assert_held(&dev_priv->drm.struct_mutex); set_bit(I915_WEDGED, &dev_priv->gpu_error.flags); i915_gem_context_lost(dev_priv); - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) i915_gem_cleanup_engine(engine); mod_delayed_work(dev_priv->wq, &dev_priv->gt.idle_work, 0); @@ -2717,6 +2719,7 @@ i915_gem_idle_work_handler(struct work_struct *work) container_of(work, typeof(*dev_priv), gt.idle_work.work); struct drm_device *dev = &dev_priv->drm; struct intel_engine_cs *engine; + enum intel_engine_id id; bool rearm_hangcheck; if (!READ_ONCE(dev_priv->gt.awake)) @@ -2739,7 +2742,7 @@ i915_gem_idle_work_handler(struct work_struct *work) if (dev_priv->gt.active_engines) goto out_unlock; - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) i915_gem_batch_pool_fini(&engine->batch_pool); GEM_BUG_ON(!dev_priv->gt.awake); @@ -2932,9 +2935,10 @@ int i915_gem_wait_for_idle(struct drm_i915_private *dev_priv, unsigned int flags) { struct intel_engine_cs *engine; + enum intel_engine_id id; int ret; - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { if (engine->last_context == NULL) continue; @@ -3174,7 +3178,7 @@ i915_gem_object_flush_gtt_write_domain(struct drm_i915_gem_object *obj) */ wmb(); if (INTEL_GEN(dev_priv) >= 6 && !HAS_LLC(dev_priv)) - POSTING_READ(RING_ACTHD(dev_priv->engine[RCS].mmio_base)); + POSTING_READ(RING_ACTHD(dev_priv->engine[RCS]->mmio_base)); intel_fb_obj_flush(obj, false, write_origin(obj, I915_GEM_DOMAIN_GTT)); @@ -4349,6 +4353,7 @@ i915_gem_init_hw(struct drm_device *dev) { struct drm_i915_private *dev_priv = to_i915(dev); struct intel_engine_cs *engine; + enum intel_engine_id id; int ret; /* Double layer security blanket, see i915_gem_init() */ @@ -4392,7 +4397,7 @@ i915_gem_init_hw(struct drm_device *dev) } /* Need to do basic initialisation of all rings first: */ - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { ret = engine->init_hw(engine); if (ret) goto out; @@ -4491,17 +4496,12 @@ i915_gem_cleanup_engines(struct drm_device *dev) { struct drm_i915_private *dev_priv = to_i915(dev); struct intel_engine_cs *engine; + enum intel_engine_id id; - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) dev_priv->gt.cleanup_engine(engine); } -static void -init_engine_lists(struct intel_engine_cs *engine) -{ - INIT_LIST_HEAD(&engine->request_list); -} - void i915_gem_load_init_fences(struct drm_i915_private *dev_priv) { @@ -4538,7 +4538,6 @@ void i915_gem_load_init(struct drm_device *dev) { struct drm_i915_private *dev_priv = to_i915(dev); - int i; dev_priv->objects = kmem_cache_create("i915_gem_object", @@ -4562,8 +4561,6 @@ i915_gem_load_init(struct drm_device *dev) INIT_LIST_HEAD(&dev_priv->mm.unbound_list); INIT_LIST_HEAD(&dev_priv->mm.bound_list); INIT_LIST_HEAD(&dev_priv->mm.fence_list); - for (i = 0; i < I915_NUM_ENGINES; i++) - init_engine_lists(&dev_priv->engine[i]); INIT_DELAYED_WORK(&dev_priv->gt.retire_work, i915_gem_retire_work_handler); INIT_DELAYED_WORK(&dev_priv->gt.idle_work, diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c index df10f4e9..481ec43 100644 --- a/drivers/gpu/drm/i915/i915_gem_context.c +++ b/drivers/gpu/drm/i915/i915_gem_context.c @@ -474,10 +474,11 @@ int i915_gem_context_init(struct drm_device *dev) void i915_gem_context_lost(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; lockdep_assert_held(&dev_priv->drm.struct_mutex); - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { if (engine->last_context) { i915_gem_context_unpin(engine->last_context, engine); engine->last_context = NULL; @@ -492,13 +493,13 @@ void i915_gem_context_lost(struct drm_i915_private *dev_priv) if (!i915_gem_context_is_default(ctx)) continue; - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) ctx->engine[engine->id].initialised = false; ctx->remap_slice = ALL_L3_SLICES(dev_priv); } - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { struct intel_context *kce = &dev_priv->kernel_context->engine[engine->id]; @@ -563,6 +564,7 @@ mi_set_context(struct drm_i915_gem_request *req, u32 hw_flags) struct drm_i915_private *dev_priv = req->i915; struct intel_ring *ring = req->ring; struct intel_engine_cs *engine = req->engine; + enum intel_engine_id id; u32 flags = hw_flags | MI_MM_SPACE_GTT; const int num_rings = /* Use an extended w/a on ivb+ if signalling from other rings */ @@ -605,7 +607,7 @@ mi_set_context(struct drm_i915_gem_request *req, u32 hw_flags) intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(num_rings)); - for_each_engine(signaller, dev_priv) { + for_each_engine(signaller, dev_priv, id) { if (signaller == engine) continue; @@ -634,7 +636,7 @@ mi_set_context(struct drm_i915_gem_request *req, u32 hw_flags) intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(num_rings)); - for_each_engine(signaller, dev_priv) { + for_each_engine(signaller, dev_priv, id) { if (signaller == engine) continue; @@ -929,8 +931,9 @@ int i915_switch_context(struct drm_i915_gem_request *req) int i915_gem_switch_to_kernel_context(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { struct drm_i915_gem_request *req; int ret; diff --git a/drivers/gpu/drm/i915/i915_gem_evict.c b/drivers/gpu/drm/i915/i915_gem_evict.c index 5b6f81c..b5e9e66 100644 --- a/drivers/gpu/drm/i915/i915_gem_evict.c +++ b/drivers/gpu/drm/i915/i915_gem_evict.c @@ -37,8 +37,9 @@ static bool gpu_is_idle(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { if (intel_engine_is_active(engine)) return false; } diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c index 72c7c18..fa4d27c 100644 --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c @@ -1599,12 +1599,12 @@ eb_select_engine(struct drm_i915_private *dev_priv, return NULL; } - engine = &dev_priv->engine[_VCS(bsd_idx)]; + engine = dev_priv->engine[_VCS(bsd_idx)]; } else { - engine = &dev_priv->engine[user_ring_map[user_ring_id]]; + engine = dev_priv->engine[user_ring_map[user_ring_id]]; } - if (!intel_engine_initialized(engine)) { + if (!engine) { DRM_DEBUG("execbuf with invalid ring: %u\n", user_ring_id); return NULL; } diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c index 0bb4232..b35701c 100644 --- a/drivers/gpu/drm/i915/i915_gem_gtt.c +++ b/drivers/gpu/drm/i915/i915_gem_gtt.c @@ -1728,8 +1728,9 @@ static void gen8_ppgtt_enable(struct drm_device *dev) { struct drm_i915_private *dev_priv = to_i915(dev); struct intel_engine_cs *engine; + enum intel_engine_id id; - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { u32 four_level = USES_FULL_48BIT_PPGTT(dev) ? GEN8_GFX_PPGTT_48B : 0; I915_WRITE(RING_MODE_GEN7(engine), _MASKED_BIT_ENABLE(GFX_PPGTT_ENABLE | four_level)); @@ -1741,6 +1742,7 @@ static void gen7_ppgtt_enable(struct drm_device *dev) struct drm_i915_private *dev_priv = to_i915(dev); struct intel_engine_cs *engine; uint32_t ecochk, ecobits; + enum intel_engine_id id; ecobits = I915_READ(GAC_ECO_BITS); I915_WRITE(GAC_ECO_BITS, ecobits | ECOBITS_PPGTT_CACHE64B); @@ -1754,7 +1756,7 @@ static void gen7_ppgtt_enable(struct drm_device *dev) } I915_WRITE(GAM_ECOCHK, ecochk); - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { /* GFX_MODE is per-ring on gen7+ */ I915_WRITE(RING_MODE_GEN7(engine), _MASKED_BIT_ENABLE(GFX_PPGTT_ENABLE)); @@ -2239,11 +2241,12 @@ static bool needs_idle_maps(struct drm_i915_private *dev_priv) void i915_check_and_clear_faults(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; if (INTEL_INFO(dev_priv)->gen < 6) return; - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { u32 fault_reg; fault_reg = I915_READ(RING_FAULT_REG(engine)); if (fault_reg & RING_FAULT_VALID) { @@ -2260,7 +2263,10 @@ void i915_check_and_clear_faults(struct drm_i915_private *dev_priv) fault_reg & ~RING_FAULT_VALID); } } - POSTING_READ(RING_FAULT_REG(&dev_priv->engine[RCS])); + + /* Engine specific init may not have been done till this point. */ + if (dev_priv->engine[RCS]) + POSTING_READ(RING_FAULT_REG(dev_priv->engine[RCS])); } static void i915_ggtt_flush(struct drm_i915_private *dev_priv) diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c index 8832f8e..74ede1f 100644 --- a/drivers/gpu/drm/i915/i915_gem_request.c +++ b/drivers/gpu/drm/i915/i915_gem_request.c @@ -256,10 +256,11 @@ static int i915_gem_check_wedge(struct drm_i915_private *dev_priv) static int i915_gem_init_seqno(struct drm_i915_private *dev_priv, u32 seqno) { struct intel_engine_cs *engine; + enum intel_engine_id id; int ret; /* Carefully retire all requests without writing to the rings */ - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { ret = intel_engine_idle(engine, I915_WAIT_INTERRUPTIBLE | I915_WAIT_LOCKED); @@ -276,7 +277,7 @@ static int i915_gem_init_seqno(struct drm_i915_private *dev_priv, u32 seqno) } /* Finally reset hw state */ - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) intel_engine_init_seqno(engine, seqno); return 0; diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c index b5b5869..8176b86 100644 --- a/drivers/gpu/drm/i915/i915_gpu_error.c +++ b/drivers/gpu/drm/i915/i915_gpu_error.c @@ -464,7 +464,7 @@ int i915_error_state_to_str(struct drm_i915_error_state_buf *m, len += scnprintf(buf + len, sizeof(buf), "%s%s", first ? "" : ", ", - dev_priv->engine[j].name); + dev_priv->engine[j]->name); first = 0; } scnprintf(buf + len, sizeof(buf), ")"); @@ -482,7 +482,7 @@ int i915_error_state_to_str(struct drm_i915_error_state_buf *m, obj = ee->batchbuffer; if (obj) { - err_puts(m, dev_priv->engine[i].name); + err_puts(m, dev_priv->engine[i]->name); if (ee->pid != -1) err_printf(m, " (submitted by %s [%d])", ee->comm, @@ -496,14 +496,14 @@ int i915_error_state_to_str(struct drm_i915_error_state_buf *m, obj = ee->wa_batchbuffer; if (obj) { err_printf(m, "%s (w/a) --- gtt_offset = 0x%08x\n", - dev_priv->engine[i].name, + dev_priv->engine[i]->name, lower_32_bits(obj->gtt_offset)); print_error_obj(m, obj); } if (ee->num_requests) { err_printf(m, "%s --- %d requests\n", - dev_priv->engine[i].name, + dev_priv->engine[i]->name, ee->num_requests); for (j = 0; j < ee->num_requests; j++) { err_printf(m, " pid %d, seqno 0x%08x, emitted %ld, head 0x%08x, tail 0x%08x\n", @@ -517,10 +517,10 @@ int i915_error_state_to_str(struct drm_i915_error_state_buf *m, if (IS_ERR(ee->waiters)) { err_printf(m, "%s --- ? waiters [unable to acquire spinlock]\n", - dev_priv->engine[i].name); + dev_priv->engine[i]->name); } else if (ee->num_waiters) { err_printf(m, "%s --- %d waiters\n", - dev_priv->engine[i].name, + dev_priv->engine[i]->name, ee->num_waiters); for (j = 0; j < ee->num_waiters; j++) { err_printf(m, " seqno 0x%08x for %s [%d]\n", @@ -532,7 +532,7 @@ int i915_error_state_to_str(struct drm_i915_error_state_buf *m, if ((obj = ee->ringbuffer)) { err_printf(m, "%s --- ringbuffer = 0x%08x\n", - dev_priv->engine[i].name, + dev_priv->engine[i]->name, lower_32_bits(obj->gtt_offset)); print_error_obj(m, obj); } @@ -546,7 +546,7 @@ int i915_error_state_to_str(struct drm_i915_error_state_buf *m, hws_page = &obj->pages[LRC_PPHWSP_PN][0]; } err_printf(m, "%s --- HW Status = 0x%08llx\n", - dev_priv->engine[i].name, hws_offset); + dev_priv->engine[i]->name, hws_offset); offset = 0; for (elt = 0; elt < PAGE_SIZE/16; elt += 4) { err_printf(m, "[%04x] %08x %08x %08x %08x\n", @@ -563,12 +563,12 @@ int i915_error_state_to_str(struct drm_i915_error_state_buf *m, if (obj) { u64 wa_ctx_offset = obj->gtt_offset; u32 *wa_ctx_page = &obj->pages[0][0]; - struct intel_engine_cs *engine = &dev_priv->engine[RCS]; + struct intel_engine_cs *engine = dev_priv->engine[RCS]; u32 wa_ctx_size = (engine->wa_ctx.indirect_ctx.size + engine->wa_ctx.per_ctx.size); err_printf(m, "%s --- WA ctx batch buffer = 0x%08llx\n", - dev_priv->engine[i].name, wa_ctx_offset); + dev_priv->engine[i]->name, wa_ctx_offset); offset = 0; for (elt = 0; elt < wa_ctx_size; elt += 4) { err_printf(m, "[%04x] %08x %08x %08x %08x\n", @@ -583,7 +583,7 @@ int i915_error_state_to_str(struct drm_i915_error_state_buf *m, if ((obj = ee->ctx)) { err_printf(m, "%s --- HW Context = 0x%08x\n", - dev_priv->engine[i].name, + dev_priv->engine[i]->name, lower_32_bits(obj->gtt_offset)); print_error_obj(m, obj); } @@ -918,7 +918,7 @@ static void gen8_record_semaphore_state(struct drm_i915_error_state *error, if (!error->semaphore) return; - for_each_engine_id(to, dev_priv, id) { + for_each_engine(to, dev_priv, id) { int idx; u16 signal_offset; u32 *tmp; @@ -1173,14 +1173,14 @@ static void i915_gem_record_rings(struct drm_i915_private *dev_priv, i915_error_object_create(dev_priv, dev_priv->semaphore); for (i = 0; i < I915_NUM_ENGINES; i++) { - struct intel_engine_cs *engine = &dev_priv->engine[i]; + struct intel_engine_cs *engine = dev_priv->engine[i]; struct drm_i915_error_engine *ee = &error->engine[i]; struct drm_i915_gem_request *request; ee->pid = -1; ee->engine_id = -1; - if (!intel_engine_initialized(engine)) + if (!engine) continue; ee->engine_id = i; @@ -1603,7 +1603,7 @@ void i915_get_engine_instdone(struct drm_i915_private *dev_priv, enum intel_engine_id engine_id, struct intel_instdone *instdone) { - u32 mmio_base = dev_priv->engine[engine_id].mmio_base; + u32 mmio_base = dev_priv->engine[engine_id]->mmio_base; int slice; int subslice; diff --git a/drivers/gpu/drm/i915/i915_guc_submission.c b/drivers/gpu/drm/i915/i915_guc_submission.c index 3106dcc..a1f76c8 100644 --- a/drivers/gpu/drm/i915/i915_guc_submission.c +++ b/drivers/gpu/drm/i915/i915_guc_submission.c @@ -917,6 +917,7 @@ static void guc_addon_create(struct intel_guc *guc) struct guc_policies *policies; struct guc_mmio_reg_state *reg_state; struct intel_engine_cs *engine; + enum intel_engine_id id; struct page *page; u32 size; @@ -944,10 +945,10 @@ static void guc_addon_create(struct intel_guc *guc) * so its address won't change after we've told the GuC where * to find it. */ - engine = &dev_priv->engine[RCS]; + engine = dev_priv->engine[RCS]; ads->golden_context_lrca = engine->status_page.ggtt_offset; - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) ads->eng_state_size[engine->guc_id] = intel_lr_context_size(engine); /* GuC scheduling policies */ @@ -960,7 +961,7 @@ static void guc_addon_create(struct intel_guc *guc) /* MMIO reg state */ reg_state = (void *)policies + sizeof(struct guc_policies); - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { reg_state->mmio_white_list[engine->guc_id].mmio_start = engine->mmio_base + GUC_MMIO_WHITE_LIST_START; @@ -1014,9 +1015,10 @@ int i915_guc_submission_init(struct drm_i915_private *dev_priv) int i915_guc_submission_enable(struct drm_i915_private *dev_priv) { struct intel_guc *guc = &dev_priv->guc; + struct drm_i915_gem_request *request; struct i915_guc_client *client; struct intel_engine_cs *engine; - struct drm_i915_gem_request *request; + enum intel_engine_id id; /* client for execbuf submission */ client = guc_client_alloc(dev_priv, @@ -1033,7 +1035,7 @@ int i915_guc_submission_enable(struct drm_i915_private *dev_priv) guc_init_doorbell_hw(guc); /* Take over from manual control of ELSP (execlists) */ - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { engine->submit_request = i915_guc_submit; /* Replay the current set of previously submitted requests */ diff --git a/drivers/gpu/drm/i915/i915_irq.c b/drivers/gpu/drm/i915/i915_irq.c index bd6c8b0..fff3265 100644 --- a/drivers/gpu/drm/i915/i915_irq.c +++ b/drivers/gpu/drm/i915/i915_irq.c @@ -1058,8 +1058,9 @@ static u32 vlv_wa_c0_ei(struct drm_i915_private *dev_priv, u32 pm_iir) static bool any_waiters(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) if (intel_engine_has_waiter(engine)) return true; @@ -1257,20 +1258,20 @@ static void ilk_gt_irq_handler(struct drm_i915_private *dev_priv, u32 gt_iir) { if (gt_iir & GT_RENDER_USER_INTERRUPT) - notify_ring(&dev_priv->engine[RCS]); + notify_ring(dev_priv->engine[RCS]); if (gt_iir & ILK_BSD_USER_INTERRUPT) - notify_ring(&dev_priv->engine[VCS]); + notify_ring(dev_priv->engine[VCS]); } static void snb_gt_irq_handler(struct drm_i915_private *dev_priv, u32 gt_iir) { if (gt_iir & GT_RENDER_USER_INTERRUPT) - notify_ring(&dev_priv->engine[RCS]); + notify_ring(dev_priv->engine[RCS]); if (gt_iir & GT_BSD_USER_INTERRUPT) - notify_ring(&dev_priv->engine[VCS]); + notify_ring(dev_priv->engine[VCS]); if (gt_iir & GT_BLT_USER_INTERRUPT) - notify_ring(&dev_priv->engine[BCS]); + notify_ring(dev_priv->engine[BCS]); if (gt_iir & (GT_BLT_CS_ERROR_INTERRUPT | GT_BSD_CS_ERROR_INTERRUPT | @@ -1340,21 +1341,21 @@ static void gen8_gt_irq_handler(struct drm_i915_private *dev_priv, u32 gt_iir[4]) { if (gt_iir[0]) { - gen8_cs_irq_handler(&dev_priv->engine[RCS], + gen8_cs_irq_handler(dev_priv->engine[RCS], gt_iir[0], GEN8_RCS_IRQ_SHIFT); - gen8_cs_irq_handler(&dev_priv->engine[BCS], + gen8_cs_irq_handler(dev_priv->engine[BCS], gt_iir[0], GEN8_BCS_IRQ_SHIFT); } if (gt_iir[1]) { - gen8_cs_irq_handler(&dev_priv->engine[VCS], + gen8_cs_irq_handler(dev_priv->engine[VCS], gt_iir[1], GEN8_VCS1_IRQ_SHIFT); - gen8_cs_irq_handler(&dev_priv->engine[VCS2], + gen8_cs_irq_handler(dev_priv->engine[VCS2], gt_iir[1], GEN8_VCS2_IRQ_SHIFT); } if (gt_iir[3]) - gen8_cs_irq_handler(&dev_priv->engine[VECS], + gen8_cs_irq_handler(dev_priv->engine[VECS], gt_iir[3], GEN8_VECS_IRQ_SHIFT); if (gt_iir[2] & dev_priv->pm_rps_events) @@ -1598,7 +1599,7 @@ static void gen6_rps_irq_handler(struct drm_i915_private *dev_priv, u32 pm_iir) if (HAS_VEBOX(dev_priv)) { if (pm_iir & PM_VEBOX_USER_INTERRUPT) - notify_ring(&dev_priv->engine[VECS]); + notify_ring(dev_priv->engine[VECS]); if (pm_iir & PM_VEBOX_CS_ERROR_INTERRUPT) DRM_DEBUG("Command parser error, pm_iir 0x%08x\n", pm_iir); @@ -2840,9 +2841,10 @@ semaphore_wait_to_signaller_ring(struct intel_engine_cs *engine, u32 ipehr, { struct drm_i915_private *dev_priv = engine->i915; struct intel_engine_cs *signaller; + enum intel_engine_id id; if (INTEL_GEN(dev_priv) >= 8) { - for_each_engine(signaller, dev_priv) { + for_each_engine(signaller, dev_priv, id) { if (engine == signaller) continue; @@ -2852,7 +2854,7 @@ semaphore_wait_to_signaller_ring(struct intel_engine_cs *engine, u32 ipehr, } else { u32 sync_bits = ipehr & MI_SEMAPHORE_SYNC_MASK; - for_each_engine(signaller, dev_priv) { + for_each_engine(signaller, dev_priv, id) { if(engine == signaller) continue; @@ -2973,8 +2975,9 @@ static int semaphore_passed(struct intel_engine_cs *engine) static void semaphore_clear_deadlocks(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) engine->hangcheck.deadlock = 0; } @@ -3101,6 +3104,7 @@ static void i915_hangcheck_elapsed(struct work_struct *work) container_of(work, typeof(*dev_priv), gpu_error.hangcheck_work.work); struct intel_engine_cs *engine; + enum intel_engine_id id; unsigned int hung = 0, stuck = 0; int busy_count = 0; #define BUSY 1 @@ -3120,7 +3124,7 @@ static void i915_hangcheck_elapsed(struct work_struct *work) */ intel_uncore_arm_unclaimed_mmio_detection(dev_priv); - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { bool busy = intel_engine_has_waiter(engine); u64 acthd; u32 seqno; @@ -4011,7 +4015,7 @@ static irqreturn_t i8xx_irq_handler(int irq, void *arg) new_iir = I915_READ16(IIR); /* Flush posted writes */ if (iir & I915_USER_INTERRUPT) - notify_ring(&dev_priv->engine[RCS]); + notify_ring(dev_priv->engine[RCS]); for_each_pipe(dev_priv, pipe) { int plane = pipe; @@ -4208,7 +4212,7 @@ static irqreturn_t i915_irq_handler(int irq, void *arg) new_iir = I915_READ(IIR); /* Flush posted writes */ if (iir & I915_USER_INTERRUPT) - notify_ring(&dev_priv->engine[RCS]); + notify_ring(dev_priv->engine[RCS]); for_each_pipe(dev_priv, pipe) { int plane = pipe; @@ -4440,9 +4444,9 @@ static irqreturn_t i965_irq_handler(int irq, void *arg) new_iir = I915_READ(IIR); /* Flush posted writes */ if (iir & I915_USER_INTERRUPT) - notify_ring(&dev_priv->engine[RCS]); + notify_ring(dev_priv->engine[RCS]); if (iir & I915_BSD_USER_INTERRUPT) - notify_ring(&dev_priv->engine[VCS]); + notify_ring(dev_priv->engine[VCS]); for_each_pipe(dev_priv, pipe) { if (pipe_stats[pipe] & PIPE_START_VBLANK_INTERRUPT_STATUS && diff --git a/drivers/gpu/drm/i915/intel_breadcrumbs.c b/drivers/gpu/drm/i915/intel_breadcrumbs.c index 495611b..23fc104 100644 --- a/drivers/gpu/drm/i915/intel_breadcrumbs.c +++ b/drivers/gpu/drm/i915/intel_breadcrumbs.c @@ -621,6 +621,7 @@ void intel_engine_fini_breadcrumbs(struct intel_engine_cs *engine) unsigned int intel_kick_waiters(struct drm_i915_private *i915) { struct intel_engine_cs *engine; + enum intel_engine_id id; unsigned int mask = 0; /* To avoid the task_struct disappearing beneath us as we wake up @@ -628,7 +629,7 @@ unsigned int intel_kick_waiters(struct drm_i915_private *i915) * RCU lock, i.e. as we call wake_up_process() we must be holding the * rcu_read_lock(). */ - for_each_engine(engine, i915) + for_each_engine(engine, i915, id) if (unlikely(intel_engine_wakeup(engine))) mask |= intel_engine_flag(engine); @@ -638,9 +639,10 @@ unsigned int intel_kick_waiters(struct drm_i915_private *i915) unsigned int intel_kick_signalers(struct drm_i915_private *i915) { struct intel_engine_cs *engine; + enum intel_engine_id id; unsigned int mask = 0; - for_each_engine(engine, i915) { + for_each_engine(engine, i915, id) { if (unlikely(READ_ONCE(engine->breadcrumbs.first_signal))) { wake_up_process(engine->breadcrumbs.signaler); mask |= intel_engine_flag(engine); diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c index a366656..df5d9f8 100644 --- a/drivers/gpu/drm/i915/intel_display.c +++ b/drivers/gpu/drm/i915/intel_display.c @@ -12242,19 +12242,19 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc, work->flip_count = I915_READ(PIPE_FLIPCOUNT_G4X(pipe)) + 1; if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev)) { - engine = &dev_priv->engine[BCS]; + engine = dev_priv->engine[BCS]; if (fb->modifier[0] != old_fb->modifier[0]) /* vlv: DISPLAY_FLIP fails to change tiling */ engine = NULL; } else if (IS_IVYBRIDGE(dev) || IS_HASWELL(dev)) { - engine = &dev_priv->engine[BCS]; + engine = dev_priv->engine[BCS]; } else if (INTEL_INFO(dev)->gen >= 7) { engine = i915_gem_active_get_engine(&obj->last_write, &obj->base.dev->struct_mutex); if (engine == NULL || engine->id != RCS) - engine = &dev_priv->engine[BCS]; + engine = dev_priv->engine[BCS]; } else { - engine = &dev_priv->engine[RCS]; + engine = dev_priv->engine[RCS]; } mmio_flip = use_mmio_flip(engine, obj); diff --git a/drivers/gpu/drm/i915/intel_engine_cs.c b/drivers/gpu/drm/i915/intel_engine_cs.c index 480584c..1685da1 100644 --- a/drivers/gpu/drm/i915/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/intel_engine_cs.c @@ -82,12 +82,17 @@ static const struct engine_info { }, }; -static struct intel_engine_cs * +static int intel_engine_setup(struct drm_i915_private *dev_priv, enum intel_engine_id id) { const struct engine_info *info = &intel_engines[id]; - struct intel_engine_cs *engine = &dev_priv->engine[id]; + struct intel_engine_cs *engine; + + GEM_BUG_ON(dev_priv->engine[id]); + engine = kzalloc(sizeof(*engine), GFP_KERNEL); + if (!engine) + return -ENOMEM; engine->id = id; engine->i915 = dev_priv; @@ -97,7 +102,8 @@ intel_engine_setup(struct drm_i915_private *dev_priv, engine->mmio_base = info->mmio_base; engine->irq_shift = info->irq_shift; - return engine; + dev_priv->engine[id] = engine; + return 0; } /** @@ -112,6 +118,8 @@ int intel_engines_init(struct drm_device *dev) struct intel_device_info *device_info = mkwrite_device_info(dev_priv); unsigned int mask = 0; int (*init)(struct intel_engine_cs *engine); + struct intel_engine_cs *engine; + enum intel_engine_id id; unsigned int i; int ret; @@ -131,7 +139,11 @@ int intel_engines_init(struct drm_device *dev) if (!init) continue; - ret = init(intel_engine_setup(dev_priv, i)); + ret = intel_engine_setup(dev_priv, i); + if (ret) + goto cleanup; + + ret = init(dev_priv->engine[i]); if (ret) goto cleanup; @@ -151,11 +163,11 @@ int intel_engines_init(struct drm_device *dev) return 0; cleanup: - for (i = 0; i < I915_NUM_ENGINES; i++) { + for_each_engine(engine, dev_priv, id) { if (i915.enable_execlists) - intel_logical_ring_cleanup(&dev_priv->engine[i]); + intel_logical_ring_cleanup(engine); else - intel_engine_cleanup(&dev_priv->engine[i]); + intel_engine_cleanup(engine); } return ret; diff --git a/drivers/gpu/drm/i915/intel_guc_loader.c b/drivers/gpu/drm/i915/intel_guc_loader.c index 7ace96b..95c6751 100644 --- a/drivers/gpu/drm/i915/intel_guc_loader.c +++ b/drivers/gpu/drm/i915/intel_guc_loader.c @@ -100,12 +100,13 @@ const char *intel_guc_fw_status_repr(enum intel_guc_fw_status status) static void guc_interrupts_release(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; int irqs; /* tell all command streamers NOT to forward interrupts or vblank to GuC */ irqs = _MASKED_FIELD(GFX_FORWARD_VBLANK_MASK, GFX_FORWARD_VBLANK_NEVER); irqs |= _MASKED_BIT_DISABLE(GFX_INTERRUPT_STEERING); - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) I915_WRITE(RING_MODE_GEN7(engine), irqs); /* route all GT interrupts to the host */ @@ -117,12 +118,13 @@ static void guc_interrupts_release(struct drm_i915_private *dev_priv) static void guc_interrupts_capture(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; int irqs; u32 tmp; /* tell all command streamers to forward interrupts (but not vblank) to GuC */ irqs = _MASKED_BIT_ENABLE(GFX_INTERRUPT_STEERING); - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) I915_WRITE(RING_MODE_GEN7(engine), irqs); /* route USER_INTERRUPT to Host, all others are sent to GuC. */ diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c index 10fcea57..bc86585 100644 --- a/drivers/gpu/drm/i915/intel_lrc.c +++ b/drivers/gpu/drm/i915/intel_lrc.c @@ -1648,9 +1648,6 @@ void intel_logical_ring_cleanup(struct intel_engine_cs *engine) { struct drm_i915_private *dev_priv; - if (!intel_engine_initialized(engine)) - return; - /* * Tasklet cannot be active at this point due intel_mark_active/idle * so this is just for documentation. @@ -1677,13 +1674,16 @@ void intel_logical_ring_cleanup(struct intel_engine_cs *engine) lrc_destroy_wa_ctx_obj(engine); engine->i915 = NULL; + dev_priv->engine[engine->id] = NULL; + kfree(engine); } void intel_execlists_enable_submission(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) engine->submit_request = execlists_submit_request; } @@ -2151,6 +2151,7 @@ void intel_lr_context_resume(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; struct i915_gem_context *ctx; + enum intel_engine_id id; /* Because we emit WA_TAIL_DWORDS there may be a disparity * between our bookkeeping in ce->ring->head and ce->ring->tail and @@ -2163,7 +2164,7 @@ void intel_lr_context_resume(struct drm_i915_private *dev_priv) * simplicity, we just zero everything out. */ list_for_each_entry(ctx, &dev_priv->context_list, link) { - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { struct intel_context *ce = &ctx->engine[engine->id]; u32 *reg; diff --git a/drivers/gpu/drm/i915/intel_overlay.c b/drivers/gpu/drm/i915/intel_overlay.c index a24bc8c..272eb93 100644 --- a/drivers/gpu/drm/i915/intel_overlay.c +++ b/drivers/gpu/drm/i915/intel_overlay.c @@ -233,7 +233,7 @@ static int intel_overlay_do_wait_request(struct intel_overlay *overlay, static struct drm_i915_gem_request *alloc_request(struct intel_overlay *overlay) { struct drm_i915_private *dev_priv = overlay->i915; - struct intel_engine_cs *engine = &dev_priv->engine[RCS]; + struct intel_engine_cs *engine = dev_priv->engine[RCS]; return i915_gem_request_alloc(engine, dev_priv->kernel_context); } diff --git a/drivers/gpu/drm/i915/intel_pm.c b/drivers/gpu/drm/i915/intel_pm.c index 7f1748a..f5e3cc5 100644 --- a/drivers/gpu/drm/i915/intel_pm.c +++ b/drivers/gpu/drm/i915/intel_pm.c @@ -5359,6 +5359,7 @@ static void gen9_enable_rps(struct drm_i915_private *dev_priv) static void gen9_enable_rc6(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; uint32_t rc6_mask = 0; /* 1a: Software RC state - RC0 */ @@ -5380,7 +5381,7 @@ static void gen9_enable_rc6(struct drm_i915_private *dev_priv) I915_WRITE(GEN6_RC6_WAKE_RATE_LIMIT, 54 << 16); I915_WRITE(GEN6_RC_EVALUATION_INTERVAL, 125000); /* 12500 * 1280ns */ I915_WRITE(GEN6_RC_IDLE_HYSTERSIS, 25); /* 25 * 1280ns */ - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) I915_WRITE(RING_MAX_IDLE(engine->mmio_base), 10); if (HAS_GUC(dev_priv)) @@ -5425,6 +5426,7 @@ static void gen9_enable_rc6(struct drm_i915_private *dev_priv) static void gen8_enable_rps(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; uint32_t rc6_mask = 0; /* 1a: Software RC state - RC0 */ @@ -5441,7 +5443,7 @@ static void gen8_enable_rps(struct drm_i915_private *dev_priv) I915_WRITE(GEN6_RC6_WAKE_RATE_LIMIT, 40 << 16); I915_WRITE(GEN6_RC_EVALUATION_INTERVAL, 125000); /* 12500 * 1280ns */ I915_WRITE(GEN6_RC_IDLE_HYSTERSIS, 25); /* 25 * 1280ns */ - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) I915_WRITE(RING_MAX_IDLE(engine->mmio_base), 10); I915_WRITE(GEN6_RC_SLEEP, 0); if (IS_BROADWELL(dev_priv)) @@ -5501,6 +5503,7 @@ static void gen8_enable_rps(struct drm_i915_private *dev_priv) static void gen6_enable_rps(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; u32 rc6vids, rc6_mask = 0; u32 gtfifodbg; int rc6_mode; @@ -5534,7 +5537,7 @@ static void gen6_enable_rps(struct drm_i915_private *dev_priv) I915_WRITE(GEN6_RC_EVALUATION_INTERVAL, 125000); I915_WRITE(GEN6_RC_IDLE_HYSTERSIS, 25); - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) I915_WRITE(RING_MAX_IDLE(engine->mmio_base), 10); I915_WRITE(GEN6_RC_SLEEP, 0); @@ -5983,6 +5986,7 @@ static void valleyview_cleanup_gt_powersave(struct drm_i915_private *dev_priv) static void cherryview_enable_rps(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; u32 gtfifodbg, val, rc6_mode = 0, pcbr; WARN_ON(!mutex_is_locked(&dev_priv->rps.hw_lock)); @@ -6009,7 +6013,7 @@ static void cherryview_enable_rps(struct drm_i915_private *dev_priv) I915_WRITE(GEN6_RC_EVALUATION_INTERVAL, 125000); /* 12500 * 1280ns */ I915_WRITE(GEN6_RC_IDLE_HYSTERSIS, 25); /* 25 * 1280ns */ - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) I915_WRITE(RING_MAX_IDLE(engine->mmio_base), 10); I915_WRITE(GEN6_RC_SLEEP, 0); @@ -6071,6 +6075,7 @@ static void cherryview_enable_rps(struct drm_i915_private *dev_priv) static void valleyview_enable_rps(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; u32 gtfifodbg, val, rc6_mode = 0; WARN_ON(!mutex_is_locked(&dev_priv->rps.hw_lock)); @@ -6110,7 +6115,7 @@ static void valleyview_enable_rps(struct drm_i915_private *dev_priv) I915_WRITE(GEN6_RC_EVALUATION_INTERVAL, 125000); I915_WRITE(GEN6_RC_IDLE_HYSTERSIS, 25); - for_each_engine(engine, dev_priv) + for_each_engine(engine, dev_priv, id) I915_WRITE(RING_MAX_IDLE(engine->mmio_base), 10); I915_WRITE(GEN6_RC6_THRESHOLD, 0x557); @@ -6793,7 +6798,7 @@ static void __intel_autoenable_gt_powersave(struct work_struct *work) if (READ_ONCE(dev_priv->rps.enabled)) goto out; - rcs = &dev_priv->engine[RCS]; + rcs = dev_priv->engine[RCS]; if (rcs->last_context) goto out; diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c index 729f373..e107455 100644 --- a/drivers/gpu/drm/i915/intel_ringbuffer.c +++ b/drivers/gpu/drm/i915/intel_ringbuffer.c @@ -1228,7 +1228,7 @@ static int gen8_rcs_signal(struct drm_i915_gem_request *req) if (ret) return ret; - for_each_engine_id(waiter, dev_priv, id) { + for_each_engine(waiter, dev_priv, id) { u64 gtt_offset = req->engine->semaphore.signal_ggtt[id]; if (gtt_offset == MI_SEMAPHORE_SYNC_INVALID) continue; @@ -1265,7 +1265,7 @@ static int gen8_xcs_signal(struct drm_i915_gem_request *req) if (ret) return ret; - for_each_engine_id(waiter, dev_priv, id) { + for_each_engine(waiter, dev_priv, id) { u64 gtt_offset = req->engine->semaphore.signal_ggtt[id]; if (gtt_offset == MI_SEMAPHORE_SYNC_INVALID) continue; @@ -1292,6 +1292,7 @@ static int gen6_signal(struct drm_i915_gem_request *req) struct intel_ring *ring = req->ring; struct drm_i915_private *dev_priv = req->i915; struct intel_engine_cs *engine; + enum intel_engine_id id; int ret, num_rings; num_rings = INTEL_INFO(dev_priv)->num_rings; @@ -1299,7 +1300,7 @@ static int gen6_signal(struct drm_i915_gem_request *req) if (ret) return ret; - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { i915_reg_t mbox_reg; if (!(BIT(engine->hw_id) & GEN6_SEMAPHORES_MASK)) @@ -2091,9 +2092,6 @@ void intel_engine_cleanup(struct intel_engine_cs *engine) { struct drm_i915_private *dev_priv; - if (!intel_engine_initialized(engine)) - return; - dev_priv = engine->i915; if (engine->buffer) { @@ -2120,13 +2118,16 @@ void intel_engine_cleanup(struct intel_engine_cs *engine) intel_ring_context_unpin(dev_priv->kernel_context, engine); engine->i915 = NULL; + dev_priv->engine[engine->id] = NULL; + kfree(engine); } void intel_legacy_submission_resume(struct drm_i915_private *dev_priv) { struct intel_engine_cs *engine; + enum intel_engine_id id; - for_each_engine(engine, dev_priv) { + for_each_engine(engine, dev_priv, id) { engine->buffer->head = engine->buffer->tail; engine->buffer->last_retired_head = -1; } diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h index 124f464..44ef6b5 100644 --- a/drivers/gpu/drm/i915/intel_ringbuffer.h +++ b/drivers/gpu/drm/i915/intel_ringbuffer.h @@ -395,12 +395,6 @@ struct intel_engine_cs { u32 (*get_cmd_length_mask)(u32 cmd_header); }; -static inline bool -intel_engine_initialized(const struct intel_engine_cs *engine) -{ - return engine->i915 != NULL; -} - static inline unsigned intel_engine_flag(const struct intel_engine_cs *engine) { @@ -421,7 +415,7 @@ intel_engine_sync_index(struct intel_engine_cs *engine, * vcs2 -> 0 = rcs, 1 = vcs, 2 = bcs, 3 = vecs; */ - idx = (other - engine) - 1; + idx = (other->id - engine->id) - 1; if (idx < 0) idx += I915_NUM_ENGINES;