From patchwork Thu Mar 26 22:35:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11461381 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B9BD6159A for ; Thu, 26 Mar 2020 22:35:34 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A2C6720787 for ; Thu, 26 Mar 2020 22:35:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A2C6720787 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D8E646E955; Thu, 26 Mar 2020 22:35:33 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id E7B596E366 for ; Thu, 26 Mar 2020 22:35:31 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from build.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 20706638-1500050 for multiple; Thu, 26 Mar 2020 22:35:03 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Thu, 26 Mar 2020 22:35:00 +0000 Message-Id: <20200326223501.19654-1-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 1/2] drm/i915/execlists: Prevent GPU death on ELSP[1] promotion to idle context X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" In what seems remarkably similar to the w/a required to not reload an idle context with HEAD==TAIL, it appears we must prevent the HW from switching to an idle context in ELSP[1], while simultaneously trying to preempt the HW to run another context and a continuation of the idle context (which is no longer idle). process_csb: vecs0: cs-irq head=0, tail=1 process_csb: vecs0: csb[1]: status=0x00000882:0x00000020 trace_ports: vecs0: preempted { 8c0:30!, 0:0 } trace_ports: vecs0: promote { 8b2:32!, 8c0:30 } trace_ports: vecs0: submit { 8b8:32, 8c0:32 } process_csb: vecs0: cs-irq head=1, tail=2 process_csb: vecs0: csb[2]: status=0x00000814:0x00000040 trace_ports: vecs0: completed { 8b2:32!, 8c0:30 } process_csb: vecs0: cs-irq head=2, tail=5 process_csb: vecs0: csb[3]: status=0x00000812:0x00000020 trace_ports: vecs0: preempted { 8c0:30!, 0:0 } trace_ports: vecs0: promote { 8b8:32!, 8c0:32 } process_csb: vecs0: csb[4]: status=0x00000814:0x00000060 trace_ports: vecs0: completed { 8b8:32!, 8c0:32 } process_csb: vecs0: csb[5]: status=0x00000818:0x00000020 trace_ports: vecs0: completed { 8c0:32, 0:0 } process_csb: vecs0: ring:{start:0x00021000, head:03f8, tail:03f8, ctl:00000000, mode:00000200} process_csb: vecs0: rq:{start:00021000, head:03c0, tail:0400, seqno:8c0:32, hwsp:30}, process_csb: vecs0: ctx:{start:00021000, head:03f8, tail:03f8}, process_csb: GEM_BUG_ON("context completed before request") Fortunately, we just so happen to have a semaphore in place to prevent the ring HEAD from proceeding past the end of a request that we can use to fix the HEAD in position as we reprogram ELSP. Closes: https://gitlab.freedesktop.org/drm/intel/issues/1501 Signed-off-by: Chris Wilson Cc: Tvrtko Ursulin Cc: Mika Kuoppala --- drivers/gpu/drm/i915/gt/intel_lrc.c | 68 ++++++++++++++--------------- 1 file changed, 34 insertions(+), 34 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c index b12355048501..132816235d8a 100644 --- a/drivers/gpu/drm/i915/gt/intel_lrc.c +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c @@ -1854,7 +1854,7 @@ static inline void clear_ports(struct i915_request **ports, int count) memset_p((void **)ports, NULL, count); } -static void execlists_dequeue(struct intel_engine_cs *engine) +static bool execlists_dequeue(struct intel_engine_cs *engine) { struct intel_engine_execlists * const execlists = &engine->execlists; struct i915_request **port = execlists->pending; @@ -1928,13 +1928,6 @@ static void execlists_dequeue(struct intel_engine_cs *engine) execlists->queue_priority_hint); record_preemption(execlists); - /* - * Don't let the RING_HEAD advance past the breadcrumb - * as we unwind (and until we resubmit) so that we do - * not accidentally tell it to go backwards. - */ - ring_set_paused(engine, 1); - /* * Note that we have not stopped the GPU at this point, * so we are unwinding the incomplete requests as they @@ -1954,7 +1947,6 @@ static void execlists_dequeue(struct intel_engine_cs *engine) last->sched.attr.priority, execlists->queue_priority_hint); - ring_set_paused(engine, 1); defer_active(engine); /* @@ -1988,7 +1980,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine) * of timeslices, our queue might be. */ start_timeslice(engine); - return; + return false; } } } @@ -2021,9 +2013,10 @@ static void execlists_dequeue(struct intel_engine_cs *engine) } if (last && !can_merge_rq(last, rq)) { + /* leave this for another sibling */ spin_unlock(&ve->base.active.lock); start_timeslice(engine); - return; /* leave this for another sibling */ + return false; } ENGINE_TRACE(engine, @@ -2193,32 +2186,30 @@ static void execlists_dequeue(struct intel_engine_cs *engine) * interrupt for secondary ports). */ execlists->queue_priority_hint = queue_prio(execlists); + if (!submit) + return false; - if (submit) { - *port = execlists_schedule_in(last, port - execlists->pending); - execlists->switch_priority_hint = - switch_prio(engine, *execlists->pending); - - /* - * Skip if we ended up with exactly the same set of requests, - * e.g. trying to timeslice a pair of ordered contexts - */ - if (!memcmp(active, execlists->pending, - (port - execlists->pending + 1) * sizeof(*port))) { - do - execlists_schedule_out(fetch_and_zero(port)); - while (port-- != execlists->pending); + *port = execlists_schedule_in(last, port - execlists->pending); + execlists->switch_priority_hint = + switch_prio(engine, *execlists->pending); - goto skip_submit; - } - clear_ports(port + 1, last_port - port); + /* + * Skip if we ended up with exactly the same set of requests, + * e.g. trying to timeslice a pair of ordered contexts + */ + if (!memcmp(active, execlists->pending, + (port - execlists->pending + 1) * sizeof(*port))) { + do + execlists_schedule_out(fetch_and_zero(port)); + while (port-- != execlists->pending); - execlists_submit_ports(engine); - set_preempt_timeout(engine, *active); - } else { -skip_submit: - ring_set_paused(engine, 0); + return false; } + clear_ports(port + 1, last_port - port); + + execlists_submit_ports(engine); + set_preempt_timeout(engine, *active); + return true; } static void @@ -2478,7 +2469,16 @@ static void __execlists_submission_tasklet(struct intel_engine_cs *const engine) lockdep_assert_held(&engine->active.lock); if (!READ_ONCE(engine->execlists.pending[0])) { rcu_read_lock(); /* protect peeking at execlists->active */ - execlists_dequeue(engine); + + /* + * Don't let the RING_HEAD advance past the breadcrumb + * as we unwind (and until we resubmit) so that we do + * not accidentally tell it to go backwards. + */ + ring_set_paused(engine, 1); + if (!execlists_dequeue(engine)) + ring_set_paused(engine, 0); + rcu_read_unlock(); } } From patchwork Thu Mar 26 22:35:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11461379 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 17B3F159A for ; Thu, 26 Mar 2020 22:35:29 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 005C420787 for ; Thu, 26 Mar 2020 22:35:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 005C420787 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BF4226E047; Thu, 26 Mar 2020 22:35:26 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id CCBBE6E047 for ; Thu, 26 Mar 2020 22:35:23 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from build.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 20706639-1500050 for multiple; Thu, 26 Mar 2020 22:35:03 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Thu, 26 Mar 2020 22:35:01 +0000 Message-Id: <20200326223501.19654-2-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200326223501.19654-1-chris@chris-wilson.co.uk> References: <20200326223501.19654-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 2/2] drm/i915/execlists: Explicitly reset both reg and context runtime X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Upon a GPU reset, we copy the default context image over top of the guilty image. This will rollback the CTX_TIMESTAMP register to before our value of ce->runtime.last. Reset both back to 0 so that we do not encounter an underflow on the next schedule out after resume. This should not be a huge issue in practice, as hangs should be rare in correct code. Signed-off-by: Chris Wilson Cc: Tvrtko Ursulin --- drivers/gpu/drm/i915/gt/intel_lrc.c | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c index 132816235d8a..987dbdf2542e 100644 --- a/drivers/gpu/drm/i915/gt/intel_lrc.c +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c @@ -238,6 +238,17 @@ __execlists_update_reg_state(const struct intel_context *ce, const struct intel_engine_cs *engine, u32 head); +static u32 intel_context_get_runtime(const struct intel_context *ce) +{ + /* + * We can use either ppHWSP[16] which is recorded before the context + * switch (and so excludes the cost of context switches) or use the + * value from the context image itself, which is saved/restored earlier + * and so includes the cost of the save. + */ + return READ_ONCE(ce->lrc_reg_state[CTX_TIMESTAMP]); +} + static void mark_eio(struct i915_request *rq) { if (i915_request_completed(rq)) @@ -1154,6 +1165,7 @@ static void restore_default_state(struct intel_context *ce, engine->context_size - PAGE_SIZE); execlists_init_reg_state(regs, ce, engine, ce->ring, false); + ce->runtime.last = intel_context_get_runtime(ce); } static void reset_active(struct i915_request *rq, @@ -1195,17 +1207,6 @@ static void reset_active(struct i915_request *rq, ce->lrc_desc |= CTX_DESC_FORCE_RESTORE; } -static u32 intel_context_get_runtime(const struct intel_context *ce) -{ - /* - * We can use either ppHWSP[16] which is recorded before the context - * switch (and so excludes the cost of context switches) or use the - * value from the context image itself, which is saved/restored earlier - * and so includes the cost of the save. - */ - return READ_ONCE(ce->lrc_reg_state[CTX_TIMESTAMP]); -} - static void st_update_runtime_underflow(struct intel_context *ce, s32 dt) { #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST) @@ -4581,6 +4582,7 @@ static void init_common_reg_state(u32 * const regs, regs[CTX_CONTEXT_CONTROL] = ctl; regs[CTX_RING_CTL] = RING_CTL_SIZE(ring->size) | RING_VALID; + regs[CTX_TIMESTAMP] = 0; } static void init_wa_bb_reg_state(u32 * const regs,