From patchwork Wed Aug 15 14:34:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 10566661 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9885213B6 for ; Wed, 15 Aug 2018 14:34:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8A0482A083 for ; Wed, 15 Aug 2018 14:34:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7E19F2A1AA; Wed, 15 Aug 2018 14:34:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3AEE72A083 for ; Wed, 15 Aug 2018 14:34:48 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 307D96E3A3; Wed, 15 Aug 2018 14:34:47 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id 593446E3A4 for ; Wed, 15 Aug 2018 14:34:43 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 12924971-1500050 for multiple; Wed, 15 Aug 2018 15:34:38 +0100 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Wed, 15 Aug 2018 15:34:31 +0100 Message-Id: <20180815143434.13675-1-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.18.0 Subject: [Intel-gfx] [PATCH 1/4] drm/i915/execlists: Assert the queue is non-empty on unsubmitting X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP In the sequence <0>[ 531.960431] drv_self-4806 7.... 527402570us : intel_gpu_reset: engine_mask=1, ret=0, retry=0 <0>[ 531.960431] drv_self-4806 7.... 527402571us : execlists_reset: rcs0 request global=115de, current=71133 <0>[ 531.960431] drv_self-4806 7d..1 527402571us : execlists_cancel_port_requests: rcs0:port0 global=71134 (fence 826b:198), (current 71133) <0>[ 531.960431] drv_self-4806 7d..1 527402572us : execlists_cancel_port_requests: rcs0:port1 global=71135 (fence 826c:53), (current 71133) <0>[ 531.960431] drv_self-4806 7d..1 527402572us : __i915_request_unsubmit: rcs0 fence 826c:53 <- global=71135, current 71133 <0>[ 531.960431] drv_self-4806 7d..1 527402579us : __i915_request_unsubmit: rcs0 fence 826b:198 <- global=71134, current 71133 <0>[ 531.960431] drv_self-4806 7.... 527402613us : intel_engine_cancel_stop_cs: rcs0 <0>[ 531.960431] drv_self-4806 7.... 527402624us : execlists_reset_finish: rcs0 we are missing the execlists_submission_tasklet() invocation before the execlists_reset_fini() implying that either the queue is empty, or we failed to schedule and run the tasklet on finish. Add an assert so we are sure that on unsubmitting the incomplete request after reset, the queue is indeed populated. Signed-off-by: Chris Wilson --- drivers/gpu/drm/i915/intel_lrc.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c index 3f90c74038ef..0e90fd831c70 100644 --- a/drivers/gpu/drm/i915/intel_lrc.c +++ b/drivers/gpu/drm/i915/intel_lrc.c @@ -344,6 +344,7 @@ static void __unwind_incomplete_requests(struct intel_engine_cs *engine) last_prio = rq_prio(rq); p = lookup_priolist(engine, last_prio); } + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root)); GEM_BUG_ON(p->priority != rq_prio(rq)); list_add(&rq->sched.link, &p->requests); From patchwork Wed Aug 15 14:34:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 10566659 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AF47F13B6 for ; Wed, 15 Aug 2018 14:34:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9E0192A083 for ; Wed, 15 Aug 2018 14:34:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8F1A02A1AA; Wed, 15 Aug 2018 14:34:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1E32A2A083 for ; Wed, 15 Aug 2018 14:34:45 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 456636E3A6; Wed, 15 Aug 2018 14:34:44 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5758A6E3A3 for ; Wed, 15 Aug 2018 14:34:43 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 12924972-1500050 for multiple; Wed, 15 Aug 2018 15:34:38 +0100 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Wed, 15 Aug 2018 15:34:32 +0100 Message-Id: <20180815143434.13675-2-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180815143434.13675-1-chris@chris-wilson.co.uk> References: <20180815143434.13675-1-chris@chris-wilson.co.uk> Subject: [Intel-gfx] [PATCH 2/4] drm/i915/execlists: Reset queue_priority on cancellation X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP If we have an available execlists port, the queue_priority should be INT_MIN to allow immediate direct submission. As we clear the ports in execlists_cancel_port_requests(), we should then reset queue_priority to show the available space. Signed-off-by: Chris Wilson --- drivers/gpu/drm/i915/intel_lrc.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c index 0e90fd831c70..178a74e29400 100644 --- a/drivers/gpu/drm/i915/intel_lrc.c +++ b/drivers/gpu/drm/i915/intel_lrc.c @@ -800,6 +800,7 @@ execlists_cancel_port_requests(struct intel_engine_execlists * const execlists) } execlists_clear_all_active(execlists); + execlists->queue_priority = INT_MIN; } static void reset_csb_pointers(struct intel_engine_execlists *execlists) From patchwork Wed Aug 15 14:34:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 10566663 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EE3131510 for ; Wed, 15 Aug 2018 14:34:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DE77D2A083 for ; Wed, 15 Aug 2018 14:34:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D29862A1AA; Wed, 15 Aug 2018 14:34:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8BC062A083 for ; Wed, 15 Aug 2018 14:34:49 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 819EA6E3A4; Wed, 15 Aug 2018 14:34:47 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id 31D326E3A4 for ; Wed, 15 Aug 2018 14:34:44 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 12924973-1500050 for multiple; Wed, 15 Aug 2018 15:34:38 +0100 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Wed, 15 Aug 2018 15:34:33 +0100 Message-Id: <20180815143434.13675-3-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180815143434.13675-1-chris@chris-wilson.co.uk> References: <20180815143434.13675-1-chris@chris-wilson.co.uk> Subject: [Intel-gfx] [PATCH 3/4] drm/i915/execlists: Include reset depth in traces X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP Show the reset depth (the tasklet disable count) in the GEM_TRACE to indicate when we might not expect tasklets to be flushed. Signed-off-by: Chris Wilson --- drivers/gpu/drm/i915/intel_guc_submission.c | 3 ++- drivers/gpu/drm/i915/intel_lrc.c | 6 ++++-- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/i915/intel_guc_submission.c b/drivers/gpu/drm/i915/intel_guc_submission.c index 195adbd0ebf7..8f43c2fe90ca 100644 --- a/drivers/gpu/drm/i915/intel_guc_submission.c +++ b/drivers/gpu/drm/i915/intel_guc_submission.c @@ -815,7 +815,8 @@ guc_reset_prepare(struct intel_engine_cs *engine) { struct intel_engine_execlists * const execlists = &engine->execlists; - GEM_TRACE("%s\n", engine->name); + GEM_TRACE("%s: depth<-%d\n", engine->name, + atomic_read(&execlists->tasklet.count)); /* * Prevent request submission to the hardware until we have diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c index 178a74e29400..257d6f5f44d1 100644 --- a/drivers/gpu/drm/i915/intel_lrc.c +++ b/drivers/gpu/drm/i915/intel_lrc.c @@ -1834,7 +1834,8 @@ execlists_reset_prepare(struct intel_engine_cs *engine) struct i915_request *request, *active; unsigned long flags; - GEM_TRACE("%s\n", engine->name); + GEM_TRACE("%s: depth<-%d\n", engine->name, + atomic_read(&execlists->tasklet.count)); /* * Prevent request submission to the hardware until we have @@ -1982,7 +1983,8 @@ static void execlists_reset_finish(struct intel_engine_cs *engine) */ __tasklet_enable_sync_once(&execlists->tasklet); - GEM_TRACE("%s\n", engine->name); + GEM_TRACE("%s: depth->%d\n", engine->name, + atomic_read(&execlists->tasklet.count)); } static int intel_logical_ring_emit_pdps(struct i915_request *rq) From patchwork Wed Aug 15 14:34:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 10566665 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 03F7F1510 for ; Wed, 15 Aug 2018 14:34:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E778C2A083 for ; Wed, 15 Aug 2018 14:34:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DC06F2A1AA; Wed, 15 Aug 2018 14:34:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 949362A083 for ; Wed, 15 Aug 2018 14:34:50 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E3A006E3AB; Wed, 15 Aug 2018 14:34:47 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id 30CDE6E3A3 for ; Wed, 15 Aug 2018 14:34:44 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 12924974-1500050 for multiple; Wed, 15 Aug 2018 15:34:38 +0100 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Wed, 15 Aug 2018 15:34:34 +0100 Message-Id: <20180815143434.13675-4-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180815143434.13675-1-chris@chris-wilson.co.uk> References: <20180815143434.13675-1-chris@chris-wilson.co.uk> Subject: [Intel-gfx] [PATCH 4/4] drm/i915/execlists: Use direct submission to restart HW after reset X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP We can bypass ksoftirqd for restarting our submission queue after a GPU reset and so avoid any inscrutable delays. Signed-off-by: Chris Wilson --- drivers/gpu/drm/i915/intel_guc_submission.c | 24 +++++++++++++++++++++ drivers/gpu/drm/i915/intel_lrc.c | 2 +- 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/intel_guc_submission.c b/drivers/gpu/drm/i915/intel_guc_submission.c index 8f43c2fe90ca..d9f9b12a0213 100644 --- a/drivers/gpu/drm/i915/intel_guc_submission.c +++ b/drivers/gpu/drm/i915/intel_guc_submission.c @@ -842,6 +842,29 @@ guc_reset_prepare(struct intel_engine_cs *engine) return i915_gem_find_active_request(engine); } +static void guc_reset_finish(struct intel_engine_cs *engine) +{ + struct intel_engine_execlists * const execlists = &engine->execlists; + + /* After a GPU reset, we may have requests to replay */ + if (!RB_EMPTY_ROOT(&execlists->queue.rb_root)) + tasklet_schedule(&execlists->tasklet); + + /* + * Flush the tasklet while we still have the forcewake to be sure + * that it is not allowed to sleep before we restart and reload a + * context. + * + * As before (with execlists_reset_prepare) we rely on the caller + * serialising multiple attempts to reset so that we know that we + * are the only one manipulating tasklet state. + */ + __tasklet_enable_sync_once(&execlists->tasklet); + + GEM_TRACE("%s: depth->%d\n", engine->name, + atomic_read(&execlists->tasklet.count)); +} + /* * Everything below here is concerned with setup & teardown, and is * therefore not part of the somewhat time-critical batch-submission @@ -1289,6 +1312,7 @@ static void guc_set_default_submission(struct intel_engine_cs *engine) engine->unpark = guc_submission_unpark; engine->reset.prepare = guc_reset_prepare; + engine->reset.finish = guc_reset_finish; engine->flags &= ~I915_ENGINE_SUPPORTS_STATS; } diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c index 257d6f5f44d1..4631d38f0b6e 100644 --- a/drivers/gpu/drm/i915/intel_lrc.c +++ b/drivers/gpu/drm/i915/intel_lrc.c @@ -1970,7 +1970,7 @@ static void execlists_reset_finish(struct intel_engine_cs *engine) /* After a GPU reset, we may have requests to replay */ if (!RB_EMPTY_ROOT(&execlists->queue.rb_root)) - tasklet_schedule(&execlists->tasklet); + execlists_submission_tasklet(execlists->tasklet.data); /* * Flush the tasklet while we still have the forcewake to be sure