From patchwork Mon Feb 1 08:56:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 12058491 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 231F4C433E0 for ; Mon, 1 Feb 2021 08:58:17 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C2D9C64E40 for ; Mon, 1 Feb 2021 08:58:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C2D9C64E40 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BF1FF6E4EC; Mon, 1 Feb 2021 08:57:43 +0000 (UTC) Received: from fireflyinternet.com (unknown [77.68.26.236]) by gabe.freedesktop.org (Postfix) with ESMTPS id 90D236E4D4 for ; Mon, 1 Feb 2021 08:57:34 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from build.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 23757732-1500050 for multiple; Mon, 01 Feb 2021 08:57:18 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Mon, 1 Feb 2021 08:56:37 +0000 Message-Id: <20210201085715.27435-19-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210201085715.27435-1-chris@chris-wilson.co.uk> References: <20210201085715.27435-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 19/57] drm/i915: Fix the iterative dfs for defering requests X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Chris Wilson Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" The current implementation of walking the children of a deferred requests lacks the backtracking required to reduce the dfs to linear. Having pulled it from execlists into the common layer, we can reuse the dfs code for priority inheritance. Signed-off-by: Chris Wilson Reviewed-by: Tvrtko Ursulin --- drivers/gpu/drm/i915/i915_scheduler.c | 56 +++++++++++++++++++-------- 1 file changed, 40 insertions(+), 16 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c index bfd37ee801fd..694ca3a3b563 100644 --- a/drivers/gpu/drm/i915/i915_scheduler.c +++ b/drivers/gpu/drm/i915/i915_scheduler.c @@ -466,8 +466,10 @@ void i915_request_set_priority(struct i915_request *rq, int prio) void __i915_sched_defer_request(struct intel_engine_cs *engine, struct i915_request *rq) { - struct list_head *pl; - LIST_HEAD(list); + struct list_head *pos = &rq->sched.waiters_list; + const int prio = rq_prio(rq); + struct i915_request *rn; + LIST_HEAD(dfs); lockdep_assert_held(&engine->active.lock); GEM_BUG_ON(!test_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags)); @@ -477,14 +479,11 @@ void __i915_sched_defer_request(struct intel_engine_cs *engine, * to those that are waiting upon it. So we traverse its chain of * waiters and move any that are earlier than the request to after it. */ - pl = lookup_priolist(engine, rq_prio(rq)); + rq->sched.dfs.prev = NULL; do { - struct i915_dependency *p; - - GEM_BUG_ON(i915_request_is_active(rq)); - list_move_tail(&rq->sched.link, pl); - - for_each_waiter(p, rq) { + list_for_each_continue(pos, &rq->sched.waiters_list) { + struct i915_dependency *p = + list_entry(pos, typeof(*p), wait_link); struct i915_request *w = container_of(p->waiter, typeof(*w), sched); @@ -500,19 +499,44 @@ void __i915_sched_defer_request(struct intel_engine_cs *engine, __i915_request_has_started(w) && !__i915_request_is_complete(rq)); - if (!i915_request_is_ready(w)) + if (!i915_request_in_priority_queue(w)) continue; - if (rq_prio(w) < rq_prio(rq)) + /* + * We also need to reorder within the same priority. + * + * This is unlike priority-inheritance, where if the + * signaler already has a higher priority [earlier + * deadline] than us, we can ignore as it will be + * scheduled first. If a waiter already has the + * same priority, we still have to push it to the end + * of the list. This unfortunately means we cannot + * use the rq_deadline() itself as a 'visited' bit. + */ + if (rq_prio(w) < prio) continue; - GEM_BUG_ON(rq_prio(w) > rq_prio(rq)); - GEM_BUG_ON(i915_request_is_active(w)); - list_move_tail(&w->sched.link, &list); + GEM_BUG_ON(rq_prio(w) != prio); + + /* Remember our position along this branch */ + rq = stack_push(w, rq, pos); + pos = &rq->sched.waiters_list; } - rq = list_first_entry_or_null(&list, typeof(*rq), sched.link); - } while (rq); + /* Note list is reversed for waiters wrt signal hierarchy */ + GEM_BUG_ON(rq->engine != engine); + GEM_BUG_ON(!i915_request_in_priority_queue(rq)); + list_move(&rq->sched.link, &dfs); + + /* Track our visit, and prevent duplicate processing */ + clear_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags); + } while ((rq = stack_pop(rq, &pos))); + + pos = lookup_priolist(engine, prio); + list_for_each_entry_safe(rq, rn, &dfs, sched.link) { + set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags); + list_add_tail(&rq->sched.link, pos); + } } static void queue_request(struct intel_engine_cs *engine,