From patchwork Sun Nov 10 18:57:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236297 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 34F9114E5 for ; Sun, 10 Nov 2019 18:58:41 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1CAC520842 for ; Sun, 10 Nov 2019 18:58:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1CAC520842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 110FB6E7C1; Sun, 10 Nov 2019 18:58:36 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id D9E646E79F for ; Sun, 10 Nov 2019 18:58:29 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151880-1500050 for multiple; Sun, 10 Nov 2019 18:58:09 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:42 +0000 Message-Id: <20191110185806.17413-1-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 01/25] drm/i915: Protect context while grabbing its name for the request X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Inside print_request(), we query the context/timeline name. Nothing immediately protects the context from being freed if the request is complete -- we rely on serialisation by the caller to keep the name valid until they finish using it. Inside intel_engine_dump(), we generally only print the requsts in the execution queue protected by the engine->active.lock, but we also show the pending execlists ports which are not protected and so require an rcu_read_lock to keep the pointer valid. [ 1695.700883] BUG: KASAN: use-after-free in i915_fence_get_timeline_name+0x53/0x90 [i915] [ 1695.700981] Read of size 8 at addr ffff8887344f4d50 by task gem_ctx_persist/2968 [ 1695.701068] [ 1695.701156] CPU: 1 PID: 2968 Comm: gem_ctx_persist Tainted: G U 5.4.0-rc6+ #331 [ 1695.701246] Hardware name: Intel Corporation NUC7i5BNK/NUC7i5BNB, BIOS BNKBL357.86A.0052.2017.0918.1346 09/18/2017 [ 1695.701334] Call Trace: [ 1695.701424] dump_stack+0x5b/0x90 [ 1695.701870] ? i915_fence_get_timeline_name+0x53/0x90 [i915] [ 1695.701964] print_address_description.constprop.7+0x36/0x50 [ 1695.702408] ? i915_fence_get_timeline_name+0x53/0x90 [i915] [ 1695.702856] ? i915_fence_get_timeline_name+0x53/0x90 [i915] [ 1695.702947] __kasan_report.cold.10+0x1a/0x3a [ 1695.703390] ? i915_fence_get_timeline_name+0x53/0x90 [i915] [ 1695.703836] i915_fence_get_timeline_name+0x53/0x90 [i915] [ 1695.704241] print_request+0x82/0x2e0 [i915] [ 1695.704638] ? fwtable_read32+0x133/0x360 [i915] [ 1695.705042] ? write_timestamp+0x110/0x110 [i915] [ 1695.705133] ? _raw_spin_lock_irqsave+0x79/0xc0 [ 1695.705221] ? refcount_inc_not_zero_checked+0x91/0x110 [ 1695.705306] ? refcount_dec_and_mutex_lock+0x50/0x50 [ 1695.705709] ? intel_engine_find_active_request+0x202/0x230 [i915] [ 1695.706115] intel_engine_dump+0x2c9/0x900 [i915] Fixes: c36eebd9ba5d ("drm/i915/gt: execlists->active is serialised by the tasklet") Signed-off-by: Chris Wilson Cc: Mika Kuoppala Cc: Tvrtko Ursulin --- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c index bf20305a1083..b9613d044393 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c @@ -1373,6 +1373,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine, } execlists_active_lock_bh(execlists); + rcu_read_lock(); for (port = execlists->active; (rq = *port); port++) { char hdr[80]; int len; @@ -1410,6 +1411,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine, if (tl) intel_timeline_put(tl); } + rcu_read_unlock(); execlists_active_unlock_bh(execlists); } else if (INTEL_GEN(dev_priv) > 6) { drm_printf(m, "\tPP_DIR_BASE: 0x%08x\n", From patchwork Sun Nov 10 18:57:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236293 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B4BCF15AB for ; Sun, 10 Nov 2019 18:58:39 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9CBDA20842 for ; Sun, 10 Nov 2019 18:58:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9CBDA20842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 83B136E7AD; Sun, 10 Nov 2019 18:58:34 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id D0B856E0B9 for ; Sun, 10 Nov 2019 18:58:27 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151882-1500050 for multiple; Sun, 10 Nov 2019 18:58:10 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:43 +0000 Message-Id: <20191110185806.17413-2-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 02/25] drm/i915/gem: Embed context/timeline name inside the GEM context X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Matthew Auld Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Use a small char buffer inside the i915_gem_context to store the user friendly name so that ctx->name has the same lifetime as the RCU protected GEM context. That is, e.g. when using print_request() that prints the timeline name (ctx->name), the name will not be prematurely freed upon the context being closed and the last reference dropped. Signed-off-by: Chris Wilson Cc: Tvrtko Ursulin Cc: Matthew Auld Reviewed-by: Mika Kuoppala --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 12 ++---------- .../gpu/drm/i915/gem/i915_gem_context_types.h | 18 +++++++++--------- .../gpu/drm/i915/gem/selftests/mock_context.c | 4 +--- 3 files changed, 12 insertions(+), 22 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index 94b51b13ef17..982770e8163d 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -277,9 +277,7 @@ static void i915_gem_context_free(struct i915_gem_context *ctx) if (ctx->timeline) intel_timeline_put(ctx->timeline); - kfree(ctx->name); put_pid(ctx->pid); - mutex_destroy(&ctx->mutex); kfree_rcu(ctx, rcu); @@ -789,12 +787,8 @@ static int gem_context_register(struct i915_gem_context *ctx, mutex_unlock(&ctx->mutex); ctx->pid = get_task_pid(current, PIDTYPE_PID); - ctx->name = kasprintf(GFP_KERNEL, "%s[%d]", - current->comm, pid_nr(ctx->pid)); - if (!ctx->name) { - ret = -ENOMEM; - goto err_pid; - } + snprintf(ctx->name, sizeof(ctx->name), "%s[%d]", + current->comm, pid_nr(ctx->pid)); /* And finally expose ourselves to userspace via the idr */ mutex_lock(&fpriv->context_idr_lock); @@ -803,8 +797,6 @@ static int gem_context_register(struct i915_gem_context *ctx, if (ret >= 0) goto out; - kfree(fetch_and_zero(&ctx->name)); -err_pid: put_pid(fetch_and_zero(&ctx->pid)); out: return ret; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h index 861d7d92fe9f..69df5459c350 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h @@ -100,15 +100,6 @@ struct i915_gem_context { */ struct pid *pid; - /** - * @name: arbitrary name - * - * A name is constructed for the context from the creator's process - * name, pid and user handle in order to uniquely identify the - * context in messages. - */ - const char *name; - /** link: place with &drm_i915_private.context_list */ struct list_head link; struct llist_node free_link; @@ -176,6 +167,15 @@ struct i915_gem_context { * per vm, which may be one per context or shared with the global GTT) */ struct radix_tree_root handles_vma; + + /** + * @name: arbitrary name, used for user debug + * + * A name is constructed for the context from the creator's process + * name, pid and user handle in order to uniquely identify the + * context in messages. + */ + char name[TASK_COMM_LEN + 8]; }; #endif /* __I915_GEM_CONTEXT_TYPES_H__ */ diff --git a/drivers/gpu/drm/i915/gem/selftests/mock_context.c b/drivers/gpu/drm/i915/gem/selftests/mock_context.c index cdcb006321a7..53e89efb09c0 100644 --- a/drivers/gpu/drm/i915/gem/selftests/mock_context.c +++ b/drivers/gpu/drm/i915/gem/selftests/mock_context.c @@ -37,9 +37,7 @@ mock_context(struct drm_i915_private *i915, if (name) { struct i915_ppgtt *ppgtt; - ctx->name = kstrdup(name, GFP_KERNEL); - if (!ctx->name) - goto err_put; + strncpy(ctx->name, name, sizeof(ctx->name)); ppgtt = mock_ppgtt(i915, name); if (!ppgtt) From patchwork Sun Nov 10 18:57:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236277 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0A03314E5 for ; Sun, 10 Nov 2019 18:58:30 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E58D820842 for ; Sun, 10 Nov 2019 18:58:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E58D820842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 795346E0B9; Sun, 10 Nov 2019 18:58:28 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0C5CB6E25F for ; Sun, 10 Nov 2019 18:58:26 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151883-1500050 for multiple; Sun, 10 Nov 2019 18:58:10 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:44 +0000 Message-Id: <20191110185806.17413-3-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 03/25] drm/i915/gem: Update context name on closing X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Update the context.name on closing so that the persistent requests are clear in debug prints. Signed-off-by: Chris Wilson Reviewed-by: Mika Kuoppala --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index 982770e8163d..72d389afa28a 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -461,11 +461,29 @@ static void kill_context(struct i915_gem_context *ctx) } } +static void set_closed_name(struct i915_gem_context *ctx) +{ + char *s; + + /* Replace '[]' with '<>' to indicate closed in debug prints */ + + s = strrchr(ctx->name, '['); + if (!s) + return; + + *s = '<'; + + s = strchr(s + 1, ']'); + if (s) + *s = '>'; +} + static void context_close(struct i915_gem_context *ctx) { struct i915_address_space *vm; i915_gem_context_set_closed(ctx); + set_closed_name(ctx); mutex_lock(&ctx->mutex); From patchwork Sun Nov 10 18:57:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236321 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5917E1850 for ; Sun, 10 Nov 2019 18:58:54 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 40CF42085B for ; Sun, 10 Nov 2019 18:58:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 40CF42085B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 692916E7D5; Sun, 10 Nov 2019 18:58:53 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id BB4AA6E51A for ; Sun, 10 Nov 2019 18:58:30 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151884-1500050 for multiple; Sun, 10 Nov 2019 18:58:10 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:45 +0000 Message-Id: <20191110185806.17413-4-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 04/25] drm/i915/execlists: Move reset_active() from schedule-out to schedule-in X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" The gem_ctx_persistence/smoketest was detecting an odd coherency issue inside the LRC context image; that the address of the ring buffer did not match our associated struct intel_ring. As we set the address into the context image when we pin the ring buffer into place before the context is active, that leaves the question of where did it get overwritten. Either the HW context save occurred after our pin which would imply that our idle barriers are broken, or we overwrote the context image ourselves. It is only in reset_active() where we dabble inside the context image outside of a serialised path from schedule-out; but we could equally perform the operation inside schedule-in which is then fully serialised with the context pin -- and remains serialised by the engine pulse with kill_context(). (The only downside, aside from doing more work inside the engine->active.lock, was the plan to merge all the reset paths into doing their context scrubbing on schedule-out needs more thought.) Fixes: d12acee84ffb ("drm/i915/execlists: Cancel banned contexts on schedule-out") Testcase: igt/gem_ctx_persistence/smoketest Signed-off-by: Chris Wilson Cc: Tvrtko Ursulin --- drivers/gpu/drm/i915/gt/intel_lrc.c | 114 ++++++++++++++-------------- 1 file changed, 57 insertions(+), 57 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c index 1975fe4775d2..a06074a26fb5 100644 --- a/drivers/gpu/drm/i915/gt/intel_lrc.c +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c @@ -1042,6 +1042,59 @@ execlists_check_context(const struct intel_context *ce, WARN_ONCE(!valid, "Invalid lrc state found before submission\n"); } +static void restore_default_state(struct intel_context *ce, + struct intel_engine_cs *engine) +{ + u32 *regs = ce->lrc_reg_state; + + if (engine->pinned_default_state) + memcpy(regs, /* skip restoring the vanilla PPHWSP */ + engine->pinned_default_state + LRC_STATE_PN * PAGE_SIZE, + engine->context_size - PAGE_SIZE); + + execlists_init_reg_state(regs, ce, engine, ce->ring, false); +} + +static void reset_active(struct i915_request *rq, + struct intel_engine_cs *engine) +{ + struct intel_context * const ce = rq->hw_context; + u32 head; + + /* + * The executing context has been cancelled. We want to prevent + * further execution along this context and propagate the error on + * to anything depending on its results. + * + * In __i915_request_submit(), we apply the -EIO and remove the + * requests' payloads for any banned requests. But first, we must + * rewind the context back to the start of the incomplete request so + * that we do not jump back into the middle of the batch. + * + * We preserve the breadcrumbs and semaphores of the incomplete + * requests so that inter-timeline dependencies (i.e other timelines) + * remain correctly ordered. And we defer to __i915_request_submit() + * so that all asynchronous waits are correctly handled. + */ + GEM_TRACE("%s(%s): { rq=%llx:%lld }\n", + __func__, engine->name, rq->fence.context, rq->fence.seqno); + + /* On resubmission of the active request, payload will be scrubbed */ + if (i915_request_completed(rq)) + head = rq->tail; + else + head = active_request(ce->timeline, rq)->head; + ce->ring->head = intel_ring_wrap(ce->ring, head); + intel_ring_update_space(ce->ring); + + /* Scrub the context image to prevent replaying the previous batch */ + restore_default_state(ce, engine); + __execlists_update_reg_state(ce, engine); + + /* We've switched away, so this should be a no-op, but intent matters */ + ce->lrc_desc |= CTX_DESC_FORCE_RESTORE; +} + static inline struct intel_engine_cs * __execlists_schedule_in(struct i915_request *rq) { @@ -1050,8 +1103,11 @@ __execlists_schedule_in(struct i915_request *rq) intel_context_get(ce); + if (unlikely(i915_gem_context_is_banned(ce->gem_context))) + reset_active(rq, engine); + if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM)) - execlists_check_context(ce, rq->engine); + execlists_check_context(ce, engine); if (ce->tag) { /* Use a fixed tag for OA and friends */ @@ -1102,59 +1158,6 @@ static void kick_siblings(struct i915_request *rq, struct intel_context *ce) tasklet_schedule(&ve->base.execlists.tasklet); } -static void restore_default_state(struct intel_context *ce, - struct intel_engine_cs *engine) -{ - u32 *regs = ce->lrc_reg_state; - - if (engine->pinned_default_state) - memcpy(regs, /* skip restoring the vanilla PPHWSP */ - engine->pinned_default_state + LRC_STATE_PN * PAGE_SIZE, - engine->context_size - PAGE_SIZE); - - execlists_init_reg_state(regs, ce, engine, ce->ring, false); -} - -static void reset_active(struct i915_request *rq, - struct intel_engine_cs *engine) -{ - struct intel_context * const ce = rq->hw_context; - u32 head; - - /* - * The executing context has been cancelled. We want to prevent - * further execution along this context and propagate the error on - * to anything depending on its results. - * - * In __i915_request_submit(), we apply the -EIO and remove the - * requests' payloads for any banned requests. But first, we must - * rewind the context back to the start of the incomplete request so - * that we do not jump back into the middle of the batch. - * - * We preserve the breadcrumbs and semaphores of the incomplete - * requests so that inter-timeline dependencies (i.e other timelines) - * remain correctly ordered. And we defer to __i915_request_submit() - * so that all asynchronous waits are correctly handled. - */ - GEM_TRACE("%s(%s): { rq=%llx:%lld }\n", - __func__, engine->name, rq->fence.context, rq->fence.seqno); - - /* On resubmission of the active request, payload will be scrubbed */ - if (i915_request_completed(rq)) - head = rq->tail; - else - head = active_request(ce->timeline, rq)->head; - ce->ring->head = intel_ring_wrap(ce->ring, head); - intel_ring_update_space(ce->ring); - - /* Scrub the context image to prevent replaying the previous batch */ - restore_default_state(ce, engine); - __execlists_update_reg_state(ce, engine); - - /* We've switched away, so this should be a no-op, but intent matters */ - ce->lrc_desc |= CTX_DESC_FORCE_RESTORE; -} - static inline void __execlists_schedule_out(struct i915_request *rq, struct intel_engine_cs * const engine) @@ -1165,9 +1168,6 @@ __execlists_schedule_out(struct i915_request *rq, execlists_context_status_change(rq, INTEL_CONTEXT_SCHEDULE_OUT); intel_gt_pm_put(engine->gt); - if (unlikely(i915_gem_context_is_banned(ce->gem_context))) - reset_active(rq, engine); - /* * If this is part of a virtual engine, its next request may * have been blocked waiting for access to the active context. From patchwork Sun Nov 10 18:57:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236275 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 952D615AB for ; Sun, 10 Nov 2019 18:58:29 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5006A2085B for ; Sun, 10 Nov 2019 18:58:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5006A2085B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 68DF16E06B; Sun, 10 Nov 2019 18:58:28 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id 056976E06B for ; Sun, 10 Nov 2019 18:58:26 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151885-1500050 for multiple; Sun, 10 Nov 2019 18:58:10 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:46 +0000 Message-Id: <20191110185806.17413-5-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 05/25] drm/i915/pmu: "Frequency" is reported as accumulated cycles X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: stable@vger.kernel.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" We report "frequencies" (actual-frequency, requested-frequency) as the number of accumulated cycles so that the average frequency over that period may be determined by the user. This means the units we report to the user are Mcycles (or just M), not MHz. Signed-off-by: Chris Wilson Cc: Tvrtko Ursulin Cc: stable@vger.kernel.org --- drivers/gpu/drm/i915/i915_pmu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_pmu.c b/drivers/gpu/drm/i915/i915_pmu.c index 4804775644bf..9b02be0ad4e6 100644 --- a/drivers/gpu/drm/i915/i915_pmu.c +++ b/drivers/gpu/drm/i915/i915_pmu.c @@ -908,8 +908,8 @@ create_event_attributes(struct i915_pmu *pmu) const char *name; const char *unit; } events[] = { - __event(I915_PMU_ACTUAL_FREQUENCY, "actual-frequency", "MHz"), - __event(I915_PMU_REQUESTED_FREQUENCY, "requested-frequency", "MHz"), + __event(I915_PMU_ACTUAL_FREQUENCY, "actual-frequency", "M"), + __event(I915_PMU_REQUESTED_FREQUENCY, "requested-frequency", "M"), __event(I915_PMU_INTERRUPTS, "interrupts", NULL), __event(I915_PMU_RC6_RESIDENCY, "rc6-residency", "ns"), }; From patchwork Sun Nov 10 18:57:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236305 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7993D14E5 for ; Sun, 10 Nov 2019 18:58:44 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6075120842 for ; Sun, 10 Nov 2019 18:58:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6075120842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F20F46E7B0; Sun, 10 Nov 2019 18:58:34 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id DC3066E7AD for ; Sun, 10 Nov 2019 18:58:28 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151886-1500050 for multiple; Sun, 10 Nov 2019 18:58:10 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:47 +0000 Message-Id: <20191110185806.17413-6-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 06/25] drm/i915/userptr: Handle unlocked gup retries X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Enable gup to retry and fault the pages outside of the mmap_sem lock in our worker. As we are inside our worker, outside of any critical path, we can allow the mmap_sem lock to be dropped in order to service a page fault; this in turn allows the mm to populate the page using a slow fault handler. Testcase: igt/gem_userptr/userfault Signed-off-by: Chris Wilson Cc: Tvrtko Ursulin --- drivers/gpu/drm/i915/gem/i915_gem_userptr.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c b/drivers/gpu/drm/i915/gem/i915_gem_userptr.c index ee65c6acf0e2..5e87126def2f 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_userptr.c @@ -459,26 +459,31 @@ __i915_gem_userptr_get_pages_worker(struct work_struct *_work) if (pvec != NULL) { struct mm_struct *mm = obj->userptr.mm->mm; unsigned int flags = 0; + int locked = 0; if (!i915_gem_object_is_readonly(obj)) flags |= FOLL_WRITE; ret = -EFAULT; if (mmget_not_zero(mm)) { - down_read(&mm->mmap_sem); while (pinned < npages) { + if (!locked) { + down_read(&mm->mmap_sem); + locked = 1; + } ret = get_user_pages_remote (work->task, mm, obj->userptr.ptr + pinned * PAGE_SIZE, npages - pinned, flags, - pvec + pinned, NULL, NULL); + pvec + pinned, NULL, &locked); if (ret < 0) break; pinned += ret; } - up_read(&mm->mmap_sem); + if (locked) + up_read(&mm->mmap_sem); mmput(mm); } } From patchwork Sun Nov 10 18:57:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236281 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8C3D015AB for ; Sun, 10 Nov 2019 18:58:32 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7468F2085B for ; Sun, 10 Nov 2019 18:58:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7468F2085B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0179D6E25F; Sun, 10 Nov 2019 18:58:32 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id DFE366E25F for ; Sun, 10 Nov 2019 18:58:29 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151887-1500050 for multiple; Sun, 10 Nov 2019 18:58:11 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:48 +0000 Message-Id: <20191110185806.17413-7-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 07/25] drm/i915: Cancel context if it hangs after it is closed X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" If we detect a hang in a closed context, just flush all of its requests and cancel any remaining execution along the context. Note that after closing the context, the last reference to the context may be dropped, leaving it only valid under RCU. Signed-off-by: Chris Wilson Reviewed-by: Mika Kuoppala --- drivers/gpu/drm/i915/gt/intel_reset.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/intel_reset.c b/drivers/gpu/drm/i915/gt/intel_reset.c index f03e000051c1..a6b0d00c3a51 100644 --- a/drivers/gpu/drm/i915/gt/intel_reset.c +++ b/drivers/gpu/drm/i915/gt/intel_reset.c @@ -81,6 +81,11 @@ static bool context_mark_guilty(struct i915_gem_context *ctx) bool banned; int i; + if (i915_gem_context_is_closed(ctx)) { + i915_gem_context_set_banned(ctx); + return true; + } + atomic_inc(&ctx->guilty_count); /* Cool contexts are too cool to be banned! (Used for reset testing.) */ @@ -124,6 +129,7 @@ void __i915_request_reset(struct i915_request *rq, bool guilty) GEM_BUG_ON(i915_request_completed(rq)); + rcu_read_lock(); /* protect the GEM context */ if (guilty) { i915_request_skip(rq, -EIO); if (context_mark_guilty(rq->gem_context)) @@ -132,6 +138,7 @@ void __i915_request_reset(struct i915_request *rq, bool guilty) dma_fence_set_error(&rq->fence, -EAGAIN); context_mark_innocent(rq->gem_context); } + rcu_read_unlock(); } static bool i915_in_reset(struct pci_dev *pdev) From patchwork Sun Nov 10 18:57:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236317 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 55E9D14E5 for ; Sun, 10 Nov 2019 18:58:52 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3D4E82085B for ; Sun, 10 Nov 2019 18:58:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3D4E82085B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C4C146E7D2; Sun, 10 Nov 2019 18:58:51 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0E5476E7AB for ; Sun, 10 Nov 2019 18:58:27 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151888-1500050 for multiple; Sun, 10 Nov 2019 18:58:11 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:49 +0000 Message-Id: <20191110185806.17413-8-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 08/25] drm/i915: Show guilty context name on GPU reset X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" We mention that we are resetting the GPU, and dump the device state for post mortem debugging. However, while that dump contains the active processes and the one flagged as causing the error, we do not always include that information in dmesg. Include the name of the guilty process in dmesg for reference. Signed-off-by: Chris Wilson Cc: Mika Kuoppala Cc: Tvrtko Ursulin Reviewed-by: Mika Kuoppala --- drivers/gpu/drm/i915/gt/intel_reset.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/intel_reset.c b/drivers/gpu/drm/i915/gt/intel_reset.c index a6b0d00c3a51..b7007cd78c6f 100644 --- a/drivers/gpu/drm/i915/gt/intel_reset.c +++ b/drivers/gpu/drm/i915/gt/intel_reset.c @@ -92,6 +92,10 @@ static bool context_mark_guilty(struct i915_gem_context *ctx) if (!i915_gem_context_is_bannable(ctx)) return false; + dev_notice(ctx->i915->drm.dev, + "%s context reset due to GPU hang\n", + ctx->name); + /* Record the timestamp for the last N hangs */ prev_hang = ctx->hang_timestamp[0]; for (i = 0; i < ARRAY_SIZE(ctx->hang_timestamp) - 1; i++) From patchwork Sun Nov 10 18:57:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236313 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 43F9E15AB for ; Sun, 10 Nov 2019 18:58:47 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2BB5120842 for ; Sun, 10 Nov 2019 18:58:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2BB5120842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 472B56E7BE; Sun, 10 Nov 2019 18:58:36 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id E2FFC6E7A2 for ; Sun, 10 Nov 2019 18:58:27 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151889-1500050 for multiple; Sun, 10 Nov 2019 18:58:11 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:50 +0000 Message-Id: <20191110185806.17413-9-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 09/25] drm/i915/icl: Refine PG_HYSTERESIS X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" After doing some measuring, Icelake behaves on a par with Broadwell, and without having to compromise for low power cores with long latencies, we can reduce the powergating hysteresis so that the powersaving is enabled faster. No impact observed on client side throughput measures (so negligible increase in extra switching), and inspection from high frequency polling using igt/gem_exec_nop/sequential, provided an estimate for the upper bound before we can measure a substantial impact on latency. Signed-off-by: Chris Wilson Cc: Mika Kuoppala Reviewed-by: Mika Kuoppala --- drivers/gpu/drm/i915/gt/intel_rc6.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_rc6.c b/drivers/gpu/drm/i915/gt/intel_rc6.c index 5ad4a92a9582..5e3ba034bd46 100644 --- a/drivers/gpu/drm/i915/gt/intel_rc6.c +++ b/drivers/gpu/drm/i915/gt/intel_rc6.c @@ -88,15 +88,12 @@ static void gen11_rc6_enable(struct intel_rc6 *rc6) * do not want the enable hysteresis to less than the wakeup latency. * * igt/gem_exec_nop/sequential provides a rough estimate for the - * service latency, and puts it around 10us for Broadwell (and other - * big core) and around 40us for Broxton (and other low power cores). - * [Note that for legacy ringbuffer submission, this is less than 1us!] - * However, the wakeup latency on Broxton is closer to 100us. To be - * conservative, we have to factor in a context switch on top (due - * to ksoftirqd). + * service latency, and puts it under 10us for Icelake, similar to + * Broadwell+, To be conservative, we want to factor in a context + * switch on top (due to ksoftirqd). */ - set(uncore, GEN9_MEDIA_PG_IDLE_HYSTERESIS, 250); - set(uncore, GEN9_RENDER_PG_IDLE_HYSTERESIS, 250); + set(uncore, GEN9_MEDIA_PG_IDLE_HYSTERESIS, 60); + set(uncore, GEN9_RENDER_PG_IDLE_HYSTERESIS, 60); /* 3a: Enable RC6 */ set(uncore, GEN6_RC_CONTROL, From patchwork Sun Nov 10 18:57:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236319 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D424015AB for ; Sun, 10 Nov 2019 18:58:53 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BBCF22085B for ; Sun, 10 Nov 2019 18:58:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BBCF22085B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 363F46E7DC; Sun, 10 Nov 2019 18:58:53 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id E4F076E7A3 for ; Sun, 10 Nov 2019 18:58:27 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151890-1500050 for multiple; Sun, 10 Nov 2019 18:58:11 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:51 +0000 Message-Id: <20191110185806.17413-10-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 10/25] drm/i915/execlists: Reduce barrier on context switch to a wmb() X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Having been forced to reduce Braswell back to using the aliasing ppgtt, the coherency issue we previously observed cannot impact us. Reduce the performance penalty imposed on all platforms from using the mfence to a mere sfence. References: cf66b8a0ba14 ("drm/i915/execlists: Apply a full mb before execution for Braswell") Signed-off-by: Chris Wilson Cc: Mika Kuoppala Cc: Joonas Lahtinen Reviewed-by: Joonas Lahtinen --- drivers/gpu/drm/i915/gt/intel_lrc.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c index a06074a26fb5..4b6d9e6b1bfd 100644 --- a/drivers/gpu/drm/i915/gt/intel_lrc.c +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c @@ -1218,13 +1218,8 @@ static u64 execlists_update_context(const struct i915_request *rq) * may not be visible to the HW prior to the completion of the UC * register write and that we may begin execution from the context * before its image is complete leading to invalid PD chasing. - * - * Furthermore, Braswell, at least, wants a full mb to be sure that - * the writes are coherent in memory (visible to the GPU) prior to - * execution, and not just visible to other CPUs (as is the result of - * wmb). */ - mb(); + wmb(); desc = ce->lrc_desc; ce->lrc_desc &= ~CTX_DESC_FORCE_RESTORE; From patchwork Sun Nov 10 18:57:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236291 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0DEB414E5 for ; Sun, 10 Nov 2019 18:58:39 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E84F820842 for ; Sun, 10 Nov 2019 18:58:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E84F820842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 172736E7AC; Sun, 10 Nov 2019 18:58:33 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id E15A56E7B0 for ; Sun, 10 Nov 2019 18:58:28 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151891-1500050 for multiple; Sun, 10 Nov 2019 18:58:11 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:52 +0000 Message-Id: <20191110185806.17413-11-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 11/25] drm/i915/gem: Silence sparse for RCU protection inside the constructor X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Inside the constructor, while cloning, we need to replace the dst->engines. Having forgotten that dst->engines is marked as RCU protected, we need to add the appropriate annotations to make sparse happy. Signed-off-by: Chris Wilson Cc: Tvrtko Ursulin --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index 72d389afa28a..907c2ec30b98 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -2003,7 +2003,8 @@ static int clone_engines(struct i915_gem_context *dst, user_engines = i915_gem_context_user_engines(src); i915_gem_context_unlock_engines(src); - free_engines(dst->engines); + /* Serialised by constructor */ + free_engines(__context_engines_static(dst)); RCU_INIT_POINTER(dst->engines, clone); if (user_engines) i915_gem_context_set_user_engines(dst); @@ -2038,7 +2039,8 @@ static int clone_sseu(struct i915_gem_context *dst, unsigned long n; int err; - clone = dst->engines; /* no locking required; sole access */ + /* no locking required; sole access under constructor*/ + clone = __context_engines_static(dst); if (e->num_engines != clone->num_engines) { err = -EINVAL; goto unlock; From patchwork Sun Nov 10 18:57:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236323 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4E44514E5 for ; Sun, 10 Nov 2019 18:58:55 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 365BE20842 for ; Sun, 10 Nov 2019 18:58:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 365BE20842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id CF3E06E7DB; Sun, 10 Nov 2019 18:58:53 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id 181AF6E7B2 for ; Sun, 10 Nov 2019 18:58:28 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151892-1500050 for multiple; Sun, 10 Nov 2019 18:58:12 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:53 +0000 Message-Id: <20191110185806.17413-12-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 12/25] drm/i915/userptr: Try to acquire the page lock around set_page_dirty() X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: stable@vger.kernel.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" set_page_dirty says: For pages with a mapping this should be done under the page lock for the benefit of asynchronous memory errors who prefer a consistent dirty state. This rule can be broken in some special cases, but should be better not to. Under those rules, it is only safe for us to use the plain set_page_dirty calls for shmemfs/anonymous memory. Userptr may be used with real mappings and so needs to use the locked version (set_page_dirty_lock). However, following a try_to_unmap() we may want to remove the userptr and so call put_pages(). However, try_to_unmap() acquires the page lock and so we must avoid recursively locking the pages ourselves -- which means that we cannot safely acquire the lock around set_page_dirty(). Since we can't be sure of the lock, we have to risk skip dirtying the page, or else risk calling set_page_dirty() without a lock and so risk fs corruption. Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=203317 Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=112012 Fixes: 5cc9ed4b9a7a ("drm/i915: Introduce mapping of user pages into video m References: cb6d7c7dc7ff ("drm/i915/userptr: Acquire the page lock around set_page_dirty()") References: 505a8ec7e11a ("Revert "drm/i915/userptr: Acquire the page lock around set_page_dirty()"") References: 6dcc693bc57f ("ext4: warn when page is dirtied without buffers") Signed-off-by: Chris Wilson Cc: Lionel Landwerlin Cc: Tvrtko Ursulin Cc: Joonas Lahtinen Cc: stable@vger.kernel.org --- drivers/gpu/drm/i915/gem/i915_gem_userptr.c | 22 ++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c b/drivers/gpu/drm/i915/gem/i915_gem_userptr.c index 5e87126def2f..54ebc7ab71bc 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_userptr.c @@ -651,8 +651,28 @@ i915_gem_userptr_put_pages(struct drm_i915_gem_object *obj, obj->mm.dirty = false; for_each_sgt_page(page, sgt_iter, pages) { - if (obj->mm.dirty) + if (obj->mm.dirty && trylock_page(page)) { + /* + * As this may not be anonymous memory (e.g. shmem) + * but exist on a real mapping, we have to lock + * the page in order to dirty it -- holding + * the page reference is not sufficient to + * prevent the inode from being truncated. + * Play safe and take the lock. + * + * However...! + * + * The mmu-notifier can be invalidated for a + * migrate_page, that is alreadying holding the lock + * on the page. Such a try_to_unmap() will result + * in us calling put_pages() and so recursively try + * to lock the page. We avoid that deadlock with + * a trylock_page() and in exchange we risk missing + * some page dirtying. + */ set_page_dirty(page); + unlock_page(page); + } mark_page_accessed(page); put_page(page); From patchwork Sun Nov 10 18:57:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236285 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AC10814E5 for ; Sun, 10 Nov 2019 18:58:34 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 936D020842 for ; Sun, 10 Nov 2019 18:58:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 936D020842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BBCC86E7A3; Sun, 10 Nov 2019 18:58:32 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id DB20B6E25F for ; Sun, 10 Nov 2019 18:58:27 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151893-1500050 for multiple; Sun, 10 Nov 2019 18:58:12 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:54 +0000 Message-Id: <20191110185806.17413-13-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 13/25] drm/i915: Taint the kernel on dumping the GEM ftrace buffer X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" As the ftrace buffer is single shot, once dumped it will not update. As such, it only provides information for the first bug and all subsequent bugs are noise. The goal of CI is to have zero bugs, so taint the kernel causing CI to reboot the machine; fix the bug and move on. Signed-off-by: Chris Wilson Cc: Mika Kuoppala Cc: Tvrtko Ursulin Reviewed-by: Joonas Lahtinen --- drivers/gpu/drm/i915/i915_gem.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_gem.h b/drivers/gpu/drm/i915/i915_gem.h index f6f9675848b8..36a8c7673c4f 100644 --- a/drivers/gpu/drm/i915/i915_gem.h +++ b/drivers/gpu/drm/i915/i915_gem.h @@ -68,9 +68,10 @@ struct drm_i915_private; pr_err(__VA_ARGS__); \ trace_printk(__VA_ARGS__); \ } while (0) -#define GEM_TRACE_DUMP() ftrace_dump(DUMP_ALL) +#define GEM_TRACE_DUMP() \ + do { ftrace_dump(DUMP_ALL); add_taint_for_CI(TAINT_WARN); } while (0) #define GEM_TRACE_DUMP_ON(expr) \ - do { if (expr) ftrace_dump(DUMP_ALL); } while (0) + do { if (expr) GEM_TRACE_DUMP(); } while (0) #else #define GEM_TRACE(...) do { } while (0) #define GEM_TRACE_ERR(...) do { } while (0) From patchwork Sun Nov 10 18:57:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236289 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3071E15AB for ; Sun, 10 Nov 2019 18:58:38 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 17D942085B for ; Sun, 10 Nov 2019 18:58:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 17D942085B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B40016E51A; Sun, 10 Nov 2019 18:58:32 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id D691E6E0C8 for ; Sun, 10 Nov 2019 18:58:27 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151894-1500050 for multiple; Sun, 10 Nov 2019 18:58:12 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:55 +0000 Message-Id: <20191110185806.17413-14-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 14/25] drm/i915/selftests: Exercise parallel blit operations on a single ctx X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Matthew Auld Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Make sure that our code is robust enough to handle multiple threads trying to clear objects for a single client context. This brings the joy of a shared GGTT to all! References: https://bugs.freedesktop.org/show_bug.cgi?id=112176 Signed-off-by: Chris Wilson Cc: Matthew Auld Reviewed-by: Matthew Auld --- .../i915/gem/selftests/i915_gem_object_blt.c | 93 ++++++++++++------- 1 file changed, 57 insertions(+), 36 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_object_blt.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_object_blt.c index dd43ea0c9025..8a0fba7d5d4f 100644 --- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_object_blt.c +++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_object_blt.c @@ -186,6 +186,8 @@ static int perf_copy_blt(void *arg) struct igt_thread_arg { struct drm_i915_private *i915; + struct i915_gem_context *ctx; + struct file *file; struct rnd_state prng; unsigned int n_cpus; }; @@ -198,24 +200,20 @@ static int igt_fill_blt_thread(void *arg) struct drm_i915_gem_object *obj; struct i915_gem_context *ctx; struct intel_context *ce; - struct file *file; unsigned int prio; IGT_TIMEOUT(end); int err; - file = mock_file(i915); - if (IS_ERR(file)) - return PTR_ERR(file); + ctx = thread->ctx; + if (!ctx) { + ctx = live_context(i915, thread->file); + if (IS_ERR(ctx)) + return PTR_ERR(ctx); - ctx = live_context(i915, file); - if (IS_ERR(ctx)) { - err = PTR_ERR(ctx); - goto out_file; + prio = i915_prandom_u32_max_state(I915_PRIORITY_MAX, prng); + ctx->sched.priority = I915_USER_PRIORITY(prio); } - prio = i915_prandom_u32_max_state(I915_PRIORITY_MAX, prng); - ctx->sched.priority = I915_USER_PRIORITY(prio); - ce = i915_gem_context_get_engine(ctx, BCS0); GEM_BUG_ON(IS_ERR(ce)); @@ -300,8 +298,6 @@ static int igt_fill_blt_thread(void *arg) err = 0; intel_context_put(ce); -out_file: - fput(file); return err; } @@ -313,24 +309,20 @@ static int igt_copy_blt_thread(void *arg) struct drm_i915_gem_object *src, *dst; struct i915_gem_context *ctx; struct intel_context *ce; - struct file *file; unsigned int prio; IGT_TIMEOUT(end); int err; - file = mock_file(i915); - if (IS_ERR(file)) - return PTR_ERR(file); + ctx = thread->ctx; + if (!ctx) { + ctx = live_context(i915, thread->file); + if (IS_ERR(ctx)) + return PTR_ERR(ctx); - ctx = live_context(i915, file); - if (IS_ERR(ctx)) { - err = PTR_ERR(ctx); - goto out_file; + prio = i915_prandom_u32_max_state(I915_PRIORITY_MAX, prng); + ctx->sched.priority = I915_USER_PRIORITY(prio); } - prio = i915_prandom_u32_max_state(I915_PRIORITY_MAX, prng); - ctx->sched.priority = I915_USER_PRIORITY(prio); - ce = i915_gem_context_get_engine(ctx, BCS0); GEM_BUG_ON(IS_ERR(ce)); @@ -431,19 +423,19 @@ static int igt_copy_blt_thread(void *arg) err = 0; intel_context_put(ce); -out_file: - fput(file); return err; } static int igt_threaded_blt(struct drm_i915_private *i915, - int (*blt_fn)(void *arg)) + int (*blt_fn)(void *arg), + unsigned int flags) +#define SINGLE_CTX BIT(0) { struct igt_thread_arg *thread; struct task_struct **tsk; + unsigned int n_cpus, i; I915_RND_STATE(prng); - unsigned int n_cpus; - unsigned int i; + struct file *file; int err = 0; n_cpus = num_online_cpus() + 1; @@ -453,13 +445,27 @@ static int igt_threaded_blt(struct drm_i915_private *i915, return 0; thread = kcalloc(n_cpus, sizeof(struct igt_thread_arg), GFP_KERNEL); - if (!thread) { - kfree(tsk); - return 0; + if (!thread) + goto out_tsk; + + thread[0].file = mock_file(i915); + if (IS_ERR(thread[0].file)) { + err = PTR_ERR(file); + goto out_thread; + } + + if (flags & SINGLE_CTX) { + thread[0].ctx = live_context(i915, thread[0].file); + if (IS_ERR(thread[0].ctx)) { + err = PTR_ERR(thread[0].ctx); + goto out_file; + } } for (i = 0; i < n_cpus; ++i) { thread[i].i915 = i915; + thread[i].file = thread[0].file; + thread[i].ctx = thread[0].ctx; thread[i].n_cpus = n_cpus; thread[i].prng = I915_RND_STATE_INITIALIZER(prandom_u32_state(&prng)); @@ -488,20 +494,33 @@ static int igt_threaded_blt(struct drm_i915_private *i915, put_task_struct(tsk[i]); } - kfree(tsk); +out_file: + fput(thread[0].file); +out_thread: kfree(thread); - +out_tsk: + kfree(tsk); return err; } static int igt_fill_blt(void *arg) { - return igt_threaded_blt(arg, igt_fill_blt_thread); + return igt_threaded_blt(arg, igt_fill_blt_thread, 0); +} + +static int igt_fill_blt_ctx0(void *arg) +{ + return igt_threaded_blt(arg, igt_fill_blt_thread, SINGLE_CTX); } static int igt_copy_blt(void *arg) { - return igt_threaded_blt(arg, igt_copy_blt_thread); + return igt_threaded_blt(arg, igt_copy_blt_thread, 0); +} + +static int igt_copy_blt_ctx0(void *arg) +{ + return igt_threaded_blt(arg, igt_copy_blt_thread, SINGLE_CTX); } int i915_gem_object_blt_live_selftests(struct drm_i915_private *i915) @@ -510,7 +529,9 @@ int i915_gem_object_blt_live_selftests(struct drm_i915_private *i915) SUBTEST(perf_fill_blt), SUBTEST(perf_copy_blt), SUBTEST(igt_fill_blt), + SUBTEST(igt_fill_blt_ctx0), SUBTEST(igt_copy_blt), + SUBTEST(igt_copy_blt_ctx0), }; if (intel_gt_is_wedged(&i915->gt)) From patchwork Sun Nov 10 18:57:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236287 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9502715AB for ; Sun, 10 Nov 2019 18:58:35 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7CC3320842 for ; Sun, 10 Nov 2019 18:58:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7CC3320842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B5E366E7A2; Sun, 10 Nov 2019 18:58:32 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id DDA476E7AF for ; Sun, 10 Nov 2019 18:58:28 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151895-1500050 for multiple; Sun, 10 Nov 2019 18:58:12 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:56 +0000 Message-Id: <20191110185806.17413-15-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 15/25] drm/i915/selftests: Perform some basic cycle counting of MI ops X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anna Karas Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Some basic information that is useful to know, such as how many cycles is a MI_NOOP. Signed-off-by: Chris Wilson Cc: Anna Karas Cc: Tvrtko Ursulin Reviewed-by: Matthew Auld --- .../i915/gem/selftests/i915_gem_object_blt.c | 15 +- drivers/gpu/drm/i915/gt/selftest_engine_cs.c | 339 +++++++++++++++++- drivers/gpu/drm/i915/i915_pci.c | 6 + drivers/gpu/drm/i915/i915_selftest.h | 4 + .../drm/i915/selftests/i915_perf_selftests.h | 13 + .../gpu/drm/i915/selftests/i915_selftest.c | 43 +++ 6 files changed, 417 insertions(+), 3 deletions(-) create mode 100644 drivers/gpu/drm/i915/selftests/i915_perf_selftests.h diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_object_blt.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_object_blt.c index 8a0fba7d5d4f..c8a8c07e07ab 100644 --- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_object_blt.c +++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_object_blt.c @@ -526,8 +526,6 @@ static int igt_copy_blt_ctx0(void *arg) int i915_gem_object_blt_live_selftests(struct drm_i915_private *i915) { static const struct i915_subtest tests[] = { - SUBTEST(perf_fill_blt), - SUBTEST(perf_copy_blt), SUBTEST(igt_fill_blt), SUBTEST(igt_fill_blt_ctx0), SUBTEST(igt_copy_blt), @@ -542,3 +540,16 @@ int i915_gem_object_blt_live_selftests(struct drm_i915_private *i915) return i915_live_subtests(tests, i915); } + +int i915_gem_object_blt_perf_selftests(struct drm_i915_private *i915) +{ + static const struct i915_subtest tests[] = { + SUBTEST(perf_fill_blt), + SUBTEST(perf_copy_blt), + }; + + if (intel_gt_is_wedged(&i915->gt)) + return 0; + + return i915_live_subtests(tests, i915); +} diff --git a/drivers/gpu/drm/i915/gt/selftest_engine_cs.c b/drivers/gpu/drm/i915/gt/selftest_engine_cs.c index 3880f07c29b8..0ad831f6a25a 100644 --- a/drivers/gpu/drm/i915/gt/selftest_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/selftest_engine_cs.c @@ -4,7 +4,344 @@ * Copyright © 2018 Intel Corporation */ -#include "../i915_selftest.h" +#include + +#include "intel_gt_pm.h" +#include "intel_rps.h" + +#include "i915_selftest.h" +#include "selftests/igt_flush_test.h" + +#define COUNT 5 + +static int cmp_u32(const void *A, const void *B) +{ + const u32 *a = A, *b = B; + + return *a - *b; +} + +static void perf_begin(struct intel_gt *gt) +{ + intel_gt_pm_get(gt); + + /* Boost gpufreq to max [waitboost] and keep it fixed */ + atomic_inc(>->rps.num_waiters); + schedule_work(>->rps.work); + flush_work(>->rps.work); +} + +static int perf_end(struct intel_gt *gt) +{ + atomic_dec(>->rps.num_waiters); + intel_gt_pm_put(gt); + + return igt_flush_test(gt->i915); +} + +static int write_timestamp(struct i915_request *rq, int slot) +{ + u32 cmd; + u32 *cs; + + cs = intel_ring_begin(rq, 4); + if (IS_ERR(cs)) + return PTR_ERR(cs); + + cmd = MI_STORE_REGISTER_MEM | MI_USE_GGTT; + if (INTEL_GEN(rq->i915) >= 8) + cmd++; + *cs++ = cmd; + *cs++ = i915_mmio_reg_offset(RING_TIMESTAMP(rq->engine->mmio_base)); + *cs++ = i915_request_timeline(rq)->hwsp_offset + slot * sizeof(u32); + *cs++ = 0; + + intel_ring_advance(rq, cs); + + return 0; +} + +static struct i915_vma *create_empty_batch(struct intel_context *ce) +{ + struct drm_i915_gem_object *obj; + struct i915_vma *vma; + u32 *cs; + int err; + + obj = i915_gem_object_create_internal(ce->engine->i915, PAGE_SIZE); + if (IS_ERR(obj)) + return ERR_CAST(obj); + + cs = i915_gem_object_pin_map(obj, I915_MAP_WB); + if (IS_ERR(cs)) { + i915_gem_object_put(obj); + return ERR_CAST(cs); + } + + cs[0] = MI_BATCH_BUFFER_END; + + i915_gem_object_flush_map(obj); + i915_gem_object_unpin_map(obj); + + vma = i915_vma_instance(obj, ce->vm, NULL); + if (IS_ERR(vma)) { + i915_gem_object_put(obj); + return vma; + } + + err = i915_vma_pin(vma, 0, 0, PIN_USER); + if (err) { + i915_gem_object_put(obj); + return ERR_PTR(err); + } + + return vma; +} + +static u32 trifilter(u32 *a) +{ + u64 sum; + + sort(a, COUNT, sizeof(*a), cmp_u32, NULL); + + sum += mul_u32_u32(a[2], 2); + sum += a[1]; + sum += a[3]; + + return sum >> 2; +} + +static int perf_mi_bb_start(void *arg) +{ + struct intel_gt *gt = arg; + struct intel_engine_cs *engine; + enum intel_engine_id id; + int err = 0; + + if (INTEL_GEN(gt->i915) < 7) /* for per-engine CS_TIMESTAMP */ + return 0; + + perf_begin(gt); + for_each_engine(engine, gt, id) { + struct intel_context *ce = engine->kernel_context; + struct i915_vma *batch; + u32 cycles[COUNT]; + int i; + + batch = create_empty_batch(ce); + if (IS_ERR(batch)) { + err = PTR_ERR(batch); + break; + } + + err = i915_vma_sync(batch); + if (err) { + i915_vma_put(batch); + break; + } + + for (i = 0; i < ARRAY_SIZE(cycles); i++) { + struct i915_request *rq; + + rq = i915_request_create(ce); + if (IS_ERR(rq)) { + err = PTR_ERR(rq); + break; + } + + err = write_timestamp(rq, 2); + if (err) + goto out; + + err = rq->engine->emit_bb_start(rq, + batch->node.start, 8, + 0); + if (err) + goto out; + + err = write_timestamp(rq, 3); + if (err) + goto out; + +out: + i915_request_get(rq); + i915_request_add(rq); + + if (i915_request_wait(rq, 0, HZ / 5) < 0) + err = -EIO; + i915_request_put(rq); + if (err) + break; + + cycles[i] = rq->hwsp_seqno[3] - rq->hwsp_seqno[2]; + } + i915_vma_put(batch); + if (err) + break; + + pr_info("%s: MI_BB_START cycles: %u\n", + engine->name, trifilter(cycles)); + } + if (perf_end(gt)) + err = -EIO; + + return err; +} + +static struct i915_vma *create_nop_batch(struct intel_context *ce) +{ + struct drm_i915_gem_object *obj; + struct i915_vma *vma; + u32 *cs; + int err; + + obj = i915_gem_object_create_internal(ce->engine->i915, SZ_64K); + if (IS_ERR(obj)) + return ERR_CAST(obj); + + cs = i915_gem_object_pin_map(obj, I915_MAP_WB); + if (IS_ERR(cs)) { + i915_gem_object_put(obj); + return ERR_CAST(cs); + } + + memset(cs, 0, SZ_64K); + cs[SZ_64K / sizeof(*cs) - 1] = MI_BATCH_BUFFER_END; + + i915_gem_object_flush_map(obj); + i915_gem_object_unpin_map(obj); + + vma = i915_vma_instance(obj, ce->vm, NULL); + if (IS_ERR(vma)) { + i915_gem_object_put(obj); + return vma; + } + + err = i915_vma_pin(vma, 0, 0, PIN_USER); + if (err) { + i915_gem_object_put(obj); + return ERR_PTR(err); + } + + return vma; +} + +static int perf_mi_noop(void *arg) +{ + struct intel_gt *gt = arg; + struct intel_engine_cs *engine; + enum intel_engine_id id; + int err = 0; + + if (INTEL_GEN(gt->i915) < 7) /* for per-engine CS_TIMESTAMP */ + return 0; + + perf_begin(gt); + for_each_engine(engine, gt, id) { + struct intel_context *ce = engine->kernel_context; + struct i915_vma *base, *nop; + u32 cycles[COUNT]; + int i; + + base = create_empty_batch(ce); + if (IS_ERR(base)) { + err = PTR_ERR(base); + break; + } + + err = i915_vma_sync(base); + if (err) { + i915_vma_put(base); + break; + } + + nop = create_nop_batch(ce); + if (IS_ERR(nop)) { + err = PTR_ERR(nop); + i915_vma_put(base); + break; + } + + err = i915_vma_sync(nop); + if (err) { + i915_vma_put(nop); + i915_vma_put(base); + break; + } + + for (i = 0; i < ARRAY_SIZE(cycles); i++) { + struct i915_request *rq; + + rq = i915_request_create(ce); + if (IS_ERR(rq)) { + err = PTR_ERR(rq); + break; + } + + err = write_timestamp(rq, 2); + if (err) + goto out; + + err = rq->engine->emit_bb_start(rq, + base->node.start, 8, + 0); + if (err) + goto out; + + err = write_timestamp(rq, 3); + if (err) + goto out; + + err = rq->engine->emit_bb_start(rq, + nop->node.start, 4096, + 0); + if (err) + goto out; + + err = write_timestamp(rq, 4); + if (err) + goto out; + +out: + i915_request_get(rq); + i915_request_add(rq); + + if (i915_request_wait(rq, 0, HZ / 5) < 0) + err = -EIO; + i915_request_put(rq); + if (err) + break; + + cycles[i] = + (rq->hwsp_seqno[4] - rq->hwsp_seqno[3]) - + (rq->hwsp_seqno[3] - rq->hwsp_seqno[2]); + } + i915_vma_put(nop); + i915_vma_put(base); + if (err) + break; + + pr_info("%s: 16K MI_NOOP cycles: %u\n", + engine->name, trifilter(cycles)); + } + if (perf_end(gt)) + err = -EIO; + + return err; +} + +int intel_engine_cs_perf_selftests(struct drm_i915_private *i915) +{ + static const struct i915_subtest tests[] = { + SUBTEST(perf_mi_bb_start), + SUBTEST(perf_mi_noop), + }; + + if (intel_gt_is_wedged(&i915->gt)) + return 0; + + return intel_gt_live_subtests(tests, &i915->gt); +} static int intel_mmio_bases_check(void *arg) { diff --git a/drivers/gpu/drm/i915/i915_pci.c b/drivers/gpu/drm/i915/i915_pci.c index 1bb701d32a5d..da3e9b5752ac 100644 --- a/drivers/gpu/drm/i915/i915_pci.c +++ b/drivers/gpu/drm/i915/i915_pci.c @@ -1003,6 +1003,12 @@ static int i915_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) return err > 0 ? -ENOTTY : err; } + err = i915_perf_selftests(pdev); + if (err) { + i915_pci_remove(pdev); + return err > 0 ? -ENOTTY : err; + } + return 0; } diff --git a/drivers/gpu/drm/i915/i915_selftest.h b/drivers/gpu/drm/i915/i915_selftest.h index 4d88205de51b..98bcb6fa0ab4 100644 --- a/drivers/gpu/drm/i915/i915_selftest.h +++ b/drivers/gpu/drm/i915/i915_selftest.h @@ -36,6 +36,7 @@ struct i915_selftest { char *filter; int mock; int live; + int perf; }; #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST) @@ -45,6 +46,7 @@ extern struct i915_selftest i915_selftest; int i915_mock_selftests(void); int i915_live_selftests(struct pci_dev *pdev); +int i915_perf_selftests(struct pci_dev *pdev); /* We extract the function declarations from i915_mock_selftests.h and * i915_live_selftests.h Add your unit test declarations there! @@ -61,6 +63,7 @@ int i915_live_selftests(struct pci_dev *pdev); #undef selftest #define selftest(name, func) int func(struct drm_i915_private *i915); #include "selftests/i915_live_selftests.h" +#include "selftests/i915_perf_selftests.h" #undef selftest struct i915_subtest { @@ -109,6 +112,7 @@ int __i915_subtests(const char *caller, static inline int i915_mock_selftests(void) { return 0; } static inline int i915_live_selftests(struct pci_dev *pdev) { return 0; } +static inline int i915_perf_selftests(struct pci_dev *pdev) { return 0; } #define I915_SELFTEST_DECLARE(x) #define I915_SELFTEST_ONLY(x) 0 diff --git a/drivers/gpu/drm/i915/selftests/i915_perf_selftests.h b/drivers/gpu/drm/i915/selftests/i915_perf_selftests.h new file mode 100644 index 000000000000..f7129a243daa --- /dev/null +++ b/drivers/gpu/drm/i915/selftests/i915_perf_selftests.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* List each unit test as selftest(name, function) + * + * The name is used as both an enum and expanded as subtest__name to create + * a module parameter. It must be unique and legal for a C identifier. + * + * The function should be of type int function(void). It may be conditionally + * compiled using #if IS_ENABLED(DRM_I915_SELFTEST). + * + * Tests are executed in order by igt/i915_selftest + */ +selftest(engine_cs, intel_engine_cs_perf_selftests) +selftest(blt, i915_gem_object_blt_perf_selftests) diff --git a/drivers/gpu/drm/i915/selftests/i915_selftest.c b/drivers/gpu/drm/i915/selftests/i915_selftest.c index a6cca4ad96f6..d3bf9eefb682 100644 --- a/drivers/gpu/drm/i915/selftests/i915_selftest.c +++ b/drivers/gpu/drm/i915/selftests/i915_selftest.c @@ -57,6 +57,12 @@ enum { #undef selftest }; +enum { +#define selftest(name, func) perf_##name, +#include "i915_perf_selftests.h" +#undef selftest +}; + struct selftest { bool enabled; const char *name; @@ -78,6 +84,12 @@ static struct selftest live_selftests[] = { }; #undef selftest +#define selftest(n, f) [perf_##n] = { .name = #n, { .live = f } }, +static struct selftest perf_selftests[] = { +#include "i915_perf_selftests.h" +}; +#undef selftest + /* Embed the line number into the parameter name so that we can order tests */ #define selftest(n, func) selftest_0(n, func, param(n)) #define param(n) __PASTE(igt__, __PASTE(__LINE__, __mock_##n)) @@ -93,6 +105,13 @@ module_param_named(id, live_selftests[live_##n].enabled, bool, 0400); #include "i915_live_selftests.h" #undef selftest_0 #undef param + +#define param(n) __PASTE(igt__, __PASTE(__LINE__, __perf_##n)) +#define selftest_0(n, func, id) \ +module_param_named(id, perf_selftests[perf_##n].enabled, bool, 0400); +#include "i915_perf_selftests.h" +#undef selftest_0 +#undef param #undef selftest static void set_default_test_all(struct selftest *st, unsigned int count) @@ -200,6 +219,27 @@ int i915_live_selftests(struct pci_dev *pdev) return 0; } +int i915_perf_selftests(struct pci_dev *pdev) +{ + int err; + + if (!i915_selftest.perf) + return 0; + + err = run_selftests(perf, pdev_to_i915(pdev)); + if (err) { + i915_selftest.perf = err; + return err; + } + + if (i915_selftest.perf < 0) { + i915_selftest.perf = -ENOTTY; + return 1; + } + + return 0; +} + static bool apply_subtest_filter(const char *caller, const char *name) { char *filter, *sep, *tok; @@ -365,3 +405,6 @@ MODULE_PARM_DESC(mock_selftests, "Run selftests before loading, using mock hardw module_param_named_unsafe(live_selftests, i915_selftest.live, int, 0400); MODULE_PARM_DESC(live_selftests, "Run selftests after driver initialisation on the live system (0:disabled [default], 1:run tests then continue, -1:run tests then exit module)"); + +module_param_named_unsafe(perf_selftests, i915_selftest.perf, int, 0400); +MODULE_PARM_DESC(perf_selftests, "Run performance orientated selftests after driver initialisation on the live system (0:disabled [default], 1:run tests then continue, -1:run tests then exit module)"); From patchwork Sun Nov 10 18:57:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236309 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E1E9C1850 for ; Sun, 10 Nov 2019 18:58:45 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C9D2420842 for ; Sun, 10 Nov 2019 18:58:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C9D2420842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 447C56E7BD; Sun, 10 Nov 2019 18:58:36 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id E0A066E79F for ; Sun, 10 Nov 2019 18:58:27 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151896-1500050 for multiple; Sun, 10 Nov 2019 18:58:12 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:57 +0000 Message-Id: <20191110185806.17413-16-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 16/25] drm/i915/selftests: Mock the engine sorting for easy validation X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" To make exploration of different sorting orders and presentation of the engines via the uabi easier, wrap the basic engine registration into a mock (aka standalone) selftest. Signed-off-by: Chris Wilson Cc: Tvrtko Ursulin --- drivers/gpu/drm/i915/gt/intel_engine_user.c | 4 + .../gpu/drm/i915/gt/selftest_engine_user.c | 86 +++++++++++++++++++ .../drm/i915/selftests/i915_mock_selftests.h | 3 +- 3 files changed, 92 insertions(+), 1 deletion(-) create mode 100644 drivers/gpu/drm/i915/gt/selftest_engine_user.c diff --git a/drivers/gpu/drm/i915/gt/intel_engine_user.c b/drivers/gpu/drm/i915/gt/intel_engine_user.c index 7f7150a733f4..15bb05aa1986 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_user.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_user.c @@ -293,3 +293,7 @@ unsigned int intel_engines_has_context_isolation(struct drm_i915_private *i915) return which; } + +#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST) +#include "selftest_engine_user.c" +#endif diff --git a/drivers/gpu/drm/i915/gt/selftest_engine_user.c b/drivers/gpu/drm/i915/gt/selftest_engine_user.c new file mode 100644 index 000000000000..d11cc6a4af09 --- /dev/null +++ b/drivers/gpu/drm/i915/gt/selftest_engine_user.c @@ -0,0 +1,86 @@ +/* + * SPDX-License-Identifier: MIT + * + * Copyright © 2019 Intel Corporation + */ + +#include "i915_drv.h" + +static void destroy_engines(struct drm_i915_private *i915) +{ + struct intel_engine_cs *engine, *next; + + rbtree_postorder_for_each_entry_safe(engine, next, + &i915->uabi_engines, uabi_node) + kfree(engine); +} + +static int mock_uabi_engines(void *arg) +{ + static const u8 limits[] = { + [RENDER_CLASS] = 1, + [COPY_ENGINE_CLASS] = 1, + [VIDEO_DECODE_CLASS] = I915_MAX_VCS, + [VIDEO_ENHANCEMENT_CLASS] = I915_MAX_VECS, + }; + struct intel_engine_cs *engine; + struct drm_i915_private *i915; + unsigned long num_engines; + unsigned long found; + int c, i; + int err = 0; + + i915 = kzalloc(sizeof(*i915), GFP_KERNEL); + if (!i915) + return -ENOMEM; + + num_engines = 0; + for (c = 0; c < ARRAY_SIZE(limits); c++) { + for (i = 0; i < limits[c]; i++) { + engine = kzalloc(sizeof(*engine), GFP_KERNEL); + if (!engine) + goto err; + + engine->i915 = i915; + + engine->class = c; + engine->instance = i; + + intel_engine_add_user(engine); + num_engines++; + } + } + +err: + /* Check as far as we got up to -- will explode if not quite right */ + intel_engines_driver_register(i915); + + found = 0; + for_each_uabi_engine(engine, i915) { + pr_info("%s (%d, %d) -> [%d, %d]\n", + engine->name, + engine->uabi_class, + engine->uabi_instance, + engine->class, + engine->instance); + found++; + } + if (found != num_engines) { + pr_err("Registered %lu engines; only found %lu uABI engines\n", + num_engines, found); + err = -EINVAL; + } + + destroy_engines(i915); + kfree(i915); + return err; +} + +int intel_engine_user_mock_selftests(void) +{ + static const struct i915_subtest tests[] = { + SUBTEST(mock_uabi_engines), + }; + + return i915_subtests(tests, NULL); +} diff --git a/drivers/gpu/drm/i915/selftests/i915_mock_selftests.h b/drivers/gpu/drm/i915/selftests/i915_mock_selftests.h index aa5a0e7f5d9e..9a2dd8350650 100644 --- a/drivers/gpu/drm/i915/selftests/i915_mock_selftests.h +++ b/drivers/gpu/drm/i915/selftests/i915_mock_selftests.h @@ -14,7 +14,8 @@ selftest(fence, i915_sw_fence_mock_selftests) selftest(scatterlist, scatterlist_mock_selftests) selftest(syncmap, i915_syncmap_mock_selftests) selftest(uncore, intel_uncore_mock_selftests) -selftest(engine, intel_engine_cs_mock_selftests) +selftest(engine_cs, intel_engine_cs_mock_selftests) +selftest(engine_user, intel_engine_user_mock_selftests) selftest(timelines, intel_timeline_mock_selftests) selftest(requests, i915_request_mock_selftests) selftest(objects, i915_gem_object_mock_selftests) From patchwork Sun Nov 10 18:57:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236315 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5EA1F15AB for ; Sun, 10 Nov 2019 18:58:48 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 464DB20842 for ; Sun, 10 Nov 2019 18:58:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 464DB20842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 745246E7CD; Sun, 10 Nov 2019 18:58:36 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id E22626E7A2 for ; Sun, 10 Nov 2019 18:58:29 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151897-1500050 for multiple; Sun, 10 Nov 2019 18:58:13 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:58 +0000 Message-Id: <20191110185806.17413-17-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 17/25] drm/i915/selftests: Fill all the drm_vma_manager holes X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Matthew Auld Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" To test mmap_offset_exhaustion, we first have to fill the entire vma manager leaving a single page. Don't assume that the vma manager is not already fragment, and fill all the holes. Signed-off-by: Chris Wilson Cc: Matthew Auld Reviewed-by: Matthew Auld --- .../drm/i915/gem/selftests/i915_gem_mman.c | 45 ++++++++++++++----- 1 file changed, 33 insertions(+), 12 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c index cabf45b1403f..b307574e3f3b 100644 --- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c +++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c @@ -607,28 +607,43 @@ static int igt_mmap_offset_exhaustion(void *arg) struct drm_i915_private *i915 = arg; struct drm_mm *mm = &i915->drm.vma_offset_manager->vm_addr_space_mm; struct drm_i915_gem_object *obj; - struct drm_mm_node resv, *hole; - u64 hole_start, hole_end; + struct drm_mm_node *hole, *next; int loop, err; /* Disable background reaper */ disable_retire_worker(i915); GEM_BUG_ON(!i915->gt.awake); + intel_gt_retire_requests(&i915->gt); + i915_gem_drain_freed_objects(i915); /* Trim the device mmap space to only a page */ - memset(&resv, 0, sizeof(resv)); - drm_mm_for_each_hole(hole, mm, hole_start, hole_end) { - resv.start = hole_start; - resv.size = hole_end - hole_start - 1; /* PAGE_SIZE units */ - mmap_offset_lock(i915); - err = drm_mm_reserve_node(mm, &resv); - mmap_offset_unlock(i915); + mmap_offset_lock(i915); + loop = 1; /* PAGE_SIZE units */ + list_for_each_entry_safe(hole, next, &mm->hole_stack, hole_stack) { + struct drm_mm_node *resv; + + resv = kzalloc(sizeof(*resv), GFP_NOWAIT); + if (!resv) { + err = -ENOMEM; + goto out_park; + } + + resv->start = drm_mm_hole_node_start(hole) + loop; + resv->size = hole->hole_size - loop; + resv->color = -1ul; + loop = 0; + + pr_debug("Reserving hole [%llx + %llx]\n", + resv->start, resv->size); + + err = drm_mm_reserve_node(mm, resv); if (err) { pr_err("Failed to trim VMA manager, err=%d\n", err); goto out_park; } - break; } + GEM_BUG_ON(!list_is_singular(&mm->hole_stack)); + mmap_offset_unlock(i915); /* Just fits! */ if (!assert_mmap_offset(i915, PAGE_SIZE, 0)) { @@ -685,9 +700,15 @@ static int igt_mmap_offset_exhaustion(void *arg) out: mmap_offset_lock(i915); - drm_mm_remove_node(&resv); - mmap_offset_unlock(i915); out_park: + drm_mm_for_each_node_safe(hole, next, mm) { + if (hole->color != -1ul) + continue; + + drm_mm_remove_node(hole); + kfree(hole); + } + mmap_offset_unlock(i915); restore_retire_worker(i915); return err; err_obj: From patchwork Sun Nov 10 18:57:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236279 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 353D015AB for ; Sun, 10 Nov 2019 18:58:31 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1CEC220842 for ; Sun, 10 Nov 2019 18:58:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1CEC220842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id AFBFF6E0C8; Sun, 10 Nov 2019 18:58:28 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id 07BC26E0B9 for ; Sun, 10 Nov 2019 18:58:26 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151898-1500050 for multiple; Sun, 10 Nov 2019 18:58:13 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:57:59 +0000 Message-Id: <20191110185806.17413-18-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 18/25] Revert "drm/i915: use a separate context for gpu relocs" X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Since commit c45e788d95b4 ("drm/i915/tgl: Suspend pre-parser across GTT invalidations"), we now disable the advanced preparser on Tigerlake for the invalidation phase at the start of the batch, we no longer need to emit the GPU relocations from a second context as they are now flushed inlined. References: 8a9a982767b7 ("drm/i915: use a separate context for gpu relocs") References: c45e788d95b4 ("drm/i915/tgl: Suspend pre-parser across GTT invalidations") Signed-off-by: Chris Wilson --- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 30 +------------------ 1 file changed, 1 insertion(+), 29 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c index e4f5c269150a..5dbfedeb0a6f 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c @@ -253,7 +253,6 @@ struct i915_execbuffer { bool has_fence : 1; bool needs_unfenced : 1; - struct intel_context *ce; struct i915_request *rq; u32 *rq_cmd; unsigned int rq_size; @@ -884,9 +883,6 @@ static void eb_destroy(const struct i915_execbuffer *eb) { GEM_BUG_ON(eb->reloc_cache.rq); - if (eb->reloc_cache.ce) - intel_context_put(eb->reloc_cache.ce); - if (eb->lut_size > 0) kfree(eb->buckets); } @@ -910,7 +906,6 @@ static void reloc_cache_init(struct reloc_cache *cache, cache->has_fence = cache->gen < 4; cache->needs_unfenced = INTEL_INFO(i915)->unfenced_needs_alignment; cache->node.flags = 0; - cache->ce = NULL; cache->rq = NULL; cache->rq_size = 0; } @@ -1180,7 +1175,7 @@ static int __reloc_gpu_alloc(struct i915_execbuffer *eb, if (err) goto err_unmap; - rq = intel_context_create_request(cache->ce); + rq = i915_request_create(eb->context); if (IS_ERR(rq)) { err = PTR_ERR(rq); goto err_unpin; @@ -1251,29 +1246,6 @@ static u32 *reloc_gpu(struct i915_execbuffer *eb, if (!intel_engine_can_store_dword(eb->engine)) return ERR_PTR(-ENODEV); - if (!cache->ce) { - struct intel_context *ce; - - /* - * The CS pre-parser can pre-fetch commands across - * memory sync points and starting gen12 it is able to - * pre-fetch across BB_START and BB_END boundaries - * (within the same context). We therefore use a - * separate context gen12+ to guarantee that the reloc - * writes land before the parser gets to the target - * memory location. - */ - if (cache->gen >= 12) - ce = intel_context_create(eb->context->gem_context, - eb->engine); - else - ce = intel_context_get(eb->context); - if (IS_ERR(ce)) - return ERR_CAST(ce); - - cache->ce = ce; - } - err = __reloc_gpu_alloc(eb, vma, len); if (unlikely(err)) return ERR_PTR(err); From patchwork Sun Nov 10 18:58:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236299 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ECC8215AB for ; Sun, 10 Nov 2019 18:58:41 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D4D9D20842 for ; Sun, 10 Nov 2019 18:58:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D4D9D20842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 34BBD6E7C6; Sun, 10 Nov 2019 18:58:36 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id EF4096E7AB for ; Sun, 10 Nov 2019 18:58:29 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151899-1500050 for multiple; Sun, 10 Nov 2019 18:58:13 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:58:00 +0000 Message-Id: <20191110185806.17413-19-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 19/25] drm/i915: Use a ctor for TYPESAFE_BY_RCU i915_request X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" As we start peeking into requests for longer and longer, e.g. incorporating use of spinlocks when only protected by an rcu_read_lock(), we need to be careful in how we reset the request when recycling and need to preserve any barriers that may still be in use as the request is reset for reuse. Signed-off-by: Chris Wilson --- drivers/gpu/drm/i915/i915_request.c | 35 ++++++++++++++++++--------- drivers/gpu/drm/i915/i915_scheduler.c | 6 +++++ drivers/gpu/drm/i915/i915_scheduler.h | 1 + drivers/gpu/drm/i915/i915_sw_fence.c | 8 ++++++ drivers/gpu/drm/i915/i915_sw_fence.h | 2 ++ 5 files changed, 41 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c index 00011f9533b6..f7cc13d9d0ae 100644 --- a/drivers/gpu/drm/i915/i915_request.c +++ b/drivers/gpu/drm/i915/i915_request.c @@ -214,7 +214,7 @@ static void remove_from_engine(struct i915_request *rq) spin_lock(&engine->active.lock); locked = engine; } - list_del(&rq->sched.link); + list_del_init(&rq->sched.link); spin_unlock_irq(&locked->active.lock); } @@ -586,6 +586,18 @@ request_alloc_slow(struct intel_timeline *tl, gfp_t gfp) return kmem_cache_alloc(global.slab_requests, gfp); } +static void __i915_request_ctor(void *arg) +{ + struct i915_request *rq = arg; + + spin_lock_init(&rq->lock); + i915_sched_node_init(&rq->sched); + i915_sw_fence_init(&rq->submit, submit_notify); + i915_sw_fence_init(&rq->semaphore, semaphore_notify); + + INIT_LIST_HEAD(&rq->execute_cb); +} + struct i915_request * __i915_request_create(struct intel_context *ce, gfp_t gfp) { @@ -655,15 +667,14 @@ __i915_request_create(struct intel_context *ce, gfp_t gfp) rq->rcustate = get_state_synchronize_rcu(); /* acts as smp_mb() */ - spin_lock_init(&rq->lock); dma_fence_init(&rq->fence, &i915_fence_ops, &rq->lock, tl->fence_context, seqno); /* We bump the ref for the fence chain */ - i915_sw_fence_init(&i915_request_get(rq)->submit, submit_notify); - i915_sw_fence_init(&i915_request_get(rq)->semaphore, semaphore_notify); + i915_sw_fence_reinit(&i915_request_get(rq)->submit); + i915_sw_fence_reinit(&i915_request_get(rq)->semaphore); - i915_sched_node_init(&rq->sched); + i915_sched_node_reinit(&rq->sched); /* No zalloc, must clear what we need by hand */ rq->file_priv = NULL; @@ -671,8 +682,6 @@ __i915_request_create(struct intel_context *ce, gfp_t gfp) rq->capture_list = NULL; rq->flags = 0; - INIT_LIST_HEAD(&rq->execute_cb); - /* * Reserve space in the ring buffer for all the commands required to * eventually emit this request. This is to guarantee that the @@ -1533,10 +1542,14 @@ static struct i915_global_request global = { { int __init i915_global_request_init(void) { - global.slab_requests = KMEM_CACHE(i915_request, - SLAB_HWCACHE_ALIGN | - SLAB_RECLAIM_ACCOUNT | - SLAB_TYPESAFE_BY_RCU); + global.slab_requests = + kmem_cache_create("i915_request", + sizeof(struct i915_request), + __alignof__(struct i915_request), + SLAB_HWCACHE_ALIGN | + SLAB_RECLAIM_ACCOUNT | + SLAB_TYPESAFE_BY_RCU, + __i915_request_ctor); if (!global.slab_requests) return -ENOMEM; diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c index 010d67f48ad9..724e96fe96e9 100644 --- a/drivers/gpu/drm/i915/i915_scheduler.c +++ b/drivers/gpu/drm/i915/i915_scheduler.c @@ -387,6 +387,10 @@ void i915_sched_node_init(struct i915_sched_node *node) INIT_LIST_HEAD(&node->signalers_list); INIT_LIST_HEAD(&node->waiters_list); INIT_LIST_HEAD(&node->link); +} + +void i915_sched_node_reinit(struct i915_sched_node *node) +{ node->attr.priority = I915_PRIORITY_INVALID; node->semaphores = 0; node->flags = 0; @@ -481,6 +485,7 @@ void i915_sched_node_fini(struct i915_sched_node *node) if (dep->flags & I915_DEPENDENCY_ALLOC) i915_dependency_free(dep); } + INIT_LIST_HEAD(&node->signalers_list); /* Remove ourselves from everyone who depends upon us */ list_for_each_entry_safe(dep, tmp, &node->waiters_list, wait_link) { @@ -491,6 +496,7 @@ void i915_sched_node_fini(struct i915_sched_node *node) if (dep->flags & I915_DEPENDENCY_ALLOC) i915_dependency_free(dep); } + INIT_LIST_HEAD(&node->waiters_list); spin_unlock_irq(&schedule_lock); } diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h index 07d243acf553..d1dc4efef77b 100644 --- a/drivers/gpu/drm/i915/i915_scheduler.h +++ b/drivers/gpu/drm/i915/i915_scheduler.h @@ -26,6 +26,7 @@ sched.link) void i915_sched_node_init(struct i915_sched_node *node); +void i915_sched_node_reinit(struct i915_sched_node *node); bool __i915_sched_node_add_dependency(struct i915_sched_node *node, struct i915_sched_node *signal, diff --git a/drivers/gpu/drm/i915/i915_sw_fence.c b/drivers/gpu/drm/i915/i915_sw_fence.c index 6a88db291252..eacc6c5ce0fd 100644 --- a/drivers/gpu/drm/i915/i915_sw_fence.c +++ b/drivers/gpu/drm/i915/i915_sw_fence.c @@ -227,6 +227,14 @@ void __i915_sw_fence_init(struct i915_sw_fence *fence, fence->flags = (unsigned long)fn; } +void i915_sw_fence_reinit(struct i915_sw_fence *fence) +{ + debug_fence_init(fence); + + atomic_set(&fence->pending, 1); + fence->error = 0; +} + void i915_sw_fence_commit(struct i915_sw_fence *fence) { debug_fence_activate(fence); diff --git a/drivers/gpu/drm/i915/i915_sw_fence.h b/drivers/gpu/drm/i915/i915_sw_fence.h index ab7d58bd0b9d..1e90d9a51bd2 100644 --- a/drivers/gpu/drm/i915/i915_sw_fence.h +++ b/drivers/gpu/drm/i915/i915_sw_fence.h @@ -54,6 +54,8 @@ do { \ __i915_sw_fence_init((fence), (fn), NULL, NULL) #endif +void i915_sw_fence_reinit(struct i915_sw_fence *fence); + #ifdef CONFIG_DRM_I915_SW_FENCE_DEBUG_OBJECTS void i915_sw_fence_fini(struct i915_sw_fence *fence); #else From patchwork Sun Nov 10 18:58:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236303 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9FC8A14E5 for ; Sun, 10 Nov 2019 18:58:43 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8746E20842 for ; Sun, 10 Nov 2019 18:58:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8746E20842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C3BFF6E7AB; Sun, 10 Nov 2019 18:58:34 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id D868A6E7AC for ; Sun, 10 Nov 2019 18:58:28 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151900-1500050 for multiple; Sun, 10 Nov 2019 18:58:13 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:58:01 +0000 Message-Id: <20191110185806.17413-20-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 20/25] drm/i915: Drop GEM context as a direct link from i915_request X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Keep the intel_context as being the primary state for i915_request, with the GEM context a backpointer from the low level state for the rarer cases we need client information. Our goal is to remove such references to clients from the backend, and leave the HW submission agnostic to client interfaces and self-contained. Signed-off-by: Chris Wilson --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 15 ++---- drivers/gpu/drm/i915/gem/i915_gem_context.h | 38 --------------- .../gpu/drm/i915/gem/i915_gem_context_types.h | 7 +-- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 8 ++-- drivers/gpu/drm/i915/gt/intel_breadcrumbs.c | 4 +- drivers/gpu/drm/i915/gt/intel_context.c | 2 +- drivers/gpu/drm/i915/gt/intel_context.h | 42 +++++++++++++++++ drivers/gpu/drm/i915/gt/intel_context_types.h | 5 +- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 6 +-- drivers/gpu/drm/i915/gt/intel_lrc.c | 47 +++++++++---------- drivers/gpu/drm/i915/gt/intel_reset.c | 40 +++++++++------- .../gpu/drm/i915/gt/intel_ring_submission.c | 10 ++-- drivers/gpu/drm/i915/gt/selftest_lrc.c | 20 ++++---- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 6 +-- drivers/gpu/drm/i915/gvt/scheduler.c | 27 +++++------ drivers/gpu/drm/i915/i915_gem.c | 6 +-- drivers/gpu/drm/i915/i915_gpu_error.c | 11 +++-- drivers/gpu/drm/i915/i915_perf.c | 4 +- drivers/gpu/drm/i915/i915_request.c | 18 +++---- drivers/gpu/drm/i915/i915_request.h | 3 +- drivers/gpu/drm/i915/i915_scheduler.c | 2 +- 21 files changed, 162 insertions(+), 159 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index 907c2ec30b98..9f756ddea790 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -69,6 +69,7 @@ #include +#include "gt/intel_context.h" #include "gt/intel_engine_heartbeat.h" #include "gt/intel_engine_user.h" #include "gt/intel_lrc_reg.h" @@ -422,15 +423,6 @@ static void kill_context(struct i915_gem_context *ctx) struct i915_gem_engines_iter it; struct intel_context *ce; - /* - * If we are already banned, it was due to a guilty request causing - * a reset and the entire context being evicted from the GPU. - */ - if (i915_gem_context_is_banned(ctx)) - return; - - i915_gem_context_set_banned(ctx); - /* * Map the user's engine back to the actual engines; one virtual * engine will be mapped to multiple engines, and using ctx->engine[] @@ -441,6 +433,9 @@ static void kill_context(struct i915_gem_context *ctx) for_each_gem_engine(ce, __context_engines_static(ctx), it) { struct intel_engine_cs *engine; + if (intel_context_set_banned(ce)) + continue; + /* * Check the current active state of this context; if we * are currently executing on the GPU we need to evict @@ -1091,7 +1086,7 @@ static void set_ppgtt_barrier(void *data) static int emit_ppgtt_update(struct i915_request *rq, void *data) { - struct i915_address_space *vm = rq->hw_context->vm; + struct i915_address_space *vm = rq->context->vm; struct intel_engine_cs *engine = rq->engine; u32 base = engine->mmio_base; u32 *cs; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.h b/drivers/gpu/drm/i915/gem/i915_gem_context.h index 18e50a769a6e..69932899803e 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.h @@ -91,26 +91,6 @@ static inline void i915_gem_context_clear_persistence(struct i915_gem_context *c clear_bit(UCONTEXT_PERSISTENCE, &ctx->user_flags); } -static inline bool i915_gem_context_is_banned(const struct i915_gem_context *ctx) -{ - return test_bit(CONTEXT_BANNED, &ctx->flags); -} - -static inline void i915_gem_context_set_banned(struct i915_gem_context *ctx) -{ - set_bit(CONTEXT_BANNED, &ctx->flags); -} - -static inline bool i915_gem_context_force_single_submission(const struct i915_gem_context *ctx) -{ - return test_bit(CONTEXT_FORCE_SINGLE_SUBMISSION, &ctx->flags); -} - -static inline void i915_gem_context_set_force_single_submission(struct i915_gem_context *ctx) -{ - __set_bit(CONTEXT_FORCE_SINGLE_SUBMISSION, &ctx->flags); -} - static inline bool i915_gem_context_user_engines(const struct i915_gem_context *ctx) { @@ -129,24 +109,6 @@ i915_gem_context_clear_user_engines(struct i915_gem_context *ctx) clear_bit(CONTEXT_USER_ENGINES, &ctx->flags); } -static inline bool -i915_gem_context_nopreempt(const struct i915_gem_context *ctx) -{ - return test_bit(CONTEXT_NOPREEMPT, &ctx->flags); -} - -static inline void -i915_gem_context_set_nopreempt(struct i915_gem_context *ctx) -{ - set_bit(CONTEXT_NOPREEMPT, &ctx->flags); -} - -static inline void -i915_gem_context_clear_nopreempt(struct i915_gem_context *ctx) -{ - clear_bit(CONTEXT_NOPREEMPT, &ctx->flags); -} - static inline bool i915_gem_context_is_kernel(struct i915_gem_context *ctx) { return !ctx->file_priv; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h index 69df5459c350..017ca803ab47 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h @@ -134,11 +134,8 @@ struct i915_gem_context { * @flags: small set of booleans */ unsigned long flags; -#define CONTEXT_BANNED 0 -#define CONTEXT_CLOSED 1 -#define CONTEXT_FORCE_SINGLE_SUBMISSION 2 -#define CONTEXT_USER_ENGINES 3 -#define CONTEXT_NOPREEMPT 4 +#define CONTEXT_CLOSED 0 +#define CONTEXT_USER_ENGINES 1 struct mutex mutex; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c index 5dbfedeb0a6f..dff5c0180bf0 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c @@ -745,9 +745,6 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb) unsigned int i, batch; int err; - if (unlikely(i915_gem_context_is_banned(eb->gem_context))) - return -EIO; - INIT_LIST_HEAD(&eb->relocs); INIT_LIST_HEAD(&eb->unbound); @@ -2050,7 +2047,7 @@ static int eb_submit(struct i915_execbuffer *eb) if (err) return err; - if (i915_gem_context_nopreempt(eb->gem_context)) + if (intel_context_nopreempt(eb->context)) eb->request->flags |= I915_REQUEST_NOPREEMPT; return 0; @@ -2136,6 +2133,9 @@ static int __eb_pin_engine(struct i915_execbuffer *eb, struct intel_context *ce) if (err) return err; + if (unlikely(intel_context_is_banned(ce))) + return -EIO; + /* * Pinning the contexts may generate requests in order to acquire * GGTT space, so do this first before we reserve a seqno for diff --git a/drivers/gpu/drm/i915/gt/intel_breadcrumbs.c b/drivers/gpu/drm/i915/gt/intel_breadcrumbs.c index 55317081d48b..2f11bee638d0 100644 --- a/drivers/gpu/drm/i915/gt/intel_breadcrumbs.c +++ b/drivers/gpu/drm/i915/gt/intel_breadcrumbs.c @@ -271,7 +271,7 @@ bool i915_request_enable_breadcrumb(struct i915_request *rq) if (test_bit(I915_FENCE_FLAG_ACTIVE, &rq->fence.flags)) { struct intel_breadcrumbs *b = &rq->engine->breadcrumbs; - struct intel_context *ce = rq->hw_context; + struct intel_context *ce = rq->context; struct list_head *pos; spin_lock(&b->irq_lock); @@ -326,7 +326,7 @@ void i915_request_cancel_breadcrumb(struct i915_request *rq) */ spin_lock(&b->irq_lock); if (test_bit(I915_FENCE_FLAG_SIGNAL, &rq->fence.flags)) { - struct intel_context *ce = rq->hw_context; + struct intel_context *ce = rq->context; list_del(&rq->signal_link); if (list_empty(&ce->signals)) diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c index ee9d2bcd2c13..625f75f7825d 100644 --- a/drivers/gpu/drm/i915/gt/intel_context.c +++ b/drivers/gpu/drm/i915/gt/intel_context.c @@ -307,7 +307,7 @@ int intel_context_prepare_remote_request(struct intel_context *ce, int err; /* Only suitable for use in remotely modifying this context */ - GEM_BUG_ON(rq->hw_context == ce); + GEM_BUG_ON(rq->context == ce); if (rcu_access_pointer(rq->timeline) != tl) { /* timeline sharing! */ err = mutex_lock_interruptible_nested(&tl->mutex, diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h index 68b3d317d959..1e607343d256 100644 --- a/drivers/gpu/drm/i915/gt/intel_context.h +++ b/drivers/gpu/drm/i915/gt/intel_context.h @@ -7,7 +7,9 @@ #ifndef __INTEL_CONTEXT_H__ #define __INTEL_CONTEXT_H__ +#include #include +#include #include "i915_active.h" #include "intel_context_types.h" @@ -153,4 +155,44 @@ static inline struct intel_ring *__intel_context_ring_size(u64 sz) return u64_to_ptr(struct intel_ring, sz); } +static inline bool intel_context_is_banned(const struct intel_context *ce) +{ + return test_bit(CONTEXT_BANNED, &ce->flags); +} + +static inline bool intel_context_set_banned(struct intel_context *ce) +{ + return test_and_set_bit(CONTEXT_BANNED, &ce->flags); +} + +static inline bool +intel_context_force_single_submission(const struct intel_context *ce) +{ + return test_bit(CONTEXT_FORCE_SINGLE_SUBMISSION, &ce->flags); +} + +static inline void +intel_context_set_single_submission(struct intel_context *ce) +{ + __set_bit(CONTEXT_FORCE_SINGLE_SUBMISSION, &ce->flags); +} + +static inline bool +intel_context_nopreempt(const struct intel_context *ce) +{ + return test_bit(CONTEXT_NOPREEMPT, &ce->flags); +} + +static inline void +intel_context_set_nopreempt(struct intel_context *ce) +{ + set_bit(CONTEXT_NOPREEMPT, &ce->flags); +} + +static inline void +intel_context_clear_nopreempt(struct intel_context *ce) +{ + clear_bit(CONTEXT_NOPREEMPT, &ce->flags); +} + #endif /* __INTEL_CONTEXT_H__ */ diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h index 6959b05ae5f8..0251edc8f568 100644 --- a/drivers/gpu/drm/i915/gt/intel_context_types.h +++ b/drivers/gpu/drm/i915/gt/intel_context_types.h @@ -54,7 +54,10 @@ struct intel_context { struct intel_timeline *timeline; unsigned long flags; -#define CONTEXT_ALLOC_BIT 0 +#define CONTEXT_ALLOC_BIT 0 +#define CONTEXT_BANNED 1 +#define CONTEXT_FORCE_SINGLE_SUBMISSION 2 +#define CONTEXT_NOPREEMPT 3 u32 *lrc_reg_state; u64 lrc_desc; diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c index b9613d044393..7f79741ad684 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c @@ -1516,9 +1516,9 @@ void intel_engine_dump(struct intel_engine_cs *engine, print_request_ring(m, rq); - if (rq->hw_context->lrc_reg_state) { + if (rq->context->lrc_reg_state) { drm_printf(m, "Logical Ring Context:\n"); - hexdump(m, rq->hw_context->lrc_reg_state, PAGE_SIZE); + hexdump(m, rq->context->lrc_reg_state, PAGE_SIZE); } } spin_unlock_irqrestore(&engine->active.lock, flags); @@ -1579,7 +1579,7 @@ int intel_enable_engine_stats(struct intel_engine_cs *engine) for (port = execlists->pending; (rq = *port); port++) { /* Exclude any contexts already counted in active */ - if (!intel_context_inflight_count(rq->hw_context)) + if (!intel_context_inflight_count(rq->context)) engine->stats.active++; } diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c index 4b6d9e6b1bfd..77efdc3af4ac 100644 --- a/drivers/gpu/drm/i915/gt/intel_lrc.c +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c @@ -887,7 +887,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine) list_move(&rq->sched.link, pl); active = rq; } else { - struct intel_engine_cs *owner = rq->hw_context->engine; + struct intel_engine_cs *owner = rq->context->engine; /* * Decouple the virtual breadcrumb before moving it @@ -1058,7 +1058,7 @@ static void restore_default_state(struct intel_context *ce, static void reset_active(struct i915_request *rq, struct intel_engine_cs *engine) { - struct intel_context * const ce = rq->hw_context; + struct intel_context * const ce = rq->context; u32 head; /* @@ -1099,11 +1099,11 @@ static inline struct intel_engine_cs * __execlists_schedule_in(struct i915_request *rq) { struct intel_engine_cs * const engine = rq->engine; - struct intel_context * const ce = rq->hw_context; + struct intel_context * const ce = rq->context; intel_context_get(ce); - if (unlikely(i915_gem_context_is_banned(ce->gem_context))) + if (unlikely(intel_context_is_banned(ce))) reset_active(rq, engine); if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM)) @@ -1131,7 +1131,7 @@ __execlists_schedule_in(struct i915_request *rq) static inline struct i915_request * execlists_schedule_in(struct i915_request *rq, int idx) { - struct intel_context * const ce = rq->hw_context; + struct intel_context * const ce = rq->context; struct intel_engine_cs *old; GEM_BUG_ON(!intel_engine_pm_is_awake(rq->engine)); @@ -1162,7 +1162,7 @@ static inline void __execlists_schedule_out(struct i915_request *rq, struct intel_engine_cs * const engine) { - struct intel_context * const ce = rq->hw_context; + struct intel_context * const ce = rq->context; intel_engine_context_out(engine); execlists_context_status_change(rq, INTEL_CONTEXT_SCHEDULE_OUT); @@ -1186,7 +1186,7 @@ __execlists_schedule_out(struct i915_request *rq, static inline void execlists_schedule_out(struct i915_request *rq) { - struct intel_context * const ce = rq->hw_context; + struct intel_context * const ce = rq->context; struct intel_engine_cs *cur, *old; trace_i915_request_out(rq); @@ -1203,7 +1203,7 @@ execlists_schedule_out(struct i915_request *rq) static u64 execlists_update_context(const struct i915_request *rq) { - struct intel_context *ce = rq->hw_context; + struct intel_context *ce = rq->context; u64 desc; ce->lrc_reg_state[CTX_RING_TAIL] = @@ -1285,19 +1285,18 @@ assert_pending_valid(const struct intel_engine_execlists *execlists, } for (port = execlists->pending; (rq = *port); port++) { - if (ce == rq->hw_context) { + if (ce == rq->context) { GEM_TRACE_ERR("Dup context:%llx in pending[%zd]\n", ce->timeline->fence_context, port - execlists->pending); return false; } - ce = rq->hw_context; + ce = rq->context; if (i915_request_completed(rq)) continue; - if (i915_active_is_idle(&ce->active) && - !i915_gem_context_is_kernel(ce->gem_context)) { + if (i915_active_is_idle(&ce->active) && ce->gem_context) { GEM_TRACE_ERR("Inactive context:%llx in pending[%zd]\n", ce->timeline->fence_context, port - execlists->pending); @@ -1361,7 +1360,7 @@ static void execlists_submit_ports(struct intel_engine_cs *engine) static bool ctx_single_port_submission(const struct intel_context *ce) { return (IS_ENABLED(CONFIG_DRM_I915_GVT) && - i915_gem_context_force_single_submission(ce->gem_context)); + intel_context_force_single_submission(ce)); } static bool can_merge_ctx(const struct intel_context *prev, @@ -1397,7 +1396,7 @@ static bool can_merge_rq(const struct i915_request *prev, (I915_REQUEST_NOPREEMPT | I915_REQUEST_SENTINEL))) return false; - if (!can_merge_ctx(prev->hw_context, next->hw_context)) + if (!can_merge_ctx(prev->context, next->context)) return false; return true; @@ -1584,7 +1583,7 @@ static unsigned long active_preempt_timeout(struct intel_engine_cs *engine) return 0; /* Force a fast reset for terminated contexts (ignoring sysfs!) */ - if (unlikely(i915_gem_context_is_banned(rq->gem_context))) + if (unlikely(intel_context_is_banned(rq->context))) return 1; return READ_ONCE(engine->props.preempt_timeout_ms); @@ -1692,7 +1691,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine) * tendency to ignore us rewinding the TAIL to the * end of an earlier request. */ - last->hw_context->lrc_desc |= CTX_DESC_FORCE_RESTORE; + last->context->lrc_desc |= CTX_DESC_FORCE_RESTORE; last = NULL; } else if (need_timeslice(engine, last) && timer_expired(&engine->execlists.timer)) { @@ -1774,7 +1773,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine) GEM_BUG_ON(rq != ve->request); GEM_BUG_ON(rq->engine != &ve->base); - GEM_BUG_ON(rq->hw_context != &ve->context); + GEM_BUG_ON(rq->context != &ve->context); if (rq_prio(rq) >= queue_prio(execlists)) { if (!virtual_matches(ve, rq, engine)) { @@ -1893,7 +1892,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine) * same LRCA, i.e. we must submit 2 different * contexts if we submit 2 ELSP. */ - if (last->hw_context == rq->hw_context) + if (last->context == rq->context) goto done; if (i915_request_has_sentinel(last)) @@ -1906,8 +1905,8 @@ static void execlists_dequeue(struct intel_engine_cs *engine) * the same context (even though a different * request) to the second port. */ - if (ctx_single_port_submission(last->hw_context) || - ctx_single_port_submission(rq->hw_context)) + if (ctx_single_port_submission(last->context) || + ctx_single_port_submission(rq->context)) goto done; merge = false; @@ -1921,8 +1920,8 @@ static void execlists_dequeue(struct intel_engine_cs *engine) } GEM_BUG_ON(last && - !can_merge_ctx(last->hw_context, - rq->hw_context)); + !can_merge_ctx(last->context, + rq->context)); submit = true; last = rq; @@ -2536,7 +2535,7 @@ static int execlists_request_alloc(struct i915_request *request) { int ret; - GEM_BUG_ON(!intel_context_is_pinned(request->hw_context)); + GEM_BUG_ON(!intel_context_is_pinned(request->context)); /* * Flush enough space to reduce the likelihood of waiting after @@ -3043,7 +3042,7 @@ static void __execlists_reset(struct intel_engine_cs *engine, bool stalled) /* We still have requests in-flight; the engine should be active */ GEM_BUG_ON(!intel_engine_pm_is_awake(engine)); - ce = rq->hw_context; + ce = rq->context; GEM_BUG_ON(!i915_vma_is_pinned(ce->state)); if (i915_request_completed(rq)) { diff --git a/drivers/gpu/drm/i915/gt/intel_reset.c b/drivers/gpu/drm/i915/gt/intel_reset.c index b7007cd78c6f..0d4c610a90b7 100644 --- a/drivers/gpu/drm/i915/gt/intel_reset.c +++ b/drivers/gpu/drm/i915/gt/intel_reset.c @@ -40,27 +40,31 @@ static void rmw_clear_fw(struct intel_uncore *uncore, i915_reg_t reg, u32 clr) static void engine_skip_context(struct i915_request *rq) { struct intel_engine_cs *engine = rq->engine; - struct i915_gem_context *hung_ctx = rq->gem_context; + struct intel_context *hung_ctx = rq->context; if (!i915_request_is_active(rq)) - return; + rq = list_first_entry(&engine->active.requests, + typeof(*rq), sched.link); lockdep_assert_held(&engine->active.lock); list_for_each_entry_continue(rq, &engine->active.requests, sched.link) - if (rq->gem_context == hung_ctx) + if (rq->context == hung_ctx) i915_request_skip(rq, -EIO); } -static void client_mark_guilty(struct drm_i915_file_private *file_priv, - const struct i915_gem_context *ctx) +static void client_mark_guilty(struct i915_request *rq, bool banned) { - unsigned int score; + struct i915_gem_context *ctx = rq->context->gem_context; + struct drm_i915_file_private *file_priv = ctx->file_priv; unsigned long prev_hang; + unsigned int score; - if (i915_gem_context_is_banned(ctx)) + if (IS_ERR_OR_NULL(file_priv)) + return; + + score = 0; + if (banned) score = I915_CLIENT_SCORE_CONTEXT_BAN; - else - score = 0; prev_hang = xchg(&file_priv->hang_timestamp, jiffies); if (time_before(jiffies, prev_hang + I915_CLIENT_FAST_HANG_JIFFIES)) @@ -75,14 +79,15 @@ static void client_mark_guilty(struct drm_i915_file_private *file_priv, } } -static bool context_mark_guilty(struct i915_gem_context *ctx) +static bool mark_guilty(struct i915_request *rq) { + struct i915_gem_context *ctx = rq->context->gem_context; unsigned long prev_hang; bool banned; int i; if (i915_gem_context_is_closed(ctx)) { - i915_gem_context_set_banned(ctx); + intel_context_set_banned(rq->context); return true; } @@ -109,18 +114,17 @@ static bool context_mark_guilty(struct i915_gem_context *ctx) if (banned) { DRM_DEBUG_DRIVER("context %s: guilty %d, banned\n", ctx->name, atomic_read(&ctx->guilty_count)); - i915_gem_context_set_banned(ctx); + intel_context_set_banned(rq->context); } - if (!IS_ERR_OR_NULL(ctx->file_priv)) - client_mark_guilty(ctx->file_priv, ctx); + client_mark_guilty(rq, banned); return banned; } -static void context_mark_innocent(struct i915_gem_context *ctx) +static void mark_innocent(struct i915_request *rq) { - atomic_inc(&ctx->active_count); + atomic_inc(&rq->context->gem_context->active_count); } void __i915_request_reset(struct i915_request *rq, bool guilty) @@ -136,11 +140,11 @@ void __i915_request_reset(struct i915_request *rq, bool guilty) rcu_read_lock(); /* protect the GEM context */ if (guilty) { i915_request_skip(rq, -EIO); - if (context_mark_guilty(rq->gem_context)) + if (mark_guilty(rq)) engine_skip_context(rq); } else { dma_fence_set_error(&rq->fence, -EAGAIN); - context_mark_innocent(rq->gem_context); + mark_innocent(rq); } rcu_read_unlock(); } diff --git a/drivers/gpu/drm/i915/gt/intel_ring_submission.c b/drivers/gpu/drm/i915/gt/intel_ring_submission.c index a47d5a7c32c9..1c2f3c3a8840 100644 --- a/drivers/gpu/drm/i915/gt/intel_ring_submission.c +++ b/drivers/gpu/drm/i915/gt/intel_ring_submission.c @@ -1485,7 +1485,7 @@ static inline int mi_set_context(struct i915_request *rq, u32 flags) *cs++ = MI_NOOP; *cs++ = MI_SET_CONTEXT; - *cs++ = i915_ggtt_offset(rq->hw_context->state) | flags; + *cs++ = i915_ggtt_offset(rq->context->state) | flags; /* * w/a: MI_SET_CONTEXT must always be followed by MI_NOOP * WaMiSetContext_Hang:snb,ivb,vlv @@ -1555,7 +1555,7 @@ static int remap_l3_slice(struct i915_request *rq, int slice) static int remap_l3(struct i915_request *rq) { - struct i915_gem_context *ctx = rq->gem_context; + struct i915_gem_context *ctx = rq->context->gem_context; int i, err; if (!ctx->remap_slice) @@ -1576,7 +1576,7 @@ static int remap_l3(struct i915_request *rq) static int switch_context(struct i915_request *rq) { - struct intel_context *ce = rq->hw_context; + struct intel_context *ce = rq->context; struct i915_address_space *vm = vm_alias(ce); int ret; @@ -1601,7 +1601,7 @@ static int switch_context(struct i915_request *rq) * is purely used for flushing user contexts. */ hw_flags = 0; - if (i915_gem_context_is_kernel(rq->gem_context)) + if (i915_gem_context_is_kernel(rq->context->gem_context)) hw_flags = MI_RESTORE_INHIBIT; ret = mi_set_context(rq, hw_flags); @@ -1648,7 +1648,7 @@ static int ring_request_alloc(struct i915_request *request) { int ret; - GEM_BUG_ON(!intel_context_is_pinned(request->hw_context)); + GEM_BUG_ON(!intel_context_is_pinned(request->context)); GEM_BUG_ON(i915_request_timeline(request)->has_initial_breadcrumb); /* diff --git a/drivers/gpu/drm/i915/gt/selftest_lrc.c b/drivers/gpu/drm/i915/gt/selftest_lrc.c index d1ed3c0f851c..7d5713ea6bde 100644 --- a/drivers/gpu/drm/i915/gt/selftest_lrc.c +++ b/drivers/gpu/drm/i915/gt/selftest_lrc.c @@ -1187,13 +1187,13 @@ static int __cancel_active0(struct live_preempt_cancel *arg) __func__, arg->engine->name)) return -EIO; - clear_bit(CONTEXT_BANNED, &arg->a.ctx->flags); rq = spinner_create_request(&arg->a.spin, arg->a.ctx, arg->engine, MI_ARB_CHECK); if (IS_ERR(rq)) return PTR_ERR(rq); + clear_bit(CONTEXT_BANNED, &rq->context->flags); i915_request_get(rq); i915_request_add(rq); if (!igt_wait_for_spinner(&arg->a.spin, rq)) { @@ -1201,7 +1201,7 @@ static int __cancel_active0(struct live_preempt_cancel *arg) goto out; } - i915_gem_context_set_banned(arg->a.ctx); + intel_context_set_banned(rq->context); err = intel_engine_pulse(arg->engine); if (err) goto out; @@ -1236,13 +1236,13 @@ static int __cancel_active1(struct live_preempt_cancel *arg) __func__, arg->engine->name)) return -EIO; - clear_bit(CONTEXT_BANNED, &arg->a.ctx->flags); rq[0] = spinner_create_request(&arg->a.spin, arg->a.ctx, arg->engine, MI_NOOP); /* no preemption */ if (IS_ERR(rq[0])) return PTR_ERR(rq[0]); + clear_bit(CONTEXT_BANNED, &rq[0]->context->flags); i915_request_get(rq[0]); i915_request_add(rq[0]); if (!igt_wait_for_spinner(&arg->a.spin, rq[0])) { @@ -1250,7 +1250,6 @@ static int __cancel_active1(struct live_preempt_cancel *arg) goto out; } - clear_bit(CONTEXT_BANNED, &arg->b.ctx->flags); rq[1] = spinner_create_request(&arg->b.spin, arg->b.ctx, arg->engine, MI_ARB_CHECK); @@ -1259,13 +1258,14 @@ static int __cancel_active1(struct live_preempt_cancel *arg) goto out; } + clear_bit(CONTEXT_BANNED, &rq[1]->context->flags); i915_request_get(rq[1]); err = i915_request_await_dma_fence(rq[1], &rq[0]->fence); i915_request_add(rq[1]); if (err) goto out; - i915_gem_context_set_banned(arg->b.ctx); + intel_context_set_banned(rq[1]->context); err = intel_engine_pulse(arg->engine); if (err) goto out; @@ -1308,13 +1308,13 @@ static int __cancel_queued(struct live_preempt_cancel *arg) __func__, arg->engine->name)) return -EIO; - clear_bit(CONTEXT_BANNED, &arg->a.ctx->flags); rq[0] = spinner_create_request(&arg->a.spin, arg->a.ctx, arg->engine, MI_ARB_CHECK); if (IS_ERR(rq[0])) return PTR_ERR(rq[0]); + clear_bit(CONTEXT_BANNED, &rq[0]->context->flags); i915_request_get(rq[0]); i915_request_add(rq[0]); if (!igt_wait_for_spinner(&arg->a.spin, rq[0])) { @@ -1322,13 +1322,13 @@ static int __cancel_queued(struct live_preempt_cancel *arg) goto out; } - clear_bit(CONTEXT_BANNED, &arg->b.ctx->flags); rq[1] = igt_request_alloc(arg->b.ctx, arg->engine); if (IS_ERR(rq[1])) { err = PTR_ERR(rq[1]); goto out; } + clear_bit(CONTEXT_BANNED, &rq[1]->context->flags); i915_request_get(rq[1]); err = i915_request_await_dma_fence(rq[1], &rq[0]->fence); i915_request_add(rq[1]); @@ -1349,7 +1349,7 @@ static int __cancel_queued(struct live_preempt_cancel *arg) if (err) goto out; - i915_gem_context_set_banned(arg->a.ctx); + intel_context_set_banned(rq[2]->context); err = intel_engine_pulse(arg->engine); if (err) goto out; @@ -1396,13 +1396,13 @@ static int __cancel_hostile(struct live_preempt_cancel *arg) return 0; GEM_TRACE("%s(%s)\n", __func__, arg->engine->name); - clear_bit(CONTEXT_BANNED, &arg->a.ctx->flags); rq = spinner_create_request(&arg->a.spin, arg->a.ctx, arg->engine, MI_NOOP); /* preemption disabled */ if (IS_ERR(rq)) return PTR_ERR(rq); + clear_bit(CONTEXT_BANNED, &rq->context->flags); i915_request_get(rq); i915_request_add(rq); if (!igt_wait_for_spinner(&arg->a.spin, rq)) { @@ -1410,7 +1410,7 @@ static int __cancel_hostile(struct live_preempt_cancel *arg) goto out; } - i915_gem_context_set_banned(arg->a.ctx); + intel_context_set_banned(rq->context); err = intel_engine_pulse(arg->engine); /* force reset */ if (err) goto out; diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c index 2498c55e0ea5..08267d2c3cfc 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c @@ -471,7 +471,7 @@ static void guc_add_request(struct intel_guc *guc, struct i915_request *rq) { struct intel_guc_client *client = guc->execbuf_client; struct intel_engine_cs *engine = rq->engine; - u32 ctx_desc = lower_32_bits(rq->hw_context->lrc_desc); + u32 ctx_desc = lower_32_bits(rq->context->lrc_desc); u32 ring_tail = intel_ring_set_tail(rq->ring, rq->tail) / sizeof(u64); guc_wq_item_append(client, engine->guc_id, ctx_desc, @@ -572,7 +572,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine) int i; priolist_for_each_request_consume(rq, rn, p, i) { - if (last && rq->hw_context != last->hw_context) { + if (last && rq->context != last->context) { if (port == last_port) goto done; @@ -677,7 +677,7 @@ static void guc_reset(struct intel_engine_cs *engine, bool stalled) stalled = false; __i915_request_reset(rq, stalled); - intel_lr_context_reset(engine, rq->hw_context, rq->head, stalled); + intel_lr_context_reset(engine, rq->context, rq->head, stalled); out_unlock: spin_unlock_irqrestore(&engine->active.lock, flags); diff --git a/drivers/gpu/drm/i915/gvt/scheduler.c b/drivers/gpu/drm/i915/gvt/scheduler.c index 5b2a7d072ec9..228c66534e21 100644 --- a/drivers/gpu/drm/i915/gvt/scheduler.c +++ b/drivers/gpu/drm/i915/gvt/scheduler.c @@ -59,7 +59,7 @@ static void set_context_pdp_root_pointer( static void update_shadow_pdps(struct intel_vgpu_workload *workload) { struct drm_i915_gem_object *ctx_obj = - workload->req->hw_context->state->obj; + workload->req->context->state->obj; struct execlist_ring_context *shadow_ring_context; struct page *page; @@ -130,7 +130,7 @@ static int populate_shadow_context(struct intel_vgpu_workload *workload) struct intel_gvt *gvt = vgpu->gvt; int ring_id = workload->ring_id; struct drm_i915_gem_object *ctx_obj = - workload->req->hw_context->state->obj; + workload->req->context->state->obj; struct execlist_ring_context *shadow_ring_context; struct page *page; void *dst; @@ -205,9 +205,9 @@ static int populate_shadow_context(struct intel_vgpu_workload *workload) return 0; } -static inline bool is_gvt_request(struct i915_request *req) +static inline bool is_gvt_request(struct i915_request *rq) { - return i915_gem_context_force_single_submission(req->gem_context); + return intel_context_force_single_submission(rq->context); } static void save_ring_hw_state(struct intel_vgpu *vgpu, int ring_id) @@ -307,7 +307,7 @@ static int copy_workload_to_ring_buffer(struct intel_vgpu_workload *workload) u32 *cs; int err; - if (IS_GEN(req->i915, 9) && is_inhibit_context(req->hw_context)) + if (IS_GEN(req->i915, 9) && is_inhibit_context(req->context)) intel_vgpu_restore_inhibit_context(vgpu, req); /* @@ -363,11 +363,10 @@ static void release_shadow_wa_ctx(struct intel_shadow_wa_ctx *wa_ctx) } static void set_context_ppgtt_from_shadow(struct intel_vgpu_workload *workload, - struct i915_gem_context *ctx) + struct intel_context *ce) { struct intel_vgpu_mm *mm = workload->shadow_mm; - struct i915_ppgtt *ppgtt = - i915_vm_to_ppgtt(i915_gem_context_get_vm_rcu(ctx)); + struct i915_ppgtt *ppgtt = i915_vm_to_ppgtt(ce->vm); int i = 0; if (mm->ppgtt_mm.root_entry_type == GTT_TYPE_PPGTT_ROOT_L4_ENTRY) { @@ -380,8 +379,6 @@ static void set_context_ppgtt_from_shadow(struct intel_vgpu_workload *workload, px_dma(pd) = mm->ppgtt_mm.shadow_pdps[i]; } } - - i915_vm_put(&ppgtt->vm); } static int @@ -529,7 +526,7 @@ static void update_wa_ctx_2_shadow_ctx(struct intel_shadow_wa_ctx *wa_ctx) container_of(wa_ctx, struct intel_vgpu_workload, wa_ctx); struct i915_request *rq = workload->req; struct execlist_ring_context *shadow_ring_context = - (struct execlist_ring_context *)rq->hw_context->lrc_reg_state; + (struct execlist_ring_context *)rq->context->lrc_reg_state; shadow_ring_context->bb_per_ctx_ptr.val = (shadow_ring_context->bb_per_ctx_ptr.val & @@ -628,7 +625,7 @@ static int prepare_workload(struct intel_vgpu_workload *workload) update_shadow_pdps(workload); - set_context_ppgtt_from_shadow(workload, s->shadow[ring]->gem_context); + set_context_ppgtt_from_shadow(workload, s->shadow[ring]); ret = intel_vgpu_sync_oos_pages(workload->vgpu); if (ret) { @@ -787,7 +784,7 @@ static void update_guest_context(struct intel_vgpu_workload *workload) struct i915_request *rq = workload->req; struct intel_vgpu *vgpu = workload->vgpu; struct intel_gvt *gvt = vgpu->gvt; - struct drm_i915_gem_object *ctx_obj = rq->hw_context->state->obj; + struct drm_i915_gem_object *ctx_obj = rq->context->state->obj; struct execlist_ring_context *shadow_ring_context; struct page *page; void *src; @@ -1232,8 +1229,6 @@ int intel_vgpu_setup_submission(struct intel_vgpu *vgpu) if (IS_ERR(ctx)) return PTR_ERR(ctx); - i915_gem_context_set_force_single_submission(ctx); - ppgtt = i915_vm_to_ppgtt(i915_gem_context_get_vm_rcu(ctx)); i915_context_ppgtt_root_save(s, ppgtt); @@ -1249,6 +1244,8 @@ int intel_vgpu_setup_submission(struct intel_vgpu *vgpu) goto out_shadow_ctx; } + intel_context_set_single_submission(ce); + if (!USES_GUC_SUBMISSION(i915)) { /* Max ring buffer size */ const unsigned int ring_size = 512 * SZ_4K; diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 143a8952b736..f4cbd379db95 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -1108,9 +1108,9 @@ static int __intel_engines_record_defaults(struct intel_gt *gt) continue; /* We want to be able to unbind the state from the GGTT */ - GEM_BUG_ON(intel_context_is_pinned(rq->hw_context)); + GEM_BUG_ON(intel_context_is_pinned(rq->context)); - state = rq->hw_context->state; + state = rq->context->state; if (!state) continue; @@ -1162,7 +1162,7 @@ static int __intel_engines_record_defaults(struct intel_gt *gt) if (!rq) continue; - ce = rq->hw_context; + ce = rq->context; i915_request_put(rq); intel_context_put(ce); } diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c index 1f2f266f26af..5c7369c55f92 100644 --- a/drivers/gpu/drm/i915/i915_gpu_error.c +++ b/drivers/gpu/drm/i915/i915_gpu_error.c @@ -1220,7 +1220,7 @@ static void error_record_engine_registers(struct i915_gpu_state *error, static void record_request(const struct i915_request *request, struct drm_i915_error_request *erq) { - const struct i915_gem_context *ctx = request->gem_context; + const struct i915_gem_context *ctx = request->context->gem_context; erq->flags = request->fence.flags; erq->context = request->fence.context; @@ -1230,7 +1230,7 @@ static void record_request(const struct i915_request *request, erq->start = i915_ggtt_offset(request->ring->vma); erq->head = request->head; erq->tail = request->tail; - erq->pid = ctx->pid ? pid_nr(ctx->pid) : 0; + erq->pid = ctx && ctx->pid ? pid_nr(ctx->pid) : 0; } static void engine_record_requests(struct intel_engine_cs *engine, @@ -1297,7 +1297,10 @@ static void error_record_engine_execlists(const struct intel_engine_cs *engine, static bool record_context(struct drm_i915_error_context *e, const struct i915_request *rq) { - const struct i915_gem_context *ctx = rq->gem_context; + const struct i915_gem_context *ctx = rq->context->gem_context; + + if (!ctx) + return false; if (ctx->pid) { struct task_struct *task; @@ -1451,7 +1454,7 @@ gem_record_rings(struct i915_gpu_state *error, struct compress *compress) capture = request_record_user_bo(request, ee, capture); capture = capture_vma(capture, - request->hw_context->state, + request->context->state, &ee->ctx); capture = capture_vma(capture, diff --git a/drivers/gpu/drm/i915/i915_perf.c b/drivers/gpu/drm/i915/i915_perf.c index 2c380aba1ce9..1924d44081e9 100644 --- a/drivers/gpu/drm/i915/i915_perf.c +++ b/drivers/gpu/drm/i915/i915_perf.c @@ -3075,7 +3075,7 @@ static void i915_perf_enable_locked(struct i915_perf_stream *stream) stream->ops->enable(stream); if (stream->hold_preemption) - i915_gem_context_set_nopreempt(stream->ctx); + intel_context_set_nopreempt(stream->pinned_ctx); } /** @@ -3101,7 +3101,7 @@ static void i915_perf_disable_locked(struct i915_perf_stream *stream) stream->enabled = false; if (stream->hold_preemption) - i915_gem_context_clear_nopreempt(stream->ctx); + intel_context_clear_nopreempt(stream->pinned_ctx); if (stream->ops->disable) stream->ops->disable(stream); diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c index f7cc13d9d0ae..8a1f1026f034 100644 --- a/drivers/gpu/drm/i915/i915_request.c +++ b/drivers/gpu/drm/i915/i915_request.c @@ -62,6 +62,8 @@ static const char *i915_fence_get_driver_name(struct dma_fence *fence) static const char *i915_fence_get_timeline_name(struct dma_fence *fence) { + const struct i915_gem_context *ctx; + /* * The timeline struct (as part of the ppgtt underneath a context) * may be freed when the request is no longer in use by the GPU. @@ -74,7 +76,8 @@ static const char *i915_fence_get_timeline_name(struct dma_fence *fence) if (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags)) return "signaled"; - return to_request(fence)->gem_context->name ?: "[i915]"; + ctx = to_request(fence)->context->gem_context; + return ctx && ctx->name ? ctx->name : "[i915]"; } static bool i915_fence_signaled(struct dma_fence *fence) @@ -272,8 +275,8 @@ bool i915_request_retire(struct i915_request *rq) remove_from_client(rq); list_del(&rq->link); - intel_context_exit(rq->hw_context); - intel_context_unpin(rq->hw_context); + intel_context_exit(rq->context); + intel_context_unpin(rq->context); free_capture_list(rq); i915_sched_node_fini(&rq->sched); @@ -376,7 +379,7 @@ bool __i915_request_submit(struct i915_request *request) if (i915_request_completed(request)) goto xfer; - if (i915_gem_context_is_banned(request->gem_context)) + if (intel_context_is_banned(request->context)) i915_request_skip(request, -EIO); /* @@ -655,8 +658,7 @@ __i915_request_create(struct intel_context *ce, gfp_t gfp) goto err_free; rq->i915 = ce->engine->i915; - rq->hw_context = ce; - rq->gem_context = ce->gem_context; + rq->context = ce; rq->engine = ce->engine; rq->ring = ce->ring; rq->execution_mask = ce->engine->mask; @@ -908,7 +910,7 @@ i915_request_await_request(struct i915_request *to, struct i915_request *from) &from->submit, I915_FENCE_GFP); } else if (intel_engine_has_semaphores(to->engine) && - to->gem_context->sched.priority >= I915_PRIORITY_NORMAL) { + to->context->gem_context->sched.priority >= I915_PRIORITY_NORMAL) { ret = emit_semaphore_wait(to, from, I915_FENCE_GFP); } else { ret = i915_sw_fence_await_dma_fence(&to->submit, @@ -1236,7 +1238,7 @@ void __i915_request_queue(struct i915_request *rq, void i915_request_add(struct i915_request *rq) { - struct i915_sched_attr attr = rq->gem_context->sched; + struct i915_sched_attr attr = rq->context->gem_context->sched; struct intel_timeline * const tl = i915_request_timeline(rq); struct i915_request *prev; diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h index 96991d64759c..b3b6534ae798 100644 --- a/drivers/gpu/drm/i915/i915_request.h +++ b/drivers/gpu/drm/i915/i915_request.h @@ -109,9 +109,8 @@ struct i915_request { * i915_request_free() will then decrement the refcount on the * context. */ - struct i915_gem_context *gem_context; struct intel_engine_cs *engine; - struct intel_context *hw_context; + struct intel_context *context; struct intel_ring *ring; struct intel_timeline __rcu *timeline; struct list_head signal_link; diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c index 724e96fe96e9..54f69494ecf7 100644 --- a/drivers/gpu/drm/i915/i915_scheduler.c +++ b/drivers/gpu/drm/i915/i915_scheduler.c @@ -213,7 +213,7 @@ static void kick_submission(struct intel_engine_cs *engine, * If we are already the currently executing context, don't * bother evaluating if we should preempt ourselves. */ - if (inflight->hw_context == rq->hw_context) + if (inflight->context == rq->context) goto unlock; engine->execlists.queue_priority_hint = prio; From patchwork Sun Nov 10 18:58:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236311 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8FA0814E5 for ; Sun, 10 Nov 2019 18:58:46 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 778AC20842 for ; Sun, 10 Nov 2019 18:58:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 778AC20842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D121C6E7D3; Sun, 10 Nov 2019 18:58:36 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id CE1C96E06B for ; Sun, 10 Nov 2019 18:58:27 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151901-1500050 for multiple; Sun, 10 Nov 2019 18:58:13 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:58:02 +0000 Message-Id: <20191110185806.17413-21-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 21/25] drm/i915: Push the use-semaphore marker onto the intel_context X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Instead of rummaging through the intel_context to peek at the GEM context in the middle of request submission to decide whether to use semaphores, store that information on the intel_context itself. Signed-off-by: Chris Wilson --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 52 +++++++++++++------ drivers/gpu/drm/i915/gt/intel_context.c | 3 ++ drivers/gpu/drm/i915/gt/intel_context.h | 15 ++++++ drivers/gpu/drm/i915/gt/intel_context_types.h | 7 +-- drivers/gpu/drm/i915/i915_request.c | 8 ++- 5 files changed, 60 insertions(+), 25 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index 9f756ddea790..e691b9efcb2a 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -1843,6 +1843,40 @@ set_persistence(struct i915_gem_context *ctx, return __context_set_persistence(ctx, args->value); } +static void __apply_priority(struct intel_context *ce, void *arg) +{ + struct i915_gem_context *ctx = arg; + + if (intel_context_use_semaphores(ce) && + ctx->sched.priority < I915_PRIORITY_NORMAL) + intel_context_clear_use_semaphores(ce); +} + +static int set_priority(struct i915_gem_context *ctx, + const struct drm_i915_gem_context_param *args) +{ + s64 priority = args->value; + + if (args->size) + return -EINVAL; + + if (!(ctx->i915->caps.scheduler & I915_SCHEDULER_CAP_PRIORITY)) + return -ENODEV; + + if (priority > I915_CONTEXT_MAX_USER_PRIORITY || + priority < I915_CONTEXT_MIN_USER_PRIORITY) + return -EINVAL; + + if (priority > I915_CONTEXT_DEFAULT_PRIORITY && + !capable(CAP_SYS_NICE)) + return -EPERM; + + ctx->sched.priority = I915_USER_PRIORITY(priority); + context_apply_all(ctx, __apply_priority, ctx); + + return 0; +} + static int ctx_setparam(struct drm_i915_file_private *fpriv, struct i915_gem_context *ctx, struct drm_i915_gem_context_param *args) @@ -1889,23 +1923,7 @@ static int ctx_setparam(struct drm_i915_file_private *fpriv, break; case I915_CONTEXT_PARAM_PRIORITY: - { - s64 priority = args->value; - - if (args->size) - ret = -EINVAL; - else if (!(ctx->i915->caps.scheduler & I915_SCHEDULER_CAP_PRIORITY)) - ret = -ENODEV; - else if (priority > I915_CONTEXT_MAX_USER_PRIORITY || - priority < I915_CONTEXT_MIN_USER_PRIORITY) - ret = -EINVAL; - else if (priority > I915_CONTEXT_DEFAULT_PRIORITY && - !capable(CAP_SYS_NICE)) - ret = -EPERM; - else - ctx->sched.priority = - I915_USER_PRIORITY(priority); - } + ret = set_priority(ctx, args); break; case I915_CONTEXT_PARAM_SSEU: diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c index 625f75f7825d..7075d03f508f 100644 --- a/drivers/gpu/drm/i915/gt/intel_context.c +++ b/drivers/gpu/drm/i915/gt/intel_context.c @@ -238,6 +238,9 @@ intel_context_init(struct intel_context *ce, rcu_read_unlock(); if (ctx->timeline) ce->timeline = intel_timeline_get(ctx->timeline); + if (ctx->sched.priority >= I915_PRIORITY_NORMAL && + intel_engine_has_semaphores(engine)) + __set_bit(CONTEXT_USE_SEMAPHORES, &ce->flags); ce->engine = engine; ce->ops = engine->cops; diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h index 1e607343d256..d7b667a26e08 100644 --- a/drivers/gpu/drm/i915/gt/intel_context.h +++ b/drivers/gpu/drm/i915/gt/intel_context.h @@ -155,6 +155,21 @@ static inline struct intel_ring *__intel_context_ring_size(u64 sz) return u64_to_ptr(struct intel_ring, sz); } +static inline bool intel_context_use_semaphores(const struct intel_context *ce) +{ + return test_bit(CONTEXT_USE_SEMAPHORES, &ce->flags); +} + +static inline void intel_context_set_use_semaphores(struct intel_context *ce) +{ + set_bit(CONTEXT_USE_SEMAPHORES, &ce->flags); +} + +static inline void intel_context_clear_use_semaphores(struct intel_context *ce) +{ + clear_bit(CONTEXT_USE_SEMAPHORES, &ce->flags); +} + static inline bool intel_context_is_banned(const struct intel_context *ce) { return test_bit(CONTEXT_BANNED, &ce->flags); diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h index 0251edc8f568..264c1efde772 100644 --- a/drivers/gpu/drm/i915/gt/intel_context_types.h +++ b/drivers/gpu/drm/i915/gt/intel_context_types.h @@ -55,9 +55,10 @@ struct intel_context { unsigned long flags; #define CONTEXT_ALLOC_BIT 0 -#define CONTEXT_BANNED 1 -#define CONTEXT_FORCE_SINGLE_SUBMISSION 2 -#define CONTEXT_NOPREEMPT 3 +#define CONTEXT_USE_SEMAPHORES 1 +#define CONTEXT_BANNED 2 +#define CONTEXT_FORCE_SINGLE_SUBMISSION 3 +#define CONTEXT_NOPREEMPT 4 u32 *lrc_reg_state; u64 lrc_desc; diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c index 8a1f1026f034..678b32089507 100644 --- a/drivers/gpu/drm/i915/i915_request.c +++ b/drivers/gpu/drm/i915/i915_request.c @@ -905,18 +905,16 @@ i915_request_await_request(struct i915_request *to, struct i915_request *from) return ret; } - if (to->engine == from->engine) { + if (to->engine == from->engine) ret = i915_sw_fence_await_sw_fence_gfp(&to->submit, &from->submit, I915_FENCE_GFP); - } else if (intel_engine_has_semaphores(to->engine) && - to->context->gem_context->sched.priority >= I915_PRIORITY_NORMAL) { + else if (intel_context_use_semaphores(to->context)) ret = emit_semaphore_wait(to, from, I915_FENCE_GFP); - } else { + else ret = i915_sw_fence_await_dma_fence(&to->submit, &from->fence, 0, I915_FENCE_GFP); - } if (ret < 0) return ret; From patchwork Sun Nov 10 18:58:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236301 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C2F8E15AB for ; Sun, 10 Nov 2019 18:58:42 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AAAF620842 for ; Sun, 10 Nov 2019 18:58:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AAAF620842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3C3EF6E7B5; Sun, 10 Nov 2019 18:58:35 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id DECA56E51A for ; Sun, 10 Nov 2019 18:58:27 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151902-1500050 for multiple; Sun, 10 Nov 2019 18:58:13 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:58:03 +0000 Message-Id: <20191110185806.17413-22-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 22/25] drm/i915: Remove i915->kernel_context X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Allocate only an internal intel_context for the kernel_context, forgoing a global GEM context for internal use as we only require a separate address space (for our own protection). Now having weaned GT from requiring ce->gem_context, we can stop referencing it entirely. This also means we no longer have to create random and unnecessary GEM contexts for internal use. GEM contexts are now entirely for tracking GEM clients, and intel_context the execution environment on the GPU. Signed-off-by: Chris Wilson --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 97 +++++------ drivers/gpu/drm/i915/gem/i915_gem_context.h | 6 +- drivers/gpu/drm/i915/gem/i915_gem_userptr.c | 6 +- .../drm/i915/gem/selftests/i915_gem_context.c | 5 +- .../gpu/drm/i915/gem/selftests/mock_context.c | 11 +- drivers/gpu/drm/i915/gt/intel_context.c | 32 +--- drivers/gpu/drm/i915/gt/intel_context.h | 9 +- drivers/gpu/drm/i915/gt/intel_context_types.h | 11 +- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 21 +-- drivers/gpu/drm/i915/gt/intel_gt.c | 25 ++- drivers/gpu/drm/i915/gt/intel_gt_types.h | 7 + drivers/gpu/drm/i915/gt/intel_lrc.c | 21 ++- drivers/gpu/drm/i915/gt/intel_lrc.h | 6 +- drivers/gpu/drm/i915/gt/intel_reset.c | 14 +- .../gpu/drm/i915/gt/intel_ring_submission.c | 4 +- drivers/gpu/drm/i915/gt/selftest_context.c | 57 ++----- .../drm/i915/gt/selftest_engine_heartbeat.c | 3 +- drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 141 +++++++--------- drivers/gpu/drm/i915/gt/selftest_lrc.c | 152 ++++++------------ .../gpu/drm/i915/gt/selftest_workarounds.c | 72 ++++----- drivers/gpu/drm/i915/gvt/scheduler.c | 16 +- drivers/gpu/drm/i915/i915_active.c | 2 + drivers/gpu/drm/i915/i915_drv.h | 3 - drivers/gpu/drm/i915/i915_gem.c | 16 +- drivers/gpu/drm/i915/i915_gem_gtt.c | 8 +- drivers/gpu/drm/i915/i915_perf.c | 3 - drivers/gpu/drm/i915/i915_request.c | 5 +- drivers/gpu/drm/i915/selftests/i915_request.c | 6 +- .../gpu/drm/i915/selftests/mock_gem_device.c | 8 +- 29 files changed, 292 insertions(+), 475 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index e691b9efcb2a..21c265af6bd5 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -208,6 +208,32 @@ context_get_vm_rcu(struct i915_gem_context *ctx) } while (1); } +static void intel_context_set_gem(struct intel_context *ce, + struct i915_gem_context *ctx) +{ + ce->gem_context = ctx; + + ce->ring = __intel_context_ring_size(SZ_16K); + + if (rcu_access_pointer(ctx->vm)) { + struct i915_address_space *vm; + + rcu_read_lock(); + vm = context_get_vm_rcu(ctx); /* hmm */ + rcu_read_unlock(); + + i915_vm_put(ce->vm); + ce->vm = vm; + } + + if (ctx->timeline) + ce->timeline = intel_timeline_get(ctx->timeline); + + if (ctx->sched.priority >= I915_PRIORITY_NORMAL && + intel_engine_has_semaphores(ce->engine)) + __set_bit(CONTEXT_USE_SEMAPHORES, &ce->flags); +} + static void __free_engines(struct i915_gem_engines *e, unsigned int count) { while (count--) { @@ -250,12 +276,14 @@ static struct i915_gem_engines *default_engines(struct i915_gem_context *ctx) GEM_BUG_ON(engine->legacy_idx >= I915_NUM_ENGINES); GEM_BUG_ON(e->engines[engine->legacy_idx]); - ce = intel_context_create(ctx, engine); + ce = intel_context_create(engine); if (IS_ERR(ce)) { __free_engines(e, e->num_engines + 1); return ERR_CAST(ce); } + intel_context_set_gem(ce, ctx); + e->engines[engine->legacy_idx] = ce; e->num_engines = max(e->num_engines, engine->legacy_idx); } @@ -705,37 +733,6 @@ i915_gem_create_context(struct drm_i915_private *i915, unsigned int flags) return ctx; } -static void -destroy_kernel_context(struct i915_gem_context **ctxp) -{ - struct i915_gem_context *ctx; - - /* Keep the context ref so that we can free it immediately ourselves */ - ctx = i915_gem_context_get(fetch_and_zero(ctxp)); - GEM_BUG_ON(!i915_gem_context_is_kernel(ctx)); - - context_close(ctx); - i915_gem_context_free(ctx); -} - -struct i915_gem_context * -i915_gem_context_create_kernel(struct drm_i915_private *i915, int prio) -{ - struct i915_gem_context *ctx; - - ctx = i915_gem_create_context(i915, 0); - if (IS_ERR(ctx)) - return ctx; - - i915_gem_context_clear_bannable(ctx); - i915_gem_context_set_persistence(ctx); - ctx->sched.priority = I915_USER_PRIORITY(prio); - - GEM_BUG_ON(!i915_gem_context_is_kernel(ctx)); - - return ctx; -} - static void init_contexts(struct i915_gem_contexts *gc) { spin_lock_init(&gc->lock); @@ -745,32 +742,12 @@ static void init_contexts(struct i915_gem_contexts *gc) init_llist_head(&gc->free_list); } -int i915_gem_init_contexts(struct drm_i915_private *i915) +void i915_gem_init_contexts(struct drm_i915_private *i915) { - struct i915_gem_context *ctx; - - /* Reassure ourselves we are only called once */ - GEM_BUG_ON(i915->kernel_context); - init_contexts(&i915->gem.contexts); - - /* lowest priority; idle task */ - ctx = i915_gem_context_create_kernel(i915, I915_PRIORITY_MIN); - if (IS_ERR(ctx)) { - DRM_ERROR("Failed to create default global context\n"); - return PTR_ERR(ctx); - } - i915->kernel_context = ctx; - DRM_DEBUG_DRIVER("%s context support initialized\n", DRIVER_CAPS(i915)->has_logical_contexts ? "logical" : "fake"); - return 0; -} - -void i915_gem_driver_release__contexts(struct drm_i915_private *i915) -{ - destroy_kernel_context(&i915->kernel_context); } static int context_idr_cleanup(int id, void *p, void *data) @@ -1522,12 +1499,14 @@ set_engines__load_balance(struct i915_user_extension __user *base, void *data) } } - ce = intel_execlists_create_virtual(set->ctx, siblings, n); + ce = intel_execlists_create_virtual(siblings, n); if (IS_ERR(ce)) { err = PTR_ERR(ce); goto out_siblings; } + intel_context_set_gem(ce, set->ctx); + if (cmpxchg(&set->engines->engines[idx], NULL, ce)) { intel_context_put(ce); err = -EEXIST; @@ -1697,12 +1676,14 @@ set_engines(struct i915_gem_context *ctx, return -ENOENT; } - ce = intel_context_create(ctx, engine); + ce = intel_context_create(engine); if (IS_ERR(ce)) { __free_engines(set.engines, n); return PTR_ERR(ce); } + intel_context_set_gem(ce, ctx); + set.engines->engines[n] = ce; } set.engines->num_engines = num_engines; @@ -2003,13 +1984,15 @@ static int clone_engines(struct i915_gem_context *dst, */ if (intel_engine_is_virtual(engine)) clone->engines[n] = - intel_execlists_clone_virtual(dst, engine); + intel_execlists_clone_virtual(engine); else - clone->engines[n] = intel_context_create(dst, engine); + clone->engines[n] = intel_context_create(engine); if (IS_ERR_OR_NULL(clone->engines[n])) { __free_engines(clone, n); goto err_unlock; } + + intel_context_set_gem(clone->engines[n], dst); } clone->num_engines = n; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.h b/drivers/gpu/drm/i915/gem/i915_gem_context.h index 69932899803e..d99fff481d59 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.h @@ -115,8 +115,7 @@ static inline bool i915_gem_context_is_kernel(struct i915_gem_context *ctx) } /* i915_gem_context.c */ -int __must_check i915_gem_init_contexts(struct drm_i915_private *i915); -void i915_gem_driver_release__contexts(struct drm_i915_private *i915); +void i915_gem_init_contexts(struct drm_i915_private *i915); int i915_gem_context_open(struct drm_i915_private *i915, struct drm_file *file); @@ -140,9 +139,6 @@ int i915_gem_context_setparam_ioctl(struct drm_device *dev, void *data, int i915_gem_context_reset_stats_ioctl(struct drm_device *dev, void *data, struct drm_file *file); -struct i915_gem_context * -i915_gem_context_create_kernel(struct drm_i915_private *i915, int prio); - static inline struct i915_gem_context * i915_gem_context_get(struct i915_gem_context *ctx) { diff --git a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c b/drivers/gpu/drm/i915/gem/i915_gem_userptr.c index 54ebc7ab71bc..5dccc9757947 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_userptr.c @@ -778,15 +778,11 @@ i915_gem_userptr_ioctl(struct drm_device *dev, return -EFAULT; if (args->flags & I915_USERPTR_READ_ONLY) { - struct i915_address_space *vm; - /* * On almost all of the older hw, we cannot tell the GPU that * a page is readonly. */ - vm = rcu_dereference_protected(dev_priv->kernel_context->vm, - true); /* static vm */ - if (!vm || !vm->has_read_only) + if (!dev_priv->gt.vm->has_read_only) return -ENODEV; } diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c index 9a509c18b7c7..106a3bfa3bb6 100644 --- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c @@ -315,7 +315,7 @@ static int live_parallel_switch(void *arg) if (!data[m].ce[0]) continue; - ce = intel_context_create(ctx, data[m].ce[0]->engine); + ce = intel_context_create(data[m].ce[0]->engine); if (IS_ERR(ce)) goto out; @@ -1239,8 +1239,7 @@ __igt_ctx_sseu(struct drm_i915_private *i915, hweight32(engine->sseu.slice_mask), hweight32(pg_sseu.slice_mask)); - ce = intel_context_create(engine->kernel_context->gem_context, - engine); + ce = intel_context_create(engine); if (IS_ERR(ce)) { ret = PTR_ERR(ce); goto out_put; diff --git a/drivers/gpu/drm/i915/gem/selftests/mock_context.c b/drivers/gpu/drm/i915/gem/selftests/mock_context.c index 53e89efb09c0..fdf2f120234c 100644 --- a/drivers/gpu/drm/i915/gem/selftests/mock_context.c +++ b/drivers/gpu/drm/i915/gem/selftests/mock_context.c @@ -96,7 +96,16 @@ live_context(struct drm_i915_private *i915, struct file *file) struct i915_gem_context * kernel_context(struct drm_i915_private *i915) { - return i915_gem_context_create_kernel(i915, I915_PRIORITY_NORMAL); + struct i915_gem_context *ctx; + + ctx = i915_gem_create_context(i915, 0); + if (IS_ERR(ctx)) + return ctx; + + GEM_BUG_ON(!i915_gem_context_is_kernel(ctx)); + i915_gem_context_clear_bannable(ctx); + + return ctx; } void kernel_context_close(struct i915_gem_context *ctx) diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c index 7075d03f508f..04aa0759be39 100644 --- a/drivers/gpu/drm/i915/gt/intel_context.c +++ b/drivers/gpu/drm/i915/gt/intel_context.c @@ -31,8 +31,7 @@ void intel_context_free(struct intel_context *ce) } struct intel_context * -intel_context_create(struct i915_gem_context *ctx, - struct intel_engine_cs *engine) +intel_context_create(struct intel_engine_cs *engine) { struct intel_context *ce; @@ -40,7 +39,7 @@ intel_context_create(struct i915_gem_context *ctx, if (!ce) return ERR_PTR(-ENOMEM); - intel_context_init(ce, ctx, engine); + intel_context_init(ce, engine); return ce; } @@ -72,8 +71,6 @@ int __intel_context_do_pin(struct intel_context *ce) ce->engine->name, ce->timeline->fence_context, ce->ring->head, ce->ring->tail); - i915_gem_context_get(ce->gem_context); /* for ctx->ppgtt */ - smp_mb__before_atomic(); /* flush pin before it is visible */ } @@ -103,7 +100,6 @@ void intel_context_unpin(struct intel_context *ce) ce->ops->unpin(ce); - i915_gem_context_put(ce->gem_context); intel_context_active_release(ce); } @@ -198,7 +194,7 @@ int intel_context_active_acquire(struct intel_context *ce) return err; /* Preallocate tracking nodes */ - if (!i915_gem_context_is_kernel(ce->gem_context)) { + if (!intel_context_is_barrier(ce)) { err = i915_active_acquire_preallocate_barrier(&ce->active, ce->engine); if (err) { @@ -219,33 +215,19 @@ void intel_context_active_release(struct intel_context *ce) void intel_context_init(struct intel_context *ce, - struct i915_gem_context *ctx, struct intel_engine_cs *engine) { - struct i915_address_space *vm; - GEM_BUG_ON(!engine->cops); + GEM_BUG_ON(!engine->gt->vm); kref_init(&ce->ref); - ce->gem_context = ctx; - rcu_read_lock(); - vm = rcu_dereference(ctx->vm); - if (vm) - ce->vm = i915_vm_get(vm); - else - ce->vm = i915_vm_get(&engine->gt->ggtt->vm); - rcu_read_unlock(); - if (ctx->timeline) - ce->timeline = intel_timeline_get(ctx->timeline); - if (ctx->sched.priority >= I915_PRIORITY_NORMAL && - intel_engine_has_semaphores(engine)) - __set_bit(CONTEXT_USE_SEMAPHORES, &ce->flags); - ce->engine = engine; ce->ops = engine->cops; ce->sseu = engine->sseu; - ce->ring = __intel_context_ring_size(SZ_16K); + ce->ring = __intel_context_ring_size(SZ_4K); + + ce->vm = i915_vm_get(engine->gt->vm); INIT_LIST_HEAD(&ce->signal_link); INIT_LIST_HEAD(&ce->signals); diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h index d7b667a26e08..80d4c2acc729 100644 --- a/drivers/gpu/drm/i915/gt/intel_context.h +++ b/drivers/gpu/drm/i915/gt/intel_context.h @@ -18,13 +18,11 @@ #include "intel_timeline_types.h" void intel_context_init(struct intel_context *ce, - struct i915_gem_context *ctx, struct intel_engine_cs *engine); void intel_context_fini(struct intel_context *ce); struct intel_context * -intel_context_create(struct i915_gem_context *ctx, - struct intel_engine_cs *engine); +intel_context_create(struct intel_engine_cs *engine); void intel_context_free(struct intel_context *ce); @@ -155,6 +153,11 @@ static inline struct intel_ring *__intel_context_ring_size(u64 sz) return u64_to_ptr(struct intel_ring, sz); } +static inline bool intel_context_is_barrier(const struct intel_context *ce) +{ + return test_bit(CONTEXT_BARRIER_BIT, &ce->flags); +} + static inline bool intel_context_use_semaphores(const struct intel_context *ce) { return test_bit(CONTEXT_USE_SEMAPHORES, &ce->flags); diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h index 264c1efde772..ba46c36a88a2 100644 --- a/drivers/gpu/drm/i915/gt/intel_context_types.h +++ b/drivers/gpu/drm/i915/gt/intel_context_types.h @@ -54,11 +54,12 @@ struct intel_context { struct intel_timeline *timeline; unsigned long flags; -#define CONTEXT_ALLOC_BIT 0 -#define CONTEXT_USE_SEMAPHORES 1 -#define CONTEXT_BANNED 2 -#define CONTEXT_FORCE_SINGLE_SUBMISSION 3 -#define CONTEXT_NOPREEMPT 4 +#define CONTEXT_BARRIER_BIT 0 +#define CONTEXT_ALLOC_BIT 1 +#define CONTEXT_USE_SEMAPHORES 2 +#define CONTEXT_BANNED 3 +#define CONTEXT_FORCE_SINGLE_SUBMISSION 4 +#define CONTEXT_NOPREEMPT 5 u32 *lrc_reg_state; u64 lrc_desc; diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c index 7f79741ad684..a27b22914fb2 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c @@ -757,11 +757,11 @@ create_kernel_context(struct intel_engine_cs *engine) struct intel_context *ce; int err; - ce = intel_context_create(engine->i915->kernel_context, engine); + ce = intel_context_create(engine); if (IS_ERR(ce)) return ce; - ce->ring = __intel_context_ring_size(SZ_4K); + __set_bit(CONTEXT_BARRIER_BIT, &ce->flags); err = intel_context_pin(ce); if (err) { @@ -798,6 +798,12 @@ int intel_engine_init_common(struct intel_engine_cs *engine) engine->set_default_submission(engine); + ret = measure_breadcrumb_dw(engine); + if (ret < 0) + return ret; + + engine->emit_fini_breadcrumb_dw = ret; + /* * We may need to do things with the shrinker which * require us to immediately switch back to the default @@ -812,18 +818,7 @@ int intel_engine_init_common(struct intel_engine_cs *engine) engine->kernel_context = ce; - ret = measure_breadcrumb_dw(engine); - if (ret < 0) - goto err_unpin; - - engine->emit_fini_breadcrumb_dw = ret; - return 0; - -err_unpin: - intel_context_unpin(ce); - intel_context_put(ce); - return ret; } /** diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c index c39b21c8d328..4c078247825d 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -74,7 +74,6 @@ int intel_gt_init_hw(struct intel_gt *gt) struct intel_uncore *uncore = gt->uncore; int ret; - BUG_ON(!i915->kernel_context); ret = intel_gt_terminally_wedged(gt); if (ret) return ret; @@ -365,6 +364,14 @@ static void intel_gt_fini_scratch(struct intel_gt *gt) i915_vma_unpin_and_release(>->scratch, 0); } +static struct i915_address_space *kernel_vm(struct intel_gt *gt) +{ + if (INTEL_PPGTT(gt->i915) > INTEL_PPGTT_ALIASING) + return &i915_ppgtt_create(gt->i915)->vm; + else + return i915_vm_get(>->ggtt->vm); +} + int intel_gt_init(struct intel_gt *gt) { int err; @@ -375,7 +382,17 @@ int intel_gt_init(struct intel_gt *gt) intel_gt_pm_init(gt); + gt->vm = kernel_vm(gt); + if (!gt->vm) { + err = -ENOMEM; + goto err_scratch; + } + return 0; + +err_scratch: + intel_gt_fini_scratch(gt); + return err; } void intel_gt_driver_remove(struct intel_gt *gt) @@ -390,6 +407,12 @@ void intel_gt_driver_unregister(struct intel_gt *gt) void intel_gt_driver_release(struct intel_gt *gt) { + struct i915_address_space *vm; + + vm = fetch_and_zero(>->vm); + if (vm) /* FIXME being called twice on error paths :( */ + i915_vm_put(vm); + intel_gt_pm_fini(gt); intel_gt_fini_scratch(gt); } diff --git a/drivers/gpu/drm/i915/gt/intel_gt_types.h b/drivers/gpu/drm/i915/gt/intel_gt_types.h index d4e14dbd172e..96890dd12b5f 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_types.h +++ b/drivers/gpu/drm/i915/gt/intel_gt_types.h @@ -90,6 +90,13 @@ struct intel_gt { struct intel_engine_cs *engine[I915_NUM_ENGINES]; struct intel_engine_cs *engine_class[MAX_ENGINE_CLASS + 1] [MAX_ENGINE_INSTANCE + 1]; + + /* + * Default address space (either GGTT or ppGTT depending on arch). + * + * Reserved for exclusive use by the kernel. + */ + struct i915_address_space *vm; }; enum intel_gt_scratch_field { diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c index 77efdc3af4ac..84b452d17ec9 100644 --- a/drivers/gpu/drm/i915/gt/intel_lrc.c +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c @@ -133,12 +133,11 @@ */ #include -#include "gem/i915_gem_context.h" - #include "i915_drv.h" #include "i915_perf.h" #include "i915_trace.h" #include "i915_vgpu.h" +#include "intel_context.h" #include "intel_engine_pm.h" #include "intel_gt.h" #include "intel_gt_pm.h" @@ -1296,7 +1295,8 @@ assert_pending_valid(const struct intel_engine_execlists *execlists, if (i915_request_completed(rq)) continue; - if (i915_active_is_idle(&ce->active) && ce->gem_context) { + if (i915_active_is_idle(&ce->active) && + !intel_context_is_barrier(ce)) { GEM_TRACE_ERR("Inactive context:%llx in pending[%zd]\n", ce->timeline->fence_context, port - execlists->pending); @@ -4427,8 +4427,7 @@ virtual_bond_execute(struct i915_request *rq, struct dma_fence *signal) } struct intel_context * -intel_execlists_create_virtual(struct i915_gem_context *ctx, - struct intel_engine_cs **siblings, +intel_execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count) { struct virtual_engine *ve; @@ -4439,13 +4438,13 @@ intel_execlists_create_virtual(struct i915_gem_context *ctx, return ERR_PTR(-EINVAL); if (count == 1) - return intel_context_create(ctx, siblings[0]); + return intel_context_create(siblings[0]); ve = kzalloc(struct_size(ve, siblings, count), GFP_KERNEL); if (!ve) return ERR_PTR(-ENOMEM); - ve->base.i915 = ctx->i915; + ve->base.i915 = siblings[0]->i915; ve->base.gt = siblings[0]->gt; ve->base.uncore = siblings[0]->uncore; ve->base.id = -1; @@ -4488,7 +4487,7 @@ intel_execlists_create_virtual(struct i915_gem_context *ctx, virtual_submission_tasklet, (unsigned long)ve); - intel_context_init(&ve->context, ctx, &ve->base); + intel_context_init(&ve->context, &ve->base); for (n = 0; n < count; n++) { struct intel_engine_cs *sibling = siblings[n]; @@ -4569,14 +4568,12 @@ intel_execlists_create_virtual(struct i915_gem_context *ctx, } struct intel_context * -intel_execlists_clone_virtual(struct i915_gem_context *ctx, - struct intel_engine_cs *src) +intel_execlists_clone_virtual(struct intel_engine_cs *src) { struct virtual_engine *se = to_virtual_engine(src); struct intel_context *dst; - dst = intel_execlists_create_virtual(ctx, - se->siblings, + dst = intel_execlists_create_virtual(se->siblings, se->num_siblings); if (IS_ERR(dst)) return dst; diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.h b/drivers/gpu/drm/i915/gt/intel_lrc.h index 04511d8ebdc1..081521f17c74 100644 --- a/drivers/gpu/drm/i915/gt/intel_lrc.h +++ b/drivers/gpu/drm/i915/gt/intel_lrc.h @@ -111,13 +111,11 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine, unsigned int max); struct intel_context * -intel_execlists_create_virtual(struct i915_gem_context *ctx, - struct intel_engine_cs **siblings, +intel_execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count); struct intel_context * -intel_execlists_clone_virtual(struct i915_gem_context *ctx, - struct intel_engine_cs *src); +intel_execlists_clone_virtual(struct intel_engine_cs *src); int intel_virtual_engine_attach_bond(struct intel_engine_cs *engine, const struct intel_engine_cs *master, diff --git a/drivers/gpu/drm/i915/gt/intel_reset.c b/drivers/gpu/drm/i915/gt/intel_reset.c index 0d4c610a90b7..9ecf24439fde 100644 --- a/drivers/gpu/drm/i915/gt/intel_reset.c +++ b/drivers/gpu/drm/i915/gt/intel_reset.c @@ -52,9 +52,8 @@ static void engine_skip_context(struct i915_request *rq) i915_request_skip(rq, -EIO); } -static void client_mark_guilty(struct i915_request *rq, bool banned) +static void client_mark_guilty(struct i915_gem_context *ctx, bool banned) { - struct i915_gem_context *ctx = rq->context->gem_context; struct drm_i915_file_private *file_priv = ctx->file_priv; unsigned long prev_hang; unsigned int score; @@ -81,11 +80,15 @@ static void client_mark_guilty(struct i915_request *rq, bool banned) static bool mark_guilty(struct i915_request *rq) { - struct i915_gem_context *ctx = rq->context->gem_context; + struct i915_gem_context *ctx; unsigned long prev_hang; bool banned; int i; + ctx = rq->context->gem_context; + if (!ctx) + return false; + if (i915_gem_context_is_closed(ctx)) { intel_context_set_banned(rq->context); return true; @@ -117,14 +120,15 @@ static bool mark_guilty(struct i915_request *rq) intel_context_set_banned(rq->context); } - client_mark_guilty(rq, banned); + client_mark_guilty(ctx, banned); return banned; } static void mark_innocent(struct i915_request *rq) { - atomic_inc(&rq->context->gem_context->active_count); + if (rq->context->gem_context) + atomic_inc(&rq->context->gem_context->active_count); } void __i915_request_reset(struct i915_request *rq, bool guilty) diff --git a/drivers/gpu/drm/i915/gt/intel_ring_submission.c b/drivers/gpu/drm/i915/gt/intel_ring_submission.c index 1c2f3c3a8840..6ddfe8a86dbe 100644 --- a/drivers/gpu/drm/i915/gt/intel_ring_submission.c +++ b/drivers/gpu/drm/i915/gt/intel_ring_submission.c @@ -1558,7 +1558,7 @@ static int remap_l3(struct i915_request *rq) struct i915_gem_context *ctx = rq->context->gem_context; int i, err; - if (!ctx->remap_slice) + if (!ctx || !ctx->remap_slice) return 0; for (i = 0; i < MAX_L3_SLICES; i++) { @@ -1601,7 +1601,7 @@ static int switch_context(struct i915_request *rq) * is purely used for flushing user contexts. */ hw_flags = 0; - if (i915_gem_context_is_kernel(rq->context->gem_context)) + if (!rq->context->gem_context) hw_flags = MI_RESTORE_INHIBIT; ret = mi_set_context(rq, hw_flags); diff --git a/drivers/gpu/drm/i915/gt/selftest_context.c b/drivers/gpu/drm/i915/gt/selftest_context.c index 14ba6ceb9177..b5ef95a99358 100644 --- a/drivers/gpu/drm/i915/gt/selftest_context.c +++ b/drivers/gpu/drm/i915/gt/selftest_context.c @@ -67,15 +67,14 @@ static int context_sync(struct intel_context *ce) return err; } -static int __live_context_size(struct intel_engine_cs *engine, - struct i915_gem_context *fixme) +static int __live_context_size(struct intel_engine_cs *engine) { struct intel_context *ce; struct i915_request *rq; void *vaddr; int err; - ce = intel_context_create(fixme, engine); + ce = intel_context_create(engine); if (IS_ERR(ce)) return PTR_ERR(ce); @@ -143,7 +142,6 @@ static int live_context_size(void *arg) { struct intel_gt *gt = arg; struct intel_engine_cs *engine; - struct i915_gem_context *fixme; enum intel_engine_id id; int err = 0; @@ -152,10 +150,6 @@ static int live_context_size(void *arg) * HW tries to write past the end of one. */ - fixme = kernel_context(gt->i915); - if (IS_ERR(fixme)) - return PTR_ERR(fixme); - for_each_engine(engine, gt, id) { struct { struct drm_i915_gem_object *state; @@ -180,7 +174,7 @@ static int live_context_size(void *arg) /* Overlaps with the execlists redzone */ engine->context_size += I915_GTT_PAGE_SIZE; - err = __live_context_size(engine, fixme); + err = __live_context_size(engine); engine->context_size -= I915_GTT_PAGE_SIZE; @@ -193,12 +187,10 @@ static int live_context_size(void *arg) break; } - kernel_context_close(fixme); return err; } -static int __live_active_context(struct intel_engine_cs *engine, - struct i915_gem_context *fixme) +static int __live_active_context(struct intel_engine_cs *engine) { struct intel_context *ce; int pass; @@ -223,7 +215,7 @@ static int __live_active_context(struct intel_engine_cs *engine, return -EINVAL; } - ce = intel_context_create(fixme, engine); + ce = intel_context_create(engine); if (IS_ERR(ce)) return PTR_ERR(ce); @@ -287,23 +279,11 @@ static int live_active_context(void *arg) { struct intel_gt *gt = arg; struct intel_engine_cs *engine; - struct i915_gem_context *fixme; enum intel_engine_id id; - struct file *file; int err = 0; - file = mock_file(gt->i915); - if (IS_ERR(file)) - return PTR_ERR(file); - - fixme = live_context(gt->i915, file); - if (IS_ERR(fixme)) { - err = PTR_ERR(fixme); - goto out_file; - } - for_each_engine(engine, gt, id) { - err = __live_active_context(engine, fixme); + err = __live_active_context(engine); if (err) break; @@ -312,8 +292,6 @@ static int live_active_context(void *arg) break; } -out_file: - fput(file); return err; } @@ -345,8 +323,7 @@ static int __remote_sync(struct intel_context *ce, struct intel_context *remote) return err; } -static int __live_remote_context(struct intel_engine_cs *engine, - struct i915_gem_context *fixme) +static int __live_remote_context(struct intel_engine_cs *engine) { struct intel_context *local, *remote; int pass; @@ -360,11 +337,11 @@ static int __live_remote_context(struct intel_engine_cs *engine, * clobber the idle-barrier. */ - remote = intel_context_create(fixme, engine); + remote = intel_context_create(engine); if (IS_ERR(remote)) return PTR_ERR(remote); - local = intel_context_create(fixme, engine); + local = intel_context_create(engine); if (IS_ERR(local)) { err = PTR_ERR(local); goto err_remote; @@ -397,23 +374,11 @@ static int live_remote_context(void *arg) { struct intel_gt *gt = arg; struct intel_engine_cs *engine; - struct i915_gem_context *fixme; enum intel_engine_id id; - struct file *file; int err = 0; - file = mock_file(gt->i915); - if (IS_ERR(file)) - return PTR_ERR(file); - - fixme = live_context(gt->i915, file); - if (IS_ERR(fixme)) { - err = PTR_ERR(fixme); - goto out_file; - } - for_each_engine(engine, gt, id) { - err = __live_remote_context(engine, fixme); + err = __live_remote_context(engine); if (err) break; @@ -422,8 +387,6 @@ static int live_remote_context(void *arg) break; } -out_file: - fput(file); return err; } diff --git a/drivers/gpu/drm/i915/gt/selftest_engine_heartbeat.c b/drivers/gpu/drm/i915/gt/selftest_engine_heartbeat.c index f665a0e23c61..2feead9b447a 100644 --- a/drivers/gpu/drm/i915/gt/selftest_engine_heartbeat.c +++ b/drivers/gpu/drm/i915/gt/selftest_engine_heartbeat.c @@ -173,8 +173,7 @@ static int __live_heartbeat_fast(struct intel_engine_cs *engine) int err; int i; - ce = intel_context_create(engine->kernel_context->gem_context, - engine); + ce = intel_context_create(engine); if (IS_ERR(ce)) return PTR_ERR(ce); diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c index d155c9374453..ff2d8af282a6 100644 --- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c +++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c @@ -377,36 +377,30 @@ static int igt_reset_nop(void *arg) struct intel_gt *gt = arg; struct i915_gpu_error *global = >->i915->gpu_error; struct intel_engine_cs *engine; - struct i915_gem_context *ctx; unsigned int reset_count, count; enum intel_engine_id id; IGT_TIMEOUT(end_time); - struct file *file; int err = 0; /* Check that we can reset during non-user portions of requests */ - file = mock_file(gt->i915); - if (IS_ERR(file)) - return PTR_ERR(file); - - ctx = live_context(gt->i915, file); - if (IS_ERR(ctx)) { - err = PTR_ERR(ctx); - goto out; - } - - i915_gem_context_clear_bannable(ctx); reset_count = i915_reset_count(global); count = 0; do { for_each_engine(engine, gt, id) { + struct intel_context *ce; int i; + ce = intel_context_create(engine); + if (IS_ERR(ce)) { + err = PTR_ERR(ce); + break; + } + for (i = 0; i < 16; i++) { struct i915_request *rq; - rq = igt_request_alloc(ctx, engine); + rq = intel_context_create_request(ce); if (IS_ERR(rq)) { err = PTR_ERR(rq); break; @@ -414,6 +408,8 @@ static int igt_reset_nop(void *arg) i915_request_add(rq); } + + intel_context_put(ce); } igt_global_reset_lock(gt); @@ -437,10 +433,7 @@ static int igt_reset_nop(void *arg) } while (time_before(jiffies, end_time)); pr_info("%s: %d resets\n", __func__, count); - err = igt_flush_test(gt->i915); -out: - fput(file); - if (intel_gt_is_wedged(gt)) + if (igt_flush_test(gt->i915)) err = -EIO; return err; } @@ -450,31 +443,22 @@ static int igt_reset_nop_engine(void *arg) struct intel_gt *gt = arg; struct i915_gpu_error *global = >->i915->gpu_error; struct intel_engine_cs *engine; - struct i915_gem_context *ctx; enum intel_engine_id id; - struct file *file; - int err = 0; /* Check that we can engine-reset during non-user portions */ if (!intel_has_reset_engine(gt)) return 0; - file = mock_file(gt->i915); - if (IS_ERR(file)) - return PTR_ERR(file); - - ctx = live_context(gt->i915, file); - if (IS_ERR(ctx)) { - err = PTR_ERR(ctx); - goto out; - } - - i915_gem_context_clear_bannable(ctx); for_each_engine(engine, gt, id) { - unsigned int reset_count, reset_engine_count; - unsigned int count; + unsigned int reset_count, reset_engine_count, count; + struct intel_context *ce; IGT_TIMEOUT(end_time); + int err; + + ce = intel_context_create(engine); + if (IS_ERR(ce)) + return PTR_ERR(ce); reset_count = i915_reset_count(global); reset_engine_count = i915_reset_engine_count(global, engine); @@ -494,7 +478,7 @@ static int igt_reset_nop_engine(void *arg) for (i = 0; i < 16; i++) { struct i915_request *rq; - rq = igt_request_alloc(ctx, engine); + rq = intel_context_create_request(ce); if (IS_ERR(rq)) { err = PTR_ERR(rq); break; @@ -525,20 +509,14 @@ static int igt_reset_nop_engine(void *arg) clear_bit(I915_RESET_ENGINE + id, >->reset.flags); pr_info("%s(%s): %d resets\n", __func__, engine->name, count); + intel_context_put(ce); + if (igt_flush_test(gt->i915)) + err = -EIO; if (err) - break; - - err = igt_flush_test(gt->i915); - if (err) - break; + return err; } - err = igt_flush_test(gt->i915); -out: - fput(file); - if (intel_gt_is_wedged(gt)) - err = -EIO; - return err; + return 0; } static int __igt_reset_engine(struct intel_gt *gt, bool active) @@ -699,43 +677,43 @@ static int active_engine(void *data) struct active_engine *arg = data; struct intel_engine_cs *engine = arg->engine; struct i915_request *rq[8] = {}; - struct i915_gem_context *ctx[ARRAY_SIZE(rq)]; - unsigned long count = 0; - struct file *file; + struct intel_context *ce[ARRAY_SIZE(rq)]; + unsigned long count; int err = 0; - file = mock_file(engine->i915); - if (IS_ERR(file)) - return PTR_ERR(file); - - for (count = 0; count < ARRAY_SIZE(ctx); count++) { - ctx[count] = live_context(engine->i915, file); - if (IS_ERR(ctx[count])) { - err = PTR_ERR(ctx[count]); + for (count = 0; count < ARRAY_SIZE(ce); count++) { + ce[count] = intel_context_create(engine); + if (IS_ERR(ce[count])) { + err = PTR_ERR(ce[count]); while (--count) - i915_gem_context_put(ctx[count]); - goto err_file; + intel_context_put(ce[count]); + return err; } } + count = 0; while (!kthread_should_stop()) { unsigned int idx = count++ & (ARRAY_SIZE(rq) - 1); struct i915_request *old = rq[idx]; struct i915_request *new; - new = igt_request_alloc(ctx[idx], engine); + new = intel_context_create_request(ce[idx]); if (IS_ERR(new)) { err = PTR_ERR(new); break; } - if (arg->flags & TEST_PRIORITY) - ctx[idx]->sched.priority = - i915_prandom_u32_max_state(512, &prng); - rq[idx] = i915_request_get(new); i915_request_add(new); + if (engine->schedule && arg->flags & TEST_PRIORITY) { + struct i915_sched_attr attr = { + .priority = + i915_prandom_u32_max_state(512, &prng), + }; + engine->schedule(rq[idx], &attr); + } + err = active_request_put(old); if (err) break; @@ -749,10 +727,10 @@ static int active_engine(void *data) /* Keep the first error */ if (!err) err = err__; + + intel_context_put(ce[count]); } -err_file: - fput(file); return err; } @@ -1300,32 +1278,21 @@ static int igt_reset_evict_ggtt(void *arg) static int igt_reset_evict_ppgtt(void *arg) { struct intel_gt *gt = arg; - struct i915_gem_context *ctx; - struct i915_address_space *vm; - struct file *file; + struct i915_ppgtt *ppgtt; int err; - file = mock_file(gt->i915); - if (IS_ERR(file)) - return PTR_ERR(file); + /* aliasing == global gtt locking, covered above */ + if (INTEL_PPGTT(gt->i915) < INTEL_PPGTT_FULL) + return 0; - ctx = live_context(gt->i915, file); - if (IS_ERR(ctx)) { - err = PTR_ERR(ctx); - goto out; - } + ppgtt = i915_ppgtt_create(gt->i915); + if (IS_ERR(ppgtt)) + return PTR_ERR(ppgtt); - err = 0; - vm = i915_gem_context_get_vm_rcu(ctx); - if (!i915_is_ggtt(vm)) { - /* aliasing == global gtt locking, covered above */ - err = __igt_reset_evict_vma(gt, vm, - evict_vma, EXEC_OBJECT_WRITE); - } - i915_vm_put(vm); + err = __igt_reset_evict_vma(gt, &ppgtt->vm, + evict_vma, EXEC_OBJECT_WRITE); + i915_vm_put(&ppgtt->vm); -out: - fput(file); return err; } diff --git a/drivers/gpu/drm/i915/gt/selftest_lrc.c b/drivers/gpu/drm/i915/gt/selftest_lrc.c index 7d5713ea6bde..0a9ea43ee6b7 100644 --- a/drivers/gpu/drm/i915/gt/selftest_lrc.c +++ b/drivers/gpu/drm/i915/gt/selftest_lrc.c @@ -53,11 +53,10 @@ static struct i915_vma *create_scratch(struct intel_gt *gt) static int live_sanitycheck(void *arg) { struct intel_gt *gt = arg; - struct i915_gem_engines_iter it; - struct i915_gem_context *ctx; - struct intel_context *ce; + struct intel_engine_cs *engine; + enum intel_engine_id id; struct igt_spinner spin; - int err = -ENOMEM; + int err = 0; if (!HAS_LOGICAL_RING_CONTEXTS(gt->i915)) return 0; @@ -65,17 +64,20 @@ static int live_sanitycheck(void *arg) if (igt_spinner_init(&spin, gt)) return -ENOMEM; - ctx = kernel_context(gt->i915); - if (!ctx) - goto err_spin; - - for_each_gem_engine(ce, i915_gem_context_lock_engines(ctx), it) { + for_each_engine(engine, gt, id) { + struct intel_context *ce; struct i915_request *rq; + ce = intel_context_create(engine); + if (IS_ERR(ce)) { + err = PTR_ERR(ce); + break; + } + rq = igt_spinner_create_request(&spin, ce, MI_NOOP); if (IS_ERR(rq)) { err = PTR_ERR(rq); - goto err_ctx; + goto out_ctx; } i915_request_add(rq); @@ -84,21 +86,21 @@ static int live_sanitycheck(void *arg) GEM_TRACE_DUMP(); intel_gt_set_wedged(gt); err = -EIO; - goto err_ctx; + goto out_ctx; } igt_spinner_end(&spin); if (igt_flush_test(gt->i915)) { err = -EIO; - goto err_ctx; + goto out_ctx; } + +out_ctx: + intel_context_put(ce); + if (err) + break; } - err = 0; -err_ctx: - i915_gem_context_unlock_engines(ctx); - kernel_context_close(ctx); -err_spin: igt_spinner_fini(&spin); return err; } @@ -106,7 +108,6 @@ static int live_sanitycheck(void *arg) static int live_unlite_restore(struct intel_gt *gt, int prio) { struct intel_engine_cs *engine; - struct i915_gem_context *ctx; enum intel_engine_id id; struct igt_spinner spin; int err = -ENOMEM; @@ -119,10 +120,6 @@ static int live_unlite_restore(struct intel_gt *gt, int prio) if (igt_spinner_init(&spin, gt)) return err; - ctx = kernel_context(gt->i915); - if (!ctx) - goto err_spin; - err = 0; for_each_engine(engine, gt, id) { struct intel_context *ce[2] = {}; @@ -144,7 +141,7 @@ static int live_unlite_restore(struct intel_gt *gt, int prio) for (n = 0; n < ARRAY_SIZE(ce); n++) { struct intel_context *tmp; - tmp = intel_context_create(ctx, engine); + tmp = intel_context_create(engine); if (IS_ERR(tmp)) { err = PTR_ERR(tmp); goto err_ce; @@ -253,8 +250,6 @@ static int live_unlite_restore(struct intel_gt *gt, int prio) break; } - kernel_context_close(ctx); -err_spin: igt_spinner_fini(&spin); return err; } @@ -309,17 +304,17 @@ emit_semaphore_chain(struct i915_request *rq, struct i915_vma *vma, int idx) static struct i915_request * semaphore_queue(struct intel_engine_cs *engine, struct i915_vma *vma, int idx) { - struct i915_gem_context *ctx; + struct intel_context *ce; struct i915_request *rq; int err; - ctx = kernel_context(engine->i915); - if (!ctx) - return ERR_PTR(-ENOMEM); + ce = intel_context_create(engine); + if (IS_ERR(ce)) + return ERR_CAST(ce); - rq = igt_request_alloc(ctx, engine); + rq = intel_context_create_request(ce); if (IS_ERR(rq)) - goto out_ctx; + goto out_ce; err = 0; if (rq->engine->emit_init_breadcrumb) @@ -332,8 +327,8 @@ semaphore_queue(struct intel_engine_cs *engine, struct i915_vma *vma, int idx) if (err) rq = ERR_PTR(err); -out_ctx: - kernel_context_close(ctx); +out_ce: + intel_context_put(ce); return rq; } @@ -2392,27 +2387,17 @@ static int nop_virtual_engine(struct intel_gt *gt, { IGT_TIMEOUT(end_time); struct i915_request *request[16]; - struct i915_gem_context *ctx[16]; struct intel_context *ve[16]; unsigned long n, prime, nc; struct igt_live_test t; ktime_t times[2] = {}; int err; - GEM_BUG_ON(!nctx || nctx > ARRAY_SIZE(ctx)); + GEM_BUG_ON(!nctx || nctx > ARRAY_SIZE(ve)); for (n = 0; n < nctx; n++) { - ctx[n] = kernel_context(gt->i915); - if (!ctx[n]) { - err = -ENOMEM; - nctx = n; - goto out; - } - - ve[n] = intel_execlists_create_virtual(ctx[n], - siblings, nsibling); + ve[n] = intel_execlists_create_virtual(siblings, nsibling); if (IS_ERR(ve[n])) { - kernel_context_close(ctx[n]); err = PTR_ERR(ve[n]); nctx = n; goto out; @@ -2421,7 +2406,6 @@ static int nop_virtual_engine(struct intel_gt *gt, err = intel_context_pin(ve[n]); if (err) { intel_context_put(ve[n]); - kernel_context_close(ctx[n]); nctx = n; goto out; } @@ -2502,7 +2486,6 @@ static int nop_virtual_engine(struct intel_gt *gt, for (nc = 0; nc < nctx; nc++) { intel_context_unpin(ve[nc]); intel_context_put(ve[nc]); - kernel_context_close(ctx[nc]); } return err; } @@ -2561,7 +2544,6 @@ static int mask_virtual_engine(struct intel_gt *gt, unsigned int nsibling) { struct i915_request *request[MAX_ENGINE_INSTANCE + 1]; - struct i915_gem_context *ctx; struct intel_context *ve; struct igt_live_test t; unsigned int n; @@ -2572,11 +2554,7 @@ static int mask_virtual_engine(struct intel_gt *gt, * restrict it to our desired engine within the virtual engine. */ - ctx = kernel_context(gt->i915); - if (!ctx) - return -ENOMEM; - - ve = intel_execlists_create_virtual(ctx, siblings, nsibling); + ve = intel_execlists_create_virtual(siblings, nsibling); if (IS_ERR(ve)) { err = PTR_ERR(ve); goto out_close; @@ -2644,7 +2622,6 @@ static int mask_virtual_engine(struct intel_gt *gt, out_put: intel_context_put(ve); out_close: - kernel_context_close(ctx); return err; } @@ -2684,7 +2661,6 @@ static int preserved_virtual_engine(struct intel_gt *gt, unsigned int nsibling) { struct i915_request *last = NULL; - struct i915_gem_context *ctx; struct intel_context *ve; struct i915_vma *scratch; struct igt_live_test t; @@ -2692,17 +2668,11 @@ static int preserved_virtual_engine(struct intel_gt *gt, int err = 0; u32 *cs; - ctx = kernel_context(gt->i915); - if (!ctx) - return -ENOMEM; - scratch = create_scratch(siblings[0]->gt); - if (IS_ERR(scratch)) { - err = PTR_ERR(scratch); - goto out_close; - } + if (IS_ERR(scratch)) + return PTR_ERR(scratch); - ve = intel_execlists_create_virtual(ctx, siblings, nsibling); + ve = intel_execlists_create_virtual(siblings, nsibling); if (IS_ERR(ve)) { err = PTR_ERR(ve); goto out_scratch; @@ -2785,8 +2755,6 @@ static int preserved_virtual_engine(struct intel_gt *gt, intel_context_put(ve); out_scratch: i915_vma_unpin_and_release(&scratch, 0); -out_close: - kernel_context_close(ctx); return err; } @@ -2838,7 +2806,6 @@ static int bond_virtual_engine(struct intel_gt *gt, #define BOND_SCHEDULE BIT(0) { struct intel_engine_cs *master; - struct i915_gem_context *ctx; struct i915_request *rq[16]; enum intel_engine_id id; unsigned long n; @@ -2846,10 +2813,6 @@ static int bond_virtual_engine(struct intel_gt *gt, GEM_BUG_ON(nsibling >= ARRAY_SIZE(rq) - 1); - ctx = kernel_context(gt->i915); - if (!ctx) - return -ENOMEM; - err = 0; rq[0] = ERR_PTR(-ENOMEM); for_each_engine(master, gt, id) { @@ -2860,7 +2823,7 @@ static int bond_virtual_engine(struct intel_gt *gt, memset_p((void *)rq, ERR_PTR(-EINVAL), ARRAY_SIZE(rq)); - rq[0] = igt_request_alloc(ctx, master); + rq[0] = i915_request_create(master->kernel_context); if (IS_ERR(rq[0])) { err = PTR_ERR(rq[0]); goto out; @@ -2880,9 +2843,7 @@ static int bond_virtual_engine(struct intel_gt *gt, for (n = 0; n < nsibling; n++) { struct intel_context *ve; - ve = intel_execlists_create_virtual(ctx, - siblings, - nsibling); + ve = intel_execlists_create_virtual(siblings, nsibling); if (IS_ERR(ve)) { err = PTR_ERR(ve); onstack_fence_fini(&fence); @@ -2960,7 +2921,6 @@ static int bond_virtual_engine(struct intel_gt *gt, if (igt_flush_test(gt->i915)) err = -EIO; - kernel_context_close(ctx); return err; } @@ -3270,8 +3230,7 @@ static int live_lrc_fixed(void *arg) return err; } -static int __live_lrc_state(struct i915_gem_context *fixme, - struct intel_engine_cs *engine, +static int __live_lrc_state(struct intel_engine_cs *engine, struct i915_vma *scratch) { struct intel_context *ce; @@ -3286,7 +3245,7 @@ static int __live_lrc_state(struct i915_gem_context *fixme, int err; int n; - ce = intel_context_create(fixme, engine); + ce = intel_context_create(engine); if (IS_ERR(ce)) return PTR_ERR(ce); @@ -3360,7 +3319,6 @@ static int live_lrc_state(void *arg) { struct intel_gt *gt = arg; struct intel_engine_cs *engine; - struct i915_gem_context *fixme; struct i915_vma *scratch; enum intel_engine_id id; int err = 0; @@ -3370,18 +3328,12 @@ static int live_lrc_state(void *arg) * intel_context. */ - fixme = kernel_context(gt->i915); - if (!fixme) - return -ENOMEM; - scratch = create_scratch(gt); - if (IS_ERR(scratch)) { - err = PTR_ERR(scratch); - goto out_close; - } + if (IS_ERR(scratch)) + return PTR_ERR(scratch); for_each_engine(engine, gt, id) { - err = __live_lrc_state(fixme, engine, scratch); + err = __live_lrc_state(engine, scratch); if (err) break; } @@ -3390,8 +3342,6 @@ static int live_lrc_state(void *arg) err = -EIO; i915_vma_unpin_and_release(&scratch, 0); -out_close: - kernel_context_close(fixme); return err; } @@ -3424,8 +3374,7 @@ static int gpr_make_dirty(struct intel_engine_cs *engine) return 0; } -static int __live_gpr_clear(struct i915_gem_context *fixme, - struct intel_engine_cs *engine, +static int __live_gpr_clear(struct intel_engine_cs *engine, struct i915_vma *scratch) { struct intel_context *ce; @@ -3441,7 +3390,7 @@ static int __live_gpr_clear(struct i915_gem_context *fixme, if (err) return err; - ce = intel_context_create(fixme, engine); + ce = intel_context_create(engine); if (IS_ERR(ce)) return PTR_ERR(ce); @@ -3503,7 +3452,6 @@ static int live_gpr_clear(void *arg) { struct intel_gt *gt = arg; struct intel_engine_cs *engine; - struct i915_gem_context *fixme; struct i915_vma *scratch; enum intel_engine_id id; int err = 0; @@ -3513,18 +3461,12 @@ static int live_gpr_clear(void *arg) * to avoid leaking any information from previous contexts. */ - fixme = kernel_context(gt->i915); - if (!fixme) - return -ENOMEM; - scratch = create_scratch(gt); - if (IS_ERR(scratch)) { - err = PTR_ERR(scratch); - goto out_close; - } + if (IS_ERR(scratch)) + return PTR_ERR(scratch); for_each_engine(engine, gt, id) { - err = __live_gpr_clear(fixme, engine, scratch); + err = __live_gpr_clear(engine, scratch); if (err) break; } @@ -3533,8 +3475,6 @@ static int live_gpr_clear(void *arg) err = -EIO; i915_vma_unpin_and_release(&scratch, 0); -out_close: - kernel_context_close(fixme); return err; } diff --git a/drivers/gpu/drm/i915/gt/selftest_workarounds.c b/drivers/gpu/drm/i915/gt/selftest_workarounds.c index d5d1e1a32187..ac1921854cbf 100644 --- a/drivers/gpu/drm/i915/gt/selftest_workarounds.c +++ b/drivers/gpu/drm/i915/gt/selftest_workarounds.c @@ -264,22 +264,15 @@ static int switch_to_scratch_context(struct intel_engine_cs *engine, struct igt_spinner *spin) { - struct i915_gem_context *ctx; struct intel_context *ce; struct i915_request *rq; int err = 0; - ctx = kernel_context(engine->i915); - if (IS_ERR(ctx)) - return PTR_ERR(ctx); - - GEM_BUG_ON(i915_gem_context_is_bannable(ctx)); - - ce = i915_gem_context_get_engine(ctx, engine->legacy_idx); - GEM_BUG_ON(IS_ERR(ce)); + ce = intel_context_create(engine); + if (IS_ERR(ce)) + return PTR_ERR(ce); rq = igt_spinner_create_request(spin, ce, MI_NOOP); - intel_context_put(ce); if (IS_ERR(rq)) { @@ -293,7 +286,6 @@ switch_to_scratch_context(struct intel_engine_cs *engine, if (err && spin) igt_spinner_end(spin); - kernel_context_close(ctx); return err; } @@ -367,20 +359,17 @@ static int check_whitelist_across_reset(struct intel_engine_cs *engine, return err; } -static struct i915_vma *create_batch(struct i915_gem_context *ctx) +static struct i915_vma *create_batch(struct i915_address_space *vm) { struct drm_i915_gem_object *obj; - struct i915_address_space *vm; struct i915_vma *vma; int err; - obj = i915_gem_object_create_internal(ctx->i915, 16 * PAGE_SIZE); + obj = i915_gem_object_create_internal(vm->i915, 16 * PAGE_SIZE); if (IS_ERR(obj)) return ERR_CAST(obj); - vm = i915_gem_context_get_vm_rcu(ctx); vma = i915_vma_instance(obj, vm, NULL); - i915_vm_put(vm); if (IS_ERR(vma)) { err = PTR_ERR(vma); goto err_obj; @@ -452,8 +441,7 @@ static int whitelist_writable_count(struct intel_engine_cs *engine) return count; } -static int check_dirty_whitelist(struct i915_gem_context *ctx, - struct intel_engine_cs *engine) +static int check_dirty_whitelist(struct intel_context *ce) { const u32 values[] = { 0x00000000, @@ -481,19 +469,17 @@ static int check_dirty_whitelist(struct i915_gem_context *ctx, 0xffff00ff, 0xffffffff, }; - struct i915_address_space *vm; + struct intel_engine_cs *engine = ce->engine; struct i915_vma *scratch; struct i915_vma *batch; int err = 0, i, v; u32 *cs, *results; - vm = i915_gem_context_get_vm_rcu(ctx); - scratch = create_scratch(vm, 2 * ARRAY_SIZE(values) + 1); - i915_vm_put(vm); + scratch = create_scratch(ce->vm, 2 * ARRAY_SIZE(values) + 1); if (IS_ERR(scratch)) return PTR_ERR(scratch); - batch = create_batch(ctx); + batch = create_batch(ce->vm); if (IS_ERR(batch)) { err = PTR_ERR(batch); goto out_scratch; @@ -518,7 +504,7 @@ static int check_dirty_whitelist(struct i915_gem_context *ctx, srm = MI_STORE_REGISTER_MEM; lrm = MI_LOAD_REGISTER_MEM; - if (INTEL_GEN(ctx->i915) >= 8) + if (INTEL_GEN(engine->i915) >= 8) lrm++, srm++; pr_debug("%s: Writing garbage to %x\n", @@ -577,7 +563,7 @@ static int check_dirty_whitelist(struct i915_gem_context *ctx, i915_gem_object_unpin_map(batch->obj); intel_gt_chipset_flush(engine->gt); - rq = igt_request_alloc(ctx, engine); + rq = intel_context_create_request(ce); if (IS_ERR(rq)) { err = PTR_ERR(rq); goto out_batch; @@ -696,7 +682,7 @@ static int check_dirty_whitelist(struct i915_gem_context *ctx, break; } - if (igt_flush_test(ctx->i915)) + if (igt_flush_test(engine->i915)) err = -EIO; out_batch: i915_vma_unpin_and_release(&batch, 0); @@ -709,38 +695,31 @@ static int live_dirty_whitelist(void *arg) { struct intel_gt *gt = arg; struct intel_engine_cs *engine; - struct i915_gem_context *ctx; enum intel_engine_id id; - struct file *file; - int err = 0; /* Can the user write to the whitelisted registers? */ if (INTEL_GEN(gt->i915) < 7) /* minimum requirement for LRI, SRM, LRM */ return 0; - file = mock_file(gt->i915); - if (IS_ERR(file)) - return PTR_ERR(file); - - ctx = live_context(gt->i915, file); - if (IS_ERR(ctx)) { - err = PTR_ERR(ctx); - goto out_file; - } - for_each_engine(engine, gt, id) { + struct intel_context *ce; + int err; + if (engine->whitelist.count == 0) continue; - err = check_dirty_whitelist(ctx, engine); + ce = intel_context_create(engine); + if (IS_ERR(ce)) + return PTR_ERR(ce); + + err = check_dirty_whitelist(ce); + intel_context_put(ce); if (err) - goto out_file; + return err; } -out_file: - fput(file); - return err; + return 0; } static int live_reset_whitelist(void *arg) @@ -830,12 +809,15 @@ static int read_whitelisted_registers(struct i915_gem_context *ctx, static int scrub_whitelisted_registers(struct i915_gem_context *ctx, struct intel_engine_cs *engine) { + struct i915_address_space *vm; struct i915_request *rq; struct i915_vma *batch; int i, err = 0; u32 *cs; - batch = create_batch(ctx); + vm = i915_gem_context_get_vm_rcu(ctx); + batch = create_batch(vm); + i915_vm_put(vm); if (IS_ERR(batch)) return PTR_ERR(batch); diff --git a/drivers/gpu/drm/i915/gvt/scheduler.c b/drivers/gpu/drm/i915/gvt/scheduler.c index 228c66534e21..b3299f88e24e 100644 --- a/drivers/gpu/drm/i915/gvt/scheduler.c +++ b/drivers/gpu/drm/i915/gvt/scheduler.c @@ -35,12 +35,12 @@ #include -#include "gem/i915_gem_context.h" #include "gem/i915_gem_pm.h" #include "gt/intel_context.h" #include "gt/intel_ring.h" #include "i915_drv.h" +#include "i915_gem_gtt.h" #include "gvt.h" #define RING_CTX_OFF(x) \ @@ -1220,16 +1220,14 @@ int intel_vgpu_setup_submission(struct intel_vgpu *vgpu) struct drm_i915_private *i915 = vgpu->gvt->dev_priv; struct intel_vgpu_submission *s = &vgpu->submission; struct intel_engine_cs *engine; - struct i915_gem_context *ctx; struct i915_ppgtt *ppgtt; enum intel_engine_id i; int ret; - ctx = i915_gem_context_create_kernel(i915, I915_PRIORITY_MAX); - if (IS_ERR(ctx)) - return PTR_ERR(ctx); + ppgtt = i915_ppgtt_create(i915); + if (IS_ERR(ppgtt)) + return PTR_ERR(ppgtt); - ppgtt = i915_vm_to_ppgtt(i915_gem_context_get_vm_rcu(ctx)); i915_context_ppgtt_root_save(s, ppgtt); for_each_engine(engine, i915, i) { @@ -1238,12 +1236,14 @@ int intel_vgpu_setup_submission(struct intel_vgpu *vgpu) INIT_LIST_HEAD(&s->workload_q_head[i]); s->shadow[i] = ERR_PTR(-EINVAL); - ce = intel_context_create(ctx, engine); + ce = intel_context_create(engine); if (IS_ERR(ce)) { ret = PTR_ERR(ce); goto out_shadow_ctx; } + i915_vm_put(ce->vm); + ce->vm = i915_vm_get(&ppgtt->vm); intel_context_set_single_submission(ce); if (!USES_GUC_SUBMISSION(i915)) { /* Max ring buffer size */ @@ -1278,7 +1278,6 @@ int intel_vgpu_setup_submission(struct intel_vgpu *vgpu) bitmap_zero(s->tlb_handle_pending, I915_NUM_ENGINES); i915_vm_put(&ppgtt->vm); - i915_gem_context_put(ctx); return 0; out_shadow_ctx: @@ -1291,7 +1290,6 @@ int intel_vgpu_setup_submission(struct intel_vgpu *vgpu) intel_context_put(s->shadow[i]); } i915_vm_put(&ppgtt->vm); - i915_gem_context_put(ctx); return ret; } diff --git a/drivers/gpu/drm/i915/i915_active.c b/drivers/gpu/drm/i915/i915_active.c index 207383dda84d..953359fb82f2 100644 --- a/drivers/gpu/drm/i915/i915_active.c +++ b/drivers/gpu/drm/i915/i915_active.c @@ -6,6 +6,7 @@ #include +#include "gt/intel_context.h" #include "gt/intel_engine_pm.h" #include "gt/intel_ring.h" @@ -708,6 +709,7 @@ void i915_request_add_active_barriers(struct i915_request *rq) struct llist_node *node, *next; unsigned long flags; + GEM_BUG_ON(!intel_context_is_barrier(rq->context)); GEM_BUG_ON(intel_engine_is_virtual(engine)); GEM_BUG_ON(i915_request_timeline(rq) != engine->kernel_context->timeline); diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h index 7e0f67babe20..7e8a039e4f28 100644 --- a/drivers/gpu/drm/i915/i915_drv.h +++ b/drivers/gpu/drm/i915/i915_drv.h @@ -960,9 +960,6 @@ struct drm_i915_private { struct pci_dev *bridge_dev; - /* Context used internally to idle the GPU and setup initial state */ - struct i915_gem_context *kernel_context; - struct intel_engine_cs *engine[I915_NUM_ENGINES]; struct rb_root uabi_engines; diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index f4cbd379db95..b24825c691f7 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -1063,8 +1063,7 @@ static int __intel_engines_record_defaults(struct intel_gt *gt) GEM_BUG_ON(!engine->kernel_context); engine->serial++; /* force the kernel context switch */ - ce = intel_context_create(engine->kernel_context->gem_context, - engine); + ce = intel_context_create(engine); if (IS_ERR(ce)) { err = PTR_ERR(ce); goto out; @@ -1217,6 +1216,7 @@ int i915_gem_init(struct drm_i915_private *dev_priv) } intel_gt_init(&dev_priv->gt); + i915_gem_init_contexts(dev_priv); ret = intel_engines_setup(&dev_priv->gt); if (ret) { @@ -1224,16 +1224,10 @@ int i915_gem_init(struct drm_i915_private *dev_priv) goto err_unlock; } - ret = i915_gem_init_contexts(dev_priv); - if (ret) { - GEM_BUG_ON(ret == -EIO); - goto err_scratch; - } - ret = intel_engines_init(&dev_priv->gt); if (ret) { GEM_BUG_ON(ret == -EIO); - goto err_context; + goto err_scratch; } intel_uc_init(&dev_priv->gt.uc); @@ -1297,9 +1291,6 @@ int i915_gem_init(struct drm_i915_private *dev_priv) intel_uc_fini(&dev_priv->gt.uc); intel_engines_cleanup(&dev_priv->gt); } -err_context: - if (ret != -EIO) - i915_gem_driver_release__contexts(dev_priv); err_scratch: intel_gt_driver_release(&dev_priv->gt); err_unlock: @@ -1364,7 +1355,6 @@ void i915_gem_driver_remove(struct drm_i915_private *dev_priv) void i915_gem_driver_release(struct drm_i915_private *dev_priv) { intel_engines_cleanup(&dev_priv->gt); - i915_gem_driver_release__contexts(dev_priv); intel_gt_driver_release(&dev_priv->gt); intel_wa_list_free(&dev_priv->gt_wa_list); diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c index 6239a9adbf14..a22a2fe409c4 100644 --- a/drivers/gpu/drm/i915/i915_gem_gtt.c +++ b/drivers/gpu/drm/i915/i915_gem_gtt.c @@ -1363,12 +1363,8 @@ static int gen8_init_scratch(struct i915_address_space *vm) * If everybody agrees to not to write into the scratch page, * we can reuse it for all vm, keeping contexts and processes separate. */ - if (vm->has_read_only && - vm->i915->kernel_context && - vm->i915->kernel_context->vm) { - struct i915_address_space *clone = - rcu_dereference_protected(vm->i915->kernel_context->vm, - true); /* static */ + if (vm->has_read_only && vm->gt->vm && !i915_is_ggtt(vm->gt->vm)) { + struct i915_address_space *clone = vm->gt->vm; GEM_BUG_ON(!clone->has_read_only); diff --git a/drivers/gpu/drm/i915/i915_perf.c b/drivers/gpu/drm/i915/i915_perf.c index 1924d44081e9..25dbd01b822f 100644 --- a/drivers/gpu/drm/i915/i915_perf.c +++ b/drivers/gpu/drm/i915/i915_perf.c @@ -2340,9 +2340,6 @@ static int lrc_configure_all_contexts(struct i915_perf_stream *stream, */ spin_lock(&i915->gem.contexts.lock); list_for_each_entry_safe(ctx, cn, &i915->gem.contexts.list, link) { - if (ctx == i915->kernel_context) - continue; - if (!kref_get_unless_zero(&ctx->ref)) continue; diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c index 678b32089507..20eeef386577 100644 --- a/drivers/gpu/drm/i915/i915_request.c +++ b/drivers/gpu/drm/i915/i915_request.c @@ -1236,8 +1236,8 @@ void __i915_request_queue(struct i915_request *rq, void i915_request_add(struct i915_request *rq) { - struct i915_sched_attr attr = rq->context->gem_context->sched; struct intel_timeline * const tl = i915_request_timeline(rq); + struct i915_sched_attr attr = {}; struct i915_request *prev; lockdep_assert_held(&tl->mutex); @@ -1247,6 +1247,9 @@ void i915_request_add(struct i915_request *rq) prev = __i915_request_commit(rq); + if (rq->context->gem_context) + attr = rq->context->gem_context->sched; + /* * Boost actual workloads past semaphores! * diff --git a/drivers/gpu/drm/i915/selftests/i915_request.c b/drivers/gpu/drm/i915/selftests/i915_request.c index 50cc7ca9afba..d29f44cf3162 100644 --- a/drivers/gpu/drm/i915/selftests/i915_request.c +++ b/drivers/gpu/drm/i915/selftests/i915_request.c @@ -741,10 +741,8 @@ static int live_empty_request(void *arg) static struct i915_vma *recursive_batch(struct drm_i915_private *i915) { - struct i915_gem_context *ctx = i915->kernel_context; struct drm_i915_gem_object *obj; const int gen = INTEL_GEN(i915); - struct i915_address_space *vm; struct i915_vma *vma; u32 *cmd; int err; @@ -753,9 +751,7 @@ static struct i915_vma *recursive_batch(struct drm_i915_private *i915) if (IS_ERR(obj)) return ERR_CAST(obj); - vm = i915_gem_context_get_vm_rcu(ctx); - vma = i915_vma_instance(obj, vm, NULL); - i915_vm_put(vm); + vma = i915_vma_instance(obj, i915->gt.vm, NULL); if (IS_ERR(vma)) { err = PTR_ERR(vma); goto err; diff --git a/drivers/gpu/drm/i915/selftests/mock_gem_device.c b/drivers/gpu/drm/i915/selftests/mock_gem_device.c index e58b0bc9cdb6..a539d8b39682 100644 --- a/drivers/gpu/drm/i915/selftests/mock_gem_device.c +++ b/drivers/gpu/drm/i915/selftests/mock_gem_device.c @@ -63,7 +63,6 @@ static void mock_device_release(struct drm_device *dev) for_each_engine(engine, &i915->gt, id) mock_engine_free(engine); - i915_gem_driver_release__contexts(i915); drain_workqueue(i915->wq); i915_gem_drain_freed_objects(i915); @@ -179,6 +178,7 @@ struct drm_i915_private *mock_gem_device(void) mock_init_contexts(i915); mock_init_ggtt(i915, &i915->ggtt); + i915->gt.vm = i915_vm_get(&i915->ggtt.vm); mkwrite_device_info(i915)->engine_mask = BIT(0); @@ -186,10 +186,6 @@ struct drm_i915_private *mock_gem_device(void) if (!i915->engine[RCS0]) goto err_unlock; - i915->kernel_context = mock_context(i915, NULL); - if (!i915->kernel_context) - goto err_engine; - if (mock_engine_init(i915->engine[RCS0])) goto err_context; @@ -198,8 +194,6 @@ struct drm_i915_private *mock_gem_device(void) return i915; err_context: - i915_gem_driver_release__contexts(i915); -err_engine: mock_engine_free(i915->engine[RCS0]); err_unlock: destroy_workqueue(i915->wq); From patchwork Sun Nov 10 18:58:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236295 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 762E414E5 for ; Sun, 10 Nov 2019 18:58:40 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5E18920842 for ; Sun, 10 Nov 2019 18:58:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5E18920842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 03E386E7B4; Sun, 10 Nov 2019 18:58:35 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id D83EA6E25F for ; Sun, 10 Nov 2019 18:58:28 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151903-1500050 for multiple; Sun, 10 Nov 2019 18:58:14 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:58:04 +0000 Message-Id: <20191110185806.17413-23-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 23/25] drm/i915: Move i915_gem_init_contexts() earlier X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" As the GEM global context setup is now independent of the GT state (although GT does currently still depending upon the global i915->kernel_context), we can move its init earlier, leaving the gt init ready to extracted. Signed-off-by: Chris Wilson --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 2 +- drivers/gpu/drm/i915/gem/i915_gem_context.h | 2 +- drivers/gpu/drm/i915/i915_gem.c | 23 +++++++++++---------- 3 files changed, 14 insertions(+), 13 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index 21c265af6bd5..239a390922ab 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -742,7 +742,7 @@ static void init_contexts(struct i915_gem_contexts *gc) init_llist_head(&gc->free_list); } -void i915_gem_init_contexts(struct drm_i915_private *i915) +void i915_gem_init__contexts(struct drm_i915_private *i915) { init_contexts(&i915->gem.contexts); DRM_DEBUG_DRIVER("%s context support initialized\n", diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.h b/drivers/gpu/drm/i915/gem/i915_gem_context.h index d99fff481d59..80a90ead4108 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.h @@ -115,7 +115,7 @@ static inline bool i915_gem_context_is_kernel(struct i915_gem_context *ctx) } /* i915_gem_context.c */ -void i915_gem_init_contexts(struct drm_i915_private *i915); +void i915_gem_init__contexts(struct drm_i915_private *i915); int i915_gem_context_open(struct drm_i915_private *i915, struct drm_file *file); diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index b24825c691f7..04350e61c86c 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -1216,18 +1216,17 @@ int i915_gem_init(struct drm_i915_private *dev_priv) } intel_gt_init(&dev_priv->gt); - i915_gem_init_contexts(dev_priv); ret = intel_engines_setup(&dev_priv->gt); if (ret) { GEM_BUG_ON(ret == -EIO); - goto err_unlock; + goto err_gt_early; } ret = intel_engines_init(&dev_priv->gt); if (ret) { GEM_BUG_ON(ret == -EIO); - goto err_scratch; + goto err_engines; } intel_uc_init(&dev_priv->gt.uc); @@ -1254,19 +1253,19 @@ int i915_gem_init(struct drm_i915_private *dev_priv) ret = intel_engines_verify_workarounds(&dev_priv->gt); if (ret) - goto err_gt; + goto err_gt_late; ret = __intel_engines_record_defaults(&dev_priv->gt); if (ret) - goto err_gt; + goto err_gt_late; ret = i915_inject_probe_error(dev_priv, -ENODEV); if (ret) - goto err_gt; + goto err_gt_late; ret = i915_inject_probe_error(dev_priv, -EIO); if (ret) - goto err_gt; + goto err_gt_late; intel_uncore_forcewake_put(&dev_priv->uncore, FORCEWAKE_ALL); @@ -1278,7 +1277,7 @@ int i915_gem_init(struct drm_i915_private *dev_priv) * HW as irrevisibly wedged, but keep enough state around that the * driver doesn't explode during runtime. */ -err_gt: +err_gt_late: intel_gt_set_wedged_on_init(&dev_priv->gt); i915_gem_suspend(dev_priv); i915_gem_suspend_late(dev_priv); @@ -1287,11 +1286,12 @@ int i915_gem_init(struct drm_i915_private *dev_priv) err_init_hw: intel_uc_fini_hw(&dev_priv->gt.uc); err_uc_init: - if (ret != -EIO) { + if (ret != -EIO) intel_uc_fini(&dev_priv->gt.uc); +err_engines: + if (ret != -EIO) intel_engines_cleanup(&dev_priv->gt); - } -err_scratch: +err_gt_early: intel_gt_driver_release(&dev_priv->gt); err_unlock: intel_uncore_forcewake_put(&dev_priv->uncore, FORCEWAKE_ALL); @@ -1382,6 +1382,7 @@ static void i915_gem_init__mm(struct drm_i915_private *i915) void i915_gem_init_early(struct drm_i915_private *dev_priv) { i915_gem_init__mm(dev_priv); + i915_gem_init__contexts(dev_priv); spin_lock_init(&dev_priv->fb_tracking.lock); } From patchwork Sun Nov 10 18:58:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236307 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4775515AB for ; Sun, 10 Nov 2019 18:58:45 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2F3B720842 for ; Sun, 10 Nov 2019 18:58:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2F3B720842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F2A4E6E7B2; Sun, 10 Nov 2019 18:58:34 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id EA4F96E7A3 for ; Sun, 10 Nov 2019 18:58:29 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151904-1500050 for multiple; Sun, 10 Nov 2019 18:58:14 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:58:05 +0000 Message-Id: <20191110185806.17413-24-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 24/25] drm/i915/uc: Use an internal buffer for firmware images X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Since the lifetime of the uc_fw is virtually identical to the current pinned range, simplify the setup to avoid using a swappable shmem file, and just use an internal bo. The immediate advantage is in removing the extra pin/unpin stages during init that are very difficult to balance along error paths. Signed-off-by: Chris Wilson --- drivers/gpu/drm/i915/gem/i915_gem_internal.c | 43 +++++++++++++++++ drivers/gpu/drm/i915/gem/i915_gem_object.h | 4 +- drivers/gpu/drm/i915/gem/i915_gem_shmem.c | 51 -------------------- drivers/gpu/drm/i915/gt/uc/intel_guc.c | 12 ++--- drivers/gpu/drm/i915/gt/uc/intel_huc.c | 12 ++--- drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c | 30 +----------- drivers/gpu/drm/i915/gt/uc/intel_uc_fw.h | 2 - 7 files changed, 56 insertions(+), 98 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_internal.c b/drivers/gpu/drm/i915/gem/i915_gem_internal.c index 9cfb0e41ff06..a8eb0eb27390 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_internal.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_internal.c @@ -198,3 +198,46 @@ i915_gem_object_create_internal(struct drm_i915_private *i915, return obj; } + +/* Allocate a new GEM object and fill it with the supplied data */ +struct drm_i915_gem_object * +i915_gem_object_create_from_data(struct drm_i915_private *i915, + const void *data, resource_size_t size) +{ + struct drm_i915_gem_object *obj; + resource_size_t offset; + struct sgt_iter it; + struct page *page; + int err; + + obj = i915_gem_object_create_internal(i915, round_up(size, PAGE_SIZE)); + if (IS_ERR(obj)) + return obj; + + GEM_BUG_ON(obj->write_domain != I915_GEM_DOMAIN_CPU); + + err = i915_gem_object_pin_pages(obj); + if (err) + goto err; + + offset = 0; + for_each_sgt_page(page, it, obj->mm.pages) { + int len = min_t(typeof(size), size - offset, PAGE_SIZE); + void *ptr; + + ptr = kmap(page); + + memcpy(ptr, data + offset, len); + if (!(obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE)) + drm_clflush_virt_range(ptr, len); + + kunmap(page); + offset += len; + } + + return obj; /* keep pages pinned */ + +err: + i915_gem_object_put(obj); + return ERR_PTR(err); +} diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.h b/drivers/gpu/drm/i915/gem/i915_gem_object.h index e5750d506cc9..ac4f45fa5234 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.h @@ -29,8 +29,8 @@ struct drm_i915_gem_object * i915_gem_object_create_shmem(struct drm_i915_private *i915, resource_size_t size); struct drm_i915_gem_object * -i915_gem_object_create_shmem_from_data(struct drm_i915_private *i915, - const void *data, resource_size_t size); +i915_gem_object_create_from_data(struct drm_i915_private *i915, + const void *data, resource_size_t size); extern const struct drm_i915_gem_object_ops i915_gem_shmem_ops; void __i915_gem_object_release_shmem(struct drm_i915_gem_object *obj, diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c index 4d69c3fc3439..0f4c8fc38bba 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c @@ -533,57 +533,6 @@ i915_gem_object_create_shmem(struct drm_i915_private *i915, size, 0); } -/* Allocate a new GEM object and fill it with the supplied data */ -struct drm_i915_gem_object * -i915_gem_object_create_shmem_from_data(struct drm_i915_private *dev_priv, - const void *data, resource_size_t size) -{ - struct drm_i915_gem_object *obj; - struct file *file; - resource_size_t offset; - int err; - - obj = i915_gem_object_create_shmem(dev_priv, round_up(size, PAGE_SIZE)); - if (IS_ERR(obj)) - return obj; - - GEM_BUG_ON(obj->write_domain != I915_GEM_DOMAIN_CPU); - - file = obj->base.filp; - offset = 0; - do { - unsigned int len = min_t(typeof(size), size, PAGE_SIZE); - struct page *page; - void *pgdata, *vaddr; - - err = pagecache_write_begin(file, file->f_mapping, - offset, len, 0, - &page, &pgdata); - if (err < 0) - goto fail; - - vaddr = kmap(page); - memcpy(vaddr, data, len); - kunmap(page); - - err = pagecache_write_end(file, file->f_mapping, - offset, len, len, - page, pgdata); - if (err < 0) - goto fail; - - size -= len; - data += len; - offset += len; - } while (size); - - return obj; - -fail: - i915_gem_object_put(obj); - return ERR_PTR(err); -} - static int init_shmem(struct intel_memory_region *mem) { int err; diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.c b/drivers/gpu/drm/i915/gt/uc/intel_guc.c index 019ae6486e8d..29c2b416bb03 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.c @@ -334,13 +334,14 @@ int intel_guc_init(struct intel_guc *guc) struct intel_gt *gt = guc_to_gt(guc); int ret; - ret = intel_uc_fw_init(&guc->fw); - if (ret) - goto err_fetch; + /* This should happen before the load! */ + GEM_BUG_ON(intel_uc_fw_is_loaded(&guc->fw)); + if (!intel_uc_fw_is_available(&guc->fw)) + return -ENOEXEC; ret = intel_guc_log_create(&guc->log); if (ret) - goto err_fw; + goto err_fetch; ret = intel_guc_ads_create(guc); if (ret) @@ -375,8 +376,6 @@ int intel_guc_init(struct intel_guc *guc) intel_guc_ads_destroy(guc); err_log: intel_guc_log_destroy(&guc->log); -err_fw: - intel_uc_fw_fini(&guc->fw); err_fetch: intel_uc_fw_cleanup_fetch(&guc->fw); DRM_DEV_DEBUG_DRIVER(gt->i915->drm.dev, "failed with %d\n", ret); @@ -399,7 +398,6 @@ void intel_guc_fini(struct intel_guc *guc) intel_guc_ads_destroy(guc); intel_guc_log_destroy(&guc->log); - intel_uc_fw_fini(&guc->fw); intel_uc_fw_cleanup_fetch(&guc->fw); } diff --git a/drivers/gpu/drm/i915/gt/uc/intel_huc.c b/drivers/gpu/drm/i915/gt/uc/intel_huc.c index 32a069841c14..f0b555fb6dd2 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_huc.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_huc.c @@ -108,9 +108,10 @@ int intel_huc_init(struct intel_huc *huc) struct drm_i915_private *i915 = huc_to_gt(huc)->i915; int err; - err = intel_uc_fw_init(&huc->fw); - if (err) - goto out; + /* This should happen before the load! */ + GEM_BUG_ON(intel_uc_fw_is_loaded(&huc->fw)); + if (!intel_uc_fw_is_available(&huc->fw)) + return -ENOEXEC; /* * HuC firmware image is outside GuC accessible range. @@ -119,12 +120,10 @@ int intel_huc_init(struct intel_huc *huc) */ err = intel_huc_rsa_data_create(huc); if (err) - goto out_fini; + goto out; return 0; -out_fini: - intel_uc_fw_fini(&huc->fw); out: intel_uc_fw_cleanup_fetch(&huc->fw); DRM_DEV_DEBUG_DRIVER(i915->drm.dev, "failed with %d\n", err); @@ -137,7 +136,6 @@ void intel_huc_fini(struct intel_huc *huc) return; intel_huc_rsa_data_destroy(huc); - intel_uc_fw_fini(&huc->fw); } /** diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c index 66a30ab7044a..3e30d7167876 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c @@ -363,7 +363,7 @@ int intel_uc_fw_fetch(struct intel_uc_fw *uc_fw, struct drm_i915_private *i915) } } - obj = i915_gem_object_create_shmem_from_data(i915, fw->data, fw->size); + obj = i915_gem_object_create_from_data(i915, fw->data, fw->size); if (IS_ERR(obj)) { err = PTR_ERR(obj); goto fail; @@ -525,34 +525,6 @@ int intel_uc_fw_upload(struct intel_uc_fw *uc_fw, struct intel_gt *gt, return err; } -int intel_uc_fw_init(struct intel_uc_fw *uc_fw) -{ - int err; - - /* this should happen before the load! */ - GEM_BUG_ON(intel_uc_fw_is_loaded(uc_fw)); - - if (!intel_uc_fw_is_available(uc_fw)) - return -ENOEXEC; - - err = i915_gem_object_pin_pages(uc_fw->obj); - if (err) { - DRM_DEBUG_DRIVER("%s fw pin-pages err=%d\n", - intel_uc_fw_type_repr(uc_fw->type), err); - intel_uc_fw_change_status(uc_fw, INTEL_UC_FIRMWARE_FAIL); - } - - return err; -} - -void intel_uc_fw_fini(struct intel_uc_fw *uc_fw) -{ - if (!intel_uc_fw_is_available(uc_fw)) - return; - - i915_gem_object_unpin_pages(uc_fw->obj); -} - /** * intel_uc_fw_cleanup_fetch - cleanup uC firmware * @uc_fw: uC firmware diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.h b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.h index 7a0a5989afc9..1d3b78cc770a 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.h +++ b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.h @@ -233,8 +233,6 @@ int intel_uc_fw_fetch(struct intel_uc_fw *uc_fw, struct drm_i915_private *i915); void intel_uc_fw_cleanup_fetch(struct intel_uc_fw *uc_fw); int intel_uc_fw_upload(struct intel_uc_fw *uc_fw, struct intel_gt *gt, u32 wopcm_offset, u32 dma_flags); -int intel_uc_fw_init(struct intel_uc_fw *uc_fw); -void intel_uc_fw_fini(struct intel_uc_fw *uc_fw); size_t intel_uc_fw_copy_rsa(struct intel_uc_fw *uc_fw, void *dst, u32 max_len); void intel_uc_fw_dump(const struct intel_uc_fw *uc_fw, struct drm_printer *p); From patchwork Sun Nov 10 18:58:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11236283 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7909514E5 for ; Sun, 10 Nov 2019 18:58:33 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 60C5C20842 for ; Sun, 10 Nov 2019 18:58:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 60C5C20842 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 72D0E6E79F; Sun, 10 Nov 2019 18:58:32 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id 09DD26E0C8 for ; Sun, 10 Nov 2019 18:58:26 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19151905-1500050 for multiple; Sun, 10 Nov 2019 18:58:14 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sun, 10 Nov 2019 18:58:06 +0000 Message-Id: <20191110185806.17413-25-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191110185806.17413-1-chris@chris-wilson.co.uk> References: <20191110185806.17413-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 25/25] drm/i915/gt: Pull GT initialisation under intel_gt_init() X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Begin pulling the GT setup underneath a single GT umbrella; let intel_gt take ownership of its engines! As hinted, the complication is the lifetime of the probed engine versus the active lifetime of the GT backends. We need to detect the engine layout early and keep it until the end so that we can sanitize state on takeover and release. Signed-off-by: Chris Wilson --- drivers/gpu/drm/i915/display/intel_overlay.c | 6 +- drivers/gpu/drm/i915/gt/intel_engine.h | 8 +- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 42 ++-- drivers/gpu/drm/i915/gt/intel_engine_types.h | 15 +- drivers/gpu/drm/i915/gt/intel_gt.c | 230 +++++++++++++++++- drivers/gpu/drm/i915/gt/intel_lrc.c | 17 +- drivers/gpu/drm/i915/gt/intel_reset.c | 9 +- .../gpu/drm/i915/gt/intel_ring_submission.c | 14 +- .../gpu/drm/i915/gt/intel_timeline_types.h | 4 +- drivers/gpu/drm/i915/gt/mock_engine.c | 16 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 9 +- drivers/gpu/drm/i915/i915_drv.c | 1 - drivers/gpu/drm/i915/i915_gem.c | 228 +---------------- drivers/gpu/drm/i915/selftests/i915_gem.c | 1 + 14 files changed, 302 insertions(+), 298 deletions(-) diff --git a/drivers/gpu/drm/i915/display/intel_overlay.c b/drivers/gpu/drm/i915/display/intel_overlay.c index 8cfb785e761c..973026f0c717 100644 --- a/drivers/gpu/drm/i915/display/intel_overlay.c +++ b/drivers/gpu/drm/i915/display/intel_overlay.c @@ -1326,12 +1326,14 @@ static int get_registers(struct intel_overlay *overlay, bool use_phys) void intel_overlay_setup(struct drm_i915_private *dev_priv) { struct intel_overlay *overlay; + struct intel_engine_cs *engine; int ret; if (!HAS_OVERLAY(dev_priv)) return; - if (!HAS_ENGINE(dev_priv, RCS0)) + engine = dev_priv->engine[RCS0]; + if (!engine || !engine->kernel_context) return; overlay = kzalloc(sizeof(*overlay), GFP_KERNEL); @@ -1339,7 +1341,7 @@ void intel_overlay_setup(struct drm_i915_private *dev_priv) return; overlay->i915 = dev_priv; - overlay->context = dev_priv->engine[RCS0]->kernel_context; + overlay->context = engine->kernel_context; GEM_BUG_ON(!overlay->context); overlay->color_key = 0x0101fe; diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h index bc3b72bfa9e3..833ddbe501c1 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine.h +++ b/drivers/gpu/drm/i915/gt/intel_engine.h @@ -181,7 +181,9 @@ void intel_engine_cleanup(struct intel_engine_cs *engine); int intel_engines_init_mmio(struct intel_gt *gt); int intel_engines_setup(struct intel_gt *gt); int intel_engines_init(struct intel_gt *gt); -void intel_engines_cleanup(struct intel_gt *gt); + +void intel_engines_release(struct intel_gt *gt); +void intel_engines_free(struct intel_gt *gt); int intel_engine_init_common(struct intel_engine_cs *engine); void intel_engine_cleanup_common(struct intel_engine_cs *engine); @@ -272,8 +274,8 @@ gen8_emit_ggtt_write(u32 *cs, u32 value, u32 gtt_offset, u32 flags) static inline void __intel_engine_reset(struct intel_engine_cs *engine, bool stalled) { - if (engine->reset.reset) - engine->reset.reset(engine, stalled); + if (engine->reset.rewind) + engine->reset.rewind(engine, stalled); engine->serial++; /* contexts lost */ } diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c index a27b22914fb2..c4c00b01f45e 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c @@ -318,12 +318,6 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id) engine->props.timeslice_duration_ms = CONFIG_DRM_I915_TIMESLICE_DURATION; - /* - * To be overridden by the backend on setup. However to facilitate - * cleanup on error during setup, we always provide the destroy vfunc. - */ - engine->destroy = (typeof(engine->destroy))kfree; - engine->context_size = intel_engine_context_size(gt->i915, engine->class); if (WARN_ON(engine->context_size > BIT(20))) @@ -389,21 +383,39 @@ static void intel_setup_engine_capabilities(struct intel_gt *gt) } /** - * intel_engines_cleanup() - free the resources allocated for Command Streamers + * intel_engines_release() - free the resources allocated for Command Streamers * @gt: pointer to struct intel_gt */ -void intel_engines_cleanup(struct intel_gt *gt) +void intel_engines_release(struct intel_gt *gt) { struct intel_engine_cs *engine; enum intel_engine_id id; + /* Decouple the backend; but keep the layout for late GPU resets */ for_each_engine(engine, gt, id) { - engine->destroy(engine); - gt->engine[id] = NULL; + if (!engine->release) + continue; + + engine->release(engine); + engine->release = NULL; + + memset(&engine->reset, 0, sizeof(engine->reset)); + gt->i915->engine[id] = NULL; } } +void intel_engines_free(struct intel_gt *gt) +{ + struct intel_engine_cs *engine; + enum intel_engine_id id; + + for_each_engine(engine, gt, id) { + kfree(engine); + gt->engine[id] = NULL; + } +} + /** * intel_engines_init_mmio() - allocate and prepare the Engine Command Streamers * @gt: pointer to struct intel_gt @@ -454,7 +466,7 @@ int intel_engines_init_mmio(struct intel_gt *gt) return 0; cleanup: - intel_engines_cleanup(gt); + intel_engines_free(gt); return err; } @@ -487,7 +499,7 @@ int intel_engines_init(struct intel_gt *gt) return 0; cleanup: - intel_engines_cleanup(gt); + intel_engines_release(gt); return err; } @@ -661,16 +673,13 @@ int intel_engines_setup(struct intel_gt *gt) if (err) goto cleanup; - /* We expect the backend to take control over its state */ - GEM_BUG_ON(engine->destroy == (typeof(engine->destroy))kfree); - GEM_BUG_ON(!engine->cops); } return 0; cleanup: - intel_engines_cleanup(gt); + intel_engines_release(gt); return err; } @@ -831,6 +840,7 @@ int intel_engine_init_common(struct intel_engine_cs *engine) void intel_engine_cleanup_common(struct intel_engine_cs *engine) { GEM_BUG_ON(!list_empty(&engine->active.requests)); + tasklet_kill(&engine->execlists.tasklet); /* flush the callback */ cleanup_status_page(engine); diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h index c5d1047a4bc5..6bdca3e7ae9f 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h @@ -389,7 +389,10 @@ struct intel_engine_cs { struct { void (*prepare)(struct intel_engine_cs *engine); - void (*reset)(struct intel_engine_cs *engine, bool stalled); + + void (*rewind)(struct intel_engine_cs *engine, bool stalled); + void (*cancel)(struct intel_engine_cs *engine); + void (*finish)(struct intel_engine_cs *engine); } reset; @@ -439,15 +442,7 @@ struct intel_engine_cs { void (*schedule)(struct i915_request *request, const struct i915_sched_attr *attr); - /* - * Cancel all requests on the hardware, or queued for execution. - * This should only cancel the ready requests that have been - * submitted to the engine (via the engine->submit_request callback). - * This is called when marking the device as wedged. - */ - void (*cancel_requests)(struct intel_engine_cs *engine); - - void (*destroy)(struct intel_engine_cs *engine); + void (*release)(struct intel_engine_cs *engine); struct intel_engine_execlists execlists; diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c index 4c078247825d..d62b1f18bc56 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -4,11 +4,13 @@ */ #include "i915_drv.h" +#include "intel_context.h" #include "intel_gt.h" #include "intel_gt_pm.h" #include "intel_gt_requests.h" #include "intel_mocs.h" #include "intel_rc6.h" +#include "intel_renderstate.h" #include "intel_rps.h" #include "intel_uncore.h" #include "intel_pm.h" @@ -372,32 +374,249 @@ static struct i915_address_space *kernel_vm(struct intel_gt *gt) return i915_vm_get(>->ggtt->vm); } +static int __engines_record_defaults(struct intel_gt *gt) +{ + struct i915_request *requests[I915_NUM_ENGINES] = {}; + struct intel_engine_cs *engine; + enum intel_engine_id id; + int err = 0; + + /* + * As we reset the gpu during very early sanitisation, the current + * register state on the GPU should reflect its defaults values. + * We load a context onto the hw (with restore-inhibit), then switch + * over to a second context to save that default register state. We + * can then prime every new context with that state so they all start + * from the same default HW values. + */ + + for_each_engine(engine, gt, id) { + struct intel_context *ce; + struct i915_request *rq; + + /* We must be able to switch to something! */ + GEM_BUG_ON(!engine->kernel_context); + engine->serial++; /* force the kernel context switch */ + + ce = intel_context_create(engine); + if (IS_ERR(ce)) { + err = PTR_ERR(ce); + goto out; + } + + rq = intel_context_create_request(ce); + if (IS_ERR(rq)) { + err = PTR_ERR(rq); + intel_context_put(ce); + goto out; + } + + err = intel_engine_emit_ctx_wa(rq); + if (err) + goto err_rq; + + err = intel_renderstate_emit(rq); + if (err) + goto err_rq; + +err_rq: + requests[id] = i915_request_get(rq); + i915_request_add(rq); + if (err) + goto out; + } + + /* Flush the default context image to memory, and enable powersaving. */ + if (intel_gt_wait_for_idle(gt, I915_GEM_IDLE_TIMEOUT) == -ETIME) { + err = -EIO; + goto out; + } + + for (id = 0; id < ARRAY_SIZE(requests); id++) { + struct i915_request *rq; + struct i915_vma *state; + void *vaddr; + + rq = requests[id]; + if (!rq) + continue; + + /* We want to be able to unbind the state from the GGTT */ + GEM_BUG_ON(intel_context_is_pinned(rq->context)); + + state = rq->context->state; + if (!state) + continue; + + /* + * As we will hold a reference to the logical state, it will + * not be torn down with the context, and importantly the + * object will hold onto its vma (making it possible for a + * stray GTT write to corrupt our defaults). Unmap the vma + * from the GTT to prevent such accidents and reclaim the + * space. + */ + err = i915_vma_unbind(state); + if (err) + goto out; + + i915_gem_object_lock(state->obj); + err = i915_gem_object_set_to_cpu_domain(state->obj, false); + i915_gem_object_unlock(state->obj); + if (err) + goto out; + + i915_gem_object_set_cache_coherency(state->obj, I915_CACHE_LLC); + + /* Check we can acquire the image of the context state */ + vaddr = i915_gem_object_pin_map(state->obj, I915_MAP_FORCE_WB); + if (IS_ERR(vaddr)) { + err = PTR_ERR(vaddr); + goto out; + } + + rq->engine->default_state = i915_gem_object_get(state->obj); + i915_gem_object_unpin_map(state->obj); + } + +out: + /* + * If we have to abandon now, we expect the engines to be idle + * and ready to be torn-down. The quickest way we can accomplish + * this is by declaring ourselves wedged. + */ + if (err) + intel_gt_set_wedged(gt); + + for (id = 0; id < ARRAY_SIZE(requests); id++) { + struct intel_context *ce; + struct i915_request *rq; + + rq = requests[id]; + if (!rq) + continue; + + ce = rq->context; + i915_request_put(rq); + intel_context_put(ce); + } + return err; +} + +static int __engines_verify_workarounds(struct intel_gt *gt) +{ + struct intel_engine_cs *engine; + enum intel_engine_id id; + int err = 0; + + if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM)) + return 0; + + for_each_engine(engine, gt, id) { + if (intel_engine_verify_workarounds(engine, "load")) + err = -EIO; + } + + return err; +} + +static void __intel_gt_disable(struct intel_gt *gt) +{ + intel_gt_set_wedged_on_init(gt); + + intel_gt_suspend_prepare(gt); + intel_gt_suspend_late(gt); + + GEM_BUG_ON(intel_gt_pm_is_awake(gt)); +} + int intel_gt_init(struct intel_gt *gt) { int err; - err = intel_gt_init_scratch(gt, IS_GEN(gt->i915, 2) ? SZ_256K : SZ_4K); + err = i915_inject_probe_error(gt->i915, -ENODEV); if (err) return err; + /* + * This is just a security blanket to placate dragons. + * On some systems, we very sporadically observe that the first TLBs + * used by the CS may be stale, despite us poking the TLB reset. If + * we hold the forcewake during initialisation these problems + * just magically go away. + */ + intel_uncore_forcewake_get(gt->uncore, FORCEWAKE_ALL); + + err = intel_gt_init_scratch(gt, IS_GEN(gt->i915, 2) ? SZ_256K : SZ_4K); + if (err) + goto out_fw; + intel_gt_pm_init(gt); gt->vm = kernel_vm(gt); if (!gt->vm) { err = -ENOMEM; - goto err_scratch; + goto err_pm; } - return 0; + err = intel_engines_setup(gt); + if (err) + goto err_vm; + + err = intel_engines_init(gt); + if (err) + goto err_engines; + + intel_uc_init(>->uc); + + err = intel_gt_init_hw(gt); + if (err) + goto err_uc_init; + + /* Only when the HW is re-initialised, can we replay the requests */ + err = intel_gt_resume(gt); + if (err) + goto err_uc_init; + + err = __engines_record_defaults(gt); + if (err) + goto err_gt; + + err = __engines_verify_workarounds(gt); + if (err) + goto err_gt; -err_scratch: + err = i915_inject_probe_error(gt->i915, -EIO); + if (err) + goto err_gt; + + goto out_fw; +err_gt: + __intel_gt_disable(gt); +err_uc_init: + intel_uc_fini(>->uc); +err_engines: + intel_engines_release(gt); +err_vm: + i915_vm_put(fetch_and_zero(>->vm)); +err_pm: + intel_gt_pm_fini(gt); intel_gt_fini_scratch(gt); +out_fw: + if (err) + intel_gt_set_wedged_on_init(gt); + intel_uncore_forcewake_put(gt->uncore, FORCEWAKE_ALL); return err; } void intel_gt_driver_remove(struct intel_gt *gt) { - GEM_BUG_ON(gt->awake); + __intel_gt_disable(gt); + + intel_uc_fini_hw(>->uc); + intel_uc_fini(>->uc); + + intel_engines_release(gt); } void intel_gt_driver_unregister(struct intel_gt *gt) @@ -422,4 +641,5 @@ void intel_gt_driver_late_release(struct intel_gt *gt) intel_uc_driver_late_release(>->uc); intel_gt_fini_reset(gt); intel_gt_fini_timelines(gt); + intel_engines_free(gt); } diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c index 84b452d17ec9..ff91849beb2f 100644 --- a/drivers/gpu/drm/i915/gt/intel_lrc.c +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c @@ -3112,7 +3112,7 @@ static void __execlists_reset(struct intel_engine_cs *engine, bool stalled) __unwind_incomplete_requests(engine); } -static void execlists_reset(struct intel_engine_cs *engine, bool stalled) +static void execlists_reset_rewind(struct intel_engine_cs *engine, bool stalled) { unsigned long flags; @@ -3130,7 +3130,7 @@ static void nop_submission_tasklet(unsigned long data) /* The driver is wedged; don't process any more events. */ } -static void execlists_cancel_requests(struct intel_engine_cs *engine) +static void execlists_reset_cancel(struct intel_engine_cs *engine) { struct intel_engine_execlists * const execlists = &engine->execlists; struct i915_request *rq, *rn; @@ -3719,12 +3719,12 @@ static void execlists_park(struct intel_engine_cs *engine) void intel_execlists_set_default_submission(struct intel_engine_cs *engine) { engine->submit_request = execlists_submit_request; - engine->cancel_requests = execlists_cancel_requests; engine->schedule = i915_schedule; engine->execlists.tasklet.func = execlists_submission_tasklet; engine->reset.prepare = execlists_reset_prepare; - engine->reset.reset = execlists_reset; + engine->reset.rewind = execlists_reset_rewind; + engine->reset.cancel = execlists_reset_cancel; engine->reset.finish = execlists_reset_finish; engine->park = execlists_park; @@ -3741,11 +3741,10 @@ void intel_execlists_set_default_submission(struct intel_engine_cs *engine) engine->flags |= I915_ENGINE_HAS_RELATIVE_MMIO; } -static void execlists_destroy(struct intel_engine_cs *engine) +static void execlists_release(struct intel_engine_cs *engine) { intel_engine_cleanup_common(engine); lrc_destroy_wa_ctx(engine); - kfree(engine); } static void @@ -3753,13 +3752,9 @@ logical_ring_default_vfuncs(struct intel_engine_cs *engine) { /* Default vfuncs which can be overriden by each engine. */ - engine->destroy = execlists_destroy; + engine->release = execlists_release; engine->resume = execlists_resume; - engine->reset.prepare = execlists_reset_prepare; - engine->reset.reset = execlists_reset; - engine->reset.finish = execlists_reset_finish; - engine->cops = &execlists_context_ops; engine->request_alloc = execlists_request_alloc; diff --git a/drivers/gpu/drm/i915/gt/intel_reset.c b/drivers/gpu/drm/i915/gt/intel_reset.c index 9ecf24439fde..c60e6361a638 100644 --- a/drivers/gpu/drm/i915/gt/intel_reset.c +++ b/drivers/gpu/drm/i915/gt/intel_reset.c @@ -666,7 +666,8 @@ static void reset_prepare_engine(struct intel_engine_cs *engine) * GPU state upon resume, i.e. fail to restart after a reset. */ intel_uncore_forcewake_get(engine->uncore, FORCEWAKE_ALL); - engine->reset.prepare(engine); + if (engine->reset.prepare) + engine->reset.prepare(engine); } static void revoke_mmaps(struct intel_gt *gt) @@ -741,7 +742,8 @@ static int gt_reset(struct intel_gt *gt, intel_engine_mask_t stalled_mask) static void reset_finish_engine(struct intel_engine_cs *engine) { - engine->reset.finish(engine); + if (engine->reset.finish) + engine->reset.finish(engine); intel_uncore_forcewake_put(engine->uncore, FORCEWAKE_ALL); intel_engine_breadcrumbs_irq(engine); @@ -818,7 +820,8 @@ static void __intel_gt_set_wedged(struct intel_gt *gt) /* Mark all executing requests as skipped */ for_each_engine(engine, gt, id) - engine->cancel_requests(engine); + if (engine->reset.cancel) + engine->reset.cancel(engine); reset_finish(gt, awake); diff --git a/drivers/gpu/drm/i915/gt/intel_ring_submission.c b/drivers/gpu/drm/i915/gt/intel_ring_submission.c index 6ddfe8a86dbe..72ffcc5671c8 100644 --- a/drivers/gpu/drm/i915/gt/intel_ring_submission.c +++ b/drivers/gpu/drm/i915/gt/intel_ring_submission.c @@ -771,7 +771,7 @@ static void reset_prepare(struct intel_engine_cs *engine) intel_uncore_read_fw(uncore, RING_HEAD(base))); } -static void reset_ring(struct intel_engine_cs *engine, bool stalled) +static void reset_rewind(struct intel_engine_cs *engine, bool stalled) { struct i915_request *pos, *rq; unsigned long flags; @@ -899,7 +899,7 @@ static int rcs_resume(struct intel_engine_cs *engine) return xcs_resume(engine); } -static void cancel_requests(struct intel_engine_cs *engine) +static void reset_cancel(struct intel_engine_cs *engine) { struct i915_request *request; unsigned long flags; @@ -1804,7 +1804,6 @@ static int gen6_ring_flush(struct i915_request *rq, u32 mode) static void i9xx_set_default_submission(struct intel_engine_cs *engine) { engine->submit_request = i9xx_submit_request; - engine->cancel_requests = cancel_requests; engine->park = NULL; engine->unpark = NULL; @@ -1816,7 +1815,7 @@ static void gen6_bsd_set_default_submission(struct intel_engine_cs *engine) engine->submit_request = gen6_bsd_submit_request; } -static void ring_destroy(struct intel_engine_cs *engine) +static void ring_release(struct intel_engine_cs *engine) { struct drm_i915_private *dev_priv = engine->i915; @@ -1830,8 +1829,6 @@ static void ring_destroy(struct intel_engine_cs *engine) intel_timeline_unpin(engine->legacy.timeline); intel_timeline_put(engine->legacy.timeline); - - kfree(engine); } static void setup_irq(struct intel_engine_cs *engine) @@ -1862,11 +1859,12 @@ static void setup_common(struct intel_engine_cs *engine) setup_irq(engine); - engine->destroy = ring_destroy; + engine->release = ring_release; engine->resume = xcs_resume; engine->reset.prepare = reset_prepare; - engine->reset.reset = reset_ring; + engine->reset.rewind = reset_rewind; + engine->reset.cancel = reset_cancel; engine->reset.finish = reset_finish; engine->cops = &ring_context_ops; diff --git a/drivers/gpu/drm/i915/gt/intel_timeline_types.h b/drivers/gpu/drm/i915/gt/intel_timeline_types.h index 98d9ee166379..34d28d7771d1 100644 --- a/drivers/gpu/drm/i915/gt/intel_timeline_types.h +++ b/drivers/gpu/drm/i915/gt/intel_timeline_types.h @@ -14,10 +14,10 @@ #include "i915_active_types.h" -struct drm_i915_private; struct i915_vma; -struct intel_timeline_cacheline; struct i915_syncmap; +struct intel_gt; +struct intel_timeline_cacheline; struct intel_timeline { u64 fence_context; diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c index 83f549d203a0..8bb5acf7199e 100644 --- a/drivers/gpu/drm/i915/gt/mock_engine.c +++ b/drivers/gpu/drm/i915/gt/mock_engine.c @@ -207,16 +207,12 @@ static void mock_reset_prepare(struct intel_engine_cs *engine) { } -static void mock_reset(struct intel_engine_cs *engine, bool stalled) +static void mock_reset_rewind(struct intel_engine_cs *engine, bool stalled) { GEM_BUG_ON(stalled); } -static void mock_reset_finish(struct intel_engine_cs *engine) -{ -} - -static void mock_cancel_requests(struct intel_engine_cs *engine) +static void mock_reset_cancel(struct intel_engine_cs *engine) { struct i915_request *request; unsigned long flags; @@ -234,6 +230,10 @@ static void mock_cancel_requests(struct intel_engine_cs *engine) spin_unlock_irqrestore(&engine->active.lock, flags); } +static void mock_reset_finish(struct intel_engine_cs *engine) +{ +} + struct intel_engine_cs *mock_engine(struct drm_i915_private *i915, const char *name, int id) @@ -265,9 +265,9 @@ struct intel_engine_cs *mock_engine(struct drm_i915_private *i915, engine->base.submit_request = mock_submit_request; engine->base.reset.prepare = mock_reset_prepare; - engine->base.reset.reset = mock_reset; + engine->base.reset.rewind = mock_reset_rewind; + engine->base.reset.cancel = mock_reset_cancel; engine->base.reset.finish = mock_reset_finish; - engine->base.cancel_requests = mock_cancel_requests; i915->gt.engine[id] = &engine->base; i915->gt.engine_class[0][id] = &engine->base; diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c index 08267d2c3cfc..63ab6a9567e0 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c @@ -658,7 +658,7 @@ cancel_port_requests(struct intel_engine_execlists * const execlists) memset(execlists->inflight, 0, sizeof(execlists->inflight)); } -static void guc_reset(struct intel_engine_cs *engine, bool stalled) +static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled) { struct intel_engine_execlists * const execlists = &engine->execlists; struct i915_request *rq; @@ -683,7 +683,7 @@ static void guc_reset(struct intel_engine_cs *engine, bool stalled) spin_unlock_irqrestore(&engine->active.lock, flags); } -static void guc_cancel_requests(struct intel_engine_cs *engine) +static void guc_reset_cancel(struct intel_engine_cs *engine) { struct intel_engine_execlists * const execlists = &engine->execlists; struct i915_request *rq, *rn; @@ -1102,11 +1102,10 @@ static void guc_set_default_submission(struct intel_engine_cs *engine) engine->park = engine->unpark = NULL; engine->reset.prepare = guc_reset_prepare; - engine->reset.reset = guc_reset; + engine->reset.rewind = guc_reset_rewind; + engine->reset.cancel = guc_reset_cancel; engine->reset.finish = guc_reset_finish; - engine->cancel_requests = guc_cancel_requests; - engine->flags &= ~I915_ENGINE_SUPPORTS_STATS; engine->flags |= I915_ENGINE_NEEDS_BREADCRUMB_TASKLET; diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c index 64f8ba3449ed..734064e355ec 100644 --- a/drivers/gpu/drm/i915/i915_drv.c +++ b/drivers/gpu/drm/i915/i915_drv.c @@ -617,7 +617,6 @@ static int i915_driver_mmio_probe(struct drm_i915_private *dev_priv) */ static void i915_driver_mmio_release(struct drm_i915_private *dev_priv) { - intel_engines_cleanup(&dev_priv->gt); intel_teardown_mchbar(dev_priv); intel_uncore_fini_mmio(&dev_priv->uncore); pci_dev_put(dev_priv->bridge_dev); diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 04350e61c86c..e19432d1795d 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -44,19 +44,11 @@ #include "gem/i915_gem_clflush.h" #include "gem/i915_gem_context.h" #include "gem/i915_gem_ioctls.h" -#include "gem/i915_gem_pm.h" #include "gt/intel_engine_user.h" #include "gt/intel_gt.h" -#include "gt/intel_gt_pm.h" -#include "gt/intel_gt_requests.h" -#include "gt/intel_mocs.h" -#include "gt/intel_reset.h" -#include "gt/intel_renderstate.h" -#include "gt/intel_rps.h" #include "gt/intel_workarounds.h" #include "i915_drv.h" -#include "i915_scatterlist.h" #include "i915_trace.h" #include "i915_vgpu.h" @@ -1039,152 +1031,6 @@ i915_gem_madvise_ioctl(struct drm_device *dev, void *data, return err; } -static int __intel_engines_record_defaults(struct intel_gt *gt) -{ - struct i915_request *requests[I915_NUM_ENGINES] = {}; - struct intel_engine_cs *engine; - enum intel_engine_id id; - int err = 0; - - /* - * As we reset the gpu during very early sanitisation, the current - * register state on the GPU should reflect its defaults values. - * We load a context onto the hw (with restore-inhibit), then switch - * over to a second context to save that default register state. We - * can then prime every new context with that state so they all start - * from the same default HW values. - */ - - for_each_engine(engine, gt, id) { - struct intel_context *ce; - struct i915_request *rq; - - /* We must be able to switch to something! */ - GEM_BUG_ON(!engine->kernel_context); - engine->serial++; /* force the kernel context switch */ - - ce = intel_context_create(engine); - if (IS_ERR(ce)) { - err = PTR_ERR(ce); - goto out; - } - - rq = intel_context_create_request(ce); - if (IS_ERR(rq)) { - err = PTR_ERR(rq); - intel_context_put(ce); - goto out; - } - - err = intel_engine_emit_ctx_wa(rq); - if (err) - goto err_rq; - - err = intel_renderstate_emit(rq); - if (err) - goto err_rq; - -err_rq: - requests[id] = i915_request_get(rq); - i915_request_add(rq); - if (err) - goto out; - } - - /* Flush the default context image to memory, and enable powersaving. */ - if (intel_gt_wait_for_idle(gt, I915_GEM_IDLE_TIMEOUT) == -ETIME) { - err = -EIO; - goto out; - } - - for (id = 0; id < ARRAY_SIZE(requests); id++) { - struct i915_request *rq; - struct i915_vma *state; - void *vaddr; - - rq = requests[id]; - if (!rq) - continue; - - /* We want to be able to unbind the state from the GGTT */ - GEM_BUG_ON(intel_context_is_pinned(rq->context)); - - state = rq->context->state; - if (!state) - continue; - - /* - * As we will hold a reference to the logical state, it will - * not be torn down with the context, and importantly the - * object will hold onto its vma (making it possible for a - * stray GTT write to corrupt our defaults). Unmap the vma - * from the GTT to prevent such accidents and reclaim the - * space. - */ - err = i915_vma_unbind(state); - if (err) - goto out; - - i915_gem_object_lock(state->obj); - err = i915_gem_object_set_to_cpu_domain(state->obj, false); - i915_gem_object_unlock(state->obj); - if (err) - goto out; - - i915_gem_object_set_cache_coherency(state->obj, I915_CACHE_LLC); - - /* Check we can acquire the image of the context state */ - vaddr = i915_gem_object_pin_map(state->obj, I915_MAP_FORCE_WB); - if (IS_ERR(vaddr)) { - err = PTR_ERR(vaddr); - goto out; - } - - rq->engine->default_state = i915_gem_object_get(state->obj); - i915_gem_object_unpin_map(state->obj); - } - -out: - /* - * If we have to abandon now, we expect the engines to be idle - * and ready to be torn-down. The quickest way we can accomplish - * this is by declaring ourselves wedged. - */ - if (err) - intel_gt_set_wedged(gt); - - for (id = 0; id < ARRAY_SIZE(requests); id++) { - struct intel_context *ce; - struct i915_request *rq; - - rq = requests[id]; - if (!rq) - continue; - - ce = rq->context; - i915_request_put(rq); - intel_context_put(ce); - } - return err; -} - -static int intel_engines_verify_workarounds(struct intel_gt *gt) -{ - struct intel_engine_cs *engine; - enum intel_engine_id id; - int err = 0; - - if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM)) - return 0; - - for_each_engine(engine, gt, id) { - if (intel_engine_verify_workarounds(engine, "load")) - err = -EIO; - } - - return err; -} - int i915_gem_init(struct drm_i915_private *dev_priv) { int ret; @@ -1201,45 +1047,12 @@ int i915_gem_init(struct drm_i915_private *dev_priv) intel_uc_fetch_firmwares(&dev_priv->gt.uc); intel_wopcm_init(&dev_priv->wopcm); - /* This is just a security blanket to placate dragons. - * On some systems, we very sporadically observe that the first TLBs - * used by the CS may be stale, despite us poking the TLB reset. If - * we hold the forcewake during initialisation these problems - * just magically go away. - */ - intel_uncore_forcewake_get(&dev_priv->uncore, FORCEWAKE_ALL); - ret = i915_init_ggtt(dev_priv); if (ret) { GEM_BUG_ON(ret == -EIO); goto err_unlock; } - intel_gt_init(&dev_priv->gt); - - ret = intel_engines_setup(&dev_priv->gt); - if (ret) { - GEM_BUG_ON(ret == -EIO); - goto err_gt_early; - } - - ret = intel_engines_init(&dev_priv->gt); - if (ret) { - GEM_BUG_ON(ret == -EIO); - goto err_engines; - } - - intel_uc_init(&dev_priv->gt.uc); - - ret = intel_gt_init_hw(&dev_priv->gt); - if (ret) - goto err_uc_init; - - /* Only when the HW is re-initialised, can we replay the requests */ - ret = intel_gt_resume(&dev_priv->gt); - if (ret) - goto err_init_hw; - /* * Despite its name intel_init_clock_gating applies both display * clock gating workarounds; GT mmio workarounds and the occasional @@ -1251,23 +1064,9 @@ int i915_gem_init(struct drm_i915_private *dev_priv) */ intel_init_clock_gating(dev_priv); - ret = intel_engines_verify_workarounds(&dev_priv->gt); - if (ret) - goto err_gt_late; - - ret = __intel_engines_record_defaults(&dev_priv->gt); - if (ret) - goto err_gt_late; - - ret = i915_inject_probe_error(dev_priv, -ENODEV); + ret = intel_gt_init(&dev_priv->gt); if (ret) - goto err_gt_late; - - ret = i915_inject_probe_error(dev_priv, -EIO); - if (ret) - goto err_gt_late; - - intel_uncore_forcewake_put(&dev_priv->uncore, FORCEWAKE_ALL); + goto err_unlock; return 0; @@ -1277,24 +1076,8 @@ int i915_gem_init(struct drm_i915_private *dev_priv) * HW as irrevisibly wedged, but keep enough state around that the * driver doesn't explode during runtime. */ -err_gt_late: - intel_gt_set_wedged_on_init(&dev_priv->gt); - i915_gem_suspend(dev_priv); - i915_gem_suspend_late(dev_priv); - - i915_gem_drain_workqueue(dev_priv); -err_init_hw: - intel_uc_fini_hw(&dev_priv->gt.uc); -err_uc_init: - if (ret != -EIO) - intel_uc_fini(&dev_priv->gt.uc); -err_engines: - if (ret != -EIO) - intel_engines_cleanup(&dev_priv->gt); -err_gt_early: - intel_gt_driver_release(&dev_priv->gt); err_unlock: - intel_uncore_forcewake_put(&dev_priv->uncore, FORCEWAKE_ALL); + i915_gem_drain_workqueue(dev_priv); if (ret != -EIO) { intel_uc_cleanup_firmwares(&dev_priv->gt.uc); @@ -1342,19 +1125,16 @@ void i915_gem_driver_remove(struct drm_i915_private *dev_priv) i915_gem_suspend_late(dev_priv); intel_gt_driver_remove(&dev_priv->gt); + dev_priv->uabi_engines = RB_ROOT; /* Flush any outstanding unpin_work. */ i915_gem_drain_workqueue(dev_priv); - intel_uc_fini_hw(&dev_priv->gt.uc); - intel_uc_fini(&dev_priv->gt.uc); - i915_gem_drain_freed_objects(dev_priv); } void i915_gem_driver_release(struct drm_i915_private *dev_priv) { - intel_engines_cleanup(&dev_priv->gt); intel_gt_driver_release(&dev_priv->gt); intel_wa_list_free(&dev_priv->gt_wa_list); diff --git a/drivers/gpu/drm/i915/selftests/i915_gem.c b/drivers/gpu/drm/i915/selftests/i915_gem.c index 657e23a8dd11..b37fc53973cc 100644 --- a/drivers/gpu/drm/i915/selftests/i915_gem.c +++ b/drivers/gpu/drm/i915/selftests/i915_gem.c @@ -9,6 +9,7 @@ #include "gem/selftests/igt_gem_utils.h" #include "gem/selftests/mock_context.h" #include "gt/intel_gt.h" +#include "gt/intel_gt_pm.h" #include "i915_selftest.h"