From patchwork Wed Dec 4 11:15:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11272757 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 39C7F138C for ; Wed, 4 Dec 2019 11:16:08 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2029D206DB for ; Wed, 4 Dec 2019 11:16:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2029D206DB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B0E8D6E8C1; Wed, 4 Dec 2019 11:16:06 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id 09AA46E8C1; Wed, 4 Dec 2019 11:16:04 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 19455310-1500050 for multiple; Wed, 04 Dec 2019 11:15:51 +0000 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Wed, 4 Dec 2019 11:15:53 +0000 Message-Id: <20191204111553.3620947-1-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0 MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH i-g-t] i915/gem_exec_balancer: Race SUBMIT_FENCE against semaphores X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: igt-dev@lists.freedesktop.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" The scheduler may use a semaphore between engines to serialise requests, and in doing so submit the request before its signalers are ready. This dependency must also be copied across any SUBMIT_FENCE so that a bonded-pair will not execute ahead of any of its implicit dependencies -- it too must wait for the same semaphores. Signed-off-by: Chris Wilson Cc: Tvrtko Ursulin --- tests/i915/gem_exec_balancer.c | 104 +++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) diff --git a/tests/i915/gem_exec_balancer.c b/tests/i915/gem_exec_balancer.c index e5e83508f..5f6e3d568 100644 --- a/tests/i915/gem_exec_balancer.c +++ b/tests/i915/gem_exec_balancer.c @@ -959,6 +959,107 @@ static void bonded_chain(int i915) gem_context_destroy(i915, ctx); } +static void __bonded_sema(int i915, uint32_t ctx, + const struct i915_engine_class_instance *siblings, + unsigned int count) +{ + const int priorities[] = { -1023, 0, 1023 }; + struct drm_i915_gem_exec_object2 batch = { + .handle = batch_create(i915), + }; + struct drm_i915_gem_execbuffer2 execbuf = { + .buffers_ptr = to_user_pointer(&batch), + .buffer_count = 1, + .rsvd1 = ctx, + }; + igt_spin_t *spin; + + for (int i = 0; i < ARRAY_SIZE(priorities); i++) { + /* A: spin forever on seperate render engine */ + spin = igt_spin_new(i915, + .flags = (IGT_SPIN_POLL_RUN | + IGT_SPIN_FENCE_OUT)); + igt_spin_busywait_until_started(spin); + + /* + * Note we replace the timelines between each execbuf, so + * that any pair of requests on the same engine could be + * re-ordered by the scheduler -- if the dependency tracking + * is subpar. + */ + + /* B: waits for A (using a semaphore) on engine 1 */ + set_load_balancer(i915, ctx, siblings, count, NULL); + execbuf.rsvd2 = spin->out_fence; + execbuf.flags = I915_EXEC_FENCE_IN | I915_EXEC_FENCE_OUT; + execbuf.flags |= 1; + gem_execbuf_wr(i915, &execbuf); + + /* B': run in parallel with B on engine 2 */ + if (priorities[i] > 0) + gem_context_set_priority(i915, ctx, priorities[i]); + set_load_balancer(i915, ctx, siblings, count, NULL); + execbuf.flags = I915_EXEC_FENCE_SUBMIT | I915_EXEC_FENCE_OUT; + execbuf.flags |= 2; + execbuf.rsvd2 >>= 32; + gem_execbuf_wr(i915, &execbuf); + gem_context_set_priority(i915, ctx, 0); + + /* Wait for any magic timeslicing or preemptions... */ + igt_assert_eq(sync_fence_wait(execbuf.rsvd2 >> 32, 1000), + -ETIME); + + igt_debugfs_dump(i915, "i915_engine_info"); + + /* + * ... which should not have happened, so everything is still + * waiting on the spinner + */ + igt_assert_eq(sync_fence_status(spin->out_fence), 0); + igt_assert_eq(sync_fence_status(execbuf.rsvd2 & 0xffffffff), 0); + igt_assert_eq(sync_fence_status(execbuf.rsvd2 >> 32), 0); + + igt_spin_free(i915, spin); + gem_sync(i915, batch.handle); + + igt_assert_eq(sync_fence_status(execbuf.rsvd2 & 0xffffffff), 1); + igt_assert_eq(sync_fence_status(execbuf.rsvd2 >> 32), 1); + + close(execbuf.rsvd2); + close(execbuf.rsvd2 >> 32); + } + + gem_close(i915, batch.handle); +} + +static void bonded_semaphore(int i915) +{ + uint32_t ctx; + + /* + * Given batches A, B and B', where B and B' are a bonded pair, with + * B' depending on B with a submit fence and B depending on A as + * an ordinary fence; prove B' cannot complete before A, with the + * difference here (wrt bonded_chain) that A is on another engine and + * so A, B and B' are expected to be inflight concurrently. + */ + igt_require(gem_scheduler_has_semaphores(i915)); + + ctx = gem_context_create(i915); + + for (int class = 1; class < 32; class++) { + struct i915_engine_class_instance *siblings; + unsigned int count; + + siblings = list_engines(i915, 1u << class, &count); + if (count > 1) + __bonded_sema(i915, ctx, siblings, count); + free(siblings); + } + + gem_context_destroy(i915, ctx); +} + static void indices(int i915) { I915_DEFINE_CONTEXT_PARAM_ENGINES(engines, I915_EXEC_RING_MASK + 1); @@ -1685,6 +1786,9 @@ igt_main igt_subtest("bonded-chain") bonded_chain(i915); + igt_subtest("bonded-semaphore") + bonded_semaphore(i915); + igt_fixture { igt_stop_hang_detector(); }