From patchwork Sat Oct 19 10:04:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11200087 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 225BD13B1 for ; Sat, 19 Oct 2019 10:05:14 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B4B6C21D7C for ; Sat, 19 Oct 2019 10:05:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B4B6C21D7C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4D9D589194; Sat, 19 Oct 2019 10:05:11 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id AE45889D7C for ; Sat, 19 Oct 2019 10:05:09 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 18893440-1500050 for multiple; Sat, 19 Oct 2019 11:04:40 +0100 From: Chris Wilson To: intel-gfx@lists.freedesktop.org Date: Sat, 19 Oct 2019 11:04:37 +0100 Message-Id: <20191019100439.24640-1-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.24.0.rc0 MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH i-g-t 1/3] i915: Exercise preemption timeout controls in sysfs X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" We [will] expose various per-engine scheduling controls. One of which, 'preempt_timeout_ms', defines how we wait for a preemption request to be honoured by the currently executing context. If it fails to relieve the GPU within the required timeout, the engine is reset and the miscreant forcibly evicted. Signed-off-by: Chris Wilson --- lib/i915/gem_context.c | 41 ++++ lib/i915/gem_context.h | 2 + tests/Makefile.sources | 1 + tests/i915/sysfs_preemption_timeout.c | 336 ++++++++++++++++++++++++++ tests/meson.build | 1 + 5 files changed, 381 insertions(+) create mode 100644 tests/i915/sysfs_preemption_timeout.c diff --git a/lib/i915/gem_context.c b/lib/i915/gem_context.c index 1fae5191f..aa083c2f0 100644 --- a/lib/i915/gem_context.c +++ b/lib/i915/gem_context.c @@ -403,3 +403,44 @@ bool gem_context_has_engine(int fd, uint32_t ctx, uint64_t engine) return __gem_execbuf(fd, &execbuf) == -ENOENT; } + +static int create_ext_ioctl(int i915, + struct drm_i915_gem_context_create_ext *arg) +{ + int err; + + err = 0; + if (igt_ioctl(i915, DRM_IOCTL_I915_GEM_CONTEXT_CREATE_EXT, arg)) { + err = -errno; + igt_assume(err); + } + + errno = 0; + return err; +} + +uint32_t gem_context_create_for_engine(int i915, unsigned int class, unsigned int inst) +{ + I915_DEFINE_CONTEXT_PARAM_ENGINES(engines, 1) = { + .engines = { { .engine_class = class, .engine_instance = inst } } + }; + struct drm_i915_gem_context_create_ext_setparam p_engines = { + .base = { + .name = I915_CONTEXT_CREATE_EXT_SETPARAM, + .next_extension = 0, /* end of chain */ + }, + .param = { + .param = I915_CONTEXT_PARAM_ENGINES, + .value = to_user_pointer(&engines), + .size = sizeof(engines), + }, + }; + struct drm_i915_gem_context_create_ext create = { + .flags = I915_CONTEXT_CREATE_FLAGS_USE_EXTENSIONS, + .extensions = to_user_pointer(&p_engines), + }; + + igt_assert_eq(create_ext_ioctl(i915, &create), 0); + igt_assert_neq(create.ctx_id, 0); + return create.ctx_id; +} diff --git a/lib/i915/gem_context.h b/lib/i915/gem_context.h index c0d4c9615..9e0a083f0 100644 --- a/lib/i915/gem_context.h +++ b/lib/i915/gem_context.h @@ -34,6 +34,8 @@ int __gem_context_create(int fd, uint32_t *ctx_id); void gem_context_destroy(int fd, uint32_t ctx_id); int __gem_context_destroy(int fd, uint32_t ctx_id); +uint32_t gem_context_create_for_engine(int fd, unsigned int class, unsigned int inst); + int __gem_context_clone(int i915, uint32_t src, unsigned int share, unsigned int flags, diff --git a/tests/Makefile.sources b/tests/Makefile.sources index 093eb57f3..dff7dac06 100644 --- a/tests/Makefile.sources +++ b/tests/Makefile.sources @@ -98,6 +98,7 @@ TESTS_progs = \ tools_test \ vgem_basic \ vgem_slow \ + i915/sysfs_preemption_timeout \ $(NULL) TESTS_progs += gem_bad_reloc diff --git a/tests/i915/sysfs_preemption_timeout.c b/tests/i915/sysfs_preemption_timeout.c new file mode 100644 index 000000000..0d24f751f --- /dev/null +++ b/tests/i915/sysfs_preemption_timeout.c @@ -0,0 +1,336 @@ +/* + * Copyright © 2019 Intel Corporation + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice (including the next + * paragraph) shall be included in all copies or substantial portions of the + * Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS + * IN THE SOFTWARE. + */ + +#include +#include +#include +#include +#include +#include +#include + +#include "drmtest.h" /* gem_quiescent_gpu()! */ +#include "i915/gem_engine_topology.h" +#include "igt_dummyload.h" +#include "igt_sysfs.h" +#include "ioctl_wrappers.h" /* igt_require_gem()! */ +#include "sw_sync.h" + +#include "igt_debugfs.h" + +static bool __enable_hangcheck(int dir, bool state) +{ + return igt_sysfs_set(dir, "enable_hangcheck", state ? "1" : "0"); +} + +static bool enable_hangcheck(int i915, bool state) +{ + bool success; + int dir; + + dir = igt_sysfs_open_parameters(i915); + if (dir < 0) /* no parameters, must be default! */ + return false; + + success = __enable_hangcheck(dir, state); + close(dir); + + return success; +} + +static void set_preempt_timeout(int engine, unsigned int value) +{ + unsigned int delay; + + igt_sysfs_printf(engine, "preempt_timeout_ms", "%u", value); + igt_sysfs_scanf(engine, "preempt_timeout_ms", "%u", &delay); + igt_assert_eq(delay, value); +} + +static void test_idempotent(int i915, int engine) +{ + unsigned int saved; + + igt_assert(igt_sysfs_scanf(engine, "preempt_timeout_ms", "%u", &saved) == 1); + igt_debug("Initial preempt_timeout_ms:%u\n", saved); + + set_preempt_timeout(engine, 0); + set_preempt_timeout(engine, 1); + set_preempt_timeout(engine, 1000); + set_preempt_timeout(engine, 1234); + set_preempt_timeout(engine, 654321); + + set_preempt_timeout(engine, saved); +} + +static void test_invalid(int i915, int engine) +{ + unsigned int saved, delay; + + igt_assert(igt_sysfs_scanf(engine, "preempt_timeout_ms", "%u", &saved) == 1); + igt_debug("Initial preempt_timeout_ms:%u\n", saved); + + igt_sysfs_printf(engine, "preempt_timeout_ms", PRIu64, -1); + igt_sysfs_scanf(engine, "preempt_timeout_ms", "%u", &delay); + igt_assert_eq(delay, saved); + + igt_sysfs_printf(engine, "preempt_timeout_ms", "%d", -1); + igt_sysfs_scanf(engine, "preempt_timeout_ms", "%u", &delay); + igt_assert_eq(delay, saved); + + igt_sysfs_printf(engine, "preempt_timeout_ms", PRIu64, 40ull << 32); + igt_sysfs_scanf(engine, "preempt_timeout_ms", "%u", &delay); + igt_assert_eq(delay, saved); +} + +static void set_unbannable(int i915, uint32_t ctx) +{ + struct drm_i915_gem_context_param p = { + .ctx_id = ctx, + .param = I915_CONTEXT_PARAM_BANNABLE, + }; + + igt_assert_eq(__gem_context_set_param(i915, &p), 0); +} + +static uint32_t create_context(int i915, unsigned int class, unsigned int inst, int prio) +{ + uint32_t ctx; + + ctx = gem_context_create_for_engine(i915, class, inst); + set_unbannable(i915, ctx); + gem_context_set_priority(i915, ctx, prio); + + return ctx; +} + +static uint64_t __test_timeout(int i915, int engine, unsigned int timeout) +{ + unsigned int class, inst; + struct timespec ts = {}; + igt_spin_t *spin[2]; + uint64_t elapsed; + uint32_t ctx[2]; + + igt_assert(igt_sysfs_scanf(engine, "class", "%u", &class) == 1); + igt_assert(igt_sysfs_scanf(engine, "instance", "%u", &inst) == 1); + + set_preempt_timeout(engine, timeout); + + ctx[0] = create_context(i915, class, inst, -1023); + spin[0] = igt_spin_new(i915, ctx[0], + .flags = (IGT_SPIN_NO_PREEMPTION | + IGT_SPIN_POLL_RUN | + IGT_SPIN_FENCE_OUT)); + igt_spin_busywait_until_started(spin[0]); + + ctx[1] = create_context(i915, class, inst, 1023); + igt_nsec_elapsed(&ts); + spin[1] = igt_spin_new(i915, ctx[1], .flags = IGT_SPIN_POLL_RUN); + igt_spin_busywait_until_started(spin[1]); + elapsed = igt_nsec_elapsed(&ts); + + igt_spin_free(i915, spin[1]); + + igt_assert_eq(sync_fence_wait(spin[0]->out_fence, 1), 0); + igt_assert_eq(sync_fence_status(spin[0]->out_fence), -EIO); + + igt_spin_free(i915, spin[0]); + + gem_context_destroy(i915, ctx[1]); + gem_context_destroy(i915, ctx[0]); + gem_quiescent_gpu(i915); + + return elapsed; +} + +static void test_timeout(int i915, int engine) +{ + int delays[] = { 1, 50, 100, 500 }; + unsigned int saved; + + igt_assert(igt_sysfs_scanf(engine, "preempt_timeout_ms", "%u", &saved) == 1); + igt_debug("Initial preempt_timeout_ms:%u\n", saved); + + gem_quiescent_gpu(i915); + igt_require(enable_hangcheck(i915, false)); + + for (int i = 0; i < ARRAY_SIZE(delays); i++) { + uint64_t elapsed; + + elapsed = __test_timeout(i915, engine, delays[i]); + igt_info("preempt_timeout_ms:%d, elapsed=%.3fms\n", + delays[i], elapsed * 1e-6); + + /* + * We need to give a couple of jiffies slack for the scheduler timeouts + * and then a little more slack fr the overhead in submitting and + * measuring. 50ms should cover all of our sins and be useful + * tolerance. + */ + igt_assert_f(elapsed / 1000 / 1000 < delays[i] + 50, + "Forced preemption timeout exceeded request!\n"); + } + + igt_assert(enable_hangcheck(i915, true)); + gem_quiescent_gpu(i915); + + set_preempt_timeout(engine, saved); +} + +static void test_off(int i915, int engine) +{ + unsigned int class, inst; + igt_spin_t *spin[2]; + unsigned int saved; + uint32_t ctx[2]; + + igt_assert(igt_sysfs_scanf(engine, "preempt_timeout_ms", "%u", &saved) == 1); + igt_debug("Initial preempt_timeout_ms:%u\n", saved); + + gem_quiescent_gpu(i915); + igt_require(enable_hangcheck(i915, false)); + + igt_assert(igt_sysfs_scanf(engine, "class", "%u", &class) == 1); + igt_assert(igt_sysfs_scanf(engine, "instance", "%u", &inst) == 1); + + set_preempt_timeout(engine, 0); + + ctx[0] = create_context(i915, class, inst, -1023); + spin[0] = igt_spin_new(i915, ctx[0], + .flags = (IGT_SPIN_NO_PREEMPTION | + IGT_SPIN_POLL_RUN | + IGT_SPIN_FENCE_OUT)); + igt_spin_busywait_until_started(spin[0]); + + ctx[1] = create_context(i915, class, inst, 1023); + spin[1] = igt_spin_new(i915, ctx[1], .flags = IGT_SPIN_POLL_RUN); + + for (int i = 0; i < 150; i++) { + igt_assert_eq(sync_fence_status(spin[0]->out_fence), 0); + sleep(1); + } + + set_preempt_timeout(engine, 1); + + igt_spin_busywait_until_started(spin[1]); + igt_spin_free(i915, spin[1]); + + igt_assert_eq(sync_fence_wait(spin[0]->out_fence, 1), 0); + igt_assert_eq(sync_fence_status(spin[0]->out_fence), -EIO); + + igt_spin_free(i915, spin[0]); + + gem_context_destroy(i915, ctx[1]); + gem_context_destroy(i915, ctx[0]); + + igt_assert(enable_hangcheck(i915, true)); + gem_quiescent_gpu(i915); + + set_preempt_timeout(engine, saved); +} + +#if 0 +static void each_engines(int fd) +{ + struct dirent *de; + DIR *dir; + + dir = fdopendir(fd); + while (dir && (de = readdir(dir))) { + int engine = openat(engines, de->d_name, O_RDONLY); + char *name; + + name = igt_sysfs_get(engine, "name"); + if (!name) + continue; + + igt_subtest_group { + igt_fixture { + igt_require(fstatat(engine, + "preempt_timeout_ms", + &st, 0) == 0); + } + } + } + closedir(dir); +} +#endif + +igt_main +{ + const struct intel_execution_engine2 *it; + int i915 = -1, engines = -1; + + igt_fixture { + int sys; + + i915 = drm_open_driver(DRIVER_INTEL); + igt_require_gem(i915); + + sys = igt_sysfs_open(i915); + igt_require(sys != -1); + + engines = openat(sys, "engine", O_RDONLY); + igt_require(engines != -1); + + close(sys); + } + + __for_each_static_engine(it) { + igt_subtest_group { + int engine = -1; + char *name = NULL; + + igt_fixture { + struct stat st; + + engine = openat(engines, it->name, O_RDONLY); + igt_require(fstatat(engine, + "preempt_timeout_ms", + &st, 0) == 0); + + name = igt_sysfs_get(engine, "name"); + } + if (!name) + continue; + + igt_subtest_f("%s-idempotent", name) + test_idempotent(i915, engine); + igt_subtest_f("%s-invalid", name) + test_invalid(i915, engine); + igt_subtest_f("%s-timeout", name) + test_timeout(i915, engine); + igt_subtest_f("%s-off", name) + test_off(i915, engine); + + free(name); + close(engine); + } + } + + igt_fixture { + close(engines); + close(i915); + } +} diff --git a/tests/meson.build b/tests/meson.build index 3f3eee277..a699377e3 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -238,6 +238,7 @@ i915_progs = [ 'i915_query', 'i915_selftest', 'i915_suspend', + 'sysfs_preemption_timeout', ] test_deps = [ igt_deps ]