From patchwork Fri Dec 16 07:47:13 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 9477419 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 383496047D for ; Fri, 16 Dec 2016 07:50:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2B7E62880A for ; Fri, 16 Dec 2016 07:50:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2050228851; Fri, 16 Dec 2016 07:50:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3C55A2880A for ; Fri, 16 Dec 2016 07:50:01 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 294F86EB3D; Fri, 16 Dec 2016 07:49:59 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-wm0-x241.google.com (mail-wm0-x241.google.com [IPv6:2a00:1450:400c:c09::241]) by gabe.freedesktop.org (Postfix) with ESMTPS id E75226EB3F; Fri, 16 Dec 2016 07:48:01 +0000 (UTC) Received: by mail-wm0-x241.google.com with SMTP id u144so3544798wmu.0; Thu, 15 Dec 2016 23:48:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=X2ppWmghsEm4Y9o32sftnwilnl9A8KKdRNK7H/fTigk=; b=gaUSxZS+hqD+pTxkxOM3a7SiEye0pcf0ViBM+QEPrmEWlnESuosyRUb5CgywIAX5Vb s+L1Jgl6ZL4zbzg6kOcimixPG3yvflX3dmCZnYG6zd2Np386atyYJX2Fa0VzesWdPlvc P9RAenhdYhF8bTfAkDkxSkS29vGiNKvivK5FVB/ilNYmbx5zD3fmo0XmFYVc5H6k7562 HeyqB6kOGGVyIZrcpj20EPEhAAC82+5yiOgoKifEWGjPU3jEFGAJMwCpdv3cf5RWd6X0 +rVz3MrKHe4qTFjGxeF//xgbzLZcJaSW2TcXv+2RYmrgw+t6oGGbzLHvjqWlW1u9fOnb TMXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=X2ppWmghsEm4Y9o32sftnwilnl9A8KKdRNK7H/fTigk=; b=feJOBj52ZVm7DHz+gPnknCz0/Fg54/Pb31Pzyz9LK3HO2UZzGFD2+8zLc8B9O0Zusr RkKyrPqCCt46pcn/+UeA8zy0FksIFh9lkiETJ9f9U6QPjm5beyf0EMvNhaGbbHiAU10/ vYOJtTVMuu8ROnjVMTToRrAxnktqHzQ+LMF8Uiw9o09V36523E5SNFMdd/i8bNJlNlrD XMYha3ozlpbiRbA//hshJT1iBMtjy4yMWJL/0v29F5VS0xm6JoLI6Tlqy79KdW2JWJ4o mh9zPUqSpOn+cNpeUgYy7q0ikWxAqpmHuxpa80oVQmew7uE10m9t4zRYF6DY2hwUUMmh MdAw== X-Gm-Message-State: AIkVDXLO9n3utntc1MQlgrLzZaBQVrbEl2MIv97v39+eSDsjATtlUtvW1WyKo2DZawr4Xw== X-Received: by 10.28.127.9 with SMTP id a9mr1961607wmd.95.1481874480100; Thu, 15 Dec 2016 23:48:00 -0800 (PST) Received: from haswell.alporthouse.com ([78.156.65.138]) by smtp.gmail.com with ESMTPSA id c202sm2792407wme.1.2016.12.15.23.47.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Dec 2016 23:47:59 -0800 (PST) From: Chris Wilson To: dri-devel@lists.freedesktop.org Subject: [PATCH v2 35/40] drm: Apply tight eviction scanning to color_adjust Date: Fri, 16 Dec 2016 07:47:13 +0000 Message-Id: <20161216074718.32500-36-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20161216074718.32500-1-chris@chris-wilson.co.uk> References: <20161216074718.32500-1-chris@chris-wilson.co.uk> Cc: intel-gfx@lists.freedesktop.org, joonas.lahtinen@linux.intel.com X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Using mm->color_adjust makes the eviction scanner much tricker since we don't know the actual neighbours of the target hole until after it is created (after scanning is complete). To work out whether we need to evict the neighbours because they impact upon the hole, we have to then check the hole afterwards - requiring an extra step in the user of the eviction scanner when they apply color_adjust. v2: Massage kerneldoc. Signed-off-by: Chris Wilson Reviewed-by: Joonas Lahtinen --- drivers/gpu/drm/drm_mm.c | 76 ++++++++++++++++++++++----------- drivers/gpu/drm/i915/i915_gem_evict.c | 7 +++ drivers/gpu/drm/selftests/test-drm_mm.c | 20 ++++++++- include/drm/drm_mm.h | 1 + 4 files changed, 77 insertions(+), 27 deletions(-) diff --git a/drivers/gpu/drm/drm_mm.c b/drivers/gpu/drm/drm_mm.c index 91c89ae09b26..22db356e3ebc 100644 --- a/drivers/gpu/drm/drm_mm.c +++ b/drivers/gpu/drm/drm_mm.c @@ -692,19 +692,21 @@ EXPORT_SYMBOL(drm_mm_replace_node); * The DRM range allocator supports this use-case through the scanning * interfaces. First a scan operation needs to be initialized with * drm_mm_scan_init() or drm_mm_scan_init_with_range(). The driver adds - * objects to the roaster (probably by walking an LRU list, but this can be - * freely implemented) until a suitable hole is found or there's no further - * evictable object. + * objects to the roster (probably by walking an LRU list, but this can be + * freely implemented) (using drm_mm_scan_add_block()) until a suitable hole + * is found or there are no further evictable objects. * * The driver must walk through all objects again in exactly the reverse * order to restore the allocator state. Note that while the allocator is used * in the scan mode no other operation is allowed. * - * Finally the driver evicts all objects selected in the scan. Adding and - * removing an object is O(1), and since freeing a node is also O(1) the overall - * complexity is O(scanned_objects). So like the free stack which needs to be - * walked before a scan operation even begins this is linear in the number of - * objects. It doesn't seem to hurt badly. + * Finally the driver evicts all objects selected (drm_mm_scan_remove_block() + * reported true) in the scan, and any overlapping nodes after color adjustment + * (drm_mm_scan_evict_color()). Adding and removing an object is O(1), and + * since freeing a node is also O(1) the overall complexity is + * O(scanned_objects). So like the free stack which needs to be walked before a + * scan operation even begins this is linear in the number of objects. It + * doesn't seem to hurt too badly. */ /** @@ -829,23 +831,8 @@ bool drm_mm_scan_add_block(struct drm_mm_scan *scan, } } - if (mm->color_adjust) { - /* If allocations need adjusting due to neighbouring colours, - * we do not have enough information to decide if we need - * to evict nodes on either side of [adj_start, adj_end]. - * What almost works is - * hit_start = adj_start + (hole_start - col_start); - * hit_end = adj_start + scan->size + (hole_end - col_end); - * but because the decision is only made on the final hole, - * we may underestimate the required adjustments for an - * interior allocation. - */ - scan->hit_start = hole_start; - scan->hit_end = hole_end; - } else { - scan->hit_start = adj_start; - scan->hit_end = adj_start + scan->size; - } + scan->hit_start = adj_start; + scan->hit_end = adj_start + scan->size; DRM_MM_BUG_ON(scan->hit_start >= scan->hit_end); DRM_MM_BUG_ON(scan->hit_start < hole_start); @@ -903,6 +890,45 @@ bool drm_mm_scan_remove_block(struct drm_mm_scan *scan, EXPORT_SYMBOL(drm_mm_scan_remove_block); /** + * drm_mm_scan_color_evict - evict overlapping nodes on either side of hole + * @scan: drm_mm scan with target hole + * + * After completing an eviction scan and removing the selected nodes, we may + * need to remove a few more nodes from either side of the target hole if + * mm.color_adjust is being used. + * + * Returns: + * A node to evict, or NULL if there are no overlapping nodes. + */ +struct drm_mm_node *drm_mm_scan_color_evict(struct drm_mm_scan *scan) +{ + struct drm_mm *mm = scan->mm; + struct drm_mm_node *hole; + u64 hole_start, hole_end; + + DRM_MM_BUG_ON(list_empty(&mm->hole_stack)); + + if (!mm->color_adjust) + return NULL; + + hole = list_first_entry(&mm->hole_stack, typeof(*hole), hole_stack); + hole_start = __drm_mm_hole_node_start(hole); + hole_end = __drm_mm_hole_node_end(hole); + + DRM_MM_BUG_ON(hole_start > scan->hit_start); + DRM_MM_BUG_ON(hole_end < scan->hit_end); + + mm->color_adjust(hole, scan->color, &hole_start, &hole_end); + if (hole_start > scan->hit_start) + return hole; + if (hole_end < scan->hit_end) + return list_next_entry(hole, node_list); + + return NULL; +} +EXPORT_SYMBOL(drm_mm_scan_color_evict); + +/** * drm_mm_init - initialize a drm-mm allocator * @mm: the drm_mm structure to initialize * @start: start of the range managed by @mm diff --git a/drivers/gpu/drm/i915/i915_gem_evict.c b/drivers/gpu/drm/i915/i915_gem_evict.c index 2741498cdf2b..50129ec1caab 100644 --- a/drivers/gpu/drm/i915/i915_gem_evict.c +++ b/drivers/gpu/drm/i915/i915_gem_evict.c @@ -108,6 +108,7 @@ i915_gem_evict_something(struct i915_address_space *vm, NULL, }, **phase; struct i915_vma *vma, *next; + struct drm_mm_node *node; int ret; lockdep_assert_held(&vm->i915->drm.struct_mutex); @@ -218,6 +219,12 @@ i915_gem_evict_something(struct i915_address_space *vm, if (ret == 0) ret = i915_vma_unbind(vma); } + + while (ret == 0 && (node = drm_mm_scan_color_evict(&scan))) { + vma = container_of(node, struct i915_vma, node); + ret = i915_vma_unbind(vma); + } + return ret; } diff --git a/drivers/gpu/drm/selftests/test-drm_mm.c b/drivers/gpu/drm/selftests/test-drm_mm.c index a9fed0637297..9899e8364350 100644 --- a/drivers/gpu/drm/selftests/test-drm_mm.c +++ b/drivers/gpu/drm/selftests/test-drm_mm.c @@ -1132,6 +1132,7 @@ static bool evict_nodes(struct drm_mm_scan *scan, struct evict_node *nodes, unsigned int *order, unsigned int count, + bool use_color, struct list_head *evict_list) { struct evict_node *e, *en; @@ -1156,6 +1157,21 @@ static bool evict_nodes(struct drm_mm_scan *scan, list_for_each_entry(e, evict_list, link) drm_mm_remove_node(&e->node); + if (use_color) { + struct drm_mm_node *node; + + while ((node = drm_mm_scan_color_evict(scan))) { + e = container_of(node, typeof(*e), node); + drm_mm_remove_node(&e->node); + list_add(&e->link, evict_list); + } + } else { + if (drm_mm_scan_color_evict(scan)) { + pr_err("drm_mm_scan_color_evict unexpectedly reported overlapping nodes!\n"); + return false; + } + } + return true; } @@ -1269,7 +1285,7 @@ static int evict_something(struct drm_mm *mm, range_start, range_end, mode->create_flags); if (!evict_nodes(&scan, - nodes, order, count, + nodes, order, count, false, &evict_list)) return -EINVAL; @@ -1854,7 +1870,7 @@ static int evict_color(struct drm_mm *mm, range_start, range_end, mode->create_flags); if (!evict_nodes(&scan, - nodes, order, count, + nodes, order, count, true, &evict_list)) return -EINVAL; diff --git a/include/drm/drm_mm.h b/include/drm/drm_mm.h index a1532eb0ffbc..6e5911fa9018 100644 --- a/include/drm/drm_mm.h +++ b/include/drm/drm_mm.h @@ -410,6 +410,7 @@ bool drm_mm_scan_add_block(struct drm_mm_scan *scan, struct drm_mm_node *node); bool drm_mm_scan_remove_block(struct drm_mm_scan *scan, struct drm_mm_node *node); +struct drm_mm_node *drm_mm_scan_color_evict(struct drm_mm_scan *scan); void drm_mm_debug_table(const struct drm_mm *mm, const char *prefix); #ifdef CONFIG_DEBUG_FS