From patchwork Mon Dec 12 11:53:45 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 9470591 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 37CC060760 for ; Mon, 12 Dec 2016 11:56:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2BFDC28472 for ; Mon, 12 Dec 2016 11:56:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2060A28478; Mon, 12 Dec 2016 11:56:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 11F8F28472 for ; Mon, 12 Dec 2016 11:56:21 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8EDE86E32A; Mon, 12 Dec 2016 11:54:57 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-wj0-x241.google.com (mail-wj0-x241.google.com [IPv6:2a00:1450:400c:c01::241]) by gabe.freedesktop.org (Postfix) with ESMTPS id 32F646E2EF; Mon, 12 Dec 2016 11:54:29 +0000 (UTC) Received: by mail-wj0-x241.google.com with SMTP id he10so11125118wjc.2; Mon, 12 Dec 2016 03:54:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=UhtVt3RFYnuRB0sY3h0avXhjv2Opjv34tQHkaAadWOM=; b=RY/TBkkjkDqOEUx1IoyYRU9UxcE0x+E2yGSEPfp8IfZdozjzojyIIlAPJk89nDjgY3 3holIE3+1tuY+k4kVkXxletNrLt0MvIvW9STrKOJw1Qdj6R2bFLub1DjkmFZkc7IUnDv qCajY2i544dcFGJCKiitkxM76CpN1CJaMdO9uchIIvaNiJEasqnvKxtXMlqZ7pAawDag vJ8rdAZaQjcR+JSHfFxgvvgx5lBxKt0knNqW7Enk0tXG5iPYqipyZ3wi9HIhFOxJwT0i QTT/Ofyooya1l9wOr5VjPBvCaFPa8qyf3TJ009mbEvgBxZqseuywkkObSwiM58UKdHeS uSCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=UhtVt3RFYnuRB0sY3h0avXhjv2Opjv34tQHkaAadWOM=; b=EhnqdowABeU0KF+5kiNolMlxOVuj7YLexBtduBhJdpCpHZ4GgcHmsUPhWxY2kMW3UW w0EOG9E8TX6nEjWU+mVPxuyJUew115GhBcXw3/v6BjqVysOVxtfV8JAHNUEc+2/yB7Dg uG6XyiD+n6jKE5wh0qB4pJADiuZD7lrCHTyA2fuStqBDF4fB4jpOVE20s5Wl4qHvDhxf fmLsDtqa1KLR6/zzmhDGvXbfyJ+6b6AfJWqSh1ZPJqWyig7JAiTWkTKAv/DhK0CcAnBA SAC4e2mUKeVg/PHPjH81/pI3zw7Dqp4RPQpkAg4jab2sgdfq8L8OseDbYfjJXHUnEbVV gLxg== X-Gm-Message-State: AKaTC02mwbdGSNx0r2nxwZXP08YE2BIvYS8q3h9L30SUqOSWS7tW6wY0zyTeJyG4GSRrHA== X-Received: by 10.194.94.166 with SMTP id dd6mr90076240wjb.88.1481543667407; Mon, 12 Dec 2016 03:54:27 -0800 (PST) Received: from haswell.alporthouse.com ([78.156.65.138]) by smtp.gmail.com with ESMTPSA id u81sm35077381wmu.10.2016.12.12.03.54.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Dec 2016 03:54:26 -0800 (PST) From: Chris Wilson To: dri-devel@lists.freedesktop.org Subject: [PATCH 29/34] drm: Compute tight evictions for drm_mm_scan Date: Mon, 12 Dec 2016 11:53:45 +0000 Message-Id: <20161212115350.780-30-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20161212115350.780-1-chris@chris-wilson.co.uk> References: <20161212115350.780-1-chris@chris-wilson.co.uk> Cc: intel-gfx@lists.freedesktop.org, joonas.lahtinen@linux.intel.com X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Compute the minimal required hole during scan and only evict those nodes that overlap. This enables us to reduce the number of nodes we need to evict to the bare minimum. Signed-off-by: Chris Wilson Reviewed-by: Joonas Lahtinen --- drivers/gpu/drm/drm_mm.c | 60 +++++++++++++++++++++++++++------ drivers/gpu/drm/etnaviv/etnaviv_mmu.c | 2 +- drivers/gpu/drm/i915/i915_gem_evict.c | 3 +- drivers/gpu/drm/selftests/test-drm_mm.c | 22 +++++++----- include/drm/drm_mm.h | 22 ++++++------ 5 files changed, 78 insertions(+), 31 deletions(-) diff --git a/drivers/gpu/drm/drm_mm.c b/drivers/gpu/drm/drm_mm.c index a1467a43eda2..eec0a46f5b38 100644 --- a/drivers/gpu/drm/drm_mm.c +++ b/drivers/gpu/drm/drm_mm.c @@ -718,10 +718,10 @@ EXPORT_SYMBOL(drm_mm_replace_node); * @color: opaque tag value to use for the allocation * @start: start of the allowed range for the allocation * @end: end of the allowed range for the allocation + * @flags: flags to specify how the allocation will be performed afterwards * * This simply sets up the scanning routines with the parameters for the desired - * hole. Note that there's no need to specify allocation flags, since they only - * change the place a node is allocated from within a suitable hole. + * hole. * * Warning: * As long as the scan list is non-empty, no other operations than @@ -733,7 +733,8 @@ void drm_mm_scan_init_with_range(struct drm_mm_scan *scan, u64 alignment, unsigned long color, u64 start, - u64 end) + u64 end, + unsigned int flags) { DRM_MM_BUG_ON(size == 0); DRM_MM_BUG_ON(mm->scan_active); @@ -743,6 +744,7 @@ void drm_mm_scan_init_with_range(struct drm_mm_scan *scan, scan->color = color; scan->alignment = alignment; scan->size = size; + scan->flags = flags; DRM_MM_BUG_ON(end <= start); scan->range_start = start; @@ -777,7 +779,7 @@ bool drm_mm_scan_add_block(struct drm_mm_scan *scan, DRM_MM_BUG_ON(node->mm != mm); DRM_MM_BUG_ON(!node->allocated); DRM_MM_BUG_ON(node->scanned_block); - node->scanned_block = 1; + node->scanned_block = true; mm->scan_active++; hole = list_prev_entry(node, node_list); @@ -799,15 +801,53 @@ bool drm_mm_scan_add_block(struct drm_mm_scan *scan, adj_start = max(col_start, scan->range_start); adj_end = min(col_end, scan->range_end); + if (adj_end <= adj_start || adj_end - adj_start < scan->size) + return false; + + if (scan->flags == DRM_MM_CREATE_TOP) + adj_start = adj_end - scan->size; + + if (scan->alignment) { + u64 rem; + + div64_u64_rem(adj_start, scan->alignment, &rem); + if (rem) { + adj_start -= rem; + if (scan->flags != DRM_MM_CREATE_TOP) + adj_start += scan->alignment; + if (adj_start < max(col_start, scan->range_start) || + max(col_end, scan->range_end) - adj_start < scan->size) + return false; + + if (adj_end <= adj_start || + adj_end - adj_start < scan->size) + return false; + } + } - if (check_free_hole(adj_start, adj_end, - scan->size, scan->alignment)) { + if (mm->color_adjust) { + /* If allocations need adjusting due to neighbouring colours, + * we do not have enough information to decide if we need + * to evict nodes on either side of [adj_start, adj_end]. + * What almost works is + * hit_start = adj_start + (hole_start - col_start); + * hit_end = adj_start + scan->size + (hole_end - col_end); + * but because the decision is only made on the final hole, + * we may underestimate the required adjustments for an + * interior allocation. + */ scan->hit_start = hole_start; scan->hit_end = hole_end; - return true; + } else { + scan->hit_start = adj_start; + scan->hit_end = adj_start + scan->size; } - return false; + DRM_MM_BUG_ON(scan->hit_start >= scan->hit_end); + DRM_MM_BUG_ON(scan->hit_start < hole_start); + DRM_MM_BUG_ON(scan->hit_end > hole_end); + + return true; } EXPORT_SYMBOL(drm_mm_scan_add_block); @@ -834,7 +874,7 @@ bool drm_mm_scan_remove_block(struct drm_mm_scan *scan, DRM_MM_BUG_ON(node->mm != scan->mm); DRM_MM_BUG_ON(!node->scanned_block); - node->scanned_block = 0; + node->scanned_block = false; DRM_MM_BUG_ON(!node->mm->scan_active); node->mm->scan_active--; @@ -844,7 +884,7 @@ bool drm_mm_scan_remove_block(struct drm_mm_scan *scan, prev_node->hole_follows = node->scanned_preceeds_hole; list_add(&node->node_list, &prev_node->node_list); - return (drm_mm_hole_node_end(node) > scan->hit_start && + return (node->start + node->size > scan->hit_start && node->start < scan->hit_end); } EXPORT_SYMBOL(drm_mm_scan_remove_block); diff --git a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c index fe1e886dcabb..2dae3169ce48 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c @@ -135,7 +135,7 @@ static int etnaviv_iommu_find_iova(struct etnaviv_iommu *mmu, } /* Try to retire some entries */ - drm_mm_scan_init(&scan, &mmu->mm, size, 0, 0); + drm_mm_scan_init(&scan, &mmu->mm, size, 0, 0, 0); found = 0; INIT_LIST_HEAD(&list); diff --git a/drivers/gpu/drm/i915/i915_gem_evict.c b/drivers/gpu/drm/i915/i915_gem_evict.c index 5415f888c63f..89da9225043f 100644 --- a/drivers/gpu/drm/i915/i915_gem_evict.c +++ b/drivers/gpu/drm/i915/i915_gem_evict.c @@ -128,7 +128,8 @@ i915_gem_evict_something(struct i915_address_space *vm, */ drm_mm_scan_init_with_range(&scan, &vm->mm, min_size, alignment, cache_level, - start, end); + start, end, + flags & PIN_HIGH ? DRM_MM_CREATE_TOP : 0); if (flags & PIN_NONBLOCK) phases[1] = NULL; diff --git a/drivers/gpu/drm/selftests/test-drm_mm.c b/drivers/gpu/drm/selftests/test-drm_mm.c index 1180033f8a65..09ead31a094d 100644 --- a/drivers/gpu/drm/selftests/test-drm_mm.c +++ b/drivers/gpu/drm/selftests/test-drm_mm.c @@ -854,7 +854,7 @@ static int igt_evict(void *ignored) struct evict_node *e; struct drm_mm_scan scan; - drm_mm_scan_init(&scan, &mm, 1, 0, 0); + drm_mm_scan_init(&scan, &mm, 1, 0, 0, 0); for (m = 0; m < size; m++) { e = &nodes[m]; list_add(&e->link, &evict_list); @@ -902,7 +902,7 @@ static int igt_evict(void *ignored) struct drm_mm_node tmp; int err; - drm_mm_scan_init(&scan, &mm, nsize, n, 0); + drm_mm_scan_init(&scan, &mm, nsize, n, 0, 0); drm_random_reorder(order, size, &lcg_state); for (m = 0; m < size; m++) { e = &nodes[order[m]]; @@ -963,7 +963,7 @@ static int igt_evict(void *ignored) int nsize = (size - n + 1) / 2; int err; - drm_mm_scan_init(&scan, &mm, nsize, n, 0); + drm_mm_scan_init(&scan, &mm, nsize, n, 0, 0); drm_random_reorder(order, size, &lcg_state); for (m = 0; m < size; m++) { e = &nodes[order[m]]; @@ -1076,7 +1076,8 @@ static int igt_evict_range(void *ignored) int err; drm_mm_scan_init_with_range(&scan, &mm, nsize, n, 0, - range_start, range_end); + range_start, range_end, + 0); drm_random_reorder(order, size, &lcg_state); for (m = 0; m < size; m++) { e = &nodes[order[m]]; @@ -1146,7 +1147,8 @@ static int igt_evict_range(void *ignored) int err; drm_mm_scan_init_with_range(&scan, &mm, nsize, n, 0, - range_start, range_end); + range_start, range_end, + 0); drm_random_reorder(order, size, &lcg_state); for (m = 0; m < size; m++) { e = &nodes[order[m]]; @@ -1640,7 +1642,7 @@ static int igt_color_evict(void *ignored) struct drm_mm_node tmp; int err; - drm_mm_scan_init(&scan, &mm, nsize, n, c); + drm_mm_scan_init(&scan, &mm, nsize, n, c, 0); drm_random_reorder(order, total_size, &lcg_state); for (m = 0; m < total_size; m++) { e = &nodes[order[m]]; @@ -1713,7 +1715,7 @@ static int igt_color_evict(void *ignored) int nsize = (total_size - n + 1) / 2; int err; - drm_mm_scan_init(&scan, &mm, nsize, n, c); + drm_mm_scan_init(&scan, &mm, nsize, n, c, 0); drm_random_reorder(order, total_size, &lcg_state); for (m = 0; m < total_size; m++) { e = &nodes[order[m]]; @@ -1844,7 +1846,8 @@ static int igt_color_evict_range(void *ignored) int err; drm_mm_scan_init_with_range(&scan, &mm, nsize, n, c, - range_start, range_end); + range_start, range_end, + 0); drm_random_reorder(order, total_size, &lcg_state); for (m = 0; m < total_size; m++) { e = &nodes[order[m]]; @@ -1919,7 +1922,8 @@ static int igt_color_evict_range(void *ignored) int err; drm_mm_scan_init_with_range(&scan, &mm, nsize, n, c, - range_start, range_end); + range_start, range_end, + 0); drm_random_reorder(order, total_size, &lcg_state); for (m = 0; m < total_size; m++) { e = &nodes[order[m]]; diff --git a/include/drm/drm_mm.h b/include/drm/drm_mm.h index 22d00515fc57..5e8350b91fcf 100644 --- a/include/drm/drm_mm.h +++ b/include/drm/drm_mm.h @@ -119,6 +119,7 @@ struct drm_mm_scan { struct drm_mm_node *prev_scanned_node; unsigned long color; + unsigned int flags; }; /** @@ -369,11 +370,9 @@ __drm_mm_interval_first(struct drm_mm *mm, u64 start, u64 last); void drm_mm_scan_init_with_range(struct drm_mm_scan *scan, struct drm_mm *mm, - u64 size, - u64 alignment, - unsigned long color, - u64 start, - u64 end); + u64 size, u64 alignment, unsigned long color, + u64 start, u64 end, + unsigned int flags); /** * drm_mm_scan_init - initialize lru scanning @@ -382,10 +381,10 @@ void drm_mm_scan_init_with_range(struct drm_mm_scan *scan, * @size: size of the allocation * @alignment: alignment of the allocation * @color: opaque tag value to use for the allocation + * @flags: flags to specify how the allocation will be performed afterwards * * This simply sets up the scanning routines with the parameters for the desired - * hole. Note that there's no need to specify allocation flags, since they only - * change the place a node is allocated from within a suitable hole. + * hole. * * Warning: * As long as the scan list is non-empty, no other operations than @@ -395,10 +394,13 @@ static inline void drm_mm_scan_init(struct drm_mm_scan *scan, struct drm_mm *mm, u64 size, u64 alignment, - unsigned long color) + unsigned long color, + unsigned int flags) { - drm_mm_scan_init_with_range(scan, mm, size, alignment, color, - 0, U64_MAX); + drm_mm_scan_init_with_range(scan, mm, + size, alignment, color, + 0, U64_MAX, + flags); } bool drm_mm_scan_add_block(struct drm_mm_scan *scan,