From patchwork Fri Dec 16 07:47:10 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 9477413 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 14E2E6047D for ; Fri, 16 Dec 2016 07:49:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0A8982880A for ; Fri, 16 Dec 2016 07:49:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F3A3428810; Fri, 16 Dec 2016 07:49:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 417242880A for ; Fri, 16 Dec 2016 07:49:46 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4E3B86EB5C; Fri, 16 Dec 2016 07:48:39 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-wj0-x244.google.com (mail-wj0-x244.google.com [IPv6:2a00:1450:400c:c01::244]) by gabe.freedesktop.org (Postfix) with ESMTPS id E99176EB34; Fri, 16 Dec 2016 07:47:58 +0000 (UTC) Received: by mail-wj0-x244.google.com with SMTP id he10so13157992wjc.2; Thu, 15 Dec 2016 23:47:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=HDTukmtTPYHit+azZ6fegVqBpEa8s2YGQT5BB/gKJS4=; b=CGZgNOv60Ik78huSXblz+IwbIOh+kBRShX5Fs9CmdYNscry6o1V3gQg1YyF+PmS1AQ KZtxh8C1dAciJTdocdO/Msq0w/zQRN7OjA7fGhtoGGARKEWTsh5VO2zSib+IY3uMd7Ih mcM3qIQqzSk7T/ACaGSsYWQbefl+dZNxOleDRq5uWWM+WI30z8xBdrK///TvIqUmipEj SDTY6Lw4ZAdeml9MJE/K9I7J9mQTbDLoGj6+fGgsL60IzZ9UD985aQyFgLwoMvpuLCUr Belv/VoO2TQlXK7BuQXuzYswPOotklOXhT5VA/Fy8AyzR6+oys+hwvnC+0otxN1dv5Ic pnrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=HDTukmtTPYHit+azZ6fegVqBpEa8s2YGQT5BB/gKJS4=; b=P82IiZFtoT8BlMfhpsXvdprnoedRcb4xdlNGkanLs3kb/CQcZj9WV/R7l1CvP8eCT5 Z48IegFDHsmfRY+cStGGBrCSmYxM3AXnVo6sV4w8u/fSyIID9v3D29dMfWI07SeQIZlc PVjv1eSe3pYJSEoutW4TKKZYUZuo4SyDxkCOGYmAfzCsqxJa+UzjZXb7hYXY7HcUB6/k rh+h95qVpGMAhBf/zakQ2RZSW0FH+YrLp7OVELppRdJDUF8kogtIajuJmoevMU9OIUZV rS07ChC0XX6dHUAVX/0hA0DlRJSGChIw8qtqjdoFRMH1nGmuE4LduTBtoPYzAfDZrf6G osrQ== X-Gm-Message-State: AKaTC03EJBfAkaNCROLpwsrPdiK8TJW0tD0h/bTpQdvx+60NwlzrGSzDrAF9XVuWicClPA== X-Received: by 10.194.106.41 with SMTP id gr9mr1656071wjb.202.1481874476866; Thu, 15 Dec 2016 23:47:56 -0800 (PST) Received: from haswell.alporthouse.com ([78.156.65.138]) by smtp.gmail.com with ESMTPSA id c202sm2792407wme.1.2016.12.15.23.47.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Dec 2016 23:47:56 -0800 (PST) From: Chris Wilson To: dri-devel@lists.freedesktop.org Subject: [PATCH v2 32/40] drm: Compute tight evictions for drm_mm_scan Date: Fri, 16 Dec 2016 07:47:10 +0000 Message-Id: <20161216074718.32500-33-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20161216074718.32500-1-chris@chris-wilson.co.uk> References: <20161216074718.32500-1-chris@chris-wilson.co.uk> Cc: intel-gfx@lists.freedesktop.org, joonas.lahtinen@linux.intel.com X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Compute the minimal required hole during scan and only evict those nodes that overlap. This enables us to reduce the number of nodes we need to evict to the bare minimum. Signed-off-by: Chris Wilson Reviewed-by: Joonas Lahtinen --- drivers/gpu/drm/drm_mm.c | 60 +++++++++++++++++++++++++++------ drivers/gpu/drm/etnaviv/etnaviv_mmu.c | 2 +- drivers/gpu/drm/i915/i915_gem_evict.c | 3 +- drivers/gpu/drm/selftests/test-drm_mm.c | 10 +++--- include/drm/drm_mm.h | 22 ++++++------ 5 files changed, 71 insertions(+), 26 deletions(-) diff --git a/drivers/gpu/drm/drm_mm.c b/drivers/gpu/drm/drm_mm.c index 956782e7b092..ff1e62c066e8 100644 --- a/drivers/gpu/drm/drm_mm.c +++ b/drivers/gpu/drm/drm_mm.c @@ -718,10 +718,10 @@ EXPORT_SYMBOL(drm_mm_replace_node); * @color: opaque tag value to use for the allocation * @start: start of the allowed range for the allocation * @end: end of the allowed range for the allocation + * @flags: flags to specify how the allocation will be performed afterwards * * This simply sets up the scanning routines with the parameters for the desired - * hole. Note that there's no need to specify allocation flags, since they only - * change the place a node is allocated from within a suitable hole. + * hole. * * Warning: * As long as the scan list is non-empty, no other operations than @@ -733,7 +733,8 @@ void drm_mm_scan_init_with_range(struct drm_mm_scan *scan, u64 alignment, unsigned long color, u64 start, - u64 end) + u64 end, + unsigned int flags) { DRM_MM_BUG_ON(start >= end); DRM_MM_BUG_ON(size == 0 || size > end - start); @@ -744,6 +745,7 @@ void drm_mm_scan_init_with_range(struct drm_mm_scan *scan, scan->color = color; scan->alignment = alignment; scan->size = size; + scan->flags = flags; DRM_MM_BUG_ON(end <= start); scan->range_start = start; @@ -778,7 +780,7 @@ bool drm_mm_scan_add_block(struct drm_mm_scan *scan, DRM_MM_BUG_ON(node->mm != mm); DRM_MM_BUG_ON(!node->allocated); DRM_MM_BUG_ON(node->scanned_block); - node->scanned_block = 1; + node->scanned_block = true; mm->scan_active++; hole = list_prev_entry(node, node_list); @@ -800,15 +802,53 @@ bool drm_mm_scan_add_block(struct drm_mm_scan *scan, adj_start = max(col_start, scan->range_start); adj_end = min(col_end, scan->range_end); + if (adj_end <= adj_start || adj_end - adj_start < scan->size) + return false; + + if (scan->flags == DRM_MM_CREATE_TOP) + adj_start = adj_end - scan->size; + + if (scan->alignment) { + u64 rem; + + div64_u64_rem(adj_start, scan->alignment, &rem); + if (rem) { + adj_start -= rem; + if (scan->flags != DRM_MM_CREATE_TOP) + adj_start += scan->alignment; + if (adj_start < max(col_start, scan->range_start) || + min(col_end, scan->range_end) - adj_start < scan->size) + return false; + + if (adj_end <= adj_start || + adj_end - adj_start < scan->size) + return false; + } + } - if (check_free_hole(adj_start, adj_end, - scan->size, scan->alignment)) { + if (mm->color_adjust) { + /* If allocations need adjusting due to neighbouring colours, + * we do not have enough information to decide if we need + * to evict nodes on either side of [adj_start, adj_end]. + * What almost works is + * hit_start = adj_start + (hole_start - col_start); + * hit_end = adj_start + scan->size + (hole_end - col_end); + * but because the decision is only made on the final hole, + * we may underestimate the required adjustments for an + * interior allocation. + */ scan->hit_start = hole_start; scan->hit_end = hole_end; - return true; + } else { + scan->hit_start = adj_start; + scan->hit_end = adj_start + scan->size; } - return false; + DRM_MM_BUG_ON(scan->hit_start >= scan->hit_end); + DRM_MM_BUG_ON(scan->hit_start < hole_start); + DRM_MM_BUG_ON(scan->hit_end > hole_end); + + return true; } EXPORT_SYMBOL(drm_mm_scan_add_block); @@ -836,7 +876,7 @@ bool drm_mm_scan_remove_block(struct drm_mm_scan *scan, DRM_MM_BUG_ON(node->mm != scan->mm); DRM_MM_BUG_ON(!node->scanned_block); - node->scanned_block = 0; + node->scanned_block = false; DRM_MM_BUG_ON(!node->mm->scan_active); node->mm->scan_active--; @@ -846,7 +886,7 @@ bool drm_mm_scan_remove_block(struct drm_mm_scan *scan, prev_node->hole_follows = node->scanned_preceeds_hole; list_add(&node->node_list, &prev_node->node_list); - return (drm_mm_hole_node_end(node) > scan->hit_start && + return (node->start + node->size > scan->hit_start && node->start < scan->hit_end); } EXPORT_SYMBOL(drm_mm_scan_remove_block); diff --git a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c index fe1e886dcabb..2dae3169ce48 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c @@ -135,7 +135,7 @@ static int etnaviv_iommu_find_iova(struct etnaviv_iommu *mmu, } /* Try to retire some entries */ - drm_mm_scan_init(&scan, &mmu->mm, size, 0, 0); + drm_mm_scan_init(&scan, &mmu->mm, size, 0, 0, 0); found = 0; INIT_LIST_HEAD(&list); diff --git a/drivers/gpu/drm/i915/i915_gem_evict.c b/drivers/gpu/drm/i915/i915_gem_evict.c index 77ded288534b..2741498cdf2b 100644 --- a/drivers/gpu/drm/i915/i915_gem_evict.c +++ b/drivers/gpu/drm/i915/i915_gem_evict.c @@ -128,7 +128,8 @@ i915_gem_evict_something(struct i915_address_space *vm, */ drm_mm_scan_init_with_range(&scan, &vm->mm, min_size, alignment, cache_level, - start, end); + start, end, + flags & PIN_HIGH ? DRM_MM_CREATE_TOP : 0); /* Retire before we search the active list. Although we have * reasonable accuracy in our retirement lists, we may have diff --git a/drivers/gpu/drm/selftests/test-drm_mm.c b/drivers/gpu/drm/selftests/test-drm_mm.c index c8cbfb88b803..a9fed0637297 100644 --- a/drivers/gpu/drm/selftests/test-drm_mm.c +++ b/drivers/gpu/drm/selftests/test-drm_mm.c @@ -1169,7 +1169,7 @@ static bool evict_nothing(struct drm_mm *mm, struct drm_mm_node *node; unsigned int n; - drm_mm_scan_init(&scan, mm, 1, 0, 0); + drm_mm_scan_init(&scan, mm, 1, 0, 0, 0); for (n = 0; n < total_size; n++) { e = &nodes[n]; list_add(&e->link, &evict_list); @@ -1216,7 +1216,7 @@ static bool evict_everything(struct drm_mm *mm, unsigned int n; int err; - drm_mm_scan_init(&scan, mm, total_size, 0, 0); + drm_mm_scan_init(&scan, mm, total_size, 0, 0, 0); for (n = 0; n < total_size; n++) { e = &nodes[n]; list_add(&e->link, &evict_list); @@ -1266,7 +1266,8 @@ static int evict_something(struct drm_mm *mm, drm_mm_scan_init_with_range(&scan, mm, size, alignment, 0, - range_start, range_end); + range_start, range_end, + mode->create_flags); if (!evict_nodes(&scan, nodes, order, count, &evict_list)) @@ -1850,7 +1851,8 @@ static int evict_color(struct drm_mm *mm, drm_mm_scan_init_with_range(&scan, mm, size, alignment, color, - range_start, range_end); + range_start, range_end, + mode->create_flags); if (!evict_nodes(&scan, nodes, order, count, &evict_list)) diff --git a/include/drm/drm_mm.h b/include/drm/drm_mm.h index 8539832fac0f..639d00eb8f28 100644 --- a/include/drm/drm_mm.h +++ b/include/drm/drm_mm.h @@ -120,6 +120,7 @@ struct drm_mm_scan { struct drm_mm_node *prev_scanned_node; unsigned long color; + unsigned int flags; }; /** @@ -376,11 +377,9 @@ __drm_mm_interval_first(const struct drm_mm *mm, u64 start, u64 last); void drm_mm_scan_init_with_range(struct drm_mm_scan *scan, struct drm_mm *mm, - u64 size, - u64 alignment, - unsigned long color, - u64 start, - u64 end); + u64 size, u64 alignment, unsigned long color, + u64 start, u64 end, + unsigned int flags); /** * drm_mm_scan_init - initialize lru scanning @@ -389,10 +388,10 @@ void drm_mm_scan_init_with_range(struct drm_mm_scan *scan, * @size: size of the allocation * @alignment: alignment of the allocation * @color: opaque tag value to use for the allocation + * @flags: flags to specify how the allocation will be performed afterwards * * This simply sets up the scanning routines with the parameters for the desired - * hole. Note that there's no need to specify allocation flags, since they only - * change the place a node is allocated from within a suitable hole. + * hole. * * Warning: * As long as the scan list is non-empty, no other operations than @@ -402,10 +401,13 @@ static inline void drm_mm_scan_init(struct drm_mm_scan *scan, struct drm_mm *mm, u64 size, u64 alignment, - unsigned long color) + unsigned long color, + unsigned int flags) { - drm_mm_scan_init_with_range(scan, mm, size, alignment, color, - 0, U64_MAX); + drm_mm_scan_init_with_range(scan, mm, + size, alignment, color, + 0, U64_MAX, + flags); } bool drm_mm_scan_add_block(struct drm_mm_scan *scan,