From patchwork Thu Feb 17 07:50:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Kasireddy, Vivek" X-Patchwork-Id: 12749555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 55501C433EF for ; Thu, 17 Feb 2022 08:06:45 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2919710E9A7; Thu, 17 Feb 2022 08:06:43 +0000 (UTC) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by gabe.freedesktop.org (Postfix) with ESMTPS id 02DA610E9A4; Thu, 17 Feb 2022 08:06:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645085202; x=1676621202; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6qHJiwLEB5wZW6ryG81zNYT+YwHCSQt+VMxOomwt5wM=; b=e1b1j3IzmenzwqW2R3ezzFM+OLJY/n4PNaZqN/BkEEYM3U4cqIsz3njJ EqtDv0ALR5mg10/YjfSduGcNHclwsnWXQsZVUad/5vVY5aW4yF/9ONWA1 AsyrDUzcIO7NMBjc3XE1P/+Gzx1K6EN9uWX6JgZ9YW54DA9vgyy/R0k+G XHHdm2Y/u2BvBVCm4s7D3FthY04vOxkYekQKHi73oluX1pvt9zI7JaEGX RXEzqhqxL1SQYyRkV1vSpt918vHc4QrnMdjpNZv8CN7U31OaV9VAMi5zG SByO+4nrPXDsBTv7znRnMYkRPTz4F9jddaAX3DEKHHxGGUb0VGDSd1rYT w==; X-IronPort-AV: E=McAfee;i="6200,9189,10260"; a="314088975" X-IronPort-AV: E=Sophos;i="5.88,375,1635231600"; d="scan'208";a="314088975" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Feb 2022 00:06:41 -0800 X-IronPort-AV: E=Sophos;i="5.88,375,1635231600"; d="scan'208";a="501531643" Received: from vkasired-desk2.fm.intel.com ([10.105.128.127]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Feb 2022 00:06:41 -0800 From: Vivek Kasireddy To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Date: Wed, 16 Feb 2022 23:50:12 -0800 Message-Id: <20220217075014.922605-2-vivek.kasireddy@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220217075014.922605-1-vivek.kasireddy@intel.com> References: <20220217075014.922605-1-vivek.kasireddy@intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v2 1/3] drm/mm: Ensure that the entry is not NULL before extracting rb_node X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" While looking for next holes suitable for an allocation, although, it is highly unlikely, make sure that the DECLARE_NEXT_HOLE_ADDR macro is using a valid node before it extracts the rb_node from it. Cc: Tvrtko Ursulin Cc: Christian König Signed-off-by: Vivek Kasireddy --- drivers/gpu/drm/drm_mm.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/drm_mm.c b/drivers/gpu/drm/drm_mm.c index 8257f9d4f619..499d8874e4ed 100644 --- a/drivers/gpu/drm/drm_mm.c +++ b/drivers/gpu/drm/drm_mm.c @@ -389,11 +389,12 @@ first_hole(struct drm_mm *mm, #define DECLARE_NEXT_HOLE_ADDR(name, first, last) \ static struct drm_mm_node *name(struct drm_mm_node *entry, u64 size) \ { \ - struct rb_node *parent, *node = &entry->rb_hole_addr; \ + struct rb_node *parent, *node; \ \ - if (!entry || RB_EMPTY_NODE(node)) \ + if (!entry || RB_EMPTY_NODE(&entry->rb_hole_addr)) \ return NULL; \ \ + node = &entry->rb_hole_addr; \ if (usable_hole_addr(node->first, size)) { \ node = node->first; \ while (usable_hole_addr(node->last, size)) \ From patchwork Thu Feb 17 07:50:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Kasireddy, Vivek" X-Patchwork-Id: 12749556 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0F6EFC433F5 for ; Thu, 17 Feb 2022 08:06:57 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1BA3810E9AC; Thu, 17 Feb 2022 08:06:46 +0000 (UTC) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by gabe.freedesktop.org (Postfix) with ESMTPS id 23FBA10E9A5; Thu, 17 Feb 2022 08:06:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645085203; x=1676621203; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BGQFnVsX3vhwrw3F1FyL947zE4+fjV2pXPUAVwKjhXs=; b=iwe+kFfX/+Wd2P2Nr6f+pZdVTF8ACKRa7ZcCV1E91EtEzOuItUIDeuQW MNd62awUbbhTDMCKgPN3/3Yz5//ig3+4LL3d6F2FukkLro13U7AafSZzW 8PrHrgVJOs9kvmftGSSkXlChFMabUo+nJu1JiwUolSciZJrErlXRXjQ98 JP6+7ppVnTn4hO4UK8NZLaoAifInQ1/ZhSy/79RxVU+LUq32u8HfxXRlY b0pCzGjPOxbNI2xCX6eYpW1NnXGJ2xzhJvlfyp/gWSxBt273GG9cB9C7h RxmG36pqzmh/AJcNGewIVJvXaewlZwQ7Ux/sjg64QsA3trWAfJkY+hBIj g==; X-IronPort-AV: E=McAfee;i="6200,9189,10260"; a="314088978" X-IronPort-AV: E=Sophos;i="5.88,375,1635231600"; d="scan'208";a="314088978" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Feb 2022 00:06:42 -0800 X-IronPort-AV: E=Sophos;i="5.88,375,1635231600"; d="scan'208";a="501531670" Received: from vkasired-desk2.fm.intel.com ([10.105.128.127]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Feb 2022 00:06:41 -0800 From: Vivek Kasireddy To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Date: Wed, 16 Feb 2022 23:50:13 -0800 Message-Id: <20220217075014.922605-3-vivek.kasireddy@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220217075014.922605-1-vivek.kasireddy@intel.com> References: <20220217075014.922605-1-vivek.kasireddy@intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v2 2/3] drm/mm: Add an iterator to optimally walk over holes for an allocation (v4) X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" This iterator relies on drm_mm_first_hole() and drm_mm_next_hole() functions to identify suitable holes for an allocation of a given size by efficiently traversing the rbtree associated with the given allocator. It replaces the for loop in drm_mm_insert_node_in_range() and can also be used by drm drivers to quickly identify holes of a certain size within a given range. v2: (Tvrtko) - Prepend a double underscore for the newly exported first/next_hole - s/each_best_hole/each_suitable_hole/g - Mask out DRM_MM_INSERT_ONCE from the mode before calling first/next_hole and elsewhere. v3: (Tvrtko) - Reduce the number of hunks by retaining the "mode" variable name v4: - Typo: s/__drm_mm_next_hole(.., hole/__drm_mm_next_hole(.., pos Reviewed-by: Tvrtko Ursulin Acked-by: Christian König Suggested-by: Tvrtko Ursulin Signed-off-by: Vivek Kasireddy --- drivers/gpu/drm/drm_mm.c | 32 +++++++++++++++----------------- include/drm/drm_mm.h | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+), 17 deletions(-) diff --git a/drivers/gpu/drm/drm_mm.c b/drivers/gpu/drm/drm_mm.c index 499d8874e4ed..f5339610361c 100644 --- a/drivers/gpu/drm/drm_mm.c +++ b/drivers/gpu/drm/drm_mm.c @@ -352,10 +352,10 @@ static struct drm_mm_node *find_hole_addr(struct drm_mm *mm, u64 addr, u64 size) return node; } -static struct drm_mm_node * -first_hole(struct drm_mm *mm, - u64 start, u64 end, u64 size, - enum drm_mm_insert_mode mode) +struct drm_mm_node * +__drm_mm_first_hole(struct drm_mm *mm, + u64 start, u64 end, u64 size, + enum drm_mm_insert_mode mode) { switch (mode) { default: @@ -374,6 +374,7 @@ first_hole(struct drm_mm *mm, hole_stack); } } +EXPORT_SYMBOL(__drm_mm_first_hole); /** * DECLARE_NEXT_HOLE_ADDR - macro to declare next hole functions @@ -411,11 +412,11 @@ static struct drm_mm_node *name(struct drm_mm_node *entry, u64 size) \ DECLARE_NEXT_HOLE_ADDR(next_hole_high_addr, rb_left, rb_right) DECLARE_NEXT_HOLE_ADDR(next_hole_low_addr, rb_right, rb_left) -static struct drm_mm_node * -next_hole(struct drm_mm *mm, - struct drm_mm_node *node, - u64 size, - enum drm_mm_insert_mode mode) +struct drm_mm_node * +__drm_mm_next_hole(struct drm_mm *mm, + struct drm_mm_node *node, + u64 size, + enum drm_mm_insert_mode mode) { switch (mode) { default: @@ -433,6 +434,7 @@ next_hole(struct drm_mm *mm, return &node->hole_stack == &mm->hole_stack ? NULL : node; } } +EXPORT_SYMBOL(__drm_mm_next_hole); /** * drm_mm_reserve_node - insert an pre-initialized node @@ -517,11 +519,11 @@ int drm_mm_insert_node_in_range(struct drm_mm * const mm, u64 size, u64 alignment, unsigned long color, u64 range_start, u64 range_end, - enum drm_mm_insert_mode mode) + enum drm_mm_insert_mode caller_mode) { struct drm_mm_node *hole; u64 remainder_mask; - bool once; + enum drm_mm_insert_mode mode = caller_mode & ~DRM_MM_INSERT_ONCE; DRM_MM_BUG_ON(range_start > range_end); @@ -534,13 +536,9 @@ int drm_mm_insert_node_in_range(struct drm_mm * const mm, if (alignment <= 1) alignment = 0; - once = mode & DRM_MM_INSERT_ONCE; - mode &= ~DRM_MM_INSERT_ONCE; - remainder_mask = is_power_of_2(alignment) ? alignment - 1 : 0; - for (hole = first_hole(mm, range_start, range_end, size, mode); - hole; - hole = once ? NULL : next_hole(mm, hole, size, mode)) { + drm_mm_for_each_suitable_hole(hole, mm, range_start, range_end, + size, mode) { u64 hole_start = __drm_mm_hole_node_start(hole); u64 hole_end = hole_start + hole->hole_size; u64 adj_start, adj_end; diff --git a/include/drm/drm_mm.h b/include/drm/drm_mm.h index ac33ba1b18bc..dff6db627807 100644 --- a/include/drm/drm_mm.h +++ b/include/drm/drm_mm.h @@ -400,6 +400,42 @@ static inline u64 drm_mm_hole_node_end(const struct drm_mm_node *hole_node) 1 : 0; \ pos = list_next_entry(pos, hole_stack)) +struct drm_mm_node * +__drm_mm_first_hole(struct drm_mm *mm, + u64 start, u64 end, u64 size, + enum drm_mm_insert_mode mode); + +struct drm_mm_node * +__drm_mm_next_hole(struct drm_mm *mm, + struct drm_mm_node *node, + u64 size, + enum drm_mm_insert_mode mode); + +/** + * drm_mm_for_each_suitable_hole - iterator to optimally walk over all + * holes that can fit an allocation of the given @size. + * @pos: &drm_mm_node used internally to track progress + * @mm: &drm_mm allocator to walk + * @range_start: start of the allowed range for the allocation + * @range_end: end of the allowed range for the allocation + * @size: size of the allocation + * @mode: fine-tune the allocation search + * + * This iterator walks over all holes suitable for the allocation of given + * @size in a very efficient manner. It is implemented by calling + * drm_mm_first_hole() and drm_mm_next_hole() which identify the + * appropriate holes within the given range by efficiently traversing the + * rbtree associated with @mm. + */ +#define drm_mm_for_each_suitable_hole(pos, mm, range_start, range_end, \ + size, mode) \ + for (pos = __drm_mm_first_hole(mm, range_start, range_end, size, \ + mode & ~DRM_MM_INSERT_ONCE); \ + pos; \ + pos = mode & DRM_MM_INSERT_ONCE ? \ + NULL : __drm_mm_next_hole(mm, pos, size, \ + mode & ~DRM_MM_INSERT_ONCE)) + /* * Basic range manager support (drm_mm.c) */ From patchwork Thu Feb 17 07:50:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Kasireddy, Vivek" X-Patchwork-Id: 12749557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 22AE1C433FE for ; Thu, 17 Feb 2022 08:07:00 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E460D10E9B1; Thu, 17 Feb 2022 08:06:51 +0000 (UTC) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by gabe.freedesktop.org (Postfix) with ESMTPS id EB8B710E9AB; Thu, 17 Feb 2022 08:06:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645085205; x=1676621205; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UNfXIENrNM+jeccVfiXuPwI2uSkqWg5Q51q8p/xRtLo=; b=UmNNmwWKw/xWtTyMasU6dksxNaUfv/Xqs+eWuAZimG2ZyUXqU006KEEK 9gtHaIQkJYGiu6JhLk+cCFytIK6zJ1ADL2ZXUCVJE9bQUK3h59VY53BLM fC3Z32oW8IIZUW88keEXHksa9SsuC124HTWGD8RPz1D7s8ZMFBFube+dm lPmuUvYzXYIYyT8y1/pIr2NPZtQ2pjcIuLPvPU/uDwfB38kJEio/dZvm2 Q/6IhX0vj2TgggHkcbeR/mqc9pV2m0M6ZeEbtfn6aKK8th6r/eWWTsd+t hLsAxNLh6Jen53zwgnQ5SWM+ClwctNsgk7l5kKDj6dItrT1LnIpb22syl w==; X-IronPort-AV: E=McAfee;i="6200,9189,10260"; a="275408514" X-IronPort-AV: E=Sophos;i="5.88,375,1635231600"; d="scan'208";a="275408514" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Feb 2022 00:06:42 -0800 X-IronPort-AV: E=Sophos;i="5.88,375,1635231600"; d="scan'208";a="501531701" Received: from vkasired-desk2.fm.intel.com ([10.105.128.127]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Feb 2022 00:06:42 -0800 From: Vivek Kasireddy To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Date: Wed, 16 Feb 2022 23:50:14 -0800 Message-Id: <20220217075014.922605-4-vivek.kasireddy@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220217075014.922605-1-vivek.kasireddy@intel.com> References: <20220217075014.922605-1-vivek.kasireddy@intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v2 3/3] drm/i915/gem: Don't try to map and fence large scanout buffers (v8) X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" On platforms capable of allowing 8K (7680 x 4320) modes, pinning 2 or more framebuffers/scanout buffers results in only one that is mappable/ fenceable. Therefore, pageflipping between these 2 FBs where only one is mappable/fenceable creates latencies large enough to miss alternate vblanks thereby producing less optimal framerate. This mainly happens because when i915_gem_object_pin_to_display_plane() is called to pin one of the FB objs, the associated vma is identified as misplaced and therefore i915_vma_unbind() is called which unbinds and evicts it. This misplaced vma gets subseqently pinned only when i915_gem_object_ggtt_pin_ww() is called without PIN_MAPPABLE. This results in a latency of ~10ms and happens every other vblank/repaint cycle. Therefore, to fix this issue, we try to see if there is space to map at-least two objects of a given size and return early if there isn't. This would ensure that we do not try with PIN_MAPPABLE for any objects that are too big to map thereby preventing unncessary unbind. Testcase: Running Weston and weston-simple-egl on an Alderlake_S (ADLS) platform with a 8K@60 mode results in only ~40 FPS. Since upstream Weston submits a frame ~7ms before the next vblank, the latencies seen between atomic commit and flip event are 7, 24 (7 + 16.66), 7, 24..... suggesting that it misses the vblank every other frame. Here is the ftrace snippet that shows the source of the ~10ms latency: i915_gem_object_pin_to_display_plane() { 0.102 us | i915_gem_object_set_cache_level(); i915_gem_object_ggtt_pin_ww() { 0.390 us | i915_vma_instance(); 0.178 us | i915_vma_misplaced(); i915_vma_unbind() { __i915_active_wait() { 0.082 us | i915_active_acquire_if_busy(); 0.475 us | } intel_runtime_pm_get() { 0.087 us | intel_runtime_pm_acquire(); 0.259 us | } __i915_active_wait() { 0.085 us | i915_active_acquire_if_busy(); 0.240 us | } __i915_vma_evict() { ggtt_unbind_vma() { gen8_ggtt_clear_range() { 10507.255 us | } 10507.689 us | } 10508.516 us | } v2: Instead of using bigjoiner checks, determine whether a scanout buffer is too big by checking to see if it is possible to map two of them into the ggtt. v3 (Ville): - Count how many fb objects can be fit into the available holes instead of checking for a hole twice the object size. - Take alignment constraints into account. - Limit this large scanout buffer check to >= Gen 11 platforms. v4: - Remove existing heuristic that checks just for size. (Ville) - Return early if we find space to map at-least two objects. (Tvrtko) - Slightly update the commit message. v5: (Tvrtko) - Rename the function to indicate that the object may be too big to map into the aperture. - Account for guard pages while calculating the total size required for the object. - Do not subject all objects to the heuristic check and instead consider objects only of a certain size. - Do the hole walk using the rbtree. - Preserve the existing PIN_NONBLOCK logic. - Drop the PIN_MAPPABLE check while pinning the VMA. v6: (Tvrtko) - Return 0 on success and the specific error code on failure to preserve the existing behavior. v7: (Ville) - Drop the HAS_GMCH(i915), DISPLAY_VER(i915) < 11 and size < ggtt->mappable_end / 4 checks. - Drop the redundant check that is based on previous heuristic. v8: - Make sure that we are holding the mutex associated with ggtt vm as we traverse the hole nodes. Cc: Ville Syrjälä Cc: Maarten Lankhorst Cc: Tvrtko Ursulin Cc: Manasi Navare Reviewed-by: Tvrtko Ursulin Signed-off-by: Vivek Kasireddy --- drivers/gpu/drm/i915/i915_gem.c | 124 +++++++++++++++++++++++--------- 1 file changed, 90 insertions(+), 34 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 2e10187cd0a0..db00e71ce328 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -49,6 +49,7 @@ #include "gem/i915_gem_pm.h" #include "gem/i915_gem_region.h" #include "gem/i915_gem_userptr.h" +#include "gem/i915_gem_tiling.h" #include "gt/intel_engine_user.h" #include "gt/intel_gt.h" #include "gt/intel_gt_pm.h" @@ -879,6 +880,92 @@ static void discard_ggtt_vma(struct i915_vma *vma) spin_unlock(&obj->vma.lock); } +static int +i915_gem_object_fits_in_aperture(struct drm_i915_gem_object *obj, + u64 alignment, u64 flags) +{ + struct drm_i915_private *i915 = to_i915(obj->base.dev); + struct i915_ggtt *ggtt = to_gt(i915)->ggtt; + struct drm_mm_node *hole; + u64 hole_start, hole_end, start, end; + u64 fence_size, fence_alignment; + unsigned int count = 0; + + /* + * If the required space is larger than the available + * aperture, we will not able to find a slot for the + * object and unbinding the object now will be in + * vain. Worse, doing so may cause us to ping-pong + * the object in and out of the Global GTT and + * waste a lot of cycles under the mutex. + */ + if (obj->base.size > ggtt->mappable_end) + return -E2BIG; + + /* + * If NONBLOCK is set the caller is optimistically + * trying to cache the full object within the mappable + * aperture, and *must* have a fallback in place for + * situations where we cannot bind the object. We + * can be a little more lax here and use the fallback + * more often to avoid costly migrations of ourselves + * and other objects within the aperture. + */ + if (!(flags & PIN_NONBLOCK)) + return 0; + + /* + * Other objects such as batchbuffers are fairly small compared + * to FBs and are unlikely to exahust the aperture space. + * Therefore, return early if this obj is not an FB. + */ + if (!i915_gem_object_is_framebuffer(obj)) + return 0; + + fence_size = i915_gem_fence_size(i915, obj->base.size, + i915_gem_object_get_tiling(obj), + i915_gem_object_get_stride(obj)); + + if (i915_vm_has_cache_coloring(&ggtt->vm)) + fence_size += 2 * I915_GTT_PAGE_SIZE; + + fence_alignment = i915_gem_fence_alignment(i915, obj->base.size, + i915_gem_object_get_tiling(obj), + i915_gem_object_get_stride(obj)); + alignment = max_t(u64, alignment, fence_alignment); + + /* + * Assuming this object is a large scanout buffer, we try to find + * out if there is room to map at-least two of them. There could + * be space available to map one but to be consistent, we try to + * avoid mapping/fencing any of them. + */ + mutex_lock(&ggtt->vm.mutex); + drm_mm_for_each_suitable_hole(hole, &ggtt->vm.mm, 0, ggtt->mappable_end, + fence_size, DRM_MM_INSERT_LOW) { + hole_start = drm_mm_hole_node_start(hole); + hole_end = hole_start + hole->hole_size; + + do { + start = round_up(hole_start, alignment); + end = min_t(u64, hole_end, ggtt->mappable_end); + + if (range_overflows(start, fence_size, end)) + break; + + if (++count >= 2) { + mutex_unlock(&ggtt->vm.mutex); + return 0; + } + + hole_start = start + fence_size; + } while (1); + } + + mutex_unlock(&ggtt->vm.mutex); + return -ENOSPC; +} + struct i915_vma * i915_gem_object_ggtt_pin_ww(struct drm_i915_gem_object *obj, struct i915_gem_ww_ctx *ww, @@ -894,36 +981,9 @@ i915_gem_object_ggtt_pin_ww(struct drm_i915_gem_object *obj, if (flags & PIN_MAPPABLE && (!view || view->type == I915_GGTT_VIEW_NORMAL)) { - /* - * If the required space is larger than the available - * aperture, we will not able to find a slot for the - * object and unbinding the object now will be in - * vain. Worse, doing so may cause us to ping-pong - * the object in and out of the Global GTT and - * waste a lot of cycles under the mutex. - */ - if (obj->base.size > ggtt->mappable_end) - return ERR_PTR(-E2BIG); - - /* - * If NONBLOCK is set the caller is optimistically - * trying to cache the full object within the mappable - * aperture, and *must* have a fallback in place for - * situations where we cannot bind the object. We - * can be a little more lax here and use the fallback - * more often to avoid costly migrations of ourselves - * and other objects within the aperture. - * - * Half-the-aperture is used as a simple heuristic. - * More interesting would to do search for a free - * block prior to making the commitment to unbind. - * That caters for the self-harm case, and with a - * little more heuristics (e.g. NOFAULT, NOEVICT) - * we could try to minimise harm to others. - */ - if (flags & PIN_NONBLOCK && - obj->base.size > ggtt->mappable_end / 2) - return ERR_PTR(-ENOSPC); + ret = i915_gem_object_fits_in_aperture(obj, alignment, flags); + if (ret) + return ERR_PTR(ret); } new_vma: @@ -935,10 +995,6 @@ i915_gem_object_ggtt_pin_ww(struct drm_i915_gem_object *obj, if (flags & PIN_NONBLOCK) { if (i915_vma_is_pinned(vma) || i915_vma_is_active(vma)) return ERR_PTR(-ENOSPC); - - if (flags & PIN_MAPPABLE && - vma->fence_size > ggtt->mappable_end / 2) - return ERR_PTR(-ENOSPC); } if (i915_vma_is_pinned(vma) || i915_vma_is_active(vma)) {