From patchwork Fri Feb 10 13:06:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 13135773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 767C7C05027 for ; Fri, 10 Feb 2023 13:07:01 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E363210E135; Fri, 10 Feb 2023 13:07:00 +0000 (UTC) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by gabe.freedesktop.org (Postfix) with ESMTPS id 57F2010E1D1; Fri, 10 Feb 2023 13:06:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676034419; x=1707570419; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=J2sRuMIZR4XQkaZEw8inbQKQmbjNad/gTnJ7tpE5A5k=; b=TxPTfGsa+hzyEPcaw90+i0HDlu7PCluHASfFkE7T6+iKA9ULRvOa6AMR 3hQnlKeTqN+ZbQ2hSMm81Qqm+wfZAO9DxHNZt3PLlpwoL8cpzDaoLd4FI R/ghLC0WQvamVv7m3x3D5zufw1hAb/xj3y/LhEJoj0uB3dwjyfPXWrBwf 1OM/YLomZ8SypUzoehHoxGbf7cf8Dtp0AKCh3g0YiO/c/MugaU4JbSwqU 0dtyuAHJzWXL+G+Zasm448RrEUacoKutuwj4qx6IhEO4j8o0Mgo7M2/oR rxvTJbSpop2qUTi+PZp7Mo9h9JvxIVSkCJEO8STF+hKI49s73ajZfcnpo g==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="329045872" X-IronPort-AV: E=Sophos;i="5.97,287,1669104000"; d="scan'208";a="329045872" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 05:06:59 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="731711283" X-IronPort-AV: E=Sophos;i="5.97,287,1669104000"; d="scan'208";a="731711283" Received: from athornbe-mobl1.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.209.249]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 05:06:57 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Date: Fri, 10 Feb 2023 13:06:43 +0000 Message-Id: <20230210130647.580135-2-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210130647.580135-1-tvrtko.ursulin@linux.intel.com> References: <20230210130647.580135-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [RFC 1/5] dma-fence: Track explicit waiters X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Tvrtko Ursulin Track how many callers are explicity waiting on a fence to signal and allow querying that via new dma_fence_wait_count() API. This provides infrastructure on top of which generic "waitboost" concepts can be implemented by individual drivers. Wait-boosting is any reactive activity, such as raising the GPU clocks, which happens while there are active external waiters. Signed-off-by: Tvrtko Ursulin --- drivers/dma-buf/dma-fence.c | 102 ++++++++++++++++------ drivers/gpu/drm/i915/gt/intel_engine_pm.c | 1 - include/linux/dma-fence.h | 14 +++ 3 files changed, 88 insertions(+), 29 deletions(-) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index 0de0482cd36e..ed43290c0bdf 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -344,6 +344,24 @@ void __dma_fence_might_wait(void) } #endif +static void incr_wait_count(struct dma_fence *fence, struct dma_fence_cb *cb) +{ + lockdep_assert_held(fence->lock); + + __set_bit(DMA_FENCE_CB_FLAG_WAITCOUNT_BIT, &cb->flags); + fence->waitcount++; + WARN_ON_ONCE(!fence->waitcount); +} + +static void decr_wait_count(struct dma_fence *fence, struct dma_fence_cb *cb) +{ + lockdep_assert_held(fence->lock); + + if (__test_and_clear_bit(DMA_FENCE_CB_FLAG_WAITCOUNT_BIT, &cb->flags)) { + WARN_ON_ONCE(!fence->waitcount); + fence->waitcount--; + } +} /** * dma_fence_signal_timestamp_locked - signal completion of a fence @@ -384,6 +402,7 @@ int dma_fence_signal_timestamp_locked(struct dma_fence *fence, list_for_each_entry_safe(cur, tmp, &cb_list, node) { INIT_LIST_HEAD(&cur->node); + decr_wait_count(fence, cur); cur->func(fence, cur); } @@ -612,35 +631,15 @@ void dma_fence_enable_sw_signaling(struct dma_fence *fence) unsigned long flags; spin_lock_irqsave(fence->lock, flags); + fence->waitcount++; + WARN_ON_ONCE(!fence->waitcount); __dma_fence_enable_signaling(fence); spin_unlock_irqrestore(fence->lock, flags); } EXPORT_SYMBOL(dma_fence_enable_sw_signaling); -/** - * dma_fence_add_callback - add a callback to be called when the fence - * is signaled - * @fence: the fence to wait on - * @cb: the callback to register - * @func: the function to call - * - * Add a software callback to the fence. The caller should keep a reference to - * the fence. - * - * @cb will be initialized by dma_fence_add_callback(), no initialization - * by the caller is required. Any number of callbacks can be registered - * to a fence, but a callback can only be registered to one fence at a time. - * - * If fence is already signaled, this function will return -ENOENT (and - * *not* call the callback). - * - * Note that the callback can be called from an atomic context or irq context. - * - * Returns 0 in case of success, -ENOENT if the fence is already signaled - * and -EINVAL in case of error. - */ -int dma_fence_add_callback(struct dma_fence *fence, struct dma_fence_cb *cb, - dma_fence_func_t func) +static int add_callback(struct dma_fence *fence, struct dma_fence_cb *cb, + dma_fence_func_t func, bool wait) { unsigned long flags; int ret = 0; @@ -655,10 +654,15 @@ int dma_fence_add_callback(struct dma_fence *fence, struct dma_fence_cb *cb, spin_lock_irqsave(fence->lock, flags); + if (wait) + incr_wait_count(fence, cb); + if (__dma_fence_enable_signaling(fence)) { cb->func = func; list_add_tail(&cb->node, &fence->cb_list); } else { + if (test_bit(DMA_FENCE_CB_FLAG_WAITCOUNT_BIT, &cb->flags)) + decr_wait_count(fence, cb); INIT_LIST_HEAD(&cb->node); ret = -ENOENT; } @@ -667,8 +671,44 @@ int dma_fence_add_callback(struct dma_fence *fence, struct dma_fence_cb *cb, return ret; } + +/** + * dma_fence_add_callback - add a callback to be called when the fence + * is signaled + * @fence: the fence to wait on + * @cb: the callback to register + * @func: the function to call + * + * Add a software callback to the fence. The caller should keep a reference to + * the fence. + * + * @cb will be initialized by dma_fence_add_callback(), no initialization + * by the caller is required. Any number of callbacks can be registered + * to a fence, but a callback can only be registered to one fence at a time. + * + * If fence is already signaled, this function will return -ENOENT (and + * *not* call the callback). + * + * Note that the callback can be called from an atomic context or irq context. + * + * Returns 0 in case of success, -ENOENT if the fence is already signaled + * and -EINVAL in case of error. + */ +int dma_fence_add_callback(struct dma_fence *fence, struct dma_fence_cb *cb, + dma_fence_func_t func) +{ + return add_callback(fence, cb, func, false); +} EXPORT_SYMBOL(dma_fence_add_callback); +int dma_fence_add_wait_callback(struct dma_fence *fence, + struct dma_fence_cb *cb, + dma_fence_func_t func) +{ + return add_callback(fence, cb, func, true); +} +EXPORT_SYMBOL(dma_fence_add_wait_callback); + /** * dma_fence_get_status - returns the status upon completion * @fence: the dma_fence to query @@ -721,8 +761,10 @@ dma_fence_remove_callback(struct dma_fence *fence, struct dma_fence_cb *cb) spin_lock_irqsave(fence->lock, flags); ret = !list_empty(&cb->node); - if (ret) + if (ret) { + decr_wait_count(fence, cb); list_del_init(&cb->node); + } spin_unlock_irqrestore(fence->lock, flags); @@ -780,6 +822,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout) cb.base.func = dma_fence_default_wait_cb; cb.task = current; + incr_wait_count(fence, &cb.base); list_add(&cb.base.node, &fence->cb_list); while (!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags) && ret > 0) { @@ -796,8 +839,10 @@ dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout) ret = -ERESTARTSYS; } - if (!list_empty(&cb.base.node)) + if (!list_empty(&cb.base.node)) { + decr_wait_count(fence, &cb.base); list_del(&cb.base.node); + } __set_current_state(TASK_RUNNING); out: @@ -875,8 +920,8 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, struct dma_fence *fence = fences[i]; cb[i].task = current; - if (dma_fence_add_callback(fence, &cb[i].base, - dma_fence_default_wait_cb)) { + if (dma_fence_add_wait_callback(fence, &cb[i].base, + dma_fence_default_wait_cb)) { /* This fence is already signaled */ if (idx) *idx = i; @@ -957,6 +1002,7 @@ dma_fence_init(struct dma_fence *fence, const struct dma_fence_ops *ops, fence->context = context; fence->seqno = seqno; fence->flags = 0UL; + fence->waitcount = 0; fence->error = 0; trace_dma_fence_init(fence); diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c index e971b153fda9..2693a0151a6b 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c @@ -218,7 +218,6 @@ static bool switch_to_kernel_context(struct intel_engine_cs *engine) * until the background request retirement running every * second or two). */ - BUILD_BUG_ON(sizeof(rq->duration) > sizeof(rq->submitq)); dma_fence_add_callback(&rq->fence, &rq->duration.cb, duration); rq->duration.emitted = ktime_get(); } diff --git a/include/linux/dma-fence.h b/include/linux/dma-fence.h index 775cdc0b4f24..d0ed923e4545 100644 --- a/include/linux/dma-fence.h +++ b/include/linux/dma-fence.h @@ -92,6 +92,7 @@ struct dma_fence { u64 seqno; unsigned long flags; struct kref refcount; + unsigned int waitcount; int error; }; @@ -116,6 +117,11 @@ typedef void (*dma_fence_func_t)(struct dma_fence *fence, struct dma_fence_cb { struct list_head node; dma_fence_func_t func; + unsigned long flags; +}; + +enum dma_fence_cb_flag_bits { + DMA_FENCE_CB_FLAG_WAITCOUNT_BIT, }; /** @@ -377,6 +383,9 @@ signed long dma_fence_default_wait(struct dma_fence *fence, int dma_fence_add_callback(struct dma_fence *fence, struct dma_fence_cb *cb, dma_fence_func_t func); +int dma_fence_add_wait_callback(struct dma_fence *fence, + struct dma_fence_cb *cb, + dma_fence_func_t func); bool dma_fence_remove_callback(struct dma_fence *fence, struct dma_fence_cb *cb); void dma_fence_enable_sw_signaling(struct dma_fence *fence); @@ -528,6 +537,11 @@ static inline int dma_fence_get_status_locked(struct dma_fence *fence) int dma_fence_get_status(struct dma_fence *fence); +static inline unsigned int dma_fence_wait_count(struct dma_fence *fence) +{ + return fence->waitcount; +} + /** * dma_fence_set_error - flag an error condition on the fence * @fence: the dma_fence From patchwork Fri Feb 10 13:06:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 13135775 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F1044C05027 for ; Fri, 10 Feb 2023 13:07:14 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0023410E251; Fri, 10 Feb 2023 13:07:02 +0000 (UTC) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8272D10E135; Fri, 10 Feb 2023 13:07:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676034420; x=1707570420; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zD70VZdjgnOic3K1qhG8DBPtshCjC6Dxxa8vF3F1mg4=; b=GzX1EMq781ZPxETaUhYmWZWbyXelIp5tSpr17CK4q1R73jhB8ruyCnrd SmaggYFJnEWYIdBsxRE82rrJfIocexHBl8tgh7bTDYVdvwJq0Oqm50APX jSEQ6H5EItP7yO41bN4kKDsS0YqR7pcVM6dxr0l7W0MpGCPTTlR204ktj 2tNzvsIaTtjz1HjnDr0R1qbkdXbxV73Fio7+gAm6H8GhYacn8FKN5Zt7J Q/tEkwBLCtMB6r1FjkEeZHRwOXg2xLngVq4uZgHy/ajObdpNt7V1AcLaW u+BDOfsf8rScLVlYRyaxTAmoLO3dBfAAXtlsiBO/gLnRadSVGUCfNtpC+ g==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="329045878" X-IronPort-AV: E=Sophos;i="5.97,287,1669104000"; d="scan'208";a="329045878" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 05:07:00 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="731711286" X-IronPort-AV: E=Sophos;i="5.97,287,1669104000"; d="scan'208";a="731711286" Received: from athornbe-mobl1.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.209.249]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 05:06:59 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Date: Fri, 10 Feb 2023 13:06:44 +0000 Message-Id: <20230210130647.580135-3-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210130647.580135-1-tvrtko.ursulin@linux.intel.com> References: <20230210130647.580135-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [RFC 2/5] drm/syncobj: Mark syncobj waits as external waiters X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Tvrtko Ursulin Use the previously added dma-fence tracking of explicit waiters. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/drm_syncobj.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_syncobj.c b/drivers/gpu/drm/drm_syncobj.c index 0c2be8360525..776b90774a64 100644 --- a/drivers/gpu/drm/drm_syncobj.c +++ b/drivers/gpu/drm/drm_syncobj.c @@ -1065,9 +1065,9 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs, if ((flags & DRM_SYNCOBJ_WAIT_FLAGS_WAIT_AVAILABLE) || dma_fence_is_signaled(fence) || (!entries[i].fence_cb.func && - dma_fence_add_callback(fence, - &entries[i].fence_cb, - syncobj_wait_fence_func))) { + dma_fence_add_wait_callback(fence, + &entries[i].fence_cb, + syncobj_wait_fence_func))) { /* The fence has been signaled */ if (flags & DRM_SYNCOBJ_WAIT_FLAGS_WAIT_ALL) { signaled_count++; From patchwork Fri Feb 10 13:06:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 13135778 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5188CC6379F for ; Fri, 10 Feb 2023 13:07:22 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F3C1010ED04; Fri, 10 Feb 2023 13:07:07 +0000 (UTC) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by gabe.freedesktop.org (Postfix) with ESMTPS id C8ABD10E1D1; Fri, 10 Feb 2023 13:07:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676034421; x=1707570421; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Q9YTKJ3u8YMMvKwr50O9Cd1SSM69KoIu6lzAW0teqGY=; b=BJrUR0dwoXrBRKCBz0eVkPCL9hPw1x6E4CATHZ/sCKZ+XPnWB2LPyFVf OqQ0HoFuyln4/sci4p+QLomLnS9tqW6n3H+RlQTSL5qqU0PG1IaXH1gcM biRIOyc74UeHQwJehwziKa26x+k4fA2pJ4sEElovDQZfDWLnnaCuWLI4i dEUz/hmK8wCgc0uOJR3h4yPRHePdwOnU/4zVp2DvNjn0CAksweowC7lO6 oeP5+aAlrAhXRVaSM/ZaNulvvUN5I+YpWSlji2EMpEgG31ePEWOvpgfFs RWcVI9ZxyNQxFvrOKWFdKBbjabNRcqU1ZoKGDz/JafNfyUMcpgL0o1ifb w==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="329045881" X-IronPort-AV: E=Sophos;i="5.97,287,1669104000"; d="scan'208";a="329045881" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 05:07:01 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="731711289" X-IronPort-AV: E=Sophos;i="5.97,287,1669104000"; d="scan'208";a="731711289" Received: from athornbe-mobl1.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.209.249]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 05:07:00 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Date: Fri, 10 Feb 2023 13:06:45 +0000 Message-Id: <20230210130647.580135-4-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210130647.580135-1-tvrtko.ursulin@linux.intel.com> References: <20230210130647.580135-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [RFC 3/5] drm/i915: Waitboost external waits X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Tvrtko Ursulin Userspace waits coming via the drm_syncobj route have so far been bypassing the waitboost mechanism. Use the previously added dma-fence wait tracking API and apply the same waitboosting logic which applies to other entry points. This should fix the perfomance regressions experience by clvk and similar userspace which relies on drm_syncobj. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/i915/i915_request.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c index 7503dcb9043b..8989f62a7fba 100644 --- a/drivers/gpu/drm/i915/i915_request.c +++ b/drivers/gpu/drm/i915/i915_request.c @@ -94,7 +94,12 @@ static bool i915_fence_signaled(struct dma_fence *fence) static bool i915_fence_enable_signaling(struct dma_fence *fence) { - return i915_request_enable_breadcrumb(to_request(fence)); + struct i915_request *rq = to_request(fence); + + if (dma_fence_wait_count(&rq->fence) && !i915_request_started(rq)) + intel_rps_boost(rq); + + return i915_request_enable_breadcrumb(rq); } static signed long i915_fence_wait(struct dma_fence *fence, From patchwork Fri Feb 10 13:06:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 13135776 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AC8EEC05027 for ; Fri, 10 Feb 2023 13:07:20 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 952B510E27B; Fri, 10 Feb 2023 13:07:05 +0000 (UTC) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by gabe.freedesktop.org (Postfix) with ESMTPS id 18C9410E259; Fri, 10 Feb 2023 13:07:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676034423; x=1707570423; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fBvOEzsEUpuULwaHPJM994RuIgwkgHj0Lxh3VNlA7mU=; b=EJRByqzJbc62FlADDlhtuDWkIkOtxDW2swtSHoKDrvpODqSMFVBtVSkG jgJ89if/BrBDGQqQvWOhneCtkxmhGcl/0w543loyWY3diqTr8ZNzbh5Mz pPVlqFTr3z4CKTY3GRZldcvFLubI5Eaz9sluCWqLiSw0vVQxdQpSUC/AC iEZO8D1ORhVnfxBTVIyCGJZu0FbJ1+LKFDRZYKMqRsuWxtBUzQCx0cYqF CXspVC3xfn2I2sZXyqtop9rBVP5AR6WjLIFsulhIbZsjyvCyaF6zK2+Qq R94nYrm8mY5HO1gfbCQZx0Zl16us26cd/EusCJwEJO8ccpcLLCzNop18X w==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="329045888" X-IronPort-AV: E=Sophos;i="5.97,287,1669104000"; d="scan'208";a="329045888" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 05:07:02 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="731711305" X-IronPort-AV: E=Sophos;i="5.97,287,1669104000"; d="scan'208";a="731711305" Received: from athornbe-mobl1.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.209.249]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 05:07:01 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Date: Fri, 10 Feb 2023 13:06:46 +0000 Message-Id: <20230210130647.580135-5-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210130647.580135-1-tvrtko.ursulin@linux.intel.com> References: <20230210130647.580135-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [RFC 4/5] drm/i915: Mark waits as explicit X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Tvrtko Ursulin Use the previously added dma-fence API to mark the direct i915 waits as explicit. This has no significant effect apart from following the new pattern. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/i915/i915_request.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c index 8989f62a7fba..488b180f8821 100644 --- a/drivers/gpu/drm/i915/i915_request.c +++ b/drivers/gpu/drm/i915/i915_request.c @@ -2046,7 +2046,8 @@ long i915_request_wait_timeout(struct i915_request *rq, intel_rps_boost(rq); wait.tsk = current; - if (dma_fence_add_callback(&rq->fence, &wait.cb, request_wait_wake)) + if (dma_fence_add_wait_callback(&rq->fence, &wait.cb, + request_wait_wake)) goto out; /* From patchwork Fri Feb 10 13:06:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 13135777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7B269C636CD for ; Fri, 10 Feb 2023 13:07:21 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 919A310E82D; Fri, 10 Feb 2023 13:07:06 +0000 (UTC) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6535810E278; Fri, 10 Feb 2023 13:07:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676034424; x=1707570424; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/CcC0aOeeAtqrOsPao/E+8cmDCKIM6s5J/aBXV6fZRc=; b=VMCwUOqVkNZXPC2VY+/qzaK/tM6tZzHfyWmhnYnkPQMEGSXROZj8vpbk Cu3UrUNKUF/VPCdJf69ZrgfWbnpEn49QJn+v7DmJIoMK8E0kb14o4KF5K xKxo6rYtfBW5NDJIUThl3XBQ3kcJPnH5irVag7u7HRwrYob839e3vIoUl GUFv4i5w20SZgNr75Ava64TB2cp8GVjKbj6ToMKnOOBlLE6PONL/OAhgM VxjYdrTMk/tGnpjidt0Uj6jXoY+QRdQpv0xp2vFOl64IbF4eULmhvPy1u DUrEReNjsx4eWw7ZQGxsW7snP0oP2XNdNwdVcpE4CBCnrYuth5S1C9hLE g==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="329045895" X-IronPort-AV: E=Sophos;i="5.97,287,1669104000"; d="scan'208";a="329045895" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 05:07:04 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="731711317" X-IronPort-AV: E=Sophos;i="5.97,287,1669104000"; d="scan'208";a="731711317" Received: from athornbe-mobl1.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.209.249]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 05:07:03 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Date: Fri, 10 Feb 2023 13:06:47 +0000 Message-Id: <20230210130647.580135-6-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210130647.580135-1-tvrtko.ursulin@linux.intel.com> References: <20230210130647.580135-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [RFC 5/5] drm/i915: Wait boost requests waited upon by others X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Tvrtko Ursulin Use the newly added dma-fence API to apply waitboost not only requests which have been marked with I915_WAIT_PRIORITY by i915, but which may be waited upon by others (such as for instance buffer sharing in multi-GPU scenarios). Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/i915/i915_request.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c index 488b180f8821..e24fac5c1567 100644 --- a/drivers/gpu/drm/i915/i915_request.c +++ b/drivers/gpu/drm/i915/i915_request.c @@ -2042,7 +2042,8 @@ long i915_request_wait_timeout(struct i915_request *rq, * but at a cost of spending more power processing the workload * (bad for battery). */ - if (flags & I915_WAIT_PRIORITY && !i915_request_started(rq)) + if (((flags & I915_WAIT_PRIORITY) || dma_fence_wait_count(&rq->fence)) + && !i915_request_started(rq)) intel_rps_boost(rq); wait.tsk = current;