From patchwork Wed Nov 17 22:49:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vinay Belgaumkar X-Patchwork-Id: 12625725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79250C433FE for ; Wed, 17 Nov 2021 22:50:25 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1F51C6101C for ; Wed, 17 Nov 2021 22:50:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1F51C6101C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3D10E6E303; Wed, 17 Nov 2021 22:50:24 +0000 (UTC) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by gabe.freedesktop.org (Postfix) with ESMTPS id 896C06E283; Wed, 17 Nov 2021 22:50:22 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6200,9189,10171"; a="221285254" X-IronPort-AV: E=Sophos;i="5.87,243,1631602800"; d="scan'208";a="221285254" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Nov 2021 14:50:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,243,1631602800"; d="scan'208";a="536467626" Received: from vbelgaum-ubuntu.fm.intel.com ([10.1.27.27]) by orsmga001.jf.intel.com with ESMTP; 17 Nov 2021 14:50:21 -0800 From: Vinay Belgaumkar To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 1/3] drm/i915/gt: Spread virtual engines over idle engines Date: Wed, 17 Nov 2021 14:49:53 -0800 Message-Id: <20211117224955.28999-2-vinay.belgaumkar@intel.com> X-Mailer: git-send-email 2.34.0 In-Reply-To: <20211117224955.28999-1-vinay.belgaumkar@intel.com> References: <20211117224955.28999-1-vinay.belgaumkar@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vinay Belgaumkar , Tvrtko Ursulin , Chris Wilson Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Chris Wilson Everytime we come to the end of a virtual engine's context, re-randomise it's siblings[]. As we schedule the siblings' tasklets in the order they are in the array, earlier entries are executed first (when idle) and so will be preferred when scheduling the next virtual request. Currently, we only update the array when switching onto a new idle engine, so we prefer to stick on the last execute engine, keeping the work compact. However, it can be beneficial to spread the work out across idle engines, so choose another sibling as our preferred target at the end of the context's execution. Signed-off-by: Chris Wilson Cc: Vinay Belgaumkar Cc: Tvrtko Ursulin Reviewed-by: Tvrtko Ursulin Acked-by: Rodrigo Vivi --- .../drm/i915/gt/intel_execlists_submission.c | 80 ++++++++++++------- 1 file changed, 52 insertions(+), 28 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c index ca03880fa7e4..b95bbc8fb91a 100644 --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c @@ -539,6 +539,41 @@ static void execlists_schedule_in(struct i915_request *rq, int idx) GEM_BUG_ON(intel_context_inflight(ce) != rq->engine); } +static void virtual_xfer_context(struct virtual_engine *ve, + struct intel_engine_cs *engine) +{ + unsigned int n; + + if (likely(engine == ve->siblings[0])) + return; + + if (!intel_engine_has_relative_mmio(engine)) + lrc_update_offsets(&ve->context, engine); + + /* + * Move the bound engine to the top of the list for + * future execution. We then kick this tasklet first + * before checking others, so that we preferentially + * reuse this set of bound registers. + */ + for (n = 1; n < ve->num_siblings; n++) { + if (ve->siblings[n] == engine) { + swap(ve->siblings[n], ve->siblings[0]); + break; + } + } +} + +static int ve_random_sibling(struct virtual_engine *ve) +{ + return prandom_u32_max(ve->num_siblings); +} + +static int ve_random_other_sibling(struct virtual_engine *ve) +{ + return 1 + prandom_u32_max(ve->num_siblings - 1); +} + static void resubmit_virtual_request(struct i915_request *rq, struct virtual_engine *ve) { @@ -578,8 +613,23 @@ static void kick_siblings(struct i915_request *rq, struct intel_context *ce) rq->execution_mask != engine->mask) resubmit_virtual_request(rq, ve); - if (READ_ONCE(ve->request)) + /* + * Reschedule with a new "preferred" sibling. + * + * The tasklets are executed in the order of ve->siblings[], so + * siblings[0] receives preferrential treatment of greedily checking + * for execution of the virtual engine. At this point, the virtual + * engine is no longer in the current GPU cache due to idleness or + * contention, so it can be executed on any without penalty. We + * re-randomise at this point in order to spread light loads across + * the system, heavy overlapping loads will continue to be greedily + * executed by the first available engine. + */ + if (READ_ONCE(ve->request)) { + virtual_xfer_context(ve, + ve->siblings[ve_random_other_sibling(ve)]); tasklet_hi_schedule(&ve->base.sched_engine->tasklet); + } } static void __execlists_schedule_out(struct i915_request * const rq, @@ -1030,32 +1080,6 @@ first_virtual_engine(struct intel_engine_cs *engine) return NULL; } -static void virtual_xfer_context(struct virtual_engine *ve, - struct intel_engine_cs *engine) -{ - unsigned int n; - - if (likely(engine == ve->siblings[0])) - return; - - GEM_BUG_ON(READ_ONCE(ve->context.inflight)); - if (!intel_engine_has_relative_mmio(engine)) - lrc_update_offsets(&ve->context, engine); - - /* - * Move the bound engine to the top of the list for - * future execution. We then kick this tasklet first - * before checking others, so that we preferentially - * reuse this set of bound registers. - */ - for (n = 1; n < ve->num_siblings; n++) { - if (ve->siblings[n] == engine) { - swap(ve->siblings[n], ve->siblings[0]); - break; - } - } -} - static void defer_request(struct i915_request *rq, struct list_head * const pl) { LIST_HEAD(list); @@ -3590,7 +3614,7 @@ static void virtual_engine_initial_hint(struct virtual_engine *ve) * NB This does not force us to execute on this engine, it will just * typically be the first we inspect for submission. */ - swp = prandom_u32_max(ve->num_siblings); + swp = ve_random_sibling(ve); if (swp) swap(ve->siblings[swp], ve->siblings[0]); } From patchwork Wed Nov 17 22:49:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vinay Belgaumkar X-Patchwork-Id: 12625727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C77BC433EF for ; Wed, 17 Nov 2021 22:50:30 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 59FE3613D0 for ; Wed, 17 Nov 2021 22:50:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 59FE3613D0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0BC196E3C6; Wed, 17 Nov 2021 22:50:28 +0000 (UTC) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6CFA46E32A; Wed, 17 Nov 2021 22:50:26 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6200,9189,10171"; a="221285265" X-IronPort-AV: E=Sophos;i="5.87,243,1631602800"; d="scan'208";a="221285265" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Nov 2021 14:50:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,243,1631602800"; d="scan'208";a="536467640" Received: from vbelgaum-ubuntu.fm.intel.com ([10.1.27.27]) by orsmga001.jf.intel.com with ESMTP; 17 Nov 2021 14:50:25 -0800 From: Vinay Belgaumkar To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 2/3] drm/i915/gt: Compare average group occupancy for RPS evaluation Date: Wed, 17 Nov 2021 14:49:54 -0800 Message-Id: <20211117224955.28999-3-vinay.belgaumkar@intel.com> X-Mailer: git-send-email 2.34.0 In-Reply-To: <20211117224955.28999-1-vinay.belgaumkar@intel.com> References: <20211117224955.28999-1-vinay.belgaumkar@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vinay Belgaumkar , Chris Wilson , Tvrtko Ursulin Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Chris Wilson Currently, we inspect each engine individually and measure the occupancy of that engine over the last evaluation interval. If that exceeds our busyness thresholds, we decide to increase the GPU frequency. However, under a load balancer, we should consider the occupancy of entire engine groups, as work may be spread out across the group. In doing so, we prefer wide over fast, power consumption is approximately proportional to the square of the frequency. However, since the load balancer is greedy, the first idle engine gets all the work, and preferrentially reuses the last active engine, under light loads all work is assigned to one engine, and so that engine appears very busy. But if the work happened to overlap slightly, the workload would spread across multiple engines, reducing each individual engine's runtime, and so reducing the rps contribution, keeping the frequency low. Instead, when considering the contribution, consider the contribution over the entire engine group (capacity). Signed-off-by: Chris Wilson Cc: Vinay Belgaumkar Cc: Tvrtko Ursulin Reviewed-by: Vinay Belgaumkar --- drivers/gpu/drm/i915/gt/intel_rps.c | 48 ++++++++++++++++++++--------- 1 file changed, 34 insertions(+), 14 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_rps.c b/drivers/gpu/drm/i915/gt/intel_rps.c index 07ff7ba7b2b7..3675ac93ded0 100644 --- a/drivers/gpu/drm/i915/gt/intel_rps.c +++ b/drivers/gpu/drm/i915/gt/intel_rps.c @@ -7,6 +7,7 @@ #include "i915_drv.h" #include "intel_breadcrumbs.h" +#include "intel_engine_pm.h" #include "intel_gt.h" #include "intel_gt_clock_utils.h" #include "intel_gt_irq.h" @@ -65,26 +66,45 @@ static void set(struct intel_uncore *uncore, i915_reg_t reg, u32 val) static void rps_timer(struct timer_list *t) { struct intel_rps *rps = from_timer(rps, t, timer); - struct intel_engine_cs *engine; - ktime_t dt, last, timestamp; - enum intel_engine_id id; + struct intel_gt *gt = rps_to_gt(rps); + ktime_t dt, last, timestamp = 0; s64 max_busy[3] = {}; + int i, j; - timestamp = 0; - for_each_engine(engine, rps_to_gt(rps), id) { - s64 busy; - int i; + /* Compare average occupancy over each engine group */ + for (i = 0; i < ARRAY_SIZE(gt->engine_class); i++) { + s64 busy = 0; + int count = 0; + + for (j = 0; j < ARRAY_SIZE(gt->engine_class[i]); j++) { + struct intel_engine_cs *engine; - dt = intel_engine_get_busy_time(engine, ×tamp); - last = engine->stats.rps; - engine->stats.rps = dt; + engine = gt->engine_class[i][j]; + if (!engine) + continue; - busy = ktime_to_ns(ktime_sub(dt, last)); - for (i = 0; i < ARRAY_SIZE(max_busy); i++) { - if (busy > max_busy[i]) - swap(busy, max_busy[i]); + dt = intel_engine_get_busy_time(engine, ×tamp); + last = engine->stats.rps; + engine->stats.rps = dt; + + if (!intel_engine_pm_is_awake(engine)) + continue; + + busy += ktime_to_ns(ktime_sub(dt, last)); + count++; + } + + if (count > 1) + busy = div_u64(busy, count); + if (busy <= max_busy[ARRAY_SIZE(max_busy) - 1]) + continue; + + for (j = 0; j < ARRAY_SIZE(max_busy); j++) { + if (busy > max_busy[j]) + swap(busy, max_busy[j]); } } + last = rps->pm_timestamp; rps->pm_timestamp = timestamp; From patchwork Wed Nov 17 22:49:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vinay Belgaumkar X-Patchwork-Id: 12625729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39ED5C433F5 for ; Wed, 17 Nov 2021 22:50:33 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 072B6613D0 for ; Wed, 17 Nov 2021 22:50:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 072B6613D0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E9F716E402; Wed, 17 Nov 2021 22:50:30 +0000 (UTC) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by gabe.freedesktop.org (Postfix) with ESMTPS id 64FC56E402; Wed, 17 Nov 2021 22:50:28 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6200,9189,10171"; a="221285274" X-IronPort-AV: E=Sophos;i="5.87,243,1631602800"; d="scan'208";a="221285274" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Nov 2021 14:50:28 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,243,1631602800"; d="scan'208";a="536467646" Received: from vbelgaum-ubuntu.fm.intel.com ([10.1.27.27]) by orsmga001.jf.intel.com with ESMTP; 17 Nov 2021 14:50:27 -0800 From: Vinay Belgaumkar To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 3/3] drm/i915/gt: Improve "race-to-idle" at low frequencies Date: Wed, 17 Nov 2021 14:49:55 -0800 Message-Id: <20211117224955.28999-4-vinay.belgaumkar@intel.com> X-Mailer: git-send-email 2.34.0 In-Reply-To: <20211117224955.28999-1-vinay.belgaumkar@intel.com> References: <20211117224955.28999-1-vinay.belgaumkar@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vinay Belgaumkar , Tvrtko Ursulin , Chris Wilson Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Chris Wilson While the power consumption is proportional to the frequency, there is also a static draw for active gates. The longer we are able to powergate (rc6), the lower the static draw. Thus there is a sweetspot in the frequency/power curve where we run at higher frequency in order to sleep longer, aka race-to-idle. This is more evident at lower frequencies, so let's look to bump the frequency if we think we will benefit by sleeping longer at the higher frequency and so conserving power. Signed-off-by: Chris Wilson Cc: Vinay Belgaumkar Cc: Tvrtko Ursulin Reviewed-by: Vinay Belgaumkar --- drivers/gpu/drm/i915/gt/intel_rps.c | 31 ++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_rps.c b/drivers/gpu/drm/i915/gt/intel_rps.c index 3675ac93ded0..6af3231982af 100644 --- a/drivers/gpu/drm/i915/gt/intel_rps.c +++ b/drivers/gpu/drm/i915/gt/intel_rps.c @@ -63,6 +63,22 @@ static void set(struct intel_uncore *uncore, i915_reg_t reg, u32 val) intel_uncore_write_fw(uncore, reg, val); } +static bool race_to_idle(struct intel_rps *rps, u64 busy, u64 dt) +{ + unsigned int this = rps->cur_freq; + unsigned int next = rps->cur_freq + 1; + u64 next_dt = next * max(busy, dt); + + /* + * Compare estimated time spent in rc6 at the next power bin. If + * we expect to sleep longer than the estimated increased power + * cost of running at a higher frequency, it will be reduced power + * consumption overall. + */ + return (((next_dt - this * busy) >> 10) * this * this > + ((next_dt - next * busy) >> 10) * next * next); +} + static void rps_timer(struct timer_list *t) { struct intel_rps *rps = from_timer(rps, t, timer); @@ -133,7 +149,7 @@ static void rps_timer(struct timer_list *t) if (!max_busy[i]) break; - busy += div_u64(max_busy[i], 1 << i); + busy += max_busy[i] >> i; } GT_TRACE(rps_to_gt(rps), "busy:%lld [%d%%], max:[%lld, %lld, %lld], interval:%d\n", @@ -141,13 +157,18 @@ static void rps_timer(struct timer_list *t) max_busy[0], max_busy[1], max_busy[2], rps->pm_interval); - if (100 * busy > rps->power.up_threshold * dt && - rps->cur_freq < rps->max_freq_softlimit) { + if (rps->cur_freq < rps->max_freq_softlimit && + race_to_idle(rps, max_busy[0], dt)) { + rps->pm_iir |= GEN6_PM_RP_UP_THRESHOLD; + rps->pm_interval = 1; + schedule_work(&rps->work); + } else if (rps->cur_freq < rps->max_freq_softlimit && + 100 * busy > rps->power.up_threshold * dt) { rps->pm_iir |= GEN6_PM_RP_UP_THRESHOLD; rps->pm_interval = 1; schedule_work(&rps->work); - } else if (100 * busy < rps->power.down_threshold * dt && - rps->cur_freq > rps->min_freq_softlimit) { + } else if (rps->cur_freq > rps->min_freq_softlimit && + 100 * busy < rps->power.down_threshold * dt) { rps->pm_iir |= GEN6_PM_RP_DOWN_THRESHOLD; rps->pm_interval = 1; schedule_work(&rps->work);