From patchwork Tue Jul 7 18:23:45 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Morten Rasmussen X-Patchwork-Id: 6738581 Return-Path: X-Original-To: patchwork-linux-pm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 89A23C05AC for ; Tue, 7 Jul 2015 19:01:46 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 9D90A205BE for ; Tue, 7 Jul 2015 19:01:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8B6C820742 for ; Tue, 7 Jul 2015 19:01:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932786AbbGGSWf (ORCPT ); Tue, 7 Jul 2015 14:22:35 -0400 Received: from foss.arm.com ([217.140.101.70]:37320 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932669AbbGGSWF (ORCPT ); Tue, 7 Jul 2015 14:22:05 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EB562390; Tue, 7 Jul 2015 11:22:31 -0700 (PDT) Received: from e105550-lin.cambridge.arm.com (e105550-lin.cambridge.arm.com [10.2.131.193]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B493F3F23A; Tue, 7 Jul 2015 11:22:02 -0700 (PDT) From: Morten Rasmussen To: peterz@infradead.org, mingo@redhat.com Cc: vincent.guittot@linaro.org, daniel.lezcano@linaro.org, Dietmar Eggemann , yuyang.du@intel.com, mturquette@baylibre.com, rjw@rjwysocki.net, Juri Lelli , sgurrappadi@nvidia.com, pang.xunlei@zte.com.cn, linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Dietmar Eggemann Subject: [RFCv5 PATCH 02/46] sched: Make load tracking frequency scale-invariant Date: Tue, 7 Jul 2015 19:23:45 +0100 Message-Id: <1436293469-25707-3-git-send-email-morten.rasmussen@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1436293469-25707-1-git-send-email-morten.rasmussen@arm.com> References: <1436293469-25707-1-git-send-email-morten.rasmussen@arm.com> Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Spam-Status: No, score=-7.7 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Dietmar Eggemann Apply frequency scale-invariance correction factor to load tracking. Each segment of the sched_avg::runnable_avg_sum geometric series is now scaled by the current frequency so the sched_avg::load_avg_contrib of each entity will be invariant with frequency scaling. As a result, cfs_rq::runnable_load_avg which is the sum of sched_avg::load_avg_contrib, becomes invariant too. So the load level that is returned by weighted_cpuload, stays relative to the max frequency of the cpu. Then, we want the keep the load tracking values in a 32bits type, which implies that the max value of sched_avg::{runnable|running}_avg_sum must be lower than 2^32/88761=48388 (88761 is the max weight of a task). As LOAD_AVG_MAX = 47742, arch_scale_freq_capacity must return a value less than (48388/47742) << SCHED_CAPACITY_SHIFT = 1037 (SCHED_SCALE_CAPACITY = 1024). So we define the range to [0..SCHED_SCALE_CAPACITY] in order to avoid overflow. Cc: Ingo Molnar Cc: Peter Zijlstra Signed-off-by: Dietmar Eggemann Acked-by: Vincent Guittot --- kernel/sched/fair.c | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 4b6e5f6..376ec75 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -2551,9 +2551,9 @@ static __always_inline int __update_entity_runnable_avg(u64 now, int cpu, int runnable, int running) { - u64 delta, periods; - u32 runnable_contrib; - int delta_w, decayed = 0; + u64 delta, scaled_delta, periods; + u32 runnable_contrib, scaled_runnable_contrib; + int delta_w, scaled_delta_w, decayed = 0; unsigned long scale_freq = arch_scale_freq_capacity(NULL, cpu); delta = now - sa->last_runnable_update; @@ -2587,11 +2587,12 @@ static __always_inline int __update_entity_runnable_avg(u64 now, int cpu, * period and accrue it. */ delta_w = 1024 - delta_w; + scaled_delta_w = (delta_w * scale_freq) >> SCHED_CAPACITY_SHIFT; + if (runnable) - sa->runnable_avg_sum += delta_w; + sa->runnable_avg_sum += scaled_delta_w; if (running) - sa->running_avg_sum += delta_w * scale_freq - >> SCHED_CAPACITY_SHIFT; + sa->running_avg_sum += scaled_delta_w; sa->avg_period += delta_w; delta -= delta_w; @@ -2609,20 +2610,23 @@ static __always_inline int __update_entity_runnable_avg(u64 now, int cpu, /* Efficiently calculate \sum (1..n_period) 1024*y^i */ runnable_contrib = __compute_runnable_contrib(periods); + scaled_runnable_contrib = (runnable_contrib * scale_freq) + >> SCHED_CAPACITY_SHIFT; + if (runnable) - sa->runnable_avg_sum += runnable_contrib; + sa->runnable_avg_sum += scaled_runnable_contrib; if (running) - sa->running_avg_sum += runnable_contrib * scale_freq - >> SCHED_CAPACITY_SHIFT; + sa->running_avg_sum += scaled_runnable_contrib; sa->avg_period += runnable_contrib; } /* Remainder of delta accrued against u_0` */ + scaled_delta = (delta * scale_freq) >> SCHED_CAPACITY_SHIFT; + if (runnable) - sa->runnable_avg_sum += delta; + sa->runnable_avg_sum += scaled_delta; if (running) - sa->running_avg_sum += delta * scale_freq - >> SCHED_CAPACITY_SHIFT; + sa->running_avg_sum += scaled_delta; sa->avg_period += delta; return decayed;