From patchwork Mon Mar 14 05:22:11 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Turquette X-Patchwork-Id: 8576431 Return-Path: X-Original-To: patchwork-linux-pm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id F15D9C0554 for ; Mon, 14 Mar 2016 05:27:05 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 0281D20429 for ; Mon, 14 Mar 2016 05:27:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E4C302043C for ; Mon, 14 Mar 2016 05:27:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933524AbcCNF0g (ORCPT ); Mon, 14 Mar 2016 01:26:36 -0400 Received: from mail-pf0-f179.google.com ([209.85.192.179]:36494 "EHLO mail-pf0-f179.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933483AbcCNF03 (ORCPT ); Mon, 14 Mar 2016 01:26:29 -0400 Received: by mail-pf0-f179.google.com with SMTP id u190so98874303pfb.3 for ; Sun, 13 Mar 2016 22:26:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=DQDXdRxBUm3p6eN5mSnIyicw8e44ODVMIcz33wa1TNw=; b=Ylq6SaItanQLzLb7B7aDGPbTshq1UM/jo47MTHlcfpyJp1f+10NDr2MoTdUjykG/NC jzXDTW0oQr0h8SQ+ugfEEmfQtsqSdr/yfjz0SyPfEIJ2nry238cfUJ9TGUyOMO9aAbd1 WksamIQrUbomp7LYjWZ14J4SGkLphUD8FQU5XNDL9lk5/nHirQz6p0G4p7kTlPjlBnCf 3GXIUbr/16qrk6l87bVcl/RF9vBNfcFEZmxNIj/9zaoIyOTk5hqSBW90Gr+UG2GT+k3I JwTvFR9TdYVos1qYoXVW3SVNfts81qnecpIu7NSRcd4PblrZ3Rf3Yvr6Gaprqj3QJp69 1GnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=DQDXdRxBUm3p6eN5mSnIyicw8e44ODVMIcz33wa1TNw=; b=BBALtTCtjiRy60ow4jiHZyRkw9nYhDMy25L8P0H2PnfCQxFCn79Ja0HdmpUnJaaoAz W9ev6Otv2Q0+Z8k9u+U5oo1By9kmJjzfzNHQCQ4BSmBrzJMXT8V9Hj1SURaxejRGz7Zn arvlmZN9Uo8iwvFAmwcZpkJnUvUB/xOQImwbBVJIa67VpoFIgjEypJ2xs1CN1jUKV7+Y 9iORi6195znUdzuUXAqRVPz9zJ/ZfP1cTiWtST6+1m09y0u6jCa1FZ3qQt3G8rAmK+Wi RZ3ddgdlgj7Q24aIWof65ewahMlt9eLXDiAmCxlNp88bSwWELXsTR+i0fO5oarw76DM1 30uQ== X-Gm-Message-State: AD7BkJJ/LbPYA3vGk22fI3753liq+cJpJu776gY9GMlVk3Y7F/a29bS7D9hbEBId/fQtmnvO X-Received: by 10.98.32.211 with SMTP id m80mr24886942pfj.3.1457933188236; Sun, 13 Mar 2016 22:26:28 -0700 (PDT) Received: from localhost (cpe-172-248-200-249.socal.res.rr.com. [172.248.200.249]) by smtp.gmail.com with ESMTPSA id ey7sm28622598pab.47.2016.03.13.22.26.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 13 Mar 2016 22:26:27 -0700 (PDT) From: Michael Turquette X-Google-Original-From: Michael Turquette To: peterz@infradead.org, rjw@rjwysocki.net Cc: linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Juri.Lelli@arm.com, steve.muckle@linaro.org, morten.rasmussen@arm.com, dietmar.eggemann@arm.com, vincent.guittot@linaro.org, Michael Turquette Subject: [PATCH 7/8] cpufreq: Frequency invariant scheduler load-tracking support Date: Sun, 13 Mar 2016 22:22:11 -0700 Message-Id: <1457932932-28444-8-git-send-email-mturquette+renesas@baylibre.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1457932932-28444-1-git-send-email-mturquette+renesas@baylibre.com> References: <1457932932-28444-1-git-send-email-mturquette+renesas@baylibre.com> Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Dietmar Eggemann Implements cpufreq_scale_freq_capacity() to provide the scheduler with a frequency scaling correction factor for more accurate load-tracking. The factor is: current_freq(cpu) << SCHED_CAPACITY_SHIFT / max_freq(cpu) In fact, freq_scale should be a struct cpufreq_policy data member. But this would require that the scheduler hot path (__update_load_avg()) would have to grab the cpufreq lock. This can be avoided by using per-cpu data initialized to SCHED_CAPACITY_SCALE for freq_scale. Signed-off-by: Dietmar Eggemann Signed-off-by: Michael Turquette --- I'm not as sure about patches 7 & 8, but I included them since I needed frequency invariance while testing. As mentioned by myself in 2014 and Rafael last month, the arch_scale_freq_capacity hook is awkward, because this behavior may vary within an architecture. I re-introduce Dietmar's generic cpufreq implementation of the frequency invariance hook in this patch, and change the preprocessor magic in sched.h to favor the cpufreq implementation over arch- or platform-specific ones in the next patch. If run-time selection of ops is needed them someone will need to write that code. I think that this negates the need for the arm arch hooks[0-2], and hopefully Morten and Dietmar can weigh in on this. [0] lkml.kernel.org/r/1436293469-25707-2-git-send-email-morten.rasmussen@arm.com [1] lkml.kernel.org/r/1436293469-25707-6-git-send-email-morten.rasmussen@arm.com [2] lkml.kernel.org/r/1436293469-25707-8-git-send-email-morten.rasmussen@arm.com drivers/cpufreq/cpufreq.c | 29 +++++++++++++++++++++++++++++ include/linux/cpufreq.h | 3 +++ 2 files changed, 32 insertions(+) diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c index b1ca9c4..e67584f 100644 --- a/drivers/cpufreq/cpufreq.c +++ b/drivers/cpufreq/cpufreq.c @@ -306,6 +306,31 @@ static void adjust_jiffies(unsigned long val, struct cpufreq_freqs *ci) #endif } +/********************************************************************* + * FREQUENCY INVARIANT CPU CAPACITY * + *********************************************************************/ + +static DEFINE_PER_CPU(unsigned long, freq_scale) = SCHED_CAPACITY_SCALE; + +static void +scale_freq_capacity(struct cpufreq_policy *policy, struct cpufreq_freqs *freqs) +{ + unsigned long cur = freqs ? freqs->new : policy->cur; + unsigned long scale = (cur << SCHED_CAPACITY_SHIFT) / policy->max; + int cpu; + + pr_debug("cpus %*pbl cur/cur max freq %lu/%u kHz freq scale %lu\n", + cpumask_pr_args(policy->cpus), cur, policy->max, scale); + + for_each_cpu(cpu, policy->cpus) + per_cpu(freq_scale, cpu) = scale; +} + +unsigned long cpufreq_scale_freq_capacity(struct sched_domain *sd, int cpu) +{ + return per_cpu(freq_scale, cpu); +} + static void __cpufreq_notify_transition(struct cpufreq_policy *policy, struct cpufreq_freqs *freqs, unsigned int state) { @@ -409,6 +434,8 @@ wait: spin_unlock(&policy->transition_lock); + scale_freq_capacity(policy, freqs); + cpufreq_notify_transition(policy, freqs, CPUFREQ_PRECHANGE); } EXPORT_SYMBOL_GPL(cpufreq_freq_transition_begin); @@ -2125,6 +2152,8 @@ static int cpufreq_set_policy(struct cpufreq_policy *policy, blocking_notifier_call_chain(&cpufreq_policy_notifier_list, CPUFREQ_NOTIFY, new_policy); + scale_freq_capacity(new_policy, NULL); + policy->min = new_policy->min; policy->max = new_policy->max; diff --git a/include/linux/cpufreq.h b/include/linux/cpufreq.h index 0e39499..72833be 100644 --- a/include/linux/cpufreq.h +++ b/include/linux/cpufreq.h @@ -583,4 +583,7 @@ unsigned int cpufreq_generic_get(unsigned int cpu); int cpufreq_generic_init(struct cpufreq_policy *policy, struct cpufreq_frequency_table *table, unsigned int transition_latency); + +struct sched_domain; +unsigned long cpufreq_scale_freq_capacity(struct sched_domain *sd, int cpu); #endif /* _LINUX_CPUFREQ_H */