From patchwork Wed May 8 17:42:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Douglas RAILLARD X-Patchwork-Id: 10936321 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B4F7C14DB for ; Wed, 8 May 2019 17:43:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A4A15285E8 for ; Wed, 8 May 2019 17:43:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 988D628610; Wed, 8 May 2019 17:43:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3BF6A285F9 for ; Wed, 8 May 2019 17:43:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729052AbfEHRnL (ORCPT ); Wed, 8 May 2019 13:43:11 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:41792 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725910AbfEHRnK (ORCPT ); Wed, 8 May 2019 13:43:10 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 16EC8A78; Wed, 8 May 2019 10:43:10 -0700 (PDT) Received: from e107049-lin.arm.com (e107049-lin.cambridge.arm.com [10.1.195.43]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 909CD3F575; Wed, 8 May 2019 10:43:08 -0700 (PDT) From: douglas.raillard@arm.com To: linux-kernel@vger.kernel.org Cc: linux-pm@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, quentin.perret@arm.com, douglas.raillard@arm.com, patrick.bellasi@arm.com, dietmar.eggemann@arm.com Subject: [RFC PATCH 1/7] PM: Introduce em_pd_get_higher_freq() Date: Wed, 8 May 2019 18:42:55 +0100 Message-Id: <20190508174301.4828-2-douglas.raillard@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190508174301.4828-1-douglas.raillard@arm.com> References: <20190508174301.4828-1-douglas.raillard@arm.com> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Douglas RAILLARD em_pd_get_higher_freq() returns a frequency greater or equal to the provided one while taking into account a given cost margin. It also skips inefficient OPPs that have a higher cost than another one with a higher frequency. Signed-off-by: Douglas RAILLARD --- include/linux/energy_model.h | 48 ++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/include/linux/energy_model.h b/include/linux/energy_model.h index aa027f7bcb3e..797b4e0f758c 100644 --- a/include/linux/energy_model.h +++ b/include/linux/energy_model.h @@ -159,6 +159,48 @@ static inline int em_pd_nr_cap_states(struct em_perf_domain *pd) return pd->nr_cap_states; } +/** + * em_pd_get_higher_freq() - Get the highest frequency that does not exceed the + * given cost margin compared to min_freq + * @pd : performance domain for which this must be done + * @min_freq : minimum frequency to return + * @cost_margin : allowed margin compared to min_freq, as a per-1024 value. + * + * Return: the chosen frequency, guaranteed to be at least as high as min_freq. + */ +static inline unsigned long em_pd_get_higher_freq(struct em_perf_domain *pd, + unsigned long min_freq, unsigned long cost_margin) +{ + unsigned long max_cost = 0; + struct em_cap_state *cs; + int i; + + if (!pd) + return min_freq; + + /* Compute the maximum allowed cost */ + for (i = 0; i < pd->nr_cap_states; i++) { + cs = &pd->table[i]; + if (cs->frequency >= min_freq) { + max_cost = cs->cost + (cs->cost * cost_margin) / 1024; + break; + } + } + + /* Find the highest frequency that will not exceed the cost margin */ + for (i = pd->nr_cap_states-1; i >= 0; i--) { + cs = &pd->table[i]; + if (cs->cost <= max_cost) + return cs->frequency; + } + + /* + * We should normally never reach here, unless min_freq was higher than + * the highest available frequency, which is not expected to happen. + */ + return min_freq; +} + #else struct em_perf_domain {}; struct em_data_callback {}; @@ -182,6 +224,12 @@ static inline int em_pd_nr_cap_states(struct em_perf_domain *pd) { return 0; } + +static inline unsigned long em_pd_get_higher_freq(struct em_perf_domain *pd, + unsigned long min_freq, unsigned long cost_margin) +{ + return min_freq; +} #endif #endif From patchwork Wed May 8 17:42:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Douglas RAILLARD X-Patchwork-Id: 10936319 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9273E14B6 for ; Wed, 8 May 2019 17:43:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8247D285E8 for ; Wed, 8 May 2019 17:43:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 76FA72861C; Wed, 8 May 2019 17:43:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1B9B8285E8 for ; Wed, 8 May 2019 17:43:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727819AbfEHRnl (ORCPT ); Wed, 8 May 2019 13:43:41 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:41800 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729059AbfEHRnM (ORCPT ); Wed, 8 May 2019 13:43:12 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CE56415AB; Wed, 8 May 2019 10:43:11 -0700 (PDT) Received: from e107049-lin.arm.com (e107049-lin.cambridge.arm.com [10.1.195.43]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 536C23F575; Wed, 8 May 2019 10:43:10 -0700 (PDT) From: douglas.raillard@arm.com To: linux-kernel@vger.kernel.org Cc: linux-pm@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, quentin.perret@arm.com, douglas.raillard@arm.com, patrick.bellasi@arm.com, dietmar.eggemann@arm.com Subject: [RFC PATCH 2/7] sched/cpufreq: Attach perf domain to sugov policy Date: Wed, 8 May 2019 18:42:56 +0100 Message-Id: <20190508174301.4828-3-douglas.raillard@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190508174301.4828-1-douglas.raillard@arm.com> References: <20190508174301.4828-1-douglas.raillard@arm.com> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Douglas RAILLARD Attach an Energy Model perf_domain to each sugov_policy to prepare the ground for energy-aware schedutil. Signed-off-by: Douglas RAILLARD --- kernel/sched/cpufreq_schedutil.c | 39 ++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index d0f4d2111bfe..82dbacf83649 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -39,6 +39,10 @@ struct sugov_policy { bool work_in_progress; bool need_freq_update; + +#ifdef CONFIG_ENERGY_MODEL + struct em_perf_domain *pd; +#endif }; struct sugov_cpu { @@ -64,6 +68,38 @@ static DEFINE_PER_CPU(struct sugov_cpu, sugov_cpu); /************************ Governor internals ***********************/ +#ifdef CONFIG_ENERGY_MODEL +static void sugov_policy_attach_pd(struct sugov_policy *sg_policy) +{ + struct em_perf_domain *pd; + struct cpufreq_policy *policy = sg_policy->policy; + + sg_policy->pd = NULL; + pd = em_cpu_get(policy->cpu); + if (!pd) + return; + + if (cpumask_equal(policy->related_cpus, to_cpumask(pd->cpus))) + sg_policy->pd = pd; + else + pr_warn("%s: Not all CPUs in schedutil policy %u share the same perf domain, no perf domain for that policy will be registered\n", + __func__, policy->cpu); +} + +static struct em_perf_domain *sugov_policy_get_pd( + struct sugov_policy *sg_policy) +{ + return sg_policy->pd; +} +#else /* CONFIG_ENERGY_MODEL */ +static void sugov_policy_attach_pd(struct sugov_policy *sg_policy) {} +static struct em_perf_domain *sugov_policy_get_pd( + struct sugov_policy *sg_policy) +{ + return NULL; +} +#endif /* CONFIG_ENERGY_MODEL */ + static bool sugov_should_update_freq(struct sugov_policy *sg_policy, u64 time) { s64 delta_ns; @@ -849,6 +885,9 @@ static int sugov_start(struct cpufreq_policy *policy) sugov_update_shared : sugov_update_single); } + + sugov_policy_attach_pd(sg_policy); + return 0; } From patchwork Wed May 8 17:42:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Douglas RAILLARD X-Patchwork-Id: 10936315 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 983FD14B6 for ; Wed, 8 May 2019 17:43:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 88B8C285E8 for ; Wed, 8 May 2019 17:43:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7CD7B2861C; Wed, 8 May 2019 17:43:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1028D285E8 for ; Wed, 8 May 2019 17:43:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729074AbfEHRnO (ORCPT ); Wed, 8 May 2019 13:43:14 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:41808 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725910AbfEHRnO (ORCPT ); Wed, 8 May 2019 13:43:14 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9045215AD; Wed, 8 May 2019 10:43:13 -0700 (PDT) Received: from e107049-lin.arm.com (e107049-lin.cambridge.arm.com [10.1.195.43]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 166C03F575; Wed, 8 May 2019 10:43:11 -0700 (PDT) From: douglas.raillard@arm.com To: linux-kernel@vger.kernel.org Cc: linux-pm@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, quentin.perret@arm.com, douglas.raillard@arm.com, patrick.bellasi@arm.com, dietmar.eggemann@arm.com Subject: [RFC PATCH 3/7] sched/cpufreq: Hook em_pd_get_higher_power() into get_next_freq() Date: Wed, 8 May 2019 18:42:57 +0100 Message-Id: <20190508174301.4828-4-douglas.raillard@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190508174301.4828-1-douglas.raillard@arm.com> References: <20190508174301.4828-1-douglas.raillard@arm.com> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Douglas RAILLARD Choose the highest OPP for a given energy cost, allowing to skip lower frequencies that would not be cheaper in terms of consumed power. These frequencies can still be interesting to keep in the energy model to give more freedom to thermal throttling, but should not be selected under normal circumstances. This also prepares the ground for energy-aware frequency boosting. Signed-off-by: Douglas RAILLARD --- kernel/sched/cpufreq_schedutil.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index 82dbacf83649..f4173a7c0f01 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -10,6 +10,7 @@ #include "sched.h" +#include #include #include @@ -200,9 +201,19 @@ static unsigned int get_next_freq(struct sugov_policy *sg_policy, struct cpufreq_policy *policy = sg_policy->policy; unsigned int freq = arch_scale_freq_invariant() ? policy->cpuinfo.max_freq : policy->cur; + struct em_perf_domain *pd = sugov_policy_get_pd(sg_policy); + + /* Maximum power we are ready to spend. */ + unsigned int cost_margin = 0; freq = map_util_freq(util, freq, max); + /* + * Try to get a higher frequency if one is available, given the extra + * power we are ready to spend. + */ + freq = em_pd_get_higher_freq(pd, freq, cost_margin); + if (freq == sg_policy->cached_raw_freq && !sg_policy->need_freq_update) return sg_policy->next_freq; From patchwork Wed May 8 17:42:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Douglas RAILLARD X-Patchwork-Id: 10936317 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B92AC933 for ; Wed, 8 May 2019 17:43:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A747F285E8 for ; Wed, 8 May 2019 17:43:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9B69828610; Wed, 8 May 2019 17:43:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 39D30285E8 for ; Wed, 8 May 2019 17:43:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726767AbfEHRne (ORCPT ); Wed, 8 May 2019 13:43:34 -0400 Received: from foss.arm.com ([217.140.101.70]:41816 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729078AbfEHRnP (ORCPT ); Wed, 8 May 2019 13:43:15 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 528C780D; Wed, 8 May 2019 10:43:15 -0700 (PDT) Received: from e107049-lin.arm.com (e107049-lin.cambridge.arm.com [10.1.195.43]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id CCBAF3F575; Wed, 8 May 2019 10:43:13 -0700 (PDT) From: douglas.raillard@arm.com To: linux-kernel@vger.kernel.org Cc: linux-pm@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, quentin.perret@arm.com, douglas.raillard@arm.com, patrick.bellasi@arm.com, dietmar.eggemann@arm.com Subject: [RFC PATCH 4/7] sched/cpufreq: Move up sugov_cpu_is_busy() Date: Wed, 8 May 2019 18:42:58 +0100 Message-Id: <20190508174301.4828-5-douglas.raillard@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190508174301.4828-1-douglas.raillard@arm.com> References: <20190508174301.4828-1-douglas.raillard@arm.com> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Douglas RAILLARD Move sugov_cpu_is_busy() static function of cpufreq_schedutil.c higher in the file so it can be used by other functions. Signed-off-by: Douglas RAILLARD --- kernel/sched/cpufreq_schedutil.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index f4173a7c0f01..a52c66559321 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -173,6 +173,19 @@ static void sugov_deferred_update(struct sugov_policy *sg_policy, u64 time, } } +#ifdef CONFIG_NO_HZ_COMMON +static bool sugov_cpu_is_busy(struct sugov_cpu *sg_cpu) +{ + unsigned long idle_calls = tick_nohz_get_idle_calls_cpu(sg_cpu->cpu); + bool ret = idle_calls == sg_cpu->saved_idle_calls; + + sg_cpu->saved_idle_calls = idle_calls; + return ret; +} +#else +static inline bool sugov_cpu_is_busy(struct sugov_cpu *sg_cpu) { return false; } +#endif /* CONFIG_NO_HZ_COMMON */ + /** * get_next_freq - Compute a new frequency for a given cpufreq policy. * @sg_policy: schedutil policy object to compute the new frequency for. @@ -462,19 +475,6 @@ static unsigned long sugov_iowait_apply(struct sugov_cpu *sg_cpu, u64 time, return max(boost, util); } -#ifdef CONFIG_NO_HZ_COMMON -static bool sugov_cpu_is_busy(struct sugov_cpu *sg_cpu) -{ - unsigned long idle_calls = tick_nohz_get_idle_calls_cpu(sg_cpu->cpu); - bool ret = idle_calls == sg_cpu->saved_idle_calls; - - sg_cpu->saved_idle_calls = idle_calls; - return ret; -} -#else -static inline bool sugov_cpu_is_busy(struct sugov_cpu *sg_cpu) { return false; } -#endif /* CONFIG_NO_HZ_COMMON */ - /* * Make sugov_should_update_freq() ignore the rate limit when DL * has increased the utilization. From patchwork Wed May 8 17:42:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Douglas RAILLARD X-Patchwork-Id: 10936313 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 870BD933 for ; Wed, 8 May 2019 17:43:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 76AD028610 for ; Wed, 8 May 2019 17:43:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6A5AA2862D; Wed, 8 May 2019 17:43:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 029B128610 for ; Wed, 8 May 2019 17:43:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729092AbfEHRnR (ORCPT ); Wed, 8 May 2019 13:43:17 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:41828 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725910AbfEHRnR (ORCPT ); Wed, 8 May 2019 13:43:17 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1476CA78; Wed, 8 May 2019 10:43:17 -0700 (PDT) Received: from e107049-lin.arm.com (e107049-lin.cambridge.arm.com [10.1.195.43]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 8EBFD3F575; Wed, 8 May 2019 10:43:15 -0700 (PDT) From: douglas.raillard@arm.com To: linux-kernel@vger.kernel.org Cc: linux-pm@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, quentin.perret@arm.com, douglas.raillard@arm.com, patrick.bellasi@arm.com, dietmar.eggemann@arm.com Subject: [RFC PATCH 5/7] sched/cpufreq: sugov_cpu_is_busy for shared policy Date: Wed, 8 May 2019 18:42:59 +0100 Message-Id: <20190508174301.4828-6-douglas.raillard@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190508174301.4828-1-douglas.raillard@arm.com> References: <20190508174301.4828-1-douglas.raillard@arm.com> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Douglas RAILLARD Allow using sugov_cpu_is_busy() from sugov_update_shared(). This means that the heuristic needs to return stable results across multiple calls for a given CPU, even if there has been no utilization change since last call. sugov_cpu_is_busy() currently both checks business status and updates the counters, so let's decouple the two actions. Signed-off-by: Douglas RAILLARD --- kernel/sched/cpufreq_schedutil.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index a52c66559321..a12b7e5bc028 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -178,12 +178,17 @@ static bool sugov_cpu_is_busy(struct sugov_cpu *sg_cpu) { unsigned long idle_calls = tick_nohz_get_idle_calls_cpu(sg_cpu->cpu); bool ret = idle_calls == sg_cpu->saved_idle_calls; + return ret; +} +static void sugov_cpu_is_busy_update(struct sugov_cpu *sg_cpu) +{ + unsigned long idle_calls = tick_nohz_get_idle_calls_cpu(sg_cpu->cpu); sg_cpu->saved_idle_calls = idle_calls; - return ret; } #else static inline bool sugov_cpu_is_busy(struct sugov_cpu *sg_cpu) { return false; } +static void sugov_cpu_is_busy_update(struct sugov_cpu *sg_cpu) {} #endif /* CONFIG_NO_HZ_COMMON */ /** @@ -503,6 +508,7 @@ static void sugov_update_single(struct update_util_data *hook, u64 time, return; busy = sugov_cpu_is_busy(sg_cpu); + sugov_cpu_is_busy_update(sg_cpu); util = sugov_get_util(sg_cpu); max = sg_cpu->max; From patchwork Wed May 8 17:43:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Douglas RAILLARD X-Patchwork-Id: 10936311 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2BA48933 for ; Wed, 8 May 2019 17:43:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 19B58285F9 for ; Wed, 8 May 2019 17:43:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0DBF12861C; Wed, 8 May 2019 17:43:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 96594285F9 for ; Wed, 8 May 2019 17:43:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729109AbfEHRnT (ORCPT ); Wed, 8 May 2019 13:43:19 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:41834 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729105AbfEHRnT (ORCPT ); Wed, 8 May 2019 13:43:19 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CB3F280D; Wed, 8 May 2019 10:43:18 -0700 (PDT) Received: from e107049-lin.arm.com (e107049-lin.cambridge.arm.com [10.1.195.43]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 5105B3F575; Wed, 8 May 2019 10:43:17 -0700 (PDT) From: douglas.raillard@arm.com To: linux-kernel@vger.kernel.org Cc: linux-pm@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, quentin.perret@arm.com, douglas.raillard@arm.com, patrick.bellasi@arm.com, dietmar.eggemann@arm.com Subject: [RFC PATCH 6/7] sched/cpufreq: Improve sugov_cpu_is_busy accuracy Date: Wed, 8 May 2019 18:43:00 +0100 Message-Id: <20190508174301.4828-7-douglas.raillard@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190508174301.4828-1-douglas.raillard@arm.com> References: <20190508174301.4828-1-douglas.raillard@arm.com> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Douglas RAILLARD Avoid assuming a CPU is busy when it has begun being idle before get_next_freq() is called. This is achieved by making sure the CPU will not be detected as busy by other CPUs whenever its utilization is decreasing. Signed-off-by: Douglas RAILLARD --- kernel/sched/cpufreq_schedutil.c | 42 ++++++++++++++++++++++++++++---- 1 file changed, 37 insertions(+), 5 deletions(-) diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index a12b7e5bc028..ce4b90cafbb5 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -62,6 +62,7 @@ struct sugov_cpu { /* The field below is for single-CPU policies only: */ #ifdef CONFIG_NO_HZ_COMMON unsigned long saved_idle_calls; + unsigned long previous_util; #endif }; @@ -181,14 +182,35 @@ static bool sugov_cpu_is_busy(struct sugov_cpu *sg_cpu) return ret; } -static void sugov_cpu_is_busy_update(struct sugov_cpu *sg_cpu) +static void sugov_cpu_is_busy_update(struct sugov_cpu *sg_cpu, + unsigned long util) { unsigned long idle_calls = tick_nohz_get_idle_calls_cpu(sg_cpu->cpu); sg_cpu->saved_idle_calls = idle_calls; + + /* + * Make sure that this CPU will not be immediately considered as busy in + * cases where the CPU has already entered an idle state. In that case, + * the number of idle_calls will not vary anymore until it exits idle, + * which would lead sugov_cpu_is_busy() to say that this CPU is busy, + * because it has not (re)entered idle since the last time we looked at + * it. + * Assuming cpu0 and cpu1 are in the same policy, that will make sure + * this sequence of events leads to right cpu1 business status from + * get_next_freq(cpu=1) + * cpu0: [enter idle] -> [get_next_freq] -> [doing nothing] -> [wakeup] + * cpu1: ... -> [get_next_freq] -> ... + */ + if (util <= sg_cpu->previous_util) + sg_cpu->saved_idle_calls--; + + sg_cpu->previous_util = util; } #else static inline bool sugov_cpu_is_busy(struct sugov_cpu *sg_cpu) { return false; } -static void sugov_cpu_is_busy_update(struct sugov_cpu *sg_cpu) {} +static void sugov_cpu_is_busy_update(struct sugov_cpu *sg_cpu + unsigned long util) +{} #endif /* CONFIG_NO_HZ_COMMON */ /** @@ -507,10 +529,9 @@ static void sugov_update_single(struct update_util_data *hook, u64 time, if (!sugov_should_update_freq(sg_policy, time)) return; - busy = sugov_cpu_is_busy(sg_cpu); - sugov_cpu_is_busy_update(sg_cpu); - util = sugov_get_util(sg_cpu); + busy = sugov_cpu_is_busy(sg_cpu); + sugov_cpu_is_busy_update(sg_cpu, util); max = sg_cpu->max; util = sugov_iowait_apply(sg_cpu, time, util, max); next_f = get_next_freq(sg_policy, util, max); @@ -545,12 +566,15 @@ static unsigned int sugov_next_freq_shared(struct sugov_cpu *sg_cpu, u64 time) struct cpufreq_policy *policy = sg_policy->policy; unsigned long util = 0, max = 1; unsigned int j; + unsigned long sg_cpu_util = 0; for_each_cpu(j, policy->cpus) { struct sugov_cpu *j_sg_cpu = &per_cpu(sugov_cpu, j); unsigned long j_util, j_max; j_util = sugov_get_util(j_sg_cpu); + if (j_sg_cpu == sg_cpu) + sg_cpu_util = j_util; j_max = j_sg_cpu->max; j_util = sugov_iowait_apply(j_sg_cpu, time, j_util, j_max); @@ -560,6 +584,14 @@ static unsigned int sugov_next_freq_shared(struct sugov_cpu *sg_cpu, u64 time) } } + /* + * Only update the business status if we are looking at the CPU for + * which a utilization change triggered a call to get_next_freq(). This + * way, we don't affect the "busy" status of CPUs that don't have any + * change in utilization. + */ + sugov_cpu_is_busy_update(sg_cpu, sg_cpu_util); + return get_next_freq(sg_policy, util, max); } From patchwork Wed May 8 17:43:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Douglas RAILLARD X-Patchwork-Id: 10936309 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9AA50933 for ; Wed, 8 May 2019 17:43:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 884A0285F9 for ; Wed, 8 May 2019 17:43:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7AE322861C; Wed, 8 May 2019 17:43:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 04519285F9 for ; Wed, 8 May 2019 17:43:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729132AbfEHRnV (ORCPT ); Wed, 8 May 2019 13:43:21 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:41840 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729122AbfEHRnV (ORCPT ); Wed, 8 May 2019 13:43:21 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8D6C0A78; Wed, 8 May 2019 10:43:20 -0700 (PDT) Received: from e107049-lin.arm.com (e107049-lin.cambridge.arm.com [10.1.195.43]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 138AB3F575; Wed, 8 May 2019 10:43:18 -0700 (PDT) From: douglas.raillard@arm.com To: linux-kernel@vger.kernel.org Cc: linux-pm@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, quentin.perret@arm.com, douglas.raillard@arm.com, patrick.bellasi@arm.com, dietmar.eggemann@arm.com Subject: [RFC PATCH 7/7] sched/cpufreq: Boost schedutil frequency ramp up Date: Wed, 8 May 2019 18:43:01 +0100 Message-Id: <20190508174301.4828-8-douglas.raillard@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190508174301.4828-1-douglas.raillard@arm.com> References: <20190508174301.4828-1-douglas.raillard@arm.com> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Douglas RAILLARD In some situations, it can be interesting to spend temporarily more power if that can give a useful frequency boost. The sugov_cpu_is_busy() heuristic is reused to check if there has been some idle time on all CPUs in the considered perf domain since last call to schedutil's get_next_freq(). If not, it is assumed that at least one CPU is in a frequency ramp up phase and the domain will be allowed to spend extra power to reach a stable OPP in a shorter amount of time. Since the extra power expenditure is bounded, it cannot skyrocket even on platforms with a large number of cores in the same frequency domain and/or very high ratio between lowest and highest OPP cost. Signed-off-by: Douglas RAILLARD --- kernel/sched/cpufreq_schedutil.c | 22 +++++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index ce4b90cafbb5..513b32bf14c5 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -218,6 +218,8 @@ static void sugov_cpu_is_busy_update(struct sugov_cpu *sg_cpu * @sg_policy: schedutil policy object to compute the new frequency for. * @util: Current CPU utilization. * @max: CPU capacity. + * @busy: true if at least one CPU in the policy is busy, which means it had no + * idle time since its last frequency change. * * If the utilization is frequency-invariant, choose the new frequency to be * proportional to it, that is @@ -231,20 +233,28 @@ static void sugov_cpu_is_busy_update(struct sugov_cpu *sg_cpu * * Take C = 1.25 for the frequency tipping point at (util / max) = 0.8. * + * An energy-aware boost is then applied if busy is true. The boost will allow + * selecting frequencies at most twice as costly in term of energy. + * * The lowest driver-supported frequency which is equal or greater than the raw * next_freq (as calculated above) is returned, subject to policy min/max and * cpufreq driver limitations. */ static unsigned int get_next_freq(struct sugov_policy *sg_policy, - unsigned long util, unsigned long max) + unsigned long util, unsigned long max, + bool busy) { struct cpufreq_policy *policy = sg_policy->policy; unsigned int freq = arch_scale_freq_invariant() ? policy->cpuinfo.max_freq : policy->cur; struct em_perf_domain *pd = sugov_policy_get_pd(sg_policy); - /* Maximum power we are ready to spend. */ - unsigned int cost_margin = 0; + /* + * Maximum power we are ready to spend. + * When one CPU is busy in the policy, we apply a boost to help it reach + * the needed frequency faster. + */ + unsigned int cost_margin = busy ? 1024/2 : 0; freq = map_util_freq(util, freq, max); @@ -534,7 +544,7 @@ static void sugov_update_single(struct update_util_data *hook, u64 time, sugov_cpu_is_busy_update(sg_cpu, util); max = sg_cpu->max; util = sugov_iowait_apply(sg_cpu, time, util, max); - next_f = get_next_freq(sg_policy, util, max); + next_f = get_next_freq(sg_policy, util, max, busy); /* * Do not reduce the frequency if the CPU has not been idle * recently, as the reduction is likely to be premature then. @@ -567,6 +577,7 @@ static unsigned int sugov_next_freq_shared(struct sugov_cpu *sg_cpu, u64 time) unsigned long util = 0, max = 1; unsigned int j; unsigned long sg_cpu_util = 0; + bool busy = false; for_each_cpu(j, policy->cpus) { struct sugov_cpu *j_sg_cpu = &per_cpu(sugov_cpu, j); @@ -577,6 +588,7 @@ static unsigned int sugov_next_freq_shared(struct sugov_cpu *sg_cpu, u64 time) sg_cpu_util = j_util; j_max = j_sg_cpu->max; j_util = sugov_iowait_apply(j_sg_cpu, time, j_util, j_max); + busy |= sugov_cpu_is_busy(j_sg_cpu); if (j_util * max > j_max * util) { util = j_util; @@ -592,7 +604,7 @@ static unsigned int sugov_next_freq_shared(struct sugov_cpu *sg_cpu, u64 time) */ sugov_cpu_is_busy_update(sg_cpu, sg_cpu_util); - return get_next_freq(sg_policy, util, max); + return get_next_freq(sg_policy, util, max, busy); } static void