From patchwork Mon Oct 9 10:36:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 13413369 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94B19E95A91 for ; Mon, 9 Oct 2023 10:36:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345877AbjJIKgb (ORCPT ); Mon, 9 Oct 2023 06:36:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46970 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345831AbjJIKga (ORCPT ); Mon, 9 Oct 2023 06:36:30 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 515D09F for ; Mon, 9 Oct 2023 03:36:28 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id ffacd0b85a97d-3296b49c546so2468674f8f.3 for ; Mon, 09 Oct 2023 03:36:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1696847787; x=1697452587; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=re7b+0McMMJs79AIsRmZVXLCw9N+5e5Qqk3aklm3ETc=; b=kmsDUtAsrjuA2Wdc3hsWu2yZ/c5ilHfeqWH+Uuy+u4uj5O8kL9oPPTgZY1WcBjFw00 2rUWYUsebVOmdH72oqY/vP81dvhWhiUfXh6lx45D1eZclu88SolVvC3xPlKMhiL4KxUL OsoXCXNSp2WTNhwCszD/iWRBsRZfzua0Xb6RpVUpMIjjE9HYaaqfL9WRUU7dMOyDe4sH vkT7+eHAg0DNZ11O6t3qkkbDkYcrT/s7n6EdvchnMJXVUeqWblDWx7NE2ATUrcMeoPGL /9dQCqKGrKIeiqxQDcC1RWTBjHngA05H9WfR8uqH5iM6z7/sFuRxb5Eg7pIC3oKUji5I SI5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696847787; x=1697452587; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=re7b+0McMMJs79AIsRmZVXLCw9N+5e5Qqk3aklm3ETc=; b=dXl9eW9A2RCFBqEyAWKiItQqgZbBqQgOtXXI3AA3UVwCB6mPfYI4udVF7/rsulWhcj +hZCpPrEI/hpJZB4DaeDU67AyXDly4IRKKS1ho2bexHjHKD75YvAyS9bTD7kzAQFvyHA Hw+Rl5X0a5EmPbsgKrTAYWfShDwXv7y/Y/G13/R0+qDwZbiWRxv9fhQvSHcJyuXRdi9l iFYRiakDN2T3aLYkLMJpfBAYZHfik0cUGvh387r+LrjAUAq+Pnc+/+2qkng4Q5kPWBsr MqnkqXdPpwfQ66PGqp5Dieta0151uM0oGNwAQJPP9JITagXHh4GT2FPawQtut562iWeg LBdg== X-Gm-Message-State: AOJu0YxW8ugmUDz0hy7j9fAmtC3o91cAemb1I4ifU+yiAa80jp71rWpy pFhoOlfWqwmGEvOkHuHZ8jyMQQ== X-Google-Smtp-Source: AGHT+IEjQw3F2QpiyYMDHKQ0Ydoc0UsVzOGqUY2ht0reqHXmAAoEP6EJ50UQeRQWWBd0aG7tZrfNCQ== X-Received: by 2002:adf:f505:0:b0:327:ffef:ba51 with SMTP id q5-20020adff505000000b00327ffefba51mr13155370wro.49.1696847786785; Mon, 09 Oct 2023 03:36:26 -0700 (PDT) Received: from vingu-book.. ([2a01:e0a:f:6020:53f1:24bc:5e47:821d]) by smtp.gmail.com with ESMTPSA id f16-20020adfdb50000000b0031ff89af0e4sm9226722wrj.99.2023.10.09.03.36.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Oct 2023 03:36:26 -0700 (PDT) From: Vincent Guittot To: linux@armlinux.org.uk, catalin.marinas@arm.com, will@kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, sudeep.holla@arm.com, gregkh@linuxfoundation.org, rafael@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, viresh.kumar@linaro.org, lukasz.luba@arm.com, ionela.voinescu@arm.com, pierre.gondois@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-pm@vger.kernel.org Cc: conor.dooley@microchip.com, suagrfillet@gmail.com, ajones@ventanamicro.com, lftan@kernel.org, Vincent Guittot Subject: [PATCH v2 1/6] sched: consolidate and cleanup access to CPU's max compute capacity Date: Mon, 9 Oct 2023 12:36:16 +0200 Message-Id: <20231009103621.374412-2-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231009103621.374412-1-vincent.guittot@linaro.org> References: <20231009103621.374412-1-vincent.guittot@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org Remove struct rq cpu_capacity_orig field and use arch_scale_cpu_capacity() instead. Scheduler uses 3 methods to get access to the CPU's max compute capacity: - arch_scale_cpu_capacity(cpu) which is the default way to get CPU's capacity. - cpu_capacity_orig field which is periodically updated with arch_scale_cpu_capacity(). - capacity_orig_of(cpu) which encapsulates rq->cpu_capacity_orig. There is no real need to save the value returned by arch_scale_cpu_capacity() in struct rq. arch_scale_cpu_capacity() returns: - either a per_cpu variable. - or a const value for systems which have only one capacity. Remove cpu_capacity_orig and use arch_scale_cpu_capacity() everywhere. No functional changes. some tests of Arm64 small SMP device (hikey): no noticeable changes HMP device (RB5): hackbench shows minor improvement (1-2%) large smp (thx2): hackbench and tbench shows minor improvement (1%) Signed-off-by: Vincent Guittot Reviewed-by: Dietmar Eggemann --- Documentation/scheduler/sched-capacity.rst | 13 +++++++------ kernel/sched/core.c | 2 +- kernel/sched/cpudeadline.c | 2 +- kernel/sched/deadline.c | 4 ++-- kernel/sched/fair.c | 18 ++++++++---------- kernel/sched/rt.c | 2 +- kernel/sched/sched.h | 6 ------ kernel/sched/topology.c | 7 +++++-- 8 files changed, 25 insertions(+), 29 deletions(-) diff --git a/Documentation/scheduler/sched-capacity.rst b/Documentation/scheduler/sched-capacity.rst index e2c1cf743158..de414b33dd2a 100644 --- a/Documentation/scheduler/sched-capacity.rst +++ b/Documentation/scheduler/sched-capacity.rst @@ -39,14 +39,15 @@ per Hz, leading to:: ------------------- Two different capacity values are used within the scheduler. A CPU's -``capacity_orig`` is its maximum attainable capacity, i.e. its maximum -attainable performance level. A CPU's ``capacity`` is its ``capacity_orig`` to -which some loss of available performance (e.g. time spent handling IRQs) is -subtracted. +``original capacity`` is its maximum attainable capacity, i.e. its maximum +attainable performance level. This original capacity is returned by +the function arch_scale_cpu_capacity(). A CPU's ``capacity`` is its ``original +capacity`` to which some loss of available performance (e.g. time spent +handling IRQs) is subtracted. Note that a CPU's ``capacity`` is solely intended to be used by the CFS class, -while ``capacity_orig`` is class-agnostic. The rest of this document will use -the term ``capacity`` interchangeably with ``capacity_orig`` for the sake of +while ``original capacity`` is class-agnostic. The rest of this document will use +the term ``capacity`` interchangeably with ``original capacity`` for the sake of brevity. 1.3 Platform examples diff --git a/kernel/sched/core.c b/kernel/sched/core.c index cf6d3fdd4eb5..a3f9cd52eec5 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -9929,7 +9929,7 @@ void __init sched_init(void) #ifdef CONFIG_SMP rq->sd = NULL; rq->rd = NULL; - rq->cpu_capacity = rq->cpu_capacity_orig = SCHED_CAPACITY_SCALE; + rq->cpu_capacity = SCHED_CAPACITY_SCALE; rq->balance_callback = &balance_push_callback; rq->active_balance = 0; rq->next_balance = jiffies; diff --git a/kernel/sched/cpudeadline.c b/kernel/sched/cpudeadline.c index 57c92d751bcd..95baa12a1029 100644 --- a/kernel/sched/cpudeadline.c +++ b/kernel/sched/cpudeadline.c @@ -131,7 +131,7 @@ int cpudl_find(struct cpudl *cp, struct task_struct *p, if (!dl_task_fits_capacity(p, cpu)) { cpumask_clear_cpu(cpu, later_mask); - cap = capacity_orig_of(cpu); + cap = arch_scale_cpu_capacity(cpu); if (cap > max_cap || (cpu == task_cpu(p) && cap == max_cap)) { diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index d98408a274e5..7039a8d5ae9b 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -132,7 +132,7 @@ static inline unsigned long __dl_bw_capacity(const struct cpumask *mask) int i; for_each_cpu_and(i, mask, cpu_active_mask) - cap += capacity_orig_of(i); + cap += arch_scale_cpu_capacity(i); return cap; } @@ -144,7 +144,7 @@ static inline unsigned long __dl_bw_capacity(const struct cpumask *mask) static inline unsigned long dl_bw_capacity(int i) { if (!sched_asym_cpucap_active() && - capacity_orig_of(i) == SCHED_CAPACITY_SCALE) { + arch_scale_cpu_capacity(i) == SCHED_CAPACITY_SCALE) { return dl_bw_cpus(i) << SCHED_CAPACITY_SHIFT; } else { RCU_LOCKDEP_WARN(!rcu_read_lock_sched_held(), diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 04fbcbda97d5..7e2027c810e7 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4713,7 +4713,7 @@ static inline void util_est_update(struct cfs_rq *cfs_rq, * To avoid overestimation of actual task utilization, skip updates if * we cannot grant there is idle time in this CPU. */ - if (task_util(p) > capacity_orig_of(cpu_of(rq_of(cfs_rq)))) + if (task_util(p) > arch_scale_cpu_capacity(cpu_of(rq_of(cfs_rq)))) return; /* @@ -4761,14 +4761,14 @@ static inline int util_fits_cpu(unsigned long util, return fits; /* - * We must use capacity_orig_of() for comparing against uclamp_min and + * We must use arch_scale_cpu_capacity() for comparing against uclamp_min and * uclamp_max. We only care about capacity pressure (by using * capacity_of()) for comparing against the real util. * * If a task is boosted to 1024 for example, we don't want a tiny * pressure to skew the check whether it fits a CPU or not. * - * Similarly if a task is capped to capacity_orig_of(little_cpu), it + * Similarly if a task is capped to arch_scale_cpu_capacity(little_cpu), it * should fit a little cpu even if there's some pressure. * * Only exception is for thermal pressure since it has a direct impact @@ -4780,7 +4780,7 @@ static inline int util_fits_cpu(unsigned long util, * For uclamp_max, we can tolerate a drop in performance level as the * goal is to cap the task. So it's okay if it's getting less. */ - capacity_orig = capacity_orig_of(cpu); + capacity_orig = arch_scale_cpu_capacity(cpu); capacity_orig_thermal = capacity_orig - arch_scale_thermal_pressure(cpu); /* @@ -7261,7 +7261,7 @@ select_idle_capacity(struct task_struct *p, struct sched_domain *sd, int target) * Look for the CPU with best capacity. */ else if (fits < 0) - cpu_cap = capacity_orig_of(cpu) - thermal_load_avg(cpu_rq(cpu)); + cpu_cap = arch_scale_cpu_capacity(cpu) - thermal_load_avg(cpu_rq(cpu)); /* * First, select CPU which fits better (-1 being better than 0). @@ -7503,7 +7503,7 @@ cpu_util(int cpu, struct task_struct *p, int dst_cpu, int boost) util = max(util, util_est); } - return min(util, capacity_orig_of(cpu)); + return min(util, arch_scale_cpu_capacity(cpu)); } unsigned long cpu_util_cfs(int cpu) @@ -9294,8 +9294,6 @@ static void update_cpu_capacity(struct sched_domain *sd, int cpu) unsigned long capacity = scale_rt_capacity(cpu); struct sched_group *sdg = sd->groups; - cpu_rq(cpu)->cpu_capacity_orig = arch_scale_cpu_capacity(cpu); - if (!capacity) capacity = 1; @@ -9371,7 +9369,7 @@ static inline int check_cpu_capacity(struct rq *rq, struct sched_domain *sd) { return ((rq->cpu_capacity * sd->imbalance_pct) < - (rq->cpu_capacity_orig * 100)); + (arch_scale_cpu_capacity(cpu_of(rq)) * 100)); } /* @@ -9382,7 +9380,7 @@ check_cpu_capacity(struct rq *rq, struct sched_domain *sd) static inline int check_misfit_status(struct rq *rq, struct sched_domain *sd) { return rq->misfit_task_load && - (rq->cpu_capacity_orig < rq->rd->max_cpu_capacity || + (arch_scale_cpu_capacity(rq->cpu) < rq->rd->max_cpu_capacity || check_cpu_capacity(rq, sd)); } diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 88fc98601413..72f0a0767059 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -471,7 +471,7 @@ static inline bool rt_task_fits_capacity(struct task_struct *p, int cpu) min_cap = uclamp_eff_value(p, UCLAMP_MIN); max_cap = uclamp_eff_value(p, UCLAMP_MAX); - cpu_cap = capacity_orig_of(cpu); + cpu_cap = arch_scale_cpu_capacity(cpu); return cpu_cap >= min(min_cap, max_cap); } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 649eb9ec0657..74195eb39eaa 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1033,7 +1033,6 @@ struct rq { struct sched_domain __rcu *sd; unsigned long cpu_capacity; - unsigned long cpu_capacity_orig; struct balance_callback *balance_callback; @@ -2967,11 +2966,6 @@ static inline void cpufreq_update_util(struct rq *rq, unsigned int flags) {} #endif #ifdef CONFIG_SMP -static inline unsigned long capacity_orig_of(int cpu) -{ - return cpu_rq(cpu)->cpu_capacity_orig; -} - /** * enum cpu_util_type - CPU utilization type * @FREQUENCY_UTIL: Utilization used to select frequency diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index a7b50bba7829..1cc595907363 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -2488,12 +2488,15 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att /* Attach the domains */ rcu_read_lock(); for_each_cpu(i, cpu_map) { + unsigned long capacity; + rq = cpu_rq(i); sd = *per_cpu_ptr(d.sd, i); + capacity = arch_scale_cpu_capacity(i); /* Use READ_ONCE()/WRITE_ONCE() to avoid load/store tearing: */ - if (rq->cpu_capacity_orig > READ_ONCE(d.rd->max_cpu_capacity)) - WRITE_ONCE(d.rd->max_cpu_capacity, rq->cpu_capacity_orig); + if (capacity > READ_ONCE(d.rd->max_cpu_capacity)) + WRITE_ONCE(d.rd->max_cpu_capacity, capacity); cpu_attach_domain(sd, d.rd, i); } From patchwork Mon Oct 9 10:36:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 13413371 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 427E5E95A8E for ; Mon, 9 Oct 2023 10:36:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345890AbjJIKgg (ORCPT ); Mon, 9 Oct 2023 06:36:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345887AbjJIKgc (ORCPT ); Mon, 9 Oct 2023 06:36:32 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6FA4AB for ; Mon, 9 Oct 2023 03:36:29 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id ffacd0b85a97d-323168869daso4295762f8f.2 for ; Mon, 09 Oct 2023 03:36:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1696847788; x=1697452588; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1IicB4ODERuC1IEqNgt/JtBz6z2h9Ys3ExqnEipNKQY=; b=AasJ6UtCQrpQt+8GtUfwZdiYa81u585pfCUctZVy8RZLsEdtULx8chhLCtjK5RLOWH /WrYoJ8QJ/R+T01N4u6AK0uWobVZGtDL+WdEN2K/rQTF0PUlo3+NMQGFwvFGlkWMSu5+ uTNvgN5ZXhoslm0foOsgUd+NcHcUeVKpIY/PEaFtd+AYCeVOqHZsgwKh7L2gh8qONXJm bz+8Su8VmsHNJ+MLh3Ff6Y1ia0CBw59dj6hFgb57L6ugeW+Gfy0J/sHq95/wUWRJa3e2 +Dvj1n0t2iT1JLGLpV/XCY1lDFhuqT8YD+rQ0A8YBnY5DtSTlApOXgD7qwjjgSk5qoyW yEOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696847788; x=1697452588; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1IicB4ODERuC1IEqNgt/JtBz6z2h9Ys3ExqnEipNKQY=; b=b3zKumtQGWi48rZa2xOWU5pZ5lGCxqQaTG4NhJBrV86AE0xx+/Ha0RzG3AnLClQ36h xCge0Rw/CR21mFPIvq5L1Un0f8HwhRU5fKiGcMFdgAIXMYo0B6dF9EWEizhHXwM53yRa +Myg/ImwvHqm5UNsM+MrqcojmtfD72PNZfiRUAxkFVQs5AyOI9/uqJ0+KS7XZMlhDgEm yReGy0oe4f5WxnqUByuhUlbPeUOs4RQK07FaVkAspb6YIoUj0e2UCXfLCwwoh3tvtHo2 0r33cuFHrUal/Lx3rVhLvuXsOock1pBmsbeMBVfHla9e7hjva5hzrBCNsjPl+nHEojdy inZQ== X-Gm-Message-State: AOJu0YwN5yBe/Ll6FUuUDSiWqDrhwfdvEz7zDTcmtEoKae+tr3wcltkF CB4MGHD6gaB2iZjR4G9BJne2vg== X-Google-Smtp-Source: AGHT+IGYCDsjL+s+eXaJYDwmVGq+Pm1XzCDysQm4pemCHQ7SCDtFRnahF5GOdbFaRBcM0Tt7ZFT2Dw== X-Received: by 2002:a5d:4d02:0:b0:317:6fb5:bafc with SMTP id z2-20020a5d4d02000000b003176fb5bafcmr12403144wrt.65.1696847788095; Mon, 09 Oct 2023 03:36:28 -0700 (PDT) Received: from vingu-book.. ([2a01:e0a:f:6020:53f1:24bc:5e47:821d]) by smtp.gmail.com with ESMTPSA id f16-20020adfdb50000000b0031ff89af0e4sm9226722wrj.99.2023.10.09.03.36.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Oct 2023 03:36:27 -0700 (PDT) From: Vincent Guittot To: linux@armlinux.org.uk, catalin.marinas@arm.com, will@kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, sudeep.holla@arm.com, gregkh@linuxfoundation.org, rafael@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, viresh.kumar@linaro.org, lukasz.luba@arm.com, ionela.voinescu@arm.com, pierre.gondois@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-pm@vger.kernel.org Cc: conor.dooley@microchip.com, suagrfillet@gmail.com, ajones@ventanamicro.com, lftan@kernel.org, Vincent Guittot Subject: [PATCH v2 2/6] topology: add a new arch_scale_freq_reference Date: Mon, 9 Oct 2023 12:36:17 +0200 Message-Id: <20231009103621.374412-3-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231009103621.374412-1-vincent.guittot@linaro.org> References: <20231009103621.374412-1-vincent.guittot@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org Create a new method to get a unique and fixed max frequency. Currently cpuinfo.max_freq or the highest (or last) state of performance domain are used as the max frequency when computing the frequency for a level of utilization but: - cpuinfo_max_freq can change at runtime. boost is one example of such change. - cpuinfo.max_freq and last item of the PD can be different leading to different results between cpufreq and energy model. We need to save the reference frequency that has been used when computing the CPUs capacity and use this fixed and coherent value to convert between frequency and CPU's capacity. In fact, we already save the frequency that has been used when computing the capacity of each CPU. We extend the precision to save khZ instead of Mhz currently and we modify the type to be aligned with other variables used when converting frequency to capacity and the other way. Signed-off-by: Vincent Guittot Reviewed-by: Lukasz Luba --- arch/arm/include/asm/topology.h | 1 + arch/arm64/include/asm/topology.h | 1 + arch/riscv/include/asm/topology.h | 1 + drivers/base/arch_topology.c | 29 ++++++++++++++--------------- include/linux/arch_topology.h | 7 +++++++ 5 files changed, 24 insertions(+), 15 deletions(-) diff --git a/arch/arm/include/asm/topology.h b/arch/arm/include/asm/topology.h index c7d2510e5a78..853c4f81ba4a 100644 --- a/arch/arm/include/asm/topology.h +++ b/arch/arm/include/asm/topology.h @@ -13,6 +13,7 @@ #define arch_set_freq_scale topology_set_freq_scale #define arch_scale_freq_capacity topology_get_freq_scale #define arch_scale_freq_invariant topology_scale_freq_invariant +#define arch_scale_freq_ref topology_get_freq_ref #endif /* Replace task scheduler's default cpu-invariant accounting */ diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h index 9fab663dd2de..a323b109b9c4 100644 --- a/arch/arm64/include/asm/topology.h +++ b/arch/arm64/include/asm/topology.h @@ -23,6 +23,7 @@ void update_freq_counters_refs(void); #define arch_set_freq_scale topology_set_freq_scale #define arch_scale_freq_capacity topology_get_freq_scale #define arch_scale_freq_invariant topology_scale_freq_invariant +#define arch_scale_freq_ref topology_get_freq_ref #ifdef CONFIG_ACPI_CPPC_LIB #define arch_init_invariance_cppc topology_init_cpu_capacity_cppc diff --git a/arch/riscv/include/asm/topology.h b/arch/riscv/include/asm/topology.h index e316ab3b77f3..61183688bdd5 100644 --- a/arch/riscv/include/asm/topology.h +++ b/arch/riscv/include/asm/topology.h @@ -9,6 +9,7 @@ #define arch_set_freq_scale topology_set_freq_scale #define arch_scale_freq_capacity topology_get_freq_scale #define arch_scale_freq_invariant topology_scale_freq_invariant +#define arch_scale_freq_ref topology_get_freq_ref /* Replace task scheduler's default cpu-invariant accounting */ #define arch_scale_cpu_capacity topology_get_cpu_scale diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c index b741b5ba82bd..9a073c2d2086 100644 --- a/drivers/base/arch_topology.c +++ b/drivers/base/arch_topology.c @@ -19,6 +19,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -26,7 +27,8 @@ static DEFINE_PER_CPU(struct scale_freq_data __rcu *, sft_data); static struct cpumask scale_freq_counters_mask; static bool scale_freq_invariant; -static DEFINE_PER_CPU(u32, freq_factor) = 1; +DEFINE_PER_CPU(unsigned long, capacity_ref_freq) = 1; +EXPORT_PER_CPU_SYMBOL_GPL(capacity_ref_freq); static bool supports_scale_freq_counters(const struct cpumask *cpus) { @@ -170,9 +172,9 @@ DEFINE_PER_CPU(unsigned long, thermal_pressure); * operating on stale data when hot-plug is used for some CPUs. The * @capped_freq reflects the currently allowed max CPUs frequency due to * thermal capping. It might be also a boost frequency value, which is bigger - * than the internal 'freq_factor' max frequency. In such case the pressure - * value should simply be removed, since this is an indication that there is - * no thermal throttling. The @capped_freq must be provided in kHz. + * than the internal 'capacity_ref_freq' max frequency. In such case the + * pressure value should simply be removed, since this is an indication that + * there is no thermal throttling. The @capped_freq must be provided in kHz. */ void topology_update_thermal_pressure(const struct cpumask *cpus, unsigned long capped_freq) @@ -183,10 +185,7 @@ void topology_update_thermal_pressure(const struct cpumask *cpus, cpu = cpumask_first(cpus); max_capacity = arch_scale_cpu_capacity(cpu); - max_freq = per_cpu(freq_factor, cpu); - - /* Convert to MHz scale which is used in 'freq_factor' */ - capped_freq /= 1000; + max_freq = arch_scale_freq_ref(cpu); /* * Handle properly the boost frequencies, which should simply clean @@ -279,13 +278,13 @@ void topology_normalize_cpu_scale(void) capacity_scale = 1; for_each_possible_cpu(cpu) { - capacity = raw_capacity[cpu] * per_cpu(freq_factor, cpu); + capacity = raw_capacity[cpu] * per_cpu(capacity_ref_freq, cpu); capacity_scale = max(capacity, capacity_scale); } pr_debug("cpu_capacity: capacity_scale=%llu\n", capacity_scale); for_each_possible_cpu(cpu) { - capacity = raw_capacity[cpu] * per_cpu(freq_factor, cpu); + capacity = raw_capacity[cpu] * per_cpu(capacity_ref_freq, cpu); capacity = div64_u64(capacity << SCHED_CAPACITY_SHIFT, capacity_scale); topology_set_cpu_scale(cpu, capacity); @@ -321,15 +320,15 @@ bool __init topology_parse_cpu_capacity(struct device_node *cpu_node, int cpu) cpu_node, raw_capacity[cpu]); /* - * Update freq_factor for calculating early boot cpu capacities. + * Update capacity_ref_freq for calculating early boot cpu capacities. * For non-clk CPU DVFS mechanism, there's no way to get the * frequency value now, assuming they are running at the same - * frequency (by keeping the initial freq_factor value). + * frequency (by keeping the initial capacity_ref_freq value). */ cpu_clk = of_clk_get(cpu_node, 0); if (!PTR_ERR_OR_ZERO(cpu_clk)) { - per_cpu(freq_factor, cpu) = - clk_get_rate(cpu_clk) / 1000; + per_cpu(capacity_ref_freq, cpu) = + clk_get_rate(cpu_clk) / HZ_PER_KHZ; clk_put(cpu_clk); } } else { @@ -411,7 +410,7 @@ init_cpu_capacity_callback(struct notifier_block *nb, cpumask_andnot(cpus_to_visit, cpus_to_visit, policy->related_cpus); for_each_cpu(cpu, policy->related_cpus) - per_cpu(freq_factor, cpu) = policy->cpuinfo.max_freq / 1000; + per_cpu(capacity_ref_freq, cpu) = policy->cpuinfo.max_freq; if (cpumask_empty(cpus_to_visit)) { topology_normalize_cpu_scale(); diff --git a/include/linux/arch_topology.h b/include/linux/arch_topology.h index a07b510e7dc5..38ca6c76af56 100644 --- a/include/linux/arch_topology.h +++ b/include/linux/arch_topology.h @@ -27,6 +27,13 @@ static inline unsigned long topology_get_cpu_scale(int cpu) void topology_set_cpu_scale(unsigned int cpu, unsigned long capacity); +DECLARE_PER_CPU(unsigned long, capacity_ref_freq); + +static inline unsigned long topology_get_freq_ref(int cpu) +{ + return per_cpu(capacity_ref_freq, cpu); +} + DECLARE_PER_CPU(unsigned long, arch_freq_scale); static inline unsigned long topology_get_freq_scale(int cpu) From patchwork Mon Oct 9 10:36:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 13413370 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B09B7E95A91 for ; Mon, 9 Oct 2023 10:36:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345914AbjJIKgf (ORCPT ); Mon, 9 Oct 2023 06:36:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345890AbjJIKgc (ORCPT ); Mon, 9 Oct 2023 06:36:32 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0CC889D for ; Mon, 9 Oct 2023 03:36:31 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id ffacd0b85a97d-32157c8e4c7so4354542f8f.1 for ; Mon, 09 Oct 2023 03:36:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1696847789; x=1697452589; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kNKKT3nz1YObI5oerkbE50Couf3/jlljKZ/fkPRsfb8=; b=KKeeHEgwkzQ7bGffr2RggX/J5Aa4SZbzSD22Y5kids44XDdQavj8NZcLWx99fYBZ44 wIcGSDTNGZfBUNPtXTPGSYvprHajbjEV5o3pkN9iozzacYoFjWK0YowgKFPjbyxhUr/N sj9jnBgC4u8sVATjbsjYf0H32w6xoMuBLH4CdlRu4BmmZkHjZvKyjwPmlp+SR6paFCiG 5vHmH2oc5AFL7ysO68tWlkmkvKm/FlEnSh7vog1EsBd11/nWf5R8L6d903K82pOls5oX OByluKj0k3RjPvFjgO9OGw3mKzpAlxhqx6BGxGRs+z2Y0e6+w0Ws8l9y2fI1DXs0KydH 5S3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696847789; x=1697452589; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kNKKT3nz1YObI5oerkbE50Couf3/jlljKZ/fkPRsfb8=; b=uxU9mLLTFrNmUq3bWj/8EZOys9e0ZhacXbadaS+DvPhxCR7EXDxt/Knm7mY+iiRMvw iNNxc/9E/ZXIpaRfa655HL+0F2RIS+B1dviX6DRLgFpUlrMyeHd24eUOhPvAE9jC+nAY SXGfFh/wwVsptDqlAsRSwtD0gi/q/tMu2cZIfzThFnucHlcgK20AKIHRsiM2pJTNhImE G4p2csknS5VkA9DOhJ/Cro5M3KpM7zunV96QdD48D1acaCYPHvgdY3KfYtdB7COu4601 Kwi+aOIUleJ6bLSySbKmlsqKGKehBLfs6X3ngpwUFTW7VTJtY+I6P16c/OwInSjbWu9V nluA== X-Gm-Message-State: AOJu0YxG1Z31aBOU5i1wjWv8BXeXCxlJmH2xPfbYjQMvz09qzRuaAZ6p WNUb10p/au4xxqEPIWPvD50rTQ== X-Google-Smtp-Source: AGHT+IG26Qla65j//2waJk5p+C3Wp1IkFGp+wjCfVC7nMqMP8R+PELrHfGujdUavK8O0HgXO8xjo4Q== X-Received: by 2002:a5d:4586:0:b0:321:5e2f:37e1 with SMTP id p6-20020a5d4586000000b003215e2f37e1mr13248647wrq.19.1696847789459; Mon, 09 Oct 2023 03:36:29 -0700 (PDT) Received: from vingu-book.. ([2a01:e0a:f:6020:53f1:24bc:5e47:821d]) by smtp.gmail.com with ESMTPSA id f16-20020adfdb50000000b0031ff89af0e4sm9226722wrj.99.2023.10.09.03.36.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Oct 2023 03:36:28 -0700 (PDT) From: Vincent Guittot To: linux@armlinux.org.uk, catalin.marinas@arm.com, will@kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, sudeep.holla@arm.com, gregkh@linuxfoundation.org, rafael@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, viresh.kumar@linaro.org, lukasz.luba@arm.com, ionela.voinescu@arm.com, pierre.gondois@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-pm@vger.kernel.org Cc: conor.dooley@microchip.com, suagrfillet@gmail.com, ajones@ventanamicro.com, lftan@kernel.org, Vincent Guittot Subject: [PATCH v2 3/6] cpufreq: use the fixed and coherent frequency for scaling capacity Date: Mon, 9 Oct 2023 12:36:18 +0200 Message-Id: <20231009103621.374412-4-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231009103621.374412-1-vincent.guittot@linaro.org> References: <20231009103621.374412-1-vincent.guittot@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org cpuinfo.max_freq can change at runtime because of boost as an example. This implies that the value could be different from the frequency that has been used to compute the capacity of a CPU. The new arch_scale_freq_ref() returns a fixed and coherent frequency that can be used to compute the capacity for a given frequency. Signed-off-by: Vincent Guittot Acked-by: Viresh Kumar Reviewed-by: Lukasz Luba --- drivers/cpufreq/cpufreq.c | 4 ++-- include/linux/cpufreq.h | 9 +++++++++ 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c index 60ed89000e82..8c4f9c2f9c44 100644 --- a/drivers/cpufreq/cpufreq.c +++ b/drivers/cpufreq/cpufreq.c @@ -454,7 +454,7 @@ void cpufreq_freq_transition_end(struct cpufreq_policy *policy, arch_set_freq_scale(policy->related_cpus, policy->cur, - policy->cpuinfo.max_freq); + arch_scale_freq_ref(policy->cpu)); spin_lock(&policy->transition_lock); policy->transition_ongoing = false; @@ -2174,7 +2174,7 @@ unsigned int cpufreq_driver_fast_switch(struct cpufreq_policy *policy, policy->cur = freq; arch_set_freq_scale(policy->related_cpus, freq, - policy->cpuinfo.max_freq); + arch_scale_freq_ref(policy->cpu)); cpufreq_stats_record_transition(policy, freq); if (trace_cpu_frequency_enabled()) { diff --git a/include/linux/cpufreq.h b/include/linux/cpufreq.h index 71d186d6933a..bbc483b4b6e5 100644 --- a/include/linux/cpufreq.h +++ b/include/linux/cpufreq.h @@ -1211,6 +1211,15 @@ void arch_set_freq_scale(const struct cpumask *cpus, { } #endif + +#ifndef arch_scale_freq_ref +static __always_inline +unsigned int arch_scale_freq_ref(int cpu) +{ + return 0; +} +#endif + /* the following are really really optional */ extern struct freq_attr cpufreq_freq_attr_scaling_available_freqs; extern struct freq_attr cpufreq_freq_attr_scaling_boost_freqs; From patchwork Mon Oct 9 10:36:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 13413372 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8ECFDE95A91 for ; Mon, 9 Oct 2023 10:36:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345922AbjJIKgh (ORCPT ); Mon, 9 Oct 2023 06:36:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47072 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345910AbjJIKge (ORCPT ); Mon, 9 Oct 2023 06:36:34 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 576FEC6 for ; Mon, 9 Oct 2023 03:36:32 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-4064876e8b8so42346745e9.0 for ; Mon, 09 Oct 2023 03:36:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1696847791; x=1697452591; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RmtdGUvSawQ4q9KDGPlbeId1/iTLhEjjDMVIwSogQm4=; b=BoQxuiosFjVPZt8rIcS2jhV7/cOVSW3fBHkNztfizdSO555RdCP+O62fPV2dAzkqy3 6gRLD5DdtjvXAlT5j5SNvqku6Bk0Y8KRTu3YOoYmj1H0+99gVGSo8YR/kpP1k2uetAOH GJEU8vOxXy7MYI0OPlghIoPpyMigXVv9+MyrEuqzAu9C8e/35KffZ03iMcvtsmTIIqGY DXz/y9Tg1YzPNZYIHRtPJrVL8QwSH/AbYuU2EvsKfPjFG2IolWIa8VE+KMioqA67CHzR sFOohL1SEGzZrUVFI0I3QF0DP0GXG2+c2HM5TNk/HMXxtKfyObT/vhE2Rm+C8mp/c70w vPRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696847791; x=1697452591; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RmtdGUvSawQ4q9KDGPlbeId1/iTLhEjjDMVIwSogQm4=; b=Ul4Uam6IGg2/Gkx1UpaGb/pZYXd05c9Bs8ze9z7akZJoYRindIj6bcPZGglrczaoFc huCg+5igysrBOFasfSwY78lZzX1YuzmgXS/Czq0ubEmob5vIWVT441glCVZ19Se9OrhQ qFXpgjF0s8/aY9Eu3Xk0Mq7CJ1hsvChQYoy9Qfm+eYKZ2avKM6I9RLqZXyj0lCnX0N6x MQzQqiKJGOols/swWz9dZrBM9hM+M/q5tnB7NR+qEIZdTCOmhEqc4+Sh+/pTH63YwYsd u6eYRhNygTS+hJ51yd9VYrSELM6HyOJk6SQzbNiam1AN8g+UcAG0UHc8kLpVXlmcNcDX MLJg== X-Gm-Message-State: AOJu0YzUnKtB2nrH9LApOv59UR+s+MYXidAZhlle6WTRIdiyQXzurxua r1nQ7QSQeeQJ/F2P9cclYqlsgw== X-Google-Smtp-Source: AGHT+IFsDHg/Rs8MQPYQNA1a7DXBKSokGpRKvXSJgz2PNurvi4j34eIaCQtgEUCXe0uK4YQzK2pDKA== X-Received: by 2002:a05:6000:91:b0:320:67:5580 with SMTP id m17-20020a056000009100b0032000675580mr12443168wrx.5.1696847790816; Mon, 09 Oct 2023 03:36:30 -0700 (PDT) Received: from vingu-book.. ([2a01:e0a:f:6020:53f1:24bc:5e47:821d]) by smtp.gmail.com with ESMTPSA id f16-20020adfdb50000000b0031ff89af0e4sm9226722wrj.99.2023.10.09.03.36.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Oct 2023 03:36:30 -0700 (PDT) From: Vincent Guittot To: linux@armlinux.org.uk, catalin.marinas@arm.com, will@kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, sudeep.holla@arm.com, gregkh@linuxfoundation.org, rafael@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, viresh.kumar@linaro.org, lukasz.luba@arm.com, ionela.voinescu@arm.com, pierre.gondois@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-pm@vger.kernel.org Cc: conor.dooley@microchip.com, suagrfillet@gmail.com, ajones@ventanamicro.com, lftan@kernel.org, Vincent Guittot Subject: [PATCH v2 4/6] cpufreq/schedutil: use a fixed reference frequency Date: Mon, 9 Oct 2023 12:36:19 +0200 Message-Id: <20231009103621.374412-5-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231009103621.374412-1-vincent.guittot@linaro.org> References: <20231009103621.374412-1-vincent.guittot@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org cpuinfo.max_freq can change at runtime because of boost as an example. This implies that the value could be different than the one that has been used when computing the capacity of a CPU. The new arch_scale_freq_ref() returns a fixed and coherent reference frequency that can be used when computing a frequency based on utilization. Use this arch_scale_freq_ref() when available and fallback to policy otherwise. Signed-off-by: Vincent Guittot Reviewed-by: Lukasz Luba --- kernel/sched/cpufreq_schedutil.c | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index 4492608b7d7f..1fa7e74add8f 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -114,6 +114,28 @@ static void sugov_deferred_update(struct sugov_policy *sg_policy) } } +/** + * cpufreq_get_capacity_ref_freq - get the reference frequency of a given CPU that + * has been used to correlate frequency and compute capacity. + * @policy: the cpufreq policy of the CPU in question. + * @use_current: Fallback to current freq instead of policy->cpuinfo.max_freq. + * + * Return: the reference CPU frequency to compute a capacity. + */ +static __always_inline +unsigned long get_capacity_ref_freq(struct cpufreq_policy *policy) +{ + unsigned int freq = arch_scale_freq_ref(policy->cpu); + + if (freq) + return freq; + + if (arch_scale_freq_invariant()) + return policy->cpuinfo.max_freq; + + return policy->cur; +} + /** * get_next_freq - Compute a new frequency for a given cpufreq policy. * @sg_policy: schedutil policy object to compute the new frequency for. @@ -140,10 +162,10 @@ static unsigned int get_next_freq(struct sugov_policy *sg_policy, unsigned long util, unsigned long max) { struct cpufreq_policy *policy = sg_policy->policy; - unsigned int freq = arch_scale_freq_invariant() ? - policy->cpuinfo.max_freq : policy->cur; + unsigned int freq; util = map_util_perf(util); + freq = get_capacity_ref_freq(policy); freq = map_util_freq(util, freq, max); if (freq == sg_policy->cached_raw_freq && !sg_policy->need_freq_update) From patchwork Mon Oct 9 10:36:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 13413373 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91715E95A91 for ; Mon, 9 Oct 2023 10:36:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346112AbjJIKgo (ORCPT ); Mon, 9 Oct 2023 06:36:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47072 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345870AbjJIKgf (ORCPT ); Mon, 9 Oct 2023 06:36:35 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2C5FDE for ; Mon, 9 Oct 2023 03:36:33 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id ffacd0b85a97d-3247d69ed2cso4201715f8f.0 for ; Mon, 09 Oct 2023 03:36:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1696847792; x=1697452592; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FXD0QUw+ffX/G//mljsP8uVRxSPaNtugfzKgqO7nW9U=; b=hnyiZ8kx7BATKD47TcsSy9fXoBlQiTLbElApM1luHBCfrVTchy3UUaK2iIBYlMQgFU k/TqLI7PtsnQNeNUPMZdf9ESFq8DoMy/mhlSehvtVc/NUYLTt6gehjOPQjYX3+jXGhzY PnF9eMW5fHW7eTxgrzlJ2CQ3ppyzfISqPHzJUW1KyrMLooc04n+SQb0nyRp9r+Ysc3fO 1Tn6QhMDa578reGzRaW71apGOiFvc5X4+yUooK6TtrsqKHzb2okpmj5Jx+qrplm7Hepg JfixR2uoI842zbNcRRV+7RWaXNoI6UO+UubLfm7feyxF5d8XOPj3iydEjojuDMvNxfFd BF5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696847792; x=1697452592; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FXD0QUw+ffX/G//mljsP8uVRxSPaNtugfzKgqO7nW9U=; b=izeQwDTKeZ8r0UM+LkvQ886ZhRjLbirsTujcGmPJUPLF621v7p9RNOsPNpNfBwwozU Qkyp/lNW7nmDT1T+EwSgvgWP4Hh7dZACCORElAOaa3dGtpM5m+r7lZGhfqa1TrLYQHgQ mZuR5tCYcb56QdYLNvlel+V7EdzJ+kX7bt+G1d9Rsy6e4Q6OOBypie5YwGe/gyZbeBHx HMJTwcKOyP227vg4Sr4g6UJiz9GH463Payx/FsljHGn6Mnf3WxNNVuoo44gOPByRJgf/ KVxn62kFLnHBT/UwN0XJKQtXkWmQUQGgKnRRzVj50Ym4UFQ18Ye9BXc9+VaiDdTPM3GP TC3g== X-Gm-Message-State: AOJu0YxMAbc7LIJ3a0MFbTbBPiyTkcpPIpTfjNOjZ6jcXYfEN3KC6eKr oQIW58DmaRfz17if6s1CoO+cJg== X-Google-Smtp-Source: AGHT+IHr3aX+9E7V3PEwqGZuQkX9W7nIP+EyPQGbKJ+lRYHFakcvBtj79RtXlZzvZ8DwRc3af55fNw== X-Received: by 2002:adf:ed10:0:b0:320:933:4524 with SMTP id a16-20020adfed10000000b0032009334524mr13404865wro.2.1696847792183; Mon, 09 Oct 2023 03:36:32 -0700 (PDT) Received: from vingu-book.. ([2a01:e0a:f:6020:53f1:24bc:5e47:821d]) by smtp.gmail.com with ESMTPSA id f16-20020adfdb50000000b0031ff89af0e4sm9226722wrj.99.2023.10.09.03.36.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Oct 2023 03:36:31 -0700 (PDT) From: Vincent Guittot To: linux@armlinux.org.uk, catalin.marinas@arm.com, will@kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, sudeep.holla@arm.com, gregkh@linuxfoundation.org, rafael@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, viresh.kumar@linaro.org, lukasz.luba@arm.com, ionela.voinescu@arm.com, pierre.gondois@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-pm@vger.kernel.org Cc: conor.dooley@microchip.com, suagrfillet@gmail.com, ajones@ventanamicro.com, lftan@kernel.org, Vincent Guittot Subject: [PATCH v2 5/6] energy_model: use a fixed reference frequency Date: Mon, 9 Oct 2023 12:36:20 +0200 Message-Id: <20231009103621.374412-6-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231009103621.374412-1-vincent.guittot@linaro.org> References: <20231009103621.374412-1-vincent.guittot@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org The last item of a performance domain is not always the performance point that has been used to compute CPU's capacity. This can lead to different target frequency compared with other part of the system like schedutil and would result in wrong energy estimation. A new arch_scale_freq_ref() is available to return a fixed and coherent frequency reference that can be used when computing the CPU's frequency for an level of utilization. Use this function to get this reference frequency. Energy model is never used without defining arch_scale_freq_ref() but can be compiled. Define a default arch_scale_freq_ref() returning 0 in such case. Signed-off-by: Vincent Guittot Reviewed-by: Lukasz Luba --- include/linux/energy_model.h | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/include/linux/energy_model.h b/include/linux/energy_model.h index b9caa01dfac4..1b0c8490d4bd 100644 --- a/include/linux/energy_model.h +++ b/include/linux/energy_model.h @@ -204,6 +204,14 @@ struct em_perf_state *em_pd_get_efficient_state(struct em_perf_domain *pd, return ps; } +#ifndef arch_scale_freq_ref +static __always_inline +unsigned int arch_scale_freq_ref(int cpu) +{ + return 0; +} +#endif + /** * em_cpu_energy() - Estimates the energy consumed by the CPUs of a * performance domain @@ -224,7 +232,7 @@ static inline unsigned long em_cpu_energy(struct em_perf_domain *pd, unsigned long max_util, unsigned long sum_util, unsigned long allowed_cpu_cap) { - unsigned long freq, scale_cpu; + unsigned long freq, ref_freq, scale_cpu; struct em_perf_state *ps; int cpu; @@ -241,11 +249,11 @@ static inline unsigned long em_cpu_energy(struct em_perf_domain *pd, */ cpu = cpumask_first(to_cpumask(pd->cpus)); scale_cpu = arch_scale_cpu_capacity(cpu); - ps = &pd->table[pd->nr_perf_states - 1]; + ref_freq = arch_scale_freq_ref(cpu); max_util = map_util_perf(max_util); max_util = min(max_util, allowed_cpu_cap); - freq = map_util_freq(max_util, ps->frequency, scale_cpu); + freq = map_util_freq(max_util, ref_freq, scale_cpu); /* * Find the lowest performance state of the Energy Model above the From patchwork Mon Oct 9 10:36:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 13413374 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B849AE95A96 for ; Mon, 9 Oct 2023 10:36:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346134AbjJIKgp (ORCPT ); Mon, 9 Oct 2023 06:36:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34682 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345926AbjJIKgh (ORCPT ); Mon, 9 Oct 2023 06:36:37 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C1F4BA for ; Mon, 9 Oct 2023 03:36:35 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id ffacd0b85a97d-3296b49c546so2468782f8f.3 for ; Mon, 09 Oct 2023 03:36:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1696847793; x=1697452593; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zLqmJToKVvGx3RPF93C/D7OTDNmDiwUSULcdREcteNg=; b=V4ndD/FvTN8lDLyDZ+iL1ssnIQXGKkAxLC7Gh+qcxuxRwg/FhAD/TVT73fKADskQrU 1cveajp1LH72QU65QuUopll3bPyK9bDBOJzUAmTmoa3nZluUL7vesFFIU+SQQR/KkuPr 8m08QwvP9FKNhgHIVeNVG2Tt0F7BqhbJHr0F7duxV63YusafmwMjcKyO6ZK3axtUsZqo w6XTcqnQ+3blMZxf4mH+kQlo60LS+gNWeoaOCXY2kmIQFg75gPCpMyc+NfeFtkhM8ijY l4L5NejI8U/JD8NdUPBtpAmSVxw4F46tlb5HhFn7m+0/Rs8dWJz1T9wSMBZ6qSoOEcyH csog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696847793; x=1697452593; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zLqmJToKVvGx3RPF93C/D7OTDNmDiwUSULcdREcteNg=; b=X6uaqMG0OUWfkuuadLDkQssUzchu1mGzp4EuNc5A8+0W71RUklrp8MIw/+L4Xfk4if GCWGWCBaZSCUxhHMdODNbI264SJJe/aMJbVlNT0ynrtWJAV5xTMtnQRoXMYWysMv8vIs i4YdReoRcZr+xwUieoR1JYXj8iPS2eUU5vNS6IO13osmVKN69hosFFEKVWFHF4ww0cCF KN2dTYV0VYeZfu8sbrhcN3AD33aKB/MfWeNEIBocpler50c43pREOqWL7WKum1ai/na2 aV/w0e5BPtTYK8+66SDRdvX/Q24r8doBBu0uf9PYv3afuSHDCrN21vg0/IwZIVSltPdy cPHw== X-Gm-Message-State: AOJu0Ywmb9s4nodaJwhlG1LW54UBktdgRY72MNjdOYQHl8mGuwjAZfbj zxp4ToqxBJNBAutmK+926I7DqA== X-Google-Smtp-Source: AGHT+IE1c9bbdHMwBLMomM9sWbKgUcq5hfVG4KlvDcosRRJkN4Ti2TteiXopQtpHeenYl1QcJ2PxJw== X-Received: by 2002:a5d:614b:0:b0:31c:8c93:61e3 with SMTP id y11-20020a5d614b000000b0031c8c9361e3mr12270603wrt.60.1696847793608; Mon, 09 Oct 2023 03:36:33 -0700 (PDT) Received: from vingu-book.. ([2a01:e0a:f:6020:53f1:24bc:5e47:821d]) by smtp.gmail.com with ESMTPSA id f16-20020adfdb50000000b0031ff89af0e4sm9226722wrj.99.2023.10.09.03.36.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Oct 2023 03:36:33 -0700 (PDT) From: Vincent Guittot To: linux@armlinux.org.uk, catalin.marinas@arm.com, will@kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, sudeep.holla@arm.com, gregkh@linuxfoundation.org, rafael@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, viresh.kumar@linaro.org, lukasz.luba@arm.com, ionela.voinescu@arm.com, pierre.gondois@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-pm@vger.kernel.org Cc: conor.dooley@microchip.com, suagrfillet@gmail.com, ajones@ventanamicro.com, lftan@kernel.org, Vincent Guittot Subject: [PATCH v2 6/6] cpufreq/cppc: set the frequency used for capacity computation Date: Mon, 9 Oct 2023 12:36:21 +0200 Message-Id: <20231009103621.374412-7-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231009103621.374412-1-vincent.guittot@linaro.org> References: <20231009103621.374412-1-vincent.guittot@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org cppc cpufreq driver can register an artificial energy model. In such case, it also have to register the frequency that is used to define the CPU capacity Signed-off-by: Vincent Guittot --- drivers/cpufreq/cppc_cpufreq.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/drivers/cpufreq/cppc_cpufreq.c b/drivers/cpufreq/cppc_cpufreq.c index fe08ca419b3d..24c6ba349f01 100644 --- a/drivers/cpufreq/cppc_cpufreq.c +++ b/drivers/cpufreq/cppc_cpufreq.c @@ -636,6 +636,21 @@ static int populate_efficiency_class(void) return 0; } + +static void cppc_cpufreq_set_capacity_ref_freq(struct cpufreq_policy *policy) +{ + struct cppc_perf_caps *perf_caps; + struct cppc_cpudata *cpu_data; + unsigned int ref_freq; + + cpu_data = policy->driver_data; + perf_caps = &cpu_data->perf_caps; + + ref_freq = cppc_cpufreq_perf_to_khz(cpu_data, perf_caps->highest_perf); + + per_cpu(capacity_ref_freq, policy->cpu) = ref_freq; +} + static void cppc_cpufreq_register_em(struct cpufreq_policy *policy) { struct cppc_cpudata *cpu_data; @@ -643,6 +658,9 @@ static void cppc_cpufreq_register_em(struct cpufreq_policy *policy) EM_ADV_DATA_CB(cppc_get_cpu_power, cppc_get_cpu_cost); cpu_data = policy->driver_data; + + cppc_cpufreq_set_capacity_ref_freq(policy); + em_dev_register_perf_domain(get_cpu_device(policy->cpu), get_perf_level_count(policy), &em_cb, cpu_data->shared_cpu_map, 0);