From patchwork Fri Apr 5 13:33:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Beata Michalska X-Patchwork-Id: 13619066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CFD1BC67861 for ; Fri, 5 Apr 2024 13:34:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=rmx8tCT/ZPruFe5E2115BPrUdTXyR/W6eODIgnoONKU=; b=tQu498+/YtbcQq f+ylRh75XCzlxESPj3qM3neP117BjeTcMIPWvSuwCEBy7NUB41Z6byDCzcUdQdGx2ZDVVgSQw52dc jW4+Gkg04jDVjOx04ccqEk+rQyK5Ri0r/UU9kloEgjYgQE9zt9x0YUCNFpS62SCDM06ossHLDzBj9 3jpEwT1SdR9V1Vdqlx0LdW45YASBAOwkufQDowwFtItzgMkpC07/jKu+HRkPtbXH0zJ284G0Mh6CM EIudnaic3iccb7FspvQLPECVhf4QtsOD8cUOpb9AiJC1MvYBz01W4dRIqYDssevvhnWM7F78VvBzk 7Y/gpPZ2BMan/+IVxKRg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rsjhX-00000007EzL-1uL5; Fri, 05 Apr 2024 13:33:51 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rsjhS-00000007Ett-3CLA for linux-arm-kernel@lists.infradead.org; Fri, 05 Apr 2024 13:33:48 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C7BE81007; Fri, 5 Apr 2024 06:34:11 -0700 (PDT) Received: from e125905.cambridge.arm.com (e125905.cambridge.arm.com [10.1.194.73]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id BD0503F7B4; Fri, 5 Apr 2024 06:33:39 -0700 (PDT) From: Beata Michalska To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, ionela.voinescu@arm.com, vanshikonda@os.amperecomputing.com Cc: sudeep.holla@arm.com, will@kernel.org, catalin.marinas@arm.com, vincent.guittot@linaro.org, sumitg@nvidia.com, yang@os.amperecomputing.com, lihuisong@huawei.com Subject: [PATCH v4 1/4] arch_topology: init capacity_freq_ref to 0 Date: Fri, 5 Apr 2024 14:33:16 +0100 Message-Id: <20240405133319.859813-2-beata.michalska@arm.com> In-Reply-To: <20240405133319.859813-1-beata.michalska@arm.com> References: <20240405133319.859813-1-beata.michalska@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240405_063346_950909_C3DEFCB1 X-CRM114-Status: UNSURE ( 9.83 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ionela Voinescu It's useful to have capacity_freq_ref initialized to 0 for users of arch_scale_freq_ref() to detect when capacity_freq_ref was not yet set. The only scenario affected by this change in the init value is when a cpufreq driver is never loaded. As a result, the only setter of a cpu scale factor remains the call of topology_normalize_cpu_scale() from parse_dt_topology(). There we cannot use the value 0 of capacity_freq_ref so we have to compensate for its uninitialized state. Signed-off-by: Ionela Voinescu Signed-off-by: Beata Michalska Reviewed-by: Vincent Guittot --- drivers/base/arch_topology.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c index 024b78a0cfc1..7d4c92cd2bad 100644 --- a/drivers/base/arch_topology.c +++ b/drivers/base/arch_topology.c @@ -27,7 +27,7 @@ static DEFINE_PER_CPU(struct scale_freq_data __rcu *, sft_data); static struct cpumask scale_freq_counters_mask; static bool scale_freq_invariant; -DEFINE_PER_CPU(unsigned long, capacity_freq_ref) = 1; +DEFINE_PER_CPU(unsigned long, capacity_freq_ref) = 0; EXPORT_PER_CPU_SYMBOL_GPL(capacity_freq_ref); static bool supports_scale_freq_counters(const struct cpumask *cpus) @@ -292,13 +292,15 @@ void topology_normalize_cpu_scale(void) capacity_scale = 1; for_each_possible_cpu(cpu) { - capacity = raw_capacity[cpu] * per_cpu(capacity_freq_ref, cpu); + capacity = raw_capacity[cpu] * + (per_cpu(capacity_freq_ref, cpu) ?: 1); capacity_scale = max(capacity, capacity_scale); } pr_debug("cpu_capacity: capacity_scale=%llu\n", capacity_scale); for_each_possible_cpu(cpu) { - capacity = raw_capacity[cpu] * per_cpu(capacity_freq_ref, cpu); + capacity = raw_capacity[cpu] * + (per_cpu(capacity_freq_ref, cpu) ?: 1); capacity = div64_u64(capacity << SCHED_CAPACITY_SHIFT, capacity_scale); topology_set_cpu_scale(cpu, capacity); From patchwork Fri Apr 5 13:33:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Beata Michalska X-Patchwork-Id: 13619067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 908D8CD11C2 for ; Fri, 5 Apr 2024 13:34:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=6oZpXRF/3W1bMGGMMnqcfPS83aXfBY5qBbbssu1WEIs=; b=YPpyiW8UDEL+Wm 8YWzuYH01eU6llrtBWi+wdFZ3QpOttYWDhgBdwQecs/YVGBbPi8l6ONzRO5IwZWqjfEMpwrITPLfT SDTLbWAzQT1LLP7q2CRNqSzzJFZK5laSeH2R8MDzyGuQlImVM7sY+D7lJSt0rf9hMBsJIVe7z6Vm/ PiqVYNcU8PZlDPDMWhKPGjyTC6j5ppTscwKSugv1thW9qA2PtqsdwgRo8XwmC0VXdomI7qrN2PU7R Gvh/9m+xeuSSFUYe+NPUR9IKhvN9fe18NEM1mEfpA67ySVYDHrNX08f65cFmmSzenjiY4wxO2/hXF GQr+7TFqhBTVK9FgsRug==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rsjhW-00000007Eym-2S3o; Fri, 05 Apr 2024 13:33:50 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rsjhS-00000007Euv-35tg for linux-arm-kernel@lists.infradead.org; Fri, 05 Apr 2024 13:33:48 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6EEB8169C; Fri, 5 Apr 2024 06:34:13 -0700 (PDT) Received: from e125905.cambridge.arm.com (e125905.cambridge.arm.com [10.1.194.73]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 645213F7B4; Fri, 5 Apr 2024 06:33:41 -0700 (PDT) From: Beata Michalska To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, ionela.voinescu@arm.com, vanshikonda@os.amperecomputing.com Cc: sudeep.holla@arm.com, will@kernel.org, catalin.marinas@arm.com, vincent.guittot@linaro.org, sumitg@nvidia.com, yang@os.amperecomputing.com, lihuisong@huawei.com Subject: [PATCH v4 2/4] arm64: Provide an AMU-based version of arch_freq_get_on_cpu Date: Fri, 5 Apr 2024 14:33:17 +0100 Message-Id: <20240405133319.859813-3-beata.michalska@arm.com> In-Reply-To: <20240405133319.859813-1-beata.michalska@arm.com> References: <20240405133319.859813-1-beata.michalska@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240405_063346_920871_6DD62934 X-CRM114-Status: GOOD ( 19.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With the Frequency Invariance Engine (FIE) being already wired up with sched tick and making use of relevant (core counter and constant counter) AMU counters, getting the current frequency for a given CPU, can be achieved by utilizing the frequency scale factor which reflects an average CPU frequency for the last tick period length. The solution is partially based on APERF/MPERF implementation of arch_freq_get_on_cpu. Suggested-by: Ionela Voinescu Signed-off-by: Beata Michalska --- arch/arm64/kernel/topology.c | 112 +++++++++++++++++++++++++++++++---- 1 file changed, 102 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index 1a2c72f3e7f8..b03fe8617721 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include @@ -88,18 +89,28 @@ int __init parse_acpi_topology(void) * initialized. */ static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) = 1UL << (2 * SCHED_CAPACITY_SHIFT); -static DEFINE_PER_CPU(u64, arch_const_cycles_prev); -static DEFINE_PER_CPU(u64, arch_core_cycles_prev); static cpumask_var_t amu_fie_cpus; +struct amu_cntr_sample { + u64 arch_const_cycles_prev; + u64 arch_core_cycles_prev; + unsigned long last_update; +}; + +static DEFINE_PER_CPU_SHARED_ALIGNED(struct amu_cntr_sample, cpu_amu_samples); + void update_freq_counters_refs(void) { - this_cpu_write(arch_core_cycles_prev, read_corecnt()); - this_cpu_write(arch_const_cycles_prev, read_constcnt()); + struct amu_cntr_sample *amu_sample = this_cpu_ptr(&cpu_amu_samples); + + amu_sample->arch_core_cycles_prev = read_corecnt(); + amu_sample->arch_const_cycles_prev = read_constcnt(); } static inline bool freq_counters_valid(int cpu) { + struct amu_cntr_sample *amu_sample = per_cpu_ptr(&cpu_amu_samples, cpu); + if ((cpu >= nr_cpu_ids) || !cpumask_test_cpu(cpu, cpu_present_mask)) return false; @@ -108,8 +119,8 @@ static inline bool freq_counters_valid(int cpu) return false; } - if (unlikely(!per_cpu(arch_const_cycles_prev, cpu) || - !per_cpu(arch_core_cycles_prev, cpu))) { + if (unlikely(!amu_sample->arch_const_cycles_prev || + !amu_sample->arch_core_cycles_prev)) { pr_debug("CPU%d: cycle counters are not enabled.\n", cpu); return false; } @@ -152,17 +163,22 @@ void freq_inv_set_max_ratio(int cpu, u64 max_rate) static void amu_scale_freq_tick(void) { + struct amu_cntr_sample *amu_sample = this_cpu_ptr(&cpu_amu_samples); u64 prev_core_cnt, prev_const_cnt; u64 core_cnt, const_cnt, scale; - prev_const_cnt = this_cpu_read(arch_const_cycles_prev); - prev_core_cnt = this_cpu_read(arch_core_cycles_prev); + prev_const_cnt = amu_sample->arch_const_cycles_prev; + prev_core_cnt = amu_sample->arch_core_cycles_prev; update_freq_counters_refs(); - const_cnt = this_cpu_read(arch_const_cycles_prev); - core_cnt = this_cpu_read(arch_core_cycles_prev); + const_cnt = amu_sample->arch_const_cycles_prev; + core_cnt = amu_sample->arch_core_cycles_prev; + /* + * This should not happen unless the AMUs have been reset and the + * counter values have not been restored - unlikely + */ if (unlikely(core_cnt <= prev_core_cnt || const_cnt <= prev_const_cnt)) return; @@ -182,6 +198,8 @@ static void amu_scale_freq_tick(void) scale = min_t(unsigned long, scale, SCHED_CAPACITY_SCALE); this_cpu_write(arch_freq_scale, (unsigned long)scale); + + amu_sample->last_update = jiffies; } static struct scale_freq_data amu_sfd = { @@ -189,6 +207,80 @@ static struct scale_freq_data amu_sfd = { .set_freq_scale = amu_scale_freq_tick, }; +#define AMU_SAMPLE_EXP_MS 20 + +unsigned int arch_freq_get_on_cpu(int cpu) +{ + struct amu_cntr_sample *amu_sample; + cpumask_var_t ref_cpumask = NULL; + unsigned long last_update; + unsigned int freq; + u64 scale; + + if (!cpumask_test_cpu(cpu, amu_fie_cpus) || !arch_scale_freq_ref(cpu)) + return 0; +retry: + amu_sample = per_cpu_ptr(&cpu_amu_samples, cpu); + + last_update = amu_sample->last_update; + + /* + * For those CPUs that are in full dynticks mode, + * and those that have not seen tick for a while + * try an alternative source for the counters (and thus freq scale), + * if available, for given policy: + * this boils down to identifying an active cpu within the same freq + * domain, if any. + */ + if (!housekeeping_cpu(cpu, HK_TYPE_TICK) || + time_is_before_jiffies(last_update + msecs_to_jiffies(AMU_SAMPLE_EXP_MS))) { + struct cpufreq_policy *policy = cpufreq_cpu_get(cpu); + int ref_cpu = cpu; + + if (!policy_is_shared(policy)) { + cpufreq_cpu_put(policy); + return 0; + } + + if (!ref_cpumask) { + if (!zalloc_cpumask_var(&ref_cpumask, GFP_KERNEL)) { + cpufreq_cpu_put(policy); + return 0; + } + + cpumask_copy(ref_cpumask, policy->cpus); + } + + cpufreq_cpu_put(policy); + + do { + cpumask_clear_cpu(ref_cpu, ref_cpumask); + ref_cpu = cpumask_first(ref_cpumask); + + } while (ref_cpu < nr_cpu_ids && idle_cpu(ref_cpu)); + + if (ref_cpu >= nr_cpu_ids) { + /* No alternative to pull info from */ + free_cpumask_var(ref_cpumask); + return 0; + } + cpu = ref_cpu; + goto retry; + } + /* + * Reversed computation to the one used to determine + * the arch_freq_scale value + * (see amu_scale_freq_tick for details) + */ + scale = arch_scale_freq_capacity(cpu); + freq = scale * arch_scale_freq_ref(cpu); + freq >>= SCHED_CAPACITY_SHIFT; + + free_cpumask_var(ref_cpumask); + + return freq; +} + static void amu_fie_setup(const struct cpumask *cpus) { int cpu; From patchwork Fri Apr 5 13:33:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Beata Michalska X-Patchwork-Id: 13619069 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4F0CDC67861 for ; Fri, 5 Apr 2024 13:34:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Oviw1/VROIJaNRi9NmRu+X5l3gE92AchzHCq0fd/5vI=; b=VSyvaph/yD6p8G ZraoDyUFsRI/rt8v5dIVt9LRBsz5fmuMF9nsFeuZTaa4XhLRCMq/yUChcJDu8fT1V5voqnKsHh2Rh rwqLasHb9nu9kZZzfzw9nVXqWPwZXvPJTVnkmCR0EWqVOOWWgnLR95q54CyobKVF4W0W15wJx2VJx rL2iDZV90DKZdZay/fPZ6tgXM3+NBInVwZuOHPlmJhj0ysHc+EGx5SkA6BQvMn6vGA9qaxpAkGVjR 73QN8MqMDo7ORXOX/Mp3QqRXctjdHjTgRdFoYklFiYKEEJJZlPbv/PuYIzgJ/u4vLjQP1XWZx4Wx/ daKY6Jj+0bryYoVZSX7g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rsjhf-00000007F5I-0Trf; Fri, 05 Apr 2024 13:33:59 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rsjhZ-00000007F0h-3Dfh for linux-arm-kernel@bombadil.infradead.org; Fri, 05 Apr 2024 13:33:53 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=5FPqy/3OnEPmkXiN7TlqbcUR52euURW8awPVaYrH47E=; b=BKav2d6yEtrNFBsJz4ZYDvDjum 6MExDUemk9rT944XHmL+ExNh5B+LucueSSz+SMtm6xPrZ2eE6WVNaBNZHeMN2ucKAefKdxf9Xm8VU L3pC7S1GzVDzUcKiRQXaWnBcDO7Mdo1Eu9a0vpIRczCTrCVjQsLTv2Rlf3qFB0cNvc6km+8TWQwdO Cvsl5sdWsroEa0S4bPuXrcRteAb+2J5TqGXgUcrSYcO7obtBWeI7M2GYRJ0s9Em9qIHJGIj6s2gtc iNE6oCXkrw3tp0o+dkF9mREW0+l+C+0w6ywjq3JwfGcZjICsmjVAxZQ+06+4Py7WIrT+VL+leGq9H vahTw7Xg==; Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rsjhV-00000005dve-0eJq for linux-arm-kernel@lists.infradead.org; Fri, 05 Apr 2024 13:33:52 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 161BB169E; Fri, 5 Apr 2024 06:34:15 -0700 (PDT) Received: from e125905.cambridge.arm.com (e125905.cambridge.arm.com [10.1.194.73]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0B3983F7B4; Fri, 5 Apr 2024 06:33:42 -0700 (PDT) From: Beata Michalska To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, ionela.voinescu@arm.com, vanshikonda@os.amperecomputing.com Cc: sudeep.holla@arm.com, will@kernel.org, catalin.marinas@arm.com, vincent.guittot@linaro.org, sumitg@nvidia.com, yang@os.amperecomputing.com, lihuisong@huawei.com Subject: [PATCH v4 3/4] arm64: Update AMU-based frequency scale factor on entering idle Date: Fri, 5 Apr 2024 14:33:18 +0100 Message-Id: <20240405133319.859813-4-beata.michalska@arm.com> In-Reply-To: <20240405133319.859813-1-beata.michalska@arm.com> References: <20240405133319.859813-1-beata.michalska@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240405_143349_596159_FC8F2AF8 X-CRM114-Status: GOOD ( 12.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that the frequency scale factor has been activated for retrieving current frequency on a given CPU, trigger its update upon entering idle. This will, to an extent, allow querying last known frequency in a non-invasive way. It will also improve the frequency scale factor accuracy when a CPU entering idle did not receive a tick for a while. As a consequence, for idle cores, the reported frequency will be the last one observed before entering the idle state. Suggested-by: Vanshidhar Konda Signed-off-by: Beata Michalska --- arch/arm64/kernel/topology.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index b03fe8617721..f204f6489f98 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -207,6 +207,19 @@ static struct scale_freq_data amu_sfd = { .set_freq_scale = amu_scale_freq_tick, }; +void arch_cpu_idle_enter(void) +{ + unsigned int cpu = smp_processor_id(); + + if (!cpumask_test_cpu(cpu, amu_fie_cpus)) + return; + + /* Kick in AMU update but only if one has not happened already */ + if (housekeeping_cpu(cpu, HK_TYPE_TICK) && + time_is_before_jiffies(per_cpu(cpu_amu_samples.last_update, cpu))) + amu_scale_freq_tick(); +} + #define AMU_SAMPLE_EXP_MS 20 unsigned int arch_freq_get_on_cpu(int cpu) @@ -232,8 +245,8 @@ unsigned int arch_freq_get_on_cpu(int cpu) * this boils down to identifying an active cpu within the same freq * domain, if any. */ - if (!housekeeping_cpu(cpu, HK_TYPE_TICK) || - time_is_before_jiffies(last_update + msecs_to_jiffies(AMU_SAMPLE_EXP_MS))) { + if (!housekeeping_cpu(cpu, HK_TYPE_TICK) || (!idle_cpu(cpu) && + time_is_before_jiffies(last_update + msecs_to_jiffies(AMU_SAMPLE_EXP_MS)))) { struct cpufreq_policy *policy = cpufreq_cpu_get(cpu); int ref_cpu = cpu; From patchwork Fri Apr 5 13:33:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Beata Michalska X-Patchwork-Id: 13619127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DBC05C67861 for ; Fri, 5 Apr 2024 14:35:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JY6urbWrJLIsg7G6X+nVE+ZmNQPs/sEpdOZ8Rfs5MzE=; b=f1XdTI1LH++Jqy Uw5wwW+J4kBScHni6BZYSWRSMV9wAs5QDwSxEYxuUwiR7l1Gw7BMOmKIB4Ml0mCqYtQTcAz6md4gD DUZgHcTr11S21O5rj7lSf2f3kli7KoaWLQna0Wp2p2sf37Al1ECBGsZMnMdrvoVZmQMUjfSSagZJ+ njW/fya0zG6mcn4ItYMlqDO6OAdT8Kr5X1AvLct8QKNpYBYIdQRrziiZde5sBw8UnRRcqxbXI9RZl dl39TW5Ja5+TaOxLOH7FkTuZRfyhJSO17j2epJHY4UQSK/mQJA9Y8GCqvkoPVhRWPJVb6dbxUob1w f0kO0P8Z7/GirXt3u5ug==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rskek-00000007UhP-1yyE; Fri, 05 Apr 2024 14:35:02 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rsjhS-00000007Evk-3UZO for linux-arm-kernel@lists.infradead.org; Fri, 05 Apr 2024 13:33:48 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BAAE916A3; Fri, 5 Apr 2024 06:34:16 -0700 (PDT) Received: from e125905.cambridge.arm.com (e125905.cambridge.arm.com [10.1.194.73]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A65843F7B4; Fri, 5 Apr 2024 06:33:44 -0700 (PDT) From: Beata Michalska To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, ionela.voinescu@arm.com, vanshikonda@os.amperecomputing.com Cc: sudeep.holla@arm.com, will@kernel.org, catalin.marinas@arm.com, vincent.guittot@linaro.org, sumitg@nvidia.com, yang@os.amperecomputing.com, lihuisong@huawei.com Subject: [PATCH v4 4/4] cpufreq: Use arch specific feedback for cpuinfo_cur_freq Date: Fri, 5 Apr 2024 14:33:19 +0100 Message-Id: <20240405133319.859813-5-beata.michalska@arm.com> In-Reply-To: <20240405133319.859813-1-beata.michalska@arm.com> References: <20240405133319.859813-1-beata.michalska@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240405_063347_042555_6115E6C2 X-CRM114-Status: UNSURE ( 9.94 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Some architectures provide a way to determine an average frequency over a certain period of time based on available performance monitors (AMU on ARM or APERF/MPERf on x86). With those at hand, enroll arch_freq_get_on_cpu into cpuinfo_cur_freq policy sysfs attribute handler, which is expected to represent the current frequency of a given CPU, as obtained by the hardware. This is the type of feedback that counters do provide. Signed-off-by: Beata Michalska --- drivers/cpufreq/cpufreq.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c index 66e10a19d76a..603533b2608f 100644 --- a/drivers/cpufreq/cpufreq.c +++ b/drivers/cpufreq/cpufreq.c @@ -795,8 +795,10 @@ store_one(scaling_max_freq, max); static ssize_t show_cpuinfo_cur_freq(struct cpufreq_policy *policy, char *buf) { - unsigned int cur_freq = __cpufreq_get(policy); + unsigned int cur_freq = arch_freq_get_on_cpu(policy->cpu); + if (!cur_freq) + cur_freq = __cpufreq_get(policy); if (cur_freq) return sprintf(buf, "%u\n", cur_freq);