From patchwork Tue Mar 12 08:34:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Beata Michalska X-Patchwork-Id: 13589608 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AB630C54E66 for ; Tue, 12 Mar 2024 08:35:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=rmx8tCT/ZPruFe5E2115BPrUdTXyR/W6eODIgnoONKU=; b=MWhe5X+RmcbWk5 jdL42zbLDwPOTET5z1LR7ERWdLuwUApsZ+lCN722rmUO/Ezj7lqcomdDQ1kEPC/4dQiEjv6/PdOte 11FPCphvX8ahe7OBsgIh5etFTPZldq2yfaSwJg4h71VvnKHmZ4Nnm+gnu0bPo+Li2SeHMPiKhD7X9 19DjotFe6+XIEL1w/adgrW0tOLMLmg2L6CaoL9mgzKVhS5LHSRp8rGJsQLBjqRqGsyirO7llzwerf 8nbQ9losDsIlg5rcUHFfpXwWssVqKabhuF6GJgDugoafXiehlZsKkBh2Sz333+Gf3QgQo2iQhcKe7 CAiJ7KnudxetkZLo+kmg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rjxbP-00000004u72-05Ad; Tue, 12 Mar 2024 08:35:15 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rjxbK-00000004u4o-1xQQ for linux-arm-kernel@lists.infradead.org; Tue, 12 Mar 2024 08:35:11 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 461561570; Tue, 12 Mar 2024 01:35:45 -0700 (PDT) Received: from e125905.cambridge.arm.com (e125905.cambridge.arm.com [10.1.194.73]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6FF2C3F73F; Tue, 12 Mar 2024 01:35:06 -0700 (PDT) From: Beata Michalska To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, ionela.voinescu@arm.com, vanshikonda@os.amperecomputing.com Cc: sudeep.holla@arm.com, will@kernel.org, catalin.marinas@arm.com, vincent.guittot@linaro.org, sumitg@nvidia.com, yang@os.amperecomputing.com, lihuisong@huawei.com Subject: [PATCH v3 1/3] arch_topology: init capacity_freq_ref to 0 Date: Tue, 12 Mar 2024 08:34:29 +0000 Message-Id: <20240312083431.3239989-2-beata.michalska@arm.com> In-Reply-To: <20240312083431.3239989-1-beata.michalska@arm.com> References: <20240312083431.3239989-1-beata.michalska@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240312_013510_574238_A847497A X-CRM114-Status: GOOD ( 10.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ionela Voinescu It's useful to have capacity_freq_ref initialized to 0 for users of arch_scale_freq_ref() to detect when capacity_freq_ref was not yet set. The only scenario affected by this change in the init value is when a cpufreq driver is never loaded. As a result, the only setter of a cpu scale factor remains the call of topology_normalize_cpu_scale() from parse_dt_topology(). There we cannot use the value 0 of capacity_freq_ref so we have to compensate for its uninitialized state. Signed-off-by: Ionela Voinescu Signed-off-by: Beata Michalska --- drivers/base/arch_topology.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c index 024b78a0cfc1..7d4c92cd2bad 100644 --- a/drivers/base/arch_topology.c +++ b/drivers/base/arch_topology.c @@ -27,7 +27,7 @@ static DEFINE_PER_CPU(struct scale_freq_data __rcu *, sft_data); static struct cpumask scale_freq_counters_mask; static bool scale_freq_invariant; -DEFINE_PER_CPU(unsigned long, capacity_freq_ref) = 1; +DEFINE_PER_CPU(unsigned long, capacity_freq_ref) = 0; EXPORT_PER_CPU_SYMBOL_GPL(capacity_freq_ref); static bool supports_scale_freq_counters(const struct cpumask *cpus) @@ -292,13 +292,15 @@ void topology_normalize_cpu_scale(void) capacity_scale = 1; for_each_possible_cpu(cpu) { - capacity = raw_capacity[cpu] * per_cpu(capacity_freq_ref, cpu); + capacity = raw_capacity[cpu] * + (per_cpu(capacity_freq_ref, cpu) ?: 1); capacity_scale = max(capacity, capacity_scale); } pr_debug("cpu_capacity: capacity_scale=%llu\n", capacity_scale); for_each_possible_cpu(cpu) { - capacity = raw_capacity[cpu] * per_cpu(capacity_freq_ref, cpu); + capacity = raw_capacity[cpu] * + (per_cpu(capacity_freq_ref, cpu) ?: 1); capacity = div64_u64(capacity << SCHED_CAPACITY_SHIFT, capacity_scale); topology_set_cpu_scale(cpu, capacity); From patchwork Tue Mar 12 08:34:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Beata Michalska X-Patchwork-Id: 13589610 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EF4AFC54E58 for ; Tue, 12 Mar 2024 08:35:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2q2kWM5KJIvF7XqOOgoDA44V+eJ1CdfUU+ODXVcLkqM=; b=sxHFl24SgVf0oX PfMyPWpVvBynz4eizXfTReXrqW89guEE/s305HZfVlTL7/WUBVIAj/4FhZgn3Wi+GRZzUztq/+22u 3ngdUHrahM4j45abXwGb8bVE5FudwvK/BqeGLEMZZviMe6mgiLnKJrjFE51Y2h33p8lP0xBYa9818 CKgSQrvGXPW2cYMbclWVwtBlM0BSuvIbs+nHT9cn8Dvu07UMA5hzpJxqEhgpWhvT/w/l8erkgwS/Q ssI2QapqIhVcyi2SU6cdcrfqEfw9aw9LG7AmS0FGBXQy+gi/Cxp5W2Y1i5JF5XknXSL99jmSEzp66 QVP+3pFvuEjQ5TCJikCA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rjxbT-00000004u9Y-3dU3; Tue, 12 Mar 2024 08:35:19 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rjxbN-00000004u6h-3QAV for linux-arm-kernel@lists.infradead.org; Tue, 12 Mar 2024 08:35:15 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 38B311595; Tue, 12 Mar 2024 01:35:47 -0700 (PDT) Received: from e125905.cambridge.arm.com (e125905.cambridge.arm.com [10.1.194.73]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 623FC3F73F; Tue, 12 Mar 2024 01:35:08 -0700 (PDT) From: Beata Michalska To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, ionela.voinescu@arm.com, vanshikonda@os.amperecomputing.com Cc: sudeep.holla@arm.com, will@kernel.org, catalin.marinas@arm.com, vincent.guittot@linaro.org, sumitg@nvidia.com, yang@os.amperecomputing.com, lihuisong@huawei.com Subject: [PATCH v3 2/3] arm64: Provide an AMU-based version of arch_freq_get_on_cpu Date: Tue, 12 Mar 2024 08:34:30 +0000 Message-Id: <20240312083431.3239989-3-beata.michalska@arm.com> In-Reply-To: <20240312083431.3239989-1-beata.michalska@arm.com> References: <20240312083431.3239989-1-beata.michalska@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240312_013513_990586_2044C096 X-CRM114-Status: GOOD ( 18.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With the Frequency Invariance Engine (FIE) being already wired up with sched tick and making use of relevant (core counter and constant counter) AMU counters, getting the current frequency for a given CPU on supported platforms can be achieved by utilizing the frequency scale factor which reflects an average CPU frequency for the last tick period length. The solution is partially based on APERF/MPERF implementation of arch_freq_get_on_cpu. Suggested-by: Ionela Voinescu Signed-off-by: Beata Michalska --- arch/arm64/kernel/topology.c | 103 +++++++++++++++++++++++++++++++---- 1 file changed, 92 insertions(+), 11 deletions(-) diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index 1a2c72f3e7f8..42cb19c31719 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -17,6 +17,8 @@ #include #include #include +#include +#include #include #include @@ -88,18 +90,31 @@ int __init parse_acpi_topology(void) * initialized. */ static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) = 1UL << (2 * SCHED_CAPACITY_SHIFT); -static DEFINE_PER_CPU(u64, arch_const_cycles_prev); -static DEFINE_PER_CPU(u64, arch_core_cycles_prev); static cpumask_var_t amu_fie_cpus; +struct amu_cntr_sample { + u64 arch_const_cycles_prev; + u64 arch_core_cycles_prev; + unsigned long last_update; + seqcount_t seq; +}; + +static DEFINE_PER_CPU_SHARED_ALIGNED(struct amu_cntr_sample, cpu_amu_samples) = { + .seq = SEQCNT_ZERO(cpu_amu_samples.seq) +}; + void update_freq_counters_refs(void) { - this_cpu_write(arch_core_cycles_prev, read_corecnt()); - this_cpu_write(arch_const_cycles_prev, read_constcnt()); + struct amu_cntr_sample *amu_sample = this_cpu_ptr(&cpu_amu_samples); + + amu_sample->arch_core_cycles_prev = read_corecnt(); + amu_sample->arch_const_cycles_prev = read_constcnt(); } static inline bool freq_counters_valid(int cpu) { + struct amu_cntr_sample *amu_sample = per_cpu_ptr(&cpu_amu_samples, cpu); + if ((cpu >= nr_cpu_ids) || !cpumask_test_cpu(cpu, cpu_present_mask)) return false; @@ -108,8 +123,8 @@ static inline bool freq_counters_valid(int cpu) return false; } - if (unlikely(!per_cpu(arch_const_cycles_prev, cpu) || - !per_cpu(arch_core_cycles_prev, cpu))) { + if (unlikely(!amu_sample->arch_const_cycles_prev || + !amu_sample->arch_core_cycles_prev)) { pr_debug("CPU%d: cycle counters are not enabled.\n", cpu); return false; } @@ -152,20 +167,27 @@ void freq_inv_set_max_ratio(int cpu, u64 max_rate) static void amu_scale_freq_tick(void) { + struct amu_cntr_sample *amu_sample = this_cpu_ptr(&cpu_amu_samples); u64 prev_core_cnt, prev_const_cnt; u64 core_cnt, const_cnt, scale; - prev_const_cnt = this_cpu_read(arch_const_cycles_prev); - prev_core_cnt = this_cpu_read(arch_core_cycles_prev); + prev_const_cnt = amu_sample->arch_const_cycles_prev; + prev_core_cnt = amu_sample->arch_core_cycles_prev; + + write_seqcount_begin(&amu_sample->seq); update_freq_counters_refs(); - const_cnt = this_cpu_read(arch_const_cycles_prev); - core_cnt = this_cpu_read(arch_core_cycles_prev); + const_cnt = amu_sample->arch_const_cycles_prev; + core_cnt = amu_sample->arch_core_cycles_prev; + /* + * This should not happen unless the AMUs have been reset and the + * counter values have not been resroted - unlikely + */ if (unlikely(core_cnt <= prev_core_cnt || const_cnt <= prev_const_cnt)) - return; + goto leave; /* * /\core arch_max_freq_scale @@ -182,6 +204,10 @@ static void amu_scale_freq_tick(void) scale = min_t(unsigned long, scale, SCHED_CAPACITY_SCALE); this_cpu_write(arch_freq_scale, (unsigned long)scale); + + amu_sample->last_update = jiffies; +leave: + write_seqcount_end(&amu_sample->seq); } static struct scale_freq_data amu_sfd = { @@ -189,6 +215,61 @@ static struct scale_freq_data amu_sfd = { .set_freq_scale = amu_scale_freq_tick, }; +#define AMU_SAMPLE_EXP_MS 20 + +unsigned int arch_freq_get_on_cpu(int cpu) +{ + struct amu_cntr_sample *amu_sample; + unsigned long last_update; + unsigned int seq; + unsigned int freq; + u64 scale; + + if (!cpumask_test_cpu(cpu, amu_fie_cpus) || !arch_scale_freq_ref(cpu)) + return 0; + +retry: + amu_sample = per_cpu_ptr(&cpu_amu_samples, cpu); + + do { + seq = raw_read_seqcount_begin(&amu_sample->seq); + last_update = amu_sample->last_update; + } while (read_seqcount_retry(&amu_sample->seq, seq)); + + /* + * For those CPUs that are in full dynticks mode, + * and those that have not seen tick for a while + * try an alternative source for the counters (and thus freq scale), + * if available for given policy + */ + if (time_is_before_jiffies(last_update + msecs_to_jiffies(AMU_SAMPLE_EXP_MS))) { + struct cpufreq_policy *policy = cpufreq_cpu_get(cpu); + int ref_cpu = nr_cpu_ids; + + if (cpumask_intersects(housekeeping_cpumask(HK_TYPE_TICK), + policy->cpus)) + ref_cpu = cpumask_nth_and(cpu, policy->cpus, + housekeeping_cpumask(HK_TYPE_TICK)); + + cpufreq_cpu_put(policy); + if (ref_cpu >= nr_cpu_ids || ref_cpu == cpu) + /* No alternative to pull info from */ + return 0; + cpu = ref_cpu; + goto retry; + } + /* + * Reversed computation to the one used to determine + * the arch_freq_scale value + * (see amu_scale_freq_tick for details) + */ + scale = arch_scale_freq_capacity(cpu); + freq = scale * arch_scale_freq_ref(cpu); + freq >>= SCHED_CAPACITY_SHIFT; + + return freq; +} + static void amu_fie_setup(const struct cpumask *cpus) { int cpu; From patchwork Tue Mar 12 08:34:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Beata Michalska X-Patchwork-Id: 13589609 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9B0C6C54E66 for ; Tue, 12 Mar 2024 08:35:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=DYB/nXSqW16TEapAXLP9T4mcYU0EyOHZrZ6d8qNu+Ds=; b=H8Y1ZQni666mC4 L1skScfp2z9Psq4jYW1Vkb1wno+Q8WjdtkMThxjocNdKmizXkqvgBdeYUTYEyharl2jX47YFVJajy ZG24azaaOXZvHLgwyYvue+iPfHW3V+8UaFn42jZk/KFn+DuIspogdPXkXOrXcjTOD9aHPC4Nzl55x XmcI6igMLKJzckqkQC5jDwLCvn1URJ2WKipYJntq0k7oyxhKQKejsHewRr57akGMsvATC28JQPz4t 0SaZWg12hBLz7ja2rIZXL+tRZp/38wD+yKAuSM8UOtbc5FOxsgmCQ0M9DVLKFILfakIZRuqizwkQB GVRcSId1OetwapceLNDg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rjxbU-00000004uAC-3GiW; Tue, 12 Mar 2024 08:35:20 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rjxbP-00000004u7b-3JSh for linux-arm-kernel@lists.infradead.org; Tue, 12 Mar 2024 08:35:17 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2AE8515A1; Tue, 12 Mar 2024 01:35:49 -0700 (PDT) Received: from e125905.cambridge.arm.com (e125905.cambridge.arm.com [10.1.194.73]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 54A093F73F; Tue, 12 Mar 2024 01:35:10 -0700 (PDT) From: Beata Michalska To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, ionela.voinescu@arm.com, vanshikonda@os.amperecomputing.com Cc: sudeep.holla@arm.com, will@kernel.org, catalin.marinas@arm.com, vincent.guittot@linaro.org, sumitg@nvidia.com, yang@os.amperecomputing.com, lihuisong@huawei.com Subject: [PATCH v3 3/3] arm64: Update AMU-based frequency scale factor on entering idle Date: Tue, 12 Mar 2024 08:34:31 +0000 Message-Id: <20240312083431.3239989-4-beata.michalska@arm.com> In-Reply-To: <20240312083431.3239989-1-beata.michalska@arm.com> References: <20240312083431.3239989-1-beata.michalska@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240312_013515_928808_65E66CD0 X-CRM114-Status: GOOD ( 12.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that the frequency scale factor has been activated for retrieving current frequency on a given CPU, trigger its update upon entering idle. This will, to an extent, allow querying last known frequency in a non-invasive way. It will also improve the frequency scale factor accuracy when a CPU entering idle did not receive a tick for a while. Suggested-by: Vanshidhar Konda Signed-off-by: Beata Michalska --- arch/arm64/kernel/topology.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index 42cb19c31719..77c6ac577cef 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -215,6 +215,18 @@ static struct scale_freq_data amu_sfd = { .set_freq_scale = amu_scale_freq_tick, }; +void arch_cpu_idle_enter(void) +{ + unsigned int cpu = smp_processor_id(); + + if (!cpumask_test_cpu(cpu, amu_fie_cpus)) + return; + + /* Kick in AMU update but only if one has not happned already */ + if (time_is_before_jiffies(per_cpu(cpu_amu_samples.last_update, cpu))) + amu_scale_freq_tick(); +} + #define AMU_SAMPLE_EXP_MS 20 unsigned int arch_freq_get_on_cpu(int cpu) @@ -242,7 +254,8 @@ unsigned int arch_freq_get_on_cpu(int cpu) * try an alternative source for the counters (and thus freq scale), * if available for given policy */ - if (time_is_before_jiffies(last_update + msecs_to_jiffies(AMU_SAMPLE_EXP_MS))) { + if (!idle_cpu(cpu) && + time_is_before_jiffies(last_update + msecs_to_jiffies(AMU_SAMPLE_EXP_MS))) { struct cpufreq_policy *policy = cpufreq_cpu_get(cpu); int ref_cpu = nr_cpu_ids;