From patchwork Thu Feb 29 16:25:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vanshidhar Konda X-Patchwork-Id: 13577355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4FA3CC48BF6 for ; Thu, 29 Feb 2024 16:26:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=g+Z8StbMyOv+W5n14zMdfAE82GsUVRiex4iMUMY1l4g=; b=XLOMRS2RXOQLqm QAwdlx7ZS77RIBMqmhgDTduFzDqn4EajQIpQSd/NiCHnRFhP7Tdxliho2YNGFwTfkp5TQNUjQy4/U oTyr+DuaBEBfm3nKXwq+Rz6wiYRVwK3zpWg98jLJjSKCo9mdvqFITlDKieItgwpKUDve66VUJe3dZ Hkk/Sb6BEuhmFb0ywgO9UOb9vE2ESGHSkY5s8D61UydAYV0V3M8Ue5PAj9Ns84Ng8fu9MTP28wIhz 7smeRbeV0cPz55Xi9H1l9C0kZGRfZ5vSt+qqcMe/ZKlF41s3virQIoUCtCr3EM4OgaFHqXT30vlv6 4ZfZoGfKtaKRLiHYWKxw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfjEC-0000000EHlg-0Rvm; Thu, 29 Feb 2024 16:25:48 +0000 Received: from mail-dm3nam02on20700.outbound.protection.outlook.com ([2a01:111:f403:2405::700] helo=NAM02-DM3-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfjDy-0000000EHgr-4A7K for linux-arm-kernel@lists.infradead.org; Thu, 29 Feb 2024 16:25:36 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ie86MZ/UPWFz2qGZruLcfjUw0cugGo8AGbhD9rNkbLByibPqXIyWi0lj7R0MxhFDyx1NUXYCGTv8pUssTK89b9173wiN/kvl6Baz2HlLIFm/rDPlABlZ3KRpXaYv6rOj8aZ/7cwR124zF1R2JCuwZp04SAWx5psEu3n2fAjR7aJtcEH5CU2/vEOns6TjQop55oPjjta9YPn0+IxZql/y++VLWiKQ5C3EoAMJcJxYteOoT6BIL6ne4rm3Mp5iPoCkjDPlSDefolQSkNdjvHhqWr/1ih+G96YB1QQ4mE8JjfE8stzGsJQxP1uIIeJDeRKQNodXoTLzONrN/4/Ehnl+TQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wSz2qDlKix/6vaArGmJ2NJgkGJOSduawb/qLuGOk+yw=; b=kMm998ZHtpK4KHFU5dwb57oCC2qkr2+yDaGwrzr6IIJQd6pApQZ3dhV35uqasAoBGM9ZcjE/0UN/dOJ0v+eoHiDGMycZRWnI4iYd5+vlFZLnMQxIph19Rn1BnYQEvMsuVwtGVO3lkALajaiTwXs3wivKr/xxslWTp0LgEAf1aGxK2mHy47ekJkRBdXc0YMvbPG2T+0KGsUqtpKYDJ7xuhS78xXfYjBQanmqlXgzoiOQzazsSoWXHd5kd8SScZBFrW1z3F+rwes4FxbCbIXWQGqPM4XjBe22n2grNKS8j7jzXFy4pZMJSNRw+KdC0s2zV4aQ9OvgyCub+jD1sjTN+Sw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wSz2qDlKix/6vaArGmJ2NJgkGJOSduawb/qLuGOk+yw=; b=MOQ5jQpnq44SpSdGORDELnLE8hw2AnegI1480zPB1Bw+SFYkLx1V4ILVuQKTH7FJgtE7kTjQ5PIZZ6YUH0OsRWJ9rDzFGp1laUUr+eyJBtmyGCEV33/6QjuO7GgM4bze4hnEQXQVk/fjw6qMXbWBVR3x/xYnJjOj5fgm5jmpGnw= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; Received: from SJ0PR01MB6509.prod.exchangelabs.com (2603:10b6:a03:294::17) by CO1PR01MB7225.prod.exchangelabs.com (2603:10b6:303:160::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.33; Thu, 29 Feb 2024 16:25:27 +0000 Received: from SJ0PR01MB6509.prod.exchangelabs.com ([fe80::3bf9:56fb:dd12:56]) by SJ0PR01MB6509.prod.exchangelabs.com ([fe80::3bf9:56fb:dd12:56%6]) with mapi id 15.20.7316.037; Thu, 29 Feb 2024 16:25:27 +0000 From: Vanshidhar Konda To: Huisong Li , Beata Michalska Cc: Vanshidhar Konda , Ionela Voinescu , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, rafael@kernel.org, sumitg@nvidia.com, zengheng4@huawei.com, yang@os.amperecomputing.com, will@kernel.org, sudeep.holla@arm.com, liuyonglong@huawei.com, zhanjie9@hisilicon.com, linux-acpi@vger.kernel.org Subject: [PATCH v1 1/3] arm64: topology: Add arch_freq_get_on_cpu() support Date: Thu, 29 Feb 2024 08:25:13 -0800 Message-ID: <20240229162520.970986-2-vanshikonda@os.amperecomputing.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229162520.970986-1-vanshikonda@os.amperecomputing.com> References: <20240229162520.970986-1-vanshikonda@os.amperecomputing.com> X-ClientProxiedBy: CH0PR03CA0393.namprd03.prod.outlook.com (2603:10b6:610:11b::21) To SJ0PR01MB6509.prod.exchangelabs.com (2603:10b6:a03:294::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR01MB6509:EE_|CO1PR01MB7225:EE_ X-MS-Office365-Filtering-Correlation-Id: 4568ed09-1d69-4a92-5f82-08dc394309ee X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qDgXxHpSro7q3jd+pPMtPTaWgidW2fviQBN0dgL4ExvOI+kJ9ix40DZLsoDe+WVF1u0OpeKewB55l5PeTqIXZ1gHtRZ6hXHNn8URmDxvE9P0p7olEB3iYYJ70XgxKIwSm9xrAHWB6rTF8JedUgMSOUkfNUKkVXY2ZFB6U4d9QXncuo97Eg9jievotf23cBwT4grMKbOAA1tQ+U7Z8pc+/TaAXNPGlz8eKqDicBHEjU7qfFM5YQMw1Fl3wQd3JvdeChLKTjV4VsLvnzb2Pq7byk1585rOZVMISOsoQnq/n5c4bO7/kztaLxO5FkJPFPQKt80HgfYgOb6YkowTzLUhgKEIYFzPtBkN+VaEGXSEGFgXG/z2GrXk9p4PONUrKjXyFujsMZCglCNj2x/FdSsdNKH3CgKYgLugUIcNnQAqFpX13gzuxvjZcFgGBKk6UVUcmND6lFLsZEv+5TUIsFocz4lmeg8g/kCSGio0zS6H77TRDHJvSNEXSisJOhLFva1IkODzeAf0/xcotRG1bKtpk478bJ+ih6SZ68VEBqIHFlZi+qlx7go5urksZ5a0Co9Re1mnk7qRDIzJ6slvUzdfXTpq6MweW/dVvUN3E1Lv+rEavfa8YH1Of0WmARygrV8hIfwTEmr3Omv0zZWD7EPwFA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR01MB6509.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230031);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: qZhycgWvOYoEDfDJCEWDI+f3nI5RSvDFD/bPv2gLLoYfN7kDzTN3B7O3bpFFmlRxFRth8XGV1M+eFNzPWKIYfXWTDOi25L8V9IiTcnC32cFJfbOrR3r/0bsSeSqSV37kl7Tchem5usWdDLG0cHEApGELACFigeGD+XnxCJsFMs/zqrt9NNXQYFyrJCX8fdyttTVd5LPnaNKWdPtpTBIvduQ02Z91Nd6HM/GPcxRrXCg7OBJc+KWWQKYSTe8lAP4jHPcf0ZE/KBY+1paFybiCv6hK0f70HufH/pqlDs4S/N11uxZ+oSiDjRhSx8zUDdpdHVof2blOeSa4oTNMyGmBPqutvn33wOeVCFmoOAAtiWgvZ0EneaKf9hwQHF4P8HhNd+Bh3YtkyLGnzDIv2gNPfDXOYDLQzprJAhxTKdK/qRgy+S7GWSHuh7EoRAhJmnM9n18yMlrRAifjvEUiu2IODjsWIi+7kR2vTN3pQXjhXWlYstdkyC+49bBkzpWY62nRDbvlnnirW+m2QhmUTyEXdagb2qZrZ5wy3phOjUPKQV63XSUIe52K9mCIoB5jjKDimCP3CFbPwpzML3H2iHwsXpjoZ4kBp2m/c74V49/uSaMMkSTThwV4ZRUvzNNOr3j0Ipc4oU6TbhdInd1EPhHbZmTk6XBfWeSJgB3JcDIM0X5EMTldIrgnFa204fvQhLx2EBzE2a2BZu7CuxOv55f1D8QMWefS/urzw3g5mZUKe+lCQRJXMumDQoviewj0fLMEgSkNHS5WX2OfpgeQt/1YSxYqOnnGkpXl5njRjK6iH4HBVpCi7udtKxJBNPYjHym6eCY4MKvH1JFaSCyZo6cIcrsmF8YKS9c+WSyKpWX/as4lbdaArZsvEHeZupOHW+bws51r6WD05cFMeAE9Y4g65ua8gEzmRnVgGwxBVwBmHaezoZ0iGHWcYDVUeJYxDwaDeq2uZ4SfiFz2qkWiYS8vCoLVi+eKTq4Y2TzcCZkWy3s6+z6RbrYXJobHvsy6qyLjoLAacpz36BFmuFWWk5JtGaM9CgTljqjTBTrVcGzYd6Nq9YaXwnNcJi9kiHLsV4ZI2H5E1fQX/HQ532r9n5t6oyMuHsT4wrXdhZiL+47zdq1BRVSrHAmlsxPrCu3eWQ8wi1ofyyr+28N4k6GIA0Z0hjz8hTA12rewJFVmbedlTZIpWYKO1mBDBzRrBjV7sQ/KQDdM58bK15xdKc2nopAR5fuogohnBoJeC4sQuq3G7CTuFHHqvqgIJcZeTiPzcTuHkxh3w8uUjsc6mzo29abTQ61tqp+cUOl0SSGy7+UEeWUr++AEr1eIqbpS5YoXi/HNwX3QWtaGIKZteYB/pFRVHrWYmDs3YVMMAdZ+reR1qCRX49ywRWLMSld/+t6RBkt7QXNLgyHWXEhgD9SVBAtrXxJhDbGDrGc3vGmUkyqIsm7GXt2Zrcng9mEHO3KEdOVGvgiF0xA6/8vbzmxc/3/Jgru1eerskl2kzVh7U2iF+fIKXhPMeqFutAgfW1cJNwR1Xs22bVPJdXbVGuhE68mOg4LeFgTGdCcbzMcjOFES0PYSem8swniwS9O0ejrXJkPN2lR/nl2DAyB7izAJcjKaaWGvfZOW8UE64DMJYt9NZDkLczr9TVxBB5IDjh32fWxJ X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4568ed09-1d69-4a92-5f82-08dc394309ee X-MS-Exchange-CrossTenant-AuthSource: SJ0PR01MB6509.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Feb 2024 16:25:27.3101 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rsuUjr8LakPzpB3G+M4Sxx0Zl4vQAhGJ9QI1jXKg8lFDJF0UBEUqM6bY05NdJwunDhSRhhMZKcPWg90Yur6m4Ay0tIVH/Ffr4/z9ltIrb4RLS9rt7kz4J8wtO4vbsB3H X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR01MB7225 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_082535_124429_38A87C14 X-CRM114-Status: GOOD ( 20.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org AMU counters are used by the Frequency Invariance Engine (FIE) to estimate the CPU utilization during each tick. The delta of the AMU counters between two ticks can also be used to estimate the average CPU frequency of each core over the tick duration. Measure the AMU counters during tick, compute the delta and store it. When the frequency of the core is queried, use the stored delta to determine the frequency. arch_freq_get_on_cpu() is used on x86 systems to estimate the frequency of each CPU. It can be wired up on arm64 for the same functionality. Signed-off-by: Vanshidhar Konda --- arch/arm64/kernel/topology.c | 114 +++++++++++++++++++++++++++++------ 1 file changed, 96 insertions(+), 18 deletions(-) diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index 1a2c72f3e7f8..db8d14525cf4 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -17,6 +17,8 @@ #include #include #include +#include +#include #include #include @@ -82,20 +84,54 @@ int __init parse_acpi_topology(void) #undef pr_fmt #define pr_fmt(fmt) "AMU: " fmt +struct amu_counters { + seqcount_t seq; + unsigned long last_update; + u64 core_cnt; + u64 const_cnt; + u64 delta_core_cnt; + u64 delta_const_cnt; +}; + /* * Ensure that amu_scale_freq_tick() will return SCHED_CAPACITY_SCALE until * the CPU capacity and its associated frequency have been correctly * initialized. */ -static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) = 1UL << (2 * SCHED_CAPACITY_SHIFT); -static DEFINE_PER_CPU(u64, arch_const_cycles_prev); -static DEFINE_PER_CPU(u64, arch_core_cycles_prev); +static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) = + 1UL << (2 * SCHED_CAPACITY_SHIFT); +static DEFINE_PER_CPU_SHARED_ALIGNED(struct amu_counters, cpu_samples) = { + .seq = SEQCNT_ZERO(cpu_samples.seq) +}; static cpumask_var_t amu_fie_cpus; void update_freq_counters_refs(void) { - this_cpu_write(arch_core_cycles_prev, read_corecnt()); - this_cpu_write(arch_const_cycles_prev, read_constcnt()); + struct amu_counters *cpu_sample = this_cpu_ptr(&cpu_samples); + u64 core_cnt, const_cnt, delta_core_cnt, delta_const_cnt; + + const_cnt = read_constcnt(); + core_cnt = read_corecnt(); + + if (unlikely(core_cnt < cpu_sample->core_cnt) || + unlikely(const_cnt < cpu_sample->const_cnt)) { + WARN(1, "AMU counter values should be monotonic.\n"); + cpu_sample->delta_const_cnt = 0; + cpu_sample->delta_core_cnt = 0; + return; + } + + delta_core_cnt = core_cnt - cpu_sample->core_cnt; + delta_const_cnt = const_cnt - cpu_sample->const_cnt; + + cpu_sample->core_cnt = core_cnt; + cpu_sample->const_cnt = const_cnt; + + raw_write_seqcount_begin(&cpu_sample->seq); + cpu_sample->last_update = jiffies; + cpu_sample->delta_const_cnt = delta_const_cnt; + cpu_sample->delta_core_cnt = delta_core_cnt; + raw_write_seqcount_end(&cpu_sample->seq); } static inline bool freq_counters_valid(int cpu) @@ -108,8 +144,7 @@ static inline bool freq_counters_valid(int cpu) return false; } - if (unlikely(!per_cpu(arch_const_cycles_prev, cpu) || - !per_cpu(arch_core_cycles_prev, cpu))) { + if (unlikely(per_cpu_ptr(&cpu_samples, cpu) == NULL)) { pr_debug("CPU%d: cycle counters are not enabled.\n", cpu); return false; } @@ -152,19 +187,15 @@ void freq_inv_set_max_ratio(int cpu, u64 max_rate) static void amu_scale_freq_tick(void) { - u64 prev_core_cnt, prev_const_cnt; - u64 core_cnt, const_cnt, scale; - - prev_const_cnt = this_cpu_read(arch_const_cycles_prev); - prev_core_cnt = this_cpu_read(arch_core_cycles_prev); + struct amu_counters *cpu_sample = this_cpu_ptr(&cpu_samples); + u64 delta_core_cnt, delta_const_cnt, scale; update_freq_counters_refs(); - const_cnt = this_cpu_read(arch_const_cycles_prev); - core_cnt = this_cpu_read(arch_core_cycles_prev); + delta_const_cnt = cpu_sample->delta_const_cnt; + delta_core_cnt = cpu_sample->delta_core_cnt; - if (unlikely(core_cnt <= prev_core_cnt || - const_cnt <= prev_const_cnt)) + if ((delta_const_cnt == 0) || (delta_core_cnt == 0)) return; /* @@ -175,15 +206,62 @@ static void amu_scale_freq_tick(void) * See validate_cpu_freq_invariance_counters() for details on * arch_max_freq_scale and the use of SCHED_CAPACITY_SHIFT. */ - scale = core_cnt - prev_core_cnt; + scale = delta_core_cnt; scale *= this_cpu_read(arch_max_freq_scale); scale = div64_u64(scale >> SCHED_CAPACITY_SHIFT, - const_cnt - prev_const_cnt); + delta_const_cnt); scale = min_t(unsigned long, scale, SCHED_CAPACITY_SCALE); this_cpu_write(arch_freq_scale, (unsigned long)scale); } +/* + * Discard samples older than the define maximum sample age of 20ms. There + * is no point in sending IPIs in such a case. If the scheduler tick was + * not running then the CPU is either idle or isolated. + */ +#define MAX_SAMPLE_AGE ((unsigned long)HZ / 50) + +unsigned int arch_freq_get_on_cpu(int cpu) +{ + struct amu_counters *cpu_sample = per_cpu_ptr(&cpu_samples, cpu); + u64 delta_const_cnt, delta_core_cnt; + unsigned int seq, freq; + unsigned long last; + + if (!freq_counters_valid(cpu)) + goto fallback; + + do { + seq = raw_read_seqcount_begin(&cpu_sample->seq); + last = cpu_sample->last_update; + delta_core_cnt = cpu_sample->delta_core_cnt; + delta_const_cnt = cpu_sample->delta_const_cnt; + } while (read_seqcount_retry(&cpu_sample->seq, seq)); + + /* + * Bail on invalid count and when the last update was too long ago, + * which covers idle and NOHZ full CPUs. + */ + if (!delta_const_cnt || ((jiffies - last) > MAX_SAMPLE_AGE)) { + if (!(housekeeping_cpu(cpu, HK_TYPE_TICK) && idle_cpu(cpu))) + goto fallback; + } + + /* + * CPU frequency = reference perf (in Hz) * (/\ delivered) / (/\ reference) + * AMU reference performance counter increment rate is equal to the rate + * of increment of the System counter, CNTPCT_EL0 and can be used to + * compute the CPU frequency. + */ + return div64_u64((delta_core_cnt * (arch_timer_get_rate() / HZ)), + delta_const_cnt); + +fallback: + freq = cpufreq_quick_get(cpu); + return freq ? freq : cpufreq_get_hw_max_freq(cpu); +} + static struct scale_freq_data amu_sfd = { .source = SCALE_FREQ_SOURCE_ARCH, .set_freq_scale = amu_scale_freq_tick, From patchwork Thu Feb 29 16:25:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vanshidhar Konda X-Patchwork-Id: 13577356 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6EB9EC54E4A for ; Thu, 29 Feb 2024 16:26:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=g3EVuyMVwqlA9eNGLygM1S7G/LiBP0GcbjxF5igsKgE=; b=ipMdn7bYR1QBZC sg3WojCFQuoJuUbPi/0oIblUmxgraKdFfCWkoQ46iCB5MjHVgVpYcfsGnb39W89GQNSTE1vBthJwZ IW9lKWSuMXI/hwCOUba2zeskXljU7u0v0TFmgwGWWNQHfcv2PnJ9zpIY3ehfvWf1MGfNXq2OztSfp AKpDoD5EOmlG97HNR5hjt989ER1FLDEYxuwZ3xmPUSrenK64Ozl2LTp2TZy0mXsiWHp/Yv3BlWoqN zW5mtoykUsnzXJJ8pe+nRKh/BzNKxvse4uNsu5E1gY63046c6lguW3L6hx1Ax4MBc/AoC1DkJc++e r/9J3yc12yfCvz41005w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfjEE-0000000EHms-1A9K; Thu, 29 Feb 2024 16:25:50 +0000 Received: from mail-dm3nam02on20700.outbound.protection.outlook.com ([2a01:111:f403:2405::700] helo=NAM02-DM3-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfjE0-0000000EHgr-2lLw for linux-arm-kernel@lists.infradead.org; Thu, 29 Feb 2024 16:25:38 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Qc7nJnPmi69LJVwbo0BpsT/7nFH3PRTE9YNVHIJ7v9cAfQiiYuVT1TL6spjA9dcR++GHMt+EnjK2BskX3jgUuR48yMtWK4okM+lj+RTkLXSkG3MQMozWWJGbTZoJTNEki+LUvAtk8tQOiKHgRBayht1yAbnJ8QFhD5D4Kwfh/XB9az1N2Dn+kCwhIczVibUShVUqqZbtD9lPqPrMGc0yzHaSKdkn2gZgW5T0tE0jEN7f90GcK8yixDWth4Fkqgbt2a1EM7TgQKZaWrsn0Yp1cPXhwAAE6i1Ti94grHVVvIpsZpmkJPLOJoZ3uAdWBnQDlULLsqT33yQR89DjHPEqwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dPf8ktNOUncWzsFgeSsltCK0HgC4tkxBS9eGBpVMFiU=; b=doSxpmUbImMpDYTeOtfcPnFyej67/3k+wJrJnw0BB6Q3coo/1+gVSx+/VvI7gKAfB2U9woAP3zuRcYy6kL1c3g2KMnhmyw2csfj1Y7pnApsAxva0TJgoecG9zLnzOqxOCTsBLNDhQUIpsWxmN6D+E0SRc86BqGT2v42cLt/sqLwor/NzH3Fd821sO/nV9WQGgX8uJNoN2zJLrlSsZQzWQj/CfDc2JvOc/tm0miF0GsBCvj+lqCKMyFckxr6sKb7AUQnyQwQc1Mp/gp6/U+B4wvQrotJE2t9G7gv8bmBNA+i9XlJ9Leeqc/pAikgqAaLCn4Nm+FqnS4k/h+Bpf4U+rQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dPf8ktNOUncWzsFgeSsltCK0HgC4tkxBS9eGBpVMFiU=; b=f7EJXINdRr4Fdo0riDk9INCbjqjuAKesblJ+Q+gOEcoXt5/lD/R66Oj4dlqC70TGbirrQAlSOznbeQCSUtYRpdldkgD2cnge28Hvr1SbajAYpDBbAB/S+2MSOPdIcpm83BYCPyu1i19IDRxoVHwIuOO4JAWa0XNsaCWdmM+E55A= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; Received: from SJ0PR01MB6509.prod.exchangelabs.com (2603:10b6:a03:294::17) by CO1PR01MB7225.prod.exchangelabs.com (2603:10b6:303:160::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.33; Thu, 29 Feb 2024 16:25:30 +0000 Received: from SJ0PR01MB6509.prod.exchangelabs.com ([fe80::3bf9:56fb:dd12:56]) by SJ0PR01MB6509.prod.exchangelabs.com ([fe80::3bf9:56fb:dd12:56%6]) with mapi id 15.20.7316.037; Thu, 29 Feb 2024 16:25:30 +0000 From: Vanshidhar Konda To: Huisong Li , Beata Michalska Cc: Vanshidhar Konda , Ionela Voinescu , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, rafael@kernel.org, sumitg@nvidia.com, zengheng4@huawei.com, yang@os.amperecomputing.com, will@kernel.org, sudeep.holla@arm.com, liuyonglong@huawei.com, zhanjie9@hisilicon.com, linux-acpi@vger.kernel.org Subject: [PATCH v1 2/3] arm64: idle: Cache AMU counters before entering idle Date: Thu, 29 Feb 2024 08:25:14 -0800 Message-ID: <20240229162520.970986-3-vanshikonda@os.amperecomputing.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229162520.970986-1-vanshikonda@os.amperecomputing.com> References: <20240229162520.970986-1-vanshikonda@os.amperecomputing.com> X-ClientProxiedBy: CH2PR04CA0004.namprd04.prod.outlook.com (2603:10b6:610:52::14) To SJ0PR01MB6509.prod.exchangelabs.com (2603:10b6:a03:294::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR01MB6509:EE_|CO1PR01MB7225:EE_ X-MS-Office365-Filtering-Correlation-Id: 0438bab8-55c4-4a1e-4fa0-08dc39430bbc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HNAtoXAAlsqmiF57P1BveLuRXGHBKDk2sI09fkflhn3NHSH55/OPxJeRuHozw+wEJFANi6kUmGAS2v1vM8IkvHrLBsgY7JsSQRIEX1n0LwPX2S3x1OEuhebut/KnPi1mILYNQ/B9bSn5v4QFu3VEj4QzNsDvNMQdc21TCXrnyyjqssNGTAEC3fGIL4cnF20cW7IZmUaIo5kaU7UxDXn5P7AaZBBpu6FMC4a6MCWi6jVSYxg6DWiZi+qjYgw17sHfmUc2t/cy7KbpZjNIK1uwPX180z9kBn/K8gCIYJrPzBrQqv1fCKkB/hcLwt+7JgyX/HJ12sKxyKyZcnQ+1GJ1VTRW/2eNB7DDy9P0KW+YXQwDK48F5UtnOcpyUhGLZRjRgb3bW0o4RL8boIEwL3sdQ5QSsWs4xVO/hX6irimnhozyW2TXNxrdzQfMr1uo8H84LivKi7EqkXLL4nNgRtWa8eYA8JTVFS5tCDT0wuHGG2z0VMl+dL4VBc7K0vLreCacXjulp7lwm5LEimpPE5VXV9aAgAHfa559/J2h/o1v0yyWa82ueHxDp+sRSYahSkz9maiT1t0L58DWasclKiQZSn2p/ZPgdJXg1a7Ieh2h8f9E6CwZoNqVReyiw9L9W7KPAP4yat4NDn923MmMkrhVBA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR01MB6509.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230031);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: mZXXGNDYfKwAyu6DqR1zsRDlosfzWAelgwxSnFnGnOJhwsiTR0fGKzrhOLanU1EoEtXIbSKxrzLWTAKHeKLiIOM9HIYfvl6SdCZcDUCnmB5TLXLIiqObTRZIzRMQXR039nJ8QXEIdMSx5fehf0g6xBencm+3WDzRqn/hV9rQdL74PkOoeu/V4I8grGZdJCU0eHrK+/nCnjiGEUgo3sBQeehRHoWqK62/0h5n8s1sWRlwPI8q26FpClPdzL7GFrDTr9GH+vi4D9LETg3Xxr3aIzgGl36TsQqznQsIEeLSwVITbIfcHLpsTLp6mcwntE1lS/h3lLmNna/QUlsmQbqQnAkxuLV6n3iYHj9eWUKR8DaBMgtAb+WywuxvTnxSwKnVZ8qZDql2NUFKzRs+0PEVRVu1q4UAmMvMeL+SLj/m31AiffvT3rm1eDCDpq/MAz0roXmW7459uSxoVX06Q6VnM4yOPe0wFGZEStioelABP9l3jbH1RBKZGl4KSxr8C5KG4dfD34mm3nsEaSvWdMAiQplWA9q2gKITt+eNvuDtukAuAvWe0icdpXnFfmypfBvifmKCr9IP2zLqLtJGwK/I3p+PlIyGlTiR/38gzKL5OrxcizC9L0eP8yPsFDXvQm7zmR0LpWiWRy6bS9607u4QJYDPOvmOxf4mb5kRFQYkks3kXzynMLvxdVRCm88ULTctRSOJBT4JSAKXPiGPiejYV4AOF6T5srCYIEs3+oRk1bpAFVHRpcJtazMVx1HY8O1n+BPSKMNH652XGtlLldvfewHnvjlnKxiAcfyFrPbWA2ZZMVYgKY8CRBk5KWnvt0rNSEfbF/MfgktnlgOsJ3RMuGxjatnwfPeow0EuURriSWIT5j+RWcapSo3TsuxexI+lI8RYDRwUSZDSWr+0ZIr4MC6gJmQr0em7hj49Ww1iXayaqakZEueBQ+0sL3rZCZgiGpkFpsi9QECaWagfvXnu2UCr/vWc2IyDFqKy84dh1+Iz4OHwnlmG+jinfYMQdtm/Zr4dZBEH2lCA/LX2nuC0Y0wueqlS4B9PYuM3dYWelSq1EvTSjwr8xJL/EtHoqwYXLh7qrDOVT3x1QkKXjLCM4Bat9TAy9eAwOH0GN1fFbnwgJDE0w2VBhDO0VDoH20vt+RH0g2QPwSOBR+pKw0d4qtNzCLjr8+W1OM86kqG0Jbv4T1cDLXl6/Rabq36+MuxkAlNlp59luBKDkp3ZlXMeENGWMAS6CJLvGSVDuoDXFWrYN+zAoZ+rmffA49gc9QXx0ggRNTVivf5E+0PdjZlV+unMTRitOKn5lnfVuVTFnaYavtp/E6O7S48hfm1VGNyFR0Kzgrdq1o3Zk/T7tTTpWa5Lz+v1qERYEDtLQvFnUNzagoP87N5n3/DKbR6Ai8kqYovv6dT0SbAoUIqMAUkyq0ybv6Rf8gvY+xzjPhY7IRTmFCG0L5nMFY6kZozEddueh1FU29daTtQMHiozmhA30SerMCeYvdJ3VVKsN4oNdKt964BgpnCr4mAtCcKSW1vdlGCcmHXcPg08sArhkqCMW9ni/nS93+1tcF1UsYcWaT4fI64bhih4NmgTd9zUTVycvAW4oCD/PFcEwllRIx6BXG4KI9rnh7nVxI9A005mpQpdOLOW5Gr2yvxPxdITioLj X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0438bab8-55c4-4a1e-4fa0-08dc39430bbc X-MS-Exchange-CrossTenant-AuthSource: SJ0PR01MB6509.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Feb 2024 16:25:30.3520 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: vH4t4MSGMXgTi+QWHeDz7xSWfUgEdaINwGF5tNJYWirCEvg0jo5fDB91IzhVVBD42+AjL/FDedJ3EmvTTtfLoU2M1o8vCi5fTu5BDntqdtX7k5G/waRyMoruItw3eV0p X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR01MB7225 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_082536_799877_0D8F5803 X-CRM114-Status: GOOD ( 14.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org AMU counters do not increment while a CPU is in idle. Saving the value of the core and constant counters prior to invoking WFI allows FIE to compute the frequency of a CPU that is idle. Signed-off-by: Vanshidhar Konda --- arch/arm64/kernel/idle.c | 10 ++++++++++ arch/arm64/kernel/topology.c | 14 ++++++++------ 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/idle.c b/arch/arm64/kernel/idle.c index 05cfb347ec26..5ed2e57188a8 100644 --- a/arch/arm64/kernel/idle.c +++ b/arch/arm64/kernel/idle.c @@ -26,6 +26,16 @@ void __cpuidle cpu_do_idle(void) arm_cpuidle_save_irq_context(&context); +#ifdef CONFIG_ARM64_AMU_EXTN + /* Update the AMU counters before entering WFI. The cached AMU counter + * value is used to determine CPU frequency while the CPU is idle + * without needing to wake up the CPU. + */ + + if (cpu_has_amu_feat(smp_processor_id())) + update_freq_counters_refs(); +#endif + dsb(sy); wfi(); diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index db8d14525cf4..8905eb0c681f 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -240,13 +240,15 @@ unsigned int arch_freq_get_on_cpu(int cpu) } while (read_seqcount_retry(&cpu_sample->seq, seq)); /* - * Bail on invalid count and when the last update was too long ago, - * which covers idle and NOHZ full CPUs. + * Bail on invalid count and when the last update was too long ago. + * This covers idle, NOHZ full and isolated CPUs. + * + * Idle CPUs don't need to be measured because AMU counters stop + * incrementing during WFI/WFE. */ - if (!delta_const_cnt || ((jiffies - last) > MAX_SAMPLE_AGE)) { - if (!(housekeeping_cpu(cpu, HK_TYPE_TICK) && idle_cpu(cpu))) - goto fallback; - } + if (!delta_const_cnt || + ((jiffies - last) > MAX_SAMPLE_AGE && !idle_cpu(cpu))) + goto fallback; /* * CPU frequency = reference perf (in Hz) * (/\ delivered) / (/\ reference) From patchwork Thu Feb 29 16:25:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vanshidhar Konda X-Patchwork-Id: 13577357 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2281C54E49 for ; Thu, 29 Feb 2024 16:26:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=b+KDhMW8kyHXBqNOsQI4tNrNqsfjmxf5lJiJf6sMfi0=; b=GARo9XPmM4YDUd LkaT0nHrKSF1BAuG6O1H5DqfuhW6FXtwSRk8kggA8Iz0t/NnJkGpN9QtS5qCfFehKrpsqWbIq7xmZ b+q8TCC0kqCYCYo3ZT4qpjmy8PQALJt+n+w/kme/r9RsmQktSc/nbuARwh3eSGatgQtSyRBJyM5fB tEsvaNAVisd6bOGdNRJtNhjTAFtQ3/JMQk/JDcqmQ+qLaG82NBMFXFdkdKYOnfWAV7D52Lb5oAZV0 m2BObU2d7/x9iZcvyUwQ6XYDH3Ja5/vxIXac34vGtcQv7tBD4pfgU2HBvXYLbrbEeen7Yr2n6hpYL rtLoMECVHFKmqTsGrjjg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfjEG-0000000EHoN-1Rdt; Thu, 29 Feb 2024 16:25:52 +0000 Received: from mail-dm3nam02on20700.outbound.protection.outlook.com ([2a01:111:f403:2405::700] helo=NAM02-DM3-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfjE2-0000000EHgr-1vPz for linux-arm-kernel@lists.infradead.org; Thu, 29 Feb 2024 16:25:40 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TUITIbcW3t4Kj/OxamBy2thdw6rr+u4xd2qqvYLKQpq46KIiwM6aRFzcnjEjAuzTyncT6sCsF7Gs4NiOPVM2pnOBIm2Cl+sld55ZedzFtxZj0b8VqymDA9BVIIcJJVtnnE5md5fVXQCdRFw4Bt6K+6o8P3lfEOWucrfYC3T4b3Lk/7Wq+9dMue4WmZ0KdbVwzu8Le5jnxZejC3p92gopCSgR4GpDiUZZsURYFDkeR/MKgBgvDevE7yxHWHOEKyyr9QzLHfsctmxjwaxZ4Lw5PYEWMNYP/aDx7sSC0ctC3+8PwmG3wHzDUQpkCbKZ/8nEIIafxTvX0X9L/vvm3gSSMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PYCcDFr9oBZ1ay1btoaoim+BeSDQ/pxZBKtFob/bDko=; b=oUUaDw2JgygriaMjvYmYenkF5megW5wskA4OguFSHJNmCIcmcdZsygq2qQnm2ZM1E2TXh6vL9UnHfmdK4O00hPJUbQNN1QIUaP0SDjB+vBc6d9N/Iax5cxgkfD6BUlvCUjvjMuKCC6ZQEsRf9HxYXOpiXDOEnzUY+lXizQZ7SVp3l5ecTwuzuy1BQcLx+uDDsIIC6qWjEyj/+4D47j/NZ6MNQ8dfrJE/5T9W3W5PXsAqBhr/VZMXL3fu7fMKfGQ4WiYcDX6gs2XRBtK58NtXNg45tedh0gPq2kwPna01BdNkyHxQOBD6VB6kIVH8hS/Mbe05l/74LHcLIKwhfy6suQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PYCcDFr9oBZ1ay1btoaoim+BeSDQ/pxZBKtFob/bDko=; b=WjllIipTdJKk1znntIHgccxOSo/SSdX6A/dE5GCSDX8VrPVlQGJxhlnK8jBQJgYT74LqIWMKOkPpJ4y0v3HhLMN3Y7cc+KlWxSmjy+PlRTGzS6eG5FVk5azf3XBl66h2nxCE6xZ916IbSb8DMs3G/mrY5yqNi2bV1hqPZWN2izQ= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; Received: from SJ0PR01MB6509.prod.exchangelabs.com (2603:10b6:a03:294::17) by CO1PR01MB7225.prod.exchangelabs.com (2603:10b6:303:160::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.33; Thu, 29 Feb 2024 16:25:33 +0000 Received: from SJ0PR01MB6509.prod.exchangelabs.com ([fe80::3bf9:56fb:dd12:56]) by SJ0PR01MB6509.prod.exchangelabs.com ([fe80::3bf9:56fb:dd12:56%6]) with mapi id 15.20.7316.037; Thu, 29 Feb 2024 16:25:33 +0000 From: Vanshidhar Konda To: Huisong Li , Beata Michalska Cc: Vanshidhar Konda , Ionela Voinescu , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, rafael@kernel.org, sumitg@nvidia.com, zengheng4@huawei.com, yang@os.amperecomputing.com, will@kernel.org, sudeep.holla@arm.com, liuyonglong@huawei.com, zhanjie9@hisilicon.com, linux-acpi@vger.kernel.org Subject: [PATCH v1 3/3] ACPI: CPPC: Read CPC FFH counters in a single IPI Date: Thu, 29 Feb 2024 08:25:15 -0800 Message-ID: <20240229162520.970986-4-vanshikonda@os.amperecomputing.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229162520.970986-1-vanshikonda@os.amperecomputing.com> References: <20240229162520.970986-1-vanshikonda@os.amperecomputing.com> X-ClientProxiedBy: CH0PR08CA0005.namprd08.prod.outlook.com (2603:10b6:610:33::10) To SJ0PR01MB6509.prod.exchangelabs.com (2603:10b6:a03:294::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR01MB6509:EE_|CO1PR01MB7225:EE_ X-MS-Office365-Filtering-Correlation-Id: b2853955-8707-48eb-1abc-08dc39430d67 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: unhMk4IPWuCmqbDoybDBisAGIAMPcwKYLyj+zFop7V1nopZsadkF/b4rXwGevwqYEkLGSU8bWlF4ZnkQZQtl4g7+Sks/DH8XFdjN4ZHO/4trl1qX3RnvCRuEl3nHZiqrKjvjxdHolaZCrY+SsSdaK8rFdhpOGdP2dDMXx0pWiwwSnr6B9/K6wMatvAuJAfKmTTi/KmzyiQwa8uwn46/PQmBd6p7F6CG6Z/514BZYWmVg35ABKlPeKnjiqdKz2zfeBT5uBiEWiEBbHHanQtJJ5IT5eeNcvSVfnX1H5HgU8/tquXv7r2jj/IeVrKfV2zTCADAdL/J/Q1ug7tAtSepkuI6oGd/YKorvcmJI7FiIs6BeJe0RHB4qAlFTGrQIx3y7JYPkqzku60bfaDGZHaNayg23/2hDQzYyDREhwBKnz7bl8qTQCfpa7ynSubie37k1yrxoyEfR7P/vXrMSQ4YUaXEu9pe54hFXKdyCG5W1pZjhh0/FA3FbbGgcDUCL7vBh4NSJLjNgXblXBV1udJHb3Y/dI5O0xEs1KU2C5lB0ZW4EsdxfIf3cGz5J5f49rsgnxfSsOKEajBwcP6pg6Zv+Zg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR01MB6509.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230031);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: tlKAnk7khcqV1FplTjEZBWLOG0bnON/NX4ki9DpOpDuJZaJDNa7dOeInpWEVb57iS26iCcJc1ahIVkzzLHyBUIc+sKDIEYtuK1bcrFS2Fc2qAhjWlch4lZuBD1qcmpoFbWCg0Vgx7+DXhvVTnXUTiLbkOnlvtPP5UA/MAMJAzIxKloVU9Q0h6lsGPQUq9QWFdd6rHNGRSIq5WmHRWu4SPFh7rMc6rgck0ouyPWIaMoPHic9/YbHqWGaRnYLLmfnNYwHNRCqsklauTBL3+ayb/qr5l1mwAcr5yjdEKa1kR/4AFAtXkgBNWgpJ2Sxsjat85ZuEJMYeImqXOGxuIQZXlRATNF6tD5QlIBHuoMEENhdpcXzNSU2PYbpd3RjakqqdpkMqPZ89ZZaFp0CsB53WFvkEpkplMFah/vJNHsC6iUjJosYVHvMvmEnY7Z8xRlHLruoZsjJzFPQnQxpMja7PGL6QksbXnmtJBnq5Oq3PcSPMGIS6KCTvRrR8x6obpDt4dHQkRe/b+y7PQ71ZCn3XBffh+/YgPuIeBjm9/g0OMnOIQU0QmE3/y63Msz2UE1wzV0q6TJTJE09zBZEklFnWQ57oZDTrPy6U9P8T5GvKu9b95mBs27EFx0EMuVMxU3ldWRusfU+SMHAYAPc0yuac15DTYF81hHACmROTbO7I5fPdOrjkdl6VSugfUNxOhk5TqLotBs/3FdUzzFCn41ff2wTuYSwH7c7vbzWj8qhrYCbG1tFEK9jvW90zV8H5qwdMfJL0pWjbGm6f2GU0Cq47+ws5m6r0spFZqufTuPqNuwaQztZkPY5bUk7IcdUAwE5eunkC3wU3SJQHApIgnfL6soP4WOzY9ntnFu+CiVXKcQllCnwuS/0rM2ETWej8IMWoRlqBAq19tJTl6KzvuVDIDrcrfm4KGwBUI8XshhooU75QsJoPMVWc+pvQFdNlhZgrHHPcm5BFgMjIiBUJ9xyjgp2Ek7rYTajvSvOMiFrBO84BsTguA+F/65oNfjBTuF7bQqhjrPVaDUolSrut2uoqrDEvd8nZZBmW7fbdyvT7qUvRe5ntschqbQOF0HdSp8HS/GrbzMdkCWQe9I0fM65J8MRjSzQ376PMGWOwUU+pBuUhuDqHoex1h7XNQrNu+cBsmenrwjzhDQpRDGR44HOpt+IA2vUAA6a4A60fQQHtYmxVW55Xg0LaBAGm7veTh8V/WYPnEGk7H1ji759u14BReS8ozFzbcHrzidY2mQuQHWaTlB8tk2oZbWP8Klp4bC+lIv6vI1LEw4an+L630f2dCagnWuow1I3zKWFdSImCVTEyEw9J6ltzqkEwlvWJidkHIEb8WSa2l17/Fdl5A8Y3hyTw117DY+RmXi26TYvf6X+n0R95EX1aZg/ywDsHTpymrkgNRe/Yapk/5A8PAm0tVmSiu1Ws0+BYGAf2VD3EiDVPzmWqdyrFA2a5G4oWu68NX3X2neFOuYwq2IdnEGQ8JVhecz2shGmS4NWx2uNfDHDML0HuENlxpn5dc22tNyX8pzu525iaI+ntjUPOyOndk1u199QK2AUWdzPGFbajS10iCDJSNAFZTR6x/Lev5mt6uhIGdVFZMe3mosaGg7SPYMKH/6b64w4QTth5pozfxO2I/EksAEe9R6EH6hS8puMl X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: b2853955-8707-48eb-1abc-08dc39430d67 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR01MB6509.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Feb 2024 16:25:33.1284 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TEH+csnGl4oqncGnu/oRx528M6BFTBZO9FcdnV675svZBmlIZI6PdxawZQSAT3BPr9RRI1zu+cbuMUtMLMcYQpz0ObJlC2XTFmtnNi0Fcb8ZgSE3g9UpaeklDnkXaKq6 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR01MB7225 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240229_082538_594660_97454155 X-CRM114-Status: GOOD ( 19.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The CPPC driver reads delivered and reference counters using cpc_read one at a time. This leads to inaccurate measurement of CPU frequency discussed in [1]. If the firmware indicates that both the registers are in the FFH interface the kernel can read the registers together in a single IPI. This has two benefits: 1. Reduces the number of IPIs needed to read the two registers 2. The two registers will be read in close proximity resulting in more accurate CPU frequency measurement [1]: https://lore.kernel.org/all/20230328193846.8757-1-yang@os.amperecomputing.com/ Signed-off-by: Vanshidhar Konda --- arch/arm64/kernel/topology.c | 37 ++++++++++++++++++++++++++++++++++++ drivers/acpi/cppc_acpi.c | 32 +++++++++++++++++++++++++++---- include/acpi/cppc_acpi.h | 13 +++++++++++++ 3 files changed, 78 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index 8905eb0c681f..8207565f43ee 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -421,6 +421,43 @@ int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val) return ret; } +static void cpc_update_freq_counters(void *info) +{ + update_freq_counters_refs(); +} + +int cpc_read_regs_ffh(int cpu, struct ffh_cpc_reg_values *ffh_regs) +{ + struct amu_counters *ctrs = per_cpu_ptr(&cpu_samples, cpu); + int idx; + + if (!cpc_ffh_supported() || !freq_counters_valid(cpu)) + return -EOPNOTSUPP; + + if (WARN_ON_ONCE(irqs_disabled())) + return -EPERM; + + if (!idle_cpu(cpu)) + smp_call_function_single(cpu, cpc_update_freq_counters, NULL, 1); + + for (idx = 0; idx < MAX_NUM_CPC_REGS_FFH; idx++) { + + if (!ffh_regs->regs[idx].reg) + continue; + + switch ((u64)(ffh_regs->regs[idx].reg->address)) { + case 0x0: + ffh_regs->regs[idx].value = ctrs->core_cnt; + break; + case 0x1: + ffh_regs->regs[idx].value = ctrs->const_cnt; + break; + } + } + + return 0; +} + int cpc_write_ffh(int cpunum, struct cpc_reg *reg, u64 val) { return -EOPNOTSUPP; diff --git a/drivers/acpi/cppc_acpi.c b/drivers/acpi/cppc_acpi.c index d155a86a8614..55ffb1915e4f 100644 --- a/drivers/acpi/cppc_acpi.c +++ b/drivers/acpi/cppc_acpi.c @@ -113,6 +113,10 @@ static DEFINE_PER_CPU(struct cpc_desc *, cpc_desc_ptr); (cpc)->cpc_entry.reg.space_id == \ ACPI_ADR_SPACE_SYSTEM_IO) +#define CPC_IN_FFH(cpc) ((cpc)->type == ACPI_TYPE_BUFFER && \ + (cpc)->cpc_entry.reg.space_id == \ + ACPI_ADR_SPACE_FIXED_HARDWARE) + /* Evaluates to True if reg is a NULL register descriptor */ #define IS_NULL_REG(reg) ((reg)->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY && \ (reg)->address == 0 && \ @@ -974,6 +978,11 @@ int __weak cpc_write_ffh(int cpunum, struct cpc_reg *reg, u64 val) return -ENOTSUPP; } +int __weak cpc_read_regs_ffh(int cpu, struct ffh_cpc_reg_values *regs) +{ + return -ENOTSUPP; +} + /* * Since cpc_read and cpc_write are called while holding pcc_lock, it should be * as fast as possible. We have already mapped the PCC subspace during init, so @@ -1317,7 +1326,7 @@ int cppc_get_perf_ctrs(int cpunum, struct cppc_perf_fb_ctrs *perf_fb_ctrs) int pcc_ss_id = per_cpu(cpu_pcc_subspace_idx, cpunum); struct cppc_pcc_data *pcc_ss_data = NULL; u64 delivered, reference, ref_perf, ctr_wrap_time; - int ret = 0, regs_in_pcc = 0; + int ret = 0, regs_in_pcc = 0, regs_read_in_ffh = 0; if (!cpc_desc) { pr_debug("No CPC descriptor for CPU:%d\n", cpunum); @@ -1353,8 +1362,23 @@ int cppc_get_perf_ctrs(int cpunum, struct cppc_perf_fb_ctrs *perf_fb_ctrs) } } - cpc_read(cpunum, delivered_reg, &delivered); - cpc_read(cpunum, reference_reg, &reference); + if (CPC_IN_FFH(delivered_reg) && CPC_IN_FFH(reference_reg)) { + struct ffh_cpc_reg_values ffh_regs; + + ffh_regs.regs[0].reg = &(delivered_reg->cpc_entry.reg); + ffh_regs.regs[1].reg = &(reference_reg->cpc_entry.reg); + ret = cpc_read_regs_ffh(cpunum, &ffh_regs); + if (!ret) { + delivered = ffh_regs.regs[0].value; + reference = ffh_regs.regs[1].value; + regs_read_in_ffh = 1; + } + } + + if (!regs_read_in_ffh) { + cpc_read(cpunum, delivered_reg, &delivered); + cpc_read(cpunum, reference_reg, &reference); + } cpc_read(cpunum, ref_perf_reg, &ref_perf); /* @@ -1366,7 +1390,7 @@ int cppc_get_perf_ctrs(int cpunum, struct cppc_perf_fb_ctrs *perf_fb_ctrs) if (CPC_SUPPORTED(ctr_wrap_reg)) cpc_read(cpunum, ctr_wrap_reg, &ctr_wrap_time); - if (!delivered || !reference || !ref_perf) { + if (!delivered || !reference || !ref_perf) { ret = -EFAULT; goto out_err; } diff --git a/include/acpi/cppc_acpi.h b/include/acpi/cppc_acpi.h index 3a0995f8bce8..0da614a50edd 100644 --- a/include/acpi/cppc_acpi.h +++ b/include/acpi/cppc_acpi.h @@ -137,6 +137,18 @@ struct cppc_cpudata { }; #ifdef CONFIG_ACPI_CPPC_LIB + +#define MAX_NUM_CPC_REGS_FFH 2 + +struct ffh_cpc_reg { + struct cpc_reg *reg; + u64 value; +}; + +struct ffh_cpc_reg_values { + struct ffh_cpc_reg regs[MAX_NUM_CPC_REGS_FFH]; +}; + extern int cppc_get_desired_perf(int cpunum, u64 *desired_perf); extern int cppc_get_nominal_perf(int cpunum, u64 *nominal_perf); extern int cppc_get_perf_ctrs(int cpu, struct cppc_perf_fb_ctrs *perf_fb_ctrs); @@ -153,6 +165,7 @@ extern unsigned int cppc_get_transition_latency(int cpu); extern bool cpc_ffh_supported(void); extern bool cpc_supported_by_cpu(void); extern int cpc_read_ffh(int cpunum, struct cpc_reg *reg, u64 *val); +extern int cpc_read_regs_ffh(int cpu, struct ffh_cpc_reg_values *regs); extern int cpc_write_ffh(int cpunum, struct cpc_reg *reg, u64 val); extern int cppc_get_epp_perf(int cpunum, u64 *epp_perf); extern int cppc_set_epp_perf(int cpu, struct cppc_perf_ctrls *perf_ctrls, bool enable);