From patchwork Tue Mar 10 21:41:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Francisco Jerez X-Patchwork-Id: 11430319 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A43861731 for ; Tue, 10 Mar 2020 21:46:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7B7C421655 for ; Tue, 10 Mar 2020 21:46:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=riseup.net header.i=@riseup.net header.b="QwxoGYVE" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727677AbgCJVqY (ORCPT ); Tue, 10 Mar 2020 17:46:24 -0400 Received: from mx1.riseup.net ([198.252.153.129]:49184 "EHLO mx1.riseup.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726733AbgCJVqX (ORCPT ); Tue, 10 Mar 2020 17:46:23 -0400 Received: from bell.riseup.net (unknown [10.0.1.178]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 48cTDt53WnzFf56; Tue, 10 Mar 2020 14:46:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1583876782; bh=x6cAIXPzlQJJ1FEmSzTDuMyrZ/621gTCGn5sj7HNE1s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QwxoGYVERuVlEo8SG/MbDnITXtREUzWy7I2n5RXmF7UIfMHf7SD+dvdH2ykAPxEY3 DLEFB/5TCMruFongfqkFR5feO66QK5fJyR7tyI8+0R6m9jH52AsK3BvSDaQKKQ3qhQ Vk7TIOdu42D0cWoJf2oQbkxnseSePq89bI+b39rI= X-Riseup-User-ID: 04B7BD3E2701800EA3348DB9A10F00A0673674E61C3EC69C9978ABF3E94FEBB4 Received: from [127.0.0.1] (localhost [127.0.0.1]) by bell.riseup.net (Postfix) with ESMTPSA id 48cTDt36kwzJsFM; Tue, 10 Mar 2020 14:46:22 -0700 (PDT) From: Francisco Jerez To: linux-pm@vger.kernel.org, intel-gfx@lists.freedesktop.org Cc: "Rafael J. Wysocki" , "Pandruvada, Srinivas" , "Vivi, Rodrigo" , Peter Zijlstra Subject: [PATCH 01/10] PM: QoS: Add CPU_RESPONSE_FREQUENCY global PM QoS limit. Date: Tue, 10 Mar 2020 14:41:54 -0700 Message-Id: <20200310214203.26459-2-currojerez@riseup.net> In-Reply-To: <20200310214203.26459-1-currojerez@riseup.net> References: <20200310214203.26459-1-currojerez@riseup.net> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org The purpose of this PM QoS limit is to give device drivers additional control over the latency/energy efficiency trade-off made by the PM subsystem (particularly the CPUFREQ governor). It allows device drivers to set a lower bound on the response latency of PM (defined as the time it takes from wake-up to the CPU reaching a certain steady-state level of performance [e.g. the nominal frequency] in response to a step-function load). It reports to PM the minimum ramp-up latency considered of use to the application, and explicitly requests PM to filter out oscillations faster than the specified frequency. It is somewhat complementary to the current CPU_DMA_LATENCY PM QoS class which can be understood as specifying an upper latency bound on the CPU wake-up time, instead of a lower bound on the CPU frequency ramp-up time. Note that even though this provides a latency constraint it's represented as its reciprocal in Hz units for computational efficiency (since it would take a 64-bit division to compute the number of cycles elapsed from a time increment in nanoseconds and a time bound, while a frequency can simply be multiplied with the time increment). This implements a MAX constraint so that the strictest (highest response frequency) request is honored. This means that PM won't provide any guarantee that frequencies greater than the specified bound will be filtered, since that might be incompatible with the constraints specified by another more latency-sensitive application (A more fine-grained result could be achieved with a scheduling-based interface). The default value needs to be equal to zero (best effort) for it to behave as identity of the MAX operation. Signed-off-by: Francisco Jerez --- include/linux/pm_qos.h | 9 +++ include/trace/events/power.h | 33 ++++---- kernel/power/qos.c | 141 ++++++++++++++++++++++++++++++++++- 3 files changed, 165 insertions(+), 18 deletions(-) diff --git a/include/linux/pm_qos.h b/include/linux/pm_qos.h index 4a69d4af3ff8..b522e2194c05 100644 --- a/include/linux/pm_qos.h +++ b/include/linux/pm_qos.h @@ -28,6 +28,7 @@ enum pm_qos_flags_status { #define PM_QOS_LATENCY_ANY_NS ((s64)PM_QOS_LATENCY_ANY * NSEC_PER_USEC) #define PM_QOS_CPU_LATENCY_DEFAULT_VALUE (2000 * USEC_PER_SEC) +#define PM_QOS_CPU_RESPONSE_FREQUENCY_DEFAULT_VALUE 0 #define PM_QOS_RESUME_LATENCY_DEFAULT_VALUE PM_QOS_LATENCY_ANY #define PM_QOS_RESUME_LATENCY_NO_CONSTRAINT PM_QOS_LATENCY_ANY #define PM_QOS_RESUME_LATENCY_NO_CONSTRAINT_NS PM_QOS_LATENCY_ANY_NS @@ -162,6 +163,14 @@ static inline void cpu_latency_qos_update_request(struct pm_qos_request *req, static inline void cpu_latency_qos_remove_request(struct pm_qos_request *req) {} #endif +s32 cpu_response_frequency_qos_limit(void); +bool cpu_response_frequency_qos_request_active(struct pm_qos_request *req); +void cpu_response_frequency_qos_add_request(struct pm_qos_request *req, + s32 value); +void cpu_response_frequency_qos_update_request(struct pm_qos_request *req, + s32 new_value); +void cpu_response_frequency_qos_remove_request(struct pm_qos_request *req); + #ifdef CONFIG_PM enum pm_qos_flags_status __dev_pm_qos_flags(struct device *dev, s32 mask); enum pm_qos_flags_status dev_pm_qos_flags(struct device *dev, s32 mask); diff --git a/include/trace/events/power.h b/include/trace/events/power.h index af5018aa9517..7e4b52e8ca3a 100644 --- a/include/trace/events/power.h +++ b/include/trace/events/power.h @@ -359,45 +359,48 @@ DEFINE_EVENT(power_domain, power_domain_target, ); /* - * CPU latency QoS events used for global CPU latency QoS list updates + * CPU latency/response frequency QoS events used for global CPU PM + * QoS list updates. */ -DECLARE_EVENT_CLASS(cpu_latency_qos_request, +DECLARE_EVENT_CLASS(pm_qos_request, - TP_PROTO(s32 value), + TP_PROTO(const char *name, s32 value), - TP_ARGS(value), + TP_ARGS(name, value), TP_STRUCT__entry( + __string(name, name ) __field( s32, value ) ), TP_fast_assign( + __assign_str(name, name); __entry->value = value; ), - TP_printk("CPU_DMA_LATENCY value=%d", - __entry->value) + TP_printk("pm_qos_class=%s value=%d", + __get_str(name), __entry->value) ); -DEFINE_EVENT(cpu_latency_qos_request, pm_qos_add_request, +DEFINE_EVENT(pm_qos_request, pm_qos_add_request, - TP_PROTO(s32 value), + TP_PROTO(const char *name, s32 value), - TP_ARGS(value) + TP_ARGS(name, value) ); -DEFINE_EVENT(cpu_latency_qos_request, pm_qos_update_request, +DEFINE_EVENT(pm_qos_request, pm_qos_update_request, - TP_PROTO(s32 value), + TP_PROTO(const char *name, s32 value), - TP_ARGS(value) + TP_ARGS(name, value) ); -DEFINE_EVENT(cpu_latency_qos_request, pm_qos_remove_request, +DEFINE_EVENT(pm_qos_request, pm_qos_remove_request, - TP_PROTO(s32 value), + TP_PROTO(const char *name, s32 value), - TP_ARGS(value) + TP_ARGS(name, value) ); /* diff --git a/kernel/power/qos.c b/kernel/power/qos.c index 32927682bcc4..018491fecaac 100644 --- a/kernel/power/qos.c +++ b/kernel/power/qos.c @@ -271,7 +271,7 @@ void cpu_latency_qos_add_request(struct pm_qos_request *req, s32 value) return; } - trace_pm_qos_add_request(value); + trace_pm_qos_add_request("CPU_DMA_LATENCY", value); req->qos = &cpu_latency_constraints; cpu_latency_qos_apply(req, PM_QOS_ADD_REQ, value); @@ -297,7 +297,7 @@ void cpu_latency_qos_update_request(struct pm_qos_request *req, s32 new_value) return; } - trace_pm_qos_update_request(new_value); + trace_pm_qos_update_request("CPU_DMA_LATENCY", new_value); if (new_value == req->node.prio) return; @@ -323,7 +323,7 @@ void cpu_latency_qos_remove_request(struct pm_qos_request *req) return; } - trace_pm_qos_remove_request(PM_QOS_DEFAULT_VALUE); + trace_pm_qos_remove_request("CPU_DMA_LATENCY", PM_QOS_DEFAULT_VALUE); cpu_latency_qos_apply(req, PM_QOS_REMOVE_REQ, PM_QOS_DEFAULT_VALUE); memset(req, 0, sizeof(*req)); @@ -424,6 +424,141 @@ static int __init cpu_latency_qos_init(void) late_initcall(cpu_latency_qos_init); #endif /* CONFIG_CPU_IDLE */ +/* Definitions related to the CPU response frequency QoS. */ + +static struct pm_qos_constraints cpu_response_frequency_constraints = { + .list = PLIST_HEAD_INIT(cpu_response_frequency_constraints.list), + .target_value = PM_QOS_CPU_RESPONSE_FREQUENCY_DEFAULT_VALUE, + .default_value = PM_QOS_CPU_RESPONSE_FREQUENCY_DEFAULT_VALUE, + .no_constraint_value = PM_QOS_CPU_RESPONSE_FREQUENCY_DEFAULT_VALUE, + .type = PM_QOS_MAX, +}; + +/** + * cpu_response_frequency_qos_limit - Return current system-wide CPU + * response frequency QoS limit. + */ +s32 cpu_response_frequency_qos_limit(void) +{ + return pm_qos_read_value(&cpu_response_frequency_constraints); +} +EXPORT_SYMBOL_GPL(cpu_response_frequency_qos_limit); + +/** + * cpu_response_frequency_qos_request_active - Check the given PM QoS request. + * @req: PM QoS request to check. + * + * Return: 'true' if @req has been added to the CPU response frequency + * QoS list, 'false' otherwise. + */ +bool cpu_response_frequency_qos_request_active(struct pm_qos_request *req) +{ + return req->qos == &cpu_response_frequency_constraints; +} +EXPORT_SYMBOL_GPL(cpu_response_frequency_qos_request_active); + +static void cpu_response_frequency_qos_apply(struct pm_qos_request *req, + enum pm_qos_req_action action, + s32 value) +{ + int ret = pm_qos_update_target(req->qos, &req->node, action, value); + + if (ret > 0) + wake_up_all_idle_cpus(); +} + +/** + * cpu_response_frequency_qos_add_request - Add new CPU response + * frequency QoS request. + * @req: Pointer to a preallocated handle. + * @value: Requested constraint value. + * + * Use @value to initialize the request handle pointed to by @req, + * insert it as a new entry to the CPU response frequency QoS list and + * recompute the effective QoS constraint for that list. + * + * Callers need to save the handle for later use in updates and removal of the + * QoS request represented by it. + */ +void cpu_response_frequency_qos_add_request(struct pm_qos_request *req, + s32 value) +{ + if (!req) + return; + + if (cpu_response_frequency_qos_request_active(req)) { + WARN(1, KERN_ERR "%s called for already added request\n", + __func__); + return; + } + + trace_pm_qos_add_request("CPU_RESPONSE_FREQUENCY", value); + + req->qos = &cpu_response_frequency_constraints; + cpu_response_frequency_qos_apply(req, PM_QOS_ADD_REQ, value); +} +EXPORT_SYMBOL_GPL(cpu_response_frequency_qos_add_request); + +/** + * cpu_response_frequency_qos_update_request - Modify existing CPU + * response frequency QoS + * request. + * @req : QoS request to update. + * @new_value: New requested constraint value. + * + * Use @new_value to update the QoS request represented by @req in the + * CPU response frequency QoS list along with updating the effective + * constraint value for that list. + */ +void cpu_response_frequency_qos_update_request(struct pm_qos_request *req, + s32 new_value) +{ + if (!req) + return; + + if (!cpu_response_frequency_qos_request_active(req)) { + WARN(1, KERN_ERR "%s called for unknown object\n", __func__); + return; + } + + trace_pm_qos_update_request("CPU_RESPONSE_FREQUENCY", new_value); + + if (new_value == req->node.prio) + return; + + cpu_response_frequency_qos_apply(req, PM_QOS_UPDATE_REQ, new_value); +} +EXPORT_SYMBOL_GPL(cpu_response_frequency_qos_update_request); + +/** + * cpu_response_frequency_qos_remove_request - Remove existing CPU + * response frequency QoS + * request. + * @req: QoS request to remove. + * + * Remove the CPU response frequency QoS request represented by @req + * from the CPU response frequency QoS list along with updating the + * effective constraint value for that list. + */ +void cpu_response_frequency_qos_remove_request(struct pm_qos_request *req) +{ + if (!req) + return; + + if (!cpu_response_frequency_qos_request_active(req)) { + WARN(1, KERN_ERR "%s called for unknown object\n", __func__); + return; + } + + trace_pm_qos_remove_request("CPU_RESPONSE_FREQUENCY", + PM_QOS_DEFAULT_VALUE); + + cpu_response_frequency_qos_apply(req, PM_QOS_REMOVE_REQ, + PM_QOS_DEFAULT_VALUE); + memset(req, 0, sizeof(*req)); +} +EXPORT_SYMBOL_GPL(cpu_response_frequency_qos_remove_request); + /* Definitions related to the frequency QoS below. */ /** From patchwork Tue Mar 10 21:41:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Francisco Jerez X-Patchwork-Id: 11430321 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CF87D18E8 for ; Tue, 10 Mar 2020 21:46:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AE9AF22B48 for ; Tue, 10 Mar 2020 21:46:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=riseup.net header.i=@riseup.net header.b="PBy4iS8P" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726283AbgCJVqX (ORCPT ); Tue, 10 Mar 2020 17:46:23 -0400 Received: from mx1.riseup.net ([198.252.153.129]:49194 "EHLO mx1.riseup.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727506AbgCJVqX (ORCPT ); Tue, 10 Mar 2020 17:46:23 -0400 Received: from bell.riseup.net (unknown [10.0.1.178]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 48cTDt6gqJzFdkG; Tue, 10 Mar 2020 14:46:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1583876783; bh=JOWEXfrb9UFLH74QE5/WubqsiwOn7tHuUTjmrbOGpZQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PBy4iS8PUU5iVlG5k0ycH3oBaAc/XqVNoOh8MY5HHxA2Vkntb6binK6Wf72gpRvO2 /4P1BY2MHQtXYsnhABzgvb/+OMB7+JiR2CmpUyCvjrDjj6mQnpcfdz5UFSTFRQM7B1 02aFINuYN8SVJlZSmK1KHrNop0teNBwmG/bl6O9A= X-Riseup-User-ID: E9D8C4D7DC82611EFFCC156B3BA45C28F31077F6DF96799252081EE137E3455D Received: from [127.0.0.1] (localhost [127.0.0.1]) by bell.riseup.net (Postfix) with ESMTPSA id 48cTDt4mw5zJsJ5; Tue, 10 Mar 2020 14:46:22 -0700 (PDT) From: Francisco Jerez To: linux-pm@vger.kernel.org, intel-gfx@lists.freedesktop.org Cc: "Rafael J. Wysocki" , "Pandruvada, Srinivas" , "Vivi, Rodrigo" , Peter Zijlstra Subject: [PATCH 02/10] drm/i915: Adjust PM QoS response frequency based on GPU load. Date: Tue, 10 Mar 2020 14:41:55 -0700 Message-Id: <20200310214203.26459-3-currojerez@riseup.net> In-Reply-To: <20200310214203.26459-1-currojerez@riseup.net> References: <20200310214203.26459-1-currojerez@riseup.net> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org This allows CPUFREQ governors to realize when the system becomes non-CPU-bound due to GPU rendering activity, and cause them to respond more conservatively to the workload by limiting their response frequency: CPU energy usage will be reduced when there isn't a good chance for system performance to scale with CPU frequency due to the GPU bottleneck. This leaves additional TDP budget available for the GPU to reach higher frequencies, which is translated into an improvement in graphics performance to the extent that the workload remains TDP-limited (Most non-trivial graphics benchmarks out there improve significantly in the TDP-constrained platforms where this is currently enabled, see the cover letter for some numbers). If the workload isn't (anymore) TDP-limited performance should stay roughly constant, but energy usage will be divided by a similar factor. Signed-off-by: Francisco Jerez --- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 1 + drivers/gpu/drm/i915/gt/intel_engine_types.h | 7 ++ drivers/gpu/drm/i915/gt/intel_gt_pm.c | 107 +++++++++++++++++++ drivers/gpu/drm/i915/gt/intel_gt_pm.h | 3 + drivers/gpu/drm/i915/gt/intel_gt_types.h | 12 +++ drivers/gpu/drm/i915/gt/intel_lrc.c | 14 +++ 6 files changed, 144 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c index 53ac3f00909a..16ebdfa1dfc9 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c @@ -504,6 +504,7 @@ void intel_engine_init_execlists(struct intel_engine_cs *engine) execlists->queue_priority_hint = INT_MIN; execlists->queue = RB_ROOT_CACHED; + atomic_set(&execlists->overload, 0); } static void cleanup_status_page(struct intel_engine_cs *engine) diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h index 80cdde712842..1b17b2f0c7a3 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h @@ -266,6 +266,13 @@ struct intel_engine_execlists { */ u8 csb_head; + /** + * @overload: whether at least two execlist ports are + * currently submitted to the hardware, indicating that CPU + * latency isn't critical in order to maintain the GPU busy. + */ + atomic_t overload; + I915_SELFTEST_DECLARE(struct st_preempt_hang preempt_hang;) }; diff --git a/drivers/gpu/drm/i915/gt/intel_gt_pm.c b/drivers/gpu/drm/i915/gt/intel_gt_pm.c index 8b653c0f5e5f..f1f859e89a8f 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_pm.c +++ b/drivers/gpu/drm/i915/gt/intel_gt_pm.c @@ -107,6 +107,102 @@ void intel_gt_pm_init_early(struct intel_gt *gt) intel_wakeref_init(>->wakeref, gt->uncore->rpm, &wf_ops); } +/** + * Time increment until the most immediate PM QoS response frequency + * update. + * + * May be in the future (return value > 0) if the GPU is currently + * active but we haven't updated the PM QoS request to reflect a + * bottleneck yet. May be in the past (return value < 0) if the GPU + * isn't fully utilized and we've already reset the PM QoS request to + * the default value. May be zero if a PM QoS request update is due. + * + * The time increment returned by this function decreases linearly + * with time until it reaches either zero or a configurable limit. + */ +static int32_t time_to_rf_qos_update_ns(struct intel_gt *gt) +{ + const uint64_t t1 = ktime_get_ns(); + const uint64_t dt1 = gt->rf_qos.delay_max_ns; + + if (atomic_read_acquire(>->rf_qos.active_count)) { + const uint64_t t0 = atomic64_read(>->rf_qos.time_set_ns); + + return min(dt1, t0 <= t1 ? 0 : t0 - t1); + } else { + const uint64_t t0 = atomic64_read(>->rf_qos.time_clear_ns); + const unsigned int shift = gt->rf_qos.delay_slope_shift; + + return -(int32_t)(t1 <= t0 ? 1 : + min(dt1, (t1 - t0) << shift)); + } +} + +/** + * Perform a delayed PM QoS response frequency update. + */ +static void intel_gt_rf_qos_update(struct intel_gt *gt) +{ + const uint32_t dt = max(0, time_to_rf_qos_update_ns(gt)); + + timer_reduce(>->rf_qos.timer, jiffies + nsecs_to_jiffies(dt)); +} + +/** + * Timer that fires once the delay used to switch the PM QoS response + * frequency request has elapsed. + */ +static void intel_gt_rf_qos_timeout(struct timer_list *timer) +{ + struct intel_gt *gt = container_of(timer, struct intel_gt, + rf_qos.timer); + const int32_t dt = time_to_rf_qos_update_ns(gt); + + if (dt == 0) + cpu_response_frequency_qos_update_request( + >->rf_qos.req, gt->rf_qos.target_hz); + else + cpu_response_frequency_qos_update_request( + >->rf_qos.req, PM_QOS_DEFAULT_VALUE); + + if (dt > 0) + intel_gt_rf_qos_update(gt); +} + +/** + * Report the beginning of a period of GPU utilization to PM. + * + * May trigger a more energy-efficient response mode in CPU PM, but + * only after a certain delay has elapsed so we don't have a negative + * impact on the CPU ramp-up latency except after the GPU has been + * continuously utilized for a long enough period of time. + */ +void intel_gt_pm_active_begin(struct intel_gt *gt) +{ + const uint32_t dt = abs(time_to_rf_qos_update_ns(gt)); + + atomic64_set(>->rf_qos.time_set_ns, ktime_get_ns() + dt); + + if (!atomic_fetch_inc_release(>->rf_qos.active_count)) + intel_gt_rf_qos_update(gt); +} + +/** + * Report the end of a period of GPU utilization to PM. + * + * Must be called once after each call to intel_gt_pm_active_begin(). + */ +void intel_gt_pm_active_end(struct intel_gt *gt) +{ + const uint32_t dt = abs(time_to_rf_qos_update_ns(gt)); + const unsigned int shift = gt->rf_qos.delay_slope_shift; + + atomic64_set(>->rf_qos.time_clear_ns, ktime_get_ns() - (dt >> shift)); + + if (!atomic_dec_return_release(>->rf_qos.active_count)) + intel_gt_rf_qos_update(gt); +} + void intel_gt_pm_init(struct intel_gt *gt) { /* @@ -116,6 +212,14 @@ void intel_gt_pm_init(struct intel_gt *gt) */ intel_rc6_init(>->rc6); intel_rps_init(>->rps); + + cpu_response_frequency_qos_add_request(>->rf_qos.req, + PM_QOS_DEFAULT_VALUE); + + gt->rf_qos.delay_max_ns = 250000; + gt->rf_qos.delay_slope_shift = 0; + gt->rf_qos.target_hz = 2; + timer_setup(>->rf_qos.timer, intel_gt_rf_qos_timeout, 0); } static bool reset_engines(struct intel_gt *gt) @@ -170,6 +274,9 @@ static void gt_sanitize(struct intel_gt *gt, bool force) void intel_gt_pm_fini(struct intel_gt *gt) { + del_timer_sync(>->rf_qos.timer); + cpu_response_frequency_qos_remove_request(>->rf_qos.req); + intel_rc6_fini(>->rc6); } diff --git a/drivers/gpu/drm/i915/gt/intel_gt_pm.h b/drivers/gpu/drm/i915/gt/intel_gt_pm.h index 60f0e2fbe55c..43f1d45fb0db 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_pm.h +++ b/drivers/gpu/drm/i915/gt/intel_gt_pm.h @@ -58,6 +58,9 @@ int intel_gt_resume(struct intel_gt *gt); void intel_gt_runtime_suspend(struct intel_gt *gt); int intel_gt_runtime_resume(struct intel_gt *gt); +void intel_gt_pm_active_begin(struct intel_gt *gt); +void intel_gt_pm_active_end(struct intel_gt *gt); + static inline bool is_mock_gt(const struct intel_gt *gt) { return I915_SELFTEST_ONLY(gt->awake == -ENODEV); diff --git a/drivers/gpu/drm/i915/gt/intel_gt_types.h b/drivers/gpu/drm/i915/gt/intel_gt_types.h index 96890dd12b5f..4bc80c55e6f0 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_types.h +++ b/drivers/gpu/drm/i915/gt/intel_gt_types.h @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -97,6 +98,17 @@ struct intel_gt { * Reserved for exclusive use by the kernel. */ struct i915_address_space *vm; + + struct { + struct pm_qos_request req; + struct timer_list timer; + uint32_t target_hz; + uint32_t delay_max_ns; + uint32_t delay_slope_shift; + atomic64_t time_set_ns; + atomic64_t time_clear_ns; + atomic_t active_count; + } rf_qos; }; enum intel_gt_scratch_field { diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c index b9b3f78f1324..a5d7a80b826d 100644 --- a/drivers/gpu/drm/i915/gt/intel_lrc.c +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c @@ -1577,6 +1577,11 @@ static void execlists_submit_ports(struct intel_engine_cs *engine) /* we need to manually load the submit queue */ if (execlists->ctrl_reg) writel(EL_CTRL_LOAD, execlists->ctrl_reg); + + if (execlists_num_ports(execlists) > 1 && + execlists->pending[1] && + !atomic_xchg(&execlists->overload, 1)) + intel_gt_pm_active_begin(&engine->i915->gt); } static bool ctx_single_port_submission(const struct intel_context *ce) @@ -2213,6 +2218,12 @@ cancel_port_requests(struct intel_engine_execlists * const execlists) clear_ports(execlists->inflight, ARRAY_SIZE(execlists->inflight)); WRITE_ONCE(execlists->active, execlists->inflight); + + if (atomic_xchg(&execlists->overload, 0)) { + struct intel_engine_cs *engine = + container_of(execlists, typeof(*engine), execlists); + intel_gt_pm_active_end(&engine->i915->gt); + } } static inline void @@ -2386,6 +2397,9 @@ static void process_csb(struct intel_engine_cs *engine) /* port0 completed, advanced to port1 */ trace_ports(execlists, "completed", execlists->active); + if (atomic_xchg(&execlists->overload, 0)) + intel_gt_pm_active_end(&engine->i915->gt); + /* * We rely on the hardware being strongly * ordered, that the breadcrumb write is From patchwork Tue Mar 10 21:41:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Francisco Jerez X-Patchwork-Id: 11430323 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 05DB11874 for ; Tue, 10 Mar 2020 21:46:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D740B21655 for ; Tue, 10 Mar 2020 21:46:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=riseup.net header.i=@riseup.net header.b="a5msvmkc" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726733AbgCJVqY (ORCPT ); Tue, 10 Mar 2020 17:46:24 -0400 Received: from mx1.riseup.net ([198.252.153.129]:49210 "EHLO mx1.riseup.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727551AbgCJVqX (ORCPT ); Tue, 10 Mar 2020 17:46:23 -0400 Received: from bell.riseup.net (unknown [10.0.1.178]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 48cTDv1DSDzFf5Q; Tue, 10 Mar 2020 14:46:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1583876783; bh=Em3LQ/i9qE+XviHhjKpHAV1sZpQvpkA6BrliyYuH/uc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=a5msvmkcIO57AAGJc89+HHSxVg2LIbloDxw/HvHznTUtSxduq7znS+NRbCQzRXf8N 84+oHm3B93XfbIv96eOdF3rcZDbFi1o3EAXlYfNtyfjQhv69SQRS6BPriZZgiaNjdR 5TptE3BvJsn21dZ6at3kNt6pbEjeLoAOPwj3pKKA= X-Riseup-User-ID: 583ABCE62A94267CFC14F2494EA6EFFB9D4D6D5C94C1D8E07D8BB2D22A43EFE3 Received: from [127.0.0.1] (localhost [127.0.0.1]) by bell.riseup.net (Postfix) with ESMTPSA id 48cTDt6PcMzJrlc; Tue, 10 Mar 2020 14:46:22 -0700 (PDT) From: Francisco Jerez To: linux-pm@vger.kernel.org, intel-gfx@lists.freedesktop.org Cc: "Rafael J. Wysocki" , "Pandruvada, Srinivas" , "Vivi, Rodrigo" , Peter Zijlstra Subject: [PATCH 03/10] OPTIONAL: drm/i915: Expose PM QoS control parameters via debugfs. Date: Tue, 10 Mar 2020 14:41:56 -0700 Message-Id: <20200310214203.26459-4-currojerez@riseup.net> In-Reply-To: <20200310214203.26459-1-currojerez@riseup.net> References: <20200310214203.26459-1-currojerez@riseup.net> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org Signed-off-by: Francisco Jerez --- drivers/gpu/drm/i915/i915_debugfs.c | 69 +++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c index 8f2525e4ce0f..e5c27b9302d9 100644 --- a/drivers/gpu/drm/i915/i915_debugfs.c +++ b/drivers/gpu/drm/i915/i915_debugfs.c @@ -1745,6 +1745,72 @@ static const struct file_operations i915_guc_log_relay_fops = { .release = i915_guc_log_relay_release, }; +static int +i915_rf_qos_delay_max_ns_set(void *data, u64 val) +{ + struct drm_i915_private *dev_priv = data; + + WRITE_ONCE(dev_priv->gt.rf_qos.delay_max_ns, val); + return 0; +} + +static int +i915_rf_qos_delay_max_ns_get(void *data, u64 *val) +{ + struct drm_i915_private *dev_priv = data; + + *val = READ_ONCE(dev_priv->gt.rf_qos.delay_max_ns); + return 0; +} + +DEFINE_SIMPLE_ATTRIBUTE(i915_rf_qos_delay_max_ns_fops, + i915_rf_qos_delay_max_ns_get, + i915_rf_qos_delay_max_ns_set, "%llu\n"); + +static int +i915_rf_qos_delay_slope_shift_set(void *data, u64 val) +{ + struct drm_i915_private *dev_priv = data; + + WRITE_ONCE(dev_priv->gt.rf_qos.delay_slope_shift, val); + return 0; +} + +static int +i915_rf_qos_delay_slope_shift_get(void *data, u64 *val) +{ + struct drm_i915_private *dev_priv = data; + + *val = READ_ONCE(dev_priv->gt.rf_qos.delay_slope_shift); + return 0; +} + +DEFINE_SIMPLE_ATTRIBUTE(i915_rf_qos_delay_slope_shift_fops, + i915_rf_qos_delay_slope_shift_get, + i915_rf_qos_delay_slope_shift_set, "%llu\n"); + +static int +i915_rf_qos_target_hz_set(void *data, u64 val) +{ + struct drm_i915_private *dev_priv = data; + + WRITE_ONCE(dev_priv->gt.rf_qos.target_hz, val); + return 0; +} + +static int +i915_rf_qos_target_hz_get(void *data, u64 *val) +{ + struct drm_i915_private *dev_priv = data; + + *val = READ_ONCE(dev_priv->gt.rf_qos.target_hz); + return 0; +} + +DEFINE_SIMPLE_ATTRIBUTE(i915_rf_qos_target_hz_fops, + i915_rf_qos_target_hz_get, + i915_rf_qos_target_hz_set, "%llu\n"); + static int i915_runtime_pm_status(struct seq_file *m, void *unused) { struct drm_i915_private *dev_priv = node_to_i915(m->private); @@ -2390,6 +2456,9 @@ static const struct i915_debugfs_files { #endif {"i915_guc_log_level", &i915_guc_log_level_fops}, {"i915_guc_log_relay", &i915_guc_log_relay_fops}, + {"i915_rf_qos_delay_max_ns", &i915_rf_qos_delay_max_ns_fops}, + {"i915_rf_qos_delay_slope_shift", &i915_rf_qos_delay_slope_shift_fops}, + {"i915_rf_qos_target_hz", &i915_rf_qos_target_hz_fops} }; int i915_debugfs_register(struct drm_i915_private *dev_priv) From patchwork Tue Mar 10 21:41:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Francisco Jerez X-Patchwork-Id: 11430327 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8A79C1731 for ; Tue, 10 Mar 2020 21:46:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 606632467F for ; Tue, 10 Mar 2020 21:46:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=riseup.net header.i=@riseup.net header.b="ahBzDH/l" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727141AbgCJVqY (ORCPT ); Tue, 10 Mar 2020 17:46:24 -0400 Received: from mx1.riseup.net ([198.252.153.129]:49224 "EHLO mx1.riseup.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727673AbgCJVqY (ORCPT ); Tue, 10 Mar 2020 17:46:24 -0400 Received: from bell.riseup.net (unknown [10.0.1.178]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 48cTDv2v8HzFf6D; Tue, 10 Mar 2020 14:46:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1583876783; bh=Jz4j4rXwmY24uT7RUVUWzIpW3YCyx81EGlePfYaHr9U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ahBzDH/lbNIX7yWW3BIUwwu26RLDxx9fEW/+/NmT2TS61OKnvx91T4khRyCf8qY68 EpVR/IoSjGWaGToHXlNS8g4K2JJferjdALjhm4EZB2K+5JjQrxIl2xNDGxmSVm1urt S8NlxoIfHDUZ2/qPY5jnHHfT2a8GsJO+EKFymWSc= X-Riseup-User-ID: C7F74CCE029929C25DCF61978D0E953392C596615385353591C57293F5D3D979 Received: from [127.0.0.1] (localhost [127.0.0.1]) by bell.riseup.net (Postfix) with ESMTPSA id 48cTDv11mnzJs07; Tue, 10 Mar 2020 14:46:23 -0700 (PDT) From: Francisco Jerez To: linux-pm@vger.kernel.org, intel-gfx@lists.freedesktop.org Cc: "Rafael J. Wysocki" , "Pandruvada, Srinivas" , "Vivi, Rodrigo" , Peter Zijlstra Subject: [PATCH 04/10] Revert "cpufreq: intel_pstate: Drop ->update_util from pstate_funcs" Date: Tue, 10 Mar 2020 14:41:57 -0700 Message-Id: <20200310214203.26459-5-currojerez@riseup.net> In-Reply-To: <20200310214203.26459-1-currojerez@riseup.net> References: <20200310214203.26459-1-currojerez@riseup.net> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org This reverts commit c4f3f70cacba2fa19545389a12d09b606d2ad1cf. A future commit will introduce a new update_util implementation, so the pstate_funcs table entry is going to be useful. Signed-off-by: Francisco Jerez --- drivers/cpufreq/intel_pstate.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c index 7fa869004cf0..8cb5bf419b40 100644 --- a/drivers/cpufreq/intel_pstate.c +++ b/drivers/cpufreq/intel_pstate.c @@ -277,6 +277,7 @@ static struct cpudata **all_cpu_data; * @get_scaling: Callback to get frequency scaling factor * @get_val: Callback to convert P state to actual MSR write value * @get_vid: Callback to get VID data for Atom platforms + * @update_util: Active mode utilization update callback. * * Core and Atom CPU models have different way to get P State limits. This * structure is used to store those callbacks. @@ -290,6 +291,8 @@ struct pstate_funcs { int (*get_aperf_mperf_shift)(void); u64 (*get_val)(struct cpudata*, int pstate); void (*get_vid)(struct cpudata *); + void (*update_util)(struct update_util_data *data, u64 time, + unsigned int flags); }; static struct pstate_funcs pstate_funcs __read_mostly; @@ -1877,6 +1880,7 @@ static struct pstate_funcs core_funcs = { .get_turbo = core_get_turbo_pstate, .get_scaling = core_get_scaling, .get_val = core_get_val, + .update_util = intel_pstate_update_util, }; static const struct pstate_funcs silvermont_funcs = { @@ -1887,6 +1891,7 @@ static const struct pstate_funcs silvermont_funcs = { .get_val = atom_get_val, .get_scaling = silvermont_get_scaling, .get_vid = atom_get_vid, + .update_util = intel_pstate_update_util, }; static const struct pstate_funcs airmont_funcs = { @@ -1897,6 +1902,7 @@ static const struct pstate_funcs airmont_funcs = { .get_val = atom_get_val, .get_scaling = airmont_get_scaling, .get_vid = atom_get_vid, + .update_util = intel_pstate_update_util, }; static const struct pstate_funcs knl_funcs = { @@ -1907,6 +1913,7 @@ static const struct pstate_funcs knl_funcs = { .get_aperf_mperf_shift = knl_get_aperf_mperf_shift, .get_scaling = core_get_scaling, .get_val = core_get_val, + .update_util = intel_pstate_update_util, }; #define ICPU(model, policy) \ @@ -2013,9 +2020,7 @@ static void intel_pstate_set_update_util_hook(unsigned int cpu_num) /* Prevent intel_pstate_update_util() from using stale data. */ cpu->sample.time = 0; cpufreq_add_update_util_hook(cpu_num, &cpu->update_util, - (hwp_active ? - intel_pstate_update_util_hwp : - intel_pstate_update_util)); + pstate_funcs.update_util); cpu->update_util_set = true; } @@ -2584,6 +2589,7 @@ static void __init copy_cpu_funcs(struct pstate_funcs *funcs) pstate_funcs.get_scaling = funcs->get_scaling; pstate_funcs.get_val = funcs->get_val; pstate_funcs.get_vid = funcs->get_vid; + pstate_funcs.update_util = funcs->update_util; pstate_funcs.get_aperf_mperf_shift = funcs->get_aperf_mperf_shift; } @@ -2750,8 +2756,11 @@ static int __init intel_pstate_init(void) id = x86_match_cpu(hwp_support_ids); if (id) { copy_cpu_funcs(&core_funcs); - if (!no_hwp) { + if (no_hwp) { + pstate_funcs.update_util = intel_pstate_update_util; + } else { hwp_active++; + pstate_funcs.update_util = intel_pstate_update_util_hwp; hwp_mode_bdw = id->driver_data; intel_pstate.attr = hwp_cpufreq_attrs; goto hwp_cpu_matched; From patchwork Tue Mar 10 21:41:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Francisco Jerez X-Patchwork-Id: 11430325 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5661D924 for ; Tue, 10 Mar 2020 21:46:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3614B222C4 for ; Tue, 10 Mar 2020 21:46:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=riseup.net header.i=@riseup.net header.b="c1z0yYRy" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727691AbgCJVqY (ORCPT ); Tue, 10 Mar 2020 17:46:24 -0400 Received: from mx1.riseup.net ([198.252.153.129]:49230 "EHLO mx1.riseup.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727506AbgCJVqY (ORCPT ); Tue, 10 Mar 2020 17:46:24 -0400 Received: from bell.riseup.net (unknown [10.0.1.178]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 48cTDv48vSzFf6s; Tue, 10 Mar 2020 14:46:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1583876783; bh=0yX01JJFStKAYRhSVlYvPVR/0Is5p2IQSnPOaQhexy0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=c1z0yYRywJe0yR/iqrAH57P7MVw0ePcf/4aZfypegv7+hwutrt7Djffd5gCFBn9rX mPZhHgYYWpxZGLkR+LIAeRnB8dizURZpcd57A89SPM8n4ve+tsqkxPCK+p84Str5rF Vio4YgXUNyjBONNcxItsVZx4f2/i7rnk8oAg3jvY= X-Riseup-User-ID: 7730B927CA4233461DB3793177BE823CBE662ADA5D87858D0448C0AEBCA798FD Received: from [127.0.0.1] (localhost [127.0.0.1]) by bell.riseup.net (Postfix) with ESMTPSA id 48cTDv2QLDzJrlc; Tue, 10 Mar 2020 14:46:23 -0700 (PDT) From: Francisco Jerez To: linux-pm@vger.kernel.org, intel-gfx@lists.freedesktop.org Cc: "Rafael J. Wysocki" , "Pandruvada, Srinivas" , "Vivi, Rodrigo" , Peter Zijlstra Subject: [PATCH 05/10] cpufreq: intel_pstate: Implement VLP controller statistics and status calculation. Date: Tue, 10 Mar 2020 14:41:58 -0700 Message-Id: <20200310214203.26459-6-currojerez@riseup.net> In-Reply-To: <20200310214203.26459-1-currojerez@riseup.net> References: <20200310214203.26459-1-currojerez@riseup.net> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org The goal of the helper code introduced here is to compute two informational data structures: struct vlp_input_stats aggregating various scheduling and PM statistics gathered in every call of the update_util() hook, and struct vlp_status_sample which contains status information derived from the former indicating whether the system is likely to have an IO or CPU bottleneck. This will be used as main heuristic input by the new variably low-pass filtering controller (AKA VLP) that will assist the HWP at finding a reasonably energy-efficient P-state given the additional information available to the kernel about I/O utilization and scheduling behavior. Signed-off-by: Francisco Jerez --- drivers/cpufreq/intel_pstate.c | 230 +++++++++++++++++++++++++++++++++ 1 file changed, 230 insertions(+) diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c index 8cb5bf419b40..12ee350db2a9 100644 --- a/drivers/cpufreq/intel_pstate.c +++ b/drivers/cpufreq/intel_pstate.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -33,6 +34,8 @@ #include #include +#include "../../kernel/sched/sched.h" + #define INTEL_PSTATE_SAMPLING_INTERVAL (10 * NSEC_PER_MSEC) #define INTEL_CPUFREQ_TRANSITION_LATENCY 20000 @@ -59,6 +62,11 @@ static inline int32_t mul_fp(int32_t x, int32_t y) return ((int64_t)x * (int64_t)y) >> FRAC_BITS; } +static inline int rnd_fp(int32_t x) +{ + return (x + (1 << (FRAC_BITS - 1))) >> FRAC_BITS; +} + static inline int32_t div_fp(s64 x, s64 y) { return div64_s64((int64_t)x << FRAC_BITS, y); @@ -169,6 +177,49 @@ struct vid_data { int32_t ratio; }; +/** + * Scheduling and PM statistics gathered by update_vlp_sample() at + * every call of the VLP update_state() hook, used as heuristic + * inputs. + */ +struct vlp_input_stats { + int32_t realtime_count; + int32_t io_wait_count; + uint32_t max_response_frequency_hz; + uint32_t last_response_frequency_hz; +}; + +enum vlp_status { + VLP_BOTTLENECK_IO = 1 << 0, + /* + * XXX - Add other status bits here indicating a CPU or TDP + * bottleneck. + */ +}; + +/** + * Heuristic status information calculated by get_vlp_status_sample() + * from struct vlp_input_stats above, indicating whether the system + * has a potential IO or latency bottleneck. + */ +struct vlp_status_sample { + enum vlp_status value; + int32_t realtime_avg; +}; + +/** + * struct vlp_data - VLP controller parameters and state. + * @sample_interval_ns: Update interval in ns. + * @sample_frequency_hz: Reciprocal of the update interval in Hz. + */ +struct vlp_data { + s64 sample_interval_ns; + int32_t sample_frequency_hz; + + struct vlp_input_stats stats; + struct vlp_status_sample status; +}; + /** * struct global_params - Global parameters, mostly tunable via sysfs. * @no_turbo: Whether or not to use turbo P-states. @@ -239,6 +290,7 @@ struct cpudata { struct pstate_data pstate; struct vid_data vid; + struct vlp_data vlp; u64 last_update; u64 last_sample_time; @@ -268,6 +320,18 @@ struct cpudata { static struct cpudata **all_cpu_data; +/** + * struct vlp_params - VLP controller static configuration + * @sample_interval_ms: Update interval in ms. + * @avg*_hz: Exponential averaging frequencies of the various + * low-pass filters as an integer in Hz. + */ +struct vlp_params { + int sample_interval_ms; + int avg_hz; + int debug; +}; + /** * struct pstate_funcs - Per CPU model specific callbacks * @get_max: Callback to get maximum non turbo effective P state @@ -296,6 +360,11 @@ struct pstate_funcs { }; static struct pstate_funcs pstate_funcs __read_mostly; +static struct vlp_params vlp_params __read_mostly = { + .sample_interval_ms = 10, + .avg_hz = 2, + .debug = 0, +}; static int hwp_active __read_mostly; static int hwp_mode_bdw __read_mostly; @@ -1793,6 +1862,167 @@ static inline int32_t get_target_pstate(struct cpudata *cpu) return target; } +/** + * Initialize the struct vlp_data of the specified CPU to the defaults + * calculated from @vlp_params. + */ +static void intel_pstate_reset_vlp(struct cpudata *cpu) +{ + struct vlp_data *vlp = &cpu->vlp; + + vlp->sample_interval_ns = vlp_params.sample_interval_ms * NSEC_PER_MSEC; + vlp->sample_frequency_hz = max(1u, (uint32_t)MSEC_PER_SEC / + vlp_params.sample_interval_ms); + vlp->stats.last_response_frequency_hz = vlp_params.avg_hz; +} + +/** + * Fixed point representation with twice the usual number of + * fractional bits. + */ +#define DFRAC_BITS 16 +#define DFRAC_ONE (1 << DFRAC_BITS) +#define DFRAC_MAX_INT (0u - (uint32_t)DFRAC_ONE) + +/** + * Fast but rather inaccurate piecewise-linear approximation of a + * fixed-point inverse exponential: + * + * exp2n(p) = int_tofp(1) * 2 ^ (-p / DFRAC_ONE) + O(1) + * + * The error term should be lower in magnitude than 0.044. + */ +static int32_t exp2n(uint32_t p) +{ + if (p < 32 * DFRAC_ONE) { + /* Interpolate between 2^-floor(p) and 2^-ceil(p). */ + const uint32_t floor_p = p >> DFRAC_BITS; + const uint32_t ceil_p = (p + DFRAC_ONE - 1) >> DFRAC_BITS; + const uint64_t frac_p = p - (floor_p << DFRAC_BITS); + + return ((int_tofp(1) >> floor_p) * (DFRAC_ONE - frac_p) + + (ceil_p >= 32 ? 0 : int_tofp(1) >> ceil_p) * frac_p) >> + DFRAC_BITS; + } + + /* Short-circuit to avoid overflow. */ + return 0; +} + +/** + * Calculate the exponential averaging weight for a new sample based + * on the requested averaging frequency @hz and the delay since the + * last update. + */ +static int32_t get_last_sample_avg_weight(struct cpudata *cpu, unsigned int hz) +{ + /* + * Approximate, but saves several 64-bit integer divisions + * below and should be fully evaluated at compile-time. + * Causes the exponential averaging to have an effective base + * of 1.90702343749, which has little functional implications + * as long as the hz parameter is scaled accordingly. + */ + const uint32_t ns_per_s_shift = order_base_2(NSEC_PER_SEC); + const uint64_t delta_ns = cpu->sample.time - cpu->last_sample_time; + + return exp2n(min((uint64_t)DFRAC_MAX_INT, + (hz * delta_ns) >> (ns_per_s_shift - DFRAC_BITS))); +} + +/** + * Calculate some status information heuristically based on the struct + * vlp_input_stats statistics gathered by the update_state() hook. + */ +static const struct vlp_status_sample *get_vlp_status_sample( + struct cpudata *cpu, const int32_t po) +{ + struct vlp_data *vlp = &cpu->vlp; + struct vlp_input_stats *stats = &vlp->stats; + struct vlp_status_sample *last_status = &vlp->status; + + /* + * Calculate the VLP_BOTTLENECK_IO state bit, which indicates + * whether some IO device driver has requested a PM response + * frequency bound, typically due to the device being under + * close to full utilization, which should cause the + * controller to make a more conservative trade-off between + * latency and energy usage, since performance isn't + * guaranteed to scale further with increasing CPU frequency + * whenever the system is close to IO-bound. + * + * Note that the maximum achievable response frequency is + * limited by the sampling frequency of the controller, + * response frequency requests greater than that will be + * promoted to infinity (i.e. no low-pass filtering) in order + * to avoid violating the response frequency constraint + * provided via PM QoS. + */ + const bool bottleneck_io = stats->max_response_frequency_hz < + vlp->sample_frequency_hz; + + /* + * Calculate the realtime statistic that tracks the + * exponentially-averaged rate of occurrence of + * latency-sensitive events (like wake-ups from IO wait). + */ + const uint64_t delta_ns = cpu->sample.time - cpu->last_sample_time; + const int32_t realtime_sample = + div_fp((uint64_t)(stats->realtime_count + + (bottleneck_io ? 0 : stats->io_wait_count)) * + NSEC_PER_SEC, + 100 * delta_ns); + const int32_t alpha = get_last_sample_avg_weight(cpu, + vlp_params.avg_hz); + const int32_t realtime_avg = realtime_sample + + mul_fp(alpha, last_status->realtime_avg - realtime_sample); + + /* Consume the input statistics. */ + stats->io_wait_count = 0; + stats->realtime_count = 0; + if (bottleneck_io) + stats->last_response_frequency_hz = + stats->max_response_frequency_hz; + stats->max_response_frequency_hz = 0; + + /* Update the state of the controller. */ + last_status->realtime_avg = realtime_avg; + last_status->value = (bottleneck_io ? VLP_BOTTLENECK_IO : 0); + + /* Update state used for tracing. */ + cpu->sample.busy_scaled = int_tofp(stats->max_response_frequency_hz); + cpu->iowait_boost = realtime_avg; + + return last_status; +} + +/** + * Collect some scheduling and PM statistics in response to an + * update_state() call. + */ +static bool update_vlp_sample(struct cpudata *cpu, u64 time, unsigned int flags) +{ + struct vlp_input_stats *stats = &cpu->vlp.stats; + + /* Update PM QoS request. */ + const uint32_t resp_hz = cpu_response_frequency_qos_limit(); + + stats->max_response_frequency_hz = !resp_hz ? UINT_MAX : + max(stats->max_response_frequency_hz, resp_hz); + + /* Update scheduling statistics. */ + if ((flags & SCHED_CPUFREQ_IOWAIT)) + stats->io_wait_count++; + + if (cpu_rq(cpu->cpu)->rt.rt_nr_running) + stats->realtime_count++; + + /* Return whether a P-state update is due. */ + return smp_processor_id() == cpu->cpu && + time - cpu->sample.time >= cpu->vlp.sample_interval_ns && + intel_pstate_sample(cpu, time); +} + static int intel_pstate_prepare_request(struct cpudata *cpu, int pstate) { int min_pstate = max(cpu->pstate.min_pstate, cpu->min_perf_ratio); From patchwork Tue Mar 10 21:41:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Francisco Jerez X-Patchwork-Id: 11430337 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0DA7218E8 for ; Tue, 10 Mar 2020 21:46:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E0FF321655 for ; Tue, 10 Mar 2020 21:46:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=riseup.net header.i=@riseup.net header.b="SsE+jQEC" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727551AbgCJVqZ (ORCPT ); Tue, 10 Mar 2020 17:46:25 -0400 Received: from mx1.riseup.net ([198.252.153.129]:49234 "EHLO mx1.riseup.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727685AbgCJVqY (ORCPT ); Tue, 10 Mar 2020 17:46:24 -0400 Received: from bell.riseup.net (unknown [10.0.1.178]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 48cTDv5qNzzFf7g; Tue, 10 Mar 2020 14:46:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1583876783; bh=KEkzgcuMGqyPvBrxJ4k0uNT7bSd/BkqPcZ27lj5X3iQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SsE+jQECn1EKJsgYYTSBo6F1id/YM6VWt39fclFqNhwoM4MoRKR8K7uFm7/SJR1TD heL7OFWXINHXDJpgE5u8Dww6bpjrPnEH2erUnKd408jVOSQDsJhHEY0uvLXmw630YI k32HpJUJvsY0yI9Ox1WvM44Zjk4oX12SVhTpmd98= X-Riseup-User-ID: AD4EAA60D84A63821C196DFB6AFBF4BEDE6F01FBB0D27219F575F5C9C11F17D0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by bell.riseup.net (Postfix) with ESMTPSA id 48cTDv3zs7zJsFM; Tue, 10 Mar 2020 14:46:23 -0700 (PDT) From: Francisco Jerez To: linux-pm@vger.kernel.org, intel-gfx@lists.freedesktop.org Cc: "Rafael J. Wysocki" , "Pandruvada, Srinivas" , "Vivi, Rodrigo" , Peter Zijlstra Subject: [PATCH 06/10] cpufreq: intel_pstate: Implement VLP controller target P-state range estimation. Date: Tue, 10 Mar 2020 14:41:59 -0700 Message-Id: <20200310214203.26459-7-currojerez@riseup.net> In-Reply-To: <20200310214203.26459-1-currojerez@riseup.net> References: <20200310214203.26459-1-currojerez@riseup.net> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org The function introduced here calculates a P-state range derived from the statistics computed in the previous patch which will be used to drive the HWP P-state range or (if HWP is not available) as basis for some additional kernel-side frequency selection mechanism which will choose a single P-state from the range. This is meant to provide a variably low-pass filtering effect that will damp oscillations below a frequency threshold that can be specified by device drivers via PM QoS in order to achieve energy-efficient behavior in cases where the system has an IO bottleneck. Signed-off-by: Francisco Jerez --- drivers/cpufreq/intel_pstate.c | 157 +++++++++++++++++++++++++++++++++ 1 file changed, 157 insertions(+) diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c index 12ee350db2a9..cecadfec8bc1 100644 --- a/drivers/cpufreq/intel_pstate.c +++ b/drivers/cpufreq/intel_pstate.c @@ -207,17 +207,34 @@ struct vlp_status_sample { int32_t realtime_avg; }; +/** + * VLP controller state used for the estimation of the target P-state + * range, computed by get_vlp_target_range() from the heuristic status + * information defined above in struct vlp_status_sample. + */ +struct vlp_target_range { + unsigned int value[2]; + int32_t p_base; +}; + /** * struct vlp_data - VLP controller parameters and state. * @sample_interval_ns: Update interval in ns. * @sample_frequency_hz: Reciprocal of the update interval in Hz. + * @gain*: Response factor of the controller relative to each + * one of its linear input variables as fixed-point + * fraction. */ struct vlp_data { s64 sample_interval_ns; int32_t sample_frequency_hz; + int32_t gain_aggr; + int32_t gain_rt; + int32_t gain; struct vlp_input_stats stats; struct vlp_status_sample status; + struct vlp_target_range target; }; /** @@ -323,12 +340,18 @@ static struct cpudata **all_cpu_data; /** * struct vlp_params - VLP controller static configuration * @sample_interval_ms: Update interval in ms. + * @setpoint_*_pml: Target CPU utilization at which the controller is + * expected to leave the current P-state untouched, + * as an integer per mille. * @avg*_hz: Exponential averaging frequencies of the various * low-pass filters as an integer in Hz. */ struct vlp_params { int sample_interval_ms; + int setpoint_0_pml; + int setpoint_aggr_pml; int avg_hz; + int realtime_gain_pml; int debug; }; @@ -362,7 +385,10 @@ struct pstate_funcs { static struct pstate_funcs pstate_funcs __read_mostly; static struct vlp_params vlp_params __read_mostly = { .sample_interval_ms = 10, + .setpoint_0_pml = 900, + .setpoint_aggr_pml = 1500, .avg_hz = 2, + .realtime_gain_pml = 12000, .debug = 0, }; @@ -1873,6 +1899,11 @@ static void intel_pstate_reset_vlp(struct cpudata *cpu) vlp->sample_interval_ns = vlp_params.sample_interval_ms * NSEC_PER_MSEC; vlp->sample_frequency_hz = max(1u, (uint32_t)MSEC_PER_SEC / vlp_params.sample_interval_ms); + vlp->gain_rt = div_fp(cpu->pstate.max_pstate * + vlp_params.realtime_gain_pml, 1000); + vlp->gain_aggr = max(1, div_fp(1000, vlp_params.setpoint_aggr_pml)); + vlp->gain = max(1, div_fp(1000, vlp_params.setpoint_0_pml)); + vlp->target.p_base = 0; vlp->stats.last_response_frequency_hz = vlp_params.avg_hz; } @@ -1996,6 +2027,132 @@ static const struct vlp_status_sample *get_vlp_status_sample( return last_status; } +/** + * Calculate the target P-state range for the next update period. + * Uses a variably low-pass-filtering controller intended to improve + * energy efficiency when a CPU response frequency target is specified + * via PM QoS (e.g. under IO-bound conditions). + */ +static const struct vlp_target_range *get_vlp_target_range(struct cpudata *cpu) +{ + struct vlp_data *vlp = &cpu->vlp; + struct vlp_target_range *last_target = &vlp->target; + + /* + * P-state limits in fixed-point as allowed by the policy. + */ + const int32_t p0 = int_tofp(max(cpu->pstate.min_pstate, + cpu->min_perf_ratio)); + const int32_t p1 = int_tofp(cpu->max_perf_ratio); + + /* + * Observed average P-state during the sampling period. The + * conservative path (po_cons) uses the TSC increment as + * denominator which will give the minimum (arguably most + * energy-efficient) P-state able to accomplish the observed + * amount of work during the sampling period. + * + * The downside of that somewhat optimistic estimate is that + * it can give a biased result for intermittent + * latency-sensitive workloads, which may have to be completed + * in a short window of time for the system to achieve maximum + * performance, even if the average CPU utilization is low. + * For that reason the aggressive path (po_aggr) uses the + * MPERF increment as denominator, which is approximately + * optimal under the pessimistic assumption that the CPU work + * cannot be parallelized with any other dependent IO work + * that subsequently keeps the CPU idle (partly in C1+ + * states). + */ + const int32_t po_cons = + div_fp((cpu->sample.aperf << cpu->aperf_mperf_shift) + * cpu->pstate.max_pstate_physical, + cpu->sample.tsc); + const int32_t po_aggr = + div_fp((cpu->sample.aperf << cpu->aperf_mperf_shift) + * cpu->pstate.max_pstate_physical, + (cpu->sample.mperf << cpu->aperf_mperf_shift)); + + const struct vlp_status_sample *status = + get_vlp_status_sample(cpu, po_cons); + + /* Calculate the target P-state. */ + const int32_t p_tgt_cons = mul_fp(vlp->gain, po_cons); + const int32_t p_tgt_aggr = mul_fp(vlp->gain_aggr, po_aggr); + const int32_t p_tgt = max(p0, min(p1, max(p_tgt_cons, p_tgt_aggr))); + + /* Calculate the realtime P-state target lower bound. */ + const int32_t pm = int_tofp(cpu->pstate.max_pstate); + const int32_t p_tgt_rt = min(pm, mul_fp(vlp->gain_rt, + status->realtime_avg)); + + /* + * Low-pass filter the P-state estimate above by exponential + * averaging. For an oscillating workload (e.g. submitting + * work repeatedly to a device like a soundcard or GPU) this + * will approximate the minimum P-state that would be able to + * accomplish the observed amount of work during the averaging + * period, which is also the optimally energy-efficient one, + * under the assumptions that: + * + * - The power curve of the system is convex throughout the + * range of P-states allowed by the policy. I.e. energy + * efficiency is steadily decreasing with frequency past p0 + * (which is typically close to the maximum-efficiency + * ratio). In practice for the lower range of P-states + * this may only be approximately true due to the + * interaction between different components of the system. + * + * - Parallelism constraints of the workload don't prevent it + * from achieving the same throughput at the lower P-state. + * This will happen in cases where the application is + * designed in a way that doesn't allow for dependent CPU + * and IO jobs to be pipelined, leading to alternating full + * and zero utilization of the CPU and IO device. This + * will give an average IO device utilization lower than + * 100% regardless of the CPU frequency, which should + * prevent the device driver from requesting a response + * frequency bound, so the filtered P-state calculated + * below won't have an influence on the controller + * response. + * + * - The period of the oscillating workload is significantly + * shorter than the time constant of the exponential + * average (1s / last_response_frequency_hz). Otherwise for + * more slowly oscillating workloads the controller + * response will roughly follow the oscillation, leading to + * decreased energy efficiency. + * + * - The behavior of the workload doesn't change + * qualitatively during the next update interval. This is + * only true in the steady state, and could possibly lead + * to a transitory period in which the controller response + * deviates from the most energy-efficient ratio until the + * workload reaches a steady state again. + */ + const int32_t alpha = get_last_sample_avg_weight( + cpu, vlp->stats.last_response_frequency_hz); + + last_target->p_base = p_tgt + mul_fp(alpha, + last_target->p_base - p_tgt); + + /* + * Use the low-pass-filtered controller response for better + * energy efficiency unless we have reasons to believe that + * some of the optimality assumptions discussed above may not + * hold. + */ + if ((status->value & VLP_BOTTLENECK_IO)) { + last_target->value[0] = rnd_fp(p0); + last_target->value[1] = rnd_fp(last_target->p_base); + } else { + last_target->value[0] = rnd_fp(p_tgt_rt); + last_target->value[1] = rnd_fp(p1); + } + + return last_target; +} + /** * Collect some scheduling and PM statistics in response to an * update_state() call. From patchwork Tue Mar 10 21:42:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Francisco Jerez X-Patchwork-Id: 11430329 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DE7DF1874 for ; Tue, 10 Mar 2020 21:46:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BD7D7222C3 for ; Tue, 10 Mar 2020 21:46:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=riseup.net header.i=@riseup.net header.b="BEQDPbgw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727506AbgCJVqY (ORCPT ); Tue, 10 Mar 2020 17:46:24 -0400 Received: from mx1.riseup.net ([198.252.153.129]:49240 "EHLO mx1.riseup.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727551AbgCJVqY (ORCPT ); Tue, 10 Mar 2020 17:46:24 -0400 Received: from bell.riseup.net (unknown [10.0.1.178]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 48cTDw0378zFf59; Tue, 10 Mar 2020 14:46:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1583876784; bh=ltwuhymiRp0VE2eGHyBbW5dhX5HoTXYQ8tBDbkFknOM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BEQDPbgw4AFgInKuxEXiyG6FyPYZvsQhtCWhvbRFbNd1QwRwlwiaAucd5RGHSlcvb +nwClGR6I18+Jef9P791Em/n0bngfm31Vd/R9TnNzvW8T/65UHLhtjtIAp30kxszpe 6xw6eKaGNg5QyLmzoVaZm/E2rOBI3pWsPvGibZFg= X-Riseup-User-ID: 27D01099CFA45474A2CF5CDB74DB5D1B94D423BF3BF6EE795CB3C483527536A2 Received: from [127.0.0.1] (localhost [127.0.0.1]) by bell.riseup.net (Postfix) with ESMTPSA id 48cTDv5PBzzJs07; Tue, 10 Mar 2020 14:46:23 -0700 (PDT) From: Francisco Jerez To: linux-pm@vger.kernel.org, intel-gfx@lists.freedesktop.org Cc: "Rafael J. Wysocki" , "Pandruvada, Srinivas" , "Vivi, Rodrigo" , Peter Zijlstra Subject: [PATCH 07/10] cpufreq: intel_pstate: Implement VLP controller for HWP parts. Date: Tue, 10 Mar 2020 14:42:00 -0700 Message-Id: <20200310214203.26459-8-currojerez@riseup.net> In-Reply-To: <20200310214203.26459-1-currojerez@riseup.net> References: <20200310214203.26459-1-currojerez@riseup.net> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org This implements a simple variably low-pass-filtering governor in control of the HWP MIN/MAX PERF range based on the previously introduced get_vlp_target_range(). See "cpufreq: intel_pstate: Implement VLP controller target P-state range estimation." for the rationale. Signed-off-by: Francisco Jerez --- drivers/cpufreq/intel_pstate.c | 79 +++++++++++++++++++++++++++++++++- 1 file changed, 77 insertions(+), 2 deletions(-) diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c index cecadfec8bc1..a01eed40d897 100644 --- a/drivers/cpufreq/intel_pstate.c +++ b/drivers/cpufreq/intel_pstate.c @@ -1905,6 +1905,20 @@ static void intel_pstate_reset_vlp(struct cpudata *cpu) vlp->gain = max(1, div_fp(1000, vlp_params.setpoint_0_pml)); vlp->target.p_base = 0; vlp->stats.last_response_frequency_hz = vlp_params.avg_hz; + + if (hwp_active) { + const uint32_t p0 = max(cpu->pstate.min_pstate, + cpu->min_perf_ratio); + const uint32_t p1 = max_t(uint32_t, p0, cpu->max_perf_ratio); + const uint64_t hwp_req = (READ_ONCE(cpu->hwp_req_cached) & + ~(HWP_MAX_PERF(~0L) | + HWP_MIN_PERF(~0L) | + HWP_DESIRED_PERF(~0L))) | + HWP_MIN_PERF(p0) | HWP_MAX_PERF(p1); + + wrmsrl_on_cpu(cpu->cpu, MSR_HWP_REQUEST, hwp_req); + cpu->hwp_req_cached = hwp_req; + } } /** @@ -2222,6 +2236,46 @@ static void intel_pstate_adjust_pstate(struct cpudata *cpu) fp_toint(cpu->iowait_boost * 100)); } +static void intel_pstate_adjust_pstate_range(struct cpudata *cpu, + const unsigned int range[]) +{ + const int from = cpu->hwp_req_cached; + unsigned int p0, p1, p_min, p_max; + struct sample *sample; + uint64_t hwp_req; + + update_turbo_state(); + + p0 = max(cpu->pstate.min_pstate, cpu->min_perf_ratio); + p1 = max_t(unsigned int, p0, cpu->max_perf_ratio); + p_min = clamp_t(unsigned int, range[0], p0, p1); + p_max = clamp_t(unsigned int, range[1], p0, p1); + + trace_cpu_frequency(p_max * cpu->pstate.scaling, cpu->cpu); + + hwp_req = (READ_ONCE(cpu->hwp_req_cached) & + ~(HWP_MAX_PERF(~0L) | HWP_MIN_PERF(~0L) | + HWP_DESIRED_PERF(~0L))) | + HWP_MIN_PERF(vlp_params.debug & 2 ? p0 : p_min) | + HWP_MAX_PERF(vlp_params.debug & 4 ? p1 : p_max); + + if (hwp_req != cpu->hwp_req_cached) { + wrmsrl(MSR_HWP_REQUEST, hwp_req); + cpu->hwp_req_cached = hwp_req; + } + + sample = &cpu->sample; + trace_pstate_sample(mul_ext_fp(100, sample->core_avg_perf), + fp_toint(sample->busy_scaled), + from, + hwp_req, + sample->mperf, + sample->aperf, + sample->tsc, + get_avg_frequency(cpu), + fp_toint(cpu->iowait_boost * 100)); +} + static void intel_pstate_update_util(struct update_util_data *data, u64 time, unsigned int flags) { @@ -2260,6 +2314,22 @@ static void intel_pstate_update_util(struct update_util_data *data, u64 time, intel_pstate_adjust_pstate(cpu); } +/** + * Implementation of the cpufreq update_util hook based on the VLP + * controller (see get_vlp_target_range()). + */ +static void intel_pstate_update_util_hwp_vlp(struct update_util_data *data, + u64 time, unsigned int flags) +{ + struct cpudata *cpu = container_of(data, struct cpudata, update_util); + + if (update_vlp_sample(cpu, time, flags)) { + const struct vlp_target_range *target = + get_vlp_target_range(cpu); + intel_pstate_adjust_pstate_range(cpu, target->value); + } +} + static struct pstate_funcs core_funcs = { .get_max = core_get_max_pstate, .get_max_physical = core_get_max_pstate_physical, @@ -2389,6 +2459,9 @@ static int intel_pstate_init_cpu(unsigned int cpunum) intel_pstate_get_cpu_pstates(cpu); + if (pstate_funcs.update_util == intel_pstate_update_util_hwp_vlp) + intel_pstate_reset_vlp(cpu); + pr_debug("controlling: cpu %d\n", cpunum); return 0; @@ -2398,7 +2471,8 @@ static void intel_pstate_set_update_util_hook(unsigned int cpu_num) { struct cpudata *cpu = all_cpu_data[cpu_num]; - if (hwp_active && !hwp_boost) + if (hwp_active && !hwp_boost && + pstate_funcs.update_util != intel_pstate_update_util_hwp_vlp) return; if (cpu->update_util_set) @@ -2526,7 +2600,8 @@ static int intel_pstate_set_policy(struct cpufreq_policy *policy) * was turned off, in that case we need to clear the * update util hook. */ - if (!hwp_boost) + if (!hwp_boost && pstate_funcs.update_util != + intel_pstate_update_util_hwp_vlp) intel_pstate_clear_update_util_hook(policy->cpu); intel_pstate_hwp_set(policy->cpu); } From patchwork Tue Mar 10 21:42:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Francisco Jerez X-Patchwork-Id: 11430333 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 836541731 for ; Tue, 10 Mar 2020 21:46:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 629EE222C3 for ; Tue, 10 Mar 2020 21:46:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=riseup.net header.i=@riseup.net header.b="sHx5S/kD" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727673AbgCJVqZ (ORCPT ); Tue, 10 Mar 2020 17:46:25 -0400 Received: from mx1.riseup.net ([198.252.153.129]:49224 "EHLO mx1.riseup.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727687AbgCJVqY (ORCPT ); Tue, 10 Mar 2020 17:46:24 -0400 Received: from bell.riseup.net (unknown [10.0.1.178]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 48cTDw1Tq3zFdmM; Tue, 10 Mar 2020 14:46:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1583876784; bh=stbRd6S/nM+SmSD6AtycsqKYC18n7tohdkTgbGMLfTg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sHx5S/kDQQsB4kBIeSgf+XlR9aeF4xQSC2VmxxtIolgRZFthiRcCT+E+j8vjwxApz Iu63iX3UWlCavj9nknOwkguhQtwh/EjY7jBaTOoWBFDopQcAz0T7kNMajLpC7j+DBX L91bx6doFKUMQ08OxBoFRh4TeBlwPlDAyd+KnNO4= X-Riseup-User-ID: 97FBB299D1AA66348E4076D3E7B7381D8F06234A9706F0A2D772C9261D8C0D82 Received: from [127.0.0.1] (localhost [127.0.0.1]) by bell.riseup.net (Postfix) with ESMTPSA id 48cTDv6tL4zJrlc; Tue, 10 Mar 2020 14:46:23 -0700 (PDT) From: Francisco Jerez To: linux-pm@vger.kernel.org, intel-gfx@lists.freedesktop.org Cc: "Rafael J. Wysocki" , "Pandruvada, Srinivas" , "Vivi, Rodrigo" , Peter Zijlstra Subject: [PATCH 08/10] cpufreq: intel_pstate: Enable VLP controller based on ACPI FADT profile and CPUID. Date: Tue, 10 Mar 2020 14:42:01 -0700 Message-Id: <20200310214203.26459-9-currojerez@riseup.net> In-Reply-To: <20200310214203.26459-1-currojerez@riseup.net> References: <20200310214203.26459-1-currojerez@riseup.net> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org For the moment the VLP controller is only enabled on ICL platforms other than server FADT profiles in order to reduce the validation effort of the initial submission. It should work on any other processors that support HWP though (and soon enough on non-HWP too): In order to override the default behavior (e.g. to test on other platforms) the VLP controller can be forcefully enabled or disabled by passing "intel_pstate=vlp" or "intel_pstate=no_vlp" respectively in the kernel command line. v2: Handle HWP VLP controller. Signed-off-by: Francisco Jerez --- .../admin-guide/kernel-parameters.txt | 5 ++++ Documentation/admin-guide/pm/intel_pstate.rst | 7 ++++++ drivers/cpufreq/intel_pstate.c | 25 +++++++++++++++++-- 3 files changed, 35 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 0c9894247015..9bc55fc2752e 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -1828,6 +1828,11 @@ per_cpu_perf_limits Allow per-logical-CPU P-State performance control limits using cpufreq sysfs interface + vlp + Force use of VLP P-state controller. Overrides selection + derived from ACPI FADT profile. + no_vlp + Prevent use of VLP P-state controller (see "vlp" parameter). intremap= [X86-64, Intel-IOMMU] on enable Interrupt Remapping (default) diff --git a/Documentation/admin-guide/pm/intel_pstate.rst b/Documentation/admin-guide/pm/intel_pstate.rst index 67e414e34f37..da6b64812848 100644 --- a/Documentation/admin-guide/pm/intel_pstate.rst +++ b/Documentation/admin-guide/pm/intel_pstate.rst @@ -669,6 +669,13 @@ of them have to be prepended with the ``intel_pstate=`` prefix. Use per-logical-CPU P-State limits (see `Coordination of P-state Limits`_ for details). +``vlp`` + Force use of VLP P-state controller. Overrides selection derived + from ACPI FADT profile. + +``no_vlp`` + Prevent use of VLP P-state controller (see "vlp" parameter). + Diagnostics and Tuning ====================== diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c index a01eed40d897..050cc8f03c26 100644 --- a/drivers/cpufreq/intel_pstate.c +++ b/drivers/cpufreq/intel_pstate.c @@ -3029,6 +3029,7 @@ static int intel_pstate_update_status(const char *buf, size_t size) static int no_load __initdata; static int no_hwp __initdata; +static int vlp __initdata = -1; static int hwp_only __initdata; static unsigned int force_load __initdata; @@ -3193,6 +3194,7 @@ static inline void intel_pstate_request_control_from_smm(void) {} #endif /* CONFIG_ACPI */ #define INTEL_PSTATE_HWP_BROADWELL 0x01 +#define INTEL_PSTATE_HWP_VLP 0x02 #define ICPU_HWP(model, hwp_mode) \ { X86_VENDOR_INTEL, 6, model, X86_FEATURE_HWP, hwp_mode } @@ -3200,12 +3202,15 @@ static inline void intel_pstate_request_control_from_smm(void) {} static const struct x86_cpu_id hwp_support_ids[] __initconst = { ICPU_HWP(INTEL_FAM6_BROADWELL_X, INTEL_PSTATE_HWP_BROADWELL), ICPU_HWP(INTEL_FAM6_BROADWELL_D, INTEL_PSTATE_HWP_BROADWELL), + ICPU_HWP(INTEL_FAM6_ICELAKE, INTEL_PSTATE_HWP_VLP), + ICPU_HWP(INTEL_FAM6_ICELAKE_L, INTEL_PSTATE_HWP_VLP), ICPU_HWP(X86_MODEL_ANY, 0), {} }; static int __init intel_pstate_init(void) { + bool use_vlp = vlp == 1; const struct x86_cpu_id *id; int rc; @@ -3222,8 +3227,19 @@ static int __init intel_pstate_init(void) pstate_funcs.update_util = intel_pstate_update_util; } else { hwp_active++; - pstate_funcs.update_util = intel_pstate_update_util_hwp; - hwp_mode_bdw = id->driver_data; + + if (vlp < 0 && !intel_pstate_acpi_pm_profile_server() && + (id->driver_data & INTEL_PSTATE_HWP_VLP)) { + /* Enable VLP controller by default. */ + use_vlp = true; + } + + pstate_funcs.update_util = use_vlp ? + intel_pstate_update_util_hwp_vlp : + intel_pstate_update_util_hwp; + + hwp_mode_bdw = (id->driver_data & + INTEL_PSTATE_HWP_BROADWELL); intel_pstate.attr = hwp_cpufreq_attrs; goto hwp_cpu_matched; } @@ -3301,6 +3317,11 @@ static int __init intel_pstate_setup(char *str) if (!strcmp(str, "per_cpu_perf_limits")) per_cpu_limits = true; + if (!strcmp(str, "vlp")) + vlp = 1; + if (!strcmp(str, "no_vlp")) + vlp = 0; + #ifdef CONFIG_ACPI if (!strcmp(str, "support_acpi_ppc")) acpi_ppc = true; From patchwork Tue Mar 10 21:42:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Francisco Jerez X-Patchwork-Id: 11430331 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 137DC18E8 for ; Tue, 10 Mar 2020 21:46:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E7178222C4 for ; Tue, 10 Mar 2020 21:46:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=riseup.net header.i=@riseup.net header.b="KmCtsL34" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727693AbgCJVqZ (ORCPT ); Tue, 10 Mar 2020 17:46:25 -0400 Received: from mx1.riseup.net ([198.252.153.129]:49230 "EHLO mx1.riseup.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727673AbgCJVqZ (ORCPT ); Tue, 10 Mar 2020 17:46:25 -0400 Received: from bell.riseup.net (unknown [10.0.1.178]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 48cTDw3BHrzFf3v; Tue, 10 Mar 2020 14:46:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1583876784; bh=lQcnxVdzWXKWrSmpl4/uUHBbYTey/h/8Iznnu2Xl5Xs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KmCtsL34b0YFEWPEFfHqRzMBNBREpOirxpkcWv+hTDvVyaOoQ6FimlCPTfVvEKV6M KcS9mEeZTeZRqFF5A+rl6bfZZSe2+V5+0uIJsgkXsi3Oq03WvQVsDyuoW7l8jHNJbl b+VG8f4+zpSKsDI7E+KX4qa8sME6xnD+b7tQa7V4= X-Riseup-User-ID: 88BB92681B8A847D8338CF869D6D9FCBBC58A96AF3BBB0CC65835658AD7ACD8C Received: from [127.0.0.1] (localhost [127.0.0.1]) by bell.riseup.net (Postfix) with ESMTPSA id 48cTDw1HpczJs07; Tue, 10 Mar 2020 14:46:24 -0700 (PDT) From: Francisco Jerez To: linux-pm@vger.kernel.org, intel-gfx@lists.freedesktop.org Cc: "Rafael J. Wysocki" , "Pandruvada, Srinivas" , "Vivi, Rodrigo" , Peter Zijlstra Subject: [PATCH 09/10] OPTIONAL: cpufreq: intel_pstate: Add tracing of VLP controller status. Date: Tue, 10 Mar 2020 14:42:02 -0700 Message-Id: <20200310214203.26459-10-currojerez@riseup.net> In-Reply-To: <20200310214203.26459-1-currojerez@riseup.net> References: <20200310214203.26459-1-currojerez@riseup.net> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org Signed-off-by: Francisco Jerez --- drivers/cpufreq/intel_pstate.c | 9 ++++++--- include/trace/events/power.h | 13 +++++++++---- 2 files changed, 15 insertions(+), 7 deletions(-) diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c index 050cc8f03c26..c4558a131660 100644 --- a/drivers/cpufreq/intel_pstate.c +++ b/drivers/cpufreq/intel_pstate.c @@ -2233,7 +2233,8 @@ static void intel_pstate_adjust_pstate(struct cpudata *cpu) sample->aperf, sample->tsc, get_avg_frequency(cpu), - fp_toint(cpu->iowait_boost * 100)); + fp_toint(cpu->iowait_boost * 100), + cpu->vlp.status.value); } static void intel_pstate_adjust_pstate_range(struct cpudata *cpu, @@ -2273,7 +2274,8 @@ static void intel_pstate_adjust_pstate_range(struct cpudata *cpu, sample->aperf, sample->tsc, get_avg_frequency(cpu), - fp_toint(cpu->iowait_boost * 100)); + fp_toint(cpu->iowait_boost * 100), + cpu->vlp.status.value); } static void intel_pstate_update_util(struct update_util_data *data, u64 time, @@ -2782,7 +2784,8 @@ static void intel_cpufreq_trace(struct cpudata *cpu, unsigned int trace_type, in sample->aperf, sample->tsc, get_avg_frequency(cpu), - fp_toint(cpu->iowait_boost * 100)); + fp_toint(cpu->iowait_boost * 100), + 0); } static int intel_cpufreq_target(struct cpufreq_policy *policy, diff --git a/include/trace/events/power.h b/include/trace/events/power.h index 7e4b52e8ca3a..e94d5e618175 100644 --- a/include/trace/events/power.h +++ b/include/trace/events/power.h @@ -72,7 +72,8 @@ TRACE_EVENT(pstate_sample, u64 aperf, u64 tsc, u32 freq, - u32 io_boost + u32 io_boost, + u32 vlp_status ), TP_ARGS(core_busy, @@ -83,7 +84,8 @@ TRACE_EVENT(pstate_sample, aperf, tsc, freq, - io_boost + io_boost, + vlp_status ), TP_STRUCT__entry( @@ -96,6 +98,7 @@ TRACE_EVENT(pstate_sample, __field(u64, tsc) __field(u32, freq) __field(u32, io_boost) + __field(u32, vlp_status) ), TP_fast_assign( @@ -108,9 +111,10 @@ TRACE_EVENT(pstate_sample, __entry->tsc = tsc; __entry->freq = freq; __entry->io_boost = io_boost; + __entry->vlp_status = vlp_status; ), - TP_printk("core_busy=%lu scaled=%lu from=%lu to=%lu mperf=%llu aperf=%llu tsc=%llu freq=%lu io_boost=%lu", + TP_printk("core_busy=%lu scaled=%lu from=%lu to=%lu mperf=%llu aperf=%llu tsc=%llu freq=%lu io_boost=%lu vlp=%lu", (unsigned long)__entry->core_busy, (unsigned long)__entry->scaled_busy, (unsigned long)__entry->from, @@ -119,7 +123,8 @@ TRACE_EVENT(pstate_sample, (unsigned long long)__entry->aperf, (unsigned long long)__entry->tsc, (unsigned long)__entry->freq, - (unsigned long)__entry->io_boost + (unsigned long)__entry->io_boost, + (unsigned long)__entry->vlp_status ) ); From patchwork Tue Mar 10 21:42:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Francisco Jerez X-Patchwork-Id: 11430335 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B8CDB924 for ; Tue, 10 Mar 2020 21:46:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8D51E2467D for ; Tue, 10 Mar 2020 21:46:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=riseup.net header.i=@riseup.net header.b="MDlpam7C" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727687AbgCJVqZ (ORCPT ); Tue, 10 Mar 2020 17:46:25 -0400 Received: from mx1.riseup.net ([198.252.153.129]:49240 "EHLO mx1.riseup.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727551AbgCJVqZ (ORCPT ); Tue, 10 Mar 2020 17:46:25 -0400 Received: from bell.riseup.net (unknown [10.0.1.178]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 48cTDw5FF6zFf4j; Tue, 10 Mar 2020 14:46:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1583876784; bh=zzzl2/tHeTCBj7TTDdp7xZuK24u3SUbkYH2qozKcfxA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MDlpam7CaEcA3Cq/AOVIEhepcmg7wlREWnqqxVv4NWx8mswp8ykYRHIAZzmvkQDYq MapvujMeR1yTiGpQLdPe9Anm30BnoKhJKrdsNfav9Z6OSiFYnKQjYf8Cmv74I/uZSA 6UMALvGwR7JAEuJEY8Fg79StzhYeBYghq6qY22+k= X-Riseup-User-ID: 5CE1B4FF9DF4CA22973D9FEF3FBB8CDD822CDE418BBD681102E554E4656EB1F0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by bell.riseup.net (Postfix) with ESMTPSA id 48cTDw2jwRzJsFM; Tue, 10 Mar 2020 14:46:24 -0700 (PDT) From: Francisco Jerez To: linux-pm@vger.kernel.org, intel-gfx@lists.freedesktop.org Cc: "Rafael J. Wysocki" , "Pandruvada, Srinivas" , "Vivi, Rodrigo" , Peter Zijlstra , Fengguang Wu , Julia Lawall Subject: [PATCH 10/10] OPTIONAL: cpufreq: intel_pstate: Expose VLP controller parameters via debugfs. Date: Tue, 10 Mar 2020 14:42:03 -0700 Message-Id: <20200310214203.26459-11-currojerez@riseup.net> In-Reply-To: <20200310214203.26459-1-currojerez@riseup.net> References: <20200310214203.26459-1-currojerez@riseup.net> MIME-Version: 1.0 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org This is not required for the controller to work but has proven very useful for debugging and testing of alternative heuristic parameters, which may offer a better trade-off between energy efficiency and latency. A warning is printed out which should taint the kernel for the non-standard calibration of the heuristic to be obvious in bug reports. v2: Use DEFINE_DEBUGFS_ATTRIBUTE rather than DEFINE_SIMPLE_ATTRIBUTE for debugfs files (Julia). Add realtime statistic threshold and averaging frequency parameters. Signed-off-by: Francisco Jerez Signed-off-by: Fengguang Wu Signed-off-by: Julia Lawall --- drivers/cpufreq/intel_pstate.c | 92 ++++++++++++++++++++++++++++++++++ 1 file changed, 92 insertions(+) diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c index c4558a131660..ab893a211746 100644 --- a/drivers/cpufreq/intel_pstate.c +++ b/drivers/cpufreq/intel_pstate.c @@ -1030,6 +1030,94 @@ static void intel_pstate_update_limits(unsigned int cpu) mutex_unlock(&intel_pstate_driver_lock); } +/************************** debugfs begin ************************/ +static void intel_pstate_reset_vlp(struct cpudata *cpu); + +static int vlp_param_set(void *data, u64 val) +{ + unsigned int cpu; + + *(u32 *)data = val; + for_each_possible_cpu(cpu) { + if (all_cpu_data[cpu]) + intel_pstate_reset_vlp(all_cpu_data[cpu]); + } + + WARN_ONCE(1, "Unsupported P-state VLP parameter update via debugging interface"); + + return 0; +} + +static int vlp_param_get(void *data, u64 *val) +{ + *val = *(u32 *)data; + return 0; +} +DEFINE_DEBUGFS_ATTRIBUTE(fops_vlp_param, vlp_param_get, vlp_param_set, + "%llu\n"); + +static struct dentry *debugfs_parent; + +struct vlp_param { + char *name; + void *value; + struct dentry *dentry; +}; + +static struct vlp_param vlp_files[] = { + {"vlp_sample_interval_ms", &vlp_params.sample_interval_ms, }, + {"vlp_setpoint_0_pml", &vlp_params.setpoint_0_pml, }, + {"vlp_setpoint_aggr_pml", &vlp_params.setpoint_aggr_pml, }, + {"vlp_avg_hz", &vlp_params.avg_hz, }, + {"vlp_realtime_gain_pml", &vlp_params.realtime_gain_pml, }, + {"vlp_debug", &vlp_params.debug, }, + {NULL, NULL, } +}; + +static void intel_pstate_update_util_hwp_vlp(struct update_util_data *data, + u64 time, unsigned int flags); + +static void intel_pstate_debug_expose_params(void) +{ + int i; + + if (pstate_funcs.update_util != intel_pstate_update_util_hwp_vlp) + return; + + debugfs_parent = debugfs_create_dir("pstate_snb", NULL); + if (IS_ERR_OR_NULL(debugfs_parent)) + return; + + for (i = 0; vlp_files[i].name; i++) { + struct dentry *dentry; + + dentry = debugfs_create_file_unsafe(vlp_files[i].name, 0660, + debugfs_parent, + vlp_files[i].value, + &fops_vlp_param); + if (!IS_ERR(dentry)) + vlp_files[i].dentry = dentry; + } +} + +static void intel_pstate_debug_hide_params(void) +{ + int i; + + if (IS_ERR_OR_NULL(debugfs_parent)) + return; + + for (i = 0; vlp_files[i].name; i++) { + debugfs_remove(vlp_files[i].dentry); + vlp_files[i].dentry = NULL; + } + + debugfs_remove(debugfs_parent); + debugfs_parent = NULL; +} + +/************************** debugfs end ************************/ + /************************** sysfs begin ************************/ #define show_one(file_name, object) \ static ssize_t show_##file_name \ @@ -2970,6 +3058,8 @@ static int intel_pstate_register_driver(struct cpufreq_driver *driver) global.min_perf_pct = min_perf_pct_min(); + intel_pstate_debug_expose_params(); + return 0; } @@ -2978,6 +3068,8 @@ static int intel_pstate_unregister_driver(void) if (hwp_active) return -EBUSY; + intel_pstate_debug_hide_params(); + cpufreq_unregister_driver(intel_pstate_driver); intel_pstate_driver_cleanup();