From patchwork Wed Apr 20 02:39:28 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Muckle X-Patchwork-Id: 8885651 Return-Path: X-Original-To: patchwork-linux-pm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id C048F9F39D for ; Wed, 20 Apr 2016 02:40:01 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id BD1B2201B4 for ; Wed, 20 Apr 2016 02:40:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AD9F1201FA for ; Wed, 20 Apr 2016 02:39:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753541AbcDTCjh (ORCPT ); Tue, 19 Apr 2016 22:39:37 -0400 Received: from mail-pf0-f179.google.com ([209.85.192.179]:33079 "EHLO mail-pf0-f179.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753482AbcDTCjf (ORCPT ); Tue, 19 Apr 2016 22:39:35 -0400 Received: by mail-pf0-f179.google.com with SMTP id 184so13045716pff.0 for ; Tue, 19 Apr 2016 19:39:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=6M7aSfLnSZtp5LGQEOxfu1sDouv4WvmXgaO1yE653k8=; b=FvnKKAAwav9AfqN3PAekl/n3NXsmiq/dMqCDv9YiVzxVbzCypJ+NbGBJSiUbLRu9/Z Ycl/0F+WMkRAWeTINISLVrjT0MXteUtX7NOx61l5Nl/8W6ZZGwnke6FSWbWzwsJnowbn YcykoqUldZow8BCxGdUMSm3gq+dznfEicjnG8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=6M7aSfLnSZtp5LGQEOxfu1sDouv4WvmXgaO1yE653k8=; b=Fj50zGgK5q3E90pgD0jTWFQTGMOMuvnkdkCGigFBZumZb0tsiqusfZyfM3KZ/wXpSJ swaK5bRZPs6IQkZO8y/QEwM914iD5HWoXdOsKgVbT8WNyAPTmlya4EYGD6j1zqGcn/Io GtErarAxdPotJ8KSY6Ne64hmczIcKYdTLRFytiFr5iv9v5BMelTiy4knjIn5HhZOyBlR XgAlnOewjSKiBMt6PX+aFdCefzwQPabIj/NilZfi3q9rEr66mHkp2RsHm9dKBTbJEmbd 9ZTVAovk4nHgMYjnWc/faXoFMVzwsc5NMw0OFkClbVbn1RQcEIlDjV1olyr+2p6HS/k2 jEVg== X-Gm-Message-State: AOPr4FWYrohCo8e0Mc2r9FDu6+jBDkx1OwpgXhBwheslIlZLxelZYvVb8NLLuZ0h4Aa8QYIq X-Received: by 10.98.66.89 with SMTP id p86mr8856115pfa.42.1461119974526; Tue, 19 Apr 2016 19:39:34 -0700 (PDT) Received: from graphite.smuckle.net (cpe-76-167-105-107.san.res.rr.com. [76.167.105.107]) by smtp.gmail.com with ESMTPSA id 9sm93833526pfm.10.2016.04.19.19.39.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 19 Apr 2016 19:39:34 -0700 (PDT) From: Steve Muckle X-Google-Original-From: Steve Muckle To: "Rafael J. Wysocki" , Viresh Kumar Cc: linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Peter Zijlstra , Ingo Molnar , Vincent Guittot , Morten Rasmussen , Dietmar Eggemann , Juri Lelli , Patrick Bellasi , Michael Turquette Subject: [RFC PATCH 3/4] intel_pstate: support scheduler cpufreq callbacks on remote CPUs Date: Tue, 19 Apr 2016 19:39:28 -0700 Message-Id: <1461119969-10371-3-git-send-email-smuckle@linaro.org> X-Mailer: git-send-email 2.4.10 In-Reply-To: <1461119969-10371-1-git-send-email-smuckle@linaro.org> References: <1461119969-10371-1-git-send-email-smuckle@linaro.org> Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Spam-Status: No, score=-7.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In preparation for the scheduler cpufreq callback happening on remote CPUs, add support for this in intel_pstate, which requires the callback run on the local CPU to be able to change the CPU frequency. Signed-off-by: Steve Muckle --- drivers/cpufreq/intel_pstate.c | 88 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 83 insertions(+), 5 deletions(-) diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c index 6c7cff13f0ed..fa49d3944aa5 100644 --- a/drivers/cpufreq/intel_pstate.c +++ b/drivers/cpufreq/intel_pstate.c @@ -162,6 +162,9 @@ struct _pid { * struct cpudata - Per CPU instance data storage * @cpu: CPU number for this instance data * @update_util: CPUFreq utility callback information + * @irq_work: Data for passing remote callbacks to the target CPU + * @time: Timestamp of CPUFreq callback + * @ipi_in_progress: Whether a remote callback IPI is outstanding * @pstate: Stores P state limits for this CPU * @vid: Stores VID limits for this CPU * @pid: Stores PID parameters for this CPU @@ -179,6 +182,9 @@ struct cpudata { int cpu; struct update_util_data update_util; + struct irq_work irq_work; + u64 time; + bool ipi_in_progress; struct pstate_data pstate; struct vid_data vid; @@ -1173,20 +1179,88 @@ static inline void intel_pstate_adjust_busy_pstate(struct cpudata *cpu) get_avg_frequency(cpu)); } +static void _intel_pstate_update_util(struct cpudata *cpu, u64 time) +{ + bool sample_taken = intel_pstate_sample(cpu, time); + + if (sample_taken && !hwp_active) + intel_pstate_adjust_busy_pstate(cpu); +} + +#ifdef CONFIG_SMP +static void intel_pstate_update_util_remote(struct irq_work *irq_work) +{ + struct cpudata *cpu = container_of(irq_work, struct cpudata, irq_work); + s64 delta_ns = cpu->time - cpu->sample.time; + + /* + * A local update may have happened while the ipi + * was in progress so re-check the time. + */ + if (delta_ns < pid_params.sample_rate_ns) + return; + + _intel_pstate_update_util(cpu, cpu->time); + + cpu->ipi_in_progress = false; +} + static void intel_pstate_update_util(struct update_util_data *data, u64 time, unsigned long util, unsigned long max) { struct cpudata *cpu = container_of(data, struct cpudata, update_util); - u64 delta_ns = time - cpu->sample.time; + s64 delta_ns = time - cpu->sample.time; - if ((s64)delta_ns >= pid_params.sample_rate_ns) { - bool sample_taken = intel_pstate_sample(cpu, time); + if (delta_ns < pid_params.sample_rate_ns) + return; - if (sample_taken && !hwp_active) - intel_pstate_adjust_busy_pstate(cpu); + if (cpu->cpu == smp_processor_id()) { + _intel_pstate_update_util(cpu, time); + } else { + /* The target CPU's rq lock is held. */ + if (cpu->ipi_in_progress) + return; + + /* Re-check sample_time which may have advanced. */ + smp_rmb(); + delta_ns = time - READ_ONCE(cpu->sample.time); + if (delta_ns < pid_params.sample_rate_ns) + return; + + cpu->ipi_in_progress = true; + cpu->time = time; + irq_work_queue_on(&cpu->irq_work, cpu->cpu); } } +static inline void intel_pstate_irq_work_sync(unsigned int cpu) +{ + irq_work_sync(&all_cpu_data[cpu]->irq_work); +} + +static inline void intel_pstate_init_irq_work(struct cpudata *cpu) +{ + init_irq_work(&cpu->irq_work, intel_pstate_update_util_remote); +} +#else /* !CONFIG_SMP */ +static inline void intel_pstate_irq_work_sync(unsigned int cpu) {} +static inline void intel_pstate_init_irq_work(struct cpudata *cpu) {} + +static void intel_pstate_update_util(struct update_util_data *data, u64 time, + unsigned long util, unsigned long max) +{ + struct cpudata *cpu = container_of(data, struct cpudata, update_util); + s64 delta_ns = time - cpu->sample.time; + + if (delta_ns < pid_params.sample_rate_ns) + return; + + _intel_pstate_update_util(cpu, time); +} +#endif + + + #define ICPU(model, policy) \ { X86_VENDOR_INTEL, 6, model, X86_FEATURE_APERFMPERF,\ (unsigned long)&policy } @@ -1273,6 +1347,7 @@ static void intel_pstate_clear_update_util_hook(unsigned int cpu) { cpufreq_remove_update_util_hook(cpu); synchronize_sched(); + intel_pstate_irq_work_sync(cpu); } static void intel_pstate_set_performance_limits(struct perf_limits *limits) @@ -1379,6 +1454,9 @@ static int intel_pstate_cpu_init(struct cpufreq_policy *policy) cpu = all_cpu_data[policy->cpu]; + intel_pstate_init_irq_work(cpu); + + if (limits->min_perf_pct == 100 && limits->max_perf_pct == 100) policy->policy = CPUFREQ_POLICY_PERFORMANCE; else