From patchwork Wed Apr 13 00:08:24 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Muckle X-Patchwork-Id: 8817691 Return-Path: X-Original-To: patchwork-linux-pm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 5CD389F39A for ; Wed, 13 Apr 2016 00:08:43 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 75F19202D1 for ; Wed, 13 Apr 2016 00:08:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7825A20154 for ; Wed, 13 Apr 2016 00:08:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933065AbcDMAIb (ORCPT ); Tue, 12 Apr 2016 20:08:31 -0400 Received: from mail-pf0-f170.google.com ([209.85.192.170]:35532 "EHLO mail-pf0-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751181AbcDMAIa (ORCPT ); Tue, 12 Apr 2016 20:08:30 -0400 Received: by mail-pf0-f170.google.com with SMTP id n1so23023815pfn.2 for ; Tue, 12 Apr 2016 17:08:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:date:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=5+pGPeDdmo4QwiNKLYpdRGgMaNs+CVhy/ZcZw0poUYc=; b=f+gyFXg/rTHK6GTrQu6V8lGp+F/4NB2vf8HANFXQHFdhrgWh32b3Vvq+PDIHFY8LQP ULHlgBTXD7MJb8a6Cq0h98m95lVWVvFo7dbPNGoCmS/ms8WIIpt/1/yStuuUJuTbQjjQ TJmLn9eng71AAiuH19rJMVoxTcIFk+FKNepE4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:date:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=5+pGPeDdmo4QwiNKLYpdRGgMaNs+CVhy/ZcZw0poUYc=; b=FSw1nmClJKWdi0rXMC/NwDiWBKgasPAg0Jzi/uKeED1wuwnUv5rwOfviLcJC9eVhum dFxMtIRrFuqQLB43smFPEwLwMCqOGx+CPoxDjf81eotSbFWZHPdpKEIj7pFKAs3NIXxl 4IUEuryjiB38Rdsrnf34JiOYQ3F3XW4XfDDbqVaoGkrwoomkfGM9dzIJJI+73sJqPD43 6UbrN0aqgtTUS56mvVYR6/f7V0Jtl+QtQbOmhoFslpNUiGYytsI6NjA0wAHqQsQ4ApDA YIdXBzI9zqr810IadS3ZTq6WP8VtjmKSgEIx9RfNmk/86YQQhmJu2PIB0hLwD2TgE+ev pH/w== X-Gm-Message-State: AOPr4FW1Y104TUEZOBKXQzQQ5WQQ8TrAYwWY+OE/f2r2jzcqDXV4CYJ7ZtxgwckfynxNtcSr X-Received: by 10.98.33.203 with SMTP id o72mr8669206pfj.96.1460506109128; Tue, 12 Apr 2016 17:08:29 -0700 (PDT) Received: from graphite.smuckle.net (cpe-75-80-155-7.san.res.rr.com. [75.80.155.7]) by smtp.gmail.com with ESMTPSA id bk8sm46274908pac.3.2016.04.12.17.08.26 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 12 Apr 2016 17:08:27 -0700 (PDT) From: Steve Muckle X-Google-Original-From: Steve Muckle Date: Tue, 12 Apr 2016 17:08:24 -0700 To: "Rafael J. Wysocki" Cc: Steve Muckle , Peter Zijlstra , Dietmar Eggemann , Ingo Molnar , Linux Kernel Mailing List , "linux-pm@vger.kernel.org" , Vincent Guittot , Morten Rasmussen , Juri Lelli , Patrick Bellasi , Michael Turquette Subject: Re: [PATCH 1/2] sched/fair: move cpufreq hook to update_cfs_rq_load_avg() Message-ID: <20160413000824.GB22643@graphite.smuckle.net> References: <56F97856.4040804@arm.com> <56F98832.3030207@linaro.org> <20160330193544.GD407@worktop> <56FC807C.80204@linaro.org> <20160331073743.GF3408@twins.programming.kicks-ass.net> <56FD95EE.6090007@linaro.org> <20160401092019.GN3430@twins.programming.kicks-ass.net> <570BFAE2.4080301@linaro.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Spam-Status: No, score=-7.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,T_TVD_MIME_EPI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Tue, Apr 12, 2016 at 04:29:06PM +0200, Rafael J. Wysocki wrote: > This is rather fundamental. > > For example, if you look at cpufreq_update_util(), it does this: > > data = rcu_dereference_sched(*this_cpu_ptr(&cpufreq_update_util_data)); > > meaning that it will run the current CPU's utilization update > callback. Of course, that won't work cross-CPU, because in principle > different CPUs may use different governors and therefore different > util update callbacks. Will something like the attached (unfinished patches) work? It seems to for me, but I haven't tested it much beyond confirming the hook is working on remote wakeups. I'm relying on the previous comment that it's up to cpufreq drivers to run stuff on the target policy's CPUs if the driver needs that. There's still some more work, fixing up some more smp_processor_id() usage in schedutil, but it should be easy (trace, slow path irq_work target). > If you want to do remote updates, I guess that will require an > irq_work to run the update on the target CPU, but then you'll probably > want to neglect the rate limit on it as well, so it looks like a > "need_update" flag in struct update_util_data will be useful for that. Why is it required to run the update on the target CPU? thanks, Steve From ae6eb75162f11674cbdc569466e3def4e0eed077 Mon Sep 17 00:00:00 2001 From: Steve Muckle Date: Tue, 12 Apr 2016 15:19:34 -0700 Subject: [PATCH 2/2] sched/fair: call cpufreq hook for remote wakeups Without calling the cpufreq hook for a remote wakeup it is possible for such a wakeup to go unnoticed by cpufreq on the target CPU for up to a full tick. Signed-off-by: Steve Muckle --- kernel/sched/fair.c | 8 +++----- kernel/sched/sched.h | 7 ++++--- 2 files changed, 7 insertions(+), 8 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b06c1e938cb9..d21a80a44b6e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -2826,15 +2826,13 @@ static inline void cfs_rq_util_change(struct cfs_rq *cfs_rq) struct rq *rq = rq_of(cfs_rq); int cpu = cpu_of(rq); - if (cpu == smp_processor_id() && &rq->cfs == cfs_rq) { + if (&rq->cfs == cfs_rq) { unsigned long max = rq->cpu_capacity_orig; /* * There are a few boundary cases this might miss but it should * get called often enough that that should (hopefully) not be - * a real problem -- added to that it only calls on the local - * CPU, so if we enqueue remotely we'll miss an update, but - * the next tick/schedule should update. + * a real problem. * * It will not get called when we go idle, because the idle * thread is a different class (!fair), nor will the utilization @@ -2845,7 +2843,7 @@ static inline void cfs_rq_util_change(struct cfs_rq *cfs_rq) * * See cpu_util(). */ - cpufreq_update_util(rq_clock(rq), + cpufreq_update_util(cpu, rq_clock(rq), min(cfs_rq->avg.util_avg, max), max); } } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 921d6e5d33b7..01faa0781099 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1808,11 +1808,12 @@ DECLARE_PER_CPU(struct update_util_data *, cpufreq_update_util_data); * * It can only be called from RCU-sched read-side critical sections. */ -static inline void cpufreq_update_util(u64 time, unsigned long util, unsigned long max) +static inline void cpufreq_update_util(int cpu, u64 time, unsigned long util, + unsigned long max) { struct update_util_data *data; - data = rcu_dereference_sched(*this_cpu_ptr(&cpufreq_update_util_data)); + data = rcu_dereference_sched(per_cpu(cpufreq_update_util_data, cpu)); if (data) data->func(data, time, util, max); } @@ -1835,7 +1836,7 @@ static inline void cpufreq_update_util(u64 time, unsigned long util, unsigned lo */ static inline void cpufreq_trigger_update(u64 time) { - cpufreq_update_util(time, ULONG_MAX, 0); + cpufreq_update_util(smp_processor_id(), time, ULONG_MAX, 0); } #else static inline void cpufreq_update_util(u64 time, unsigned long util, unsigned long max) {} -- 2.4.10