From patchwork Tue Aug 26 11:06:44 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 4780371 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 1582D9F383 for ; Tue, 26 Aug 2014 11:10:27 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id EB60C201DC for ; Tue, 26 Aug 2014 11:10:25 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 212C6201C0 for ; Tue, 26 Aug 2014 11:10:25 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1XMEbo-0002lj-33; Tue, 26 Aug 2014 11:08:16 +0000 Received: from mail-wi0-f169.google.com ([209.85.212.169]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1XMEbh-0002ci-5B for linux-arm-kernel@lists.infradead.org; Tue, 26 Aug 2014 11:08:09 +0000 Received: by mail-wi0-f169.google.com with SMTP id n3so4995321wiv.4 for ; Tue, 26 Aug 2014 04:07:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=C7Jfcu30eKk0LnATot3nho16OKsJbJCldDQkbpgpwk0=; b=JZqF8LYBGrNe7RNzFfuZ/lg3bsg3jLgSt8OFQiJS1yeQWDYnv7dxqbIlbC2ENCRIbw AXLvKmnPem8B5RYW4EckmGSY88z+UOj1SOvr1G+zBnzQ1HK0BFEdqubMdso6iZIyVwpt k6BJkQ6B95d/U/D/xRqDcaWAyb9rARbr4yOT5TVeoinmKIRS7Ey7lWd5peKMEJvDobcq y4XULclZ97jzHTD1OgFCTraN7ymi3nPpxfLaKH2sGxmgFO2wzRSyLo7PWoK6/vlyT3B3 c3w3YjDv6uecjf4wKVEOxj1JfLlrK2Mpgvg5te7aWvBbqtpM00JVc+pLb371g08qvUh4 yb/g== X-Gm-Message-State: ALoCoQkrpGQiJRGyMiSE+XcT+gVmrh0tkkwPxhhxndR6TT4MVE/Y46/o4GjnRGVxkrm64ycJxQR9 X-Received: by 10.194.202.165 with SMTP id kj5mr2629578wjc.50.1409051265814; Tue, 26 Aug 2014 04:07:45 -0700 (PDT) Received: from lmenx30s.lme.st.com (LPuteaux-656-01-48-212.w82-127.abo.wanadoo.fr. [82.127.83.212]) by mx.google.com with ESMTPSA id q6sm2494891wjy.47.2014.08.26.04.07.43 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 26 Aug 2014 04:07:44 -0700 (PDT) From: Vincent Guittot To: peterz@infradead.org, mingo@kernel.org, linux-kernel@vger.kernel.org, preeti@linux.vnet.ibm.com, linux@arm.linux.org.uk, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 01/12] sched: fix imbalance flag reset Date: Tue, 26 Aug 2014 13:06:44 +0200 Message-Id: <1409051215-16788-2-git-send-email-vincent.guittot@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1409051215-16788-1-git-send-email-vincent.guittot@linaro.org> References: <1409051215-16788-1-git-send-email-vincent.guittot@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20140826_040809_343410_23048D90 X-CRM114-Status: GOOD ( 17.58 ) X-Spam-Score: -0.7 (/) Cc: nicolas.pitre@linaro.org, riel@redhat.com, daniel.lezcano@linaro.org, Vincent Guittot , efault@gmx.de, dietmar.eggemann@arm.com, linaro-kernel@lists.linaro.org, Morten.Rasmussen@arm.com X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The imbalance flag can stay set whereas there is no imbalance. Let assume that we have 3 tasks that run on a dual cores /dual cluster system. We will have some idle load balance which are triggered during tick. Unfortunately, the tick is also used to queue background work so we can reach the situation where short work has been queued on a CPU which already runs a task. The load balance will detect this imbalance (2 tasks on 1 CPU and an idle CPU) and will try to pull the waiting task on the idle CPU. The waiting task is a worker thread that is pinned on a CPU so an imbalance due to pinned task is detected and the imbalance flag is set. Then, we will not be able to clear the flag because we have at most 1 task on each CPU but the imbalance flag will trig to useless active load balance between the idle CPU and the busy CPU. We need to reset of the imbalance flag as soon as we have reached a balanced state. If all tasks are pinned, we don't consider that as a balanced state and let the imbalance flag set. Reviewed-by: Preeti U Murthy Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d3427a8..085e853 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6765,10 +6765,8 @@ static int load_balance(int this_cpu, struct rq *this_rq, if (sd_parent) { int *group_imbalance = &sd_parent->groups->sgc->imbalance; - if ((env.flags & LBF_SOME_PINNED) && env.imbalance > 0) { + if ((env.flags & LBF_SOME_PINNED) && env.imbalance > 0) *group_imbalance = 1; - } else if (*group_imbalance) - *group_imbalance = 0; } /* All tasks on this runqueue were pinned by CPU affinity */ @@ -6779,7 +6777,7 @@ static int load_balance(int this_cpu, struct rq *this_rq, env.loop_break = sched_nr_migrate_break; goto redo; } - goto out_balanced; + goto out_all_pinned; } } @@ -6853,6 +6851,23 @@ static int load_balance(int this_cpu, struct rq *this_rq, goto out; out_balanced: + /* + * We reach balance although we may have faced some affinity + * constraints. Clear the imbalance flag if it was set. + */ + if (sd_parent) { + int *group_imbalance = &sd_parent->groups->sgc->imbalance; + + if (*group_imbalance) + *group_imbalance = 0; + } + +out_all_pinned: + /* + * We reach balance because all tasks are pinned at this level so + * we can't migrate them. Let the imbalance flag set so parent level + * can try to migrate them. + */ schedstat_inc(sd, lb_balanced[idle]); sd->nr_balance_failed = 0;