From patchwork Fri Jul 15 14:49:11 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dario Faggioli X-Patchwork-Id: 9232157 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0485F6075D for ; Fri, 15 Jul 2016 14:51:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EAC4727AC2 for ; Fri, 15 Jul 2016 14:51:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DF92B27F94; Fri, 15 Jul 2016 14:51:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 41D6E27AC2 for ; Fri, 15 Jul 2016 14:51:52 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bO4QV-0005hR-JT; Fri, 15 Jul 2016 14:49:15 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bO4QV-0005hF-9B for xen-devel@lists.xenproject.org; Fri, 15 Jul 2016 14:49:15 +0000 Received: from [85.158.137.68] by server-1.bemta-3.messagelabs.com id 76/C7-28758-AE7F8875; Fri, 15 Jul 2016 14:49:14 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrAIsWRWlGSWpSXmKPExsXiVRvkrPvye0e 4QcduY4vvWyYzOTB6HP5whSWAMYo1My8pvyKBNaPjzkfGgsVSFSsXrmdsYHwm0sXIxSEkMI1R 4mLzb6YuRk4OFoE1rBJfm31AEhICl1glbvxezQ6SkBCIkfj37i8bhF0lsW/7O7C4kICKxM3tq 5gg7CVMEocuZYLYwgJ6EkeO/mCHsL0kvnzrB+tlEzCQeLNjLyuILSKgJHFv1WSwXmaBhYwSze 2BXYwcQEeoSlxrdgYJ8wp4Szw62s4MYnMK+Eo8v7+GBWKVj8SLu7PBWkUF5CRWXm5hhagXlDg 58wkLyBhmAU2J9bv0IabLS2x/O4d5AqPILCRVsxCqZiGpWsDIvIpRozi1qCy1SNfIQC+pKDM9 oyQ3MTNH19DAWC83tbg4MT01JzGpWC85P3cTIzD06xkYGHcwNp/wO8QoycGkJMq7MaUjXIgvK T+lMiOxOCO+qDQntfgQowwHh5IEb843oJxgUWp6akVaZg4wCmHSEhw8SiK8G0DSvMUFibnFme kQqVOMuhxbpt5byyTEkpeflyolzrsQpEgApCijNA9uBCwhXGKUlRLmZWRgYBDiKUgtys0sQZV /xSjOwagkzLsJZApPZl4J3KZXQEcwAR1hbd4OckRJIkJKqoGRd1nvzLr/ApI3nlxgfbc2K27z Xy2P5KObd695saF31ZH+S3cZ7zN1sG6+yaPJ/WDzZt61+2yuFK06KHhD42vuxytRz6aIrJCuS mxk+ZPWJnWYI39hvsfcRMGS8yy7z79SFJv9fO3crFOfhNJfHambPFFq7STr+PK7Sf7VuW+Sf/ fsZPJ9Nn1RsBJLcUaioRZzUXEiAAIH2FMDAwAA X-Env-Sender: raistlin.df@gmail.com X-Msg-Ref: server-7.tower-31.messagelabs.com!1468594153!43284799!1 X-Originating-IP: [74.125.82.67] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 8.77; banners=-,-,- X-VirusChecked: Checked Received: (qmail 46546 invoked from network); 15 Jul 2016 14:49:13 -0000 Received: from mail-wm0-f67.google.com (HELO mail-wm0-f67.google.com) (74.125.82.67) by server-7.tower-31.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 15 Jul 2016 14:49:13 -0000 Received: by mail-wm0-f67.google.com with SMTP id i5so2466378wmg.2 for ; Fri, 15 Jul 2016 07:49:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=buK3DhuAlvmsvcNzwRpT8gk8q424VzTpSJL7BPWcIqU=; b=u0g772TPB+2aFmrvyV7dhfX9Twt+OH14rogsTeeQMKfhaoyH3M8mZnYoDITUHnoN+b jaKO9ODrebc+mY9WAzowkZUX0POpHJhFRp+KNVjT6CBapMMGkb0KSLQroX+2yOjQ10nE PLktrU3lvFeBcdBBieQQhbmd4I1Rk2UkNMk6OigkMxOv8vIVOhakAalNrDj7tHzBw1cI 7TMvEEwt5Km78sFIHSaPw2L/G5ES+SXytKNcNJvbsubc4zCcLrvFGL1u424cIrjIaDNV 0YOOKJ4OBx2x+flq7s4bC8CpgUr36sZe92UFGZhMJvfAiEDMpldZgWDCT1T/bCktDkuE zDjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=buK3DhuAlvmsvcNzwRpT8gk8q424VzTpSJL7BPWcIqU=; b=LWMB12WJ3DixLTgINVwUCOfSZiVONBB2Am+dN0Mni+QyyssAZClomU2J0xzpLL+cU+ VwS7+2HwB3N2WFo1BIqoYhWoHe3A6f9KXK7VahAYEDWmdUPUFX5aX6o5T4/BlaM1oVAf 0BKg9x0xENxuQLdelAMWG+CGWb9v2uDjMET/Ysrgp36Vczimik9YC/bhtCKaLvmKHB+B NxzgSyeeY1OyP66uKik+3eJr4+7utOBbfDpNjtzZCl5ODzQWH2LgYyexyYYdjdDa8+qR Gn7fYnrL4gfes60E8zBF//cvo2KTxS0YzY9bol4dzz7a1cbQoHQfuguzAO4OUGf6BOdC +K0A== X-Gm-Message-State: ALyK8tJJK92fCSQE9fHYZgWRo26HwboaS0RTZB8LnlZWNML7fuk3VBQI5SR0QKA+ceb1yw== X-Received: by 10.28.128.15 with SMTP id b15mr24664845wmd.100.1468594153240; Fri, 15 Jul 2016 07:49:13 -0700 (PDT) Received: from Solace.fritz.box (net-188-217-84-158.cust.vodafonedsl.it. [188.217.84.158]) by smtp.gmail.com with ESMTPSA id kb10sm1107999wjc.31.2016.07.15.07.49.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 15 Jul 2016 07:49:12 -0700 (PDT) From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Fri, 15 Jul 2016 16:49:11 +0200 Message-ID: <146859415105.10217.6564561975071969210.stgit@Solace.fritz.box> In-Reply-To: <146859397891.10217.10155969474613302167.stgit@Solace.fritz.box> References: <146859397891.10217.10155969474613302167.stgit@Solace.fritz.box> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: Anshul Makkar , George Dunlap , Jan Beulich , David Vrabel Subject: [Xen-devel] [PATCH v2 01/11] xen: sched: leave CPUs doing tasklet work alone. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP In both Credit1 and Credit2, stop considering a pCPU idle, if the reason why the idle vCPU is being selected, is to do tasklet work. Not doing so means that the tickling and load balancing logic, seeing the pCPU as idle, considers it a candidate for picking up vCPUs. But the pCPU won't actually pick up or schedule any vCPU, which would then remain in the runqueue, which is bad, especially if there were other, truly idle pCPUs, that could execute it. The only drawback is that we can't assume that a pCPU is in always marked as idle when being removed from an instance of the Credit2 scheduler (csched2_deinit_pdata). In fact, if we are in stop-machine (i.e., during suspend or shutdown), the pCPUs are running the stopmachine_tasklet and hence are actually marked as busy. On the other hand, when removing a pCPU from a Credit2 pool, it will indeed be idle. The only thing we can do, therefore, is to remove the BUG_ON() check. Signed-off-by: Dario Faggioli Reviewed-by: George Dunlap --- Cc: George Dunlap Cc: Jan Beulich Cc: Anshul Makkar Cc: David Vrabel --- Changes in v1: * reorg. the Credit1 code for minimal churn, as requested during review; * don't move out: label up in Credit1, as requested during review. --- As code changed, I did not apply Anshul's R-b tag. Anshul, re-issue it if you want (if you feel like it, of course). --- xen/common/sched_credit.c | 2 +- xen/common/sched_credit2.c | 14 ++++++++++---- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c index ac22746..d547716 100644 --- a/xen/common/sched_credit.c +++ b/xen/common/sched_credit.c @@ -1827,7 +1827,7 @@ csched_schedule( * Update idlers mask if necessary. When we're idling, other CPUs * will tickle us when they get extra work. */ - if ( snext->pri == CSCHED_PRI_IDLE ) + if ( !tasklet_work_scheduled && snext->pri == CSCHED_PRI_IDLE ) { if ( !cpumask_test_cpu(cpu, prv->idlers) ) cpumask_set_cpu(cpu, prv->idlers); diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c index 8b95a47..7e572bf 100644 --- a/xen/common/sched_credit2.c +++ b/xen/common/sched_credit2.c @@ -1923,8 +1923,16 @@ csched2_schedule( } else { - /* Update the idle mask if necessary */ - if ( !cpumask_test_cpu(cpu, &rqd->idle) ) + /* + * Update the idle mask if necessary. Note that, if we're scheduling + * idle in order to carry on some tasklet work, we want to play busy! + */ + if ( tasklet_work_scheduled ) + { + if ( cpumask_test_cpu(cpu, &rqd->idle) ) + cpumask_clear_cpu(cpu, &rqd->idle); + } + else if ( !cpumask_test_cpu(cpu, &rqd->idle) ) cpumask_set_cpu(cpu, &rqd->idle); /* Make sure avgload gets updated periodically even * if there's no activity */ @@ -2304,8 +2312,6 @@ csched2_deinit_pdata(const struct scheduler *ops, void *pcpu, int cpu) /* No need to save IRQs here, they're already disabled */ spin_lock(&rqd->lock); - BUG_ON(!cpumask_test_cpu(cpu, &rqd->idle)); - printk("Removing cpu %d from runqueue %d\n", cpu, rqi); cpumask_clear_cpu(cpu, &rqd->idle);