From patchwork Fri Jun 17 23:11:22 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dario Faggioli X-Patchwork-Id: 9185127 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 744BE601C0 for ; Fri, 17 Jun 2016 23:13:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 639CE27EED for ; Fri, 17 Jun 2016 23:13:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 58688282EE; Fri, 17 Jun 2016 23:13:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3E27D27F07 for ; Fri, 17 Jun 2016 23:13:16 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bE2v9-0002yM-So; Fri, 17 Jun 2016 23:11:27 +0000 Received: from mail6.bemta6.messagelabs.com ([85.158.143.247]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bE2v8-0002y3-8V for xen-devel@lists.xenproject.org; Fri, 17 Jun 2016 23:11:26 +0000 Received: from [85.158.143.35] by server-3.bemta-6.messagelabs.com id 52/0F-22092-D9384675; Fri, 17 Jun 2016 23:11:25 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrPIsWRWlGSWpSXmKPExsXiVRvkrDunOSX cYNUnE4vvWyYzOTB6HP5whSWAMYo1My8pvyKBNWP5o1+MBdNlKr49ecXWwPhBtIuRi0NIYAaj xPxJjxm7GDk5WATWsEo83McBYksIXGKVuNLCD2HHSOyYf58Zwq6ReHJuHVi9kICKxM3tq5gg7 EVMEosnG4DYwgJ6EkeO/mDvYuQAst0l2h4YgYTZBAwk3uzYywpiiwgoSdxbNZkJ5AZmgQZGiQ mPdzFB3KAqcf/XAjCbV8Bb4t7pjWA2p4CvxOoru9ggdvlIXDz3CMwWFZCTWHm5hRWiXlDi5Mw nLCB7mQU0Jdbv0gcJMwvIS2x/O4d5AqPILCRVsxCqZiGpWsDIvIpRvTi1qCy1SNdYL6koMz2j JDcxM0fX0MBMLze1uDgxPTUnMalYLzk/dxMjMPAZgGAHY8c/p0OMkhxMSqK8VypTwoX4kvJTK jMSizPii0pzUosPMcpwcChJ8Bo0AeUEi1LTUyvSMnOAMQiTluDgURLhrQJJ8xYXJOYWZ6ZDpE 4x6nJsmXpvLZMQS15+XqqUOK8mSJEASFFGaR7cCFg6uMQoKyXMywh0lBBPQWpRbmYJqvwrRnE ORiVh3kiQKTyZeSVwm14BHcEEdITmvGSQI0oSEVJSDYwV2WdCjr+Um110Ju+Bzbc/csd/q0xO TtbwDfdfqv+dfam9F3P2SquFBv7vVzSsZ5h6xFa1kPXajsQNYjccFnV4zPp0XteT77Z+1aeJy zMk4zpXXrj/UqCr+tGkg54V7g6/q67vq+EO9vxnt7083PvTtyXMVYfWu76t9/JQfJv64sHMqL 2P7lspsRRnJBpqMRcVJwIAuzMXDAIDAAA= X-Env-Sender: raistlin.df@gmail.com X-Msg-Ref: server-5.tower-21.messagelabs.com!1466205084!19559709!1 X-Originating-IP: [74.125.82.67] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 8.46; banners=-,-,- X-VirusChecked: Checked Received: (qmail 10643 invoked from network); 17 Jun 2016 23:11:24 -0000 Received: from mail-wm0-f67.google.com (HELO mail-wm0-f67.google.com) (74.125.82.67) by server-5.tower-21.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 17 Jun 2016 23:11:24 -0000 Received: by mail-wm0-f67.google.com with SMTP id k184so1081342wme.2 for ; Fri, 17 Jun 2016 16:11:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=pr/2Md4l5FyHYEvzKZsSnObc7rCYK9yOrPzljWa/UE0=; b=pnonsl5mmb+pp8clyiSwW9YKcbaa2DHMahTA2FsKNXKWirP/NtoFpoBnQQgTUV3XU7 l53SZqfU3jgpdqjUBYXcE8F53JKfhT+f7WPOS7iiOfOO7/zo8OfRA4AG4bnjOHuCBe8L +bracN8/JSoVMsUhHMdlzbi2fNR7QWPgLMUoeAJfsNRKRpE3VVxPzCYkg61awbdBRC6F AQJF4BYkqvPGCNQW3nqZqQtJd+fmYHRrCL+D623vPG1osII9MrBfu37gwCFWs2elLoEe W+6aQhNlI8hESATrYCH45EYQ+b40yWe/euhLfdGjsOHoeBw2dgxCWR54Ix4CuJc3NEQc FN3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=pr/2Md4l5FyHYEvzKZsSnObc7rCYK9yOrPzljWa/UE0=; b=Rjs6epBBJNFiVqi0Q94X91fEW7M+rOth/w19E0LdapFiSiJE2VXgWNBAa0qbYGNfOd OKAQGj06QE4ygmCXqEfx4E0t10GCTu7BabDJvnDNZ+DAq4+cayTt7u2PbCG11FKTXqy0 cz/jKvYEWOZAh91bh8GMGH1dCZO8Yd/ZbQG0Zu0hC/ZZ7gTU4uoub0tR5tVWTbNDZ4BB OBylhg9kpCA+mWnQsc5o71EAj9kSSMX8CUzJQshJUQCceBDGAqKlGcwFUwJWY8C5mv9u BW3rYcErG4vCQYZyDfAxHRo8na520sl1islvbtEQAlqyYyKqGIaMFe/UGOX/HZaIBJnk jXBw== X-Gm-Message-State: ALyK8tIj5W1VhB4cUkyj5jp4xX31SCj1hPtpkmmHJl6wESc/ONjv6LmtU3NlbqIVZmO6GA== X-Received: by 10.194.216.106 with SMTP id op10mr4820292wjc.178.1466205084489; Fri, 17 Jun 2016 16:11:24 -0700 (PDT) Received: from Solace.fritz.box (net-93-65-149-191.cust.vodafonedsl.it. [93.65.149.191]) by smtp.gmail.com with ESMTPSA id ae10sm7187566wjc.13.2016.06.17.16.11.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Jun 2016 16:11:23 -0700 (PDT) From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Sat, 18 Jun 2016 01:11:22 +0200 Message-ID: <146620508199.29766.7439697154591443704.stgit@Solace.fritz.box> In-Reply-To: <146620492155.29766.10321123657058307698.stgit@Solace.fritz.box> References: <146620492155.29766.10321123657058307698.stgit@Solace.fritz.box> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: Anshul Makkar , George Dunlap , David Vrabel Subject: [Xen-devel] [PATCH 01/19] xen: sched: leave CPUs doing tasklet work alone. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP In both Credit1 and Credit2, stop considering a pCPU idle, if the reason why the idle vCPU is being selected, is to do tasklet work. Not doing so means that the tickling and load balancing logic, seeing the pCPU as idle, considers it a candidate for picking up vCPUs. But the pCPU won't actually pick up or schedule any vCPU, which would then remain in the runqueue, which is bas, especially if there were other, truly idle pCPUs, that could execute it. The only drawback is that we can't assume that a pCPU is in always marked as idle when being removed from an instance of the Credit2 scheduler (csched2_deinit_pdata). In fact, if we are in stop-machine (i.e., during suspend or shutdown), the pCPUs are running the stopmachine_tasklet and hence are actually marked as busy. On the other hand, when removing a pCPU from a Credit2 pool, it will indeed be idle. The only thing we can do, therefore, is to remove the BUG_ON() check. Signed-off-by: Dario Faggioli Reviewed-by: Anshul Makkar --- Cc: George Dunlap Cc: Anshul Makkar Cc: David Vrabel --- xen/common/sched_credit.c | 12 ++++++------ xen/common/sched_credit2.c | 14 ++++++++++---- 2 files changed, 16 insertions(+), 10 deletions(-) diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c index a38a63d..a6645a2 100644 --- a/xen/common/sched_credit.c +++ b/xen/common/sched_credit.c @@ -1819,24 +1819,24 @@ csched_schedule( else snext = csched_load_balance(prv, cpu, snext, &ret.migrated); + out: /* * Update idlers mask if necessary. When we're idling, other CPUs * will tickle us when they get extra work. */ - if ( snext->pri == CSCHED_PRI_IDLE ) + if ( tasklet_work_scheduled || snext->pri != CSCHED_PRI_IDLE ) { - if ( !cpumask_test_cpu(cpu, prv->idlers) ) - cpumask_set_cpu(cpu, prv->idlers); + if ( cpumask_test_cpu(cpu, prv->idlers) ) + cpumask_clear_cpu(cpu, prv->idlers); } - else if ( cpumask_test_cpu(cpu, prv->idlers) ) + else if ( !cpumask_test_cpu(cpu, prv->idlers) ) { - cpumask_clear_cpu(cpu, prv->idlers); + cpumask_set_cpu(cpu, prv->idlers); } if ( !is_idle_vcpu(snext->vcpu) ) snext->start_time += now; -out: /* * Return task to run next... */ diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c index 1933ff1..cf8455c 100644 --- a/xen/common/sched_credit2.c +++ b/xen/common/sched_credit2.c @@ -1910,8 +1910,16 @@ csched2_schedule( } else { - /* Update the idle mask if necessary */ - if ( !cpumask_test_cpu(cpu, &rqd->idle) ) + /* + * Update the idle mask if necessary. Note that, if we're scheduling + * idle in order to carry on some tasklet work, we want to play busy! + */ + if ( tasklet_work_scheduled ) + { + if ( cpumask_test_cpu(cpu, &rqd->idle) ) + cpumask_clear_cpu(cpu, &rqd->idle); + } + else if ( !cpumask_test_cpu(cpu, &rqd->idle) ) cpumask_set_cpu(cpu, &rqd->idle); /* Make sure avgload gets updated periodically even * if there's no activity */ @@ -2291,8 +2299,6 @@ csched2_deinit_pdata(const struct scheduler *ops, void *pcpu, int cpu) /* No need to save IRQs here, they're already disabled */ spin_lock(&rqd->lock); - BUG_ON(!cpumask_test_cpu(cpu, &rqd->idle)); - printk("Removing cpu %d from runqueue %d\n", cpu, rqi); cpumask_clear_cpu(cpu, &rqd->idle);