From patchwork Fri Mar 18 19:06:12 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dario Faggioli X-Patchwork-Id: 8623381 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id AAE879F6E1 for ; Fri, 18 Mar 2016 19:08:41 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 78F952026F for ; Fri, 18 Mar 2016 19:08:40 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2FCC420123 for ; Fri, 18 Mar 2016 19:08:39 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1agzj2-00015o-NC; Fri, 18 Mar 2016 19:06:20 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1agzj1-00014p-B6 for xen-devel@lists.xenproject.org; Fri, 18 Mar 2016 19:06:19 +0000 Received: from [85.158.137.68] by server-13.bemta-3.messagelabs.com id 68/36-03443-AA15CE65; Fri, 18 Mar 2016 19:06:18 +0000 X-Env-Sender: raistlin.df@gmail.com X-Msg-Ref: server-7.tower-31.messagelabs.com!1458327977!22658516!1 X-Originating-IP: [74.125.82.68] X-SpamReason: No, hits=0.7 required=7.0 tests=BODY_RANDOM_LONG, RCVD_ILLEGAL_IP X-StarScan-Received: X-StarScan-Version: 8.11; banners=-,-,- X-VirusChecked: Checked Received: (qmail 15794 invoked from network); 18 Mar 2016 19:06:18 -0000 Received: from mail-wm0-f68.google.com (HELO mail-wm0-f68.google.com) (74.125.82.68) by server-7.tower-31.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 18 Mar 2016 19:06:18 -0000 Received: by mail-wm0-f68.google.com with SMTP id x188so8154048wmg.0 for ; Fri, 18 Mar 2016 12:06:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=XRdPFrJuZrqmjRQV46KstfM8rs41SDbF3uA/HQHMsCA=; b=cK6OyK9NDJ+LDWgn5HQ2Xtei3aGcrCKjm4rija+lUqZ7Buk+5SZnoGlnujd7kLjwCl ReptAEziOZzyTdVTU0ePJd0TVq1w2vxl0+Fa+FDjUqXhNq0TuQykzKf6jKvskVZppZ+y iqBpcyiClCa/bmMWRMOb5/1ZK15xPy+CfdLLgYF02GjAvntQVYWsOnPYaZ2qF7/3g5t8 HMTjbfsFyPjVR6opQSLHCrHNcqK55RIZvlImzLAh019uxCve9uZ4LpU49Lz+yNwKdtTs 65jD1YOaCvGyyAHAPJEG/QuBW/nUPzx3C8XUeBneX6M68Evs6Ddn91ERxKzU64yMdmUX Euwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=XRdPFrJuZrqmjRQV46KstfM8rs41SDbF3uA/HQHMsCA=; b=EbUcLd2XYogGzTp/phuxIfG/VkbQ4pg9/woWlxuqgCYWBg5WlTOHo9Ao9BcYri8dLO 2jFienMI8LGAAChxzgBHBlIyZADt2kEYP5V8MDEFPwhHKvokyLLneOgDE9yGaXojszBV LY9VqKc/a3/3hYQi5QbSTctG1un0lv/wozrUuuSu/2OrOdGRpmzg7seGku+3psQcoHeI F1pIVGRRI/lShA1olHep6wrNZTNqnuS1KS+dF5yTxraRWDkaQxhj23fX8ou8Qyu/fCe3 QwuD2LG/KeSD1aVADBWQ3/TKAwCn3QZdsaOdzzVmToP/y6R1U+eM0P87P+qNIHICp6Iz dOYQ== X-Gm-Message-State: AD7BkJJcpTpwizdXLgOP12/KcPCjBXcXVI/2rObB/7yTZLJo/qj7n3nfvkugaEV7nrwmwA== X-Received: by 10.28.13.79 with SMTP id 76mr968643wmn.5.1458327977783; Fri, 18 Mar 2016 12:06:17 -0700 (PDT) Received: from Solace.station (net-2-35-170-8.cust.vodafonedsl.it. [2.35.170.8]) by smtp.gmail.com with ESMTPSA id lz5sm13445346wjb.5.2016.03.18.12.06.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 18 Mar 2016 12:06:17 -0700 (PDT) From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Fri, 18 Mar 2016 20:06:12 +0100 Message-ID: <20160318190612.8117.79354.stgit@Solace.station> In-Reply-To: <20160318185524.8117.74837.stgit@Solace.station> References: <20160318185524.8117.74837.stgit@Solace.station> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: George Dunlap , Justin Weaver Subject: [Xen-devel] [PATCH 16/16] xen: sched: implement vcpu hard affinity in Credit2 X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Status: No, score=-4.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Justin Weaver as it was still missing. Note that this patch "only" implements hard affinity, i.e., the possibility of specifying on what pCPUs a certain vCPU can run. Soft affinity (which express a preference for vCPUs to run on certain pCPUs) is still not supported by Credit2, even after this patch. Signed-off-by: Justin Weaver Signed-off-by: Dario Faggioli Acked-by: George Dunlap --- Cc: George Dunlap --- xen/common/sched_credit2.c | 131 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 102 insertions(+), 29 deletions(-) diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c index a650216..3190eb3 100644 --- a/xen/common/sched_credit2.c +++ b/xen/common/sched_credit2.c @@ -327,6 +327,36 @@ struct csched2_dom { uint16_t nr_vcpus; }; +/* + * When a hard affinity change occurs, we may not be able to check some + * (any!) of the other runqueues, when looking for the best new processor + * for svc (as trylock-s in choose_cpu() can fail). If that happens, we + * pick, in order of decreasing preference: + * - svc's current pcpu; + * - another pcpu from svc's current runq; + * - any cpu. + */ +static int get_fallback_cpu(struct csched2_vcpu *svc) +{ + int cpu; + + if ( likely(cpumask_test_cpu(svc->vcpu->processor, + svc->vcpu->cpu_hard_affinity)) ) + return svc->vcpu->processor; + + cpumask_and(cpumask_scratch, svc->vcpu->cpu_hard_affinity, + &svc->rqd->active); + cpu = cpumask_first(cpumask_scratch); + if ( likely(cpu < nr_cpu_ids) ) + return cpu; + + cpumask_and(cpumask_scratch, svc->vcpu->cpu_hard_affinity, + cpupool_domain_cpumask(svc->vcpu->domain)); + + ASSERT(!cpumask_empty(cpumask_scratch)); + + return cpumask_first(cpumask_scratch); +} /* * Time-to-credit, credit-to-time. @@ -560,8 +590,9 @@ runq_tickle(const struct scheduler *ops, unsigned int cpu, struct csched2_vcpu * goto tickle; } - /* Get a mask of idle, but not tickled */ + /* Get a mask of idle, but not tickled, that new is allowed to run on. */ cpumask_andnot(&mask, &rqd->idle, &rqd->tickled); + cpumask_and(&mask, &mask, new->vcpu->cpu_hard_affinity); /* If it's not empty, choose one */ i = cpumask_cycle(cpu, &mask); @@ -572,9 +603,11 @@ runq_tickle(const struct scheduler *ops, unsigned int cpu, struct csched2_vcpu * } /* Otherwise, look for the non-idle cpu with the lowest credit, - * skipping cpus which have been tickled but not scheduled yet */ + * skipping cpus which have been tickled but not scheduled yet, + * that new is allowed to run on. */ cpumask_andnot(&mask, &rqd->active, &rqd->idle); cpumask_andnot(&mask, &mask, &rqd->tickled); + cpumask_and(&mask, &mask, new->vcpu->cpu_hard_affinity); for_each_cpu(i, &mask) { @@ -1124,9 +1157,8 @@ choose_cpu(const struct scheduler *ops, struct vcpu *vc) d2printk("%pv -\n", svc->vcpu); clear_bit(__CSFLAG_runq_migrate_request, &svc->flags); } - /* Leave it where it is for now. When we actually pay attention - * to affinity we'll have to figure something out... */ - return vc->processor; + + return get_fallback_cpu(svc); } /* First check to see if we're here because someone else suggested a place @@ -1137,45 +1169,56 @@ choose_cpu(const struct scheduler *ops, struct vcpu *vc) { printk("%s: Runqueue migrate aborted because target runqueue disappeared!\n", __func__); - /* Fall-through to normal cpu pick */ } else { - d2printk("%pv +\n", svc->vcpu); - new_cpu = cpumask_cycle(vc->processor, &svc->migrate_rqd->active); - goto out_up; + cpumask_and(cpumask_scratch, vc->cpu_hard_affinity, + &svc->migrate_rqd->active); + new_cpu = cpumask_any(cpumask_scratch); + if ( new_cpu < nr_cpu_ids ) + { + d2printk("%pv +\n", svc->vcpu); + goto out_up; + } } + /* Fall-through to normal cpu pick */ } - /* FIXME: Pay attention to cpu affinity */ - min_avgload = MAX_LOAD; /* Find the runqueue with the lowest instantaneous load */ for_each_cpu(i, &prv->active_queues) { struct csched2_runqueue_data *rqd; - s_time_t rqd_avgload; + s_time_t rqd_avgload = MAX_LOAD; rqd = prv->rqd + i; - /* If checking a different runqueue, grab the lock, - * read the avg, and then release the lock. + /* + * If checking a different runqueue, grab the lock, check hard + * affinity, read the avg, and then release the lock. * * If on our own runqueue, don't grab or release the lock; * but subtract our own load from the runqueue load to simulate - * impartiality */ + * impartiality. + * + * Note that, if svc's hard affinity has changed, this is the + * first time when we see such change, so it is indeed possible + * that none of the cpus in svc's current runqueue is in our + * (new) hard affinity! + */ if ( rqd == svc->rqd ) { - rqd_avgload = rqd->b_avgload - svc->avgload; + if ( cpumask_intersects(vc->cpu_hard_affinity, &rqd->active) ) + rqd_avgload = rqd->b_avgload - svc->avgload; } else if ( spin_trylock(&rqd->lock) ) { - rqd_avgload = rqd->b_avgload; + if ( cpumask_intersects(vc->cpu_hard_affinity, &rqd->active) ) + rqd_avgload = rqd->b_avgload; + spin_unlock(&rqd->lock); } - else - continue; if ( rqd_avgload < min_avgload ) { @@ -1184,12 +1227,14 @@ choose_cpu(const struct scheduler *ops, struct vcpu *vc) } } - /* We didn't find anyone (most likely because of spinlock contention); leave it where it is */ + /* We didn't find anyone (most likely because of spinlock contention). */ if ( min_rqi == -1 ) - new_cpu = vc->processor; + new_cpu = get_fallback_cpu(svc); else { - new_cpu = cpumask_cycle(vc->processor, &prv->rqd[min_rqi].active); + cpumask_and(cpumask_scratch, vc->cpu_hard_affinity, + &prv->rqd[min_rqi].active); + new_cpu = cpumask_any(cpumask_scratch); BUG_ON(new_cpu >= nr_cpu_ids); } @@ -1269,7 +1314,12 @@ static void migrate(const struct scheduler *ops, on_runq=1; } __runq_deassign(svc); - svc->vcpu->processor = cpumask_any(&trqd->active); + + cpumask_and(cpumask_scratch, svc->vcpu->cpu_hard_affinity, + &trqd->active); + svc->vcpu->processor = cpumask_any(cpumask_scratch); + BUG_ON(svc->vcpu->processor >= nr_cpu_ids); + __runq_assign(svc, trqd); if ( on_runq ) { @@ -1283,6 +1333,17 @@ static void migrate(const struct scheduler *ops, } } +/* + * It makes sense considering migrating svc to rqd, if: + * - svc is not already flagged to migrate, + * - if svc is allowed to run on at least one of the pcpus of rqd. + */ +static bool_t vcpu_is_migrateable(struct csched2_vcpu *svc, + struct csched2_runqueue_data *rqd) +{ + return !(svc->flags & CSFLAG_runq_migrate_request) && + cpumask_intersects(svc->vcpu->cpu_hard_affinity, &rqd->active); +} static void balance_load(const struct scheduler *ops, int cpu, s_time_t now) { @@ -1391,8 +1452,7 @@ retry: __update_svc_load(ops, push_svc, 0, now); - /* Skip this one if it's already been flagged to migrate */ - if ( push_svc->flags & CSFLAG_runq_migrate_request ) + if ( !vcpu_is_migrateable(push_svc, st.orqd) ) continue; list_for_each( pull_iter, &st.orqd->svc ) @@ -1404,8 +1464,7 @@ retry: __update_svc_load(ops, pull_svc, 0, now); } - /* Skip this one if it's already been flagged to migrate */ - if ( pull_svc->flags & CSFLAG_runq_migrate_request ) + if ( !vcpu_is_migrateable(pull_svc, st.lrqd) ) continue; consider(&st, push_svc, pull_svc); @@ -1421,8 +1480,7 @@ retry: { struct csched2_vcpu * pull_svc = list_entry(pull_iter, struct csched2_vcpu, rqd_elem); - /* Skip this one if it's already been flagged to migrate */ - if ( pull_svc->flags & CSFLAG_runq_migrate_request ) + if ( !vcpu_is_migrateable(pull_svc, st.lrqd) ) continue; /* Consider pull only */ @@ -1461,11 +1519,22 @@ csched2_vcpu_migrate( /* Check if new_cpu is valid */ BUG_ON(!cpumask_test_cpu(new_cpu, &CSCHED2_PRIV(ops)->initialized)); + ASSERT(cpumask_test_cpu(new_cpu, vc->cpu_hard_affinity)); trqd = RQD(ops, new_cpu); + /* + * Do the actual movement toward new_cpu, and update vc->processor. + * If we are changing runqueue, migrate() takes care of everything. + * If we are not changing runqueue, we need to update vc->processor + * here. In fact, if, for instance, we are here because the vcpu's + * hard affinity changed, we don't want to risk leaving vc->processor + * pointing to a pcpu where we can't run any longer. + */ if ( trqd != svc->rqd ) migrate(ops, svc, trqd, NOW()); + else + vc->processor = new_cpu; } static int @@ -1685,6 +1754,10 @@ runq_candidate(struct csched2_runqueue_data *rqd, { struct csched2_vcpu * svc = list_entry(iter, struct csched2_vcpu, runq_elem); + /* Only consider vcpus that are allowed to run on this processor. */ + if ( !cpumask_test_cpu(cpu, svc->vcpu->cpu_hard_affinity) ) + continue; + /* If this is on a different processor, don't pull it unless * its credit is at least CSCHED2_MIGRATE_RESIST higher. */ if ( svc->vcpu->processor != cpu