From patchwork Thu Jul 27 12:05:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dario Faggioli X-Patchwork-Id: 9866693 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C5D6360382 for ; Thu, 27 Jul 2017 12:08:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B7D9728789 for ; Thu, 27 Jul 2017 12:08:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ACB0028803; Thu, 27 Jul 2017 12:08:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 96FC528789 for ; Thu, 27 Jul 2017 12:07:59 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dahY7-0005Wx-K3; Thu, 27 Jul 2017 12:05:51 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dahY6-0005Wi-9x for xen-devel@lists.xenproject.org; Thu, 27 Jul 2017 12:05:50 +0000 Received: from [85.158.143.35] by server-9.bemta-6.messagelabs.com id 0B/EB-03406-D17D9795; Thu, 27 Jul 2017 12:05:49 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmplleJIrShJLcpLzFFi42K5GNpwSFfmemW kwfm9Zhbft0xmcmD0OPzhCksAYxRrZl5SfkUCa8bVu3OZCl5aVnQfn83awNiq08XIxSEkMJNR 4uakdewgDovAGlaJHXeesIE4EgKXWCXOrj/N0sXICeTESezaOZcZwq6U2Ht9LVhcSEBF4ub2V UwQo74zSrRvOg+WEBbQkzhy9Ac7hB0i0b39OSuIzSZgIPFmx14wW0RASeLeqslgzcwCzYwSXR +/M4EkWARUJR5Pvg9WxCvgINGz7CrYUE4BJ4ntF5rYIDY7Stw++xNsgaiAnMTKyy1Q9YISJ2c +AarnABqqKbF+lz5ImFlAXmL72znMExhFZiGpmoVQNQtJ1QJG5lWM6sWpRWWpRbqGeklFmekZ JbmJmTm6hgZmermpxcWJ6ak5iUnFesn5uZsYgRHAAAQ7GHc+dzrEKMnBpCTKO8m0IlKILyk/p TIjsTgjvqg0J7X4EKMGB4fAhLNzpzNJseTl56UqSfDOv1oZKSRYlJqeWpGWmQOMUZhSCQ4eJR HedpA0b3FBYm5xZjpE6hSjMceVK+u+MHFMObD9C5MQ2CQpcd5nIKUCIKUZpXlwg2Cp4xKjrJQ wLyPQmUI8BalFuZklqPKvGMU5GJWEeW+CTOHJzCuB2/cK6BQmoFMmNoGdUpKIkJJqYNTherbf 7Rp7XURX4b2C17lsbfbnJzWdedTrLGjW6RB29nnsNz/zi9c4PX+7f3jTfIDbN++/lPrJ2VwTO XZpHT6U9S0sYG3BzZWtNzP5zFfGzAudwXlVXOLksxIhb72F304KXWQLXLZ0F7fLuw26VodfHP woE8dccUb3bKGa3LJywavdxUtM3yixFGckGmoxFxUnAgAK3MTBGAMAAA== X-Env-Sender: raistlin.df@gmail.com X-Msg-Ref: server-8.tower-21.messagelabs.com!1501157148!75113458!1 X-Originating-IP: [209.85.128.194] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.4.25; banners=-,-,- X-VirusChecked: Checked Received: (qmail 35688 invoked from network); 27 Jul 2017 12:05:48 -0000 Received: from mail-wr0-f194.google.com (HELO mail-wr0-f194.google.com) (209.85.128.194) by server-8.tower-21.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 27 Jul 2017 12:05:48 -0000 Received: by mail-wr0-f194.google.com with SMTP id y43so24194771wrd.0 for ; Thu, 27 Jul 2017 05:05:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=UJaZ0HaHGC553Qap3ZjHdvCCKob5sxwrU1zSSUOFAjE=; b=nDG0/Zbi8NaieE3/+RT7doYgVLRrp1HpmS05RFZrll/IFREyxA2FOWOFLQjGJibExV Xzn2CL11GtqMcQFpP89k/lUefCEEHeOxI3hcixcp8rrAhSZ93/79NZedUDchLH4bofn0 inmQ41Q8AEeluZbtz5Wy0A4+Rykt0VAllbAsu9S3VRx/XFcbDBIEY2IrbZm+4lcAzVCT lZ4pL6S5HkVah/l+S/VIwQHYZrRAeWUn5CMaK6FY8CdA3z65bLqPpknQDMcMvr68LqmF 3S9kXtesPEQsh8DndY3pBPi/IYOvnsbRQivwwKFcI+qtN16l6rMV0oYswGH4TkUebBeO ts2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=UJaZ0HaHGC553Qap3ZjHdvCCKob5sxwrU1zSSUOFAjE=; b=t2IWyo2TvOjkKZUXfL1sYerMldOzTz0dBl47rtREWTwYEa7lyiXE7kVnvV4SBrITbB YrcJGxe54g24vhY7BIL/C0dFWx1znmxCR9vvSuc3JhymBBlUj+Cb8C0zLZcbtHao+YtQ FCqeVrfBHyBIeRIelyh09TQWVFmebdK6sGL3tVlwi39BqW4u3MYbbENqKq+RNFlybbOw E7E3yTyob7fKsK1im4OA58zbZqw6hf95gHhZ1zBOIdQDEawsvfJBqKTun5oKfGjB1cIY F1NNEfivDKXeOu1lzBzcV+oOXph+KZfkF5THybRBkbMZj0+l8LYSPKfsiPVXZL2ei7hX N7CQ== X-Gm-Message-State: AIVw110ubXy7NvWoJEPa58tMwVBayn6qWbqjJE+SOriLiYEhYMUzgqB+ m6f0ZWa0NjgYMQ== X-Received: by 10.223.167.73 with SMTP id e9mr3260071wrd.205.1501157148400; Thu, 27 Jul 2017 05:05:48 -0700 (PDT) Received: from [192.168.0.31] ([80.66.223.212]) by smtp.gmail.com with ESMTPSA id o8sm8183413wra.42.2017.07.27.05.05.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 27 Jul 2017 05:05:47 -0700 (PDT) From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Thu, 27 Jul 2017 14:05:46 +0200 Message-ID: <150115714632.6767.2456973254047459392.stgit@Solace> In-Reply-To: <150115657192.6767.15778617807307106582.stgit@Solace> References: <150115657192.6767.15778617807307106582.stgit@Solace> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: George Dunlap , "Justin T. Weaver" , Anshul Makkar Subject: [Xen-devel] [PATCH v2 2/6] xen: credit2: soft-affinity awareness in gat_fallback_cpu() X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP By, basically, moving all the logic of the function inside the usual two steps (soft-affinity step and hard-affinity step) loop. While there, add two performance counters (in cpu_pick and in get_fallback_cpu() itself), in order to be able to tell how frequently it happens that we need to look for a fallback cpu. Signed-off-by: Dario Faggioli Signed-off-by: Justin T. Weaver Reviewed-by: George Dunlap --- Cc: Anshul Makkar Cc: George Dunlap --- Changes from v1: - as discussed during review, only consider hard-affinity for the last stand. The idea is not moving the vcpu to a diffrent runqueue because of soft-affinity, as a part of finding a fallback cpu; - as discussed during review, added the performance counters; - BUG_ON(1) turned into ASSERT_UNREACHABLE(), as suggested during review; - return something same and random enough, at the end of the function (in case we somehow manage to get there). --- xen/common/sched_credit2.c | 101 +++++++++++++++++++++++++++++++++--------- xen/include/xen/perfc_defn.h | 2 + 2 files changed, 82 insertions(+), 21 deletions(-) diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c index 57e77df..aa8f169 100644 --- a/xen/common/sched_credit2.c +++ b/xen/common/sched_credit2.c @@ -549,36 +549,93 @@ void smt_idle_mask_clear(unsigned int cpu, cpumask_t *mask) } /* - * When a hard affinity change occurs, we may not be able to check some - * (any!) of the other runqueues, when looking for the best new processor - * for svc (as trylock-s in csched2_cpu_pick() can fail). If that happens, we - * pick, in order of decreasing preference: - * - svc's current pcpu; - * - another pcpu from svc's current runq; - * - any cpu. + * In csched2_cpu_pick(), it may not be possible to actually look at remote + * runqueues (the trylock-s on their spinlocks can fail!). If that happens, + * we pick, in order of decreasing preference: + * 1) svc's current pcpu, if it is part of svc's soft affinity; + * 2) a pcpu in svc's current runqueue that is also in svc's soft affinity; + * 3) svc's current pcpu, if it is part of svc's hard affinity; + * 4) a pcpu in svc's current runqueue that is also in svc's hard affinity; + * 5) just one valid pcpu from svc's hard affinity + * + * Of course, 1, 2 and 3 makes sense only if svc has a soft affinity. Also + * note that at least 6 is guaranteed to _always_ return at least one pcpu. */ static int get_fallback_cpu(struct csched2_vcpu *svc) { struct vcpu *v = svc->vcpu; - int cpu = v->processor; + unsigned int bs; - cpumask_and(cpumask_scratch_cpu(cpu), v->cpu_hard_affinity, - cpupool_domain_cpumask(v->domain)); + SCHED_STAT_CRANK(need_fallback_cpu); - if ( likely(cpumask_test_cpu(cpu, cpumask_scratch_cpu(cpu))) ) - return cpu; - - if ( likely(cpumask_intersects(cpumask_scratch_cpu(cpu), - &svc->rqd->active)) ) + for_each_affinity_balance_step( bs ) { - cpumask_and(cpumask_scratch_cpu(cpu), &svc->rqd->active, - cpumask_scratch_cpu(cpu)); - return cpumask_first(cpumask_scratch_cpu(cpu)); - } + int cpu = v->processor; - ASSERT(!cpumask_empty(cpumask_scratch_cpu(cpu))); + if ( bs == BALANCE_SOFT_AFFINITY && + !has_soft_affinity(v, v->cpu_hard_affinity) ) + continue; - return cpumask_first(cpumask_scratch_cpu(cpu)); + affinity_balance_cpumask(v, bs, cpumask_scratch_cpu(cpu)); + cpumask_and(cpumask_scratch_cpu(cpu), cpumask_scratch_cpu(cpu), + cpupool_domain_cpumask(v->domain)); + + /* + * This is cases 1 or 3 (depending on bs): if v->processor is (still) + * in our affinity, go for it, for cache betterness. + */ + if ( likely(cpumask_test_cpu(cpu, cpumask_scratch_cpu(cpu))) ) + return cpu; + + /* + * This is cases 2 or 4 (depending on bs): v->processor isn't there + * any longer, check if we at least can stay in our current runq. + */ + if ( likely(cpumask_intersects(cpumask_scratch_cpu(cpu), + &svc->rqd->active)) ) + { + cpumask_and(cpumask_scratch_cpu(cpu), cpumask_scratch_cpu(cpu), + &svc->rqd->active); + return cpumask_first(cpumask_scratch_cpu(cpu)); + } + + /* + * We may well pick any valid pcpu from our soft-affinity, outside + * of our current runqueue, but we decide not to. In fact, changing + * runqueue is slow, affects load distribution, and is a source of + * overhead for the vcpus running on the other runqueue (we need the + * lock). So, better do that as a consequence of a well informed + * decision (or if we really don't have any other chance, as we will, + * at step 6, if we get to there). + * + * Also, being here, looking for a fallback, is an unfortunate and + * infrequent event, while the decision of putting us in the runqueue + * wehere we are was (likely) made taking all the relevant factors + * into account. So let's not disrupt that, just for the sake of + * soft-affinity, and let's wait here to be able to made (hopefully, + * soon), another similar well informed decision. + */ + if ( bs == BALANCE_SOFT_AFFINITY ) + continue; + + /* + * This is cases 6: last stand, just one valid pcpu from our hard + * affinity. It's guaranteed that there is at least one valid cpu, + * and therefore we are sure that we return it, and never really + * exit the loop. + */ + ASSERT(bs == BALANCE_HARD_AFFINITY && + !cpumask_empty(cpumask_scratch_cpu(cpu))); + cpu = cpumask_first(cpumask_scratch_cpu(cpu)); + if ( likely(cpu < nr_cpu_ids) ) + return cpu; + } + ASSERT_UNREACHABLE(); + /* + * We can't be here. But if that somehow happen (in non-debug builds), + * at least return something which both online and in our hard-affinity. + */ + return cpumask_any(cpumask_scratch_cpu(v->processor)); } /* @@ -1715,6 +1772,8 @@ csched2_cpu_pick(const struct scheduler *ops, struct vcpu *vc) ASSERT(!cpumask_empty(&prv->active_queues)); + SCHED_STAT_CRANK(pick_cpu); + /* Locking: * - Runqueue lock of vc->processor is already locked * - Need to grab prv lock to make sure active runqueues don't diff --git a/xen/include/xen/perfc_defn.h b/xen/include/xen/perfc_defn.h index 53849af..c135bf8 100644 --- a/xen/include/xen/perfc_defn.h +++ b/xen/include/xen/perfc_defn.h @@ -66,6 +66,8 @@ PERFCOUNTER(migrate_on_runq, "csched2: migrate_on_runq") PERFCOUNTER(migrate_no_runq, "csched2: migrate_no_runq") PERFCOUNTER(runtime_min_timer, "csched2: runtime_min_timer") PERFCOUNTER(runtime_max_timer, "csched2: runtime_max_timer") +PERFCOUNTER(pick_cpu, "csched2: pick_cpu") +PERFCOUNTER(need_fallback_cpu, "csched2: need_fallback_cpu") PERFCOUNTER(migrated, "csched2: migrated") PERFCOUNTER(migrate_resisted, "csched2: migrate_resisted") PERFCOUNTER(credit_reset, "csched2: credit_reset")