From patchwork Tue May 28 10:33:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?SsO8cmdlbiBHcm/Dnw==?= X-Patchwork-Id: 10964667 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B577792A for ; Tue, 28 May 2019 10:35:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A309E28755 for ; Tue, 28 May 2019 10:35:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 93710287EF; Tue, 28 May 2019 10:35:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D1CD2286CF for ; Tue, 28 May 2019 10:35:41 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hVZRV-0007hs-LH; Tue, 28 May 2019 10:34:53 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hVZQJ-00053Z-A7 for xen-devel@lists.xenproject.org; Tue, 28 May 2019 10:33:39 +0000 X-Inumbo-ID: 0a641504-8134-11e9-ae0e-4b41fbb2c99e Received: from mx1.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 0a641504-8134-11e9-ae0e-4b41fbb2c99e; Tue, 28 May 2019 10:33:32 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 277F1B0B3; Tue, 28 May 2019 10:33:32 +0000 (UTC) From: Juergen Gross To: xen-devel@lists.xenproject.org Date: Tue, 28 May 2019 12:33:13 +0200 Message-Id: <20190528103313.1343-61-jgross@suse.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190528103313.1343-1-jgross@suse.com> References: <20190528103313.1343-1-jgross@suse.com> Subject: [Xen-devel] [PATCH 60/60] xen/sched: add scheduling granularity enum X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Tim Deegan , Stefano Stabellini , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Dario Faggioli , Julien Grall , Jan Beulich MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Add a scheduling granularity enum ("cpu", "core", "socket") for specification of the scheduling granularity. Initially it is set to "cpu", this can be modified by the new boot parameter (x86 only) "sched-gran". According to the selected granularity sched_granularity is set after all cpus are online. A test is added for all sched resources holding the same number of cpus. Fall back to core- or cpu-scheduling in that case. Signed-off-by: Juergen Gross Reviewed-by: Dario Faggioli --- RFC V2: - fixed freeing of sched_res when merging cpus - rename parameter to "sched-gran" (Jan Beulich) - rename parameter option from "thread" to "cpu" (Jan Beulich) V1: - rename scheduler_smp_init() to scheduler_gran_init(), let it be called by cpupool_init() - avoid using literal cpu number 0 in scheduler_percpu_init() (Jan Beulich) - style correction (Jan Beulich) - fallback to smaller granularity instead of panic in case of unbalanced cpu configuration --- xen/common/cpupool.c | 2 ++ xen/common/schedule.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++ xen/include/xen/sched-if.h | 1 + 3 files changed, 81 insertions(+) diff --git a/xen/common/cpupool.c b/xen/common/cpupool.c index 4dac7dedca..5a01f90d08 100644 --- a/xen/common/cpupool.c +++ b/xen/common/cpupool.c @@ -868,6 +868,8 @@ static int __init cpupool_init(void) unsigned int cpu; int err; + scheduler_gran_init(); + cpupool0 = cpupool_create(0, 0, &err); BUG_ON(cpupool0 == NULL); cpupool_put(cpupool0); diff --git a/xen/common/schedule.c b/xen/common/schedule.c index fed71b26d2..9e9b7f5f48 100644 --- a/xen/common/schedule.c +++ b/xen/common/schedule.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include #include @@ -61,6 +62,23 @@ unsigned int sched_granularity = 1; bool sched_disable_smt_switching; cpumask_var_t sched_res_mask; +#ifdef CONFIG_X86 +static int __init sched_select_granularity(const char *str) +{ + if (strcmp("cpu", str) == 0) + opt_sched_granularity = SCHED_GRAN_cpu; + else if (strcmp("core", str) == 0) + opt_sched_granularity = SCHED_GRAN_core; + else if (strcmp("socket", str) == 0) + opt_sched_granularity = SCHED_GRAN_socket; + else + return -EINVAL; + + return 0; +} +custom_param("sched-gran", sched_select_granularity); +#endif + /* Common lock for free cpus. */ static DEFINE_SPINLOCK(sched_free_cpu_lock); @@ -2492,6 +2510,66 @@ const cpumask_t *sched_get_opt_cpumask(enum sched_gran opt, unsigned int cpu) return mask; } +static unsigned int __init sched_check_granularity(void) +{ + unsigned int cpu; + unsigned int siblings, gran = 0; + + if ( opt_sched_granularity == SCHED_GRAN_cpu ) + return 1; + + for_each_online_cpu ( cpu ) + { + siblings = cpumask_weight(sched_get_opt_cpumask(opt_sched_granularity, + cpu)); + if ( gran == 0 ) + gran = siblings; + else if ( gran != siblings ) + return 0; + } + + sched_disable_smt_switching = true; + + return gran; +} + +/* Setup data for selected scheduler granularity. */ +void __init scheduler_gran_init(void) +{ + unsigned int gran = 0; + const char *fallback = NULL; + + while ( gran == 0 ) + { + gran = sched_check_granularity(); + + if ( gran == 0 ) + { + switch ( opt_sched_granularity ) + { + case SCHED_GRAN_core: + opt_sched_granularity = SCHED_GRAN_cpu; + fallback = "Asymmetric cpu configuration.\n" + "Falling back to sched-gran=cpu.\n"; + break; + case SCHED_GRAN_socket: + opt_sched_granularity = SCHED_GRAN_core; + fallback = "Asymmetric cpu configuration.\n" + "Falling back to sched-gran=core.\n"; + break; + default: + ASSERT_UNREACHABLE(); + break; + } + } + } + + if ( fallback ) + warning_add(fallback); + + sched_granularity = gran; +} + /* Initialise the data structures. */ void __init scheduler_init(void) { diff --git a/xen/include/xen/sched-if.h b/xen/include/xen/sched-if.h index c125236068..e16884fdf5 100644 --- a/xen/include/xen/sched-if.h +++ b/xen/include/xen/sched-if.h @@ -614,5 +614,6 @@ affinity_balance_cpumask(const struct sched_unit *unit, int step, void sched_rm_cpu(unsigned int cpu); const cpumask_t *sched_get_opt_cpumask(enum sched_gran opt, unsigned int cpu); +void scheduler_gran_init(void); #endif /* __XEN_SCHED_IF_H__ */