[60/60] xen/sched: add scheduling granularity enum
diff mbox series

Message ID 20190528103313.1343-61-jgross@suse.com
State New, archived
Headers show
Series
  • xen: add core scheduling support
Related show

Commit Message

Jürgen Groß May 28, 2019, 10:33 a.m. UTC
Add a scheduling granularity enum ("cpu", "core", "socket") for
specification of the scheduling granularity. Initially it is set to
"cpu", this can be modified by the new boot parameter (x86 only)
"sched-gran".

According to the selected granularity sched_granularity is set after
all cpus are online.

A test is added for all sched resources holding the same number of
cpus. Fall back to core- or cpu-scheduling in that case.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
RFC V2:
- fixed freeing of sched_res when merging cpus
- rename parameter to "sched-gran" (Jan Beulich)
- rename parameter option from "thread" to "cpu" (Jan Beulich)

V1:
- rename scheduler_smp_init() to scheduler_gran_init(), let it be called
  by cpupool_init()
- avoid using literal cpu number 0 in scheduler_percpu_init() (Jan Beulich)
- style correction (Jan Beulich)
- fallback to smaller granularity instead of panic in case of
  unbalanced cpu configuration
---
 xen/common/cpupool.c       |  2 ++
 xen/common/schedule.c      | 78 ++++++++++++++++++++++++++++++++++++++++++++++
 xen/include/xen/sched-if.h |  1 +
 3 files changed, 81 insertions(+)

Comments

Jan Beulich May 28, 2019, 11:51 a.m. UTC | #1
>>> On 28.05.19 at 12:33, <jgross@suse.com> wrote:
> @@ -61,6 +62,23 @@ unsigned int sched_granularity = 1;
>  bool sched_disable_smt_switching;
>  cpumask_var_t sched_res_mask;
>  
> +#ifdef CONFIG_X86
> +static int __init sched_select_granularity(const char *str)
> +{
> +    if (strcmp("cpu", str) == 0)
> +        opt_sched_granularity = SCHED_GRAN_cpu;
> +    else if (strcmp("core", str) == 0)
> +        opt_sched_granularity = SCHED_GRAN_core;
> +    else if (strcmp("socket", str) == 0)
> +        opt_sched_granularity = SCHED_GRAN_socket;
> +    else
> +        return -EINVAL;
> +
> +    return 0;
> +}
> +custom_param("sched-gran", sched_select_granularity);
> +#endif

I'm surprised by the x86 dependency here: I didn't think HT or multi-
core are x86-only concepts. Even if Arm may not want this right now,
I think it would be better to have a dedicated Kconfig setting, which
for now only x86 would select.

Also there are several missing blanks here.

Jan
Jürgen Groß May 28, 2019, 12:02 p.m. UTC | #2
On 28/05/2019 13:51, Jan Beulich wrote:
>>>> On 28.05.19 at 12:33, <jgross@suse.com> wrote:
>> @@ -61,6 +62,23 @@ unsigned int sched_granularity = 1;
>>  bool sched_disable_smt_switching;
>>  cpumask_var_t sched_res_mask;
>>  
>> +#ifdef CONFIG_X86
>> +static int __init sched_select_granularity(const char *str)
>> +{
>> +    if (strcmp("cpu", str) == 0)
>> +        opt_sched_granularity = SCHED_GRAN_cpu;
>> +    else if (strcmp("core", str) == 0)
>> +        opt_sched_granularity = SCHED_GRAN_core;
>> +    else if (strcmp("socket", str) == 0)
>> +        opt_sched_granularity = SCHED_GRAN_socket;
>> +    else
>> +        return -EINVAL;
>> +
>> +    return 0;
>> +}
>> +custom_param("sched-gran", sched_select_granularity);
>> +#endif
> 
> I'm surprised by the x86 dependency here: I didn't think HT or multi-
> core are x86-only concepts. Even if Arm may not want this right now,
> I think it would be better to have a dedicated Kconfig setting, which
> for now only x86 would select.

Fine with me.

I just wanted to highlight that ARM is missing some code in the context
switching area.

> Also there are several missing blanks here.

Oh, indeed!


Juergen
Dario Faggioli July 19, 2019, 6:31 p.m. UTC | #3
On Tue, 2019-05-28 at 12:33 +0200, Juergen Gross wrote:
> Add a scheduling granularity enum ("cpu", "core", "socket") for
> specification of the scheduling granularity. Initially it is set to
> "cpu", this can be modified by the new boot parameter (x86 only)
> "sched-gran".
> 
> According to the selected granularity sched_granularity is set after
> all cpus are online.
> 
> A test is added for all sched resources holding the same number of
> cpus. Fall back to core- or cpu-scheduling in that case.
>
Might be me (non-native speaker), but this reads weird.

Perhaps the second sentence should have ended with "is that _is_not_
the case" ?

As in, "if it is not the case that all sched resources hold the same
number of cpus, we fallback". While right now it seems to me to say
that we fall back in the case that the resources hold the same number
of cpus...

> Signed-off-by: Juergen Gross <jgross@suse.com>
>
With the above clarified:

Reviewed-by: Dario Faggioli <dfaggioli@suse.com>

Regards

Patch
diff mbox series

diff --git a/xen/common/cpupool.c b/xen/common/cpupool.c
index 4dac7dedca..5a01f90d08 100644
--- a/xen/common/cpupool.c
+++ b/xen/common/cpupool.c
@@ -868,6 +868,8 @@  static int __init cpupool_init(void)
     unsigned int cpu;
     int err;
 
+    scheduler_gran_init();
+
     cpupool0 = cpupool_create(0, 0, &err);
     BUG_ON(cpupool0 == NULL);
     cpupool_put(cpupool0);
diff --git a/xen/common/schedule.c b/xen/common/schedule.c
index fed71b26d2..9e9b7f5f48 100644
--- a/xen/common/schedule.c
+++ b/xen/common/schedule.c
@@ -34,6 +34,7 @@ 
 #include <xen/cpu.h>
 #include <xen/preempt.h>
 #include <xen/event.h>
+#include <xen/warning.h>
 #include <public/sched.h>
 #include <xsm/xsm.h>
 #include <xen/err.h>
@@ -61,6 +62,23 @@  unsigned int sched_granularity = 1;
 bool sched_disable_smt_switching;
 cpumask_var_t sched_res_mask;
 
+#ifdef CONFIG_X86
+static int __init sched_select_granularity(const char *str)
+{
+    if (strcmp("cpu", str) == 0)
+        opt_sched_granularity = SCHED_GRAN_cpu;
+    else if (strcmp("core", str) == 0)
+        opt_sched_granularity = SCHED_GRAN_core;
+    else if (strcmp("socket", str) == 0)
+        opt_sched_granularity = SCHED_GRAN_socket;
+    else
+        return -EINVAL;
+
+    return 0;
+}
+custom_param("sched-gran", sched_select_granularity);
+#endif
+
 /* Common lock for free cpus. */
 static DEFINE_SPINLOCK(sched_free_cpu_lock);
 
@@ -2492,6 +2510,66 @@  const cpumask_t *sched_get_opt_cpumask(enum sched_gran opt, unsigned int cpu)
     return mask;
 }
 
+static unsigned int __init sched_check_granularity(void)
+{
+    unsigned int cpu;
+    unsigned int siblings, gran = 0;
+
+    if ( opt_sched_granularity == SCHED_GRAN_cpu )
+        return 1;
+
+    for_each_online_cpu ( cpu )
+    {
+        siblings = cpumask_weight(sched_get_opt_cpumask(opt_sched_granularity,
+                                                        cpu));
+        if ( gran == 0 )
+            gran = siblings;
+        else if ( gran != siblings )
+            return 0;
+    }
+
+    sched_disable_smt_switching = true;
+
+    return gran;
+}
+
+/* Setup data for selected scheduler granularity. */
+void __init scheduler_gran_init(void)
+{
+    unsigned int gran = 0;
+    const char *fallback = NULL;
+
+    while ( gran == 0 )
+    {
+        gran = sched_check_granularity();
+
+        if ( gran == 0 )
+        {
+            switch ( opt_sched_granularity )
+            {
+            case SCHED_GRAN_core:
+                opt_sched_granularity = SCHED_GRAN_cpu;
+                fallback = "Asymmetric cpu configuration.\n"
+                           "Falling back to sched-gran=cpu.\n";
+                break;
+            case SCHED_GRAN_socket:
+                opt_sched_granularity = SCHED_GRAN_core;
+                fallback = "Asymmetric cpu configuration.\n"
+                           "Falling back to sched-gran=core.\n";
+                break;
+            default:
+                ASSERT_UNREACHABLE();
+                break;
+            }
+        }
+    }
+
+    if ( fallback )
+        warning_add(fallback);
+
+    sched_granularity = gran;
+}
+
 /* Initialise the data structures. */
 void __init scheduler_init(void)
 {
diff --git a/xen/include/xen/sched-if.h b/xen/include/xen/sched-if.h
index c125236068..e16884fdf5 100644
--- a/xen/include/xen/sched-if.h
+++ b/xen/include/xen/sched-if.h
@@ -614,5 +614,6 @@  affinity_balance_cpumask(const struct sched_unit *unit, int step,
 
 void sched_rm_cpu(unsigned int cpu);
 const cpumask_t *sched_get_opt_cpumask(enum sched_gran opt, unsigned int cpu);
+void scheduler_gran_init(void);
 
 #endif /* __XEN_SCHED_IF_H__ */