From patchwork Wed Nov 27 01:49:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?546L6LSH?= X-Patchwork-Id: 11263209 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3B22D138C for ; Wed, 27 Nov 2019 01:51:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0FC6F20722 for ; Wed, 27 Nov 2019 01:51:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726655AbfK0BvL (ORCPT ); Tue, 26 Nov 2019 20:51:11 -0500 Received: from out30-57.freemail.mail.aliyun.com ([115.124.30.57]:36569 "EHLO out30-57.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726295AbfK0BvL (ORCPT ); Tue, 26 Nov 2019 20:51:11 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04420;MF=yun.wang@linux.alibaba.com;NM=1;PH=DS;RN=16;SR=0;TI=SMTPD_---0TjBB3tB_1574819461; Received: from testdeMacBook-Pro.local(mailfrom:yun.wang@linux.alibaba.com fp:SMTPD_---0TjBB3tB_1574819461) by smtp.aliyun-inc.com(127.0.0.1); Wed, 27 Nov 2019 09:51:03 +0800 Subject: [PATCH v2 1/3] sched/numa: advanced per-cgroup numa statistic From: =?utf-8?b?546L6LSH?= To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Luis Chamberlain , Kees Cook , Iurii Zaikin , =?utf-8?q?Michal_Koutn=C3=BD?= , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, "Paul E. McKenney" References: <743eecad-9556-a241-546b-c8a66339840e@linux.alibaba.com> <207ef46c-672c-27c8-2012-735bd692a6de@linux.alibaba.com> Message-ID: <9354ffe8-81ba-9e76-e0b3-222bc942b3fc@linux.alibaba.com> Date: Wed, 27 Nov 2019 09:49:34 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: <207ef46c-672c-27c8-2012-735bd692a6de@linux.alibaba.com> Content-Language: en-US Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Currently there are no good approach to monitoring the per-cgroup numa efficiency, this could be a trouble especially when groups are sharing CPUs, it's impossible to tell which one caused the remote-memory access by reading hardware counter since multiple workloads could sharing the same CPU, which make it painful when one want to find out the root cause and fix the issue. In order to address this, we introduced new per-cgroup statistic for numa: * the numa locality to imply the numa balancing efficiency * the numa execution time on each node The task locality is the local page accessing ratio traced on numa balancing PF, and the group locality is the topology of task execution time, sectioned by the locality into 7 regions. For example the new entry 'cpu.numa_stat' show: locality 39541 60962 36842 72519 118605 721778 946553 exectime 1220127 1458684 Here we know the workloads in hierarchy executed 1220127ms on node_0 and 1458684ms on node_1 in total, tasks with locality around 0~13% executed for 39541 ms, and tasks with locality around 86~100% executed for 946553 ms, which imply most of the memory access are local access. By monitoring the new statistic, we will be able to know the numa efficiency of each per-cgroup workloads on machine, whatever they sharing the CPUs or not, we will be able to find out which one introduced the remote access mostly. Besides, per-node memory topology from 'memory.numa_stat' become more useful when we have the per-node execution time, workloads always executing on node_0 while it's memory is all on node_1 is usually a bad case. Cc: Mel Gorman Cc: Peter Zijlstra Cc: Michal Koutný Signed-off-by: Michael Wang --- include/linux/sched.h | 18 ++++++++- include/linux/sched/sysctl.h | 6 +++ init/Kconfig | 9 +++++ kernel/sched/core.c | 91 ++++++++++++++++++++++++++++++++++++++++++++ kernel/sched/fair.c | 33 ++++++++++++++++ kernel/sched/sched.h | 17 +++++++++ kernel/sysctl.c | 11 ++++++ 7 files changed, 184 insertions(+), 1 deletion(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 8f6607cd40ac..505b041594ef 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1118,9 +1118,25 @@ struct task_struct { * numa_faults_locality tracks if faults recorded during the last * scan window were remote/local or failed to migrate. The task scan * period is adapted based on the locality of the faults with different - * weights depending on whether they were shared or private faults + * weights depending on whether they were shared or private faults. + * + * Counter id stand for: + * 0 -- remote faults + * 1 -- local faults + * 2 -- page migration failure + * + * Extra counters when CONFIG_CGROUP_NUMA_STAT enabled: + * 3 -- remote page accessing + * 4 -- local page accessing + * + * The 'remote/local faults' records the cpu-page relationship before + * page migration, while the 'remote/local page accessing' is after. */ +#ifndef CONFIG_CGROUP_NUMA_STAT unsigned long numa_faults_locality[3]; +#else + unsigned long numa_faults_locality[5]; +#endif unsigned long numa_pages_migrated; #endif /* CONFIG_NUMA_BALANCING */ diff --git a/include/linux/sched/sysctl.h b/include/linux/sched/sysctl.h index 89f55e914673..2d6a515df544 100644 --- a/include/linux/sched/sysctl.h +++ b/include/linux/sched/sysctl.h @@ -102,4 +102,10 @@ extern int sched_energy_aware_handler(struct ctl_table *table, int write, loff_t *ppos); #endif +#ifdef CONFIG_CGROUP_NUMA_STAT +extern int sysctl_cg_numa_stat(struct ctl_table *table, int write, + void __user *buffer, size_t *lenp, + loff_t *ppos); +#endif + #endif /* _LINUX_SCHED_SYSCTL_H */ diff --git a/init/Kconfig b/init/Kconfig index 4d8d145c41d2..b31d2b560493 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -817,6 +817,15 @@ config NUMA_BALANCING_DEFAULT_ENABLED If set, automatic NUMA balancing will be enabled if running on a NUMA machine. +config CGROUP_NUMA_STAT + bool "Advanced per-cgroup NUMA statistics" + default n + depends on CGROUP_SCHED && NUMA_BALANCING + help + This option adds support for per-cgroup NUMA locality/execution + statistics, for monitoring NUMA efficiency of per-cgroup workloads + on NUMA platforms with NUMA Balancing enabled. + menuconfig CGROUPS bool "Control Group support" select KERNFS diff --git a/kernel/sched/core.c b/kernel/sched/core.c index aaa1740e6497..eabcab25be50 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -7657,6 +7657,84 @@ static u64 cpu_rt_period_read_uint(struct cgroup_subsys_state *css, } #endif /* CONFIG_RT_GROUP_SCHED */ +#ifdef CONFIG_CGROUP_NUMA_STAT +DEFINE_STATIC_KEY_FALSE(sched_cg_numa_stat); + +#ifdef CONFIG_PROC_SYSCTL +int sysctl_cg_numa_stat(struct ctl_table *table, int write, + void __user *buffer, size_t *lenp, loff_t *ppos) +{ + struct ctl_table t; + int err; + int state = static_branch_likely(&sched_cg_numa_stat); + + if (write && !capable(CAP_SYS_ADMIN)) + return -EPERM; + + t = *table; + t.data = &state; + err = proc_dointvec_minmax(&t, write, buffer, lenp, ppos); + if (err < 0 || !write) + return err; + + if (state) + static_branch_enable(&sched_cg_numa_stat); + else + static_branch_disable(&sched_cg_numa_stat); + + return err; +} +#endif + +static inline struct cfs_rq *tg_cfs_rq(struct task_group *tg, int cpu) +{ + return tg == &root_task_group ? &cpu_rq(cpu)->cfs : tg->cfs_rq[cpu]; +} + +static int cpu_numa_stat_show(struct seq_file *sf, void *v) +{ + int nr; + struct task_group *tg = css_tg(seq_css(sf)); + + if (!static_branch_likely(&sched_cg_numa_stat)) + return 0; + + seq_puts(sf, "locality"); + for (nr = 0; nr < NR_NL_INTERVAL; nr++) { + int cpu; + u64 sum = 0; + + for_each_possible_cpu(cpu) + sum += tg_cfs_rq(tg, cpu)->nstat.locality[nr]; + + seq_printf(sf, " %u", jiffies_to_msecs(sum)); + } + seq_putc(sf, '\n'); + + seq_puts(sf, "exectime"); + for_each_online_node(nr) { + int cpu; + u64 sum = 0; + + for_each_cpu(cpu, cpumask_of_node(nr)) + sum += tg_cfs_rq(tg, cpu)->nstat.jiffies; + + seq_printf(sf, " %u", jiffies_to_msecs(sum)); + } + seq_putc(sf, '\n'); + + return 0; +} + +static __init int cg_numa_stat_setup(char *opt) +{ + static_branch_enable(&sched_cg_numa_stat); + + return 0; +} +__setup("cg_numa_stat", cg_numa_stat_setup); +#endif + static struct cftype cpu_legacy_files[] = { #ifdef CONFIG_FAIR_GROUP_SCHED { @@ -7706,6 +7784,12 @@ static struct cftype cpu_legacy_files[] = { .seq_show = cpu_uclamp_max_show, .write = cpu_uclamp_max_write, }, +#endif +#ifdef CONFIG_CGROUP_NUMA_STAT + { + .name = "numa_stat", + .seq_show = cpu_numa_stat_show, + }, #endif { } /* Terminate */ }; @@ -7887,6 +7971,13 @@ static struct cftype cpu_files[] = { .seq_show = cpu_uclamp_max_show, .write = cpu_uclamp_max_write, }, +#endif +#ifdef CONFIG_CGROUP_NUMA_STAT + { + .name = "numa_stat", + .flags = CFTYPE_NOT_ON_ROOT, + .seq_show = cpu_numa_stat_show, + }, #endif { } /* terminate */ }; diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 81eba554db8d..d5fa038d07d3 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -2465,6 +2465,18 @@ void task_numa_fault(int last_cpupid, int mem_node, int pages, int flags) p->numa_faults[task_faults_idx(NUMA_MEMBUF, mem_node, priv)] += pages; p->numa_faults[task_faults_idx(NUMA_CPUBUF, cpu_node, priv)] += pages; p->numa_faults_locality[local] += pages; + +#ifdef CONFIG_CGROUP_NUMA_STAT + if (!static_branch_unlikely(&sched_cg_numa_stat)) + return; + + /* + * We want to record the real local/remote page access statistic + * here, so use 'mem_node' which is the real residential node of + * page after migrate_misplaced_page(). + */ + p->numa_faults_locality[3 + !!(mem_node == numa_node_id())] += pages; +#endif } static void reset_ptenuma_scan(struct task_struct *p) @@ -4285,6 +4297,23 @@ static void put_prev_entity(struct cfs_rq *cfs_rq, struct sched_entity *prev) cfs_rq->curr = NULL; } +#ifdef CONFIG_CGROUP_NUMA_STAT +static void update_numa_statistics(struct cfs_rq *cfs_rq) +{ + int idx; + unsigned long remote = current->numa_faults_locality[3]; + unsigned long local = current->numa_faults_locality[4]; + + cfs_rq->nstat.jiffies++; + + if (!remote && !local) + return; + + idx = (NR_NL_INTERVAL - 1) * local / (remote + local); + cfs_rq->nstat.locality[idx]++; +} +#endif + static void entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued) { @@ -4298,6 +4327,10 @@ entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued) */ update_load_avg(cfs_rq, curr, UPDATE_TG); update_cfs_group(curr); +#ifdef CONFIG_CGROUP_NUMA_STAT + if (static_branch_unlikely(&sched_cg_numa_stat)) + update_numa_statistics(cfs_rq); +#endif #ifdef CONFIG_SCHED_HRTICK /* diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 05c282775f21..87080c7164ac 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -486,6 +486,16 @@ struct cfs_bandwidth { }; #endif /* CONFIG_CGROUP_SCHED */ +#ifdef CONFIG_CGROUP_NUMA_STAT +/* NUMA Locality Interval, 7 buckets for cache align */ +#define NR_NL_INTERVAL 7 + +struct numa_statistics { + u64 jiffies; + u64 locality[NR_NL_INTERVAL]; +}; +#endif + /* CFS-related fields in a runqueue */ struct cfs_rq { struct load_weight load; @@ -575,6 +585,9 @@ struct cfs_rq { struct list_head throttled_list; #endif /* CONFIG_CFS_BANDWIDTH */ #endif /* CONFIG_FAIR_GROUP_SCHED */ +#ifdef CONFIG_CGROUP_NUMA_STAT + struct numa_statistics nstat ____cacheline_aligned; +#endif }; static inline int rt_bandwidth_enabled(void) @@ -1601,6 +1614,10 @@ static const_debug __maybe_unused unsigned int sysctl_sched_features = extern struct static_key_false sched_numa_balancing; extern struct static_key_false sched_schedstats; +#ifdef CONFIG_CGROUP_NUMA_STAT +extern struct static_key_false sched_cg_numa_stat; +#endif + static inline u64 global_rt_period(void) { return (u64)sysctl_sched_rt_period * NSEC_PER_USEC; diff --git a/kernel/sysctl.c b/kernel/sysctl.c index 50373984a5e2..2df9a5e7f875 100644 --- a/kernel/sysctl.c +++ b/kernel/sysctl.c @@ -428,6 +428,17 @@ static struct ctl_table kern_table[] = { .extra2 = SYSCTL_ONE, }, #endif /* CONFIG_NUMA_BALANCING */ +#ifdef CONFIG_CGROUP_NUMA_STAT + { + .procname = "cg_numa_stat", + .data = NULL, /* filled in by handler */ + .maxlen = sizeof(unsigned int), + .mode = 0644, + .proc_handler = sysctl_cg_numa_stat, + .extra1 = SYSCTL_ZERO, + .extra2 = SYSCTL_ONE, + }, +#endif /* CONFIG_CGROUP_NUMA_STAT */ #endif /* CONFIG_SCHED_DEBUG */ { .procname = "sched_rt_period_us", From patchwork Wed Nov 27 01:50:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?546L6LSH?= X-Patchwork-Id: 11263211 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D5ECE14ED for ; Wed, 27 Nov 2019 01:51:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BD0072068E for ; Wed, 27 Nov 2019 01:51:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726858AbfK0Bvf (ORCPT ); Tue, 26 Nov 2019 20:51:35 -0500 Received: from out30-132.freemail.mail.aliyun.com ([115.124.30.132]:49171 "EHLO out30-132.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725871AbfK0Bve (ORCPT ); Tue, 26 Nov 2019 20:51:34 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R111e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04423;MF=yun.wang@linux.alibaba.com;NM=1;PH=DS;RN=16;SR=0;TI=SMTPD_---0TjBZuPt_1574819487; Received: from testdeMacBook-Pro.local(mailfrom:yun.wang@linux.alibaba.com fp:SMTPD_---0TjBZuPt_1574819487) by smtp.aliyun-inc.com(127.0.0.1); Wed, 27 Nov 2019 09:51:29 +0800 Subject: [PATCH v2 2/3] sched/numa: expose per-task pages-migration-failure From: =?utf-8?b?546L6LSH?= To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Luis Chamberlain , Kees Cook , Iurii Zaikin , =?utf-8?q?Michal_Koutn=C3=BD?= , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, "Paul E. McKenney" References: <743eecad-9556-a241-546b-c8a66339840e@linux.alibaba.com> <207ef46c-672c-27c8-2012-735bd692a6de@linux.alibaba.com> Message-ID: <3931bf05-2939-0499-7660-8cc9a6f71c9a@linux.alibaba.com> Date: Wed, 27 Nov 2019 09:50:01 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: <207ef46c-672c-27c8-2012-735bd692a6de@linux.alibaba.com> Content-Language: en-US Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org NUMA balancing will try to migrate pages between nodes, which could caused by memory policy or numa group aggregation, while the page migration could failed too for eg when the target node run out of memory. Since this is critical to the performance, admin should know how serious the problem is, and take actions before it causing too much performance damage, thus this patch expose the counter as 'migfailed' in '/proc/PID/sched'. Cc: Peter Zijlstra Cc: Michal Koutný Suggested-by: Mel Gorman Signed-off-by: Michael Wang Acked-by: Mel Gorman --- kernel/sched/debug.c | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index f7e4579e746c..73c4809c8f37 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -848,6 +848,7 @@ static void sched_show_numa(struct task_struct *p, struct seq_file *m) P(total_numa_faults); SEQ_printf(m, "current_node=%d, numa_group_id=%d\n", task_node(p), task_numa_group_id(p)); + SEQ_printf(m, "migfailed=%lu\n", p->numa_faults_locality[2]); show_numa_stats(p, m); mpol_put(pol); #endif From patchwork Wed Nov 27 01:50:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?546L6LSH?= X-Patchwork-Id: 11263213 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F1B28138C for ; Wed, 27 Nov 2019 01:52:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C8C4C2068E for ; Wed, 27 Nov 2019 01:52:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727008AbfK0BwR (ORCPT ); Tue, 26 Nov 2019 20:52:17 -0500 Received: from out30-56.freemail.mail.aliyun.com ([115.124.30.56]:53567 "EHLO out30-56.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725871AbfK0BwR (ORCPT ); Tue, 26 Nov 2019 20:52:17 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R321e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e07486;MF=yun.wang@linux.alibaba.com;NM=1;PH=DS;RN=16;SR=0;TI=SMTPD_---0TjBBq.j_1574819522; Received: from testdeMacBook-Pro.local(mailfrom:yun.wang@linux.alibaba.com fp:SMTPD_---0TjBBq.j_1574819522) by smtp.aliyun-inc.com(127.0.0.1); Wed, 27 Nov 2019 09:52:05 +0800 Subject: [PATCH v2 3/3] sched/numa: documentation for per-cgroup numa stat From: =?utf-8?b?546L6LSH?= To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Luis Chamberlain , Kees Cook , Iurii Zaikin , =?utf-8?q?Michal_Koutn=C3=BD?= , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, "Paul E. McKenney" References: <743eecad-9556-a241-546b-c8a66339840e@linux.alibaba.com> <207ef46c-672c-27c8-2012-735bd692a6de@linux.alibaba.com> Message-ID: Date: Wed, 27 Nov 2019 09:50:35 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: <207ef46c-672c-27c8-2012-735bd692a6de@linux.alibaba.com> Content-Language: en-US Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Since v1: * thanks to Iurii for the better sentence * thanks to Jonathan for the better format Add the description for 'cg_numa_stat', also a new doc to explain the details on how to deal with the per-cgroup numa statistics. Cc: Peter Zijlstra Cc: Michal Koutný Cc: Mel Gorman Cc: Jonathan Corbet Cc: Iurii Zaikin Signed-off-by: Michael Wang --- Documentation/admin-guide/cg-numa-stat.rst | 163 ++++++++++++++++++++++++ Documentation/admin-guide/index.rst | 1 + Documentation/admin-guide/kernel-parameters.txt | 4 + Documentation/admin-guide/sysctl/kernel.rst | 9 ++ 4 files changed, 177 insertions(+) create mode 100644 Documentation/admin-guide/cg-numa-stat.rst diff --git a/Documentation/admin-guide/cg-numa-stat.rst b/Documentation/admin-guide/cg-numa-stat.rst new file mode 100644 index 000000000000..6f505f46fe00 --- /dev/null +++ b/Documentation/admin-guide/cg-numa-stat.rst @@ -0,0 +1,163 @@ +=============================== +Per-cgroup NUMA statistics +=============================== + +Background +---------- + +On NUMA platforms, remote memory accessing always has a performance penalty, +although we have NUMA balancing working hard to maximize the access locality, +there are still situations it can't help. + +This could happen in modern production environment. When a large number of +cgroups are used to classify and control resources, this creates a complex +configuration for memory policy, CPUs and NUMA nodes. In such cases NUMA +balancing could end up with the wrong memory policy or exhausted local NUMA +node, which would lead to low percentage of local page accesses. + +We need to detect such cases, figure out which workloads from which cgroup +has introduced the issues, then we get chance to do adjustment to avoid +performance degradation. + +However, there are no hardware counters for per-task local/remote accessing +info, we don't know how many remote page accesses have occurred for a +particular task. + +Statistics +---------- + +Fortunately, we have NUMA Balancing which scans task's mapping and triggers PF +periodically, gives us the opportunity to record per-task page accessing info. + +By "echo 1 > /proc/sys/kernel/cg_numa_stat" at runtime or adding boot parameter +'cg_numa_stat', we will enable the accounting of per-cgroup numa statistics, +the 'cpu.numa_stat' entry of CPU cgroup will show statistics:: + + locality -- execution time sectioned by task NUMA locality (in ms) + exectime -- execution time sectioned by NUMA node (in ms) + +We define 'task NUMA locality' as:: + + nr_local_page_access * 100 / (nr_local_page_access + nr_remote_page_access) + +this per-task percentage value will be updated on the ticks for current task, +and the access counter will be updated on task's NUMA balancing PF, so only +the pages which NUMA Balancing paid attention to will be accounted. + +On each tick, we acquire the locality of current task on that CPU, accumulating +the ticks into the counter of corresponding locality region, tasks from the +same group sharing the counters, becoming the group locality. + +Similarly, we acquire the NUMA node of current CPU where the current task is +executing on, accumulating the ticks into the counter of corresponding node, +becoming the per-cgroup node execution time. + +Note that the accounting is hierarchical, which means the numa statistics for +a given group represents not only the workload of this group, but also the +workloads of all it's descendants. + +For example the 'cpu.numa_stat' show:: + + locality 39541 60962 36842 72519 118605 721778 946553 + exectime 1220127 1458684 + +The locality is sectioned into 7 regions, approximately as:: + + 0-13% 14-27% 28-42% 43-56% 57-71% 72-85% 86-100% + +And exectime is sectioned into 2 nodes, 0 and 1 in this case. + +Thus we know the workload of this group and it's descendants have totally +executed 1220127ms on node_0 and 1458684ms on node_1, tasks with locality +around 0~13% executed for 39541 ms, and tasks with locality around 87~100% +executed for 946553 ms, which imply most of the memory access are local. + +Monitoring +---------- + +By monitoring the increments of these statistics, we can easily know whether +NUMA balancing is working well for a particular workload. + +For example we take a 5 secs sample period, and consider locality under 27% +is bad, then on each sampling we have:: + + region_bad = region_1 + region_2 + region_all = region_1 + region_2 + ... + region_7 + +and we have the increments as:: + + region_bad_diff = region_bad - last_region_bad + region_all_diff = region_all - last_region_all + +which finally become:: + + region_bad_percent = region_bad_diff * 100 / region_all_diff + +we can plot a line for region_bad_percent, when the line close to 0 things +are good, when getting close to 100% something is wrong, we can pick a proper +watermark to trigger warning message. + +You may want to drop the data if the region_all is too small, which implies +there are not many available pages for NUMA Balancing, ignoring would be fine +since most likely the workload is insensitive to NUMA. + +Monitoring root group helps you control the overall situation, while you may +also want to monitor all the leaf groups which contain the workloads, this +helps to catch the mouse. + +The exectime could be useful when NUMA Balancing is disabled, or when locality +becomes too small, for NUMA node X we have:: + + exectime_X_diff = exectime_X - last_exectime_X + exectime_all_diff = exectime_all - last_exectime_all + +try to put your workload into a memory cgroup which providing per-node memory +consumption by 'memory.numa_stat' entry, then we could get:: + + memory_percent_X = memory_X * 100 / memory_all + exectime_percent_X = exectime_X_diff * 100 / exectime_all_diff + +These two percentages are usually matched on each node, workload should execute +mostly on the node contain most of it's memory, but it's not guaranteed. + +The workload may only access a small part of it's memory, in such cases although +the majority of memory are remotely, locality could still be good. + +Thus to tell if things are fine or not depends on the understanding of system +resource deployment, however, if you find node X got 100% memory percent but 0% +exectime percent, definitely something is wrong. + +Troubleshooting +--------------- + +After identifying which workload introduced the bad locality, check: + +1). Is the workload bound to a particular NUMA node? +2). Has any NUMA node run out of resources? + +There are several ways to bind task's memory with a NUMA node, the strict way +like the MPOL_BIND memory policy or 'cpuset.mems' will limiting the memory +node where to allocate pages, in this situation, admin should make sure the +task is allowed to run on the CPUs of that NUMA node, and make sure there are +available CPU resource there. + +There are also ways to bind task's CPU with a NUMA node, like 'cpuset.cpus' or +sched_setaffinity() syscall, in this situation, NUMA Balancing help to migrate +pages into that node, admin should make sure there are available memory there. + +Admin could try rebind or unbind the NUMA node to erase the damage, make a +change then observe the statistics see if things get better until the situation +is acceptable. + +Highlights +---------- + +For some tasks, NUMA Balancing may found no necessary to scan pages, and +locality could always be 0 or small number, don't pay attention to them +since they most likely insensitive to NUMA. + +There are no accounting until the option turned on, so enable it in advance +if you want to have the whole history. + +We have per-task migfailed counter to tell how many page migration has been +failed for a particular task, you will find it in /proc/PID/sched entry. diff --git a/Documentation/admin-guide/index.rst b/Documentation/admin-guide/index.rst index 4405b7485312..c75a3fdfcd94 100644 --- a/Documentation/admin-guide/index.rst +++ b/Documentation/admin-guide/index.rst @@ -112,6 +112,7 @@ configure specific aspects of kernel behavior to your liking. video-output wimax/index xfs + cg-numa-stat .. only:: subproject and html diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 0945611b3877..674c1a4bae14 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3227,6 +3227,10 @@ numa_balancing= [KNL,X86] Enable or disable automatic NUMA balancing. Allowed values are enable and disable + cg_numa_atat [KNL] Enable advanced per-cgroup numa statistics. + Useful to debug NUMA efficiency problems when there are + lots of per-cgroup workloads. + numa_zonelist_order= [KNL, BOOT] Select zonelist order for NUMA. 'node', 'default' can be specified This can be set from sysctl after boot. diff --git a/Documentation/admin-guide/sysctl/kernel.rst b/Documentation/admin-guide/sysctl/kernel.rst index 7e203b3ed331..918a26d2bd77 100644 --- a/Documentation/admin-guide/sysctl/kernel.rst +++ b/Documentation/admin-guide/sysctl/kernel.rst @@ -572,6 +572,15 @@ rate for each task. numa_balancing_scan_size_mb is how many megabytes worth of pages are scanned for a given scan. +cg_numa_stat: +============= + +Enables/disables advanced per-cgroup NUMA statistic. + +0: disabled (default). +1: enabled. + +Check Documentation/admin-guide/cg-numa-stat.rst for details. osrelease, ostype & version: ============================