| Message ID | 20240423051826.791934-3-shakeel.butt@linux.dev (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | memcg: reduce memory consumption by memcg stats | expand |
Hi Shakeel, kernel test robot noticed the following build errors: [auto build test ERROR on next-20240422] [cannot apply to akpm-mm/mm-everything linus/master v6.9-rc5 v6.9-rc4 v6.9-rc3 v6.9-rc5] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information] url: https://github.com/intel-lab-lkp/linux/commits/Shakeel-Butt/mm-rearrange-node_stat_item-to-put-memcg-stats-at-start/20240423-132451 base: next-20240422 patch link: https://lore.kernel.org/r/20240423051826.791934-3-shakeel.butt%40linux.dev patch subject: [PATCH 2/4] memcg: reduce memory for the lruvec and memcg stats config: x86_64-buildonly-randconfig-002-20240423 (https://download.01.org/0day-ci/archive/20240423/202404232230.94gQwAI2-lkp@intel.com/config) compiler: clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240423/202404232230.94gQwAI2-lkp@intel.com/reproduce) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp@intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202404232230.94gQwAI2-lkp@intel.com/ All errors (new ones prefixed by >>): >> mm/memcontrol.c:1651:2: error: call to '__compiletime_assert_963' declared with 'error' attribute: BUILD_BUG_ON failed: ARRAY_SIZE(memory_stats) != MEMCG_NR_STAT - 1 1651 | BUILD_BUG_ON(ARRAY_SIZE(memory_stats) != MEMCG_NR_STAT - 1); | ^ include/linux/build_bug.h:50:2: note: expanded from macro 'BUILD_BUG_ON' 50 | BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition) | ^ include/linux/build_bug.h:39:37: note: expanded from macro 'BUILD_BUG_ON_MSG' 39 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg) | ^ include/linux/compiler_types.h:460:2: note: expanded from macro 'compiletime_assert' 460 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__) | ^ include/linux/compiler_types.h:448:2: note: expanded from macro '_compiletime_assert' 448 | __compiletime_assert(condition, msg, prefix, suffix) | ^ include/linux/compiler_types.h:441:4: note: expanded from macro '__compiletime_assert' 441 | prefix ## suffix(); \ | ^ <scratch space>:40:1: note: expanded from here 40 | __compiletime_assert_963 | ^ 1 error generated. vim +1651 mm/memcontrol.c 1645 1646 static void memcg_stat_format(struct mem_cgroup *memcg, struct seq_buf *s) 1647 { 1648 int i; 1649 1650 /* Reduce by 1 for MEMCG_SWAP as that is not exposed in v2. */ > 1651 BUILD_BUG_ON(ARRAY_SIZE(memory_stats) != MEMCG_NR_STAT - 1); 1652 1653 /* 1654 * Provide statistics on the state of the memory subsystem as 1655 * well as cumulative event counters that show past behavior. 1656 * 1657 * This list is ordered following a combination of these gradients: 1658 * 1) generic big picture -> specifics and details 1659 * 2) reflecting userspace activity -> reflecting kernel heuristics 1660 * 1661 * Current memory state: 1662 */ 1663 mem_cgroup_flush_stats(memcg); 1664 1665 for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { 1666 u64 size; 1667 1668 size = memcg_page_state_output(memcg, memory_stats[i].idx); 1669 seq_buf_printf(s, "%s %llu\n", memory_stats[i].name, size); 1670 1671 if (unlikely(memory_stats[i].idx == NR_SLAB_UNRECLAIMABLE_B)) { 1672 size += memcg_page_state_output(memcg, 1673 NR_SLAB_RECLAIMABLE_B); 1674 seq_buf_printf(s, "slab %llu\n", size); 1675 } 1676 } 1677 1678 /* Accumulated memory events */ 1679 seq_buf_printf(s, "pgscan %lu\n", 1680 memcg_events(memcg, PGSCAN_KSWAPD) + 1681 memcg_events(memcg, PGSCAN_DIRECT) + 1682 memcg_events(memcg, PGSCAN_KHUGEPAGED)); 1683 seq_buf_printf(s, "pgsteal %lu\n", 1684 memcg_events(memcg, PGSTEAL_KSWAPD) + 1685 memcg_events(memcg, PGSTEAL_DIRECT) + 1686 memcg_events(memcg, PGSTEAL_KHUGEPAGED)); 1687 1688 for (i = 0; i < ARRAY_SIZE(memcg_vm_event_stat); i++) { 1689 if (memcg_vm_event_stat[i] == PGPGIN || 1690 memcg_vm_event_stat[i] == PGPGOUT) 1691 continue; 1692 1693 seq_buf_printf(s, "%s %lu\n", 1694 vm_event_name(memcg_vm_event_stat[i]), 1695 memcg_events(memcg, memcg_vm_event_stat[i])); 1696 } 1697 1698 /* The above should easily fit into one page */ 1699 WARN_ON_ONCE(seq_buf_has_overflowed(s)); 1700 } 1701
Hi Shakeel,
kernel test robot noticed the following build errors:
[auto build test ERROR on next-20240422]
[cannot apply to akpm-mm/mm-everything linus/master v6.9-rc5 v6.9-rc4 v6.9-rc3 v6.9-rc5]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Shakeel-Butt/mm-rearrange-node_stat_item-to-put-memcg-stats-at-start/20240423-132451
base: next-20240422
patch link: https://lore.kernel.org/r/20240423051826.791934-3-shakeel.butt%40linux.dev
patch subject: [PATCH 2/4] memcg: reduce memory for the lruvec and memcg stats
config: sh-allmodconfig (https://download.01.org/0day-ci/archive/20240424/202404240415.fucxk6Ix-lkp@intel.com/config)
compiler: sh4-linux-gcc (GCC) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240424/202404240415.fucxk6Ix-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202404240415.fucxk6Ix-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from <command-line>:
In function 'memcg_stat_format',
inlined from 'memory_stat_format.constprop' at mm/memcontrol.c:1707:3:
>> include/linux/compiler_types.h:460:45: error: call to '__compiletime_assert_633' declared with attribute error: BUILD_BUG_ON failed: ARRAY_SIZE(memory_stats) != MEMCG_NR_STAT - 1
460 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^
include/linux/compiler_types.h:441:25: note: in definition of macro '__compiletime_assert'
441 | prefix ## suffix(); \
| ^~~~~~
include/linux/compiler_types.h:460:9: note: in expansion of macro '_compiletime_assert'
460 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^~~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:39:37: note: in expansion of macro 'compiletime_assert'
39 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
| ^~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:50:9: note: in expansion of macro 'BUILD_BUG_ON_MSG'
50 | BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition)
| ^~~~~~~~~~~~~~~~
mm/memcontrol.c:1651:9: note: in expansion of macro 'BUILD_BUG_ON'
1651 | BUILD_BUG_ON(ARRAY_SIZE(memory_stats) != MEMCG_NR_STAT - 1);
| ^~~~~~~~~~~~
In function 'memcg_stat_format',
inlined from 'memory_stat_format' at mm/memcontrol.c:1707:3,
inlined from 'memory_stat_show' at mm/memcontrol.c:6946:2:
>> include/linux/compiler_types.h:460:45: error: call to '__compiletime_assert_633' declared with attribute error: BUILD_BUG_ON failed: ARRAY_SIZE(memory_stats) != MEMCG_NR_STAT - 1
460 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^
include/linux/compiler_types.h:441:25: note: in definition of macro '__compiletime_assert'
441 | prefix ## suffix(); \
| ^~~~~~
include/linux/compiler_types.h:460:9: note: in expansion of macro '_compiletime_assert'
460 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^~~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:39:37: note: in expansion of macro 'compiletime_assert'
39 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
| ^~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:50:9: note: in expansion of macro 'BUILD_BUG_ON_MSG'
50 | BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition)
| ^~~~~~~~~~~~~~~~
mm/memcontrol.c:1651:9: note: in expansion of macro 'BUILD_BUG_ON'
1651 | BUILD_BUG_ON(ARRAY_SIZE(memory_stats) != MEMCG_NR_STAT - 1);
| ^~~~~~~~~~~~
vim +/__compiletime_assert_633 +460 include/linux/compiler_types.h
eb5c2d4b45e3d2 Will Deacon 2020-07-21 446
eb5c2d4b45e3d2 Will Deacon 2020-07-21 447 #define _compiletime_assert(condition, msg, prefix, suffix) \
eb5c2d4b45e3d2 Will Deacon 2020-07-21 448 __compiletime_assert(condition, msg, prefix, suffix)
eb5c2d4b45e3d2 Will Deacon 2020-07-21 449
eb5c2d4b45e3d2 Will Deacon 2020-07-21 450 /**
eb5c2d4b45e3d2 Will Deacon 2020-07-21 451 * compiletime_assert - break build and emit msg if condition is false
eb5c2d4b45e3d2 Will Deacon 2020-07-21 452 * @condition: a compile-time constant condition to check
eb5c2d4b45e3d2 Will Deacon 2020-07-21 453 * @msg: a message to emit if condition is false
eb5c2d4b45e3d2 Will Deacon 2020-07-21 454 *
eb5c2d4b45e3d2 Will Deacon 2020-07-21 455 * In tradition of POSIX assert, this macro will break the build if the
eb5c2d4b45e3d2 Will Deacon 2020-07-21 456 * supplied condition is *false*, emitting the supplied error message if the
eb5c2d4b45e3d2 Will Deacon 2020-07-21 457 * compiler has support to do so.
eb5c2d4b45e3d2 Will Deacon 2020-07-21 458 */
eb5c2d4b45e3d2 Will Deacon 2020-07-21 459 #define compiletime_assert(condition, msg) \
eb5c2d4b45e3d2 Will Deacon 2020-07-21 @460 _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
eb5c2d4b45e3d2 Will Deacon 2020-07-21 461
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 9aba0d0462ca..d68db7a0e829 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -32,7 +32,7 @@ struct kmem_cache; /* Cgroup-specific page state, on top of universal node page state */ enum memcg_stat_item { - MEMCG_SWAP = NR_VM_NODE_STAT_ITEMS, + MEMCG_SWAP = NR_VM_NODE_MEMCG_STAT_ITEMS, MEMCG_SOCK, MEMCG_PERCPU_B, MEMCG_VMALLOC, @@ -92,21 +92,21 @@ struct mem_cgroup_reclaim_iter { struct lruvec_stats_percpu { /* Local (CPU and cgroup) state */ - long state[NR_VM_NODE_STAT_ITEMS]; + long state[NR_VM_NODE_MEMCG_STAT_ITEMS]; /* Delta calculation for lockless upward propagation */ - long state_prev[NR_VM_NODE_STAT_ITEMS]; + long state_prev[NR_VM_NODE_MEMCG_STAT_ITEMS]; }; struct lruvec_stats { /* Aggregated (CPU and subtree) state */ - long state[NR_VM_NODE_STAT_ITEMS]; + long state[NR_VM_NODE_MEMCG_STAT_ITEMS]; /* Non-hierarchical (CPU aggregated) state */ - long state_local[NR_VM_NODE_STAT_ITEMS]; + long state_local[NR_VM_NODE_MEMCG_STAT_ITEMS]; /* Pending child counts during tree propagation */ - long state_pending[NR_VM_NODE_STAT_ITEMS]; + long state_pending[NR_VM_NODE_MEMCG_STAT_ITEMS]; }; /* diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 989ca97402c6..59592f3c7d9b 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -192,8 +192,12 @@ enum node_stat_item { NR_SHMEM_THPS, NR_FILE_THPS, NR_ANON_THPS, - /* No memcg stats for the following fields. */ - NR_SHMEM_PMDMAPPED, + /* + * No memcg stats for the following fields. Please add stats which have + * memcg counterpart above NR_VM_NODE_MEMCG_STAT_ITEMS. + */ + NR_VM_NODE_MEMCG_STAT_ITEMS, + NR_SHMEM_PMDMAPPED = NR_VM_NODE_MEMCG_STAT_ITEMS, NR_FILE_PMDMAPPED, NR_WRITEBACK_TEMP, /* Writeback using temporary buffers */ NR_VMSCAN_WRITE, diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 833d09c1d523..bb1bbf417a46 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1648,6 +1648,9 @@ static void memcg_stat_format(struct mem_cgroup *memcg, struct seq_buf *s) { int i; + /* Reduce by 1 for MEMCG_SWAP as that is not exposed in v2. */ + BUILD_BUG_ON(ARRAY_SIZE(memory_stats) != MEMCG_NR_STAT - 1); + /* * Provide statistics on the state of the memory subsystem as * well as cumulative event counters that show past behavior. @@ -5869,7 +5872,7 @@ static void mem_cgroup_css_rstat_flush(struct cgroup_subsys_state *css, int cpu) lstatc = per_cpu_ptr(pn->lruvec_stats_percpu, cpu); - for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { + for (i = 0; i < NR_VM_NODE_MEMCG_STAT_ITEMS; i++) { delta = pn->lruvec_stats.state_pending[i]; if (delta) pn->lruvec_stats.state_pending[i] = 0;
At the moment, the amount of memory allocated for stats related structs in the mem_cgroup corresponds to the size of enum node_stat_item. However not all fields in enum node_stat_item has corresponding memcg stats. The fields of enum node_stat_item is sorted in such a way that all the fields with corresponding memcg stats are at the start of the enum node_stat_item. So, let's just make an explicit boundary within enum node_stat_item and use that boundary to allocate memory for stats related structs of memcgs. For a given x86_64 config, the size of stats with and without patch is: structs size in bytes w/o with struct lruvec_stats 1128 648 struct lruvec_stats_percpu 752 432 struct memcg_vmstats 1832 1352 struct memcg_vmstats_percpu 1280 960 The memory savings is further compounded by the fact that these structs are allocated for each cpu and for node. To be precise, for each memcg, the memory saved would be: Memory saved = ((21 * 3 * NR_NODES) + (21 * 2 * NR_NODS * NR_CPUS) + (21 * 3) + (21 * 2 * NR_CPUS)) * sizeof(long) Where 21 is the number of fields eliminated. Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev> --- include/linux/memcontrol.h | 12 ++++++------ include/linux/mmzone.h | 8 ++++++-- mm/memcontrol.c | 5 ++++- 3 files changed, 16 insertions(+), 9 deletions(-)