| Message ID | 20201208095132.79383-1-songmuchun@bytedance.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v2] mm: memcontrol: optimize per-lruvec stats counter memory usage | expand |
On Tue, Dec 8, 2020 at 1:53 AM Muchun Song <songmuchun@bytedance.com> wrote: > > The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), so the type of s32 > of lruvec_stat_cpu is enough. And introduce struct per_cpu_lruvec_stat > to optimize memory usage. > > The size of struct lruvec_stat is 304 bytes on 64 bits system. As it > is a per-cpu structure. So with this patch, we can save 304 / 2 * ncpu > bytes per-memcg per-node where ncpu is the number of the possible CPU. > If there are c memory cgroup (include dying cgroup) and n NUMA node in > the system. Finally, we can save (152 * ncpu * c * n) bytes. > > Signed-off-by: Muchun Song <songmuchun@bytedance.com> Few nits below: Reviewed-by: Shakeel Butt <shakeelb@google.com> > --- > Changes in v1 -> v2: > - Update the commit log to point out how many bytes that we can save. > > include/linux/memcontrol.h | 6 +++++- > mm/memcontrol.c | 10 +++++++++- > 2 files changed, 14 insertions(+), 2 deletions(-) > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > index 3febf64d1b80..290d6ec8535a 100644 > --- a/include/linux/memcontrol.h > +++ b/include/linux/memcontrol.h > @@ -92,6 +92,10 @@ struct lruvec_stat { > long count[NR_VM_NODE_STAT_ITEMS]; > }; > > +struct per_cpu_lruvec_stat { lruvec_stat is also per-cpu, so the name per_cpu_lruvec_stat does not really tell why it is different from lruvec. Maybe name is batched_lruvec_stat or something else. > + s32 count[NR_VM_NODE_STAT_ITEMS]; > +}; > + > /* > * Bitmap of shrinker::id corresponding to memcg-aware shrinkers, > * which have elements charged to this memcg. > @@ -111,7 +115,7 @@ struct mem_cgroup_per_node { > struct lruvec_stat __percpu *lruvec_stat_local; A comment for the above why it still needs to be lruvec_stat. > > /* Subtree VM stats (batched updates) */ > - struct lruvec_stat __percpu *lruvec_stat_cpu; > + struct per_cpu_lruvec_stat __percpu *lruvec_stat_cpu; > atomic_long_t lruvec_stat[NR_VM_NODE_STAT_ITEMS]; > > unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > index eec44918d373..da6dc6ca388d 100644 > --- a/mm/memcontrol.c > +++ b/mm/memcontrol.c > @@ -5198,7 +5198,7 @@ static int alloc_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) > return 1; > } > > - pn->lruvec_stat_cpu = alloc_percpu_gfp(struct lruvec_stat, > + pn->lruvec_stat_cpu = alloc_percpu_gfp(struct per_cpu_lruvec_stat, > GFP_KERNEL_ACCOUNT); > if (!pn->lruvec_stat_cpu) { > free_percpu(pn->lruvec_stat_local); > @@ -7089,6 +7089,14 @@ static int __init mem_cgroup_init(void) > { > int cpu, node; > > + /* > + * Currently s32 type (can refer to struct per_cpu_lruvec_stat) is > + * used for per-memcg-per-cpu caching of per-node statistics. In order > + * to work fine, we should make sure that the overfill threshold can't > + * exceed S32_MAX / PAGE_SIZE. > + */ > + BUILD_BUG_ON(MEMCG_CHARGE_BATCH > S32_MAX / PAGE_SIZE); > + > cpuhp_setup_state_nocalls(CPUHP_MM_MEMCQ_DEAD, "mm/memctrl:dead", NULL, > memcg_hotplug_cpu_dead); > > -- > 2.11.0 >
On Tue, Dec 08, 2020 at 05:51:32PM +0800, Muchun Song wrote: > The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), so the type of s32 > of lruvec_stat_cpu is enough. And introduce struct per_cpu_lruvec_stat > to optimize memory usage. > > The size of struct lruvec_stat is 304 bytes on 64 bits system. As it > is a per-cpu structure. So with this patch, we can save 304 / 2 * ncpu > bytes per-memcg per-node where ncpu is the number of the possible CPU. > If there are c memory cgroup (include dying cgroup) and n NUMA node in > the system. Finally, we can save (152 * ncpu * c * n) bytes. Honestly, I'm not convinced. Say, ncpu = 32, n = 2, c = 500. We're saving <5Mb of memory. If the machine has 128Gb of RAM, it's .000000003%. Using longs (s64) allows not to think too much about overflows and can also be slightly faster on 64-bit machines. Thanks!
On Tue, Dec 08, 2020 at 06:21:18PM -0800, Roman Gushchin wrote: > On Tue, Dec 08, 2020 at 05:51:32PM +0800, Muchun Song wrote: > > The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), so the type of s32 > > of lruvec_stat_cpu is enough. And introduce struct per_cpu_lruvec_stat > > to optimize memory usage. > > > > The size of struct lruvec_stat is 304 bytes on 64 bits system. As it > > is a per-cpu structure. So with this patch, we can save 304 / 2 * ncpu > > bytes per-memcg per-node where ncpu is the number of the possible CPU. > > If there are c memory cgroup (include dying cgroup) and n NUMA node in > > the system. Finally, we can save (152 * ncpu * c * n) bytes. > > Honestly, I'm not convinced. > Say, ncpu = 32, n = 2, c = 500. We're saving <5Mb of memory. > If the machine has 128Gb of RAM, it's .000000003%. My bad, it's actually (32*2*500*152)/(128*1024*1024*1024) = .0035% but it's still in the noise. > > Using longs (s64) allows not to think too much about overflows > and can also be slightly faster on 64-bit machines. > > Thanks!
On Wed, Dec 9, 2020 at 10:21 AM Roman Gushchin <guro@fb.com> wrote: > > On Tue, Dec 08, 2020 at 05:51:32PM +0800, Muchun Song wrote: > > The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), so the type of s32 > > of lruvec_stat_cpu is enough. And introduce struct per_cpu_lruvec_stat > > to optimize memory usage. > > > > The size of struct lruvec_stat is 304 bytes on 64 bits system. As it > > is a per-cpu structure. So with this patch, we can save 304 / 2 * ncpu > > bytes per-memcg per-node where ncpu is the number of the possible CPU. > > If there are c memory cgroup (include dying cgroup) and n NUMA node in > > the system. Finally, we can save (152 * ncpu * c * n) bytes. > > Honestly, I'm not convinced. > Say, ncpu = 32, n = 2, c = 500. We're saving <5Mb of memory. > If the machine has 128Gb of RAM, it's .000000003%. Hi Roman, When the cpu hotplug is enabled, the ncpu can be 256 on some configurations. Also, the c can be more large when there are many dying cgroup in the system. So the savings depends on the environment and configurations. Right? > > Using longs (s64) allows not to think too much about overflows > and can also be slightly faster on 64-bit machines. > > Thanks! -- Yours, Muchun
On Wed, Dec 09, 2020 at 10:31:55AM +0800, Muchun Song wrote: > On Wed, Dec 9, 2020 at 10:21 AM Roman Gushchin <guro@fb.com> wrote: > > > > On Tue, Dec 08, 2020 at 05:51:32PM +0800, Muchun Song wrote: > > > The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), so the type of s32 > > > of lruvec_stat_cpu is enough. Actually the threshold can be as big as MEMCG_CHARGE_BATCH * PAGE_SIZE. It still fits into s32, but without explicitly saying it it's hard to understand why not choosing s8, as in vmstat.c. > > > > > > The size of struct lruvec_stat is 304 bytes on 64 bits system. As it > > > is a per-cpu structure. So with this patch, we can save 304 / 2 * ncpu > > > bytes per-memcg per-node where ncpu is the number of the possible CPU. > > > If there are c memory cgroup (include dying cgroup) and n NUMA node in > > > the system. Finally, we can save (152 * ncpu * c * n) bytes. > > > > Honestly, I'm not convinced. > > Say, ncpu = 32, n = 2, c = 500. We're saving <5Mb of memory. > > If the machine has 128Gb of RAM, it's .000000003%. > > Hi Roman, > > When the cpu hotplug is enabled, the ncpu can be 256 on > some configurations. Also, the c can be more large when > there are many dying cgroup in the system. > > So the savings depends on the environment and > configurations. Right? Of course, but machines with more CPUs tend to have more RAM as well. Thanks!
On Wed, Dec 9, 2020 at 11:52 AM Roman Gushchin <guro@fb.com> wrote: > > On Wed, Dec 09, 2020 at 10:31:55AM +0800, Muchun Song wrote: > > On Wed, Dec 9, 2020 at 10:21 AM Roman Gushchin <guro@fb.com> wrote: > > > > > > On Tue, Dec 08, 2020 at 05:51:32PM +0800, Muchun Song wrote: > > > > The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), so the type of s32 > > > > of lruvec_stat_cpu is enough. > > Actually the threshold can be as big as MEMCG_CHARGE_BATCH * PAGE_SIZE. > It still fits into s32, but without explicitly saying it it's hard to > understand why not choosing s8, as in vmstat.c. Yeah, here I need to update the commit log. > > > > > > > > > The size of struct lruvec_stat is 304 bytes on 64 bits system. As it > > > > is a per-cpu structure. So with this patch, we can save 304 / 2 * ncpu > > > > bytes per-memcg per-node where ncpu is the number of the possible CPU. > > > > If there are c memory cgroup (include dying cgroup) and n NUMA node in > > > > the system. Finally, we can save (152 * ncpu * c * n) bytes. > > > > > > Honestly, I'm not convinced. > > > Say, ncpu = 32, n = 2, c = 500. We're saving <5Mb of memory. > > > If the machine has 128Gb of RAM, it's .000000003%. > > > > Hi Roman, > > > > When the cpu hotplug is enabled, the ncpu can be 256 on > > some configurations. Also, the c can be more large when > > there are many dying cgroup in the system. > > > > So the savings depends on the environment and > > configurations. Right? > > Of course, but machines with more CPUs tend to have more RAM as well. Here I mean possible CPU not online CPU. The number of possible CPUs may be greater than online CPUs. The per-cpu allocator is based on the number of possible CPUs. Right? Thanks. > > Thanks!
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 3febf64d1b80..290d6ec8535a 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -92,6 +92,10 @@ struct lruvec_stat { long count[NR_VM_NODE_STAT_ITEMS]; }; +struct per_cpu_lruvec_stat { + s32 count[NR_VM_NODE_STAT_ITEMS]; +}; + /* * Bitmap of shrinker::id corresponding to memcg-aware shrinkers, * which have elements charged to this memcg. @@ -111,7 +115,7 @@ struct mem_cgroup_per_node { struct lruvec_stat __percpu *lruvec_stat_local; /* Subtree VM stats (batched updates) */ - struct lruvec_stat __percpu *lruvec_stat_cpu; + struct per_cpu_lruvec_stat __percpu *lruvec_stat_cpu; atomic_long_t lruvec_stat[NR_VM_NODE_STAT_ITEMS]; unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index eec44918d373..da6dc6ca388d 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5198,7 +5198,7 @@ static int alloc_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) return 1; } - pn->lruvec_stat_cpu = alloc_percpu_gfp(struct lruvec_stat, + pn->lruvec_stat_cpu = alloc_percpu_gfp(struct per_cpu_lruvec_stat, GFP_KERNEL_ACCOUNT); if (!pn->lruvec_stat_cpu) { free_percpu(pn->lruvec_stat_local); @@ -7089,6 +7089,14 @@ static int __init mem_cgroup_init(void) { int cpu, node; + /* + * Currently s32 type (can refer to struct per_cpu_lruvec_stat) is + * used for per-memcg-per-cpu caching of per-node statistics. In order + * to work fine, we should make sure that the overfill threshold can't + * exceed S32_MAX / PAGE_SIZE. + */ + BUILD_BUG_ON(MEMCG_CHARGE_BATCH > S32_MAX / PAGE_SIZE); + cpuhp_setup_state_nocalls(CPUHP_MM_MEMCQ_DEAD, "mm/memctrl:dead", NULL, memcg_hotplug_cpu_dead);
The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), so the type of s32 of lruvec_stat_cpu is enough. And introduce struct per_cpu_lruvec_stat to optimize memory usage. The size of struct lruvec_stat is 304 bytes on 64 bits system. As it is a per-cpu structure. So with this patch, we can save 304 / 2 * ncpu bytes per-memcg per-node where ncpu is the number of the possible CPU. If there are c memory cgroup (include dying cgroup) and n NUMA node in the system. Finally, we can save (152 * ncpu * c * n) bytes. Signed-off-by: Muchun Song <songmuchun@bytedance.com> --- Changes in v1 -> v2: - Update the commit log to point out how many bytes that we can save. include/linux/memcontrol.h | 6 +++++- mm/memcontrol.c | 10 +++++++++- 2 files changed, 14 insertions(+), 2 deletions(-)