| Message ID | 152594593798.22949.6730606876057040426.stgit@localhost.localdomain (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Thu, May 10, 2018 at 12:52:18PM +0300, Kirill Tkhai wrote: > The patch introduces shrinker::id number, which is used to enumerate > memcg-aware shrinkers. The number start from 0, and the code tries > to maintain it as small as possible. > > This will be used as to represent a memcg-aware shrinkers in memcg > shrinkers map. > > Since all memcg-aware shrinkers are based on list_lru, which is per-memcg > in case of !SLOB only, the new functionality will be under MEMCG && !SLOB > ifdef (symlinked to CONFIG_MEMCG_SHRINKER). Using MEMCG && !SLOB instead of introducing a new config option was done deliberately, see: http://lkml.kernel.org/r/20151210202244.GA4809@cmpxchg.org I guess, this doesn't work well any more, as there are more and more parts depending on kmem accounting, like shrinkers. If you really want to introduce a new option, I think you should call it CONFIG_MEMCG_KMEM and use it consistently throughout the code instead of MEMCG && !SLOB. And this should be done in a separate patch. > diff --git a/fs/super.c b/fs/super.c > index 122c402049a2..16c153d2f4f1 100644 > --- a/fs/super.c > +++ b/fs/super.c > @@ -248,6 +248,9 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags, > s->s_time_gran = 1000000000; > s->cleancache_poolid = CLEANCACHE_NO_POOL; > > +#ifdef CONFIG_MEMCG_SHRINKER > + s->s_shrink.id = -1; > +#endif No point doing that - you are going to overwrite the id anyway in prealloc_shrinker(). > s->s_shrink.seeks = DEFAULT_SEEKS; > s->s_shrink.scan_objects = super_cache_scan; > s->s_shrink.count_objects = super_cache_count; > diff --git a/mm/vmscan.c b/mm/vmscan.c > index 10c8a38c5eef..d691beac1048 100644 > --- a/mm/vmscan.c > +++ b/mm/vmscan.c > @@ -169,6 +169,47 @@ unsigned long vm_total_pages; > static LIST_HEAD(shrinker_list); > static DECLARE_RWSEM(shrinker_rwsem); > > +#ifdef CONFIG_MEMCG_SHRINKER > +static DEFINE_IDR(shrinker_idr); > + > +static int prealloc_memcg_shrinker(struct shrinker *shrinker) > +{ > + int id, ret; > + > + down_write(&shrinker_rwsem); > + ret = id = idr_alloc(&shrinker_idr, shrinker, 0, 0, GFP_KERNEL); > + if (ret < 0) > + goto unlock; > + shrinker->id = id; > + ret = 0; > +unlock: > + up_write(&shrinker_rwsem); > + return ret; > +} > + > +static void del_memcg_shrinker(struct shrinker *shrinker) Nit: IMO unregister_memcg_shrinker() would be a better name as it matches unregister_shrinker(), just like prealloc_memcg_shrinker() matches prealloc_shrinker(). > +{ > + int id = shrinker->id; > + > + if (id < 0) > + return; Nit: I think this should be BUG_ON(id >= 0) as this function is only called for memcg-aware shrinkers AFAICS. > + > + down_write(&shrinker_rwsem); > + idr_remove(&shrinker_idr, id); > + up_write(&shrinker_rwsem); > + shrinker->id = -1; > +}
On 13.05.2018 08:15, Vladimir Davydov wrote: > On Thu, May 10, 2018 at 12:52:18PM +0300, Kirill Tkhai wrote: >> The patch introduces shrinker::id number, which is used to enumerate >> memcg-aware shrinkers. The number start from 0, and the code tries >> to maintain it as small as possible. >> >> This will be used as to represent a memcg-aware shrinkers in memcg >> shrinkers map. >> >> Since all memcg-aware shrinkers are based on list_lru, which is per-memcg >> in case of !SLOB only, the new functionality will be under MEMCG && !SLOB >> ifdef (symlinked to CONFIG_MEMCG_SHRINKER). > > Using MEMCG && !SLOB instead of introducing a new config option was done > deliberately, see: > > http://lkml.kernel.org/r/20151210202244.GA4809@cmpxchg.org > > I guess, this doesn't work well any more, as there are more and more > parts depending on kmem accounting, like shrinkers. If you really want > to introduce a new option, I think you should call it CONFIG_MEMCG_KMEM > and use it consistently throughout the code instead of MEMCG && !SLOB. > And this should be done in a separate patch. What do you mean under "consistently throughout the code"? Should I replace all MEMCG && !SLOB with CONFIG_MEMCG_KMEM over existing code? >> diff --git a/fs/super.c b/fs/super.c >> index 122c402049a2..16c153d2f4f1 100644 >> --- a/fs/super.c >> +++ b/fs/super.c >> @@ -248,6 +248,9 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags, >> s->s_time_gran = 1000000000; >> s->cleancache_poolid = CLEANCACHE_NO_POOL; >> >> +#ifdef CONFIG_MEMCG_SHRINKER >> + s->s_shrink.id = -1; >> +#endif > > No point doing that - you are going to overwrite the id anyway in > prealloc_shrinker(). Not so, this is done deliberately. alloc_super() has the only "fail" label, and it handles all the allocation errors there. The patch just behaves in the same style. It sets "-1" to make destroy_unused_super() able to differ the cases, when shrinker is really initialized, and when it's not. If you don't like this, I can move "s->s_shrink.id = -1;" into prealloc_memcg_shrinker() instead of this. >> s->s_shrink.seeks = DEFAULT_SEEKS; >> s->s_shrink.scan_objects = super_cache_scan; >> s->s_shrink.count_objects = super_cache_count; > >> diff --git a/mm/vmscan.c b/mm/vmscan.c >> index 10c8a38c5eef..d691beac1048 100644 >> --- a/mm/vmscan.c >> +++ b/mm/vmscan.c >> @@ -169,6 +169,47 @@ unsigned long vm_total_pages; >> static LIST_HEAD(shrinker_list); >> static DECLARE_RWSEM(shrinker_rwsem); >> >> +#ifdef CONFIG_MEMCG_SHRINKER >> +static DEFINE_IDR(shrinker_idr); >> + >> +static int prealloc_memcg_shrinker(struct shrinker *shrinker) >> +{ >> + int id, ret; >> + >> + down_write(&shrinker_rwsem); >> + ret = id = idr_alloc(&shrinker_idr, shrinker, 0, 0, GFP_KERNEL); >> + if (ret < 0) >> + goto unlock; >> + shrinker->id = id; >> + ret = 0; >> +unlock: >> + up_write(&shrinker_rwsem); >> + return ret; >> +} >> + >> +static void del_memcg_shrinker(struct shrinker *shrinker) > > Nit: IMO unregister_memcg_shrinker() would be a better name as it > matches unregister_shrinker(), just like prealloc_memcg_shrinker() > matches prealloc_shrinker(). > >> +{ >> + int id = shrinker->id; >> + > >> + if (id < 0) >> + return; > > Nit: I think this should be BUG_ON(id >= 0) as this function is only > called for memcg-aware shrinkers AFAICS. See comment to alloc_super(). >> + >> + down_write(&shrinker_rwsem); >> + idr_remove(&shrinker_idr, id); >> + up_write(&shrinker_rwsem); >> + shrinker->id = -1; >> +} Kirill
On Mon, May 14, 2018 at 12:03:38PM +0300, Kirill Tkhai wrote: > On 13.05.2018 08:15, Vladimir Davydov wrote: > > On Thu, May 10, 2018 at 12:52:18PM +0300, Kirill Tkhai wrote: > >> The patch introduces shrinker::id number, which is used to enumerate > >> memcg-aware shrinkers. The number start from 0, and the code tries > >> to maintain it as small as possible. > >> > >> This will be used as to represent a memcg-aware shrinkers in memcg > >> shrinkers map. > >> > >> Since all memcg-aware shrinkers are based on list_lru, which is per-memcg > >> in case of !SLOB only, the new functionality will be under MEMCG && !SLOB > >> ifdef (symlinked to CONFIG_MEMCG_SHRINKER). > > > > Using MEMCG && !SLOB instead of introducing a new config option was done > > deliberately, see: > > > > http://lkml.kernel.org/r/20151210202244.GA4809@cmpxchg.org > > > > I guess, this doesn't work well any more, as there are more and more > > parts depending on kmem accounting, like shrinkers. If you really want > > to introduce a new option, I think you should call it CONFIG_MEMCG_KMEM > > and use it consistently throughout the code instead of MEMCG && !SLOB. > > And this should be done in a separate patch. > > What do you mean under "consistently throughout the code"? Should I replace > all MEMCG && !SLOB with CONFIG_MEMCG_KMEM over existing code? Yes, otherwise it looks messy - in some places we check !SLOB, in others we use CONFIG_MEMCG_SHRINKER (or whatever it will be called). > > >> diff --git a/fs/super.c b/fs/super.c > >> index 122c402049a2..16c153d2f4f1 100644 > >> --- a/fs/super.c > >> +++ b/fs/super.c > >> @@ -248,6 +248,9 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags, > >> s->s_time_gran = 1000000000; > >> s->cleancache_poolid = CLEANCACHE_NO_POOL; > >> > >> +#ifdef CONFIG_MEMCG_SHRINKER > >> + s->s_shrink.id = -1; > >> +#endif > > > > No point doing that - you are going to overwrite the id anyway in > > prealloc_shrinker(). > > Not so, this is done deliberately. alloc_super() has the only "fail" label, > and it handles all the allocation errors there. The patch just behaves in > the same style. It sets "-1" to make destroy_unused_super() able to differ > the cases, when shrinker is really initialized, and when it's not. > If you don't like this, I can move "s->s_shrink.id = -1;" into > prealloc_memcg_shrinker() instead of this. Yes, please do so that we don't have MEMCG ifdefs in fs code. Thanks.
diff --git a/fs/super.c b/fs/super.c index 122c402049a2..16c153d2f4f1 100644 --- a/fs/super.c +++ b/fs/super.c @@ -248,6 +248,9 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags, s->s_time_gran = 1000000000; s->cleancache_poolid = CLEANCACHE_NO_POOL; +#ifdef CONFIG_MEMCG_SHRINKER + s->s_shrink.id = -1; +#endif s->s_shrink.seeks = DEFAULT_SEEKS; s->s_shrink.scan_objects = super_cache_scan; s->s_shrink.count_objects = super_cache_count; diff --git a/include/linux/shrinker.h b/include/linux/shrinker.h index 6794490f25b2..d8f3fc833e6e 100644 --- a/include/linux/shrinker.h +++ b/include/linux/shrinker.h @@ -66,6 +66,10 @@ struct shrinker { /* These are for internal use */ struct list_head list; +#ifdef CONFIG_MEMCG_SHRINKER + /* ID in shrinker_idr */ + int id; +#endif /* objs pending delete, per node */ atomic_long_t *nr_deferred; }; diff --git a/init/Kconfig b/init/Kconfig index 1706d963766b..09e201c2ada9 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -680,6 +680,11 @@ config MEMCG_SWAP_ENABLED select this option (if, for some reason, they need to disable it then swapaccount=0 does the trick). +config MEMCG_SHRINKER + bool + depends on MEMCG && !SLOB + default y + config BLK_CGROUP bool "IO controller" depends on BLOCK diff --git a/mm/vmscan.c b/mm/vmscan.c index 10c8a38c5eef..d691beac1048 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -169,6 +169,47 @@ unsigned long vm_total_pages; static LIST_HEAD(shrinker_list); static DECLARE_RWSEM(shrinker_rwsem); +#ifdef CONFIG_MEMCG_SHRINKER +static DEFINE_IDR(shrinker_idr); + +static int prealloc_memcg_shrinker(struct shrinker *shrinker) +{ + int id, ret; + + down_write(&shrinker_rwsem); + ret = id = idr_alloc(&shrinker_idr, shrinker, 0, 0, GFP_KERNEL); + if (ret < 0) + goto unlock; + shrinker->id = id; + ret = 0; +unlock: + up_write(&shrinker_rwsem); + return ret; +} + +static void del_memcg_shrinker(struct shrinker *shrinker) +{ + int id = shrinker->id; + + if (id < 0) + return; + + down_write(&shrinker_rwsem); + idr_remove(&shrinker_idr, id); + up_write(&shrinker_rwsem); + shrinker->id = -1; +} +#else /* CONFIG_MEMCG_SHRINKER */ +static int prealloc_memcg_shrinker(struct shrinker *shrinker) +{ + return 0; +} + +static void del_memcg_shrinker(struct shrinker *shrinker) +{ +} +#endif /* CONFIG_MEMCG_SHRINKER */ + #ifdef CONFIG_MEMCG static bool global_reclaim(struct scan_control *sc) { @@ -306,6 +347,7 @@ unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru, int zone int prealloc_shrinker(struct shrinker *shrinker) { size_t size = sizeof(*shrinker->nr_deferred); + int ret; if (shrinker->flags & SHRINKER_NUMA_AWARE) size *= nr_node_ids; @@ -313,11 +355,26 @@ int prealloc_shrinker(struct shrinker *shrinker) shrinker->nr_deferred = kzalloc(size, GFP_KERNEL); if (!shrinker->nr_deferred) return -ENOMEM; + + if (shrinker->flags & SHRINKER_MEMCG_AWARE) { + ret = prealloc_memcg_shrinker(shrinker); + if (ret) + goto free_deferred; + } + return 0; + +free_deferred: + kfree(shrinker->nr_deferred); + shrinker->nr_deferred = NULL; + return -ENOMEM; } void free_prealloced_shrinker(struct shrinker *shrinker) { + if (shrinker->flags & SHRINKER_MEMCG_AWARE) + del_memcg_shrinker(shrinker); + kfree(shrinker->nr_deferred); shrinker->nr_deferred = NULL; } @@ -347,6 +404,8 @@ void unregister_shrinker(struct shrinker *shrinker) { if (!shrinker->nr_deferred) return; + if (shrinker->flags & SHRINKER_MEMCG_AWARE) + del_memcg_shrinker(shrinker); down_write(&shrinker_rwsem); list_del(&shrinker->list); up_write(&shrinker_rwsem);
The patch introduces shrinker::id number, which is used to enumerate memcg-aware shrinkers. The number start from 0, and the code tries to maintain it as small as possible. This will be used as to represent a memcg-aware shrinkers in memcg shrinkers map. Since all memcg-aware shrinkers are based on list_lru, which is per-memcg in case of !SLOB only, the new functionality will be under MEMCG && !SLOB ifdef (symlinked to CONFIG_MEMCG_SHRINKER). Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com> --- fs/super.c | 3 ++ include/linux/shrinker.h | 4 +++ init/Kconfig | 5 ++++ mm/vmscan.c | 59 ++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 71 insertions(+)