| Message ID | 20171123143537.ztpxpk3sjbpo72rf@dhcp22.suse.cz (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Thu, Nov 23, 2017 at 03:35:37PM +0100, Michal Hocko wrote: > Hopefully less screwed version. But as I've said I am not really > familiar with the code and do not feel competent to change it so please > be very careful here. I've moved the shrinker registration to > alloc_super which turned out to be simpler. I don't get it. Why the hell do we need all that PITA in the first place? Just make sget_userns() end with if (unlikely(regsiter_shrinker(&s->s_shrink) != 0)) { deactivate_locked_super(s); s = ERR_PTR(-ENOMEM); } return s; and be done with that. All there is to it...
On Thu 23-11-17 14:55:40, Al Viro wrote: > On Thu, Nov 23, 2017 at 03:35:37PM +0100, Michal Hocko wrote: > > Hopefully less screwed version. But as I've said I am not really > > familiar with the code and do not feel competent to change it so please > > be very careful here. I've moved the shrinker registration to > > alloc_super which turned out to be simpler. > > I don't get it. Why the hell do we need all that PITA in the first place? > Just make sget_userns() end with > if (unlikely(regsiter_shrinker(&s->s_shrink) != 0)) { > deactivate_locked_super(s); > s = ERR_PTR(-ENOMEM); > } > return s; > and be done with that. All there is to it... Who is going to unregister that shrinker on other failure paths?
Al Viro wrote: > On Thu, Nov 23, 2017 at 03:35:37PM +0100, Michal Hocko wrote: > > Hopefully less screwed version. But as I've said I am not really > > familiar with the code and do not feel competent to change it so please > > be very careful here. I've moved the shrinker registration to > > alloc_super which turned out to be simpler. > > I don't get it. Why the hell do we need all that PITA in the first place? > Just make sget_userns() end with > if (unlikely(regsiter_shrinker(&s->s_shrink) != 0)) { > deactivate_locked_super(s); > s = ERR_PTR(-ENOMEM); > } > return s; > and be done with that. All there is to it... > Doesn't deactivate_locked_super() call unregister_shrinker() ?
On Fri, Nov 24, 2017 at 12:04:23AM +0900, Tetsuo Handa wrote: > Al Viro wrote: > > On Thu, Nov 23, 2017 at 03:35:37PM +0100, Michal Hocko wrote: > > > Hopefully less screwed version. But as I've said I am not really > > > familiar with the code and do not feel competent to change it so please > > > be very careful here. I've moved the shrinker registration to > > > alloc_super which turned out to be simpler. > > > > I don't get it. Why the hell do we need all that PITA in the first place? > > Just make sget_userns() end with > > if (unlikely(regsiter_shrinker(&s->s_shrink) != 0)) { > > deactivate_locked_super(s); > > s = ERR_PTR(-ENOMEM); > > } > > return s; > > and be done with that. All there is to it... > > > > Doesn't deactivate_locked_super() call unregister_shrinker() ? And? unregister_shrinker() will do list_del() on empty list_head and kfree(NULL); where's the problem with that?
On Thu 23-11-17 16:02:11, Michal Hocko wrote: > On Thu 23-11-17 14:55:40, Al Viro wrote: > > On Thu, Nov 23, 2017 at 03:35:37PM +0100, Michal Hocko wrote: > > > Hopefully less screwed version. But as I've said I am not really > > > familiar with the code and do not feel competent to change it so please > > > be very careful here. I've moved the shrinker registration to > > > alloc_super which turned out to be simpler. > > > > I don't get it. Why the hell do we need all that PITA in the first place? > > Just make sget_userns() end with > > if (unlikely(regsiter_shrinker(&s->s_shrink) != 0)) { > > deactivate_locked_super(s); > > s = ERR_PTR(-ENOMEM); > > } > > return s; > > and be done with that. All there is to it... > > Who is going to unregister that shrinker on other failure paths? Scratch that. I've mixed destroy_unused_super with deactivate_locked_super. Go with whatever works...
Al Viro wrote: > On Fri, Nov 24, 2017 at 12:04:23AM +0900, Tetsuo Handa wrote: > > Al Viro wrote: > > > On Thu, Nov 23, 2017 at 03:35:37PM +0100, Michal Hocko wrote: > > > > Hopefully less screwed version. But as I've said I am not really > > > > familiar with the code and do not feel competent to change it so please > > > > be very careful here. I've moved the shrinker registration to > > > > alloc_super which turned out to be simpler. > > > > > > I don't get it. Why the hell do we need all that PITA in the first place? > > > Just make sget_userns() end with > > > if (unlikely(regsiter_shrinker(&s->s_shrink) != 0)) { > > > deactivate_locked_super(s); > > > s = ERR_PTR(-ENOMEM); > > > } > > > return s; > > > and be done with that. All there is to it... > > > > > > > Doesn't deactivate_locked_super() call unregister_shrinker() ? > > And? unregister_shrinker() will do list_del() on empty list_head > and kfree(NULL); where's the problem with that? > The problem is that calling unregister_shrinker() without successful register_shrinker() causes crash due to s_shrink.list.{prev,next} == NULL.
On Fri, Nov 24, 2017 at 12:35:29AM +0900, Tetsuo Handa wrote: > Al Viro wrote: > > On Fri, Nov 24, 2017 at 12:04:23AM +0900, Tetsuo Handa wrote: > > > Al Viro wrote: > > > > On Thu, Nov 23, 2017 at 03:35:37PM +0100, Michal Hocko wrote: > > > > > Hopefully less screwed version. But as I've said I am not really > > > > > familiar with the code and do not feel competent to change it so please > > > > > be very careful here. I've moved the shrinker registration to > > > > > alloc_super which turned out to be simpler. > > > > > > > > I don't get it. Why the hell do we need all that PITA in the first place? > > > > Just make sget_userns() end with > > > > if (unlikely(regsiter_shrinker(&s->s_shrink) != 0)) { > > > > deactivate_locked_super(s); > > > > s = ERR_PTR(-ENOMEM); > > > > } > > > > return s; > > > > and be done with that. All there is to it... > > > > > > > > > > Doesn't deactivate_locked_super() call unregister_shrinker() ? > > > > And? unregister_shrinker() will do list_del() on empty list_head > > and kfree(NULL); where's the problem with that? > > > The problem is that calling unregister_shrinker() without successful > register_shrinker() causes crash due to s_shrink.list.{prev,next} == NULL. *shrug* shrinker->nr_deferred = kzalloc(size, GFP_KERNEL); if (!shrinker->nr_deferred) { /* make sure it's in consistent state */ INIT_LIST_HEAD(&shrinker->list); return -ENOMEM; }
Al Viro wrote: > On Fri, Nov 24, 2017 at 12:35:29AM +0900, Tetsuo Handa wrote: > > Al Viro wrote: > > > On Fri, Nov 24, 2017 at 12:04:23AM +0900, Tetsuo Handa wrote: > > > > Al Viro wrote: > > > > > On Thu, Nov 23, 2017 at 03:35:37PM +0100, Michal Hocko wrote: > > > > > > Hopefully less screwed version. But as I've said I am not really > > > > > > familiar with the code and do not feel competent to change it so please > > > > > > be very careful here. I've moved the shrinker registration to > > > > > > alloc_super which turned out to be simpler. > > > > > > > > > > I don't get it. Why the hell do we need all that PITA in the first place? > > > > > Just make sget_userns() end with > > > > > if (unlikely(regsiter_shrinker(&s->s_shrink) != 0)) { > > > > > deactivate_locked_super(s); > > > > > s = ERR_PTR(-ENOMEM); > > > > > } > > > > > return s; > > > > > and be done with that. All there is to it... > > > > > > > > > > > > > Doesn't deactivate_locked_super() call unregister_shrinker() ? > > > > > > And? unregister_shrinker() will do list_del() on empty list_head > > > and kfree(NULL); where's the problem with that? > > > > > The problem is that calling unregister_shrinker() without successful > > register_shrinker() causes crash due to s_shrink.list.{prev,next} == NULL. > > *shrug* > shrinker->nr_deferred = kzalloc(size, GFP_KERNEL); > if (!shrinker->nr_deferred) { > /* make sure it's in consistent state */ > INIT_LIST_HEAD(&shrinker->list); > return -ENOMEM; > } > > Yes, that will work. Michal, like Al thinks, making unregister_shrinker() no-op if register_shrinker() failed simplifies things. Can we go with http://lkml.kernel.org/r/1511265853-15654-1-git-send-email-penguin-kernel@I-love.SAKURA.ne.jp with patch description updated to include Syzbot report?
On Fri 24-11-17 06:51:09, Tetsuo Handa wrote: > Al Viro wrote: > > On Fri, Nov 24, 2017 at 12:35:29AM +0900, Tetsuo Handa wrote: > > > Al Viro wrote: > > > > On Fri, Nov 24, 2017 at 12:04:23AM +0900, Tetsuo Handa wrote: > > > > > Al Viro wrote: > > > > > > On Thu, Nov 23, 2017 at 03:35:37PM +0100, Michal Hocko wrote: > > > > > > > Hopefully less screwed version. But as I've said I am not really > > > > > > > familiar with the code and do not feel competent to change it so please > > > > > > > be very careful here. I've moved the shrinker registration to > > > > > > > alloc_super which turned out to be simpler. > > > > > > > > > > > > I don't get it. Why the hell do we need all that PITA in the first place? > > > > > > Just make sget_userns() end with > > > > > > if (unlikely(regsiter_shrinker(&s->s_shrink) != 0)) { > > > > > > deactivate_locked_super(s); > > > > > > s = ERR_PTR(-ENOMEM); > > > > > > } > > > > > > return s; > > > > > > and be done with that. All there is to it... > > > > > > > > > > > > > > > > Doesn't deactivate_locked_super() call unregister_shrinker() ? > > > > > > > > And? unregister_shrinker() will do list_del() on empty list_head > > > > and kfree(NULL); where's the problem with that? > > > > > > > The problem is that calling unregister_shrinker() without successful > > > register_shrinker() causes crash due to s_shrink.list.{prev,next} == NULL. > > > > *shrug* > > shrinker->nr_deferred = kzalloc(size, GFP_KERNEL); > > if (!shrinker->nr_deferred) { > > /* make sure it's in consistent state */ > > INIT_LIST_HEAD(&shrinker->list); > > return -ENOMEM; > > } > > > > > > Yes, that will work. > > Michal, like Al thinks, making unregister_shrinker() no-op if > register_shrinker() failed simplifies things. Can we go with > http://lkml.kernel.org/r/1511265853-15654-1-git-send-email-penguin-kernel@I-love.SAKURA.ne.jp > with patch description updated to include Syzbot report? I am not opposed to that patch. I just want it to make sure callers _do_ handle the error because a missing shrinker can cause memory pressure realated issues. unregister_shrinker definitely shouldn't blow up but I wanted it to warn. This would however trigger a false positive in this path, right? It is true that the allocation failure would already trigger warning so the clean up path could be more relaxed. It can be still quite some time between those two events. In any case. I do not have a strong preference. If relying on deactivate_locked_super is really seem much better for the vfs code then let's go with your patch without warning.
On Thu 23-11-17 14:55:40, Al Viro wrote: > On Thu, Nov 23, 2017 at 03:35:37PM +0100, Michal Hocko wrote: > > Hopefully less screwed version. But as I've said I am not really > > familiar with the code and do not feel competent to change it so please > > be very careful here. I've moved the shrinker registration to > > alloc_super which turned out to be simpler. > > I don't get it. Why the hell do we need all that PITA in the first place? > Just make sget_userns() end with > if (unlikely(regsiter_shrinker(&s->s_shrink) != 0)) { > deactivate_locked_super(s); > s = ERR_PTR(-ENOMEM); > } > return s; > and be done with that. All there is to it... Al, do you plan to push this fix? Tetsuo's unregister_shrinker fortification is already in the mmotm tree.
On Wed, Nov 29, 2017 at 12:55:15PM +0100, Michal Hocko wrote: > On Thu 23-11-17 14:55:40, Al Viro wrote: > > On Thu, Nov 23, 2017 at 03:35:37PM +0100, Michal Hocko wrote: > > > Hopefully less screwed version. But as I've said I am not really > > > familiar with the code and do not feel competent to change it so please > > > be very careful here. I've moved the shrinker registration to > > > alloc_super which turned out to be simpler. > > > > I don't get it. Why the hell do we need all that PITA in the first place? > > Just make sget_userns() end with > > if (unlikely(regsiter_shrinker(&s->s_shrink) != 0)) { > > deactivate_locked_super(s); > > s = ERR_PTR(-ENOMEM); > > } > > return s; > > and be done with that. All there is to it... > > Al, do you plan to push this fix? Tetsuo's unregister_shrinker > fortification is already in the mmotm tree. Is it in any git branch I could pull from? Or I could just throw it into vfs.git#for-linus before the fix above - up to you, folks...
On Sat, Dec 09, 2017 at 08:59:22PM +0000, Al Viro wrote: > On Wed, Nov 29, 2017 at 12:55:15PM +0100, Michal Hocko wrote: > > On Thu 23-11-17 14:55:40, Al Viro wrote: > > > On Thu, Nov 23, 2017 at 03:35:37PM +0100, Michal Hocko wrote: > > > > Hopefully less screwed version. But as I've said I am not really > > > > familiar with the code and do not feel competent to change it so please > > > > be very careful here. I've moved the shrinker registration to > > > > alloc_super which turned out to be simpler. > > > > > > I don't get it. Why the hell do we need all that PITA in the first place? > > > Just make sget_userns() end with > > > if (unlikely(regsiter_shrinker(&s->s_shrink) != 0)) { > > > deactivate_locked_super(s); > > > s = ERR_PTR(-ENOMEM); > > > } > > > return s; > > > and be done with that. All there is to it... > > > > Al, do you plan to push this fix? Tetsuo's unregister_shrinker > > fortification is already in the mmotm tree. > > Is it in any git branch I could pull from? Or I could just throw it > into vfs.git#for-linus before the fix above - up to you, folks... Actually, looking at mmotm... I don't see it there. Which patch is it in?
Al Viro wrote: > On Sat, Dec 09, 2017 at 08:59:22PM +0000, Al Viro wrote: > > On Wed, Nov 29, 2017 at 12:55:15PM +0100, Michal Hocko wrote: > > > On Thu 23-11-17 14:55:40, Al Viro wrote: > > > > On Thu, Nov 23, 2017 at 03:35:37PM +0100, Michal Hocko wrote: > > > > > Hopefully less screwed version. But as I've said I am not really > > > > > familiar with the code and do not feel competent to change it so please > > > > > be very careful here. I've moved the shrinker registration to > > > > > alloc_super which turned out to be simpler. > > > > > > > > I don't get it. Why the hell do we need all that PITA in the first place? > > > > Just make sget_userns() end with > > > > if (unlikely(regsiter_shrinker(&s->s_shrink) != 0)) { > > > > deactivate_locked_super(s); > > > > s = ERR_PTR(-ENOMEM); > > > > } > > > > return s; > > > > and be done with that. All there is to it... > > > > > > Al, do you plan to push this fix? Tetsuo's unregister_shrinker > > > fortification is already in the mmotm tree. > > > > Is it in any git branch I could pull from? Or I could just throw it > > into vfs.git#for-linus before the fix above - up to you, folks... > > Actually, looking at mmotm... I don't see it there. Which patch > is it in? > My unregister_shrinker() fortification patch ( http://lkml.kernel.org/r/1511523385-6433-1-git-send-email-penguin-kernel@I-love.SAKURA.ne.jp ) is not yet in the mmotm tree due to disagreement between Michal and I, but you can throw your sget_userns() patch into vfs.git#for-linus anyway. We will eventually apply unregister_shrinker() fortification patch. I'm observing whether people notice my __must_check annotation patch ( http://lkml.kernel.org/r/1511523385-6433-2-git-send-email-penguin-kernel@I-love.SAKURA.ne.jp ) in linux-next.git and fix register_shrinker() callers. If people do not fix the callers, both patches need to be sent to linux.git.
On Sun 10-12-17 11:33:18, Tetsuo Handa wrote: > Al Viro wrote: > > On Sat, Dec 09, 2017 at 08:59:22PM +0000, Al Viro wrote: > > > On Wed, Nov 29, 2017 at 12:55:15PM +0100, Michal Hocko wrote: > > > > On Thu 23-11-17 14:55:40, Al Viro wrote: > > > > > On Thu, Nov 23, 2017 at 03:35:37PM +0100, Michal Hocko wrote: > > > > > > Hopefully less screwed version. But as I've said I am not really > > > > > > familiar with the code and do not feel competent to change it so please > > > > > > be very careful here. I've moved the shrinker registration to > > > > > > alloc_super which turned out to be simpler. > > > > > > > > > > I don't get it. Why the hell do we need all that PITA in the first place? > > > > > Just make sget_userns() end with > > > > > if (unlikely(regsiter_shrinker(&s->s_shrink) != 0)) { > > > > > deactivate_locked_super(s); > > > > > s = ERR_PTR(-ENOMEM); > > > > > } > > > > > return s; > > > > > and be done with that. All there is to it... > > > > > > > > Al, do you plan to push this fix? Tetsuo's unregister_shrinker > > > > fortification is already in the mmotm tree. > > > > > > Is it in any git branch I could pull from? Or I could just throw it > > > into vfs.git#for-linus before the fix above - up to you, folks... > > > > Actually, looking at mmotm... I don't see it there. Which patch > > is it in? > > > > My unregister_shrinker() fortification patch > ( http://lkml.kernel.org/r/1511523385-6433-1-git-send-email-penguin-kernel@I-love.SAKURA.ne.jp ) > is not yet in the mmotm tree due to disagreement between Michal and I, but > you can throw your sget_userns() patch into vfs.git#for-linus anyway. > We will eventually apply unregister_shrinker() fortification patch. I've acked the patch http://lkml.kernel.org/r/20171124122148.qevmiogh3pzr4zix@dhcp22.suse.cz I disagreed with your must_check patch which has nothing to do with the patch disussed here. I've also suggested some changelog clarifications. Please repost and I am pretty sure Andew will pick it up.
On Sun, Dec 10, 2017 at 11:05:15AM +0100, Michal Hocko wrote: > > My unregister_shrinker() fortification patch > > ( http://lkml.kernel.org/r/1511523385-6433-1-git-send-email-penguin-kernel@I-love.SAKURA.ne.jp ) > > is not yet in the mmotm tree due to disagreement between Michal and I, but > > you can throw your sget_userns() patch into vfs.git#for-linus anyway. > > We will eventually apply unregister_shrinker() fortification patch. > > I've acked the patch http://lkml.kernel.org/r/20171124122148.qevmiogh3pzr4zix@dhcp22.suse.cz > I disagreed with your must_check patch which has nothing to do with the > patch disussed here. I've also suggested some changelog clarifications. > Please repost and I am pretty sure Andew will pick it up. Hell, I will pick it up, add sget one on top of that and put both into #for-linus.
diff --git a/fs/super.c b/fs/super.c index d4e33e8f1e6f..29edf3d1875b 100644 --- a/fs/super.c +++ b/fs/super.c @@ -155,11 +155,19 @@ static void destroy_super_rcu(struct rcu_head *head) schedule_work(&s->destroy_work); } -/* Free a superblock that has never been seen by anyone */ +/* + * Free a superblock that has never been seen by anyone. Note that shrinkers + * could have been invoked already but we rely on s_umount to not actually + * touch it. + */ static void destroy_unused_super(struct super_block *s) { if (!s) return; + + if (!list_empty(&s->s_shrink.list)) + unregister_shrinker(&s->s_shrink); + up_write(&s->s_umount); list_lru_destroy(&s->s_dentry_lru); list_lru_destroy(&s->s_inode_lru); @@ -190,6 +198,7 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags, return NULL; INIT_LIST_HEAD(&s->s_mounts); + INIT_LIST_HEAD(&s->s_shrink.list); s->s_user_ns = get_user_ns(user_ns); if (security_sb_alloc(s)) @@ -252,6 +261,8 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags, s->s_shrink.count_objects = super_cache_count; s->s_shrink.batch = 1024; s->s_shrink.flags = SHRINKER_NUMA_AWARE | SHRINKER_MEMCG_AWARE; + if (register_shrinker(&s->s_shrink)) + goto fail; return s; fail: @@ -518,7 +529,6 @@ struct super_block *sget_userns(struct file_system_type *type, hlist_add_head(&s->s_instances, &type->fs_supers); spin_unlock(&sb_lock); get_filesystem(type); - register_shrinker(&s->s_shrink); return s; }