| Message ID | 20190403022858.97584-1-cai@lca.pw (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | slab: store tagged freelist for off-slab slabmgmt | expand |
On Wed, Apr 3, 2019 at 4:29 AM Qian Cai <cai@lca.pw> wrote: > > The commit 51dedad06b5f ("kasan, slab: make freelist stored without > tags") calls kasan_reset_tag() for off-slab slab management object > leading to freelist being stored non-tagged. However, cache_grow_begin() > -> alloc_slabmgmt() -> kmem_cache_alloc_node() which assigns a tag for > the address and stores in the shadow address. As the result, it causes > endless errors below during boot due to drain_freelist() -> > slab_destroy() -> kasan_slab_free() which compares already untagged > freelist against the stored tag in the shadow address. Since off-slab > slab management object freelist is such a special case, so just store it > tagged. Non-off-slab management object freelist is still stored untagged > which has not been assigned a tag and should not cause any other > troubles with this inconsistency. Hi Qian, Could you share the config (or other steps) you used to reproduce this? Thanks! > > BUG: KASAN: double-free or invalid-free in slab_destroy+0x84/0x88 > Pointer tag: [ff], memory tag: [99] > > CPU: 0 PID: 1376 Comm: kworker/0:4 Tainted: G W > 5.1.0-rc3+ #8 > Hardware name: HPE Apollo 70 /C01_APACHE_MB , BIOS > L50_5.13_1.0.6 07/10/2018 > Workqueue: cgroup_destroy css_killed_work_fn > Call trace: > dump_backtrace+0x0/0x450 > show_stack+0x20/0x2c > dump_stack+0xe0/0x16c > print_address_description+0x74/0x2a4 > kasan_report_invalid_free+0x80/0xc0 > __kasan_slab_free+0x204/0x208 > kasan_slab_free+0xc/0x18 > kmem_cache_free+0xe4/0x254 > slab_destroy+0x84/0x88 > drain_freelist+0xd0/0x104 > __kmem_cache_shrink+0x1ac/0x224 > __kmemcg_cache_deactivate+0x1c/0x28 > memcg_deactivate_kmem_caches+0xa0/0xe8 > memcg_offline_kmem+0x8c/0x3d4 > mem_cgroup_css_offline+0x24c/0x290 > css_killed_work_fn+0x154/0x618 > process_one_work+0x9cc/0x183c > worker_thread+0x9b0/0xe38 > kthread+0x374/0x390 > ret_from_fork+0x10/0x18 > > Allocated by task 1625: > __kasan_kmalloc+0x168/0x240 > kasan_slab_alloc+0x18/0x20 > kmem_cache_alloc_node+0x1f8/0x3a0 > cache_grow_begin+0x4fc/0xa24 > cache_alloc_refill+0x2f8/0x3e8 > kmem_cache_alloc+0x1bc/0x3bc > sock_alloc_inode+0x58/0x334 > alloc_inode+0xb8/0x164 > new_inode_pseudo+0x20/0xec > sock_alloc+0x74/0x284 > __sock_create+0xb0/0x58c > sock_create+0x98/0xb8 > __sys_socket+0x60/0x138 > __arm64_sys_socket+0xa4/0x110 > el0_svc_handler+0x2c0/0x47c > el0_svc+0x8/0xc > > Freed by task 1625: > __kasan_slab_free+0x114/0x208 > kasan_slab_free+0xc/0x18 > kfree+0x1a8/0x1e0 > single_release+0x7c/0x9c > close_pdeo+0x13c/0x43c > proc_reg_release+0xec/0x108 > __fput+0x2f8/0x784 > ____fput+0x1c/0x28 > task_work_run+0xc0/0x1b0 > do_notify_resume+0xb44/0x1278 > work_pending+0x8/0x10 > > The buggy address belongs to the object at ffff809681b89e00 > which belongs to the cache kmalloc-128 of size 128 > The buggy address is located 0 bytes inside of > 128-byte region [ffff809681b89e00, ffff809681b89e80) > The buggy address belongs to the page: > page:ffff7fe025a06e00 count:1 mapcount:0 mapping:01ff80082000fb00 > index:0xffff809681b8fe04 > flags: 0x17ffffffc000200(slab) > raw: 017ffffffc000200 ffff7fe025a06d08 ffff7fe022ef7b88 01ff80082000fb00 > raw: ffff809681b8fe04 ffff809681b80000 00000001000000e0 0000000000000000 > page dumped because: kasan: bad access detected > page allocated via order 0, migratetype Unmovable, gfp_mask > 0x2420c0(__GFP_IO|__GFP_FS|__GFP_NOWARN|__GFP_COMP|__GFP_THISNODE) > prep_new_page+0x4e0/0x5e0 > get_page_from_freelist+0x4ce8/0x50d4 > __alloc_pages_nodemask+0x738/0x38b8 > cache_grow_begin+0xd8/0xa24 > ____cache_alloc_node+0x14c/0x268 > __kmalloc+0x1c8/0x3fc > ftrace_free_mem+0x408/0x1284 > ftrace_free_init_mem+0x20/0x28 > kernel_init+0x24/0x548 > ret_from_fork+0x10/0x18 > > Memory state around the buggy address: > ffff809681b89c00: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe > ffff809681b89d00: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe > >ffff809681b89e00: 99 99 99 99 99 99 99 99 fe fe fe fe fe fe fe fe > ^ > ffff809681b89f00: 43 43 43 43 43 fe fe fe fe fe fe fe fe fe fe fe > ffff809681b8a000: 6d fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe > > Fixes: 51dedad06b5f ("kasan, slab: make freelist stored without tags") > Signed-off-by: Qian Cai <cai@lca.pw> > --- > mm/slab.c | 1 - > 1 file changed, 1 deletion(-) > > diff --git a/mm/slab.c b/mm/slab.c > index 329bfe67f2ca..46a6e084222b 100644 > --- a/mm/slab.c > +++ b/mm/slab.c > @@ -2374,7 +2374,6 @@ static void *alloc_slabmgmt(struct kmem_cache *cachep, > /* Slab management obj is off-slab. */ > freelist = kmem_cache_alloc_node(cachep->freelist_cache, > local_flags, nodeid); > - freelist = kasan_reset_tag(freelist); > if (!freelist) > return NULL; > } else { > -- > 2.17.2 (Apple Git-113) >
On Wed, 2019-04-03 at 13:23 +0200, Andrey Konovalov wrote: > On Wed, Apr 3, 2019 at 4:29 AM Qian Cai <cai@lca.pw> wrote: > > > > The commit 51dedad06b5f ("kasan, slab: make freelist stored without > > tags") calls kasan_reset_tag() for off-slab slab management object > > leading to freelist being stored non-tagged. However, cache_grow_begin() > > -> alloc_slabmgmt() -> kmem_cache_alloc_node() which assigns a tag for > > the address and stores in the shadow address. As the result, it causes > > endless errors below during boot due to drain_freelist() -> > > slab_destroy() -> kasan_slab_free() which compares already untagged > > freelist against the stored tag in the shadow address. Since off-slab > > slab management object freelist is such a special case, so just store it > > tagged. Non-off-slab management object freelist is still stored untagged > > which has not been assigned a tag and should not cause any other > > troubles with this inconsistency. > > Hi Qian, > > Could you share the config (or other steps) you used to reproduce this? https://git.sr.ht/~cai/linux-debug/blob/master/config Additional command-line option to boot: page_poison=on crashkernel=768M earlycon page_owner=on numa_balancing=enable systemd.unified_cgroup_hierarchy=1 debug_guardpage_minorder=1
On Wed, Apr 3, 2019 at 3:07 PM Qian Cai <cai@lca.pw> wrote: > > On Wed, 2019-04-03 at 13:23 +0200, Andrey Konovalov wrote: > > On Wed, Apr 3, 2019 at 4:29 AM Qian Cai <cai@lca.pw> wrote: > > > > > > The commit 51dedad06b5f ("kasan, slab: make freelist stored without > > > tags") calls kasan_reset_tag() for off-slab slab management object > > > leading to freelist being stored non-tagged. However, cache_grow_begin() > > > -> alloc_slabmgmt() -> kmem_cache_alloc_node() which assigns a tag for > > > the address and stores in the shadow address. As the result, it causes > > > endless errors below during boot due to drain_freelist() -> > > > slab_destroy() -> kasan_slab_free() which compares already untagged > > > freelist against the stored tag in the shadow address. Since off-slab > > > slab management object freelist is such a special case, so just store it > > > tagged. Non-off-slab management object freelist is still stored untagged > > > which has not been assigned a tag and should not cause any other > > > troubles with this inconsistency. > > > > Hi Qian, > > > > Could you share the config (or other steps) you used to reproduce this? > > https://git.sr.ht/~cai/linux-debug/blob/master/config > > Additional command-line option to boot: > > page_poison=on crashkernel=768M earlycon page_owner=on numa_balancing=enable > systemd.unified_cgroup_hierarchy=1 debug_guardpage_minorder=1 Reproduced, thanks! As far as my understanding of how SLAB works goes, this change looks good to me. Reviewed-by: Andrey Konovalov <andreyknvl@google.com> Thanks, Qian! > > -- > You received this message because you are subscribed to the Google Groups "kasan-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com. > To post to this group, send email to kasan-dev@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/1554296870.26196.32.camel%40lca.pw. > For more options, visit https://groups.google.com/d/optout.
diff --git a/mm/slab.c b/mm/slab.c index 329bfe67f2ca..46a6e084222b 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -2374,7 +2374,6 @@ static void *alloc_slabmgmt(struct kmem_cache *cachep, /* Slab management obj is off-slab. */ freelist = kmem_cache_alloc_node(cachep->freelist_cache, local_flags, nodeid); - freelist = kasan_reset_tag(freelist); if (!freelist) return NULL; } else {
The commit 51dedad06b5f ("kasan, slab: make freelist stored without tags") calls kasan_reset_tag() for off-slab slab management object leading to freelist being stored non-tagged. However, cache_grow_begin() -> alloc_slabmgmt() -> kmem_cache_alloc_node() which assigns a tag for the address and stores in the shadow address. As the result, it causes endless errors below during boot due to drain_freelist() -> slab_destroy() -> kasan_slab_free() which compares already untagged freelist against the stored tag in the shadow address. Since off-slab slab management object freelist is such a special case, so just store it tagged. Non-off-slab management object freelist is still stored untagged which has not been assigned a tag and should not cause any other troubles with this inconsistency. BUG: KASAN: double-free or invalid-free in slab_destroy+0x84/0x88 Pointer tag: [ff], memory tag: [99] CPU: 0 PID: 1376 Comm: kworker/0:4 Tainted: G W 5.1.0-rc3+ #8 Hardware name: HPE Apollo 70 /C01_APACHE_MB , BIOS L50_5.13_1.0.6 07/10/2018 Workqueue: cgroup_destroy css_killed_work_fn Call trace: dump_backtrace+0x0/0x450 show_stack+0x20/0x2c dump_stack+0xe0/0x16c print_address_description+0x74/0x2a4 kasan_report_invalid_free+0x80/0xc0 __kasan_slab_free+0x204/0x208 kasan_slab_free+0xc/0x18 kmem_cache_free+0xe4/0x254 slab_destroy+0x84/0x88 drain_freelist+0xd0/0x104 __kmem_cache_shrink+0x1ac/0x224 __kmemcg_cache_deactivate+0x1c/0x28 memcg_deactivate_kmem_caches+0xa0/0xe8 memcg_offline_kmem+0x8c/0x3d4 mem_cgroup_css_offline+0x24c/0x290 css_killed_work_fn+0x154/0x618 process_one_work+0x9cc/0x183c worker_thread+0x9b0/0xe38 kthread+0x374/0x390 ret_from_fork+0x10/0x18 Allocated by task 1625: __kasan_kmalloc+0x168/0x240 kasan_slab_alloc+0x18/0x20 kmem_cache_alloc_node+0x1f8/0x3a0 cache_grow_begin+0x4fc/0xa24 cache_alloc_refill+0x2f8/0x3e8 kmem_cache_alloc+0x1bc/0x3bc sock_alloc_inode+0x58/0x334 alloc_inode+0xb8/0x164 new_inode_pseudo+0x20/0xec sock_alloc+0x74/0x284 __sock_create+0xb0/0x58c sock_create+0x98/0xb8 __sys_socket+0x60/0x138 __arm64_sys_socket+0xa4/0x110 el0_svc_handler+0x2c0/0x47c el0_svc+0x8/0xc Freed by task 1625: __kasan_slab_free+0x114/0x208 kasan_slab_free+0xc/0x18 kfree+0x1a8/0x1e0 single_release+0x7c/0x9c close_pdeo+0x13c/0x43c proc_reg_release+0xec/0x108 __fput+0x2f8/0x784 ____fput+0x1c/0x28 task_work_run+0xc0/0x1b0 do_notify_resume+0xb44/0x1278 work_pending+0x8/0x10 The buggy address belongs to the object at ffff809681b89e00 which belongs to the cache kmalloc-128 of size 128 The buggy address is located 0 bytes inside of 128-byte region [ffff809681b89e00, ffff809681b89e80) The buggy address belongs to the page: page:ffff7fe025a06e00 count:1 mapcount:0 mapping:01ff80082000fb00 index:0xffff809681b8fe04 flags: 0x17ffffffc000200(slab) raw: 017ffffffc000200 ffff7fe025a06d08 ffff7fe022ef7b88 01ff80082000fb00 raw: ffff809681b8fe04 ffff809681b80000 00000001000000e0 0000000000000000 page dumped because: kasan: bad access detected page allocated via order 0, migratetype Unmovable, gfp_mask 0x2420c0(__GFP_IO|__GFP_FS|__GFP_NOWARN|__GFP_COMP|__GFP_THISNODE) prep_new_page+0x4e0/0x5e0 get_page_from_freelist+0x4ce8/0x50d4 __alloc_pages_nodemask+0x738/0x38b8 cache_grow_begin+0xd8/0xa24 ____cache_alloc_node+0x14c/0x268 __kmalloc+0x1c8/0x3fc ftrace_free_mem+0x408/0x1284 ftrace_free_init_mem+0x20/0x28 kernel_init+0x24/0x548 ret_from_fork+0x10/0x18 Memory state around the buggy address: ffff809681b89c00: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe ffff809681b89d00: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe >ffff809681b89e00: 99 99 99 99 99 99 99 99 fe fe fe fe fe fe fe fe ^ ffff809681b89f00: 43 43 43 43 43 fe fe fe fe fe fe fe fe fe fe fe ffff809681b8a000: 6d fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe Fixes: 51dedad06b5f ("kasan, slab: make freelist stored without tags") Signed-off-by: Qian Cai <cai@lca.pw> --- mm/slab.c | 1 - 1 file changed, 1 deletion(-)