From patchwork Mon Feb 28 15:09:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hyeonggon Yoo <42.hyeyoo@gmail.com> X-Patchwork-Id: 12763471 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A4D0C433EF for ; Mon, 28 Feb 2022 15:10:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 130428D0002; Mon, 28 Feb 2022 10:10:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0DEFA8D0001; Mon, 28 Feb 2022 10:10:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F0F378D0002; Mon, 28 Feb 2022 10:10:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E2D3B8D0001 for ; Mon, 28 Feb 2022 10:10:04 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 9690F96F34 for ; Mon, 28 Feb 2022 15:10:04 +0000 (UTC) X-FDA: 79192523928.29.F2407EA Received: from mail-pg1-f180.google.com (mail-pg1-f180.google.com [209.85.215.180]) by imf07.hostedemail.com (Postfix) with ESMTP id 0537E4000A for ; Mon, 28 Feb 2022 15:10:03 +0000 (UTC) Received: by mail-pg1-f180.google.com with SMTP id o26so11044285pgb.8 for ; Mon, 28 Feb 2022 07:10:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:to:cc:subject:message-id:mime-version:content-disposition :in-reply-to; bh=/IzYX9ekXXf7onTNDRhy0rzD0OI0XgiFY+K3R+MvA9A=; b=ZGIn/OXsY7QB0Sykq85jAfBBii7x+DAFUHBo7NHAudLxTsc/Unb8bKEFWoqST8WBpW bw94JTVLzK0mn8ztcZhov9MXKHyGOOwbI70uHhcBZOtpoU5oafVBEP+KOeLCU0ev9dGu QcEZM2EuIVFq9qVSGgzdj9jnqTBiv2dVZqWPReaWwHdGy1gdqmDn7oMTQOTUBYMrm4kH 7eRTEmq2C6bq8/xae6yoaqQa45g73ts/DgJrKuxRIfD66i1hf1Xxzat3o81X9wY3erRa 4cvJvMcJZLXK+ITjHuzNkMEZ9UtcL1Vq1LUMOxaiL6dvIxfvD0rgcprLUrOA57gpF60H /Xnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version :content-disposition:in-reply-to; bh=/IzYX9ekXXf7onTNDRhy0rzD0OI0XgiFY+K3R+MvA9A=; b=lQNkKJ66k2Qz0pX7uBwMEwhOLLeegecs36lQXbpLVy5N8dj9Xwhy4MY2j1SWiETpOc 2NfbQsBThfDlLQWnbP3wkdkq9O+kmO3MIaz675gkZWJd5ULNdw+0bLmRq6jmVcKm1wD3 nVaSXkirS+kbmbKUgVcXMpAIpzNvTANqjwCA507yjoiDVMX+i5SctDJpumh9CqMshmqG Glv4vjXiZ0cvFGOzZv3mQAgOZbVeElEdbq6Dt533ziueeOBS9fLoZPbIaX2nDVc21SMj LbkmcjoVxrNZ8wZyQ55PVJ8IBAAaYodrnzDMNH2lBrChjBBofrG3+mOQusNExwsLKGk6 xdkg== X-Gm-Message-State: AOAM530yfrvWTb6vD9OJjeA5gaQv3KF3CeFQW/KdnoB31eGeqBMSjlYL UnIcrbNByrH0aM2J0vCKg/0= X-Google-Smtp-Source: ABdhPJwOXeWsUUvwtVxFSOTG/liBvry3WbgiNRSaXCk4O/6AoD9vAhicr3tugHtb0p7pOx8qnDjRWQ== X-Received: by 2002:a63:d904:0:b0:372:9faf:5467 with SMTP id r4-20020a63d904000000b003729faf5467mr17491373pgg.196.1646061002852; Mon, 28 Feb 2022 07:10:02 -0800 (PST) Received: from ip-172-31-19-208.ap-northeast-1.compute.internal (ec2-18-181-137-102.ap-northeast-1.compute.amazonaws.com. [18.181.137.102]) by smtp.gmail.com with ESMTPSA id q8-20020a056a00088800b004bca31c8e56sm14786206pfj.115.2022.02.28.07.09.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Feb 2022 07:10:02 -0800 (PST) Date: Mon, 28 Feb 2022 15:09:55 +0000 From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To: Marco Elver Cc: Vlastimil Babka , David Rientjes , Christoph Lameter , Joonsoo Kim , Pekka Enberg , Roman Gushchin , Andrew Morton , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Oliver Glitta , Faiyaz Mohammed , Dmitry Vyukov , Eric Dumazet , Jarkko Sakkinen , Johannes Berg , Yury Norov , Arnd Bergmann , James Bottomley , Matteo Croce , Andrey Konovalov , Imran Khan , Zqiang Subject: [PATCH] mm/slub: initialize stack depot in boot process Message-ID: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 0537E4000A X-Stat-Signature: n144t17umfbiaaemaebxjmf5jeb94zts Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="ZGIn/OXs"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf07.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.215.180 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com X-HE-Tag: 1646061003-56802 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: commit ba10d4b46655 ("mm/slub: use stackdepot to save stack trace in objects") initializes stack depot while creating cache if SLAB_STORE_USER flag is set. This can make kernel crash because a cache can be created in various contexts. For example if user sets slub_debug=U, kernel crashes because create_boot_cache() calls stack_depot_init(), which tries to allocate hash table using memblock_alloc() if slab is not available. But memblock is also not available at that time. This patch solves the problem by initializing stack depot early in boot process if SLAB_STORE_USER debug flag is set globally or the flag is set to at least one cache. [ elver@google.com: initialize stack depot depending on slub_debug parameter instead of allowing stack_depot_init() can be called in kmem_cache_init() for simplicity. ] Link: https://lkml.org/lkml/2022/2/28/238 Fixes: ba10d4b46655 ("mm/slub: use stackdepot to save stack trace in objects") Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- include/linux/slab.h | 1 + init/main.c | 1 + mm/slab.c | 4 ++++ mm/slob.c | 4 ++++ mm/slub.c | 28 +++++++++++++++++++++++++--- 5 files changed, 35 insertions(+), 3 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 37bde99b74af..023f3f71ae35 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -139,6 +139,7 @@ struct mem_cgroup; /* * struct kmem_cache related prototypes */ +void __init kmem_cache_init_early(void); void __init kmem_cache_init(void); bool slab_is_available(void); diff --git a/init/main.c b/init/main.c index 65fa2e41a9c0..4fdb7975a085 100644 --- a/init/main.c +++ b/init/main.c @@ -835,6 +835,7 @@ static void __init mm_init(void) kfence_alloc_pool(); report_meminit(); stack_depot_early_init(); + kmem_cache_init_early(); mem_init(); mem_init_print_info(); kmem_cache_init(); diff --git a/mm/slab.c b/mm/slab.c index ddf5737c63d9..80a6d01aab06 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -1196,6 +1196,10 @@ static void __init set_up_node(struct kmem_cache *cachep, int index) } } +void __init kmem_cache_init_early(void) +{ +} + /* * Initialisation. Called after the page allocator have been initialised and * before smp_init(). diff --git a/mm/slob.c b/mm/slob.c index 60c5842215f1..00e323af8be4 100644 --- a/mm/slob.c +++ b/mm/slob.c @@ -715,6 +715,10 @@ struct kmem_cache kmem_cache_boot = { .align = ARCH_KMALLOC_MINALIGN, }; +void __init kmem_cache_init_early(void) +{ +} + void __init kmem_cache_init(void) { kmem_cache = &kmem_cache_boot; diff --git a/mm/slub.c b/mm/slub.c index a74afe59a403..40bcd18143b6 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4221,9 +4221,6 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags) s->remote_node_defrag_ratio = 1000; #endif - if (s->flags & SLAB_STORE_USER && IS_ENABLED(CONFIG_STACKDEPOT)) - stack_depot_init(); - /* Initialize the pre-computed randomized freelist if slab is up */ if (slab_state >= UP) { if (init_cache_random_seq(s)) @@ -4810,6 +4807,31 @@ static struct kmem_cache * __init bootstrap(struct kmem_cache *static_cache) return s; } +/* Initialize stack depot if needed */ +void __init kmem_cache_init_early(void) +{ +#ifdef CONFIG_STACKDEPOT + slab_flags_t block_flags; + char *next_block; + char *slab_list; + + if (slub_debug & SLAB_STORE_USER) + goto init_stack_depot; + + next_block = slub_debug_string; + while (next_block) { + next_block = parse_slub_debug_flags(next_block, &block_flags, &slab_list, false); + if (block_flags & SLAB_STORE_USER) + goto init_stack_depot; + } + + return; + +init_stack_depot: + stack_depot_init(); +#endif +} + void __init kmem_cache_init(void) { static __initdata struct kmem_cache boot_kmem_cache,