From patchwork Sun Feb 27 09:23:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hyeonggon Yoo <42.hyeyoo@gmail.com> X-Patchwork-Id: 12761562 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55F99C433F5 for ; Sun, 27 Feb 2022 09:23:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 823B78D0002; Sun, 27 Feb 2022 04:23:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7D2C38D0001; Sun, 27 Feb 2022 04:23:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 69B0C8D0002; Sun, 27 Feb 2022 04:23:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.27]) by kanga.kvack.org (Postfix) with ESMTP id 5ADF78D0001 for ; Sun, 27 Feb 2022 04:23:33 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 1651020E2B for ; Sun, 27 Feb 2022 09:23:33 +0000 (UTC) X-FDA: 79188021906.03.BA5EF3A Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf23.hostedemail.com (Postfix) with ESMTP id 83436140004 for ; Sun, 27 Feb 2022 09:23:31 +0000 (UTC) Received: by mail-pf1-f175.google.com with SMTP id a5so7603668pfv.9 for ; Sun, 27 Feb 2022 01:23:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:to:cc:subject:message-id:mime-version:content-disposition :in-reply-to; bh=YKnxtZE2J4v12iS0fkHm9fdiEXM2m35F8Fk6VzqHDJs=; b=gsFQBqioXHjAw6hD6Yd7BIthDv0o6558Jjlxjdj60UHr/TUw5mbmVDNW3Mw4mfLBqJ y18TDRuBREtjEciK7pvIJk+Yml0exrMOuIfFfSGsRBS+kO5NQ3bLZGXbwIRKKX9yezdw vtInvg6o/fRAGW9EWIg42jh/4u0Oh1R+AQwYlFa6K6rTTc5I9KGnXseZqdTm5iLCam4O hhtPsxlQFlhr5MYHGq90OLuZG+6dQtA8oLuXPZps1CdB3WSZIuye4w7Hu27jYq/4Q0X1 ikiQUqdnK6BgGLuP7/XO07U9YE1oM8VlBgzcm6Ou2/fMdpgodE1Z8/1/qvdHX78t5YLe eX8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version :content-disposition:in-reply-to; bh=YKnxtZE2J4v12iS0fkHm9fdiEXM2m35F8Fk6VzqHDJs=; b=iTaIKb8jpwUtLv4b2gDBf4QI3Z4jjcIWDYitXcxxArHQ5gvZh03gCIQ/gKeJPwQ63g 7bplnvqFSScTE7AzfXxF/fX54j05OfihOnP5SRL4Ec2No6O72CWsHLSk1I1d2+PIo0eQ b6kk9wGgpRGXPO5Y63STaaJbutje4rMiU3qvIulm0I7zVNFMKBIlRFcfZAMLabledmKR 4xqQhkNCoHPKJwCQAYVSTlOjiuCvLH+ECOMjs6PPaYwgmvDk0jV7oMMSvcBqiBqTJmP1 iL5EmH3KiWk1lXcyOLb/VPuU1eA/mDpogW0zOEcoQbk2Or+IsbWT2VIDgqVmT3n96XF9 vdIw== X-Gm-Message-State: AOAM531uEkRtqToZ9NUPX4yV2oxsruwxVpkGmWPJf7ZRiPUXN1NbIjo8 ER2xQHQ90PjGmXv0/UItoFg= X-Google-Smtp-Source: ABdhPJzMawut+N0Hibcl7C82p1CHJqf3vz7xVupBicFzhSKFRDvmHb0hc3h1/wky4JSeuPTlAy9Mrw== X-Received: by 2002:a05:6a00:24ca:b0:4e1:cb76:32da with SMTP id d10-20020a056a0024ca00b004e1cb7632damr15794574pfv.81.1645953810461; Sun, 27 Feb 2022 01:23:30 -0800 (PST) Received: from ip-172-31-19-208.ap-northeast-1.compute.internal (ec2-18-181-137-102.ap-northeast-1.compute.amazonaws.com. [18.181.137.102]) by smtp.gmail.com with ESMTPSA id e14-20020a056a001a8e00b004e136d54a15sm9584444pfv.105.2022.02.27.01.23.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Feb 2022 01:23:30 -0800 (PST) Date: Sun, 27 Feb 2022 09:23:22 +0000 From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To: Vlastimil Babka Cc: David Rientjes , Christoph Lameter , Joonsoo Kim , Pekka Enberg , Roman Gushchin , Andrew Morton , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Oliver Glitta , Faiyaz Mohammed , Dmitry Vyukov , Eric Dumazet , Jarkko Sakkinen , Johannes Berg , Yury Norov , Arnd Bergmann , James Bottomley , Matteo Croce , Marco Elver , Andrey Konovalov , Imran Khan , Zqiang Subject: [PATCH v2] lib/stackdepot: Use page allocator if both slab and memblock is unavailable Message-ID: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 83436140004 X-Stat-Signature: i6tdpppxupkdopgtnceysp1fqz7ti6ew Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=gsFQBqio; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf23.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com X-HE-Tag: 1645953811-590513 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: After commit 2dba5eb1c73b ("lib/stackdepot: allow optional init and stack_table allocation by kvmalloc()"), stack_depot_init() is called later if CONFIG_STACKDEPOT_ALWAYS_INIT=n to remove unnecessary memory usage. It allocates stack_table using memblock_alloc() or kvmalloc() depending on availability of slab allocator. But when stack_depot_init() is called while creating boot slab caches, both slab allocator and memblock is not available. So kernel crashes. Allocate stack_table from page allocator when both slab allocator and memblock is unavailable. Limit size of stack_table when using page allocator because vmalloc() is also unavailable in kmem_cache_init(). It must not be larger than (PAGE_SIZE << (MAX_ORDER - 1)). This patch was tested on both CONFIG_STACKDEPOT_ALWAYS_INIT=y and n. [ lkp@intel.com: Fix W=1 build warning ] Fixes: 2dba5eb1c73b ("lib/stackdepot: allow optional init and stack_table allocation by kvmalloc()") Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- lib/stackdepot.c | 28 +++++++++++++++++++++------- 1 file changed, 21 insertions(+), 7 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index bf5ba9af0500..a96f8fd78c42 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -73,6 +73,14 @@ static int next_slab_inited; static size_t depot_offset; static DEFINE_RAW_SPINLOCK(depot_lock); +static size_t stack_hash_size = (1 << CONFIG_STACK_HASH_ORDER); +static inline size_t stack_hash_mask(void) +{ + return stack_hash_size - 1; +} + +#define STACK_HASH_SEED 0x9747b28c + static bool init_stack_slab(void **prealloc) { if (!*prealloc) @@ -142,10 +150,6 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) return stack; } -#define STACK_HASH_SIZE (1L << CONFIG_STACK_HASH_ORDER) -#define STACK_HASH_MASK (STACK_HASH_SIZE - 1) -#define STACK_HASH_SEED 0x9747b28c - static bool stack_depot_disable; static struct stack_record **stack_table; @@ -172,18 +176,28 @@ __ref int stack_depot_init(void) mutex_lock(&stack_depot_init_mutex); if (!stack_depot_disable && !stack_table) { - size_t size = (STACK_HASH_SIZE * sizeof(struct stack_record *)); + size_t size = (stack_hash_size * sizeof(struct stack_record *)); int i; if (slab_is_available()) { pr_info("Stack Depot allocating hash table with kvmalloc\n"); stack_table = kvmalloc(size, GFP_KERNEL); + } else if (totalram_pages() > 0) { + /* Reduce size because vmalloc may be unavailable */ + size = min_t(size_t, size, PAGE_SIZE << (MAX_ORDER - 1)); + stack_hash_size = size / sizeof(struct stack_record *); + + pr_info("Stack Depot allocating hash table with __get_free_pages\n"); + stack_table = (struct stack_record **) + __get_free_pages(GFP_KERNEL, get_order(size)); } else { pr_info("Stack Depot allocating hash table with memblock_alloc\n"); stack_table = memblock_alloc(size, SMP_CACHE_BYTES); } + if (stack_table) { - for (i = 0; i < STACK_HASH_SIZE; i++) + pr_info("Stack Depot hash table size=%zu\n", stack_hash_size); + for (i = 0; i < stack_hash_size; i++) stack_table[i] = NULL; } else { pr_err("Stack Depot hash table allocation failed, disabling\n"); @@ -363,7 +377,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, goto fast_exit; hash = hash_stack(entries, nr_entries); - bucket = &stack_table[hash & STACK_HASH_MASK]; + bucket = &stack_table[hash & stack_hash_mask()]; /* * Fast path: look the stack trace up without locking.