From patchwork Sun Jul 30 15:16:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13333455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CC40C001E0 for ; Sun, 30 Jul 2023 15:16:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A54E1900004; Sun, 30 Jul 2023 11:16:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A2BD3900002; Sun, 30 Jul 2023 11:16:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 87D6E900004; Sun, 30 Jul 2023 11:16:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 77D37900002 for ; Sun, 30 Jul 2023 11:16:17 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 3C70D40310 for ; Sun, 30 Jul 2023 15:16:17 +0000 (UTC) X-FDA: 81068629194.21.33B99E1 Received: from mail-wm1-f49.google.com (mail-wm1-f49.google.com [209.85.128.49]) by imf26.hostedemail.com (Postfix) with ESMTP id 5EC20140007 for ; Sun, 30 Jul 2023 15:16:15 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=TkyhfbVG; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf26.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.128.49 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690730175; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SFM4pR1QacGlm/8xlIUmCJcy4VSy/s+0OM0YFxjuA8k=; b=ywkpbunMXKgo6N5t+6ggMTfsYCJXwYQA/aFJL2Y2FQaD467jT8jGH835/yGCglUdFDWfho 5+ULMU3cMZEs7TKYr8LJIcUXhEUoadJxBE7EtfzqH9wiokNhWQJIsEi9aoamS6VsA94a+9 P59qmOldkeU+0/Eqc7Yeb9teqBDQJZM= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=TkyhfbVG; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf26.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.128.49 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690730175; a=rsa-sha256; cv=none; b=kt33kv9CENl+88yXERN9rAuKWVR4LRqWlW2okmXwE6HaJNjOawGI5QjiggcGbjbdAixfWV zni0KPYKPUBLm5BwwMMxsimwEnnfBbyr5yarF9NhJJEbyvGyDIkiofB2YjidEYfNyQbrqh /tHtdTZnV+HfdM8u+cIBO44kTCmC+R4= Received: by mail-wm1-f49.google.com with SMTP id 5b1f17b1804b1-3fe167d4a18so14344375e9.0 for ; Sun, 30 Jul 2023 08:16:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1690730174; x=1691334974; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SFM4pR1QacGlm/8xlIUmCJcy4VSy/s+0OM0YFxjuA8k=; b=TkyhfbVGe1Y7YHfHksu1wqG6hgQhYRmidCb5+Y61F1QRRs/h47YJFt0AIJh54Q0FKL gByq+10/h43JlVcCEVZjC6aUzw/Omy08uMjBB7CgI/FZQXN3z5WuJzdScj4/HnraGiyu il7QDEWZYGSpogyg92dYwrJa+Eqeb9M65dGnmL7nnCP2F14JH377zSxW/hWjC6XaZnyk O4M74hvreh6XC7+2F9vY3dYi8dCjt4haamygyIK9l6ZMqTa22DtdrGIfwCNlLo2xUbbo yHp+23v36B0XyqElADWdUdmxDLaLZYjFnCSrKJy9muMAQ+U2qOTBbbF4yWjE1y0Ngwfe EPHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690730174; x=1691334974; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SFM4pR1QacGlm/8xlIUmCJcy4VSy/s+0OM0YFxjuA8k=; b=kkjBnHLaC7c9rd1ZDqrNIvbt/HGoDFqR0krB1xvDD4wvz0/aIFVleCyw4i/9em+QXr 7O4FVBuZsXq0VFmwvqFHWQck7I+f1Q/RIhosI5FsSaPkyAPaMchJeJSbXTcIi+S3dcRX SQv861YUDi67/fgCZmT4moKG2exw4vPSVY1ySii3SxS/MisKBXzas84U8v/LsPYCpHeu MzkxzkZN3DmbiJLzK6K7QwgjkqWPNuyHa1V8xmA+qw40Qk5oA8ZfN7+rdasgCBViTt58 dhNTl/7lUFzbiMwlErL+97wVM7inI5OZTadHJYogE2oKgW8fcBzdeHSNgLR984tcw0Kh c4nQ== X-Gm-Message-State: ABy/qLbrxwhq5lCNsUhZg/3YE4PRLQmzpqQGbJNcc3wF6B16DP+BN8WZ hc5oAkMxjOBDsc2ryB9A1eTigx1PSmy40z9BcA0= X-Google-Smtp-Source: APBJJlGH28j3h1b9uEfHTlqYomCBicErb0vATQvL1NGT50ZdEoNVQZEd6E1WZNtnMG3Mxzbn8725YA== X-Received: by 2002:a05:600c:d5:b0:3fb:e206:ca5f with SMTP id u21-20020a05600c00d500b003fbe206ca5fmr5709508wmm.31.1690730173889; Sun, 30 Jul 2023 08:16:13 -0700 (PDT) Received: from localhost.localdomain ([2a02:6b6a:b465:0:eda5:aa63:ce24:dac2]) by smtp.gmail.com with ESMTPSA id f17-20020a7bcc11000000b003fd2d33ea53sm9123027wmh.14.2023.07.30.08.16.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Jul 2023 08:16:13 -0700 (PDT) From: Usama Arif To: linux-mm@kvack.org, muchun.song@linux.dev, mike.kravetz@oracle.com, rppt@kernel.org Cc: linux-kernel@vger.kernel.org, fam.zheng@bytedance.com, liangma@liangbit.com, simon.evans@bytedance.com, punit.agrawal@bytedance.com, Usama Arif Subject: [v2 5/6] mm: move allocation of gigantic hstates to the start of mm_core_init Date: Sun, 30 Jul 2023 16:16:05 +0100 Message-Id: <20230730151606.2871391-6-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230730151606.2871391-1-usama.arif@bytedance.com> References: <20230730151606.2871391-1-usama.arif@bytedance.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: rz45dj5okf1pdj49bhfjrcpscfgetp8i X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 5EC20140007 X-HE-Tag: 1690730175-593719 X-HE-Meta: U2FsdGVkX18EaNSDxKnRgECReaoNN4VLzSKHvFm6QOSyNVyeKAQMA6GrXzJnF4QGqQN6nSrulV82sDA7BilPcIkKTwCVvtYxIDxR0T15Fw+aCod9/5J7KEzXA6WnxfAA3Y9Os3kMQD6XdkcP3P5zd9nVYnBoGQliXFWrFgoEu9cWgHU7M+9ConUNlqhTMDXU67yb6QSlqjtgAGkKfmjN4lNnxvxSK0u4s7N5P6HDLurTVWlSdOxsXW+ValHq4odOImbon2kEGPxivhEIBgdQyjpGmct2ZDkB5qIh9OMr+k/+FqiClapgQmCg1Arw+hT7zfoHCLa3xY2KUNqOYBICNsQYtqcLUvG9td8lI7867JV7twUw/tHFbW6K/kIiXeLoLI+0aphI0OOf50ePfwLdhqKExqCXoDyJ3rBV7wHsEZ4eRP8ep2cMK2ApuyzMSkJARI66eEwlU+5xRfWFAFuFmfMj7e4foOdhiMQZEUeuR2tdhLz7ZpTzhrB/Ukr5chJdXhSYPfh65O+rrsg5Qu0LPlOd7Lki39/dCNUxYZ+M5wOvXgefzUr+vLJrHT1dE1sLB+0UkTDdHNPJFPZ3NTO8e/Psu1Y6y1+HfYqOoWqhny7Q3vZi5Joqhl1o92RV+oxh3mmgUERNMo7aS6ABq+C+8CXZY1gLM0qVShGeUMoKto7SxLM5886IAg4naneTNfRT2TttYID0P255jXW9Sv9QDi7340adalTjXWs1nrX5S9Y+Q1nsP0S7XWTl+BLVsC78Zdx9D29EavMKL3UeolJzAkP1xURYpAmhCc/AL+Zf/Ep9EkKX93H9leG8JFpZR6NKidZ3zNBvdbyCDjV6Qzf9sVKHK1pKX8QdhC7CiXIcvcPvjXgKwsywM9bAu6ueWHXwv62JJ2s5gVkXuuU8VSQ5VOJnzkddDGBoYxAqZFSyof0LDDxpa+9dtgM/6L+s+qwyWX0H/cLO0kJ3isAHIT1 rAoPCo9N fSKtUadQwxgXVmUAoqRrrUBVCA4a+xVCIpT3QfcLw0w4JzxfHWWoc8iQsFi6uh6bOteIGy+fuxGu9mznLEUWIQCmxUoyHDsEEgZnliBYxuNybkq8CrYO8kemz3xS32/LzS9wNrceSVhGD5cbQ6L8PDgPddyA2LqxaRrbvscXq++6a7ubXBVPj0bVk+6DNaIkBKyVRyLBWH2xVOV/VgTcE5HmFkYltqJ4nvNrEnXD05P9ytMMHU5gWCjzA2fKTOmb0ukL8tRgt6KrajPFtKKMUYgPD0AdXQXba7dvd2KGudJm2FGY1HHLcf8cIpIzM3t0qk8326zNZ+yj4L9tB/fXB0p3ksSEiqI4wY+rVd2N5PhhXeQSacGgyklZ5SDnUrBblEpbfLQt3qZgJ4Kx/AlZ+n7ikwoknjCKHlL1WixunqKb08hvJq+5bq+aQSxihyh0nzKJWHfE4M4FEHW/kKhtyMm7bmg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000032, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Whether the initialization of tail struct pages of a hugepage happens or not will become dependent on the commandline parameter hugetlb_free_vmemmap in the future. Hence, hugetlb_hstate_alloc_pages needs to be after command line parameters are parsed and the start of mm_core_init is a good point. Signed-off-by: Usama Arif --- mm/hugetlb.c | 18 ++++++++++-------- mm/internal.h | 9 +++++++++ mm/mm_init.c | 6 ++++++ 3 files changed, 25 insertions(+), 8 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 541c07b6d60f..bf60545496d7 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -4400,14 +4400,6 @@ static int __init hugepages_setup(char *s) } } - /* - * Global state is always initialized later in hugetlb_init. - * But we need to allocate gigantic hstates here early to still - * use the bootmem allocator. - */ - if (hugetlb_max_hstate && hstate_is_gigantic(parsed_hstate)) - hugetlb_hstate_alloc_pages(parsed_hstate); - last_mhp = mhp; return 1; @@ -4419,6 +4411,16 @@ static int __init hugepages_setup(char *s) } __setup("hugepages=", hugepages_setup); +void __init hugetlb_hstate_alloc_gigantic_pages(void) +{ + int i; + + for (i = 0; i < HUGE_MAX_HSTATE; i++) { + if (hstate_is_gigantic(&hstates[i])) + hugetlb_hstate_alloc_pages(&hstates[i]); + } +} + /* * hugepagesz command line processing * A specific huge page size can only be specified once with hugepagesz. diff --git a/mm/internal.h b/mm/internal.h index a7d9e980429a..692bb1136a39 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1102,4 +1102,13 @@ struct vma_prepare { struct vm_area_struct *remove; struct vm_area_struct *remove2; }; + +#ifdef CONFIG_HUGETLBFS +void __init hugetlb_hstate_alloc_gigantic_pages(void); +#else +static inline void __init hugetlb_hstate_alloc_gigantic_pages(void); +{ +} +#endif /* CONFIG_HUGETLBFS */ + #endif /* __MM_INTERNAL_H */ diff --git a/mm/mm_init.c b/mm/mm_init.c index a1963c3322af..f2751ccd7d99 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -26,6 +26,7 @@ #include #include #include +#include #include "internal.h" #include "slab.h" #include "shuffle.h" @@ -2768,6 +2769,11 @@ static void __init mem_init_print_info(void) */ void __init mm_core_init(void) { + /* + * We need to allocate gigantic hstates here early to still use the bootmem + * allocator. Non gigantic hstates are initialized later in hugetlb_init. + */ + hugetlb_hstate_alloc_gigantic_pages(); /* Initializations relying on SMP setup */ build_all_zonelists(NULL); page_alloc_init_cpuhp();