From patchwork Wed Mar 2 08:37:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12765617 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 067B2C433EF for ; Wed, 2 Mar 2022 08:38:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7D27A8D0005; Wed, 2 Mar 2022 03:38:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7826C8D0001; Wed, 2 Mar 2022 03:38:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 671998D0005; Wed, 2 Mar 2022 03:38:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0199.hostedemail.com [216.40.44.199]) by kanga.kvack.org (Postfix) with ESMTP id 593AA8D0001 for ; Wed, 2 Mar 2022 03:38:21 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 0F6289F325 for ; Wed, 2 Mar 2022 08:38:21 +0000 (UTC) X-FDA: 79198794402.19.5AFD0DB Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) by imf31.hostedemail.com (Postfix) with ESMTP id 98FA320005 for ; Wed, 2 Mar 2022 08:38:20 +0000 (UTC) Received: by mail-pj1-f41.google.com with SMTP id mg21-20020a17090b371500b001bef9e4657cso810724pjb.0 for ; Wed, 02 Mar 2022 00:38:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oqE92Z30jLRB5qj639p7ReXjqJAONBgi3Xr8rqugwWc=; b=G5CCJorcD+mvdq6Rx3y5zrdO9Uswo4f2HKicBJt0MzQd5dDmOGdcpF5Q0T5Sb+0dMe 5Gh1k3gXFNEWbH8soeJvSqYskqaD8UIrfXhIbIZBJUvCoyOAUIlV90YNP9qy8+iB/ocD Xb6khWfXY6hJSuTWWhVIJQ5JO67wyCMgGKs/udM/CHgixLogCFxvXwpqr0oet2xKig0W 0HSsd5g99k3fdPHKiaprYso/2/8m4zlPpcL+YcAUzQ2mbDElyCAN1eccs9UdJZqorU/p mNdMO7mjcCYhtJPqtnIo8RbIlPjV8CS2QQbKSEsPE6xsiveggCAQsm8VeiNcWAjZ85av U3cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oqE92Z30jLRB5qj639p7ReXjqJAONBgi3Xr8rqugwWc=; b=7osUpUtST0h940UHok0ey69F0qtk7V1JoL0CqxDTuDLxdLm2qvyndnfMTNLFtAi93J XBhLXMNqhdSBrffRXD9RsC9E1qv2r1woRjLqCsp7IxwM1HV/GurPtpevZEGwZcAOozvm OdsfEFKLAzawG8WWwOobJd00a1CF+G6BP91oV90dsw8zJ7bA2vcBlZOU3EZSpK8mdG5h XnY2vvyrx2bLzQCMSibpd497F5uuycj6wRGIXPzlZaOOjYMQkK58wXa815N/6MG7+z66 bGuB7TsO8xSxHjfMDSYL2RTH8uZFaC1smc8N5Cb9zm8csdlBHfYbkKU2aYwGmjrlgSf6 XR9A== X-Gm-Message-State: AOAM533J8vRSU/CYMBorNHDhQ/HJzWsmVwpEIYJ6hx8UdjYzoXlgNvc5 Oa2YAdqtDJfsAf2ZLlm//obEJQ== X-Google-Smtp-Source: ABdhPJxo987uh2sy4iVBcpuO3RBDBmmhqeJ+aQAyhMLaM+8AbMNcaI4YlJ2ED9/26vh1UTb91s0bdw== X-Received: by 2002:a17:90b:4b90:b0:1be:fd89:5402 with SMTP id lr16-20020a17090b4b9000b001befd895402mr769942pjb.117.1646210299670; Wed, 02 Mar 2022 00:38:19 -0800 (PST) Received: from FVFYT0MHHV2J.bytedance.net ([61.120.150.70]) by smtp.gmail.com with ESMTPSA id 17-20020a056a00071100b004f0f941d1e8sm18705141pfl.24.2022.03.02.00.38.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Mar 2022 00:38:19 -0800 (PST) From: Muchun Song To: corbet@lwn.net, mike.kravetz@oracle.com, akpm@linux-foundation.org, mcgrof@kernel.org, keescook@chromium.org, yzaikin@google.com Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, smuchun@gmail.com, Muchun Song Subject: [PATCH v2 1/3] mm: hugetlb: disable freeing vmemmap pages when struct page crosses page boundaries Date: Wed, 2 Mar 2022 16:37:56 +0800 Message-Id: <20220302083758.32528-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220302083758.32528-1-songmuchun@bytedance.com> References: <20220302083758.32528-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 98FA320005 X-Rspam-User: Authentication-Results: imf31.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=G5CCJorc; spf=pass (imf31.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.216.41 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Stat-Signature: 5xu69mzy7a1x4snxx3btphquy4wj53oh X-HE-Tag: 1646210300-286912 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON is enabled and the size of "struct page" is not power of two, we cannot optimize vmemmap pages of HugeTLB pages. We should disable this feature in this case. Signed-off-by: Muchun Song --- mm/hugetlb_vmemmap.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index b3118dba0518..836d1117f08b 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -121,6 +121,17 @@ void __init hugetlb_vmemmap_init(struct hstate *h) if (!hugetlb_free_vmemmap_enabled()) return; + if (IS_ENABLED(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON) && + !is_power_of_2(sizeof(struct page))) { + /* + * The hugetlb_free_vmemmap_enabled_key can be enabled when + * CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON. It should + * be disabled if "struct page" crosses page boundaries. + */ + static_branch_disable(&hugetlb_free_vmemmap_enabled_key); + return; + } + vmemmap_pages = (nr_pages * sizeof(struct page)) >> PAGE_SHIFT; /* * The head page is not to be freed to buddy allocator, the other tail From patchwork Wed Mar 2 08:37:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12765618 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32341C433EF for ; Wed, 2 Mar 2022 08:38:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C18CC8D0008; Wed, 2 Mar 2022 03:38:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BC76C8D0007; Wed, 2 Mar 2022 03:38:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB7128D0008; Wed, 2 Mar 2022 03:38:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0109.hostedemail.com [216.40.44.109]) by kanga.kvack.org (Postfix) with ESMTP id 9D5A18D0007 for ; Wed, 2 Mar 2022 03:38:27 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5B10B180927DB for ; Wed, 2 Mar 2022 08:38:27 +0000 (UTC) X-FDA: 79198794654.23.884A6EC Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by imf25.hostedemail.com (Postfix) with ESMTP id 3CCEBA0002 for ; Wed, 2 Mar 2022 08:38:28 +0000 (UTC) Received: by mail-pf1-f172.google.com with SMTP id u16so1265128pfg.12 for ; Wed, 02 Mar 2022 00:38:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6H6PLOlODlELuUrXrCLYoGeXhxxcGUB9kEy/fShHjn4=; b=kGOrPFJKIY2zeFcU6Bd2P7TZK+xqFiHSSY0aFvJaKv9cIpsG2U7WfOIWyPfgPKW0t1 7TG3A/OoFGobN9N1LfM1tkJ22xS2FqOVBkUry4O75X/IcXg6ZBFwiB1b4YG0RO2+jbkO SxB44nqufytorpbdgh3WgaY72Q/fMd3wXEsT5juGljg+xBvYwxny4kcaK+N50Ab60Y6t 8pOFo9uTJnX3LVpPKLbU33wZj3DhXV5+AaUAgbSZB7KWNjalg+E4EAMxk/wU/aQuE890 uoJRknbjxvl645OSqL8L8xPvmnsnn3QHd4eQ7mg83FiSwyN6zB5+5Sw+NNDrID0YVfCk zeLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6H6PLOlODlELuUrXrCLYoGeXhxxcGUB9kEy/fShHjn4=; b=S57OSoC/6Qm/7FzgI48jxnaIyn2Ap/zWe4yzQsTVOiZHqqI0GEQ6Tpz+j/Aq/brjOs 7d0jLqY+fJyUPqJndcDtA/NfiARsTYmaUhCmDMpGf23h6sgMAB6PZ9vPpRXbJrfCegXE rpC8nD+rWP1040yIkbmX2vceGGuDDcPbaGdrB/8uc8G9y/SqHOVkxBaXxJQ95QWn0xd+ I22Sw9ShEkG1rE0S9zMz7KS2eG+0gz5xKQt75BtSryiDLC9qBLn+MAbqvjN2w/qIRyKs ixlrbQ4TrAJiGnbTtPRuu59UfLjlyOtKpjgRCsxunGOpsQEwWMTtU0XRj2MZKqP9f5Dv r4mw== X-Gm-Message-State: AOAM532ihVCpzXxtZiMq//RfD2PLSD5roibsGvbDrgwoTx7RyHc3SXpn VfbfGUo/S0KzL1CrfZdLfg6ZIA== X-Google-Smtp-Source: ABdhPJzgdFgcCMX/hRjpvLUm5sqK8u+KnVSknTrF5zgAinrtJQ3utZbE65PdsaG5Q1sXGQgiS85mdg== X-Received: by 2002:a63:8ac2:0:b0:341:1e45:26c with SMTP id y185-20020a638ac2000000b003411e45026cmr25687808pgd.31.1646210306168; Wed, 02 Mar 2022 00:38:26 -0800 (PST) Received: from FVFYT0MHHV2J.bytedance.net ([61.120.150.70]) by smtp.gmail.com with ESMTPSA id 17-20020a056a00071100b004f0f941d1e8sm18705141pfl.24.2022.03.02.00.38.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Mar 2022 00:38:25 -0800 (PST) From: Muchun Song To: corbet@lwn.net, mike.kravetz@oracle.com, akpm@linux-foundation.org, mcgrof@kernel.org, keescook@chromium.org, yzaikin@google.com Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, smuchun@gmail.com, Muchun Song Subject: [PATCH v2 2/3] sysctl: allow to set extra1 to SYSCTL_ONE Date: Wed, 2 Mar 2022 16:37:57 +0800 Message-Id: <20220302083758.32528-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220302083758.32528-1-songmuchun@bytedance.com> References: <20220302083758.32528-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Rspam-User: X-Stat-Signature: e85wuxgqs9pbzd88ni6xx1ebrzp7uqcm Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=kGOrPFJK; spf=pass (imf25.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Rspamd-Queue-Id: 3CCEBA0002 X-HE-Tag: 1646210308-659337 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Some sysctls only allow to be enabled and cannot be set back to be disabled. But proc_do_static_key() does not consider this situation, which set ->extra1 to SYSCTL_ZERO unconditionally. This patch add the ability to set ->extra1 to SYSCTL_ONE, which will be used in the next patch. Signed-off-by: Muchun Song --- kernel/sysctl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sysctl.c b/kernel/sysctl.c index 64065abf361e..ab3e9c937268 100644 --- a/kernel/sysctl.c +++ b/kernel/sysctl.c @@ -1631,7 +1631,7 @@ int proc_do_static_key(struct ctl_table *table, int write, .data = &val, .maxlen = sizeof(val), .mode = table->mode, - .extra1 = SYSCTL_ZERO, + .extra1 = table->extra1 == SYSCTL_ONE ? SYSCTL_ONE : SYSCTL_ZERO, .extra2 = SYSCTL_ONE, }; From patchwork Wed Mar 2 08:37:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12765619 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3932C433FE for ; Wed, 2 Mar 2022 08:38:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E7C68D000B; Wed, 2 Mar 2022 03:38:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 697A48D0009; Wed, 2 Mar 2022 03:38:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 560638D000B; Wed, 2 Mar 2022 03:38:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0175.hostedemail.com [216.40.44.175]) by kanga.kvack.org (Postfix) with ESMTP id 452D98D0009 for ; Wed, 2 Mar 2022 03:38:34 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 02D5E180927C3 for ; Wed, 2 Mar 2022 08:38:34 +0000 (UTC) X-FDA: 79198794906.21.F6BA421 Received: from mail-pg1-f178.google.com (mail-pg1-f178.google.com [209.85.215.178]) by imf10.hostedemail.com (Postfix) with ESMTP id 7B30FC0005 for ; Wed, 2 Mar 2022 08:38:33 +0000 (UTC) Received: by mail-pg1-f178.google.com with SMTP id o26so1036284pgb.8 for ; Wed, 02 Mar 2022 00:38:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FsguA/yBosoWVXOBb/NOWtMqUfetLJMnNLuw820odF8=; b=aoo/dmpx/Y3ySAakcLWA1JW/gsNtfCZsFCp7C9m9iPc/FMnmQ5SHJYSFN4vnU/+xhl GX8jo5gIoywnNFv8qbAzJPx4v1w9ge2Fv0/hAigO6k5OVdT0qutEtXtRk4K9EdHEMSRi 6Isz27oMi4I6u/onl+n345h1XkIFKARGlkXjzRyvyPTTxs7pyyBo8f+YCebRUgiQjOeG zYX/2zCZgHwcgM31YNTvSXCEcemDhgeygmYYaFSjQOsLechWkyWmNINL6YxeXJI/LDLO QxJYX7xQjChx8lf9G5HruPNL+o2GhaI3EmXgiev/8nO4SaNF57632fNeYaq0baUoD/Pf lhXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FsguA/yBosoWVXOBb/NOWtMqUfetLJMnNLuw820odF8=; b=AT2WXjMF4i/jQLM4eVpzs+2QHvRVtxQ952hbRyjfGsQnvWiQPf4SUlNiuabtqumYxV CzVzE6mgnn3Zp4c+xk1qfjGHoHjAY66/H0fytGH1Ez7ps4kka2lAWvwWVYyXGannq0zn 5wvDi1r2Qh4nyd+jcgsY7herfquyr6Cjl/0fxtw+1koXgpW0jT2oT2XyRBwLD3LdDV0K stXcl2TpwZRiulf633T6mv4O94m1DEtoB8BEaDTb0e4o17RbL6Gb2RQpBObcpw5E90ty Ka4n3ERtYCbzoHFH7VUDM6x3LeB8BuDjW390HWHzm8tOtoBoNrWHCjat++trMnkgpyoC btMg== X-Gm-Message-State: AOAM532rNCq4CW5WRhHtyMuyVU8vcO66bcoNWCyWFrhyKAyFyoAksmwa UBN4lQwRfQHaXF79AfAInbXg3luhDMZ9xR64wfg= X-Google-Smtp-Source: ABdhPJzzyoMiaWMi1HQFsB5f52pvjW1fFVtWj8+EB7226uejAhqYNfoWYUPN+0gKyHoYqQxixEsFqg== X-Received: by 2002:a63:134a:0:b0:378:9ef8:950a with SMTP id 10-20020a63134a000000b003789ef8950amr12445378pgt.548.1646210312444; Wed, 02 Mar 2022 00:38:32 -0800 (PST) Received: from FVFYT0MHHV2J.bytedance.net ([61.120.150.70]) by smtp.gmail.com with ESMTPSA id 17-20020a056a00071100b004f0f941d1e8sm18705141pfl.24.2022.03.02.00.38.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Mar 2022 00:38:32 -0800 (PST) From: Muchun Song To: corbet@lwn.net, mike.kravetz@oracle.com, akpm@linux-foundation.org, mcgrof@kernel.org, keescook@chromium.org, yzaikin@google.com Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, smuchun@gmail.com, Muchun Song Subject: [PATCH v2 3/3] mm: hugetlb: add hugetlb_free_vmemmap sysctl Date: Wed, 2 Mar 2022 16:37:58 +0800 Message-Id: <20220302083758.32528-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220302083758.32528-1-songmuchun@bytedance.com> References: <20220302083758.32528-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 7B30FC0005 X-Stat-Signature: iezir67ikmqxzic88h9i74ruducr15ou Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b="aoo/dmpx"; spf=pass (imf10.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.215.178 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1646210313-25859 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We must add "hugetlb_free_vmemmap=on" to boot cmdline and reboot the server to enable the feature of freeing vmemmap pages of HugeTLB pages. Rebooting usually taske a long time. Add a sysctl to enable the feature at runtime and do not need to reboot. Signed-off-by: Muchun Song --- Documentation/admin-guide/sysctl/vm.rst | 13 ++++++++++ include/linux/memory_hotplug.h | 9 +++++++ mm/hugetlb_vmemmap.c | 42 ++++++++++++++++++++++++++++----- mm/hugetlb_vmemmap.h | 4 +++- mm/memory_hotplug.c | 5 ++++ 5 files changed, 66 insertions(+), 7 deletions(-) diff --git a/Documentation/admin-guide/sysctl/vm.rst b/Documentation/admin-guide/sysctl/vm.rst index f4804ce37c58..01f18e6cc227 100644 --- a/Documentation/admin-guide/sysctl/vm.rst +++ b/Documentation/admin-guide/sysctl/vm.rst @@ -561,6 +561,19 @@ Change the minimum size of the hugepage pool. See Documentation/admin-guide/mm/hugetlbpage.rst +hugetlb_free_vmemmap +==================== + +A toggle value indicating if vmemmap pages are allowed to be optimized. +If it is off (0), then it can be set true (1). Once true, the vmemmap +pages associated with each HugeTLB page will be optimized, and the toggle +cannot be set back to false. It only optimizes the subsequent allocation +of HugeTLB pages from buddy system, while already allocated HugeTLB pages +will not be optimized. + +See Documentation/admin-guide/mm/hugetlbpage.rst + + nr_hugepages_mempolicy ====================== diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h index e0b2209ab71c..20d7edf62a6a 100644 --- a/include/linux/memory_hotplug.h +++ b/include/linux/memory_hotplug.h @@ -351,4 +351,13 @@ void arch_remove_linear_mapping(u64 start, u64 size); extern bool mhp_supports_memmap_on_memory(unsigned long size); #endif /* CONFIG_MEMORY_HOTPLUG */ +#ifdef CONFIG_MHP_MEMMAP_ON_MEMORY +bool mhp_memmap_on_memory(void); +#else +static inline bool mhp_memmap_on_memory(void) +{ + return false; +} +#endif + #endif /* __LINUX_MEMORY_HOTPLUG_H */ diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 836d1117f08b..3bcc8f25bd50 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -10,6 +10,7 @@ #define pr_fmt(fmt) "HugeTLB: " fmt +#include #include "hugetlb_vmemmap.h" /* @@ -118,17 +119,14 @@ void __init hugetlb_vmemmap_init(struct hstate *h) BUILD_BUG_ON(__NR_USED_SUBPAGE >= RESERVE_VMEMMAP_SIZE / sizeof(struct page)); - if (!hugetlb_free_vmemmap_enabled()) - return; - - if (IS_ENABLED(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON) && - !is_power_of_2(sizeof(struct page))) { + if (!is_power_of_2(sizeof(struct page))) { /* * The hugetlb_free_vmemmap_enabled_key can be enabled when * CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON. It should * be disabled if "struct page" crosses page boundaries. */ - static_branch_disable(&hugetlb_free_vmemmap_enabled_key); + if (IS_ENABLED(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON)) + static_branch_disable(&hugetlb_free_vmemmap_enabled_key); return; } @@ -147,3 +145,35 @@ void __init hugetlb_vmemmap_init(struct hstate *h) pr_info("can free %d vmemmap pages for %s\n", h->nr_free_vmemmap_pages, h->name); } + +static struct ctl_table hugetlb_vmemmap_sysctls[] = { + { + .procname = "hugetlb_free_vmemmap", + .data = &hugetlb_free_vmemmap_enabled_key.key, + .mode = 0644, + /* only handle a transition from default "0" to "1" */ + .proc_handler = proc_do_static_key, + .extra1 = SYSCTL_ONE, + .extra2 = SYSCTL_ONE, + }, + { } +}; + +static __init int hugetlb_vmemmap_sysctls_init(void) +{ + /* + * The vmemmap pages cannot be optimized if + * "memory_hotplug.memmap_on_memory" is enabled unless + * "hugetlb_free_vmemmap" is enabled as well since + * "hugetlb_free_vmemmap" takes precedence over + * "memory_hotplug.memmap_on_memory". + */ + if (mhp_memmap_on_memory() && !hugetlb_free_vmemmap_enabled()) + return 0; + + if (is_power_of_2(sizeof(struct page))) + register_sysctl_init("vm", hugetlb_vmemmap_sysctls); + + return 0; +} +late_initcall(hugetlb_vmemmap_sysctls_init); diff --git a/mm/hugetlb_vmemmap.h b/mm/hugetlb_vmemmap.h index cb2bef8f9e73..b67a159027f4 100644 --- a/mm/hugetlb_vmemmap.h +++ b/mm/hugetlb_vmemmap.h @@ -21,7 +21,9 @@ void hugetlb_vmemmap_init(struct hstate *h); */ static inline unsigned int free_vmemmap_pages_per_hpage(struct hstate *h) { - return h->nr_free_vmemmap_pages; + if (hugetlb_free_vmemmap_enabled()) + return h->nr_free_vmemmap_pages; + return 0; } #else static inline int alloc_huge_page_vmemmap(struct hstate *h, struct page *head) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index c226a337c1ef..c2115e566abc 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -50,6 +50,11 @@ static bool memmap_on_memory __ro_after_init; #ifdef CONFIG_MHP_MEMMAP_ON_MEMORY module_param(memmap_on_memory, bool, 0444); MODULE_PARM_DESC(memmap_on_memory, "Enable memmap on memory for memory hotplug"); + +bool mhp_memmap_on_memory(void) +{ + return memmap_on_memory; +} #endif enum {