From patchwork Sun Sep 22 04:32:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13809000 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9F45CF9C6C for ; Sun, 22 Sep 2024 04:32:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A2FEE6B007B; Sun, 22 Sep 2024 00:32:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9DFF06B0082; Sun, 22 Sep 2024 00:32:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A7636B0085; Sun, 22 Sep 2024 00:32:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 6C4176B007B for ; Sun, 22 Sep 2024 00:32:39 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id C7271C0C6F for ; Sun, 22 Sep 2024 04:32:38 +0000 (UTC) X-FDA: 82591103196.12.5FD8707 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by imf15.hostedemail.com (Postfix) with ESMTP id 152B0A0002 for ; Sun, 22 Sep 2024 04:32:32 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="REj3Mh7/"; spf=pass (imf15.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726979442; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=AJIqkPeSg0kDGuLUkbSeWADD2GKVOIFf/Ecw0K1tnO0=; b=0Q+c0v0yjuNuY2G+9AHOpAc770GiIIc4CWbils2H7d56XoAbfKoVkjJ0B2a7a/9qrgtg6o 8aiFeGZwGAQ95rxEpBJtWLMxJ9cbxR0fQOZCn5plbJwi+uqLENNOiGe67yDFIOEUSgQ0KK N2a3G7FvcAnKRKg5OL4HFQJIbmQVKsA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726979442; a=rsa-sha256; cv=none; b=ZXaHZr2c5YR/TmtqQ2I7waM7NsynnB0VHwpv2RivmHaf3ZP+VT83ts2mQoduHFHYl/ZLOA faJSXUHzNO6ngWQfLUzYoGbkDz6c6uLSNoiSnqNsDIDvt7TOIItbNzCV1uAkLoljGVolg4 UBMuOuzdG7EZqD+pw9nUkAYdnHE/g7E= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b="REj3Mh7/"; spf=pass (imf15.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1726979542; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=AJIqkPeSg0kDGuLUkbSeWADD2GKVOIFf/Ecw0K1tnO0=; b=REj3Mh7/yTxruMGmnCGAClZNkFHUWAiLKU1LgsvgMgwUl9zqN9mG6WL5gP856UAVhFZblwt/1dNlu/CcToZL7BlBqfxAIUfrwH4NEONAU6YO+MGoa2/mVrVzx+eiDV4296D1HJncK0wykNtBDo1rWodVi0nS5g60gzm/ercjQ2M= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WFPfeQH_1726979540) by smtp.aliyun-inc.com; Sun, 22 Sep 2024 12:32:21 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, 21cnbao@gmail.com, ryan.roberts@arm.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3] mm: shmem: fix khugepaged activation policy for shmem Date: Sun, 22 Sep 2024 12:32:13 +0800 Message-Id: <9b9c6cbc4499bf44c6455367fd9e0f6036525680.1726978977.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 MIME-Version: 1.0 X-Rspamd-Queue-Id: 152B0A0002 X-Stat-Signature: shuiyf44nw4p9mry8thuado8uj9derk8 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1726979552-484330 X-HE-Meta: U2FsdGVkX1/IMlZ+hNh2P8yqtG1TtCTuPlbU4Mhp5Tfs7KWpLVVoHNbh5y7UTx5nC2w72YRy1BGXcPQqjpN3294pCyxEsQYmmJyE/BQo7MHO2OMF3AzwT3Mf+aB6FP5VbubZNpHKA+8BCmXZHmIImu5o8Zs//EeHVr5Jp28k7Rkn/m8AuHuA0Y5KOVwDwFG5EwQh0WklRRbketOvPyIuLhO5OIdp+Kh/uxlP3rSE2sam1hesRxPNdWyV/0Z7n0eMoAEBWk1HUgYXukKGuEi5R/rQZSfIxL/18eCRfXM4jqBMSNLMIOat6R1FPq5jv2/7hVUYvwn5l3ktdvz7y+XcWFbVeNPhsGxOc2xH6GqRUHzrFWyOGQlhHONVoRmRT/NUBrP+CXuvJyjTDCQyABlFVa/TRF4dpAjvIl2ctT9HIoGgyJN0vzP+upCRqw0Ak9edJcFwPB1evxP6gt8f4otUs9rCE5Cnu+F1fAr79G3imGvpj2z4Uuy1NMvBKZJESVhHVnhqM26Uv/LJ6jOfyjOMd8aa1hMXEs7c06S6EYnyXKfyj8M/KCMKbpPAr3/7dq9qPTQ4rV25VvsH4e4S2QdDk/qi4aDtQv9zfY90A19CLq+vZpTswslLNfik0kcPHvPogkPJIHGLhalmOtxDZDn4zVd64iuRanAirRbNcJiI6IRksF3Bbmo9JiOUtVUACyzd2hrtVnKfY2X2oyVDozx5nAvjJK0dupjRSL6K9+Vuup4HQtxyZirS9uyE8JaEvmbfTRjWv8968FX3Jq89TdrN9/uj9c7T1IwJaqO+E2SiSy9CEa9l7rHI9wjzAdILJe6c5zwiUj1/+9BU67eA/ZgXV7omzH6mhg1sJLT6oMCowMxjuMdboz1jO5In72lT04gnPmFwedwILYv88E/O5mxSV+GXDjTTQxGHQjzL5jxhPsT+j0YcGs9th0alrL8+pINqvSv97/x/COKAcY6skxS 06vMFYW3 uFvGaTIMrNPLaPc9MJfEOB/aztKAP2qAvnvT4OLJkTrF587jeh2YJwSO9eTmQ698J9jiH2a+kseOhPFjkfj2NmLPvOG6MTVJh2ql0HUA4zTN7ZErSYUCPsKacrp0Ynjinqq7qiQyxpMa3LViTSJlpfmofvpSbRAFZEr7jnusY+Y7ScxX6Aykhr88792V65sEU8vfb3VDm0z1S+oJQyPVfEBqcTmFnjBLXd/GggOO5tVHfCippxSjQqNfGfEdbmN+0ihkVjL0d3uv9CwR4mVh7I+lFAjniPM+BuC2M X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Shmem has a separate interface (different from anonymous pages) to control huge page allocation, that means shmem THP can be enabled while anonymous THP is disabled. However, in this case, khugepaged will not start to collapse shmem THP, which is unreasonable. To fix this issue, we should call start_stop_khugepaged() to activate or deactivate the khugepaged thread when setting shmem mTHP interfaces. Moreover, add a new helper shmem_hpage_pmd_enabled() to help to check whether shmem THP is enabled, which will determine if khugepaged should be activated. Reported-by: Ryan Roberts Signed-off-by: Baolin Wang Reviewed-by: Ryan Roberts Acked-by: David Hildenbrand --- Changes from v2: - Fix building errors reported by kernel test robot. Changes from v1: - Add reviewed tag from Ryan. Thanks. - Add some shmem comments per Ryan. --- include/linux/shmem_fs.h | 6 ++++++ mm/khugepaged.c | 6 +++++- mm/shmem.c | 29 +++++++++++++++++++++++++++-- 3 files changed, 38 insertions(+), 3 deletions(-) diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h index 515a9a6a3c6f..ee6635052383 100644 --- a/include/linux/shmem_fs.h +++ b/include/linux/shmem_fs.h @@ -114,6 +114,7 @@ int shmem_unuse(unsigned int type); unsigned long shmem_allowable_huge_orders(struct inode *inode, struct vm_area_struct *vma, pgoff_t index, loff_t write_end, bool shmem_huge_force); +bool shmem_hpage_pmd_enabled(void); #else static inline unsigned long shmem_allowable_huge_orders(struct inode *inode, struct vm_area_struct *vma, pgoff_t index, @@ -121,6 +122,11 @@ static inline unsigned long shmem_allowable_huge_orders(struct inode *inode, { return 0; } + +static inline bool shmem_hpage_pmd_enabled(void) +{ + return false; +} #endif #ifdef CONFIG_SHMEM diff --git a/mm/khugepaged.c b/mm/khugepaged.c index f9c39898eaff..feb101f465c6 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -416,9 +416,11 @@ static inline int hpage_collapse_test_exit_or_disable(struct mm_struct *mm) static bool hugepage_pmd_enabled(void) { /* - * We cover both the anon and the file-backed case here; file-backed + * We cover the anon, shmem and the file-backed case here; file-backed * hugepages, when configured in, are determined by the global control. * Anon pmd-sized hugepages are determined by the pmd-size control. + * Shmem pmd-sized hugepages are also determined by its pmd-size control, + * except when the global shmem_huge is set to SHMEM_HUGE_DENY. */ if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && hugepage_global_enabled()) @@ -430,6 +432,8 @@ static bool hugepage_pmd_enabled(void) if (test_bit(PMD_ORDER, &huge_anon_orders_inherit) && hugepage_global_enabled()) return true; + if (IS_ENABLED(CONFIG_SHMEM) && shmem_hpage_pmd_enabled()) + return true; return false; } diff --git a/mm/shmem.c b/mm/shmem.c index 361affdf3990..181b1b051070 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1653,6 +1653,23 @@ static gfp_t limit_gfp_mask(gfp_t huge_gfp, gfp_t limit_gfp) } #ifdef CONFIG_TRANSPARENT_HUGEPAGE +bool shmem_hpage_pmd_enabled(void) +{ + if (shmem_huge == SHMEM_HUGE_DENY) + return false; + if (test_bit(HPAGE_PMD_ORDER, &huge_shmem_orders_always)) + return true; + if (test_bit(HPAGE_PMD_ORDER, &huge_shmem_orders_madvise)) + return true; + if (test_bit(HPAGE_PMD_ORDER, &huge_shmem_orders_within_size)) + return true; + if (test_bit(HPAGE_PMD_ORDER, &huge_shmem_orders_inherit) && + shmem_huge != SHMEM_HUGE_NEVER) + return true; + + return false; +} + unsigned long shmem_allowable_huge_orders(struct inode *inode, struct vm_area_struct *vma, pgoff_t index, loff_t write_end, bool shmem_huge_force) @@ -5036,7 +5053,7 @@ static ssize_t shmem_enabled_store(struct kobject *kobj, struct kobj_attribute *attr, const char *buf, size_t count) { char tmp[16]; - int huge; + int huge, err; if (count + 1 > sizeof(tmp)) return -EINVAL; @@ -5060,7 +5077,9 @@ static ssize_t shmem_enabled_store(struct kobject *kobj, shmem_huge = huge; if (shmem_huge > SHMEM_HUGE_DENY) SHMEM_SB(shm_mnt->mnt_sb)->huge = shmem_huge; - return count; + + err = start_stop_khugepaged(); + return err ? err : count; } struct kobj_attribute shmem_enabled_attr = __ATTR_RW(shmem_enabled); @@ -5137,6 +5156,12 @@ static ssize_t thpsize_shmem_enabled_store(struct kobject *kobj, ret = -EINVAL; } + if (ret > 0) { + int err = start_stop_khugepaged(); + + if (err) + ret = err; + } return ret; }