From patchwork Sun Aug 11 22:49:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13759902 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D318C52D7F for ; Sun, 11 Aug 2024 22:50:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B3AC86B0092; Sun, 11 Aug 2024 18:50:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AEA9A6B0098; Sun, 11 Aug 2024 18:50:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 967936B009A; Sun, 11 Aug 2024 18:50:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 77A3C6B0092 for ; Sun, 11 Aug 2024 18:50:06 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 06198A050D for ; Sun, 11 Aug 2024 22:50:06 +0000 (UTC) X-FDA: 82441459212.01.F553740 Received: from mail-oo1-f51.google.com (mail-oo1-f51.google.com [209.85.161.51]) by imf01.hostedemail.com (Postfix) with ESMTP id 369E64000B for ; Sun, 11 Aug 2024 22:50:04 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=IAtv7GcK; spf=pass (imf01.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.161.51 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723416550; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cEgDKE4ERpo3w8nI4WSUoZx58a4oMDZtRCwQ+K3zmag=; b=bOeZcV0cQevvWESh/qk15G+w3Gghl2S2bcE+snpuPSkI4TF5LAR52RhrWUWpbQjr9PoktJ BNGqYTJ3eTWXLdCsuNKnI6vsQ6hdQh6JxfD96ID0gSpAnZTR8q2e38S9KeTtaKEZV+g8R2 G3VuVnGAEj/2lJfjn7/qmAWGK+cVh6o= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=IAtv7GcK; spf=pass (imf01.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.161.51 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723416550; a=rsa-sha256; cv=none; b=ON7S/VS9C0+97DBbkuk/I5Ei+2OnI1DvmrgocMYhJmi22tARq7BNi7nyF325XCC3EKjjw/ wOKgkYSgetknZVlyoFMTPn92WMcfVAHTUNKvXeC+NlHCTUoRUVWNG/KsKDzqsvvJmWihRk AwkA/TFXxVbhzTlThqeL5pisbo5Or8Y= Received: by mail-oo1-f51.google.com with SMTP id 006d021491bc7-5d5b22f97b7so3114445eaf.2 for ; Sun, 11 Aug 2024 15:50:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723416603; x=1724021403; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cEgDKE4ERpo3w8nI4WSUoZx58a4oMDZtRCwQ+K3zmag=; b=IAtv7GcKVuGZmduI6gLGsOfjXOOtxXHPm+3xBIQcyBStsm+QdTK15FvIUZGs9h7JlL +yA8AT0bfPijGde7xLF6YsbiuVjVRLpV8lI4v7CxHJFw3HvPTIga+kZxDEPMwyIGZ9WX eqmrRwdmkgCixBab64JnwOJUh0/OJMg7FBWG8DxUPqV06wOuikHifawTx9apslyu8Rdc Wd63URV+jis6EdO34bKs1I3OmdhFzXjaEcUhOJC3d2XP6502OgUBOCeFvwcA9gCUkYr5 hioUG8tzE4zPISZZtKWeCGfv3Ga9OtX/VC6o2H9hY0/GwWMvstdFqKble19IWhxP8uLh 7R2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723416603; x=1724021403; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cEgDKE4ERpo3w8nI4WSUoZx58a4oMDZtRCwQ+K3zmag=; b=Jc5tDdFgaIqDMhEP7eZyaqYXhjpICMQO3mmCeM46Te0wazplMJRMfqGhNTkrdSTwFf PG9hjE2ZCViS33bsrPZlWCFjYJ0vcXeNTFvpnVwTrU41CPDftoTKYVr0XB2GL1NrsF4u EveMCosD8FWytQn9DXmiPrZ0FU5k4wBw1SAZtxCajYq9HGHWMoj6BPHvgmtiL4S9AC0E FigYuzUV3DQUiP0ucX4uvB4hfqGarknoWgjNXMoU5RyyMXIatZlQZQfuBv/NnHprS1MY Ls3RwUzjiOLp8AIVbJTnmeH+yrAZfRLexARxVzP2WbqImFR6zHFMEPS3b1Ys7F0ewVjB 3uXQ== X-Forwarded-Encrypted: i=1; AJvYcCVsXtaokZfG4/gaiZFL3QwYqd/73H+fsjwKDwRS5PAgoT16yzS+wVIg7m4zf6vUmZrKtz63kwgYEDBDnzhGtsy32mA= X-Gm-Message-State: AOJu0Ywc1Y9VAaeQrHuNyGJvyUkUllWR3ciiq0C8jnby1VLjaZgmEt7r pWSdVKNr4HTpILwG8Oz4mcqfTVicuxzztv+VQdgUShP6egTJ+h4d X-Google-Smtp-Source: AGHT+IE9ZC8p2j6VQW0lR0KF1Vkmg+7U5zG9HQ83zY+w5vCE3eQKU6K5sDiWTcNZ91l2hRRlI6QHmQ== X-Received: by 2002:a05:6358:280d:b0:1ac:1499:34ab with SMTP id e5c5f4694b2df-1b176f838a2mr1070001055d.14.1723416602917; Sun, 11 Aug 2024 15:50:02 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-710e5874ddcsm2822993b3a.28.2024.08.11.15.49.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 11 Aug 2024 15:50:02 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hanchuanhua@oppo.com, ioworker0@gmail.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, ryan.roberts@arm.com, v-songbaohua@oppo.com, ziy@nvidia.com, yuanshuai@oppo.com Subject: [PATCH v2 1/2] mm: collect the number of anon large folios Date: Mon, 12 Aug 2024 10:49:39 +1200 Message-Id: <20240811224940.39876-2-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240811224940.39876-1-21cnbao@gmail.com> References: <20240811224940.39876-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Stat-Signature: edb3xoqrjrugftyk1wxk9sz1ncfsrkfw X-Rspam-User: X-Rspamd-Queue-Id: 369E64000B X-Rspamd-Server: rspam02 X-HE-Tag: 1723416604-480178 X-HE-Meta: U2FsdGVkX1+aAlMpf97LabCvdIboVVa+nTTAcWhXFHBrZbMGu787SEvnpfK0c78WKMsfDqxUsw6aCrnlDQgyNlMz/B3hDovw8mdE6+3HQO0g7vLlAJ5uKzUSDUHzbWFd96PiX2LEaYeNYLm8wePlJ/8kyFl0dsn6yYhLq2cvFt100yW1L+UsUCk6RlJrXYSMzhLS0cT9opjk8ARFsXdr97VnoHDHCPHyyG5dRO1dQsJ8ltLKlWFo4jaknBQ0L7uoplS5eVPQEkx1q4fS5HqW9emP/i2OVw8aGxVF9jCoxBvbGCLAxStekxGg5aQIvUU9YGfKN0Pqaf90JoycPDlh8aGR/X3G5iBENUsX4CeD+CH+gpyGZkK/xYi3s1PxUht2+lJBUlijXiT0qJcDL4vypAGoLY1xKqbps7ZKcB6S4JcIutmJ72LRUouSkDlS6HWvFYJnXIKBB3WV/4Wn5ryLZctmMr+qmUig8YBDLsoWm8lP1ZDqGo8Iivp3MZGiqNLMDvdavhBr1zxVbwD8FxM6d584K+y1xiurW/8FeDlgTsllO6fRab+kHCq58NyyhZwtLFADh98gY/7BKBT+E7mn/96b0tXf2xcmC7XKU98w5++OiuNqE+d/Cuo42mPWElnJdW0/6eQI9LP5E1Lxn39bYdt17TXJGCFVHi+Ta8X6PV5LdWh945gnaaYfx5BLudIZdai0XnTNPaJjkCtFRHSzOdecK7eo8YpEpuz1LJpq38Fa4wdw8Ijd2OKEasrLkfBd8BhHS7peivcNxfFuekaI0ZqvF/0OBOZQiW8IXNUqg1UHGgZtgW8gNbN7JtGnD0yXM0xy5FCaJXObVe2J8RBgSCIHbBRTaaXRUBfPz0zwuFtDJKWPtM9v1rr+pCdERfjxQ+E2YG+bL6sQJQzt2JVcbYsi9IE46gs1Yms00jzisHZBXA7RZTOIkEp7SbYciZzsB4sgtNFzWPWR0nKRt62 XTlueuN0 qDuI+c3fhS0yB/6ulS0M8pCF5Zz3N2FEVgEN8YkTIIWC9McAphqzQnrvQV8KVB//MlUT9kkGQ4gtSoecvzrFOsXjBRQj6TXE+FJHvS5bQ76QQqCXfE47evM8KhefdbmSAfm6jYgGRfshWfDhGYgSR4NXCIVf86Gto4YAQX3cOG6Vr2KBxPNUjoKfdvwMMikMY0aqw+zhNw+A8kaDBxIF1IiSQqumWMxxNRvolADPVBWarRPaMtOQQvI/84/+w1jqV829gyCmzbSu7VlfukYuir9smrs2qqfplshMI515Z4vlvuylcPmZgm6z3t455HthmTUPhfMsZHgbjTIB7qqVnwxmdhZOS7Ps2kGvdsP266aMQP8eabXHk93wYB6qu083CNDduLC1L9+l577Wp9jMEyhRqDTAnF3qWj1k0KgLD3fmiiSAyLln/mz8+mg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song Anon large folios come from three places: 1. new allocated large folios in PF, they will call folio_add_new_anon_rmap() for rmap; 2. a large folio is split into multiple lower-order large folios; 3. a large folio is migrated to a new large folio. In all above three counts, we increase nr_anon by 1; Anon large folios might go either because of be split or be put to free, in these cases, we reduce the count by 1. Folios that have been added to the swap cache but have not yet received an anon mapping won't be counted. This is consistent with the AnonPages statistics in /proc/meminfo. Signed-off-by: Barry Song Acked-by: David Hildenbrand --- Documentation/admin-guide/mm/transhuge.rst | 5 +++++ include/linux/huge_mm.h | 15 +++++++++++++-- mm/huge_memory.c | 13 ++++++++++--- mm/migrate.c | 4 ++++ mm/page_alloc.c | 5 ++++- mm/rmap.c | 1 + 6 files changed, 37 insertions(+), 6 deletions(-) diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst index 058485daf186..9fdfb46e4560 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -527,6 +527,11 @@ split_deferred it would free up some memory. Pages on split queue are going to be split under memory pressure, if splitting is possible. +nr_anon + the number of anon huge pages we have in the whole system. + These huge pages could be entirely mapped or have partially + unmapped/unused subpages. + As the system ages, allocating huge pages may be expensive as the system uses memory compaction to copy data around memory to free a huge page for use. There are some counters in ``/proc/vmstat`` to help diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 4c32058cacfe..2ee2971e4e10 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -126,6 +126,7 @@ enum mthp_stat_item { MTHP_STAT_SPLIT, MTHP_STAT_SPLIT_FAILED, MTHP_STAT_SPLIT_DEFERRED, + MTHP_STAT_NR_ANON, __MTHP_STAT_COUNT }; @@ -136,14 +137,24 @@ struct mthp_stat { DECLARE_PER_CPU(struct mthp_stat, mthp_stats); -static inline void count_mthp_stat(int order, enum mthp_stat_item item) +static inline void mod_mthp_stat(int order, enum mthp_stat_item item, int delta) { if (order <= 0 || order > PMD_ORDER) return; - this_cpu_inc(mthp_stats.stats[order][item]); + this_cpu_add(mthp_stats.stats[order][item], delta); +} + +static inline void count_mthp_stat(int order, enum mthp_stat_item item) +{ + mod_mthp_stat(order, item, 1); } + #else +static inline void mod_mthp_stat(int order, enum mthp_stat_item item, int delta) +{ +} + static inline void count_mthp_stat(int order, enum mthp_stat_item item) { } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d90d6e94a800..afb911789df8 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -596,6 +596,7 @@ DEFINE_MTHP_STAT_ATTR(shmem_fallback_charge, MTHP_STAT_SHMEM_FALLBACK_CHARGE); DEFINE_MTHP_STAT_ATTR(split, MTHP_STAT_SPLIT); DEFINE_MTHP_STAT_ATTR(split_failed, MTHP_STAT_SPLIT_FAILED); DEFINE_MTHP_STAT_ATTR(split_deferred, MTHP_STAT_SPLIT_DEFERRED); +DEFINE_MTHP_STAT_ATTR(nr_anon, MTHP_STAT_NR_ANON); static struct attribute *anon_stats_attrs[] = { &anon_fault_alloc_attr.attr, @@ -608,6 +609,7 @@ static struct attribute *anon_stats_attrs[] = { &split_attr.attr, &split_failed_attr.attr, &split_deferred_attr.attr, + &nr_anon_attr.attr, NULL, }; @@ -3216,8 +3218,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, struct deferred_split *ds_queue = get_deferred_split_queue(folio); /* reset xarray order to new order after split */ XA_STATE_ORDER(xas, &folio->mapping->i_pages, folio->index, new_order); - struct anon_vma *anon_vma = NULL; + bool is_anon = folio_test_anon(folio); struct address_space *mapping = NULL; + struct anon_vma *anon_vma = NULL; int order = folio_order(folio); int extra_pins, ret; pgoff_t end; @@ -3229,7 +3232,7 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, if (new_order >= folio_order(folio)) return -EINVAL; - if (folio_test_anon(folio)) { + if (is_anon) { /* order-1 is not supported for anonymous THP. */ if (new_order == 1) { VM_WARN_ONCE(1, "Cannot split to order-1 folio"); @@ -3269,7 +3272,7 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, if (folio_test_writeback(folio)) return -EBUSY; - if (folio_test_anon(folio)) { + if (is_anon) { /* * The caller does not necessarily hold an mmap_lock that would * prevent the anon_vma disappearing so we first we take a @@ -3382,6 +3385,10 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, } } + if (is_anon) { + mod_mthp_stat(order, MTHP_STAT_NR_ANON, -1); + mod_mthp_stat(new_order, MTHP_STAT_NR_ANON, 1 << (order - new_order)); + } __split_huge_page(page, list, end, new_order); ret = 0; } else { diff --git a/mm/migrate.c b/mm/migrate.c index 7e1267042a56..bde573ec2af8 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -423,6 +423,8 @@ static int __folio_migrate_mapping(struct address_space *mapping, /* No turning back from here */ newfolio->index = folio->index; newfolio->mapping = folio->mapping; + if (folio_test_anon(folio) && folio_test_large(folio)) + mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON, 1); if (folio_test_swapbacked(folio)) __folio_set_swapbacked(newfolio); @@ -447,6 +449,8 @@ static int __folio_migrate_mapping(struct address_space *mapping, */ newfolio->index = folio->index; newfolio->mapping = folio->mapping; + if (folio_test_anon(folio) && folio_test_large(folio)) + mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON, 1); folio_ref_add(newfolio, nr); /* add cache reference */ if (folio_test_swapbacked(folio)) { __folio_set_swapbacked(newfolio); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 84a7154fde93..382c364d3efa 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1084,8 +1084,11 @@ __always_inline bool free_pages_prepare(struct page *page, (page + i)->flags &= ~PAGE_FLAGS_CHECK_AT_PREP; } } - if (PageMappingFlags(page)) + if (PageMappingFlags(page)) { + if (PageAnon(page) && compound) + mod_mthp_stat(order, MTHP_STAT_NR_ANON, -1); page->mapping = NULL; + } if (is_check_pages_enabled()) { if (free_page_is_bad(page)) bad++; diff --git a/mm/rmap.c b/mm/rmap.c index a6b9cd0b2b18..b7609920704c 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1467,6 +1467,7 @@ void folio_add_new_anon_rmap(struct folio *folio, struct vm_area_struct *vma, } __folio_mod_stat(folio, nr, nr_pmdmapped); + mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON, 1); } static __always_inline void __folio_add_file_rmap(struct folio *folio, From patchwork Sun Aug 11 22:49:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13759903 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A9B5C52D7F for ; Sun, 11 Aug 2024 22:50:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 19CFA6B009A; Sun, 11 Aug 2024 18:50:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 14D876B009E; Sun, 11 Aug 2024 18:50:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 014F06B009F; Sun, 11 Aug 2024 18:50:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D81AE6B009A for ; Sun, 11 Aug 2024 18:50:11 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 9E8741402C4 for ; Sun, 11 Aug 2024 22:50:11 +0000 (UTC) X-FDA: 82441459422.06.38AE2BF Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf22.hostedemail.com (Postfix) with ESMTP id D85B0C000D for ; Sun, 11 Aug 2024 22:50:09 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=AdC3yUlC; spf=pass (imf22.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723416575; a=rsa-sha256; cv=none; b=FPZ8J36wVlHZUBnGiF+eSKpQkEvGKFY61SSdwFEVn3G6rIJooVkz9nqwGITSAiyywp3g71 JNkrCZlrCFqIx9/SpoaBsh+ZIcv7G49mZwHDpiDrRFERzrqZh6FD2w1jWo/D501o1oYnaW M3Pj3mZsz/F3N9t6rO+iNH5xO3fmeWA= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=AdC3yUlC; spf=pass (imf22.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723416575; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Tw0JgRH12Wo8KdGqNK+rncsTqLHooD07nmRGRawWDls=; b=vrie93W4f8gGvdDoD1QWR0SoX0uxmCQCOpY55rra3xwtKDKQ9hI7KUt7Qd8E+bxG4kw2ye V1fKz+xb/nErMMuVR5ihwTGCOACb42AICVgdrgsdIaFe4rTP580FGPIVAsRceOKzITWVMh 0uivkQwNjDrKjkUYX8TJXw9HtgMRtj0= Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-70d1cbbeeaeso2650468b3a.0 for ; Sun, 11 Aug 2024 15:50:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723416609; x=1724021409; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Tw0JgRH12Wo8KdGqNK+rncsTqLHooD07nmRGRawWDls=; b=AdC3yUlCVq9mJn5xQDPSWUsgJHpHnwUU1mdgBaJbGB2DNE3+yWCcfzYuVGgV1G7XrZ CuFu+66FojbA7udfE/lwpt6GNA5YGFiaDWP7tOSphPLjz1jkLyLPtMZaLPcUp9hiyd6K mEJaZDF4QUcPzd4zbJPJ2kgJjWkd21Dy3CWzGj1EBSufJRmlETxuBwffQ/9do3EWdm97 ooNRwxdaqh7j0CYQwJoSUTie0TLRcOEGLrGMeEVUjWYoRanqkaZMmFMpegMwXryiPsN3 l3kt9LRqTx+qa7EK0DogrxCy84e8PT1Nh6UYDJ+iXyoY189sIOmF3jbM93dzJgcBDXzp vJLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723416609; x=1724021409; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Tw0JgRH12Wo8KdGqNK+rncsTqLHooD07nmRGRawWDls=; b=bVFZDxFMkqnGoziIYJtztd/mahXhHiibbBLnPFKJxRTDuNAGLCgx/iBuacl/CvllhA Ih+btZH/5JLnoorMK9xjqOHm53PtKqhoFuc1lZe584Ck7tMtZ+3Rt1mPpBUhirhTORuS K0sNy9gpGn5qOOSXk6Rt8EucZy2UYbGcm0t3ZYDYJA+YcEgT+NGoaeMtNgahp815ull6 bYSi2i/2WMt9KDijBdHY9JCw8T6Epjn61YbIJ6M4gsLEFkTe+67xoYftiZJVebpQaOa4 vhioPmnbazfx5vBQ8NSXXjMCD5eSVkgci/7cmJqycqM1/9QskQCS5lCrryfhTsLbj2S2 RoHA== X-Forwarded-Encrypted: i=1; AJvYcCWKB29St+gid+GGBm7OyAXEDtvN0fmBTbPuDDVw2ZsuQax4X92SIqvOg2ZaJ4AlzpaBZfOEcp+VUA==@kvack.org X-Gm-Message-State: AOJu0YzGXXG5H/BJtPaZEUYQyrNU6Ia6wwMozugpslbXbGqOrErWTdPq VFLvEnPtWvyFF1Ha8tmNncUWjmbi4XJPFX2yc2QFLx7Wg6O0RrIO X-Google-Smtp-Source: AGHT+IGAQoUx/mSmccF5tcTHEJNHLzHr1Ne7T77+kVl0PpmsVLNxjtn54anVGm8f20OzojHM35fnSQ== X-Received: by 2002:a05:6a00:2e17:b0:705:9ddb:db6b with SMTP id d2e1a72fcca58-710dc6e549amr6012262b3a.13.1723416608551; Sun, 11 Aug 2024 15:50:08 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-710e5874ddcsm2822993b3a.28.2024.08.11.15.50.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 11 Aug 2024 15:50:08 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hanchuanhua@oppo.com, ioworker0@gmail.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, ryan.roberts@arm.com, v-songbaohua@oppo.com, ziy@nvidia.com, yuanshuai@oppo.com Subject: [PATCH v2 2/2] mm: collect the number of anon large folios on split_deferred list Date: Mon, 12 Aug 2024 10:49:40 +1200 Message-Id: <20240811224940.39876-3-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240811224940.39876-1-21cnbao@gmail.com> References: <20240811224940.39876-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Stat-Signature: 9o4dcohr6depjj884odbz8uy6wrs74yu X-Rspamd-Queue-Id: D85B0C000D X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1723416609-961275 X-HE-Meta: U2FsdGVkX1/GGQJTrCmfucJ0Z58DO/CcE9ro17bpIrZu9Lz1Y3mQju2bxPsRRDogsQy1TR22B8fMoyIyWKfcrCgBQ9kvZbqV/0cXoSENig9ir/OWS2RiMmwSZAWFaatApsH4PY3Z+oieeRZkMSL0D26dbetfJZ4fk+ykXnhS0AjBHdP/a8nQBr/kOaZ7IG6+upz8+TSxA9Dz6cYpzSwxmhEg974P6ivxbA2o0Rn4iHz7zDZWSnB0UJK4jwjIM4k5JYPriEgA2HMXq7Ews4+CoqL1qlUDi8q5HJcK/zvVcEsmbUf2x64UFJu6F8J30GJ+6tGMVSQnQoJ9yUWTFdrw5rgYz1lvW4pGSeZv/K2mPwitKyYj6st+9dM5IkjL5OT9PE7BF0e9RbFw0FGadmXv8StnZy1mDMFdCgvQUfvcIcP/qnm3BfB3M8iTSXBP0sKUFSc1cxNhqXVqgojSMCNKDHzXNxl1YJqcO8dzZ2iQmfHUsKuhOr2Ck7ucjETYW/mOphcgaXkGUE8QaZyfj76h03OnYfLasI8csSU8l3Nm1FkheNsyGHo9noQuDu9aGX7TuZYbvEkvi2po23jfwNJDf1Vs6Vu4lDKZ2XelA4VReAC/jBHES8bi/fGuxaWFa8kEnFG2z9M9jdzbi+y7nGq+1Kidr5SQxyECO/gZCCPabUFFh3j6EiMxvZc5W2NkQKkK5THaGRnfDjfBp43iQMYn8kjbeskxFtA3AtBET/QzZcywdUyeckDY7JJ2Q3z6cnp3liwPxm5HERKfWRHJhXp9WE+hM9b8+S8N0b2U/rYDHKh6jwI9eiWvA06c2SlZL45uiRPLEroLTqmPKZcobv/EbuAwfx9QqRWmsRsU3rNqDvu/hG02nUxFywI1jQIrIwteBHFXd9TSSCkdxbpQ3hYZeH+7zYQmxm83w3bHxnpP/m1HvRCmRMgpLtNq9mIuwdhVE8E8NlAlBKknZ2Vw2VY Ctv/Gj5A SeEBT2HQ0fPAz3R+jBB6KHV22rdKOT5gHq7qkbbV3qtlAcY5lTRImKU9atbCaemv0R+3BxodC9ci8O6PLC/NavSWZgJeNzITdrw9MZgGV9EoTNlqYMjt6itDR6GPEAhkWITlpshBZj4zCoMlc9nkdolIabd2jsvhIsDoAddB2wkctaF/y+XX5zXrkkhHEPg8qLByj4IQAUx8ZpPkjz7m3lvJplBgaPNizN8K99z6Q3805iUyxyONdGJskI8kN1r3Q/Nj1guwIhYZl05CuREcGw2TaIR6hfNv+V96HAbbeSSnq1UpJrNknQT0wlby/4x73/kCd2gHC2uQnPh2+Fp7rHQhlec+xSejLG1+fVkUdgR0/NECy7b0dLWHQPhnE2XPLT+d3EncAsbVRztaaA0AE8q7MGKUYz4c8aY6DLqPijybZZ76JN2y7Vra0dAKBgVaspsQVp5Dbe2ecp9w50xeiaNP8IGVNAV8n75Hy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song When an mTHP is added to the deferred_list, its partial pages are unused, leading to wasted memory and potentially increasing memory reclamation pressure. Detailing the specifics of how unmapping occurs is quite difficult and not that useful, so we adopt a simple approach: each time an mTHP enters the deferred_list, we increment the count by 1; whenever it leaves for any reason, we decrement the count by 1. Signed-off-by: Barry Song --- Documentation/admin-guide/mm/transhuge.rst | 5 +++++ include/linux/huge_mm.h | 1 + mm/huge_memory.c | 6 ++++++ 3 files changed, 12 insertions(+) diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst index 9fdfb46e4560..7072469de8a8 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -532,6 +532,11 @@ nr_anon These huge pages could be entirely mapped or have partially unmapped/unused subpages. +nr_split_deferred + the number of anon huge pages which have been partially unmapped + and put onto split queue. Those unmapped subpages are also unused + and temporarily wasting memory. + As the system ages, allocating huge pages may be expensive as the system uses memory compaction to copy data around memory to free a huge page for use. There are some counters in ``/proc/vmstat`` to help diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 2ee2971e4e10..1e2d5dbe82c5 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -127,6 +127,7 @@ enum mthp_stat_item { MTHP_STAT_SPLIT_FAILED, MTHP_STAT_SPLIT_DEFERRED, MTHP_STAT_NR_ANON, + MTHP_STAT_NR_SPLIT_DEFERRED, __MTHP_STAT_COUNT }; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index afb911789df8..1a12c011e2df 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -597,6 +597,7 @@ DEFINE_MTHP_STAT_ATTR(split, MTHP_STAT_SPLIT); DEFINE_MTHP_STAT_ATTR(split_failed, MTHP_STAT_SPLIT_FAILED); DEFINE_MTHP_STAT_ATTR(split_deferred, MTHP_STAT_SPLIT_DEFERRED); DEFINE_MTHP_STAT_ATTR(nr_anon, MTHP_STAT_NR_ANON); +DEFINE_MTHP_STAT_ATTR(nr_split_deferred, MTHP_STAT_NR_SPLIT_DEFERRED); static struct attribute *anon_stats_attrs[] = { &anon_fault_alloc_attr.attr, @@ -610,6 +611,7 @@ static struct attribute *anon_stats_attrs[] = { &split_failed_attr.attr, &split_deferred_attr.attr, &nr_anon_attr.attr, + &nr_split_deferred_attr.attr, NULL, }; @@ -3359,6 +3361,7 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, if (folio_order(folio) > 1 && !list_empty(&folio->_deferred_list)) { ds_queue->split_queue_len--; + mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_SPLIT_DEFERRED, -1); /* * Reinitialize page_deferred_list after removing the * page from the split_queue, otherwise a subsequent @@ -3425,6 +3428,7 @@ void __folio_undo_large_rmappable(struct folio *folio) spin_lock_irqsave(&ds_queue->split_queue_lock, flags); if (!list_empty(&folio->_deferred_list)) { ds_queue->split_queue_len--; + mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_SPLIT_DEFERRED, -1); list_del_init(&folio->_deferred_list); } spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags); @@ -3466,6 +3470,7 @@ void deferred_split_folio(struct folio *folio) if (folio_test_pmd_mappable(folio)) count_vm_event(THP_DEFERRED_SPLIT_PAGE); count_mthp_stat(folio_order(folio), MTHP_STAT_SPLIT_DEFERRED); + mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_SPLIT_DEFERRED, 1); list_add_tail(&folio->_deferred_list, &ds_queue->split_queue); ds_queue->split_queue_len++; #ifdef CONFIG_MEMCG @@ -3513,6 +3518,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, list_move(&folio->_deferred_list, &list); } else { /* We lost race with folio_put() */ + mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_SPLIT_DEFERRED, -1); list_del_init(&folio->_deferred_list); ds_queue->split_queue_len--; }