From patchwork Thu Oct 31 22:45:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13858417 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 284B0E6748D for ; Thu, 31 Oct 2024 22:46:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5A3B36B0095; Thu, 31 Oct 2024 18:46:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 579DF6B0096; Thu, 31 Oct 2024 18:46:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 442386B0098; Thu, 31 Oct 2024 18:46:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2249F6B0095 for ; Thu, 31 Oct 2024 18:46:57 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id BE0DAC19EB for ; Thu, 31 Oct 2024 22:46:56 +0000 (UTC) X-FDA: 82735383192.10.3584A6F Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf04.hostedemail.com (Postfix) with ESMTP id 01DFD40005 for ; Thu, 31 Oct 2024 22:46:19 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=xwt81EE3; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf04.hostedemail.com: domain of 33QgkZwgKCL4omrwi2lsksskpi.gsqpmry1-qqozego.svk@flex--kinseyho.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=33QgkZwgKCL4omrwi2lsksskpi.gsqpmry1-qqozego.svk@flex--kinseyho.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730414695; a=rsa-sha256; cv=none; b=Zi9+RUQsRlOe6JcDplrpWGrnK+rWNUYAlz5OG7e2wLGHXmIQ2OqyFznOIO3nJCIZHvc/kr XJl9DC1T5afbjXXsOo7hunU2TihayzrDEKf05sj76PX1mRnhm7FI7fBTqTJ0/QLWACDAu3 PCyfWN6DRnslqALwc3e6S/8jVfATP1c= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=xwt81EE3; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf04.hostedemail.com: domain of 33QgkZwgKCL4omrwi2lsksskpi.gsqpmry1-qqozego.svk@flex--kinseyho.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=33QgkZwgKCL4omrwi2lsksskpi.gsqpmry1-qqozego.svk@flex--kinseyho.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730414695; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=guCTOxzKlbAV7mFNRx7TvPMOy3sTYQa/R9Du6hcfImY=; b=r5tvNND03SpVS05rDCSBrUYPL9vJx+X4stQ1Lv/dS6FEc5e7D/FsMY8bUtPIHznmIDuhH/ 31l+1bqmTrJAtaew0pqyisWY2mP0sKdq9W00809ND4LrXL4apDs9N0GnRMWfoekOL7w8v8 H1Rx7WfXVbhgSxXa1gV06yUl45SFpD8= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e321d26b38so27165197b3.2 for ; Thu, 31 Oct 2024 15:46:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1730414814; x=1731019614; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=guCTOxzKlbAV7mFNRx7TvPMOy3sTYQa/R9Du6hcfImY=; b=xwt81EE3mreyOJlCoPxiVwu7JswuotdzBW+ABvBcoWnZImcSGEg7kOaZ/A0PghJ0UB l3K+1DtAyyMSLOqZriwAIqNpndkw+ZEpc3s+KGzWclKopMmj4NfwnagAoH16d/Q594EF 0q+qB8FpyismKgUTeclWLaFY8qVyHOB4dtsLu1oo3pjIv5vy49ioouaIRiQ+Klx1VW12 XuoHlEJMMxuIUi+9DEwbehhZQfX/7B+QoVUfB7A+xSZURlFlA6affBsrXKInEzk2eklu Mi67hHRjS/YbhQw3E4pdUbnht2/v6ALpaS+bPExiZkfBVVthalHo6OOQhmxlVX7pswGm v03w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730414814; x=1731019614; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=guCTOxzKlbAV7mFNRx7TvPMOy3sTYQa/R9Du6hcfImY=; b=KaxbExBqwZvVL7c0d1usmpqkZKvxlTp/mnxCpQMEdxLr0os9r7wEJY2NU6j+t5oOGQ yV4G3cwyJb+dTwrcMABaSVmZggkpJT6WY/1SyQevfXAe25nRUxWEmt9HgNZ700K0fLT2 qnqUK1ym3FR2UAz1NOCYvD2LS8xx5dfDYKpN0htqpSjWqQhx2F2nKVQIcdmEQpExZnJ3 hfNxgScwjjp7Ck/DqpNFRMBuavfJwGa2IMOg98YR6GQdY2q2z4WlmxPwQFSf2uQUrnDC TBDY1OT80M7N1K1iOVBxIcu1lwfauNMGSbz6bKrShjY4qTsBRXwx0LAsV57RpULvtAch xONQ== X-Forwarded-Encrypted: i=1; AJvYcCW6UovYeHzoA3ANrbC4V0Am5EUvOIgdF57gdzhE+EAWBOlBcPhFxgeXpK+weI1ItUdpmY5SewT0Aw==@kvack.org X-Gm-Message-State: AOJu0YysLBjeQBU34DKRkc5SIA2cngTM0Pykzh+TSZKC+3xY5K2Dr+Gm BO7I2QbHDmsIsHfiegTGnnFLiS+uiR+Z+NKre75tUSVSXX/J8dDGja8KWkkRSBB9cT6aIPOFNiW COSwA672+lw== X-Google-Smtp-Source: AGHT+IGA5sjZe8vpyl9lON3SR0LDoe9B+CE24eA4rKmGB1vEzw7Pi4vx0K181caj3vNLVFXCXdibXtjqBQbgGQ== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:a1:836b:ac13:31a5]) (user=kinseyho job=sendgmr) by 2002:a25:6b49:0:b0:e0e:8b26:484e with SMTP id 3f1490d57ef6-e30e5b41dffmr2685276.8.1730414813939; Thu, 31 Oct 2024 15:46:53 -0700 (PDT) Date: Thu, 31 Oct 2024 22:45:50 +0000 In-Reply-To: <20241031224551.1736113-1-kinseyho@google.com> Mime-Version: 1.0 References: <20241031224551.1736113-1-kinseyho@google.com> X-Mailer: git-send-email 2.47.0.163.g1226f6d8fa-goog Message-ID: <20241031224551.1736113-2-kinseyho@google.com> Subject: [PATCH mm-unstable v1 1/2] mm: add generic system-wide page counters From: Kinsey Ho To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Pasha Tatashin , David Rientjes , willy@infradead.org, Vlastimil Babka , David Hildenbrand , Kinsey Ho , Joel Granados , Kaiyang Zhao , Sourav Panda , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org X-Rspamd-Queue-Id: 01DFD40005 X-Stat-Signature: 65eomf8khjap34dr5n6yai8wt376dcax X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1730414779-405255 X-HE-Meta: U2FsdGVkX1+2kq1QB53V0iQqtHkkR2gJBBnOWQiGOVT/vLSJ34pbtvrNonW2K3vPBe2GAVH9nXsN2TAGr43c6imBaqpqVme63Pu9Ul3xClHKzpveoFGA9STeNoBZQjob09sVh5Xn56faovJUvZP+thQZtFai9FL7e/W+TjS1+BJ6ZtA0cL1UDnWmA2FjMZWPVocn+tlwSVanVUGpD9Gdr4obvF/jmvNzHG9xjM4thaK3cFnt9FTnQO1Ece5aTletE1NWn44Q8As6lbihtz15wWPQCxMj55FxydumMBQtn58Uv85FK9sVRo+DACVAdUTOtDvp2tUEhQmeqRDW1ETVihKVQWp+Xz1qerp+VOfxgnXrLZXNW75XtePvlvPFBKR8zxIOvYZmMnWbP084qo5nST/ButqXhavgaCpCPW0k+u4V3XEuMBE5uFjQS70yz8I43WtX85DxCKnYSBkEVXABl6snGQF5PuLTu+qBIfYNxpscDnAaoc3M/rBXHVrALwAtgwjhGmmRo8u8CDYC6ZCQAYQf2VIriyReZ9JOpsc/0VluhnrqGODbRzzLufnSN1ACXqcGnuESpE1perBgOcdrA6UocEFHRS+CI2aJbSp0CAYOErbO1vF1qPma3z6+xASFyXI8CYT+g7LWwKKxpBnLqRlPUp2wt8zcTKkhT/GFyg56dxzcCG5EKhSkGYsNOWhtZB0IRDXbWgiazXFn/s3ID1uWbDMxkXVuulD5dG87oxv6PZJWRYJ+ZxipudAVkP5b9cfRdNEY+iyiyOcZUhRXZX9rHicgqNrS5t43Q3TDe3DtEO106+aXi8xL77TBFKbteb17TxC+eR/lE/upfXHbFENrXmJ1fT/+GsXD1Kz8pJh+bgEdooabhliQh2ML8hAJp6vN9cgsgCS07XDKq6c7S1Q6wCbU/GIzO+f/BKxUI8E9BTNtUcBMXK3zP3LcC1pHHF/siDDA4R/sGCSPhEr XAJ+j6Lv C2BunCOwLUysdMZFV+0O5E3uFFlRG5RM8aDS/m2zlcCLBbGudYa+YHQ7CmDxfdTyxYYfzkqc6CMRiQ9hOrofDbH5PXMDUFbiX6In6U6J8MdCwfEIST7e2ZdKkyClwVd7MDYV51HVdmd/8897D06QZMsozd+4/EHeNAiOnNsimbVF5oOE9XbotE5nglnN9G9BecuL1SueekHo2tyDeXcK3oKXwH0PySmTLz3IvLSB5+54xURUHnRrSqObRSs+ejQocoHcmJmHODEb3cLWRjUULksuLPcY94EieiPcgUsJcRGwWhZl1RGNIO6pGb1I++u1IpCLgpuHESL6tB1pn71ObhKfAxI6wX6k158coUR6LIMxuO4mMNawQQbPKfpZFoO4j53uzuCvAb6aJzmQfg7+OZUmocowQVcrnaTiDXuZ5N+aFWOdRBvKzZFUoi8XyADchfD4GYuLfhhcd7YafuBah7srOe4NFQwkMmsSF06e2ZrJD80TWg0khhzojK+Echeiu/Zsur4FrcASLYEvUYnTzNZJzxs3SpqQARrYbyfB7+cYL96yQOcr9ZkS7ow== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: commit f4cb78af91e3 ("mm: add system wide stats items category") and commit 9d8573111024 ("mm: don't account memmap per-node") renamed NR_VM_WRITEBACK_STAT_ITEMS to NR_VM_STAT_ITEMS to track memmap/memmap_boot pages system wide. Extend the implementation so that the system wide page statistics can be tracked using a generic interface. This patch is in preparation for the next patch which adds a rarely modified system wide vmstat. Note that this implementation uses global atomic fields with no per-cpu optimizations as the existing usecase (memmap pages) is rarely modified as well. Signed-off-by: Kinsey Ho --- include/linux/vmstat.h | 8 ++++++++ mm/vmstat.c | 32 +++++++++++++++++++++++--------- 2 files changed, 31 insertions(+), 9 deletions(-) diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h index d2761bf8ff32..ac4d42c4fabd 100644 --- a/include/linux/vmstat.h +++ b/include/linux/vmstat.h @@ -145,6 +145,11 @@ extern atomic_long_t vm_zone_stat[NR_VM_ZONE_STAT_ITEMS]; extern atomic_long_t vm_node_stat[NR_VM_NODE_STAT_ITEMS]; extern atomic_long_t vm_numa_event[NR_VM_NUMA_EVENT_ITEMS]; +/* + * Global page accounting (no per cpu differentials). + */ +extern atomic_long_t vm_global_stat[NR_VM_STAT_ITEMS]; + #ifdef CONFIG_NUMA static inline void zone_numa_event_add(long x, struct zone *zone, enum numa_stat_item item) @@ -491,6 +496,9 @@ static inline void node_stat_sub_folio(struct folio *folio, mod_node_page_state(folio_pgdat(folio), item, -folio_nr_pages(folio)); } +void mod_global_page_state(enum vm_stat_item item, long nr); +unsigned long global_page_state(enum vm_stat_item item); + extern const char * const vmstat_text[]; static inline const char *zone_stat_name(enum zone_stat_item item) diff --git a/mm/vmstat.c b/mm/vmstat.c index 22a294556b58..e5a6dd5106c2 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -161,9 +161,11 @@ void vm_events_fold_cpu(int cpu) */ atomic_long_t vm_zone_stat[NR_VM_ZONE_STAT_ITEMS] __cacheline_aligned_in_smp; atomic_long_t vm_node_stat[NR_VM_NODE_STAT_ITEMS] __cacheline_aligned_in_smp; +atomic_long_t vm_global_stat[NR_VM_STAT_ITEMS] __cacheline_aligned_in_smp; atomic_long_t vm_numa_event[NR_VM_NUMA_EVENT_ITEMS] __cacheline_aligned_in_smp; EXPORT_SYMBOL(vm_zone_stat); EXPORT_SYMBOL(vm_node_stat); +EXPORT_SYMBOL(vm_global_stat); #ifdef CONFIG_NUMA static void fold_vm_zone_numa_events(struct zone *zone) @@ -1033,22 +1035,34 @@ unsigned long node_page_state(struct pglist_data *pgdat, } #endif +void mod_global_page_state(enum vm_stat_item item, long nr) +{ + atomic_long_add(nr, &vm_global_stat[item]); +} + +unsigned long global_page_state(enum vm_stat_item item) +{ + long x = atomic_long_read(&vm_global_stat[item]); +#ifdef CONFIG_SMP + if (x < 0) + x = 0; +#endif + return x; +} + /* * Count number of pages "struct page" and "struct page_ext" consume. - * nr_memmap_boot_pages: # of pages allocated by boot allocator - * nr_memmap_pages: # of pages that were allocated by buddy allocator + * NR_MEMMAP_BOOT_PAGES: # of pages allocated by boot allocator + * NR_MEMMAP_PAGES: # of pages that were allocated by buddy allocator */ -static atomic_long_t nr_memmap_boot_pages = ATOMIC_LONG_INIT(0); -static atomic_long_t nr_memmap_pages = ATOMIC_LONG_INIT(0); - void memmap_boot_pages_add(long delta) { - atomic_long_add(delta, &nr_memmap_boot_pages); + mod_global_page_state(NR_MEMMAP_BOOT_PAGES, delta); } void memmap_pages_add(long delta) { - atomic_long_add(delta, &nr_memmap_pages); + mod_global_page_state(NR_MEMMAP_PAGES, delta); } #ifdef CONFIG_COMPACTION @@ -1880,8 +1894,8 @@ static void *vmstat_start(struct seq_file *m, loff_t *pos) global_dirty_limits(v + NR_DIRTY_BG_THRESHOLD, v + NR_DIRTY_THRESHOLD); - v[NR_MEMMAP_PAGES] = atomic_long_read(&nr_memmap_pages); - v[NR_MEMMAP_BOOT_PAGES] = atomic_long_read(&nr_memmap_boot_pages); + for (int i = NR_MEMMAP_PAGES; i < NR_VM_STAT_ITEMS; i++) + v[i] = global_page_state(i); v += NR_VM_STAT_ITEMS; #ifdef CONFIG_VM_EVENT_COUNTERS