From patchwork Wed Nov 27 02:57:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yuanchu Xie X-Patchwork-Id: 13886508 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D38CAD66BAE for ; Wed, 27 Nov 2024 02:57:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4FE106B00A0; Tue, 26 Nov 2024 21:57:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4AFE88D0002; Tue, 26 Nov 2024 21:57:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 326708D0001; Tue, 26 Nov 2024 21:57:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 12DFA6B00A0 for ; Tue, 26 Nov 2024 21:57:50 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id BBBA64167C for ; Wed, 27 Nov 2024 02:57:49 +0000 (UTC) X-FDA: 82830364806.03.7DDB76B Received: from mail-oa1-f74.google.com (mail-oa1-f74.google.com [209.85.160.74]) by imf25.hostedemail.com (Postfix) with ESMTP id 24BCCA0011 for ; Wed, 27 Nov 2024 02:57:43 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=VURwKIuT; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of 3qopGZwcKCBsPL1E38L7FF7C5.3FDC9ELO-DDBM13B.FI7@flex--yuanchu.bounces.google.com designates 209.85.160.74 as permitted sender) smtp.mailfrom=3qopGZwcKCBsPL1E38L7FF7C5.3FDC9ELO-DDBM13B.FI7@flex--yuanchu.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732676265; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5lhbaogLbZu3E/Vkjc13k2hWVq6P2Y+49H1lvwk45sc=; b=W8Rj7Wba7sWU6TYpjsWFSK5WR9wGceIp5QwQvNbF0gRxC7WLGijXNAG2E21i+klCOYtCdj A6X7B32/UWBbQhapGhWfroeDij5uTyP0duMztK8YK3STqrJLWWBBouRSdZnR67laKN0V8z za2a7kx+YhrMai4MXXe3VU0QD4fCtKg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732676265; a=rsa-sha256; cv=none; b=bo3iEX2/XvhUzuhPDcoB09/QFXRrvfNVsmM423djJbc7bURveVpLHCTsdygPHlyVzg1SnI mJy1uWa2SMp7W6TvzYZkGEZwVofoW627u4qC7nkRM0xzEZk6GpCBC4yubG3orF7g564XAn pFa63PtKZAHkEJyF81WKExcC8TYgQ+U= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=VURwKIuT; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of 3qopGZwcKCBsPL1E38L7FF7C5.3FDC9ELO-DDBM13B.FI7@flex--yuanchu.bounces.google.com designates 209.85.160.74 as permitted sender) smtp.mailfrom=3qopGZwcKCBsPL1E38L7FF7C5.3FDC9ELO-DDBM13B.FI7@flex--yuanchu.bounces.google.com Received: by mail-oa1-f74.google.com with SMTP id 586e51a60fabf-2975e172b3bso2353302fac.1 for ; Tue, 26 Nov 2024 18:57:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732676267; x=1733281067; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=5lhbaogLbZu3E/Vkjc13k2hWVq6P2Y+49H1lvwk45sc=; b=VURwKIuTZH5BbQre9hEUI+eZmIAmetlPkchXhi/nvzy+QGcLq1BCTPSK2peIjI4ExX neZ2TKzgmUTqDyzefjs6MgpKriRh/1tA1/LnwyoQcdz+qzuun2uNGWPhZAg87ImrbE8e Uplb0gkM2pPxje2Cb80XN/dNhFxikYDhWhOIjLX+fGYo4SKECF6JCD8tsMqXANmiO0jB frcki8tj8b537WXU8f2sUH2PgdI5AV3n5/vqe9tu7KewwwTddcamC49XGEh83zQirfbk pktV2geQqf3+Pxpp8OEF1N/N1rWPDEZwPwDhdWA2ieRyHM6AmNMtTLYC58VYv2N28q7N bezg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732676267; x=1733281067; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=5lhbaogLbZu3E/Vkjc13k2hWVq6P2Y+49H1lvwk45sc=; b=s0y6jxYTkbVkY/P6UpMcxFKljL+XAlxwZGdb5J74qg5SnI3y+Y4U2jFhVgNiDhHx2E tM8TCGCQycaiXVhzSpitMqV6dFc0OCDHf4/rH7k3+e3hCEYrnpiu01rdxNpcgmwlHOaS V6n3uPrlbsOyVBd3kVP0GgRQdBdCAAyQR5hr4jpw8Wuq7HXF4NMxOj4NMiUHyoL4hBzt aRmjrdf7kW2XxjJ6LinqMOcoCdCv5X2PzwDJ1tc4CzdRctujocWMHKR2xvXBCdYOOZnP aF/Yzri63ZHCUShNbgI6+CqQh/SRqqdU5XjnjogAfitjLJUgW0ezBtw+Dsx4wete83ac rjqA== X-Forwarded-Encrypted: i=1; AJvYcCXdC/87eUpcupxN9Lb93GZBJrA2AtGh9NZvw/kIa6Iy4bUB3qD7mmh6zkLj55qrpNwsosh660OfBQ==@kvack.org X-Gm-Message-State: AOJu0YzxcDBk+CAAd8fSdHYc/Qi0Zf1uW4Mutxibl6lUWSgt2z03e/ZX eQ+xOkRkGPShp/8LhkhgL92texEoDV7sd2u5m0Hc4HQW9zzRwG+x6VbZnN3N1luWsugyRu0CO/r p61cRuw== X-Google-Smtp-Source: AGHT+IEx9U6wR1hCMpTboiP7q8J5ECqe/TnGoNjy6O4ISdQoe4PI0qJfC/k0oduc+Ab8ibmO8UdzD0uyW92D X-Received: from oabqq6.prod.google.com ([2002:a05:6870:ef06:b0:296:e2da:79c8]) (user=yuanchu job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6871:a002:b0:296:a7c7:6a08 with SMTP id 586e51a60fabf-29dc4012ca6mr1026883fac.15.1732676266733; Tue, 26 Nov 2024 18:57:46 -0800 (PST) Date: Tue, 26 Nov 2024 18:57:21 -0800 In-Reply-To: <20241127025728.3689245-1-yuanchu@google.com> Mime-Version: 1.0 References: <20241127025728.3689245-1-yuanchu@google.com> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241127025728.3689245-3-yuanchu@google.com> Subject: [PATCH v4 2/9] mm: use refresh interval to rate-limit workingset report aggregation From: Yuanchu Xie To: Andrew Morton , David Hildenbrand , "Aneesh Kumar K.V" , Khalid Aziz , Henry Huang , Yu Zhao , Dan Williams , Gregory Price , Huang Ying , Lance Yang , Randy Dunlap , Muhammad Usama Anjum Cc: Tejun Heo , Johannes Weiner , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Jonathan Corbet , Greg Kroah-Hartman , "Rafael J. Wysocki" , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , " =?utf-8?q?Eugenio_P=C3=A9rez?= " , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Mike Rapoport , Shuah Khan , Christian Brauner , Daniel Watson , Yuanchu Xie , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux.dev, linux-mm@kvack.org, linux-kselftest@vger.kernel.org X-Stat-Signature: wnfri1ajwi4fbk8s34te9573brrxtf59 X-Rspamd-Queue-Id: 24BCCA0011 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1732676263-677679 X-HE-Meta: U2FsdGVkX1+V5pingA1CJPmr/jBj/rQEG95KPjgeeThumoIhF7fFypoBkcCFILmGxU9sVmtSIm649LmewPty/TJtVdxEchd3zYyyIA447uM7beZxRcMIQhyd2bfbjwDdoT0xD5AOvPLZ6veubM88RyEY/n+fHRD0bnoK4FlakGlfpsn+ldJ6VZjWlwwhH4iYWpxMdAo8eusVwngmIpxngCsO323D7Wk5v+ytuawWz+sbllA0SIEH4HeGonSq1hcGBJnVNu/rmQFORiGwXCt3lXU/CYu/g3hUYgpp/s1EwzOuqAxgNOszO5Es+4azGBnHLTCSSiw044DNvbr2RQELGU66CrXMPcgJtHmUzxexjUKTBW7Ge3bDH3clnq0SLiKTK70n1wO6t+zDV/qdKVjlSrIjiP1jetQbKu99ArGHqXH2kbCt+mZZCvFhhKByUhiFuR3vl79VcBnIuYzneGhoS1op7qqQmGX03dXShdJdXB+SLqXlfDs5sQ+fxI1qSDHFVCbBS/meg7ECOriJsW+1lGFAAOP9DN8R1s4i+3JhJiDWrXTIUvo7j/l1yUXAtdUfhk3bbzO4rjCjGPfIhn8gY6RawwwQpZLhUY1dv/qQrCwb0M7LUBmibgIaN6JhKlxXDcr2YnLqmwOOdEmrfJtK+4AxSahAIIr9JFCa6vQYSk51u5VRPyQQqypObpqEdqrHPorZDySzNYv1DAvbHTIzGrWdb8ZSfdXO4N73sz3Y4N233Ej/G1sowTGgImxDgaHijhMSMnDfrSk2/wf7oMXureAjqK0RpvebBCb4ORAzM7AeG3GUrd1TO3UdHYEXGNDZ9rce4puRL4/k3wZ73uBydYp88BI8EP5555upWlfkxfY3K+gl532LB+EbUTGTaetLo2TlkYliJVC295prXq7fGBmVIAc0X11Z8RGbr/OUW1ndf4mHQ8IUk8sCLItxGTyLgR5lwUqNzzZZ41oZTsQ oGdJ0R9U WddNFCH5rA1YBGuR5bA/CuUMPhPlTVK+MGFAIWjI6Kh78U3evLaCJ2uSxC9jw3tMk1gMYMdU/Bru0Jz2TbvJEBAM9IKxI6JkDQvm2a2rCV8m95Em5sQB0zLclheuVWfvEFT7ULNbug3ZlBJORon74JIetDjczKVlQVFTyXUCtWFUil89xJkXOLYv0qx5CdKwZAQerg8oeZTjugLOm125jeTk1L6bx1MjtzCUIjw1biyBp8c8GRIbvEF3WBfFioUSwn+BbsiNVvuQisB7kH3cpgciKZND5hhWdB/tqokd+JnhlEOKgHS0aIKjrr8fp1M72TFrYj5hmHARZB+5m9x5/FvkqaU+jMy2FRbEwADOWYRHvD63X3BMNFHl1nIHQOd/sUutBwtGG8UCcjjsFrNIZ6pBFOrG0mAWvWcAANj8gthY5DTk/l245yNREMyP5auiAOHpeA5mNRKILtRzk/J8QEzE1ShFzXy1CFMJFa9LrqTHzxR43oZP3i7p8nNUIEOWCT6oN01LxpGUJ+p6y35K9lDbA4WR+qBlb+e3rKiwg1ar1544m5th/dh5D8w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The refresh interval is a rate limiting factor to workingset page age histogram reads. When a workingset report is generated, the oldest timestamp of all the lruvecs is stored as the timestamp of the report. The same report will be read until the report expires beyond the refresh interval, at which point a new report is generated. Sysfs interface /sys/devices/system/node/nodeX/workingset_report/refresh_interval time in milliseconds specifying how long the report is valid for Signed-off-by: Yuanchu Xie --- include/linux/workingset_report.h | 1 + mm/workingset_report.c | 101 ++++++++++++++++++++++++------ 2 files changed, 83 insertions(+), 19 deletions(-) diff --git a/include/linux/workingset_report.h b/include/linux/workingset_report.h index d7c2ee14ec87..8bae6a600410 100644 --- a/include/linux/workingset_report.h +++ b/include/linux/workingset_report.h @@ -37,6 +37,7 @@ struct wsr_page_age_histo { }; struct wsr_state { + unsigned long refresh_interval; /* breakdown of workingset by page age */ struct mutex page_age_lock; struct wsr_page_age_histo *page_age; diff --git a/mm/workingset_report.c b/mm/workingset_report.c index a4dcf62fcd96..8678536ccfc7 100644 --- a/mm/workingset_report.c +++ b/mm/workingset_report.c @@ -174,9 +174,11 @@ static void collect_page_age_type(const struct lru_gen_folio *lrugen, * Assume the heuristic that pages are in the MGLRU generation * through uniform accesses, so we can aggregate them * proportionally into bins. + * + * Returns the timestamp of the youngest gen in this lruvec. */ -static void collect_page_age(struct wsr_page_age_histo *page_age, - const struct lruvec *lruvec) +static unsigned long collect_page_age(struct wsr_page_age_histo *page_age, + const struct lruvec *lruvec) { int type; const struct lru_gen_folio *lrugen = &lruvec->lrugen; @@ -191,11 +193,14 @@ static void collect_page_age(struct wsr_page_age_histo *page_age, for (type = 0; type < ANON_AND_FILE; type++) collect_page_age_type(lrugen, bin, max_seq, min_seq[type], curr_timestamp, type); + + return READ_ONCE(lruvec->lrugen.timestamps[lru_gen_from_seq(max_seq)]); } /* First step: hierarchically scan child memcgs. */ static void refresh_scan(struct wsr_state *wsr, struct mem_cgroup *root, - struct pglist_data *pgdat) + struct pglist_data *pgdat, + unsigned long refresh_interval) { struct mem_cgroup *memcg; unsigned int flags; @@ -208,12 +213,15 @@ static void refresh_scan(struct wsr_state *wsr, struct mem_cgroup *root, do { struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); unsigned long max_seq = READ_ONCE((lruvec)->lrugen.max_seq); + int gen = lru_gen_from_seq(max_seq); + unsigned long birth = READ_ONCE(lruvec->lrugen.timestamps[gen]); /* * setting can_swap=true and force_scan=true ensures * proper workingset stats when the system cannot swap. */ - try_to_inc_max_seq(lruvec, max_seq, true, true); + if (time_is_before_jiffies(birth + refresh_interval)) + try_to_inc_max_seq(lruvec, max_seq, true, true); cond_resched(); } while ((memcg = mem_cgroup_iter(root, memcg, NULL))); @@ -228,6 +236,7 @@ static void refresh_aggregate(struct wsr_page_age_histo *page_age, { struct mem_cgroup *memcg; struct wsr_report_bin *bin; + unsigned long oldest_lruvec_time = jiffies; for (bin = page_age->bins; bin->idle_age != WORKINGSET_INTERVAL_MAX; bin++) { @@ -241,11 +250,15 @@ static void refresh_aggregate(struct wsr_page_age_histo *page_age, memcg = mem_cgroup_iter(root, NULL, NULL); do { struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); + unsigned long lruvec_time = + collect_page_age(page_age, lruvec); + + if (time_before(lruvec_time, oldest_lruvec_time)) + oldest_lruvec_time = lruvec_time; - collect_page_age(page_age, lruvec); cond_resched(); } while ((memcg = mem_cgroup_iter(root, memcg, NULL))); - WRITE_ONCE(page_age->timestamp, jiffies); + WRITE_ONCE(page_age->timestamp, oldest_lruvec_time); } static void copy_node_bins(struct pglist_data *pgdat, @@ -270,17 +283,25 @@ bool wsr_refresh_report(struct wsr_state *wsr, struct mem_cgroup *root, struct pglist_data *pgdat) { struct wsr_page_age_histo *page_age; + unsigned long refresh_interval = READ_ONCE(wsr->refresh_interval); if (!READ_ONCE(wsr->page_age)) return false; - refresh_scan(wsr, root, pgdat); + if (!refresh_interval) + return false; + mutex_lock(&wsr->page_age_lock); page_age = READ_ONCE(wsr->page_age); - if (page_age) { - copy_node_bins(pgdat, page_age); - refresh_aggregate(page_age, root, pgdat); - } + if (!page_age) + goto unlock; + if (page_age->timestamp && + time_is_after_jiffies(page_age->timestamp + refresh_interval)) + goto unlock; + refresh_scan(wsr, root, pgdat, refresh_interval); + copy_node_bins(pgdat, page_age); + refresh_aggregate(page_age, root, pgdat); +unlock: mutex_unlock(&wsr->page_age_lock); return !!page_age; } @@ -299,6 +320,52 @@ static struct wsr_state *kobj_to_wsr(struct kobject *kobj) return &mem_cgroup_lruvec(NULL, kobj_to_pgdat(kobj))->wsr; } +static ssize_t refresh_interval_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + struct wsr_state *wsr = kobj_to_wsr(kobj); + unsigned int interval = READ_ONCE(wsr->refresh_interval); + + return sysfs_emit(buf, "%u\n", jiffies_to_msecs(interval)); +} + +static ssize_t refresh_interval_store(struct kobject *kobj, + struct kobj_attribute *attr, + const char *buf, size_t len) +{ + unsigned int interval; + int err; + struct wsr_state *wsr = kobj_to_wsr(kobj); + + err = kstrtouint(buf, 0, &interval); + if (err) + return err; + + mutex_lock(&wsr->page_age_lock); + if (interval && !wsr->page_age) { + struct wsr_page_age_histo *page_age = + kzalloc(sizeof(struct wsr_page_age_histo), GFP_KERNEL); + + if (!page_age) { + err = -ENOMEM; + goto unlock; + } + wsr->page_age = page_age; + } + if (!interval && wsr->page_age) { + kfree(wsr->page_age); + wsr->page_age = NULL; + } + + WRITE_ONCE(wsr->refresh_interval, msecs_to_jiffies(interval)); +unlock: + mutex_unlock(&wsr->page_age_lock); + return err ?: len; +} + +static struct kobj_attribute refresh_interval_attr = + __ATTR_RW(refresh_interval); + static ssize_t page_age_intervals_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { @@ -382,13 +449,6 @@ static ssize_t page_age_show(struct kobject *kobj, struct kobj_attribute *attr, int ret = 0; struct wsr_state *wsr = kobj_to_wsr(kobj); - - mutex_lock(&wsr->page_age_lock); - if (!wsr->page_age) - wsr->page_age = - kzalloc(sizeof(struct wsr_page_age_histo), GFP_KERNEL); - mutex_unlock(&wsr->page_age_lock); - wsr_refresh_report(wsr, NULL, kobj_to_pgdat(kobj)); mutex_lock(&wsr->page_age_lock); @@ -414,7 +474,10 @@ static ssize_t page_age_show(struct kobject *kobj, struct kobj_attribute *attr, static struct kobj_attribute page_age_attr = __ATTR_RO(page_age); static struct attribute *workingset_report_attrs[] = { - &page_age_intervals_attr.attr, &page_age_attr.attr, NULL + &refresh_interval_attr.attr, + &page_age_intervals_attr.attr, + &page_age_attr.attr, + NULL }; static const struct attribute_group workingset_report_attr_group = {