From patchwork Wed Nov 27 02:57:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yuanchu Xie X-Patchwork-Id: 13886509 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 318C3D66BAD for ; Wed, 27 Nov 2024 02:57:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 153418D0002; Tue, 26 Nov 2024 21:57:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 104FE8D0001; Tue, 26 Nov 2024 21:57:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E52068D0002; Tue, 26 Nov 2024 21:57:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C4B608D0001 for ; Tue, 26 Nov 2024 21:57:51 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 582DA4164E for ; Wed, 27 Nov 2024 02:57:51 +0000 (UTC) X-FDA: 82830365100.28.52A4C8B Received: from mail-oa1-f74.google.com (mail-oa1-f74.google.com [209.85.160.74]) by imf04.hostedemail.com (Postfix) with ESMTP id 34D7340016 for ; Wed, 27 Nov 2024 02:57:43 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=nNovH3Di; spf=pass (imf04.hostedemail.com: domain of 3rIpGZwcKCB0RN3G5AN9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--yuanchu.bounces.google.com designates 209.85.160.74 as permitted sender) smtp.mailfrom=3rIpGZwcKCB0RN3G5AN9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--yuanchu.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732676264; a=rsa-sha256; cv=none; b=UKdPufD5YhXHsU3pUlHT5s83lXOaz2dPydlV5aloPyv9pubIp1pDMcM/Rl5doUiuejbSxG MaJXiSYxGuqIRl04mZ77ZYvp3d3rFVnJp6xkqgGeE/yqMa3YL1wrnr2by4dTMuNrLhYY5P 37vimVsoS8myEtkegCfFVr6Yu0FMkqI= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=nNovH3Di; spf=pass (imf04.hostedemail.com: domain of 3rIpGZwcKCB0RN3G5AN9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--yuanchu.bounces.google.com designates 209.85.160.74 as permitted sender) smtp.mailfrom=3rIpGZwcKCB0RN3G5AN9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--yuanchu.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732676264; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jJsIXJ522dse8Oa1czRmZ0i3siB+gYc3GobTJqrZlLM=; b=L5zkmeyexU5nTThlBl7AFOByUX6x+23+wa1DLDbs0jw9lA83PvObPRLrKCw6Q4dGL8YZvw Cce9QZkDCOBOmeLHE4eisHGUwwjYB757jzWzVAz0BtH3yIs7Y07XvjuHSiTF/B9uSym8Pa 8sRn8f/wu8LPkkup3X3ZvRmVgshPE48= Received: by mail-oa1-f74.google.com with SMTP id 586e51a60fabf-2969ecd4315so4690290fac.3 for ; Tue, 26 Nov 2024 18:57:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732676268; x=1733281068; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=jJsIXJ522dse8Oa1czRmZ0i3siB+gYc3GobTJqrZlLM=; b=nNovH3DiiOVid7okcpjgzoNdRrDAnJWRBgfmvcGz7n4/aheQJoE6CC+98jsVnWtVDk MilySeJ3iFvIBfs/vx8SPsdcCGHbRThzlqMzA5AD1lssK4V7QNpa8bWnoMyXilm9iquR sN0Wn2Ds6A2IKZTqOjonhLLawgLBbprqmy/0blsn2E5AyuXLvaaV6tlOIcMFj8y2wCH3 gB3OSaJiaC3PceMWIBHMjN+eC9okWA5byW2+B3uf3CeJaoElLSCSG17kQFOlCl5zifc/ wyDcEgtUzJFfDrHBX9H4gcoq7u+G/rgDVh/m18YlwW3HuyM3KeSBqVoN70wMeLTFEbEd X+rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732676268; x=1733281068; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=jJsIXJ522dse8Oa1czRmZ0i3siB+gYc3GobTJqrZlLM=; b=BgzSu237neeFvzbFushElCm7w4sOrqDtW8K/xLCC/rZbFtaJR503NK0qjbUxxomgA9 HtlB2O5KcwNlUOkdSLLW/Xp7IEI/WLpIp6b1Z/bABCDvGgMaCYo0JF8MSjxLuhbVopvA f7gXJuVvdsgoNlsdN6NmVzJewoVFw7vulnei9FGU7uOXQph8B4qhKUJceMpobB1Resdc W4Sb/jgHAtiKsKADGx7SSEigAxPmUACkCPu/H78gn8DAd9nA4whWK5orvls/SFYAa5fQ nXJUPEnPlDlUH3Gkcto2TxmRjT2D8uxrZp9wkOJ1vfHYythYSrFFs6IxTEvZBuSjwGlN Ppyg== X-Forwarded-Encrypted: i=1; AJvYcCW0t9WLuNb7Zc7bCcqywAeaYWgFSMYLAe0k1H3SfpLLiKe8ICMBmTCGDPFmzxHjfIYyCQq9eHoIoQ==@kvack.org X-Gm-Message-State: AOJu0Yz6MqSqnHaywe4YDvYNreUgJlz9cIbdXVdcqHeu6F0yr/HdBPJ6 I+UQtgmk9MrZXPEdtWfUBOk0XjF6OoV50wau4HY7aupL46vs3rccD7uGGLnD7eCOYGFloW3Nlm0 xhiTJTA== X-Google-Smtp-Source: AGHT+IHpxh9NC9fR/Y+VVMw56gHiKhQb3M845RK+LO+TsitSupQfuVS+RxLJBMoZkNtQHfRh61ZaI/968chM X-Received: from oabxa1.prod.google.com ([2002:a05:6870:7f01:b0:295:f4c4:2bdd]) (user=yuanchu job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6870:9a98:b0:297:27c6:d50f with SMTP id 586e51a60fabf-29dc3f9ee1emr1306226fac.2.1732676268691; Tue, 26 Nov 2024 18:57:48 -0800 (PST) Date: Tue, 26 Nov 2024 18:57:22 -0800 In-Reply-To: <20241127025728.3689245-1-yuanchu@google.com> Mime-Version: 1.0 References: <20241127025728.3689245-1-yuanchu@google.com> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241127025728.3689245-4-yuanchu@google.com> Subject: [PATCH v4 3/9] mm: report workingset during memory pressure driven scanning From: Yuanchu Xie To: Andrew Morton , David Hildenbrand , "Aneesh Kumar K.V" , Khalid Aziz , Henry Huang , Yu Zhao , Dan Williams , Gregory Price , Huang Ying , Lance Yang , Randy Dunlap , Muhammad Usama Anjum Cc: Tejun Heo , Johannes Weiner , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Jonathan Corbet , Greg Kroah-Hartman , "Rafael J. Wysocki" , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , " =?utf-8?q?Eugenio_P=C3=A9rez?= " , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Mike Rapoport , Shuah Khan , Christian Brauner , Daniel Watson , Yuanchu Xie , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux.dev, linux-mm@kvack.org, linux-kselftest@vger.kernel.org X-Rspamd-Queue-Id: 34D7340016 X-Stat-Signature: 5bmip8kqdqhe4zctj3zaoiz4fs4wugdc X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1732676263-942823 X-HE-Meta: U2FsdGVkX19vy1B7hcuclx2cbbKrrNwmfFR6voVs2yF/hlJXBw5Ab0deng6kvPHpHwGq/nmceZaw71NbfxOdi6GjnvL1s1QgrzsiMsOPuWQ9PApjj/XIXdNadgITaIikmqNvBeLlhD2dcgB00Hm8fzV1SMmcBtSfrvcucDP0oRrXvOZCMN57iKkqITyUyz877Ev0Gw+aRLCFW9Qou2gMdI9gHHr7B1mt3OYO5aNm4KX2c7pc7eN6ZagyBrazjfqiuKQVJYdoV9V8SLpaDpkLE7HWjVIbUqrUIZbNx2Y1O1gYDw3EWVSQwlg7oLAGhaD0c2GKW2kQTOMau1/6oHwSzURfm877IziVz3tuL80ee1XiZ0ZkYn9NY97xjcO0dL9+Y18IeV7chWAphZAlitytoKv/7TJnDdQeYP17RmNP7wt+rQIRWYrigDbmqW+oDJTsUmG4OTX+GF/t4+oueVU7k4Qx6J+xiEtZpl6hB+smAMcKccJLdGUFPgMzKo8a6Ls6W9YnOWS6lHnO2oUjNGBAmKyr6Q4A01zU3BEFd+iy/7Hhm0W6xTu1/AukHqTq2iJ52mHor7fWlc3zfnajN/D58JY022y3RlKVfwibGIG/40DipPRsHfGLfG+XIXdtOjrZVcQWupDJzvJx8BLmkTBKCxEFkk+Q8/3ihrYvm0QeB5B6V7oHDF6Os3K/uutYWZJozEl2CmwdGKm1nMBbtBr1suhYiJbYO2hvgKN938/G4GOKjQrhv+pdL3Iyd5VtAz8mXWR8ZGQz+TiGnyNQoxaTn82/foRyR7TaGWdYzHGI+Y4KNpicVjs3/UgrdaVYNRtve1PL1ZVOIvT6jYzdhOFcHWj11XeMaNurYkO2u/ypPb7ukU82QyEXDQUuQMczooChCUWzYKsrLQ0TedltiBLlaxAv1mflzCu3Qb/PKFGsUfs/VL3J+cs0m/rq/Q+fEnpDGyDEAFCyz6XXv6tJX6o xSmOD7M+ aGwWgJQaI4Xow1Uh2uztIRTt0/+542oTqKBlS81HcsJS/Y58FqRG7M5HDV0oXL3r0VQbUGoiF/xH6HXzPL3E9jTj00UggTApauqj4Erb2qSqTjZkg/WmMqvaA6axVbf7xOpZdDz8mvHLjMHUKovBb9PJS2enSHmwAcsMiQhcnFuuM5bwvTPq1iIlfmYQrwtzosBQZuzEOgCR2hqiQl+SQ6tShnPfIKE6Euwu+HQcXrG0M7eYEtSy2UofuC457c3pQyIpp6Vezgjhk/UNecAyAJf50aWURavo6v4lkIWrSnb3o5dWiFr7A1hbIAMR4cdH2DPzD8DOwgHgOz479K7JBDGNTr0DoqdmVUtp9t5CYt14Zgaizc4DDm097gkpnNilDpLu68MWljq0EWOQ+IJME8ZtbNBDeFX595rq9t8eRnpcsEUGcEN7kqmW2XacfB7sUAV1cEg46jboewY9lyMWtBEWcZK3/0EB210rjmFDp+pR+pCElQlWZVZ2t5Xb/v7XrVOYAloyLQQn50GUcojbZ2o+rVRetUIHfFKAzC9ijrFinGOU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When a node reaches its low watermarks and wakes up kswapd, notify all userspace programs waiting on the workingset page age histogram of the memory pressure, so a userspace agent can read the workingset report in time and make policy decisions, such as logging, oom-killing, or migration. Sysfs interface: /sys/devices/system/node/nodeX/workingset_report/report_threshold time in milliseconds that specifies how often the userspace agent can be notified for node memory pressure. Signed-off-by: Yuanchu Xie --- include/linux/workingset_report.h | 4 +++ mm/internal.h | 12 ++++++++ mm/vmscan.c | 46 +++++++++++++++++++++++++++++++ mm/workingset_report.c | 43 ++++++++++++++++++++++++++++- 4 files changed, 104 insertions(+), 1 deletion(-) diff --git a/include/linux/workingset_report.h b/include/linux/workingset_report.h index 8bae6a600410..2ec8b927b200 100644 --- a/include/linux/workingset_report.h +++ b/include/linux/workingset_report.h @@ -37,7 +37,11 @@ struct wsr_page_age_histo { }; struct wsr_state { + unsigned long report_threshold; unsigned long refresh_interval; + + struct kernfs_node *page_age_sys_file; + /* breakdown of workingset by page age */ struct mutex page_age_lock; struct wsr_page_age_histo *page_age; diff --git a/mm/internal.h b/mm/internal.h index bbd3c1501bac..508b7d9937d6 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -479,6 +479,18 @@ bool try_to_inc_max_seq(struct lruvec *lruvec, unsigned long seq, bool can_swap, bool force_scan); void set_task_reclaim_state(struct task_struct *task, struct reclaim_state *rs); +#ifdef CONFIG_WORKINGSET_REPORT +/* + * in mm/wsr.c + */ +void notify_workingset(struct mem_cgroup *memcg, struct pglist_data *pgdat); +#else +static inline void notify_workingset(struct mem_cgroup *memcg, + struct pglist_data *pgdat) +{ +} +#endif + /* * in mm/rmap.c: */ diff --git a/mm/vmscan.c b/mm/vmscan.c index 89da4d8dfb5f..2bca81271d15 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2578,6 +2578,15 @@ static bool can_age_anon_pages(struct pglist_data *pgdat, return can_demote(pgdat->node_id, sc); } +#ifdef CONFIG_WORKINGSET_REPORT +static void try_to_report_workingset(struct pglist_data *pgdat, struct scan_control *sc); +#else +static inline void try_to_report_workingset(struct pglist_data *pgdat, + struct scan_control *sc) +{ +} +#endif + #ifdef CONFIG_LRU_GEN #ifdef CONFIG_LRU_GEN_ENABLED @@ -4004,6 +4013,8 @@ static void lru_gen_age_node(struct pglist_data *pgdat, struct scan_control *sc) set_initial_priority(pgdat, sc); + try_to_report_workingset(pgdat, sc); + memcg = mem_cgroup_iter(NULL, NULL, NULL); do { struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); @@ -5649,6 +5660,38 @@ static int __init init_lru_gen(void) }; late_initcall(init_lru_gen); +#ifdef CONFIG_WORKINGSET_REPORT +static void try_to_report_workingset(struct pglist_data *pgdat, + struct scan_control *sc) +{ + struct mem_cgroup *memcg = sc->target_mem_cgroup; + struct wsr_state *wsr = &mem_cgroup_lruvec(memcg, pgdat)->wsr; + unsigned long threshold = READ_ONCE(wsr->report_threshold); + + if (sc->priority == DEF_PRIORITY) + return; + + if (!threshold) + return; + + if (!mutex_trylock(&wsr->page_age_lock)) + return; + + if (!wsr->page_age) { + mutex_unlock(&wsr->page_age_lock); + return; + } + + if (time_is_after_jiffies(wsr->page_age->timestamp + threshold)) { + mutex_unlock(&wsr->page_age_lock); + return; + } + + mutex_unlock(&wsr->page_age_lock); + notify_workingset(memcg, pgdat); +} +#endif /* CONFIG_WORKINGSET_REPORT */ + #else /* !CONFIG_LRU_GEN */ static void lru_gen_age_node(struct pglist_data *pgdat, struct scan_control *sc) @@ -6200,6 +6243,9 @@ static void shrink_zones(struct zonelist *zonelist, struct scan_control *sc) if (zone->zone_pgdat == last_pgdat) continue; last_pgdat = zone->zone_pgdat; + + if (!sc->proactive) + try_to_report_workingset(zone->zone_pgdat, sc); shrink_node(zone->zone_pgdat, sc); } diff --git a/mm/workingset_report.c b/mm/workingset_report.c index 8678536ccfc7..bbefb0046669 100644 --- a/mm/workingset_report.c +++ b/mm/workingset_report.c @@ -320,6 +320,33 @@ static struct wsr_state *kobj_to_wsr(struct kobject *kobj) return &mem_cgroup_lruvec(NULL, kobj_to_pgdat(kobj))->wsr; } +static ssize_t report_threshold_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + struct wsr_state *wsr = kobj_to_wsr(kobj); + unsigned int threshold = READ_ONCE(wsr->report_threshold); + + return sysfs_emit(buf, "%u\n", jiffies_to_msecs(threshold)); +} + +static ssize_t report_threshold_store(struct kobject *kobj, + struct kobj_attribute *attr, + const char *buf, size_t len) +{ + unsigned int threshold; + struct wsr_state *wsr = kobj_to_wsr(kobj); + + if (kstrtouint(buf, 0, &threshold)) + return -EINVAL; + + WRITE_ONCE(wsr->report_threshold, msecs_to_jiffies(threshold)); + + return len; +} + +static struct kobj_attribute report_threshold_attr = + __ATTR_RW(report_threshold); + static ssize_t refresh_interval_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { @@ -474,6 +501,7 @@ static ssize_t page_age_show(struct kobject *kobj, struct kobj_attribute *attr, static struct kobj_attribute page_age_attr = __ATTR_RO(page_age); static struct attribute *workingset_report_attrs[] = { + &report_threshold_attr.attr, &refresh_interval_attr.attr, &page_age_intervals_attr.attr, &page_age_attr.attr, @@ -495,8 +523,13 @@ void wsr_init_sysfs(struct node *node) wsr = kobj_to_wsr(kobj); - if (sysfs_create_group(kobj, &workingset_report_attr_group)) + if (sysfs_create_group(kobj, &workingset_report_attr_group)) { pr_warn("Workingset report failed to create sysfs files\n"); + return; + } + + wsr->page_age_sys_file = + kernfs_walk_and_get(kobj->sd, "workingset_report/page_age"); } EXPORT_SYMBOL_GPL(wsr_init_sysfs); @@ -509,6 +542,14 @@ void wsr_remove_sysfs(struct node *node) return; wsr = kobj_to_wsr(kobj); + kernfs_put(wsr->page_age_sys_file); sysfs_remove_group(kobj, &workingset_report_attr_group); } EXPORT_SYMBOL_GPL(wsr_remove_sysfs); + +void notify_workingset(struct mem_cgroup *memcg, struct pglist_data *pgdat) +{ + struct wsr_state *wsr = &mem_cgroup_lruvec(memcg, pgdat)->wsr; + + kernfs_notify(wsr->page_age_sys_file); +}