From patchwork Wed Nov 27 02:57:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yuanchu Xie X-Patchwork-Id: 13886510 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86E06D66BAC for ; Wed, 27 Nov 2024 02:57:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 524F38D0003; Tue, 26 Nov 2024 21:57:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 45F268D0001; Tue, 26 Nov 2024 21:57:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D9808D0003; Tue, 26 Nov 2024 21:57:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 079508D0001 for ; Tue, 26 Nov 2024 21:57:54 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id B0AAC120DBA for ; Wed, 27 Nov 2024 02:57:53 +0000 (UTC) X-FDA: 82830365100.15.B0839C5 Received: from mail-ot1-f73.google.com (mail-ot1-f73.google.com [209.85.210.73]) by imf04.hostedemail.com (Postfix) with ESMTP id 84EF240012 for ; Wed, 27 Nov 2024 02:57:45 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=smvHYIUM; spf=pass (imf04.hostedemail.com: domain of 3ropGZwcKCB8TP5I7CPBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--yuanchu.bounces.google.com designates 209.85.210.73 as permitted sender) smtp.mailfrom=3ropGZwcKCB8TP5I7CPBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--yuanchu.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732676269; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nc9u6Gr3t/eex7ob4WWy8qTSgpDGth3PA2eslkY0ahg=; b=fv0NGqY62dMzqyku+ZQ1qmShcLoxGP46ZYsH/VC9eJ4tb/xzMkUw8uRshKDf4uPtfFcypz sK59UWgwk8Gb1RIUjDHEHm74SFz0HJaVOKXoPa+2D5ApY6lmGH86EjssPxBkdjz4O3oYDZ KcRVGjSBTMp6PzhN8Vfk0pzRAQlPlCc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732676269; a=rsa-sha256; cv=none; b=DHzpk/zP4ZTiAYOle8LMIvYzQ1M7CR1O2XUHXJZh/UQepRKTOKH1KKJjW0krAwdP9bNvMG RAgYlYcXwWS4Z3zc0AhyzBZndgH+nQtKrs6NdLvzOkhkLLpoZUlNtOs9fSBtDegk5PxH8o 1Scjg7EcCV1oPMTIE0syctAAeWUapnQ= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=smvHYIUM; spf=pass (imf04.hostedemail.com: domain of 3ropGZwcKCB8TP5I7CPBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--yuanchu.bounces.google.com designates 209.85.210.73 as permitted sender) smtp.mailfrom=3ropGZwcKCB8TP5I7CPBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--yuanchu.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-ot1-f73.google.com with SMTP id 46e09a7af769-71d557437e0so1959463a34.3 for ; Tue, 26 Nov 2024 18:57:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732676271; x=1733281071; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=nc9u6Gr3t/eex7ob4WWy8qTSgpDGth3PA2eslkY0ahg=; b=smvHYIUM4wHBpMM+/9JXfp4R+PFVKtRvFT48UuF4L9MKY+7uQB+pCvcpj+wePh1/9E I2gbzjejVxBGyQY2TsnfqgbC6BjY0SheRgEUPUh7LSG/N148WO89DMdxv7G09aQbllYg GtaC1ZYKPtuEB4+EQETM8MirEn4wHYxeGheBDxxHwhs6jisiwh1nh0Ru9XilOjD2Hs7P 1dmOmhnIBWvQFKGEpuKT262Ri15WvprM8oh1iivi2zPYAMk3vTLxZLiRhlWh2VRssOeK 16UIRr7oyah5E7xTYjsQX9cDbzI58+1MJaNZ5sa7zSocbDr/UTndnwxB4Kt4NKOMyXrD z7Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732676271; x=1733281071; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=nc9u6Gr3t/eex7ob4WWy8qTSgpDGth3PA2eslkY0ahg=; b=ZegN7oDURx1kpNrQuRu7A4Lnmdqnp8JfLj7V9P9zX2kMhO3xwjmd7MMlG/mfcPFlYx aR9UcuEYkMQE0XU7FFRXEqRQj1YNCvJ2SDTtrk+dkKV4dt2+sW4DkQ7lSX+3VVxiz/mS cdYe2eVAOLWe1qh93g8v0lxvYcQ/WEZsQ28ioH6cDAdZsU9UrBklLVQU7nIW0BIch2sn Szi1Ji6qFhTKQHJHSW73zUz8DS8WdHVIiXHU8HgdPeoQPmiRYi52p5sCbwfsjsaV29g9 d9T5g16a+L50Bf8McvaRHpoky7WCLLGIOa8cl8zuVgCIS1V3EAq1zMfdY8bJ7hlJ0vA2 XkkQ== X-Forwarded-Encrypted: i=1; AJvYcCWXXj6S1pJ1L+l8JXbaAWqJxDRdWAx8+bOEYcSJFaO0lp3B3ZOnVTDYWxfzZ71+BLMHhk0oARJhtg==@kvack.org X-Gm-Message-State: AOJu0YzDRGJh5r3KK0NYhC0GTwrVPIw6tVLH5bUMuY3Ah9hAL0TFm6+b R2UowvmvVRff+JgsxGkAYmDXTYPHlRIR+s5hWufOh/sNp38v5tWlR2Apnbngp8QuL12q1QfYIi4 FmhcDMw== X-Google-Smtp-Source: AGHT+IHgj0Am4IyS+M4OwpOyz4MtxJ9EoZ2BKJ1SYuM2EnZPBNIt4U8MU/D28Yi7V0kFKazRJu4A5wWgVUd3 X-Received: from pgbcz4.prod.google.com ([2002:a05:6a02:2304:b0:7fc:26c4:ff6c]) (user=yuanchu job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6359:4ca0:b0:1ca:9e4a:d06 with SMTP id e5c5f4694b2df-1cab15f4bd5mr150435955d.13.1732676270571; Tue, 26 Nov 2024 18:57:50 -0800 (PST) Date: Tue, 26 Nov 2024 18:57:23 -0800 In-Reply-To: <20241127025728.3689245-1-yuanchu@google.com> Mime-Version: 1.0 References: <20241127025728.3689245-1-yuanchu@google.com> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241127025728.3689245-5-yuanchu@google.com> Subject: [PATCH v4 4/9] mm: extend workingset reporting to memcgs From: Yuanchu Xie To: Andrew Morton , David Hildenbrand , "Aneesh Kumar K.V" , Khalid Aziz , Henry Huang , Yu Zhao , Dan Williams , Gregory Price , Huang Ying , Lance Yang , Randy Dunlap , Muhammad Usama Anjum Cc: Tejun Heo , Johannes Weiner , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Jonathan Corbet , Greg Kroah-Hartman , "Rafael J. Wysocki" , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , " =?utf-8?q?Eugenio_P=C3=A9rez?= " , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Mike Rapoport , Shuah Khan , Christian Brauner , Daniel Watson , Yuanchu Xie , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux.dev, linux-mm@kvack.org, linux-kselftest@vger.kernel.org X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 84EF240012 X-Stat-Signature: 7ggsngdqk8recq7t7gg9d7zymh33y5hs X-Rspam-User: X-HE-Tag: 1732676265-541682 X-HE-Meta: U2FsdGVkX19KOo0bP4XQSvHy8xi875Cymoj11aCkEitIoL4yhQGWDJaiSlOF+jc2uXVkku5rJ9rw0fIjy2HcsfOZ/DV1K1mRR8u8hU7vjTpiK0KWIZ2PSu8arakWDuyeQKsde5E/rjBo06p3cOZQYpzUMityqYJGm8Cq6J1ypuVIJsDTpqUn8Xb9q/5sXPLQ+xa+jcrIW3rVHNr6BVSVaskHuflFgTe91KfomCZnwJE7jaBJ9sW3sii43dcRLyAlvZuTr+H2y8To8hixqss7j6bfGZ+4jFPwtXwRFjxEX1QC41fJPR0duFsjqyEQxC5XzVGgymcwdK969Nx6PVhddHZHYBdnBaQnkKDI7AOYjXKXFX9VnCUmLfXryIRBQ1O+wYgL55yvHXj1f3IOrwi3b/VD6ec61QFlmEM5opYshYioDfHsV6fwsw2G9AXLiJWVFZPw91gWU3ABOE7eSAQcsM2EWxjHBkKl0cWsgQsDIx8iOf3FdoU7YW82zyoJjCyE0rgxqFgbpa+8dt3A4DhUSZb1bWUWhu1AiWmUd2Lap3jp07ucLe/K94eOxX1s5LAisXy0PW3sqT+oLRYmMcAvGKf6BKBvqYoGOnW+yrEFEED/CIZJ7J2c7c/9LJsz8qo4GX+SbzmrA33vO4gsH44nWD17O+blqIK+/mVaTI70UYQZ0T5bka31akvSPsAm5vhxqVPooqQLZ4b2aqkQ2y9HVt9ImMZxvZlU7fiMI5FRL+1II4uQ/aD9Yz1JpHalZd3QFSBLgPnZd2hqml/8MO2Wp5Jc1b+WSrwxtLkZS/0n/rfdNEzEXOBbEaAMJ5+O1nI2A2othCNaHVL7U5T9H+L+oBPoVDM2bnv0yaNmkCvGl/AUuVnPfhyZXYkKGXlQ6QMoe6Yuvzsd7NLkVZOs/jAu+LXOHSwY5PbAHlltnuUT9MHsoDjAF754voiurXVxTRpUN9YiNyoBzR5zOEBBZ9p LHGg0rF4 5Y19R4i1nB33LeEfhK/kycQ9RR3bVEQOWUKsJYw78RBQZCECF0VdmEkzPHN6ptQnzxoVufDeyw3nBOlufezQ8Y7TbOA6BpuFWJ6vBFuZHUOzxGvDBlqrdw3xCqCr510nQPHVFOd9N7wGSTWkACzu7uPe05yQqoG2wFAUrU2wx+Xouu0qm+b6vwpox0/sDlASTsbB4qKlMvOcN6RKlk/CXhzn29kMNL9r7vtCGVMHxaPPBWKdmpT7m5kTcTs1ySfeAqCbRwjxN6fiFCwn6CbNGUiviQx1+5Ad3FOzA0EExYkXm738iuAlgRreKEalfAYG0TiDM9vz8Ol7es42wwqq6JL1CPtDHjaWrqze3hbuXGgMYAAvOLnrnSlG/IpnJ5EjevHmTByLFQBGmTm1xLZbQtbz2TPpNNGK1kCjr9KvzJEhT17Y4YqyEU0U1LIbI+XF6H/0xRe69qhAVmBAKieZpBdA4HcoOOrmSzBzRklmpIENxCmiYqg5k4m8pkNvCdXjj8jIppKdvLqbu44/EaJgOmGo7x5F4xRZ8YVyFy8YrmxqpWZBJQFQoPHaGRQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Break down the system-wide workingset report into per-memcg reports, which aggregages its children hierarchically. The per-node workingset reporting histograms and refresh/report threshold files are presented as memcg files, showing a report containing all the nodes. The per-node page age interval is configurable in sysfs and not available per-memcg, while the refresh interval and report threshold are configured per-memcg. Memcg interface: /sys/fs/cgroup/.../memory.workingset.page_age The memcg equivalent of the sysfs workingset page age histogram breaks down the workingset of this memcg and its children into page age intervals. Each node is prefixed with a node header and a newline. Non-proactive direct reclaim on this memcg can also wake up userspace agents that are waiting on this file. e.g. N0 1000 anon=0 file=0 2000 anon=0 file=0 3000 anon=0 file=0 4000 anon=0 file=0 5000 anon=0 file=0 18446744073709551615 anon=0 file=0 /sys/fs/cgroup/.../memory.workingset.refresh_interval The memcg equivalent of the sysfs refresh interval. A per-node number of how much time a page age histogram is valid for, in milliseconds. e.g. echo N0=2000 > memory.workingset.refresh_interval /sys/fs/cgroup/.../memory.workingset.report_threshold The memcg equivalent of the sysfs report threshold. A per-node number of how often userspace agent waiting on the page age histogram can be woken up, in milliseconds. e.g. echo N0=1000 > memory.workingset.report_threshold Signed-off-by: Yuanchu Xie --- include/linux/memcontrol.h | 21 ++++ include/linux/workingset_report.h | 15 ++- mm/internal.h | 2 + mm/memcontrol.c | 160 +++++++++++++++++++++++++++++- mm/workingset_report.c | 50 +++++++--- 5 files changed, 230 insertions(+), 18 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index e1b41554a5fb..fd595b33a54d 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -323,6 +323,11 @@ struct mem_cgroup { spinlock_t event_list_lock; #endif /* CONFIG_MEMCG_V1 */ +#ifdef CONFIG_WORKINGSET_REPORT + /* memory.workingset.page_age file */ + struct cgroup_file workingset_page_age_file; +#endif + struct mem_cgroup_per_node *nodeinfo[]; }; @@ -1094,6 +1099,16 @@ static inline void memcg_memory_event_mm(struct mm_struct *mm, void split_page_memcg(struct page *head, int old_order, int new_order); +static inline struct cgroup_file * +mem_cgroup_page_age_file(struct mem_cgroup *memcg) +{ +#ifdef CONFIG_WORKINGSET_REPORT + return &memcg->workingset_page_age_file; +#else + return NULL; +#endif +} + #else /* CONFIG_MEMCG */ #define MEM_CGROUP_ID_SHIFT 0 @@ -1511,6 +1526,12 @@ void count_memcg_event_mm(struct mm_struct *mm, enum vm_event_item idx) static inline void split_page_memcg(struct page *head, int old_order, int new_order) { } + +static inline struct cgroup_file * +mem_cgroup_page_age_file(struct mem_cgroup *memcg) +{ + return NULL; +} #endif /* CONFIG_MEMCG */ /* diff --git a/include/linux/workingset_report.h b/include/linux/workingset_report.h index 2ec8b927b200..616be6469768 100644 --- a/include/linux/workingset_report.h +++ b/include/linux/workingset_report.h @@ -9,6 +9,8 @@ struct mem_cgroup; struct pglist_data; struct node; struct lruvec; +struct cgroup_file; +struct wsr_state; #ifdef CONFIG_WORKINGSET_REPORT @@ -40,7 +42,10 @@ struct wsr_state { unsigned long report_threshold; unsigned long refresh_interval; - struct kernfs_node *page_age_sys_file; + union { + struct kernfs_node *page_age_sys_file; + struct cgroup_file *page_age_cgroup_file; + }; /* breakdown of workingset by page age */ struct mutex page_age_lock; @@ -60,6 +65,9 @@ void wsr_remove_sysfs(struct node *node); */ bool wsr_refresh_report(struct wsr_state *wsr, struct mem_cgroup *root, struct pglist_data *pgdat); + +int wsr_set_refresh_interval(struct wsr_state *wsr, + unsigned long refresh_interval); #else static inline void wsr_init_lruvec(struct lruvec *lruvec) { @@ -79,6 +87,11 @@ static inline void wsr_init_sysfs(struct node *node) static inline void wsr_remove_sysfs(struct node *node) { } +static inline int wsr_set_refresh_interval(struct wsr_state *wsr, + unsigned long refresh_interval) +{ + return 0; +} #endif /* CONFIG_WORKINGSET_REPORT */ #endif /* _LINUX_WORKINGSET_REPORT_H */ diff --git a/mm/internal.h b/mm/internal.h index 508b7d9937d6..50ca0c6e651c 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -484,6 +484,8 @@ void set_task_reclaim_state(struct task_struct *task, struct reclaim_state *rs); * in mm/wsr.c */ void notify_workingset(struct mem_cgroup *memcg, struct pglist_data *pgdat); +int workingset_report_intervals_parse(char *src, + struct wsr_report_bins *bins); #else static inline void notify_workingset(struct mem_cgroup *memcg, struct pglist_data *pgdat) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 096856b35fbc..d1032c6efc66 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -4348,6 +4348,144 @@ static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, return nbytes; } +#ifdef CONFIG_WORKINGSET_REPORT +static int memory_ws_refresh_interval_show(struct seq_file *m, void *v) +{ + int nid; + struct mem_cgroup *memcg = mem_cgroup_from_seq(m); + + for_each_node_state(nid, N_MEMORY) { + struct wsr_state *wsr = + &mem_cgroup_lruvec(memcg, NODE_DATA(nid))->wsr; + + seq_printf(m, "N%d=%u ", nid, + jiffies_to_msecs(READ_ONCE(wsr->refresh_interval))); + } + seq_putc(m, '\n'); + + return 0; +} + +static ssize_t memory_wsr_threshold_parse(char *buf, size_t nbytes, + unsigned int *nid_out, + unsigned int *msecs) +{ + char *node, *threshold; + unsigned int nid; + int err; + + buf = strstrip(buf); + threshold = buf; + node = strsep(&threshold, "="); + + if (*node != 'N') + return -EINVAL; + + err = kstrtouint(node + 1, 0, &nid); + if (err) + return err; + + if (nid >= nr_node_ids || !node_state(nid, N_MEMORY)) + return -EINVAL; + + err = kstrtouint(threshold, 0, msecs); + if (err) + return err; + + *nid_out = nid; + + return nbytes; +} + +static ssize_t memory_ws_refresh_interval_write(struct kernfs_open_file *of, + char *buf, size_t nbytes, + loff_t off) +{ + unsigned int nid, msecs; + struct wsr_state *wsr; + struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); + ssize_t ret = memory_wsr_threshold_parse(buf, nbytes, &nid, &msecs); + + if (ret < 0) + return ret; + + wsr = &mem_cgroup_lruvec(memcg, NODE_DATA(nid))->wsr; + + ret = wsr_set_refresh_interval(wsr, msecs); + return ret ?: nbytes; +} + +static int memory_ws_report_threshold_show(struct seq_file *m, void *v) +{ + int nid; + struct mem_cgroup *memcg = mem_cgroup_from_seq(m); + + for_each_node_state(nid, N_MEMORY) { + struct wsr_state *wsr = + &mem_cgroup_lruvec(memcg, NODE_DATA(nid))->wsr; + + seq_printf(m, "N%d=%u ", nid, + jiffies_to_msecs(READ_ONCE(wsr->report_threshold))); + } + seq_putc(m, '\n'); + + return 0; +} + +static ssize_t memory_ws_report_threshold_write(struct kernfs_open_file *of, + char *buf, size_t nbytes, + loff_t off) +{ + unsigned int nid, msecs; + struct wsr_state *wsr; + struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); + ssize_t ret = memory_wsr_threshold_parse(buf, nbytes, &nid, &msecs); + + if (ret < 0) + return ret; + + wsr = &mem_cgroup_lruvec(memcg, NODE_DATA(nid))->wsr; + WRITE_ONCE(wsr->report_threshold, msecs_to_jiffies(msecs)); + return ret; +} + +static int memory_ws_page_age_show(struct seq_file *m, void *v) +{ + int nid; + struct mem_cgroup *memcg = mem_cgroup_from_seq(m); + + for_each_node_state(nid, N_MEMORY) { + struct wsr_state *wsr = + &mem_cgroup_lruvec(memcg, NODE_DATA(nid))->wsr; + struct wsr_report_bin *bin; + + if (!READ_ONCE(wsr->page_age)) + continue; + + wsr_refresh_report(wsr, memcg, NODE_DATA(nid)); + mutex_lock(&wsr->page_age_lock); + if (!wsr->page_age) + goto unlock; + seq_printf(m, "N%d\n", nid); + for (bin = wsr->page_age->bins; + bin->idle_age != WORKINGSET_INTERVAL_MAX; bin++) + seq_printf(m, "%u anon=%lu file=%lu\n", + jiffies_to_msecs(bin->idle_age), + bin->nr_pages[0] * PAGE_SIZE, + bin->nr_pages[1] * PAGE_SIZE); + + seq_printf(m, "%lu anon=%lu file=%lu\n", WORKINGSET_INTERVAL_MAX, + bin->nr_pages[0] * PAGE_SIZE, + bin->nr_pages[1] * PAGE_SIZE); + +unlock: + mutex_unlock(&wsr->page_age_lock); + } + + return 0; +} +#endif + static struct cftype memory_files[] = { { .name = "current", @@ -4419,7 +4557,27 @@ static struct cftype memory_files[] = { .flags = CFTYPE_NS_DELEGATABLE, .write = memory_reclaim, }, - { } /* terminate */ +#ifdef CONFIG_WORKINGSET_REPORT + { + .name = "workingset.refresh_interval", + .flags = CFTYPE_NOT_ON_ROOT | CFTYPE_NS_DELEGATABLE, + .seq_show = memory_ws_refresh_interval_show, + .write = memory_ws_refresh_interval_write, + }, + { + .name = "workingset.report_threshold", + .flags = CFTYPE_NOT_ON_ROOT | CFTYPE_NS_DELEGATABLE, + .seq_show = memory_ws_report_threshold_show, + .write = memory_ws_report_threshold_write, + }, + { + .name = "workingset.page_age", + .flags = CFTYPE_NOT_ON_ROOT | CFTYPE_NS_DELEGATABLE, + .file_offset = offsetof(struct mem_cgroup, workingset_page_age_file), + .seq_show = memory_ws_page_age_show, + }, +#endif + {} /* terminate */ }; struct cgroup_subsys memory_cgrp_subsys = { diff --git a/mm/workingset_report.c b/mm/workingset_report.c index bbefb0046669..1e1bdb8bf75b 100644 --- a/mm/workingset_report.c +++ b/mm/workingset_report.c @@ -37,9 +37,12 @@ void wsr_destroy_pgdat(struct pglist_data *pgdat) void wsr_init_lruvec(struct lruvec *lruvec) { struct wsr_state *wsr = &lruvec->wsr; + struct mem_cgroup *memcg = lruvec_memcg(lruvec); memset(wsr, 0, sizeof(*wsr)); mutex_init(&wsr->page_age_lock); + if (memcg && !mem_cgroup_is_root(memcg)) + wsr->page_age_cgroup_file = mem_cgroup_page_age_file(memcg); } void wsr_destroy_lruvec(struct lruvec *lruvec) @@ -51,8 +54,8 @@ void wsr_destroy_lruvec(struct lruvec *lruvec) memset(wsr, 0, sizeof(*wsr)); } -static int workingset_report_intervals_parse(char *src, - struct wsr_report_bins *bins) +int workingset_report_intervals_parse(char *src, + struct wsr_report_bins *bins) { int err = 0, i = 0; char *cur, *next = strim(src); @@ -356,20 +359,14 @@ static ssize_t refresh_interval_show(struct kobject *kobj, return sysfs_emit(buf, "%u\n", jiffies_to_msecs(interval)); } -static ssize_t refresh_interval_store(struct kobject *kobj, - struct kobj_attribute *attr, - const char *buf, size_t len) +int wsr_set_refresh_interval(struct wsr_state *wsr, + unsigned long refresh_interval) { - unsigned int interval; - int err; - struct wsr_state *wsr = kobj_to_wsr(kobj); - - err = kstrtouint(buf, 0, &interval); - if (err) - return err; + int err = 0; + unsigned long old_interval = 0; mutex_lock(&wsr->page_age_lock); - if (interval && !wsr->page_age) { + if (refresh_interval && !wsr->page_age) { struct wsr_page_age_histo *page_age = kzalloc(sizeof(struct wsr_page_age_histo), GFP_KERNEL); @@ -377,16 +374,34 @@ static ssize_t refresh_interval_store(struct kobject *kobj, err = -ENOMEM; goto unlock; } + page_age->bins[0].idle_age = WORKINGSET_INTERVAL_MAX; wsr->page_age = page_age; } - if (!interval && wsr->page_age) { + if (!refresh_interval && wsr->page_age) { kfree(wsr->page_age); wsr->page_age = NULL; } - WRITE_ONCE(wsr->refresh_interval, msecs_to_jiffies(interval)); + old_interval = READ_ONCE(wsr->refresh_interval); + WRITE_ONCE(wsr->refresh_interval, msecs_to_jiffies(refresh_interval)); unlock: mutex_unlock(&wsr->page_age_lock); + return err; +} + +static ssize_t refresh_interval_store(struct kobject *kobj, + struct kobj_attribute *attr, + const char *buf, size_t len) +{ + unsigned int interval; + int err; + struct wsr_state *wsr = kobj_to_wsr(kobj); + + err = kstrtouint(buf, 0, &interval); + if (err) + return err; + + err = wsr_set_refresh_interval(wsr, interval); return err ?: len; } @@ -551,5 +566,8 @@ void notify_workingset(struct mem_cgroup *memcg, struct pglist_data *pgdat) { struct wsr_state *wsr = &mem_cgroup_lruvec(memcg, pgdat)->wsr; - kernfs_notify(wsr->page_age_sys_file); + if (mem_cgroup_is_root(memcg)) + kernfs_notify(wsr->page_age_sys_file); + else + cgroup_file_notify(wsr->page_age_cgroup_file); }