From patchwork Fri Apr 8 04:57:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12806062 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8D30C433FE for ; Fri, 8 Apr 2022 04:57:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 33AD66B0072; Fri, 8 Apr 2022 00:57:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C3456B0074; Fri, 8 Apr 2022 00:57:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 164338D0001; Fri, 8 Apr 2022 00:57:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.27]) by kanga.kvack.org (Postfix) with ESMTP id 08AEC6B0072 for ; Fri, 8 Apr 2022 00:57:49 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id C993D1203CA for ; Fri, 8 Apr 2022 04:57:48 +0000 (UTC) X-FDA: 79332504216.19.4C9762B Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf25.hostedemail.com (Postfix) with ESMTP id 66BD4A0002 for ; Fri, 8 Apr 2022 04:57:48 +0000 (UTC) Received: by mail-pj1-f74.google.com with SMTP id mm2-20020a17090b358200b001bf529127dfso4759965pjb.6 for ; Thu, 07 Apr 2022 21:57:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=NjSiOM/35CoR66GddgJGkPcuLjrAQk87146PcI+7fw0=; b=aSNVD8Dv1pAjAIp1MJf2LmZJVyOsK+n7bFr2jOZjil/rmLRiCSWTwcrXHDdy7/oHe/ Bv37xIDzncGkL6U8iAB8xE0qgotdzer+GesJprbrbN3tE3/AcvGuyaa2hqNOnYLnd5ez RR5OJTb4hxOJBAzLiZvp0SfHpR9AC0CGAa57NYjTeSonjqhTEd/aYSa7+PW2l0M9an7c oGf7opkEGVC2n1NR9o+AT++usov2GpzMn2bsSu7Vq/6dkvjUVtG4dOiR8RH1fTj7Zdy1 i6ipH575Xw22gZ6xNEZXfmXS7bA+BGy4czwQICZYNXytSU6mfDYit92feNqqhuD7m9zA SUUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=NjSiOM/35CoR66GddgJGkPcuLjrAQk87146PcI+7fw0=; b=gP3qkGssSro71VQ8PI0Hctre+qD3fVXYxIagbPcK8W3KD2+ks2WMhh4mx/2PQu5DOJ Ws2YMOEhyMM6svix38n8/nWpvEDi9Ci9ph2AZhuhzJFhNmmGZXAL6sbjdRgJ/x/mgnvb ms4Zqkh1AuIuexrAN6W/nD9J8YGNq5sUA+sje5K6UIW1JGQcdSscTK/P4gVvZyVoVIqq 5q4nuufmS48qE+onIercwVYr/1CjF4PR/2V7vpW6oBHzilcPJ8zXGqOPJwiycW1WHxHw nmfz1lxrgZ7lAT4GouR+YUl7fQPK4YCb2jhAzs9bShx9W3G2VsReJ7VCDyYgsqPGDp+p Y1gw== X-Gm-Message-State: AOAM532VGtgjspj/0zJWVf+jv6eToRKfkfwTBE4kuHIU7dws5UFPjM+0 A1WWA9OswXwY8RBuJTsB9JLMGhZ7q2CQcXN6 X-Google-Smtp-Source: ABdhPJywussi7i/xLF5mokT59v1xMo0FnOaWa7Wq5Iefmfbat7pCbV7jYYk7OWFFGxZnl4nnGMYwR5ffLABQzYhx X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a63:1117:0:b0:399:2df0:7fb9 with SMTP id g23-20020a631117000000b003992df07fb9mr14473499pgl.40.1649393867335; Thu, 07 Apr 2022 21:57:47 -0700 (PDT) Date: Fri, 8 Apr 2022 04:57:40 +0000 In-Reply-To: <20220408045743.1432968-1-yosryahmed@google.com> Message-Id: <20220408045743.1432968-2-yosryahmed@google.com> Mime-Version: 1.0 References: <20220408045743.1432968-1-yosryahmed@google.com> X-Mailer: git-send-email 2.35.1.1178.g4f1659d476-goog Subject: [PATCH v3 1/4] memcg: introduce per-memcg reclaim interface From: Yosry Ahmed To: Johannes Weiner , Michal Hocko , Shakeel Butt , Andrew Morton , Roman Gushchin Cc: David Rientjes , Tejun Heo , Zefan Li , Jonathan Corbet , Shuah Khan , Yu Zhao , Dave Hansen , Wei Xu , Greg Thelen , Chen Wandun , Vaibhav Jain , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Tim Chen , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Yosry Ahmed , Michal Hocko X-Stat-Signature: 9wyeqkpdukd9eb4xh8krctqyd96gzstz Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=aSNVD8Dv; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of 3y8BPYgoKCKQcSWVcELQIHKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--yosryahmed.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3y8BPYgoKCKQcSWVcELQIHKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--yosryahmed.bounces.google.com X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 66BD4A0002 X-HE-Tag: 1649393868-769091 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Shakeel Butt Introduce a memcg interface to trigger memory reclaim on a memory cgroup. Use case: Proactive Reclaim --------------------------- A userspace proactive reclaimer can continuously probe the memcg to reclaim a small amount of memory. This gives more accurate and up-to-date workingset estimation as the LRUs are continuously sorted and can potentially provide more deterministic memory overcommit behavior. The memory overcommit controller can provide more proactive response to the changing behavior of the running applications instead of being reactive. A userspace reclaimer's purpose in this case is not a complete replacement for kswapd or direct reclaim, it is to proactively identify memory savings opportunities and reclaim some amount of cold pages set by the policy to free up the memory for more demanding jobs or scheduling new jobs. A user space proactive reclaimer is used in Google data centers. Additionally, Meta's TMO paper recently referenced a very similar interface used for user space proactive reclaim: https://dl.acm.org/doi/pdf/10.1145/3503222.3507731 Benefits of a user space reclaimer: ----------------------------------- 1) More flexible on who should be charged for the cpu of the memory reclaim. For proactive reclaim, it makes more sense to be centralized. 2) More flexible on dedicating the resources (like cpu). The memory overcommit controller can balance the cost between the cpu usage and the memory reclaimed. 3) Provides a way to the applications to keep their LRUs sorted, so, under memory pressure better reclaim candidates are selected. This also gives more accurate and uptodate notion of working set for an application. Why memory.high is not enough? ------------------------------ - memory.high can be used to trigger reclaim in a memcg and can potentially be used for proactive reclaim. However there is a big downside in using memory.high. It can potentially introduce high reclaim stalls in the target application as the allocations from the processes or the threads of the application can hit the temporary memory.high limit. - Userspace proactive reclaimers usually use feedback loops to decide how much memory to proactively reclaim from a workload. The metrics used for this are usually either refaults or PSI, and these metrics will become messy if the application gets throttled by hitting the high limit. - memory.high is a stateful interface, if the userspace proactive reclaimer crashes for any reason while triggering reclaim it can leave the application in a bad state. - If a workload is rapidly expanding, setting memory.high to proactively reclaim memory can result in actually reclaiming more memory than intended. The benefits of such interface and shortcomings of existing interface were further discussed in this RFC thread: https://lore.kernel.org/linux-mm/5df21376-7dd1-bf81-8414-32a73cea45dd@google.com/ Interface: ---------- Introducing a very simple memcg interface 'echo 10M > memory.reclaim' to trigger reclaim in the target memory cgroup. The interface is introduced as a nested-keyed file to allow for future optional arguments to be easily added to configure the behavior of reclaim. Possible Extensions: -------------------- - This interface can be extended with an additional parameter or flags to allow specifying one or more types of memory to reclaim from (e.g. file, anon, ..). - The interface can also be extended with a node mask to reclaim from specific nodes. This has use cases for reclaim-based demotion in memory tiering systens. - A similar per-node interface can also be added to support proactive reclaim and reclaim-based demotion in systems without memcg. - Add a timeout parameter to make it easier for user space to call the interface without worrying about being blocked for an undefined amount of time. For now, let's keep things simple by adding the basic functionality. [yosryahmed@google.com: refreshed to current master, updated commit message based on recent discussions and use cases] Signed-off-by: Shakeel Butt Signed-off-by: Yosry Ahmed Acked-by: Johannes Weiner Acked-by: Michal Hocko Acked-by: Wei Xu Acked-by: Roman Gushchin --- Documentation/admin-guide/cgroup-v2.rst | 21 ++++++++++++++ mm/memcontrol.c | 37 +++++++++++++++++++++++++ 2 files changed, 58 insertions(+) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 69d7a6983f78..19bcd73cad03 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1208,6 +1208,27 @@ PAGE_SIZE multiple when read back. high limit is used and monitored properly, this limit's utility is limited to providing the final safety net. + memory.reclaim + A write-only nested-keyed file which exists for all cgroups. + + This is a simple interface to trigger memory reclaim in the + target cgroup. + + This file accepts a single key, the number of bytes to reclaim. + No nested keys are currently supported. + + Example:: + + echo "1G" > memory.reclaim + + The interface can be later extended with nested keys to + configure the reclaim behavior. For example, specify the + type of memory to reclaim from (anon, file, ..). + + Please note that the kernel can over or under reclaim from + the target cgroup. If less bytes are reclaimed than the + specified amount, -EAGAIN is returned. + memory.oom.group A read-write single value file which exists on non-root cgroups. The default value is "0". diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 725f76723220..2b214b66d333 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6355,6 +6355,38 @@ static ssize_t memory_oom_group_write(struct kernfs_open_file *of, return nbytes; } +static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, + size_t nbytes, loff_t off) +{ + struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); + unsigned int nr_retries = MAX_RECLAIM_RETRIES; + unsigned long nr_to_reclaim, nr_reclaimed = 0; + int err; + + buf = strstrip(buf); + err = page_counter_memparse(buf, "", &nr_to_reclaim); + if (err) + return err; + + while (nr_reclaimed < nr_to_reclaim) { + unsigned long reclaimed; + + if (signal_pending(current)) + break; + + reclaimed = try_to_free_mem_cgroup_pages(memcg, + nr_to_reclaim - nr_reclaimed, + GFP_KERNEL, true); + + if (!reclaimed && !nr_retries--) + break; + + nr_reclaimed += reclaimed; + } + + return nr_reclaimed < nr_to_reclaim ? -EAGAIN : nbytes; +} + static struct cftype memory_files[] = { { .name = "current", @@ -6413,6 +6445,11 @@ static struct cftype memory_files[] = { .seq_show = memory_oom_group_show, .write = memory_oom_group_write, }, + { + .name = "reclaim", + .flags = CFTYPE_NS_DELEGATABLE, + .write = memory_reclaim, + }, { } /* terminate */ }; From patchwork Fri Apr 8 04:57:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12806063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FA0FC433F5 for ; Fri, 8 Apr 2022 04:57:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C62096B0074; Fri, 8 Apr 2022 00:57:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C12538D0001; Fri, 8 Apr 2022 00:57:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A16406B0078; Fri, 8 Apr 2022 00:57:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id 8EE176B0074 for ; Fri, 8 Apr 2022 00:57:50 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 53B601B4 for ; Fri, 8 Apr 2022 04:57:50 +0000 (UTC) X-FDA: 79332504300.16.BDE62AC Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) by imf10.hostedemail.com (Postfix) with ESMTP id CDE09C0002 for ; Fri, 8 Apr 2022 04:57:49 +0000 (UTC) Received: by mail-pf1-f202.google.com with SMTP id g5-20020a62f945000000b0050578328060so980541pfm.17 for ; Thu, 07 Apr 2022 21:57:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=RpFXlKvwb4pUJyIrx0BwPq/Sz7TwgzSMaU3KXpjsnFg=; b=PZfwTndHaHFeiId6kJJWIk58eh5ykRjdyumVYKw/V0Oluj34d7Lv6G9QfbAO6MG68X 5Cp8uWVnKwShcF0BFnxZHLWp6ZB5aWPOXG/Oix4YaoRQJT/TamzL0Z+Q0C1feePcwpoK FENgXFWEzQHHDlbUa6+OWFMfZy/Zs5THpeXzBDDUoSOpdHazwli+xVssStSpQX6dXP87 +gEZWfq2i6qSv+b+zRxuTxamA08CMwUpQtHbpqJp/ibE+JIapaGcj0/Y8FuPUv9ANtvB K64P3pZuES8BJSknRtTorjKzVC0kiVkym17TcMpzn9ih6gm1W5AFO0y+l2ijh3LMEALt vIDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=RpFXlKvwb4pUJyIrx0BwPq/Sz7TwgzSMaU3KXpjsnFg=; b=dcbfmsJvPwJzwq623seJifvtE5oWcrGmpgdqSWB6Yj+ZOHSUjaU5EjHTvqaxGL402h nG095Xnbl9Ke49OShyLlxtJNtiHBNXp4UgnCMGHOdPhZvh4FvPJH3/YfmF4L417ZWGOd wSCN0SsYZdY6TNttdoie6AO61crNEfTNWQ2YLpNkNZWO9oS+ec+YDoxyUU7VeAr4js21 BYeU2N+22oPOVfumaidfAbiHrIuEhVDOZjZCpT2Nt/xJXoMMCf5yP5ZLkrFMmUtDKPCm Zpn1/SWSHK82PoPIXeQj/aOUBBzGbyYyaBa9DsObOUz0y0+v1OXEwiZMfDNfhboczu9a 1FTg== X-Gm-Message-State: AOAM5322/3+lxX+uFgMoM2wpX4I90YwDPCtUdmi25d1qZNrytBwDeCrN DPAA+8yJ74avmPJGqUFcU0AoCA1M1ZtjqWRW X-Google-Smtp-Source: ABdhPJw5mVvKa75yNWq8X1toeYD179Hk+kkkpDn6jYwbzqwx+XJ9Se+9zdydnBR6hBfnjvMnIUQQ2/3DjI+R68dT X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:902:ecd1:b0:154:5393:aa7e with SMTP id a17-20020a170902ecd100b001545393aa7emr17225851plh.79.1649393868836; Thu, 07 Apr 2022 21:57:48 -0700 (PDT) Date: Fri, 8 Apr 2022 04:57:41 +0000 In-Reply-To: <20220408045743.1432968-1-yosryahmed@google.com> Message-Id: <20220408045743.1432968-3-yosryahmed@google.com> Mime-Version: 1.0 References: <20220408045743.1432968-1-yosryahmed@google.com> X-Mailer: git-send-email 2.35.1.1178.g4f1659d476-goog Subject: [PATCH v3 2/4] selftests: cgroup: return the errno of write() in cg_write() on failure From: Yosry Ahmed To: Johannes Weiner , Michal Hocko , Shakeel Butt , Andrew Morton , Roman Gushchin Cc: David Rientjes , Tejun Heo , Zefan Li , Jonathan Corbet , Shuah Khan , Yu Zhao , Dave Hansen , Wei Xu , Greg Thelen , Chen Wandun , Vaibhav Jain , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Tim Chen , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Yosry Ahmed X-Stat-Signature: zgkmnba39r5s5eoaxqg65y9tgeby3m57 Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=PZfwTndH; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf10.hostedemail.com: domain of 3zMBPYgoKCKUdTXWdFMRJILTTLQJ.HTRQNSZc-RRPaFHP.TWL@flex--yosryahmed.bounces.google.com designates 209.85.210.202 as permitted sender) smtp.mailfrom=3zMBPYgoKCKUdTXWdFMRJILTTLQJ.HTRQNSZc-RRPaFHP.TWL@flex--yosryahmed.bounces.google.com X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: CDE09C0002 X-HE-Tag: 1649393869-656562 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000071, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, cg_write() returns 0 on success and -1 on failure. Modify it to return the errno of write() syscall when write() fails. Signed-off-by: Yosry Ahmed Acked-by: Roman Gushchin --- tools/testing/selftests/cgroup/cgroup_util.c | 32 +++++++++++--------- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/tools/testing/selftests/cgroup/cgroup_util.c b/tools/testing/selftests/cgroup/cgroup_util.c index dbaa7aabbb4a..3b6bb09985fa 100644 --- a/tools/testing/selftests/cgroup/cgroup_util.c +++ b/tools/testing/selftests/cgroup/cgroup_util.c @@ -38,23 +38,23 @@ static ssize_t read_text(const char *path, char *buf, size_t max_len) return len; } -static ssize_t write_text(const char *path, char *buf, ssize_t len) +/* + * Returns: + * success -> 0 + * open() failure -> -1 + * write() failure -> errno + */ +static int write_text(const char *path, char *buf, ssize_t len) { - int fd; + int fd, ret; fd = open(path, O_WRONLY | O_APPEND); if (fd < 0) return fd; - len = write(fd, buf, len); - if (len < 0) { - close(fd); - return len; - } - + ret = write(fd, buf, len) < 0 ? errno : 0; close(fd); - - return len; + return ret; } char *cg_name(const char *root, const char *name) @@ -177,17 +177,19 @@ long cg_read_lc(const char *cgroup, const char *control) return cnt; } +/* + * Returns: + * success -> 0 + * open() failure -> -1 + * write() failure -> errno + */ int cg_write(const char *cgroup, const char *control, char *buf) { char path[PATH_MAX]; ssize_t len = strlen(buf); snprintf(path, sizeof(path), "%s/%s", cgroup, control); - - if (write_text(path, buf, len) == len) - return 0; - - return -1; + return write_text(path, buf, len); } int cg_find_unified_root(char *root, size_t len) From patchwork Fri Apr 8 04:57:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12806064 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C050C433EF for ; Fri, 8 Apr 2022 04:57:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E9C36B0075; Fri, 8 Apr 2022 00:57:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 471128D0002; Fri, 8 Apr 2022 00:57:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2764D8D0001; Fri, 8 Apr 2022 00:57:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id 168486B0075 for ; Fri, 8 Apr 2022 00:57:52 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D9C82353 for ; Fri, 8 Apr 2022 04:57:51 +0000 (UTC) X-FDA: 79332504342.04.E5FB87C Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf03.hostedemail.com (Postfix) with ESMTP id 6B99020003 for ; Fri, 8 Apr 2022 04:57:51 +0000 (UTC) Received: by mail-pj1-f73.google.com with SMTP id q1-20020a17090a750100b001cb1d8cb9e9so1643843pjk.1 for ; Thu, 07 Apr 2022 21:57:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=gA8fl4VBLSIuPoY/kPZyFR8DNphzGSE/geYX3/dgg00=; b=Pcpd2Rbrapa7M7Za73CnE1bEFC5wI7dwEloVVr2sNZaIVqKCy5cB4naL+d9PVU9Kq9 o27VPMESu81Lm/IKCtpsqDvuaOrAbNZ4Z3JlkNVWBzI0rgkIQUh8AgXWl+jQU788ZJmX l6A6jyeZdEjndJ2j2ErDuXmLudkDIPIY2lKK43CSnl2C6npVfgu2m4i619MVJM7BRrwk AY/5jLUI62lflwZYHGLXDAiENTVnD5dnA32uP1ue3g0CVNMJg6oMYm2jAGzmng8hs1ih 0Riamlr+TJ4j2W4TmwXqsgJmdKZCloPwc1OqNvr5som55EVUiCPihm0xB908z/cfbhad eBpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=gA8fl4VBLSIuPoY/kPZyFR8DNphzGSE/geYX3/dgg00=; b=In3sUMzEAYH7ir8/f315zoieGaR/fO8I/MXc/RegrxKz1fLYSKkY+RAualjdBy4y0z BkdRQyqLUw0b/IuGzVrnH25NtNCfI95tpqqfMh8k9xXNcaHWCdvXoY+1uHkGm2fcJDvG l00iLtX3P7Nd7zPyDcWnZsLzJTXTRG65d4nv7YG6/lqUUvwwKZR0j9mzdZGK1gYfbWRm n78s2f2O+xYW/YLcJoCQYya+zulZJNB6+0m5pnWO8gBGFPIbCmNYgL5A5Rcdcgj5nGuI DO2eLExLig+6cHuHLgYyKUSfHgfl6M2OJRm46mX6lMzGrgdRa+3r1+xgcxjtlyT3cLYb PbRQ== X-Gm-Message-State: AOAM531tGe5YGtQOE1R5Z/OnBu0RuEagFLjyZP1iq2ODdsqfNoMZWSVu mf1ovWFp9oYr250H+aaY9gbEV8qkYH8fclRi X-Google-Smtp-Source: ABdhPJwveDjBEYrVRy38w4adlQ9PbmM5YsL9jL+O+3mQ7Gci02SY9egKz08a0oG2xCNWEbJ8Q67ORfFnzPtKwKxT X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:902:6bc4:b0:154:6b3d:a720 with SMTP id m4-20020a1709026bc400b001546b3da720mr17230778plt.104.1649393870424; Thu, 07 Apr 2022 21:57:50 -0700 (PDT) Date: Fri, 8 Apr 2022 04:57:42 +0000 In-Reply-To: <20220408045743.1432968-1-yosryahmed@google.com> Message-Id: <20220408045743.1432968-4-yosryahmed@google.com> Mime-Version: 1.0 References: <20220408045743.1432968-1-yosryahmed@google.com> X-Mailer: git-send-email 2.35.1.1178.g4f1659d476-goog Subject: [PATCH v3 3/4] selftests: cgroup: fix alloc_anon_noexit() instantly freeing memory From: Yosry Ahmed To: Johannes Weiner , Michal Hocko , Shakeel Butt , Andrew Morton , Roman Gushchin Cc: David Rientjes , Tejun Heo , Zefan Li , Jonathan Corbet , Shuah Khan , Yu Zhao , Dave Hansen , Wei Xu , Greg Thelen , Chen Wandun , Vaibhav Jain , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Tim Chen , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Yosry Ahmed Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Pcpd2Rbr; spf=pass (imf03.hostedemail.com: domain of 3zsBPYgoKCKcfVZYfHOTLKNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--yosryahmed.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3zsBPYgoKCKcfVZYfHOTLKNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: pgyn7s6e78xfqdq51jyxqpqosh4taqx6 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 6B99020003 X-HE-Tag: 1649393871-926191 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, alloc_anon_noexit() calls alloc_anon() which instantly frees the allocated memory. alloc_anon_noexit() is usually used with cg_run_nowait() to run a process in the background that allocates memory. It makes sense for the background process to keep the memory allocated and not instantly free it (otherwise there is no point of running it in the background). Signed-off-by: Yosry Ahmed Acked-by: Roman Gushchin --- tools/testing/selftests/cgroup/test_memcontrol.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/cgroup/test_memcontrol.c b/tools/testing/selftests/cgroup/test_memcontrol.c index 36ccf2322e21..f2ffb3a30194 100644 --- a/tools/testing/selftests/cgroup/test_memcontrol.c +++ b/tools/testing/selftests/cgroup/test_memcontrol.c @@ -211,13 +211,17 @@ static int alloc_pagecache_50M_noexit(const char *cgroup, void *arg) static int alloc_anon_noexit(const char *cgroup, void *arg) { int ppid = getppid(); + size_t size = (unsigned long)arg; + char *buf, *ptr; - if (alloc_anon(cgroup, arg)) - return -1; + buf = malloc(size); + for (ptr = buf; ptr < buf + size; ptr += PAGE_SIZE) + *ptr = 0; while (getppid() == ppid) sleep(1); + free(buf); return 0; } From patchwork Fri Apr 8 04:57:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12806065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE4E9C4332F for ; Fri, 8 Apr 2022 04:57:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A180A6B0078; Fri, 8 Apr 2022 00:57:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9CC708D0002; Fri, 8 Apr 2022 00:57:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F3AA8D0001; Fri, 8 Apr 2022 00:57:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.27]) by kanga.kvack.org (Postfix) with ESMTP id 71FEF6B0078 for ; Fri, 8 Apr 2022 00:57:53 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 3D668253B2 for ; Fri, 8 Apr 2022 04:57:53 +0000 (UTC) X-FDA: 79332504426.02.DAC99DF Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf05.hostedemail.com (Postfix) with ESMTP id D7686100003 for ; Fri, 8 Apr 2022 04:57:52 +0000 (UTC) Received: by mail-pj1-f74.google.com with SMTP id gq2-20020a17090b104200b001ca93cdc0f2so4100954pjb.7 for ; Thu, 07 Apr 2022 21:57:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=3pTs/VmZPxq5q5wYSF5wNcPHcga+GtsVQl1Ey1cMTYA=; b=rvhnsrqMv4sgVKJh1vywJDwJ5U1YIrm4qeZ+xvYMN5iNrp3UBtoPAwLYGST3GoaiMm WHOaYmhgN89ZSkqq/rrlyrpsoAk9snev93OB3iD6xwnbay5f9TY/LZvY0DCAezc6XBkq xL4SfAAQ1Asd+mUQruSX/RHQfET7mMtUa5hZPgL9lbYKPNV6DK0EbhmuK9i82PR/5pAt /NhixDWU/pG6CKN2oFTyr8dDewwdERP0qYAzylJbweyHFeoMsu6trfoKBuDfsepu7Rek a8aShEUZ343hISjyHiMcMdIBoVEqcQ1IoADLU9STJVAAr7sxoIvQ8281ix2aOcgAz9HZ /kVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=3pTs/VmZPxq5q5wYSF5wNcPHcga+GtsVQl1Ey1cMTYA=; b=ATya4NH47p4+tTk/weu9HoGu8jlayib6uzEZPq78HxA/QQG2UXzQcpyFrlR26xbLxr twM1GqkzTRUt7gcNpr68xUwSD61YkeVI6OTJ+mwPiLDOxLC9I86n6ASUlRiM3NVLVi5x wjT7uIyCEu+CYm4XlVjIA45SLM1GE2XzBnJRDl7lfwTqADlIgJ4UaqRHWB0dTcdUM5HS g5oq63rT1xiHaOY31DKSmoV3Og9qBzY3WWiBiTHAEsvWQFW2/arH28Jx7QNzf0efKTYc LWDYbIeLz/2gIq+k0LmRwDpcB+NPlRf0GJEQ/wXQuNhNSvdZ2eIvTwOgb50T9QB8shuj ywtA== X-Gm-Message-State: AOAM5338qcOJ3dfwsO6Ki0nBaasxAGoFp0TYLEqWWu6JubsmOx0GuNMx z2zWG4UGwuZHD5L9PUR35j2vc5ILnNYfuD2o X-Google-Smtp-Source: ABdhPJwrP7dagj8dnlFgC14HxELMunGPx+PFv2E1bWL6b6NRaIqpjG704ThFr7Q8pI5XpMISvNAO5NCYkQKyivj5 X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:903:22c4:b0:154:8399:5285 with SMTP id y4-20020a17090322c400b0015483995285mr17325689plg.56.1649393871741; Thu, 07 Apr 2022 21:57:51 -0700 (PDT) Date: Fri, 8 Apr 2022 04:57:43 +0000 In-Reply-To: <20220408045743.1432968-1-yosryahmed@google.com> Message-Id: <20220408045743.1432968-5-yosryahmed@google.com> Mime-Version: 1.0 References: <20220408045743.1432968-1-yosryahmed@google.com> X-Mailer: git-send-email 2.35.1.1178.g4f1659d476-goog Subject: [PATCH v3 4/4] selftests: cgroup: add a selftest for memory.reclaim From: Yosry Ahmed To: Johannes Weiner , Michal Hocko , Shakeel Butt , Andrew Morton , Roman Gushchin Cc: David Rientjes , Tejun Heo , Zefan Li , Jonathan Corbet , Shuah Khan , Yu Zhao , Dave Hansen , Wei Xu , Greg Thelen , Chen Wandun , Vaibhav Jain , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Tim Chen , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Yosry Ahmed X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: D7686100003 X-Stat-Signature: dbw7wwxx4x3zpnm6cmbthqptcy7iwfch Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=rvhnsrqM; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf05.hostedemail.com: domain of 3z8BPYgoKCKggWaZgIPUMLOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--yosryahmed.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3z8BPYgoKCKggWaZgIPUMLOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--yosryahmed.bounces.google.com X-Rspam-User: X-HE-Tag: 1649393872-494730 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add a new test for memory.reclaim that verifies that the interface correctly reclaims memory as intended, from both anon and file pages. Signed-off-by: Yosry Ahmed Acked-by: Roman Gushchin --- .../selftests/cgroup/test_memcontrol.c | 85 +++++++++++++++++++ 1 file changed, 85 insertions(+) diff --git a/tools/testing/selftests/cgroup/test_memcontrol.c b/tools/testing/selftests/cgroup/test_memcontrol.c index f2ffb3a30194..75ae441c01c3 100644 --- a/tools/testing/selftests/cgroup/test_memcontrol.c +++ b/tools/testing/selftests/cgroup/test_memcontrol.c @@ -760,6 +760,90 @@ static int test_memcg_max(const char *root) return ret; } +/* + * This test checks that memory.reclaim reclaims the given + * amount of memory (from both anon and file). + */ +static int test_memcg_reclaim(const char *root) +{ + int ret = KSFT_FAIL, fd; + char *memcg; + long current, to_reclaim; + char buf[64]; + + memcg = cg_name(root, "memcg_test"); + if (!memcg) + goto cleanup; + + if (cg_create(memcg)) + goto cleanup; + + current = cg_read_long(memcg, "memory.current"); + if (current != 0) + goto cleanup; + + cg_run_nowait(memcg, alloc_anon_noexit, (void *) MB(50)); + sleep(1); + + fd = get_temp_fd(); + if (fd < 0) + goto cleanup; + + cg_run_nowait(memcg, alloc_pagecache_50M_noexit, (void *)(long)fd); + sleep(1); + + current = cg_read_long(memcg, "memory.current"); + if (!values_close(current, MB(100), 10)) + goto cleanup; + + /* + * Reclaim until current reaches 30M, make sure to reclaim over 50M to + * hit both anon and file. + */ + while (true) { + int err; + + current = cg_read_long(memcg, "memory.current"); + to_reclaim = current - MB(30); + + /* + * We only keep looping if we get EAGAIN, which means we could + * not reclaim the full amount. + */ + if (to_reclaim <= 0) + goto cleanup; + + + snprintf(buf, sizeof(buf), "%ld", to_reclaim); + err = cg_write(memcg, "memory.reclaim", buf); + if (!err) { + /* + * If writing succeeds, then the written amount should have been + * fully reclaimed (and maybe more). + */ + current = cg_read_long(memcg, "memory.current"); + if (!values_close(current, MB(30), 3) && current > MB(30)) + goto cleanup; + break; + } + + /* The kernel could not reclaim the full amount, try again. */ + if (err == EAGAIN) + continue; + + /* We got an unexpected error. */ + goto cleanup; + } + + ret = KSFT_PASS; +cleanup: + cg_destroy(memcg); + free(memcg); + close(fd); + + return ret; +} + static int alloc_anon_50M_check_swap(const char *cgroup, void *arg) { long mem_max = (long)arg; @@ -1263,6 +1347,7 @@ struct memcg_test { T(test_memcg_high), T(test_memcg_high_sync), T(test_memcg_max), + T(test_memcg_reclaim), T(test_memcg_oom_events), T(test_memcg_swap_max), T(test_memcg_sock),