From patchwork Mon Apr 25 19:00:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12826098 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E320C433F5 for ; Mon, 25 Apr 2022 19:00:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 833386B0080; Mon, 25 Apr 2022 15:00:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E2476B0081; Mon, 25 Apr 2022 15:00:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 65D1A6B0082; Mon, 25 Apr 2022 15:00:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.27]) by kanga.kvack.org (Postfix) with ESMTP id 58C9B6B0080 for ; Mon, 25 Apr 2022 15:00:57 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 25D132865C for ; Mon, 25 Apr 2022 19:00:57 +0000 (UTC) X-FDA: 79396318554.01.DABC0EF Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) by imf25.hostedemail.com (Postfix) with ESMTP id 114FEA0046 for ; Mon, 25 Apr 2022 19:00:49 +0000 (UTC) Received: by mail-pg1-f201.google.com with SMTP id c32-20020a631c60000000b0039cec64e9f1so9744250pgm.3 for ; Mon, 25 Apr 2022 12:00:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=fBFNWoBuZ+rUFYDa2D6wGphqTYvQjamUdIjSsw6TIq0=; b=smHSwP6GCAdbnpTWe49B9oKF0iYhl1bzB1K3BDIn9UhpSm9W7gWtmt2DRS6JQKMy8o BajDe1FI0jRx+0D1XfBG9Tjf8xi8/2gRrmgadMgDjdPr01tIohPi8twnF3DK6BGgIVvi 3yOdRTAnVNgeawrn30eQ9SYwTdBmFzJ6tY5eY0W6C6U4q9dlHlnFVJU6QSb8bqBoq2P5 R85Z1HAt6KgkwQKubfSsF9s5YBGQN4nGEmwB9Jg3JkRFLErxcvUGrbweooRE4a+KXyK+ F2xMNNMm5DykM7dmSOdrjgcVgow14TDKqX495S1XlPEkUL5iomDIlLABrbbmXI8Ja1Oe JrIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=fBFNWoBuZ+rUFYDa2D6wGphqTYvQjamUdIjSsw6TIq0=; b=Rc4hpadGfKPkeqyXndOyGuFSe5irr927L5lHntRAiRUOxKlZ7yP7KWlMDAy//Rgnbp GRGRnA7LPsoTqefb3DbKKvNUf64EKavKmjKI7B7m8rDI7fiaNY9nyv8OialNTHPUPbMY MYQMOil6PobaznofLUO2GYfE42cTj/evnNzyNjvzVPn7Ug5xpkFHUBI8e2vg+I+LABsh yZe4e1Gke3RBGLvzI2H1I7gJKIvub4HIhOAFzozWJoLCVS0U8pdoM9N4O39JbpSTHq9w FeBwwF5hIxHn+nNXGbEhmNDcNOydSbURlsLxYoL2PithHgsuCxJLNTddpM/5SFrivzX2 jnGw== X-Gm-Message-State: AOAM531GOS/Oe1yNyCLECQUvPHi19aF7xVRjIrFYH9OfMY1O9bpj1H+E ty/WcwEftuXxIKbly91kLkeJfzaVgSH1MRPB X-Google-Smtp-Source: ABdhPJxt8xE4NWnQfnST2T2fwscg58NT92N6YrUwlKz1bvN3nvLWPKcU8F4b6X/rtWS2BqN9NjKAgFqXj2Zk/cq6 X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a65:6e88:0:b0:39d:2647:f75d with SMTP id bm8-20020a656e88000000b0039d2647f75dmr16444204pgb.523.1650913255418; Mon, 25 Apr 2022 12:00:55 -0700 (PDT) Date: Mon, 25 Apr 2022 19:00:37 +0000 In-Reply-To: <20220425190040.2475377-1-yosryahmed@google.com> Message-Id: <20220425190040.2475377-2-yosryahmed@google.com> Mime-Version: 1.0 References: <20220425190040.2475377-1-yosryahmed@google.com> X-Mailer: git-send-email 2.36.0.rc2.479.g8af0fa9b8e-goog Subject: [PATCH v5 1/4] memcg: introduce per-memcg reclaim interface From: Yosry Ahmed To: Johannes Weiner , Michal Hocko , Shakeel Butt , Andrew Morton , Roman Gushchin Cc: David Rientjes , Tejun Heo , Zefan Li , Jonathan Corbet , Shuah Khan , Yu Zhao , Dave Hansen , Wei Xu , Greg Thelen , Chen Wandun , Vaibhav Jain , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Tim Chen , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Yosry Ahmed , Michal Hocko X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 114FEA0046 Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=smHSwP6G; spf=pass (imf25.hostedemail.com: domain of 35-9mYgoKCHw3txw3fmrjilttlqj.htrqnsz2-rrp0fhp.twl@flex--yosryahmed.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=35-9mYgoKCHw3txw3fmrjilttlqj.htrqnsz2-rrp0fhp.twl@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-Stat-Signature: ewjb8dced9k893g6fzhu4axf7y7argdi X-HE-Tag: 1650913249-18889 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Shakeel Butt Introduce a memcg interface to trigger memory reclaim on a memory cgroup. Use case: Proactive Reclaim --------------------------- A userspace proactive reclaimer can continuously probe the memcg to reclaim a small amount of memory. This gives more accurate and up-to-date workingset estimation as the LRUs are continuously sorted and can potentially provide more deterministic memory overcommit behavior. The memory overcommit controller can provide more proactive response to the changing behavior of the running applications instead of being reactive. A userspace reclaimer's purpose in this case is not a complete replacement for kswapd or direct reclaim, it is to proactively identify memory savings opportunities and reclaim some amount of cold pages set by the policy to free up the memory for more demanding jobs or scheduling new jobs. A user space proactive reclaimer is used in Google data centers. Additionally, Meta's TMO paper recently referenced a very similar interface used for user space proactive reclaim: https://dl.acm.org/doi/pdf/10.1145/3503222.3507731 Benefits of a user space reclaimer: ----------------------------------- 1) More flexible on who should be charged for the cpu of the memory reclaim. For proactive reclaim, it makes more sense to be centralized. 2) More flexible on dedicating the resources (like cpu). The memory overcommit controller can balance the cost between the cpu usage and the memory reclaimed. 3) Provides a way to the applications to keep their LRUs sorted, so, under memory pressure better reclaim candidates are selected. This also gives more accurate and uptodate notion of working set for an application. Why memory.high is not enough? ------------------------------ - memory.high can be used to trigger reclaim in a memcg and can potentially be used for proactive reclaim. However there is a big downside in using memory.high. It can potentially introduce high reclaim stalls in the target application as the allocations from the processes or the threads of the application can hit the temporary memory.high limit. - Userspace proactive reclaimers usually use feedback loops to decide how much memory to proactively reclaim from a workload. The metrics used for this are usually either refaults or PSI, and these metrics will become messy if the application gets throttled by hitting the high limit. - memory.high is a stateful interface, if the userspace proactive reclaimer crashes for any reason while triggering reclaim it can leave the application in a bad state. - If a workload is rapidly expanding, setting memory.high to proactively reclaim memory can result in actually reclaiming more memory than intended. The benefits of such interface and shortcomings of existing interface were further discussed in this RFC thread: https://lore.kernel.org/linux-mm/5df21376-7dd1-bf81-8414-32a73cea45dd@google.com/ Interface: ---------- Introducing a very simple memcg interface 'echo 10M > memory.reclaim' to trigger reclaim in the target memory cgroup. The interface is introduced as a nested-keyed file to allow for future optional arguments to be easily added to configure the behavior of reclaim. Possible Extensions: -------------------- - This interface can be extended with an additional parameter or flags to allow specifying one or more types of memory to reclaim from (e.g. file, anon, ..). - The interface can also be extended with a node mask to reclaim from specific nodes. This has use cases for reclaim-based demotion in memory tiering systens. - A similar per-node interface can also be added to support proactive reclaim and reclaim-based demotion in systems without memcg. - Add a timeout parameter to make it easier for user space to call the interface without worrying about being blocked for an undefined amount of time. For now, let's keep things simple by adding the basic functionality. [yosryahmed@google.com: worked on versions v2 onwards, refreshed to current master, updated commit message based on recent discussions and use cases] Signed-off-by: Shakeel Butt Co-developed-by: Yosry Ahmed Signed-off-by: Yosry Ahmed Acked-by: Johannes Weiner Acked-by: Michal Hocko Acked-by: Wei Xu Acked-by: Roman Gushchin Acked-by: David Rientjes Reviewed-by: Davidlohr Bueso --- Documentation/admin-guide/cgroup-v2.rst | 21 ++++++++++++ mm/memcontrol.c | 45 +++++++++++++++++++++++++ 2 files changed, 66 insertions(+) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 69d7a6983f78..19bcd73cad03 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1208,6 +1208,27 @@ PAGE_SIZE multiple when read back. high limit is used and monitored properly, this limit's utility is limited to providing the final safety net. + memory.reclaim + A write-only nested-keyed file which exists for all cgroups. + + This is a simple interface to trigger memory reclaim in the + target cgroup. + + This file accepts a single key, the number of bytes to reclaim. + No nested keys are currently supported. + + Example:: + + echo "1G" > memory.reclaim + + The interface can be later extended with nested keys to + configure the reclaim behavior. For example, specify the + type of memory to reclaim from (anon, file, ..). + + Please note that the kernel can over or under reclaim from + the target cgroup. If less bytes are reclaimed than the + specified amount, -EAGAIN is returned. + memory.oom.group A read-write single value file which exists on non-root cgroups. The default value is "0". diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 725f76723220..879f1716d6e9 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6355,6 +6355,46 @@ static ssize_t memory_oom_group_write(struct kernfs_open_file *of, return nbytes; } +static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, + size_t nbytes, loff_t off) +{ + struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); + unsigned int nr_retries = MAX_RECLAIM_RETRIES; + unsigned long nr_to_reclaim, nr_reclaimed = 0; + int err; + + buf = strstrip(buf); + err = page_counter_memparse(buf, "", &nr_to_reclaim); + if (err) + return err; + + while (nr_reclaimed < nr_to_reclaim) { + unsigned long reclaimed; + + if (signal_pending(current)) + return -EINTR; + + /* + * This is the final attempt, drain percpu lru caches in the + * hope of introducing more evictable pages for + * try_to_free_mem_cgroup_pages(). + */ + if (!nr_retries) + lru_add_drain_all(); + + reclaimed = try_to_free_mem_cgroup_pages(memcg, + nr_to_reclaim - nr_reclaimed, + GFP_KERNEL, true); + + if (!reclaimed && !nr_retries--) + return -EAGAIN; + + nr_reclaimed += reclaimed; + } + + return nbytes; +} + static struct cftype memory_files[] = { { .name = "current", @@ -6413,6 +6453,11 @@ static struct cftype memory_files[] = { .seq_show = memory_oom_group_show, .write = memory_oom_group_write, }, + { + .name = "reclaim", + .flags = CFTYPE_NS_DELEGATABLE, + .write = memory_reclaim, + }, { } /* terminate */ };