From patchwork Thu Nov 2 20:02:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13444171 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D92FC4332F for ; Thu, 2 Nov 2023 20:02:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A5A26280006; Thu, 2 Nov 2023 16:02:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9E0B08D000F; Thu, 2 Nov 2023 16:02:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 85ACB280006; Thu, 2 Nov 2023 16:02:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6FF968D000F for ; Thu, 2 Nov 2023 16:02:07 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 4BE291CA954 for ; Thu, 2 Nov 2023 20:02:07 +0000 (UTC) X-FDA: 81414085494.08.2676EBC Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf27.hostedemail.com (Postfix) with ESMTP id 6143840014 for ; Thu, 2 Nov 2023 20:02:04 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=nWND0WvP; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf27.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698955324; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=ZuHikmRqCjHm13by+oAQr/aA53zai8UWN0Q4ojl/okQ=; b=kO+ww0I7HKWkSL7Woy6eOuez1nabPWEATVWd5d56PcrB2uYWakqL7kPvO32VfFriz5UrVy ToxPvdw95WAQuMeL2ar1HBprPNKodLSD4EhDYIzVD55KqSQ2wZ0Z3SfWoHSWUiAe1ZSwU8 7CZZDj4uteNEhPLfPf2a2DvD1t63o04= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=nWND0WvP; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf27.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698955324; a=rsa-sha256; cv=none; b=MrG3uNdAeB2qHyM1DnDefdWwTvwoY4h2NuddWD2FrLCe2FqYCc6BD5Hts8GDLquXMuOVWJ S54nA+I/aOefvdCIzifWXgWr3muwFCXfFWYu1g5DvI9Y8M1JZeh5oVoHQAe81EKFF4Hq1o 5XM+U5wz3Km0sHC+PT/Rfmh9d3omRbQ= Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-6c34e87b571so365753b3a.3 for ; Thu, 02 Nov 2023 13:02:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1698955323; x=1699560123; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=ZuHikmRqCjHm13by+oAQr/aA53zai8UWN0Q4ojl/okQ=; b=nWND0WvPyAalyzdWoIdhX38vxp0so2wdmFywqvYy6A27CisQ3eP9SXrDXreGvRJryu YRTpoNWChSnottdTcfeztgw7Czo3AK1eVwOaKjpat3xMXPQVJeW2BBHWo1NrfCVn2z3S xWEPMOVlmn5RcIx9jO/N6wxABbNVBxyAiMn6J9uUv02VJPYcfu13loj6i569attZfGWI zfUMjZeJEuIXfL+KkOnLvbnMmXTzwmt0z9o8YPHSJI0JQ20Q7Z+A/t9sA3Y569mbTWZK FUgsQgJCEUl5SEEnN+9dRYWrhu3w4aQLlQgnE5FU00aWjwtL+XXf3ul3lxM6imjA8hkB 6sLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698955323; x=1699560123; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ZuHikmRqCjHm13by+oAQr/aA53zai8UWN0Q4ojl/okQ=; b=ECeTUTgShgiRdLnpIs7umhciLXcjb4BQkJU+UgscSTyEK4XEngPDT4OGtGypYRM2sC FMAKS+5n8E/R8uI9S5nkhL6ZieKFOnLRI1YExFO+0PsQCTYN3WIW0gtIqqbTEVMLio5i yjuH4tH9tC7Wf9cJJpZcfc650hFhFsEfY/aipA4g7IT44oIEiy9TDrsOb+22HJ4zwlXa 1DDqw+eb30ICCKbbRgbtP1fD9uZwoX/itBLw2/RMMN+Lc5TpjyKji4kppzg2vkvuxilq 6CdqQb5TYr3upkza0TpO1rbB0bhRk9euCC8wE1zb2gizxA8jJh+ESdHJI/6UnnkyD9ba UKDQ== X-Gm-Message-State: AOJu0YybgcxAlUjSWrhNBid2yZQxZS+QB5MZog9n2Yqg3jhHrOjTvqLK dFuBkparHFT3TA+DcS21gm4= X-Google-Smtp-Source: AGHT+IGh0NPW9G4Gl6NZHVh96UIpJkOaC5QTzf5jtxemRwXtmKKMWA/99myN8VpWsfw0G4aEXKNrBg== X-Received: by 2002:a05:6a20:9189:b0:156:e1ce:d4a1 with SMTP id v9-20020a056a20918900b00156e1ced4a1mr20346801pzd.9.1698955322912; Thu, 02 Nov 2023 13:02:02 -0700 (PDT) Received: from localhost (fwdproxy-prn-019.fbsv.net. [2a03:2880:ff:13::face:b00c]) by smtp.gmail.com with ESMTPSA id k13-20020a056a00134d00b006c34274f66asm129637pfu.102.2023.11.02.13.02.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Nov 2023 13:02:02 -0700 (PDT) From: Nhat Pham To: akpm@linux-foundation.org Cc: tj@kernel.org, lizefan.x@bytedance.com, hannes@cmpxchg.org, cerasuolodomenico@gmail.com, yosryahmed@google.com, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, hughd@google.com, corbet@lwn.net, konrad.wilk@oracle.com, senozhatsky@chromium.org, rppt@kernel.org, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, david@ixit.cz Subject: [RFC PATCH v2] zswap: memcontrol: implement zswap writeback disabling Date: Thu, 2 Nov 2023 13:02:02 -0700 Message-Id: <20231102200202.920461-1-nphamcs@gmail.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Rspamd-Queue-Id: 6143840014 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: mnfy7x4pmmyffho6cd3n9c3thxqyy1fd X-HE-Tag: 1698955324-338358 X-HE-Meta: U2FsdGVkX18a+o6Walr0syFFJDo4QZgkRuDW23Kef+CLUMoF/UHKHCfEUG9gpuPjBIUU8DHwwwwrNBhmhqrFV5bFt3gAe339umPI1RfpybwVZ+ao2f2QC9Qk17Wi7kRgcYelMbrwmMOI2MAJv6DbWEjMiZ1yAXBQ7BA/neUxQV4iJMojG8tEUz5GMfA5/VF2kMAKMcvbFYJs1boD/UjIxtjj3/ldFkNI23QVlfI7X1WpoHEQMzLlGzxcnU0O3n+QBZovpsx3FW544Ahe2i0fxHERvYJU728ivm5cdP4kS0x6RVgNFHy8fMdLajQdU6fjTDrgHJZ7LhynwOQEijrlSjYjjuqiGIJA4TnOz6lNSoKRzsRYFdzlnQWM/hOP0dJT4WWrwGR3wXLsPsOoKH03YQywOQmmOf2/J0AitvFUpMckhhAwsLmCZWgKUPcyQ1fz2r5y6rj2fWcZB9sZMeBepjZJMVaYwVagFKtu0YZYLGGuKJ4ngWyFjypWckOwZGHl6XoLVme4Tdk/nxRfmbt/gjq+e7V5aiJUBZA+fSm54qJjHJqb9wgBOybJl5yc8rn28wY1H9eA7qlscOtrO+EvYch2X9hrRXcfHqqZWmx9F8KhRqaLDZgSIKcUeMdOIHEsSfVlt8s/kf65SsDqYiwIPlLLrsTlP5ua7coQwEUI2Xo9EU6hiGCWpHOtQNoGYCMPpjU9lETSDYcu2tXZB8vuwu5boiwTRSwt/H3teG4W8eWiOjuzGpeuuXzIu95NeinsKQQCrFT8T9+ek7QsjTjt1pXYW+QhyLWi7aI9oTmE+0F/9o3fTCaS9CFDtaDmyPoZuR2Q/HZbXbMvXwPVx541S+SY3L9pfwKYUUV/GR43YZtwJU7S1TXFwU1xl6Irzf6nEp9uPxkYOH7tVSLaoxu3lDyqBnTUJiqEtGQLplFNfmumXR0ZbKBGeZW8gcu92rmsMDR4M3Yvg+v7vQNL4zL lfso9XQy DAeR+iSOZdJZ1qCKNd1vvgNj/DBp5ho6YZqgok8tD+SXNrO8q2LQVs52htp2JmwDafFNA7jX4flzUzjcoOq+qmBJLLtOeT6hDVBYKqFPJf4WUR1eWpyyIIchwR6FYlnS/cyGVlHY8NdSwmUqvCuV8NR75sckipAMK7/PIkfHOUEkOFBKgwGM8BoEgE6aimsWOlv+aZ1ab7IW8MEIivFtm8AUTHZAYUa/RJlSQIZjMgzkWByf1bEXR1zb+XY2iCFhYrXRdPs6U7+rmyBvzql9bnw/RprzJgXkFTmlmPbmHcRjWK3VHPzNvGk5ujziZ3t2azwSjbRhMJisbm7iGMCrPyLwoEG/bjHEpKr/MZuFuX4VR4qdsLTVVKAMTMTNDqTdgndGtRclwa0CYJiKXPdWDyGnbeiMfm7ZUn+58I1OGuvkppUZy/WxL3swKMRGxoGX8Gf6NiHY/rmej2kF42aew8x/oGnPQrUAhLdsZHFIBc1w6ROA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: During our experiment with zswap, we sometimes observe swap IOs due to occasional zswap store failures and writebacks-to-swap. These swapping IOs prevent many users who cannot tolerate swapping from adopting zswap to save memory and improve performance where possible. This patch adds the option to disable this behavior entirely: do not writeback to backing swapping device when a zswap store attempt fail, and do not write pages in the zswap pool back to the backing swap device (both when the pool is full, and when the new zswap shrinker is called). This new behavior can be opted-in/out on a per-cgroup basis via a new cgroup file. By default, writebacks to swap device is enabled, which is the previous behavior. Note that this is subtly different from setting memory.swap.max to 0, as it still allows for pages to be stored in the zswap pool (which itself consumes swap space in its current form). Suggested-by: Johannes Weiner Signed-off-by: Nhat Pham --- Documentation/admin-guide/cgroup-v2.rst | 11 +++++++ Documentation/admin-guide/mm/zswap.rst | 6 ++++ include/linux/memcontrol.h | 17 +++++++++++ mm/memcontrol.c | 38 +++++++++++++++++++++++++ mm/page_io.c | 6 ++++ mm/shmem.c | 3 +- mm/zswap.c | 9 ++++++ 7 files changed, 88 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 606b2e0eac4b..18c4171392ea 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1672,6 +1672,17 @@ PAGE_SIZE multiple when read back. limit, it will refuse to take any more stores before existing entries fault back in or are written out to disk. + memory.zswap.writeback + A read-write single value file which exists on non-root + cgroups. The default value is "1". + + When this is set to 0, all swapping attempts to swapping devices + are disabled. This included both zswap writebacks, and swapping due + to zswap store failure. + + Note that this is subtly different from setting memory.swap.max to + 0, as it still allows for pages to be written to the zswap pool. + memory.pressure A read-only nested-keyed file. diff --git a/Documentation/admin-guide/mm/zswap.rst b/Documentation/admin-guide/mm/zswap.rst index 522ae22ccb84..b987e58edb70 100644 --- a/Documentation/admin-guide/mm/zswap.rst +++ b/Documentation/admin-guide/mm/zswap.rst @@ -153,6 +153,12 @@ attribute, e. g.:: Setting this parameter to 100 will disable the hysteresis. +Some users cannot tolerate the swapping that comes with zswap store failures +and zswap writebacks. Swapping can be disabled entirely (without disabling +zswap itself) on a cgroup-basis as follows: + + echo 0 > /sys/fs/cgroup//memory.zswap.writeback + When there is a sizable amount of cold memory residing in the zswap pool, it can be advantageous to proactively write these cold pages to swap and reclaim the memory for other use cases. By default, the zswap shrinker is disabled. diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 95f6c9e60ed1..e3a3a06727dc 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -219,6 +219,12 @@ struct mem_cgroup { #if defined(CONFIG_MEMCG_KMEM) && defined(CONFIG_ZSWAP) unsigned long zswap_max; + + /* + * Prevent pages from this memcg from being written back from zswap to + * swap, and from being swapped out on zswap store failures. + */ + bool zswap_writeback; #endif unsigned long soft_limit; @@ -1615,6 +1621,12 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order, { return 0; } + +static inline bool mem_cgroup_swap_disk_enabled(struct mem_cgroup *memcg) +{ + return false; +} + #endif /* CONFIG_MEMCG */ static inline void __inc_lruvec_kmem_state(void *p, enum node_stat_item idx) @@ -1931,6 +1943,7 @@ static inline void count_objcg_event(struct obj_cgroup *objcg, bool obj_cgroup_may_zswap(struct obj_cgroup *objcg); void obj_cgroup_charge_zswap(struct obj_cgroup *objcg, size_t size); void obj_cgroup_uncharge_zswap(struct obj_cgroup *objcg, size_t size); +bool mem_cgroup_zswap_writeback_enabled(struct mem_cgroup *memcg); #else static inline bool obj_cgroup_may_zswap(struct obj_cgroup *objcg) { @@ -1944,6 +1957,10 @@ static inline void obj_cgroup_uncharge_zswap(struct obj_cgroup *objcg, size_t size) { } +static inline bool mem_cgroup_zswap_writeback_enabled(struct mem_cgroup *memcg) +{ + return false; +} #endif #endif /* _LINUX_MEMCONTROL_H */ diff --git a/mm/memcontrol.c b/mm/memcontrol.c index e43b5aba8efc..b68c613c23a9 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5545,6 +5545,7 @@ mem_cgroup_css_alloc(struct cgroup_subsys_state *parent_css) WRITE_ONCE(memcg->soft_limit, PAGE_COUNTER_MAX); #if defined(CONFIG_MEMCG_KMEM) && defined(CONFIG_ZSWAP) memcg->zswap_max = PAGE_COUNTER_MAX; + WRITE_ONCE(memcg->zswap_writeback, true); #endif page_counter_set_high(&memcg->swap, PAGE_COUNTER_MAX); if (parent) { @@ -8177,6 +8178,12 @@ void obj_cgroup_uncharge_zswap(struct obj_cgroup *objcg, size_t size) rcu_read_unlock(); } +bool mem_cgroup_zswap_writeback_enabled(struct mem_cgroup *memcg) +{ + return cgroup_subsys_on_dfl(memory_cgrp_subsys) && memcg + && READ_ONCE(memcg->zswap_writeback); +} + static u64 zswap_current_read(struct cgroup_subsys_state *css, struct cftype *cft) { @@ -8209,6 +8216,31 @@ static ssize_t zswap_max_write(struct kernfs_open_file *of, return nbytes; } +static int zswap_writeback_show(struct seq_file *m, void *v) +{ + struct mem_cgroup *memcg = mem_cgroup_from_seq(m); + + seq_printf(m, "%d\n", READ_ONCE(memcg->zswap_writeback)); + return 0; +} + +static ssize_t zswap_writeback_write(struct kernfs_open_file *of, + char *buf, size_t nbytes, loff_t off) +{ + struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); + int zswap_writeback; + ssize_t parse_ret = kstrtoint(strstrip(buf), 0, &zswap_writeback); + + if (parse_ret) + return parse_ret; + + if (zswap_writeback != 0 && zswap_writeback != 1) + return -EINVAL; + + WRITE_ONCE(memcg->zswap_writeback, zswap_writeback); + return nbytes; +} + static struct cftype zswap_files[] = { { .name = "zswap.current", @@ -8221,6 +8253,12 @@ static struct cftype zswap_files[] = { .seq_show = zswap_max_show, .write = zswap_max_write, }, + { + .name = "zswap.writeback", + .flags = CFTYPE_NOT_ON_ROOT, + .seq_show = zswap_writeback_show, + .write = zswap_writeback_write, + }, { } /* terminate */ }; #endif /* CONFIG_MEMCG_KMEM && CONFIG_ZSWAP */ diff --git a/mm/page_io.c b/mm/page_io.c index cb559ae324c6..5e606f1aa2f6 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -201,6 +201,12 @@ int swap_writepage(struct page *page, struct writeback_control *wbc) folio_end_writeback(folio); return 0; } + + if (!mem_cgroup_zswap_writeback_enabled(folio_memcg(folio))) { + folio_mark_dirty(folio); + return AOP_WRITEPAGE_ACTIVATE; + } + __swap_writepage(&folio->page, wbc); return 0; } diff --git a/mm/shmem.c b/mm/shmem.c index cab053831fea..e5044678de8b 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1514,8 +1514,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) mutex_unlock(&shmem_swaplist_mutex); BUG_ON(folio_mapped(folio)); - swap_writepage(&folio->page, wbc); - return 0; + return swap_writepage(&folio->page, wbc); } mutex_unlock(&shmem_swaplist_mutex); diff --git a/mm/zswap.c b/mm/zswap.c index 260e01180ee0..42a478d1a21f 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -590,6 +590,9 @@ static unsigned long zswap_shrinker_scan(struct shrinker *shrinker, struct zswap_pool *pool = shrinker->private_data; bool encountered_page_in_swapcache = false; + if (!mem_cgroup_zswap_writeback_enabled(sc->memcg)) + return SHRINK_STOP; + nr_protected = atomic_long_read(&lruvec->zswap_lruvec_state.nr_zswap_protected); lru_size = list_lru_shrink_count(&pool->list_lru, sc); @@ -620,6 +623,9 @@ static unsigned long zswap_shrinker_count(struct shrinker *shrinker, struct lruvec *lruvec = mem_cgroup_lruvec(memcg, NODE_DATA(sc->nid)); unsigned long nr_backing, nr_stored, nr_freeable, nr_protected; + if (!mem_cgroup_zswap_writeback_enabled(memcg)) + return 0; + #ifdef CONFIG_MEMCG_KMEM cgroup_rstat_flush(memcg->css.cgroup); nr_backing = memcg_page_state(memcg, MEMCG_ZSWAP_B) >> PAGE_SHIFT; @@ -935,6 +941,9 @@ static int shrink_memcg(struct mem_cgroup *memcg) struct zswap_pool *pool; int nid, shrunk = 0; + if (!mem_cgroup_zswap_writeback_enabled(memcg)) + return -EINVAL; + /* * Skip zombies because their LRUs are reparented and we would be * reclaiming from the parent instead of the dead memcg.