From patchwork Mon Aug 5 23:22:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13754220 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41FB3C3DA4A for ; Mon, 5 Aug 2024 23:22:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C67F46B0083; Mon, 5 Aug 2024 19:22:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C17CF6B0085; Mon, 5 Aug 2024 19:22:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ADF3D6B0088; Mon, 5 Aug 2024 19:22:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 8ECB36B0083 for ; Mon, 5 Aug 2024 19:22:47 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0397580803 for ; Mon, 5 Aug 2024 23:22:46 +0000 (UTC) X-FDA: 82419768774.26.4B21E4E Received: from mail-yw1-f169.google.com (mail-yw1-f169.google.com [209.85.128.169]) by imf30.hostedemail.com (Postfix) with ESMTP id 4C71E80023 for ; Mon, 5 Aug 2024 23:22:45 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=QiOTXQIB; spf=pass (imf30.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.128.169 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722900134; a=rsa-sha256; cv=none; b=6Ei3vwI04ncZPeleJX9dp5HPUsTC9POe4vlelQEH4jcqXdNr+oFegsYxGbTjJLBRulsb4W Vix7KjYm6sv6+Jjw/JELxRkWU3hnf6sjbqjWTYoWx/yzIB7vYqd9K94RUG/2oJjkpnk9tt i1WsowmmEdJON0V/GWxE1pHffin39wc= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=QiOTXQIB; spf=pass (imf30.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.128.169 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722900134; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=HPYAMMsP0uRNissPNWZSJAta6wImuOv5d58cqRo/XhU=; b=Ik2A5uzaR+qeXxtKVCvfmUdP1x0+5Ih8uuLp7+9xtshmIqpvN7vepWJk101k1nmO9KkCHR xbzDeGIVT2KExZKGzWDcHnEmyLYXZOsFKwy8MwS1JF6O/UxflSW1l1tSoWyyqEMbr4hcYx BskRWjbJyc6InS0kxHpVpi4ZeoDx7Zc= Received: by mail-yw1-f169.google.com with SMTP id 00721157ae682-65f9708c50dso1404497b3.2 for ; Mon, 05 Aug 2024 16:22:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722900164; x=1723504964; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=HPYAMMsP0uRNissPNWZSJAta6wImuOv5d58cqRo/XhU=; b=QiOTXQIBomU0hWy8zcFSaO/7oG/Q24kwXD9miZbyCRjzMuPb12JjfEn4+wHRbj15wO vYDQIaXhe5TTi65R0maRBHwGyIOiu7+eNrrLzKDYMQGyFL6l2w7a8hR7xFbREHmJM9pr JRs8+PnRraxTsWVKx8DCWFWlTNitnFVLQKjCrNRdLhH+Udz/Yd55SfAV32ZK22usYJ4z lhoFxdQnavbFQTcV2LGzYHKU1NoLEsMk42Y2whkd0oXArU6smz7vQgghrbgFvAOXWLlh NoBydBTSadoP+btUxDtyQoZTKBrRtTpDHV4Opg0ucH5s5Mf/R1L/yfMFwcN7rzG0ChlF RgMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722900164; x=1723504964; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=HPYAMMsP0uRNissPNWZSJAta6wImuOv5d58cqRo/XhU=; b=IT6XYPfNfVlrNBtIo4sN7okJPtfAgzXwPCMcM8dpvwgZ1/+PX21B8uiU5p4XWeU7KH 4lamLfeEbY4KvZ95yhN5q+g7CO0w1EbX2BSZIx44SXWdkSqZGh5oU9lC+4shs5/XxBh8 8MnBoYMVRT+IHtvVD7dbEa14gXNrE7Da4Jg8/JMRzyWerRqkzW+QGaNExCdVgFbaAUjg WVkOi7vH4LnKpEn40kYc5y9+zg1N5gsH63hB6snYOj9KOdOPL+uFuffHQIE3SuO9HeCp GhEj77xtB9fA3SiL6us42li4zjXjKOa5+Uhno2MxOOytisCz4XTU8LAgfj66KJgLS4Sd j7vw== X-Forwarded-Encrypted: i=1; AJvYcCU4ycb/1W0j6dEQeyZYkBQgG/RNo0GhiSRb9q7fv5cTd4npCjj2EnEuKKMrfIF6LRL1K42FVWhtL9Zp+xflZLWV6rI= X-Gm-Message-State: AOJu0YwBxar6ldUgxMKuDNiSmQfTcc3Cvt/xK3xP5cvVgq8TSf+8czyl wCYavm21nQcEVmGaDyzYe1f1zuLjzXdDF0yV2igC/9/A+yt/ZDCS X-Google-Smtp-Source: AGHT+IFPrHEO/KgU/leyHw2+KcPvaanTNyZl+zRAzc8NLNxP+vaHpLq7G/+mghISbxSoo5k/0kOQtg== X-Received: by 2002:a81:8501:0:b0:673:b39a:92f0 with SMTP id 00721157ae682-6896077619dmr143576897b3.11.1722900164182; Mon, 05 Aug 2024 16:22:44 -0700 (PDT) Received: from localhost (fwdproxy-nha-005.fbsv.net. [2a03:2880:25ff:5::face:b00c]) by smtp.gmail.com with ESMTPSA id 00721157ae682-68a0f4193desm13652377b3.6.2024.08.05.16.22.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Aug 2024 16:22:43 -0700 (PDT) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, yosryahmed@google.com, shakeel.butt@linux.dev, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, flintglass@gmail.com, chengming.zhou@linux.dev Subject: [PATCH v3 0/2] improving dynamic zswap shrinker protection scheme Date: Mon, 5 Aug 2024 16:22:41 -0700 Message-ID: <20240805232243.2896283-1-nphamcs@gmail.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-Stat-Signature: pahcjum8kcgxac6upm6ymjxrn6mxw1n5 X-Rspamd-Queue-Id: 4C71E80023 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1722900165-240726 X-HE-Meta: U2FsdGVkX1+uE+C/7nz08X1QXmKYhN6F+mqalOAeXVm/zaYfjq1ovA7Bxam4IzdCmzECv+36AWQp8eJFxYY4yMGXuQFnw7Z/iVcl4oAAIoCv2pN6RzF1r7BOQUAaQEnqwiT3+J2sxIU5MZEsgxhE6S+N1ei/MfBaDAgjHYZPdodhj8xD5FcJpNmJylsTcF5svkwbXoZ737LnbJGI0YnRRPKJwrPCELPtotX6Q2ufxxCXfRCDL0WGc387I/vwzD78cAJ0BqiYSuqklDTA9S7UhtzJRJ7ZObIJh621F6qDdo2PRP2qUJky+Rh8tAtxROKPmQxTIXpZV+2Vy32KjsDpXINXrLXZwTk3+dgN8DXD/h/C9SW3K8MfqN2vl0+AplP53Lhl0kGe8KNCqnSqG9vUa7XVjWkP4tUMpm5Q5C5oAHt0cGLC51fEj//i50fcQaTCguTHWuvFTJPAcVmAOMcXGPo7Rc3qvF2K8Q7J+WnLgDvOpNg0sVMjXYuzVpMDgie03KjEFhk+GzrQ3t9GNgXl8M7HzVc1gw5qOMZW/WDO5cD5e7y0U+HBeSOpeToa8u2COtU4zeuWVf7l/kGkVZGvBYD8PI46gs0cmp2yMqR+Zrxww9cexvPahwPObYRbmIEshFCVHcHLnDeJjrO6JSQ5sPpYHCOcoWIwIAR1bl6eCHZaLeldn/A3twMaWZt+tlq/+t9iWfJUjE2g+t8druaaBbk2EcahBYXG/CKrTF8JnaHdjZf2TQOY4RLvmagkgPnx2WiC+qJlhw0lH+3/axCnHdg7Qm/gVjG4+WiJoSVE6b/C5Mt9wl50v7s4rjaB4vWMEvfh3W+7Hcau5AlF9TBf1W6j2vQncplCtkpRFYQ7JYSq0Rx8/X8RIvmQrB6WxvC2Rd8z/wy9AB5fd+0mlKrV79PDCF5v7kBIyMtFZB5ypl6ZaUbYMwU01KuHYJQ7IMMXLj/l1kn2OzInCoKiEuc u7nqPwyV R+vJDZsBFLwHPl9k7y4gq1VEARlTJXvhEiYCzj1QbfBwg51Y0XJBq24if/zT+hiUo9pUlRiwF4syim627BmBoicezL/0X4+T3SWTfax20rUKXArKodfW6AJKtXWaHDjav5DsxIRJCgYpiSKkkofaednAipqEhIdk9A/VsfrrZDSCqdsqZcViHBgEtKUxB97dUZrv9QlpE+jN71J/KO5ob4+6P+vwdkJRlOctLQd3yiY8GXjM0zMmyKINa0jYsF9QLzDFREztqf2NpO/BxVGfPYMGdmSbK9XlaT1FtEsfYQT7T2w2cPFMK2e4E9oAi9i6Xxc7L/UXm6U0lcvXgQA4j2pGA5+E272yr/cxKqt8HHrpQtHISmc286TWX182LXRcCCQ0ScY8tK4u7pSI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.051973, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: v3: No (intended) functional change * Small cleanups, renamings, etc. (suggested by Yosry Ahmed) v2: * Add more details in comments, patch changelog, documentation, etc. about the second chance scheme and its ability to modulate the writeback rate (patch 1) (suggested by Yosry Ahmed). * Move the referenced bit (patch 1) (suggested by Yosry Ahmed). When experimenting with the memory-pressure based (i.e "dynamic") zswap shrinker in production, we observed a sharp increase in the number of swapins, which led to performance regression. We were able to trace this regression to the following problems with the shrinker's warm pages protection scheme: 1. The protection decays way too rapidly, and the decaying is coupled with zswap stores, leading to anomalous patterns, in which a small batch of zswap stores effectively erase all the protection in place for the warmer pages in the zswap LRU. This observation has also been corroborated upstream by Takero Funaki (in [1]). 2. We inaccurately track the number of swapped in pages, missing the non-pivot pages that are part of the readahead window, while counting the pages that are found in the zswap pool. To alleviate these two issues, this patch series improve the dynamic zswap shrinker in the following manner: 1. Replace the protection size tracking scheme with a second chance algorithm. This new scheme removes the need for haphazard stats decaying, and automatically adjusts the pace of pages aging with memory pressure, and writeback rate with pool activities: slowing down when the pool is dominated with zswpouts, and speeding up when the pool is dominated with stale entries. 2. Fix the tracking of the number of swapins to take into account non-pivot pages in the readahead window. With these two changes in place, in a kernel-building benchmark without any cold data added, the number of swapins is reduced by 64.12%. This translate to a 10.32% reduction in build time. We also observe a 3% reduction in kernel CPU time. In another benchmark, with cold data added (to gauge the new algorithm's ability to offload cold data), the new second chance scheme outperforms the old protection scheme by around 0.7%, and actually written back around 21% more pages to backing swap device. So the new scheme is just as good, if not even better than the old scheme on this front as well. [1]: https://lore.kernel.org/linux-mm/CAPpodddcGsK=0Xczfuk8usgZ47xeyf4ZjiofdT+ujiyz6V2pFQ@mail.gmail.com/ Nhat Pham (2): zswap: implement a second chance algorithm for dynamic zswap shrinker zswap: track swapins from disk more accurately include/linux/zswap.h | 16 +++---- mm/page_io.c | 11 ++++- mm/swap_state.c | 8 +--- mm/zswap.c | 108 ++++++++++++++++++++++++------------------ 4 files changed, 82 insertions(+), 61 deletions(-) base-commit: cca1345bd26a67fc61a92ff0c6d81766c259e522