From patchwork Thu Jun 27 10:55:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13714183 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7046C2BD09 for ; Thu, 27 Jun 2024 10:57:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2560B6B00A1; Thu, 27 Jun 2024 06:57:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2056C6B00A2; Thu, 27 Jun 2024 06:57:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0A64D6B00A3; Thu, 27 Jun 2024 06:57:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id DAF576B00A1 for ; Thu, 27 Jun 2024 06:57:35 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 99218A4685 for ; Thu, 27 Jun 2024 10:57:35 +0000 (UTC) X-FDA: 82276367670.20.21E4402 Received: from mail-yb1-f169.google.com (mail-yb1-f169.google.com [209.85.219.169]) by imf22.hostedemail.com (Postfix) with ESMTP id D7F51C000D for ; Thu, 27 Jun 2024 10:57:33 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=kl7qKAT5; spf=pass (imf22.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.219.169 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719485845; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=WsD/FARGvPOf7vgRzJuB2goC5N5PTWOZHU6FalHWVEQ=; b=lOM6dJSGFDSKd3vKw1IXy94Yn9raxl+rZbafAcfGtW+8VkCQrkjjmXbAvsbjmd/f2fxzQi hc0dPXUWMOKFWYhaUGztmjOK9y9jBqKsb9ctCN93b8rRymsfUheZYJMqo2kiwyghsbsbNs qhbeTZMOsLbM+7AFeG2rTVFBceeQ4VU= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=kl7qKAT5; spf=pass (imf22.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.219.169 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719485845; a=rsa-sha256; cv=none; b=v+n+xTPC82NwCcdtnnwrtnNj/QYzYgI9mYaV4vYVVP608KImYs8z56lg2YjZ+JR2SVxpTM 7JT9dRn+IQa2TATn/IdVyOIX0qOdOCqj7IjJkIsvrlxSy65PFhSjsI4/NSs155x0voZedx mXBsYZPZxcl2tHPABbTIN+upPxWxtmY= Received: by mail-yb1-f169.google.com with SMTP id 3f1490d57ef6-dfb05bcc50dso6843659276.0 for ; Thu, 27 Jun 2024 03:57:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719485853; x=1720090653; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=WsD/FARGvPOf7vgRzJuB2goC5N5PTWOZHU6FalHWVEQ=; b=kl7qKAT5AH3Kr5QPUJXnCBQMgVKPSswxuMstiNTHr9FTjYJAn9m4L5WvaK4KjhkLUq FfXGEkCXgYFLa67hFHhq57N9Np3fqmXf1ezQ7ioHu22lv5XL5KjPl9FwrukNq7NO7JJ7 oQMSG0Cn+IVUCeLr3aTe3Yzw9Hsm0EuDMjZY7kH9u30aVkxKVeCKu9EaKYflcqyIG5pr uBeWcms15SIRpxMWjkOxonBmdjcWY6vdceqk45KJ4KcJ5M3NDt5saW/D2DITzaL8lTNq Nqg4z4cOvTCwr2By/H7oK/Qnjfr4sOgm3k7DyW1Gpy08BYkG/HriV6SLg0GQGMyzsG/j V0vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719485853; x=1720090653; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=WsD/FARGvPOf7vgRzJuB2goC5N5PTWOZHU6FalHWVEQ=; b=M1UnerlPLJVnlVjPdtr0Yi6xQS7Tg9i/ZRqPJr39RbC0oBbZ4JNlZa5mB4Z6E0Mock kKhEqexvLMM7TzCLdclAPrzjElruKcIEFeht7FFuQ6d4GtKLEMJ9h4P0KTzpSumeEzPO o5vHfUL9yKe+a2Efcs8UyR2ZgPZLsBJG9dtWk0eq+cwgsaGNWrN76veWOJ0OAiT/Utv8 xN2HWninUFhyyRMV8ut1qUrzymJwdGtnNJpMHXkJewZ4dKUluQEwkcgIk2W2Gkox/+dh 5nwAKoXPyx5xqw67kuOokav/h3Ta3FAOBvJXJxwK7TibLi77DaRojM7a9qnBRgwQIvcH ym/Q== X-Forwarded-Encrypted: i=1; AJvYcCVNxRtENWs/BMMa4+AQUtt4/0J+F6QekIuZxhvPgeMESavmOiviqidLQyWfLEWDYhJFY9pest10f2rm2Ih7QjEf5Vg= X-Gm-Message-State: AOJu0Yw2DTbZCYuQVjJSJkZ/Om2WCKzBsbcs8gTWaQWp7YFxlzdCAmuU wu1opTf861hBVbFCriXL0E2aBB1Q1eXvv9jVfO1zxhNKcGqI+E4X X-Google-Smtp-Source: AGHT+IGwVk/Fd58WrBQnRqaISapyqM7miJqYT0qpOXhZSoqezURiR7HMbvfIYj8GFrb4XddqStxYsQ== X-Received: by 2002:a25:2e4b:0:b0:dff:d79:cbb9 with SMTP id 3f1490d57ef6-e0303ff4702mr11132219276.63.1719485852807; Thu, 27 Jun 2024 03:57:32 -0700 (PDT) Received: from localhost (fwdproxy-ash-004.fbsv.net. [2a03:2880:20ff:4::face:b00c]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-44641fd6fc9sm4742631cf.81.2024.06.27.03.57.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Jun 2024 03:57:32 -0700 (PDT) From: Usama Arif To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, shakeel.butt@linux.dev, david@redhat.com, ying.huang@intel.com, hughd@google.com, willy@infradead.org, yosryahmed@google.com, nphamcs@gmail.com, chengming.zhou@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, Usama Arif Subject: [PATCH v7 0/2] mm: store zero pages to be swapped out in a bitmap Date: Thu, 27 Jun 2024 11:55:28 +0100 Message-ID: <20240627105730.3110705-1-usamaarif642@gmail.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: D7F51C000D X-Stat-Signature: nhrm8yitiygcpf31egs6bwgb37ycak5f X-HE-Tag: 1719485853-820221 X-HE-Meta: U2FsdGVkX19U6FN/M/A0VYib4+0YIJSTnJbLtv1HY+uOzXqdgMvveiVG0q2DHqIpSXZEresN6BAfeZWe4QNXBUV6l+5w3t5fRZgoH0zq9UqZEajCSSIzit3TX/Y14QqHZJR89+GEKMKhNceXtMhx/X9e/lwM/9QFsVZs4FkqZCTKbl0J3ZLvs0o6s//Dm6cM8wCTEhBOM9JDCjwUWPTpo0IZK366N5kppO6jtpbsQvefAbKrxWGNJF4RhFsjFQNu1p0sabXloH9Yw0FJuM9UwixRh7NEtF5ukhRSKgqa2TkeSE5i1ModYTABRX3COvKKPBTiaCmz0hJSDWwy58wFOjEbQrl6icQYht024zrnAMYXCKFSVOAkkTcUo3kDmGK6CuaD0lC/Uc1a3mTHTJ6E/N4zsDODxYrbS9JfD+pEFxJbz+jKFGyBGp3XT/AVpDv1W29PiB7OJnCO3hhmmuzPidy6NwsazvzoOA5RKL6BXFt0UyCMjKqlJhfqLvUNeNnMml9opBHLIzAoLPmCYE0i5Vnbn6wFRqifItFhmTeJRc2/MjImm0F9gEPp8YjXpqUl3IN/EAwiBHaKuwo98SwoFJX30/sEr4lwklHLXocniy1kz38SLdGOKS6yPI4jIBAhhHUB+O4X7hzNfAHO+08ycrG+qwR0M0A2u8MugYktJ3LeRxMdvFFUu0Zy9fXlPwQEohzf0dFhodje0s+KaRlQG/ycc6pR1tiPuZbr8Ztxurey6OBsr2S102ZeXezPRhlhtjGKYNSB6Y0sgLXbxFGlb4iAMCZ/Zb/P15k63/C/AlqjWWxKy41tQ59xVQhdjkaGq40//7hGHdiRDWD6fPK1sNUX31WJHoT1jC0/+j+YDqYFIn9c44j+k9tIYvRogk7SChj5nLKlQCckW9SvcD2GU9E8VEAJuMf99YatWJshEJkQXBoDCrUPbosHaFhYqmAP5he1wvsnI7qYHis3tXR WMjDegfk dbSSE0IU/d/lyeBwTFZs5sknSIMpqNDjpIQGfWNmCJIJz+bJkURU6TEU5s+QpqK8W384z9wECFflAYgK6Bp1Q35Z68xjmQzf/RRMtSAoO0DoLkTXC4ytSYlmn92c38gMeR/m2eE8yd5Pn43rSMnWUNO1iEUpHT/oyahiva0Q6xoup8618B9IM9qPSlBZ3CjCIGrnP/Vgk9YFxZGJM4BiUdwbFZW1Hy4+PSbL3iyb8G600c4lqVppcEATnLw1Kt+tCkQ/iCLKZOFZMB+Na/y78Hq4s597tJS4fBUyiZ4LKLb7BAe4aVzezWleQsUX5JXb+5fTQdN5qBTIJ14AQhMZbu5VlcS2DfBytZ7SHLBDnBHpBuJoRw5BYVeq30yRMrnpk/04L X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: As shown in the patchseries that introduced the zswap same-filled optimization [1], 10-20% of the pages stored in zswap are same-filled. This is also observed across Meta's server fleet. By using VM counters in swap_writepage (not included in this patchseries) it was found that less than 1% of the same-filled pages to be swapped out are non-zero pages. For conventional swap setup (without zswap), rather than reading/writing these pages to flash resulting in increased I/O and flash wear, a bitmap can be used to mark these pages as zero at write time, and the pages can be filled at read time if the bit corresponding to the page is set. When using zswap with swap, this also means that a zswap_entry does not need to be allocated for zero filled pages resulting in memory savings which would offset the memory used for the bitmap. A similar attempt was made earlier in [2] where zswap would only track zero-filled pages instead of same-filled. This patchseries adds zero-filled pages optimization to swap (hence it can be used even if zswap is disabled) and removes the same-filled code from zswap (as only 1% of the same-filled pages are non-zero), simplifying code. This patchseries is based on mm-unstable. [1] https://lore.kernel.org/all/20171018104832epcms5p1b2232e2236258de3d03d1344dde9fce0@epcms5p1/ [2] https://lore.kernel.org/lkml/20240325235018.2028408-1-yosryahmed@google.com/ --- v6 -> v7: (Yosry and David): - Change to kvmalloc_array for zeromap allocation instead of kvzalloc as it does an additional overflow check, and use sizeof(unsigned long) for allocation size calculation to take into account 32 bit kernels. v5 -> v6 (kernel test robot ): - change bitmap_zalloc/free to kvzalloc/free as a very large swap file will result in the allocation order to exceed MAX_PAGE_ORDER retulting in bitmap_zalloc to fail. v4 -> v5 (Yosry): - Correct comment about using clear_bit instead of bitmp_clear. - Remove clearing the zeromap from swap_cluster_schedule_discard and swap_do_scheduled_discard. v3 -> v4: - remove folio_start/end_writeback when folio is zero filled at swap_writepage (Matthew) - check if a large folio is partially in zeromap and return without folio_mark_uptodate so that an IO error is emitted, rather than checking zswap/disk (Yosry) - clear zeromap in swap_free_cluster (Nhat) v2 -> v3: - Going back to the v1 version of the implementation (David and Shakeel) - convert unatomic bitmap_set/clear to atomic set/clear_bit (Johannes) - use clear_highpage instead of folio_page_zero_fill (Yosry) v1 -> v2: - instead of using a bitmap in swap, clear pte for zero pages and let do_pte_missing handle this page at page fault. (Yosry and Matthew) - Check end of page first when checking if folio is zero filled as it could lead to better performance. (Yosry) Usama Arif (2): mm: store zero pages to be swapped out in a bitmap mm: remove code to handle same filled pages include/linux/swap.h | 1 + mm/page_io.c | 113 ++++++++++++++++++++++++++++++++++++++++++- mm/swapfile.c | 20 ++++++++ mm/zswap.c | 86 +++----------------------------- 4 files changed, 141 insertions(+), 79 deletions(-)