From patchwork Fri Aug 2 12:20:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13751514 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25AEAC52D6D for ; Fri, 2 Aug 2024 12:21:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 67B346B007B; Fri, 2 Aug 2024 08:21:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 62AFF6B0083; Fri, 2 Aug 2024 08:21:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4CB976B0085; Fri, 2 Aug 2024 08:21:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 30ED56B007B for ; Fri, 2 Aug 2024 08:21:00 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C3450407AA for ; Fri, 2 Aug 2024 12:20:59 +0000 (UTC) X-FDA: 82407214638.20.FE79C13 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf02.hostedemail.com (Postfix) with ESMTP id D0D1880025 for ; Fri, 2 Aug 2024 12:20:57 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=C41183ui; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722601193; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6q/Vgd/HJjjNMutQgVmDvwDRdznDYvNQkKPVmlkjWvI=; b=ViCZSUJ0UOkp63t3lZabrmmgf0AdDOpfQIgF2QEIRzKYeSm6ZS42b9cfK1gvZZAXeYqvQS eB0F49tsuh04Z9zWpnz3xQuDQkAxKg/M889AHU5E4Po3TZm83ECg5EMA0ctTfPSS8SCj7A s8tNTdpkm+cXNGv6FYNlVokyaLqLBTc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722601193; a=rsa-sha256; cv=none; b=ygSpgh5kS4bTZmCUd2RFRUlbBZj7eVttRekS9SjKlUwaBS0hI9Ir0q3XZjj1awMvQ472hh VL66fMZ4BAYeY0y1P2lxhDJ1M0EGrJiJxUbMXVCONuea8R4J6PoHD94yAxXSV7NO7WiPN6 yevf8fKJmxZ50HZd1DwxVZMFCHAXyoA= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=C41183ui; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=21cnbao@gmail.com Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-1fc47abc040so56119625ad.0 for ; Fri, 02 Aug 2024 05:20:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722601256; x=1723206056; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6q/Vgd/HJjjNMutQgVmDvwDRdznDYvNQkKPVmlkjWvI=; b=C41183uiwo4IRvKB54XfceIcatndFH7JXQF7+XASnYbs53EIosfAgdodRDJVhK9n46 bGnGMUyyh0YCI1lmXrmrmX88ZKJweuIQLZjw0oFTuXzGVOile4NP1MefOsEM7QIZuGXs J1CBjNvclLUS307hcokw8qF3hPxkEeRdtJKlLRj1wvYiYh7M2YdsySVEaiHJbfG5xOgt STTdWL0OsTvwqpRtIU+RVBGNXk92VDdOn/64eXypjMu1lgoZUVtWmBpGXgKBuyAfJH9Q kS8WgROa9RANDPq9jj6lvdI/53uuNTgyf3duTWG/NzitlUlF4/+rNtp6YzsAO0/XNZRL HtTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722601256; x=1723206056; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6q/Vgd/HJjjNMutQgVmDvwDRdznDYvNQkKPVmlkjWvI=; b=vFxTpR5PLVWE8Yw3m1udQdsbX6KuZPVtZX876Pui5YjbDxKMEO1ngvZQ2URSKjIzp+ BThqyRdnRwDxVvTXEeYzIzZFaOhcQMb2vBwRSeWeiy9R2gVN47lfZaUIm9bkYQtHvZ/1 e4TnhW+qKs/k0xPuJgm/nZ17SfVPGFCP7ExwngQy84QI5/U2Z+bDeWfFQN/YBV4N1m40 H/205xR/2Gw6NiUPeyWLIjX0Hb79OnN7dkQOZagBnrrTlBQGII/MDlL5Cx78rj+Sr8cq B848Av6ARUKdIoJ40mkyZPV8bTHsolkSvZtjSWHtiQ5dZOOHUjelSYNsD2GSP3ZiFLG1 K0sA== X-Forwarded-Encrypted: i=1; AJvYcCWzINKg9cxkCa5ccrv5ih0JMS1CJzbIbW9AssG4y4vHjKG37tbizfouHjvy6baDrtq3hKeiSXUpeLZZ5fmJXSH6zgA= X-Gm-Message-State: AOJu0Yzp7DWTrMrj/7AhLlE9V8WJ1joBDNdW//eHi0tlZYjU91kHeQII oHgVptfzt4PdAmHuSs6HiaafO8A4Wsz9UgLafPxgPULNoeypkfUT X-Google-Smtp-Source: AGHT+IHvCwKcMa1WaAaEyOSrgbVMbhDS/GTyK57FYlX0r3AtXnqExmED+EmlnKkgiLfMX0DbO2tSVQ== X-Received: by 2002:a17:903:1c2:b0:1fd:63d7:5d21 with SMTP id d9443c01a7336-1ff574218c3mr37981545ad.47.1722601256210; Fri, 02 Aug 2024 05:20:56 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1ff590600e4sm15841875ad.144.2024.08.02.05.20.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Aug 2024 05:20:55 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hannes@cmpxchg.org, hughd@google.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, mhocko@suse.com, minchan@kernel.org, nphamcs@gmail.com, ryan.roberts@arm.com, senozhatsky@chromium.org, shakeel.butt@linux.dev, shy828301@gmail.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, xiang@kernel.org, ying.huang@intel.com, yosryahmed@google.com, hch@infradead.org Subject: [PATCH v6 0/2] mm: Ignite large folios swap-in support Date: Sat, 3 Aug 2024 00:20:29 +1200 Message-Id: <20240802122031.117548-1-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240726094618.401593-1-21cnbao@gmail.com> References: <20240726094618.401593-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: D0D1880025 X-Stat-Signature: 67q8nibxp5k9r64o1ds8tdendofc89fw X-Rspam-User: X-HE-Tag: 1722601257-878890 X-HE-Meta: U2FsdGVkX1/LevlIQND/2ncB0MBcm97iSFGQOeqWSSYmhnXN2fFLL3DVQ2MZ+Hf+rUxMBz1hKLvNr1WEdy0zj8mi1Ur6i5+fq0QteGhPaTz5pAy7+NcqEFxfB6LB+KKmtWcooLiuDONt6uytF0yw5UlX2vI4K0gj9TQptvcbmBqwv4dLmvBWzTUZQwsyaXbiratMzTdYO30/fC8WH2bwHt09nnJufFkSJmZ5qnBymIN/9rUyKJStjixFD7/FKZhMUQFe59io6KXwkQZ3sz5cbGt8PLyamqLyflIDC8kr0qw6uUGV5uh3k77/+cmvn5kQEUpF6F0Z96HFxqzgWHGPw6W0XuzBgcjdB0crz3Tm0doaK6E+5tlF2sCq9409qDd1ZpKslSPjNIJ7hzuT/vZuy5RX1++X6nNbogHPAoGqWt4rM35wI4ladekvF6FHRIG3e03raNu5agRZTGlgtm9/gyysF6VpcbIrCH6sddyB/b8KcMGBY7TNEfdmqJZpbK673lrqviVDtn9q3OwynFXNbH0iNICEP9souHD/l1rGkaLyHViOcvlqddUHsmDiaginmOxB/b0lXgGXm0I7O8QD3bjfKvCP6xVxiQEuShhRR5dwZaUeQywOStFe2wglYN3lacXcrx8SsXvT/Q29SxhfHKxnBXczZc7lwXmhgcVauZSrz+eV/aM2acsQqFtIRy/rUL5gugVgRKOVv3+byuVPjR/2eDdiaJHuF4X/BMKQx7pM29aSdBVFKqqC0tEU4KPZE114VPQYzvPgFx0/aqCocdZJsirrdzxGgzhQwPTjgDu66ylc4lurKExypJkODJrhVTr4VXeWSZ6qwnlWymqTITN2pyBlJnJAnMPgIkeGhvsJ3D21xrU6hGomcRUqsmCYHJ5byEpgtqlhApr0MtL5z6I2A979Xet50oqwcXdCsrvAISnSRG9kkKB8ngeCjfGQToPtN+Bs0qkHblSjfUo a5xV3NYk MpKQMAxaEWD3dWaH4oD2MTCOWzLHgI1eHW33JLH1a0/oB46j8Z0INKkUR5+oC5YHsGt6L4GYnNuwGPJ27Pe+HnujvKuVRN9uJy3Ok3ApgBLzmTgJ8eAe/fvYSp7+c44H1lAKY8q97PygE9Vnena/plbNLTCDEYMNZ3XYf/q+eKrcFlisAiMqdMck06cHZ/c7oAP3FiXjqZC5Ay6KdjDp0tLDU3Ns88x1qJQxBeEsTITprnApT4RJPFYulEFJwybudcg6iRY6AQqhgLQnSVawtGbL3nRA4pnbnwW5HnP7aiYG61GCsy9bgFH/wOqyC70S/kB3InEGo5BPlsrorQd3oDdmufdUIRmZkNsce5o4UiMccSL8RsuL5MY0979BN6pTPlVHagG6XPgcjSgGoe8qKEE+GglwmXuQY6qXtEpr9c2AFzdD1bbXrn11KNxtglN+OE4lNclIo9v05OXIiumfD9s2V0FIPFKxsNCsU9JOKWsAhBmNw4Ur4p8t9a7jHml8eSpZe X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song Currently, we support mTHP swapout but not swapin. This means that once mTHP is swapped out, it will come back as small folios when swapped in. This is particularly detrimental for devices like Android, where more than half of the memory is in swap. The lack of mTHP swapin functionality makes mTHP a showstopper in scenarios that heavily rely on swap. This patchset introduces mTHP swap-in support. It starts with synchronous devices similar to zRAM, aiming to benefit as many users as possible with minimal changes. -v6: * remove the swapin control added in v5, per Willy, Christoph; The original reason for adding the swpin_enabled control was primarily to address concerns for slower devices. Currently, since we only support fast sync devices, swap-in size is less of a concern. We’ll gain a clearer understanding of the next steps while more devices begin to support mTHP swap-in. * add nr argument in mem_cgroup_swapin_uncharge_swap() instead of adding new API, Willy; * swapcache_prepare() and swapcache_clear() large folios support is also removed as it has been separated per Baolin's request, right now has been in mm-unstable. * provide more data in changelog. -v5: https://lore.kernel.org/linux-mm/20240726094618.401593-1-21cnbao@gmail.com/ * Add swap-in control policy according to Ying's proposal. Right now only "always" and "never" are supported, later we can extend to "auto"; * Fix the comment regarding zswap_never_enabled() according to Yosry; * Filter out unaligned swp entries earlier; * add mem_cgroup_swapin_uncharge_swap_nr() helper -v4: https://lore.kernel.org/linux-mm/20240629111010.230484-1-21cnbao@gmail.com/ Many parts of v3 have been merged into the mm tree with the help on reviewing from Ryan, David, Ying and Chris etc. Thank you very much! This is the final part to allocate large folios and map them. * Use Yosry's zswap_never_enabled(), notice there is a bug. I put the bug fix in this v4 RFC though it should be fixed in Yosry's patch * lots of code improvement (drop large stack, hold ptl etc) according to Yosry's and Ryan's feedback * rebased on top of the latest mm-unstable and utilized some new helpers introduced recently. -v3: https://lore.kernel.org/linux-mm/20240304081348.197341-1-21cnbao@gmail.com/ * avoid over-writing err in __swap_duplicate_nr, pointed out by Yosry, thanks! * fix the issue folio is charged twice for do_swap_page, separating alloc_anon_folio and alloc_swap_folio as they have many differences now on * memcg charing * clearing allocated folio or not -v2: https://lore.kernel.org/linux-mm/20240229003753.134193-1-21cnbao@gmail.com/ * lots of code cleanup according to Chris's comments, thanks! * collect Chris's ack tags, thanks! * address David's comment on moving to use folio_add_new_anon_rmap for !folio_test_anon in do_swap_page, thanks! * remove the MADV_PAGEOUT patch from this series as Ryan will intergrate it into swap-out series * Apply Kairui's work of "mm/swap: fix race when skipping swapcache" on large folios swap-in as well * fixed corrupted data(zero-filled data) in two races: zswap and a part of entries are in swapcache while some others are not in by checking SWAP_HAS_CACHE while swapping in a large folio -v1: https://lore.kernel.org/all/20240118111036.72641-1-21cnbao@gmail.com/#t Barry Song (1): mm: add nr argument in mem_cgroup_swapin_uncharge_swap() helper to support large folios Chuanhua Han (1): mm: support large folios swap-in for zRAM-like devices include/linux/memcontrol.h | 5 +- mm/memcontrol.c | 7 +- mm/memory.c | 211 +++++++++++++++++++++++++++++++++---- mm/swap_state.c | 2 +- 4 files changed, 196 insertions(+), 29 deletions(-)