From patchwork Tue Jun 18 06:54:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13701835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1371EC27C4F for ; Tue, 18 Jun 2024 06:54:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8F5316B013F; Tue, 18 Jun 2024 02:54:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8A0926B0148; Tue, 18 Jun 2024 02:54:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 741E16B0144; Tue, 18 Jun 2024 02:54:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 515716B013A for ; Tue, 18 Jun 2024 02:54:35 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C3731A04B8 for ; Tue, 18 Jun 2024 06:54:34 +0000 (UTC) X-FDA: 82243096068.24.8E2E643 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by imf23.hostedemail.com (Postfix) with ESMTP id CE914140016 for ; Tue, 18 Jun 2024 06:54:31 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=lKyP5K6G; spf=pass (imf23.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718693669; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=DxaCp5bOC4hyWB6FKdEFK/JUDfrUbLNO+z6WPPrt+Vw=; b=TbjzunHCYVq7lUrAd3Oi+zo5ESdhBWqenOFwuAXa5NGU/htiHwLBSxCVmXWWajfC1c9mwr lvd3fhTfMZsLouoe46sMXIFPM7M6nGdkAzD0hjgyHgBu66p9ZIdU/4CrdIY9AOPYvI7z2X frLwOlqWC6QLeYnLFrTbl39FjvyTd4Y= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=lKyP5K6G; spf=pass (imf23.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718693669; a=rsa-sha256; cv=none; b=LZz6pcwBEEaQcIxzRXGFhc/OErV+IvkVSsO+sPnLX/vp2XMk4kyBjFEEfW8F234qixRWom Z1c4c0qkTLDtce1dEWyZK+fZdccOy5P/b1F7Aj/LoBJmCeNqQUsu+J0gvhdXqGdrxjuuB+ Hu2yExGF/y4ul5TFxBBsR10uTMwGnas= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1718693669; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=DxaCp5bOC4hyWB6FKdEFK/JUDfrUbLNO+z6WPPrt+Vw=; b=lKyP5K6GrpOdFAsIyYXpaiCuDU374+QUNURhr/XPZCF+8FPwIySdQw/WvEGbMsPKQ+ixoQcitc8FRFXCTWtotN5k1kYCqwOqX/GR6q3UaakDo0C+hZDjz8WNI9c3voRHTi9hRDpqZp2q/eD59pqSwZMWJ5ppnTtzXElN5xs0U0s= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R101e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033022160150;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W8jFFIj_1718693666; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W8jFFIj_1718693666) by smtp.aliyun-inc.com; Tue, 18 Jun 2024 14:54:27 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 0/9] support large folio swap-out and swap-in for shmem Date: Tue, 18 Jun 2024 14:54:12 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: CE914140016 X-Stat-Signature: d3fd68azqinna63iufhikzu8hknura9r X-Rspam-User: X-HE-Tag: 1718693671-572190 X-HE-Meta: U2FsdGVkX182dzB8XahEEkfPAkC7ak8JF5C0oP8FybCZbtiugh529H6GrxZkBaf5dMi1H58VN18QVxkFRIBnZT66WNGBthoW0qvRrQ17XVauvQRhR0HeIJ41/hAWADxGjuPR5zZ5zaNlVLbgVY4Exzi47re7ej+otfBsvsUUVEP7cGE0D0M/hinR1V7NLpU4SOBR7KhvnP83dxSwrQ4+PEdxg7GC5UmIcgwGpmxexkONJFdONb6H5c00rdZHo+bnFlYllOyUkBBAEzHgPl5BTol9Pia+kqEnlYpbFiXfpjRbpSnra5kxEJU+HBB/Ufthj7ObHDAPDTF7l9l9b2le8eeDA9eNbg6+lQqBXV0JslBzd3Vu2cyKwtwR820fHyGaFDDGV1HulN94Wc5RJZS9bFkBNLZYv/f4hl5CkmuyZT4LBRkgefPG2eUdpQp3nnjCf5D/bkicqCo7PPU9/+8fICcouy2UZHwJJbo4KSTWDqWCQOoecXyNkg7fNTQoSbibFv8vXRtDAIgQA+erRMNu46ZrI4M88GgfX/WFhzkxL+ENcw8qXJDD2NLOHe2+LN3r+bl3pVi3GNuSZlLdoGL1K7aM+Owo61XXTjCwCDJMia6XS0jg5CIUzoCOPLG2ZanKyW7TOuOmvcANSx7G0U+SshQgYXkCDA70s/uVh2XicGuNZKhWGZKykpe10yF/V2XZjXL0SoNRSlIrgS/CC4ewHixtKN+ZYNXeHvTs0p0UUcDPE770nym+pJ8P3wLPH/eFnyGM6Xxtgfk6mSGEOXQrzx4/3Up7Chch397GQADhImIl5SaU25/K8PloNtlm9Tj8e5RxQa7qyiUxwd6YRcnfvFkHkUqWQZblhp8d1gCansFBy5Rqqfg2j0Fn4Z9eaZixTPmDZzysm4TJ4Ys0yiQad1zEVMPeLDIRUsMVzfv4nRTUkoCWWfwccidkri1Ow7LuVGQILFwx8c/yqZ7ro1I vP8er9yF 6jqWzDhJKFtqO4KjFkEG3BzBWSrR36/vk6u0f19DYr+so/x3GW5BRZ2J+RUiK4sZli1wxvvycm+cV7MClRvGCfttB+6CNsXHPDHMGznR3u/WIXZinBCX6Zj3zUv1oj6D1gVBx/CIal5Uv3fet2Y7A1gWa88WJPlRexmMAHd7gmXTX+/aOBlSaznq7+YIJ2JtDLB0G2rmXzICPemqmo+/ZCKz8EFnc1GaC82bZpKFjnexpn6F1GTrsnyEjQG3w9WvrNxNrjjdtrLTAxRZnDwwsYsxykAoBoWODq6QakOXSwujRLi5+9WxhUjpT9OeoPGSUykbWn4PreB7Oc7Ca4+XdG/e2OZD5W3pVtAOWANBmtFzJ2qNsZWXPsVPDNA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Shmem will support large folio allocation [1] [2] to get a better performance, however, the memory reclaim still splits the precious large folios when trying to swap-out shmem, which may lead to the memory fragmentation issue and can not take advantage of the large folio for shmeme. Moreover, the swap code already supports for swapping out large folio without split, and large folio swap-in[3] series is queued into mm-unstable branch. Hence this patch set also supports the large folio swap-out and swap-in for shmem. Functional testing ================== I use the latest mm-unstable branch to test with reverting Chris's "mm: swap: mTHP swap allocator base on swap cluster order" series which can cause some problems (Hugh also reported in [4]). Machine environment: 32 Arm cores, 120G memory and 50G swap device. 1. Run xfstests suite to test tmpfs filesystem, and I did not catch any regressions with this patch set. FSTYP=tmpfs export TEST_DIR=/mnt/tempfs_mnt export TEST_DEV=/mnt/tempfs_mnt export SCRATCH_MNT=/mnt/scratchdir export SCRATCH_DEV=/mnt/scratchdir 2. Run all mm selftests in tools/testing/selftests/mm/, and no regressions found. 3. I also wrote several shmem swap test cases, including shmem splitting, shmem swapout, shmem swapin, swapoff during shmem swapout, shmem reclaim, shmem swapin replacement, etc. I tested these cases under 4K and 64K shmem folio sizes with a swap device, and shmem swap functionality works well on my machine (I can share these test cases if needed). Hugh, I think this version has fixed the reference issue you pointed out before, and I have also fixed some issues related to shmem swapin replacement. So could you help to take a look at this series when you find some time? Thanks a lot. [1] https://lore.kernel.org/all/cover.1717495894.git.baolin.wang@linux.alibaba.com/ [2] https://lore.kernel.org/all/20240515055719.32577-1-da.gomez@samsung.com/ [3] https://lore.kernel.org/all/20240508224040.190469-6-21cnbao@gmail.com/T/ [4] https://lore.kernel.org/all/8db63194-77fd-e0b8-8601-2bbf04889a5b@google.com/ Changes from v1: - Remove useless 'order' variable in shmem_partial_swap_usage(), per Daniel. - Add a new patch to return number of pages beeing freed in shmem_free_swap(), per Daniel. - Drop 'orders' parameter for find_get_entries() and find_lock_entries(). - Round down the index when adding the swapin folio into the pagecache, suggested by Hugh. - Fix the reference issue when removing folio from pagecache in patch 8. - Fix replacing old folio in swap cache in patch 7. Changes from RFC: - Rebased to the latest mm-unstable. - Drop the counter name fixing patch, which was queued into mm-hotfixes-stable branch. Baolin Wang (8): mm: vmscan: add validation before spliting shmem large folio mm: swap: extend swap_shmem_alloc() to support batch SWAP_MAP_SHMEM flag setting mm: shmem: extend shmem_partial_swap_usage() to support large folio swap mm: filemap: use xa_get_order() to get the swap entry order mm: shmem: use swap_free_nr() to free shmem swap entries mm: shmem: support large folio allocation for shmem_replace_folio() mm: shmem: drop folio reference count using 'nr_pages' in shmem_delete_from_page_cache() mm: shmem: support large folio swap out Daniel Gomez (1): mm: shmem: return number of pages beeing freed in shmem_free_swap drivers/gpu/drm/i915/gem/i915_gem_shmem.c | 1 + include/linux/swap.h | 4 +- include/linux/writeback.h | 1 + mm/filemap.c | 4 + mm/shmem.c | 107 +++++++++++++--------- mm/swapfile.c | 98 ++++++++++---------- mm/vmscan.c | 22 ++++- 7 files changed, 142 insertions(+), 95 deletions(-)