From patchwork Fri Feb 14 17:57:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13975435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75463C02198 for ; Fri, 14 Feb 2025 17:58:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A467A6B007B; Fri, 14 Feb 2025 12:58:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9F6516B0085; Fri, 14 Feb 2025 12:58:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8BDF26B0088; Fri, 14 Feb 2025 12:58:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 68FE86B007B for ; Fri, 14 Feb 2025 12:58:26 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 186F81C6F5A for ; Fri, 14 Feb 2025 17:58:26 +0000 (UTC) X-FDA: 83119309812.09.22C9FD0 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf11.hostedemail.com (Postfix) with ESMTP id 50A5840015 for ; Fri, 14 Feb 2025 17:58:24 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=E7KpcZ3v; spf=pass (imf11.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739555904; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=fH8wa9mDFKKjF6AAEuB3tgHwOgdezUr8uBme/mmnzog=; b=ueu0gKkTBPthfSalKhNkKRjYFHSzujFusSH8lcdG203th+4Pjlpr+oXDXpz1zQNnvEtxgs 0ByxGzIm4T9jSmHUp04pIIcrBoU/hbzt5KoNTHNramxb9gpmiFT+UEXPkLF1GhCMBbipvS qb8TtLzxXcZyPjDSBBuNAA2pjyHPS7A= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=E7KpcZ3v; spf=pass (imf11.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739555904; a=rsa-sha256; cv=none; b=G9/qAqLFG1RV9E0s9ELf//+BGI1GqFrEVdvc+45YMYBpH8mHbA1/uB9cZ8rq/siGBgwTjl haOSNtHLHec88MOvo6mW7OHKRq4jrPdBEmkvAsJyxrik8OjuB9bt6ksdOhW1v+ajt86k7S LpQc21VSFRg3DVJaTDffUifs3k0T6Ag= Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-220c8f38febso44344285ad.2 for ; Fri, 14 Feb 2025 09:58:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739555902; x=1740160702; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:message-id:date :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=fH8wa9mDFKKjF6AAEuB3tgHwOgdezUr8uBme/mmnzog=; b=E7KpcZ3voSI4I8vVUpVhpukYXNlXdTsEhd1yazRYoRMsM7FN3jty4q1+wuLp0DUkSz L7OqcUU3Lx5foJqeKl+eL4FRGkbV6zAX+xRg6mKy/ilUYG4BmWIhHnTkuSOkPYwpYBXv HPDGVeXhQ+fZnRPUvGfu1DXpt3ECVorxDzNTHjUWOtW24oI/PqVjxfLyUfWqrTDCkDCY Em+9MhUtmeL9bCOPThV8U9LHIMp0qktwzZ/qmDaWPrdpORNhhkJx0BNKjsjXRbH6M1m6 maCN35u27nGDBr+Zyni26M20QPss4pMd38+2tTdcSWP3kYeatxMSnXrHUKMcb33fF0bw gnjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739555902; x=1740160702; h=content-transfer-encoding:mime-version:reply-to:message-id:date :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=fH8wa9mDFKKjF6AAEuB3tgHwOgdezUr8uBme/mmnzog=; b=lAhjD6MlAcYzsuOragGD7jKuT1ha3JNial6SPOfEPEj1QuD5/lAgd4OYiKmubk2IP0 APC1BrVg2AiX0ugxh8mrCwJ8zW/78ao07M96knsXQZ5QaQn3qh8dd3kjT6v0kqaz+8WH moers9XnQbybY8djAN9h0Z9Xn3Hon48XNofLQio0mJqVNJE+CMQ+S24IEqxMcSn4JIwd S4IxB301xKMDq0IHpQDZ//1KDI1RP6CEOR5GHbRnB7u7bBxaacP0PizqbKgZZWjlMjqp Mv20okIua0mWnTVXhsT+By9A70b0UYoVqPvr5+pOlfvjJKnF4k961G2Zgg2xUgRd4XsJ M93g== X-Gm-Message-State: AOJu0YyEgvVgdPo/3P/YJqTHYcdcS3iS9ET7JWfWCXx5Wm2nusycKapZ xT4eORABToGmi5/ygNGpQvs0eA38cD9NIiyyxYjyYeSA9GnuVmqBj79p8SieExY= X-Gm-Gg: ASbGnctx/OYOUp+2+0pDHkCzlI9a7y14dkG2Bixb/nSaQAf+t0WQatRUSkWz6HsRsZ7 KEIObJqMvZIWi1OOsAHWo1QGGSMTIDKdTcb0q/kzG+NEiGZ0B4o+Lmd6RLIjM/+Cn9BlzauekFB 99g2FkQ10Mn+N3X1lQmhA1JXUZEcd8fUo2hJ9kT1U094RQIHIxYK3+YUVLF135+7syqy1/vVqjQ jFt8iwmZ91TqKIOIv3SnLgiKrvAxWibiFe1DlLUhl1zoS1nzYo4cPTZKZODZceCKkRUDX0j28PE t5zpKXrFmZhLMMoy1vaX+jiE0ADY1dsk+dbz X-Google-Smtp-Source: AGHT+IGc//Yf3KcYa3gkm02REt+Cv1CxSx5XjNqoq3oIpbzENLsmetvp4vbwpFAd27eNTXoz2k2JEQ== X-Received: by 2002:a17:902:cccc:b0:21f:5e6f:a432 with SMTP id d9443c01a7336-221040409damr3063105ad.24.1739555902166; Fri, 14 Feb 2025 09:58:22 -0800 (PST) Received: from KASONG-MC4.tencent.com ([106.37.120.226]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d55943b5sm31216605ad.246.2025.02.14.09.58.17 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 14 Feb 2025 09:58:21 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Barry Song , Hugh Dickins , Yosry Ahmed , "Huang, Ying" , Baoquan He , Nhat Pham , Johannes Weiner , Kalesh Singh , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 0/7] mm, swap: remove swap slot cache Date: Sat, 15 Feb 2025 01:57:02 +0800 Message-ID: <20250214175709.76029-1-ryncsn@gmail.com> X-Mailer: git-send-email 2.48.1 Reply-To: Kairui Song MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 50A5840015 X-Stat-Signature: ptezu4reuq97wtj7mbbazirb77e1isqk X-HE-Tag: 1739555904-910437 X-HE-Meta: U2FsdGVkX1/sdgFHevF+MgpZqYyVxtc5mWI8XFDgBikeMqaHameC26+GWKShYvy7drzsU3Lj+JV1mhut4znqvKaO6nVdJHS3AGGG5k3MlJT8qSdQcdqUBGUFY9w7sCvOL17F17aHLzC0mA7VABm4F3Q8h0YAgoTHYo8u0ob0lq7miSAGCsu/Ry8R/i7ZYKG3nm5zK+d14yyuXL1/VQT1rGV/z/BnhORQPu3Rm3Ek3NcW96gZNU0jhznjQf4yeMtVBYOUzJtIQ4pRAMosBTm8NTLC7jvTlkpx4Yq43d4OdS7gTzilZ8GPHq6kVvUwlpFZHAM+TjRx3vX2SNCUOhkYvF88T3zE1MyRX1x8T8dsPHSjbFdJ08cSnV5hhzOdzoi6GyjcHJBCk/r+NkMd4tAGfeDMRiRUDnYUChXVU53ieCohoWCJRimoPfEcBzt6/AEUQfdJXHUWu4snYHf+z3gBC1QcQz3YU+HLSws0thHXBPhNeDEvONRsAC74qu0LeuKnSpzL/tr69yTxH2BxuIxebHsXCk7qmOvvyQmWOaX4G1RkSOws8HaQCm617Fuzpt/ZJEl4kkMRbwNRT8dFqx0GBW5Aih/ofLeIvlwZLh3VpCMurMFYElJk/VctbZ7xaUzvDyHAvjg5arkWVmJavK3GaTEZPnqoMAlzLJpLIvOVZuNuD8U52ODrRQMXpGJiBvZ9Rt79+UIqW2t1kHcdP9K4Egh9SmC5K1QAsSdo36iYbpbbRYBkU3ktwm7+10W4k4Uovwr3x88c92uzywEcu8qgEflqcN3ZuOvmyWV6bWSgRhRQu0cPsxjssx4A/fKcJwPKxU/bCRpXOrlb3eKg/NVk3+EftLzKTVpOBn8NnTV17M/00FJkl7FE5MDchFB9lCeu7kwxukceGuF1tt+5ssX3m0CZPFSBJcLa6TaLahZJdJ/HI+hPMlp71FN3XvBaT/wEkaidm+1OCgfApXX3pcg Ye+CEc9h 2aae+IOn4ODAAb9AnwLp67juXAiyOMKt0VyIILPRLTGc5QXMm5zxsxRVwruDH/7xmrYIlhZcJyn9+q+jd6GN28h7zsX43sE1W7dVXCEtfc87B+4qHEzUhasVD484u7BBMIas+ZkNdxbYrfFpi+6bEteTItVVcxn3aLDLHEbfFXItFzzw70F8H4sOBUZ4ffShIiYA7AVFYqHp61DAGGHbvjC7AQGEIKRQs6jf/Hs2iOyT0RHNYgYJ2RZlb8D+frUF+R5QBrk1I3+vh5upLA/6oFtnvzv5wuQrRbH2vRFF9DBdyQBqnCUgpK4nWqq8DWRErVZ0jB1FT3Jf8NdOkiTNY7mRJsPf7hjOVYI1RnqWzKsSwy6nQYjebKMFh414TjpG9l6RBrM4f4ylHaPSdaGzk+SgLtbxAU9qK0lDFYPWHjP6LZUeSUhtCtqbNK4mnqtkj0+q2LHSTH1f13BiAR8tO4ibOkP7SJ8HXBGvjS9FsqMpIuMkMMuHH3IOWpXVgH4VX+7N0lY3rC6G2P5D4T264c62aeDJTXxqgL2WLASmlKGkYsnMzcRZQF5IXrrXxBa8t+62xuhQuuQmTIF5/Pch6NMhuaiAzF27cCcPxpL9X9Y68EYA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000027, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Slot cache was initially introduced by commit 67afa38e012e ("mm/swap: add cache for swap slots allocation") to reduce the lock contention of si->lock. Previous series "mm, swap: rework of swap allocator locks" [1] removed swap slot cache for freeing path as freeing path no longer touches si->lock in most cased. Allocation path also have slight to none contention on si->lock since that series, but slot cache still helps to reduce other overheads, like counters and the plist. This series removes the slot cache from allocation path too, by using the cluster as allocation fast path and also reduce other overheads. Now slot cache is completely gone, the code is much simplified without obvious feature or performance change, also clean up related workaround. Also this should avoid other potential issues, e.g. the long pinning of swap slots: swap slot cache pins swap slots with HAS_CACHE, causing reclaim or allocation fail to use these slots on scanning. The only behavior change is the swap device allocation rotation mechanism, as explained in the patch "mm, swap: use percpu cluster as allocation fast path". Test results are looking good after deleting the swap slot cache: - vm-scalability with: `usemem --init-time -O -y -x -R -31 1G`, 12G memory cgroup using simulated pmem as SWAP (32G pmem, 32 CPUs), 16 test runs for each case, measuring the total throughput: Before (KB/s) (stdev) After (KB/s) (stdev) Random (4K): 424907.60 (24410.78) 414745.92 (34554.78) Random (64K): 163308.82 (11635.72) 167314.50 (18434.99) Sequential (4K, !-R): 6150056.79 (103205.90) 6321469.06 (115878.16) - Build linux kernel with make -j96, using 4K folio with 1.5G memory cgroup limit and 64K folio with 2G memory cgroup limit, on top of tmpfs, 12 test runs, measuring the system time: Before (s) (stdev) After (s) (stdev) make -j96 (4K): 6445.69 (61.95) 6408.80 (69.46) make -j96 (64K): 6841.71 (409.04) 6437.99 (435.55) The performance is unchanged, slightly better in some cases. [1] https://lore.kernel.org/linux-mm/20250113175732.48099-1-ryncsn@gmail.com/ Kairui Song (7): mm, swap: avoid reclaiming irrelevant swap cache mm, swap: drop the flag TTRS_DIRECT mm, swap: avoid redundant swap device pinning mm, swap: don't update the counter up-front mm, swap: use percpu cluster as allocation fast path mm, swap: remove swap slot cache mm, swap: simplify folio swap allocation include/linux/swap.h | 21 +-- include/linux/swap_slots.h | 28 ---- mm/Makefile | 2 +- mm/shmem.c | 21 +-- mm/swap.h | 6 - mm/swap_slots.c | 295 ---------------------------------- mm/swap_state.c | 79 ++-------- mm/swapfile.c | 315 ++++++++++++++++++------------------- mm/vmscan.c | 16 +- mm/zswap.c | 6 + 10 files changed, 196 insertions(+), 593 deletions(-) delete mode 100644 include/linux/swap_slots.h delete mode 100644 mm/swap_slots.c