From patchwork Mon Aug 5 16:07:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: zhiguojiang X-Patchwork-Id: 13753803 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0038CC3DA7F for ; Mon, 5 Aug 2024 16:08:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8B4236B00A8; Mon, 5 Aug 2024 12:08:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 862A86B00B3; Mon, 5 Aug 2024 12:08:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68E3D6B00B4; Mon, 5 Aug 2024 12:08:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 46F126B00A8 for ; Mon, 5 Aug 2024 12:08:14 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E537D1202FB for ; Mon, 5 Aug 2024 16:08:13 +0000 (UTC) X-FDA: 82418673666.20.1C6F7A2 Received: from HK3PR03CU002.outbound.protection.outlook.com (mail-eastasiaazon11011021.outbound.protection.outlook.com [52.101.129.21]) by imf17.hostedemail.com (Postfix) with ESMTP id CEAFC40025 for ; Mon, 5 Aug 2024 16:08:10 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=vivo.com header.s=selector2 header.b=iKoZwQun; spf=pass (imf17.hostedemail.com: domain of justinjiang@vivo.com designates 52.101.129.21 as permitted sender) smtp.mailfrom=justinjiang@vivo.com; dmarc=pass (policy=quarantine) header.from=vivo.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722874042; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=iKrcm7D1+w1/f9EZUAes0m1eh314T9WAUIVPzQ6rzNE=; b=tk7oCo2BC7bFBH/qDoT3Nkd8b9kcp8Bt7sorDyxBaFOqsFGAtJObgc/rvFa25TAc0H/kfK GKw6llz1X/jh4EhLDT2Ne0jkxYns8XwmjMWJ8vyOSsxTXhGojX1xfxl/EVnlCK4QYzl/3X uEr9vsPV03j8c0pKSSyb3GvrdybyMpc= ARC-Authentication-Results: i=2; imf17.hostedemail.com; dkim=pass header.d=vivo.com header.s=selector2 header.b=iKoZwQun; spf=pass (imf17.hostedemail.com: domain of justinjiang@vivo.com designates 52.101.129.21 as permitted sender) smtp.mailfrom=justinjiang@vivo.com; dmarc=pass (policy=quarantine) header.from=vivo.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1722874042; a=rsa-sha256; cv=pass; b=VU2jal0SeB92tiGeJ6VFoqS1paVpd1Df1v5+wJAQ218EESF4TLY59Xw1Z0GFq840xzPQpj c9GTzKmgwUPoVSaUFSRfcTS/OLflQ4Cn6pZyQiUt43lECxt9L7Zp1g9m9IfzdeW4zaLXdL ttyAAEac+ludnVk947ODcOBq/B95hCo= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=w04zAj1I84Uj/r8HdA1+uVOo2/EFm+W1AK1C1qlFTM6UKTDTsKkb0JXeBWcFtcvoMZu0ozwphNCZlNafq8TeFObWSWOEu3kENun/5aUQf77q+jEe5Z4n7Mjxg+R95ITiQexPjYJky5u0LFQYf8R2YTcXZ/SF4ZVincG083/lMOhFSilKsQsaHxQB1jREmlJedyBRwt2FAV6KNDmfR9I/pzywoO2kImeqPScL5UJGl92ixEhEmHZ1+H5v/3nf9YVIFmCWg9yD0yt9TL2+9QolVC8i/SJCfEyfVR2m7ODvVmlTnyk1mxhH8WLcBwCKH7ZAPuPXqrCQ4M5epcv8tevPXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=iKrcm7D1+w1/f9EZUAes0m1eh314T9WAUIVPzQ6rzNE=; b=eMjeJLhSOVmt0dcWw8MSVlBhlH84UhusQXrNmv0V+6CRQSHp2i4ca5Ei3usDW5oHddojvVh7VpjEOAfFW10c9H/GMqRzdSLzvy+AxI6MpbFMNW9HEa/Oy6ZlrheUuUHLnnAcuaCiIEDZmnpjGaI7lZfpq0V0Jo5oe8hRoeb9vZEOL87jn7tTp5//TeeVNWHJUOgA2cgpFVao5t32eHPyUUkZdOl8TP5umVw3EdJsrYY3R7jTdqvvcih1Vl8Zaej1G0nnbASNDApOI6dsPHOOgHNhcWgGndOqCAx8ZiC/u0Qa1TyrJTcxqDwwzb206ZDZBV0ut6t+RX5EBwmKgB9iRw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iKrcm7D1+w1/f9EZUAes0m1eh314T9WAUIVPzQ6rzNE=; b=iKoZwQunSCnhOawB4eT3/b17sYlHr7yCxW6vgb65MMsgz8u9YTIWB8nHVQhfN3djd7o+yd2mAoqYrM74SNO0qQMSSGDfHhFbYnceW7zK8CHiLKnunmuHMAYO7DnmbZjO4SKJG2jCn8+AqhZ2z+plk00noybYV2hzXq7w/0eKu5zldFXe1xThyhqBvmoDI1LsZf93+87Hc1rcihATI1wFUwbL0DFrNmPhIZOj9RIPQWBpo2qzYVklKADUKjOJc8jY+JdlnLHQJozbTF4wrXrL9xp229eF5x1cnc8rZ8CKq+VXTq5SohrcPgmfvMHg8AndPOwDm1jc52tHkrGWsd2R2g== Received: from JH0PR06MB6849.apcprd06.prod.outlook.com (2603:1096:990:47::12) by KL1PR0601MB5614.apcprd06.prod.outlook.com (2603:1096:820:9b::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7828.26; Mon, 5 Aug 2024 16:08:02 +0000 Received: from JH0PR06MB6849.apcprd06.prod.outlook.com ([fe80::ed24:a6cd:d489:c5ed]) by JH0PR06MB6849.apcprd06.prod.outlook.com ([fe80::ed24:a6cd:d489:c5ed%3]) with mapi id 15.20.7828.023; Mon, 5 Aug 2024 16:08:02 +0000 From: Zhiguo Jiang To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Barry Song <21cnbao@gmail.com>, Chris Li , Zhiguo Jiang Cc: opensource.kernel@vivo.com Subject: [PATCH] mm: swap: mTHP frees entries as a whole Date: Tue, 6 Aug 2024 00:07:54 +0800 Message-ID: <20240805160754.1081-1-justinjiang@vivo.com> X-Mailer: git-send-email 2.41.0.windows.3 X-ClientProxiedBy: SG2PR02CA0059.apcprd02.prod.outlook.com (2603:1096:4:54::23) To JH0PR06MB6849.apcprd06.prod.outlook.com (2603:1096:990:47::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: JH0PR06MB6849:EE_|KL1PR0601MB5614:EE_ X-MS-Office365-Filtering-Correlation-Id: 38a9ac8c-f468-44f0-ce2d-08dcb568c86f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|52116014|38350700014; X-Microsoft-Antispam-Message-Info: unWLF5sZ7htz6RNjFL5dhoNaqrGdeUTsHB3u2nIo8DuMZ849iec6yErurkjGNjwbsv+QqYXKnugAxU47U9g/Go/oC5x7NmyAE0vcDrG5R6Bed2sOwVWbJJppPbsQGd5urRnAxi0NlWndxmMfh9mpzZsK3aByCwnpB1OlFwkvpyW/rHgrPm9vzhfK5xMKwUr/UtVUhyJlr4ZxQAFgWdXjDBAJavosdYXjjK1eSwXGTxI5W4aWeEBZLQOLPcrA2mKlAbHg5OMcYToVXf6uSBTaIJR4IvIBoqPiwEhqMYYk9mSZNq/m0wuMXgJZyWKN0I816zho/i4PnmksXSIAhJPfaW8oUFmI+EdLElIvjmPYtpT99hb7p4QZC+i/K9PF0VVaPyHuEWwkf5nYAiPnYrK3WOPrBHqSq6HiXWnPfS7yzpAVNxu2iHs5HAGETkHT1G69oJMYoLQoK8e/EaTyCOqhfzN98/QbBGgHyr8QvgLIGXKtp9VpKoDxA6uwWJ+BtjPXVR0umHlMenRXuN9lyncrtJ5AW4X9AXinnq7H6Hls8D9Qw++WhNPoskw1Cnf9Wra++gPEKy7ZB+L6448w1IINJ2ADsC36IotD+8DlPmvkYAivbT0P8RnWP+wvC0QmHf4Gb2L+fyvTPc1llGYdxuyoh5M3+RUqQczZ7PxoXP4xUQsBEW8vCClPP/F6OlOOd1mf48uO20BvgVyCrQzQMlWf4Eys2o0eWm16RfHe3dJWWxtggzeF5w6kwKrJvH9iLDQ7W2VToIFZFBVK2VlUcFQ7i2XrxvKj0FhhzCDsCkj+H+CXwIAOpZQJNB4zwACRFDc1K0PgZcFzh3u7PKMFalX/DeUomg0MmR7P3BD/1pgGlHx0vmY5bugnC1/9VUO1DVvPbUbLC+UIRQH8TvvZTgx0wBYPqdlC05DELrOC3c/67x5aPmaxRYKaeHM2JpBBL0Sr36wFvBFMnb7VAiqELWpmEGOVlc8Yo3Lf0ZA0WrhrJSuvmwGHN9kGjYcH/ZA4ZhKgUqCLkcVAk30xyujJC5QP9OPVAUxvevdYQfAgoST0IUXnRfOrft7SA5ZsqQavoA0fuT7i2nrB/fQMkL+LfqcvtTtCrNaGKzfx8Gbah2M2egGhxzODeel1FjqVByx6vkMmJ6r8bViVR7XBLj7JFwpLJXvGnuugceJJ5PelbL4WHfp3UEabFwXLVjVRTEgezl9PyJSsvivQbB29P5tLBRUcjlGnTv7H49hUyPFLMolRufU3H2NQkotOSZmYkK2FhpbU/utTcsoAIee00KQLqjiVq45Wh/0nkyJaDL2y6ODFUdMjqptpdrfGMPj5kyUc6FhJoWz/i3Y0IA4S3WQ65noejEKUncYZhFMaekQPIltZEJMc8B7WWKsW+/gXBE9+hUc1cn+hSpENjLMkjQZGReNiAw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:JH0PR06MB6849.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(52116014)(38350700014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Apbgo0TE4k9ZcgKsCJz+x2crfOwA5J7XuIMSjH2DKX9ZeenM1k/L7bRbXskl3N4GIp4MoMiWqrTSVD1Aku/lgn1Qa0JatFTGDZbmYtp/9lNJTGfMOXyaW0Lc7z3SslWRIkyTb3ajONcLB6tGDNfjQ9Aj29cVwf47AUAqWqGfJi6RRxCUJFI1yQt3a/jzCxZz05Iw7PrInYFltp9eouihl6LJEOB5CiX16vRvEG01S8gNkQAtGxsPCE3xs3ODy7xEC+/NfzLKozzVUKm+G9u18M6kbMORv7icqXtAqpMTdb1en+9E4H68j4q0rvv+QVhOG/O7PFVWvCCnn59Bi+heNcKU7Hg1/ul0QFVDt4oVDSn8dbXHbD60uelMaJK40/vu1VvrlO8S3blnkbppcnMWyIhUEQywYybl9rCNHuNnr1H7pVc3fdiXpZmeoKzopIMEIrxHDFfvb+j7IbYNgTR3daRvQj1/oHJFjRh8M83PwlsnwWXAXIoAKT6Z+LdREOIe2buptV1d4bkWCGUJpq6zBwMhKjVZbB3Z3LeJ7Q7L6xYqx2gKMT0njaKKECYs2qOGICye4xJNbo7NV6Zus0F3bkzk4bcMwSv0Lm2Uhq3ClxtoiTvnea/7QnwigSSnolECJAxwLuhZ4VX/goHBGdcXSI3t4y1vEE2nQQSbTTeQgf+6KjFN9cwfoKkT78ATMVjU6cCYXSuk0HcEYMQ9fXSahqGbCoiPsapIMevi4FKu6Nh/LK+/KR3cfRR6XJ++M5kg6m6z5zEMwSPp0wpY8aTL9gfB0NDN0+IH9CaOlsU0fdmIcCYB/XcySxMsvlgQ/TYzNoHch05uuuj/Y6Iz3/ZEpYLX3ya8O6quM62osEAenP5ghmaLjQ6qGJp/qI8CG0qJs2A4v1kw2qFT8Q4AFhimoTKuhQBQAf2yTomx2BpCePTS69h1gmLL7+ZXgw3nsPGkn0EzQ83qdSGu2sGXhoddg5pKkrqIIr2eWH24HOpCmembjoLFsMgLNT6KT9S1TKs1BAImCD43bfM/0lv45l3wTUgIyJYlyYmKN683I8w75kOOEnzh4nndVz9xwswDcsOPLkwsqzgIWRabRN3IL4P+UfZVSiQgQkBgAHSwRQV2MbTVpgaItc14pyjPLFXqyi+9wFnkb4M3fMnV5KqKvuWUHTeVu4llCZP3f0CjahVicqDimI1ufUPC6urWZMm4E73zGb+FZLcW8Iu0kN2gLUcnROm5PIKMia7shWbTp9TcBznRB/hQpGKJGG6djEvrVejJYk+5ZzuhxfV/djDT96/QZKuLH1focj13m60eqLC/wbk3NpHSUOQPvmpxhfta8U1Zm3VuJQBnp1Dh/PGyQXp1P+z+zrGL2OeOOKsUGyoVnA73XkT4xVQ1/si6nrNMsmvc+FW9jqV3n04T92G2pa2XCazdek32wUWa1xyvxJ75P1GQHl2N/OR2jnk3/lrExM0SJGrSLmCaMVjjpvU4bL/R+beS7yjdFomnluOyO0vftqUPtecd0YzNSaQKazZZ18nWuL2cnckEwFD3oKP0CLq2iLWQhH30ORFrdyJgrOetPt/ZYva/6RIYsoex0I1435sj X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 38a9ac8c-f468-44f0-ce2d-08dcb568c86f X-MS-Exchange-CrossTenant-AuthSource: JH0PR06MB6849.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Aug 2024 16:08:02.4743 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: L53GOATE5PQehqAvjgFB5UFjbtsJu94D6c2LQmCCOloLrmXjInPRSnFJ3IWfFk1VRpv20UPg0uJGXDygyv/Qkg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: KL1PR0601MB5614 X-Stat-Signature: 6kinjysja5umse476x4okiumxzafu1o8 X-Rspam-User: X-Rspamd-Queue-Id: CEAFC40025 X-Rspamd-Server: rspam02 X-HE-Tag: 1722874090-172817 X-HE-Meta: U2FsdGVkX1/06kNZScK/nzEcCb43asDhg4Uyc0/+TSwGc1XYhnORJ1I7tjXqvK91IUvOwcVcGouC0K2YuxiiMlP5psSicm3LziedGNUjy/Pv+/oPmYyLTc5dIsjt5HwWgTDOzLLGhAhI85XE7/lNNSNyE+Nr3eApyEdyADDSTU/8XtKDSY136HSfk64t+8vABhS2DcmbBKgQ1A0WCMXLC+sxyEnFkQTT7inbwKcUdv2IxuXRHYJNGZiZ5BVXy6HK5f+L73cAiliyVmmsFlyu+Bexj8oKScPpwKfYDUe7ADt7xEMo8tuATJcj9UwZVHU7EvNiVaJwWGAg+Qk4PpiCOpZ6x/oixaHiYPKZZg4gxi+/vSJujeH28XUSFqfdzeHuvwfBKi+Jqtw1fvarfVtBlM27zP9vEC/MZ/1vNzWH6K2xhrI4YURrJq4KKbGUYiYitW72AZfir7qpU0ijMOeiKQOSEfsYoSH/Ize6fNGAur7pRvdcCNcCiq4w7QfqYYuJ9/Ot09CHj3XoxLsMQekW1jbWU3jK2Gb+tlvp6SKnrClXYq78ia2IwyNIn1UyTYnqjGfELQunU9C7dke1u6krtIzt3ok8yZtl/S0j9QrO9VtzDMnhL2wTTL/khUp86gAIMBmQ1YW+UTzrENKjrE49JSdYv7GLXrtalMYkbT/SBeWuTmpJoo8D5ydihe5TWi2E4Ekqsf3It+C3BIXFY+7u5Yg5MdU0k0mNY7/HLsATlXPJG8C1/NZywfVlGL2PUVXGUWaykXhaKa+8kGirlxxzAl0Legrp7zamdpXrErtsnyVv6zVjw20zf/IfFe4x4rzDQUOxExjFs4cFnRZ1yMBHOgAUPlaNaAHWCA6p+Em10qAAQjwodGsiClnvnxTGV9oEo7VcmuQ/miCuk5YO12BmHvB+xg4/Sy2VfN+9uFZmbhwl6E+n5mkORyCz8yMgnvEkTlH50TvVgnpWtrKPJjs nXHaXbNQ yhgudiMT0soZ9JQ4MVXS73yxtOiHQvLS3JrpbADGoQwPDCFsY58/5uTUpI3KrcDHDghiRhaSUrY5uaABgG5xRPjNM7FgFLHPaQwxKGWoRjyROQUOrmW7Fg8Q5JXCZXPCunuwYIWn5LWYWM/1cAHxTFTSYymshv9uNS7ikaOXlFejedP8UkPCRz8LOBZNFFb32lVxDa42aPeQqnTthzxHNmps+0oHZFogqXZXTvQI1YPbMb8fRuZ9+kegAmjc0VPEV1vW5vLp/CiPVtBaiKFA5vQOgi4TgVORa8mJz1WqWzP7ODTU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Support mTHP's attempt to free swap entries as a whole, which can avoid frequent swap_info locking for every individual entry in swapcache_free_entries(). When the swap_map count values corresponding to all contiguous entries are all zero excluding SWAP_HAS_CACHE, the entries will be freed directly by skippping percpu swp_slots caches. Signed-off-by: Zhiguo Jiang --- mm/swapfile.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/mm/swapfile.c b/mm/swapfile.c index ea023fc25d08..829fb4cfb6ec --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1493,6 +1493,58 @@ static void swap_entry_range_free(struct swap_info_struct *p, swp_entry_t entry, swap_range_free(p, offset, nr_pages); } +/* + * Free the contiguous swap entries as a whole, caller have to + * ensure all entries belong to the same folio. + */ +static void swap_entry_range_check_and_free(struct swap_info_struct *p, + swp_entry_t entry, int nr, bool *any_only_cache) +{ + const unsigned long start_offset = swp_offset(entry); + const unsigned long end_offset = start_offset + nr; + unsigned long offset; + DECLARE_BITMAP(to_free, SWAPFILE_CLUSTER) = { 0 }; + struct swap_cluster_info *ci; + int i = 0, nr_setbits = 0; + unsigned char count; + + /* + * Free and check swap_map count values corresponding to all contiguous + * entries in the whole folio range. + */ + WARN_ON_ONCE(nr > SWAPFILE_CLUSTER); + ci = lock_cluster_or_swap_info(p, start_offset); + for (offset = start_offset; offset < end_offset; offset++, i++) { + if (data_race(p->swap_map[offset])) { + count = __swap_entry_free_locked(p, offset, 1); + if (!count) { + bitmap_set(to_free, i, 1); + nr_setbits++; + } else if (count == SWAP_HAS_CACHE) { + *any_only_cache = true; + } + } else { + WARN_ON_ONCE(1); + } + } + unlock_cluster_or_swap_info(p, ci); + + /* + * If the swap_map count values corresponding to all contiguous entries are + * all zero excluding SWAP_HAS_CACHE, the entries will be freed directly by + * skippping percpu swp_slots caches, which can avoid frequent swap_info + * locking for every individual entry. + */ + if (nr > 1 && nr_setbits == nr) { + spin_lock(&p->lock); + swap_entry_range_free(p, entry, nr); + spin_unlock(&p->lock); + } else { + for_each_set_bit(i, to_free, SWAPFILE_CLUSTER) + free_swap_slot(swp_entry(p->type, start_offset + i)); + } +} + static void cluster_swap_free_nr(struct swap_info_struct *sis, unsigned long offset, int nr_pages, unsigned char usage) @@ -1808,6 +1860,14 @@ void free_swap_and_cache_nr(swp_entry_t entry, int nr) if (WARN_ON(end_offset > si->max)) goto out; + /* + * Try to free all contiguous entries about mTHP as a whole. + */ + if (IS_ENABLED(CONFIG_THP_SWAP) && nr > 1) { + swap_entry_range_check_and_free(si, entry, nr, &any_only_cache); + goto free_cache; + } + /* * First free all entries in the range. */ @@ -1821,6 +1881,7 @@ void free_swap_and_cache_nr(swp_entry_t entry, int nr) } } +free_cache: /* * Short-circuit the below loop if none of the entries had their * reference drop to zero.