From patchwork Thu Feb 13 03:43:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13972777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32B2BC021A0 for ; Thu, 13 Feb 2025 03:45:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B9065280007; Wed, 12 Feb 2025 22:45:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B1761280003; Wed, 12 Feb 2025 22:45:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 96A48280007; Wed, 12 Feb 2025 22:45:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 73C31280003 for ; Wed, 12 Feb 2025 22:45:39 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 2751AC028B for ; Thu, 13 Feb 2025 03:45:39 +0000 (UTC) X-FDA: 83113531998.19.3D8AE9D Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2072.outbound.protection.outlook.com [40.107.237.72]) by imf25.hostedemail.com (Postfix) with ESMTP id 634E5A0004 for ; Thu, 13 Feb 2025 03:45:36 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=X63yJOHz; spf=pass (imf25.hostedemail.com: domain of ziy@nvidia.com designates 40.107.237.72 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739418336; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=arKOQe462duZLGgRb8IwStyimqTscy81tSdsFMgbflI=; b=NPVN1CoT6Tf9yr7HLhJgUzQuK4MFdEALSVlVU5JbbQxF/+K1SNDUrPPxWYUCUMICHWpKge 6hOIKu8UGhoARqI9Bk+OeRqwwqsN3je2dsKIDMPQE8X7TzlXQH7XIubAvW7wYdPltmoHkX +ze9BMvFu9pV6xDUEhmIC+YJEW+La5k= ARC-Authentication-Results: i=2; imf25.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=X63yJOHz; spf=pass (imf25.hostedemail.com: domain of ziy@nvidia.com designates 40.107.237.72 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1739418336; a=rsa-sha256; cv=pass; b=ijKLPkQsEJedc5YAQ5wK3uQkyujrXe+Xl9uBe0uvGSWiCLf6S8ksHOH4DGQrd/LJmcIhq9 5V5cmo9hUNmhcGzmXcZCoSexjaqQIkfQXpGtlCFxs/n2jvIXr/VrI3j6qDvPZ4dbdngSEZ L55TTQkQaosszVDGZqXHCqOfH8OXQuQ= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=DnDzcwYZFgHZdgkCu8OUbAu7ZMCfOPKmgpESS/wkws81zragX3PHd4EMzCv4ngvnYtubOh0ZP6ZHNhJTijGebWDsGoYHj2+WwJhqru7cg5NLGqNVHQc5CNKYq6ocWNeaDHd11Lx5lQGnXbc67M5PlRJSJiOuhkQ2Z5IienhV22q+BRzWfGalsWptO/97+EuvM6om+t3lg0NVQXfSUAfWZZ4rjNYZWdTtVHPOGHSjSDF/pEuIGwnOQFlCo+swDGZguU3AflvDeNTlcTNylVdnI9SY3gp5N5PcHpy4VRUTAR/QO1JNg+T9+OzSZV4gPM0N4os+wZksG5/VRhtL2IjaQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=arKOQe462duZLGgRb8IwStyimqTscy81tSdsFMgbflI=; b=dWjNbMxbquwjLsKnqNh2y5kkjUTHYPW90E22tlO0z4hZJlE9dRHmhLlP1D9XXZVnoaNTa/jXPC+7ezULhtvNzzFpYfRLvv+hWpfZ2bTjjd9aSojKGdSvnyo/mwGAcTk6wKo9r0sz29PLIoGbOfbHU/HYy9m+zbPmBksEJHEEQWdC+vt/oIixW0OHRiQ9TuWaempKjJChemz11NNlEAIuTsGjd06a5vZ/7QYlwUacoRnpJ2zW2SDdEYZuF4T9T0Oc/jPzGgoCF3xXIhgTgTTJyCKfnwNN1CTob7HxPbTqUyqXyRBsgx5Og5a6Xt0oMgpCTYzAvPQyDxhMceAJPDyp6w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=arKOQe462duZLGgRb8IwStyimqTscy81tSdsFMgbflI=; b=X63yJOHzrEafK8k5+NfGb2tcgfeikEiL1augvQ5Y4jpEwhzRsZP50xrv8bOE5U12f1Ls/l8yvlaK1HeqBz5DF809gaJvJr70l76FefJv8xEw0klJVpG/z9PhPXgjGFmEm6lQsrN6O/fQrfbQ9YHh4hRKxKak2GsAhSvI4rD/qyaStqa+Okn/AX8esr2p8PIYIsjCyzF2yUeTjI4qyGRPNFtiLJ3HMgNHu0g1kSw/uP1gkqJNSXZYoYmsDmE9XTN2jWPdKbnkMA2CcNsOe2m8dxE4+DUgE0UCHoklVYsNhiu0Xr5eKjqtpu7R+unm9b3P2CWO8eFM4eM/h4D34nEg/Q== Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by DM3PR12MB9434.namprd12.prod.outlook.com (2603:10b6:0:4b::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8422.12; Thu, 13 Feb 2025 03:44:07 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8445.013; Thu, 13 Feb 2025 03:44:07 +0000 From: Zi Yan To: Matthew Wilcox , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: Andrew Morton , Hugh Dickins , Baolin Wang , Kairui Song , Miaohe Lin , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH 1/2] mm/filemap: use xas_try_split() in __filemap_add_folio(). Date: Wed, 12 Feb 2025 22:43:54 -0500 Message-ID: <20250213034355.516610-2-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250213034355.516610-1-ziy@nvidia.com> References: <20250213034355.516610-1-ziy@nvidia.com> X-ClientProxiedBy: BN0PR07CA0007.namprd07.prod.outlook.com (2603:10b6:408:141::24) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|DM3PR12MB9434:EE_ X-MS-Office365-Filtering-Correlation-Id: 4f726e5c-e83c-4600-6b4c-08dd4be0aaf7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014; X-Microsoft-Antispam-Message-Info: JaNN4P14DDN9Ts/m3M7ieaSND9q+KihDDWGPx2xLfsQfqfxe3FM2lV/LlkmsRa35U4W/hT2jwYpbBWg9hwfCGTfnaCPM7jvUVFpeOL4YKvWKWmQfT3ZrD1YC1OWrZ9GrcqCItTl526TkNCqN5LHkgrK1Nd+Ep8PqitNM8Eg7xneucCrg6l8zkJcHhwXJAFs0A+v4VVps+8nMNHZaOO26hNWKcXUTYo0rovA1BDjCiN8C33vEhZd4C8y2vrJJHSOnKtUPaPHtx5qkhf5wHje07wKEixJ7hTqbJtcCVwlQjKPPttWrOogj89uF7yGrvdv7f3Ykkl6jMHcMY7URFHhRNMnltQxGWeM7SkMRuYeiMoZUUe/Ctba0IgVGlFvx3yhaQboF+PCvtKheF3sivDcDnn5R3kFhfh6o2Gs8lnde/2BwwMwewngt/3I0Mideb8fn153JcKznKphq4jixIEHxfTbqu6y8lPn/AlZyqR+O/fV8ayxXAUUzpbXd8D+BvkTKa/EK1eyQQ9FRMSI9Jfx8H7S0p5WGluTwWOG2LaL63T5vBWs+7b0CCj6+JP+vgD3MKqxq7wYmd8lICc7UE52U2FFz8XBdgIGdzyB2TBxn8mWLAbIrjvMDW2QTg6qT3zp8QWc5LjqFB9UIn+aNSz+1MF7Z06ttwjtdxnI4GZBrbMUcPEENt6PCmaciK4eT+JuACqha/GKliOm60Y8G8USKxPbWn/r3euzejFPkXbONPJWgNDeMze6gx+ZX1iCeuNc7F5DzIgeVcQZ5zPABHGU2mLegreZt/z2/wTRx39zlgMXxgw7u1sM7V0zDdz3qvwwrnyVWnTihZ4Y9qo5+CAUiLZt+oni22ITQm1gL6gt7bKZ+1eUXYTf6XzUTWc2ma4V3NqMPQF93HB2lL4hraLZKbyVC5ZlNAeeJohX1zhYm/4HqJVdfRMs7z4WSS3nxOOCuadOObbY3vTN5fCbhSc46eGXayQBZhQFT5BwyUsHUrlNm5P1mcUWG4oo5J1cZLTo1d9+R9Ub7YAJus36P5ebArOsHb8OMXhjG8k7dMFuoL4MfYzOkZCcBXqGipSY1+ql/3Zmf6MTebkwCV7pjhhlhLxImjReBiSDXshUCOBcf/eEci4VXhBa3FB/R/OPP+fGhB0vLOww4JNXWlDmbzcurMTOKMs86DMp3ue3eN8h9L4HzwCne5R7svenAafqdfZx5ZJ22IpFllpN7Mx4JXW48l0Hg3ODNIF9o8+6B6A88yvVU5QvAZhWWi7eGm0GTxduFMiOSituSgyQ2dCAiS2WC1z1I63el52ArEXe49Q1Facyp/7+4ZXJ7mVXQiptpUjnDKVnJPDI7v4sxYDHVYaage7tmeW9v3UEvFHsfhVI4qiYWJfbkAg5zybDG3z1zfWyd X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: oXajlNh9K0jvbG8HNGCTudk5but1SJnjN/gajXcJ7faXTJCbdUDZ0FROgarJk9Rj//axjghncrxoA3nuEaRWZV0zVoIdOnXk5EZVemYeEREUC+EYnTSxx4k/VJcycF75RE7396VC7LFy8RrAunPwk6f35h1XoTFJG32kMJ3aIK5CbvLD0tB99nKyov+vxvD1YXlrjo93NJtASx0EdVSOkEt7BrmC+oJd4VBGdgFgomkPen1B2nwPLFhQJ4ULXWoso/wITKvEzDBTnCamiMtdwn1/2KY533zfmoURXlc1mJ/dTaZSBnAIOUTh6gg6CQLp1C0zgRD4RQGUJiez470ITDZs33KMlFUF5PzE1hJNeo6ovQVfCoCGZRudvLPlBr4XMJOXJKgs84B0+nLQEY0unDTg0RliQUpM4MQR73SugE4ksWYYack22a/Zj1RWFUYb5bQ8FGBD3vwEVvLOVIA3hZ4k3jIwwZacwtBZBz6mHwykwTNoqzNJIFZbv85etIWWQGt07e/xh7Oo55SSxesci/kOFOFritL9QuTheTYOCE19WAS0JJgv5rghm7h1x4kDYOvjh1XNTDddTAXeVAUhm470dsyVNvkZUSm5t8i9YOMQyl75A2ERgva7cLY23EsF03IUHd08U9p0df/We54x8eLLLVhIg/mQ2zvNBoL8b3J+oIcwSv9yufvcJwYlumI64doMlt/pR9/9feRU5TabNcDb4NeQmJBCAHXjGdLH0ssswzImI6O+MHZFXXJw2+fSvzwwcPuOea4SLxDMARbvsaVYi3CLemWvQuzo5FH3dPgBLxrU/JihYMsmsPYar0H8rIjFMfO++4jaDIQVTM0X0VSI4jNzqzV3cibMeImIdopePFecRyoEO1ePGV+pRrtlILBtnrE5VKNhEt92itBQ1r2PhN9HHhTRP68PhzXiaxHY7hV7dmvfaj/shLCE2cfx288vVESFb5SFTREjdOS3MIYmjiuST+m2x+9FKIBaXqG1EmpIxivT+bblwtFuS7Rs30HUlp6R1jKXX2TqTJ4XqrbBHdQDJfw2z8IhAQAhtsYQd/oji5yYmjePMQ88OMrtIQ0vwub4zSHa2/9OX6cY6XKpMtHY+Q3boRIOM2aSO2kp3jVeUejtj0UHsPktpm/BP3oL6SzzODIS2WD27flw+utsYRRyrPmL1ut4Xvq4z4AzmgloStT3DkTtjVyaTxjrDhRwOc0S1eviSYDlPOKuYVaVt9cpSUKUXp8Lz0EznxTkVipSZpYffSmozewray8SMnVIPqs2T8wr1OexWWJvPdY4UVjWuWiOE2EsewexznRdwgwCc73suhFdQtnAw6c0O5Or75By8QwQEqMx3kKHP7yt71i3vxx6SEz/H7D/RiuFpUnmwp7NSjE9JFUQE9y0v/12Me4T8ZKkvql+i7cV9TIfEKWhZa44S8laKwIjyeqqroldZIgPjmszyoOxNBUtAMRwzgnYZ2YH1kqoAR75TuLyapP/1EXd3jICYByATnBedkpklJTNKn/4SSRpAoywaVp1SZ5x7JfOKn2f/+agb+6Zg9PTqjAj3O8JN9uR2Gg5suJpqc1QhBHUCgH+NoOV X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4f726e5c-e83c-4600-6b4c-08dd4be0aaf7 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2025 03:44:07.0605 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: K6HKpw9gcrWuHmhGhmT891TxB1+4wEt9JQw1AkBCOHxxj1Gu3/V4xok0btbwM9p4 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR12MB9434 X-Rspam-User: X-Rspamd-Queue-Id: 634E5A0004 X-Stat-Signature: a9dsnc5gezmk7cgdm7uw3z6913863nhk X-Rspamd-Server: rspam03 X-HE-Tag: 1739418336-814979 X-HE-Meta: U2FsdGVkX19O5NSKSu33krss4sgpIyclFw1HRb3A/NoOaFOTKKLRdMPR0YT01bdZ8A9kfQzH/stcQdOfYHXWe11JVSumCf0tWwQTfC8MtkoW2GzlO+elC1/F2kWc43Ljo1ZzezP3b0VDEd6cKL5gnQ7KNAdIjtRj5u07+hM2Z+iM+LIxNsdjkAfP9hEqSU89kCfQ+lFG30apik9OhsMBBQoGLp3fWcp0yHY4mwEogQGyQC3cq3EkqqPkQQaGP0ZIQCZSbvQsYPsruVQVz4xYp/PPpkExcqSjtuMYAZNGD/rZthVYMrg1zsKsb73JxmLx82knXlCAdj8b0BFHLUQQsJrT6E9I832SXtAjAqAOQgO4smbEJYY/fCxknCTYf9YNGMqSgTr2mjmvu6JUIQmnD93lgp2Nq2aodZtm9eWWaWorIYlYc/BcH7dulJxyaMWOYNuCP9ZBZiMWADHNGv0PsZGwSjcvMGUjPGGCwMiowUn619KmEoHukN4s5a7b3pH4/eCvxx6HCuaf/eRTRwsqPVzhRg2zt8wbGwlCYMW0V4q7bO/GITdhTUwqBd2csLD4jXLl1eXLjMIKDcQ531YOtjqMAmw+WDBOO2SPOhgxuYUU9saLezyNo9j7OZqHR7r84f5tx8KG2MtZLuAevlgH25V7+UBsfcBtJoXlihgPVoEdVklZjbfMlPAuPjByi4ust0JHk73U5dmA4XxamcdzP5AV4dDY51AeTLRcspe7Fw7X43BEPeiaVV6juZUolaREcJ/f8iqvJwEf3qKAfEqOt7XfY2vG9VW+M/31keCz+/i+88P0xTuERq6ng0pZ1Gwpm1H8NiUpr9ABzpBjxSMFxNzRg7SHfORCeUWI/qCao+RLpnFQeIOIAZDm2jNxqcktB8UCSKiCn7YC/0CrGYKCJnRDBLF4JrQVHb3LDcEe1VynAe6brXfNm5n5PfsFgUXW93SxZ1aYD/igwX+LVql aBwBmenb 9dQTXOdel30jnddqFlPGmTgyYyG7eVX7ylBfNcatqlT3wfUhoJ75qIXL1X4BPRWQSbU/ygv036NAnUmWVyyn0nVl7mF69j49+o+0l/oSxt6MXBOhp9nfuXTXMKCzLq03xwgbt72KnW31rZxESvjCqnJyCcODwyJoqQx7sV6xU6rMgS+zxg/u6VinKdIghJcG4Mhd9fo6lA1PWO3o3fm4UcOTQXkNYIDXtl4LU4hggRcLHLZLSYIOcVNmTdZx5jDbYxK1lRJ4bRX/D+/NBd0sciCL8K4owI62j0TiFRIpblpa7GRQ55j1Bv25M8UBZ5bt5oV8I7fOKEqAX5at9yTU8rcevVCYjA8tLBqK/0FBZqz0cVQ1FsUWVQz7g3sWMJXRd4XI1QYS/x2/UfsA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: During __filemap_add_folio(), a shadow entry is covering n slots and a folio covers m slots with m < n is to be added. Instead of splitting all n slots, only the m slots covered by the folio need to be split and the remaining n-m shadow entries can be retained with orders ranging from m to n-1. This method only requires (n/XA_CHUNK_SHIFT) - (m/XA_CHUNK_SHIFT) new xa_nodes instead of (n % XA_CHUNK_SHIFT) * ((n/XA_CHUNK_SHIFT) - (m/XA_CHUNK_SHIFT)) new xa_nodes, compared to the original xas_split_alloc() + xas_split() one. For example, to insert an order-0 folio when an order-9 shadow entry is present (assuming XA_CHUNK_SHIFT is 6), 1 xa_node is needed instead of 8. xas_try_split_min_order() is introduced to reduce the number of calls to xas_try_split() during split. Signed-off-by: Zi Yan --- include/linux/xarray.h | 7 +++++++ lib/xarray.c | 25 +++++++++++++++++++++++ mm/filemap.c | 46 +++++++++++++++++------------------------- 3 files changed, 51 insertions(+), 27 deletions(-) diff --git a/include/linux/xarray.h b/include/linux/xarray.h index 9eb8c7425090..6ef3d682b189 100644 --- a/include/linux/xarray.h +++ b/include/linux/xarray.h @@ -1557,6 +1557,7 @@ void xas_split(struct xa_state *, void *entry, unsigned int order); void xas_split_alloc(struct xa_state *, void *entry, unsigned int order, gfp_t); void xas_try_split(struct xa_state *xas, void *entry, unsigned int order, gfp_t gfp); +unsigned int xas_try_split_min_order(unsigned int order); #else static inline int xa_get_order(struct xarray *xa, unsigned long index) { @@ -1583,6 +1584,12 @@ static inline void xas_try_split(struct xa_state *xas, void *entry, unsigned int order, gfp_t gfp) { } + +static inline unsigned int xas_try_split_min_order(unsigned int order) +{ + return 0; +} + #endif /** diff --git a/lib/xarray.c b/lib/xarray.c index c38beca77830..1805fde1c361 100644 --- a/lib/xarray.c +++ b/lib/xarray.c @@ -1133,6 +1133,28 @@ void xas_split(struct xa_state *xas, void *entry, unsigned int order) } EXPORT_SYMBOL_GPL(xas_split); +/** + * xas_try_split_min_order() - Minimal split order xas_try_split() can accept + * @order: Current entry order. + * + * xas_try_split() can split a multi-index entry to smaller than @order - 1 if + * no new xa_node is needed. This function provides the minimal order + * xas_try_split() supports. + * + * Return: the minimal order xas_try_split() supports + * + * Context: Any context. + * + */ +unsigned int xas_try_split_min_order(unsigned int order) +{ + if (order % XA_CHUNK_SHIFT == 0) + return order == 0 ? 0 : order - 1; + + return order - (order % XA_CHUNK_SHIFT); +} +EXPORT_SYMBOL_GPL(xas_try_split_min_order); + /** * xas_try_split() - Try to split a multi-index entry. * @xas: XArray operation state. @@ -1145,6 +1167,9 @@ EXPORT_SYMBOL_GPL(xas_split); * be allocated, the function will use @gfp to get one. If more xa_node are * needed, the function gives EINVAL error. * + * NOTE: use xas_try_split_min_order() to get next split order instead of + * @order - 1 if you want to minmize xas_try_split() calls. + * * Context: Any context. The caller should hold the xa_lock. */ void xas_try_split(struct xa_state *xas, void *entry, unsigned int order, diff --git a/mm/filemap.c b/mm/filemap.c index 804d7365680c..e28a7a623889 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -860,11 +860,10 @@ EXPORT_SYMBOL_GPL(replace_page_cache_folio); noinline int __filemap_add_folio(struct address_space *mapping, struct folio *folio, pgoff_t index, gfp_t gfp, void **shadowp) { - XA_STATE(xas, &mapping->i_pages, index); - void *alloced_shadow = NULL; - int alloced_order = 0; + XA_STATE_ORDER(xas, &mapping->i_pages, index, folio_order(folio)); bool huge; long nr; + unsigned int forder = folio_order(folio); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); VM_BUG_ON_FOLIO(folio_test_swapbacked(folio), folio); @@ -873,7 +872,6 @@ noinline int __filemap_add_folio(struct address_space *mapping, mapping_set_update(&xas, mapping); VM_BUG_ON_FOLIO(index & (folio_nr_pages(folio) - 1), folio); - xas_set_order(&xas, index, folio_order(folio)); huge = folio_test_hugetlb(folio); nr = folio_nr_pages(folio); @@ -883,7 +881,7 @@ noinline int __filemap_add_folio(struct address_space *mapping, folio->index = xas.xa_index; for (;;) { - int order = -1, split_order = 0; + int order = -1; void *entry, *old = NULL; xas_lock_irq(&xas); @@ -901,21 +899,26 @@ noinline int __filemap_add_folio(struct address_space *mapping, order = xas_get_order(&xas); } - /* entry may have changed before we re-acquire the lock */ - if (alloced_order && (old != alloced_shadow || order != alloced_order)) { - xas_destroy(&xas); - alloced_order = 0; - } - if (old) { - if (order > 0 && order > folio_order(folio)) { + if (order > 0 && order > forder) { + unsigned int split_order = max(forder, + xas_try_split_min_order(order)); + /* How to handle large swap entries? */ BUG_ON(shmem_mapping(mapping)); - if (!alloced_order) { - split_order = order; - goto unlock; + + while (order > forder) { + xas_set_order(&xas, index, split_order); + xas_try_split(&xas, old, order, + GFP_NOWAIT); + if (xas_error(&xas)) + goto unlock; + order = split_order; + split_order = + max(xas_try_split_min_order( + split_order), + forder); } - xas_split(&xas, old, order); xas_reset(&xas); } if (shadowp) @@ -939,17 +942,6 @@ noinline int __filemap_add_folio(struct address_space *mapping, unlock: xas_unlock_irq(&xas); - /* split needed, alloc here and retry. */ - if (split_order) { - xas_split_alloc(&xas, old, split_order, gfp); - if (xas_error(&xas)) - goto error; - alloced_shadow = old; - alloced_order = split_order; - xas_reset(&xas); - continue; - } - if (!xas_nomem(&xas, gfp)) break; } From patchwork Thu Feb 13 03:43:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13972776 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C59BC021A0 for ; Thu, 13 Feb 2025 03:44:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DB0836B0093; Wed, 12 Feb 2025 22:44:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D39A56B0095; Wed, 12 Feb 2025 22:44:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B64FE6B0096; Wed, 12 Feb 2025 22:44:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 95F076B0093 for ; Wed, 12 Feb 2025 22:44:25 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 4E784451A2 for ; Thu, 13 Feb 2025 03:44:25 +0000 (UTC) X-FDA: 83113528890.20.2D9F7D3 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2041.outbound.protection.outlook.com [40.107.237.41]) by imf03.hostedemail.com (Postfix) with ESMTP id 89E5220002 for ; Thu, 13 Feb 2025 03:44:22 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=AtO1V+VF; spf=pass (imf03.hostedemail.com: domain of ziy@nvidia.com designates 40.107.237.41 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739418262; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=K8pPmk9cmC65qEEnIXqv8Or4vcMEJrsAI/eN68/fODY=; b=N5dtDo3JkSftTPx9Rbu0hEvMbMG2UVivnyHbmTDou3TP6KfNO10SFtwzSx5i/GF74b2qwn MWHWL4pBilAluW4bngk61h3RZJHr46Jpx+MPLnXFu/FXpZ+vLDHGmC41hoZhMMghh5G0XS sJi9SYSsk176AoH6J7EopqmUpbnP4+Q= ARC-Authentication-Results: i=2; imf03.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=AtO1V+VF; spf=pass (imf03.hostedemail.com: domain of ziy@nvidia.com designates 40.107.237.41 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1739418262; a=rsa-sha256; cv=pass; b=QDi5VuiL5nsVbtT8nwTVu6d78MEWhaBhAKL7Gq4syLf5PK0ytuivvHPCmfEvezu4WA7znR c/3PFSEVIhR1GJGsQYO7wEURsY40oK+POpNwv0XMLJ1atScmjPdw7Zc42jMlWeBtNxPIOR uGU94tnAuAreQDxlKHx/Cg5pfxj9O4U= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=y5FXUbWcTkZr5xYAU76xQZQ/h6lugZWz5qVq+HHd7S1kD+Yp7cJCBx4+z5Zkj2EaBuXLXwUX1HUVExpn9shJbYN2N2HNrZvNiJZwqlW+7LNZikZCf/64GyQu/Q+64ku8mA8swLCqc348AWxn/n670bpgIhtkKWnYue9zyE0Pd9nIKPz8MHFAVAAjhUbB+NeLtg5EInORhPXBg5zF9x50trcLZ/iI7Ws1VQmj1GjMAvcOetUK8+ZfpGSukMUSEZdh/hssokWcPqdttLZrVBTyB39CVhWfIxfsTiGs4Tx2i4QgNop9gB0YgVDdY3EOAdtONp2eOnxzU6sIP/SLSQH3sA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=K8pPmk9cmC65qEEnIXqv8Or4vcMEJrsAI/eN68/fODY=; b=OXXXwI6XaJa3bLr/HSJNkxOH6EYgMq7Bxf6GwhwwAUl7Tgnyo538PtWwdPeVCwathPCJ6gZCp4uvamjnJ46toan8nYf6nc9fdgqGR6VGNaob6iIx9qOEIOTnYxVxQgwrhKPZ1iElcQ7XPi4hS9RarC4CssAQMfmX3zsn+75r9NyhUGDWV7UD9y96TLKJY62Ts8dGLj9/kM9Ww363xIfTJL7gXT8yEwDat+hs4avm0Yvj66ZcwOYRNA7cw36swKKuW8qM1ADy9A3AxwzewmU9xAyxhflqnvLeFCsusAyy80Zx37Ak3MMnbfrvk4RsGkXRicp0ZSJtMOXgITe5KzZpoQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K8pPmk9cmC65qEEnIXqv8Or4vcMEJrsAI/eN68/fODY=; b=AtO1V+VFfUKGVfydLWeq0Nf4mUBP2d2C1+D5jG7POyW0sstFEOdLMkn1b9l1ypgT12yGFXRrDP55j6acg5LFN15k5OzjtUOQY4Y19UqLFQYS27kY70cc13UyZHCUHqIsD7N1OtJyBCqo8lpSwrVKPthZkDFW5WZ8laIYkXwyRJSv5vzHQRIajZyRYKauyw0NY8eB/T2qlGT7VBad6RbCOQ9TJ2D6FyeCUOzWvTj/O6lw5SbXL03O+zhRNRjLx3bNmx37A5fAadbgxXER39fyxL1hzEkwmKlpBGIBMipz30JoAlgj0KpglH9UTJIm173Rtry/dzFpc2Ct9/RQ4iTUgQ== Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by DM3PR12MB9434.namprd12.prod.outlook.com (2603:10b6:0:4b::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8422.12; Thu, 13 Feb 2025 03:44:08 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8445.013; Thu, 13 Feb 2025 03:44:08 +0000 From: Zi Yan To: Matthew Wilcox , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: Andrew Morton , Hugh Dickins , Baolin Wang , Kairui Song , Miaohe Lin , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH 2/2] mm/shmem: use xas_try_split() in shmem_split_large_entry(). Date: Wed, 12 Feb 2025 22:43:55 -0500 Message-ID: <20250213034355.516610-3-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250213034355.516610-1-ziy@nvidia.com> References: <20250213034355.516610-1-ziy@nvidia.com> X-ClientProxiedBy: BN0PR07CA0011.namprd07.prod.outlook.com (2603:10b6:408:141::32) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|DM3PR12MB9434:EE_ X-MS-Office365-Filtering-Correlation-Id: fa7ebe06-6ad3-43af-15df-08dd4be0abb6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014; X-Microsoft-Antispam-Message-Info: I6Bl8IM0zgjgpoIi4iyWvVVZ1eW+9lCkt/CPeMXrVxBRAUbSRPyuAspxTAtIhtNGGsyihqeosQHXwB3Pn7wRBENBUsyw9108dLLaHKa/NBLt0fFlMCTBzNaNjrjUB9Rk1kOWfoHbNj8y1gnO9ZkM8PEH/vBbktM2Z1NGGpKloqpy6qSfcGlFzDCGk+oBQFQ61//1QtKBwE6S2yUhFbyBN3eJtR3g3ketNYxgG7V0/IJB6L6CSCJgaWa9g0RBLvpOuZ9/4sKlWSsTbYSbQUMVlKkHcoW4hGxX9HPkf7lgLms4zomT+P9t0C6rcTzYUDiknZG9nmoWpqtl1qbpKItxXceBinj2cClRr60iuraxq6+OR8uvsYcBMBkU+BIwrpWuug/p+wipP5/t9ZtyJJjLxvxarR+DB1qF978gnH0Iz57bYBPCBEhX5npn5QPwBupQDM/glGcJkntCiNlO7irMvDlw6kHjtC/mIGCrGHpXyjOMpVo8Inc2A2mksoFYiAcFNHG7qFroYEJ0b6LQYkLO50fEqv3RfcXNKhfZJ6GNQiZMbnzagvbB/AsypzzEHGVfSeWL1JmqM3Y6Aa2D67/2Tnn17ooRLHjFBxM46dzvzIUPwqI5kRuDUqijN3DzbnKPEfRWutJvlFP9O3JAUVebefHTiYrN3cU6gUiJ5/of+sHGpIEEOi9tfcCsMVCfvNSqp9MK9fbPfMFKV1raMx1nBn5xSly4NwjkBcNOQWcpZtsUZyFr3BLJZFviu6O2sXloo+zRZmQz5QwVl9oIFgN6P6caOpTXPjyS5W7yfLN6DFXag8uv9J+3GWeP/oPcxQuHGeCUZsirzNXi6UYeZ6nElM9Y3fTPAczn+oB2+d91swj4t08JZo62Qd5o/MNE1EmGJOu3oaCyXwHDWxBzYMV3guDPe/huIHSprMxxyMC54rXTZ/b8k3XUwgqobiisGaXv4mvB6E4jFK6WcBnr9s60a0sZvwYjoSEjr9ZZxCRu97lCisM5bI1AK3sapNjV8WFW+JJEUadnXwIHW1VvCHOXif9hSRXGwpTLXa0LqxF0d8sLz5GNr8PcVokWC6kk/cN8VHvPFh+mt9uZzHkF8VgOI+uknjRxgQW0zgBFMtlUcvrfrj/nOXDvA4OD1yQDxlm8P2QHdxwCSOhz896+ytqo9crhpDZQ8+pC+y545FWcz7ljgcaKSK2YvPKPDWT6GK8ES6xhuXW3GhDQrf7BaBuwaE3w/5hQtmwXfwuba59jKjhI/gkleL+akXe2SOJoDRLx1ZICbbeEBYG2bz+X20WIYpmAL19/MwyKll22IZDqdrrdyw+18ll4GEFG7k+OLTgVjAOhIj79bsTiEP3l7/7RFDE6O8kuYsF5MuaAtyYf8SS2jZhTKq3bukGZzL/Uw1f3 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: pu1aAhawNWxpfR39NTwNKZVyKYEBJfSY6Nr+fnunAyWEomSjpn8RLXzAkKX4oaZly/rRHyiVq+vIqn5NjO0NDZ1CfC4I3nhK9v5CUQ27tt/UviZfv49ZgzOzIgsjz4G6ZfNao+zDtfca+AMkqfV1T1+fyod2hUqyz/JUsVQrPNzqRuYscrQwZbfKFeIp9qLybATu+S2hxiLljosO1qqA3uB/tEfcE9eNVw5CwAdYvz7KlvjnUBmKhP1/koZxEZ6kPebcSkdqMWI4Es5gib4Qp2p/Idf4ttsm0uy1Wt2kEKlE9Nnv047nOlSvKp4DnXkiGuPonOZwL3KGkNUMgQPk7oGh0CydLv0dgDfCoXkEk19tL5193wHchbxF3qS1X9ajR0Bo8icbb+U4BZMmIY4zVY9EHyB6/sESn+qE2o58Y9u5q+MzM0AaG7G2sBl+Gr36mSsty5rM+F/IRakTYrYd39hPlZdCFbF79SoORpNu9NFNMJuGCfezqFlvBRWKKHEuG3DwLSwy4IYPDBWEFdXOBpSw7B0GblO2ACpE2dsYx3TDxDS5d53h9hj8HlhsMpBV9wy3tBCfQiVyWNfsAPCiQtWaQZ+kj2wOapQXYIUfWNWX8l2Mw9nwozwvrrlcarwT3q9WHhxGvD2KGvbnUf9bQQRyp6uOQcaThOCl5qVZO7MA561XVrKkV9OhCwfKGUGA/17l4yufE/r55UhNWZJB+fpphCMUvJ3SfyAQuIiZ8BGkoq6ff150U0BwPvHThz/fKBbb4neIPwAzvaPIhbOIfQNA4EN6S1PPTY+a4H+sOEMiFAnMdIb8pbOvYFWeMU0yeD0s5D+QitGStVjFeHm3J7z/wCnudTyXFQYr4r/oOSTY4fjj+yyPIZqnkgZVu9ps7jrK7lF743vnMpEz2MVVzasW4JvoOsAv9NUkHAKGPr9nN6hcNA9kLHt9DG07dzMHwF/1Bo3v5DFGITGPVDQodhMqZ7vNk1xV8IlyGHVGKW45G0VWSFJzMHiP0ktoDoVLOkFjp+wKpONvPY7k7aQujDee1lJuxMxdxy/rakQkOGbCyFyZ3Xcl1Gu3NVkFPC4+Bpvcg5KXbHnxctxl8yIimk+ZmTxAXHGR+oeJ6EZYXR1tUvxenCXQl+VAOFJPqdwlsYL+GgmiNHxl6Hvnyc9gHxW/6ZSm6bNxEJlabAEU5I8L/CGeZAI3IMapjmZ2o4rXvcystnKLkOd/xkpd4DJbC54jHyK9LM8u14KQvzlMugK8+7Tpmj1oz66csKZutvb3K3/gXfesESnf1GYKpmZnE85V7p/88/UX9hIR3XzaEax03GKyFx1J3eZwVcR+rRYgqyWZSmj1706kyHymy0llFzll/5y/wZrzcV79WEDWayHNrUUTpNKB1cs+kWv9lZqX2/qM44CIS/QSH29xK3VMsuX13hvN+0m5LSMBq13BVpVz4DCAMJ/SNLz1qhHaPMoYMxsN5pznnHnUcO+VwhajmI47GvV7s65PpCkLxnvkk9rbOKRxRhFwDhcf89QTT8VV7ARyZvt2rbLPBJ9RqdKl3SIX6YTY1Fyu9y+2ysjwLoHViCj31GC7ehrHy+sKgbVZ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: fa7ebe06-6ad3-43af-15df-08dd4be0abb6 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2025 03:44:08.3285 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 28aYuASFhEo6U16wM1riwEvgEE/MwUWaiH55VCmp5iSRZ3wG08QheRC7JrWIawQL X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR12MB9434 X-Rspam-User: X-Rspamd-Queue-Id: 89E5220002 X-Stat-Signature: p5o9inu9tygh9h1sbata7zqtkoasinwn X-Rspamd-Server: rspam03 X-HE-Tag: 1739418262-181954 X-HE-Meta: U2FsdGVkX18CeL2uziUmaeO8ZApDphsV8eZ3rVQj1zPJQ61q+bgAneR+S1EbXaWYyb6Ma+a8Q6rcENGXpVVPGvygLTbq4A7/wwvZJ0d2ofz/w30JfSAZO3FNNdq6/dIR4wwtsWWUqrdu4SFfYqCnrtgy7apQhZMGbWizTrTWQ0uNGvVV76UYWAF9jBRkFZZHtASl5q2rh4x0hnunvtmeWiXxBSjj0mu0mV7yBPUA8VJPxvKOOUpi1aq+VJBUU9Egezv2WaRAp1PVbN4NJxRdqb9Y+uxnP3DGUBF6FinzGCgER2GrxcVcYf6dF/07mpOtCZsQg/UthnzLLuPfPap2eBkfWBGYpbZAIMuTOWvgzxMydkf2llIE/w/F39G3QEZRIfrDJvKNBreFIrSdnkURmSqpvIWW50V9IYCsV16/AZgWXDlC9wwGTW/ESWgrXmW6KrOdjQx5ls2onMPnKgOLB30zDk9MqJdJKY+EYIM+KUWbi6+p1tISwD+g8cMN5ABeizrrpc/vDe/0YHCS5Eaz7uQk26w/uBr2p3Nd+n33wKTysN07uJP4qb8WBkM6iO++yA2VZ8yl26Ppxpryj+K9tdc2sO4Ph+ZiLTFl6Jggi63jlR1TuaFqCiqx6JzU6ySa+Icqp+SUTT3NS1WRA57U6Y01ti0Sq6qFSlx9hpY2dwsldgZuS3t6usLtL6psKSnGrxfQmNXuUmN9aVVG+32u+I+g4WAXfPsl76XS3HJKUyeaDvqJY+Igs7Te1Hajo5g3wv8F43BFVpbw4nppq+uKFyhRBzzw2ifYmookAhAxZSD717KTYt0bMywRaHQet70NdF3v+c75M3Z3ss3twi3PH1Y4Yd/i49eY35FjMjCVbSxbMkYAe8CopQ/ps9x0Zejwa9p40x8Jum/412Upt0yFu9Q21M228nwzqJrSWXc0f+vnn+X83QkDYk1V3Te7z5yw4lZ0GVzXO/HIhkL1lHS afAu+aIh BTXTLF8JTzoLW4ezwQpDBcBPGYlXyWnZIjY30J8xpoecgTkvQ9AXTSyBr19okUCda3uBp2xmkFesHadcO7nniL7duKdXvSP7CU8aR4IsDNJLBT15X+DUy0LroKtUGFD4TKGQO15+/uAB2rvxYi4IQT3u1NpB80qpXj9NZauDv7DIzkxnsiDsTcqNnXPXI/a/Tnys9a1IdQKZtZ/SLj0hZFK5eahY1ydNPI/GGZsqiNHTAJN28ZCy9uhji24KmO7ZXnCdmhc5sv4iJDoOg6jaKBu5zUmZ0MmH8a7eUmcTvzHNlxU7XOKf4aaJNLF7L3XvtclQYFRvO4i/eA0huKiUkogXhpqwyAW0hZfv9l1E6/BM8DgFNBApNN9/qvU3PtxJsLal6Y1zprNIgOSA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: During shmem_split_large_entry(), large swap entries are covering n slots and an order-0 folio needs to be inserted. Instead of splitting all n slots, only the 1 slot covered by the folio need to be split and the remaining n-1 shadow entries can be retained with orders ranging from 0 to n-1. This method only requires (n/XA_CHUNK_SHIFT) new xa_nodes instead of (n % XA_CHUNK_SHIFT) * (n/XA_CHUNK_SHIFT) new xa_nodes, compared to the original xas_split_alloc() + xas_split() one. For example, to split an order-9 large swap entry (assuming XA_CHUNK_SHIFT is 6), 1 xa_node is needed instead of 8. xas_try_split_min_order() is used to reduce the number of calls to xas_try_split() during split. Signed-off-by: Zi Yan --- mm/shmem.c | 43 ++++++++++++++++--------------------------- 1 file changed, 16 insertions(+), 27 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 671f63063fd4..b35ba250c53d 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2162,14 +2162,14 @@ static int shmem_split_large_entry(struct inode *inode, pgoff_t index, { struct address_space *mapping = inode->i_mapping; XA_STATE_ORDER(xas, &mapping->i_pages, index, 0); - void *alloced_shadow = NULL; - int alloced_order = 0, i; + int split_order = 0; + int i; /* Convert user data gfp flags to xarray node gfp flags */ gfp &= GFP_RECLAIM_MASK; for (;;) { - int order = -1, split_order = 0; + int order = -1; void *old = NULL; xas_lock_irq(&xas); @@ -2181,20 +2181,21 @@ static int shmem_split_large_entry(struct inode *inode, pgoff_t index, order = xas_get_order(&xas); - /* Swap entry may have changed before we re-acquire the lock */ - if (alloced_order && - (old != alloced_shadow || order != alloced_order)) { - xas_destroy(&xas); - alloced_order = 0; - } - /* Try to split large swap entry in pagecache */ if (order > 0) { - if (!alloced_order) { - split_order = order; - goto unlock; + int cur_order = order; + + split_order = xas_try_split_min_order(cur_order); + + while (cur_order > 0) { + xas_set_order(&xas, index, split_order); + xas_try_split(&xas, old, cur_order, GFP_NOWAIT); + if (xas_error(&xas)) + goto unlock; + cur_order = split_order; + split_order = + xas_try_split_min_order(split_order); } - xas_split(&xas, old, order); /* * Re-set the swap entry after splitting, and the swap @@ -2213,26 +2214,14 @@ static int shmem_split_large_entry(struct inode *inode, pgoff_t index, unlock: xas_unlock_irq(&xas); - /* split needed, alloc here and retry. */ - if (split_order) { - xas_split_alloc(&xas, old, split_order, gfp); - if (xas_error(&xas)) - goto error; - alloced_shadow = old; - alloced_order = split_order; - xas_reset(&xas); - continue; - } - if (!xas_nomem(&xas, gfp)) break; } -error: if (xas_error(&xas)) return xas_error(&xas); - return alloced_order; + return split_order; } /*