From patchwork Mon Jan 6 16:55:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13927685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E77A9E77198 for ; Mon, 6 Jan 2025 16:57:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 78B4E6B0095; Mon, 6 Jan 2025 11:57:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 712606B009F; Mon, 6 Jan 2025 11:57:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 564B36B00A0; Mon, 6 Jan 2025 11:57:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 391586B0095 for ; Mon, 6 Jan 2025 11:57:31 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E6F6212011E for ; Mon, 6 Jan 2025 16:57:30 +0000 (UTC) X-FDA: 82977633060.23.AF100E8 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2075.outbound.protection.outlook.com [40.107.95.75]) by imf01.hostedemail.com (Postfix) with ESMTP id 1A67740013 for ; Mon, 6 Jan 2025 16:57:27 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=SZuQG6rh; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf01.hostedemail.com: domain of ziy@nvidia.com designates 40.107.95.75 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736182648; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SPvVoJvT4mBBzU5IHfOt+qRzccUAAB1+CNSz4Sg4YUY=; b=jW4TxYrBJh7shNTY6VjShsBxBF8vIqq8ch378f0rHOuNaQoVplr8azQy1+cipfd+9O1KNQ 7VLNuqAsnTmYE4jyK1k6IWRkodxeOgB/kHhrc/uJsZNl2lVubdEdH0SwNzC0bASjnDqO/1 E4sEU2UvkxZaL4fhWRt086B74bav16Q= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1736182648; a=rsa-sha256; cv=pass; b=gVRri23J870/39tcnnYMpV7UJdN6nNqSP+vyt8KEXrNFXUEZ5C5FbEDxxvFL5jA9QgPUzH yIVlxXHF20Nh8etbln+MrmBltoYFtBju0qlZU5v7TRnvwYFRtLrGxZJfUVGLQVrF7lIdHQ x2eTkiql1IHTTE6OB2LZM/Dk0Ovbcs4= ARC-Authentication-Results: i=2; imf01.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=SZuQG6rh; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf01.hostedemail.com: domain of ziy@nvidia.com designates 40.107.95.75 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=FtUorw4fnz7gfZIeiP17bNOg9tqBoV9Aq+GPXxl+Hiut3HLB1CADXzHw7H0c1L9NeF2bce0NvvDgZ3IS6HOY21khS4wpc87kabSpFtpdqbIBB1/8j4TLBpyUmhymEt9l7R2uPYpG2P9B6R+wXCNXa5RgI759isMgyggYBa35N17WNQITcfKcXSEeUN4UwizEDMpqz0TIRcVBu7u8ZZLKZ3gHfKvmC2nqiWBOqbOC83tyrTeN+9X5y9qco6NAURywQEu/vTW7R4PUF0GCEiC12XH7NkZO/cWn0oAm08j4Jo8s9kPA4ijVdyiIGfdaZh13T00ccORlybkwYn4THl0Tbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SPvVoJvT4mBBzU5IHfOt+qRzccUAAB1+CNSz4Sg4YUY=; b=J8MzkTQTeqzfa15TpUiYMyapCuwNajPvyXm3yanQNnS3Q/EWrT8c+PdZiiIP1Hb4HAIRV3In6yl8ItlQQj+36bZnLonYXK+d0O/qjY6sEENfGkGCIzucm+dANbNdz7D7euzKx5wKu3Hu+bJBf/KdwhIUKEc7gtPnIBqphDvoiY77tJPGaARLXRO+ZZM5Ij1QOjV9RrWNKUuFPhbM9Hb/zMAyNGvSvyABLywjpUHiNVyiRhm7kIa2Vn0wD0SWg904qERsK0Yup3LIpwcI2ktL4RX7LYCMGn6CALvN7eESZmnBLtSOjBk8mwoZVQcn2R68SWghYVGOVfO1wBJajpQweA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SPvVoJvT4mBBzU5IHfOt+qRzccUAAB1+CNSz4Sg4YUY=; b=SZuQG6rh6hqu+FEmnrS0uHer/Z/LSQS4dQtNEq0j+M2haqwC+Eb1BAhJOqvzTa7J5n+WjRAKLJziwBF3zaN9UJ2yBXb9+a4aoBr9MCU1VvShU+MhSULzpvp7jb759HO8EdBeswGHobao/PZ0J1WQPVYK15nbg+lX69UUDACj4K95ULL2Wp9q/+D1QBTpBMa1zAnrSFPVb1BJOVTpFVrb2NowipOhEpDu+t0VjvJRBiFIIN5nESBnikjKscxTyEGXm/mlxwY2lKggF60CQkhmhcNhQ2BXhZHnbqoBHFXy4Exyz3HZqHoF7t1WICyjaTEi5TtTmra/U8Vq62ncPanBZA== Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by PH7PR12MB6634.namprd12.prod.outlook.com (2603:10b6:510:211::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8314.17; Mon, 6 Jan 2025 16:55:27 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%3]) with mapi id 15.20.8314.018; Mon, 6 Jan 2025 16:55:27 +0000 From: Zi Yan To: linux-mm@kvack.org, "Kirill A . Shutemov" , "Matthew Wilcox (Oracle)" Cc: Ryan Roberts , Hugh Dickins , David Hildenbrand , Yang Shi , Miaohe Lin , Kefeng Wang , Yu Zhao , John Hubbard , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v4 05/10] mm/huge_memory: move folio split common code to __folio_split() Date: Mon, 6 Jan 2025 11:55:08 -0500 Message-ID: <20250106165513.104899-6-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250106165513.104899-1-ziy@nvidia.com> References: <20250106165513.104899-1-ziy@nvidia.com> X-ClientProxiedBy: MN2PR08CA0022.namprd08.prod.outlook.com (2603:10b6:208:239::27) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|PH7PR12MB6634:EE_ X-MS-Office365-Filtering-Correlation-Id: 88cb1e27-229b-4cc7-01dd-08dd2e72ebdf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: DqEU+yVDVswf/iqKrnHlgBDcHIFARiocXPoBx2D6Ugj4W3Pp48J3iVil9XNejJTaDEpYRbs9n+yh/EzpdEIj0/VnJqf6dQ2Qzqft/cn0umbXvc6bI5gj455YL/HgGhDHpDjBooZINh6uokeE3X3Y7fXHJCI8dERHBFD+VZ1Y2W31nOOsJzkVksSxEHvoq9xEhJrctYwe8smMNqj+I3aA9MJk9Tyz3QssuwIV5PwYxvObb55o/l6zWA6VwgKgE0bCBNObgJcb5YflqPAkIFz4k2RmawQ0qbnqsGc1+YQqwOsErfvNaIt/d8PwqzI+7filzsDtIqhQy2uaocXJhj+CqSx84HDtOnluD2o5mElGVc/VA0kmesLVc14pI7v2YT8vNss9pdEnD9jWU67ihLl+DP1kLK1O2jg3V8vzPszGuS+PaKx3/lgelbqz9SILNwD/gFZ8N+Y2O6pTycP9ZrNNPba6bpu/r4XUsxiHAnOzantPzyQns8S5sLHdMlkycDKianHUiiMJrZn/8wbzMEUA51x9aqdtyU6Nw00nTrVG43mB6XXQkTXJH9+ygb1OtiNHCVez5kXfxdHiA2QN37t9/0qsNY+xWQtuTaB4IGfs4CvBdT5mxuVZWRTeNO2xG49u43ebueBX0WkC6gnLY2hEMr97heStK3CoZnZVGqZK6YHOLNryQYNqJ3RkaYODJz7L6vpdffcZiAi1TL784FuV2Mt4Kdf4EqF/81UVRegLDyFAilKLYZRhyIqFDLSE3YsTNuE1qxRAT3bSMQz2dwkwsk30v8+fJkAFU+XFLKfeJzX044w0xO1kHTTzEiLANfwaYwZq0KtzFJaSVqh9lRzxt8Zh5xCkALRgOk4dx7gy+NimrJUNFpG2Hye4nayQQ9tJElAoDVCdjiQeqDg9f0AF5FjA8qhQ2FQTWjSAbV0fQ5Oo192sSDbfVt5CP5KlazLrEG4ikvARn/EfkjCzJCM58Beur6MnnlbfVZWGD6shyJZVLXyyMpwH6G3aSYszT4Ou+jkp9dc9NikMyUJB05XgKDexxFOXYQGuYKOa/3c7a7vsrjLwYYltMkA5okczu5KKq2FQ9YTVPdyvsbdGDnJGqz/X+YFS4IJll9YsXQAbnsmoeNGa0l+cK+UNWgpP/jntIo6bodDs2lCA7s15ZAkC4WowxTFBngwq8kXCqP1nfA1ZkoVAYThHK2G0FS2x34XD1knXs1V0nFl3Z3xaiEXKJKyCSKHLDKaj6Ne/Qr2TpPzifwAxMrHnhfrOkgCwG0GNURoM495Fwd2gPLo0uMwIdbi+VpQiogIhLAfjFMNZi1fEDPqKNRdfoBfvvusLLdFlbUOjBEZSlt+IfteCWRi/Zv5pjaB8PXTVSJX7cNtHT1kzyFizBu4a0ISse8QCeQTY X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: NQ3X1QnXhu920HW/Ke2kCZAiyZKTYX2gOT7Zzs3WbxbE9oMdLb4S6YGGrOHhCbp2+hZNwCaPGkZlRCyv5tNkvzuJCakazBMonD3E9AsrIKR7pSLZD6ftn9cu/PsZuqj1Vm1KUjhEVICxPNrMr/Ws4cfmYqfRQjqHRnVnY99snkScBRFJ8jiNc1N8yEqq7uxoK9g+4kNZ0ddcNICoqBhxlXdynDaSXl7YchCnx7Anj2ryLyIipZv3KB9gR+g7L0aPgX2JtYQPAfwWtgasXiO5v3JgNoRahKqDqq7RMwWAUK4ItXCEdFGsvKny0loFgzyYd66MCmnHBtUN363ExtgREj0cONWRty1GxAR8NtTQAxJ35HFiH7hjv4yibG1425NrbMTHZYWBah4WyoJtMFlKGsWLZWglUdJbeS6fyfkn5xiGwRWz+XY4jdm/a51sBjJrPtMHfIhe3SCLOE2L8hkFfK/OjdBLN2zoqCN6aJF6qDA23h17pJvcSGFDA/qf7FRaorDHeIdwy/ZqqblyZC3HBjCfUpT92MEuGQYTuoAW1we9Wfp/8RGpvQjILfX9ptys3F9L1vq8WIsnQ/X4e6EPgO2yFPq8B2+wT4J5MgK3tQH0GjVzUmoocXnCiKif0RII/f8azGxJvklCQtxxEIl2xCJfH/Iu+y5dhQ9E1xcRsvutQORMh/bC/e7BpSinHvTH54mCX4AIbQ3YRkJvYSO7xZsPFLElNFjQSr5vznZzWsKklWSPH9qGpg/wW1fvns1q2YNNi2taeTRntzEwLtE5hPJLE/VEm34stdUe2W+y+4LM8SuRxy8zQD/cWoghGcYANHn4Ku4Ybd0E7k8aP3PaCrrSZIk+GVIEcS7FX+WPPeP3DnoWRDVPgaClyaGKL0XvRoHYDkUHMlXCQkJm1HQwblF3uGp939+pbxxFx6TDRYrAb7l8K58SNnX/OwWb31LmPtQGImoRlPRw5t7tqDWGprN2V90B6p4RJVrxHPuzULKVrZwMhbZJxoOIZuTLkYAjQrXZHYsv24bfYnEEQ5ZjAu7+hvjQ5rf+V3ZgQMy9RVm70bUBd/MPdjznyn+IEVS2si0FM1QtwiXBtC4lpjMfZiO5c4SjTF5LY2tRE7nrYO2x/xz6PgXxRLNfgmHGHKFIfPweJaAiNeqamuwuKo7E2m34tFNrjpHHhYByG10OM+Jw6ruG0fR45ECmz4T+4W77nbBhWxv0o2OnavPLWpSA/RFI2b+PmaxY5HUHl1s4zttu1qynWiAhdFr0zd5cnzfn4fOkcG4e7fL7NWoyh7dlo233wA2Xtpa6ltSr/D5VeLRzWCW8GZVmZIGAOI/Ke3u+1++lq8FfEvIy4e6ss+Q6/3Un3MfjsbYEIW8B5UinQae+R1HIY2/Rh4/pUm3sbGjMKSEpJpyexrueb9cQ6etqsLj/wjNGhAy1FlKlahmafSyX4e5alSWFGyG5rui1SS2m5YhHj0mJTayBRW6/bTnLmiIGYF1w/zbAubUKPhiR3HMnbvFi7ZE2iB2IHWye+WMvc65E5bTuK/kTrn3b6CTJbHhRRU81E0HacpkcOPtZrprKdtKnhHsWd9ShIm3+hXVs X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 88cb1e27-229b-4cc7-01dd-08dd2e72ebdf X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jan 2025 16:55:27.6119 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: eHS5XiGVcKiTcYLOQ9lOclptGe2xgJOHMFq4iy11H9ikwkX/7Wls+Rs6EMyXd0jn X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6634 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 1A67740013 X-Stat-Signature: x8ehms641m3i4czsuxbcxqunur7z3szm X-Rspam-User: X-HE-Tag: 1736182647-28494 X-HE-Meta: U2FsdGVkX1+yaP1jpKrleu8B9TJHf48oWWMGbXqjJSokk2pgMdVzsRbzFyECWw8IqKUPHX3u5dRjPXrojztgt7qucQAjgvCdm4372y+g+CUQh4V/+T8Wt8ktrYA2rbn7f0q+BkKRrn7vYJqku8wQdUObPmH04+HuuJD7QGZHHg8yFtQAxPUETzxDmhumCR3bU3paoBI2IFXRPnlUU0727wo0zjPCRNetl94CGgbumDA2LUVEADNAxmKafEHSSvDkDIiiTtEmTFZ6HAEnmeBoxr7XWGSmpXgWzamywlZhgShpXhnAv7g2H5ldxs5Z3v1tr8LQNz3MVdnZWPzAcGjp9fNkMmU6DKraaBIN+kb4oECp19Q5nww45lRa0uecieIOVUIwJ8XglOIVyQ86c7w4/Fc4st4kUF7oVF97yC2XvvIj56PzqJjyvEqKs25815k0wJ41+17JQzMs9SZ1lIims2wBNjOcgD94FhAKvsj9AqQitvRl/pFfbt/K6MFc0HAZOClG8WQjAFitk8lS37s3KsM8m8AcnYkGNvaPlC7y3gq66oRrXB3qyKLRDa1mhcesLLs4QkuS0zdcjEZu4G4psrhMJIKBSDI4ADI9YRyk/Av2CDFQXE9u6Vm2Oc5g1gjMJDWVzzeH2k/N74uClSZ5vODOhkFobWsPHLFbMh2y1znNjMZR5jakHNCN9h5Fnx7VecldP1MTgv4/kgJGvrqW0BXvEgq2hsrK+b0LPl/BLr3uQ5EVyU6GAIyDL4s58ujHCEYrCSFrAEhhjDSj5T3DBM8WAHlq2GJKibMGiskbb450N0Bejf3v7lysgKW2bsUotlprSFmwIeRVlziHhFAkB9lzH8Ll/ZxE9Stg9ovC+8d54V/h4uKAxN9dKym3Nloblkty10ROowPbpJrnoIvvxNooNZdtBjMgz7EuG4NC6ytA1nBpiIxDhAAAIpTi9hopq61v1LF2PjbGwSYKB6y ihL1/1C2 WSgIGVdmON9NR3DWVJ8Su18ebYFiBWf+UlnMGwQkPR6/4BjIyxOq0bIuPA7hgM3aIc8KkvD9qPOSBg2VX+lBZTctZA/Lpy4Us67W/OuW12Rk9sfqKHulCgJ6WWEntYFazJHIFI+pEoqokDtrW5PsiWs+gfiLpNXafPcpJu4xAojxu+UtWlXdmFVQKxnsiucKcExx/xQCOQc7BLkOv3s2jCtNLJcklYGNGPMYLMVe5n8TUsXccYjTwWg6gmp9PlElJGuPCyqK/o/PZcaR448zYoVg9nmV/RkPPGB3PKlOHufLTD6Aty3eTzw6qfPei9rRAUzMBUbBzBDb9vC1Ppk04mgtaTH3uifr7UokjID4hi9aNp8GAfyzDha8XAz9RlEVAO6y35oEE3ol0Opw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is a preparation patch for folio_split(). In the upcoming patch folio_split() will share folio unmapping and remapping code with split_huge_page_to_list_to_order(), so move the code to a common function __folio_split() first. Signed-off-by: Zi Yan --- mm/huge_memory.c | 107 +++++++++++++++++++++++++---------------------- 1 file changed, 57 insertions(+), 50 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d8e743f81e76..586870e60003 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3724,57 +3724,9 @@ static int __split_unmapped_folio(struct folio *folio, int new_order, return ret; } -/* - * This function splits a large folio into smaller folios of order @new_order. - * @page can point to any page of the large folio to split. The split operation - * does not change the position of @page. - * - * Prerequisites: - * - * 1) The caller must hold a reference on the @page's owning folio, also known - * as the large folio. - * - * 2) The large folio must be locked. - * - * 3) The folio must not be pinned. Any unexpected folio references, including - * GUP pins, will result in the folio not getting split; instead, the caller - * will receive an -EAGAIN. - * - * 4) @new_order > 1, usually. Splitting to order-1 anonymous folios is not - * supported for non-file-backed folios, because folio->_deferred_list, which - * is used by partially mapped folios, is stored in subpage 2, but an order-1 - * folio only has subpages 0 and 1. File-backed order-1 folios are supported, - * since they do not use _deferred_list. - * - * After splitting, the caller's folio reference will be transferred to @page, - * resulting in a raised refcount of @page after this call. The other pages may - * be freed if they are not mapped. - * - * If @list is null, tail pages will be added to LRU list, otherwise, to @list. - * - * Pages in @new_order will inherit the mapping, flags, and so on from the - * huge page. - * - * Returns 0 if the huge page was split successfully. - * - * Returns -EAGAIN if the folio has unexpected reference (e.g., GUP) or if - * the folio was concurrently removed from the page cache. - * - * Returns -EBUSY when trying to split the huge zeropage, if the folio is - * under writeback, if fs-specific folio metadata cannot currently be - * released, or if some unexpected race happened (e.g., anon VMA disappeared, - * truncation). - * - * Callers should ensure that the order respects the address space mapping - * min-order if one is set for non-anonymous folios. - * - * Returns -EINVAL when trying to split to an order that is incompatible - * with the folio. Splitting to order 0 is compatible with all folios. - */ -int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, - unsigned int new_order) +static int __folio_split(struct folio *folio, unsigned int new_order, + struct page *page, struct list_head *list) { - struct folio *folio = page_folio(page); struct deferred_split *ds_queue = get_deferred_split_queue(folio); /* reset xarray order to new order after split */ XA_STATE_ORDER(xas, &folio->mapping->i_pages, folio->index, new_order); @@ -3984,6 +3936,61 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, return ret; } +/* + * This function splits a large folio into smaller folios of order @new_order. + * @page can point to any page of the large folio to split. The split operation + * does not change the position of @page. + * + * Prerequisites: + * + * 1) The caller must hold a reference on the @page's owning folio, also known + * as the large folio. + * + * 2) The large folio must be locked. + * + * 3) The folio must not be pinned. Any unexpected folio references, including + * GUP pins, will result in the folio not getting split; instead, the caller + * will receive an -EAGAIN. + * + * 4) @new_order > 1, usually. Splitting to order-1 anonymous folios is not + * supported for non-file-backed folios, because folio->_deferred_list, which + * is used by partially mapped folios, is stored in subpage 2, but an order-1 + * folio only has subpages 0 and 1. File-backed order-1 folios are supported, + * since they do not use _deferred_list. + * + * After splitting, the caller's folio reference will be transferred to @page, + * resulting in a raised refcount of @page after this call. The other pages may + * be freed if they are not mapped. + * + * If @list is null, tail pages will be added to LRU list, otherwise, to @list. + * + * Pages in @new_order will inherit the mapping, flags, and so on from the + * huge page. + * + * Returns 0 if the huge page was split successfully. + * + * Returns -EAGAIN if the folio has unexpected reference (e.g., GUP) or if + * the folio was concurrently removed from the page cache. + * + * Returns -EBUSY when trying to split the huge zeropage, if the folio is + * under writeback, if fs-specific folio metadata cannot currently be + * released, or if some unexpected race happened (e.g., anon VMA disappeared, + * truncation). + * + * Callers should ensure that the order respects the address space mapping + * min-order if one is set for non-anonymous folios. + * + * Returns -EINVAL when trying to split to an order that is incompatible + * with the folio. Splitting to order 0 is compatible with all folios. + */ +int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, + unsigned int new_order) +{ + struct folio *folio = page_folio(page); + + return __folio_split(folio, new_order, page, list); +} + int min_order_for_split(struct folio *folio) { if (folio_test_anon(folio))