From patchwork Wed Feb 5 03:14:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13960494 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2077.outbound.protection.outlook.com [40.107.212.77]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D883421422B; Wed, 5 Feb 2025 03:14:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.212.77 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738725285; cv=fail; b=erM54jSGSvywTLNsQf3fFM2nWq4WOathThZ3Kkj8lHs/mkCowUvfzuxc0vjG8+l1T6OyVbZidTGlLXTo2QOiXK900Q8302EJy54s8BuG28eKSe6hNBOWKiMmXyjYTzkPEQRPE1VaW73APBPdLMP00yLvINtTNeIveMj77/JXq54= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738725285; c=relaxed/simple; bh=1d+NvaOvoKvlX9KzmLvvhy1ae5cs7bf5tRsjdEZoy5M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=m5IVvhv9HmooaauVt4RkTtVqGFPtR0ijH/xUPdtrF0HmnQbQF9BCiWv8EUgvwrUS4S9BuTVOjMvN+71LrYKmRtQxCOdnvRNdB9MfRps5lSYVt6iTRGoIUh0HktkHl5/2jS9u9wKhxpEQVDEXUrocxfTz4EMVx4ZwKt2/q9003y4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=SPzXvQEV; arc=fail smtp.client-ip=40.107.212.77 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="SPzXvQEV" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=eLaAIBJoN7XenzwludgH2JkvonCO6zkalTWIELsuFfrgHLPYJCn182Yk3kMe+Y64TUl9M3IccHbQWNyqFxqxxTU6k0bW5ig9pciP9crAxYapLzXXoxuPudNktVuEDiX2siq1cgRKm6Yv7YgRrfpVyGN+O/EtmxWsKm6dg69H/7QvBvVkJ/GVxuQm94etNYXlEIXnl6wSyphL8dUA7E9FFX/GBhB4UJdan8Bfvn0V9YInqIiMnxNB4KEp5W/6Y67LQbPGCUIHNgCJixHSH3rjM4MFDtrS1M6MmpDbX3+xPrJUbpHBHVtWlk2tLuff5B0UcKcJq1cBM6bj8wQyM4Rtzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jW2+/ILTH9BszGEwh1OHt2pRusN1IYxi52xgipsQPkk=; b=tslar8kePCXBR9Q5lbVE+PaxIY9vOVJXVvhOb9SWDyocX67TPOS6rjwB1mVnZye5p2bA+QvttL7pzwXqSfkqo2oGQP5a1Sjb3Gbe4qUICoXC3hsvDG/nd+gb4R/eOWyVnRUIwNMYHKsDzgLbwKnp/jvNw9a0wdfkvHnvypCrdZmKhtm9wKk0AELNFE+SebOfXXRJn+OdVfpoOc6cMXg7BdbVgs0qGygty7i/7SXljX0CoEh0AyvbuXsOy7ZsKlXbsjZf7FUuSpXkAWlDsxQR7Qwqg/Z2JCkuISPfWvGHoTkP30/OxG9GAYkdE8NIajZBoUU2f5TSBySJYFPWhhR21A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jW2+/ILTH9BszGEwh1OHt2pRusN1IYxi52xgipsQPkk=; b=SPzXvQEVpmWOddzzZKkCaoqUcaRmo8PVsxv70kPr7SUhwKvu/nA9ERktvIo1gBxjzYo6JA3K/8mobqbUl++6fOVyS3lwfrL64euLOPMC4X+uA5sKiCoa+Q58+Oz3AVzx5f66a3sGkecYwQ+wDYosInUGlcU4NQutp0b7OHlxQhvZxm0kcOPU8/SLI2b9RhE5hCQj9Fbtq52WnWf9Kuq6kB9ZSy6iIlLAmAp+w27YYMLDlV3G7mwdeqJZB/UqaEiq2EsqloC77JiGQe24GWPXLycxF77FSh/1b/w5wUzM/GFh8gh6bfjWF539A/r1lgfsDkT90O3Rxr87iEKpOLxflQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by MW6PR12MB8865.namprd12.prod.outlook.com (2603:10b6:303:23b::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8398.24; Wed, 5 Feb 2025 03:14:37 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%6]) with mapi id 15.20.8398.021; Wed, 5 Feb 2025 03:14:37 +0000 From: Zi Yan To: linux-mm@kvack.org, Andrew Morton , "Kirill A . Shutemov" , "Matthew Wilcox (Oracle)" Cc: Ryan Roberts , Hugh Dickins , David Hildenbrand , Yang Shi , Miaohe Lin , Kefeng Wang , Yu Zhao , John Hubbard , Baolin Wang , linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v6 2/7] mm/huge_memory: move folio split common code to __folio_split() Date: Tue, 4 Feb 2025 22:14:12 -0500 Message-ID: <20250205031417.1771278-3-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250205031417.1771278-1-ziy@nvidia.com> References: <20250205031417.1771278-1-ziy@nvidia.com> X-ClientProxiedBy: BN9PR03CA0171.namprd03.prod.outlook.com (2603:10b6:408:f4::26) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|MW6PR12MB8865:EE_ X-MS-Office365-Filtering-Correlation-Id: 0aab9b85-bee2-407a-b150-08dd45933906 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|7416014|376014; X-Microsoft-Antispam-Message-Info: CvpErN0+xu8cdCDZkKTVKFtEx63dvMMkG+BORhaZg3QhrCVNg65zSb4eB31q7EG3ua63paypEeoWGsaagufmRKJHuHO3nCqUeL06M0pRimOiNwiq0h2I9Ij4fb9Cdsa0Ou421fcVDqGA2soVjQzCem2xw90boziwPLKf3L6l0wXhGtVJnQk04VkQF8Lc77fPVsft2FO+/X3mYnTCbBw+0D+loHAvIL+y6hE/iwPo9+TghFotAkjoUAVndCsOY2BoZHclrzWzFunLoBRqDaKyrD4YhBu78aC3cjUlS2IMJ2VDfDKfBfGp8Akmb8gxs1x8vo9E+5hW+aqPZgrco2XIPNt8b8mML9W9HuJ5W4n8K4iI9XxtLM81aq8iSCJ3vBIoq44F3MGBnVQ5BbPudERc+Ot/hF/fKy4I004xpyPKqm5r5ugVFOg5UoblXiwFYNfC5tqAbdapPXbQ70z44Sd0Zs1ogAflU/nnOU/7PRWk/2l/5mGwEvFnjt/9eEd62KACoZk7zyYwSmSXZt7SaMIwQU2Z9e525FLVsl6o0pKgvQsFxepbhfYUeiBZ7rbKt5idZvr3YbNfGMJe5Ef3q/YIXqhbCfvrFQvPSpPs0LaEGlai8LH4GgfqOdTR2QtJxbokSeC0vjdJMBwQCMBquaj1f58okTTGZIK+wreVvIdxiEYm5cygR9ubY7rZbxPsZO+whOKZzpmGDDsuUgFwi1OGO5nmhMl0OOOTyTCwGwPzn5hBy8ZgJhi4y/LpcvNvEOdvnLqi4//ivkrRtFwXKPnAZv6Dus2kse1QvbVL9KeHw+cB2CtKBlswolFgHAmqdmqhmFJ3WzWW6FDSNZMtNXe6VaAswVOUHESik6spM7TPJ0eDAcwMjCvzvHhhaD2TbiJtxCFN0/fjm96RLfSMut7WbUbWpnNxQ8iAFeFzAPgaBqokR0ekgPu9daBGXHdv7Qr8b46+tS8+ZilbfX6ouD9VEU9ABDTdJpEf6drXLRTB8jttqTy5wRMMuu4eMfqEGX8exNlQ1e501eqhkcoWDoy7I5YVBmkNSQeg3R3SXCpktXRQJDA5aiHwrWF6eoNqUfhtIUzYWy6hIZnM+ce4lh67bXzY5LCLk3M8OfvIRhrwcUtDuiD4EcYVoz8zSibzGOBpLL8udcj+oby69xrFGI3xyP0N5YACvCZ0ep39wGrydUUFzojaKI5OGHXXw5oWIdEDqIqHRTCGTO4ADAJTc947ZSgxs9MO6FhMbUbsBrbfmF4U2kbnZ9K92jfMMdKF66yDyD9chS9vX/omiLkpgqJGJJ89LaWKfMGVXqy3C5FQqdT1TCQpklt0g1eseB+3/BSr7VoF+VzbjBKGDWDXFCbhcaGF1KOks6zuIbITu02J7yWzXibsahH29k202HgUi3BY X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(7416014)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: NHrkXLebJi81Pp153/HLPvJRlbkgW4zSQ7AnCsi/HIexMRQ0RSt7chiPKWG2TWNHl3+s0r6V9g4vEDqpbCAehWhjae1ul739bkWKaflEkDfXHNm8vqe3tV0au4vKkRJCz4fU3Kz96vZ16Lq5c8Xx6wrXTf8E+VwL77g8JWL8+AaOmIrgidXudfbV7Y9Brr15th/oGa8jp5ZgOBAqMKbs402obosWGhakgds1J8wzptVx1BZ2wktBZF+wLgSb5Z2BY/nIEaBvQKKXaDh2kHOR9PgNUYNeMQ3fKE7eIqu1oPNCPX81lbatxX5/4X9n6+kPQdmK1zLA08bEELNZYzs4KMGxyNaXCASY/DwJqx0R9osEjZOLan+80WBl2P3iDhc2uGqnaFn78Uok+jZbFk+MytZgQm1zopPcSZ4CxlGsp9BWYYjDwAuJnMUNVPHAI1oETTWihafzlyt1tYJcvkWLeP21b8/anRC6JHPBy9EQRUm5hZi7FY9krkWijVJY8X9kAZVz/jX1bCO2n8AKyI7GA5RmjsgBmqe0DxwA3btIQOFXBGvoqVxsryACUFvxg3AJ7S4iviSa/h30Mf0Cr7SGGMDMzXNDqMBXmsMlp+7H4OvYt9aBvzeQW1E/pYgVTPWZRr7an9O32Tv2bA77Csa399OcqRrq8EPd0eq3FWSvwEuxphSaN0xVXCtaOirSBNMo8GChCSyF3ALLgA9OTDCOPzVEfCd+U0MPs+RBX2aDJCSe+reIq7l9d+gU15FOfHroAi1l2+zMhhj5DhHtA2xJ7W6h45OGW3FCRLdwt2DEQ1gWj8zqAiKvjBDN2+P0Z5t6MWudwdtWmnbiLZ4Z9y0gCBCWhA1a3HFZngi421FIwAMiOM0dIs1c2nAGh1oJiyICDs8ui6+zQhPx6+kpUWBUBIWpEZCUHy6b4Q2Jw5taHkHSe8lTQDLGPICyurAOLvaCks6uCTxEYuFcwdr/sUC3wZW2k5hArNyuJNG7xtgIZrkbTRoXxXXGVOQhX48lOgMzEejoNzzP3Eq2L62k5MArLYb7DfeTcpneyqJxvxZrN1AqxeAfrf86k3zkqSgAvMM0NfAPrKaer9hqaPPgY7OwAO58KmBdGoSc2qcmKmBgT0di7AQucyMISsTGKvJF65Tbr1Aa+P7XEmd9zVkEhcd5YSLNbyfFt5EgvItRev0z0i1aaFuRZgcbSyQBdXhtwym4XFpAWJ3Cy1dP2R7lC73C+VH1cMnNnbYL9+PK2jMg0IUkcqGm5KYI0NKmvhuXUangBCr9S2/EpZEQKrKemUvXY2BX7kzak+80NLhg4E7uTUg1hFx8S5mrfhy1WNNeq28S2XghdRE+mPcCbIZFmz3M9iChermszQhZcQc+EuX8z5yievgeoXFm+vVO0X0ehD4OA4nU1iqsHhQPc4oQPAgE+UW5EjCKtJ6gYUGIoX9p2APkohV28jUglh+aGMtuRaUDFTX+52Ce38LNS+9EbeBTkLQ4SjiIOJUTSoezcO/Gm04B02hN/bmd9P76mrdzSxh7LDAHAPKqBjbvx5kh9O69ZspoWqs6sByn03tI30HB59rXlWRoT20AC06cVFuS9ioq X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0aab9b85-bee2-407a-b150-08dd45933906 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Feb 2025 03:14:37.6949 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: V/ElNChP3AbTR5isYhkDh+1/43nOCjuQyDsFtQITf63yFz+4HCuL3M1niN5cSnbl X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR12MB8865 This is a preparation patch for folio_split(). In the upcoming patch folio_split() will share folio unmapping and remapping code with split_huge_page_to_list_to_order(), so move the code to a common function __folio_split() first. Signed-off-by: Zi Yan --- mm/huge_memory.c | 107 +++++++++++++++++++++++++---------------------- 1 file changed, 57 insertions(+), 50 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1948d86ac4ce..848bf297e307 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3741,57 +3741,9 @@ static int __split_unmapped_folio(struct folio *folio, int new_order, return ret; } -/* - * This function splits a large folio into smaller folios of order @new_order. - * @page can point to any page of the large folio to split. The split operation - * does not change the position of @page. - * - * Prerequisites: - * - * 1) The caller must hold a reference on the @page's owning folio, also known - * as the large folio. - * - * 2) The large folio must be locked. - * - * 3) The folio must not be pinned. Any unexpected folio references, including - * GUP pins, will result in the folio not getting split; instead, the caller - * will receive an -EAGAIN. - * - * 4) @new_order > 1, usually. Splitting to order-1 anonymous folios is not - * supported for non-file-backed folios, because folio->_deferred_list, which - * is used by partially mapped folios, is stored in subpage 2, but an order-1 - * folio only has subpages 0 and 1. File-backed order-1 folios are supported, - * since they do not use _deferred_list. - * - * After splitting, the caller's folio reference will be transferred to @page, - * resulting in a raised refcount of @page after this call. The other pages may - * be freed if they are not mapped. - * - * If @list is null, tail pages will be added to LRU list, otherwise, to @list. - * - * Pages in @new_order will inherit the mapping, flags, and so on from the - * huge page. - * - * Returns 0 if the huge page was split successfully. - * - * Returns -EAGAIN if the folio has unexpected reference (e.g., GUP) or if - * the folio was concurrently removed from the page cache. - * - * Returns -EBUSY when trying to split the huge zeropage, if the folio is - * under writeback, if fs-specific folio metadata cannot currently be - * released, or if some unexpected race happened (e.g., anon VMA disappeared, - * truncation). - * - * Callers should ensure that the order respects the address space mapping - * min-order if one is set for non-anonymous folios. - * - * Returns -EINVAL when trying to split to an order that is incompatible - * with the folio. Splitting to order 0 is compatible with all folios. - */ -int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, - unsigned int new_order) +static int __folio_split(struct folio *folio, unsigned int new_order, + struct page *page, struct list_head *list) { - struct folio *folio = page_folio(page); struct deferred_split *ds_queue = get_deferred_split_queue(folio); /* reset xarray order to new order after split */ XA_STATE_ORDER(xas, &folio->mapping->i_pages, folio->index, new_order); @@ -4001,6 +3953,61 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, return ret; } +/* + * This function splits a large folio into smaller folios of order @new_order. + * @page can point to any page of the large folio to split. The split operation + * does not change the position of @page. + * + * Prerequisites: + * + * 1) The caller must hold a reference on the @page's owning folio, also known + * as the large folio. + * + * 2) The large folio must be locked. + * + * 3) The folio must not be pinned. Any unexpected folio references, including + * GUP pins, will result in the folio not getting split; instead, the caller + * will receive an -EAGAIN. + * + * 4) @new_order > 1, usually. Splitting to order-1 anonymous folios is not + * supported for non-file-backed folios, because folio->_deferred_list, which + * is used by partially mapped folios, is stored in subpage 2, but an order-1 + * folio only has subpages 0 and 1. File-backed order-1 folios are supported, + * since they do not use _deferred_list. + * + * After splitting, the caller's folio reference will be transferred to @page, + * resulting in a raised refcount of @page after this call. The other pages may + * be freed if they are not mapped. + * + * If @list is null, tail pages will be added to LRU list, otherwise, to @list. + * + * Pages in @new_order will inherit the mapping, flags, and so on from the + * huge page. + * + * Returns 0 if the huge page was split successfully. + * + * Returns -EAGAIN if the folio has unexpected reference (e.g., GUP) or if + * the folio was concurrently removed from the page cache. + * + * Returns -EBUSY when trying to split the huge zeropage, if the folio is + * under writeback, if fs-specific folio metadata cannot currently be + * released, or if some unexpected race happened (e.g., anon VMA disappeared, + * truncation). + * + * Callers should ensure that the order respects the address space mapping + * min-order if one is set for non-anonymous folios. + * + * Returns -EINVAL when trying to split to an order that is incompatible + * with the folio. Splitting to order 0 is compatible with all folios. + */ +int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, + unsigned int new_order) +{ + struct folio *folio = page_folio(page); + + return __folio_split(folio, new_order, page, list); +} + int min_order_for_split(struct folio *folio) { if (folio_test_anon(folio))