From patchwork Thu Dec 5 00:18:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13894607 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 490BAE7716D for ; Thu, 5 Dec 2024 00:20:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C54F36B009B; Wed, 4 Dec 2024 19:20:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C04766B009C; Wed, 4 Dec 2024 19:20:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A56B56B009D; Wed, 4 Dec 2024 19:20:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 820F16B009B for ; Wed, 4 Dec 2024 19:20:49 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 4834216133C for ; Thu, 5 Dec 2024 00:20:49 +0000 (UTC) X-FDA: 82858999230.06.754199B Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2084.outbound.protection.outlook.com [40.107.92.84]) by imf21.hostedemail.com (Postfix) with ESMTP id EA70B1C0010 for ; Thu, 5 Dec 2024 00:20:17 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=Ak7Kk8SW; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf21.hostedemail.com: domain of ziy@nvidia.com designates 40.107.92.84 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733358036; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=a+3spWHdN2EMOLZhAL3VZ/AE7EL0e8oC5PEvi8fneRU=; b=oBIZ663/937rgTxEOl/AWLUyo1+GPOF1//Vo0VnrLVkO4+kffI+PIBNUTaZcd2bZO2eBXx AaI3l84AXaJFjM7Z4w5XiiwvsGm/ZGjlKhobLjpcO8VccU4yUXj2gnMqV22OdMT+TIcFN0 lTLES9eR6zg0ygv4o0ajdqn3LUNN6vk= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1733358036; a=rsa-sha256; cv=pass; b=UoCP71XceeWDThH+mGxcAUXZVHXfI094cJphtUKT/vJVbYt2o1JfNhslJWgAQWLnkVb/8H 3O4j9TDUxnFCeuRdmkdxYIIMQLWUINPm1u9/fnPx4YYOsUZ+7VpeNfYkD1B6EhbmFC8Ba+ +V/uogGpzgVTC2nItwYjvkXcST71AUE= ARC-Authentication-Results: i=2; imf21.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=Ak7Kk8SW; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf21.hostedemail.com: domain of ziy@nvidia.com designates 40.107.92.84 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=oN+5qogiez2NgcifIL1b/9Uxg6pTc8GGnmWDogYvYA8xmY41gvuTbBlbSoWAvewSZ+iP5z0aQqAcLNsHvJnIo2lmkxw2rBTwIjSuyZdDctBojkG6tmsuiIM6DRAvWE3doDuoegJ8ZCLUBdpbPVVNrAMZ89NXrtgX6e2O0K5saaDK+3Pwidev7N0Y8vayGd+SU6MzgTZb838yoUAV1HYEkP5euMWZaniU/cmDZiVIc9gR7x5xqOX0LnJwgrJjn3IBLVXXT6gGBGrk6U5+6QRAfh7J0NC8TuxhReBJ5nbmOQ0A7GtJ6NrZIVhesvTt8ReeTUwc3CSlpMqr3SGk74GQ/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=a+3spWHdN2EMOLZhAL3VZ/AE7EL0e8oC5PEvi8fneRU=; b=LcWylKiZrXb8x+tfXeGnqZM8GNc0nRD0x7vZ7EYba748KtAW/cmdpY5Ny7oCIiL3lB8ox9cLSlFErgo+yZVU1uaFNE8eNHAUpMoFOJCFfJk1NRbb9cRMH6x+XZn9G6AbhIyuCV126AfdGAJ5G2LnnQlgqL0086KrcNx8b5EiwBDsjkqLnyzTFMsoSjiid8OvI4gdfY7t7KZ9PnGdEaWRJW0XAvlB/PWKWSFlZtsLQ/VtvOngF4kjqRUlSD1DdLOKCIi0xwxkJLGeJBimM7eeXSzO50joZUZvEfOCA2fUWbsViA0KWGPIpaBahJGY3/SuI+9+Dgmq2Pz/rWDMFScJlg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=a+3spWHdN2EMOLZhAL3VZ/AE7EL0e8oC5PEvi8fneRU=; b=Ak7Kk8SWZLEsM0pq+jI56diIOKrSoYgm80zOCxHpDx2p4InJL15fUfcVfXf0nRMNPw4EeJS4PMKC4ECAiWmiBOQa4GdYbKeFIe7C3MG6kGP3UXV1wsVcdCXUv+SW40MvvTG81BV/2/giOlZ0Cxt9rT+Khoxz3w0JGd05vJyu3ww7NqjlcqFHxV0fYtYwBXPUVKpc4PyWyYnmGIHYheGGOfsndZ7TcZNqVeCXbd5Nml0ToqK+i3ddCYyU3q98F4E+PhRyucr/gWiM4EwixmeNWqUgJALhEsauVpnADHdHvlyt4crwzjdfn4q/BtjJ9tGuYGJ7mN5fdz7Je3ZMVsJ1Jg== Received: from BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) by IA1PR12MB7589.namprd12.prod.outlook.com (2603:10b6:208:42b::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8230.12; Thu, 5 Dec 2024 00:18:44 +0000 Received: from BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9]) by BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9%6]) with mapi id 15.20.8230.010; Thu, 5 Dec 2024 00:18:44 +0000 From: Zi Yan To: linux-mm@kvack.org, "Kirill A . Shutemov" , "Matthew Wilcox (Oracle)" Cc: Ryan Roberts , Hugh Dickins , David Hildenbrand , Yang Shi , Miaohe Lin , Kefeng Wang , Yu Zhao , John Hubbard , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH RESEND v3 1/9] mm/huge_memory: add two new (not yet used) functions for folio_split() Date: Wed, 4 Dec 2024 19:18:31 -0500 Message-ID: <20241205001839.2582020-2-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241205001839.2582020-1-ziy@nvidia.com> References: <20241205001839.2582020-1-ziy@nvidia.com> X-ClientProxiedBy: BN0PR04CA0063.namprd04.prod.outlook.com (2603:10b6:408:ea::8) To BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL4PR12MB9478:EE_|IA1PR12MB7589:EE_ X-MS-Office365-Filtering-Correlation-Id: 36185e56-4f05-46a8-1c71-08dd14c26166 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|7416014|1800799024; X-Microsoft-Antispam-Message-Info: s6pyskzxu24QNu2sQsXfCShkcCx3pS2RuJVk3nURn2Ua+DCxBOG7gLcbkmJ3ht6bDhe6fBo7M8vipL3IZaqbAH2q+T37nil6W2vKTs8zHHQvX9H/CrRufVr6J7aNd97wX8jKW49xy3A9CVleOS7EOfJNfZxUlEzGWGIrEnIBgvjQYUu8/fHarRdK9MZRb9MThCEgw5pNoONcXVywvXQQUklFL6aLynUa/914gZku35cf32OypWArV5y68SHWARSDuEE2/r0cMP9Wxfpx8RRCo2M3bglQ1niKQOMrCqF0sfGnP+GV1e2R4G9/mGStLKPZiZGP0E6v68ozMMR00YqVi8mHGEMPMCZBryfi9zbZi4KpwTOTrjbuy2DjCdugpMvjwO3emfqlQhHceh3+Hbork5DqgcITsjHoxHG+63JDEycEGRkDAK3AWgRFbOb00q6ElFVu0EQrmb1qC3UzQcqBIU58A1QKfXx+gVkDdB0ayF1bjOLS1RMp1tzLLA7276eSv62ZvRW+OLu6Gg42FHGzgQMVBPsVw1z3zQk2uaqtFR3cW/GACyjB3EPRYVX/ouubEuzDGEGYrnnPl2nH1uq3BNuYxJmBEP73AHXtMuOmu1IaBAnq9YXBaogC/x0tiUOacxo+CJs22bZTNCGphBJWq84g0LsF7QUK91kQkdqoBkh3J/QuxSAo7bhvYj+DYSEQ51SVIfl8u4O4q428qaZ6Co0YJGyl1GeOM9hmyTaR4aR+atJxZWFkbOFK1WEZ0eMQwltpZyrv7/OfrrNj/DR5oHAwAMmKG37Eau2ooU7FrqWp1PF/7xS4xl/jXSaM35oaCxOhaLPMWN6cFmWhNTUvEthvyqw8ZXgK0I8BnUmp/KM/MkS3kzt9VloBW3ukTBLK/Z0PFhzadc6zMrKvTYzyALkiPDnsWA6e3Aeiosm/73NZ6aQpMHMwCjAE9RSzsTwqNQ88NT4ET8D3j6gQwNIb6xWhEL/mwEwQN+hKF0UlS2I5BxlCOCxlFqI4CV0HT79HNBg7QSpbTGtB/mYApvWuLH7voO8JQj+PF4NUjPgC+uzdgI1YoylDIef8zQZdZlzA2jO/1U/netmLt5Hj48QwjB8z5tGxhgR7cI5yVe8L5ZJrA56d9d0tysYWZ49yI4yQp0UnhC8R6yDnz+fngVfR/leLWt+9rPAvr/aY6gyrsiXlhNsOq+KV6adWm//IM3juH6xG+Dvr1nzPkYKcD3wqAttXNma0/m/2rQvPbfjQ4wTptzQhL9DQZY3qByab1zbaIMDKMPzC2bvPU0ogVC1iwyfHzyfp1ZokBMuah35XTeS7Q874fmojPXLrJ+0e1urwzZNGSHpWlWxx6ZrlB+Sr5g== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL4PR12MB9478.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(7416014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: nir4wRO24tGE7X/Ms5NgF1e9GpVg8wmwtl96KRQbztBELwfPphRJkmiAZjU7deN//wDuthZDVudBbtGJ4aj3lIygL4e7HF3ZLbhu2ZwB9r2sfo6tYDjY2sAFUKxofefjj5Hy3MbpDqnSo4i1boK/EAAWPyQwLy8ZxSkOatKw7eWZvhIeWcAWkTBhKCaXmzamnVJ5SOO7+2fmPmMznRGb17lSjyKHtHDGe9sgdpN8a49Pwqpv4R3OUSHHZDnyX9SKQa5LLKFXVeG/WAAVRET13TinRPzgSoZLuxHfRPGnr/wcGyveVGwzFpWp2D5gTiwA7Sosqgk/11f/AVIRdxAqriWd1kV63qxzs4QQdyh/5owxstvpV/KXAKuzRKmSeXk7hrKxse+fTqm5R7zldOXhjWu7pSr/pfkemQqv3PwHxuq1HtIL9lU0dGWnClvh7hn/V14URkwGgsAeUxsDntjpgMmoR4dHVj/dTAcfAgH9EB1gBdChDbOXLkl3Wn1D2DOmnsrP8OtVK19etEjrCyiWlOGGfhNktRDqbX55JCvytRLMKrTz/kxUwAqY8/F1pufJ5g8DLfvBQGo/2O+wkaGADpbPmqtgNeYE56vdOorViLDn7RezCJKKn+MSDzIRX/c08ooeIBoSyJAbnPSQOllgmaHiGKGzNY9GRZMeBCVG5EQP9qPtzcvjDRJ5GTzchLJdNBIDjuVbSCo8D5SVVr762qZV0c4EkCtLdI3cZ3E/avBEw2wTbedwZqE5OHjUIAgLBDh//YClR8Emgq6S+ofnBKfUIuPdUrJ4Hn8bq31/sMtCm1s5GhZJnMpmPHkuYoHbC4GHyCpNjwem9htEf+rzQEKc1Yisejy7KAEi8jlEGfo9ztscMWsGgrYh6Nyfd+MHjrqo5MCAYrG7BktlJa+Yxm/hpJlgJcV94htKx15K6pukV23DYQpCKdqRgY4qdvr8QH35A1DphRstbjVTSk3c9SHFEeAc7MnuYGs+WSY9o+hkJTrvG9HfUzQ9fq4ivEWCI7sLnEIZZOjl2PMVUDyxPFmdtZmrbdu1hzTVZ6S2twP0aECnhqUDtSmnFGEf4TOyb/Q1ocB8k4GsRZxvZA88hhExxJxyXDyz/wntQ740p5qQ0PVHLZagZjlUJQQsIDjDBsvqRoURrkwSIhWcFCNNQhoKAjKR4VI0soQYyEPj+PmDzeeeP5vupwrhEAEfNdOJ2tBj1aixHXliHXXHABLJ6KZJRQn6K/eX7wyldsPO2/FV1wqu8NvoAoPfRU5G6EcQG1Njbd1CNuUYk07os30astJUuan5I5/D5mPcaSfsJupzHjtr5w6Ej7QwEyD6ZjY+CjUIgdIbCaGVBXZNwW1pAG9daW979I53AfDkmGns2RsNTRWA6L8VJyawW2I/Y6xg85ao6LryK8rTZ4Jp1icsgbU0YdQKvXMiRoWpMNDQyqdyxDzVI+Xd8liBDbMeWZA1WYBJokxuZXgZx+l/2lcIu7EZWtKjArEOCK3RWJ3WK7xqVLm7Dp/9igNDHQgDnePjjnTuvmJhcqd67HkyJDhORc/ltchG3eEbHlCOyoFPY+3tYJ7cOh8heJOPC4an3Iy/ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 36185e56-4f05-46a8-1c71-08dd14c26166 X-MS-Exchange-CrossTenant-AuthSource: BL4PR12MB9478.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2024 00:18:44.7556 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GloXxXsUr9TmYrGIEXCwtbGIIAT9yudJcykIsFMLm9lnEise6T/iQOEaWfTwuLrm X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7589 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: EA70B1C0010 X-Stat-Signature: rkq1wbfbum71sqpc4gi1i3npdo9t8hzf X-Rspam-User: X-HE-Tag: 1733358017-695128 X-HE-Meta: U2FsdGVkX1/3tuYEwh7qTuzgxTfrNZ7ueWiTwHtpGl6zuhH34M5URWva7K7DYrT+DwF6YK26j6+PAfRdA0+16nj/0rg0Q/3h48FlSOM6f/slNykPs+yLKv6gAaS0dagl6adZAdCH+wKzXCBkxtLdTQ/yIhUiyfWUd3NRAOIaBYEhaEXfcyYlP+b1RryG6mdo8BBHd6N8jzCuEueBJwsZL8nIyekg+mi+H3+25MJTTHR7WeK4a3fSJ/uwsebwZRBg6aPVdjATJosaqqL00WYeFYaR/N1aPLBUPjTy4q44mkQBFtJWxxJpsSiXXQyoOanr/kvon6pw9xvQOM05roqURo42kVPaeJVFH2psRUSKaURqKns5ROivVOXhoB8IbJnox6BchTs1Hx8sRTZWLAozjnWn7NDTyQFQYY1qPgaOQT2lRNkfWpXHD9mpynuLJPo0f/afe1lbUP0Mn9JzY8MZbOAHJar6P3gYvQHydSXmIDcbl49cSxdakG60fGvOu51PZkoSZDug56eQfMqkpMtL+UBEQigou3XftPFZz745AxDSlBIq8bITRrPrMex/h+/NXrR4eOB8RMeViHbBnaq0UvOiT5ECPbAlznCNuDMoq/ZvvyfG5DKAPiOR6Drh1xfKEHmvDoZGBERaCOLhC928y5UJCXS4m7J4SWGvkSDpxDWaozvRoyNJobUotoFngHgAsQdraurgmVw2T4gIITob3MJA0L1WsIuURhW+N6lx4D8tlTBzeorkqwHHhXyCWfZGaOd+XqtsLNcY8edxbGv6gDEwc4BujNijL59Q4hcED8jzgGCNYys/og9710x0dJNhUvSXLrjNSMU1Fb/XkcLiL1XUKXe+Zrngncb6rCd6dbWawm0d6R6iHKT+qA2Q7D8eTjlqyD6nelNWu9jsH5w7t+H5zjfzv+m6RuNjdbmvUHH+r0K0V33DQXN/HanuCliEL+1C6KiMIyuJ3WXArVj ifeeQEuS yYZeLRc1pPMUtqJwnAmSbWOKiC9Sakf1M/riUpfYeAsO6Vb53sTyNwmhnfrcJrIKHb+D5lIaNsCKjc7/sHbYlf7L0qvPbgsaCs1rbksZwN3KTXtDY+QE1axpToZH8aYARjGVeLu8dkcaoGJD6dFvuac7Bs9CweNZ27z7KgPGTCYTILsPBz3ilByJMUy1mnFM7n3AANOuPY1xJ2po9cCQ+GieKgwMpM9mAu0B/avL1+/adm4FHTImD5u8VRLbsD+DAlE6aYRDU3o8ePGG1+NBSfKES6dz/TfXsMS0KiAPOjm0j+MyJey8okgPT+ySCaQU36gKLIAExN02jggwT+8riwMa1idSQXwTyRDWcB83rc670OfoogajaOnruGMe68ytL4dMoXhIB0m2ryyI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is a preparation patch, both added functions are not used yet. The added __split_unmapped_folio() is able to split a folio with its mapping removed in two manners: 1) uniform split (the existing way), and 2) buddy allocator like split. The added __split_folio_to_order() can split a folio into any lower order. For uniform split, __split_unmapped_folio() calls it once to split the given folio to the new order. For buddy allocator split, __split_unmapped_folio() calls it (folio_order - new_order) times and each time splits the folio containing the given page to one lower order. Signed-off-by: Zi Yan --- mm/huge_memory.c | 361 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 360 insertions(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index ab46ef718b44..0cde13286bb0 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3127,7 +3127,6 @@ static void remap_page(struct folio *folio, unsigned long nr, int flags) static void lru_add_page_tail(struct folio *folio, struct page *tail, struct lruvec *lruvec, struct list_head *list) { - VM_BUG_ON_FOLIO(!folio_test_large(folio), folio); VM_BUG_ON_FOLIO(PageLRU(tail), folio); lockdep_assert_held(&lruvec->lru_lock); @@ -3371,6 +3370,366 @@ bool can_split_folio(struct folio *folio, int caller_pins, int *pextra_pins) caller_pins; } +static long page_in_folio_offset(struct page *page, struct folio *folio) +{ + long nr_pages = folio_nr_pages(folio); + unsigned long pages_pfn = page_to_pfn(page); + unsigned long folios_pfn = folio_pfn(folio); + + if (pages_pfn >= folios_pfn && pages_pfn < (folios_pfn + nr_pages)) + return pages_pfn - folios_pfn; + + return -EINVAL; +} + +/* + * It splits @folio into @new_order folios and copies the @folio metadata to + * all the resulting folios. + */ +static int __split_folio_to_order(struct folio *folio, int new_order) +{ + int curr_order = folio_order(folio); + long nr_pages = folio_nr_pages(folio); + long new_nr_pages = 1 << new_order; + long index; + + if (curr_order <= new_order) + return -EINVAL; + + /* + * Skip the first new_nr_pages, since the new folio from them have all + * the flags from the original folio. + */ + for (index = new_nr_pages; index < nr_pages; index += new_nr_pages) { + struct page *head = &folio->page; + struct page *new_head = head + index; + + /* + * Careful: new_folio is not a "real" folio before we cleared PageTail. + * Don't pass it around before clear_compound_head(). + */ + struct folio *new_folio = (struct folio *)new_head; + + VM_BUG_ON_PAGE(atomic_read(&new_head->_mapcount) != -1, new_head); + + /* + * Clone page flags before unfreezing refcount. + * + * After successful get_page_unless_zero() might follow flags change, + * for example lock_page() which set PG_waiters. + * + * Note that for mapped sub-pages of an anonymous THP, + * PG_anon_exclusive has been cleared in unmap_folio() and is stored in + * the migration entry instead from where remap_page() will restore it. + * We can still have PG_anon_exclusive set on effectively unmapped and + * unreferenced sub-pages of an anonymous THP: we can simply drop + * PG_anon_exclusive (-> PG_mappedtodisk) for these here. + */ + new_head->flags &= ~PAGE_FLAGS_CHECK_AT_PREP; + new_head->flags |= (head->flags & + ((1L << PG_referenced) | + (1L << PG_swapbacked) | + (1L << PG_swapcache) | + (1L << PG_mlocked) | + (1L << PG_uptodate) | + (1L << PG_active) | + (1L << PG_workingset) | + (1L << PG_locked) | + (1L << PG_unevictable) | +#ifdef CONFIG_ARCH_USES_PG_ARCH_2 + (1L << PG_arch_2) | +#endif +#ifdef CONFIG_ARCH_USES_PG_ARCH_3 + (1L << PG_arch_3) | +#endif + (1L << PG_dirty) | + LRU_GEN_MASK | LRU_REFS_MASK)); + + /* ->mapping in first and second tail page is replaced by other uses */ + VM_BUG_ON_PAGE(new_nr_pages > 2 && new_head->mapping != TAIL_MAPPING, + new_head); + new_head->mapping = head->mapping; + new_head->index = head->index + index; + + /* + * page->private should not be set in tail pages. Fix up and warn once + * if private is unexpectedly set. + */ + if (unlikely(new_head->private)) { + VM_WARN_ON_ONCE_PAGE(true, new_head); + new_head->private = 0; + } + + if (folio_test_swapcache(folio)) + new_folio->swap.val = folio->swap.val + index; + + /* Page flags must be visible before we make the page non-compound. */ + smp_wmb(); + + /* + * Clear PageTail before unfreezing page refcount. + * + * After successful get_page_unless_zero() might follow put_page() + * which needs correct compound_head(). + */ + clear_compound_head(new_head); + if (new_order) { + prep_compound_page(new_head, new_order); + folio_set_large_rmappable(new_folio); + + folio_set_order(folio, new_order); + } + + if (folio_test_young(folio)) + folio_set_young(new_folio); + if (folio_test_idle(folio)) + folio_set_idle(new_folio); + + folio_xchg_last_cpupid(new_folio, folio_last_cpupid(folio)); + } + + if (!new_order) + ClearPageCompound(&folio->page); + + return 0; +} + +/* + * It splits an unmapped @folio to lower order smaller folios in two ways. + * @folio: the to-be-split folio + * @new_order: the smallest order of the after split folios (since buddy + * allocator like split generates folios with orders from @folio's + * order - 1 to new_order). + * @page: in buddy allocator like split, the folio containing @page will be + * split until its order becomes @new_order. + * @list: the after split folios will be added to @list if it is not NULL, + * otherwise to LRU lists. + * @end: the end of the file @folio maps to. -1 if @folio is anonymous memory. + * @xas: xa_state pointing to folio->mapping->i_pages and locked by caller + * @mapping: @folio->mapping + * @uniform_split: if the split is uniform or not (buddy allocator like split) + * + * + * 1. uniform split: the given @folio into multiple @new_order small folios, + * where all small folios have the same order. This is done when + * uniform_split is true. + * 2. buddy allocator like split: the given @folio is split into half and one + * of the half (containing the given page) is split into half until the + * given @page's order becomes @new_order. This is done when uniform_split is + * false. + * + * The high level flow for these two methods are: + * 1. uniform split: a single __split_folio_to_order() is called to split the + * @folio into @new_order, then we traverse all the resulting folios one by + * one in PFN ascending order and perform stats, unfreeze, adding to list, + * and file mapping index operations. + * 2. buddy allocator like split: in general, folio_order - @new_order calls to + * __split_folio_to_order() are called in the for loop to split the @folio + * to one lower order at a time. The resulting small folios are processed + * like what is done during the traversal in 1, except the one containing + * @page, which is split in next for loop. + * + * After splitting, the caller's folio reference will be transferred to the + * folio containing @page. The other folios may be freed if they are not mapped. + * + * In terms of locking, after splitting, + * 1. uniform split leaves @page (or the folio contains it) locked; + * 2. buddy allocator like split leaves @folio locked. + * + * + * For !uniform_split, when -ENOMEM is returned, the original folio might be + * split. The caller needs to check the input folio. + */ +static int __split_unmapped_folio(struct folio *folio, int new_order, + struct page *page, struct list_head *list, pgoff_t end, + struct xa_state *xas, struct address_space *mapping, + bool uniform_split) +{ + struct lruvec *lruvec; + struct address_space *swap_cache = NULL; + struct folio *origin_folio = folio; + struct folio *next_folio = folio_next(folio); + struct folio *new_folio; + struct folio *next; + int order = folio_order(folio); + int split_order; + int start_order = uniform_split ? new_order : order - 1; + int nr_dropped = 0; + int ret = 0; + bool stop_split = false; + + if (folio_test_anon(folio) && folio_test_swapcache(folio)) { + /* a swapcache folio can only be uniformly split to order-0 */ + if (!uniform_split || new_order != 0) + return -EINVAL; + + swap_cache = swap_address_space(folio->swap); + xa_lock(&swap_cache->i_pages); + } + + if (folio_test_anon(folio)) + mod_mthp_stat(order, MTHP_STAT_NR_ANON, -1); + + /* lock lru list/PageCompound, ref frozen by page_ref_freeze */ + lruvec = folio_lruvec_lock(folio); + + /* + * split to new_order one order at a time. For uniform split, + * folio is split to new_order directly. + */ + for (split_order = start_order; + split_order >= new_order && !stop_split; + split_order--) { + int old_order = folio_order(folio); + struct folio *release; + struct folio *end_folio = folio_next(folio); + int status; + + /* order-1 anonymous folio is not supported */ + if (folio_test_anon(folio) && split_order == 1) + continue; + if (uniform_split && split_order != new_order) + continue; + + if (mapping) { + /* + * uniform split has xas_split_alloc() called before + * irq is disabled, since xas_nomem() might not be + * able to allocate enough memory. + */ + if (uniform_split) + xas_split(xas, folio, old_order); + else { + xas_set_order(xas, folio->index, split_order); + xas_set_err(xas, -ENOMEM); + if (xas_nomem(xas, GFP_NOWAIT)) + xas_split(xas, folio, old_order); + else { + stop_split = true; + ret = -ENOMEM; + goto after_split; + } + } + } + + /* complete memcg works before add pages to LRU */ + split_page_memcg(&folio->page, old_order, split_order); + split_page_owner(&folio->page, old_order, split_order); + pgalloc_tag_split(folio, old_order, split_order); + + status = __split_folio_to_order(folio, split_order); + + if (status < 0) { + stop_split = true; + ret = -EINVAL; + } + +after_split: + /* + * Iterate through after-split folios and perform related + * operations. But in buddy allocator like split, the folio + * containing the specified page is skipped until its order + * is new_order, since the folio will be worked on in next + * iteration. + */ + for (release = folio, next = folio_next(folio); + release != end_folio; + release = next, next = folio_next(next)) { + /* + * for buddy allocator like split, the folio containing + * page will be split next and should not be released, + * until the folio's order is new_order or stop_split + * is set to true by the above xas_split() failure. + */ + if (page_in_folio_offset(page, release) >= 0) { + folio = release; + if (split_order != new_order && !stop_split) + continue; + } + if (folio_test_anon(release)) { + mod_mthp_stat(folio_order(release), + MTHP_STAT_NR_ANON, 1); + } + + /* + * Unfreeze refcount first. Additional reference from + * page cache. + */ + folio_ref_unfreeze(release, + 1 + ((!folio_test_anon(origin_folio) || + folio_test_swapcache(origin_folio)) ? + folio_nr_pages(release) : 0)); + + if (release != origin_folio) + lru_add_page_tail(origin_folio, &release->page, + lruvec, list); + + /* Some pages can be beyond EOF: drop them from page cache */ + if (release->index >= end) { + if (shmem_mapping(origin_folio->mapping)) + nr_dropped++; + else if (folio_test_clear_dirty(release)) + folio_account_cleaned(release, + inode_to_wb(origin_folio->mapping->host)); + __filemap_remove_folio(release, NULL); + folio_put(release); + } else if (!folio_test_anon(release)) { + __xa_store(&origin_folio->mapping->i_pages, + release->index, &release->page, 0); + } else if (swap_cache) { + __xa_store(&swap_cache->i_pages, + swap_cache_index(release->swap), + &release->page, 0); + } + } + xas_destroy(xas); + } + + unlock_page_lruvec(lruvec); + + if (folio_test_anon(origin_folio)) { + if (folio_test_swapcache(origin_folio)) + xa_unlock(&swap_cache->i_pages); + } else + xa_unlock(&mapping->i_pages); + + /* Caller disabled irqs, so they are still disabled here */ + local_irq_enable(); + + if (nr_dropped) + shmem_uncharge(mapping->host, nr_dropped); + + remap_page(origin_folio, 1 << order, + folio_test_anon(origin_folio) ? + RMP_USE_SHARED_ZEROPAGE : 0); + + /* + * At this point, folio should contain the specified page. + * For uniform split, it is left for caller to unlock. + * For buddy allocator like split, the first after-split folio is left + * for caller to unlock. + */ + for (new_folio = origin_folio, next = folio_next(origin_folio); + new_folio != next_folio; + new_folio = next, next = folio_next(next)) { + if (uniform_split && new_folio == folio) + continue; + if (!uniform_split && new_folio == origin_folio) + continue; + + folio_unlock(new_folio); + /* + * Subpages may be freed if there wasn't any mapping + * like if add_to_swap() is running on a lru page that + * had its mapping zapped. And freeing these pages + * requires taking the lru_lock so we do the put_page + * of the tail pages after the split is complete. + */ + free_page_and_swap_cache(&new_folio->page); + } + return ret; +} + /* * This function splits a large folio into smaller folios of order @new_order. * @page can point to any page of the large folio to split. The split operation From patchwork Thu Dec 5 00:18:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13894602 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 990C2E7716D for ; Thu, 5 Dec 2024 00:18:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 32B396B0089; Wed, 4 Dec 2024 19:18:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2DAC36B008A; Wed, 4 Dec 2024 19:18:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 156536B008C; Wed, 4 Dec 2024 19:18:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id EA79E6B0089 for ; Wed, 4 Dec 2024 19:18:57 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A81D4A12C8 for ; Thu, 5 Dec 2024 00:18:57 +0000 (UTC) X-FDA: 82858994022.20.456CC7F Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2069.outbound.protection.outlook.com [40.107.94.69]) by imf01.hostedemail.com (Postfix) with ESMTP id 68A8A40012 for ; Thu, 5 Dec 2024 00:18:43 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="Arg/tCuF"; spf=pass (imf01.hostedemail.com: domain of ziy@nvidia.com designates 40.107.94.69 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733357918; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=o3FI5qHRFlXzJea8dxSObFOP35mYmvtNR3JUb0l4oa0=; b=PTcb4mpb8uqMgy/FVAOAhWoluAYIeeKTiRO4AWEv05iPlZVVMhuGOR7tbF7ZARi8CHT2Oq R3fsIfFTcvnxGtycjIMLjO9755KHuiGgUKl3HXT9aNepV36g89V+DBwIJDrSzdZKKiHbfF qujz/nsrTbd3+vlCloeYQZhFVXYyJpo= ARC-Authentication-Results: i=2; imf01.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="Arg/tCuF"; spf=pass (imf01.hostedemail.com: domain of ziy@nvidia.com designates 40.107.94.69 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1733357918; a=rsa-sha256; cv=pass; b=xdSjMWz2Szpqv8Rso2lKkv40kQF2VOEF2C6TDeA/3QdkJwUKa0aPJdbK5pazP4AQLS3+hi g7tMoW7xBvUH/PFvHkUPl/QR7+Ly6eXZ0NviKyWCo+UAKhhbt8KHUW39knk4W/QwN/OKlf 4h9WdI0MCzU1HoJYBhXp/8zesHapdgc= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=WYAxkO5X1ZpCnVGef2IYXN355Vc51xdVwFcK2UlggAckh9Nlc979K3crexg9mhP44hl5SZMlitAg+vJFIZwIUn3tC92EIdoZWHMiMSvmQmfMq9esw2vFrgJd0geKWlU9Z/GU7WwcqP0vA9w9NVm3r3cFIPC93dIHMcq1OPuM8mTo18HE/+ob1/bjFpdl7+TgCKFNGWUWo/KpIauQClBX9ko6CEqGWszR8KoRNVeExk2IzOfreqdvthRhLYjIq2kxBImGCElbjf4A8HcPP4hKstOp5w9Pjl3ZJ7Y0aQNSNuExtYf+V3MNMzk47lj6Nyr8hWvYxTxpJZNoQHivEuDIfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=o3FI5qHRFlXzJea8dxSObFOP35mYmvtNR3JUb0l4oa0=; b=yOdDw4ZUpPXfPA9z9Bo1QrWKIyLY40csLJKk9wbYTA3VggaIiYxAXd3dPHUSASuXnqvbu6F2h2B5ib/zPVNHT348kaREIU0gOACEGDsNPot89If/7Xzgmio/ZEwI44JvY86ehaNu+3WblNfg6iO9BbsD+yUEKTE9Pcc8BLmFQIXqS4L9DHTx/e9FKfh88uGm2XKdk9GqANoNs5YP1DmcKdvyeFUZtdA3lNF1vYr4vXphpYs5TYg7GvEh3ccJas320T/NdlGZaChEUI2f8oFOqTL/+XjJjBVGMEy7CU8wlAngQ2WPeP1FB4pRkNd5NNIHNin0hlyjhsbArK0PaBVV4w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=o3FI5qHRFlXzJea8dxSObFOP35mYmvtNR3JUb0l4oa0=; b=Arg/tCuFbFCM8LjwI1hy42LgmlnpJC96nx+vYnN/SqZ8WtMrJ5uEJvVAnrwWFa/Qz7s+gtTDBWh9JTU6cg1M6lBdJLLdS+AE2//aBR0RX4i+XSwgqs8BQyM3E+d4yF87IMvr7DRqlT15PtGak04ClyXsnYYPOak2evBZCBeCVJiY2lj19HsBLN08AiM2vupbqmqomHjj4sJzPpZdEwnVyVR9owQxgIwFVmLXHfTolPJGQpSn2jJdlAHKG4us7g/Q/WlYg+yhFv8SD4J0YsNhRiNtpgYMxEDYNBkE57KnXvcTI7aSTAQnz7oi5jHX/a5pfdBphJ1SBfkrqjWCVUF4fQ== Received: from BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) by IA1PR12MB7589.namprd12.prod.outlook.com (2603:10b6:208:42b::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8230.12; Thu, 5 Dec 2024 00:18:45 +0000 Received: from BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9]) by BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9%6]) with mapi id 15.20.8230.010; Thu, 5 Dec 2024 00:18:45 +0000 From: Zi Yan To: linux-mm@kvack.org, "Kirill A . Shutemov" , "Matthew Wilcox (Oracle)" Cc: Ryan Roberts , Hugh Dickins , David Hildenbrand , Yang Shi , Miaohe Lin , Kefeng Wang , Yu Zhao , John Hubbard , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH RESEND v3 2/9] mm/huge_memory: move folio split common code to __folio_split() Date: Wed, 4 Dec 2024 19:18:32 -0500 Message-ID: <20241205001839.2582020-3-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241205001839.2582020-1-ziy@nvidia.com> References: <20241205001839.2582020-1-ziy@nvidia.com> X-ClientProxiedBy: BN0PR04CA0088.namprd04.prod.outlook.com (2603:10b6:408:ea::33) To BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL4PR12MB9478:EE_|IA1PR12MB7589:EE_ X-MS-Office365-Filtering-Correlation-Id: a2f1fb4d-54d1-4876-80c8-08dd14c26200 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|7416014|1800799024; X-Microsoft-Antispam-Message-Info: 1QolBZRmmAEHgGrtorOG8B6yOOdrb/VPCJazNbulkZaYXmPCGu94LXY+R+CwVWOCkJJ1JjMEm2ub4jbEZYe9y+2e6uxlIs/k5pytUl8Dg1P9chelq0zC+tAlJ9+iq4PTQ/9qNQ1KFGylwL+Vuu55aqwuQn0GANb2XxkyU6o6car7QQhGI+0P7tfj1D4ky6SqZf14JAis0KIvSUz88uggHjmi7qSYGA5mBsA/iP603T8J8MwAAzL3Lk2hMZ89hhoo9g5JVqCcTKva+NI5XCey6hM75ya+Z+uLK4CTpG586iHMAIWXfLS4bZbhkHzwf0FxZ82bZ0ovBRYyPhA8ETf5XDiEkNc6HFvcvjP24SMfbuzJ77Ri+u5O06dtSyROvA5D0Ag1Sppwv4OqguWle+d7cUeKS+6MpqFNNaBq5qFkv7f13ljFbIBYT+kR3HitYcN5r2LzsqkIZi0RdPgJD0zmDcl46W8sb2N+5iM84+oJdPIqB71dvtUj+DnmJkv1GtEpu83wDoveTQBoH8zqUL78atE+WstgRhVz45ha7J0spEMQWg4uRyUmnpBN7wkr2rDS2ZSBaPO2DGfP0ITgx46/3Zz9KKFUWpQ+UwPFquOjw6vgOvDDa1bJBuYzwIRVS0X27gZwfPeOnOaxP+WHeJ+7QHDVQ3GIxk5m8kMwveyF3FAg8r9jFbqKPPAdEEhkm2ambpjpXtQU7+GmnMZqR5vsqLiq4v8a2xuMYJfgaVGUlk546b2rDZLcexJoW98gDEZOnyxYQw2E8VOvVaounTr1frDS8H1X1NkFAxrKAplNaPrP+6wWgELJgkzvibZJqjDtnIzed4L5TbMqQPAg29APw9gU3QY2xpw0b8Hws68RuRyhOQrk+8uOaawAzid1sb99BC8lMUepQcdG4cnrBCAFdX8JjXGig36pOYiEiCXQAbymtdbN6JhFprIID6UJgjm9x/SYnp/nWy3WNh/1FhQaZNX3SJOxC9EA4pa2L1/Y7wWbyNUzE8xNAxfIsauAmcF+QqEFRqDc8PyrtOTOydqOAVCwHgWGMc1Pekeqcz9HL5hnIe7V3qgegGeLsCYRuESSOPdvDSZ+8X9RZTcbobYiZrv7ijRWW9Debzeb8v4fd0usP/aHBTokq+o6/m6Mg6DpLMIoKScGO+/BasAL2st7cyXq5G+4TtvjPfyZRD0PxqDuH7NUG6jv1k91qHKWdRmpOdzXQL4pxOPJxdJnKekxwcPFIboI0TLJTiSqmtpwymCC1Y5vutiz1RDOGRA8ZWuDkaKmoI3uAQcIYaCF9JsNfg+gnOe9ML1x9BQ39r0r7EJen4Ba23jdeEWk/aCkD8dYLHRENgC6Sht7/8Wkye2nTg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL4PR12MB9478.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(7416014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 5Jf/1vSd6GO9UXzUD44HeU7v16p5gBh1Dbf0E4ug2pAtGfeZHlPvwGudrDQ/9FkMXpdEzx1d7zONNHtoiFHgJbvYBnpfJSmTpaGP8lNR143rlONjFZVym0L4JOLGbl7tVOkNQf1x61LztEbtOEZDyL5PPOkW0jxRRJzWKOTt4t9RuYFJ+CCWe6vaWvYy1/0wfQ/DOWsXpVZ/43HG2Mw4x+FrAC3GiJZ6y+fDOCEaxGn4sdlh7Lm0DjVsi6Qfc9P9Qv04nXVrfHKwzUysCl8ESJljbzK0xom7YGC5I2a4k+0pwDSZVEqPGFwcpXT0SzF0QC8W3EsE3u3QRPNFf0xpXCRWJR4h8vnqmFP57Z6iahCHjH7hBtWOa3JQoMRrqi94KA99Twpt8QaMlRcBTB+Q7cL+l0Jo9oY1JVzZiEMjEcKl4nKib3aPPg7bf2hzSkaVtK1+xsnWPFM9dum6rpmM6O1a47CDKSL/RTgmqrDPjn9dT2ax/6VW23RYtwkUbFGdkE+/8ZBKzzdrPDML0cVYKPC1K4fJiwRqCxBelRcaE21vnu9n+jr0lm7vGN+8ZS5s8dLhF5p+fnMFP55BEk41YvC9GSmJIcf8ecxBeYQs9lV/hovpJLXv++1ZIFBOnOqwG/mz/dJAut7+matmHpZ85Gt4u3ToUP/2XKxCYlL6EX+jIUCzZMFqR/yFrwh4hRdAv9fALjUm8YRLYm0AcQdxyCg7gJzRkkXehqQa0Ndf7nMGQOy+q9ZKIn6cn8GhDyPq8UMvTbJHpMdZL4dyupZlpGzGnbSVzNnmkMHiSxTxUA/2C0QDWsBxl90l1DAnvMkYzwVQjJhyo09xV1uKbch3ghCueMNFLQj25+Q5QFiCGJvwZqO9JbUcStkk+VX/iUGMSAJja04YL4GatSy8P6cK/sQMdgFQHOvZjr4G6YsHI1ozxBMp0dG4KDcH1z4fUTMqcqI7KCZ96vpcmjDT8k2whBQdb7wad2RUDIkN49l8cRSAf4BBg0lipqsRz5s19XM0Oc9mwQOsbGARZXrTrencS2T+IrWU5GqpBTlkKqMWGjp1UQ3l2HD8RcMXz/wGOwOD2/ZHPJdcycEUbfPjyzMoMH/C9Z/u3bLB4H1BzBtaMVRL6pefZ1R1WzioJpeTnJeKa7xB4y8OYTQdrnSoJwYQvanXXMOtWvjL1r19q72oahj/DLm1bik0g4ZvioCJm3PaUdnjX1CaV16qy6NXZtxX5fNVm3QJuxcYnFkmI8M+pdd1dCOw+fg5XfRJGgcbGQBIMJ2GcCDAMJ1VnHk9Ug11l9nDlZ5M29qtrlvXPEdu9QRI1szOQKRnjKALuG5AqNYB1EZWCzU0mJVh61oYJEF/Ev0xhCytKqry37EjAxdbexMhO+JnZmW4p+hWqMILzOVHeSbh+I2bmrcYjGhHZrvNy8Yc3+7ifR6/up2UFAWwpx//vzizXU6T1CM0/XwaB5DyV7BeDj1VhfF47lbwWRCyOuLsx0aiWK6HTL5Dr2zq8zKTBX8LqhTqNGe0Y6jDe3o8Q/NjozCGcy3riRXxPim+XBdNd1a9eYH2v6e3yxug4LCEsTBZcbR8LNsDIFfnsVhE X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: a2f1fb4d-54d1-4876-80c8-08dd14c26200 X-MS-Exchange-CrossTenant-AuthSource: BL4PR12MB9478.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2024 00:18:45.7880 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: dIhU24BY9d6ANMjCa8XUQ9wvRdxYxMQ/vVBeV6i3VAKXVLyk6k+Ts555wfi0eDzh X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7589 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 68A8A40012 X-Rspam-User: X-Stat-Signature: ztk3oe7cwiqu3rhews5ocg5jsgm1uszk X-HE-Tag: 1733357923-328291 X-HE-Meta: U2FsdGVkX1/qwW8wVr7iRP6lemnUqpyExvU2WkV+AbsWpTNaRowQi0jQssPzObVI0m4RAVRLL5tYIvBWblaAeUFatBVDxYfkwRU8C0dQBe52GmFbIiP0r3XwV0u+qMobxB7thUikKc9jZj3W5PV++mp4hiD3Qe+sR3KlwipBCZsD64kADrnaRBiyq9V+GHivYSydTrUNpkl+IxXGjlNTQKVJTpUrKOsBp/85ft5cTVJhht3ARURZW5hiImhLi/RzCWajDlTn0/M7aBCRXwnTQuZZQgpNpEUUVW9zrchu64QG6RW7nS6m60WyNL1WGXW68KFrTAo0rsXo/ImIVgOxThKrmz3Zr26sb8FEgyRpz/O1gYB8J8cIkOp3a/RW8kpr+u1UrIHX5m6lSIBEZoK3H/zr3d3O0+hdAXXAMxzYU2+cte3r5CCmsdLJ6UjjQpobUmkaoYDFeozbSFVuI7hyt04h/oI/F5ftcd2GOfH4dD4HqbzURwKtWV7D/KGhbO4YOJ1uF4O5UkU12+IfDypUP3XuI/70KcO5+I/lKkRCIBLssjWOB0v0iMkEAhVVP2brvW5VEXGp6sTRjqPRJR7OqLXWxiauL/KN4p5KvnjK5eh6TZZLq+ORl8n+P8eGOHV6bIr99hgB6nQbeZZJyfKrrvGY2BxRfOflDGeBsOxQaZoLpZ+cU8wWBaIbewgPF+4vN6eyWygO0Tzce9bn/PkPUNexGhgJQtoAMr3Vor8LI9T9C6JyBGzkXZixZZCD3M1cM6kXVIKXdP337hPWaVDXsJqvIsRAKNMnVH1dLxi83egip+OSjHCYL9BtVo6SinpnDusWAwUjOzxPNQkL9YfgOsyVTVWKdiJL0yvTXYPEhq6mAIempIAC5eU6qFLBmb9IU3TnnnhZT40v5urmWNaOEPAz/SE8+OO+LzjXaF1fhhmIouA+JoGhTM9H2s3ZjYNXMUqYE4Dqwn0rlRlzYlT tmCbHDQq aYNLE46ellPy9EkyWzM+wW03Or7cbVxwLFfcHhBvyn5oHd2KRS/opj21j5/RHej4rLcriw9yJJVfS1naYl57eP6fLjHlvOJK+CRgV/yIlhE/1qdxs/Fzcr1a0VuamBabviYjyl6zMWtQgxBoIGANyulh9ztt3YaXP3z4+aJh7+k9SPGvD5Aq9vlfbYc848P0aEfi2AFXqN8YEkf0dhTgG6MaVSf8ZSy4TCri6+D4QUC+L6PB6ijKn7tjn31MJc/S9pH0cwsvdtUzpL/yaTlhlAhDDfD3nn/UyfUHdnL8L2IJmOana4MMkA8rTsREC/0ZRBfJXVvGUeRulH+uhPiRcbmdv/KVfKccKooWloYupvFsrYr0KQ4HsyU0FHtDFolMAEe3jc4IeEKpjjHU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is a preparation patch for folio_split(). In the upcoming patch folio_split() will share folio unmapping and remapping code with split_huge_page_to_list_to_order(), so move the code to a common function __folio_split() first. Signed-off-by: Zi Yan --- mm/huge_memory.c | 107 +++++++++++++++++++++++++---------------------- 1 file changed, 57 insertions(+), 50 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 0cde13286bb0..e928082be3b2 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3730,57 +3730,9 @@ static int __split_unmapped_folio(struct folio *folio, int new_order, return ret; } -/* - * This function splits a large folio into smaller folios of order @new_order. - * @page can point to any page of the large folio to split. The split operation - * does not change the position of @page. - * - * Prerequisites: - * - * 1) The caller must hold a reference on the @page's owning folio, also known - * as the large folio. - * - * 2) The large folio must be locked. - * - * 3) The folio must not be pinned. Any unexpected folio references, including - * GUP pins, will result in the folio not getting split; instead, the caller - * will receive an -EAGAIN. - * - * 4) @new_order > 1, usually. Splitting to order-1 anonymous folios is not - * supported for non-file-backed folios, because folio->_deferred_list, which - * is used by partially mapped folios, is stored in subpage 2, but an order-1 - * folio only has subpages 0 and 1. File-backed order-1 folios are supported, - * since they do not use _deferred_list. - * - * After splitting, the caller's folio reference will be transferred to @page, - * resulting in a raised refcount of @page after this call. The other pages may - * be freed if they are not mapped. - * - * If @list is null, tail pages will be added to LRU list, otherwise, to @list. - * - * Pages in @new_order will inherit the mapping, flags, and so on from the - * huge page. - * - * Returns 0 if the huge page was split successfully. - * - * Returns -EAGAIN if the folio has unexpected reference (e.g., GUP) or if - * the folio was concurrently removed from the page cache. - * - * Returns -EBUSY when trying to split the huge zeropage, if the folio is - * under writeback, if fs-specific folio metadata cannot currently be - * released, or if some unexpected race happened (e.g., anon VMA disappeared, - * truncation). - * - * Callers should ensure that the order respects the address space mapping - * min-order if one is set for non-anonymous folios. - * - * Returns -EINVAL when trying to split to an order that is incompatible - * with the folio. Splitting to order 0 is compatible with all folios. - */ -int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, - unsigned int new_order) +static int __folio_split(struct folio *folio, unsigned int new_order, + struct page *page, struct list_head *list) { - struct folio *folio = page_folio(page); struct deferred_split *ds_queue = get_deferred_split_queue(folio); /* reset xarray order to new order after split */ XA_STATE_ORDER(xas, &folio->mapping->i_pages, folio->index, new_order); @@ -3996,6 +3948,61 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, return ret; } +/* + * This function splits a large folio into smaller folios of order @new_order. + * @page can point to any page of the large folio to split. The split operation + * does not change the position of @page. + * + * Prerequisites: + * + * 1) The caller must hold a reference on the @page's owning folio, also known + * as the large folio. + * + * 2) The large folio must be locked. + * + * 3) The folio must not be pinned. Any unexpected folio references, including + * GUP pins, will result in the folio not getting split; instead, the caller + * will receive an -EAGAIN. + * + * 4) @new_order > 1, usually. Splitting to order-1 anonymous folios is not + * supported for non-file-backed folios, because folio->_deferred_list, which + * is used by partially mapped folios, is stored in subpage 2, but an order-1 + * folio only has subpages 0 and 1. File-backed order-1 folios are supported, + * since they do not use _deferred_list. + * + * After splitting, the caller's folio reference will be transferred to @page, + * resulting in a raised refcount of @page after this call. The other pages may + * be freed if they are not mapped. + * + * If @list is null, tail pages will be added to LRU list, otherwise, to @list. + * + * Pages in @new_order will inherit the mapping, flags, and so on from the + * huge page. + * + * Returns 0 if the huge page was split successfully. + * + * Returns -EAGAIN if the folio has unexpected reference (e.g., GUP) or if + * the folio was concurrently removed from the page cache. + * + * Returns -EBUSY when trying to split the huge zeropage, if the folio is + * under writeback, if fs-specific folio metadata cannot currently be + * released, or if some unexpected race happened (e.g., anon VMA disappeared, + * truncation). + * + * Callers should ensure that the order respects the address space mapping + * min-order if one is set for non-anonymous folios. + * + * Returns -EINVAL when trying to split to an order that is incompatible + * with the folio. Splitting to order 0 is compatible with all folios. + */ +int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, + unsigned int new_order) +{ + struct folio *folio = page_folio(page); + + return __folio_split(folio, new_order, page, list); +} + int min_order_for_split(struct folio *folio) { if (folio_test_anon(folio)) From patchwork Thu Dec 5 00:18:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13894609 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DB16E7716F for ; Thu, 5 Dec 2024 00:21:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 131186B009F; Wed, 4 Dec 2024 19:21:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0E1826B00A0; Wed, 4 Dec 2024 19:21:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E9D866B00A1; Wed, 4 Dec 2024 19:21:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C667F6B009F for ; Wed, 4 Dec 2024 19:21:06 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 88C9AAED0E for ; Thu, 5 Dec 2024 00:21:06 +0000 (UTC) X-FDA: 82858999818.19.8321EB1 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2051.outbound.protection.outlook.com [40.107.243.51]) by imf23.hostedemail.com (Postfix) with ESMTP id B787E140004 for ; Thu, 5 Dec 2024 00:20:53 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=ZUrlaFgw; spf=pass (imf23.hostedemail.com: domain of ziy@nvidia.com designates 40.107.243.51 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733358053; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aecB1bzSv7lUIxU7S6qbIrcsPh+7bGet1pzIbTFfV5s=; b=NC1fZyRvjOlM/Q/IbFjU7m3ywDqQ2CdhFt71kZSCESuXMRH33OCZLSTtAz91bdIjhVpg1R EBRyjO3wF233WUmely61VsraLVC7JYcpKpt7YnbZDbXtLsqm4PAMXkbfH9xtbRB/j1378D aPmJ/SLQ+gvLzh4AkymQoB7EbuSMiIE= ARC-Authentication-Results: i=2; imf23.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=ZUrlaFgw; spf=pass (imf23.hostedemail.com: domain of ziy@nvidia.com designates 40.107.243.51 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1733358053; a=rsa-sha256; cv=pass; b=fRja1IiyDu8szvtk4wRmOFmKTN9ZPmUOdzn4hk8sDtx77+Z6jOCGr2hhNIxAW48Yt3cSlp NxyGT3Na846QYK+V9hb4QpcX0rGGk/wEpGwY3eUnI9ry9oyGqEhIvEZ5TYfnmuzzEas693 Yc/Ee2u8S2DJK/2SzT9yekYQQM84UnY= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=A6jE9GEgRd+teJ/e5Dcuio5DT+TsLXpnJ5UI8Jw3EWgE0HMHz9zlVNjGtsSqNmvO67RkkIlOzl8UQVbZr58WSByEyBtpMnf7bxIlgpNFJJJP9AUtP06p2JHCzFPJUujD7iy7pGnk/IajMf8GBIbjedQoaquPEUT49RUaoO4URNyQeNsU5NPHqFeIvhwwwhviG1NxbRLRp3H1SP97n8MZy9BLivpvucj0NxA91OkF+CHVRBucCJCis74hZuhHXzD1LByA5K4NVIPTeYaNNzaIRT4SOI2Pvd/K5zjj7E1mHFIQOoYOpVXQzIX3VkmCBb251N9JugoSq7mKfd+eqU9p0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=aecB1bzSv7lUIxU7S6qbIrcsPh+7bGet1pzIbTFfV5s=; b=lEdtsd5i+MatIalLwsN7qLi5PhRnvn3CsmmR34c1RnsSeShv5fDpOROiPDGXTpQ+VKn9yF7YJwDT6gaJxvFs/AhdQ2uQn/Wc7OcBT5j2thNvRlMXv08fX/glQ1ovmtANa4U02B99OzhiYuoK19UyRne4ioxEokOp4U6s2FlcPWSrlP0wAM9wgNlFAkgE5wiEj+PKhoMpdDyVsRGvJbQqgvpzqpR3c7FweLBLLdHJ17zyqmeIdcc8M07EP4b/y/xeXgJsV8G2shkYqCIe+WXf0+u8VIN1jcEgpcTt4coYJigvP/cn9F/RXZ5WoLmAa+ts8e4a29XtC6JpfaasdaDgkw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aecB1bzSv7lUIxU7S6qbIrcsPh+7bGet1pzIbTFfV5s=; b=ZUrlaFgwaKRBFg48x+1ITBRlj0TUjkgbDOFooR194CoKjTOWpB764aYlJscZ6/NGWnpdWHW94VChbDMKquWfqJlT5rNWW6O1z+XoB2Sgu8kU8dF4Ob8hfeBEp8g7CQwMQKB+psVKGh168R3OI7YN5+3bRZTqRoye8TITqw2nsqKsXCM4SyARRycQKtHCPzLxxo13OiSea3t2xaN3q/YV5ptkny1V7c/TFrjiEdYWJj1XeSkywJ4s5z1PbhmBjr8213YmveKOZ+eFPIDwGLwEsBmpYNaMW9YSWVscZlbZveTK93B3QFIld9APk9XItwjcpyp6za53rtFI70qMid9pWg== Received: from BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) by IA1PR12MB7589.namprd12.prod.outlook.com (2603:10b6:208:42b::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8230.12; Thu, 5 Dec 2024 00:18:47 +0000 Received: from BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9]) by BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9%6]) with mapi id 15.20.8230.010; Thu, 5 Dec 2024 00:18:47 +0000 From: Zi Yan To: linux-mm@kvack.org, "Kirill A . Shutemov" , "Matthew Wilcox (Oracle)" Cc: Ryan Roberts , Hugh Dickins , David Hildenbrand , Yang Shi , Miaohe Lin , Kefeng Wang , Yu Zhao , John Hubbard , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH RESEND v3 3/9] mm/huge_memory: add buddy allocator like folio_split() Date: Wed, 4 Dec 2024 19:18:33 -0500 Message-ID: <20241205001839.2582020-4-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241205001839.2582020-1-ziy@nvidia.com> References: <20241205001839.2582020-1-ziy@nvidia.com> X-ClientProxiedBy: BN0PR04CA0084.namprd04.prod.outlook.com (2603:10b6:408:ea::29) To BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL4PR12MB9478:EE_|IA1PR12MB7589:EE_ X-MS-Office365-Filtering-Correlation-Id: e77bfd14-1ad6-4963-57a7-08dd14c262be X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|7416014|1800799024; X-Microsoft-Antispam-Message-Info: Z1fvhnqYpg4VtTjRMkmHYsA9KR0tVqTf0HA1/FKxnAzfVZZsp7xQSYLfy84k/wS6CnQeAgpe8GWy7X5ierCuV2hrwvUZYr53MvXBAoqfKHypm1uFzDYvH7cig8Ue7yqBflM2lF0cnN/OyQgLFa8Xx/KnBBq5Kw2Kxozjy6uzY5f+FHGchoWCxo0Vy/RnF5BrdaqGZK7d4sOY96SU+QAn6m33yUbtzZbfx6UwuY1am6NzSiUQhoRlTR3zWyPutEo6epqsiGUfE9BJY+hc+LZ45YsODrZfXolSt+VKIvOoEXhpnl8Y3w2vHFevN3dWU+7DBd2Ep//zT/Qz2mrU1F7jLoS7sj3Mz0Fipeuwg3aUYfP6Jo0zZ1C7rkBJlWDxHFa9GoTmbxNfEniEz1yZw9kN9TGvsiCi0lxpDxJRsxUJM64DY/59Uh+4zPirvFz44Ahg4d3OVuBN6mZjJmrpZXyV8Deqz+9ZpszaXc5l6BzphHBUHgIGtiEWpJLucA8kwTNC4OvGqyk+WHYdfCgjoeJgg2GvNBs0vnP7WAIouu5OTvezWvuenmyAylU3KcR5zosFSR1u6In2FR66kd4LGXMz3stz9drEpaQJQUq9BudvTpJJCWkNaZ6XHm/2fqX3HfcYXTO/9Ablw/bcB0F48Y7CyvDxfD4UqOOQPxe6fnh8o2q0lWXz+s9axieR4D4fcmh5boi1Am3wodZgSXeo6GLyyxga5crfwhyIFtkkCTQ5tNtnOEjg2s67raoQghPxAhyNW7r/2hnQsmOShEtunoBEQrPukHg9f9U3nAIxKOchPnSeTu8npoVa/uFMyxLtbisP2t3xGeYRXRsnf9ZZbfYFq1U4iUW9KbJJ1y8sH011wVWgSvkKFHEWK8b6VwrP5tzZGrtIGgXTUdnzbs0jQCGD0pPFrQlJvgJnsnJrAKPdTct6BpLJuhez8Cc72UMMg85hse+T1RQsdljnY2AjRLNefTelY/GADOGmC6yf4nLm76lJCclpuXWMVAjUMuyWWg8EqJMItMpKm/KrNSReOpAdFX9kYIionoAAzT2/HxUSx2jDoCi5yeUkqtyM9Fv3gfUwjlAtP9wtDSvBf1PvsPSRCD5RqimNbPCRQPbR1bm2EHFvAuHxwByAF9HsQ7K4OE0jqS6WD56c9kGWgV+latAiqLQC7UPeDYmp7RkUGO4wnkhWDUFrE8L1xMJqWtZ5tr5E2Wsd/ryVWRmDCPFXfTJCS/bTkdJ72Yjg6mnwsdNMhfcFo+Wt/3SwBNO3YWbzVogok0bufP/XlYEkVA6fCN7bALg6H6HHe+8XkgH8gvd1eujibk2s8eOcIKI7F208PR3GIjEugcBEnllNVZT2aKUK3g== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL4PR12MB9478.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(7416014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: hNDNGPSm8DlF8Li1QB4WiuJ5v0uu++9w0jo+TPELyeHwumBs8WfdkfFSiin9/er4bzpKXVdA67VJsil6QBZo/ox7jiwLScfpM6gA6k8j9uqW78T6Vpj+iuALvH+WR/9ypj26UhuNMDbVW75RVHCu02u77Am9JzKNfKbkk0IWqvYp4Xz+dMuHyv65UW195vxsfoedN5OZ7bvbL33Up91vmscP9MJSzrKDzmjGZ3ImgdDYUh9QrZWDOFzK4+6OE0sBTPxNKRTk+L7a1k1bJw93DsrQ7A2IEc/GTjyeacZzrhJb1DyFWbYBocqDFCT1r7+wtRsGgvbmnH6PgIJxl5fADQd4hmwkYsexunIt0aWj8MRReakepBw3ClsRD/PzmmCCqih7elhZkiV6NKBJeLMkfZ+AHywcZ98LppjgWCFGkQc4cWI4BZpFYFPG1vHTxS0BJqF7Jgol9tgdpw8i/HC6vTigtPHca00xA6DE8ybdYBRh42DAVgFHbIZHLH5sQ2BMnAyeiCFjkOYZ0IcDJwlZgLVux46ym/g92YCekyUntDida2Tm2wlSES7U6Te0hdboTbk6s/Dix5yg4jbH8qVtoQDv90kJVJNSXIZqLHu2ojNtfzZLj+aTOEko/zH0UPPC0KzpObzirdJKXElyB/NbMm0y3RT6C9ynlO28aDj/KzCbbx/7CoKioQn03wkOXVPKDZKcjvJK1IBdg8b8pUuz4+CkGAxMT7CiJWbnEivG3wLh5VZOnGDbWZ9ycUq95pWoEotlti+mU3H0/aQCuNbDdvj/984Khk2o9lNaz1iMQnXyS59ZEzyVUmomf/8GhFx46Cj0ut6Lk/HleC9RE5tWMk76mD8+Ov3oR7+LM/fNUjyzkXVnYWDzyr5XltdBs48z28DHxlPycOUOyYxcfTpF+nF2BqKW4lhKz9QmDlgjE5itZ5UrgpJKxWvilIj1xN1VHzDvHvw6RDQE0FEdvylkrmb1CR9mwz3/Ukhy3hqo7Dxjf8ssyIieqFUALSW5yWCKQYAgrJe0DlahqjYwVxBgdndMxKv+Mgbo2kvSquR99WMfUlm+RxxFfXJLn3M2qgmQFvhLF8zy2DP6XTEX5Xxq2C16IsqCzS7vBX+cvocR4NvJkWbDFf194KaBQWAwoIjrOL3yyKjGqvW6Bo4/V3E7MG/q/Q78fNyzk7FYAs1/FoclLpziMt52hLwWzKR8j8qD/y54jDQy6LXLCIKsENF5sf+Jd0KrB+/hlcr9g8h3L0MSzGJ2bhCOnezqQzF6koKxUv2l0NN4T8LQhlGy4EyDD+IzNEG5+J860XrHG2uq9hw4a6aLspGU99lwIiL7Nok87WaTolW1W7PxhZFr+ctikic4Dm8lTxMGo2xICvqlN2vU+p1PpxfxftrzKmEhG7VgutGMeuhAAErFauRRVSygEZgld1SsdI57soM+C6LFohIZc7m22iiSDwug6xxXqEYjhu+E1WR+K1TO0tM1BucIL2d9uZyvxVWQtX9moZXTw8AJiX22afGvRPyryjb8UYKoKyqmqhM1dvzFttWiXk0qGLcXf2QyoP80jM5GBfq4zCf/NGZkZbiC6zBc7vC+tI36 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e77bfd14-1ad6-4963-57a7-08dd14c262be X-MS-Exchange-CrossTenant-AuthSource: BL4PR12MB9478.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2024 00:18:47.0320 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UHrPxPob2I8r/EODqtdVkCgs1zISpQHa+QwglnI0OSdQH1VdK2eRH6lFovU9tV7s X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7589 X-Rspamd-Server: rspam05 X-Stat-Signature: 3ob1oy76ryk1wr6fai4med7xhjr5n3yu X-Rspamd-Queue-Id: B787E140004 X-Rspam-User: X-HE-Tag: 1733358053-908897 X-HE-Meta: U2FsdGVkX18OjCozfYyChNHAbpOg9c2GhaSTF4MWmoEyauS/ZJOVu6TDXTUzTMxY3VlQJ+ejlncp1j8d5MXGt9sitRwAfgyAvXxhAQ7/PcGnX/rzsv1aWY0WL/Ko7f2qbPk48lvKq66I/RHGqlQqWyQEq4QGxsWnJBGrCPxTGHqRlf14EoVnp/g9NfeV7MODQZJjd1p9DDcY0ThGpfySsbW7+NrP7CDDggR7kSKzW+yGeAsegFa5jzS3Yfvp1dkXPby/HB378Y/VK42FMllFPUYuYBe+hfNDmfvne2X8PZiUzCjgbAVaKnl41JjmPuO/95FdS2kYBr5B/tgthiBjDhuSmhcQun1YJtb2zzotWFhDKNsVBqTKGd9/hxvmsVvegVqKevRViSm7OzfLsQJi0III39KqR7og6il63lz1D7UO3XihWdYMmFU5ig2cxa/iNGvH7Y1O5bTQDHOhkjrZ9iOrcFz8xZzF1LF/lq7cOW28sSpSpq+EuOgcWmyatQaRtwUFe1BSwQcbNepn9oCWgvaWSYldS+Ox755ql7ZJcrGgqaQTZ12QKBhrd2zZna/EOP8pnRK9PErBYHGWdsKb4DVR999ZOPdhuMdnsghkManA6F8uevPp9OGORXBhCK6dIYFxZ4jg2lk7vjy7QYvuzfqxZVXMj62tpCdwdD3MhVbh931AqDuzJVlhvlURpMSghYrQpQWwRvzfkGNRXry3KQyc3CNFmWrtSvmdxy60e1fdor6g0Lnb/v9VnKnTjoegmkL3noP6m4mJT+l1CtoXaRf1LOd9HmM8jWi83W8Hq8aqoylcDqbxwvmnXYZcr23uKdufBEJx/ptZtUsnfBauAQowoORFxxUwhmnnA+kB4FsKC9wYAFv4DY7YsoDFGP0FufGRZLUaUzR/nE2ZgL8G0hf2G0lBvtpC311cHaUdz9S05/vBefRCVUF/rJ86ONwX81/Ggt/WhymirxYo9ie yIekWVNl hpLJl/e6I62MgQJsxSC/L47K81qpH2Gm9oemO5tSxh7CoSYJPRyivHNSFW6oNqKgz1nsdKFrm5xilkMWLSgZEe6+24ad37Cu9IeGnlnphTCsT5u7UKGXC7A88gOL8EQ9JQICdqoV3AIeKa0p2wpiJWlljG4D1RfeNor/+cGcFvlhiGOjC5DxeZH2z2MkZbiX2R1IJB8UJavzwHChkpOgjnPIIASnwZzOctb+PgnvcC0uV+1YbdYokOCwtiJ9PeDEcIvTqqfoAaW9Uv5wSLOp2Lc5jQupF/bzodudDd1PosVTVBjoJf4hBy+MvnFn/oPEHZGQdOsZVK4vy76aIPGvIAC191BynsRnV7qDuz/ta6QQIBYvkHKxwwVe+enqSlPjm4Bqi6KgbISnG5NE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: folio_split() splits a large folio in the same way as buddy allocator splits a large free page for allocation. The purpose is to minimize the number of folios after the split. For example, if user wants to free the 3rd subpage in a order-9 folio, folio_split() will split the order-9 folio as: O-0, O-0, O-0, O-0, O-2, O-3, O-4, O-5, O-6, O-7, O-8 if it is anon O-1, O-0, O-0, O-2, O-3, O-4, O-5, O-6, O-7, O-9 if it is pagecache Since anon folio does not support order-1 yet. It generates fewer folios than existing page split approach, which splits the order-9 to 512 order-0 folios. folio_split() and existing split_huge_page_to_list_to_order() share the folio unmapping and remapping code in __folio_split() and the common backend split code in __split_unmapped_folio() using uniform_split variable to distinguish their operations. Signed-off-by: Zi Yan --- mm/huge_memory.c | 56 +++++++++++++++++++++++++++++++++++------------- 1 file changed, 41 insertions(+), 15 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index e928082be3b2..4f4f3b50c537 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3731,11 +3731,10 @@ static int __split_unmapped_folio(struct folio *folio, int new_order, } static int __folio_split(struct folio *folio, unsigned int new_order, - struct page *page, struct list_head *list) + struct page *page, struct list_head *list, bool uniform_split) { struct deferred_split *ds_queue = get_deferred_split_queue(folio); - /* reset xarray order to new order after split */ - XA_STATE_ORDER(xas, &folio->mapping->i_pages, folio->index, new_order); + XA_STATE(xas, &folio->mapping->i_pages, folio->index); bool is_anon = folio_test_anon(folio); struct address_space *mapping = NULL; struct anon_vma *anon_vma = NULL; @@ -3756,9 +3755,10 @@ static int __folio_split(struct folio *folio, unsigned int new_order, VM_WARN_ONCE(1, "Cannot split to order-1 folio"); return -EINVAL; } - } else if (new_order) { + } else { /* Split shmem folio to non-zero order not supported */ - if (shmem_mapping(folio->mapping)) { + if ((!uniform_split || new_order) && + shmem_mapping(folio->mapping)) { VM_WARN_ONCE(1, "Cannot split shmem folio to non-0 order"); return -EINVAL; @@ -3769,7 +3769,7 @@ static int __folio_split(struct folio *folio, unsigned int new_order, * CONFIG_READ_ONLY_THP_FOR_FS. But in that case, the mapping * does not actually support large folios properly. */ - if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && + if (new_order && IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && !mapping_large_folio_support(folio->mapping)) { VM_WARN_ONCE(1, "Cannot split file folio to non-0 order"); @@ -3778,7 +3778,7 @@ static int __folio_split(struct folio *folio, unsigned int new_order, } /* Only swapping a whole PMD-mapped folio is supported */ - if (folio_test_swapcache(folio) && new_order) + if (folio_test_swapcache(folio) && (!uniform_split || new_order)) return -EINVAL; is_hzp = is_huge_zero_folio(folio); @@ -3835,10 +3835,13 @@ static int __folio_split(struct folio *folio, unsigned int new_order, goto out; } - xas_split_alloc(&xas, folio, folio_order(folio), gfp); - if (xas_error(&xas)) { - ret = xas_error(&xas); - goto out; + if (uniform_split) { + xas_set_order(&xas, folio->index, new_order); + xas_split_alloc(&xas, folio, folio_order(folio), gfp); + if (xas_error(&xas)) { + ret = xas_error(&xas); + goto out; + } } anon_vma = NULL; @@ -3903,7 +3906,6 @@ static int __folio_split(struct folio *folio, unsigned int new_order, if (mapping) { int nr = folio_nr_pages(folio); - xas_split(&xas, folio, folio_order(folio)); if (folio_test_pmd_mappable(folio) && new_order < HPAGE_PMD_ORDER) { if (folio_test_swapbacked(folio)) { @@ -3921,8 +3923,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order, mod_mthp_stat(order, MTHP_STAT_NR_ANON, -1); mod_mthp_stat(new_order, MTHP_STAT_NR_ANON, 1 << (order - new_order)); } - __split_huge_page(page, list, end, new_order); - ret = 0; + ret = __split_unmapped_folio(page_folio(page), new_order, + page, list, end, &xas, mapping, uniform_split); } else { spin_unlock(&ds_queue->split_queue_lock); fail: @@ -4000,7 +4002,31 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, { struct folio *folio = page_folio(page); - return __folio_split(folio, new_order, page, list); + return __folio_split(folio, new_order, page, list, true); +} + +/* + * folio_split: split a folio at offset_in_new_order to a new_order folio + * @folio: folio to split + * @new_order: the order of the new folio + * @page: a page within the new folio + * + * return: 0: successful, <0 failed (if -ENOMEM is returned, @folio might be + * split but not to @new_order, the caller needs to check) + * + * Split a folio at offset_in_new_order to a new_order folio, leave the + * remaining subpages of the original folio as large as possible. For example, + * split an order-9 folio at its third order-3 subpages to an order-3 folio. + * There are 2^6=64 order-3 subpages in an order-9 folio and the result will be + * a set of folios with different order and the new folio is in bracket: + * [order-4, {order-3}, order-3, order-5, order-6, order-7, order-8]. + * + * After split, folio is left locked for caller. + */ +int folio_split(struct folio *folio, unsigned int new_order, + struct page *page, struct list_head *list) +{ + return __folio_split(folio, new_order, page, list, false); } int min_order_for_split(struct folio *folio) From patchwork Thu Dec 5 00:18:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13894603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9013EE7716D for ; Thu, 5 Dec 2024 00:19:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 268616B008C; Wed, 4 Dec 2024 19:19:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 217516B0092; Wed, 4 Dec 2024 19:19:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 06A006B0093; Wed, 4 Dec 2024 19:19:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D68836B008C for ; Wed, 4 Dec 2024 19:19:04 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 4F8B8121278 for ; Thu, 5 Dec 2024 00:19:04 +0000 (UTC) X-FDA: 82858994316.20.BA7ACC1 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2084.outbound.protection.outlook.com [40.107.92.84]) by imf22.hostedemail.com (Postfix) with ESMTP id 3398AC0009 for ; Thu, 5 Dec 2024 00:18:45 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=auf8t8LU; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf22.hostedemail.com: domain of ziy@nvidia.com designates 40.107.92.84 as permitted sender) smtp.mailfrom=ziy@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1733357927; a=rsa-sha256; cv=pass; b=FqaEF+o3hUe6SK6nV/RBDSVXh+qj97bK8YelTPhoVH/sNwFnfpw0VOsFm3dj0oFLXKRpbR bC8hdso8cIiZU4SGxJOtXm9XfzZ3mgXzj2WZZWQos+k1pVZ0G1Xi4RHxQnDz0/kCuk9FPQ 5zUJE8XiyUzFseP20WEnJSzJyVwRK3U= ARC-Authentication-Results: i=2; imf22.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=auf8t8LU; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf22.hostedemail.com: domain of ziy@nvidia.com designates 40.107.92.84 as permitted sender) smtp.mailfrom=ziy@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733357927; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BJYcxHpUZ2kDcL+Ml+XnRav8rWac6pv0uhEIokdawrs=; b=HYU6ch4NxyDqY60/AtUOJLaouzwHBRdVGlhcynGpmkehFIz8WFC0F15gZHU04nZtEb17aH x6/o+kSx0o5V/HVvmWF9uu34T2xklYdy/e3BCYJyLojX6SRdfm33C0xorxYP/A4QsagqfP lGzf7ddInQvVkWTgfPv/uF1XLOWohBU= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=GGB1RlvVMzCZ2ldmwBlA6ppFKr1jriFEYnQDhV9GrZxr91eE0cNQDScWaxlzIfJ9FrotUnIeMVeOh5tmObXpGBQHC9F9pGFJf3j3ms5dxf63gNpLyVBl3D7tYDjJyXqc2gQCtNugM9DH9JPyHKJiNDis4mYsm1vJiqbdUxLg2MMOrNrt3gcuiFEpHQXlN2PWeEogqzzu4My+f3ldO+K86wr2LgG2zo/a7CKJXxOKRdrIsFpwkjzRUYAOR1mV9KwZDmDUECLaOstqyKaybsOaAXQfv2D0rn8mOXn8APRekefK6Rv46WiTLVJzereAJcJawTtN/dzUyQMzVkGPF60hMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BJYcxHpUZ2kDcL+Ml+XnRav8rWac6pv0uhEIokdawrs=; b=BE6wxrjxTOKpt3izZZMgPwEaHrpjFVwrfvry7ef1mBoDFQYpE7Nh0sx1bJdjdVjnfzBaKC6aFKoonQgOgPZCeWCNhNOVYvS9x784H1BjZe3gEecLwQ/eCGK8p2+iSpE5w72LcNqhs32DC7DhAlzys/jqCgfVcPUGz1ldafQDbIngPt1lXGI07KI8mNh6GScUekBFCbGx9cyWi374ZGWezg/WjvtGlWmM5I6mjeWfe7MfX3Nu6FK3RPyt69s5cHUvxegOH60vd2MfCQ9zX/tbjfpSvDuHtn54E5tZb+LpMb5vACCiYy6m21BNpsWk4tFJhV1XPKFDVDn/tMowrUT/SQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BJYcxHpUZ2kDcL+Ml+XnRav8rWac6pv0uhEIokdawrs=; b=auf8t8LUzkLq6G2xb9M0VquuXfaVJx54weuYwDO1cVmy1JadxN2Diao2TIXHUskIckOyObnfD2FFREj1DrydVtzPzOwIgCzqrY/KyHdJXp1VOtRzEwYXEkwRJYOQjGRUeo6d1W0nmRsmRXPXa2ljV7WypoExa2kcph1yLIFCPaCl/6XX4/FLzq12zTTCZyC6jWEzOc5xuThi6n06dIew/+EFnJcDjj4f1ex7Og/ghwm9rklJPEOlshbooPERozMtbKJ4qHTyh1cLwY9j8G8kp/Uj5O93Mt3Rp1q4cwU/1z0b43CVbJNn3NANZkqgpT3PBuOBOUGQJorsPh/8+IcbIw== Received: from BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) by IA1PR12MB7589.namprd12.prod.outlook.com (2603:10b6:208:42b::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8230.12; Thu, 5 Dec 2024 00:18:48 +0000 Received: from BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9]) by BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9%6]) with mapi id 15.20.8230.010; Thu, 5 Dec 2024 00:18:48 +0000 From: Zi Yan To: linux-mm@kvack.org, "Kirill A . Shutemov" , "Matthew Wilcox (Oracle)" Cc: Ryan Roberts , Hugh Dickins , David Hildenbrand , Yang Shi , Miaohe Lin , Kefeng Wang , Yu Zhao , John Hubbard , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH RESEND v3 4/9] mm/huge_memory: remove the old, unused __split_huge_page() Date: Wed, 4 Dec 2024 19:18:34 -0500 Message-ID: <20241205001839.2582020-5-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241205001839.2582020-1-ziy@nvidia.com> References: <20241205001839.2582020-1-ziy@nvidia.com> X-ClientProxiedBy: BN0PR04CA0090.namprd04.prod.outlook.com (2603:10b6:408:ea::35) To BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL4PR12MB9478:EE_|IA1PR12MB7589:EE_ X-MS-Office365-Filtering-Correlation-Id: 60bc5f44-aa45-4352-1102-08dd14c26360 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|7416014|1800799024; X-Microsoft-Antispam-Message-Info: w4r1cMEiYkc7KtL22I6MAdHot6xMep07BHeARTD/GpqKlrGNH67CjeOxHKuHQ4iFTgzO544lG3ZskwSttAitR9CBayf5B6MacBypO7nO7Gru/w+Z8krz3qVU7lkQx6z7Jy8EmEEVyMZGDIW7c0g0moaBIRCfKX3qsXLf2cnTZf93utXbDQeILb2KArQBTR3LC8nclejyu1tdlf+hvz+k+LlXJTYLQtn94U+Qm1us7wTe7XIPE/TRv/bFAC+tGB1CIJ8Rkxc416rKW89s4FpzllHgjlyNNBrKM2lBZVt1r7sr/i0i9uHVy+5YTWYMdgtv9iqvdYmiRHkkRS9JHzUpFhet5eB0IcRcsWij8K05p/tykF+qY7ooFMQuOseCtg/kBCEKi0vz2YJ7MKlWd7i2m/47Adn5CNrhrygzc05wkTeuxTb/ofZyTrf10qeViCaIObdZpfrQekuiYXUB5WPzLvHT5g/XngdOf+bBSGBj0DJGnloh66WRvoD+jsQFSmDOGE8OTX2jr3yB3bsvJUNsZugtE+2bgjMXkHI/yQqx7/xceNH4HiLXUtDMKmhIHtQLo0ykco7VfU/KHnZf6Io/hDNE+DuRC6dhSem65G4NkOOpI1wAgTwvnJwiJp7SknluEGsan/H70c4HbKp8mr2UCtp6o+wQziHSV/UWUx+PgQSgqdXzssGr1S0W3WqmGqr0RGfWZRavsvkjNzkr+wh4Ec2PnnmaZI7KvGtjrS0fkrdwInoOGPPIPTWnb1Xpvr5bopjOMZ6ZHzA+DY6vHDCDfqER9SfgmxIzmMFWhtaG2oh0r3TLoGDoQfKyZxoUsv+BfNhgqiQVtAcEVl5IFWpm/9BofRMVDWm5a3vrWS6ga1y7ROCLiriC2zyQcf06cCwzJDhKoU4d948qO0wtC9WcBI6Txuo46lqIzqkp6p2qKFPEzVedy/WoEG+53vJPA+JM2n2Y02Xy7leaXpS7xERqrKVUEkRxvS19eisOiOvxXad1vUUMi7mniUXubJyWtvgnLdrXVuS3wrDBwCd3FW7be6AQeukgayVMY+aOa3JztmwlQT4uNRj935PwegBhQJxxjYxI+/TucdEjlVdJXFgV4QlWgGGOqkcavBJfRHqB/0V0NXszEp4Ng3VcFr3JxKNB9px5sThSPcis0Bz/bZwtvmXr2sDEaHB9Xl/xXYE3vSit1QzWHqRhVvDFdJ1T4r6lqr2J94WnW5p6V0FApelvBfg7Cdg0NMNeZ/6mPGuz0DvwiXaO2d4ugrX92WsTujKjqGk5lGqTRMlo+wi9KD7zpUUlGOIRpv2pODx2EXbti90JlyfzXIJ/7QKOMQtX9T+me0fzi0xDtkkAA52edt2mMQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL4PR12MB9478.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(7416014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: OjMZ6QtTP9yPFh8vszEpVjq9hneW5+dI1jHhK87njnmAIeUKNCX1QIkGh+lxZrXaK7Tr5iAvPnGxi6FMXBNjbyPGcKI6923UglDx6192ulwL9y4DZfClhSh1S07PXmuZHfwo9AjO9XhpEXskoEjg1LXYXM/n/FujCVDe7GZRFa9jIw0M2ACP1bt1B+ShjwKlyvNWnyahWvm8AIJcMHlat+WFRMb5Y6XaX62AnKbnki7foHNkUoWnFBOuhtEmJwc3nKZGXMm9Xlwp8Gu8XZ0WwiuMUzaTQr0ld/fLIhmvCojRJARkXckiyJWRU+VA5CzQl6dxl2xYSfKu0b1tL7WPUpXfhrHuxsmHM9JCftr9Ak7chLbiAtujbgyzNAEUFK3iXRQow2o4batthdtAkKZU+whvLi1Pjy81+Jk8Ehd2mS/yTOvwUf3ULQGFJhxMHCSt2cXCHpUmxbR7K3xGTdlQtqqYlJUzj5hTbPv7c/LHF7EB9quQa6U3Hr2pZTm2EsEKZVeU7P+jZfz52x6EVx2zEQpU5qQeDzb49k+oUyB0+E5/1FBX79szMQ6/rWOfXM7kJP9K97mbuq+BYABPUPii1kzp8g2ErIUwDvYEX0Nmdra1ywe8pG1LeOQLcG2KFn+yCCpqfSu00evGZcBRK1+UqGfkWSRSdShgEab+WMKao21P93qrR84GdyE7G5H3LuVJQpfoValmpGQ+ZL8DJzel0f31vSqYgAWDzwLu5bQJXYtpYcI5ymRr5zWmmz5sRTlyWVBVi+KYPpi8wWUAaykxJfKEpT8cLeejpR2L7dtlAomJ1PWPnx6J26JEXrW04gD9Xqmk3+olRRnvx9+/ig4rmCJ3Jl4UPRowrJLWy7t+WXk6dXDM8G0neYRLBnBLKxmQ00+cyKuPBXhs5cUVC8DM3msog1GfNQs9V2ZhiVxnsmxbGS1RNiwSPMdEmLeu6KXqR9q0aBPWyfmNCZX+S2Tj8553ehroDFBq9/obo5ItZhSsYOg+lJOXDDA7ZqWbdHLBeOEHUoeWqdZLocEQAndTwYVwEdUKlMxYCL9FtUs9315m4mzS0vhSmTPvNJAG0CRVf1xzZmAGTwU51T5zFzNxNnAmnuOhxsC0/cEaoLdcz4cCLCOu52YRfCdFp3o7ZTKN/ZYg0+BrEYNFjIBmzSBRraJ3K+nZhmL3pqvBsMs4G5ns7iQUNMmUn1KU+eoZbL+aj1Su/vJ1OMxq7O3d3WR/TvuTUABcfE7hbSivK+115wY4tITTi3cVnr4j6eGUO+EwsoJe5octTh1CMFgNl7F6gTf7wAA9UcZQie24SyqsOhDNf21lVAf3GKtyPElQzkFgocMyi3/2pFXDUkKoCz4QOhK77mwRmrISdPlqQ2TDt3m1TDBa/++KtmIepBtZGvPckGdGZjaC5Uwpyz65FhcWiqqddUPmWSNG5CmgaULqtuC4gZA20UAI29RmtvYfXVlh2ahdt9Hb77iVKxW8E4uFbTJNcF8YhghqrASvAfVujR/hezsRJ3LVokWdqzTdSm8nQTYwCqiPIpQR4tSzxBejYRU96g6FAw3NG/ey9x4XGxqhOIfinvwFTGS0SrI++S+R X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 60bc5f44-aa45-4352-1102-08dd14c26360 X-MS-Exchange-CrossTenant-AuthSource: BL4PR12MB9478.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2024 00:18:48.0863 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: sK27t186KaweXtQ+609hYOdXpHdfWc6rCIpcjgz7vg6LgwEgxnFJyTIiuII4Vgme X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7589 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 3398AC0009 X-Stat-Signature: rwzugb1tgwp6i8g8p4neqhwomc5o714f X-Rspam-User: X-HE-Tag: 1733357925-647863 X-HE-Meta: U2FsdGVkX1/7oy6xsvSVl2LUgyiH049PuGXq8LvvjDkudCGg8AZuKeVscVrvN1Ew5rm8EoblS38cU44HrfAd+Kms/N1r6Tp3GENKdLoRPBLiqOODbtu2MQImuMOBRhTRIaeWq7u1G6713dNKjE5RdBVqS2Qtt5W6omOLHNUgjSSWBiAcTCN00jA+cVapxW3V+jfPprQG7S47GjbVTKn0MLNpmWb8hScvgRiQxlDeWwBSsHayVDcKDLRpDl4zD4BWDSERymMKP8XmHwlRwWsN6tIdhAMX6j87E1fS4lnNF0pUVUPQtJ8snx0JzpoM52bjsY8eWi4aewCkAsWqcH1c7d5MvZaTBM1Z5v/x5aOp0B7OM7geqyqZnhvamKJap2vakNnUyBF493/CR29X0pH9VAko+k0auyoIQumAjjH1qqjBtIfIcE08G6i383TT15s+o7nI4Iwf/b+ZDnbj7fS/MRjPoL+AYnVZUPcIETkjizs6/uZdDTIPg8o6JRs7PYRiIVrUPLXb6WQ4kGJ2rVKC3Dbh8NY+4VKaoKMiDlR4uHMxBMI/EkGlv8FOgjfqmXUUieVhiRGMqVyOty4RkDEhEV1U9SmnmoLNnIVaIeOGFCQZalFGjd4D+y/0GkfJOYKpyDIMcLHCX8YEsejDtcj+aHWS6RvPRzSrqosEmX8j052HeiHe6SefSz4bBiCe5TmgJSv3w9VSzC2RYWMCNHdSOlxZ4V4XrHz66gTQ+XJRFyoorW8j6iuvvbxbFzkiXUy4rEE2ZBbZ5jJRqtwbu15CleRCqO+IhM8QPvuEyjDYrDFCjCTe2UlxVl78xZbWUla50ie3mgUHit1OnxmjxahizFoJYEli+QjBDMmuuVyA3FcryXdE9bDIQ68GJGpLhHbNldYK13SxvT5TKV2DYX+2K9vYGOrCWDRRktbEz0NbdlhjtLgUVGTOaP8GUkWilM7I5y5I+CEALylITKjmEjv i/+ZmXCV J0sfyE+ZfGm3cf/7nMPZPRA7d5TwsXWgvrCEsSyQzGl48QMJppzOEwfws6K70Yg1bc4+6W9zB8tcA9ZAkknc9tltOE6govM/03dfU4KMrupKWZRAi2ikZJeEiRRPK37QVyxODy5u/msn7wyxL4Hmtdc4OEUtqW9xC8pp1dV5FHgArnJhzlMyui2vvmf/M7cHZsHCYtkwMtP4z6EeAcNo5sgiFokH3Weal5Sqy/kR+LueOCWroaVa9cfuafr5poMACLYDci0x2raywhwNjfDtkP5/kGeIGJedqE/hhhmOe0OasENdyQFrCb5stpXKCZElPlS0nFaUtGsv3PxIRnh+5W9yuHzJXMZgIVWlfRYEl5TbFW9AsNKtLinTtT4YRetwBvI70iX0tcD/JxmSAsPhzHhuyEg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now split_huge_page_to_list_to_order() uses the new backend split code in __folio_split_without_mapping(), the old __split_huge_page() and __split_huge_page_tail() can be removed. Signed-off-by: Zi Yan --- mm/huge_memory.c | 207 ----------------------------------------------- 1 file changed, 207 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 4f4f3b50c537..eed4ff97e235 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3146,213 +3146,6 @@ static void lru_add_page_tail(struct folio *folio, struct page *tail, } } -static void __split_huge_page_tail(struct folio *folio, int tail, - struct lruvec *lruvec, struct list_head *list, - unsigned int new_order) -{ - struct page *head = &folio->page; - struct page *page_tail = head + tail; - /* - * Careful: new_folio is not a "real" folio before we cleared PageTail. - * Don't pass it around before clear_compound_head(). - */ - struct folio *new_folio = (struct folio *)page_tail; - - VM_BUG_ON_PAGE(atomic_read(&page_tail->_mapcount) != -1, page_tail); - - /* - * Clone page flags before unfreezing refcount. - * - * After successful get_page_unless_zero() might follow flags change, - * for example lock_page() which set PG_waiters. - * - * Note that for mapped sub-pages of an anonymous THP, - * PG_anon_exclusive has been cleared in unmap_folio() and is stored in - * the migration entry instead from where remap_page() will restore it. - * We can still have PG_anon_exclusive set on effectively unmapped and - * unreferenced sub-pages of an anonymous THP: we can simply drop - * PG_anon_exclusive (-> PG_mappedtodisk) for these here. - */ - page_tail->flags &= ~PAGE_FLAGS_CHECK_AT_PREP; - page_tail->flags |= (head->flags & - ((1L << PG_referenced) | - (1L << PG_swapbacked) | - (1L << PG_swapcache) | - (1L << PG_mlocked) | - (1L << PG_uptodate) | - (1L << PG_active) | - (1L << PG_workingset) | - (1L << PG_locked) | - (1L << PG_unevictable) | -#ifdef CONFIG_ARCH_USES_PG_ARCH_2 - (1L << PG_arch_2) | -#endif -#ifdef CONFIG_ARCH_USES_PG_ARCH_3 - (1L << PG_arch_3) | -#endif - (1L << PG_dirty) | - LRU_GEN_MASK | LRU_REFS_MASK)); - - /* ->mapping in first and second tail page is replaced by other uses */ - VM_BUG_ON_PAGE(tail > 2 && page_tail->mapping != TAIL_MAPPING, - page_tail); - new_folio->mapping = folio->mapping; - new_folio->index = folio->index + tail; - - /* - * page->private should not be set in tail pages. Fix up and warn once - * if private is unexpectedly set. - */ - if (unlikely(page_tail->private)) { - VM_WARN_ON_ONCE_PAGE(true, page_tail); - page_tail->private = 0; - } - if (folio_test_swapcache(folio)) - new_folio->swap.val = folio->swap.val + tail; - - /* Page flags must be visible before we make the page non-compound. */ - smp_wmb(); - - /* - * Clear PageTail before unfreezing page refcount. - * - * After successful get_page_unless_zero() might follow put_page() - * which needs correct compound_head(). - */ - clear_compound_head(page_tail); - if (new_order) { - prep_compound_page(page_tail, new_order); - folio_set_large_rmappable(new_folio); - } - - /* Finally unfreeze refcount. Additional reference from page cache. */ - page_ref_unfreeze(page_tail, - 1 + ((!folio_test_anon(folio) || folio_test_swapcache(folio)) ? - folio_nr_pages(new_folio) : 0)); - - if (folio_test_young(folio)) - folio_set_young(new_folio); - if (folio_test_idle(folio)) - folio_set_idle(new_folio); - - folio_xchg_last_cpupid(new_folio, folio_last_cpupid(folio)); - - /* - * always add to the tail because some iterators expect new - * pages to show after the currently processed elements - e.g. - * migrate_pages - */ - lru_add_page_tail(folio, page_tail, lruvec, list); -} - -static void __split_huge_page(struct page *page, struct list_head *list, - pgoff_t end, unsigned int new_order) -{ - struct folio *folio = page_folio(page); - struct page *head = &folio->page; - struct lruvec *lruvec; - struct address_space *swap_cache = NULL; - unsigned long offset = 0; - int i, nr_dropped = 0; - unsigned int new_nr = 1 << new_order; - int order = folio_order(folio); - unsigned int nr = 1 << order; - - /* complete memcg works before add pages to LRU */ - split_page_memcg(head, order, new_order); - - if (folio_test_anon(folio) && folio_test_swapcache(folio)) { - offset = swap_cache_index(folio->swap); - swap_cache = swap_address_space(folio->swap); - xa_lock(&swap_cache->i_pages); - } - - /* lock lru list/PageCompound, ref frozen by page_ref_freeze */ - lruvec = folio_lruvec_lock(folio); - - ClearPageHasHWPoisoned(head); - - for (i = nr - new_nr; i >= new_nr; i -= new_nr) { - struct folio *tail; - __split_huge_page_tail(folio, i, lruvec, list, new_order); - tail = page_folio(head + i); - /* Some pages can be beyond EOF: drop them from page cache */ - if (tail->index >= end) { - if (shmem_mapping(folio->mapping)) - nr_dropped++; - else if (folio_test_clear_dirty(tail)) - folio_account_cleaned(tail, - inode_to_wb(folio->mapping->host)); - __filemap_remove_folio(tail, NULL); - folio_put(tail); - } else if (!folio_test_anon(folio)) { - __xa_store(&folio->mapping->i_pages, tail->index, - tail, 0); - } else if (swap_cache) { - __xa_store(&swap_cache->i_pages, offset + i, - tail, 0); - } - } - - if (!new_order) - ClearPageCompound(head); - else { - struct folio *new_folio = (struct folio *)head; - - folio_set_order(new_folio, new_order); - } - unlock_page_lruvec(lruvec); - /* Caller disabled irqs, so they are still disabled here */ - - split_page_owner(head, order, new_order); - pgalloc_tag_split(folio, order, new_order); - - /* See comment in __split_huge_page_tail() */ - if (folio_test_anon(folio)) { - /* Additional pin to swap cache */ - if (folio_test_swapcache(folio)) { - folio_ref_add(folio, 1 + new_nr); - xa_unlock(&swap_cache->i_pages); - } else { - folio_ref_inc(folio); - } - } else { - /* Additional pin to page cache */ - folio_ref_add(folio, 1 + new_nr); - xa_unlock(&folio->mapping->i_pages); - } - local_irq_enable(); - - if (nr_dropped) - shmem_uncharge(folio->mapping->host, nr_dropped); - remap_page(folio, nr, PageAnon(head) ? RMP_USE_SHARED_ZEROPAGE : 0); - - /* - * set page to its compound_head when split to non order-0 pages, so - * we can skip unlocking it below, since PG_locked is transferred to - * the compound_head of the page and the caller will unlock it. - */ - if (new_order) - page = compound_head(page); - - for (i = 0; i < nr; i += new_nr) { - struct page *subpage = head + i; - struct folio *new_folio = page_folio(subpage); - if (subpage == page) - continue; - folio_unlock(new_folio); - - /* - * Subpages may be freed if there wasn't any mapping - * like if add_to_swap() is running on a lru page that - * had its mapping zapped. And freeing these pages - * requires taking the lru_lock so we do the put_page - * of the tail pages after the split is complete. - */ - free_page_and_swap_cache(subpage); - } -} - /* Racy check whether the huge page can be split */ bool can_split_folio(struct folio *folio, int caller_pins, int *pextra_pins) { From patchwork Thu Dec 5 00:18:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13894606 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C596E7716F for ; Thu, 5 Dec 2024 00:19:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1CBA66B0099; Wed, 4 Dec 2024 19:19:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 17C356B009A; Wed, 4 Dec 2024 19:19:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F377B6B009B; Wed, 4 Dec 2024 19:19:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id D6F9B6B0099 for ; Wed, 4 Dec 2024 19:19:35 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 937631A0EE7 for ; Thu, 5 Dec 2024 00:19:35 +0000 (UTC) X-FDA: 82858996122.06.AAD2ADE Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2078.outbound.protection.outlook.com [40.107.237.78]) by imf19.hostedemail.com (Postfix) with ESMTP id 8C50F1A0002 for ; Thu, 5 Dec 2024 00:19:16 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="E2/RxZYn"; spf=pass (imf19.hostedemail.com: domain of ziy@nvidia.com designates 40.107.237.78 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733357965; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Bxq/htjbbRs6vBjKE+Rb2qk9i5PlZjDkb7fkaknEWEQ=; b=ov2dsO5DWUm7mKbzfJwLj3h3ddTlDnhZ373N+rA8C1Orh/wXXVtQytmV0muuGQNi/ueBTA cfSq0X5MIOvSZjsfAvRBKDC8naJgu3Fnd5U1R+lHNSGhvl7cFoe46H6Yx6fBe8tkf7EdrV aAMT0Iit9uVxU/5sxv2NzNvUkIHjv1w= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1733357965; a=rsa-sha256; cv=pass; b=q0ADFQ0CkLZip56sLUQtYNYlGCwoEHRl9vyuOTNpmfWuebeszAav5DX7acl1dViPLTR15q 1TKhGOeOOW52qs0NHUJV3XVgkOwB7P6bwruHrIWXrEebpo9AA29S9dxElYLGrvN3iVBqiw tieBkjoIh3kEb9agwQ+8nuO+OSTFStU= ARC-Authentication-Results: i=2; imf19.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="E2/RxZYn"; spf=pass (imf19.hostedemail.com: domain of ziy@nvidia.com designates 40.107.237.78 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=W+OVIbXjN5L2B/bHf0jy5nPenVtZ/uhnGuTCSpELjN93oQsf88LK/ePL2iH/nlUMpEzE0N06HYbnIv3ZGnQS8IvckPo60nlDgnLA9C60vCzJ3FpWN3HB4znRNYv9EmQNr3xjzGxMoz4c4W63Pcq9qLTgQrOCMXLo7fQAMLVo2Kr+7jMRjd19augOLW+NIoy9S11zZc+xxzIknkFy2BgYUf9bsCQobU6hD2Z7P/02PV8yNauBPEaA+7zALVsuAQsWN0HfXya8Wp1AAQLtsHlzjo4hTk5Z9qvHEnCcVd0utIfdkqw77IOXWhdVpb9yms3WEwE+wp5X4kEqh6W6Kpi6BA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Bxq/htjbbRs6vBjKE+Rb2qk9i5PlZjDkb7fkaknEWEQ=; b=pfiFbjnrkmSF16wPMDJRtEeez6Budke2f9L1ceH2u2YIlzOksgKxSPgJhJsmL19/X1a82hdTCXqc6DeMFTN0Fiw1ZZkLqg9DW6wnyTCf26/ivHOdPsk8cYUt5HUr9un/Aa8gAjLx2okSaHk8viDnSwOkJQFXkZ0v0/uk0q2rpLDU/U834hWeoBlB2dVKDFKutG1x5kYspaZxx+Iu9mTn6dNJFQEle3LeQBl9f138gf0oj+BrtRdj0w4stle1Gl+TO5O4aMAHLKsP2qjJGTSpVgOjjAkxDiJqjrI5eykjukES0vbmbrLmUjKP+jiPbqTGgECKGoTm/moI1JXEsDIzrg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Bxq/htjbbRs6vBjKE+Rb2qk9i5PlZjDkb7fkaknEWEQ=; b=E2/RxZYnr3DdEs6tTLW8ENyOYHzHY4aC6taZRHsd+MMxUFWfQYnMPvUf1tjiV85Kba4W8x1ITiX7EmTWZxwiksJxL0uwczhR8V6DqolhTh5rkIRB6yLih7UTF2zOdHhunYGcN9g/aBQsYM4sT1L8hNsGOsiGW8lBJAVo18kMrWlI5sGyZtMKFro51pfNX0A6nC/DEh22KJqnqWgGxeUE4VA41Upb23Sq7BsnPff+M9/V72BS5Ttd2CgMnWcAo7a8VzYwUOnG/XN7g1wb3yHUlckC9A++GTfB+Yhm3HidB27tYjK7iPar9cPsCYpbnhOMnuFB4KDfyVyC+6zpWmjJZw== Received: from BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) by IA1PR12MB7589.namprd12.prod.outlook.com (2603:10b6:208:42b::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8230.12; Thu, 5 Dec 2024 00:18:48 +0000 Received: from BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9]) by BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9%6]) with mapi id 15.20.8230.010; Thu, 5 Dec 2024 00:18:48 +0000 From: Zi Yan To: linux-mm@kvack.org, "Kirill A . Shutemov" , "Matthew Wilcox (Oracle)" Cc: Ryan Roberts , Hugh Dickins , David Hildenbrand , Yang Shi , Miaohe Lin , Kefeng Wang , Yu Zhao , John Hubbard , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH RESEND v3 5/9] mm/huge_memory: add folio_split() to debugfs testing interface. Date: Wed, 4 Dec 2024 19:18:35 -0500 Message-ID: <20241205001839.2582020-6-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241205001839.2582020-1-ziy@nvidia.com> References: <20241205001839.2582020-1-ziy@nvidia.com> X-ClientProxiedBy: BN0PR04CA0072.namprd04.prod.outlook.com (2603:10b6:408:ea::17) To BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL4PR12MB9478:EE_|IA1PR12MB7589:EE_ X-MS-Office365-Filtering-Correlation-Id: f75e2074-40a4-497c-f60d-08dd14c263df X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|7416014|1800799024; X-Microsoft-Antispam-Message-Info: dcvr9gTrsRvR8Rij58+lK1ZYcmV8TaKNXdV6yIYjmS6XwEV/FDNyGtHxiabF52exnnBFygtP+t0K9oSfLaDMmI+izkci0lR2wsQzvj3JvEkYD7AROZPw51M19e/wwfg79+eXwhhrxZj8f2sTZ/8F8uTqtoVX5yEqyLXI9eb5isknKp280CwqOAQOJkjs1Yb+a4dwfIL5nt4zUKp/UpuHdMY5hXGe2wMle9bcgc/J2sbVGiQuxLUYP9bGAXrOLT2herAmsW83fxf8qjDQn+uhauPuqgQ4Qi1yM0W1GOOSAqNhwl4EpztRJ6cHfpOYO13GgKPVwrtZO2imiPYgklNfLt7Yr1+1JaB9gTuVNlZzNDEFn8MQ61JDKLvaU1BpLYF9vMlWVZsNjZhJ0ibDgLjd8Etbr24GBLlEfNCuNpvOVOs0g77MgCACsk34GEZn/P84z+VM1xW2Kbd+K/+WEZ249IeJ2MQq4oTzKLlFFuPt+n2x+DGgVXXc7dhSIukda3SvM83jzACksOkAvvPWOS4TpEsEaisTOxfxZOdqW9CbjMIIDyg1jBKPZmSbBD3Hr9xNFj+jLVJNv8++VU3Lwy5fc9g3FSQxdrGqUu7IhLoF9XOuDx7uJrHrE5w1QCfOMbrCdNM0m0A+qtLE753BhBEAVhE0fEYOSAQSwk2eNuFNDMB4ZZN0y6XHXBiUAkd3FgOz6j569uox8ZRQvYamV3sL5as3yX7oMWKj5u08KuKuBa1o3LRx6tbhcLcpxyM0rqJOzPFPWSzUu2hjHT2zPJ1ypkb9ZnaHsc40w7ocelicWWZ54IBEu4U5ORXyMha7AWR3FASVYR/pJHCyu6d8MVyKAMIr4O0bfJcgBsPQW+IP7Vx7i2Kit/2X8K5/cJCQlVchcDuiNeQYtO/TI3tYYvGmQmFHuQnrI7kWuEGoLn62DfIH5sdr+mFuOyH8r4YWpiJ+rR6WrLfmC8FaeQPHeWczS/UZxeP+tBH+5CBp1G1jJjsGR2Rwla4bIao84fPeFgfiWCk0mc/qeaYqg4hQEnfx9I3+YcT8fuiIzMsyL20SmIsZz1m+GsX6/g123FHEKMvt1c4IZ5g4v4VqhTMmMWBJ9LkCeu2BlxrRGeb46cZnnBbh3JsIyxILTWDhsxRtbjpXJKsSQqB0d2iAa/3yezrydrKmUjeMc1s2ewBdma2iYOl3TmIq57GJ5OghSOKhZ4XEaYIQPowI4ODJREquCb1tWtffniivQqIOacVLsCZKMtA/Y0qmjg5CzLRMPD+7/r04x+GR/IjV9J+VTh2F0/xF9+Di9qnh7+C5EmxIFC8kO/ZRd6L3bwl5J4cSnSwMdFBUTXOUHSfWD8nkZAfvdu1Z7A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL4PR12MB9478.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(7416014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: R0URncAtiweNE7zq9b3E1L/MfmkLFw1u9uethL3oypOuHJxqKYmlAD9ysYGaBqZSbpvS2+o0Ip6aWSkYw3pOKmtyoDskRf6N9Q5iIYgGwe2wfmxaeURf+wmrIseSBFEIJ+DmKD4KDUWdQqaVJyHPH/blNbHs8J+CeXbB5y2QpqUkhFRKRfRhlbw+U6sXZLzJe/Y2VOYn1cvywMnsploFwAHrs3vJII15IvZxU7YEI4r7s9XW/oCAs9s4iwLx7/rtwqjD8W/GXsd2P3SRfdNnki3MJvdWgAQXas48KUdC3b9Q1dsj6nsSlWvgBYpxKzyw3qG58UPLMKw3n+NqTTZpfPVdH7K5dKykbI4Lyii3JLgvr+CyFFh/n5XqW9qvip/pgoBDR3odZvCjIQ3vHi6FZdWrXz7b+irltqAGQueeUbjMfq07addHRErmAcaegLRNcROcVzKKlxv8L52bHx6+QqyoLt+g8rGTJuU5kLY0OaBp6RW1VnwbUkVRHa8oYjxm/EAoqSWxFJI9d4P0Bi5ho2oAlILnC4ucy9f+DF31W+GeMEb7hlJa2XFtIsRUp4xBM6zUuxNMobbg/NTZeCkX0gZMzaFK8qqLMuaJLNyms+WbWdkdTpzUmx3sOtcSBTMf+eIjDucEP8PIfKgXCxG/dbgHkhMdbSSArhtMidVyvKQ/+ubiPm5Xodz3CSBY2lz4FlMMdT7li7mDPK1EdfuI2GBp9MIy49o+jthEck4XGkOAcI03ftyh8JUGxRCvva5xX4hOr92S34xxdnWc5NirPkPSWmJtV/SRev1epZRgHyY28S+10Kiqqni0J6Sx+/sq40Dm3o7Au7Btu5rnnTKf/ssOy0bunoELHj3vNcsoVgKv6oiuPLFp3hBoJpZyN2XKfM5ESMvqQUiHel9T0vVRqilPSnAo2daw3qK9Dsy+5a0syCyXaXSesqxxwQeaWap+iCXr6hgJCfI1TbdGgW97TXijLTKr6zMGNfdgKhcHuq6KECNgRNnaOIqEA0ZDvZb3qaEU4/13EQzGHXhLZ8sCe0Hh17ou0jeYh6CSBfONEznubLta0uahUJCy7QM+vTcx7oIxplzblbdEZ8n5mV+1ls4Igx0X0kyDTkeRH6szihf20v8djclL7WuovjwIdQFSLFjiz85rcrl9lYKRd5lx48Szea1BDnXchk4INyqBrLGIvhwwlMTYP/SPdrTOc7N3DH5sc/vUgCzJNZL40rxrVrnHptjN226sO0NEqejUtJfYWWiQCwnpYudeFD1lq2vzBo8/qAuZaG/ovxVG4LF53cKH3LAmjvtXz2OxuZAnhGPCnP0mjB4AFCzCWC1w9noorTYdp93tmrDZDlm+ayBaMJW0eDYOnXOy77u4T/RrT3yRcSfnIj7oLG8vBUDcjK7gE0vZyAu5kcM3VhqSuVlY3Ji680jpXG47o551zqeVP+KOQ04B5uyYWaXw9YM3ZvMA+7Etca3efcAq7FlU7ufzjK5QA1FxC2+V+pa/lSbg/IA845od+yvYSwU5dx3med+Vuu4WHU7YkalNi0DvJjFOqiBoLWnQgM1VE9QP3Dp8oT7BWt3OPfAmBkfzbxb/NlmH X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: f75e2074-40a4-497c-f60d-08dd14c263df X-MS-Exchange-CrossTenant-AuthSource: BL4PR12MB9478.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2024 00:18:48.9246 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3U7wsB7U+JW9ht2kVHgKFmwX1CR+PitJ3DJ7sFHid+yn9Vl+6kNy9yNmhu8jZl6j X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7589 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 8C50F1A0002 X-Stat-Signature: 44qug654c4wejzj1fc98yebatt3xczm8 X-Rspam-User: X-HE-Tag: 1733357956-486769 X-HE-Meta: U2FsdGVkX18UTBawd2h5oBUGLl3oOPIpj6cE6Q5HiVPUc6mJfk/jhBkY/mGCOoEgpqb9P3+k06aAd/P1Kxaau9xzW1qZ5aU7vQIo74v41tbc0ZCiXByXV0OryieXl4KiaDIIht3fDOVS4CkDDnegmWDjube0Bfz8CL5VB6UkFUrUo0pdU/WY9gqXkltsBj5vZMmNGdKwY2sv4GR8owb2Cnwt/G5vUFE5rOU0IqUa4Uy3p1jmECDA9We39emn6PZqs+AFhTZ8p0QHLSLeVTu2f8pbtmyH3+fyTlJZEdf2so9CJNSCLOGHXW/wwuapRY6oxQkrsXUvkh2Al9h+s1B6n1jk2kZZSo7AQIVvmFBx4TKkwY6mugnfo119gEE7cUEcMdsk8jLo4pBopOXkhlf40VwvKnS56KBAgaskz9KyvpAPGwYj0reYCWdjqzrf1MFoQ1kefpSwK4cWOxDZrJIi6MCjzDDqEeKcg2vS1FE2SQaxKL7ocs7oC3IuckAoTuXjXTkkhasi2zu33s1iTwiS2Nr+OKWYYkzZ02W7ObV+A+Rz6sfWPwfhlcx2JX8LKt6kPlA4GnbxR0qVUVXQ6BTIyZk7/ZjZRxOehJaB6j0H06TwcSy9sCyKCj0pKnoBlI2fY6qR+dGopnc7OsaDCZL0PrvyYN4MkZW6U/N6Kb3kaIO/7YoqqyO8Xb7Qr9AWtALF5/wxkUX3CkPHNTTK8U4Fklh1wYWoVb7Ei1gX4ArlMtYadOBcX+N+L2NV92Tz3iaJDQWrHe7e9pmqvXBnAZ3q3FY5tpjWyVTT1T+Bn9nDAng3fCicZq+NLDlHII1cDl2fuAW4NF1+aE2uBr+aQM28i9zp0CBRv2tzmHid0Eu7VXNbCkr0pSSO0gdXRVHTVwsTmO0nUnQIygPdYSK7QeN2jCWBfYKQhFXQAZ3gPHaJQzqiw40+l/EwIBTpxNWMylpKA82wXeppwVkIKvY7ayG rHHQg9b3 cQJFL5NpRMu4tsXFw7y37BxBtgG4JGju72V1qMJmKn1jp7qTZAQSMt06KtzqDT9+UmYq7vRfmFeMEATWS/hyiT+vVG9xxmoX3pF3wYj4k/8A5UXUhe9iE6Ulg9t2+dje7I1hzDn6imO1CKNHNb+9G1NRPIdcAURclQvCKIIkRN3z9qSv5pHlBS1Z5tvXGvOIYwyx200/5nmthsAMWZoOICGKYEWrkjtH5pOZPxuMczLmLIiB2WAy2QLZ1qUgl6wvjeKTiZPFLI7LWCY5ftCvusW/EljVUgtPhh0oxYOy8C/vCUQXLy5LtEuDtdLmC3wTbDRl3OQa+gTVI6+40bjQwBvh9GQj/YS5pcKf4PtEQLgZ1BkANtRRelG0QnKzB5rz/ZlVTFePlVGiU8VQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This allows to test folio_split() by specifying an additional in folio page offset parameter to split_huge_page debugfs interface. Signed-off-by: Zi Yan --- mm/huge_memory.c | 46 ++++++++++++++++++++++++++++++++++------------ 1 file changed, 34 insertions(+), 12 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index eed4ff97e235..d6ee18327cae 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -4141,7 +4141,8 @@ static inline bool vma_not_suitable_for_thp_split(struct vm_area_struct *vma) } static int split_huge_pages_pid(int pid, unsigned long vaddr_start, - unsigned long vaddr_end, unsigned int new_order) + unsigned long vaddr_end, unsigned int new_order, + long in_folio_offset) { int ret = 0; struct task_struct *task; @@ -4225,8 +4226,16 @@ static int split_huge_pages_pid(int pid, unsigned long vaddr_start, if (!folio_test_anon(folio) && folio->mapping != mapping) goto unlock; - if (!split_folio_to_order(folio, target_order)) - split++; + if (in_folio_offset < 0 || + in_folio_offset >= folio_nr_pages(folio)) { + if (!split_folio_to_order(folio, target_order)) + split++; + } else { + struct page *split_at = folio_page(folio, + in_folio_offset); + if (!folio_split(folio, target_order, split_at, NULL)) + split++; + } unlock: @@ -4249,7 +4258,8 @@ static int split_huge_pages_pid(int pid, unsigned long vaddr_start, } static int split_huge_pages_in_file(const char *file_path, pgoff_t off_start, - pgoff_t off_end, unsigned int new_order) + pgoff_t off_end, unsigned int new_order, + long in_folio_offset) { struct filename *file; struct file *candidate; @@ -4298,8 +4308,15 @@ static int split_huge_pages_in_file(const char *file_path, pgoff_t off_start, if (folio->mapping != mapping) goto unlock; - if (!split_folio_to_order(folio, target_order)) - split++; + if (in_folio_offset < 0 || in_folio_offset >= nr_pages) { + if (!split_folio_to_order(folio, target_order)) + split++; + } else { + struct page *split_at = folio_page(folio, + in_folio_offset); + if (!folio_split(folio, target_order, split_at, NULL)) + split++; + } unlock: folio_unlock(folio); @@ -4332,6 +4349,7 @@ static ssize_t split_huge_pages_write(struct file *file, const char __user *buf, int pid; unsigned long vaddr_start, vaddr_end; unsigned int new_order = 0; + long in_folio_offset = -1; ret = mutex_lock_interruptible(&split_debug_mutex); if (ret) @@ -4360,29 +4378,33 @@ static ssize_t split_huge_pages_write(struct file *file, const char __user *buf, goto out; } - ret = sscanf(buf, "0x%lx,0x%lx,%d", &off_start, &off_end, &new_order); - if (ret != 2 && ret != 3) { + ret = sscanf(buf, "0x%lx,0x%lx,%d,%ld", &off_start, &off_end, + &new_order, &in_folio_offset); + if (ret != 2 && ret != 3 && ret != 4) { ret = -EINVAL; goto out; } - ret = split_huge_pages_in_file(file_path, off_start, off_end, new_order); + ret = split_huge_pages_in_file(file_path, off_start, off_end, + new_order, in_folio_offset); if (!ret) ret = input_len; goto out; } - ret = sscanf(input_buf, "%d,0x%lx,0x%lx,%d", &pid, &vaddr_start, &vaddr_end, &new_order); + ret = sscanf(input_buf, "%d,0x%lx,0x%lx,%d,%ld", &pid, &vaddr_start, + &vaddr_end, &new_order, &in_folio_offset); if (ret == 1 && pid == 1) { split_huge_pages_all(); ret = strlen(input_buf); goto out; - } else if (ret != 3 && ret != 4) { + } else if (ret != 3 && ret != 4 && ret != 5) { ret = -EINVAL; goto out; } - ret = split_huge_pages_pid(pid, vaddr_start, vaddr_end, new_order); + ret = split_huge_pages_pid(pid, vaddr_start, vaddr_end, new_order, + in_folio_offset); if (!ret) ret = strlen(input_buf); out: From patchwork Thu Dec 5 00:18:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13894604 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E62B3E7716D for ; Thu, 5 Dec 2024 00:19:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7C0336B0093; Wed, 4 Dec 2024 19:19:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 76F4C6B0095; Wed, 4 Dec 2024 19:19:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5C21A6B0096; Wed, 4 Dec 2024 19:19:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 3C7ED6B0093 for ; Wed, 4 Dec 2024 19:19:12 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id AFB578124C for ; Thu, 5 Dec 2024 00:19:11 +0000 (UTC) X-FDA: 82858995240.30.AEDC5BA Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2040.outbound.protection.outlook.com [40.107.94.40]) by imf18.hostedemail.com (Postfix) with ESMTP id 2BB9D1C0007 for ; Thu, 5 Dec 2024 00:19:01 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=B3r0mMGn; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf18.hostedemail.com: domain of ziy@nvidia.com designates 40.107.94.40 as permitted sender) smtp.mailfrom=ziy@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1733357935; a=rsa-sha256; cv=pass; b=KnBve1ig1gS8Z4SODkoQ0CtPvMuuxkLDx5zVKMxs1BZw2m06PuN3gNHLlv3xJMVbksi1JZ +gM3aWoxE633RnGoXn4TxD4dVPwPhYd5x1QIsbgy81fZG5SsEbQo9bG7HHAZ4rYOOdklnl I7qFEnR/ZlLFgoWPRTnWE1voUEr+cBg= ARC-Authentication-Results: i=2; imf18.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=B3r0mMGn; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf18.hostedemail.com: domain of ziy@nvidia.com designates 40.107.94.40 as permitted sender) smtp.mailfrom=ziy@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733357935; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ScO3EO7bE1reYb9jG/eBQkGrULymO/6W7BTzXOMGCLQ=; b=HiQyxjbQmyzy7u/38WqyO755FEd1nHz5VE32ekhzgsJ5oEANL/lvX1CoNyhddbU+L7Gx/K baIqI364KKKoPUmd34XZbVfI8IZvzZXr/tVj7XOz+QXI/iLfoOHYqrulgK63tMqCDKHolM Usfc3eyMNxQ2/+LuHFZeAkjYF0s4CFY= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Xs3g6oOZ6htcTISdZVUZUJo+mxpVfdqpBWxZe0yQ4HcbuwqXTM+pZzLvMCIymSXHlb2jkkaAnfw2yKh697n8j/3R0VW/CM1W6t6GtR7KU1SERIfHUZgBX7ItrY5HcKp1dzX3X2TotDtyElT4XZGjuLhm3uTTwMfGWoPBjPlftZRueoaZWsbubAaBZ0I1fVnDV9RzJ9PliGYOKPkHZbq53SIMcmoJvIpCWXZLrz6VpubUXfKYI+LP4PMs2qEB+bDkoTw8CoWnkY/e5mYnVmBhJSdrY7lJ46sbFXG04iCAmz/WJnyJjDSiVb4rdCEe2eRYqjJUn4FtzwEC3eJDXiB1Ag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ScO3EO7bE1reYb9jG/eBQkGrULymO/6W7BTzXOMGCLQ=; b=JAc3tM9CopY0rL5VrIbHpBEoTzjUKpuF5NCeHH9kFXjPzCe4HEiB9WtNSM7tkLQEhaTNBAtgBK2rDqurlUeilPpyYao8Y/bfmW7ci2UWtprzYHLcwgDJaB1wTy/TbkooUU3jWSYnYUQY9bX6GRFDIIbMexlvrHKZ1I7/+e5bmn10TLiBivNwyfbtLQqovxy9qKNwGlopkegQrP36EbboauS+a2ZYEOKEvD0K7opikE8Rf6nPk2ImNAJDMAGB7VtzySJOCK34uGlSzbsr8PLjc06urImmj7VyVBsZfIflYOMH1t+5ZNT0glSWSm3ahXLEnLfuryeiMCMusZSNKZmT6A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ScO3EO7bE1reYb9jG/eBQkGrULymO/6W7BTzXOMGCLQ=; b=B3r0mMGnAD8KNmtFPQOwr7PN48Z3LQs60uh+lnzixeSHimBzpTLwZpMhwKhnGB+Ip08IsuY5ZT6Kb0f3Nye77jJMIGBFM4M+mHPJ2i7z1dlroOpheGL5KjlZysBfjAJLCvatNkhleCQflxFRunzfbnhtYbBhtZgK1/JcJrx1v76nEmAjErgVW4QE4Zyi+aKVFVSx9aea24j1pLPTbuYxkVSzE8omo1Ze37IurWo6lxMv9cjaWOoXoccnK84a52U+Z598TFtPTs+denIc/t5TN3euHwrMaQ4ptRzkVSI6zILqnh7CCpu9sZgdT262j6WmqI5zYf2Skb/3Mkyjvruc2w== Received: from BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) by IA1PR12MB7589.namprd12.prod.outlook.com (2603:10b6:208:42b::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8230.12; Thu, 5 Dec 2024 00:18:50 +0000 Received: from BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9]) by BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9%6]) with mapi id 15.20.8230.010; Thu, 5 Dec 2024 00:18:50 +0000 From: Zi Yan To: linux-mm@kvack.org, "Kirill A . Shutemov" , "Matthew Wilcox (Oracle)" Cc: Ryan Roberts , Hugh Dickins , David Hildenbrand , Yang Shi , Miaohe Lin , Kefeng Wang , Yu Zhao , John Hubbard , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH RESEND v3 6/9] mm/truncate: use folio_split() for truncate operation. Date: Wed, 4 Dec 2024 19:18:36 -0500 Message-ID: <20241205001839.2582020-7-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241205001839.2582020-1-ziy@nvidia.com> References: <20241205001839.2582020-1-ziy@nvidia.com> X-ClientProxiedBy: BN0PR04CA0073.namprd04.prod.outlook.com (2603:10b6:408:ea::18) To BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL4PR12MB9478:EE_|IA1PR12MB7589:EE_ X-MS-Office365-Filtering-Correlation-Id: f2e236ec-df4a-42dc-898d-08dd14c264a5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|7416014|1800799024; X-Microsoft-Antispam-Message-Info: EpHA/NyAKWq7xPgwK/z96oBkzI9C16OQUotJFcKbg2XVkkHyL5VKHczgiI37WjXk8bEkEtY4vwRVAJVHZH04oNHt00wVisAkDSR1qDsJcOt1Y2B7Bo0q0l70UcRw3PMnbJq2fdYfRkCdvazumcA8nUpzSCQ9KBKkFEvLAx+JtyhPgy8fLWBC3K9IexGAl+8nCVvNSG8ZiiGsujGkaD/Xc63BK+CeQO9tbCKgx08Iwi+RkbUBF0WrPCfV2HA/r6pXhWizJmZf8+s796babPMnGHyZ5vSq6PzsOMc+gHYn59BG8mQm2DysV4iK1Xcd+O3aOh9AEOUOfkBqOsQ6Q0u4gzmDAH1kzdGitCfhum80iku1HiR078Yazvi0sFHRt6r9YJfN7koIfFftmhPwbBTlc5XvZRPHB1IB6apyNdpI2n/okSqRcF2EkCjY7kVxzwiEagW6oOOyw8bVdv2/475Nv3C/p6SV7cPJvCJ2rrU27OTGF/k5h6XRuaxWn48xv3tofFrn6A6suddADQctVVcNwySeTh3YPR5+pq5ahV+PThumnyichHWfF3m4Rm4mP9P8hGZNwIAnmdOEXLjkN0V5cKzjbEUDRW+TNk9Obx+aO1R+DPGoTyRiiPefEqrOPT/DiHl/S1qWcchPzG8MHdqX5jNR7bow/7XwriGzGLCKEZ6z1AC1Ahg/98XH3FcVjpffuvLFNHSDXDV/GhkNW4XKoOpuMZgwCF5lQkLeXa8Bi9YjViYEqylf/JRGPEvXYqZPNVD3htf10H+Nz1RZkKNJ83okt6U0pLU9AUo8JCKdoS1OCepvrmqA+LRoA2KDDj6QEuCZNz8ZWlgk/GyigLfGQDyfFtJ6hM987xiRh+hRpMHxXKXrYsXQSJyRubTZV1VGrr0y+LDTClzUvJsuQ+aTwRnv4FzWB1WRAlC+Cvwy7OYNl6/X/QsVGBBsmcd/ZhF860RIOlWOaFUvemgDX6nh14lyPaOoOd5T/xXnIwF5qLIZrctynYF7qv0HGy+2B/YLyuP0xVj1iwNneHMhvL4xtW7NBrBteE7I+AOj+oH0YLzrU89qstRw8EkuaLOMSpov07rzdjxeV5lzBqCnT4vt21RNTq7FfVZtLLBMYF6H6AxZpAdviTsUIw+B3Sdy4Tp2RlkT9tcm12Z9V6l26t67f06WCTmi6awzcSzOlFIkanfjJsl9OPcn3090AcSJXIYHT3TEINStcA7C0femzzCRjUfg9pWMqHxWlpdpVDsAC2SRQUySjnoXymUfBYOdlR2EUOevdwnqLtNTpRIbJbpwvhXM7wUJjvK/W0ufkuFdOnIU85VQlng2jxIHh/wIZpi+iBoUgKj07GUu1LlsxlFqPQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL4PR12MB9478.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(7416014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: cgzM+UQcnads1cEpigAqjrabxL2OQn8OwZu3TpzIPeo+RNwdbZcMkoUD9sn4jZgsGu1KjBArRor3K4PmyuWnNDirx5uSMSq41k3UjfQa5RPGQEgrTHJjjBGXN6dkJ1/+De8F7Ia9g/Vtv1ottbJT/xmant4lj8YtK77BJrrQrG4JPO/TS9kpaRmo0c2Rv2YmlcrirqIzw1fu8KDQQe6vxfQdVv6tsdmeBlhwGqpA/qK3smb/k9u+w7ClZZnfDZjP7k42OTy+fvvPNdRGkb3g4PyS81l8caRBL1Yp3d1jwkvEpTTdGN2U0IbXKl1scsSVENlNku3Per9BGqfwSufC4mPdpJXIdFv1L5pvjStezzZPtxV/qOj7KuTy4nuUhvl4RtoOvBcXSxDxhuTrJVBjq7LVBhUUj5Vl9HG1lbM/shF/HCeYKVQ9vYv7zsacnwtb6+kuxmD/Dqmk3iqY/OftZsNeIUj1X2xtYjEayV44EuSG1euSYhp69Glpikc+f8bUQRWxW/47jXFK+wSCQtSl/nQooUQ+9/odf5S9aN7dqF9xUi52QXmVc3D6n66cRE/2vihvCQ1JHCt/j0TirywiBUchgRP9TpNGNMttfj8SXPdlNfXM8n5XGcThKbO/ja4X9Wqmwz/UXvfn8KeB2KETaHAIpSQjsDpqy0tTvrfc1R0DPmt3J+Mbc6ZrkcMro1s3zLpvuxfSLoy6TxoQnLZGMK1ObCHK9AFWfkO0qam90tEghWkV7qOGdiuy2JlLuhXB0KNOXVY1vFicyDfMc3QDHaOwYU+cIaHAW2Z+CSFxvNPW08UgWahzlOnN+MiQz1j967nScrmFmQaMGWdWt/JvMDdpJOlPPacdmGQKp6ehk11H5Ji5dzgSY6aYj389ahk74oW3mmRr1fMhr8oflJpKPbHliHqam1spnCrKkudd1adi4VwBiBRunLqyK0tnVzpOPOn1Otq8ZlcpUZlxYM2ysYfNIyUWXf51cKRQePWNlXswOGR4ofSqoxDV6x6pyw8wY4NoSZ+EuRIPI+otVgcL6Olm2i1GqnXSOrZUbcIoYG1Z62tcj09IY3CBmshTjh00dYipBBx5e+pOCdaWoUdyJIKNz5q4DFlZACTpdkwdUpDTdHqrFe0lGbfExD/FD2w1Bjq0bOZ8FPRmFaA5OxE6E4NnrRehBM8MofWiJ2LVprgVBHCyOHl//JxWRCDBwL+d844nJGkQlqEPCj3PAXmg75K+jFYTYoi4F/fgovcCWE/usbox3zoqwaO1PfzWhs/SEVYp13vp0RqaRnXHRNxnTAR/xNYzPPZlEZ9NzEOA1Ug2Kx+tSvB3PWQJEfGfX1moI0fjpWT/aOPEh1a5BlJ8PZZO0e/lFCrRUKVFVvgWHPHzPPf5ysT68kZ/jw42vlk6BrcAJzJTokEZhQFt3MfDmz8S4C+wtKsCyM1r/8tjyNUXn/tBvTEnuqjrfWreo33E2C07dG1n8flf5ucUhh+dkT8/uVehPdRRuaIkRH4pVKJc/ZZDbl4xJV/vqkhi0PaE25qGVCWFX7vdhohfVf8bFK0gWZD01aVhbN6eBRPQFRrziUemRd0fupFv+0CMrOT+ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: f2e236ec-df4a-42dc-898d-08dd14c264a5 X-MS-Exchange-CrossTenant-AuthSource: BL4PR12MB9478.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2024 00:18:50.2388 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 45UrZFgpBllT22CXE3zA7dyNgQ12FJui3eGkM7WtjM/Ie2MxbLL8Ua9DrRFpwH4z X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7589 X-Stat-Signature: ekosrsw13upreb59i1fknco9qxxhijje X-Rspam-User: X-Rspamd-Queue-Id: 2BB9D1C0007 X-Rspamd-Server: rspam08 X-HE-Tag: 1733357941-659651 X-HE-Meta: U2FsdGVkX1/PGneZe0ajKKWftHFF+Kr8uGLhdHkEbH0Dv/N7FhPwSwE9qlQ39uX18tkAjoyvotVw36MAA6viGfeepsALcnYbbbcN+DDyiR1PCrxb9FHyfOH/GR5CPJqTsvYnD1IMFXMyOqFtFX2rJPXTaYnurfhLpS2+cpsdqa/UKP02OGVA1wIRkY4jPxK4KruXQXgUzOIksJkDr8YoQlVNPYQDEaAoVynhK/miQRqwCEShhkC5r4VdUGceizly4uFnUap8588gBrnMFyoP15cPtmfdK24OkLtZ91K/VMnrY/lzmOE9ArTcu2b3rOfylmO0fjBhMpFVEd1kqZeTgczLKvoyPcuk+CjMhUB2fZfqracchofQ0LlKeZsT9UO6irYp0/O6LXVWyzulsgwH7YL9danudzbinn1ogQ/iWRdm2bxkXAr2xnYsUic+Aj868/lIdPbQsHHL4FCTxIYwNHtJUqlUPjHGkEMAdpMb0he3A6JaQyGC0E8zeeB61Ud3+N+gEPvoJmCCh/kCl9ejYtbhmuIr/5s2AYTrWoQ55xO6izR559nr++V3DH9EmV2J9/7zLlMvyED4Ufo9zRRX5lM3oKehs9okHFzZ9AW22a/b4NMKvmOSN6WoLox9XBcGGZ+EKcvTOAY9T8QEu5SOP2Ic/433X7RcwBFOJznEBiUnkoBaMnhXyXA+USNnlBx/jmyxkEBuQsZ1LOWjrD177xZWnmS38Immq+fRE1ZzA6lnDPRZEbJKznsLuYLNc+19aKoaNzrhs41Jbim7VCZLVSXo1redCTPMHAOWM6KnqcDWM0C5VF96snUHXknq8Sh4/4UKal1v3G1rMoiAh2HMHd/QA1Gwfk/SmREGw76Qm2g1RHfouNFmTXuaoLqyZGkw8XThtR8tM6uD+TwRHoCoUAu1O0P8AtgyCb+QU26Ewf0nDwocCIuMN1u6cVGrBQ2f/kSwiPoLboe57TMXsOV sFUSiiry O2NbiH9KxKW8CZKKpeZLziLzHgBYix0RvQO3UTEKzBfYlxdJ24X1YL5e1S5WD+jb8yV0QAxl/GoxewGNyqQrcIBU8RSbeNMR1emORI71KBc37t+hMSuUx0LIfo84sVqvt4AhHnPGfw1gfUacpgeAMtca1PGdT6GNyASNRa2Qnqx/DcZPoPyXaRzqNFBayx1MLneaJXKxaedu/5wIMjbt/4goirP0U+gtRJ7p7/AtTt6+aBmTsMZ980XezkkHq4Z1JzBr68486qItVP/D4l28SLX4TdeHrVIaPpsOHEL+Ua2ChNWlRSwpuwtrO+1cS+m018cWtDE9acCn2f6fkIIZvek1U0xqwa5Pqe3C8oAaHl19LkDfyTwojlC9WKBdWWIFJJjhKrtyfo3LZN4c= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Instead of splitting the large folio uniformly during truncation, use buddy allocator like split at the start of truncation range to minimize the number of resulting folios. For example, to truncate a order-4 folio [0, 1, 2, 3, 4, 5, ..., 15] between [3, 10] (inclusive), folio_split() splits the folio to [0,1], [2], [3], [4..7], [8..15] and [3], [4..7] can be dropped and [8..15] is kept with zeros in [8..10]. It is possible to further do a folio_split() at 10, so more resulting folios can be dropped. But it is left as future possible optimization if needed. Another possible optimization is to make folio_split() to split a folio based on a given range, like [3..10] above. But that complicates folio_split(), so it will investigated when necessary. Signed-off-by: Zi Yan --- include/linux/huge_mm.h | 18 ++++++++++++++++++ mm/truncate.c | 5 ++++- 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index b94c2e8ee918..29accb5d93b8 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -339,6 +339,18 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, unsigned int new_order); int min_order_for_split(struct folio *folio); int split_folio_to_list(struct folio *folio, struct list_head *list); +int folio_split(struct folio *folio, unsigned int new_order, struct page *page, + struct list_head *list); +static inline int split_folio_at(struct folio *folio, struct page *page, + struct list_head *list) +{ + int ret = min_order_for_split(folio); + + if (ret < 0) + return ret; + + return folio_split(folio, ret, page, list); +} static inline int split_huge_page(struct page *page) { struct folio *folio = page_folio(page); @@ -531,6 +543,12 @@ static inline int split_folio_to_list(struct folio *folio, struct list_head *lis return 0; } +static inline int split_folio_at(struct folio *folio, struct page *page, + struct list_head *list) +{ + return 0; +} + static inline void deferred_split_folio(struct folio *folio, bool partially_mapped) {} #define split_huge_pmd(__vma, __pmd, __address) \ do { } while (0) diff --git a/mm/truncate.c b/mm/truncate.c index 7c304d2f0052..9f33d6821748 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -178,6 +178,7 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end) { loff_t pos = folio_pos(folio); unsigned int offset, length; + long in_folio_offset; if (pos < start) offset = start - pos; @@ -207,7 +208,9 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end) folio_invalidate(folio, offset, length); if (!folio_test_large(folio)) return true; - if (split_folio(folio) == 0) + + in_folio_offset = PAGE_ALIGN_DOWN(offset) / PAGE_SIZE; + if (split_folio_at(folio, folio_page(folio, in_folio_offset), NULL) == 0) return true; if (folio_test_dirty(folio)) return false; From patchwork Thu Dec 5 00:18:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13894610 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EEB1E7716D for ; Thu, 5 Dec 2024 00:22:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AA7FD6B00A1; Wed, 4 Dec 2024 19:22:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A59656B00A2; Wed, 4 Dec 2024 19:22:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D1906B00A3; Wed, 4 Dec 2024 19:22:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6D7886B00A1 for ; Wed, 4 Dec 2024 19:22:20 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 20EE712127E for ; Thu, 5 Dec 2024 00:22:20 +0000 (UTC) X-FDA: 82859003010.07.F2D8F94 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2050.outbound.protection.outlook.com [40.107.243.50]) by imf09.hostedemail.com (Postfix) with ESMTP id 23424140017 for ; Thu, 5 Dec 2024 00:22:06 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="E/CEDCJ8"; spf=pass (imf09.hostedemail.com: domain of ziy@nvidia.com designates 40.107.243.50 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733358121; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1lwTVNAVLwi1gyG41osM9o7VeGen/aJvxEZw1mntkp0=; b=H109Ln5dUgJTlkLTnf8dnoiW42nqu/mcQthXOO8ASVQHTvFiTofiaJonF8a2/DPaanypM4 o0N3d27yKFEzvetHskhwEPc/cx66WrJygUE9qrSPEKMS47dp2Wzidp2V7JfIF5KJh6ZzJN iN3qqQzARAeOoUzGKm8gaAmafhILAJU= ARC-Authentication-Results: i=2; imf09.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="E/CEDCJ8"; spf=pass (imf09.hostedemail.com: domain of ziy@nvidia.com designates 40.107.243.50 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1733358121; a=rsa-sha256; cv=pass; b=nFgd77+mtXl17x452jtNGJQ69lNXSBs0+29B46Ohn19XjFnEsHBT2PlvwOzxc1DPYOTmUH abmu2sVLhXk6wuPqYPirdniJTalG5KMKTlC/bKMxhdi+e+gDeYFFXWyXErkAIz2UXjk3nB 3rst2Lq7SJWzOnNv/wLiM+s76QTgJJk= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NbfzLZsX8Q+jDDOh6uEnOU+DG4B1JHRJFHn0vwij6EFymekfceBtHORnz32JpXJSMs8z/Bhf5qAtjan1/0nvP4UkgZcdnj30Q5Cjp45JAwKhHzgIG5kR4i4KCWTvR0tkls/VVp2xYrLiu73HY8ywL4yoib05rWIcslK6VbetJeKbSsrgMR4Fpa84vKHHVXcIKsYMzo62bAapcqoiK0KyVbTUy7KrEKNYnX5gjN29d3/3TPPXPGzx8ncXOeaIn+sUDih+WJ7Mmbfffm4DF2vArNiWkB5Fpo4jSIvk6tup+fmbwWOr517S84j0HRC/dBJfVJZ22OepbBfcUmiNKU2acw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1lwTVNAVLwi1gyG41osM9o7VeGen/aJvxEZw1mntkp0=; b=t1KoGDx2lWHh9HBvy8ZG3X9/U1xE6hyYcZyetorJYhYHotmtFfq/bYRRx0LCMiqcDttMYS6IceEHAUGvV47hukFt0Bf4j4kZw1yYFev6H3rEgzZfydrmevXvZJZsxocZBBs97P1Xx4mN3OQLeraGKvb7X0TP/dvk4lqDFOuZiGLB8VH2z/V64DPqmAU9NECf0w12hTeLxibFm3FQuDvDt8XIWw+bIKo4O9on1Ho1HMlsiIoq81a16QHFKEK9YQq+/GPey085APyEUUSGWAecSRGpGleoHMVJFzw5J6qyVtEhQfcnLhDgRcOQVLVZMCTPx3PqI8YlT2RUHKoOrMWDmQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1lwTVNAVLwi1gyG41osM9o7VeGen/aJvxEZw1mntkp0=; b=E/CEDCJ8Mpv1c5FJSBZnYgl23WxGntcmd/x2t3/mBBoCRpzzRaqEag/up2bqVCh9EO81uY2zREP+g0tPD5ntr5jaU932XNeyniiwBNpLNBSYgyWuhqI/0VSSi447fIXSJegHZ0V4ap2PiJerX25ukiA0+RDgMYMJOJWWyYBb8oflOT/opuFzE9yBrDjNAOaxpgiSdKuVd3usJK/YCRC4GHTm2qfzyDIg44F4DPfr1UxYO89zwPn18b8dhmakD9k9qPEIxpodPBJHDxnyyk0p1mY2Yznj/sBDQ6LGduIGgewNqQ2Vczx16r+POceeg7gEWK6rt+mNjUM7RDOISzKitA== Received: from BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) by IA1PR12MB7589.namprd12.prod.outlook.com (2603:10b6:208:42b::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8230.12; Thu, 5 Dec 2024 00:18:51 +0000 Received: from BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9]) by BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9%6]) with mapi id 15.20.8230.010; Thu, 5 Dec 2024 00:18:51 +0000 From: Zi Yan To: linux-mm@kvack.org, "Kirill A . Shutemov" , "Matthew Wilcox (Oracle)" Cc: Ryan Roberts , Hugh Dickins , David Hildenbrand , Yang Shi , Miaohe Lin , Kefeng Wang , Yu Zhao , John Hubbard , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH RESEND v3 7/9] selftests/mm: use selftests framework to print test result. Date: Wed, 4 Dec 2024 19:18:37 -0500 Message-ID: <20241205001839.2582020-8-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241205001839.2582020-1-ziy@nvidia.com> References: <20241205001839.2582020-1-ziy@nvidia.com> X-ClientProxiedBy: BN0PR04CA0089.namprd04.prod.outlook.com (2603:10b6:408:ea::34) To BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL4PR12MB9478:EE_|IA1PR12MB7589:EE_ X-MS-Office365-Filtering-Correlation-Id: abc82ebe-cb1b-4d9e-9500-08dd14c26566 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|7416014|1800799024; X-Microsoft-Antispam-Message-Info: nOI9qPciSE6Ax4E3KhmQJgr9XBlzJdscRYTQdjRAjAmLFZRg5gBQIqPJiNLM/5PYs0s5I/1QXNsJLVmkgiyPF5gJ8QctkGx8ovmEIZHVgANcqKo8rhCjr7AiG3YOn1BVPo27XJ0V9E0wIqgzgzIpsQaMY+oNmrYbrUWerlnxm8Cn4HJ1mkxGj9XuzlYTKVbMemUEe4XuMJRCye8LI2pdHg6M15T68TNNFOhTBRRE6NQaCqpQBTDBkMx5p352H/WuHL4RdxEu7V7aINKur2WQLK73KAFDzyg74aS/Ur41ChQTSexL1GukmG/Zkfd9VzLzN2I9/TetCRZFTL+de6U9PZbth++LeaHy6KtNXVLBYxzBKqUmDkN/9n8i+b5+1bFTcSYiG0OHksH2+BpsvOWA/fLmw8cQVGeID8V4uY7ymdMojFwuovsz8o+L5HDKRbWUmlewmOJd0V2w22T0piOiX7kTfNVIwPJQCxcrYK2NqVvV04+LgQ0K4PmxvLQGcOP9czDjKWx1f+UpCfDh24kKzSOL99baO3I9dHXwx2RpmuFnKqtil5tBGVT4rq/EKB/As/I/s5xGocLixawF92XFxjunlb69KREgxn5fRgnl7DGi9Rw8wyonj2t2W4F4eWvaYRsgZAJ61Esprh84FiT78h5cIi5vO6JrTDzkbAvmmgve4PCZt3nLF6nvlH8LJJlZ5NwRaIFf2nYwswGustJfhce+SUCHl1g2RdWOPaf7GXkZvmWxq8lt1MR0u3KpEBIJbCUPCnLUP+/kGAD4j0os46QJwaYlwUGTJuPZRp/+0SjdwAWN8RLZRwv4C2AP32atRqEwmFi3Yfh7Nr8i4yJCLIK5mkvS7M4BK21Fr7eq3FQ+XOU0UELTHb1S6nrfHNaF9fZEC1lOBSc7p2xPssSAKTYHVLHGd1id0AUfuXIiAaWjWW8Qb+nAFCOee+4klnl1pqQeRpZAdO2oAb4SudHTK7+prNAKG0x1Y54GV/D0nLWW2FLrysYIV3UxaT+zD2/cPkUbVZxJocw/S7XC5698S4peRzfTmwSJ69KdmuQuXD2qQqD4lO4dn+8QYlTPTnkoCCZXlbLWoyuQBQT9i0wwbyqeH3o37yUKyzEnCjnhcM4DR/nDBBgGQVFo4C6//FP7nkFGS/YVbsI4MMghcB+eOyVk06q3ejDxCKjeSW+oMl+WUcQeu2XPLmwYJg3m7O9pz1wRRB6GDMUYoulKuOu0eE0yk+/JCGXywbX+EZcWBWHLTHIHmPAbi9+LCdnV9k0RJAvYFj3la5oL93pX1uxS2AvYQZdU1C6qBpjdZswmk6Gg9vpR8lbWx/DuxMT9QCebDM2D7ZYEWSj6aFPH0z933A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL4PR12MB9478.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(7416014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: B5vCxJJSYYVqDOnAWmSHa7qFuwFjiW1IwkzoosVU65PrJqbBBKYsnxEP9QryA3+ey6NYwP3T442SeUBEL2wMd29trs7eE6eLZS3GS/14iXhLWj8agsv0QN2ohlvDuPAtWbjp/4UhAWMZkVX4utn5g1p0iP3TJdLNfGgEdWB3yIAUVDt4JDGJsjuwCTRKoNAqRBewCVsmhnPtYemUkz7FVGa/3pJ/+6/QQuwwSzGNsz97I7jB7dO/sBD3trPrBUvs7QeFy0gppmumL02PMIQm7hZ7Wk5coBPSTTZq9xjbCT4/hyMIJjSB4jZUUWWSqTJSdMdeiUz4kJf7TTj6CWUlEjPSKlV0QyTxrxMeK3Wyc7SoNAJbk6uHDhFXDNbOUimKclHaNTRpdpNoJl51mPoY3r6PAN55ew9vGRplF44weIwkuEo9wx2X2qWFFeAuMcCdmg4JPnFT9R2cevC2iFi2rmlKJUkeEyV8C+fDYgxDtXEySKFVDc3pFQFKjhk6AsvyNDPWTOj/e+GzlFWAbyHUb9RBcnOwhpbKvJ8nAefAejWz+323fBCV4nKDVjDUE/9nDOyzTaxQAJ8puQsx049TB8IbVGB9c6QDp/gC+OyVKiiAbFIxBxLe/E7xeweuoNA4hSpFv6p7nFAH9MDRr3XkxvSwW67c3brJezSdptoHrFVkfQTCLRCJ13xA9mAcOpccv/H4ORDeU7H26whl3B2dpXfaL4NK/r69x6j/HrqI3nnmTrtocvmc8pyM2P30DGtpJkmBJhdomi5bO+0Adp7ziWAzC3Z1YMkOirkplmRWPVSgRmPNMbxk3gus2uGjAQClCRFX6sJbdY0wwzjYwECPPdgnTDJyIdAOGwAMhtmEWYbNqj/ocex4nMw0uaMWoMKGd5MPYjghMoUBNPKGjDqx88GknySW/RVGEEsbvvgyUsnR72T6dvFmpx+fqmnmOF/IlSJfz1Fzu06GNtjtiIx0SnrdqRjEy1bLCDEdO/xvhV1jCv01SZumWymODQfNZq+6i3xNOIb0ArRAjMODNt243oBdXKaG3XCwZP/d/ntwRHjsO4+RU5sZckoPCXoJKjJZFJA1ne08mYUZBsmNtAzBzahN8E/Q+tkDLDSmuPpStLQeFrEpOXGkbgdtC8HzT92tq/oD/FNAs7jUttsru0pW9LIYSZWBgDVTIyZt5Bz/hs3p5bBb1YO5OChoDHYOBV6gGhmpzHe9vBdB7WH0A6QYr0GA+99jr+PGFRJPQy90MSnHYFZA/+yN3xwHKYKEUP3TaLKsNCtZ/HCOcf+VUnqeapU/zLpsfMyJJoDbxJShnt1V0zeP1CG3YbtsYQZ/4n9tGbMF5tAn5uTNZ0YI7CFb41Tjh0FFUe/aUOscqKaHHiCC39tJHJctNcajIup+GlsUH/GD4yDVSl4tPsjsxjc9mi0FcEB3+rvs2Hlbm9LV89q/h8O79hIflBxvsWjlIlYNLgsavHjol4qnqwSaOLMB7kb/W5G5t1zuce7ynFqQgjTtACuEsgBUt8xIbrUCqjwAJw9e0EseupTuQdNSpPPtrRBPrRYV0lNTkPmIastLWta4hsKM4qoWWXo+bu5wLBua X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: abc82ebe-cb1b-4d9e-9500-08dd14c26566 X-MS-Exchange-CrossTenant-AuthSource: BL4PR12MB9478.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2024 00:18:51.4678 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /Vc4AQyCBTkQqPySVU7RAzV/kLcSmkKLDMB8ttIk3NZDJ1bi7Hssd1nHHEKOWtOF X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7589 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 23424140017 X-Rspam-User: X-Stat-Signature: ri7cw84uxe4hdp4oq9cui5hojkjr6r8r X-HE-Tag: 1733358126-564807 X-HE-Meta: U2FsdGVkX18se0ASXz+5OqbI7VTIUxqSGq6Far2WYQZDwoXC+cZahtGzcZdgMJ3pJiC6HYekSYpIRMHVIAg1ZXfzk/f6W4sbjlURKIDJU50Rq0ruuRi3sW7yEkRG39HHWHHicQqfAlwzZUE28C24hHyPgrWVdZSWYC7yq6IEdb8V6vi/3hjh5gURd7QX/oRXpWMO6QTvKa7vYpg9UlHKnhYhSZrZCuchEsf6K8sRPO4MZTlFvYV/dd9iXc8j0Iw0M5sYx6p+8ovjLynONcE8R5codPK5BSB66niyOGTKk+bcnDHfV6YlxOwttynX1BGNTlvUHmMJuQw5JG6ywiQAqIfzCBCaAza1ZNjcud8uayBzKuGppfj0dpxOLjGGK2YrT7J9/xDOctKHIXlBkaeSFVCYHWR4IQ58d05y4VuiG5nr0DydG3VGAt6ZUi72TNxqgXcz5FC58IRAIyk6/io11RTwgmMO6xEeSuv1g5jSddumRGNBvxOh1OcK2aRnt1ahA30n5W/diSIDaxT7pnPBh2DpGM73Z/c0+es0ajO/mWbOzkts8zLAmAp3s/gp/bNniqRYGdy2aeyOOqedc640GPCsOF5fagbwDKWs6CrJbErCMRHVsrlVzSFpcrDZtc6WmkkUjewhkQfe7E1oqPGfJ6+/+px43MdT8yqNELplnBw+aMMbslofiYCowDxjrZ7ufDQKp8hRLZnOoUO3wc1NXywPVa56DC7ZboYPHZWItqJBII/oYhsRSf4rrDzR7wKo6clpzaD38bvdBSBz1QKeqnPDlMJ6jV471oTcfdB1hiHWY8GVRrctM/Bi1yRUC3/xguKBm89BkWcGaoXGxT/lyamjjzxtxPN3okl/M0fwb+afRqdhxqPBO1VphsNwnyWaoVXwRyvkBLq3LLIe7d+jz3VtXk97uwJfQc5V0DAVkSzSHkWcmpEOy8Oz/lsUvQjTaihReh5H5MmsVylMgfd oX+e9CsD YeiYfReh1cvKf0NuLZ83oVD8aTYFOzUxR3EITomsoTAPzVaS3Ju78818bEnUsHTcuXt5xEfVDJ1012WH92fnPlXS4D9fIVc9wQ05yMkF2o7Mz9fgp4cCM3AtjzaqxZtT7cdKkPtKE77eMnIT2TLmPU0mpsXNUaXq4U+KbllgTDn3nwbWNvEmcOxYy9Vc410tuuXy2uJNpcMOwqo378TdhrlE5s4VPdZg3NBCeUNVUj8TW17ZAvT5zX4D2ezMAQIL7MIhjmLYbNTjSl94Cyxog1CI471A61mMWEBMWwL3aePqHwNXUWvKTTvyFXVetwyjgU5UrgAnJrcNphjeRLYHhZQjsgQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Otherwise the number of tests does not match the reality. Fixes: 391e86971161 ("mm: selftest to verify zero-filled pages are mapped to zeropage") Signed-off-by: Zi Yan --- .../selftests/mm/split_huge_page_test.c | 34 +++++++------------ 1 file changed, 12 insertions(+), 22 deletions(-) diff --git a/tools/testing/selftests/mm/split_huge_page_test.c b/tools/testing/selftests/mm/split_huge_page_test.c index eb6d1b9fc362..cd74ea9b1295 100644 --- a/tools/testing/selftests/mm/split_huge_page_test.c +++ b/tools/testing/selftests/mm/split_huge_page_test.c @@ -108,38 +108,28 @@ static void verify_rss_anon_split_huge_page_all_zeroes(char *one_page, int nr_hp unsigned long rss_anon_before, rss_anon_after; size_t i; - if (!check_huge_anon(one_page, 4, pmd_pagesize)) { - printf("No THP is allocated\n"); - exit(EXIT_FAILURE); - } + if (!check_huge_anon(one_page, 4, pmd_pagesize)) + ksft_exit_fail_msg("No THP is allocated\n"); rss_anon_before = rss_anon(); - if (!rss_anon_before) { - printf("No RssAnon is allocated before split\n"); - exit(EXIT_FAILURE); - } + if (!rss_anon_before) + ksft_exit_fail_msg("No RssAnon is allocated before split\n"); /* split all THPs */ write_debugfs(PID_FMT, getpid(), (uint64_t)one_page, (uint64_t)one_page + len, 0); for (i = 0; i < len; i++) - if (one_page[i] != (char)0) { - printf("%ld byte corrupted\n", i); - exit(EXIT_FAILURE); - } + if (one_page[i] != (char)0) + ksft_exit_fail_msg("%ld byte corrupted\n", i); - if (!check_huge_anon(one_page, 0, pmd_pagesize)) { - printf("Still AnonHugePages not split\n"); - exit(EXIT_FAILURE); - } + if (!check_huge_anon(one_page, 0, pmd_pagesize)) + ksft_exit_fail_msg("Still AnonHugePages not split\n"); rss_anon_after = rss_anon(); - if (rss_anon_after >= rss_anon_before) { - printf("Incorrect RssAnon value. Before: %ld After: %ld\n", + if (rss_anon_after >= rss_anon_before) + ksft_exit_fail_msg("Incorrect RssAnon value. Before: %ld After: %ld\n", rss_anon_before, rss_anon_after); - exit(EXIT_FAILURE); - } } void split_pmd_zero_pages(void) @@ -150,7 +140,7 @@ void split_pmd_zero_pages(void) one_page = allocate_zero_filled_hugepage(len); verify_rss_anon_split_huge_page_all_zeroes(one_page, nr_hpages, len); - printf("Split zero filled huge pages successful\n"); + ksft_test_result_pass("Split zero filled huge pages successful\n"); free(one_page); } @@ -491,7 +481,7 @@ int main(int argc, char **argv) if (argc > 1) optional_xfs_path = argv[1]; - ksft_set_plan(3+9); + ksft_set_plan(4+9); pagesize = getpagesize(); pageshift = ffs(pagesize) - 1; From patchwork Thu Dec 5 00:18:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13894608 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73307E7716D for ; Thu, 5 Dec 2024 00:20:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F3B0A6B009D; Wed, 4 Dec 2024 19:20:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EEA236B009F; Wed, 4 Dec 2024 19:20:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D20C96B009E; Wed, 4 Dec 2024 19:20:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B01506B009C for ; Wed, 4 Dec 2024 19:20:53 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id F2F391412D3 for ; Thu, 5 Dec 2024 00:20:52 +0000 (UTC) X-FDA: 82858999482.23.12DE288 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2083.outbound.protection.outlook.com [40.107.243.83]) by imf11.hostedemail.com (Postfix) with ESMTP id DB6CC40014 for ; Thu, 5 Dec 2024 00:20:35 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=eFHdjs+R; spf=pass (imf11.hostedemail.com: domain of ziy@nvidia.com designates 40.107.243.83 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733358043; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FZxMul1WU308N00KYNlcxD/DyDEr6jnTyKBgAiwblKE=; b=vr6C5TlRhOaPP9C8pfeW+CQLPkN/FcuscEyLlxGwZ0H/z1uqOWzIagJ5SwVjQboW00dIcd zLap1xyL08PP3VefLgiuBYa9uFGJga88d6n026YD5Y6wh4aqpTOWyofxzLltlsBoExOyi8 IZLjnhpHcsv8F9gM9YNl+2jHwL96h+c= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1733358043; a=rsa-sha256; cv=pass; b=qorNX87lTfeKJ0rJd03+C66WtevjSOY5Izup9p94OOx2IbMXO6OQCNdvnJRBwnV8RgRHVy yIR0s59gefsPFF6P3tAH6NdHKFvSCY66B0tZcNHLfw9+hKxX4YptoUEMf0ZY0+GCocBDHP ztXdhgMwxGbvoUIWYfHUeRa0sa01XyQ= ARC-Authentication-Results: i=2; imf11.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=eFHdjs+R; spf=pass (imf11.hostedemail.com: domain of ziy@nvidia.com designates 40.107.243.83 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=bqtA2kC5x8kCmRjmBrPZpGYQYbBmEeEPpzufMAnjHz5waEzkopJXpm+wBjvC7iwcvyq1r24RW/TU4hoaMnmolp2y4k6fT1gT/WD0mt5Vt+JzL+pKXwe/uyaK6u+JFbMOMuf2zxOEgUqpElYGpOXzexDiYDDg5QCI3oaV228cdYJtWVWVGyk9nvvwy9M98yjtAe7q7DA5HVM4WHAlpS7yVDPKcgoujCf7ykUEFsAhwtOrHEt56HYLVmNPzRZc16NRpKLz5YDh3bhcIcwN5eksw1q7Z7EC6QU1u8kCgtdY8l2kGcgJ5L90Is01Qg9i39k4od6a60qa2nvlcKtcDQxbuw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FZxMul1WU308N00KYNlcxD/DyDEr6jnTyKBgAiwblKE=; b=psvpCPFnO+tnf9Y49zocp1Ogdh9VmArxS6Mc0Qgl0dRKXZhfIsoRlv1pREYlex7ae2gobW11Ip5yojbZ0MIRr/A9OkSwuvFOvX4NpjQL/iwE4N4H6S2pR2OBM2TXAa9vl0KCV44zVtQW4DfOeT2Kf/xH525AMC8vIqDY3lHDyKvjqWY05fVmd09RMvpVuvfoLWsxGSmKNW27xNrH0zGTFv5OyO8SjcUB7oCFjuVE3VsY0uYDEernhCQa3yyFiE7EDNdY10XTzCD1SNrTVuXGjYOnfQ0h78se+E+B5bihIDC/aAZoZW2RGgn1hfaUI4x7FTay6YVViDaxzaN8EnlSTg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FZxMul1WU308N00KYNlcxD/DyDEr6jnTyKBgAiwblKE=; b=eFHdjs+RZe3gDyB1rGV8cXic5AZvT//3HL5kVkRhQzKGS3gyCrZMJ7fcL6PUZOxE/SV2wPPCLKl3uy2Q1rUxIxUGu6+how62O8q2udEHQpqIaL7QDF5AG4mL4LBRE17O4fQmnABrMOhRpgdexC7+aAb0Dc4NdO7wQDWaRmtZT2F95ds707gA7nizj0XB/cNEpjTuca3Xj0ViEdFJEZHsts1az8dWlp4goMG8JVFczA+pYa5b/D+2L4EoMTdf16pCMTIYHdpDWJ2IA9ho5ju+6E26ndmchRTumKtbNWnD/9LFO5QBZosZR1FYp86gEeCuIWpqbfg9Erf7/t4TtV/CUA== Received: from BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) by IA1PR12MB7589.namprd12.prod.outlook.com (2603:10b6:208:42b::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8230.12; Thu, 5 Dec 2024 00:18:52 +0000 Received: from BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9]) by BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9%6]) with mapi id 15.20.8230.010; Thu, 5 Dec 2024 00:18:52 +0000 From: Zi Yan To: linux-mm@kvack.org, "Kirill A . Shutemov" , "Matthew Wilcox (Oracle)" Cc: Ryan Roberts , Hugh Dickins , David Hildenbrand , Yang Shi , Miaohe Lin , Kefeng Wang , Yu Zhao , John Hubbard , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH RESEND v3 8/9] selftests/mm: add tests for splitting pmd THPs to all lower orders. Date: Wed, 4 Dec 2024 19:18:38 -0500 Message-ID: <20241205001839.2582020-9-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241205001839.2582020-1-ziy@nvidia.com> References: <20241205001839.2582020-1-ziy@nvidia.com> X-ClientProxiedBy: BN0PR04CA0075.namprd04.prod.outlook.com (2603:10b6:408:ea::20) To BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL4PR12MB9478:EE_|IA1PR12MB7589:EE_ X-MS-Office365-Filtering-Correlation-Id: e7c2ee51-40c3-47b6-a3bd-08dd14c26606 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|7416014|1800799024; X-Microsoft-Antispam-Message-Info: U2BL0nui5mvzM8czRcS4cP5DUQzQ5dM/JhuNsRgmHyYtq7inVqUq6MTDUN9hd5e7O7cu0qwZtHqxWQou3w7NX3WZLzYVNY2KRUHwUklSHDQm9zctkgqwcPBisbiYjo/l5wHnSoZVO1+s2AHfkeMvRr6bVBCs+wO3Vg9eHJh42nru8rluijdC1BrxbCq7ZFDOMZ4fP3Jqx53jINwHE7EejYOuo7gafLH8BKQFIuZ2TpgxOxHKUtpiHgmxPlc/NCzARSXHGqGhjchKmRmYTs3N08xGMg1lJSeJlSerZfRKzCgvNt795Jog/4oDMyWPAhoW35p7wJ7MfVBko0gV4fdOsC8tf0p9q4YlrcYeDx6lsmFBs998jSw+GEdrDf9VLl9/UgAwklkz+pBJryernXwYpzDLeprIsbFWOKDj4i2zg4Gc/H3MXfYJy4Fh+dUiAfWrHja7MJSMP5scQkbMGiVwousiEmPRiHeMILmBbxOiRaO0m7xv9FGJ/yaVyNJGDedsXdTwTABvs33bCjIb07L2t1IH8IkIy/J2c94XdWF+i+YHwMzAOF/TIu+e8QMjAq7+53ZA1t0ogjRCqCCJxgWiK7v740MZImWTqe3VfC1UJmmCIa6EtEN9ebBT8T5w1nVXj7T+l6kwPPg3ZSPzvcB1EVR5soZP5HLAIYaRIueJehI26w3dHKQJQZ8xF0/H3uaBwr0Jl1AOc67jg/cn/RwDwZTCQRkQf5z+QgtnBZPLaJKrHPWlBepXYn+KknBAHK32r89USkD5Ifd+x992/Rhppdfu14SfpsU2O5NSL6Jvr+772+SnrFos2gFwBpHRbQKMG1QEZpONzL/pnompw7q6BqNrcfXG5G1UvEbhh8ZXMd5Ntyd4KzOE4gZSFjnTBmdoL+zMKqzmkFsLuKyYp1N6zm8JS8bWoJe0WnONPon6Qi5m/of8bkeVmsOJ1cP3ScQzWtKE5qHUFDTFi9r3rM2gpJ2GIQbCBKHQlSumaYdzbngVWfS5I2OGBZlT745gRRN+VrXlGtaBwy90siSuUX/Oo8DF8MD+W1KD0R9Ih8ih5DnPM9RWTLDwT6cHaptRGHaR8LO3tfDdkgnLNQ8wyCvTBHZN7S0QZOdIbMmiq/xiLTFOfmcxXi8nflWYSRbZa5AekYU7C2DWmfFJZ3rXSE43+AXUe0Tv6RoiRZpZh17NPYQW30EsGOYEjGYiMsRpb125thHoL6mXGJlUPdmrXiFvG1+pOr5+xi32uktXwzvaQkWvTk3hQsLxi7QdP9HDJI9XkMdfR6kIXRSK5xCRTw5CyJ5/mI8+rIflrognT/2M5Lztd30e0Qnqs7HskzQba/0qabTW/IcGdyO/ik/4iVsTCw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL4PR12MB9478.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(7416014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: vVDlrDfsJsVE5jB5WAkGQPNomvbfZFjcKqldYjCwADcHUlwyCKk1aBb5XX/dIZ/qGyyHZWk/ypmZVnyY+eXbVWk5femJ0MT1+WZy16i5yOD6UZy0ju8VAsMvalq1Dm/X8/zZfnZCGvp2HLkcnGKe09LWmS5KgLitFyFBt6BED9jSb7odIKLi2bXpITs/LHj8N3h9Y/6wbIihfvh0uCUMv9ex6cEOIITI1bYnNsW6vH14NKBb2xFt9veFX9vivqT3cAffVJQEd2AixWEN0d4jT6xDahDQcujx/e2z2spOspjRRU0qa3Vo4eYwy4LdNEpgWkeDMc8AAHGYKrlLdMV5jxaK37Ikeo7hHSLDUYIMiWMy44OjsDdYCxF0v5+V/aLGTqdiXivDvoRZQLK9P2BfKRR/cjTZdcBsJiAL2fyiEJXWYENPJQQyxYQrAST/ihPqVF0T/HlfqGeYeyIfap9QDFBT7vGHViAK4c16snTZDNE7ywM4v0eDOxGCIUrsaFJp3ovDAMTHQXGdZzOvaf3+w4PIK3uH1562zHErQIqd2qYQUIuDvstqcv3zJsJYir11PN259ZXvN/1/aMhA9WVT7DJxreNPIOW5BS4TiN28iUYfGlFrDgcGSWX3OPv/Ye6+pzf2nVVkhHEfwKNC7lq5CVkaryhPA+2etm+TuCUsB1ryUMEFqSIWJ79HDpk9qr/uJqooxYEJit/3yXrEtch3ezRmRRovyBW7B29mn73jk2MOgycneyS9l8vDt5+ZpBIGhUyZNSxmzNLhzAh5Ul4xUodSzzgiNysqd3mw2zot6zHnX708aFIFogdBnd3dm7ha3Fjzj+K9ykTW0peQ7Mi+9VaIyTWQX748ejoDVtSW94fcfrWPhNvswuIEkBfAK6ze6W50L2jQI3+MulA1E+xOv2SuJA6aYvEpzu+YlibBFIVovPwha303RlktnuZGbSm24cN0sbKcrkfFDGzxmWDhqB7HH3CJq8ZOwnWZ0ansd7VAI/UfoOPUgRt/GDG7khi1pph06RvFzSiJIEDyNSiAVrBvZkDo6NMPehVuiJnDDXcDBiG8QT3M2T1lnW8vFVuNF15+mGK9khUyveTMPk/aFANRJ0+qyRnG0gVs28ULvFzhAUGrKxq1Sn0qFQV5IcznO903mrbWTQMQDpmYShJxlQsbeketQB2z0keHuEjCJA6d0SEwLMZl12GdLTjon0MbGE7kEtsPPqKVm8Im+MsKh6rKeL9syh4Oo11lRuG0l/pKKwdaHLcEGNR7PvA80xycvKiUb86PtIhO8HtaAOuGadrEqqleCdEaNz4ZxhhLACHJj/UOG0Lcw0Q96yFECvbbN6dx6Vd6qAVIjPxYiVPaiiWF9CBgJHy4qC9J1cy+A9cW69LTAMAw11vrdvWS3KdIDPpdpUvHPkoX4ZAmGJYfjopr1dKQVWcseU6+cq2StzGsMVEBQGp50uhVXJ8s6M+hurIeHQ8piUfDCkWsoywWxbQ2RqvDb/2ranZUoBH1CodQ134p9pjCQoaPT0daMgmoEUeDxOrQ3cI9djoKUxlSxXgzU90E1S2YqPifbxzhVrd2Efi6RE52rr0SFv0f89ed X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e7c2ee51-40c3-47b6-a3bd-08dd14c26606 X-MS-Exchange-CrossTenant-AuthSource: BL4PR12MB9478.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2024 00:18:52.5388 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: AM5wsiLeIf/f6wfb5R7a/NnCXOti6ztmCm5r/QX/ruhKduvJtKc0x5bkfB6gEhKe X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7589 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: DB6CC40014 X-Stat-Signature: aj9qa91pfowe9s4z5o5n7k6tq3cc5mdh X-Rspam-User: X-HE-Tag: 1733358035-500069 X-HE-Meta: U2FsdGVkX19ATaUHs0562wLzbKtk0n5EXZjP/RHrJNSksHFy0BuMcp93wbcjql33tnN5FvxVlECmbO62wpZDThKxm0LbYzdS7WvNPxsgSWaoQDJgE6/8Bp/GJSsmtyYkDsTyOV8sLhjfO6J1dzRPxZQ4EJBMRuik/M/LtrfKpV37L2c8LIwmwCVhze3yEmdtkE8gRvaSn78QSSD92k3WEx0LYDqzn3BTr7DJBGqB5Mu611aN2rfT/ZwXhIn3gHQs+f6U1GHOxFnl/WuqA3rp9eTabOJzZOJo87toz47I/ak5TOqhnXdjD1p84MPP/ypvC7//IaZlQ2xYxcGrws5KD0Sf8HerXGaDaqL8RM9iBnwXoVP3S2HRknr6qz7vknv0LrGq2fbYUena65VSz+EhwAgRPOb3/+ee4BsuTZFxKzLylQVWb+4ac8Sbm9ezqRuIplQHlf8nUbgJGt0GFr+IBKQIjHXnGk+luAFTyGjHFjeHN9IovfXettx4HOZR6paXqheubiGiPlmrNlI3MF/tsnqtOVXxB34ed+maf+LBP1ez8py1zRg9znch9vLcv0jXg7Jmc7UEKnG5J39a75XSgjF0/c1fIu+Pzn1qlUl5KvDpcHks2haTNy/vRYAVV7qj5gR1rGZQrdlL4tWQaRrMCjPv5ot4vYKq79fRb+CWjIWzRm0P1bk+54OzkdRCgRmn19V6ivmh98+HtbC22FwBQAMPP7Mx41/HpME4fa5dWpKjjgEtSMbxQBmGuzZIu7w5C9ENTdb+SMtLGvafyy0bHnRVTl2mnNI3dkaIMwwrXyY6ik5GLJQTuEyu+XqOmOq4MAaCGFBBKdUBXl6a8E+gpGqzdy4zIvJ60mGwo4Co4bHfKoj82Zl1EBwPrVMEY6ES2Nq5PNqcBFa+4jmL/AooFs7nn3phRIv1dmSkIOzGheCk6KU0eDFPn6UQcLKILiGkRGsjCGOSoNio2asWhHN YXVsorsj pkpBnmjICbqwIS/pVHC2q6aQCzKseaNDxBCNWVJOWwx3la/nNOhy0+oNjPpWyYJ/4rkAAuLA/bYWULtg0O2ye+Lr52ZpqIFXmF+eWVqMwPlmhgdwUAV6qaauEPn7Cvf7GF6QC00MF51L4l6/A652Hg+kM3o6VKLJ2zth3coE+RMPAst3gbWm2pKLYwzDrUTyHlP2tRF/dXHPnbRWUUoMvGNSI8x+26gahERYzQbNqtbx+DtYxcqpLJK8QcdxZ5hljIJ5rnD8zpiKqP/X92O6qY9bKaGCbseKaJIJ2gVWVhSXUaMDAFz0Fc7DhGCrpmXyXEl7OVdUmt5GctYxLQCfgLmhjGQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Kernel already supports splitting a folio to any lower order. Test it. Signed-off-by: Zi Yan --- tools/testing/selftests/mm/split_huge_page_test.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/mm/split_huge_page_test.c b/tools/testing/selftests/mm/split_huge_page_test.c index cd74ea9b1295..5bb159ebc83d 100644 --- a/tools/testing/selftests/mm/split_huge_page_test.c +++ b/tools/testing/selftests/mm/split_huge_page_test.c @@ -144,7 +144,7 @@ void split_pmd_zero_pages(void) free(one_page); } -void split_pmd_thp(void) +void split_pmd_thp_to_order(int order) { char *one_page; size_t len = 4 * pmd_pagesize; @@ -164,7 +164,7 @@ void split_pmd_thp(void) /* split all THPs */ write_debugfs(PID_FMT, getpid(), (uint64_t)one_page, - (uint64_t)one_page + len, 0); + (uint64_t)one_page + len, order); for (i = 0; i < len; i++) if (one_page[i] != (char)i) @@ -174,7 +174,7 @@ void split_pmd_thp(void) if (!check_huge_anon(one_page, 0, pmd_pagesize)) ksft_exit_fail_msg("Still AnonHugePages not split\n"); - ksft_test_result_pass("Split huge pages successful\n"); + ksft_test_result_pass("Split huge pages to order %d successful\n", order); free(one_page); } @@ -481,7 +481,7 @@ int main(int argc, char **argv) if (argc > 1) optional_xfs_path = argv[1]; - ksft_set_plan(4+9); + ksft_set_plan(1+9+2+9); pagesize = getpagesize(); pageshift = ffs(pagesize) - 1; @@ -492,7 +492,10 @@ int main(int argc, char **argv) fd_size = 2 * pmd_pagesize; split_pmd_zero_pages(); - split_pmd_thp(); + + for (i = 0; i < 9; i++) + split_pmd_thp_to_order(i); + split_pte_mapped_thp(); split_file_backed_thp(); From patchwork Thu Dec 5 00:18:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13894605 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A154E7716F for ; Thu, 5 Dec 2024 00:19:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F3096B0096; Wed, 4 Dec 2024 19:19:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A32E6B0098; Wed, 4 Dec 2024 19:19:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 01CB66B0099; Wed, 4 Dec 2024 19:19:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D58456B0096 for ; Wed, 4 Dec 2024 19:19:22 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 557D0AF04F for ; Thu, 5 Dec 2024 00:19:22 +0000 (UTC) X-FDA: 82858995702.22.B47C1DA Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2079.outbound.protection.outlook.com [40.107.243.79]) by imf13.hostedemail.com (Postfix) with ESMTP id 769AC2000D for ; Thu, 5 Dec 2024 00:19:03 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=PqyAdWMn; spf=pass (imf13.hostedemail.com: domain of ziy@nvidia.com designates 40.107.243.79 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733357943; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7vfodItUcrcyLrOMU+QPVr1fQyMNwn6ddi+gcaz99x8=; b=KjhhtdCfjFWJQh2NAguljjRu1svgJsAiws6cVR3IEslBsGu6C0MpAYp71wpYSW1rOq8Hfy 4z0IlWRM7mVIBqv0oNJhxDUV2RQQPkIbvJHR9TimXnJKSUuhbt63SGKCnqTymgFV/YVLqJ w3Cgwc4X4AjbOsUJj8ZtAOnbR6E7nFs= ARC-Authentication-Results: i=2; imf13.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=PqyAdWMn; spf=pass (imf13.hostedemail.com: domain of ziy@nvidia.com designates 40.107.243.79 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1733357943; a=rsa-sha256; cv=pass; b=2XjCXgdF7DOQ8qevunimIl8z+111Atok1yPjnadg9Wd39j7BRmtxMwaopiDBmBWVd1E5uY 5AC547ZdUzr6mKZgPr9KsP21Q7Se4BI4mcoO1E02T7Yjbi8nro3HKyANGdBdZAoc9raTht Z8SStAW5nr3rcljlEmM0i0FaiMButn8= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=hHNUzRVgwFg8A4RRgWviXnzlu+ZiI6wYj3P2+KdOgo4MjQwA6DKE46EN68lvrNLR0WFI/DKGQR0F1XoXHMI+gP1Pe3WT8ttzOHTZPbkJmeA0udt7TezeSpEALsIffb7gWaGAsdyC6NhkaaA1FTlluAwSPA735It/+Q/zRXOnF54pANvoerxMqL3znt9dgQcnYBq6fhhF6bQayjaepWA0/3mwlFAJigAauo3H4uJKubmtLNU21kKANNxx1z3xqOq+RQYswVDlMnZpa8OFTpU8rPdP/6cWiCgEwG7PSdkNufPSUdyy1MqBSlUqPk71lsXFuta/GuOyNzEyKgEzNL28UA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7vfodItUcrcyLrOMU+QPVr1fQyMNwn6ddi+gcaz99x8=; b=pVUOiejGDBxT3abvIvFFUReqImlZaG4i/Xj8n3VInt3kn8Zsod7cFPO1x/BKgNTWuEwfnueeA5Wgz0brE41VzO56ZiHJ2biaPNQYgt37esWcGDPymPNsrBDwZHuz01jHRNR++li5i9jrSLaW6c/A6x21deTCrIkLs5ymXhPEhazqXihX+MKGkOMI5orwP0knmK5/p8hK8JvVQ8rcm3lNQCrZXfTDdQnEZnZBVyt0onI/lmW5cTEZ2ZJIUwFKZQzQd6uJAYQi+88BmyjQsNrTeKsS/optIWlJyZ/jz6Q1428Aeai8YZf/qd5C3J506y2O/jwFAwHYv2HGtAL7VXakmA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7vfodItUcrcyLrOMU+QPVr1fQyMNwn6ddi+gcaz99x8=; b=PqyAdWMnljyW/pwUeOuZ7+3jjCNQWCKvDEoB/OaKaN4G+0+yhyzLH76jbf8vNReTYicbV732Cc5bsYGX+zLgkjXGv65VtNVwV0TKupaSMJnk2Pz0KRtpheuUvgfoJCP2nI858xDKtdIWpwkYX8E7/86rDkfCKIWO18xrPdkh6qXlPIm12Jg/4DeVjWzAgnCGQbVIh5WhcHAJxDY7Sq+Vxlhbt9/MDQVYBW8W6KzWCg3b0jJrbYH26TK6WYx3lM1bpbLpbAfjGexGaEr85svABG3w2zDd9S9EzJ6wg+cuvPxE0GB1ZPMaUnSRbUTaOMVp5astPdXI1rn3OSF9y0x8kQ== Received: from BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) by IA1PR12MB7589.namprd12.prod.outlook.com (2603:10b6:208:42b::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8230.12; Thu, 5 Dec 2024 00:18:53 +0000 Received: from BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9]) by BL4PR12MB9478.namprd12.prod.outlook.com ([fe80::b90:212f:996:6eb9%6]) with mapi id 15.20.8230.010; Thu, 5 Dec 2024 00:18:53 +0000 From: Zi Yan To: linux-mm@kvack.org, "Kirill A . Shutemov" , "Matthew Wilcox (Oracle)" Cc: Ryan Roberts , Hugh Dickins , David Hildenbrand , Yang Shi , Miaohe Lin , Kefeng Wang , Yu Zhao , John Hubbard , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH RESEND v3 9/9] selftests/mm: add tests for folio_split(), buddy allocator like split. Date: Wed, 4 Dec 2024 19:18:39 -0500 Message-ID: <20241205001839.2582020-10-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241205001839.2582020-1-ziy@nvidia.com> References: <20241205001839.2582020-1-ziy@nvidia.com> X-ClientProxiedBy: BLAPR05CA0008.namprd05.prod.outlook.com (2603:10b6:208:36e::11) To BL4PR12MB9478.namprd12.prod.outlook.com (2603:10b6:208:58e::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL4PR12MB9478:EE_|IA1PR12MB7589:EE_ X-MS-Office365-Filtering-Correlation-Id: 02ab22a0-e71a-4537-e1b7-08dd14c266d2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|7416014|1800799024; X-Microsoft-Antispam-Message-Info: hViy2l2HHk7GFDeu19JPbjRzBLsxo06U+Hs0GZTs5+dEZtLxdtI1nRx8F17FjUNlItriPF6bVLLcsDM81mnYa5TF9S6ISh3SE39m8VwMQ3Us8hXfFjIabHkmErvYGCv7HNLgjiOTsPfnq28D9ZUK+Las/iE7AJTDOP+bG/orBTaUmeksYnIafZB1qnDIbTMPfi9wLKkljP/3DhkzKp5+xeUjTXtTbSpwDm4lcnob03j6khg90hvFXcNv0hAqPfeF4OIeA5d1DKVJZg3/YJ+wRTu+Wyu42xG5infkSz4SDLaZFdbRViAV7knUxQBLeK6oMMJJU8F70jMlsHHEjwb/QwDPzZAPFi/YQV/CT8mXKMN0nnrxlqvso66KmLyrHI+RqQJdB44yq4WxDZwsWn4uYRO5gQXHhaY9fjcSsrWevhDeOBZwxr5Yuy8EMoLXeCDGwX06ynN6NlsLG6UM8s6y3mXnPgLXnKSbcwhDAqJ9WEG5auENoLs8Xmq/n+JAc2cuI1rhPY6l0Kyyq06CadJ2sC0sGjBOx7X59Rx3hiHhQ5gVBPQ82Gnybnbu8Od44ZJQKtYNiuKi88XNDwlDaV2rRpcmTzIldI0n8aCGuO+JSVsiy7knUOXoVLaUjgNouw75mLaUAhizfNCGGMLa8vLVjsn5jOX3/7ussqEXpj2wyqxNrj9pQxB1+uNe8E9MAoZndcnRcUjFW5d4mTe0q4K37CShjubIs2y47voYsGIM9diDn2A/QYl+HECN5jpdHSXy7YdGtdG/dowrfgvd4d+UWPWfPQNMZR4+WNSJ3pjQKcGwHrImHlNBx/3+EQGXIFYVpJ5U4etI9eO4mrW+7IrR8YpvKdNgfavnKZ3T+gFzQM5mjpAOxzlm0+eeT3qj3dkGptAeM94UjtgsHVqzSNh5Ea9wWZGMaw+QnWqVFghOpnKJC6m+i/8yuLGqDO82anGn2u1afl9vlVrv2hFV8CxnxO5OC7IkqKsXOloLPUBuNElstMMRosBkW+6GZqw4JgJK4rmlv0EcdH5+dQlYAlb992tl3kyAh1MiG60wo60xiiFEW60SqHNHlJ5iIh6maRxGbVpYWDFOniB2jJYh2PqdAG28Ls9Pibs2Uo34QVh2MeKUh0Dy9T45WgYSQXbvIlSDnqsq5A8SjBWygCdMjpjphhdq2HBd8b9WWe0qdJTlvsZp0maW4cL+qDzOU1JlJTT0MTjK5Rql09OzawJMjAbEkdoFISz2BR0TIkIL882bYJ61zRS7Nl/V73zqdxpS4SY+iszkGF2CFKjYAGKKha+TOX8Z3jnXThFaaUelcGZK5hJWoVDJg9dxcszD4Ii4g5CeLmRWG0O9VgK7RxuYzPjWGg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL4PR12MB9478.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(7416014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 2wQNE0WbdANjHio81QVMoE+EzdpQglpopbIPcLOVsjGGU4HPKBeIrr8Kd8pS9d61ydyqEkQEhe6siBoZdIY7QwtcoAyj49rtEXvugxnjiSlgUzD3iZmS2IMziJXQRYaeM608H1bmh2N1/rVORVNHI2yDIp9yi2sSQvfm7p2O2VNRJSAuJZCSq+PCGd+/nXSoVJFm06uxTMUNCO1aSbn986jo8Udg9hd4ucQ2r2dhzDmkR7kemULo13VIQy5ylR8u0n7QP0QzUJSTF2W6BRuibtGaq22j44u3BTexnDscpykBGZxJV40SoEiLOMzT606eLXG8rdbC+LPFWcU9ioV8nFbVPcfedhUpyB575L3V4RoE2cI/5t48JeQ97MrE65/zEhKMoYoHTeEi5qYpe4Sm3QFVM6tv+C1cAYfpbOUmiLpJq9qPp0XHI5ALMxgPgDN0Jh0dwsVYxrNklb6jqZJh2yKOhYdhLrpLnO25NTuOMIJiRbqLbnCB0A3oKRZeyPtqtKxWelx5BLPiZ8rZmLmCDmHOMF/GmNYqsgyLUhuux1OrfXWDqu2p8q4AfbnXo48MukQ5cfX9a1TJSCc6bBJ5Aiz/uRfwn/tKl8NprP4sD3oR6TJervQDNVKesIErCVKCcOCB5yMafJ7j19t3DJ6oWxvAoo01OoeJZtvK8H/Hjz8rFMfQQaSdfMD2DERNZdpr8sTYLP+8JhAaSsJOudcp2ZwnfH05yUGn8gDmklbjy59tUsiiAY1WLXAT+GVcYdjQMGwoCEicJ7aduEt1KFNOS/vkn4GKPorWHIQS4cfk8MEfd5x1ZUg/wrBuk9KnADyDDtbVII6r1AIW1xBmriRaKBi21fWILxkl636MCbtFFwg5OW5mKz+qFusmW/efTndzOXxbnpLsWKLynseJ1EHiVuKcbI4VzJp4CO4kfAQG//cVLoi9TJtXhbiAVimp78hGQRlMoaxaCyXBy5d3sxiD0vD37EqxvyNUWb8kyxUrD898appulatGJGEjiP8sc0Fs5q1ATNQScQcmm4tOY98QDBoG4mrJIQ93G+duhBZmGpy329nQtIvEEJHhxiNUK8KOlFw5hE3Wf6mOt3JiC/oeprgP9R+J+phHKDyNOMlCXzwb7gmNQUmsI9mu6+trwVaJH/Y+87/Ja1uV2L/gFWmFt2d8TtuAczB4ggpncRRW3EOfMG6RR+n0GVs0BhJYRSiFFDUe7W8BOP6V0d+2C2aV9dfY4UUHGEK2fYYdLL9318Q+MfZPlMBycRXOVN446aJsxZ9RdVitst87A73fkX9CMwIpNI8BZayjS/HBGyFivhVBKbkjBYSIQDSvBNWbKtj3VFdCkCdwCittwaD3q22SWPS2jLUUv9gMV967b0wolzSKRCJieBIThEXhVYn8zgWFp3wv56liZlxGngs9l/HCeQ9Bm3tug6nefC65c0LZF9hnwcDI0b5aLXKLgekBo10yqLw2vURJD7oiUvoufT5sN31qFxbgqmkbTfElupuhXRO7x01mvFdvux8J1Osu7JOEpX18HF/GX+j4+wP0ZpY21ZYH/kuqXmvC8hld7ICN79nkyd+L/yXNBZQeBLdU7+GK X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 02ab22a0-e71a-4537-e1b7-08dd14c266d2 X-MS-Exchange-CrossTenant-AuthSource: BL4PR12MB9478.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2024 00:18:53.8816 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: a6nem3fB92JhKy0W1Zd6VXfj2/MxjMENbZoK3BbSQLTMM58MmqdmsD8sEHHHVUEj X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7589 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 769AC2000D X-Rspam-User: X-Stat-Signature: 37jucr65ifepgx3dd881zn33uspt1pu1 X-HE-Tag: 1733357943-708346 X-HE-Meta: U2FsdGVkX18yVcSb+3OXsvHaif2lsb54fdlkRdfGud+LptgBmQPvwupFVqeyzGFQtdiIw7LOAioKg2/wEdWL3fQZqK1VJ/pp2963cEYpsscgx5va4Sfadg68S4+WZs/EBDjCkp7kTTMmznYGYxzQ5kv8s/3jTbTa0KWF4DlUUkAUwge64Gug8SYOciRYqgV0l2P/0Jjxc08D4GIh1vgjrT0GPEKva9UJcMv5H6p+rMDXD8z+F21z7X+EQwqFAQU2Hzk3RIo4wxpVaxIhg+tQTWwNj1RWmFck1W0wZO/yN/t8bNG9CApARr68v5YzXU/MJEzzAOemPTxwARgfVmBD1Eh8PE7QRIit9bat+hWECJgesLDc5VEapCyc8hAy0XvWW8Utw0fUa2uzGUPI8Bs5khhDrcTymSwV0KLfNe22JZ8+Y05EHK3hFTg/2YwM6ryKqO/Cu/nlvxlpC7xhsO/2rw9J90SbJ0mlZzA9YN39vpEymb0A0vTQHo5DUkPudfgM37POYzpx/JWiV1oFSRCa/VvhFEe+o4jWAI8vrbOn9GZfGWXd2W/u3oY7FCxW7HTOT/6OJE/byTQShq4mavl94dVCUllBSzR69HDDcrmM6bgLygqkBcHZh5Jzi+O8UT37dTidZtU1mQk6c3U49CI0GOjpbz/PcC/nmDo3kFdxh+S28XgUMQfNLEdHgF8QqyLNNtK+Ww2mwrRqTzatczEamlYJvHuH1F10PPrAlgUWnb5k8VZSPYH0IX5PX8yHIYn7teOdWM+7oZATsT4jceQNFliApBb/5Ca6dvr06JhOzVfB5Jq2dq7y+VeMO6+Pa0g42TxADw+6Rgt2DGy4W4gx4nz/ehbNZ5AZRyBKYViAGZjF1UnLuYzrl7PEgD+wigIwMkNEhLQkDyzQ0Z4Tz6Xj4PhWnGIhBW/hVpvzx85ad3PzWOiGV0aTJZ2k7mIEYBk+cPwY7YqMhOV/qiTL+xw HAz6+FxN rxWJXKHp3zz+swWelYwYAIDa4X8Q+v9jadOmGyKzGiZGEHESEsM0hOzgTHPxIRgZdNQPJ6slQwg6p9y3U+6cAcVMkJYSz0Wi62L1CrjCDWfTWk10wuaq+buDwKu/1f233cU3HmfPn/Some6DvU8DkM5tetO3veyxPK9PJDvk0N817lSDnEIPH7MrT4AgSL3H52cx25HYYy9V9dsVZEip6PGrV3H8Pjbxg9xRiMy5ZcvQspnzfVQ8LTVwhQaWSlRwBOucC9NnwLGQvZ+Kn39LX+Ag4xQYoBQaM84DmtmVQ9rxnjO8zJVwqLxUM7jRLApId5s7EWGKg6tNvubjVkhsffHnCXfI4XtzCvoJTnIVBq0I/CcS1t20JYI86bWcwgyBMbrXFeL/jnaAAM6s= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: It splits page cache folios to orders from 0 to 8 at different in-folio offset. Signed-off-by: Zi Yan --- .../selftests/mm/split_huge_page_test.c | 29 ++++++++++++++----- 1 file changed, 22 insertions(+), 7 deletions(-) diff --git a/tools/testing/selftests/mm/split_huge_page_test.c b/tools/testing/selftests/mm/split_huge_page_test.c index 5bb159ebc83d..1af8d6fa4465 100644 --- a/tools/testing/selftests/mm/split_huge_page_test.c +++ b/tools/testing/selftests/mm/split_huge_page_test.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -420,7 +421,8 @@ int create_pagecache_thp_and_fd(const char *testfile, size_t fd_size, int *fd, return -1; } -void split_thp_in_pagecache_to_order(size_t fd_size, int order, const char *fs_loc) +void split_thp_in_pagecache_to_order_at(size_t fd_size, const char *fs_loc, + int order, int offset) { int fd; char *addr; @@ -438,7 +440,12 @@ void split_thp_in_pagecache_to_order(size_t fd_size, int order, const char *fs_l return; err = 0; - write_debugfs(PID_FMT, getpid(), (uint64_t)addr, (uint64_t)addr + fd_size, order); + if (offset == -1) + write_debugfs(PID_FMT, getpid(), (uint64_t)addr, + (uint64_t)addr + fd_size, order); + else + write_debugfs(PID_FMT, getpid(), (uint64_t)addr, + (uint64_t)addr + fd_size, order, offset); for (i = 0; i < fd_size; i++) if (*(addr + i) != (char)i) { @@ -458,8 +465,8 @@ void split_thp_in_pagecache_to_order(size_t fd_size, int order, const char *fs_l close(fd); unlink(testfile); if (err) - ksft_exit_fail_msg("Split PMD-mapped pagecache folio to order %d failed\n", order); - ksft_test_result_pass("Split PMD-mapped pagecache folio to order %d passed\n", order); + ksft_exit_fail_msg("Split PMD-mapped pagecache folio to order %d at in-folio offset %d failed\n", order, offset); + ksft_test_result_pass("Split PMD-mapped pagecache folio to order %d at in-folio offset %d passed\n", order, offset); } int main(int argc, char **argv) @@ -470,6 +477,7 @@ int main(int argc, char **argv) char fs_loc_template[] = "/tmp/thp_fs_XXXXXX"; const char *fs_loc; bool created_tmp; + int offset; ksft_print_header(); @@ -481,7 +489,7 @@ int main(int argc, char **argv) if (argc > 1) optional_xfs_path = argv[1]; - ksft_set_plan(1+9+2+9); + ksft_set_plan(1+8+2+9+8*4+2); pagesize = getpagesize(); pageshift = ffs(pagesize) - 1; @@ -494,7 +502,8 @@ int main(int argc, char **argv) split_pmd_zero_pages(); for (i = 0; i < 9; i++) - split_pmd_thp_to_order(i); + if (i != 1) + split_pmd_thp_to_order(i); split_pte_mapped_thp(); split_file_backed_thp(); @@ -502,7 +511,13 @@ int main(int argc, char **argv) created_tmp = prepare_thp_fs(optional_xfs_path, fs_loc_template, &fs_loc); for (i = 8; i >= 0; i--) - split_thp_in_pagecache_to_order(fd_size, i, fs_loc); + split_thp_in_pagecache_to_order_at(fd_size, fs_loc, i, -1); + + for (i = 0; i < 9; i++) + for (offset = 0; + offset < pmd_pagesize / pagesize; + offset += MAX(pmd_pagesize / pagesize / 4, 1 << i)) + split_thp_in_pagecache_to_order_at(fd_size, fs_loc, i, offset); cleanup_thp_fs(fs_loc, created_tmp); ksft_finished();