From patchwork Thu Mar 6 04:42:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Balbir Singh X-Patchwork-Id: 14003810 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB5FFC28B23 for ; Thu, 6 Mar 2025 04:44:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 016BD6B0092; Wed, 5 Mar 2025 23:44:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EDFA76B0093; Wed, 5 Mar 2025 23:44:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D0D6E6B0095; Wed, 5 Mar 2025 23:44:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AA4B56B0092 for ; Wed, 5 Mar 2025 23:44:04 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 70D311A0F7E for ; Thu, 6 Mar 2025 04:44:04 +0000 (UTC) X-FDA: 83189884008.06.6D43CD5 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2061.outbound.protection.outlook.com [40.107.236.61]) by imf05.hostedemail.com (Postfix) with ESMTP id B119D100002 for ; Thu, 6 Mar 2025 04:44:01 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=V49qwEit; spf=pass (imf05.hostedemail.com: domain of balbirs@nvidia.com designates 40.107.236.61 as permitted sender) smtp.mailfrom=balbirs@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741236241; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EfAQ5J7KgEMnhpzY/3xJHeeTxTuaO5ru8T/vnoUf9xs=; b=UW9aODtcvtRJx8jnGGuo80tvwXPp9UyRHll6yVKV20iL083l5sS2S5AX4m6BiZzoZCmYiz urz4PfLFtOdIqsDy8EYxtj4fG8fH4PQOD77BudfeNmhoaHxSyAm2Ls6w0rayILQEjE1lTa Rh0dAZVkmwKsH7bHOtwUZJeAE/I9YUw= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1741236241; a=rsa-sha256; cv=pass; b=H/E+Jab55SftR5btwiesQtIATKzssc85hzB3n0A80ZJCAWTB7UVYiwRqinUeMtAnK0V4nJ xwZoO513CCFmCwvB78zjrPwiWNM8hz2dAVW4PTR35Q6HTUsedDrCvv5JyM3PTRnMtUAj89 Cza0er6NJ5zEd0pHjdm2NXiAaCDoJjk= ARC-Authentication-Results: i=2; imf05.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=V49qwEit; spf=pass (imf05.hostedemail.com: domain of balbirs@nvidia.com designates 40.107.236.61 as permitted sender) smtp.mailfrom=balbirs@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=U0nDYrCKuq7xGImWwsj35l/Ohmzj7T6/sABUtsz2UzHCJxpLk2eGY90YL5JLWCPf7TKnicX5jvzM/EG6UlsTQzLqCRHrgh4gOaf6CGLaho5E8dsqxhHx7Jvmu3B1lPtI+2DNGQDCquIEGYaeUEpSs8NRQln5la5l0rPkNLMOBVdvzUGJzGRFq8OGZenF2uzf5IxmBdtkZnWtwKj2lFWOyoPXLQH+AOm28gaEihFG5eYSCWYautepu4norttczu8oK7skkGHzgG6gQQ53A6qq5sglAsqTFEAoZqtoB13YAatYSv+ffDZOyNzz8KGLj/P+GJn2V26bMCDS4//azzsQaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EfAQ5J7KgEMnhpzY/3xJHeeTxTuaO5ru8T/vnoUf9xs=; b=WxgtvaXyuPKMK7dyp+7Hu9okHQrekdLSP8GLMAubPOJlonspnnCcwsw/+QlVxGmCIZkplODywoDjt+YLqSd5K8e1ptqmwTgl0R6HWxex0CFqFuaedgoE4eow0NmetZvLvWrqsTh0pjyRbe9Mx1yu0fXyWPHNczVt7Cj3Ikrs/dcILe8UkX3OOa6U/4H2H2QUV+8w+mwz5XUji9Mk4P+VKxchoKVS9YqRAZUQ31EjWhM5gGZQavK1RmXgT5S9pgTH50JchSWaLxC7DQx4nuKGlyKy4QTJFmhGgf2ZP4OheUtHcmEwy1lplxmh2UHLfa+9usf3RTDdYCmQK78MxUVpZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EfAQ5J7KgEMnhpzY/3xJHeeTxTuaO5ru8T/vnoUf9xs=; b=V49qwEitCKnA4xtOwdqZZO66PEAUffYOacKfoxXFRLLiDBTbPLrYRffyEjNb16VZTmgLEfXFnoWWawraw4wH9UFxgcXOCntKT5QCx2XuIyEOggMd4C7fHElASQkiLDFTdsyFM8NUb/kX0SEtg22nfZZU0ktNTd3E7x1I++GtU/PqGM6HVoGBohJtzmMBmwiU38oNf/9CQExydTUXx7IjwwqwSkCJp3XQcKfR6jbRdZ6Jq/iuGydh34+7D92Lme9R2tVdFM0sTBiqst4N3lzthXkziNJ/B6n/vBedN40c7dMpmndPH7R/BFraR55YvcVDxuc+NT6TaoEYLn6rgaH1Gg== Received: from SA1PR12MB7272.namprd12.prod.outlook.com (2603:10b6:806:2b6::7) by SJ0PR12MB5674.namprd12.prod.outlook.com (2603:10b6:a03:42c::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8511.17; Thu, 6 Mar 2025 04:43:58 +0000 Received: from SA1PR12MB7272.namprd12.prod.outlook.com ([fe80::a970:b87e:819a:1868]) by SA1PR12MB7272.namprd12.prod.outlook.com ([fe80::a970:b87e:819a:1868%7]) with mapi id 15.20.8489.028; Thu, 6 Mar 2025 04:43:57 +0000 From: Balbir Singh To: linux-mm@kvack.org, akpm@linux-foundation.org Cc: dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, Balbir Singh , Karol Herbst , Lyude Paul , Danilo Krummrich , David Airlie , Simona Vetter , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Shuah Khan , David Hildenbrand , Barry Song , Baolin Wang , Ryan Roberts , Matthew Wilcox , Peter Xu , Zi Yan , Kefeng Wang , Jane Chu , Alistair Popple , Donet Tom Subject: [RFC 07/11] mm/memremap: Add folio_split support Date: Thu, 6 Mar 2025 15:42:35 +1100 Message-ID: <20250306044239.3874247-8-balbirs@nvidia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250306044239.3874247-1-balbirs@nvidia.com> References: <20250306044239.3874247-1-balbirs@nvidia.com> X-ClientProxiedBy: BYAPR21CA0006.namprd21.prod.outlook.com (2603:10b6:a03:114::16) To SA1PR12MB7272.namprd12.prod.outlook.com (2603:10b6:806:2b6::7) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA1PR12MB7272:EE_|SJ0PR12MB5674:EE_ X-MS-Office365-Filtering-Correlation-Id: c3a2bc03-9d0c-4634-a6f7-08dd5c6981bc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|7416014|376014; X-Microsoft-Antispam-Message-Info: ZSmTMkc52VQF95t8ntgQ7R5W9IiunuTeSlY7hkVZ2Nstf7/4vVfjbAKQn6qJuroAdeRNphEsd+73x5UWGqAbiAzm3i3CQ2oovtQTGNejTDHX9bTv77cJhM6BX2jp+STyGTnAcmCGZrCok7ssefx87J8dfbb6BrKFYcFbnLT3pugMnfY22FS+3jjsgYR8GO8fJ8ZWMuPNinUlnxA0hijImc0G5dINaFYdR5n2PW7VN+uPkLVnr3Lf5ksttg/KWpeepEk5gnTgIoRi60+YjoX6a1YEzOjiu3i9TCTx/KkC94zHAnJUBhzvO+amrgvtukG5LfA+KnuGbfrjb8BlR2Ug2aVlBjUKOCpKwLi/V+dCD1scMP7sCUlun+BO57xV+pCNQuOzHjQhR9lJe+KFmEW1kYHe/FhC0VIZVH0yA4HOjqeKK4VNeId2xX9gnxwNeKrQTxyiQ6exKuwh0zalL2uOi4B6lFbsYuc3MZGkSWsvxvUtIMr1JoAAXtPn/sfT2IMScvBtYMeQNn9bHF4LGdBQjzZI7qdtS69W/jUuDNth1IhSrv5LFusN8dthmyir/NuW+FGV/4gt5AbingzNzhg8idX+FxpETHecLgl5+yjvf+oXK0J7nscwQhwESlvE35BnjAn5A+9v7nS0/09Mkn2kPgKZioQ/eoCqop+/mFN3vXRZQnlv2dlirR1RMG4g9yH408TvV5BiByb0ZvdE3gKW6PtsLuONFMNNh+D3x58CqSrn2ALsvzCoF6l6XaQHJu1Wi11HHr84A5dM0Md0IlEqdr3wOigpnir8uMzrUfKuJr6K4pvdIY5LGLPIPn42w+xJeOjhjh5/WrxLmiq7J5o3RpFhi4EkYxn8UBf3o0AVZKAIXgZBSlWBbtl3pNdUF9fCENOMen7q0IoU8m9PuyJ/4PozM7HaeyMe/Q14flpLUpe5zBovjpKi+poR6GW0JANLH0bSBvdvRx1pW50ZzcMu6zc624M8sMidfs/gXKCxCv9qDpd+1LLHpf1vvaU6oMT6je+U9b107Z1DKXkLjife4DDvmxxB/PkDOpusMypOD5BTx2aOdLGbg06K+MtRISHxCSNZyLn3oYqav/Hfn8q+iXLKkED4sD717q66QhZNpiYFtGKDTbjcwz7jEtmiHSXu/gVpz5h6iquHDq8wQfvfCNoebfYiazSu5Wz7dEpSFjOadQw/5Fj00k8kvue1uNtFLT2AP+vJO39GhLMz2/sgF2r6L6soD2RE4a8URDGFRIts0h6k11UWh8HuoaeaDV6g1rQuO4ZamLrLjxBl6TlOUQHzJQGEBR1+nsoNNvEaQbfgZ02ADh+w87tnP2YfibjlyANx/5MEWs1ZXLg8atQc2dRiTA1/ZX+whGaOLCPaJRvttuqdZfGQP+rRrrfxQ9xR X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SA1PR12MB7272.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(7416014)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: /9MFiabw7JK7w6ItwPVRnRemiz6usjUeAo0mrEEigGUI3YGKi3SKtn/c2D2fS3PFR9mOA1TxvON6yaw2y3nff8fLUtUuzKUsb4OjNrxmpAi4dfRckp8KXuqk8oRlhmnj7yvesxpwO9J4U6h45eausyQitbDyPtzIzp1LAZdel2fKaC22OPpE59iDHCLTptV7I+suie+4D2CPKj57+VyvLIllJBYZJkaja/3pOjAN0onYPZoHDonJ+vjnGo6Oz8VREbers3dmDEoS0GefZxmKe5Kmr/0Dqz+ypHxN/3AKTNSwSqLFXm4N6LaIrkVVJvryaphVZhewToPr9eef57OxyLX4x2ruKwKTx0BlRNashWpsgAd3zjG7DwZthkpOfHEk0isE8ruLsbNmt80Z1/f7Zw193Tv7va5n+OohIG0W8LY4OXBgk9MHP67u/z2Iv6Jw6oNE0erXVE2e9yXE0wpAEpGz9NwLMb3sBghFJ7LaD0ebJxhSBiay1mt/ecuqYsL/lt7Ncpav2dFO+Mp6qk10JWj4jC56v6BaRr12gRX2NT8jxUoCERvICYocnEisZ6Uqr4kXW3kYpUGrVLG40PY63uQHVkwlNVmsLtnIv5WA5DI2AzK/Or5MyXlt+QBucQN8r2HMGkIMsjxK32eRMy1nJTUdLu6NuYQuWrCEzmTn5cbFX65v1G1ZOym5/tvAoUEJpL6M5gAzPayfsCU2NCFOc+RAVoXFDjtwvIK8dGuBzuamqutY4tyOet+Qax2mNJLpp1Ng/yEGGYNKWe5homE3FPw3sV5hR+WrNFZJLJT0/QwohtuJBb6rCDBtSaZPImOl+Ct9GaaKMFWx/6s7AbASWaS058+KUNv8UJIM7efNvRb/HLiGvfQ3WuOitqnA7ZmObIpkLyrb1kLfb52/Ht3yzX4HM9AqqYWU8iAXsgI38Wckg93buEYNsPr0h1g7o/E/th/4Lhdzj+zl9j7ujqjm/Qn98ZYD8vAcvGDzR59MNDcHHz313FZGpR/VVyW8upm1xHgH2AcDt18W9y6QmTnqHZnTEL2dJfUwlcspizNv9M6M0qfjkGBzvQ+oUTETAU05gbEO6rNPOnVoP7VZmN1apX0VM0Fy/30iO+NFmqYZjJpmXmzTYTY2ZVPQ1T8s3AlLmk/8voDvbbjMY59xqJPCef/zM2x1dQtvKukE8oM7P6i0deb4MgpRgg/nOqNL2ty8Ythuqn898a/gF5mNDmNjiW7h4GEQXrn496s84DC58Dd+BLuh8zeInJhzmagHTyp4e1z3kEHER5vO5kfZ4y2FvUOdtGo67RLwm5lYqjRcW0dM5UUpbk/v5O+gGs+cqh19FmmbUbvYpu+85KQAw1ywH+01k/kd+fIdgHryL70ujZsFT6A0JuwvjLtdni+0CNKgAD6IeuK4rG4J0iOTEyqga6vg5EtqDib92hRbtIZse+KtHkfF9o572j3LzYvGk1ob4W2NIvUSVEHKq6xZbkI/Tn0rw6Mfc9eSBchYE0xaN2ApZ0Zr7GGeVVUsz9/Lhqrz1lv7NL8BYjQrpb0F/zOvbJn2cx9KCZ0eDMEkGIBK4IwWNwzDBRBte27K3EFXUk53 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: c3a2bc03-9d0c-4634-a6f7-08dd5c6981bc X-MS-Exchange-CrossTenant-AuthSource: SA1PR12MB7272.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Mar 2025 04:43:57.6752 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: j1hrAssSrWSairo8gUxXMWQIHQnspsMWEYEyOO3yLBL+KFkMoU0VdIvybEJvMsiStx8F8KJKR6izTptse9wEkA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB5674 X-Stat-Signature: snae3wuzqwdre8qqcwhobpidiacmjtdu X-Rspamd-Queue-Id: B119D100002 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1741236241-202304 X-HE-Meta: U2FsdGVkX1+Chw1q4cwtHMLd/cGfaUGSGUW//VVCyHsRYarV5DVRfZMDo9hebs8SeiClXsp6RNkKpcZZIknGYiwU67fl9yKbAtf++Ex551vQvDJbngEReByTDwVfElWU1WM7/8BVUOSEZ1dA7p361nyFmyxDKBViCQ39zd5/MR9T9wJ1kXkatyxhoKfY9+TQNDvzETVrPCrVKcIs50yNp4ZSUKYvz/nMlIYGq7EMIEWODJS4QTEk4Eag0tltP1kYJSrhIdnhpUTg5QCTkJxM/SIcjxOmH1y1BDSYLDOLqC45fbnKDvQSVVRdxmuHhUTQ1hyYFWm1CbL/bfgYRTLOfcepC3pX6CtllGZfcoVwTpYgjk3TYw9P5Sz6vvkhzHhEQrrUJCetCdzlNYWHlsH09IHQVpNKF3hmjtcPwQcLDDR9rnD6FZsFml2UYPMlmnoTolJLFvz8bq4TnEmKXa/HkJW5nnbQL+CJEIaZ20MCVHW7WTA4sqTQ6GpERAXZcLLWsxd6XTnvALWDFL0CylDQKSosn3XJECwZcEv1SZoBc76+9zkkxpeZa24wLFHhzldwyh0BogQAvA073OmFoT5h/pev3uJGJ0PPBzskq/y6k9sxLqPUUG0DBRmqwYplt3gj3KU4HoixuGldYJYgQNR1L0dqeg+QbR7+CBtUcl6iFs0j56VriT0DdY7CJ6SL/QdxQ0FCESaEbwXhni6LEiSyA+XbD29D5Xs7qpBxV9hRXxkdS1fYlwYRfC4ywTxqOhOjTEX+kA1acIZ5HbJSaQbwNn5JaoiBe5PFAR2/MruDN3InBXD+MQnLKpNa9NJNXQO0pYFkDBrtFCNnqNNtRa9pC/aelmP6WZmqylZct3vMb8I25z1Ar9azy86bcqKqSvg9FLWJntqPwZiixjTXlzcuvaLgiCM34OZTjNfzF5ubcTlwfluJB1hQKN2/Odk797xg2QSFQbqqaYC5cDTh1DF JGTCiAji mzcchQxQ/eO4HIXhukJCHpjlvWktAmFCWhEaOG0V3H6jZLKzYadWETOrCSeSkjGopgHCaa4F47ET5KBBi5IEMch0+YP66GPOA9KxzxlRVNR1IMqelQZGxJpMeICymMnWyVaZB/r2jk98bUuVy8yd9zzHgP9BvRv5Bv1XV0DKXufuT7fEUPC4hq2wzwqu3ZF6ZlhY8vwRSB0nhvTUDZWS7IPtUK0HZQXsE8bBTgN0LuDkm7lwzRmGr0btch2QAdj8qqYsznTzDF6rojr3oRactMpg78loI1yAcf6zfqWHtqlSPg6wsfYQtDk38DKRbrMIOxvu5hFrCBnaAvqgpWF/bOy8q+yJBUyLzN3o7jOWkcOx3pQHHmBw1NG2uvAL/WCHaXSBhWWLf2eX4z+VfEfQUM/B//DsJ08CfdU1O/N1ij5+PFVGmPBAAzFdDtQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When a zone device page is split (via huge pmd folio split). The driver callback for folio_split is invoked to let the device driver know that the folio size has been split into a smaller order. The HMM test driver has been updated to handle the split, since the test driver uses backing pages, it requires a mechanism of reorganizing the backing pages (backing pages are used to create a mirror device) again into the right sized order pages. This is supported by exporting prep_compound_page(). Signed-off-by: Balbir Singh --- include/linux/memremap.h | 7 +++++++ include/linux/mm.h | 1 + lib/test_hmm.c | 35 +++++++++++++++++++++++++++++++++++ mm/huge_memory.c | 5 +++++ mm/page_alloc.c | 1 + 5 files changed, 49 insertions(+) diff --git a/include/linux/memremap.h b/include/linux/memremap.h index 11d586dd8ef1..2091b754f1da 100644 --- a/include/linux/memremap.h +++ b/include/linux/memremap.h @@ -100,6 +100,13 @@ struct dev_pagemap_ops { */ int (*memory_failure)(struct dev_pagemap *pgmap, unsigned long pfn, unsigned long nr_pages, int mf_flags); + + /* + * Used for private (un-addressable) device memory only. + * This callback is used when a folio is split into + * a smaller folio + */ + void (*folio_split)(struct folio *head, struct folio *tail); }; #define PGMAP_ALTMAP_VALID (1 << 0) diff --git a/include/linux/mm.h b/include/linux/mm.h index 98a67488b5fe..3d0e91e0a923 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1415,6 +1415,7 @@ static inline struct folio *virt_to_folio(const void *x) void __folio_put(struct folio *folio); void split_page(struct page *page, unsigned int order); +void prep_compound_page(struct page *page, unsigned int order); void folio_copy(struct folio *dst, struct folio *src); int folio_mc_copy(struct folio *dst, struct folio *src); diff --git a/lib/test_hmm.c b/lib/test_hmm.c index a81d2f8a0426..18b6a7b061d7 100644 --- a/lib/test_hmm.c +++ b/lib/test_hmm.c @@ -1640,10 +1640,45 @@ static vm_fault_t dmirror_devmem_fault(struct vm_fault *vmf) return ret; } + +static void dmirror_devmem_folio_split(struct folio *head, struct folio *tail) +{ + struct page *rpage = BACKING_PAGE(folio_page(head, 0)); + struct folio *new_rfolio; + struct folio *rfolio; + unsigned long offset = 0; + + if (!rpage) { + folio_page(tail, 0)->zone_device_data = NULL; + return; + } + + offset = folio_pfn(tail) - folio_pfn(head); + rfolio = page_folio(rpage); + new_rfolio = page_folio(folio_page(rfolio, offset)); + + folio_page(tail, 0)->zone_device_data = folio_page(new_rfolio, 0); + + if (folio_pfn(tail) - folio_pfn(head) == 1) { + if (folio_order(head)) + prep_compound_page(folio_page(rfolio, 0), + folio_order(head)); + folio_set_count(rfolio, 1); + } + clear_compound_head(folio_page(new_rfolio, 0)); + if (folio_order(tail)) + prep_compound_page(folio_page(new_rfolio, 0), + folio_order(tail)); + folio_set_count(new_rfolio, 1); + folio_page(new_rfolio, 0)->mapping = folio_page(rfolio, 0)->mapping; + tail->pgmap = head->pgmap; +} + static const struct dev_pagemap_ops dmirror_devmem_ops = { .page_free = dmirror_devmem_free, .migrate_to_ram = dmirror_devmem_fault, .page_free = dmirror_devmem_free, + .folio_split = dmirror_devmem_folio_split, }; static int dmirror_device_init(struct dmirror_device *mdevice, int id) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 995ac8be5709..518a70d1b58a 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3655,6 +3655,11 @@ static int __split_unmapped_folio(struct folio *folio, int new_order, MTHP_STAT_NR_ANON, 1); } + if (folio_is_device_private(origin_folio) && + origin_folio->pgmap->ops->folio_split) + origin_folio->pgmap->ops->folio_split( + origin_folio, release); + /* * Unfreeze refcount first. Additional reference from * page cache. diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 17ea8fb27cbf..563f7e39aa79 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -573,6 +573,7 @@ void prep_compound_page(struct page *page, unsigned int order) prep_compound_head(page, order); } +EXPORT_SYMBOL_GPL(prep_compound_page); static inline void set_buddy_order(struct page *page, unsigned int order) {