From patchwork Fri Feb 14 15:42:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13975151 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87753C02198 for ; Fri, 14 Feb 2025 15:44:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C3D10280002; Fri, 14 Feb 2025 10:44:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BEC1A6B008A; Fri, 14 Feb 2025 10:44:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A1799280002; Fri, 14 Feb 2025 10:44:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 834FD6B0089 for ; Fri, 14 Feb 2025 10:44:11 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 295D8805D0 for ; Fri, 14 Feb 2025 15:44:00 +0000 (UTC) X-FDA: 83118971040.30.351140C Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2083.outbound.protection.outlook.com [40.107.93.83]) by imf23.hostedemail.com (Postfix) with ESMTP id 508C114000C for ; Fri, 14 Feb 2025 15:43:57 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=ndcCe9LN; spf=pass (imf23.hostedemail.com: domain of ziy@nvidia.com designates 40.107.93.83 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739547837; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AebmBmJkIAuQ7h16HolSR4JegMPa7EHSV1KqsaK63Ow=; b=d58w/g6mmD7TTN0AEd+5xxUV4lZ7bHfkcvtoOGA1hJhZ8XjDew1KBZrEEd9oB9sfMXLggx 2R1w1s7DC8TXkfHXYAoBiqi+WeS/4/9PGqSTgQ/ZXcZOaTr+nw8FGvMKcyuYk4dePBCKPE MAIojfEpPFfRRsnmCnWD+QRh7RJTWOc= ARC-Authentication-Results: i=2; imf23.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=ndcCe9LN; spf=pass (imf23.hostedemail.com: domain of ziy@nvidia.com designates 40.107.93.83 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1739547837; a=rsa-sha256; cv=pass; b=mh0bM7wIRH7kVInhCrb8a+JuGzsKKUORTcgxnGN27EjG6KB7wzfHv7/Q4/RvVTF7AEa5xf TfoEZmYQX6ap8EQ6TyMn7Zrb347YX7zfnJQ6/1LvlpnQvWsaXyipLN/mRWV49PKVkJ3dx/ YzeUbXKGWwsau/7NLSv+LeFdfV+3TX0= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=JPHg5r7ppzyYSV2+BCHWue17bD8SpU+9hKde0Mpvzd2hQ4Gcgiqlshh7r+x+/sZ6/cbw7661/j0jwek6f/+lGvEOPKvR9XmLwC9III1DeZU4R/8SA5ZA/XGnhEq7AoEj0vghcXRlH6RAuJFm8fkrRhzPTS6JNtHYmr2KdK0g12hrX1bW/5R8iUhza66p6MwRwGMsO9OTurjSZ1+WB2iy1WdilS3yYFxOs3a/fpXhp/Gual5bZXLAq2NQiT6LNKoxdMrjtONKQPBbThdrIIoUqLA0dVgypK66u3euhJ7/ATxouuCc4CuwzoDIVv4KldDoZLXEN4niYHzt+1k6nFYF8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AebmBmJkIAuQ7h16HolSR4JegMPa7EHSV1KqsaK63Ow=; b=oQeKuA88qfEU3vyO/V7GMhWE1JZRGfEM8qvwh+kBGYW/VJ4Mc0RpPYxoLuQwW04ReFDx1r9TisuxKwvlfJxlOFqYXxUR+SKTSt8fIe2FR0nzPTg2IRvikILA5ONGGR0BTXpJj6rgP19UKMnS1gmjGrfgfP/iRyb/feFvLmHiRgyYnU/mbAF5ZXKUMW3RK3e7mnWdD1XCmuRqHxR5xUCSsvb0PTy/nbfd32xVZV7U+k5KCzcTdBEFvxKoFI4zA08nFSs0fJMiNwh4hnSvnWNz8jcEeRuueRMRr8396afBsCkk1wMX3OqU7e+RCMoBHTPBSbbW4KnQKsEIb3+Lwc4ueQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AebmBmJkIAuQ7h16HolSR4JegMPa7EHSV1KqsaK63Ow=; b=ndcCe9LNbuiXBcEVO7xZfNKtkrXt4mzQP3JP1rfh8uOlitgLwpFb6E4KiiljeiMsspM7jtG9a8Vo0HA4c9yJ41oaZPgR7WPiSVZw6rKByhoFlkpY5EKj1C8UVT1qHh82g5XzR3w2jH2M0UiLtwR1lohbU1qm2RwSr+hdW+yjgEjRgzSwaR3M5nU86RTIkIcOOg5mRZZVOsXM9cw2tVytWBKbBWxchHBtqvQNV9z5p7yijYGJV5tMD4j7BnBVJaXFyvKF6npWhnfMudb377dPO4APcFCPnpc+lGgW4YU1hpP5W8clVWcRe/EVL7yrLGZUjILrgjbFKpW4qfZS5HgzJQ== Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by CY8PR12MB8339.namprd12.prod.outlook.com (2603:10b6:930:7e::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.14; Fri, 14 Feb 2025 15:42:24 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8445.016; Fri, 14 Feb 2025 15:42:24 +0000 From: Zi Yan To: linux-mm@kvack.org, David Hildenbrand Cc: Oscar Salvador , Vlastimil Babka , Johannes Weiner , Baolin Wang , "Kirill A. Shutemov" , Mel Gorman , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v2 1/4] mm/page_isolation: make page isolation a standalone bit. Date: Fri, 14 Feb 2025 10:42:12 -0500 Message-ID: <20250214154215.717537-2-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250214154215.717537-1-ziy@nvidia.com> References: <20250214154215.717537-1-ziy@nvidia.com> X-ClientProxiedBy: MN0PR04CA0009.namprd04.prod.outlook.com (2603:10b6:208:52d::30) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|CY8PR12MB8339:EE_ X-MS-Office365-Filtering-Correlation-Id: f2ba1d9f-37c0-476f-09b7-08dd4d0e2d56 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: lxmdowwLmZYoXQfU1A24v2y++AMfGflHKUgGPAM9v4ZygP7eLkowhmD/YW12YMHDA8pW4zh0GsNWxQ8SlMiRBR2LgiIruSNPEPCunDsOgNNNdUE2D2o89ab4G5ZVKDQ3tG0bLQTBUaayFvYR9klgfcV9BaCmS8z++AYavghnKHWVvdBOztXL7kwFepXhf5t5PBHtM5d5rQKy+FJoOY+YYxRA/ObJ8Gw8xQpxqJSad4yx5oWjkW0sQoK4bHnnv/ihF4KbCIlEq54dQg6yjLbLJtfvQhJNc6gXGhkYXxuAMJmH5Faa5rsHEOhdM/LXTLIVxZcKneb/4FPy0Z7i0Ac8cSRg5go0Wh82+UevSesdhKrMhZdUGcH1H3XzIr6rypHFaCcRGMMaSaC8yx6XiGzyaE6TET69mmc0xIAkrnGtrJCxcb6TclgkioQ5Dd4mj9KJlr2xWKpdPpDfdELurA2Y1m8eplMYo2M5il1i39taUlTZWZW/NZEU/svFJZbI31v1+Vwq9MF81+jI4z1lZ90XUxxUq8ZmXwjyhWPfWq1fen4vIWgIjNJ+Nc5ggI96gBZGN4k2EiAruBFaebGIlbrcIsr8dHl7KHTChLfKicWH5ZPYt+aVEL8GdvY8zQ+R7U9HA3JLWl7HPj1Cl0q0uQqNRY6eStQ7TVPn9flVsB4GrPRVZUEzymV0tE+lI8GwXfbG2V+S5mWIwkSpoNDvrJfgij723XBt0MibxR9fiXYkGtyDrdeq94B+AHsexsFWZLGG2/IeSPixCaOqLZHSaJmrJNfuHmq5uWVMYj8z2zTNKUBEgbV/INaI6edHDF3CCQ98dWn06fHMRRdCrNpZozvbIIZLNofP1WSrzyMKoxWirWd004fywG7hCpJjzvA9VTFxPIZjlFrXDH3jqKOTXtMCh7/7suVCD2srisOSfV9S4VijU9488PYZqErDbT0QNwiAF6tDj25XYFe22jzXpEaVGsP2B6f6YX5lDFBQNj6lB5idPI8hhO7/Xosub/hnT1mO91yUOwnfrGGeGAb24NuuPWRPdS/zIDOJ9S4K/oU3UBuT4m39+igy0vFTjSdbaucgUm+oCsKtT6zm7lVT1un3+y9TLPDT3mGktu/DvF1DlTwatwlNruFWmzricjdZBEfem0y8AmZ1NqmIazkzS6GxeuoIHBjr/cWA1/SWDrYpaAFpJmkLdl77nZK2j4a8f0Kz2zyoKSt94BiRNsUDaeC2TfMXqpo1Ocr8YM51xeIwgHvkdTrYsG7i7IHCMTevqTApqLkOMH8Oxla7bG0+Vc98qLSCLwVu/TUIr1UrHtrAIqCRBxSjAsYTiB8pcvsda5z92MhzghqCnASqKtiGH0v3VAByvEea1nq73B7/TzUwwOnl1DmLnmgVWLyLI9viTKEq X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8UW9o0egujEIDx9Hat6VaHAeAqpnhtDpooDZCaodjDnI2E3ZAIHQrDuvIMXjvgUzDi6kIJoaFSC/5gAUsoTjE8G6U6ay33sJM/w0BxyCK8+2t9YRDgmFOacXA5ZT8vmujNYWCf1d3Ey1SaDrvjCKsntT/gLJVFla18VqUoFQHn5emHjEVqeSicpXcTMfTrmfT41rnceGRW7FeA5N091+qt+i6/vS2tfyjrKQ4Nl3pUhDvDxFf2JRHFBgSpolnU+N0iOAMnppSvUgzdDRrtWmrdSWyvrXB/mkNtjntz+OC/eAu+d3+jmJUyJaeXr64uxA9yGK32GPg8WaCC10deMthawcwLoXpPGXaBqwURUTpT5Kbz0kfwsBx/bNBXWLVWaK5gcc9/ozLZtrjIno/ABS+l4MQg2T/atwRmPbhiCETU4f45WUeQKr+kZKdZj4SZfp0wotfKuhthavBliXGmudW9ZGKjme86fFfqU2Rxau4N3CThtma/J4SqcTCtcFHct/jxExGt4XnwGLtLvqpCHlelZy/+sCYBJfmohC0HornGHjW6XKjLR01DffN4iLYkfWl/SzUoxFfaDY7Rgdlamh+BxGRJF7CcZyz/Oaa9gnGVjrxnVd3hH7lquGRpSJ7KiwdnyJYdfgyvi4in1l1tk7ab9t+U+wMiZGQFjpWVgNQQtnAVqkG063Oe6JAgkG12z29zKyaDJOFVf+jUQ1ocJWQ5pTgVxMJMiAHudM3JPHXcktYR0W88+Bbk5IK4QtQihH92qIKCCuY6mURuQBFiDdjcKThhGLZudWa3VgKDS3+2vq3sk3gU4TftK1PcEvSl3QRZ09RrVJGW6eVeHYyuWiRpxl9MTiRi587ahX9UKcEYZUf1q88ZQOxQTMuqEjomUJKQjO35ykwgKQXmCOkpg62SOqDtQuW1oEJJ3FB4VL+B3kyUmJ7YcxO55hjNp/mHdeIzbMGQWoipgO9NjDnUWXnrxxWAmNIshw4T9/AJ8/yTql/XWVT1c2Aq5WWuMuzY8+6AmR7znT+OP47DviGjpPxTxac8tduq179N7EJL3q2zr7+9mmSGMu9eGQDWvLtTP0GBmUlmCIIXF9wrdL2GWGnUMtbhacXg+a1KGOiJONSt6Jml+fzXkywwO/ONOTbu6NSoKP0C0vcHM2ye/bh+23fcd98EWSmrcJcjE2eHSBGjNN7vdzbWyiYFTYvXP4YyLURBH7gWWjq6B+TKkwxwOYSdBZ7n22sVm+88HguY/97AtTQu/l8OHgkWgKd5bKIZRPb4a5WVi4rwaarva7t/kdyljmWrdNi0iR0DzAVgMhYXX4xnT1y63STu8jX3nlbORnAlJbqO5wgwuYTZFBPA5UN4lcxCXhKDTQEbI0imIvsxiMp81KCOmYvr+pGOIe7RqLwKX8dvY60WWWZeN/X/+vOrfgnYA+3Hu62v8gyS63PdvdOeyhVO9kuaN200bYXEYDCiZFY97DNu4uPRDvAb5DJFUf59IkSe+QrPrsSpugTXr9rNaMsMefm8PTQDBwG9cieMKIqSnpjw41lOIV9dL0fnA+rXA53ttkAwWuqqVRBj8Jb4T4l7LJssyvjXUfkq14 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: f2ba1d9f-37c0-476f-09b7-08dd4d0e2d56 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2025 15:42:24.3016 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9BQljV9wq+4Fgs4DmRO1GFk0C538OvTrgMr2IS2dZ4JqZoJEyyOoHzLamhDfD2QK X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB8339 X-Rspam-User: X-Rspamd-Queue-Id: 508C114000C X-Stat-Signature: ft9ho3p6q5d6p7b39rgjgasi9ut8cppg X-Rspamd-Server: rspam03 X-HE-Tag: 1739547837-21183 X-HE-Meta: U2FsdGVkX1/ztYhh5Fpnto7UjfwV8eM1jxq5vuooIJEMSA1Jtt+3vSWeuTqncIkRXkCTj+PYjo5crauJKpbL9H/x5PDLmJ/xjwsBaH3c4s4AMSdXpqnFrLLmIHz8Xn/48SFYK8pVQ5YC9s2fZ4Q5663yUHHuvhP5whKYiQxAbiv6qQYVZAk3Am2t+zYsuamRPE3Vmte3BpE0f2bZS6Ja235no7k26iXU6iZhKpfBlklc3iK69FjWS7opAx63BC7nnXurQJQLg15jrOF002HlTiLprNxoRTPSGGa9slSJTpa30XT9UPYXqQQ7FUOVHP22zNa6zu2OZu4T5nihs6N+TdRDtAaoetuYcBsdCVf4huqLx1S1gTFTNR58gzLUXqXMrIjFX8JMjmI6BXOTmytAk7eBOXBWA1elej7sFWL268WlcRDEP2ywTUIl49GMRpNLFLLjFbr76BvmrZyjWgh1IKfUfbivP0dugvwc3lvsk1HCZhgCkZlCPXMN2y5qKKnX5eYNQt/vgDrdKkkmG5ycb3z4eZlURPHOKZrPhOvaZsbuIVttkCMziIoopMTAWq259PF8AstHkiNtH6/s0ekidR7A6lITRr7AcrdikWmNN+6xjGc4idjJVTw0tj4pTPRqweJ+ZUjVuRwlPXqFP94uN6jEBJWWX2x9GaosENttyv1qjidCHNB6VihAwQXeIl7UUMSYk7y922NkPD5Drlo4dXuVg/gmiEqoEESDjFZxI9HGTj64GnnSuyql945Uqibb22AFmqlDEvruDv2Ns03vHVpLrBWzE/csOiGGnIh7wa1pNmMd4xzNPwEMkKr4x2e3Vp7bdK6jqm6TBvnJJdTyjupUeqBYJ/GYEO22wqnROrxWrYd4Q3XR8+5nhsZc+zF8dDpuTPvThKOtY2NUyIVYjThhSfmBPmQgwv+CPYGPenNsL0NWsfkuVx0L/jHv68wtGQyKysRUF93+6meSLbA uCKfxYfu e4SSWMdj/ZwCLFid6aG/rjuRN850Xgo75tymSfZwRrLp0Fa82xCpKkLMu64zGeYQNMoCJRPm3l3ia+l21ueVL5yWGU98MqgsTgTC7AU95v/U+Xa8Wsbb3p5Bf8ILJyXPF5TUVJ6Dear1twDtDgdDjGJ/Gj/C7MWq3adFhVz7gBHwdwQ7ibUG2HmTYUk5NYcIUErvt0T//EzZ0cj7f5e8C+sLfuRiUlhOcO5sBdH6PFyvPxJE4sDjKqg98RPEzjtyM8ApE62XZpoh6t+L38WgiqABtYh4tss2gLABv+d+FJAVj6DLz182W0TjjBuJIkImbG4PO8d3c0SMu14czo1JET7+yww2QGlDryS6O95RBJKSDioJYnosbTcIpbE9Ly31YzZIK0VVVuQoQurQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: During page isolation, the original migratetype is overwritten, since MIGRATE_* are enums. Change MIGRATE_ISOLATE to be a standalone bit like PB_migrate_skip. pageblock bits needs to be word aligned, so expand the number of pageblock bits from 4 to 8 and make migrate isolate bit 7. Signed-off-by: Zi Yan --- include/linux/mmzone.h | 18 +++++++++++++----- include/linux/page-isolation.h | 2 +- include/linux/pageblock-flags.h | 33 ++++++++++++++++++++++++++++++++- mm/page_alloc.c | 21 +++++++++++++++++++-- 4 files changed, 65 insertions(+), 9 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 8aecbbb0b685..3c7d3f22ccb2 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -106,14 +106,22 @@ static inline bool migratetype_is_mergeable(int mt) extern int page_group_by_mobility_disabled; -#define MIGRATETYPE_MASK ((1UL << PB_migratetype_bits) - 1) +#ifdef CONFIG_MEMORY_ISOLATION +#define MIGRATETYPE_NO_ISO_MASK (BIT(PB_migratetype_bits) - 1) +#define MIGRATETYPE_MASK (MIGRATETYPE_NO_ISO_MASK | PB_migrate_isolate_bit) +#else +#define MIGRATETYPE_NO_ISO_MASK (BIT(PB_migratetype_bits) - 1) +#define MIGRATETYPE_MASK (BIT(PB_migratetype_bits) - 1) +#endif -#define get_pageblock_migratetype(page) \ - get_pfnblock_flags_mask(page, page_to_pfn(page), MIGRATETYPE_MASK) +#define get_pageblock_migratetype(page) \ + get_pfnblock_flags_mask(page, page_to_pfn(page), \ + MIGRATETYPE_MASK) -#define folio_migratetype(folio) \ - get_pfnblock_flags_mask(&folio->page, folio_pfn(folio), \ +#define folio_migratetype(folio) \ + get_pfnblock_flags_mask(&folio->page, folio_pfn(folio), \ MIGRATETYPE_MASK) + struct free_area { struct list_head free_list[MIGRATE_TYPES]; unsigned long nr_free; diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 898bb788243b..51797dc39cbc 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -5,7 +5,7 @@ #ifdef CONFIG_MEMORY_ISOLATION static inline bool is_migrate_isolate_page(struct page *page) { - return get_pageblock_migratetype(page) == MIGRATE_ISOLATE; + return get_pageblock_isolate(page); } static inline bool is_migrate_isolate(int migratetype) { diff --git a/include/linux/pageblock-flags.h b/include/linux/pageblock-flags.h index fc6b9c87cb0a..d6fe17bf0c9b 100644 --- a/include/linux/pageblock-flags.h +++ b/include/linux/pageblock-flags.h @@ -20,7 +20,10 @@ enum pageblock_bits { PB_migrate_end = PB_migrate + PB_migratetype_bits - 1, /* 3 bits required for migrate types */ PB_migrate_skip,/* If set the block is skipped by compaction */ - +#ifdef CONFIG_MEMORY_ISOLATION + PB_migrate_isolate = 7, /* If set the block is isolated */ + /* set it to 7 to make pageblock bit word aligned */ +#endif /* * Assume the bits will always align on a word. If this assumption * changes then get/set pageblock needs updating. @@ -28,6 +31,10 @@ enum pageblock_bits { NR_PAGEBLOCK_BITS }; +#ifdef CONFIG_MEMORY_ISOLATION +#define PB_migrate_isolate_bit BIT(PB_migrate_isolate) +#endif + #if defined(CONFIG_HUGETLB_PAGE) #ifdef CONFIG_HUGETLB_PAGE_SIZE_VARIABLE @@ -99,4 +106,28 @@ static inline void set_pageblock_skip(struct page *page) } #endif /* CONFIG_COMPACTION */ +#ifdef CONFIG_MEMORY_ISOLATION +#define get_pageblock_isolate(page) \ + get_pfnblock_flags_mask(page, page_to_pfn(page), \ + (1 << (PB_migrate_isolate))) +#define clear_pageblock_isolate(page) \ + set_pfnblock_flags_mask(page, 0, page_to_pfn(page), \ + (1 << PB_migrate_isolate)) +#define set_pageblock_isolate(page) \ + set_pfnblock_flags_mask(page, (1 << PB_migrate_isolate), \ + page_to_pfn(page), \ + (1 << PB_migrate_isolate)) +#else +static inline bool get_pageblock_isolate(struct page *page) +{ + return false; +} +static inline void clear_pageblock_isolate(struct page *page) +{ +} +static inline void set_pageblock_isolate(struct page *page) +{ +} +#endif /* CONFIG_MEMORY_ISOLATION */ + #endif /* PAGEBLOCK_FLAGS_H */ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 16dfcf7ade74..f17f4acc38c6 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -362,6 +362,7 @@ unsigned long get_pfnblock_flags_mask(const struct page *page, unsigned long *bitmap; unsigned long bitidx, word_bitidx; unsigned long word; + unsigned long flags; bitmap = get_pageblock_bitmap(page, pfn); bitidx = pfn_to_bitidx(page, pfn); @@ -373,7 +374,13 @@ unsigned long get_pfnblock_flags_mask(const struct page *page, * racy, are not corrupted. */ word = READ_ONCE(bitmap[word_bitidx]); - return (word >> bitidx) & mask; + flags = (word >> bitidx) & mask; + +#ifdef CONFIG_MEMORY_ISOLATION + if (flags & PB_migrate_isolate_bit) + return MIGRATE_ISOLATE; +#endif + return flags; } static __always_inline int get_pfnblock_migratetype(const struct page *page, @@ -397,8 +404,18 @@ void set_pfnblock_flags_mask(struct page *page, unsigned long flags, unsigned long bitidx, word_bitidx; unsigned long word; +#ifdef CONFIG_MEMORY_ISOLATION + BUILD_BUG_ON(NR_PAGEBLOCK_BITS != 8); + /* keep other migratetype bits if MIGRATE_ISOLATE is set */ + if (flags == MIGRATE_ISOLATE) { + mask &= ~((1UL << PB_migratetype_bits) - 1); + flags = PB_migrate_isolate_bit; + } +#else BUILD_BUG_ON(NR_PAGEBLOCK_BITS != 4); - BUILD_BUG_ON(MIGRATE_TYPES > (1 << PB_migratetype_bits)); +#endif + /* extra one for MIGRATE_ISOLATE */ + BUILD_BUG_ON(MIGRATE_TYPES > (1 << PB_migratetype_bits) + 1); bitmap = get_pageblock_bitmap(page, pfn); bitidx = pfn_to_bitidx(page, pfn);