From patchwork Mon Jul 22 17:29:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13738979 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 259EDC3DA59 for ; Mon, 22 Jul 2024 17:29:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 82D7E6B0082; Mon, 22 Jul 2024 13:29:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7DDE56B0083; Mon, 22 Jul 2024 13:29:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 62F616B0085; Mon, 22 Jul 2024 13:29:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 44B206B0082 for ; Mon, 22 Jul 2024 13:29:31 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E335E14199D for ; Mon, 22 Jul 2024 17:29:30 +0000 (UTC) X-FDA: 82368075300.20.A274669 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2051.outbound.protection.outlook.com [40.107.102.51]) by imf13.hostedemail.com (Postfix) with ESMTP id 07FB420007 for ; Mon, 22 Jul 2024 17:29:27 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=oixHRNUy; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf13.hostedemail.com: domain of ziy@nvidia.com designates 40.107.102.51 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721669322; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=i8Gzw7y7sG97NEPUr6qENMLG1Y85frlPpN/bV4VSYFs=; b=SYN63r1jbzzJcaYmWptnO135nq8ZsdzhFSTlotH9GGMAxtBVZXMTQXnbSfeXQEhSKTsVnd Y3W3vHIFmfyB+HFfvzU5M/zZrQDfDGIOr0YWYKgX7vOoOsyjShtz01QPNY7hkFUKPvqM+/ eNMawK/wjXkfWdU03c6ewOyiMjPRsmY= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1721669322; a=rsa-sha256; cv=pass; b=mJayILNfNZGCVrw6h9snNEMp+tUtDx7PEjgCquip+qiRSD2poNpUvTm38R+Qh+t2HU492g F5G5CKYiL3RG99GaR4Iv06bC3uKaDDttbsrzoFtOlt39ML9PMjqJrxwbI21HBctXt93CnG YCmlmmCQFEqf73nZCrDLgEi5HoQVxCM= ARC-Authentication-Results: i=2; imf13.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=oixHRNUy; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf13.hostedemail.com: domain of ziy@nvidia.com designates 40.107.102.51 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kI2TQ9WSkW+wpvcuCMnkx7hAYfWnnHxMKCFberp7VBQTfRWyfYnw7K39KS6GsE3ewxo6qNe2FMPyrpKjI1pJ4QM5Vb0Q0xPEMrlrdHh6O8wshQU29Omlp5BsJDqabx1ev6OF/V75Dw7pajSjAdHxlQ1CefbXQ+dtbEKFXJdtPGIiwvXZ3G6PFi8ovpQbVNNEf8IkIxAWA8Q/LL7YQxE4GVtVXVakeo+LIG0KVXH0SZIBtSDZXSYOvRP+R2PTNlttiiZx/0tw268LaWo91fePZkEfP1SmIFYZrpxcHl/DkoVqK0HTmGrmebpDY0HX5DLJ11d8UodvOCkyf1L8qGxCWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=i8Gzw7y7sG97NEPUr6qENMLG1Y85frlPpN/bV4VSYFs=; b=AW5d3PRlWA4H4YnVIvJsJzlH5N6rnimKHBy68p5gtQGa/OtZocqatjH/LV57HjO1WOKTcZVCe4/9U5ZWbd7GeEC+D1DcLxfY7qUi18WHNArVlbFPCdpEalj2fWRcBdbBnbLmdDXIdAMQWFWUKW89dpmNbf38JFUDelUu+nQe/mIFoe3blkGy7xgE3Qbv0BiEGWj3bSJN+uK+MNpmqo92HdySs8Cbc6AbB/Lyl/FOZ09XteC+2bb4TykltV6vS3dRTKbkWlm1QThQj1gIqtPYvUA7i5SuWfbpxdPffWjvCIra4A0WN72LaC0svi8ghC2FJHUrPtvrMidyWrNUgWsNfA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i8Gzw7y7sG97NEPUr6qENMLG1Y85frlPpN/bV4VSYFs=; b=oixHRNUyxtYJ8ZiT3DOVxm/ogEioAThIPzfNa8rkP0bdnMG2LaJMXcBCJJsrW6yf/U6A6ml0PSu3KUQpf4Fwg49ASVCjWMrYugfhATZ7GNDkcBLqieM55tXTYIcerk4iLA/zc+wp6WPkqnsj2DyOZMdfliWH6OpbnmWYOt8TSrFzJYBCQE+WtfUjQCw4KmE+mhg4oCJ1SyBnvxobjRmlhAxOoFZRZpYjr4gy71jk8Ef/qvhy8u2Ok0e1iucspoSGcM30Xam2qGinngvgdlmxAx3g1hdGLruc29spLp2z9Rm4Mf6//3qOV7PGSOYYSwW3RC32aD9MpmLLOOUo+FhAmA== Received: from DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) by SJ2PR12MB7944.namprd12.prod.outlook.com (2603:10b6:a03:4c5::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7784.14; Mon, 22 Jul 2024 17:29:22 +0000 Received: from DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e]) by DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e%4]) with mapi id 15.20.7784.013; Mon, 22 Jul 2024 17:29:22 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , "Huang, Ying" , Baolin Wang , Kefeng Wang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v2 1/3] memory tiering: read last_cpupid correctly in do_huge_pmd_numa_page() Date: Mon, 22 Jul 2024 13:29:15 -0400 Message-ID: <20240722172917.503370-1-ziy@nvidia.com> X-Mailer: git-send-email 2.43.0 X-ClientProxiedBy: BLAPR03CA0085.namprd03.prod.outlook.com (2603:10b6:208:329::30) To DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB5744:EE_|SJ2PR12MB7944:EE_ X-MS-Office365-Filtering-Correlation-Id: 48175789-c3b4-4923-7a53-08dcaa73d35d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: ifPb+0xJDLfklnWWQRko1xOMmmsPe6+01/9ocyQWCtVdsSMS8zD5yL7GR5s+f/sHtUpUhAy5iytsNm6iIRL08u7y5MspDZnoT480DG7GfZUGZ5UrjDVOFgySuGTBh29KrXLYahOiqZ28hH9lBWxXvrpuot2y5+L7BW6mu9B+MM/yvGnTVqNlIbN/r0Qt7uNmJPm/+3FPAVsCF/72/u4EQnjCXiYxwejzdV349v293PypP1qpjNPLoaUe8eccg7J2OQ7N6Md6+UuaL/evVje3dSgktG8DzO3tCBb4pbQCf8z8iodmmMHwuo2eDyAD8a7ZlRCbDZuhaKjwf9mzGMSRhWNETGl3XqlUZLATOqL3lc70iYKoWMeLePNQPpmUejeP2iW9D7njG5yUt8zrKlL3aFx9i6iqUniXDQjnDjXhqc2UU8T4awlMwdckkLErdXA7oYdDHjgQ/qJgVwqSqXaaAwQ4pSdJ6rEe+uWHD2kmAYkOEYTQK5Lg8LQVuymVw2+9TM4pZSGa8do/t4r2+vVsqC0qjaErI/B6H0Yi2p3jaNTbjBGNiTtkl/p1Gv5TmjdNjEq6q26VlOGqjDd6GzHbkuZmKZDPyEOrgcpRXEao3UEkuz9bZ2O5RrWny2pr/nS2GWf1aTdyE/h0yRGQwkUGFXZq3026hgmDfyPPdwjjI2vlAJkPTVd0rVTDiOS/bZp9Hq6eOj0Sk0ByYbhbMueGBNw8Tr2xcrqySJpM60zWIAse07hgH+U/2FTik2/Y7rnMVARGR5T86NusGMdZAwC9biUUjXJxCrJ0r3jK9EcgW4JseWF5oeX1CZAJQv+wcvzT2xJLhz8kYkSUsKZEFdd5DSjvxzTXEo9nXQVNHODGR+4PLxmh3xZ70gJLV0OdkAKJMFH5+axU4XaUh1DonYItk+jBIdcXm0TqW7MiMNA1AzCAS+kR3QY07AcKQDvUB0g++jsyboy9kvWXdySTZ9uu1gkzqWjOoYYqHeT96v/Zhjip1einlEH34t6y81fv+H8g2gqeEV2NUrqrZ1/Ujn2iknQ9Lto0qLQSPZeSot2j4qNJcMuKwt0VtIFq0vron7XDimgpgIUa8q0CrdLE7gUVqK3SLYpwJjNlsWO+Orac9NgB+5AmGkWrMyEh2LwXCIobvq937KqFfG5U2ZsI+D2ywm0BprFrVNfjL879USw8phjSrfLIp8fWD3+pGfDCN9jORkXRu4RmgBLdOMZXsg4uhR1G5Q1OP5FwjggjGgw2nDVC9FkxZxNrq08Mh/IaN+DxmluTvmqLA1oMfLcFaucv23IvHJhVJ29SRynO9Z58renFtP3ssxo4ZAU33A3zLKbQNeIFzTJc3hKmeZuK3enr1A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB5744.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: tpwmDIS3jChw8UwWoWF08Uq2KmBjly6fuiRcb5rGpzBC21t/SIQWFAT3fHvC6j0Ud0IwRIrxK0Ll8lAV/XMrmD7rDnKQYcaGYq930vmrQLRXgvpzphrMScBrODK/OBPqWJ8ncR5AwbIK7+1TwoCrBh0s6Z7c2md52Drix+Ik89ac2nlTGMhFeu1l9D4dhf2jqztR0RxRosQx8mz49Mn/GKXUY7wNmqFSfq0rlrc3nf2GqrZfYzWlmciEQoyO2pRJCit1TUisLNSyYoqMe1nktNUxH93WRAvYf+a3oxqCODAygnuvH5XtjlYC/MqAwXzLRSKOr4MWuCRqFfs6yDXPxTU44xQskdP+r/tm3jdpUcpzGi4EROLLrPcca9Fq3pm8h2PCM0c/2IceR6WyiT9oo2tC+tEezzqXPJ0JooF1mWGgBCtHO+t0Obn/8bNuAjyiLbrYxOAJglXoIIOr+8Y0LL1OIfZt1+IGLyKCfMQGML7J55FvsW9w5LEtDhl24yj/QJqOs/OzzyxD5Qb2+bQ/glVFL/aNPzbQD8QeoyCBYp0TZ+hCQ08GFr+u5FOdzRiorafDuNpM75XbB3vhkufP19qlcKMV+9jhaWDBfUDBMyWayTikCFjjrgHpLjPyIOaIjLGKnCG72CEy6UPR3LyAiiT7x8JuuQLVfpEXUFZjIKx8x/CmRMASGN2jov1dfsDku03KIi+JEDTajqs7g1jVH6B6LZbkI09Ie+zTVhAnMJLxyDiZIFKW+HKMKgshLxgpr0+458HtMRZnB/Sa/GjbH9jCcOn6VKlqiao4y/pdIQpaxqJDFBAwmnTZAhp3eEnkdVU3FgfY6x42Zf2NDoL9bstOzUOaieXdsgOIlAHhaVh9UzSGfAjV+U2o54A5jnnRnUe1Cx3Vb/6+whsofI/p5ZvaF/2tRz49rS3gmEmJ60qjdv9PGftRFzpSamHcEKL+lqYZxQlBUZ71ADa+H2uOsygBg/ePv1e71E0a2CrvQVLTAFdnR8TVUroH+WPXXVHeeiMLIDO5u5kEsD/Y+sxlceZhXjEklbaL9oGxkDeKBbd8GODA4N4qUn7GeNuGbLX62se4mJMAVvmviOIvEaV99eXM192iqkda9N5hK15n7iM+PlCk/UrKoRwjFzwsK/OElkYcd0qqOQy30IeTAdTFamjyQ9XO3wArcP5CFp/45JoV/Erp6cFj+C0ESp/K/DAgAIMblYT65Rsp5dayWuFMeBK73IoAr9x7m4Gh5lwloquhGqe2Iwrwbb6WD2jwGEd5030nxyLxunQFyHLXhNq6eFFEsl22fj2SEjgumTX+w0r6q7Pg937b4h4Ls1ObbBQqs/u9CoRIlmcK/pJ2eP0FqDtQ3CEloFBezgb7Qso32thcsncXG5QtYcsPS7OR6EiTEb46ZWL/xKh21gDyIQrvXWz0udU4AqJM0u0wixQxomn1jL2zNosM3ikRE95e+i0AdMymhRKxLfrvwnLo/kMD2LFvd3zwhZj+GM9P5EensiunoB/O6vtFIKN0unPNQBQOVJ3Zxbdvh3g54XlRa8gM2dddqKJfdo1wniOEADCQkHs= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 48175789-c3b4-4923-7a53-08dcaa73d35d X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB5744.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jul 2024 17:29:22.4941 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: loKXtc20IkCMCkoNVdshOPqDOrFWX+DE28F7ruoiYfQa5NUoRvHPY6DmISLukdl4 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7944 X-Stat-Signature: idukk89j7i8w49ck13quf8mophbekpg1 X-Rspamd-Queue-Id: 07FB420007 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1721669367-930864 X-HE-Meta: U2FsdGVkX1/PwQixUy7+UIRBADLrFfUWw42fWY8SFXCOLaofrGKw6ZmO9l+qfu2fYurJuPyr5e8sWI5JBzAwgmqCj+5s8dEeb0zRb7BCPHzpVzbwVv7Hcq0O4LYcM9MBB+XcmpxQgysgKPQh+nP2ZBW4N+sFnl1ql360PpLz5UckWMQTiG+Ai4Nka+66OMvICOGS6InHFg6RJQ+Xg6JlROfkhMWdDpFhRtDFpSb/iXATCT+O/r2I1hZXYuMEte/Ac+SPlRidLCaa/a3qeLi8fIbhaA34CM7hrHz3J2cB6bkzNMIs1BUkspj1l6wgLsQWfJIhuVphRBMjV9bNd7+Dz7B4fDSHq2UmytANJFIK+46xD0H2LRqsg37hlvW6IUqAkISdtE/oM1xAB8gotL0jYvezGaWwxSjgwHlGGvJY54gxf3HqZjQ1uuIZJ4958Ox/cSVZCEyM0Ww/5+/6Jj9nnqeLGaxaU8re9FAWDDSrwIIKPETfmMaTv2wOacB0szCF4F5tyD6IZroneWVDCje078rBi/UGwptlnai1gtcAfDj06WKVEX1eTDVJLjBhimonMlOZK1WhvuRwn5YaXUjUJfhbWMgDnMNCUBMXq1pRnIwpTtRw8oVwJIlOA6nUrzswOjbGS1A/rT0PvqD5bgLn7yeFHeHKg5MzypQHlVNlu3QZKMwztbf8L+OFYjMsLDkYDivD250u0uT2ZDegOXffx77CWbh0dTQbYH1/yReUZkznUlExXQtbBszGvMMPJWw5UEdrv4jmL3UFdiBGg+qH+WhsAkhbOzThP4N7xuNVeqvmtEY7THVV3s0qADXDo6Vj/5zdA0hqEX29ENdyB0Xs0B1+9nnUmEO6tKAwKUZ0zLXcgkGHl39kKXAQrt2xHSBWZWdn0V+xGpRvLfalJSdRpuCLpnqdZtMDgFWWRXcqgTX5RFU+Ti2ghAbawQyx8gs6hgP0KEYlKK69yir5GS/ TXMcrMY+ IWmta9+oZAfkPWeia236ACNW5ck8iKBHvOM0ZlK0Txv2PjQ9In9iGpz72QNpBRfORz3NRJouKFV3twFfCVaL3tcWr8UJff4j+cZxlPH4T50cM68SodYo+ZblBGfDOF5aQWhRpJE5/J+YFnsCeiV/G6PzqSGIJhgboPAQ0Xtr0t+x/LIfmqLE1BvnPO3Is5ZDLYaaQhUa8Ahad03fsFAKS4/1Vrsfcz9JrSQC5gaEWeYR4P4UixkROUW99wzcfAe5UHKxTQ5EWSh/ymqd68t6//JIXbe0phf+UHwECFiIH5Pj0e4m8DYEUwAmYf/mHcj74nklJ6VDe9fKCbf+Q1Eg7drWlNg29LHgUoqNkNlI6mx2+cBtLuMo9wBFPT5iAN4vk3o0MWbrlaNRslxP3V0o0m9sVlbFG/diqf30rY4CRhOujqZ/rZrmCL2USBSJWax1WPlKWgS5KReyidwllvciVhQSZ5g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: last_cpupid is only available when memory tiering is off or the folio is in toptier node. Complete the check to read last_cpupid when it is available. Before the fix, the default last_cpupid will be used even if memory tiering mode is turned off at runtime instead of the actual value. This can prevent task_numa_fault() from getting right numa fault stats, but should not cause any crash. User might see performance changes after the fix. Reported-by: David Hildenbrand Closes: https://lore.kernel.org/linux-mm/9af34a6b-ca56-4a64-8aa6-ade65f109288@redhat.com/ Fixes: 33024536bafd ("memory tiering: hot page selection with hint page fault latency") Signed-off-by: Zi Yan Reviewed-by: "Huang, Ying" Reviewed-by: Baolin Wang Acked-by: David Hildenbrand Reviewed-by: Kefeng Wang --- mm/huge_memory.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f4be468e06a4..825317aee88e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1712,7 +1712,8 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ - if (node_is_toptier(nid)) + if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) || + node_is_toptier(nid)) last_cpupid = folio_last_cpupid(folio); target_nid = numa_migrate_prep(folio, vmf, haddr, nid, &flags); if (target_nid == NUMA_NO_NODE) From patchwork Mon Jul 22 17:29:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13738981 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 313D6C3DA59 for ; Mon, 22 Jul 2024 17:31:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AC6586B0089; Mon, 22 Jul 2024 13:30:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A76636B008A; Mon, 22 Jul 2024 13:30:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8EFEA6B008C; Mon, 22 Jul 2024 13:30:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 70FB46B0089 for ; Mon, 22 Jul 2024 13:30:59 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 19166161745 for ; Mon, 22 Jul 2024 17:30:59 +0000 (UTC) X-FDA: 82368079038.14.6795FD1 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2067.outbound.protection.outlook.com [40.107.102.67]) by imf05.hostedemail.com (Postfix) with ESMTP id 5293310001B for ; Mon, 22 Jul 2024 17:30:56 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=ksPGxcbn; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf05.hostedemail.com: domain of ziy@nvidia.com designates 40.107.102.67 as permitted sender) smtp.mailfrom=ziy@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721669405; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=77dpwOV1mf8CfX1TKVYlaP3iGgjjwq7Xmbx0eFVU8xk=; b=iE1HdHGluaTPDv58SMFJldGh7IisiXWQYeQu0Y3nnb7tpk2G2gRJA2RFH0xYwwiBehk5vK g0ulV1GG2lygo/qyA9b4+d+yEmxB+SkjoSNAXfFASoXfBTNggMhChF+7G5NIdz4EIlXsDZ 6vXJVjfIE5YKaNKnzdMrSRAZANGFG0U= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1721669405; a=rsa-sha256; cv=pass; b=HoyZXFqkgQaMMKSwltqY87P5MogKvCu5gNw3aPbwQo1wtWHJpQjXaINLEnqfra6lRGoc0d M18W9yfBx4i5fTSgM5ma0eOEBKAFJElIExF5RX50xikpwVw62NyfSHWxXwuuM3IPqWinrz ToZ1iWY0EDDjooOAH7wApPlskjvZ3c4= ARC-Authentication-Results: i=2; imf05.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=ksPGxcbn; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf05.hostedemail.com: domain of ziy@nvidia.com designates 40.107.102.67 as permitted sender) smtp.mailfrom=ziy@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=tbGLR68w2qxynFeG+2lTC8gCCmE9L1yWVJstD/dW4/Gq5NDzmC8JyRkWD84MDATQOYaYXEF1NaxplTs1fHtequg+ZG1Y6iBTUJ32O1AfJFC429Odft6kjpo77KOT8TZSpw7Nm0ewC/qap1lxOG3p9kpT/lMQ/tEsm+vVJ5+JrRJY0Y+80YYV1vHJ4VuoAYMaGqL+9JSaZPn19OkrNfmq0zlxKjgYcod7RC99QfG5H/VohhxfEcLfqpLt+YfpNV+6iuVBFxPMuXUYJ3FMgZk8TXXCc/97I4kQYaCFtROfYe4UbWS1Lwq4pvSRZl0H/C0ArjPAc11dlJvl/WF1TGFV4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=77dpwOV1mf8CfX1TKVYlaP3iGgjjwq7Xmbx0eFVU8xk=; b=vCvucg15FAmhNs5+K1KGyGL+rZexPLdj3zIn2/VfO4CKJG3dVosv40NeRJhBjAO2cKcT7vV9IYzbyNBARQZ+fPjldYRof+UloOiSYGkQ5C5kqltDt6I5RzJNcbf2bqCn0MJ3S26mqzlVD1IRFHcWBFZ3XuJ2qzeBwGT0Q27Djo3pmt50xHUyDDXtmviC6mMAGKBckH/33rTm16/sXEg6qZR3onIAEe5vH3h+z+qXN9f/VHHoyOLLgA2YvkbnU4LWslWo97TP3dL/APfDGytjwKKKHbKXn+ItjnPLYl+oCyx+Y9mLLu9m/12nmdJUpTMN3xzo9pjU3dY/yv+AHUusCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=77dpwOV1mf8CfX1TKVYlaP3iGgjjwq7Xmbx0eFVU8xk=; b=ksPGxcbnz5maQoBUsWSPBvZA0nn+4dJESTSRH6LzEsUgYAFJUOu7JAI/RkMnzlDoer2ouYFYjCCha9PepgAV3r9G8RN91lOjhYndRuj8lRXnojV5yS3doAxfgOWvLWcYOWg2+ztwhC9Lj7w7wGCTw5Tv3H7GzVN0E/8SY0gxKbISyjaa12M2vkjgJJv2bHOVAbrtyB+lw3ViOFR/gSY4nyZYVNoZFyo3m4GarjIG3F/aQ8F0oJPBwsPXahSYD5Cu3iXuVufNPEH/lZR6cVfb82whXwzwDprbUG5uQTsTbDA53vAjgzkJBO1xg4CTqGFyEJlZekQZz3kn3c3P3d6aDg== Received: from DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) by SJ2PR12MB7944.namprd12.prod.outlook.com (2603:10b6:a03:4c5::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7784.14; Mon, 22 Jul 2024 17:29:24 +0000 Received: from DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e]) by DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e%4]) with mapi id 15.20.7784.013; Mon, 22 Jul 2024 17:29:24 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , "Huang, Ying" , Baolin Wang , Kefeng Wang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v2 2/3] memory tiering: introduce folio_has_cpupid() check Date: Mon, 22 Jul 2024 13:29:16 -0400 Message-ID: <20240722172917.503370-2-ziy@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240722172917.503370-1-ziy@nvidia.com> References: <20240722172917.503370-1-ziy@nvidia.com> X-ClientProxiedBy: BLAPR03CA0078.namprd03.prod.outlook.com (2603:10b6:208:329::23) To DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB5744:EE_|SJ2PR12MB7944:EE_ X-MS-Office365-Filtering-Correlation-Id: 3ed615e1-d660-41b3-4ea2-08dcaa73d4b6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: thTG0HbTUUzV20Qf18YnacBVhWtbTuk94UKVeyYXQrrz/LmXzXpJJBKqE5RcWUJhYkIIU9YrzVERZDN+pzrLEMso/HsUPCPjnQET2JDwPd4EaUbF72QcEw6N+xOf0PoMvi5sfNOKpYIFEkPwm3QBZU/qCG9LMxWSfvwRLdN2iL/2eKV/VHRHwD87rx/1U5ZhIUJD2Ceh9/UwxGjii7/FOLvIcXg/HyJzonGyOAZvJRldCJRuqSmMJvMT/0LQya9QZZVzM1dU4N4kpRBjp1vU2IzKlyyWSTrR+ROENGjEsrZgnBMYuUyEGwuj0d9UyPsggPRavJ/xOO3RjiDuj/eioQSzVUCcb+q51yb9iYfAS25dN0cRMYX6PpabeuAJIytzxOY5H3LJfAXT+PIN5XUYZR3iL4zFstPhDYrswEf3EVwX3zinty1oqG6f6FK0NRNN8oCMvI8UdA9GfxjQI8worS6d8MOJKfiV6e7ssNalLBmauxRllKNK8vCfq2+kJDcEVFW7LIyq1MO+pWqYanaOI5ivv6gDHN4eYKRJhnOv7FfGaWlhHIzLs7gn0R4l2aKP7RYM3I2X1+G6N8zRRIJSriOrCmpYjiwPKf3TyWTwBwA065HlDhkIpdimXeJgq8OOqwZ32n8RMMLkeCQUZgRMz2n951oH63z7VfNhDKQNliDznr/8noVMbuFYh3ufU0wWNklBU5KLpMvJL1GeZVTI2ysEEegw6OPvVcLRTWMr2B91TQp7nC+Q0PfG7IltORpOgroBYpMBjHeA5GM/R0kE09tVqHbikI3HvTS6MipfQu4k2lVq5X8c9jF/MI6DImYK7WT6SVJjnNtFLaWgBmQYPF0j6JD66IrmsQqSs3Ioo+H5Hvrkjmyovl/xSXbr1gIXHxQQxlrsjgmUEGlr42Nk4wFhkKA/62/JojNpIDDhnX8xXhWDSzHvYaljx5ZwtsQeMondBRBawTnWp8kJa01rlPGOSn58bSa05qvuEcwLYKmh/MefxZxFDqzwNmNtliVlD1wjzqB/m29xa/nBV+cSqPlXckDjm7I+1RtGyvC7e0kKUddsuNm8ZzoTb/EKyDgYli7XgUx+jxHLWI9cmUQqfqXbOzzOqN8ezuUYfNbtfxis09j0byZpGjy6CnRoVx2M+vSUIBD1vqgo2w1JNsICcZU3VwVxbGQCr74NInnszUp/n2cV2gQYAlVhKjxpkADn1tZQLP0VA23mOC+A37RROt478PZdpVRcfOMAjGkZ6sxGreXYWk0dYVv1bRKrTND/ljS5UpgL5pUoMpNLsARZNw8lG9Crv9kC0R56E5OsbkxFPlGxglTHbajHojf+dw3w06qPVKxh1qYgYT1uoNpIEA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB5744.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: +37r/LRMNW2pH/Ju/uEK96NOThkceTG0otMZgkgnEtSLStSsW/XvAlOn1/oz6M8/If0xKoJLgPWNCX9zf0zmTj9A6ls2V78JQEhVfvYAesXOCAwOLG/6anl7uvZM2hluSM16yfuD7Y/dC58CRtOfyf9zP5aWuWJPaHWVWSsxHtzY+E1Iad6Gz4YKZfZ/PvzW49MaHEtih53SI8LPkvaGaxYJpNvaXn8RTNWS8hV3V18QC4fE1DRtLN+0DyVaHSzKTziRh835ahJaOJw+RA/wq0D7nTAyNnRJ9tDccbuM7Rdmt7THNbMmFz9jdcYMgF+4It/mUlvPLSCBO6oa4aGn5eD5NWwZAJAvOZzt3x31WJrPQjskQRITskbUn4K+dYT5vkhNlZJZ5CGr8xHehJomlaeZdwWE5CnoJLLZ5oa9X6GbIujNSgqc3NJftE7svDkJbYQPqo6UTIUfL3ZeBzjBqsQIS8gF/mlRMsBpTrXQCBl0Zk4JVrKNK+GiTM0xpuktOuWugMb8rMuQEWXXqrfDfwTb6zukVVE8fFBXr4S+XMpvQL6lv9GVMQ6YaQQ0nIDw1Hn8DdCM2ahOJWv+w2bRwruq+M9JKPDAKf/CfDENu+j8IFWO59XOP4n49HS7wbZ9Uh1wDbc52k3bELGfWitkZ3ShHT2w15pVRTQZI/7S0eokJKrJPeTeyBlMeF6Q5Kgfg55co0FqISsYiXkE7kvaAph5DAb8uEz7VClD6pIcqAxNlb7ujtKvUjRdz/UVYdW2vJim0Eb0ONC4dASMDcDkCyQe7MI3vJBsEzKHmo09pKmPyIj3N0D5vU8K0ZOeOxPRoMgZYISYd1x6gubM9+rSJh+3XZvuSvgjBAn4yZenYTckdwgVP6j74e7zJgj6YLn7t4jTifgUDcM34YrBfpMyNMkqQyo1GIpLiQ3AsasLyK/Uta+2h4voNxusvTIDHQFWRfPhs1xiWxiLkf04dJRoeKEEMyrLadQcg2B3ZzOevo+bOevJ4YyffdsHx4QqFgQPbdvHmLbSLEcyIryPcf5c9RS2xDB2XUFQtLAU69JiO22LLm12nTDK3vIw2AMay8Xvm8hHzv5fA3u8exxKT4Hd8140BjNL/4W5iSfcERsIoT7QadSh2eLEHx90xKtQDAOZ7/EnbRgMJIck6r2O8s4PKvepDjOxUrZlZVk4yWs8cETExx1JAqiUSGgPzjMEiC3z99TOE6QWFpkvDb6k2xkjCNEsQyp9GLMlqnsCmmPgy/KK+JZ2a4bUgzCPnKLeGjYy8wdhVNiIoQ4zlUB9wwHDGpYm51qGjW1RNRJMIsFBbKf8RiuGymSOk99Zo+OhPuUHz5qgKy6xCxmiuyHiQsR7OY25164LcIja2y4mZ8hfYU4yI4aB8QLL94uhO+OPSxc1ydi7QWkl305GL/5IMKOBekneXWjsmGlehs03H18mxduMpXZcDwk4dP/Y/Dq9kgbDVEABpJDvlKpSH7BOAOnznSBphzW6mezjW52FWJW0Nnq+TJ5vVkBiuRsp/jeYWfDxRxDUfhLjnp+qK9X8rEl3gjBQVuD8xL4qT3ssSEAHh1E= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3ed615e1-d660-41b3-4ea2-08dcaa73d4b6 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB5744.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jul 2024 17:29:24.7681 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lNFAjMLwiZGC8QBP1OSwHuAebtZPTYic+bufBp3c6b3MH+2molyHBb3ibdXisQRZ X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7944 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 5293310001B X-Stat-Signature: qq3ya4pjjzcmqrnafpxhuq94x4wszcgj X-Rspam-User: X-HE-Tag: 1721669456-341580 X-HE-Meta: U2FsdGVkX19mnEbVj0v0wrFwGCHNv+5EHK9TYUriXOiXjGyToQ0NkOg+Q4tLw/HWX4s4AjAA9pKyKqpv0GtaGSWN7oHmC9J3Pna2qSmP5S3Yf0sFS6JZgyB/LjW73USE0fUtbXIkkJNkUeZVlACJZ8DqkSvYM67Wl06b/ZZJ2/dEHEYJtBZltf9mFXbyVHXv+8NVTX15ny3UA1uwIOHKAIQwCsHJy2zrLI+Q96cDYnzhOs8Rlhv65388hpxLL/7E1W9sTl29ZtRVSoIx+Sa+I8oLVGHad74IF31Q3aEnfpy6NrwDJQeHUc/6IXiockQyBrug9+3DfgyvYSHvgdYD9T53H08FkOFXKjxFTyZt0wUsZgzEyGU7H9yCF6Zlx8n5U5YQb8h7OcBB7yLfe7Ll6kxoUZm3qj+Ub8iN3a9cpkXVQWZYutvzBq9KxZ9IKLUPR7RzpPHR07CtZdYF7o7EWePlrPrz9e5v4wen7je9h5XIUN0Lorq+vw/V4VpRqwzeE4ITQYW8OnBgyaDX53Csj+qIxSPNjGJmxyTGgIWLfcJKUFLRmpBT58HQPqCCuK1ntVohkL/0ftTSKgREuV5er/XQpmD1eaNtUk4NnYCG4iOKur74ydyIdBcM9+3GsGsuBh6q5J9sAPYI9TJvTK75THunL4o3QDu8sxPhHzSf8+QFErUjb/7zsQMubcOUDPH979wvQsOSga647SvoCCfSI7apltfxq+Fm9y1ycMHhjTTlQAXbxse+/KRelwIBk55SAEZ8Z3ky168LyMQnUYOcKmtWa8oVxZgymfoHWwcp5sjEjk6btyTUWy/uoGpDxtaiukNQuV0ZR7lT/LH4zUn92eoOhdpYzg0nS7Ae7lfTtbQg8Eq3Oa9N9h9SdlOsYq2TrpOpbIZInkP1+jUw+PKYIgtD+SmpzbytZQ9pT+tP9a2+jXC50X+BRMVxATW64CxYRwFv82owIrlRQGKF3bL 3PRcmu3r ZvQd3/FEa3/TWzWGChkZ2i4CNbjaBQpgjsmK4BujyK+vV01c+M+AY+8aYClWs/4xYid1reyJGM4ezxCmzmh6d3rU/tCY5Imgtf6QtGgEoVxW2ovp/0qdYfNF1tYrJ8nSAkKvbPW/uNqsgH3ZP2ioRaGwdRgI1CfHeoukRVjJJlpPIb+TSuiTyIGs42A0+c2Oqigv1ifxf/6fd3bcp82oYhF13ghLw6SMTlNf/IcdsYozzwRscGwxJHu5S+Z3mIkjnjaTNEDEn561C9C/zI2Jnh3VLwTAacNYKdsACZQej0+2MNfbHLPHjcHpVUHh53KpFAEdVbsYFNTxku6p3mXYxfeLZfQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Instead of open coded check for if memory tiering mode is on and a folio is in the top tier memory, use a function to encapsulate the check. Signed-off-by: Zi Yan Reviewed-by: "Huang, Ying" Acked-by: David Hildenbrand --- include/linux/mm.h | 6 ++++++ kernel/sched/fair.c | 3 +-- mm/huge_memory.c | 6 ++---- mm/memory-tiers.c | 17 +++++++++++++++++ mm/memory.c | 3 +-- mm/mprotect.c | 3 +-- 6 files changed, 28 insertions(+), 10 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index c227f22ba810..048b2a56d8a3 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1738,6 +1738,8 @@ static inline void vma_set_access_pid_bit(struct vm_area_struct *vma) __set_bit(pid_bit, &vma->numab_state->pids_active[1]); } } + +bool folio_has_cpupid(struct folio *folio); #else /* !CONFIG_NUMA_BALANCING */ static inline int folio_xchg_last_cpupid(struct folio *folio, int cpupid) { @@ -1791,6 +1793,10 @@ static inline bool cpupid_match_pid(struct task_struct *task, int cpupid) static inline void vma_set_access_pid_bit(struct vm_area_struct *vma) { } +static inline bool folio_has_cpupid(struct folio *folio) +{ + return true; +} #endif /* CONFIG_NUMA_BALANCING */ #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 8a5b1ae0aa55..03de808cb3cc 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1840,8 +1840,7 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, * The pages in slow memory node should be migrated according * to hot/cold instead of private/shared. */ - if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && - !node_is_toptier(src_nid)) { + if (!folio_has_cpupid(folio)) { struct pglist_data *pgdat; unsigned long rate_limit; unsigned int latency, th, def_th; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 825317aee88e..d925a93bb9ed 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1712,8 +1712,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ - if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) || - node_is_toptier(nid)) + if (folio_has_cpupid(folio)) last_cpupid = folio_last_cpupid(folio); target_nid = numa_migrate_prep(folio, vmf, haddr, nid, &flags); if (target_nid == NUMA_NO_NODE) @@ -2066,8 +2065,7 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, toptier) goto unlock; - if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && - !toptier) + if (!folio_has_cpupid(folio)) folio_xchg_access_time(folio, jiffies_to_msecs(jiffies)); } diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c index 4775b3a3dabe..7f0360d4e3a0 100644 --- a/mm/memory-tiers.c +++ b/mm/memory-tiers.c @@ -6,6 +6,7 @@ #include #include #include +#include #include "internal.h" @@ -50,6 +51,22 @@ static const struct bus_type memory_tier_subsys = { .dev_name = "memory_tier", }; +/** + * folio_has_cpupid - check if a folio has cpupid information + * @folio: folio to check + * + * folio's _last_cpupid field is repurposed by memory tiering. In memory + * tiering mode, cpupid of slow memory folio (not toptier memory) is used to + * record page access time. + * + * Return: the folio _last_cpupid is used as cpupid + */ +bool folio_has_cpupid(struct folio *folio) +{ + return !(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) || + node_is_toptier(folio_nid(folio)); +} + #ifdef CONFIG_MIGRATION static int top_tier_adistance; /* diff --git a/mm/memory.c b/mm/memory.c index 802d0d8a40f9..105e1a0157dd 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5337,8 +5337,7 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ - if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && - !node_is_toptier(nid)) + if (!folio_has_cpupid(folio)) last_cpupid = (-1 & LAST_CPUPID_MASK); else last_cpupid = folio_last_cpupid(folio); diff --git a/mm/mprotect.c b/mm/mprotect.c index 222ab434da54..787c3c2bf1b6 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -161,8 +161,7 @@ static long change_pte_range(struct mmu_gather *tlb, if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_NORMAL) && toptier) continue; - if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && - !toptier) + if (!folio_has_cpupid(folio)) folio_xchg_access_time(folio, jiffies_to_msecs(jiffies)); } From patchwork Mon Jul 22 17:29:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13738980 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28FC6C3DA5D for ; Mon, 22 Jul 2024 17:29:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BAF366B0085; Mon, 22 Jul 2024 13:29:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B5F706B0088; Mon, 22 Jul 2024 13:29:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9D91B6B0089; Mon, 22 Jul 2024 13:29:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7F8606B0085 for ; Mon, 22 Jul 2024 13:29:45 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 299691C417C for ; Mon, 22 Jul 2024 17:29:45 +0000 (UTC) X-FDA: 82368075930.10.21F114B Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2068.outbound.protection.outlook.com [40.107.101.68]) by imf20.hostedemail.com (Postfix) with ESMTP id 4B12F1C0010 for ; Mon, 22 Jul 2024 17:29:42 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=byzqGfah; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf20.hostedemail.com: domain of ziy@nvidia.com designates 40.107.101.68 as permitted sender) smtp.mailfrom=ziy@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1721669348; a=rsa-sha256; cv=pass; b=rfLKVT9McSVGO2/iejotoGoDUuY3sN0jmdKiT1ICXE+51iSwy60zsrTda+grwpx9XToeqw fgh2t6uJlw9iCjs/uo7X3YvtwK/sVfD9B8Ey8TOVUD/E04ygO1tCJPNq/YakQfmZvbaoU9 k40Am8zOIRi7+RrOKUXys3kA+nbRClg= ARC-Authentication-Results: i=2; imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=byzqGfah; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf20.hostedemail.com: domain of ziy@nvidia.com designates 40.107.101.68 as permitted sender) smtp.mailfrom=ziy@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721669348; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=abeVhQpv8bi5AHWi4oC4YqHkrMU7IYRkJWoM71dK2iw=; b=32LG9/miQxMtQCQ72DHYnYCq/yDmLNzYEvNyv/lGeSnvWGO7l5LodlR792qNjxgiO+VcY2 fPBw+80VEKmqV3ukzpyVqb0Ahv6fFj+7YoNbQv9/BRj4co8D5Gy5aQ2GHViaVfm+vZmXuh 6FlvGlSPdbumg45jHzbMCxPHrPTUIFw= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=lMKZoxxs2UmzmJa93plXr5tBFLdQm0oa7IN87ivdkMUrFD6zR5Cs+K7rANtAZvX6ehFWtKWCkBg/on+rD2wGk9LSEV0rJP8niNxGnaWNGH2oo1r0Z3vTKcLAaZfX2ytd99Qoa0+HXK3D4oNGPdqeFr12B8MtaFHTiB3hoqyQMgZugb0oLc8VhlbdwN+7DThwanVZjVMWG5X9Bh3+g7z/ecbgIMAqvtmyabo96vDCX6esNfpPvLVwuZJ9Iu8Pi9sVfSdKmwkFdaaRYGxtnLQbOV79nFW5opx9XIIbyZj9hJDBT9qmma+LgWnUB6il7NDbSFXJj1/BlyZtnuv8YxmGWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=abeVhQpv8bi5AHWi4oC4YqHkrMU7IYRkJWoM71dK2iw=; b=bg/yKSx61a34F769A3Afxu1fID3xGFULncLzoS159XNfsg9CUcEiOEfpEKzgvNDGGE5Olf2wzD5ZqlRHFp4HwyEsAjqqgOmRXH5EFhSERuXJzgtnaiFOfEFzr0rT4SZH1mIbudN22qgI3aMkg1P/uYvM7vijOKKIbeaUn6cO2bzac+EzQrvOK6oYEoElqMzgNcnT6BALAEYkIjw+xDNvoZjgdOkuOcGnLKwv8nfYb6MehpR5s2vJcJugLvclfga9Ic4c/jWab4oXRQnd8/a0WHEfL9OSClDzqvBY0dYBme3tUVaUqDV6BCQnXJzzTUk9n3v1JMbzkEZgh3uc8r2Tmg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=abeVhQpv8bi5AHWi4oC4YqHkrMU7IYRkJWoM71dK2iw=; b=byzqGfahkj5ucKceEntuVWOzlETRgUhaK7s8NEoyPQddLeRlS+uD/Jcluf6emmXx7RSDlh9EYEbNgymSD0wXyLUt8qEoyqfJIt53FgsAI6K9vDJa+Pl99IkQzfL4mrnr47u3WzTGOZpjQlRYPtMnCgvwaudpgMjGXFxApIKtoRnUvrRp58eprjQLYBa0Dcr2T+jVZBSYSMrqUHpIKLGsL/cARH4f7OOKMaWMbAYmSkDwk7PYOSZ4wB53tqbWbIjIMBXz2JAse3GhbIA8pDz/kaza2qaCjRd6Z6/Ej7oBOIOSaqLnO81bGVnF9sUa8/yS/F9LE8sBI10M8QNChl0wLQ== Received: from DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) by SJ2PR12MB7944.namprd12.prod.outlook.com (2603:10b6:a03:4c5::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7784.14; Mon, 22 Jul 2024 17:29:27 +0000 Received: from DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e]) by DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e%4]) with mapi id 15.20.7784.013; Mon, 22 Jul 2024 17:29:26 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , "Huang, Ying" , Baolin Wang , Kefeng Wang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v2 3/3] memory tiering: count PGPROMOTE_SUCCESS when mem tiering is enabled. Date: Mon, 22 Jul 2024 13:29:17 -0400 Message-ID: <20240722172917.503370-3-ziy@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240722172917.503370-1-ziy@nvidia.com> References: <20240722172917.503370-1-ziy@nvidia.com> X-ClientProxiedBy: BLAPR03CA0064.namprd03.prod.outlook.com (2603:10b6:208:329::9) To DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB5744:EE_|SJ2PR12MB7944:EE_ X-MS-Office365-Filtering-Correlation-Id: fe09ea3b-d43f-447f-4757-08dcaa73d598 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: gI+yc9wiB7dIsN4/nuMz840TS6L/bx9ws2gfrKckFKMJPnAlDSn085UmE2/XYtZNdzDlz/oPRQhUzglEOtj2ePK5CvF+L/Lzr/JGEdkVagNI3V/Ft79xQ0iW8VfjyKZnzv5gczvVjx7zncKcv6D1MzwTmmJnMgQ1sCUfPyTTOmqD4V0sRtjbuIOdjwFTJRUp/EzjEepqHR0zkD8warGpQVHHMMwrU6t/Aq5g95QOKcHmkURIrQqqH9vklN1LlENtjMeiC+dLq9mQ1/3h3D2UfbucgoJZDuxto4gxEdEq31paYgNX0IAeH2/iYZOaFZsvD3fuZdiA50WDZ4pT6gQkZHvrJ+U2QwX5r2KAvlx4ZNRM/d2LxBitCGMDYjymmp6eO8vjjk2hMipR/+/jLPAaY9P75zQeUya0HZS87HHARrfKdrW13MIwoulMO6+uvCbOBrcK5d1efelZPAyZokT4GHvzb+x8Hl8Z/XZqS3yDtbIWeR/H8PFY9Zi3KYOsqQsDxLhFrzhxvOWgE65vWz0h6+zKEQjxXIdSzLUIy4bzv0IKYyLjFdVNT6P7wcggmbsKNaS9YkKT89FCfdC7diMJW44m4W1CQ0F1mOwEGpTHEH4ONP+kP0Yk9G8huwtmTxIS0dU3DN4PCTOS2W/8aZ1EG+Zutay1/m9FZw1o7DlWcKBiEfJXkrNuy37+D8VcVyvQPPzjD4nWEMLNMctIMRxyceO1cYHRQ5LpudwwjPNcdgoV8mqiqrRdrS5rWzToxHgGmASsf+w+DHivk8gw23E8GkOA2aVvYFJGnJ7BLQTBPlg6yU20m7CGgqSp2GzCzNJ5zcy6iSaCnS1gtsB3n+5f85zLi7V26cThTEOflOhsTBE1VjRBTsJSm/1sjXtpQ+TI4gPgciEXhf5iLC+aSBCsSdkyRcOcCE5fykgaxKA8uPADypInSOkuE/ShMn2GctOSdokubL/TK+T6aQvtRrAQ/5OMlL4v3QuZtxdUB6XzzFvkvope4BiOrp94wlQuIOD4hUvSTrTWvysiAxzy+qSj0IYLPavE70++VHSuPob61fJS5pgCwdhqgtg5GbfGtk4Khcbc6hTO/Jq34zKz8A+LhOGbd359/mE5vYv6xshEXeb4hhIdgX1JWywMRBFxERm4GSP6svcKkg7uaoeZdoJkicj/h+BnG14Oi/t1A+ZgRAm91pOFr+iH1tHWYBhXGUr1OCf7PiDDKHaZgIljpax/rKvWE3jkUBLbep8Hcr8t9V8ghAKqpMRb8wn5eSKwvuPnnodBInzgZdb2Kgg/0/3hF5wWq/6n6S+CJhoHD2oWYkYWKzASargS1HUBubABrt51ZA4F6pMLnsAXRh/ZW9hSTUqhnNNVWHq2GXobW7u8pl8= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB5744.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: pZ7HZ8Xi++lCd+ZMkvDThWzUVzdKzDGPtIvo2/eNWKN7arqG7GRLMiW7vSQFqio9ckKJoxSWjDaj6mQxhelUmNHDggiAKWkVcwCcViP1QdWr2Ojcop/O0kEzuYWdQbEcE8senDrUCpJflSF5rqKswSqtF6V1puVCZTphsF/CygSUZJi63DN9qvy6WWeVaJAllPosYikfras/z2mXTM6jWRJFRrYgAH9lYESwnVs+B41O7rz8EK7FXVgKpvCvUbSGqcOP9HHkT6c5+C+KytgxXNa6IdIF2ZUePlj0FmjTiYDZh7iE6jTXFj1/K27NsP6FXU4Mf/XbPniwFHTh1YHV+NOgqWaQYQupeX1l3z9YberrS/Ho/dkoIyzkLL/R8VFrBp0SEUw9V+i7ZmHBKmQWDIBRPet1dpgM2U/h32sAPraCmWNHtK99OAsgTxfFEveBYGySj00n6mR980j07xBPrjZDXk6Zdk6knHR+M5gsyHt6UqvZ4sN08lfcMl4iiaiOi+sHqkOzisYuc29zE4DbXWd+1lrvom12OJ2uCSGWavPlaS10vfSLmwHDNtrrcjy/pvyY3FzRy3BVkW2VJvju01AXN6I+V+oM1Y+TjwjeG2pJ7S7ABXhNGjy9KMuShDIjE3qRPE15WizWXomQBsVlsIsFnWnTr9jlaZU88kbZqvYMG3BKWKTfYNnqedXNorXUw82wztbf9OG61EsOvwSp/IyP80NBxdHlrTeFkUMrmwANcdMJoR9fCEhHEDvGZnaTOyTgVhU9SGuSKc1dDq8Vi9meDUBwZb5lXVtAFPji/3TBfbosUy2rvSrzp9AR4gziSHyuuu16d2r6W0nexqzD0N7J1PnCCc4VcAqnmyxhUYs7pGgZlh5KQPa0lss5e54O1k4oMs9g35JiO+7OmhK+zwWqU6YJ4AvpGQfSuofsCG92OJV2qsX7CATtLYnTR9ZF1zVV3MHiVt9jcEuAsYLcKrqcOQ1Ntw6Ew08DSwIT/Lcq1KF1dBzTmb54QcU9aZMpXctfdtA2a0Id1HOZw29nQAt6ZFgW2YnBCr1T6YmaR7Fla28q2wVAuhHxAGDNKjc4XyfGxCAXsnmqrcA8qNzCHWbTAfVA9J6P3fFJatM6lIhF/DfkDXns6+o41OyFRPtluL3f7BeiBNRDBkEDPmWPVkFU73FTc27AUO0bVeJilb3F2EH71v7KAeWsOmBDj8WBSGxxwAPAAfBNVWExTuMFB6Dc36KLkUKkrK9YO6MOhaGDAGfVaxCQgzd9oIjzE5WNgNi7oegajxusZsRjqjV1G1I0P+8+5SZUuP1OkePazrzfRweJxU+KsLUXdoq5NFDAedcorT2Mv0qLtAx5EOz9tliD8Dr1fekiV251J9XJTPnYm0UKkdCpX5njahtxLPI7v7m9ulQlbRPwmE6xWCqr2gB4QWJMUf6EW0RvN51P5ZzFEtJuj8xPqwma52H1AxP4/71bS+7yIYA7i7PAFJagua1q4B46Ka0M+UpN/MhLkT01qBXl6e9NHCVrrVmVWVcB2mQ52nEE9nllGybNxwoEAoW/WBsfODPAsT0ksFFN4rw= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: fe09ea3b-d43f-447f-4757-08dcaa73d598 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB5744.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jul 2024 17:29:26.2515 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: WKquAQ8tIuZ7Q4OejS+hs7g68br9fL/R+ErAR1n3SJub9FOFb43xF8OW1csG0xd5 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7944 X-Rspamd-Queue-Id: 4B12F1C0010 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: a7bsz1grawnptebkicuy8ogy345u7gg8 X-HE-Tag: 1721669382-195355 X-HE-Meta: U2FsdGVkX1+tp9+HtEsgBNFQWHOnGs1hbxyBDP01MM3SJn5KERy8JSI7Wa3LcHzk8oVnYjqgX4k+dpCUflqR4cy3b3nFbWK/7231oQtr0YUjfqs2rPVyYAnFG7b9fAJrKWW7wyUkmof+CBuCfimottSUkm8bVXKZAqbabzqRwl+PdPNgRMXWqme2sc2txYMboCtD1pH2IMf7bTOpB70CT2RodwekDsinc3qenhECBWBworBqKagjI+URATWTOOYJFccS82D2xMI4r7LXCeVvp7hTFFllMRlgmLIpBXwEXpejBQMJhjJb3Ln6aRseMeT7tv5L+RV4ahhLv6SSl2A4o1CfTGV71dEzGCxGVROmGvXjdh7WxOg1Ng59e3i120GaHV9JLqrGkwIOq+9xbcjOrSmKeNXwEcv8+8nMJdhPxqfQU2rg47Btg1+HNEbHEiAuZY2E7uPMr7osXWQCNooBxe18NlFcybILnPjQOIf/1zfKPfCg+PyZ8ZFRxD4XCd1ZqrLM0W+lV2+SFJqhaYUxRsjcAz2E3ZFTthLlb/t/bo29XwjQckhQL6FLMm42BmS7u2OgEVOCXismSlzBWyXBX8pd1TMEdd7q0NG0R7/pZSghANJxxPRBrlIzA/RfrRN9JPv5YXoJyCPSoqmFE3qMiefjKqdEZxcnxLtYaB08TNYuA+DZBzwJ3Mi4LhxiEg563fyMJb/A+mUtn2CSs7AV6h9Gdw99JZpXFFGwszHQVOFDagpcR+RguuhXQmXqbM03bQLoiTJm6esrBqsUUeaBmK9/rOTiwaGIL8Rq+hwi1uyT85yRAIBwfiMUmbcMF6PhFVXS7AicdhQnHsDLRdTmTQr7bEcPKzZo8Cz5EI10j84aaf+Th6Uvsd/1nhH684DMdFwm6LpYfyoZCJM3a3vD7a9iau/WLKcpfT0YHg9GBnQTcFFtC0jHKAoKOxk148qb+Nra8oO5ZAltdTuCt0m aq6e6Lvt VPSogiiIW+2Cttdmd6FArXl5XUAVOpXwv4S/sFCOViAvyJxjH4+ZGBkH8/bQm5Qg7JsKL3k7beUgzgXiOacxvaSW5iNUk43re/tgHrP77PQZxkDebC3tytAL2n1w7hyZVues2Puhz6qOUBNOogNzx1JR4Pk5Wh5QbdPLJpJXa7i6puYKx99YZkHoIgKZ+v8GISjvbXfSiPsIilMq+Da6uA3vJU6FOeqW0CR0Gd4TfVeif8U3OWY+BE4bhaqdNuu7NhVDKVe1ANaqJEm7aHekkC1J0e01UFlElFGHyuOQzgI0H65DOMvG3y/CK3xpd9aVx1V8CwVeGiEYe7Voh7F6ikz9qlwxaI8IhGnp6lTTSTZ3IYwOIfpAn7vNr5PqlC4m6C52VBbU3WnLhlJk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: memory tiering can be enabled/disabled at runtime and sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING is used to check it. In migrate_misplaced_folio(), the check is missing when PGPROMOTE_SUCCESS is incremented. Add the missing check. Reported-by: Kefeng Wang Closes: https://lore.kernel.org/linux-mm/f4ae2c9c-fe40-4807-bdb2-64cf2d716c1a@huawei.com/ Fixes: 33024536bafd ("memory tiering: hot page selection with hint page fault latency") Signed-off-by: Zi Yan Reviewed-by: Kefeng Wang --- mm/migrate.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mm/migrate.c b/mm/migrate.c index bdbb5bb04c91..b819809da470 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2630,7 +2630,9 @@ int migrate_misplaced_folio(struct folio *folio, struct vm_area_struct *vma, putback_movable_pages(&migratepages); if (nr_succeeded) { count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_succeeded); - if (!node_is_toptier(folio_nid(folio)) && node_is_toptier(node)) + if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) + && !node_is_toptier(folio_nid(folio)) + && node_is_toptier(node)) mod_node_page_state(pgdat, PGPROMOTE_SUCCESS, nr_succeeded); }