From patchwork Fri Jul 19 14:43:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13737365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5DDAC3DA59 for ; Fri, 19 Jul 2024 14:43:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 711E96B008C; Fri, 19 Jul 2024 10:43:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6C2026B0092; Fri, 19 Jul 2024 10:43:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 53C036B0093; Fri, 19 Jul 2024 10:43:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 35E2D6B008C for ; Fri, 19 Jul 2024 10:43:32 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A81C31A065A for ; Fri, 19 Jul 2024 14:43:31 +0000 (UTC) X-FDA: 82356770622.30.C2A5B40 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2077.outbound.protection.outlook.com [40.107.244.77]) by imf10.hostedemail.com (Postfix) with ESMTP id EB9A4C001E for ; Fri, 19 Jul 2024 14:43:28 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="h2o/fcHt"; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf10.hostedemail.com: domain of ziy@nvidia.com designates 40.107.244.77 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721400167; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=Ok5Z7WUVVc0vxIQhoTCNqnFoukcLJLOjfLxV4AqhyVQ=; b=mbjZ7kjtabVIV8x5I/b65CDMLzblU42M6Tsnh5IFqDIAYBQV+dKYUdbxjXuxGsd+diyns0 MXH4C4eAUbzDmbsaowkNQUlf4V2gUBryv35s8W5Zsv8HPmndRapCGqzK15caykgamIcGvU Si0FcP8K8zJVWuqg1Y7KbhnaC9nd9GQ= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1721400167; a=rsa-sha256; cv=pass; b=cE4g3cDA3v9q1MlKsxYVfp7GQAfKRA7UIfodYU32UDqTYUXf/7vm5DJgn/jU3HSamnqfaH 752hi8lW5Xg5dEqdJEJBXTFKK9FTx3naGcfJpP6vfPfIdWfGAWi86R0Qn7oa5u9DafW3I+ EFiOk9wGmolI/SGC0U0YAY4jg7i0qeU= ARC-Authentication-Results: i=2; imf10.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="h2o/fcHt"; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf10.hostedemail.com: domain of ziy@nvidia.com designates 40.107.244.77 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=aOQQkJnQ0JclTI4QLLBJWL+CTvQpuM92QfOVXbuwvX6OGx76cIMVB+43dIbzUHFomQTCqg9XM43vTbwvjd7k7heIRCox7OYtillSYlCd95BZxq04fYo1/YSz34WN3NEPNWvjGNjY/ezwJHcEBNy+z3iDQZQZum4fL/nvUJQ8lziLZz74GjWHOxtHf3eo0QZkU3TN7TSK9kG2p6OGnK+5kwOc7a8gTpJ2pU89HkoK53bHx865cTW0ub5Yx1S/siS4Bw6jG6FQgIw7lCknHQgVOnK4K+NlhiGSsUBood7+5c2c4iNasgUde6PRnoNUfpJgaDaqvYqSmL0u9EbpU6HUKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ok5Z7WUVVc0vxIQhoTCNqnFoukcLJLOjfLxV4AqhyVQ=; b=rUZujmKRYpEQZSwuQyB2Wtf309XRM8ZLIe9F3zjDmc8Z9rDDGVKJcvK3q93E/ubP9zpklCLtugPE6UJqD+UgA21eLmlPSCsiQiN1zNR/UwOuooC5+FhSJ/VcZvmhD6TD07P8kUgHu/lwSWAu6saM6NefT5eOkvyWzZmIyCycvV+Fxz2p7u2EJjN4t+SS7QYdPSwFurdQDBcSCqCWTBmB4HAj7gIkLwmOAWpBcjHT929scKV+yAfzYys/YtvRe2fS9H9gPQiBDPTteUIS9/yMQeoXNbyYJVjSUXVSmtAICv55V2Dzg36PkVJLgqPcnZ5y2Te1AERLdpjZi2On1LrEEQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ok5Z7WUVVc0vxIQhoTCNqnFoukcLJLOjfLxV4AqhyVQ=; b=h2o/fcHt4rRQDCqoWc8t3hov1XfEsmA+KlHGzpnGwIzy/qma5NA/wue17qmoDGqeI25DelZwyZY0+0gs3J3VPQwKisFmyWlEbpwV60ADwvbx+mzvAfpvQ6oaJnvMWY+en6Ri+//zbQX5Swl6peNwbfNfzuWd+/6KA3em4+zGAyrh0WB50uSQJQ2Ityz/xHP98iq9ToyKvJDHmPuGGnVagHUeDQLjBOW/wyc6qRVGvCy9+OS8BiJSwKKWggqEUddxBIi/3cT4mfXxtsSurV1umMNKunVakp1ujUmjMH6hz+lCffIY89RiNRfbwyAAI2+D5mipVkUzd9J77HL/jRF17g== Received: from DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) by MN2PR12MB4079.namprd12.prod.outlook.com (2603:10b6:208:1d5::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7784.16; Fri, 19 Jul 2024 14:43:09 +0000 Received: from DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e]) by DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e%4]) with mapi id 15.20.7784.013; Fri, 19 Jul 2024 14:43:09 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , "Huang, Ying" , Baolin Wang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH 1/2] memory tiering: read last_cpupid correctly in do_huge_pmd_numa_page() Date: Fri, 19 Jul 2024 10:43:05 -0400 Message-ID: <20240719144306.258018-1-ziy@nvidia.com> X-Mailer: git-send-email 2.43.0 X-ClientProxiedBy: BL1PR13CA0001.namprd13.prod.outlook.com (2603:10b6:208:256::6) To DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB5744:EE_|MN2PR12MB4079:EE_ X-MS-Office365-Filtering-Correlation-Id: 81c4dc1b-e18f-4458-0a52-08dca8011bb8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: 7iokqxxnl5nqHFIvqn/5sEXMaxKsc8QU8zyH27MI4UfGtwqp+Y3MZW0ePjD0b6iYQoPXxpTaBjXuG0rbOHp5Yllp6QvyvQZF55SuTNPwEjD+1vV6UXNPKAuQw/tWavAqh6Zxt9YHjazokCknlxveapnL2ufghQWDA3Ur6IJDW0Zrgbl+hbO1Pb2I/s3JFQIBjkJrrEA6GmQwN5bjYnVjdTyvpUGhaqqDOlU7Xry8OXHvN9KT8vvh868wt/Fy6ucEWanlC5ugWRsvFXCpqX0fDtU0WACsB2lVyms7kwnAKJbc7a17/9+f2x/sowmgrnLReBNS0EH2srU08GUpMrgpdY68OhbG2uJMHDccv1oiYrRhr8YMazfxMyXxuxglWZYm7vY4N/Ig4fqm4FGWCXI4/ZVkIj25rSuZpzk8QyeLru5+Bvgz28XBqu2toGFtxM4fHvNl2FuW4FcN9YDi8oDAT5qlQjDUHjeKB8S4z0tCnhzmvcGK5Lc71+U5SDFMGPhy1laqZRfh7GAdDOmMmZCRHS0wNg5CRUV9Tdt5YCFz35B23n+wEI8yIufzc/0wx9XR6gtWJx/cuFE3WO93EtXyGksTJ4pNTkcuWW8nEogkXAkJVraRyz9ySEb/CfWMwRCaciEDnMHCbXYoYe1N6FIiOrxqjzh0yRX5KNnY6FpIs5FCguXsKZObKzWg1afojjXzx/qcHqxJp6xI4SQwIQcPBSI2rsab1OfqNGqT/LUQ+ud6cGoV7WJDobSYh/r340SjzmPmic8OMiV89duNxJPMqgH87anC0uVAcQnN0bZ4LvOGjTvOUbhfneqk1GImhM5yZ0fpHP6wXzMl8LVhyh4AM1KEZ1z7WG4TlpFPenafltWqFRm8WlsdPUBRFnOgr3ipg9ZUR7A6YNeuJKy1EhCvXvmgGGAWlDwFsQhJUEjwUhVTWXrNrZLhe1t6ID566Ed+mr3HS6tNgDXDYLd02b176oKLqUC2M28NWdIbFkjnWGN/twgRQPq8tBL05zqQvXWYTU/PPibeucFIh7HaMNkyruTGDZ3PnXDgHCQXkA2/a7Bmmry8Nt1ME1UxSfNmOP/Y1nB3Juo9kNLj03mctdzbTZJM+7JI9aAiS1uBE4EWb97tDZpjrZwq55tbtLIsEora4kVnaT5VNI2eepcnI9Lw4eyKyqOPGRKmzAzcmDeIjMnfCC9VIHwXjG9nyzLgaNHaKOYFIJIWVOrxYZ8BvW/SXXIxFJw6Ov0APXW8Ro3r5zWaBF0ek42jzaJ5oEzOJgABKZ/NVWH6yKuxGp3Dbjh+96yNKmsjY/ZPq9TTZONkMxvwbYvCfMGlz/eu4/kmzB0hCYCrEHt8SElG6h5Iy2MGSA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB5744.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 09N2uVU32yBuwAfkSJGDfvkeTW0cTjHSiqVzKmvgtgiteYAQY6uCwHmN32yBM+wa4C4EnYUNpQoL9y895j2AutruB2FkLdbnsQIoSM07vmVGJfqlp5Ljk1bkl6TSnKnOU0uNSFNSS6XxVACXl9G/DR0Yk26d3OOiQmzHvxwK6x0uhdEzRGtinp56jm6eFgjqt/dg36gZZKAzlXjknFCGv4xHO7ux667poTBXlHlFppPoMLSaMVKRgKvSWxVlqUjiszdXAD1h1DAJi00yDg1b+m+AWUcZKOm/GHxT6nxfp/kZ1giOGyiLvA2mEnqAT1gWkcAF3V+ATgqgVGMM6SaGtKuACi2uSOCChhlJ9iNg6rJ8dOKCsLZjAcgwHmxifY6furAD3ODv/P9HgamHi41rLWrYk6bP3P3eU58jNGj+WhZQ/NHb0BEjSKoikUoeO0LdUwU+YOpAIp7Kdx2rUJmsjla23ejWJH5C4xmoHvbisNNMNnOjihpinLzSxrGqTUcPVZ1OoGZ65rLdYvDZPYDJO4U+52H/hc36V7vZZT7WwldXSQ0sM/7V2/Yd2HZAcUzbyADoB11UrXV9GInx9977ywy7KXRSEdiTVpULnwTnJFMwr35ZOiQfsqNi0vA9NnI4tRxc0QR70sYM8HSQWj7j8bLNaJ9NGI0R6nLmCkinQIujXP0SehVIa2Tc806xjajfg5BEZKLnFoF7wUQwPGgplTOMPAG3dhy9VaQEFCeVH5Ws6jG4Mj6jorshDWjmO64rlLoQDEHuwWjbjmDm1b5/c6XpBnvcdz2AYVMyVhOdSAtgC7d9PhO1vmCIAmM1+cFwrzUIHjTqeV+vJZgPC9RvXxuxpoOkZPkqKm+dUik5ctddSJrCluwRuaLF4lX8n3ZL7xpPdtL9CscfRLswF13ULtg7pq5Bq6p+TyQz6MVG65GUJUVdZ6VYw7T/bnxUjqgxb2TWcGT6PTiGUw7WYLo+Umr3Qjf3E8sdYkjaztiAwDS3grHmDExv7yCU9+NNH8JMpXT9+XvKBJ+ZolfnGYSyK8DuqhahopjeCnQ6Us0kMDyhYlfAKK+qMoVkR8BuQLLmjuXUT4/FIWP3OH5Q0c0mNJvzm8TwfVekdQtSMpHWTmA6OoyFuGkUTTwFjVwskid9VxzBV07QNoe0vdOOFinaCzYQxvks00BkIUto7Io+E2Q7OY0lj4q8pG+GXuYUtVTtf8MjaHsseQ6FvWM3f5iGunFma/4h5QvFQ4FjWyHniYsv9zQ6tVv3oKD0wBcpoynh51WUq304e2z3P1GzLTmgfRBXEMItO+r5u9DiL2oujbG1ED7FUKIOJkwn1OlmmgRCKqw4MNkCgCQY9aCBXntMo2FvbQ9xxrVJ+o7D8eMxPj7yut6zaG48Z+YOaMUgW3TNYl5OUbNPU9PPZnVCnSXZC35hPiy3L5H+IO2F2HGbW1bezeK4oY/T16sBrYJCv2ZCWSfOMFPq+S2qIVHgisMuqdxtgq2jZRSfNaDDcaAwEWbFkHFIjnMoCvdhnIZ/+BfsVJ/1fFIX2acH/i5iFdzLJUJmXtMfLwEYjGgh3u/58Rc= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 81c4dc1b-e18f-4458-0a52-08dca8011bb8 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB5744.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2024 14:43:09.4650 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2HIPptyy+Z+Aer4jsZvEt+DY1IMxMDtHe6iCPSJPZJGyhPbC+R6wvfZ/b2Pojs+C X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4079 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: EB9A4C001E X-Stat-Signature: iq6b7up4nisys5aezag6gh48qckpj368 X-HE-Tag: 1721400208-307422 X-HE-Meta: U2FsdGVkX1/hVSiVddR6xWWhBJrV09yCQ8kSSSAIpb/bC+VQaIAanaTbMTGBLeW5jrGtpFchVBcBaboxlQIsphAyrslXqIIAN/CztRMANR+wqd/fpBmJmkYGGeGfoRgeeZuxnbXE/do3WyAZyHUcIcYqO5fbaYTq4wPTeLYbyPz/Xa6KFeq87eNYT21d0cyisSAwg+SqhY0vgNPkj5GalMC5CDb9oSZWoslXLFjknBCuzt8Dh9uEtuyAPuIF1XfZ+7GLC1Le2EEy0svqQSpBoLco/wJnS56f2pywtin648EmozOkHDM1rilMgp93wbyw9ljFzFrigxbeHziE6tHv++JqeC/CmEELSi8/udXMUjvBxbwfvS7u5gSW7QwTnRUPCcLkgpmsu3ZkxCGfFbygqSsKEpIrfvbww6W0cHFUc2qbwQhxVFF7Csgb4WZ0nwqZ3keFF+L9goDnyEZaKKFB7kKCi7bAfKMM04ufi20IqiPZ2PYM5sQtFKOtVTrM6h0NI69cLA/UWAPqoXwifktNvwYZs/MPTkBGZOQ1kbo+rvjjOG7ioMXAQyZbdtzJXeCDJyFT2JATr0tVUIWaWZQKPgtfCwc1MMn2bmSDFv5OBRefQlPC5Sbox/9wzKqsNUbI2JAsgifocIW+KEF5Jnus79aiAO1ZJT5362R4E8Z3wzjzBloepi6xL9jP5LCFOPTFlLwrKtSssgThKeE1zNZ7qS4ZwpxUGJOXRyUVIdR9s+ML+YS2zsmjT1lLBLD0+k3mMGlnAohj8hez3IiV880TGs/Ah96Z3yN8BQeqO7Ns5+UZWmuG1ynnb25eliaEW9TVIEsBZJM7eQTqL+r4P5WWwjVN1P4n+PZSiv2cAJCsjIksk5XjwW3zmXgUb7rMXxi6JRJ/RvoMZt5ypfuqZbCYQstmmBSTzThPv/LbBd3mvXTHTrINoLvEIk8kU8Dt66rItevHYZRbFm+PS5bOQsH ZuSMln9e qF3BwlYxfOjeajTkp3buI96l8UyQ6+t1qrfzKM0CP1XFwj51SMNylNoqC4h8Et1/6OHaaVGC95AQlc4NTPgzEzMvVIGp2GEP0UY0UtEBvul9qdtxkDihMw5NII9Dh+sjOW2rsnX2VQGdR7y7p//kbKeCEbpoutpkIFYkf9Fn3Iu1sMqRBM3rJZDxuV7LhjyvHBe4CaoEbdH4RwC1iKdn0oxsmGNNgLR64uQ3O6M71Fsw9NqkaTP0Iw52sbJ+PbgCCCZcHJUqcd8FestPLKayYcdN/zIOzuJeP7Ecf0/Jdwl3vZOmPDuZruieTExeVnkVya6SNc3ZbX7/A9u8uvA5CqIPftPPr62BVicmDNhCp4H6sIPOI7VqURilUjSBrtxT5lE4NH8pueXL1e1Ieu/Xb7XkJgiji3TooLPRDSiGJZp8eV0P922WHBPJihqAD5lhvIu3O X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: last_cpupid is only available when memory tiering is off or the folio is in toptier node. Complete the check to read last_cpupid when it is available. Before the fix, the default last_cpupid will be used even if memory tiering mode is turned off at runtime instead of the actual value. This can prevent task_numa_fault() from getting right numa fault stats, but should not cause any crash. User might see performance changes after the fix. Reported-by: David Hildenbrand Closes: https://lore.kernel.org/linux-mm/9af34a6b-ca56-4a64-8aa6-ade65f109288@redhat.com/ Fixes: 33024536bafd ("memory tiering: hot page selection with hint page fault latency") Signed-off-by: Zi Yan Reviewed-by: "Huang, Ying" Reviewed-by: Baolin Wang Acked-by: David Hildenbrand Reviewed-by: Kefeng Wang --- mm/huge_memory.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f4be468e06a4..825317aee88e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1712,7 +1712,8 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ - if (node_is_toptier(nid)) + if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) || + node_is_toptier(nid)) last_cpupid = folio_last_cpupid(folio); target_nid = numa_migrate_prep(folio, vmf, haddr, nid, &flags); if (target_nid == NUMA_NO_NODE) From patchwork Fri Jul 19 14:43:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13737364 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 000EDC3DA5D for ; Fri, 19 Jul 2024 14:43:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 84A616B0088; Fri, 19 Jul 2024 10:43:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7FA436B0089; Fri, 19 Jul 2024 10:43:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 673406B008C; Fri, 19 Jul 2024 10:43:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 48FB36B0088 for ; Fri, 19 Jul 2024 10:43:18 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id BCF2980725 for ; Fri, 19 Jul 2024 14:43:17 +0000 (UTC) X-FDA: 82356770034.07.5B590AC Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2064.outbound.protection.outlook.com [40.107.96.64]) by imf22.hostedemail.com (Postfix) with ESMTP id CFC9BC0021 for ; Fri, 19 Jul 2024 14:43:14 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=rweOgWAC; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf22.hostedemail.com: domain of ziy@nvidia.com designates 40.107.96.64 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721400152; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=I7ArFvuePSWI2SLkIisr3FxXyJh02NNwl14vvvrY/bw=; b=yWme5Hi1MYwfwooOKN0US76euxdYxIw0wcjo7PkIOauwuX7oe94d//pDyEhSfib/MaMyGC XKFiOrehZYTqBRTc9ZnB4Rb1sZon8RSMcJu5i9eZSzlgY2WdvBI8MbVi+iVt9JTgXK1QVO lod78+EnBMI6BQctHU+adtRAHxPcF+4= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1721400152; a=rsa-sha256; cv=pass; b=xRC4/K4GE9rUgEM1l5pVcJMH/RW/xUUEyoWDN6H/MrAFjFnzvpH3gTN2J6c6wx3k99J2GG 5GChpKq35ayywQkeV0PiF/tEy9+ovj0EdCPvWc2G0uC+hGeApwkWq2MgjHjaVVtMTBfB79 gjXqrXy1ITFhUb4QSrziHQe9FxhPUSM= ARC-Authentication-Results: i=2; imf22.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=rweOgWAC; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf22.hostedemail.com: domain of ziy@nvidia.com designates 40.107.96.64 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=tqxLZQSjT9EX4VdpruIkiICEMs0t4bO7abTNiAllwnh3LfxdbLIUG47udIjCbSSbNXzejybnRHvk8yxJnfdxQZR60tsLeWmo/0sFFgNU2zloWDPUVRTBUUiJxaHG9VtXEIbLCL+e7LqA9i6zdnZH+qJNcFZrfK4Is/+VLI7T0ZsSPgrfTk71Q81P3qcCb2ROJViYgoGac3zc33C4z3ge3YaIH5Xa7vxGrdDguNI4eMW48loeIqdlXldISew3zpoiXiQKiKpmpX6d1phH7r7+hmBBAVRdlGhJzcDINMAKmTVeZ1zpsy61dbflVJwEMeUDhCZY3BQSjXXgrlFARbc1gA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=I7ArFvuePSWI2SLkIisr3FxXyJh02NNwl14vvvrY/bw=; b=GULG3zNCwYrH6VBbH44TZKq5v5LgE44S3cYDZ2DHBYFJmKmCYwRge01xK7b79tn9UPTsgSslU+ZBbF/0N3HsKLhPwO81htiRpKDqbn7aLGOW2b0vI/VqqA33vCkNxWppP+7DlL0vHkSsOd9hfONdTr7+c7dXsw36/7vpKQFhm20P+AyybsypmauecZerjgQWtIBlGPyXhfm0fUKOdnoUndB6/wy4d9DM2hQIjYXEiRiQgoNyWmfs5n56Bsvyx1aaxAp0FXIKyJ4F0a3Xe/lEan1bsx/XoARcV3cRGfFfw268Rbi5bwQgQtkniyhA373SmdGELr7YOrb/yjWDoF0EpQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=I7ArFvuePSWI2SLkIisr3FxXyJh02NNwl14vvvrY/bw=; b=rweOgWACw7miKnvht3vAmCN393VSYPNFD2ViA0qRn16sCbnEM//96/gBYjxG9ueFZf+K17AU2DahRTg5BetcJA18xTTPapZF5eytKLQeBarn9IyDNaA/0K4G/TDXQED60AwLEbtXxaIJ8eWshAF27PL/wo3Mq6PEQcVbKODwYZqI1nrxzYBW8J5ibg0GSdNvw3pltLV5mcRtNPilOmrOOIIYLJQOeAdTqsz8FDYnVr26BxrTnQgr8bY+SESZIUS3ynkLHXAVBeOlMg61KTq+vGVLN8yIVf6bt/WNea66Bd1scBzZOSDEklt7nXoMkwDLcqwyrjO/dzHn/apLEzOZSQ== Received: from DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) by MN2PR12MB4079.namprd12.prod.outlook.com (2603:10b6:208:1d5::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7784.16; Fri, 19 Jul 2024 14:43:10 +0000 Received: from DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e]) by DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e%4]) with mapi id 15.20.7784.013; Fri, 19 Jul 2024 14:43:10 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , "Huang, Ying" , Baolin Wang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH 2/2] memory tiering: introduce folio_has_cpupid() check Date: Fri, 19 Jul 2024 10:43:06 -0400 Message-ID: <20240719144306.258018-2-ziy@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240719144306.258018-1-ziy@nvidia.com> References: <20240719144306.258018-1-ziy@nvidia.com> X-ClientProxiedBy: BL1PR13CA0001.namprd13.prod.outlook.com (2603:10b6:208:256::6) To DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB5744:EE_|MN2PR12MB4079:EE_ X-MS-Office365-Filtering-Correlation-Id: d87e9ce8-72dc-4a2b-3113-08dca8011c6d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: ryJqqZP6Lab9I/2cgGFFWYR4yq10N79LEcRCmfi08pTgYMCVRPNsBdRP6bfveESEX28v9RKHgMb0sSFbIsviHqhITLDNNiYp01AcbpvXK7E7Hw3nYz1ALprrEnnKfDKGhrYcVFRLSJXvx7dLIhk0D6FTwwRuLzWuPtk1xa0miBpRVSE99eK8GFgeiR3Y7C5hiUEoMQYCMsEIQb5Tn8DZn6LjgtFPpgC47ycc0YwhyT/H/HlFN5Cl9WWPqcIaFAOy29oi0T9XioFtCvBOZ4a0y5dGyUkiUgbq3+ud22qqJlWMrw0QcpowF/NoeBev74F4p86F94nvEpyGLIDDY9KQzg1rfWJaBycca1fe/VdddLTVMxyJW3B5CjUtQxO67+cGjlMPRBVMSP7VfI1xMHG/dqKXSlbXBWi3oJzV8MLYHmvzhJhiadfes1PP4oEiEBIjn6pfx9k6QLlhewy5jT56cmVQxRARhT/PtL0ILweobAJtSsmBGuT1T3sxZyIIDTafoYEvbc2UVoaCS97jYN1m5IhQ6NUwhlifggBVNmyAauUqWsxDXbsT/gY/PDUtmoFgToIz3BAcAeAkC54j6pEBqYy8iLw8VLbFxrmt3lN/CpkTymzjiT0XDEoLHBndI3VcGeQJE+yKlERcS7jO+Xwel0V7TQmJJn7DXLx7OVrmoZXMAOi3zlD0CizosXH4BJsfMUb8bt9yzefGfkZSAcpBw05enoMROEIbZel53JfzzvenMecEryICxiTS7L+rxZZkcauRPtZ77JHnPgbvWXi0/c+ya4/cxO+juym/MPeq/Ww6mpGQ+kMjTUXs2lrLxNKiBvuKyiyKV+qwu6++iikjF7JGOYDSI/WElNDpH0uEzPkzhvsmA/MWQsKn4OWkLrUfDn5bB/DwxSQVTB28DkodS1POO9qM49oCtqgahEJzMqVB2GXjUZ4L2a449VGhMkTTY80LH0GvJX37UUY0QVS/Fzx4K+wF61CDQqVT2uZ9f/OWaVR1fQVY+dZCQzqhf3ZuzvxK/FQrhRjcEO2Bu/1Arge5nXwYVVItFaZehtlCNu/GagswVEViCXfT2zZATMpIQXAV034OinyyDnwJsYJhNcswxznIki1WHw4xYcGQ8vhBfuTFdWm+4eQWc3SKIyn/YgReaNgHAMhgCYfYVN747OFrOsQ3YINkEs0TusytwUEm2PlhGPdDW/VZqYUpgtV74hPg1Gbp+wj224hSJlFQs7rrGdVgIKdn35Ebs5T6KsW8jQwm4wXJ/hdAPNIRmJQqkGRcQ8WRrLSYCQSvFLvJb1fuzTNPWxCXcbBgTOpyUtyo1XedITsCHdLCO/R8OxF4F6h6yyu51cO0hbQ/NcN29A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB5744.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: JBVHmu/dkNfBDuqmJY7/Xv6PuOrl0wI6wcrJGKMqf1CQrNvuVSz1kDoHukKth+c092eqyGM4hpR/POKc5jaF3ACnEHkPVk7UswGDAoeEUyxdAkgp3vUSv6qsUX0mujFXsPk2xKMizrnwqMkCOoX5sYnpxVNZ1Ko1YvXTEFnoatM5pF9efDubafF0NbC6XK5TtKkaViGZirm1P3UuOEq+//ZPILCsDUBZLNBEo1u35KaET0BtYYAtYmTXPWD/lPXjG7TDLTcHAvQKDWz65S8/CK+/y/cUkKhfxPByr+CLai6CcL9P0CPoGl7kbVG3sxqkuwqYkIO0E8htA5d49xLR1rTczQyVeGTlrFNYY9FhpyiiBTkvka1/V3TeDZ7RPKmU8WdpGnjK5LcAfI4znTKopSK/wwPLljkEnLDP1ntcSwkdoywW3RuAIM8yYKVfFGCNZhuXmSqKnC8xukIuWh0050rRCixvG3Yqo+yka2a+k6ip4tunK4BihhYK98UASrcRm+D0SPzpPZ3clnZm8akZq6ImdmP06MPfS3MDJOQndF/VTk5sN4NEnVifXoowlnTf4RwUdNHhg1VQa8jK7pUVEBk1PBMheuZbuuELmqXCM16NfiFY5QNquQVwofn8qPO92PJVE1ssQk4lbKSqRfIT2/uVNMFcyBIo5PhJ4w1k/PwZYHoIG5spo/Rzd8XxXymjyRbDrBirpTdhGTvHZnCEb+y5CpyQYDUXwDk1m/S0eZ/ju64ECEsN4Km9X2yU6jw3/vGSPWeo26Anz+t82gQzZZfcNu4NIrS2/M4phv1QhTlzV5Y+xOTbY6MI7BUj/Rjm0ivTsHeeO8HbBU9PaG3bncJjZDi8C6kmJGrxhPRSJGAO0PM9zgt4ilk03cIIvilQdutlZ3s8xnrvmZFP6NC/dNr6jl3bxDwsGbuUIHQlZZrOlSkOUNymLgtQzT/51dj4owROgsO7cB8oX0F6PMK5aDGgPRsQ7d331UsuEligBFBbcMsBbaYmU/ebUDCHC6Vq6iKazsTnceaewEM11OOaSQdQCm/fdAU7+awzBoukWPuYwft5v9PtIzFSo4TcaJElX6X6nIuz5+L+1S7eQRk5Km0Wf3tFrPd9/6to1/CLcv6IRdb024Rxr7PI/rHVj+95I0cmxERY5RIrYArrzCulblaGiefx6C2idyzfeShoao+Px0EYFB9mdI21dgyn0c1LK7X2Blpztt1YhWH1bWszhO305mDbuZheuYd1Rsiju2rGBD5dl9N8rp2fWt9OKZ7whj/Awy5rccuvIoOS5P9OlgbnUOIe46y6tJfs4aVdNt43taRuwcNt/QvD3LPfvAc2HWaTzkT7h/IYM38XTvlAJjbAsiz2zHeDVXsGHL1JgS9/GUwx2GgrAVyIuyq/UojzsBE8S8YREPQqV9RH2Vj+/IxuHji90YJ2dsDFYJwv7gBskL/Xmft0LrnAY6/UGH6W7KFKQjuJ4pU+A1L6mzb+2ehblyIZra7DcKkXPS/Is4Z30EXjW5pWnf53CT9lQnIC7+hQYrpn42q4HVHwOSQZx4KfGVqUtnFvfg0sbkG7aL4= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: d87e9ce8-72dc-4a2b-3113-08dca8011c6d X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB5744.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2024 14:43:10.6293 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rfCvhpCFUX4GXx0czpa9OJZuNZAI3M+AOXSjRw9tP7vz9PBMxpwzmNDk9osuL7nC X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4079 X-Stat-Signature: 48tbaiuny55jx1zu7bguxeuxa61b9djg X-Rspamd-Queue-Id: CFC9BC0021 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1721400194-454454 X-HE-Meta: U2FsdGVkX199k4VxGM+s0qBbM6Z5KJ8yJHIS+DjWkYdS08vGnl0e/ZJaDTZNoIGIuiB7xN2mY7gpsIXWJ0OFtjIxntb6WH+Yt9/3DF3xeeoqObTk3MlqneEMPBHdPMRtk9WC9UZDdoQXWvzw9mUCgzoty5ukWYYx66iwvH0xw7QULCbTtAfQ96uM2eUqjFLw9/xvYLywwF4U9gLrKxmH4Hw5810LgVci+9XFwCnDgr4D0phNlGFCz5ju8AEMCr0d94bJ61r9FIbEhoJoVOrvaC2WFSuUVd44GpHu/j99ue0Rbek/RCVppuTvnZkqBrlrJ0jrGncCoBJMwGx9a2ewbzkEk7ptw1o/1EA/x7VEIQV4f6+kp33VGMAZdmPq7kokGlEu9dUiW14oa3SQfiyW8YmDZTka7u6f+FgLvNRFob39qFhIyqg29YQJkVQe6wBTQF5Gvljr263/ekTRSGAd7Fwo5WoKJSGBc+OwlXrilNPLX7eM/B1UjR2/kkAeP8MoGiF94+xNILq3f13HDuJXLx3YEaCVNmgjl972BXrKNCDfmxIPdCdhcOzKCJj6Tw82m5G5OrC74w24D8aL6QGYBrZe6axlQ1I0B1fXWc5ZX3l45Zm+/X5+P+2690VsKBXmTt5g01GB1DWXPYM8a5aefT6OAWth0g0EupRaA18uMklHcuX/qFGteXjQY4kyhzbN5gJZy5LCebNH/byFt6GL3uJiu9cP3nXm4FsRV0kaN5vdvfveeIOuCeyBE9vyauEv+kRe+ZqdLGTQ5ZrmGrkJMU57+/xwa1SwlImcTMH2WILsaiWFdDEAowDy5TIQlIixm+XWdq3H0loRGn36HtK9vtCfHTXKpcHKWKkW/HkDpcwZWxjmIyxrPkOfU7AQRAOk9Zfc6hN09inYp07f7E3mB1qTmHs+FZZ3BnXxP20Q3OwpuF2jfwaahEm2A5Okpr/6pJeNiCKkwf7NYdgZQCN d04RnC+d fMCSY4DfX1TTqMxBrY3jWzYd0W81KjaxI3kI3PGWTQJe6+Qh+nkAe5R2IN1lg1Tr0l6bvkY/6WtmLoaphX0S0f0gyIONh54hKr10BlLlyh4oicDMW9iY0f4v7uexsM5JUhC53iaD/DXN6ewBmCSyI72Y6OSRgLgNLIXGGOUw0EEceLvlHL14ZH9Ad+DsuyzHqj4iqwagPabAE/l7n6IIKO958eyAEuvEBdYXvdy3XvZS7PsqJ3Z6dsAKh9RuECvz4rxe53kjrocj8Sw8Z4ErV6eH6iHghn6E/yALeD89CWA8c1Oc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Instead of open coded check for if memory tiering mode is on and a folio is in the top tier memory, use a function to encapsulate the check. Signed-off-by: Zi Yan Reviewed-by: "Huang, Ying" --- include/linux/memory-tiers.h | 8 ++++++++ kernel/sched/fair.c | 3 +-- mm/huge_memory.c | 6 ++---- mm/memory-tiers.c | 17 +++++++++++++++++ mm/memory.c | 3 +-- mm/mprotect.c | 3 +-- 6 files changed, 30 insertions(+), 10 deletions(-) diff --git a/include/linux/memory-tiers.h b/include/linux/memory-tiers.h index 0dc0cf2863e2..10c127d461c4 100644 --- a/include/linux/memory-tiers.h +++ b/include/linux/memory-tiers.h @@ -73,6 +73,10 @@ static inline bool node_is_toptier(int node) } #endif + +bool folio_has_cpupid(struct folio *folio); + + #else #define numa_demotion_enabled false @@ -151,5 +155,9 @@ static inline struct memory_dev_type *mt_find_alloc_memory_type(int adist, static inline void mt_put_memory_types(struct list_head *memory_types) { } +static inline bool folio_has_cpupid(struct folio *folio) +{ + return true; +} #endif /* CONFIG_NUMA */ #endif /* _LINUX_MEMORY_TIERS_H */ diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 8a5b1ae0aa55..03de808cb3cc 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1840,8 +1840,7 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, * The pages in slow memory node should be migrated according * to hot/cold instead of private/shared. */ - if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && - !node_is_toptier(src_nid)) { + if (!folio_has_cpupid(folio)) { struct pglist_data *pgdat; unsigned long rate_limit; unsigned int latency, th, def_th; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 825317aee88e..d925a93bb9ed 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1712,8 +1712,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ - if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) || - node_is_toptier(nid)) + if (folio_has_cpupid(folio)) last_cpupid = folio_last_cpupid(folio); target_nid = numa_migrate_prep(folio, vmf, haddr, nid, &flags); if (target_nid == NUMA_NO_NODE) @@ -2066,8 +2065,7 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, toptier) goto unlock; - if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && - !toptier) + if (!folio_has_cpupid(folio)) folio_xchg_access_time(folio, jiffies_to_msecs(jiffies)); } diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c index 4775b3a3dabe..7f0360d4e3a0 100644 --- a/mm/memory-tiers.c +++ b/mm/memory-tiers.c @@ -6,6 +6,7 @@ #include #include #include +#include #include "internal.h" @@ -50,6 +51,22 @@ static const struct bus_type memory_tier_subsys = { .dev_name = "memory_tier", }; +/** + * folio_has_cpupid - check if a folio has cpupid information + * @folio: folio to check + * + * folio's _last_cpupid field is repurposed by memory tiering. In memory + * tiering mode, cpupid of slow memory folio (not toptier memory) is used to + * record page access time. + * + * Return: the folio _last_cpupid is used as cpupid + */ +bool folio_has_cpupid(struct folio *folio) +{ + return !(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) || + node_is_toptier(folio_nid(folio)); +} + #ifdef CONFIG_MIGRATION static int top_tier_adistance; /* diff --git a/mm/memory.c b/mm/memory.c index 802d0d8a40f9..105e1a0157dd 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5337,8 +5337,7 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ - if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && - !node_is_toptier(nid)) + if (!folio_has_cpupid(folio)) last_cpupid = (-1 & LAST_CPUPID_MASK); else last_cpupid = folio_last_cpupid(folio); diff --git a/mm/mprotect.c b/mm/mprotect.c index 222ab434da54..787c3c2bf1b6 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -161,8 +161,7 @@ static long change_pte_range(struct mmu_gather *tlb, if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_NORMAL) && toptier) continue; - if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && - !toptier) + if (!folio_has_cpupid(folio)) folio_xchg_access_time(folio, jiffies_to_msecs(jiffies)); }