From patchwork Fri Jul 19 14:43:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13737365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5DDAC3DA59 for ; Fri, 19 Jul 2024 14:43:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 711E96B008C; Fri, 19 Jul 2024 10:43:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6C2026B0092; Fri, 19 Jul 2024 10:43:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 53C036B0093; Fri, 19 Jul 2024 10:43:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 35E2D6B008C for ; Fri, 19 Jul 2024 10:43:32 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A81C31A065A for ; Fri, 19 Jul 2024 14:43:31 +0000 (UTC) X-FDA: 82356770622.30.C2A5B40 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2077.outbound.protection.outlook.com [40.107.244.77]) by imf10.hostedemail.com (Postfix) with ESMTP id EB9A4C001E for ; Fri, 19 Jul 2024 14:43:28 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="h2o/fcHt"; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf10.hostedemail.com: domain of ziy@nvidia.com designates 40.107.244.77 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721400167; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=Ok5Z7WUVVc0vxIQhoTCNqnFoukcLJLOjfLxV4AqhyVQ=; b=mbjZ7kjtabVIV8x5I/b65CDMLzblU42M6Tsnh5IFqDIAYBQV+dKYUdbxjXuxGsd+diyns0 MXH4C4eAUbzDmbsaowkNQUlf4V2gUBryv35s8W5Zsv8HPmndRapCGqzK15caykgamIcGvU Si0FcP8K8zJVWuqg1Y7KbhnaC9nd9GQ= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1721400167; a=rsa-sha256; cv=pass; b=cE4g3cDA3v9q1MlKsxYVfp7GQAfKRA7UIfodYU32UDqTYUXf/7vm5DJgn/jU3HSamnqfaH 752hi8lW5Xg5dEqdJEJBXTFKK9FTx3naGcfJpP6vfPfIdWfGAWi86R0Qn7oa5u9DafW3I+ EFiOk9wGmolI/SGC0U0YAY4jg7i0qeU= ARC-Authentication-Results: i=2; imf10.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="h2o/fcHt"; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf10.hostedemail.com: domain of ziy@nvidia.com designates 40.107.244.77 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=aOQQkJnQ0JclTI4QLLBJWL+CTvQpuM92QfOVXbuwvX6OGx76cIMVB+43dIbzUHFomQTCqg9XM43vTbwvjd7k7heIRCox7OYtillSYlCd95BZxq04fYo1/YSz34WN3NEPNWvjGNjY/ezwJHcEBNy+z3iDQZQZum4fL/nvUJQ8lziLZz74GjWHOxtHf3eo0QZkU3TN7TSK9kG2p6OGnK+5kwOc7a8gTpJ2pU89HkoK53bHx865cTW0ub5Yx1S/siS4Bw6jG6FQgIw7lCknHQgVOnK4K+NlhiGSsUBood7+5c2c4iNasgUde6PRnoNUfpJgaDaqvYqSmL0u9EbpU6HUKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ok5Z7WUVVc0vxIQhoTCNqnFoukcLJLOjfLxV4AqhyVQ=; b=rUZujmKRYpEQZSwuQyB2Wtf309XRM8ZLIe9F3zjDmc8Z9rDDGVKJcvK3q93E/ubP9zpklCLtugPE6UJqD+UgA21eLmlPSCsiQiN1zNR/UwOuooC5+FhSJ/VcZvmhD6TD07P8kUgHu/lwSWAu6saM6NefT5eOkvyWzZmIyCycvV+Fxz2p7u2EJjN4t+SS7QYdPSwFurdQDBcSCqCWTBmB4HAj7gIkLwmOAWpBcjHT929scKV+yAfzYys/YtvRe2fS9H9gPQiBDPTteUIS9/yMQeoXNbyYJVjSUXVSmtAICv55V2Dzg36PkVJLgqPcnZ5y2Te1AERLdpjZi2On1LrEEQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ok5Z7WUVVc0vxIQhoTCNqnFoukcLJLOjfLxV4AqhyVQ=; b=h2o/fcHt4rRQDCqoWc8t3hov1XfEsmA+KlHGzpnGwIzy/qma5NA/wue17qmoDGqeI25DelZwyZY0+0gs3J3VPQwKisFmyWlEbpwV60ADwvbx+mzvAfpvQ6oaJnvMWY+en6Ri+//zbQX5Swl6peNwbfNfzuWd+/6KA3em4+zGAyrh0WB50uSQJQ2Ityz/xHP98iq9ToyKvJDHmPuGGnVagHUeDQLjBOW/wyc6qRVGvCy9+OS8BiJSwKKWggqEUddxBIi/3cT4mfXxtsSurV1umMNKunVakp1ujUmjMH6hz+lCffIY89RiNRfbwyAAI2+D5mipVkUzd9J77HL/jRF17g== Received: from DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) by MN2PR12MB4079.namprd12.prod.outlook.com (2603:10b6:208:1d5::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7784.16; Fri, 19 Jul 2024 14:43:09 +0000 Received: from DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e]) by DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e%4]) with mapi id 15.20.7784.013; Fri, 19 Jul 2024 14:43:09 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , "Huang, Ying" , Baolin Wang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH 1/2] memory tiering: read last_cpupid correctly in do_huge_pmd_numa_page() Date: Fri, 19 Jul 2024 10:43:05 -0400 Message-ID: <20240719144306.258018-1-ziy@nvidia.com> X-Mailer: git-send-email 2.43.0 X-ClientProxiedBy: BL1PR13CA0001.namprd13.prod.outlook.com (2603:10b6:208:256::6) To DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB5744:EE_|MN2PR12MB4079:EE_ X-MS-Office365-Filtering-Correlation-Id: 81c4dc1b-e18f-4458-0a52-08dca8011bb8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: 7iokqxxnl5nqHFIvqn/5sEXMaxKsc8QU8zyH27MI4UfGtwqp+Y3MZW0ePjD0b6iYQoPXxpTaBjXuG0rbOHp5Yllp6QvyvQZF55SuTNPwEjD+1vV6UXNPKAuQw/tWavAqh6Zxt9YHjazokCknlxveapnL2ufghQWDA3Ur6IJDW0Zrgbl+hbO1Pb2I/s3JFQIBjkJrrEA6GmQwN5bjYnVjdTyvpUGhaqqDOlU7Xry8OXHvN9KT8vvh868wt/Fy6ucEWanlC5ugWRsvFXCpqX0fDtU0WACsB2lVyms7kwnAKJbc7a17/9+f2x/sowmgrnLReBNS0EH2srU08GUpMrgpdY68OhbG2uJMHDccv1oiYrRhr8YMazfxMyXxuxglWZYm7vY4N/Ig4fqm4FGWCXI4/ZVkIj25rSuZpzk8QyeLru5+Bvgz28XBqu2toGFtxM4fHvNl2FuW4FcN9YDi8oDAT5qlQjDUHjeKB8S4z0tCnhzmvcGK5Lc71+U5SDFMGPhy1laqZRfh7GAdDOmMmZCRHS0wNg5CRUV9Tdt5YCFz35B23n+wEI8yIufzc/0wx9XR6gtWJx/cuFE3WO93EtXyGksTJ4pNTkcuWW8nEogkXAkJVraRyz9ySEb/CfWMwRCaciEDnMHCbXYoYe1N6FIiOrxqjzh0yRX5KNnY6FpIs5FCguXsKZObKzWg1afojjXzx/qcHqxJp6xI4SQwIQcPBSI2rsab1OfqNGqT/LUQ+ud6cGoV7WJDobSYh/r340SjzmPmic8OMiV89duNxJPMqgH87anC0uVAcQnN0bZ4LvOGjTvOUbhfneqk1GImhM5yZ0fpHP6wXzMl8LVhyh4AM1KEZ1z7WG4TlpFPenafltWqFRm8WlsdPUBRFnOgr3ipg9ZUR7A6YNeuJKy1EhCvXvmgGGAWlDwFsQhJUEjwUhVTWXrNrZLhe1t6ID566Ed+mr3HS6tNgDXDYLd02b176oKLqUC2M28NWdIbFkjnWGN/twgRQPq8tBL05zqQvXWYTU/PPibeucFIh7HaMNkyruTGDZ3PnXDgHCQXkA2/a7Bmmry8Nt1ME1UxSfNmOP/Y1nB3Juo9kNLj03mctdzbTZJM+7JI9aAiS1uBE4EWb97tDZpjrZwq55tbtLIsEora4kVnaT5VNI2eepcnI9Lw4eyKyqOPGRKmzAzcmDeIjMnfCC9VIHwXjG9nyzLgaNHaKOYFIJIWVOrxYZ8BvW/SXXIxFJw6Ov0APXW8Ro3r5zWaBF0ek42jzaJ5oEzOJgABKZ/NVWH6yKuxGp3Dbjh+96yNKmsjY/ZPq9TTZONkMxvwbYvCfMGlz/eu4/kmzB0hCYCrEHt8SElG6h5Iy2MGSA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB5744.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 09N2uVU32yBuwAfkSJGDfvkeTW0cTjHSiqVzKmvgtgiteYAQY6uCwHmN32yBM+wa4C4EnYUNpQoL9y895j2AutruB2FkLdbnsQIoSM07vmVGJfqlp5Ljk1bkl6TSnKnOU0uNSFNSS6XxVACXl9G/DR0Yk26d3OOiQmzHvxwK6x0uhdEzRGtinp56jm6eFgjqt/dg36gZZKAzlXjknFCGv4xHO7ux667poTBXlHlFppPoMLSaMVKRgKvSWxVlqUjiszdXAD1h1DAJi00yDg1b+m+AWUcZKOm/GHxT6nxfp/kZ1giOGyiLvA2mEnqAT1gWkcAF3V+ATgqgVGMM6SaGtKuACi2uSOCChhlJ9iNg6rJ8dOKCsLZjAcgwHmxifY6furAD3ODv/P9HgamHi41rLWrYk6bP3P3eU58jNGj+WhZQ/NHb0BEjSKoikUoeO0LdUwU+YOpAIp7Kdx2rUJmsjla23ejWJH5C4xmoHvbisNNMNnOjihpinLzSxrGqTUcPVZ1OoGZ65rLdYvDZPYDJO4U+52H/hc36V7vZZT7WwldXSQ0sM/7V2/Yd2HZAcUzbyADoB11UrXV9GInx9977ywy7KXRSEdiTVpULnwTnJFMwr35ZOiQfsqNi0vA9NnI4tRxc0QR70sYM8HSQWj7j8bLNaJ9NGI0R6nLmCkinQIujXP0SehVIa2Tc806xjajfg5BEZKLnFoF7wUQwPGgplTOMPAG3dhy9VaQEFCeVH5Ws6jG4Mj6jorshDWjmO64rlLoQDEHuwWjbjmDm1b5/c6XpBnvcdz2AYVMyVhOdSAtgC7d9PhO1vmCIAmM1+cFwrzUIHjTqeV+vJZgPC9RvXxuxpoOkZPkqKm+dUik5ctddSJrCluwRuaLF4lX8n3ZL7xpPdtL9CscfRLswF13ULtg7pq5Bq6p+TyQz6MVG65GUJUVdZ6VYw7T/bnxUjqgxb2TWcGT6PTiGUw7WYLo+Umr3Qjf3E8sdYkjaztiAwDS3grHmDExv7yCU9+NNH8JMpXT9+XvKBJ+ZolfnGYSyK8DuqhahopjeCnQ6Us0kMDyhYlfAKK+qMoVkR8BuQLLmjuXUT4/FIWP3OH5Q0c0mNJvzm8TwfVekdQtSMpHWTmA6OoyFuGkUTTwFjVwskid9VxzBV07QNoe0vdOOFinaCzYQxvks00BkIUto7Io+E2Q7OY0lj4q8pG+GXuYUtVTtf8MjaHsseQ6FvWM3f5iGunFma/4h5QvFQ4FjWyHniYsv9zQ6tVv3oKD0wBcpoynh51WUq304e2z3P1GzLTmgfRBXEMItO+r5u9DiL2oujbG1ED7FUKIOJkwn1OlmmgRCKqw4MNkCgCQY9aCBXntMo2FvbQ9xxrVJ+o7D8eMxPj7yut6zaG48Z+YOaMUgW3TNYl5OUbNPU9PPZnVCnSXZC35hPiy3L5H+IO2F2HGbW1bezeK4oY/T16sBrYJCv2ZCWSfOMFPq+S2qIVHgisMuqdxtgq2jZRSfNaDDcaAwEWbFkHFIjnMoCvdhnIZ/+BfsVJ/1fFIX2acH/i5iFdzLJUJmXtMfLwEYjGgh3u/58Rc= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 81c4dc1b-e18f-4458-0a52-08dca8011bb8 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB5744.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2024 14:43:09.4650 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2HIPptyy+Z+Aer4jsZvEt+DY1IMxMDtHe6iCPSJPZJGyhPbC+R6wvfZ/b2Pojs+C X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4079 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: EB9A4C001E X-Stat-Signature: iq6b7up4nisys5aezag6gh48qckpj368 X-HE-Tag: 1721400208-307422 X-HE-Meta: U2FsdGVkX1/hVSiVddR6xWWhBJrV09yCQ8kSSSAIpb/bC+VQaIAanaTbMTGBLeW5jrGtpFchVBcBaboxlQIsphAyrslXqIIAN/CztRMANR+wqd/fpBmJmkYGGeGfoRgeeZuxnbXE/do3WyAZyHUcIcYqO5fbaYTq4wPTeLYbyPz/Xa6KFeq87eNYT21d0cyisSAwg+SqhY0vgNPkj5GalMC5CDb9oSZWoslXLFjknBCuzt8Dh9uEtuyAPuIF1XfZ+7GLC1Le2EEy0svqQSpBoLco/wJnS56f2pywtin648EmozOkHDM1rilMgp93wbyw9ljFzFrigxbeHziE6tHv++JqeC/CmEELSi8/udXMUjvBxbwfvS7u5gSW7QwTnRUPCcLkgpmsu3ZkxCGfFbygqSsKEpIrfvbww6W0cHFUc2qbwQhxVFF7Csgb4WZ0nwqZ3keFF+L9goDnyEZaKKFB7kKCi7bAfKMM04ufi20IqiPZ2PYM5sQtFKOtVTrM6h0NI69cLA/UWAPqoXwifktNvwYZs/MPTkBGZOQ1kbo+rvjjOG7ioMXAQyZbdtzJXeCDJyFT2JATr0tVUIWaWZQKPgtfCwc1MMn2bmSDFv5OBRefQlPC5Sbox/9wzKqsNUbI2JAsgifocIW+KEF5Jnus79aiAO1ZJT5362R4E8Z3wzjzBloepi6xL9jP5LCFOPTFlLwrKtSssgThKeE1zNZ7qS4ZwpxUGJOXRyUVIdR9s+ML+YS2zsmjT1lLBLD0+k3mMGlnAohj8hez3IiV880TGs/Ah96Z3yN8BQeqO7Ns5+UZWmuG1ynnb25eliaEW9TVIEsBZJM7eQTqL+r4P5WWwjVN1P4n+PZSiv2cAJCsjIksk5XjwW3zmXgUb7rMXxi6JRJ/RvoMZt5ypfuqZbCYQstmmBSTzThPv/LbBd3mvXTHTrINoLvEIk8kU8Dt66rItevHYZRbFm+PS5bOQsH ZuSMln9e qF3BwlYxfOjeajTkp3buI96l8UyQ6+t1qrfzKM0CP1XFwj51SMNylNoqC4h8Et1/6OHaaVGC95AQlc4NTPgzEzMvVIGp2GEP0UY0UtEBvul9qdtxkDihMw5NII9Dh+sjOW2rsnX2VQGdR7y7p//kbKeCEbpoutpkIFYkf9Fn3Iu1sMqRBM3rJZDxuV7LhjyvHBe4CaoEbdH4RwC1iKdn0oxsmGNNgLR64uQ3O6M71Fsw9NqkaTP0Iw52sbJ+PbgCCCZcHJUqcd8FestPLKayYcdN/zIOzuJeP7Ecf0/Jdwl3vZOmPDuZruieTExeVnkVya6SNc3ZbX7/A9u8uvA5CqIPftPPr62BVicmDNhCp4H6sIPOI7VqURilUjSBrtxT5lE4NH8pueXL1e1Ieu/Xb7XkJgiji3TooLPRDSiGJZp8eV0P922WHBPJihqAD5lhvIu3O X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: last_cpupid is only available when memory tiering is off or the folio is in toptier node. Complete the check to read last_cpupid when it is available. Before the fix, the default last_cpupid will be used even if memory tiering mode is turned off at runtime instead of the actual value. This can prevent task_numa_fault() from getting right numa fault stats, but should not cause any crash. User might see performance changes after the fix. Reported-by: David Hildenbrand Closes: https://lore.kernel.org/linux-mm/9af34a6b-ca56-4a64-8aa6-ade65f109288@redhat.com/ Fixes: 33024536bafd ("memory tiering: hot page selection with hint page fault latency") Signed-off-by: Zi Yan Reviewed-by: "Huang, Ying" Reviewed-by: Baolin Wang Acked-by: David Hildenbrand Reviewed-by: Kefeng Wang --- mm/huge_memory.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f4be468e06a4..825317aee88e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1712,7 +1712,8 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ - if (node_is_toptier(nid)) + if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) || + node_is_toptier(nid)) last_cpupid = folio_last_cpupid(folio); target_nid = numa_migrate_prep(folio, vmf, haddr, nid, &flags); if (target_nid == NUMA_NO_NODE)