From patchwork Wed Jul 24 13:01:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13740915 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DAE6C3DA61 for ; Wed, 24 Jul 2024 13:01:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7DEC86B008A; Wed, 24 Jul 2024 09:01:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 78DC96B008C; Wed, 24 Jul 2024 09:01:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 612236B0093; Wed, 24 Jul 2024 09:01:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 419AD6B008A for ; Wed, 24 Jul 2024 09:01:43 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id DAAEF12093C for ; Wed, 24 Jul 2024 13:01:42 +0000 (UTC) X-FDA: 82374658044.04.E3F36B1 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2053.outbound.protection.outlook.com [40.107.220.53]) by imf04.hostedemail.com (Postfix) with ESMTP id D577F400AE for ; Wed, 24 Jul 2024 13:01:26 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=tchEVv18; spf=pass (imf04.hostedemail.com: domain of ziy@nvidia.com designates 40.107.220.53 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721826063; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RF4UiEA4wU82ykwfKNn44Zy0CR5GrMNLSl7SIWdKIUg=; b=TlH0qUjms9nRJM4P9l9PpHQ1rbVO6CtlxByJopEZ05BzwKeQKOEZzf4FV/CEumwX/1j+PM SG2pgrOrHwZ7lhQvVWSPNwF9ZhRionNYFkAloKIwJ9EIYPUSi6TvPWwPOQ9754QFMHLSyF yvWAT6EQju5K1XXtrxmBqPzcE16qs20= ARC-Authentication-Results: i=2; imf04.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=tchEVv18; spf=pass (imf04.hostedemail.com: domain of ziy@nvidia.com designates 40.107.220.53 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1721826063; a=rsa-sha256; cv=pass; b=r/XcC3237+5WaDR2OycIeFtdhIlhjpmU0buslE4jeBmat1AeqgmrZVQf3emaOLWelMbiEt SdyG7wk3omEj5QA3nKyEICXfFY9i8dww5B68xI/SJNXQ2DW8qk+AzkCWsotI1fqNetmSVH QsY6Wzk0FXO0lpayAl7NGDC+KJWfJD4= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=f9z4ZGbwwXS9S5F/jfmyudOgkHbciwouJoRA9UBNHyMGJHgOag1t3mBSVDbw1zAgsN3jpj+3ypfPbQRnbzIHq+NHW4j5xcgLFvxQUTAERetUbVzP9NPXr3kmDrILa6N5nsACCyL0BYrGbGV6mu08a6Lz69cKZQcdwtZtnM0sdxiQgFmfbNRYVa0D+2aEdJK4RAtzhW9yhewLtwlAJoGXsC4BzJKbAJx9oX7fd6myCQLaox1THurIGxdcygQvcK8NS5AGDSej06MNSRTWj20A9r6UoNCXgNHjA8qeKs3cL0/QAA6KaYA9nikPX3ivAQ8bOkXBALL6aEXXmGtnv9TF9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RF4UiEA4wU82ykwfKNn44Zy0CR5GrMNLSl7SIWdKIUg=; b=yolt4R2LgHQOAJsaubsZMxi6ppuEk8m1FucJ7dTBrSDYkWYpJ9JZeOevhNbausPa5vbvEhUXnnPmDVoEcu6J2WlpbUmcw5peOBz41+FQXOcN9nwNhc4ZKpETnMYzOmiUr1aREglRqaGRkyTrIZnL6M4EGnklN/JTxFO5aCqH0YqbaSrJaFyhZwMvrw0msNQC1ue0Ol++8AIztVhAtq4So6LYFY2ttrOFbaLJ/hBftXtpwZ3abmdh1s6NFBTYytE5FlJp8Lsn5NUY4LWv7HMs/l+QcZn/kQMaLiBkX93bv3dETZ5BtKgkMj4ijCCOUbW//a5QlXolLwPxnuVm8MwIuQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RF4UiEA4wU82ykwfKNn44Zy0CR5GrMNLSl7SIWdKIUg=; b=tchEVv186QOETpCUKilziV2rEPCKlCE3Zlb9yx33o1d1GMK8TPpOwxUEfm+aySITGZy8Fcyo02kBC20XhNG5KqPuHBay5UAhUtj38j9jS509/lAX9/av6eshVNw9hhS+CWHOJ3Gf2KgMZsr3xAvqYiMIJYC8fI9LJzvKcJxn5wbi0562NYf+8tWIKvgElupGE3tCd3ygQfUMuIG8lHyDdiMnJ8CdViy14Lh1uqCwZNURXTRU4ss38eXRXl/TpwfGPik1HpksN84WP4qj/Kk3eZdUUVMMhxeSqQ9A9p7jPh8Ni6yAe5yTiLs526V1/7dqHu+jFg59FT313poKNc4QrA== Received: from DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) by SJ0PR12MB6709.namprd12.prod.outlook.com (2603:10b6:a03:44a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28; Wed, 24 Jul 2024 13:01:20 +0000 Received: from DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e]) by DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e%4]) with mapi id 15.20.7784.017; Wed, 24 Jul 2024 13:01:20 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , "Huang, Ying" , Baolin Wang , Kefeng Wang , Lorenzo Stoakes , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v3 1/3] memory tiering: read last_cpupid correctly in do_huge_pmd_numa_page() Date: Wed, 24 Jul 2024 09:01:13 -0400 Message-ID: <20240724130115.793641-2-ziy@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240724130115.793641-1-ziy@nvidia.com> References: <20240724130115.793641-1-ziy@nvidia.com> X-ClientProxiedBy: CH0PR04CA0026.namprd04.prod.outlook.com (2603:10b6:610:76::31) To DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB5744:EE_|SJ0PR12MB6709:EE_ X-MS-Office365-Filtering-Correlation-Id: 2f1ca61c-409a-49d5-ee69-08dcabe0b601 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: 4lVY2L5sPY1vBz9ZJcog+R5da7NaPth5CGMN0oPa96S1yTjWzAOk8x9MFe2Pt09Jn6rCxxdEvcP/omH9NeBPebdJJcASeSki8KwMELHn6OCcg8MroyB2miUNpT90EqABIV2fHF5YTkNppsKIfJYtUWTp6Ir9h5XhRyYobSFNr3ZiqedSSih6OB+yY6cG/YKDbZxVvdmNAiXNJO8yI8jdIbNUOy4DBiuCgbMJHDMyCizBWxmcezAR+5eRvuSPP+cRBTkNjBqlVgBacwIbE+7Jc+h5K5nTuGLmXwYu42PyHpzHhmo66o/D6fhuMPkT70X2I/qQLi+zX6a/Zv1FyLaZcPzGXmCVac4mN2n5ioyDq++0R8/GGgZtkkZXronYuEYtAlgwGDW8lFVv/L8+gNSDCOtZqoHnKseehiwHcIqScRCzzYBCWNcv2/AIf02Q6Newiqb65G4aK1BirUxZQe8cA/4X3HMlrkGig1JC3ehhEU6Jxv+aZT0Cpjmhiuwm6PGZzONRFQnKgppGibLGi1jLOcteWoznlMCv/ABfPoUedP1SNSDW7ZDxLoTxLMzq4IDQ/+Ixbcy27wVFxhtLz7FmgDyRWydtdKHG8E1EmLUyMvhTnXyAQMkENPNuajVk9Q72j/uvLeRErXfxwrPzm9HcBGV+C8TJJNvg/tQee4DHo8Tvr4NBczYaauW+F5WWFBCr2XUw9dspGDckx8krau7CDoBUjWuWJJ8UzyCJN4/86qQEDLu2/kgxu+9Xel3blB5Uh3Kgz2+79BGbLPwRNHTmZjYH4u4y3t+3IEWTNc5QfEJYcPfCeGmeKE1Mn/0iruZun39gmWseMtt6pRxfRRPZZbrExuPUBpbXzmfB17108FlnLV6+plIrIk10OHxyZYjY4S1beE9tu2hGhaROkglSU0OxvHhXPwkPW6w40L1bU4Ref69CHYCo9Yw6pi1U1fk2ko371enplLTkpSlK+kVXo6gFmlxh47kf5WWXRJlnw+HSS+HvFxeio+GjKHG1ZUypD9CK0Y2C8RkAo9qIJECdQkqvXAWziLPJiGnKHZeAHNvdFVkMSkL/pOJ2KvHYHxXIWJd3Ut17wGkPJTO4h+db3lhcrdleMK8u/8S643M7Wg1X7oLfGLKthvc9UO1FxF7clvM+KQdbW7NTpVybBoh35iDDWRspTNVUe8QZYMMzel4WOCs+Ku/WdZN6Nm+D6JHPM0A6CQUfJKeSho00feS4q37UHCSKl7GNrcea45wWSfM0FTbtx/cCipdliHMeJUlesUqLecG1+HxwrXLd/BNZ5Nv2dim3xAxKH6UpQytDHI18xXkdH5N2/QhZgod4LLHqCeVvnJSL2La5PdXyEsdJAA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB5744.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: k+BradzlQMkJcPuGzShRCMifRUJo8E8VQeYwim3XTATTZCxESEU0RslJvtihnZcZmPrWv6jDT8OttmeF6Fstowlfe+ll917bPcfF+wXS83M2bzG6m8YZIoxWdT75VA2dGvXSgcdGok9X5XRKDXPTfHVE/6tQf9LuHUplzcwW/9MVUD0iURx0Hyhqf5ZavxBtshF/MkbttYMvvyAOZEClIb43fi+HShotCkyrGN8adInWUTDnRXbwj+/GWHMwk2wuiJVS1FdlzHyVajarX7bKi6TC+PZ5UDB+BIofRWitZNMZbGUlIAhK95AiPp6JvSJwLNX6RM8jDa3mCeqUhytdvNJLqb7MZF+fY1BxVdiVbccIc0uBfcEK3fPoVBlI8+QPIhPxK4eOZbr1P7Of3oVz/4p1ufNZWguQmnWWCEQMV/xNWzSFOShPwRDZzGXexW4H16/1HwI2Oh9dNcKeJe64dYY3946ZWsIXDgz8i6hdvIqSJd/9+4zOK1wYvAt1GxSZJS/5ujZRS+fc3acVn0KUOP+2/ZoDrVfyy+7C6sIsiqWclWTsTXaRgKBxinXKAht8S8dGJF4F/QdFq03diNe/n4AKBwNXII5toDvinKS62KXH/Uiw7cKAlVlgisYWz3CbbSUZiPxN3j9Q2juDe8qki3RvAe9SZg0a/jbSVYpK1dRvYOIMpeqd0S2Tp0JPlzrukWGOfiOfOfk4MoJyheFMGjMEl29MbRz7hBhiSjl+FRWRPr9eCmSAqErHLNjs7mzmN5wasLpVcdUoZMVr/RD52RLToa65sjiTJDRQLTS6zyCjgWZ8Zlx/8x94ign1IoIi6HtJG658aq4OECtWzZb3NG0AE1VEPqFZPlCzEIorQ+A3eRvIIsZW6wHyDbKz3Pzc+KCClQjIzm8iIkcFMR9iG6qf2vfjCZ8RTPLPYfmW9AGAc+eDWWyGKQSIxhGc3316xqNCNjdkc+TPtLItpRFjQKQWvsJHeZ3vi23Xheb8TWUgDPWFUF7vnySF/KvqzLg++SW0Mhzc80o8Il+Yn0jYnRTf7JT38cBpnwvvj2xJswp4HVNWvvpcXXuZnu9BWIplXAcjS3B4DYRYsbH5nKCvRnKU048idoueRJjngS0YGB7jgSjszOmm6Qg14cfkD32LlAU1TLVSL5Dvi2AyAgTvoeBfTwzPJnPQeiSB5rgcF69sZg2GmmlH4vDRL/w3Vb4aycSfNRKWLsIGy4PPcg9Q2pPuHSXcVxYNzgcUGbSo2Qn86mS/2AwxmmPGMoUdOcMn0Rm0lX95cNaJDPpG3VZd3x/WkwD5YCNo7U5VBYSYV2wT2eKCyqIg14mvG3CVWx1OuuhmRJ2OUHO2AF1py41onQ89ZkcN4VdUbDqOhC/K6ftwh288xxZ6LhXVX7Hnu2X7XKQ1ffwxAXLqu/f7MAGcSiv2G0NrlcTImjimbjQtt5iZUHdp65lIV1ItGFsgKSSS3jBnFEZR0BdXXEpQoa4RTAmSoKcHPl1rIM68a2dj1PkT/te9GLjHEDfTHXTVSbJ9xaH/Ct0wUmo+6iHA7PUthiogvuJf3xFxMpdhFOdNK1k= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2f1ca61c-409a-49d5-ee69-08dcabe0b601 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB5744.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jul 2024 13:01:19.5805 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9y6nMqlNhwrk0oEiD/g1cVpkYSBaPTJTKg1G9uvxozmeVdRr2wfQCH407TShuFGM X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6709 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: D577F400AE X-Stat-Signature: s84w7ortungee5t4gzd1xjsjyo3tjrbr X-HE-Tag: 1721826086-937293 X-HE-Meta: U2FsdGVkX18hNkFFPhfMv+lK+/90eCci/igI7JvjC8fhI7vLuJ8/hT+qps6JmzPxL9aALQVvG/XIetA541TawTsmznRxmHrxW0XG6a8qYwpPp1tQuSl5eluprTGeD/2c7zC1B6x9vzO3lfZJaeiBZEP79daZ4V8EoYkgBF4o7q1i2DDottZZFZFw9Wz37RqZ5lcVEIbKIFUXU2v18y5BTLDvKtw7PqCrEmFYrXBxmhKEiW18A+m3lnAq5x5A/3IAODEB0jlRpjDirNxHLZ/P6IHNepxqjVdtqBA1rEA8IXYDHfnLohlvx8sZdpDwXoGnbL6eRVXR62RUmhDcQqfi+txqzzM/fQJBtW5D9uGFY95aEeVA7voF5UqPZjKaeWNneUyIrZQKnaFU9vLNa3xJ+Ts3Pefu1vu7jPcAllEGBZalNH1qBP86tY7cuPkh7YT5rMfrM0ZpSw9gg6KuwDopet2EdhAUyw3vNhFB0B4nVaocen26TTUZTBJSHbPeZP9fC0E9FRoZn5xeHKsD1uFYBFItIctjtFdBMppMF7sR99POOYtyNChJGHP+JMpu73Vs4VqjVHOqyMfOcSekGpIAGawv2tkhP59Nb5rwMKBMjHsDdxQaQcoUoJKtOK27T64bF6PKAmpLN/n3vUS2Ujm9M4YdsfLoUFH3C9o4u1HEzNW5aMq5YINuCgm3jBSSFE0jobC0NafEKuI1UShdVeVaCf4kefsQ3pqNupftSizRXJcfwnC3opKPGjKyjC+X2uLDYzUb2hFESwx5lXuUtIoSUsGGF3u4KZ5Zr8xI49OhLMo74dRLmL9qK9zQjODaGwh5353JLGa1h26cMRtKdY/vxLlbKEgG3+Dlu0FByQGcWNveU/EQy7Z91ehoZqC3pR10as3z43p3jCgWznpy1cJIMjBcJRXUnhbAOqFEYSvPwPitO0uoGv+ftBcCqoXdFc9ABkSHcLrrgc/VJYQ33z0 53VKVc4w ti3IsGJ6X3CcCD8cL7VLVMQCArWb0EAATajFP7Zt2JDSHPEKqDcMjjTafLzZIX1pyANwd1YsSvzwuRrue3CcMGujzWiXzolFVdp+78D+GesuHUwb9WSJp9G62vTJANV+xjN3YfFsZJH2z8LFxqmLpKwMyfgiCoM4ILNZJ5ocX4PMRkwobZCt7nQGwTnmUluDvD2Gq8EmeLpv8Mg0C3jcbLmhrXfN6Dj8ZTJALduxOCnsRoriFumWTy5er1InVYs/QtGLHbiSy5etcusCV84RW4GWYpcd1keXbtiHr4EzRxeMO9Q6sC4wclRzaigKGVEvuhvId+Q6DB2g9/UfK+n4malVj/+Knn1/hkmEOrA9msJX++m9ztIj0RoUwc5aPONsWg53x6wGaKV5jNTNeAhgGFNNsMrowDC4Qaz8afZchkYAWZAM2+wXe1ruAyCacwgRE25s810TWtyCf68di+MEPOydigQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: last_cpupid is only available when memory tiering is off or the folio is in toptier node. Complete the check to read last_cpupid when it is available. Before the fix, the default last_cpupid will be used even if memory tiering mode is turned off at runtime instead of the actual value. This can prevent task_numa_fault() from getting right numa fault stats, but should not cause any crash. User might see performance changes after the fix. Reported-by: David Hildenbrand Closes: https://lore.kernel.org/linux-mm/9af34a6b-ca56-4a64-8aa6-ade65f109288@redhat.com/ Fixes: 33024536bafd ("memory tiering: hot page selection with hint page fault latency") Signed-off-by: Zi Yan Reviewed-by: "Huang, Ying" Reviewed-by: Baolin Wang Acked-by: David Hildenbrand Reviewed-by: Kefeng Wang --- mm/huge_memory.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 88dffc0023b5..15234b2e252e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1707,7 +1707,8 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ - if (node_is_toptier(nid)) + if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) || + node_is_toptier(nid)) last_cpupid = folio_last_cpupid(folio); target_nid = numa_migrate_prep(folio, vmf, haddr, nid, &flags); if (target_nid == NUMA_NO_NODE) From patchwork Wed Jul 24 13:01:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13740916 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A46C1C3DA63 for ; Wed, 24 Jul 2024 13:01:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2AE086B0095; Wed, 24 Jul 2024 09:01:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 25F416B0096; Wed, 24 Jul 2024 09:01:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0894F6B0099; Wed, 24 Jul 2024 09:01:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id DBB796B0095 for ; Wed, 24 Jul 2024 09:01:49 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 86E21160938 for ; Wed, 24 Jul 2024 13:01:49 +0000 (UTC) X-FDA: 82374658338.19.9CA2AD0 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2066.outbound.protection.outlook.com [40.107.92.66]) by imf20.hostedemail.com (Postfix) with ESMTP id F093D1C0029 for ; Wed, 24 Jul 2024 13:01:42 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=qstCRrGE; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf20.hostedemail.com: domain of ziy@nvidia.com designates 40.107.92.66 as permitted sender) smtp.mailfrom=ziy@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1721826064; a=rsa-sha256; cv=pass; b=ui60C+3663cyntk9ZZaNfkTT359lqDa/OYDDDPJk9FqjTPro3esVzJOLyXkc4r+rTHDzwe Ylrq2lXeX+ryVZZCVDIpquWR4IyNu34uHpiIVQsi6PyL4e+SI8ETZkPNgG6DSxOSOQ3Msi 6ENdfx/ToN87716JjZ5t5fVyKVu19Eg= ARC-Authentication-Results: i=2; imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=qstCRrGE; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf20.hostedemail.com: domain of ziy@nvidia.com designates 40.107.92.66 as permitted sender) smtp.mailfrom=ziy@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721826064; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=de3HtV97/3HfU1Wwqz2WEK7UWyyj5aobEIaoBUZtd2I=; b=TglkMa8EyYlcR48XbMbDPkQrKNrEmZJXxn8ABUj/zvV1sUvocA2JNU+CGEoWSDxC4Hy4Rn flHB8SXL/lqMjATyc1hRPBfCGRfND7QeuewhJXjVxwisIofeNexiU8OfIopjaCtj7LzS+i ixE0REIOoO4V/WoiSFzLGMi0/490X0A= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=iQUX2ghHrM5OKpN6mlllpqbSjxg8AGuU1bv+7m0T7GZGWgOQwpe5419hI5fWhexQkCqYSUMc9Z4zwePSm7Z91Lf/Wij8D+mDONilKyZDnjmBSEauhyYRD4bScgQ8BSjWvzQya4tHKNa/UO5XnnbdsmubbnJV3hP0ne2ABNlK19L1boakZCA2iBOz45RhaoTAvYF7NEmWzIRzMtT48m+AYmLTPwLKa+DdnCYA2GdkJpv2zbYH8vcUE6NZ4RQYMz/9Tqs4sLOgZ/+807ODB2NROvxKBiQxmOLaTGBA5HVwTZbZd6EMGGU1LUeW3kj1b+s3y7r2gZtezGR5sEZECdaVyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=de3HtV97/3HfU1Wwqz2WEK7UWyyj5aobEIaoBUZtd2I=; b=kh9GUWVzbVpJHzbVLeQM6m7Q89u59cKJA1TKEkJDX9jEA1OPiYKK235KD0mjjfLeP1+prh1NjaLk5N1k4vdh6VmYBwBgTXz8YYxFKs477+AnNY5CQ6AjF00K2QZw1bRMGMC8lcUJJqhs5tuECF8QbiguZ6VQ1OgzE+3DblFLBTgQ31016Ky7cq/EdsGIMyaiCeaohYmvTn+F92Cyv1MiG0Aq0IZs2qCY1g9EIX39ReFYUzDZNci7vObDJzSvEucVelVeKIjw5dSpLRimo8Kylf+35ZGHkzJ6sjRumXzNXJ9Oxi0ogFzTsfYk1tM07Tftwf46Xs1a/BWO+HNbYyQK4w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=de3HtV97/3HfU1Wwqz2WEK7UWyyj5aobEIaoBUZtd2I=; b=qstCRrGE4Nx6o0ooMqjKn0DTR7PgDBleyJ8YHQUw+b94ShTkW+s7TR2GEPiDgrz7g5nc/7SgRkgt+mQrTlzQsO8IVwRnG9gyO74cJBFXNIxBjKcNWWEuszzyJwM9e4fP491l7R9RQkiEFZAzR5Gw5mcJggdTre4SPzVBvRLGs0599uG0xumkiKlUbePnEoxG+T/8qPXEKn+uC/1hKan+7GgmYs9+WixZj0zzYJxtckZmWrFVNqA7NKtK95uchfg4qypMtGUJJ6HohkMjn1Cu6TZDj7ZzYq09vM1JF/YIjzHupshyoC+s5rw3BOjY0cyJCmXay2+j6bQh1lmDpTTgqQ== Received: from DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) by SJ0PR12MB6709.namprd12.prod.outlook.com (2603:10b6:a03:44a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28; Wed, 24 Jul 2024 13:01:21 +0000 Received: from DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e]) by DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e%4]) with mapi id 15.20.7784.017; Wed, 24 Jul 2024 13:01:21 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , "Huang, Ying" , Baolin Wang , Kefeng Wang , Lorenzo Stoakes , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v3 2/3] memory tiering: introduce folio_use_access_time() check Date: Wed, 24 Jul 2024 09:01:14 -0400 Message-ID: <20240724130115.793641-3-ziy@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240724130115.793641-1-ziy@nvidia.com> References: <20240724130115.793641-1-ziy@nvidia.com> X-ClientProxiedBy: CH0PR04CA0007.namprd04.prod.outlook.com (2603:10b6:610:76::12) To DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB5744:EE_|SJ0PR12MB6709:EE_ X-MS-Office365-Filtering-Correlation-Id: 82296d42-9d23-40f2-3fe4-08dcabe0b6d5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: apb+s7tSbTXjL5nvvoIW4QtKIEZTFkZ57bc7CpbmYIo63CRLAKtkBcffFr0T3yhdMUPrtzbLXuoUx8fRXvX2OR4YKScJNujVU8N1PV4Y9cwENCQ0Br0Bx9YfkZRbzHNH31oqiVFcuq1TwBU3LWZ5YirmJ85/wYsXhP/8pctF/Z2V/zs1MEqRW5E38SxmC4QJtkgMLLwaSvtWzcV+6cXeNUkdThS/zPJ8mvsAO6qHudGdhjDJbD8zKDfAwJyNE5N5swQTPZHkjZL9BidbJJ7JPa7c9GvIMApMAH2bmH14fdh2Qry+KP1s+5/+i4mRXG24KBi4w/j6QNY43zR3jGYVqArxxT0H2iOQyaxPztgA2+yd9ooqHJuHlnMtnvqHxFPN5AXJiX7T5bs9Njvqvqm5vm48MtN/rwFstCmVR+hpqNOIDi5vuZrmXTHyIdAkb1vaB9E0uCCwMOujCsNbXCnQwaqKriOXAxt+8mNPemmWwQtm7XeAnnH8xkj/BuOZO235OPRr5doGBucoT5agrNIfcV4M92CumSmEtJaMD5DPfVygzRjlNNBqMnzeOKYUDEtQVyh+hRL2g+HcxEOYeRwUMur6PP3lUH2F4ybd0OFF6p2tbztVzgw1wGnX0mDFM1RCn+1mt5DHGe77byvKmaoILapWKV0HPd19jgde8nDVQxNcJAkJ3BEPIsuGATD+Jza8wl68rTvDziFduTOMYKap7uZHtu7mOUr+NUmwIcA+6fjpLYiplF6W5IoLuln6DbPOLuc9FCLPTruDq81I5PPbYkM8TnhuoET/zxt+mrbpPWCxqffO/ygIAe7aNzsmOiN00JJAYGF/iyRYl6KknEBWO0r7QFIktk5M0o5xCgbt5i8xyLfBIRVEG9j2C70qb76E5oYwe2bmsqZjnwhjDKeITbR5Ma6TxBDa3mYP7/KuPN1LraoCeEznzIGsm+TQmll6Cu9NJKNOJvIY9+oFpMEXM8Q6E3QqND9BiThOwHGlpBAjlrnZJlBTVJ/xMhrY0/kix2RXDowBnZhF4+1Be4idDNnN7VqpNCiMr67Yph9LNhGqO7zAuktol5pKRgQROYB72VBzTWvL7lGLpLAy5CQLpqSpZ20uEJNygEygR//qR1A08z0J6vAKxHI/gfCygtNlqSGwjp6FGWzucwR2JwObxeqMTA6NWfso3jZOkxLkgO3Qi4ZsGyvubY8RgP28YBfATRahY1pmDnwOpcEmEnMlaGpAlp1nXPML/Z51jH6p823TQm/JY72u62jbTyBK5Aedf0oG7lGnA4cOsBwP8OKvYuX6dYo/AmvuBOFLkO7N8goAEslJFYDU0Hc8s8K0YpHgHNzLM4VqMmELpe0UKyRMAw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB5744.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: M8E3r/N+6BRIya+hBsfajwiI0IXubOWiFRwJjr59foP3qFoYdEWtyIP2RSFOdE9eZH2CClhiVquNue0zG8oRtY/SSAxlzzgNGzjjK9KBy+LN2GqDNqAHC8hM+YYabIv+t04a3rCVomkNmqTMGKECmoCr2NVwYciukKrgz7hM251zZkiiUBEjaEGduxwgrz0wTVBMvWJul9zwBYVDyqYmphhDywye8odnR7tLluLyH4VAtabnora9r/U0/1eJp/jyIUzI5wwPJBbCO0f4eOKK38lfwl7T65meHdSjq1do19L/WpfpQyEIm2hmj+1Yz/2DD2hoZMy5Yzs5jCWFmtUxORmlbpKHI3V6u7LNWfEqyqH+ISluanX+njeFtwT0BlKkwAL7z4TBhnOE+k2kpVReVtGQk5Afw27MAmg3nLsHeQBcOT0G2PoleJ15sJNi/zUs51tohPQS8rmxb+dX/Ot7HNH+3TGKCq+l9Y3wMMu1exs/OdJzOLCj2KJMzRBaF32V6q42VfzLCWBecTldUHN6H4xWUKNT6FV1C4UpkDDU7/vixkABzwG3wEIdMEtnY9IFsjRSK0mqrqgytqFU89zok/BAv6f5YmxL5faLsxDciSYWY4FD3bQiA3AA6QzBBzpjY4BCZ/bK2t1R1l8+Xqfqo2PXfVYHYIaYom9Luve9CHREsgBL+C/kc6pVsWnFHCTz0jgmPJowg8vBClo2p7X56gwxwWmdIePxufXwZvFzBtNKreTkevqhhAk7gY6ukdTxghjCww8fBNfRLLXSp5QV34Mbv5ZH0BVrPYl8eWCjIrEq9kWE59TdYHU9MpEYAGsS7Lw7bLcDsx3YinXO2UuqQp5oyoQpvLdRyua84JVtnLETLiSmuY2ehdR0vmFx0AO3duUhamrhyOzd6hyaVzKhSBAUNLaOW9/PH6nB6g19c1eWTL/y9CVoPlHF8fffrZkTfveVL29HNIZr5Afa1SSsfSr4vhkGQ042iuDKuD3FNLhFqnj3VTNOLHHHT1g8Jx9ANa+zgQk8W+2Vpdj1HbTas7l2XmXV6cO9qf9HOnAnmZe4RfeTwZ2aXNTeHi46rh2lZiO7jB023QXVZmHzRLum4RvKv5mzvzj/PGHJhCe+VE7KPovFlxX/UrWBZbxmrjLHddiZViB9nJtzQarlbwXx5Q7NJJuJuWkN2qUHSMGKqCwoSY0yLgvd3GAoK5gZiR6DN7LgtM/TMIGv6N0IAj1QTCTyek9hlNEHKlyzTAaERhtJk13KKt+1PgRhO27wRKYv3LZfr7KX9/PKvAsoo3ad8enLFH0iI4XIkSXbL4edbtZwx28B9oGg684AR9fVpUbeM9wG+a2dfpEapquv0dk3NV+ya1L4Ap+E676eLuyswF6wg1rnpULw6nBx/326t48oLUrZX2EkDPni/nXX0+uOEJfgEo7JoDCMytlZmOvqaZ1xOYp50MqrzHnD6KrGyjBRo+ijm6/pne4pKeWt5/z1eghZC/xPZrLTu4ikXq7pP8DGADmpnV43A2/6Ev+b0d/iVwMrE5GdVcuM86VKkG4Hy1/+M8flo/Olq/QH19jCcqQ= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 82296d42-9d23-40f2-3fe4-08dcabe0b6d5 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB5744.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jul 2024 13:01:20.9717 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TBpMoBRiLx/QDGcrqF6nwlt+WjghURwjXy7NFjFHORPQSiX2HCw5Oo5OYT8gv441 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6709 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: F093D1C0029 X-Stat-Signature: bgabma7csamrwta3eykdhgpiu6z14h6j X-Rspam-User: X-HE-Tag: 1721826102-285914 X-HE-Meta: U2FsdGVkX19EA35An0bKiiaOfXwbZD0hPAQ80GCojOsfUGmn8QgcS0rcRa+U4O7u/msqBpXjzw9G0IjDf31G5HM9i//wibF8a7YIF5nru70H7+Y3ejlOW1ijIqEGNYJl/boKFGuN6LMec4RzsSfqbvBJW/l45Y47fZ3YjrSeC46IWI4FoPbPZCc9ZSgrjCqvPeQYuEtppzql7IP5AEM4TtEdUX1gcsvqRPU3/70pfW9fb6hs6gJRgV370gwbE4bedNOHHEDQwe6+nsaEBQlQ5Z5GIyqIHiz5sJ1q/8gGJeBvLAMdTx1zWmeXwIe4uN7RGph4EXRt4sWFSJK3u7QHH41GNCB7CLu97I60eu/Tpu73+Q1YQhlbGPWQ2oSZ+Tx1qHJsiKXQGSnTOiZKI95p47s2ZT658Lm55oWfTPBTPAdFSSLaTuBQwjOoif8jzk15g9ol4OVLnL1FhDPfQw1h5o3zrPK9ern4pjJd0AOGz8uGLZisMmyH6phnkYgazlB1r5OYn1JPovc17iyaEKcq01qGCPFqBa8EDykxNYuNJE8R8Xpx5KGDVGQ2DY/F4qMA2IpIr+TlvYTB8tzfAaskB/s8PQ13l1U9pqN6FZ282ybMgk85m8kVyaf2cB5wEGl8UC/JxhSqp0Jvj94SsN91iBGeB4cXdbXWxaSTSxkBsBwr8/eN9rSGYRdNaRefsbLGfjnuGC9z6n89CdloFj6GOdjBF6flUm336l6CGCsOEHSqQ+4WmBUfbdWMp+yFRS5K9zDkDctclCltchPSQx/XM9oeqy791bouTf4pG9qAbaudEayL78EzzFh4OVDT+qu9CqR1PfKDa4ZS6iODZ8i9rpHOQHxcSKl1c95GENihWftE/9Gtfk9VhRhiZ3sH7+Wokj7vCX4Zg7G2XesUjL8t65vFRyeQOdJlA81a6Qjp/vLlP8piU22cEGbopIigypwXD4Cx9z28JvfxqwWc39V vVPeS8nK 59fim756Hrga2T9eU+knRWF9aB2ncN2UkoAg59AF4kgkgxcqzD2Aok6quo4bNWk1bYoYUWJrxQyvlhugoF6ghWOt7jDJv1pylRQxNHwuexHLNvdGVGZEqFJZZemk0QG1oyiuupzZdWBFVPCeyBNT63x/g84FFEyVp5KIA235yTuLC3k8CNxriFtGgGPGcBeYRNP4+iUjY6/CaMF3HtZcSa6odc1CPyw8IHn9pyKt4Qt+Vkc8jXjQGSnj+rc0zzDPjpyJQKRypiS9fj7I43fANq0ZfaT+jLH3HMry9AxqkuTjkiLOvkMsemRrD/DNRMUH00lxaiqp7PN/OtyNJX4MxoKplQCqn67vcFR+2853YIQuRvSY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If memory tiering mode is on and a folio is not in the top tier memory, folio's cpupid field is repurposed to store page access time. Instead of an open coded check, use a function to encapsulate the check. Signed-off-by: Zi Yan Reviewed-by: "Huang, Ying" Acked-by: David Hildenbrand Reviewed-by: Kefeng Wang --- include/linux/mm.h | 6 ++++++ kernel/sched/fair.c | 3 +-- mm/huge_memory.c | 6 ++---- mm/memory-tiers.c | 19 +++++++++++++++++++ mm/memory.c | 3 +-- mm/mprotect.c | 3 +-- 6 files changed, 30 insertions(+), 10 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 17753a463e01..2c6ccf088c7b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1717,6 +1717,8 @@ static inline void vma_set_access_pid_bit(struct vm_area_struct *vma) __set_bit(pid_bit, &vma->numab_state->pids_active[1]); } } + +bool folio_use_access_time(struct folio *folio); #else /* !CONFIG_NUMA_BALANCING */ static inline int folio_xchg_last_cpupid(struct folio *folio, int cpupid) { @@ -1770,6 +1772,10 @@ static inline bool cpupid_match_pid(struct task_struct *task, int cpupid) static inline void vma_set_access_pid_bit(struct vm_area_struct *vma) { } +static inline bool folio_use_access_time(struct folio *folio) +{ + return false; +} #endif /* CONFIG_NUMA_BALANCING */ #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 9057584ec06d..416e29b56cc4 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1840,8 +1840,7 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, * The pages in slow memory node should be migrated according * to hot/cold instead of private/shared. */ - if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && - !node_is_toptier(src_nid)) { + if (folio_use_access_time(folio)) { struct pglist_data *pgdat; unsigned long rate_limit; unsigned int latency, th, def_th; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 15234b2e252e..5c0a6a4e3a6e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1707,8 +1707,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ - if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) || - node_is_toptier(nid)) + if (!folio_use_access_time(folio)) last_cpupid = folio_last_cpupid(folio); target_nid = numa_migrate_prep(folio, vmf, haddr, nid, &flags); if (target_nid == NUMA_NO_NODE) @@ -2061,8 +2060,7 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, toptier) goto unlock; - if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && - !toptier) + if (folio_use_access_time(folio)) folio_xchg_access_time(folio, jiffies_to_msecs(jiffies)); } diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c index 4775b3a3dabe..2a642ea86cb2 100644 --- a/mm/memory-tiers.c +++ b/mm/memory-tiers.c @@ -6,6 +6,7 @@ #include #include #include +#include #include "internal.h" @@ -50,6 +51,24 @@ static const struct bus_type memory_tier_subsys = { .dev_name = "memory_tier", }; +#ifdef CONFIG_NUMA_BALANCING +/** + * folio_use_access_time - check if a folio reuses cpupid for page access time + * @folio: folio to check + * + * folio's _last_cpupid field is repurposed by memory tiering. In memory + * tiering mode, cpupid of slow memory folio (not toptier memory) is used to + * record page access time. + * + * Return: the folio _last_cpupid is used to record page access time + */ +bool folio_use_access_time(struct folio *folio) +{ + return (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && + !node_is_toptier(folio_nid(folio)); +} +#endif + #ifdef CONFIG_MIGRATION static int top_tier_adistance; /* diff --git a/mm/memory.c b/mm/memory.c index 802d0d8a40f9..833d2cad6eb2 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5337,8 +5337,7 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ - if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && - !node_is_toptier(nid)) + if (folio_use_access_time(folio)) last_cpupid = (-1 & LAST_CPUPID_MASK); else last_cpupid = folio_last_cpupid(folio); diff --git a/mm/mprotect.c b/mm/mprotect.c index 222ab434da54..37cf8d249405 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -161,8 +161,7 @@ static long change_pte_range(struct mmu_gather *tlb, if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_NORMAL) && toptier) continue; - if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && - !toptier) + if (folio_use_access_time(folio)) folio_xchg_access_time(folio, jiffies_to_msecs(jiffies)); } From patchwork Wed Jul 24 13:01:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13740941 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07442C3DA63 for ; Wed, 24 Jul 2024 13:03:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9DA4F6B0099; Wed, 24 Jul 2024 09:03:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 98A416B009A; Wed, 24 Jul 2024 09:03:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 82B486B009B; Wed, 24 Jul 2024 09:03:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 61CCF6B0099 for ; Wed, 24 Jul 2024 09:03:01 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 1E90C80972 for ; Wed, 24 Jul 2024 13:03:01 +0000 (UTC) X-FDA: 82374661362.10.1FD3B10 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2054.outbound.protection.outlook.com [40.107.93.54]) by imf27.hostedemail.com (Postfix) with ESMTP id 214C24002D for ; Wed, 24 Jul 2024 13:02:57 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=s0vQVOpe; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf27.hostedemail.com: domain of ziy@nvidia.com designates 40.107.93.54 as permitted sender) smtp.mailfrom=ziy@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721826124; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FetlK/722fVi6ZVNeIZFKwGEKahpMrmor89WHb8Wxko=; b=QmXVeDQkjEsFyIlYba4oMcHnEjF7bLxEmIG5TFv/oJP5AYAWyVCF6RGL2K9RHybnr7AUaR RJd2paHhezCJ9CbGt/hP1OmOx71YtwONhE2UGmDpQsXY6y8GKOeGMRw9jSNDXwxQaihkpo p2pxra8bdKxTqdgS8YnFMUisy5Ma5CE= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1721826124; a=rsa-sha256; cv=pass; b=np5c/W/RQbtdomhdywGToLOO3h1lbkYeEQEI1dPRn4kxk5DY6BICLvAtTq4USrLKDPynpj ooHSCNNWEa/cbuX1soHqx6YX65gF1rJ+azOdsqlsYXQ3Mz0+fR54MnQ2kcbj4NL3b8JBBm fgDUNaMsXVhVVZhkL+bqVEOEgxPwxms= ARC-Authentication-Results: i=2; imf27.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=s0vQVOpe; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf27.hostedemail.com: domain of ziy@nvidia.com designates 40.107.93.54 as permitted sender) smtp.mailfrom=ziy@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=oTXQPqJNxFxP6o17/mkav4CxxEShfcrWQuM9qYsOm6BRvtQKC26A64WhWXRQ05/Cisnhalb4r9h/8g6KXGbZVnyY3c4IzpFLVTebne0tThpBP5HafW6v2K1CIJOlOBY41ihYt5HXjMjOa4gHRxUt/wj85A5V55s2dgZ+jPMcG3YRa+UBMRJu6JrMW2D1ff/M5+teKIGYDXJ2M/zgNrrco0EZa5ypnfTH3RMCSuyxXhpcYgJFG0f3bdkIVqmst3afh6mV75jSAvwwswCgOSmAix6K5Jak0poEQdq2/jEQuqyXCls9GHPxE+3UEbMewM/CRCgNJBj6MYZzntsJb7kodQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FetlK/722fVi6ZVNeIZFKwGEKahpMrmor89WHb8Wxko=; b=POdlAqhXttN4PRJRPmH5IxnikKN4/itPBOkPYZB+6f/rQp7j2od+bTXwxqDp8XAoggdbMgcAT4vvwu+yuh5i4LI8Eg8I30iOoHOR5YOsYtnK8FzxeSMbQywSVthT+Lud41f6RF/7HhQtxOqW7jUESzwc/eIkMwgAvO/5Mn+Qvua+W//e+YAlOxAR/+0g2Jk+OZsnDl2YDTrJv4bBDFqVkO5gfHyO43Tg+JccoY+RTUWBuctv5d7HtiM5AdeU5vUrgvwTIBRT7MhErzM2Q3fIzsWIfnMHjquHte/BtUA5b+3od9/v3ElJdZR1GLGDFQpfq0MI0JoY6MZHH15EP7z8vQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FetlK/722fVi6ZVNeIZFKwGEKahpMrmor89WHb8Wxko=; b=s0vQVOpedj38HyNtjrjdCuIwlXmgFVUn+DRELWIoHNSIVmF4a7CLnAKq0ooPoPMZFs9ZdUe4UShITheGTsx4Q7ED4FbO/XpJGbNbmhxY3ZZ/z6YynrUAdUkbz77rvjY7kzEo9M0qdgkPkL9/dbVEhc5Diy3P4xz44VIJeYRPmpDTWciBZYF9vYj6s69WOqQxk0DlIc+Atwc29dQakDrXnaCUggslUi4rzjWpFJRKJHyVqWQi3NFcbHMaA+47+fLg5VffYq4eT3QFKEm/pTW1bKJ3MC5EjjV7cqs4mSHlE4oTkQ3LtQ8/y+VIChHnQ7sBDk6Qs18VjrYW4eMvuaByzg== Received: from DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) by SJ0PR12MB6709.namprd12.prod.outlook.com (2603:10b6:a03:44a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28; Wed, 24 Jul 2024 13:01:22 +0000 Received: from DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e]) by DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e%4]) with mapi id 15.20.7784.017; Wed, 24 Jul 2024 13:01:22 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , "Huang, Ying" , Baolin Wang , Kefeng Wang , Lorenzo Stoakes , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v3 3/3] memory tiering: count PGPROMOTE_SUCCESS when mem tiering is enabled. Date: Wed, 24 Jul 2024 09:01:15 -0400 Message-ID: <20240724130115.793641-4-ziy@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240724130115.793641-1-ziy@nvidia.com> References: <20240724130115.793641-1-ziy@nvidia.com> X-ClientProxiedBy: CH0PR04CA0010.namprd04.prod.outlook.com (2603:10b6:610:76::15) To DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB5744:EE_|SJ0PR12MB6709:EE_ X-MS-Office365-Filtering-Correlation-Id: 3da80656-9e10-43cc-8961-08dcabe0b79c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: MnTThk9tzg2CzTQwcX0JG8vwlbLhvLKjoRcJsBF9KM8hFvuwrWGLZ4SEiIHfQb1+2MSu1zm7rBG4lUd5d1omzx1w1P3f0RihwIXRR6si/gbh9iAB8iFImALDq1HQnl6fM+JEhz7zXCJQOgllnH6AvFq6zdIL0SP1Bs/n1IzqmTneupGbD06Woweb0fB+cBKlMHWsj7WBcoZNzaU3Huue3jWfrnVJK2Z+yHHhIVe+Cd9iE79XQ8KJeKsLEWkTv5pujwBPdMnq8sxrnjsoXQVJsGvgwLnQvuODWxWYA1ieZFnCYWnjK40g3//QqQo2GocCNZKIPrNp4+LESPPtmxKehvA3AE6GrMi453hAlilsRT1DyKrDbKhseIQ/7hQzN/6FUdkByMwXA7Z14ZzSANIIlF5IT7kG/vkmp9AZ4oVSPJ6MeJ5nO/7JTKZBQ8qWYJQ4nq657fK4HImK+bgrWKYMvV6AjeujumrjtdeHZBut6mXT5300NpT0pnsFXczWvqpb5WJmK7dejsbijRPZ8dvHmZJ02cBvEE57iJafUqFEZuEf/GgspEtv17Fv8WUvjMdzwXynpGVf71ysuT+OGpXw68eWrZ4tP8MxRqIEDIKHgLoZ7nQT1JC4bIulz1J6NFQe/w8cWNIEeuVSZtEkXRzwuHnOEA3wAsSpJxLkIn86FGoPIbUCnqyf0KwaH9p2EUXgagdhrqRdI8/Yq0to9lNluTbLxV/l3jpBk8b8zW0ohZsEbErKJxZ/4VVM1oIsPIYs3etr1LVBEV/D/W3pszYb8kybkvqZk1mVrgJneuYocivXzQLtccHqdQ1tt0o9CqA8JifmoD773qSO5SAjgDGAF/KpvSOueMl6ujS0x46Xpm1Gac5XDwZTwDRrvdEh1bIJx9EOkyOqq5fqLO05DOM3nrCocJ+H+NzAwa21tEnYAuA/XuZGUOrtiqwuBdT9rgGkhnRpGPbCmEnXXBmYnW3CAk/BFp0zrtrMSSkEnXantDFvX1MBQw4ZbkGuZBHbcxCdOhBpG5mqwCi2/EZFwWeNrFhDMahd0E3JfrZDHgJq4QC/RD9atob151OLwcefrAzIz2G/I8sORAMft4FKtIT3ODKtXSLG8/+hFuOPTzfIWPXTnz10mDgiXEthc2TYC+9FrrPObO/egv8QDHNAYsaS4oIXAQyLsmJKhcHJ3UHN81QcIvo7RbWEj3QuavjHI0Cz39pBM1rtXOS/REFh8j1vhPSdEQ8oEpxGMWE4RyGvizSdGtJzny3DzcayRgCgL7TVrPqvfmYqV5FG7jwbvzQqBnVtAowXUYEcH5gGnsoqEBdaDd7HckIscUhS8mCozr7mV265ncBUakRYT8iQIFf1DxvauCEnSZthXtXT7hQY6qo= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB5744.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: zzQ1yI72LD6yMPbozV0+u377efR9JRHC/i/T5QGO6ERNk25V2cqhGmWPLuvvYvbwrlODVPXdgypVFy8HfDGdtk/g8+35Ir3DZb0QRXz5K4PB/VPkBhjd6n3FP23v+X/LdEN+gRh11sdYfIAJ5AxOhkEmPfGSQ36O2vLVhTFRzWyQMG8AOjBJfCGZ62CpAikNF+PAmXt7ykn0dyyWwv8EbuiXwSe1iOWf/DnjeUsBVHrZb8XFywYeWYfhzUN8tYVGlyT8hjiKhKNafyOFShXCArrTmFIrQWfUq2wQwvO17dMEWNKxt7GuXAAci2eMox2Dq6esra6eFH8/yiob28yZRRzNhokiGdi73YbH2Vx9MDTv8QCAHdQEaI+4N8Pq3pu2lOmPNbvjv5i9j43K2L0C4H4Gwe1t6G+pU6pVIu/kYaNFUQoEJbOxWcEqpFc4Ag4bVmkmWcR4ji6sDkchAGKubcDfrBqZNp/ElTGCiAaV9+n3nAMNEYPKrCgGprlTZ43Q7VUhMv4yeb5F3VXfD0NPZb/LgYjl7U4wC1H3y/SxP/ET8QWdYKJ4gHdQTi3ZRB9Snbm6lKUebtMahYo4KE/aaXls7OfRXPH0/G7tnrmhP9zW6IKPWew19GRl8Sqcv5sjkmqCy58CgT5VIg0foUN4GLrBGQyvuNLqo19vXNK/mIp1/4VJBcvNCPjFbeMXLjCSQ2VsFhkhGbRk97iCVMrscbeEJw3il/LSIr98PAUHDh+0/q+Eqv35OD0RlFVnaTIGDeF4VLG89QVauohqu2qHffLhXDDG714oUyF7WbmFh77uvZLOd2kg6JBLk8sBY1bbY60XPgIjqXoSKmJXl6xmrPTBkA19m8pg3b6EUVvXT6pnH3I8Fwiz5F5gkr7/+7FYKSounLg66T0QUKQb91KcK91f+e/rwfZfxlPakVZEwEH9OhFtMuYjCbVtzMtMdTD7Ph67ZQrMTmip+9wms7/+QSN7nCNOkuSiGGK/0hVnQq7+oklAbh1+w2grAGBGlshaVfxdWFDFEiZo7847kWLOQnf+h+Q/wZapjg3Dg/5Gz4U6Na3IIEYNiIYNYRFFlVDOc97Rb7GKxm/Wu8dbc3rOnl6na5ayLs669lyEWaM/0lAGsRRCfN7f+wmef9Tn/qvMa87rB4+AW6BJ7njyDDnL6wU8Yh1p8gjr10CwvxpQMzK/YNUQHkggxTp9AZmRu8ibSqisUZswNih3SyziRWHphMuesNg3tSV69jI1slXp9A1syGD5KTVfFXrA6B3st2s/niGpSPWRNbCPhQJq7Th1XwbyPdRliVpq9v+1DgxTtsVZnnV4j5QXNbZaTWRudcJQccYduLcc/3us9ZDhDrzCqimdUvEMw4Vak5RNurBV0BtpZe5eLe2ruBUBzPOZwdIOS8du/tYweWHYWK95BnzOjO0YhE6zWAZ+IySZLIj0L0xK2OMerRHi2LgrBnRTHFbchlrNicZJzIPrmRluPpbOT4Fzx6G6ehT7/P1C555WNCMu1eObd8NjKovtnMk1QSpDvVIBNTU6yB8Hg2VMi+8UQwgJqQTf/krJP4Pe9t/ybcU= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3da80656-9e10-43cc-8961-08dcabe0b79c X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB5744.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jul 2024 13:01:22.3983 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: F75X5XeByluzv5oykT1mLdYpXLwtwhry+BFd7ffSyE7sQMuCoBXyX0c8saSm6aoV X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6709 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 214C24002D X-Stat-Signature: rz5bhmhgpddmkdwbqtm4id4r8f1pde5a X-Rspam-User: X-HE-Tag: 1721826177-788762 X-HE-Meta: U2FsdGVkX1+x8RjghbezKJDLjjVboOw4K6aU47YbQQydHhlTC3615V3onNX4R1bszIxWpbCiydDmB9/9dRpY0qbqpFoergiW2+ou+3zs+Qrvewt/sCQOHTFYhsyFMqOFnEvE5yWIAvGmpc6OGn/1uR+ec/Nqpwb2dIHyf3GIzxaKXD6J1+KUM3LUQnW2fEKsKhamrF784aAbNEzIJQkLz8RVnx7McU/c13cedB+srdF2fu+KbaVIuRsgDAYG91elt6M7SY9xx51NclkBO685Pw9JWRxltq5sok+Nf/jq93VdjEYl5Ntvaa7v055tWk+DzWtciPiQIDWSRGx6oHXy18XAFK6PYjE7+1uXSjO7QnHRHl5haKiqX5QwpUm7iKtOyT8Oi6QfWgPZPSs4OcG4OAhSsrWJjV646A6pJSYnn3tSTtWT9ywf/1EUu7/m0Y8ohaZIHqSpkgfoSvWwIG07kYNEsTTVGjWWDmHrBHwm7KbfoSDqYGp0dnoQT8or8AK36n51fJj4SroPblXY4SH6d1VYD9Z+numfrKIDEmLe2ERoIA/qX7XGnMVBwkUzGKjRpJGiqun5YNzVSCxJ++/T8nHlkaoXsRHI1hfkICf4sqGUH3vfF2vz3NsVhr/AB5reTfnshw02TSPtrEQiLg+wcta2bGEMhGc4xunIoOrDb9YSfl2ShcSrK37XSQXZuetJhdx13H2zKI7H7BOHwmr3eYCZLvbDdE/0PRplHo6S5CO2taQ8l5+3P1sgHp2h2iGSr/SPXUe9E8TI3K7uilL0U0OcieiKXXO3acG95/olxe1hBxx6NVAi9maou+8stt3+wO45zQ92w8xzXG/g1LVp/azgcNMrf3dL84nntC2n7p2l+3V8z+A3A05SGc5fqKvqnb6FIkHM3KxPJx1WCmMhBQ11vXPZ4Z1k+CqKrhs5tBcfIgIAwRVHIDTl3kYtFdX3/tdaTyea1Zs+Hk1w6QA skQJ1bL4 nRG3FMxXj+pKMrz2shaFC8t4IBGPBip6UOo3VVZCgjRjnOwTfl8t8pusVIZsXtHvLsan6JBGgOcGvkqbzdjg4KnLNyIOolt6ttqPOmnmWGmN/Af7Bzok58jRHTBx6f7yFtw12tsmugCja/q3bUMotytr/f+5oR/ZOFxDUDQRWpw5KAKsVeSBmT4jDAJpa4I8oAtkh/XOmf4hLvILDNSPmvdosXgaEmQ246FF5TXO4HhjXZl82zUCmv3d/HpRy2S9NHoIwpmLup5CqooM429lXhcZpOYOkQHBgEFgohdGgqF6u3Cxt4gGLDcGVApX1VxtG9BAmT7XU9VeTlGhrzg57tXMnWlxTwUVZXArQyRWFfWPTlPuPLguEU+SAUF1g2Eu7c4PJ/5MFJSdWGQE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: memory tiering can be enabled/disabled at runtime and sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING is used to check it. In migrate_misplaced_folio(), the check is missing when PGPROMOTE_SUCCESS is incremented. Add the missing check. Reported-by: Kefeng Wang Closes: https://lore.kernel.org/linux-mm/f4ae2c9c-fe40-4807-bdb2-64cf2d716c1a@huawei.com/ Fixes: 33024536bafd ("memory tiering: hot page selection with hint page fault latency") Signed-off-by: Zi Yan Acked-by: David Hildenbrand Reviewed-by: Kefeng Wang --- mm/migrate.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mm/migrate.c b/mm/migrate.c index e7296c0fb5d5..826afb82ca56 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2630,7 +2630,9 @@ int migrate_misplaced_folio(struct folio *folio, struct vm_area_struct *vma, putback_movable_pages(&migratepages); if (nr_succeeded) { count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_succeeded); - if (!node_is_toptier(folio_nid(folio)) && node_is_toptier(node)) + if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) + && !node_is_toptier(folio_nid(folio)) + && node_is_toptier(node)) mod_node_page_state(pgdat, PGPROMOTE_SUCCESS, nr_succeeded); }