From patchwork Thu Sep 21 07:44:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13393707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2469ACD342E for ; Thu, 21 Sep 2023 07:46:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B67226B0179; Thu, 21 Sep 2023 03:46:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B15576B0184; Thu, 21 Sep 2023 03:46:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A045E6B0188; Thu, 21 Sep 2023 03:46:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 922346B0179 for ; Thu, 21 Sep 2023 03:46:20 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6C7A11CAD81 for ; Thu, 21 Sep 2023 07:46:20 +0000 (UTC) X-FDA: 81259821720.04.8F3E02B Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by imf07.hostedemail.com (Postfix) with ESMTP id EC09140018 for ; Thu, 21 Sep 2023 07:46:17 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695282378; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5ydsHzT7tGpO+G+ATUtNBq2QvjNbDU45vaayLl6WEyk=; b=jdLOcK1BUxQLUP63G3Q8OLY1DgT24a4xWEe93bRcp3UbcbJteUlN3XSX+2m6t8TSJKN1ov OE+iQvSJmGtHE1OjeTIj8j8Yl14PGkLAyuChc8mER4GnQM7OpJktaAVZZQJt3nhgNDhq1k q98GYE2e/7/Gzfrm/p02XrYUt468YXI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695282378; a=rsa-sha256; cv=none; b=7D9he+5P2mPvQclXVh4EPUTlZdVhf2T0leUeWBYhQBsBC/pxDovdKvdC09Mf2xmkXTOHim CF+6e1nEpmlKsLlPsFxE6bkoDK002Z0KPXU6tstYR/RIIYYce6d2vWVgx2//lYRd6s4AJA tLa4OtsdRvDav6dDnpzITAGYJjbAjK8= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4RrnRc37NyzMlkl; Thu, 21 Sep 2023 15:42:20 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Thu, 21 Sep 2023 15:45:56 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Mike Kravetz , , Kefeng Wang Subject: [PATCH v2 1/6] mm: memory: add vm_normal_folio_pmd() Date: Thu, 21 Sep 2023 15:44:12 +0800 Message-ID: <20230921074417.24004-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230921074417.24004-1-wangkefeng.wang@huawei.com> References: <20230921074417.24004-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Stat-Signature: e3xbaihk3oshpn35fqerhqisn8b9ejpu X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: EC09140018 X-Rspam-User: X-HE-Tag: 1695282377-407276 X-HE-Meta: U2FsdGVkX18F+sBW9CO68Rltk9z0p/G3XcBycqbxbX3KmTGK0fJI6blXaH8U/39cfKVAb3709gq2bVxea59GrUrR9RRYseD5VmAcDyJ3m81Yg2ramdn253OIFFWpOjsGLF+bcez9HijoKRiIqQAiHDauFWaAp42DoWQfcBQrtbsDGhQBrfFpVbUjiBVixmaCGZrhBw3ZFZoXsRTGeBK8vpR7oFUcxCJMl+fLVeSJjPGti0c4EcnNeQJ/2XbyIgOnzXoTm7aujrH0M01jHDz+weZw2VV+EFqt5Y9p2QFv95ndlLAfwKPJxdpon86jw62tPo6gm6xAqheWdpicoE9q4eeGffdlhJnalwsDDgMlCwQTWQ4IhnIAmkXQ8P/vBOrcyE9f0jcn8BaNEOEnV/11H0seIh9TRGAD63ZksPYE4Wt7pKd/tmoUNZzQfy4jgnf44F76WEpgx/Sq5nnhhUkHDGXmw8wgNfJ2mDmvfWhxjowi6XzxFeY2R5J1VzaVqqzdw+3Z6rwaI7sueiKHJVUsFEXBFQ3dnSMpzEZadhpcHbps3aQwjXY4crU9harY+3Rn42W9SXWYoWVU+TyPhmsmhfrmDVC/ymMNn4Lp7lblYbAUU2RVaAX7IbRbVauZTt7MMA77c+ohdbZY04LwbeTby61t++vCF+unD+nfc6FiKSkcB+LtdL1NeRlSm8nM5dcRh8f5igv/qm0QGyKKgNE0g+z+8RAmwQDg70KhYHnsPP4Am3poslhW4C7PLEjot8DkyouuGvDzOiOFfuSBV2nA/YRgA/8oahXbbmxLxzsHl2nmG8NsfQkRuTSW/HdEQeb+ANpg6LZK2C4NlowMyeTn2NBiffUnczmfEGnU+zOr5weCYJnYsHhjWdg0FAgcuLd+w4sx4vluDNUNnqOumjwOIRf0Qetq/4TuJUatFTIGxzv0fzrAa25arHeN/fdoTVVVpdbCkDpXYbWOPvD947H 84kZEpZ5 +yvq9hl5xdDmZpEwPOPvupakprNiQpIW6QPA2sgt62yZ+f3fnyEyE7sxsazF7gEWFLErKtj185HdVrAyq4NO4MByJjzeXJJDERyp5ipLEl8AUpsaYuNny2B6Vnj3Mg4qScGkaRmS29YBw6H6o5qCb0b4mvf91OyPmgYhsXX42akl4V+IPUP5HzKAFghOTP1CufjDI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The new vm_normal_folio_pmd() wrapper is similar to vm_normal_folio(), which allow them to completely replace the struct page variables with struct folio variables. Signed-off-by: Kefeng Wang --- include/linux/mm.h | 2 ++ mm/memory.c | 10 ++++++++++ 2 files changed, 12 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 1f1d0d6b8f20..a1d0c82ac9a7 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2325,6 +2325,8 @@ struct folio *vm_normal_folio(struct vm_area_struct *vma, unsigned long addr, pte_t pte); struct page *vm_normal_page(struct vm_area_struct *vma, unsigned long addr, pte_t pte); +struct folio *vm_normal_folio_pmd(struct vm_area_struct *vma, + unsigned long addr, pmd_t pmd); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr, pmd_t pmd); diff --git a/mm/memory.c b/mm/memory.c index 983a40f8ee62..dbc7b67eca68 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -689,6 +689,16 @@ struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr, out: return pfn_to_page(pfn); } + +struct folio *vm_normal_folio_pmd(struct vm_area_struct *vma, + unsigned long addr, pmd_t pmd) +{ + struct page *page = vm_normal_page_pmd(vma, addr, pmd); + + if (page) + return page_folio(page); + return NULL; +} #endif static void restore_exclusive_pte(struct vm_area_struct *vma, From patchwork Thu Sep 21 07:44:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13393709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02FA4CD342E for ; Thu, 21 Sep 2023 07:46:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5C0A16B0188; Thu, 21 Sep 2023 03:46:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 597EE6B018A; Thu, 21 Sep 2023 03:46:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 486886B0192; Thu, 21 Sep 2023 03:46:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 37A026B0188 for ; Thu, 21 Sep 2023 03:46:37 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 07A1B1A10C3 for ; Thu, 21 Sep 2023 07:46:37 +0000 (UTC) X-FDA: 81259822434.17.7623141 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf10.hostedemail.com (Postfix) with ESMTP id BA2DBC0002 for ; Thu, 21 Sep 2023 07:46:34 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=none; spf=pass (imf10.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695282395; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ok1PI54IQsKeQymGDTd8xDFL/XYgPYhYno6RdRByNRM=; b=InzfldM2yxxWybZllMWvL5/mSykHPWHfUexi9ctKpK7D+rB7SPrmSWzOOPKHBKQn3sCf2x gjVEwEE2VtJLjh6RJ2h+lNZOHc1+cstL+synxQzWaRwTL/b7O3R3eQXz378RG/cV67s8Kp LtynTf3C8oXUzVBYLBjb6limbfI19Wc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695282395; a=rsa-sha256; cv=none; b=wfMh1GXv7CrdJgwVGDNZHjkwVcPCHEa+KNIuu3q/uCjH926Jknl4eZo/HRJVvmR2sOIKdh qakMoo2YKlSOd7Tpl9iYkv/nBxEwQ9fRlrP4qrq1g+zmWJX8FjKGg9a8oR4PomJOmVVdxD 5Vo99q8RElQZwVzM7Ghirgbt9cQyay8= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none; spf=pass (imf10.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4RrnTK2HcnzrSxT; Thu, 21 Sep 2023 15:43:49 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Thu, 21 Sep 2023 15:45:56 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Mike Kravetz , , Kefeng Wang Subject: [PATCH v2 2/6] mm: huge_memory: use a folio in do_huge_pmd_numa_page() Date: Thu, 21 Sep 2023 15:44:13 +0800 Message-ID: <20230921074417.24004-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230921074417.24004-1-wangkefeng.wang@huawei.com> References: <20230921074417.24004-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: BA2DBC0002 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: r964m3hf1syhhdrwtj6bk9fdnwa5geta X-HE-Tag: 1695282394-534739 X-HE-Meta: U2FsdGVkX1/8PgQsxmVmXWsA4E8Jqy/oqI1+ueGOdmVDtEJtmQtifRo9i5Ea38Sv9EgOP4m2hzoQoLz3e3a/H0JG68bAu66OV2Ugqt29+4/V46uk1WOlOJItCyeBNtvjwTzJu0DiQdprZ7eQlGe9sVfXrmnnBY3w4ZT1HrKW6rKt7JGOF0X/N1WK3/JP7OqKxbkT0lZzO3aLCcsp/wZIZODVgmTNhVUIsu5P4YypFhPdkm3Y9CyKraboft+R/8Mr5d5hyWD2yEco431iJ6RunlsZvp9NXeMAAfD7ZEupN1fUW9VF5Gc+GCuaXmIruikaJwmqOuLIp5r57vgxp2fQuWWGX75IF1mjUoknfM2ym2OSDTV2w/XWAYwbRutWOEqN0Of8FCR2kOlOt4zBag3fC1yv+L/Nnri/KzmM6sXKYVuxv7yFaojoE2w703usRQmodiODFzbI8yfJAahmEXoIjY+9nU3cJYGKlIljY3vuUQyx9Wnznw4HHoCoJovE5ihvUjojxYyDKGkhGGYWi/Cz2lsepcND9cbWr8+jSFWY7a+KcO1wJR4TrVI/EyybRb6ptg9QC/sCo6TCSlUtro/zFdzYy+Zb3gag2hIBFDEfSk8xCwO4KiWwTDZsV7Afjr68XgPXu3Jr81D2RVUbQ+yguHaR1mCllajwTmft3kN8hZFjn/Kl0f1IFeQbtV8HEQwU5QTy3UAObHgr67A1xRevOabJqsqpl/vDFpxd4z94T0RZbDbx0tJm0Pw55vB51hsiPAK8s9olKd/SmhkAB6JYMBAZuDOtJtg31JwrPFIQoVCqR3tqf4Qbg7bRYBHbYugGwhEvJwtBZsNDBmj909rZmmwmasMBmef1zt1x+A1Bf+ry/vCtZpJoQV3KMlT3ZqCjULgr4I+mgTihmde3f8RHSkrsgIo5Vvv5kxQ+V+xzrSuIRToPZkqSuiEuSjtSf/4CR87DwYHzyK9uRL1ruXj ZYCKz9BE AkeY/FAxSF421LtAsP1iKvdM5antbQPiewcqQTDT6pf+q9GP63rPTZ5rnsT0yJB+bpTFwCuYZhJDfMnE3+mCR7z4b74fmlwpIKF0nwFgmEAYSWpHPmtUMnyrM44/P3jok3ezl3Zqpl6ccoVe0KyblBtaawPEtlWHtxx9BKwLWEPRk11DBRDt76hVTl9NYRM/0eU1/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use a folio in do_huge_pmd_numa_page(), reduce three page_folio() calls to one, no functional change intended. Signed-off-by: Kefeng Wang --- mm/huge_memory.c | 29 +++++++++++++---------------- 1 file changed, 13 insertions(+), 16 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 0da653080d8c..53472e34a761 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1517,9 +1517,9 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) struct vm_area_struct *vma = vmf->vma; pmd_t oldpmd = vmf->orig_pmd; pmd_t pmd; - struct page *page; + struct folio *folio; unsigned long haddr = vmf->address & HPAGE_PMD_MASK; - int page_nid = NUMA_NO_NODE; + int nid = NUMA_NO_NODE; int target_nid, last_cpupid = (-1 & LAST_CPUPID_MASK); bool migrated = false, writable = false; int flags = 0; @@ -1541,36 +1541,34 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) can_change_pmd_writable(vma, vmf->address, pmd)) writable = true; - page = vm_normal_page_pmd(vma, haddr, pmd); - if (!page) + folio = vm_normal_folio_pmd(vma, haddr, pmd); + if (!folio) goto out_map; /* See similar comment in do_numa_page for explanation */ if (!writable) flags |= TNF_NO_GROUP; - page_nid = page_to_nid(page); + nid = folio_nid(folio); /* * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ - if (node_is_toptier(page_nid)) - last_cpupid = page_cpupid_last(page); - target_nid = numa_migrate_prep(page, vma, haddr, page_nid, - &flags); - + if (node_is_toptier(nid)) + last_cpupid = page_cpupid_last(&folio->page); + target_nid = numa_migrate_prep(&folio->page, vma, haddr, nid, &flags); if (target_nid == NUMA_NO_NODE) { - put_page(page); + folio_put(folio); goto out_map; } spin_unlock(vmf->ptl); writable = false; - migrated = migrate_misplaced_folio(page_folio(page), vma, target_nid); + migrated = migrate_misplaced_folio(folio, vma, target_nid); if (migrated) { flags |= TNF_MIGRATED; - page_nid = target_nid; + nid = target_nid; } else { flags |= TNF_MIGRATE_FAIL; vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd); @@ -1582,9 +1580,8 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) } out: - if (page_nid != NUMA_NO_NODE) - task_numa_fault(last_cpupid, page_nid, HPAGE_PMD_NR, - flags); + if (nid != NUMA_NO_NODE) + task_numa_fault(last_cpupid, nid, HPAGE_PMD_NR, flags); return 0; From patchwork Thu Sep 21 07:44:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13393710 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC910CD5BD5 for ; Thu, 21 Sep 2023 07:46:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7E3EF6B0175; Thu, 21 Sep 2023 03:46:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 793E96B018A; Thu, 21 Sep 2023 03:46:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 682D76B0192; Thu, 21 Sep 2023 03:46:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 580206B0175 for ; Thu, 21 Sep 2023 03:46:48 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 3A4FD1CAD10 for ; Thu, 21 Sep 2023 07:46:48 +0000 (UTC) X-FDA: 81259822896.14.0C1AE7C Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf22.hostedemail.com (Postfix) with ESMTP id C4561C0016 for ; Thu, 21 Sep 2023 07:46:45 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695282406; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e8LXOfIchvrTg88g1fmJqfa/w5XDt2uRV+C4ET9YPS8=; b=WEAb31jFl7+vRj2SIOSSNdiGBJc1AH0v86UaJ13sv8Xe/8dwVOHKFF6CpM1PvjdszQgau8 5je1PGyieHnZBxaM9XlLWC1qixA4LorF0WmcmTQmvp5nIea+DPIJQ3rY5/HkfHTn9+DHqo NmuTOd1sBLQtkWUROSrhW4MB817pYDI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695282406; a=rsa-sha256; cv=none; b=DhhIy2COLQqgcgh0fGnd+NiBr4QbMVoU8vG1LCaBKaWerx6pW/7uNtcM6yuDexMM/EUGER dn/wENsYGdD3mWx07GyNtLg7BbGoZioGLv6PBHjWhyFAw4B+OLCWhSr+oA/qIrkww533x9 6j7N1s8/GxxstD32p1fTLm49VF/1Jvg= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4RrnQt2BRGztSwP; Thu, 21 Sep 2023 15:41:42 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Thu, 21 Sep 2023 15:45:57 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Mike Kravetz , , Kefeng Wang Subject: [PATCH v2 3/6] mm: memory: use a folio in do_numa_page() Date: Thu, 21 Sep 2023 15:44:14 +0800 Message-ID: <20230921074417.24004-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230921074417.24004-1-wangkefeng.wang@huawei.com> References: <20230921074417.24004-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: C4561C0016 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: sx8wsma73bkydjuj5g6der5975mgpup5 X-HE-Tag: 1695282405-22984 X-HE-Meta: U2FsdGVkX187MREPG1Kwe64R6eAD9CJ2kwzOxVAHXH6C5OXt1KfUhGEZDihhFRNOyMQkZVXlgRy2lILOdYnyuBjs2tMkJJgL1JY1JbbJZoyYHIqHobDPC4ZMsG9ZBQyBrKLGy+hkFc9BehiAR157VIlpIBE8LJ47eQaJj7VD+LsKcPNcZTNwo6pNZhWtS9qYgzdQfYQmL8+r9bXXX9qIW7i0a262rTNpZ9K1RSpmjw+4RdK46oN4ZBUA2+ga1U5+e5J//wKDC+x62RH8l26VTDz9ek7OPOvKyh+ZmVDv1KcFmGIqj1Xd+g3IXXLgUGywjO1MQ/cQJKMb+plB4JpKCjTd2AytwPMJosFzGaD4zryYr+i9uCuyT/Q06EzTsZe5KGz7YcUP+2PdXSxHgfz9XVJnNGVCiB3trMrzsRSMRZyCXODLLZSO1S3/rlOE7ouzZ+dx8sRAAxGwR8Lh97dcjaY3eHAHuUJMoxQs4LCGDNV0oANa5rCyVDdFd/DtPpisggPYvS6P8S0ACaTQRTKmQBkHrdoVcsZNXyIgpytTyttK1i5cgrH7ihmUJkhXREP0+EPNFIuVM7GXzDps853p4tQHR3y5S0rDnGkHGs1/nH4Sa7k/OAciUHqONs83VRz51DKa1JcAwQoiTHUPDq28MfEq0V+4T1RcVyWEY0aGDoEptwQTA72fCUd0zP/uLa+Wx1PEvJIRyVrCRNhPAoVRPP++mOjwYyR/RYGo3iXsh3CZ/fHx+pmcKf2e2gsnwbQtfD6dEUtPm+tSApkYvmNzp6lropaTJpAVet79hrfe/xHjlhI+3XR3PIVxXs0O/qMRY5TUvD/HH8TIKqjPqRfgG+Ot+JkRrumg9AixdgaU6ufEErR3hNDY95TTjafi2458oyAr+WxWvJWook62wHN3Lg+I3UPMB7MrEiA42iMslYwU2ZYO6o19maIX2n1B/nTDBvqjnEwsqzPzhIHSyYM YddFhPFy rNJF9XWxNj2X4twwMbuPEzkJjXGVodrxDFWAw06/RmJzft+cmvZwyHP3/bXxd3QDGkjqmWx0d9VfXO1DOgA1M7pfpNJLL7stkk0QKw5yzRYJ4OhiDSTV9QB97SVbqI2O+VvuTIMka1p6koJsNfI/Gk4xW3nEcnw5FcPkhfGJXczzSwd+JxPDNQDPKuq80kwBVO3mR X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Numa balancing only try to migrate non-compound page in do_numa_page(), use a folio in it to save several compound_head calls, note we use folio_estimated_sharers(), it is enough to check the folio sharers since only normal page is handled, if large folio numa balancing is supported, a precise folio sharers check would be used, no functional change intended. Signed-off-by: Kefeng Wang --- mm/memory.c | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index dbc7b67eca68..a05cfb6be36d 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4747,8 +4747,8 @@ int numa_migrate_prep(struct page *page, struct vm_area_struct *vma, static vm_fault_t do_numa_page(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; - struct page *page = NULL; - int page_nid = NUMA_NO_NODE; + struct folio *folio = NULL; + int nid = NUMA_NO_NODE; bool writable = false; int last_cpupid; int target_nid; @@ -4779,12 +4779,12 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) can_change_pte_writable(vma, vmf->address, pte)) writable = true; - page = vm_normal_page(vma, vmf->address, pte); - if (!page || is_zone_device_page(page)) + folio = vm_normal_folio(vma, vmf->address, pte); + if (!folio || folio_is_zone_device(folio)) goto out_map; /* TODO: handle PTE-mapped THP */ - if (PageCompound(page)) + if (folio_test_large(folio)) goto out_map; /* @@ -4799,34 +4799,34 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) flags |= TNF_NO_GROUP; /* - * Flag if the page is shared between multiple address spaces. This + * Flag if the folio is shared between multiple address spaces. This * is later used when determining whether to group tasks together */ - if (page_mapcount(page) > 1 && (vma->vm_flags & VM_SHARED)) + if (folio_estimated_sharers(folio) > 1 && (vma->vm_flags & VM_SHARED)) flags |= TNF_SHARED; - page_nid = page_to_nid(page); + nid = folio_nid(folio); /* * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && - !node_is_toptier(page_nid)) + !node_is_toptier(nid)) last_cpupid = (-1 & LAST_CPUPID_MASK); else - last_cpupid = page_cpupid_last(page); - target_nid = numa_migrate_prep(page, vma, vmf->address, page_nid, - &flags); + last_cpupid = page_cpupid_last(&folio->page); + target_nid = numa_migrate_prep(&folio->page, vma, vmf->address, nid, + &flags); if (target_nid == NUMA_NO_NODE) { - put_page(page); + folio_put(folio); goto out_map; } pte_unmap_unlock(vmf->pte, vmf->ptl); writable = false; /* Migrate to the requested node */ - if (migrate_misplaced_folio(page_folio(page), vma, target_nid)) { - page_nid = target_nid; + if (migrate_misplaced_folio(folio, vma, target_nid)) { + nid = target_nid; flags |= TNF_MIGRATED; } else { flags |= TNF_MIGRATE_FAIL; @@ -4842,8 +4842,8 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) } out: - if (page_nid != NUMA_NO_NODE) - task_numa_fault(last_cpupid, page_nid, 1, flags); + if (nid != NUMA_NO_NODE) + task_numa_fault(last_cpupid, nid, 1, flags); return 0; out_map: /* From patchwork Thu Sep 21 07:44:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13393706 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 780F8CD5BD5 for ; Thu, 21 Sep 2023 07:46:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1544E6B0175; Thu, 21 Sep 2023 03:46:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 103516B0179; Thu, 21 Sep 2023 03:46:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F34E46B0184; Thu, 21 Sep 2023 03:46:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E32B36B0175 for ; Thu, 21 Sep 2023 03:46:04 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id A139E80FE7 for ; Thu, 21 Sep 2023 07:46:04 +0000 (UTC) X-FDA: 81259821048.24.C57A39A Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf22.hostedemail.com (Postfix) with ESMTP id 2B9A7C0009 for ; Thu, 21 Sep 2023 07:46:01 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695282362; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uPXv4KuxwnoLpD+Fz5M9itVod0PMy/6XrloeJ6kolxs=; b=QfULnbyfEi6fOMHFdm+qRPOE6jlrsHaqmswvtdpLep9AxBEa5SD185mGvSsXcmOO5rZopp /hwYYQ8TuCP77gkD5hOyiZuaiQyQwxXRRJVK2nDdXwMlbHZOT3ZdsQlWwWx/WOFj7dx7EI Y6SAmfEdoKr7TMCcuONQkgUJ1KCUJo8= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695282362; a=rsa-sha256; cv=none; b=iX7QecA0v8rq5BN5GpqzKu31sE9yilBh6yPVhua0Uwlh3qECZ8l31Oru27ggbB59K3Jw60 N7R4uv49leIcl/OWWcLQUx6xYctzpiqLO/xLsHaLRaQUp5Ie/aRb55cwsqHqEG2EJAMA0d MxYmVrn7ZytmzB+LZo5a1kgcIoEacEU= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RrnSL6bjnzVl9m; Thu, 21 Sep 2023 15:42:58 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Thu, 21 Sep 2023 15:45:57 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Mike Kravetz , , Kefeng Wang Subject: [PATCH v2 4/6] mm: memory: make numa_migrate_prep() to take a folio Date: Thu, 21 Sep 2023 15:44:15 +0800 Message-ID: <20230921074417.24004-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230921074417.24004-1-wangkefeng.wang@huawei.com> References: <20230921074417.24004-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 2B9A7C0009 X-Rspam-User: X-Stat-Signature: i8cqjzzzkphyp5tnqkwuqt1yei5j3x3n X-Rspamd-Server: rspam01 X-HE-Tag: 1695282361-808834 X-HE-Meta: U2FsdGVkX1+b/DO9WV+R/5w/olVg2qCI7D94HDgeRyNiTNSzGCVsFPcGOVnoG6dghUkn0JIqm62WzSlmXTKD7ihp+Z0TjZhg5TPZ5x4rs+3vgA3v4LDQBSsh/KthPhCKYtuMm10L1XZkxN58ylLAdezstNgxQYchX5UCXhl7hKaSKVG1H/uNEO3jXjwTsufdI5M8n06Kt3NRAGtw6kDMV3p/qV9DtAHT95ELt0gRWS43smO5DeJbr3NW1ViFFjSyPmiO1ytBvMZ9kGHhAn8OkoCLrzHbDwcDVdETGlYM8f1A8oZ6z6JdwlSOYcXage5laJREsFDgHrBjoAXsCnAqQJpyqjpbqotWL2NgsAsfgBO9tLmdcfe+t9POy/0R8rKAgCBKSD/c+5FD4mhSs+W/nYz9UV3aoh67Bt4V2ZLJKmtUAR+n4Y9DmGiC71A9cI0yfyV16aUMzck+XT1Zj871cXzL+4ZYOeE5ocj8rXGojiLB+01m9/IqHooW/GFBxUFP2f9EkHqbZ9igR3nRvPQ81FXPATVjL3uXXJOj8l7iNGCw73nCgEUqTdSJVB4BqB39k13SWeDqKs5ghKxjt57kGsARJJBL5CbxZd3aVx5NadhgyDh4QVJoxVbJAtxzqzDt9pRXhxBGnCkEqDQYOj4n3aleUUY1AtRGb56J8CNyZ5feIca63P0AOBMyplap1RdrYTtxqjarHQMQ3+nOFfJ4WEDL688x5mQnbeZmOupq63PqGx4l5FdzktchMBCpla4HSnz7PE8Wsrz8HFz73Tz5kPXw6xTzB4tvqRe+qMxFwTelEKM5+xeMXI6LCf0XzPjJWTxrhKeGIGb/pw6IcsXB+T/+91obB3iVHwAEucy46v5NoGsJu+zzxJurYo0UzjSrVxnaPRjjIucjfovnSdO+1yzf/3qLn1i0p0c2zcXGq/4duoydFLIOOxpHdGA4NLgAUE8lOTWg6owv1weFXQP 5g5jA7+h ybcTNlaxil3a3XBK3U6punH0MhzjQCXc+3cAgG5Ub5f2OjGyn7/U4iqqc+W+ImApxbooasSEV/YDhpekjoOnGfMocEGkBFJmr2G4tJo83t/MrYN1OA4f5RvFKZosExbBHhE6I0Hbzgo3vVnOx3BsMZAL6fzHFB0Ro1lYxuDlYadChEdIGAOU7FBqvr7t4kDybxvVK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for large folio numa balancing, make numa_migrate_prep() to take a folio, no functional change intended. Signed-off-by: Kefeng Wang --- mm/huge_memory.c | 2 +- mm/internal.h | 2 +- mm/memory.c | 9 ++++----- 3 files changed, 6 insertions(+), 7 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 53472e34a761..0f93a73115f7 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1556,7 +1556,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) */ if (node_is_toptier(nid)) last_cpupid = page_cpupid_last(&folio->page); - target_nid = numa_migrate_prep(&folio->page, vma, haddr, nid, &flags); + target_nid = numa_migrate_prep(folio, vma, haddr, nid, &flags); if (target_nid == NUMA_NO_NODE) { folio_put(folio); goto out_map; diff --git a/mm/internal.h b/mm/internal.h index 7a961d12b088..d7916f1e9e98 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -984,7 +984,7 @@ void vunmap_range_noflush(unsigned long start, unsigned long end); void __vunmap_range_noflush(unsigned long start, unsigned long end); -int numa_migrate_prep(struct page *page, struct vm_area_struct *vma, +int numa_migrate_prep(struct folio *folio, struct vm_area_struct *vma, unsigned long addr, int page_nid, int *flags); void free_zone_device_page(struct page *page); diff --git a/mm/memory.c b/mm/memory.c index a05cfb6be36d..93ce8bcbe9d7 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4727,10 +4727,10 @@ static vm_fault_t do_fault(struct vm_fault *vmf) return ret; } -int numa_migrate_prep(struct page *page, struct vm_area_struct *vma, +int numa_migrate_prep(struct folio *folio, struct vm_area_struct *vma, unsigned long addr, int page_nid, int *flags) { - get_page(page); + folio_get(folio); /* Record the current PID acceesing VMA */ vma_set_access_pid_bit(vma); @@ -4741,7 +4741,7 @@ int numa_migrate_prep(struct page *page, struct vm_area_struct *vma, *flags |= TNF_FAULT_LOCAL; } - return mpol_misplaced(page, vma, addr); + return mpol_misplaced(&folio->page, vma, addr); } static vm_fault_t do_numa_page(struct vm_fault *vmf) @@ -4815,8 +4815,7 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) last_cpupid = (-1 & LAST_CPUPID_MASK); else last_cpupid = page_cpupid_last(&folio->page); - target_nid = numa_migrate_prep(&folio->page, vma, vmf->address, nid, - &flags); + target_nid = numa_migrate_prep(folio, vma, vmf->address, nid, &flags); if (target_nid == NUMA_NO_NODE) { folio_put(folio); goto out_map; From patchwork Thu Sep 21 07:44:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13393708 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31DD9CD5BD5 for ; Thu, 21 Sep 2023 07:46:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C9D886B0184; Thu, 21 Sep 2023 03:46:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C4E596B0188; Thu, 21 Sep 2023 03:46:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B3CCD6B018A; Thu, 21 Sep 2023 03:46:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id A6E246B0184 for ; Thu, 21 Sep 2023 03:46:36 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 71B5116104F for ; Thu, 21 Sep 2023 07:46:36 +0000 (UTC) X-FDA: 81259822392.16.8700870 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf29.hostedemail.com (Postfix) with ESMTP id D2D2B12001A for ; Thu, 21 Sep 2023 07:46:33 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695282394; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3bwHOsF7AvN9Rh08QFZbRWN+H4js2n8PQQGzoYl0bE0=; b=rxPHkODq7U3thI1YmLATb/PdmxqWcL1uh6k2S7EDWIt+aPI5tgn820YlnV1Zi6xX88Yio+ GPIu6mMtYvn+eUR1kJSpTN5txpFabSzpn+dL5ecNaLIf0IGzj/1zOhK/VuOllyzRHL00ts Xlj580L8amuXD97yuoXzYX0BW8kamA8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695282394; a=rsa-sha256; cv=none; b=cJARZRMMMAfDiC5kQMPZDC0aQvb7qU5nBmYAuSfSuHM996YDauPPe1NHw1OYwngvhiuONs o5m44NYANkNn74EgZ0amTt1xiOhvvuqajVsWgHcakuPUVzbn+ymCW1AeuGFxcgRENLAh0p V2HYZEgQXF2RsHLpNe5EJnb99mtrup0= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4RrnQv46tyztSwc; Thu, 21 Sep 2023 15:41:43 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Thu, 21 Sep 2023 15:45:58 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Mike Kravetz , , Kefeng Wang Subject: [PATCH v2 5/6] mm: mempolicy: make mpol_misplaced() to take a folio Date: Thu, 21 Sep 2023 15:44:16 +0800 Message-ID: <20230921074417.24004-6-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230921074417.24004-1-wangkefeng.wang@huawei.com> References: <20230921074417.24004-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Stat-Signature: kgpohsa6bu1i5394ea31h6oysgetoa3h X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: D2D2B12001A X-Rspam-User: X-HE-Tag: 1695282393-503916 X-HE-Meta: U2FsdGVkX1+B5Zh39KvocMunApFhEV5f+kb0vybh3eAxLZAbmp7ojVSrdGikaj97gWyJ4qEwvEpKhWqat38GvS9wUPgV1jRydUeEXdUQ7nNdEe/BrJuvyRUwK8t+M+C0wa82mN2btDFJKoN9LbEkQvkAW05lzk+Hw4B8pu6MOqk/aP/pd3fqNAV9zvBmuha+8bFSvuFNhCmMlwD1LAKGfTSZS/CVxx27hOHpKeQcyzDWOK9+KeBRyAZQDXY2R0ZDNtfZFHyD/cTohnVooU2Zdjs8tU0CM7EyzHAzJYlFJr7GU4ZE0pj1dCZJE6+xDxAyB4s4/e0NNKCPM9nlz+Yam1p5laAJXD7Hh1wEOaeEW1uRv8WhsFP+zGiM52TwExgUney9idDb7zZfeZl4sAZ+MuON/nHsGZ8AHmJ0qeyjvRfrgKvEQISTm5pXMkL+c3joMD3SQRStQ5JkP04fJ4sxeUTSJ+WaMzHgYhi8PPKyQXCrosVOsmcLbo78PpT9ipqa97LV7AGLGOE9FByqZcGf2E6Gg0rCF9yZwcC2rIwMtZ53EESRrzlxUeLFtxYCfaSHrQ4UN9sztZfWibcIHbdTbssfXV5bfIky1JA3mEShKK9kbsYDU/iW2y8JJW/YqICI5lHM/4q9OxTWIkOKufVOqsXvws4dD64oPNuIrn06LrGohpjn6GFSp7S2bOr16jHs3SWdoPdGbS7c6KScAAT0UrcTtSDKVRpuJM3fu/pOznS3H9vOsmFMWFC0DK6nHxNkgLkhZq7ELcyiVk3iRlJ22mAzbgIw5r97Tuv5vtMwVXFDk+xWvtPt2Y5a8ydNGG1+QQcuKQNWvRmomgat3tRT5K4ZU7g0yMQpKmtrcu8mF9hkz7UP1UeV8WWd4yehRuHjTGMoFJR8HeoCYKgq3ZrtNdOD1KSt/6QDpUxSVRVTmFQUoRPIUEXFpF8fNYjol656nEtOudS07hXyc5OxSNk B5e+DohR oD1xANHdauEV3WsJUVeQUfB7ACssf4cm1Jo4rNH5clC+2YOApH4wLz1jBqhNavo+raYiEOuS5ngOrzErSu3NTNpqqzBpdWfl8wj2zjhvkc25Lk+mr8XAnUXoz10NY6txQJgmASUaDJb/kMATN5D1t7TdrEd8aOFpQGw8JKIOT0Bw4I75EPwfJwIYyfmmkj6QI4VUY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for large folio numa balancing, make mpol_misplaced() to take a folio, no functional change intended. Signed-off-by: Kefeng Wang --- include/linux/mempolicy.h | 5 +++-- mm/memory.c | 2 +- mm/mempolicy.c | 22 ++++++++++++---------- 3 files changed, 16 insertions(+), 13 deletions(-) diff --git a/include/linux/mempolicy.h b/include/linux/mempolicy.h index d232de7cdc56..6c2754d7bfed 100644 --- a/include/linux/mempolicy.h +++ b/include/linux/mempolicy.h @@ -174,7 +174,7 @@ extern void mpol_to_str(char *buffer, int maxlen, struct mempolicy *pol); /* Check if a vma is migratable */ extern bool vma_migratable(struct vm_area_struct *vma); -extern int mpol_misplaced(struct page *, struct vm_area_struct *, unsigned long); +int mpol_misplaced(struct folio *, struct vm_area_struct *, unsigned long); extern void mpol_put_task_policy(struct task_struct *); static inline bool mpol_is_preferred_many(struct mempolicy *pol) @@ -278,7 +278,8 @@ static inline int mpol_parse_str(char *str, struct mempolicy **mpol) } #endif -static inline int mpol_misplaced(struct page *page, struct vm_area_struct *vma, +static inline int mpol_misplaced(struct folio *folio, + struct vm_area_struct *vma, unsigned long address) { return -1; /* no node preference */ diff --git a/mm/memory.c b/mm/memory.c index 93ce8bcbe9d7..29c5618c91e5 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4741,7 +4741,7 @@ int numa_migrate_prep(struct folio *folio, struct vm_area_struct *vma, *flags |= TNF_FAULT_LOCAL; } - return mpol_misplaced(&folio->page, vma, addr); + return mpol_misplaced(folio, vma, addr); } static vm_fault_t do_numa_page(struct vm_fault *vmf) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 98fae2bfc851..ecf06ce3a5dd 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2572,24 +2572,25 @@ static void sp_free(struct sp_node *n) } /** - * mpol_misplaced - check whether current page node is valid in policy + * mpol_misplaced - check whether current folio node is valid in policy * - * @page: page to be checked - * @vma: vm area where page mapped - * @addr: virtual address where page mapped + * @folio: folio to be checked + * @vma: vm area where folio mapped + * @addr: virtual address in @vma for shared policy lookup and interleave policy * - * Lookup current policy node id for vma,addr and "compare to" page's + * Lookup current policy node id for vma,addr and "compare to" folio's * node id. Policy determination "mimics" alloc_page_vma(). * Called from fault path where we know the vma and faulting address. * * Return: NUMA_NO_NODE if the page is in a node that is valid for this - * policy, or a suitable node ID to allocate a replacement page from. + * policy, or a suitable node ID to allocate a replacement folio from. */ -int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long addr) +int mpol_misplaced(struct folio *folio, struct vm_area_struct *vma, + unsigned long addr) { struct mempolicy *pol; struct zoneref *z; - int curnid = page_to_nid(page); + int curnid = folio_nid(folio); unsigned long pgoff; int thiscpu = raw_smp_processor_id(); int thisnid = cpu_to_node(thiscpu); @@ -2645,11 +2646,12 @@ int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long BUG(); } - /* Migrate the page towards the node whose CPU is referencing it */ + /* Migrate the folio towards the node whose CPU is referencing it */ if (pol->flags & MPOL_F_MORON) { polnid = thisnid; - if (!should_numa_migrate_memory(current, page, curnid, thiscpu)) + if (!should_numa_migrate_memory(current, &folio->page, curnid, + thiscpu)) goto out; } From patchwork Thu Sep 21 07:44:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13393711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50A80CD5BD7 for ; Thu, 21 Sep 2023 07:46:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E99446B018A; Thu, 21 Sep 2023 03:46:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E49DD6B0192; Thu, 21 Sep 2023 03:46:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3A456B0198; Thu, 21 Sep 2023 03:46:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C55C36B018A for ; Thu, 21 Sep 2023 03:46:51 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A4C2C1A10CA for ; Thu, 21 Sep 2023 07:46:51 +0000 (UTC) X-FDA: 81259823022.19.D423E68 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf04.hostedemail.com (Postfix) with ESMTP id B222A40013 for ; Thu, 21 Sep 2023 07:46:48 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695282410; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6nA4CXbR7MXdLwEB7Owk9QkLMvz0OmzW1L385S+xh+k=; b=4IiD+ba9jfLeJP8GUqVEDgEEqiZSBOyvgYXXVrx4VmzeAIVCElC9rPQHGK9rlyJ7GDOOUr yWpMsrCKTCgnBADH8BD0dRWcAEu26y3tZfYKKiDqwu8tl7nLsy3dkBFwUkrt9AdY9GCg/a Gy3DZS+V9kyEZwObt+fFLbzdU0DA+mU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695282410; a=rsa-sha256; cv=none; b=eSYuJzyc1GE0oOur3cenC+x2LgjMcmS1gBIuaKG4ykGIAyzhrkTOccMxBVNBtfMM0L2K42 roF5PQqXABXCVi+LHYqMqwrROf5A1hSUtMb4NihZd2OIcu5189vCkzc07OCQLiJMwa2CVK Qlxvz7ewNSR4G/V5fyG/Xpm7hGGLw3g= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RrnSM73xmzVjNt; Thu, 21 Sep 2023 15:42:59 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Thu, 21 Sep 2023 15:45:58 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Mike Kravetz , , Kefeng Wang Subject: [PATCH v2 6/6] sched/numa, mm: make numa migrate functions to take a folio Date: Thu, 21 Sep 2023 15:44:17 +0800 Message-ID: <20230921074417.24004-7-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230921074417.24004-1-wangkefeng.wang@huawei.com> References: <20230921074417.24004-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: B222A40013 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: u9j9baxsa7oy4masqybnyyhwpcbrcx1b X-HE-Tag: 1695282408-20647 X-HE-Meta: U2FsdGVkX19sV1n/RuoVkG8hUP+geH1+QwiQahZJoYEjSSX1SwToyMbcggXJ6ps+Yv+ViHB5M6YBDwNzotv/wIw2cKyDr8zj6acZmQhKMc4DPevK7vTGzCU2vuFf/nPaojnyLmi5/h9cjSH2b/r+faBwBqiZMERdRx8eEjollzImL80K5KeRLj9kjrGM/xdf+rJ8sUpGq/4v1+9n1C+CzVqARNtFetNxp+O03pjd/EtSNE4roswynB0XSbBbI9ERmZeus8jE05UgQyW37UQ0yr8MaLRts34ny2yntGBU8zmfk2J2YiRnDW78dCPYIaWBfl3HWNW3dpCfL0NPQo3nLfNwq7u59hzujqgEuP9ZHWa/dT/RizlcuXTHGnodezqcH5MkCOboPr/NiIOyQSSLYCmqUdBtQSC3LPedzkYecGUce8tbeiU7FRekaKdqrG8hJowElVKY6JAxWXyUbR8DHUH6ksaToL4Gbt3K6Y8XEVfVBaXzqNKZqgd4sWpGRoc84YXxnnm6FK7D3tTu9RkC9xXvT/VGYbrpZz6CmAms5QFAhbt6SKRby6BCQEN4EzJoidhVt5hhtLTm3E847PZXyhk3YudeVRbHzazvpfX1veZ2FV/9glaTAUOT7LxeIfpJZoQUEoNuazP26bHenkihelGQnwFSa7nIY3usLNPZeFc9i5E8n53Qhx5ETgge7eXpI7PeXIcduAOOuKEzVYNq7Ps09PH5uECPyTe0Gj6ffDkYQwoAWgSGK23PYLk1YFO22xYEVqeYX0IkzUGGncktrvXcMkJJXCldyHPa9bSmO9lxxaetxbUPD/m/9cKuH1iQWN2SoZltAt3XlkULmqS2TJGeEFSf9ToqtKB59dijEJ7EXogR4Kn+4vSUhlzjsdoawf1putdy2d/sQwKt3iQQSLPT7wesOQ8GAwgZBMs0NJMJuGEoP7qbpRjwkyXgWAQ3mvYkeTBPB5iteeDUaVZ rO4pFPOv PF7hWwxsuwarOaHLDZRGCfvbDQ4+RVpEUvdqIbM1oIEkKXHTS7w3d/ujFPJQuFFBchfoFDtFTX7mObazw8BDyNqW0wcl5WNb5anqKN2KJgv79J1XIlv7LO5gCdcFsr/nyDCQdn3VSyqPgZTrS3+vyouVrWQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The cpupid(or access time) is stored in the head page for THP, so it is safely to make should_numa_migrate_memory() and numa_hint_fault_latency() to take a folio. This is in preparation for large folio numa balancing. Signed-off-by: Kefeng Wang --- include/linux/sched/numa_balancing.h | 6 +++--- kernel/sched/fair.c | 12 ++++++------ mm/mempolicy.c | 2 +- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/include/linux/sched/numa_balancing.h b/include/linux/sched/numa_balancing.h index 3988762efe15..06a9d35650f0 100644 --- a/include/linux/sched/numa_balancing.h +++ b/include/linux/sched/numa_balancing.h @@ -20,8 +20,8 @@ extern void task_numa_fault(int last_node, int node, int pages, int flags); extern pid_t task_numa_group_id(struct task_struct *p); extern void set_numabalancing_state(bool enabled); extern void task_numa_free(struct task_struct *p, bool final); -extern bool should_numa_migrate_memory(struct task_struct *p, struct page *page, - int src_nid, int dst_cpu); +bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, + int src_nid, int dst_cpu); #else static inline void task_numa_fault(int last_node, int node, int pages, int flags) @@ -38,7 +38,7 @@ static inline void task_numa_free(struct task_struct *p, bool final) { } static inline bool should_numa_migrate_memory(struct task_struct *p, - struct page *page, int src_nid, int dst_cpu) + struct folio *folio, int src_nid, int dst_cpu) { return true; } diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index a502e3255392..75c9a58632a4 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1722,12 +1722,12 @@ static bool pgdat_free_space_enough(struct pglist_data *pgdat) * The smaller the hint page fault latency, the higher the possibility * for the page to be hot. */ -static int numa_hint_fault_latency(struct page *page) +static int numa_hint_fault_latency(struct folio *folio) { int last_time, time; time = jiffies_to_msecs(jiffies); - last_time = xchg_page_access_time(page, time); + last_time = xchg_page_access_time(&folio->page, time); return (time - last_time) & PAGE_ACCESS_TIME_MASK; } @@ -1784,7 +1784,7 @@ static void numa_promotion_adjust_threshold(struct pglist_data *pgdat, } } -bool should_numa_migrate_memory(struct task_struct *p, struct page * page, +bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, int src_nid, int dst_cpu) { struct numa_group *ng = deref_curr_numa_group(p); @@ -1814,16 +1814,16 @@ bool should_numa_migrate_memory(struct task_struct *p, struct page * page, numa_promotion_adjust_threshold(pgdat, rate_limit, def_th); th = pgdat->nbp_threshold ? : def_th; - latency = numa_hint_fault_latency(page); + latency = numa_hint_fault_latency(folio); if (latency >= th) return false; return !numa_promotion_rate_limit(pgdat, rate_limit, - thp_nr_pages(page)); + folio_nr_pages(folio)); } this_cpupid = cpu_pid_to_cpupid(dst_cpu, current->pid); - last_cpupid = page_cpupid_xchg_last(page, this_cpupid); + last_cpupid = page_cpupid_xchg_last(&folio->page, this_cpupid); if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && !node_is_toptier(src_nid) && !cpupid_valid(last_cpupid)) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index ecf06ce3a5dd..f4e76a887db8 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2650,7 +2650,7 @@ int mpol_misplaced(struct folio *folio, struct vm_area_struct *vma, if (pol->flags & MPOL_F_MORON) { polnid = thisnid; - if (!should_numa_migrate_memory(current, &folio->page, curnid, + if (!should_numa_migrate_memory(current, folio, curnid, thiscpu)) goto out; }