From patchwork Sat Aug 17 08:49:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13767069 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43E1BC52D7D for ; Sat, 17 Aug 2024 08:54:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 57C546B017F; Sat, 17 Aug 2024 04:54:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4D56A6B0282; Sat, 17 Aug 2024 04:54:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 235706B017F; Sat, 17 Aug 2024 04:54:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id CCA986B0283 for ; Sat, 17 Aug 2024 04:54:07 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 7441A1A0537 for ; Sat, 17 Aug 2024 08:54:07 +0000 (UTC) X-FDA: 82461125334.03.77A9AF2 Received: from szxga07-in.huawei.com (szxga07-in.huawei.com [45.249.212.35]) by imf03.hostedemail.com (Postfix) with ESMTP id BC0DA20009 for ; Sat, 17 Aug 2024 08:54:03 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf03.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.35 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723884784; a=rsa-sha256; cv=none; b=p2E8QBaT0vg1OsvYU6Mm5PEPJox5HAmBs2OGyuKI43WggTA291iUa0A9iKiT3zyWOK7Wcf DQNavaatpw5LIASBB4sfiKbRpc/WzPv/YyP15rjxYjB4rTPtJ5lHor/u9xoWlbpMshAI04 Qiysg2ciota/kBQGRyBdjPTo882UbAY= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf03.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.35 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723884784; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r+c1gcN4OuvNzeGJthnLiu58/EiznLaJlYlDrS3BRXs=; b=O1aKuHgoWO3e5VESjyuUpxeRKKnjfjS2Qxeghqaox8YjkY2sxXL2BGa60cjbXRFr5GgR/t o5WqUCjrVB4OfygiEHTVgLpnwC/Pch1m4LXJsa64jq3FECURU4lGuIqNBoaFeheg9dI+XL 77I1dZsp2M5w5NZdPLRgSXQoDy6iwmA= Received: from mail.maildlp.com (unknown [172.19.162.112]) by szxga07-in.huawei.com (SkyGuard) with ESMTP id 4WmCFq4SYVz1S7c9; Sat, 17 Aug 2024 16:49:03 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id B5B42140154; Sat, 17 Aug 2024 16:53:59 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 17 Aug 2024 16:53:59 +0800 From: Kefeng Wang To: Andrew Morton CC: David Hildenbrand , Oscar Salvador , Miaohe Lin , Naoya Horiguchi , , Kefeng Wang Subject: [PATCH v2 1/5] mm: memory_hotplug: remove head variable in do_migrate_range() Date: Sat, 17 Aug 2024 16:49:37 +0800 Message-ID: <20240817084941.2375713-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240817084941.2375713-1-wangkefeng.wang@huawei.com> References: <20240817084941.2375713-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemf100008.china.huawei.com (7.185.36.138) X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: BC0DA20009 X-Stat-Signature: u6fgerxhj3yceqkfafk7g5xpas5y5er1 X-Rspam-User: X-HE-Tag: 1723884843-710625 X-HE-Meta: U2FsdGVkX1/UdT0kwHxx6yRCHDKCrGsVMsaMS4MY98tP9xSaX551dokk+DE9szI3MlzntdKSbl8+rem9hV7HIwstmbstLdBxc1N0hgoYjfOlsWUXervKpG/f3MOh6aEwJX4XbcVOAKDVb1qzbsG2OYqQaPgOujbu7r0WHDBsyj0pRMECDXjOICEz7+Gzn9/tH1xJY0zfA++Th7iyZm7D8WhXV8jtDCa4xoIyUBUUTrb2PWYzZEEX+dIHWbw9ItSXLxIn5WQkgwzScF0gv7DqMN4XhGheWkviBWvX0v2ruzU30IVY5mBfmm28Ar53zRkJS9JqNfympeOmdMuX4Xt/fhJpEfB/cbfUlJjoSa0d0uwQg93gKzicLO4wfl5OUhw2Q0P9jP1VkHRV8zHHjpK9ASG6UpcdzO4QHay0hA5famgOFcpRrnPYebM4t/R0NI6HLaJQy+XXIhKYeLYzavp7cRHkCyDZR/5xETJ9TP8IZV/s5ezgKNCGrUfUSaiuK3sflap61iWvjw0vMMhuPMa2E8DKmxjsRK1nEO27m2NZOiYSHSKcIYmFv17zlMsHCss6J6vl2zdI0vt+hJ0W5/kTk7D5RqzgaQWyaj2blPy0krq8AH/BLkCpLSKb3k7v1Bv8kEu1KPhH73ErlyFjPdRWIAvk7Ac+3qhSBBl7XVcnpoJ3KdMk61G6AMnj4ZdYChzMUaehSjjXuM8gI5hpA7GBe0y8rSn7Kd5UT2ml9l4As4ygu1D439oVUPfahZO4vV+mi7ZucJOZP6FzEnnRqXKS9vahNa7iWHA+2YCXFYPb3GcU3JeE9Cfrm6uQxvh8qdaKUDRo7EeQg3QVblL3Lj2WqMtvor43sNH+WKocdEe+c91ArjewOWTEPNKrRwXkWoNdFCCZDOA8wBd8gO40TOPoKymJHEoSw6FbN+RJpPf/WvoU5Qj1VgzX6fmlDyJAc4CamKzSDNrIjE/JJp7f5is X1yd4qel n/2lNba8rvbo5oYl/tH5lkw3kpuPBzg7g4bnqHI54N1afXmOPXihHrNopfEficGCauiDPKFQ2xg311Jm0JFJafNn6eQ8GEl02fqNKjMDqrR3oD32o0zYDy3m/1wpIEePLCgacrsV7zeBvfKQ/1/YHR8jSTBXXKrOKgaCPT72+tRoYBKhHc2EmwLxKHYOwiH8egaXw0URFtSmkxPYS08AZzo+J2PgmtfMAHm+Z6dsg6d5a/uqmtttZDpsmc7G71TnPWKtA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Directly use a folio for HugeTLB and THP when calculate the next pfn, then remove unused head variable. Signed-off-by: Kefeng Wang Reviewed-by: Miaohe Lin Acked-by: David Hildenbrand --- mm/memory_hotplug.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index df291f2e509d..dc19b0e28fbc 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1773,7 +1773,7 @@ static int scan_movable_pages(unsigned long start, unsigned long end, static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) { unsigned long pfn; - struct page *page, *head; + struct page *page; LIST_HEAD(source); static DEFINE_RATELIMIT_STATE(migrate_rs, DEFAULT_RATELIMIT_INTERVAL, DEFAULT_RATELIMIT_BURST); @@ -1786,14 +1786,20 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) continue; page = pfn_to_page(pfn); folio = page_folio(page); - head = &folio->page; - if (PageHuge(page)) { - pfn = page_to_pfn(head) + compound_nr(head) - 1; - isolate_hugetlb(folio, &source); - continue; - } else if (PageTransHuge(page)) - pfn = page_to_pfn(head) + thp_nr_pages(page) - 1; + /* + * The folio hold no reference or lock, it might be changed + * concurrently(eg, split), folio_nr_pages() may read garbage, + * but out loop could handle that as it revisits the split + * folio later. + */ + if (folio_test_large(folio)) { + pfn = folio_pfn(folio) + folio_nr_pages(folio) - 1; + if (folio_test_hugetlb(folio)) { + isolate_hugetlb(folio, &source); + continue; + } + } /* * HWPoison pages have elevated reference counts so the migration would From patchwork Sat Aug 17 08:49:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13767068 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC058C52D7F for ; Sat, 17 Aug 2024 08:54:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 23F036B027D; Sat, 17 Aug 2024 04:54:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1238F8D00CC; Sat, 17 Aug 2024 04:54:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C45206B027D; Sat, 17 Aug 2024 04:54:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 918CB6B017B for ; Sat, 17 Aug 2024 04:54:07 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 2BAB7A054E for ; Sat, 17 Aug 2024 08:54:07 +0000 (UTC) X-FDA: 82461125334.19.2439F93 Received: from szxga05-in.huawei.com (szxga05-in.huawei.com [45.249.212.191]) by imf08.hostedemail.com (Postfix) with ESMTP id E749316000F for ; Sat, 17 Aug 2024 08:54:03 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723884831; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pTr0PZ0fKJ44mNTHvsm/hg1N1X6FUexpWOV9fIQRECM=; b=VIAKrLUpbYKIsE2RHOdiDuEvHEBq7NzaJxv9a/yO0bTyXxWyI6n2aaNA4I5XP574jM1pXG 950iclLvAXd0RW6eHFw6StTkihWwFRfA+Oa2H0eaYTcjun332IhQWvfJImZOjOHKcbYvCg vDAU/ZtIwUX2NGNNZx2cZd+Af46Gt/Y= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723884831; a=rsa-sha256; cv=none; b=V1GRexcQtiqH4cI8luqE6KHR+KTT87BLyXVTxSDsVRzPqgT3/1/M9kKjQewe3jBOb2Xez6 uI91JI9jb/ruhlOHDkoFxHtwBjm5Kori3Vih8BWNauZQADeMJxBPOBSusnKrmikDa2FyLw ai6/5lWwYVdwAneMS8/YI7RmkabQiTI= Received: from mail.maildlp.com (unknown [172.19.88.234]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4WmCHw4MRjz1HGHc; Sat, 17 Aug 2024 16:50:52 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id 353F11401E9; Sat, 17 Aug 2024 16:54:00 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 17 Aug 2024 16:53:59 +0800 From: Kefeng Wang To: Andrew Morton CC: David Hildenbrand , Oscar Salvador , Miaohe Lin , Naoya Horiguchi , , Kefeng Wang Subject: [PATCH v2 2/5] mm: memory-failure: add unmap_posioned_folio() Date: Sat, 17 Aug 2024 16:49:38 +0800 Message-ID: <20240817084941.2375713-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240817084941.2375713-1-wangkefeng.wang@huawei.com> References: <20240817084941.2375713-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemf100008.china.huawei.com (7.185.36.138) X-Rspam-User: X-Stat-Signature: 61u36bf3ez439b11q65njyehs3k9ym3d X-Rspamd-Queue-Id: E749316000F X-Rspamd-Server: rspam11 X-HE-Tag: 1723884843-900667 X-HE-Meta: U2FsdGVkX1/RRDQnAhHtUOQp6VNSScWZ/BNQMpfDXnvRa5XOYGZsQM1TSevbUF1djU0qtazBXO1U101OE5b9nOWagWLo0pZto8dIg3CppXm9ENxTgA0DM9zNM8JPXKoq3CkY/PyzMsJe4BSymL99RH3jSNNo64XEzv7ip7zh2MSEQ84OF1IOu/ow8yM27Zwc7t3V3PRPFWWk81bFCgFaAIqBlvJ/G7KyqqZie9uxgu/wYrsd+8ysHWbnxbk4Qcu0AkknBw8UlIDLZVpT9hSa0QWtmEB8gbBD+F8iuZvJiZRx88jPoVl49vwO34wBdtnEn8QnT+nU182NYk8MArSd6Bw1hpXDC114vY3ZUMjs9Qg6fM4VV0F7UUv+Qbycmctg2xwK47f7Pmoeej4LKhTwH7H/GIDvQB/VTTMbKcfhDYeBl4oaPcVhVySvjGmyp4aNGJG1Pmrm5I7FazFjqyQ1VvDqkjmyZG5xVyn1hTHaqgQG0xmAmevrkUvwTkhDNDwpI4bKeZC27kT8dGnWJXHXmHO9IEfUaWWT06GXXtmeDyDEhb2WPKgoOPjShfWJDyklkqeYmU15ZEZ3uw8jmh/O8DdHdMKSO9OJORMAl8/gKbGvZWCpYh/Gf6iy1Mh5O5IGVVa07S7RaC3YHLvWv288ynlbkJvYkfeCWcWVJiRKg44xjrGovlgI93B3HIwwiffO7VdcMNumQ/jSJ1pTqjmtjMRPcmPQeFjaPXIAh1k7bcE2h5Qv7Sjx9dPQihd6bUtjOwZSSxHsjS0RTWOb/lCmbktwPRIJ03JjNp5DzBwuptv2idR+NZQVVqDxS1oLrBmlZNa2yPTpg0XytRQhVdAkNMZZ3Z3/yRmRc5TcaqCC+aSL+heHpIYBY1KTxRFYnsFgBRDXXWDIhXQ4wuRbtp/TEUs4comB18CgX2aMc+8tf7F0+XrxuvvAaW50tYF/v29qMZRBUNhnyaR6XRZrsN9 NEWQODeO voTPj4nksFE4O+qRUhz5sn35LwFl7Extw5cxFjsJHZ6TyyPeUnzcB/ERZivrVGaWFZjnUferSA4n5IcbNkDWbNtkCUbeFIHwFcG0VhJvNUge1dUQZEp+Vkrpa/U3TD9prrH32BVid1c8yKSzCK6iZuBAF5Ne/dSYwyGV0Rl2Jlqs1AUtGfzaaJQ7x3KwbBMyhLL3yg7lw0vSRTkDZN/8p4o8llVIHWe/iORrGM4nmaVOBEduUXitx8s2DDda7XlJMDb6I/PS2d3nAttdIWCVdzp6iyt73oyemoZXj X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add unmap_posioned_folio() helper which will be reused by do_migrate_range() from memory hotplug soon. Acked-by: David Hildenbrand Signed-off-by: Kefeng Wang --- mm/internal.h | 9 +++++++++ mm/memory-failure.c | 43 ++++++++++++++++++++++++++----------------- 2 files changed, 35 insertions(+), 17 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index d7aac802efa5..74490b8ac63d 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1045,6 +1045,8 @@ static inline int find_next_best_node(int node, nodemask_t *used_node_mask) /* * mm/memory-failure.c */ +#ifdef CONFIG_MEMORY_FAILURE +int unmap_posioned_folio(struct folio *folio, enum ttu_flags ttu); void shake_folio(struct folio *folio); extern int hwpoison_filter(struct page *p); @@ -1065,6 +1067,13 @@ void add_to_kill_ksm(struct task_struct *tsk, struct page *p, unsigned long ksm_addr); unsigned long page_mapped_in_vma(struct page *page, struct vm_area_struct *vma); +#else +static inline int unmap_posioned_folio(struct folio *folio, enum ttu_flags ttu) +{ + return 0; +} +#endif + extern unsigned long __must_check vm_mmap_pgoff(struct file *, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 353254537b54..93848330de1f 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1554,6 +1554,30 @@ static int get_hwpoison_page(struct page *p, unsigned long flags) return ret; } +int unmap_posioned_folio(struct folio *folio, enum ttu_flags ttu) +{ + if (folio_test_hugetlb(folio) && !folio_test_anon(folio)) { + struct address_space *mapping; + /* + * For hugetlb pages in shared mappings, try_to_unmap + * could potentially call huge_pmd_unshare. Because of + * this, take semaphore in write mode here and set + * TTU_RMAP_LOCKED to indicate we have taken the lock + * at this higher level. + */ + mapping = hugetlb_folio_mapping_lock_write(folio); + if (!mapping) + return -EAGAIN; + + try_to_unmap(folio, ttu|TTU_RMAP_LOCKED); + i_mmap_unlock_write(mapping); + } else { + try_to_unmap(folio, ttu); + } + + return 0; +} + /* * Do all that is necessary to remove user space mappings. Unmap * the pages and send SIGBUS to the processes if the data was dirty. @@ -1615,23 +1639,8 @@ static bool hwpoison_user_mappings(struct folio *folio, struct page *p, */ collect_procs(folio, p, &tokill, flags & MF_ACTION_REQUIRED); - if (folio_test_hugetlb(folio) && !folio_test_anon(folio)) { - /* - * For hugetlb pages in shared mappings, try_to_unmap - * could potentially call huge_pmd_unshare. Because of - * this, take semaphore in write mode here and set - * TTU_RMAP_LOCKED to indicate we have taken the lock - * at this higher level. - */ - mapping = hugetlb_folio_mapping_lock_write(folio); - if (mapping) { - try_to_unmap(folio, ttu|TTU_RMAP_LOCKED); - i_mmap_unlock_write(mapping); - } else - pr_info("%#lx: could not lock mapping for mapped huge page\n", pfn); - } else { - try_to_unmap(folio, ttu); - } + if (unmap_posioned_folio(folio, ttu)) + pr_info("%#lx: could not lock mapping for mapped huge page\n", pfn); unmap_success = !folio_mapped(folio); if (!unmap_success) From patchwork Sat Aug 17 08:49:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13767071 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B3CEC52D7F for ; Sat, 17 Aug 2024 08:54:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 235516B0290; Sat, 17 Aug 2024 04:54:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1BDF86B0414; Sat, 17 Aug 2024 04:54:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 011786B02A3; Sat, 17 Aug 2024 04:54:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id CE5DB6B0290 for ; Sat, 17 Aug 2024 04:54:08 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7EF4FC0545 for ; Sat, 17 Aug 2024 08:54:08 +0000 (UTC) X-FDA: 82461125376.20.8DA8C1D Received: from szxga07-in.huawei.com (szxga07-in.huawei.com [45.249.212.35]) by imf21.hostedemail.com (Postfix) with ESMTP id 045BA1C0002 for ; Sat, 17 Aug 2024 08:54:05 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.35 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723884771; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S1fVY/WKJzMRDG5Xb+nNm7fDXazsjCAm5noFkzuJuxg=; b=U1LM3eDV5iXpJPTIjtoZgDAlAaSaqbm+KlusW7Wsm5bGlJsXmjr7UQfdiElpqG/nMWlDw5 oSVikgXquzyDleN+EqcKXuSDZSUK4tHZ+rY+LUxVWlRec1FuqDUwFi+3upBtFI9d5RR39I iE7KB4MVs518Ex/qaeCxvJpsuVJm3z0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723884771; a=rsa-sha256; cv=none; b=0aUU1eeYomIwEzM1hruzeHiu72YB0/YUcNKO/uWNy6inA6zovO5ObtCfxK/kDUoHrj9WTI wzui7VA54f3RQjY/66/VHKkob2gaCpZXCXnk4y9bcH075mKj0pkFM0jUIcn4hRv1VDpwKk gOl7Nxf+XWVuZxwqgk43gy591yD3w+c= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.35 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from mail.maildlp.com (unknown [172.19.88.214]) by szxga07-in.huawei.com (SkyGuard) with ESMTP id 4WmCFr3m8yz1S7qQ; Sat, 17 Aug 2024 16:49:04 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id 9D7ED1A016C; Sat, 17 Aug 2024 16:54:00 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 17 Aug 2024 16:54:00 +0800 From: Kefeng Wang To: Andrew Morton CC: David Hildenbrand , Oscar Salvador , Miaohe Lin , Naoya Horiguchi , , Kefeng Wang Subject: [PATCH v2 3/5] mm: memory_hotplug: check hwpoisoned page firstly in do_migrate_range() Date: Sat, 17 Aug 2024 16:49:39 +0800 Message-ID: <20240817084941.2375713-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240817084941.2375713-1-wangkefeng.wang@huawei.com> References: <20240817084941.2375713-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemf100008.china.huawei.com (7.185.36.138) X-Stat-Signature: 11h1qomuqcfiahj714usfk8ob3qszoan X-Rspamd-Queue-Id: 045BA1C0002 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1723884845-900839 X-HE-Meta: U2FsdGVkX18WfqBqn6JwGfJ+nYDc3NILR4n6eNhvk5GqA13hkAKkvYl2vFChiU84uIjkFoorHM4A+iZ+kZ62SwzQsf4bhCM58UiKrBWkolS3fQWEy4o1bJdF7NWeOVSamWkfbnkFyEp+lKsL8+iuD6HDVCSDBQzjtoRBXbTiC0sgcrXEA545WB4uGPRNeBb7Wsd/NOGjSb1P+SdUhd8lRZ0vXROhqxI6vEplSGq5LvQsnZz+KzwigbbZ34QK+SaxV6Ho2kRdUbl6+NJn8TsIiqeyiJcHiMA94yMau+9oa5sKvD3f3OtB8kjswTNb/iNSD2uwkO1nfv56QqIwrLc1deHlZoadOwfUjgJP2j29r7x2tNuW9jdKk5V65plYFerE70diYRg3fN5bBiFI0c1HaXNIrQMGngBkvQ9g3X+QeOlFG6AM2BEbVXTNYWHR8tBlP6Wd10oMYxiRMQI8tsgBmmP0CdEkBJtKdLExSbn6BlfAYjXSriwaQOWgfLvvB8DxU7EI5c2umzi31ZxV7Yd0UHCW0zrMtGOWVA4nJiXLApG0Sca532dP/XDFtauiUOFZdvxFTsWt3V0C7H0VDBDD79SER1b6u7kGIREjlItkUM9FCPHbGSNQJc2VDwdn52X8KhXV/1hdPlKDTIvjILnUga6uNe1RFDHiQh6izbjKuhvKPb74+5lHhV4npSkKQihW8euCMmlEJDHYc1Yoi2288lvUUjiCwdddjbrKxN++w/e651RXa1mPp/nf2zXS94iXNiEHRydmhVavMxSrTbf6CAfcAYRyAIRWzGojFzoVzakVTNYzJOyST9WcBa/L6WCfLs7ieWbiZM07blkT+DR011ciyLsl8pjtukdIcQNHVFYZbFhEs32oNbJfWJaI6VwIPdYl4OFTWgg2IRh9YdfKT+pNdoJHBRQ+5DOf9WZb96R3bYff9oJ1gBA3o0Z5C1tk4zGOFShOJoROx57eBvQ vpT3fCuH 6KwoJbkmWnLffqxLq9N5WXgZ0QXgHVoVG+MF9C4+NiK/lTF92pOVyfWFHuTHUaqVMGmDsC6SiLnWKBgbRIWMDe9+v9dYK64XbpSYv55GS6pCBkkGDV2L2pC7Y4lERTcNF0iMWiLaZXbbCAuzAJf9sgWmsQUIaKHXax+mPaoVWYKfBEVyxlT1cvM9ZTsLDtzlsWnTEo3faq6uGRMqTEloDUeWQKhj6jI2P8Sj/EFsgOVFXu3H8/eOpuXQEndfs4YWLbGl4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The commit b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined") don't handle the hugetlb pages, the endless loop still occur if offline a hwpoison hugetlb, luckly, with the commit e591ef7d96d6 ("mm,hwpoison,hugetlb,memory_hotplug: hotremove memory section with hwpoisoned hugepage") section with hwpoisoned hugepage"), the HPageMigratable of hugetlb page will be clear, and the hwpoison hugetlb page will be skipped in scan_movable_pages(), so the endless loop issue is fixed. However if the HPageMigratable() check passed(without reference and lock), the hugetlb page may be hwpoisoned, it won't cause issue since the hwpoisoned page will be handled correctly in the next movable pages scan loop, and it will be isolated in do_migrate_range() but fails to migrate. In order to avoid the unnecessary isolation and unify all hwpoisoned page handling, let's unconditionally check hwpoison firstly, and if it is a hwpoisoned hugetlb page, try to unmap it as the catch all safety net like normal page does. Signed-off-by: Kefeng Wang Acked-by: David Hildenbrand --- mm/memory_hotplug.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index dc19b0e28fbc..02a0d4fbc3fe 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1793,13 +1793,8 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) * but out loop could handle that as it revisits the split * folio later. */ - if (folio_test_large(folio)) { + if (folio_test_large(folio)) pfn = folio_pfn(folio) + folio_nr_pages(folio) - 1; - if (folio_test_hugetlb(folio)) { - isolate_hugetlb(folio, &source); - continue; - } - } /* * HWPoison pages have elevated reference counts so the migration would @@ -1808,11 +1803,17 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) * (e.g. current hwpoison implementation doesn't unmap KSM pages but keep * the unmap as the catch all safety net). */ - if (PageHWPoison(page)) { + if (folio_test_hwpoison(folio) || + (folio_test_large(folio) && folio_test_has_hwpoisoned(folio))) { if (WARN_ON(folio_test_lru(folio))) folio_isolate_lru(folio); if (folio_mapped(folio)) - try_to_unmap(folio, TTU_IGNORE_MLOCK); + unmap_posioned_folio(folio, TTU_IGNORE_MLOCK); + continue; + } + + if (folio_test_hugetlb(folio)) { + isolate_hugetlb(folio, &source); continue; } From patchwork Sat Aug 17 08:49:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13767070 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66185C531DE for ; Sat, 17 Aug 2024 08:54:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8C9E86B0282; Sat, 17 Aug 2024 04:54:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 78AEE6B0290; Sat, 17 Aug 2024 04:54:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 36E236B0283; Sat, 17 Aug 2024 04:54:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C901D6B017B for ; Sat, 17 Aug 2024 04:54:07 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 62F641402C0 for ; Sat, 17 Aug 2024 08:54:07 +0000 (UTC) X-FDA: 82461125334.08.16D07F6 Received: from szxga05-in.huawei.com (szxga05-in.huawei.com [45.249.212.191]) by imf23.hostedemail.com (Postfix) with ESMTP id BFC2B14000B for ; Sat, 17 Aug 2024 08:54:04 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723884770; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OGnlXOuzeF9DWayfTGT5N92iiXXehz6xI1tDTpAiTb4=; b=EzRnE/slNuTs/sJ4Gxe/tWrCN2U+xQZUGu4ogiianbv2BEQHjZKNTRKAmhtNgON2cSmODU xpMjucJMrDJhabxbBUoWkiQXKR05PGs2uwea5Z/A1+608YNfOcj1pjmroFFNGuB4kHawx0 tlIH4cpdWGrwk7N4w9QUWNSS+V/a0gM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723884770; a=rsa-sha256; cv=none; b=yIvEegDYKqdPof1sU0n20dzaa8Vcc5vrqeboVVgOlm0d48iNqkWlDWzbi6kPcDQXC6/sjC 2ho1MPVddQtTQCCv8vLlIWFDquTXGTm26Jo3G8OdYxkgVNXfMW8WrxeQi3RfToonzuU957 lENZx0aCAkePrNZCm2V7eK9flnXT0X0= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from mail.maildlp.com (unknown [172.19.163.44]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4WmCHx3Vxkz1HGLJ; Sat, 17 Aug 2024 16:50:53 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id 17C54140138; Sat, 17 Aug 2024 16:54:01 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 17 Aug 2024 16:54:00 +0800 From: Kefeng Wang To: Andrew Morton CC: David Hildenbrand , Oscar Salvador , Miaohe Lin , Naoya Horiguchi , , Kefeng Wang Subject: [PATCH v2 4/5] mm: migrate: add isolate_folio_to_list() Date: Sat, 17 Aug 2024 16:49:40 +0800 Message-ID: <20240817084941.2375713-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240817084941.2375713-1-wangkefeng.wang@huawei.com> References: <20240817084941.2375713-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemf100008.china.huawei.com (7.185.36.138) X-Stat-Signature: dwkhnxcqs4c6s8dobe7z3g7sa9mi5d59 X-Rspamd-Queue-Id: BFC2B14000B X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1723884844-49500 X-HE-Meta: U2FsdGVkX19r204XQs6ZcXN5CfQupf7CJ3QTtQKOIEtZLZoG27N9rl8oui8RfcvhBDiItNVCxcxPu9bYDtXfV6M56lTSmn49RJgnq55BP4e18d/4xubC8+VMc82nu4UrN/aBtVeTrRPsKvUp9XJxPc6iyiHGxnLEpYpd0Tp3JZjcS5ITzzd4xvPnj9w/FiPcfc8u8t6YKagsHx9kyEU/O796RDBp/PYmCTKp5RhVxHVAR844JQM20ZRfsG6IP8lpGPm1XeXKsb2EoGZn6QagIO3Xxb245FnGoAZDSDFlaBIszX0RYDClxkiRo64Ipozf4WIFiwUri/rn7s72Gt2Zsb1io/trSjqIzfK4HWe4NOwaYH8HFjGdBbkMd3cc9+F7c/fgNZSzjvHta8cLiK6ihA5NP+tchT7i4xX2Y8G2BEPVCETmgp6jg5FYay86QhGhIhjnnorZLcZ/Fq4BsvFJ+n6bxZZ1gvPPe6aGQHX1MXYZJDVyy6oirXafNrWZ2E+LxnZL+LgmfVMRn9hZ4r+rB+7DTa0pH+qbLiBaooaMyXQhAMpsSvQ4kGUHqEHTBwM7sYQRZhewYDMAeuf5YbkJa3rnbrazeP9sMuRz9nveBr1G6dN6rXyq1kebBKuWlcbq6uMkuGKKs6mMp/w1aEqabvV1FqT1lOcC0M4ZGIJ4QakK0EbX2e7ZMEfgJX14X4DQocIHtHqjvhDAIkyYNSGHadpAovrJ72SvYu/P2bjNKyaXpLHoY+cYJlf7M1qRPVyQeXXhQAsXlMFaRIj07EuNzc2UOo86wsxhs0AxU3t2kNCKawEZKsGFt8fA68YJ5ec2jN2SniftN/V2wDVyWXWhrC4gmdr5XNQD57eTqPCXvBA7Humu1pOmqiQNyoeU8UJWdAZ7QFME8sO66PVjDJAisUZe/UZM8I8J36dkucEKdfZIaKJdujaTStrNxfOAyk8DM3HjsITr9aCAsysvv5J 3ONonnKO 8ixEl3s0qJk5uL4GQk2d6TqIdbj82q/be5D4BEIa7nluUAuWW8Fv1qnh0kNMOUPILzFmgEulRj8+S7DlvD41d5mUTMoe6iGzzJRFf7NO/7s4d9kx6dvZ55JYlabctA7aCN5Nsba7NdVzyS4ZAeFEKuvXktbvc8cFUbGJlW/rc8UA+gPq4fBM6mKFn1wIst0v1/h3Pj4uRawY/78JDiTemcrx0fzk7YzUluC+XOTPq05F2bI3RLKheMIzAwwz39zpo4wsII8P2CzKP4jv20K1ZjjhkW0g88gDKnvu+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add isolate_folio_to_list() helper to try to isolate HugeTLB, no-LRU movable and LRU folios to a list, which will be reused by do_migrate_range() from memory hotplug soon, also drop the mf_isolate_folio() since we could directly use new helper in the soft_offline_in_use_page(). Acked-by: David Hildenbrand Signed-off-by: Kefeng Wang Acked-by: Miaohe Lin Tested-by: Miaohe Lin --- include/linux/migrate.h | 3 +++ mm/memory-failure.c | 46 ++++++++++------------------------------- mm/migrate.c | 27 ++++++++++++++++++++++++ 3 files changed, 41 insertions(+), 35 deletions(-) diff --git a/include/linux/migrate.h b/include/linux/migrate.h index 644be30b69c8..002e49b2ebd9 100644 --- a/include/linux/migrate.h +++ b/include/linux/migrate.h @@ -70,6 +70,7 @@ int migrate_pages(struct list_head *l, new_folio_t new, free_folio_t free, unsigned int *ret_succeeded); struct folio *alloc_migration_target(struct folio *src, unsigned long private); bool isolate_movable_page(struct page *page, isolate_mode_t mode); +bool isolate_folio_to_list(struct folio *folio, struct list_head *list); int migrate_huge_page_move_mapping(struct address_space *mapping, struct folio *dst, struct folio *src); @@ -91,6 +92,8 @@ static inline struct folio *alloc_migration_target(struct folio *src, { return NULL; } static inline bool isolate_movable_page(struct page *page, isolate_mode_t mode) { return false; } +static inline bool isolate_folio_to_list(struct folio *folio, struct list_head *list) + { return false; } static inline int migrate_huge_page_move_mapping(struct address_space *mapping, struct folio *dst, struct folio *src) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 93848330de1f..d8298017bd99 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -2659,40 +2659,6 @@ EXPORT_SYMBOL(unpoison_memory); #undef pr_fmt #define pr_fmt(fmt) "Soft offline: " fmt -static bool mf_isolate_folio(struct folio *folio, struct list_head *pagelist) -{ - bool isolated = false; - - if (folio_test_hugetlb(folio)) { - isolated = isolate_hugetlb(folio, pagelist); - } else { - bool lru = !__folio_test_movable(folio); - - if (lru) - isolated = folio_isolate_lru(folio); - else - isolated = isolate_movable_page(&folio->page, - ISOLATE_UNEVICTABLE); - - if (isolated) { - list_add(&folio->lru, pagelist); - if (lru) - node_stat_add_folio(folio, NR_ISOLATED_ANON + - folio_is_file_lru(folio)); - } - } - - /* - * If we succeed to isolate the folio, we grabbed another refcount on - * the folio, so we can safely drop the one we got from get_any_page(). - * If we failed to isolate the folio, it means that we cannot go further - * and we will return an error, so drop the reference we got from - * get_any_page() as well. - */ - folio_put(folio); - return isolated; -} - /* * soft_offline_in_use_page handles hugetlb-pages and non-hugetlb pages. * If the page is a non-dirty unmapped page-cache page, it simply invalidates. @@ -2744,7 +2710,7 @@ static int soft_offline_in_use_page(struct page *page) return 0; } - if (mf_isolate_folio(folio, &pagelist)) { + if (isolate_folio_to_list(folio, &pagelist)) { ret = migrate_pages(&pagelist, alloc_migration_target, NULL, (unsigned long)&mtc, MIGRATE_SYNC, MR_MEMORY_FAILURE, NULL); if (!ret) { @@ -2766,6 +2732,16 @@ static int soft_offline_in_use_page(struct page *page) pfn, msg_page[huge], page_count(page), &page->flags); ret = -EBUSY; } + + /* + * If we succeed to isolate the folio, we grabbed another refcount on + * the folio, so we can safely drop the one we got from get_any_page(). + * If we failed to isolate the folio, it means that we cannot go further + * and we will return an error, so drop the reference we got from + * get_any_page() as well. + */ + folio_put(folio); + return ret; } diff --git a/mm/migrate.c b/mm/migrate.c index dbfa910ec24b..53f8429a8ebe 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -178,6 +178,33 @@ void putback_movable_pages(struct list_head *l) } } +/* Must be called with an elevated refcount on the non-hugetlb folio */ +bool isolate_folio_to_list(struct folio *folio, struct list_head *list) +{ + bool isolated = false; + + if (folio_test_hugetlb(folio)) { + isolated = isolate_hugetlb(folio, list); + } else { + bool lru = !__folio_test_movable(folio); + + if (lru) + isolated = folio_isolate_lru(folio); + else + isolated = isolate_movable_page(&folio->page, + ISOLATE_UNEVICTABLE); + + if (isolated) { + list_add(&folio->lru, list); + if (lru) + node_stat_add_folio(folio, NR_ISOLATED_ANON + + folio_is_file_lru(folio)); + } + } + + return isolated; +} + static bool try_to_map_unused_to_zeropage(struct page_vma_mapped_walk *pvmw, struct folio *folio, unsigned long idx) From patchwork Sat Aug 17 08:49:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13767072 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1081C52D7D for ; Sat, 17 Aug 2024 08:54:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 89DBB6B02A0; Sat, 17 Aug 2024 04:54:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 826F26B041A; Sat, 17 Aug 2024 04:54:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6791D6B041B; Sat, 17 Aug 2024 04:54:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 166A36B02A0 for ; Sat, 17 Aug 2024 04:54:09 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id CE6EC1A051D for ; Sat, 17 Aug 2024 08:54:08 +0000 (UTC) X-FDA: 82461125376.09.07D3998 Received: from szxga06-in.huawei.com (szxga06-in.huawei.com [45.249.212.32]) by imf25.hostedemail.com (Postfix) with ESMTP id 40865A0002 for ; Sat, 17 Aug 2024 08:54:05 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.32 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723884810; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k4PijjS54tHclFDjnkXcHqSsxQhhlIxc0hCoJeNbyJA=; b=a2LXL7h4Rjf4T1pXTtKBQkzbdB7PPrLHxzkFq2d0PbpwRYhVg+TthFOlpepCxFZbSnlQPc cwyC1Z4CRYw7ioH0YofsYloYxEt8VEm/asdwenQr92Bi9deadRcIirilD22KJdO07OLjte 7hcgWS8lpOWnNqN7sUjkmku3V7m3huo= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.32 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723884810; a=rsa-sha256; cv=none; b=UjQ/52Crrn0MG32xr/jfDtZzKLMI1lfctc057x+/3mGx3I59bQRTHBD6n/AQQk0hyqlR54 GOWsL16G5iR2LqdVy9hJtm2thA3jOVI4XplnFKaFdevoEhQ8OdTrf/2NkSkIb0R2QoYqQn bK/jsf2UCsrlDJnhRZx3Io6GvbYccMQ= Received: from mail.maildlp.com (unknown [172.19.163.44]) by szxga06-in.huawei.com (SkyGuard) with ESMTP id 4WmCKN5rmrz1xvDn; Sat, 17 Aug 2024 16:52:08 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id 86D1A140138; Sat, 17 Aug 2024 16:54:01 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sat, 17 Aug 2024 16:54:01 +0800 From: Kefeng Wang To: Andrew Morton CC: David Hildenbrand , Oscar Salvador , Miaohe Lin , Naoya Horiguchi , , Kefeng Wang Subject: [PATCH v2 5/5] mm: memory_hotplug: unify Huge/LRU/non-LRU movable folio isolation Date: Sat, 17 Aug 2024 16:49:41 +0800 Message-ID: <20240817084941.2375713-6-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240817084941.2375713-1-wangkefeng.wang@huawei.com> References: <20240817084941.2375713-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemf100008.china.huawei.com (7.185.36.138) X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 40865A0002 X-Stat-Signature: tz1atqkhx8b49c875tw3c69gifj7wjtn X-HE-Tag: 1723884845-987712 X-HE-Meta: U2FsdGVkX18oCP5GSNyi7ovodzr12r88ZxejA4yAZcqchDFIPdfsVw0s6lYHCQbJDnaoqZGVj/ckptQXuvKZA+AXPYhCBpJT47OiuumYaQAMZzA9mrPszsOzlCFl7YOQwDLIPABnrtMuNktnAKosd5EFA0p2TScQLAKCQndmAGMTxHoABWlI2k7mMLCa5chEsHtzZ+GXZsofK97SYBMoY08d0WHAaZQiJSbq8xlhq8jNPQMcRwv3Z4cQseUBbQt8jNq6OJkxSo/HUuwX9kYGa8T2meyhE8Dua2khVNPF85Tz55MSz0inM7mwnGDyU/Kc/dsjHB9KBdfpgSqlyEIPAew07f33+5jQptiEkajcLxGPuIMOkBMFGH+igyiDJTxLhlU4dHsRq5p/L7aNTr62PiKfaKT4+0QS4AeJyeAZpXVjh7jdO+4yvrVaDABkzDHCgN0JVy212KriChUnpexaNpX2L9ZzXRNrHMrRXJm33dYk7CPv7nWf9LR/7IDFVtAYLR/7o/m8416d4Eb7ipu6fre/L1R7tffKOO/F6Mtup/KOWJl9WcNHI6Rs2d3fAgR77+0tbL9jDAbAttffiyDZJ5bZzhDTXk7aAeX/Lf+jgQb1kMhBboQII03k+40+HNe4NzAki4hTtvD/14mbmuzfArYtNdaMxPprIGRr83n9mSYLgyy0ZS4qkE7OfrXVMI71DUOofwNl882xDCDN2qmIiBhPcIvFLgz8eTb8vRK5/+mbvRQVRJBHefdSmCT6cMyjDfGa39oIcTI/2hFMITXaQcBNLp0TY7YNrWsRed7GEUkl+zh1/h0efOUzXbGaAMvNSA2kthBLaTzzde65aXqUg3psjOKWwV+CpKbspZEPxocoBA+4RycqAJTZh82MwlDrIa4kW8kMsB97MkHrboQkZxqqe0xARWkL4FGeZfaH1QTV0OLUEmbSdrULk+wNVVydODNqddRST7eF2dZok1t SHF1KvGH ioPiJfuhlZmtYc1Wi+BE+Hl3ONMnfX+ff5Xc3ryoMxpfGwNxHHkzgn+CvSEEYRroiUUMCbk08FNB4ERnst8aJQGjKRoz7Rj5ghP0kDmOehzPexWH+CNkw63ENUafVGRNW3o1RkiIKnMPUHnPRk3ttmfOWCPuWvUWvYDS4VJ/AAcW8G1yAANgErPPQR/nTc2+/NXGtRxD8OpjetDz657XbCSQQzASHjhlFk2TatZ7gDDntNGKPvyIpaqNERLRPgL7njxa2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use the isolate_folio_to_list() to unify hugetlb/LRU/non-LRU folio isolation, which cleanup code a bit and save a few calls to compound_head(). Signed-off-by: Kefeng Wang --- mm/memory_hotplug.c | 45 +++++++++++++++++---------------------------- 1 file changed, 17 insertions(+), 28 deletions(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 02a0d4fbc3fe..cc9c16db2f8c 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1773,14 +1773,14 @@ static int scan_movable_pages(unsigned long start, unsigned long end, static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) { unsigned long pfn; - struct page *page; LIST_HEAD(source); + struct folio *folio; static DEFINE_RATELIMIT_STATE(migrate_rs, DEFAULT_RATELIMIT_INTERVAL, DEFAULT_RATELIMIT_BURST); for (pfn = start_pfn; pfn < end_pfn; pfn++) { - struct folio *folio; - bool isolated; + struct page *page; + bool huge; if (!pfn_valid(pfn)) continue; @@ -1812,34 +1812,22 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) continue; } - if (folio_test_hugetlb(folio)) { - isolate_hugetlb(folio, &source); - continue; + huge = folio_test_hugetlb(folio); + if (!huge) { + folio = folio_get_nontail_page(page); + if (!folio) + continue; } - if (!get_page_unless_zero(page)) - continue; - /* - * We can skip free pages. And we can deal with pages on - * LRU and non-lru movable pages. - */ - if (PageLRU(page)) - isolated = isolate_lru_page(page); - else - isolated = isolate_movable_page(page, ISOLATE_UNEVICTABLE); - if (isolated) { - list_add_tail(&page->lru, &source); - if (!__PageMovable(page)) - inc_node_page_state(page, NR_ISOLATED_ANON + - page_is_file_lru(page)); - - } else { + if (!isolate_folio_to_list(folio, &source)) { if (__ratelimit(&migrate_rs)) { pr_warn("failed to isolate pfn %lx\n", pfn); dump_page(page, "isolation failed"); } } - put_page(page); + + if (!huge) + folio_put(folio); } if (!list_empty(&source)) { nodemask_t nmask = node_states[N_MEMORY]; @@ -1854,7 +1842,7 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) * We have checked that migration range is on a single zone so * we can use the nid of the first page to all the others. */ - mtc.nid = page_to_nid(list_first_entry(&source, struct page, lru)); + mtc.nid = folio_nid(list_first_entry(&source, struct folio, lru)); /* * try to allocate from a different node but reuse this node @@ -1867,11 +1855,12 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) ret = migrate_pages(&source, alloc_migration_target, NULL, (unsigned long)&mtc, MIGRATE_SYNC, MR_MEMORY_HOTPLUG, NULL); if (ret) { - list_for_each_entry(page, &source, lru) { + list_for_each_entry(folio, &source, lru) { if (__ratelimit(&migrate_rs)) { pr_warn("migrating pfn %lx failed ret:%d\n", - page_to_pfn(page), ret); - dump_page(page, "migration failure"); + folio_pfn(folio), ret); + dump_page(&folio->page, + "migration failure"); } } putback_movable_pages(&source);