From patchwork Fri Jun 14 22:15:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shivank Garg X-Patchwork-Id: 13699148 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 297BBC27C6E for ; Fri, 14 Jun 2024 22:21:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3B6426B0163; Fri, 14 Jun 2024 18:16:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 365B26B016A; Fri, 14 Jun 2024 18:16:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1DEAD6B016B; Fri, 14 Jun 2024 18:16:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id EA4E06B0163 for ; Fri, 14 Jun 2024 18:16:35 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 695B9C0998 for ; Fri, 14 Jun 2024 22:16:35 +0000 (UTC) X-FDA: 82230904350.11.DB2B6B0 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2057.outbound.protection.outlook.com [40.107.102.57]) by imf28.hostedemail.com (Postfix) with ESMTP id 99A77C0004 for ; Fri, 14 Jun 2024 22:16:31 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b="lCA//9EI"; spf=pass (imf28.hostedemail.com: domain of shivankg@amd.com designates 40.107.102.57 as permitted sender) smtp.mailfrom=shivankg@amd.com; dmarc=pass (policy=quarantine) header.from=amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718403389; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UFtwicY4CyhIt3hTfES90NWrdOR1HL5qCnm9jVePVUs=; b=1bH6nECpZ6GWyNophQCv/cN7Zxda0w5GmlYQH068SH0aZR93bFLOz0SZLL5yBxATtLnc7c nq4hoqzvBxCMtFp7G2D5gyDdK7AnKs+c4DBfKN8SzmKcV8wvE4c3mT5LuudTZJvzmZv7GP E8K2HcwO4TCxqLVo8DKQt9OgCvEAhzI= ARC-Authentication-Results: i=2; imf28.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b="lCA//9EI"; spf=pass (imf28.hostedemail.com: domain of shivankg@amd.com designates 40.107.102.57 as permitted sender) smtp.mailfrom=shivankg@amd.com; dmarc=pass (policy=quarantine) header.from=amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1718403389; a=rsa-sha256; cv=pass; b=5Vx2pkB4YnIGSIE20wH5VCjdE/TA/OsX4qE1kGEgxWiQQ6K/m3I2xmrbCJVZJcfWkAg9Gl 50rof1HB+lB4XWKjAlF7uxGZTBy97g3CauVkrA7+NA17e8qkeZtyr25oLBwr+ojzQTdYVu 0XdzmTPBy2TEDKKLdnM+VwTMttDb18U= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EzStBe8T974ZdKnRVH83dkDInGyfO5RGmn43iudb7fsK2fkB4bGDJnWG109qZzIF6ts7nDo2YdTd3ACseWL88rrNDb5y9ci2DhxrWnJJbNuMtakB2f3H+3Y1P07dlL9H2W2DESFn2HRlJuEQicuUh0S2/7iPdkLQbohGYCnWm57/pB1ALOE0wVPJ+H/O0QYicP1yBjtBUq2jiAb7z3VBE7UpZUsYUCHtqka2An08vIn+8Z3ba65X+SD33N8dNcqCaBubmlrdLSTZJxzLsmNsMZfcDmZAo7AueFBMig8XW6z8tcLIs68WSrk5lUxxDW9K+KmlJM5h/xrxKe8GyGEDpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UFtwicY4CyhIt3hTfES90NWrdOR1HL5qCnm9jVePVUs=; b=Y21oqrxVOcP8CpuVRQTtpQYqH9YlwRmHUhfuNv74KQG4UG2tXL3Ru+XgeAUOjx3MlINVxspyJ4gaNdMgLG0yKHS8x1clhAhdmXyefxu3K0YDXPN3mfxbh+v1ijHobPoBvyv+WUTYsZsgB2HcRFKbhb5uRHQKF1KCzzmc2eja/dkzFFzmrAFVu714B6d5W438BWvYC9Rmo1Fo+vLL+J/2O4rM9Hlcg5t4HC58uVElr/2y4V8j/g9aiUQbniIbqdn7AY3/nz7aadI8p6nVRXtPTqN8aisdflQhLyN5YQXtJGuOy+jUw4rhy8jwTVfzFdQ2kY/lOpJCblJn6sLdGReMeg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linux-foundation.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UFtwicY4CyhIt3hTfES90NWrdOR1HL5qCnm9jVePVUs=; b=lCA//9EI+Hdd7ew2z9isp3J6U3/VRmWIeLir/s2IZCJPCQ/fhIrp8xK5mzzB82EfJrMwjmmp/Cg8ZjO/gH31Yi1uv3QUZzUPWQa7m1mG1Z3xDhiJ0h6025LzgTSz/Nwj7pgFnkdGjCKyOY1aTQFzgtvTAbsMZzcKwMNfyqx8sgs= Received: from DM6PR13CA0025.namprd13.prod.outlook.com (2603:10b6:5:bc::38) by IA1PR12MB8537.namprd12.prod.outlook.com (2603:10b6:208:453::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.26; Fri, 14 Jun 2024 22:16:27 +0000 Received: from DS1PEPF00017091.namprd03.prod.outlook.com (2603:10b6:5:bc:cafe::31) by DM6PR13CA0025.outlook.office365.com (2603:10b6:5:bc::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.25 via Frontend Transport; Fri, 14 Jun 2024 22:16:27 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DS1PEPF00017091.mail.protection.outlook.com (10.167.17.133) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7677.15 via Frontend Transport; Fri, 14 Jun 2024 22:16:26 +0000 Received: from BLR-L-SHIVAGAR.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Fri, 14 Jun 2024 17:16:21 -0500 From: Shivank Garg To: , , CC: , , , , , , Byungchul Park Subject: [RFC PATCH 1/5] mm: separate move/undo doing on folio list from migrate_pages_batch() Date: Sat, 15 Jun 2024 03:45:21 +0530 Message-ID: <20240614221525.19170-2-shivankg@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240614221525.19170-1-shivankg@amd.com> References: <20240614221525.19170-1-shivankg@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS1PEPF00017091:EE_|IA1PR12MB8537:EE_ X-MS-Office365-Filtering-Correlation-Id: 8f60414b-8c3a-4634-6e58-08dc8cbfa231 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230037|82310400023|1800799021|36860700010|376011; X-Microsoft-Antispam-Message-Info: 5dh8prkNJ+fATbJRmMwsCID/1cj2gcVR+EGg8eZWJSv6WJLmvZiq09OtiQjSthzDGY+VEsF73llLue+jTbQbT/MfoOKzqOu/oMzqvGDTblAJdOTrAHTo/go7fX4kBKJ0aa49Lpbp20socpHoAdJXp9UzIXXG1YuoVohGxl78jhM2IaNxU/p0/SaonSSCe41Yx5j+Rll4FbPS5XQaWEAiFu9CgMyRrKXUOLgcRK5ecNRO/hwn23gfbPl4fh9U1R4w31hFqQusnmTnh41IqUHojXQ61gSa8FJGK7BiDFWrekZYKpOz5pke+NIJ0p7HorjsgW5mwl4F+/lNcUhR5WoJFh/8T4ASIvmijAKKcUq1GgR+OAJThmu4za7chdDRSs3jbUctOipoaNYY3zhhj1tJpux9uhsHpnd4qiFeEbGSHMskoAP7yS9ja3BhJsKWfWnM5K4HWiqn4KipRerejkTXVn3VzWg9jkFSwOUJmugwoWCPQ0NfLMXJWv2uDl9134GeTdJ/EcPXovu2tELMYdUs7wGT8GJoawHdfO+JXq13mHfOlIXJb7C//gB4VSJAOt7mSmNL6+FZ8RtcP9kVSCZq6m8K+W92yVkwYsODUJIepj2ig0hjt/RwKaPOJjOaLrR++nVAy1c1glk5PczOK9/3+JX5Zr0wxq2tuRl8qDma6r5TT6fqoxOWSXRHx2/GxmGXuSaMzQpEWrjFOQqkCnxSdYvPHwwdtNPkcF3Fkir/WjaButropI7RmvxmVJC+tvd2Aqa5cguLUIjs242jESJEozODf2eJd4HNjY4Ln/8oFTIOSN4WSb/3vpJRUpgle9BYEMfrLEXvpJffhAOhtFEiM8qVG/8lMuLyvYu8JatU+nsaPAu5hvGBWuQ/gOYFHdP2uLj2f+aWW/zi3B1PI7ld10hI3N6FKv9zbxjIFeFg3pjXfWaN/O7/SwZc6qJtHN3T/z7y6JTVWdzlX/XMtpEcE5rs5wEk6KgXJFQPZcJeiRRVFLqULe6+w/HpUtirXr9ae2EJq5nN5ib2T9rq+8J/++nBfMKIpsz8IBEsYPG42QLfrIXgqNUHIzbNQwIpAYfLgxU0tgdPSbDR4VXxAgy3AQtMrS1GAILuIOP3nMRSKio9cKN4w4RnpbMHsi4jflB/yBp0mV8KRlfL5G5Y5T4I8zL9d7a1nMogSHqRIyNIREr9bof6C0oKDrxKDH7cWRoeg1sXB0cJluLeYzuzyuQf0rLcl8usS5u7ouXC2pbBDh0g1yBz9yVxsarvDsGyTybsZ7xoH/vGv3LGFOYwBBjwFUWhy/5tZugPzOhGrKKIUV8ItiiSQbqhyu1yDrFz80RO87MYOWKev0bjteZUqHsy/qSrFk3AYZMeWHEvt0u9Bx8LMv8Y5CyfuAT/3G1Tnbds/kdGg+jzKG7JvB1LtJcAMQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230037)(82310400023)(1800799021)(36860700010)(376011);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2024 22:16:26.5058 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8f60414b-8c3a-4634-6e58-08dc8cbfa231 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DS1PEPF00017091.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8537 X-Stat-Signature: dup5ej8ncoptjgett6yaorg7gp8nyc3k X-Rspam-User: X-Rspamd-Queue-Id: 99A77C0004 X-Rspamd-Server: rspam02 X-HE-Tag: 1718403391-398668 X-HE-Meta: U2FsdGVkX1/dpeCTnb4s4BZSf1i4MwEqenZhD4bvAvuFIlYA5wyJN42wQ0Q+AyIK082pIv2i8SgKyQUt0YRBBcBalh49ksPlgESNur4bpf1ciZvxTsCjCK8XVWgBieYmQ8KcftqXg84dIcVwiREc0aier8FteO3eVqRUyPwiOhuczrM9YYCo+B8LUbrqLEISYg13eQlNHc8bJOpJ2ktVSpzW4a6RinTzbBxc6CX+NRoeyJVDKamnH9zy65VDCo3IOBD7osub2YBOSzGuUgQEQd3CL0ZnQhEdACQ3oOH7JZi3Jutcvl6R+UxtupUh0mHweAAfgEgdVKx3IZd2PCZByY4zSCg0NvsSUns6wyjOdEBGMLRhV50k4ciP/rrHVI2X6R3GnQDSKQ5eylq9v3GlYkB8j05HtS2zgGnUgrFrRr2ISXLJWocoxldzqPy+QmcI0g7rFCbLacsQcGK3G9PZj1qBstPRuXZYNQUHrHiAlubwlMf2qWU+OXlofphGPzTzp/6TTz+x/xrHDnGSmLQj/1hE3qcpGnYpdTVlPsAQnpKpFIH4ST/sXdUwLgLJuoPnpezFLNwDB47QV2+gEXV9JMDD/D9SvtbrY55Gab1qmYCrIEi810w8iTPkiM9os6no/bA0pZHLIHzQWMnNZWWt5AK3si79lbL9Xy6IOu4Qof4q8/ngYvbvaR29eAJVcoI8oF7qaivfzYMEBtcfL+NZLV36E9ZIUNskxOFTZAUQtIjDoYQfycSoT4ahVfy4TUfZMWaSIzBfu7EdUPK64lJ9ij3UrxTnwDnq88O5pSU7jYxLqjr4nk9k2c0WBHNUl22J9k+2ggFKJxdbjeb252hrMVSRWAS7crTgqKQ3GAmnn9/KQt3EN1gMfnHMc67h6D9RKEdfDCLYEwCFYH4DJUp6O8kAvFhPsXFzGy/mXQ5MJZAj48cosSP6fTngoUOWqoOrAsugiHYio4uwLoeednS hBXVIJOz M+kMrwONo2OKXPvjekxBWesa4cXu3frb2zcaX4OqlOWQEL+cTtzuK9P0bl8LN3Ult8r1/hd6Hi6TkMtSTff1oVdQdp7BDucyDWJPxnUu9UsE93wraJBdoH7CooVHRWVPXqrQAHQf0MPtlVPnwocIU21Zf2gaBJAwRIinMgDKyP1tb8OYf7QWr00mb+/T+tt9PIIa3v8UYj2w0o9pQ0MVZIED4//eXT6fke3vxvikXmJyU5g58tBTkaFZqP7Y558zZ4LP2JCrQKiZtKQOr8a8caD7I3Y2CPeVFsoj1LrsPVWJC0u4Jkblre/IuFC7mxrAEp71in0TA2l+JCk2YqbiQkjVu4Lu3NYfBk2xKP2niqT7qco4un1LtT0UHYAygQ0oXksoNv3y3WppNbemI2/xrrxkEZCWRlnqVnixU6+B9gPGK6vKFsWREa3n8wBfmcJIhq+UV X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Byungchul Park Functionally, no change. This is a preparatory patch picked from luf (lazy unmap flush) patch series. This patch improve code organization and readability for steps involving migrate_folio_move(). Refactored migrate_pages_batch() and separated move and undo parts operating on folio list, from migrate_pages_batch(). Signed-off-by: Byungchul Park Signed-off-by: Shivank Garg --- mm/migrate.c | 134 +++++++++++++++++++++++++++++++-------------------- 1 file changed, 83 insertions(+), 51 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index c27b1f8097d4..6c36c6e0a360 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1606,6 +1606,81 @@ static int migrate_hugetlbs(struct list_head *from, new_folio_t get_new_folio, return nr_failed; } +static void migrate_folios_move(struct list_head *src_folios, + struct list_head *dst_folios, + free_folio_t put_new_folio, unsigned long private, + enum migrate_mode mode, int reason, + struct list_head *ret_folios, + struct migrate_pages_stats *stats, + int *retry, int *thp_retry, int *nr_failed, + int *nr_retry_pages) +{ + struct folio *folio, *folio2, *dst, *dst2; + bool is_thp; + int nr_pages; + int rc; + + dst = list_first_entry(dst_folios, struct folio, lru); + dst2 = list_next_entry(dst, lru); + list_for_each_entry_safe(folio, folio2, src_folios, lru) { + is_thp = folio_test_large(folio) && folio_test_pmd_mappable(folio); + nr_pages = folio_nr_pages(folio); + + cond_resched(); + + rc = migrate_folio_move(put_new_folio, private, + folio, dst, mode, + reason, ret_folios); + /* + * The rules are: + * Success: folio will be freed + * -EAGAIN: stay on the unmap_folios list + * Other errno: put on ret_folios list + */ + switch (rc) { + case -EAGAIN: + *retry += 1; + *thp_retry += is_thp; + *nr_retry_pages += nr_pages; + break; + case MIGRATEPAGE_SUCCESS: + stats->nr_succeeded += nr_pages; + stats->nr_thp_succeeded += is_thp; + break; + default: + *nr_failed += 1; + stats->nr_thp_failed += is_thp; + stats->nr_failed_pages += nr_pages; + break; + } + dst = dst2; + dst2 = list_next_entry(dst, lru); + } +} + +static void migrate_folios_undo(struct list_head *src_folios, + struct list_head *dst_folios, + free_folio_t put_new_folio, unsigned long private, + struct list_head *ret_folios) +{ + struct folio *folio, *folio2, *dst, *dst2; + + dst = list_first_entry(dst_folios, struct folio, lru); + dst2 = list_next_entry(dst, lru); + list_for_each_entry_safe(folio, folio2, src_folios, lru) { + int old_page_state = 0; + struct anon_vma *anon_vma = NULL; + + __migrate_folio_extract(dst, &old_page_state, &anon_vma); + migrate_folio_undo_src(folio, old_page_state & PAGE_WAS_MAPPED, + anon_vma, true, ret_folios); + list_del(&dst->lru); + migrate_folio_undo_dst(dst, true, put_new_folio, private); + dst = dst2; + dst2 = list_next_entry(dst, lru); + } +} + /* * migrate_pages_batch() first unmaps folios in the from list as many as * possible, then move the unmapped folios. @@ -1628,7 +1703,7 @@ static int migrate_pages_batch(struct list_head *from, int pass = 0; bool is_thp = false; bool is_large = false; - struct folio *folio, *folio2, *dst = NULL, *dst2; + struct folio *folio, *folio2, *dst = NULL; int rc, rc_saved = 0, nr_pages; LIST_HEAD(unmap_folios); LIST_HEAD(dst_folios); @@ -1764,42 +1839,11 @@ static int migrate_pages_batch(struct list_head *from, thp_retry = 0; nr_retry_pages = 0; - dst = list_first_entry(&dst_folios, struct folio, lru); - dst2 = list_next_entry(dst, lru); - list_for_each_entry_safe(folio, folio2, &unmap_folios, lru) { - is_thp = folio_test_large(folio) && folio_test_pmd_mappable(folio); - nr_pages = folio_nr_pages(folio); - - cond_resched(); - - rc = migrate_folio_move(put_new_folio, private, - folio, dst, mode, - reason, ret_folios); - /* - * The rules are: - * Success: folio will be freed - * -EAGAIN: stay on the unmap_folios list - * Other errno: put on ret_folios list - */ - switch(rc) { - case -EAGAIN: - retry++; - thp_retry += is_thp; - nr_retry_pages += nr_pages; - break; - case MIGRATEPAGE_SUCCESS: - stats->nr_succeeded += nr_pages; - stats->nr_thp_succeeded += is_thp; - break; - default: - nr_failed++; - stats->nr_thp_failed += is_thp; - stats->nr_failed_pages += nr_pages; - break; - } - dst = dst2; - dst2 = list_next_entry(dst, lru); - } + /* Move the unmapped folios */ + migrate_folios_move(&unmap_folios, &dst_folios, + put_new_folio, private, mode, reason, + ret_folios, stats, &retry, &thp_retry, + &nr_failed, &nr_retry_pages); } nr_failed += retry; stats->nr_thp_failed += thp_retry; @@ -1808,20 +1852,8 @@ static int migrate_pages_batch(struct list_head *from, rc = rc_saved ? : nr_failed; out: /* Cleanup remaining folios */ - dst = list_first_entry(&dst_folios, struct folio, lru); - dst2 = list_next_entry(dst, lru); - list_for_each_entry_safe(folio, folio2, &unmap_folios, lru) { - int old_page_state = 0; - struct anon_vma *anon_vma = NULL; - - __migrate_folio_extract(dst, &old_page_state, &anon_vma); - migrate_folio_undo_src(folio, old_page_state & PAGE_WAS_MAPPED, - anon_vma, true, ret_folios); - list_del(&dst->lru); - migrate_folio_undo_dst(dst, true, put_new_folio, private); - dst = dst2; - dst2 = list_next_entry(dst, lru); - } + migrate_folios_undo(&unmap_folios, &dst_folios, + put_new_folio, private, ret_folios); return rc; }