From patchwork Sat Jan 25 06:53:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ge Yang X-Patchwork-Id: 13950227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCBF6C02182 for ; Sat, 25 Jan 2025 06:55:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 279882800D2; Sat, 25 Jan 2025 01:55:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2292D2800D1; Sat, 25 Jan 2025 01:55:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 119292800D2; Sat, 25 Jan 2025 01:55:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E80AF2800D1 for ; Sat, 25 Jan 2025 01:55:05 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 7FA281A1BD8 for ; Sat, 25 Jan 2025 06:55:05 +0000 (UTC) X-FDA: 83045062170.07.E1A29C7 Received: from m16.mail.126.com (m16.mail.126.com [220.197.31.8]) by imf14.hostedemail.com (Postfix) with ESMTP id ACD9810000E for ; Sat, 25 Jan 2025 06:55:01 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=126.com header.s=s110527 header.b=XWgPeoOG; dmarc=pass (policy=none) header.from=126.com; spf=pass (imf14.hostedemail.com: domain of yangge1116@126.com designates 220.197.31.8 as permitted sender) smtp.mailfrom=yangge1116@126.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737788104; a=rsa-sha256; cv=none; b=v9+z7AoXQhw0/s0ts3vzSUTDCFbugEOYo0+D19MthJfEweRYtEsqFgLGxiUhhomDsfc8q7 ROFgUYd81sshIjpe7e4BwQH3SUrznWBoYqqsao27LLvAs2ynubBOZ7v0fHyXHzJOVjixyS FrPPkQpVDDtZq1XuHIqw7XtE1fU8w7w= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=126.com header.s=s110527 header.b=XWgPeoOG; dmarc=pass (policy=none) header.from=126.com; spf=pass (imf14.hostedemail.com: domain of yangge1116@126.com designates 220.197.31.8 as permitted sender) smtp.mailfrom=yangge1116@126.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737788104; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references:dkim-signature; bh=93DSEUT6o3kFOhY6PPQRtcQaJ1bgtNlRNVj2p9OOI0Q=; b=PP6wLM+vyd9MXgjUJ7+lwrYeutjZFrcRCPDZiEUyLpe8rNU8FMSorlqqyZ7TQU9qZ0muYs hvYWx3bFYsiMdwz1rszZC1bTXWUOA1/yQKACo43ehnanxSnnaQVAI/snPOt6VdwiVW8Hcv 4Wj6gAmfQ1xH7lqacFUC+QIEdX4TuqI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=126.com; s=s110527; h=From:Subject:Date:Message-Id; bh=93DSEUT6o3kFOhY6PP QRtcQaJ1bgtNlRNVj2p9OOI0Q=; b=XWgPeoOGB6eOhJTwzDqO3Asf7Z8bHC3YnN ZNrka7Fz/KKcKAGqYSs1wFedWzjcDtSPmYq9awA1XjSdhifpyqqsd/EYFXkO4RLI qaRcKEZAtD3keTbHbsOese4qQFoJgFCywxMgtMHF7BRLV8xKvqo3Tqsh+lQ0kEOx ZxXr7mR94= Received: from hg-OptiPlex-7040.hygon.cn (unknown []) by gzga-smtp-mtada-g0-3 (Coremail) with SMTP id _____wDXL7WGipRn9ibpAA--.18559S2; Sat, 25 Jan 2025 14:53:59 +0800 (CST) From: yangge1116@126.com To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, 21cnbao@gmail.com, david@redhat.com, baolin.wang@linux.alibaba.com, hannes@cmpxchg.org, vbabka@suse.cz, liuzixing@hygon.cn, yangge Subject: [PATCH V4] mm: compaction: use the proper flag to determine watermarks Date: Sat, 25 Jan 2025 14:53:57 +0800 Message-Id: <1737788037-8439-1-git-send-email-yangge1116@126.com> X-Mailer: git-send-email 2.7.4 X-CM-TRANSID: _____wDXL7WGipRn9ibpAA--.18559S2 X-Coremail-Antispam: 1Uf129KBjvJXoWxZrW3CFyxAFy5Cr13Kr1DGFg_yoW7Jw1rpa y8uw13C395XF43uF4xta1kuF1Ygw4xJF1UJFyIyw18Zw4akFn2v3WDta45AF4UXry3JFWj vFZ09F1DCanxAaDanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x0zRoGQDUUUUU= X-Originating-IP: [112.64.138.194] X-CM-SenderInfo: 51dqwwjhrrila6rslhhfrp/1tbifg-fG2eUgakg3wABsQ X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: ACD9810000E X-Stat-Signature: 7qwyigkgseh3ejtkemdzwb7s4djhe9yu X-HE-Tag: 1737788101-401776 X-HE-Meta: U2FsdGVkX18RuRui0dOJ7xV4LUgzwFwKBZDfANJCpWCRrqv5ll8jasfDu42nhiPqITpKIHo0IL3CbTdCQEVDkMWvL+LrJHEewtyr4k1ztMyiAQl61h1fNAKN3CQudEM9m4InbwdVIM7WE2PX4gC9EYSV1NygxPB5YMZJM0jc3dtiJ2JvQzXgoqSfoFW3T20uIDeMC4IDC5TQmPs4/VJIhZj9CHQwddiiQD9kICD5KYBS9eFoRi70DULfI6yW4CpAYQV+KDYVQTDpp5QGVrcIb38ivuXSIcnLHrl8oq+fpwN82rRptmwRMvDJFj7N5qmBSLRnMSb+Wc0go2nKx7EzUDC70gwj5YIKX7NuvJEDvxwjUWBqWBSe4L7+xdkei71dKYrCTUdJ64QCNltWRxrlyyeNnZoIfwXov9TXmwjuCvkb7UzgZYfKLjlNkhvK6A/NCTxrz9lLhi/L/WJtDS2weY/8dee9nca/yY1ELD8Er5HubyQrVghPSuFyOgWjlmUOySKpsdfN9OpmTt0zzI9PyVTd0t68o/JgG3Ah0XvqCZwBxeqL1/QAr+RXk+Y4OtIWN4IaUFbrBxP0N8nBVwPgf4tPjnL7UBWoYI4x/ONX6brqKaQmu5vKLR6qannpBQmjne1VsXNRHn97bL3P+T5mHFCmedyQfz40Wjlh6O+nV+uF1xzJTd43mcY1rLvB2KP4bGo6FfYUKJJLnrNnJMaP48w0bXdj/7f29OAzsrqVQvV9LN10Wh+fZY7+eQ9uFak3njvoEEntCYmh1bbj+W4pbgy1eoxb29ceyfyKCf0tcZxjuL+GUkG5Qsrd2qUNtEode9PRvtj7S0rV+EVaZLtoL6IS238IfjkyeiOOlrlIrXoHDjnMqSQkG+9uVUfK+YASUPXTM9a/L1eOgM45ZiAKPT0/lUK4Q7rhUVGPx3s8c08qbK92x3cGP0wPJ3rg+iC79h6/fDC3qhkQiMpN/Hx R+5vuOb6 q7jM+oQ1qkdkSUGRmj21obQO5q3xnCJ4+zDFOP0GtVND0/lAgO/IcL5Va9hYk5b6Ek0o3e4plW/+bxYemlrSTeMi7pBs9O8kTkXksjANWI1KHZAca6gNhAHqCnb3pzjcuzhtKjsGqHNNqkIzk/I//45h+3suiitNXLIka5qeIKHrgZQMMJqSUxhK4taVfILnHu8l3TF8n1fFerNmI1Jn07p8fY+SqNi+7k6BYU84YKBixWi+isV0CdQoqPuuGOXwcGrs2m4K6phgVI35IpSMVKTQ+vnnYq5Mffl+pN7NA5LgHmW6kJAxt/akCcC5V1vwnsQ6CG1gCvY2Uh2W/9PJLyqO9JYjMemEUxa1pNM8PpM8MAlDk5Fw8txNyVoduWL59UIis X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: yangge There are 4 NUMA nodes on my machine, and each NUMA node has 32GB of memory. I have configured 16GB of CMA memory on each NUMA node, and starting a 32GB virtual machine with device passthrough is extremely slow, taking almost an hour. Long term GUP cannot allocate memory from CMA area, so a maximum of 16 GB of no-CMA memory on a NUMA node can be used as virtual machine memory. There is 16GB of free CMA memory on a NUMA node, which is sufficient to pass the order-0 watermark check, causing the __compaction_suitable() function to consistently return true. For costly allocations, if the __compaction_suitable() function always returns true, it causes the __alloc_pages_slowpath() function to fail to exit at the appropriate point. This prevents timely fallback to allocating memory on other nodes, ultimately resulting in excessively long virtual machine startup times. Call trace: __alloc_pages_slowpath if (compact_result == COMPACT_SKIPPED || compact_result == COMPACT_DEFERRED) goto nopage; // should exit __alloc_pages_slowpath() from here We could use the real unmovable allocation context to have __zone_watermark_unusable_free() subtract CMA pages, and thus we won't pass the order-0 check anymore once the non-CMA part is exhausted. There is some risk that in some different scenario the compaction could in fact migrate pages from the exhausted non-CMA part of the zone to the CMA part and succeed, and we'll skip it instead. But only __GFP_NORETRY allocations should be affected in the immediate "goto nopage" when compaction is skipped, others will attempt with DEF_COMPACT_PRIORITY anyway and won't fail without trying to compact-migrate the non-CMA pageblocks into CMA pageblocks first, so it should be fine. After this fix, it only takes a few tens of seconds to start a 32GB virtual machine with device passthrough functionality. Link: https://lore.kernel.org/lkml/1736335854-548-1-git-send-email-yangge1116@126.com/ Signed-off-by: yangge Acked-by: Vlastimil Babka Reviewed-by: Baolin Wang Acked-by: Johannes Weiner --- V4: - reduce the length of the subject line. V3: - collect Baolin's reviewed-by and Johannes's acked-by, thanks! V2: - update code and message suggested by Vlastimil mm/compaction.c | 29 +++++++++++++++++++++++++---- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 07bd227..3de7b67 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -2490,7 +2490,8 @@ bool compaction_zonelist_suitable(struct alloc_context *ac, int order, */ static enum compact_result compaction_suit_allocation_order(struct zone *zone, unsigned int order, - int highest_zoneidx, unsigned int alloc_flags) + int highest_zoneidx, unsigned int alloc_flags, + bool async) { unsigned long watermark; @@ -2499,6 +2500,23 @@ compaction_suit_allocation_order(struct zone *zone, unsigned int order, alloc_flags)) return COMPACT_SUCCESS; + /* + * For unmovable allocations (without ALLOC_CMA), check if there is enough + * free memory in the non-CMA pageblocks. Otherwise compaction could form + * the high-order page in CMA pageblocks, which would not help the + * allocation to succeed. However, limit the check to costly order async + * compaction (such as opportunistic THP attempts) because there is the + * possibility that compaction would migrate pages from non-CMA to CMA + * pageblock. + */ + if (order > PAGE_ALLOC_COSTLY_ORDER && async && + !(alloc_flags & ALLOC_CMA)) { + watermark = low_wmark_pages(zone) + compact_gap(order); + if (!__zone_watermark_ok(zone, 0, watermark, highest_zoneidx, + 0, zone_page_state(zone, NR_FREE_PAGES))) + return COMPACT_SKIPPED; + } + if (!compaction_suitable(zone, order, highest_zoneidx)) return COMPACT_SKIPPED; @@ -2534,7 +2552,8 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) if (!is_via_compact_memory(cc->order)) { ret = compaction_suit_allocation_order(cc->zone, cc->order, cc->highest_zoneidx, - cc->alloc_flags); + cc->alloc_flags, + cc->mode == MIGRATE_ASYNC); if (ret != COMPACT_CONTINUE) return ret; } @@ -3037,7 +3056,8 @@ static bool kcompactd_node_suitable(pg_data_t *pgdat) ret = compaction_suit_allocation_order(zone, pgdat->kcompactd_max_order, - highest_zoneidx, ALLOC_WMARK_MIN); + highest_zoneidx, ALLOC_WMARK_MIN, + false); if (ret == COMPACT_CONTINUE) return true; } @@ -3078,7 +3098,8 @@ static void kcompactd_do_work(pg_data_t *pgdat) continue; ret = compaction_suit_allocation_order(zone, - cc.order, zoneid, ALLOC_WMARK_MIN); + cc.order, zoneid, ALLOC_WMARK_MIN, + false); if (ret != COMPACT_CONTINUE) continue;