From patchwork Tue Sep 12 16:28:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13381934 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7900CA0EEB for ; Tue, 12 Sep 2023 16:28:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9928C6B012E; Tue, 12 Sep 2023 12:28:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F2E36B012F; Tue, 12 Sep 2023 12:28:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 745406B0130; Tue, 12 Sep 2023 12:28:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 624AB6B012E for ; Tue, 12 Sep 2023 12:28:38 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 3B87AC02F9 for ; Tue, 12 Sep 2023 16:28:38 +0000 (UTC) X-FDA: 81228478716.09.86D8B69 Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by imf16.hostedemail.com (Postfix) with ESMTP id 1B52B180031 for ; Tue, 12 Sep 2023 16:28:35 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=sent.com header.s=fm1 header.b=QqdKC9Fa; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=AN0P2K2R; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf16.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.26 as permitted sender) smtp.mailfrom=zi.yan@sent.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694536116; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=byTyyd05JmDbEdKIibYodeYHRw97CddSTrRNeSPfl1g=; b=4aaXQtDnTXmskscFORsd+r0WC7okZlA3D1kZIB7V+PiNN1ZcjGWT9gUujuBSoO3WPuzB4O JhTc6BeBenXT2HWEfqGf76MQqlTj9vhZDtbRnxbL2iQIQ3oo5pz880WzN2byWn9Ag1lDYC brokVP+TVVe/UFACAv130ER1hW0SH7I= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=sent.com header.s=fm1 header.b=QqdKC9Fa; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=AN0P2K2R; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf16.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.26 as permitted sender) smtp.mailfrom=zi.yan@sent.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694536116; a=rsa-sha256; cv=none; b=u+WD63QVsoU1N7YQU3tVs0wnNfp5P7WODxhr+lJUF0WrZtwaRVFsVEtP7gcr8mr+du8dS4 GRqgzFk+YZU/aOBQW0/Vi4HU8RPhcMHHfmnL3UHtBmexT2vm7ySIRlRvFL8RlMkw7xuLKM 5sVbXOei9kJqKgY4pVEYNuc+un8up0g= Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 7034B5C023E; Tue, 12 Sep 2023 12:28:35 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Tue, 12 Sep 2023 12:28:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:reply-to:sender:subject:subject:to:to; s=fm1; t= 1694536115; x=1694622515; bh=byTyyd05JmDbEdKIibYodeYHRw97CddSTrR NeSPfl1g=; b=QqdKC9Fa63nIIylhuX3PCGQuwV9dmtWfR/mazLyGKQ2+4OZ9meo 6u0MbfmLELVh08tPW3VpY7eE0jUGpD0f0hBCcPsYbrayTPcKt/3g1vA79iRGirU0 0DDb98OuKZ+PT4Apw5Utx1zuXFyQY16Z0/Xa27jNbU38TSEDSAa5eVX1SRabMaK3 rU5n6S7L6TLL+OXTLhBTgYibdxYKKNvNlyE2JwryH+czxTQVES2TV7N744KECOgg Q99ZR3+1ToHDH6Px8nmnpvYBezdgguG/oJPurUO7UvHR36q6uUvVvj2jSaeqOSNh M2o69Dm4EJYBpPihti1BcLJfCecaK0eNMHg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1694536115; x=1694622515; bh=byTyyd05JmDbEdKIibYodeYHRw97CddSTrR NeSPfl1g=; b=AN0P2K2RSLzgy6uNFWDc4/uLPjZXoOm9SaCVkKxTpK1h/+O5YvW RH69TUb9/Pk7kdQYyfQeDpfaqTKnldImPdLpBLg9I0r1pC5C9YOe/zFQRLyjcjLE 6CLXwwzTneyVPkCjv0P3DNAcfh/58tJXgnOFP80Yru1KuPKDPNQdE3qR4Uvj3JqR UVuLKi632KaIKJwUpl2haQIKZ6+sIBk9SHD1GUADGTtx8GlNig7dqgijjl9pMXs0 U0yMjxL/RaikpR8MC+LgeXuFHTAAgRRMhoivHoE4sfhDdPGdIZu8QjRvxXE8Lg5X 2B1jHKabqKGec0yBj+SglEIA9NrhPDanLKA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeiiedguddtudcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvfevufffkffojghfrhgggfestdhqredtredttdenucfhrhhomhepkghi ucgjrghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepge eghedugfduuddvleehheetgeeltdetieevuefhffevkefhveeufeeiieejgedvnecuvehl uhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepiihirdihrghnse hsvghnthdrtghomh X-ME-Proxy: Feedback-ID: iccd040f4:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 12 Sep 2023 12:28:34 -0400 (EDT) From: Zi Yan To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Zi Yan , Ryan Roberts , Andrew Morton , "Matthew Wilcox (Oracle)" , David Hildenbrand , "Yin, Fengwei" , Yu Zhao , Vlastimil Babka , Johannes Weiner , Baolin Wang , Kemeng Shi , Mel Gorman , Rohan Puri , Mcgrof Chamberlain , Adam Manzanares , John Hubbard Subject: [RFC PATCH 3/4] mm/compaction: optimize >0 order folio compaction by sorting source pages. Date: Tue, 12 Sep 2023 12:28:14 -0400 Message-Id: <20230912162815.440749-4-zi.yan@sent.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230912162815.440749-1-zi.yan@sent.com> References: <20230912162815.440749-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 1B52B180031 X-Stat-Signature: irpygxsmisfw7h9fadxehbdjf3gttbfd X-HE-Tag: 1694536115-135356 X-HE-Meta: U2FsdGVkX18PJgHkPtVf1pL0FYEsIq0SbHTdbwgrJ0VdKbg/Z0SfrC8vhCMCrcx2wOgCyu+BTM5dpsB1mfYCktbunyE6xiG0KriGv/KXsAPEo5eDO1H5P5A76GJustzsDXQfhTNe58y8IKG1AA4LqODV8H4nSvVkB8q+9kjR/IteQGDs5MzMgmHOOsNLIQeepoHw05i36nJ/zTr8DNfg5BXC1gDIUlxPe2JrH4JrPwmlp1UTGqb0JagYE1n5aV6QcbeeygPJcYd1zjMRVu6biBJWm7DrelhFCqcmrCTx1QN4wJuXWLtW0E/RilK6yBKOHnp2UmmX3k4rYp8NL8xXELLuJLQDkfyej0WTuNwg0ucL3fDI4qitupAQB/6nc2riF66mhs/VJisDyWVcyH+Ls8o/yYYGVn0yhQgvqPc/JqhxT2eV6V0sitMOC+fBnctmDtDOqY3QZIBfWyqCwCStiOb3TcXUqGsUUtyiBo/NU7Zi1uX0Uyg0M+0cTmsJUm0CAHbVt/cIGVr2HneAqfUWhrsyI/pI0OUc4snhkECnlDPKDlJgeY/kZy7dOJn+EMfxqR9Xr17fdfAhrgKnVXXbgj4RvGKkyG024D6E58wYOWOdo7jOddQzmvZb8m318wcUriGzDZtE7V+CTvB0tLGKHoe+bb3dlOs4JKzDRaZod1qG2tx8/f2vJRsxd6nmpnIp/H6O48+YZonwjzcFdVo1YZbwiVyuKkOFssW29L3KMiIHfT6MxuVihHbGz6cw7VAKWYoKSett1+PKWORw+9cu/4ULJC8Q9rUBdtcMTsqbh2S6UHYFRPuwHbgrxrTOobtgV0wbQjCQdTPaO2UFMiGGa11chKiXvWGjfaPy9g9HmLC8/RWVcFNT3jkeA1yWN/J8LWM1XdAB9i0Rqti9LLhP3nghmjnclpAY0iz84nvTU6hUvok4hZ6v5GBeeBSwEJe+Fircd6SEBQZ21sQWNsV 2M4P7kG6 UyCnyyxwCU6cifZGC84jI/doZTII2HLiDE8fxTzpVZW+n6R0ecHTGNa+pHMYTrbP6KFTwG4XQqd7J0VfOqOWoksVtvKHq13Gz3E1ga50O1HNl1y/9FVR7PTzSPR4zc80jLHrINGxr3ro9KKDCvPgvOZkcIAtGYQty4+s4NxBtaCXHlfUpO1uy6UM41K7Flm6a0T+B1Dx2ijundATEixe9g4dhUgDNOCMb8frDCf+W7mKKL/w6X0mHuVKIPdh5iQM6/O60enJrP36zvjSgzzvwRnK4DgYaFEaMeTl1j752aTXPDXFqqABrWtMQjRrcEHYa0ctG3tITLN+4FuCCgHoynMqs0wG0KfgXD9rI3aY7nXGSy3Yl0ZX9YbQlYZWVGCm9jyuhNe1pfgnuzBO3yEWPbdUTyFpnso8q96JFJvW/UXjZied2YDLxMRghAN0g7rPep11LSNZ6W6s1h3RcQlxQ3yAFUaYuBGIL4whpIY70gQlRTjPX7UgUIDNqpdoNflKJDzREJK2gVIsoUpHJqIQW0maDxCf3oGv1THIYGn5994t2mzk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zi Yan It should maximize high order free page use and minimize free page splits. It might be useful before free page merging is implemented. Signed-off-by: Zi Yan --- mm/compaction.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/mm/compaction.c b/mm/compaction.c index 45747ab5f380..4300d877b824 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -145,6 +145,38 @@ static void sort_free_pages(struct list_head *src, struct free_list *dst) } } +static void sort_folios_by_order(struct list_head *pages) +{ + struct free_list page_list[MAX_ORDER + 1]; + int order; + struct folio *folio, *next; + + for (order = 0; order <= MAX_ORDER; order++) { + INIT_LIST_HEAD(&page_list[order].pages); + page_list[order].nr_free = 0; + } + + list_for_each_entry_safe(folio, next, pages, lru) { + order = folio_order(folio); + + if (order > MAX_ORDER) + continue; + + list_move(&folio->lru, &page_list[order].pages); + page_list[order].nr_free++; + } + + for (order = MAX_ORDER; order >= 0; order--) { + if (page_list[order].nr_free) { + + list_for_each_entry_safe(folio, next, + &page_list[order].pages, lru) { + list_move_tail(&folio->lru, pages); + } + } + } +} + #ifdef CONFIG_COMPACTION bool PageMovable(struct page *page) { @@ -2636,6 +2668,8 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) pageblock_start_pfn(cc->migrate_pfn - 1)); } + sort_folios_by_order(&cc->migratepages); + err = migrate_pages(&cc->migratepages, compaction_alloc, compaction_free, (unsigned long)cc, cc->mode, MR_COMPACTION, &nr_succeeded);