From patchwork Mon Nov 13 17:01:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13454189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9FDBC4167D for ; Mon, 13 Nov 2023 17:02:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 548628D0030; Mon, 13 Nov 2023 12:02:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F70A8D002E; Mon, 13 Nov 2023 12:02:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 372E58D0030; Mon, 13 Nov 2023 12:02:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 254BD8D0001 for ; Mon, 13 Nov 2023 12:02:29 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 03F6940600 for ; Mon, 13 Nov 2023 17:02:28 +0000 (UTC) X-FDA: 81453549618.18.A76750A Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by imf13.hostedemail.com (Postfix) with ESMTP id 43E922004B for ; Mon, 13 Nov 2023 17:02:25 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=sent.com header.s=fm3 header.b=LLVv3ylo; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=BY18zVII; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf13.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.26 as permitted sender) smtp.mailfrom=zi.yan@sent.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699894945; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nzOUpjfyn/JJJn/Enob781hONORF3mJcpBIoO1y6HOc=; b=CO0el7r403+JvHIH9qxQPhZZs56EbvvbXM999Kp+U2M9rXpgy7z9FrgbOPgqIGGhVQANMD I+AJ8yVCy1HzY3jCxJ0DZIllp9iClLrZc3NoJiNK8yRX491lWRIQQMYXPEoMQywnvNmgwc IwZkK4lopDErTbcz4Il7i5AzZ2diTQo= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=sent.com header.s=fm3 header.b=LLVv3ylo; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=BY18zVII; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf13.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.26 as permitted sender) smtp.mailfrom=zi.yan@sent.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699894945; a=rsa-sha256; cv=none; b=i1fIM96MfurqRbn46OkRdlqE40vvI5d4lXlEDRCtcvAwlbAGsO5TK+9RIlNUgPppOZcG4s +7EHeSFgbisE32mIZ0bWwwgSojbi6pEa/jKWjlX8yzAIsjDGcz3J7AvKfd6gaovvOEaCnm JcACw4wqU0f1d33mYNfGLvJXRGfrUU0= Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 829795C0099; Mon, 13 Nov 2023 12:02:24 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Mon, 13 Nov 2023 12:02:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:reply-to:sender:subject:subject:to:to; s= fm3; t=1699894944; x=1699981344; bh=nzOUpjfyn/JJJn/Enob781hONORF 3mJcpBIoO1y6HOc=; b=LLVv3yloNFUmMZ6xGakYy4VMI5jabZP1oX9qTx7H16gO 1CZG/Vu4y682pbGiU601G2tvZkKoGS/cQelvSjNl8IyVRCvBxY6ZDp/I8Z0El/7X 9ZfcY15EYusoFRVwvMbdZP6TQvbCHhLECsmgGAFbJf9HFgYb01aD31ORb3xofy3Y 7MIOzM0o59YHJxOvwcxZHA0/sbuL5Je60bq6A/+EhOnwDSY4kDNInc0UL2GwUU2y KoMrFOjoHe/NngSo17AreCt7IaANttkYOr1/Q8o2fk4fb2YI47vmTdRL2ARxdrcs 37aplOkVJOt1jGNxORVOB3T9vRyR7rMhKXKxozk+5g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1699894944; x=1699981344; bh=nzOUpjfyn/JJJn/Enob781hONORF 3mJcpBIoO1y6HOc=; b=BY18zVIIhn/w0FEXwxzpd9RefEDeDi6eMLhNRbpMZAVb 9dGnvGsCq+Z6W22Gby2w0+eOoExdovlBnvJ84qpX5Plpj10dvXTreo4EGwYu7EXN C0G1KpE+tNFcW/GqNrybEqajzff+3jNMkIQdJaYehnaeJl2Dwkc2E0poBSk45Qbk nCPaOWGnMh/SBtyKU3va+yhTH7DvRNKe9NfqaeaXfGUbEQDjCaGg6XRfUbGaQ1YU 3hnkKo6HsZhFLBGIbuxZHZCximCfxM2GOv+SU6jOJyn/jEb74S9EN0ECVc+h43Bf xVLHDT/QAI2Qqt1sAZQ5wYKxhrWruO0RmuXr7juCMQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudeftddgleefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhhrgggtgfesthekredtredtjeenucfhrhhomhepkghi ucgjrghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepje ekteekffelleekudfftdefvddtjeejuedtuedtteegjefgvedtfedujeekieevnecuvehl uhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepiihirdihrghnse hsvghnthdrtghomh X-ME-Proxy: Feedback-ID: iccd040f4:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 13 Nov 2023 12:02:22 -0500 (EST) From: Zi Yan To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Zi Yan , "Huang, Ying" , Ryan Roberts , Andrew Morton , "Matthew Wilcox (Oracle)" , David Hildenbrand , "Yin, Fengwei" , Yu Zhao , Vlastimil Babka , "Kirill A . Shutemov" , Johannes Weiner , Baolin Wang , Kemeng Shi , Mel Gorman , Rohan Puri , Mcgrof Chamberlain , Adam Manzanares , "Vishal Moola (Oracle)" Subject: [PATCH v1 4/4] mm/compaction: optimize >0 order folio compaction by sorting source pages. Date: Mon, 13 Nov 2023 12:01:57 -0500 Message-ID: <20231113170157.280181-5-zi.yan@sent.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231113170157.280181-1-zi.yan@sent.com> References: <20231113170157.280181-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 43E922004B X-Stat-Signature: 7ymcqkk5fdkupq6t8i38u76xobh5x4g1 X-HE-Tag: 1699894945-829744 X-HE-Meta: U2FsdGVkX19DV7JV7pkRSg19cowLPSqohXLrVtef0lev0aTB0awy8kqIbH2wZ88/LC0+gc+otZQTePKz6+TQaTYqliUpwu1xmttq8mRRt6FV2KfBrK7y/KCuTa7pYvfCDBUi8Ew8U5IZo2oQUxE82QGbiBiFxvcPohQBP7WJq1/W00dvDWy9pT97lkF5cYIY1Xl+sF3v0GqX/PRj08SO2cUGDFXHm93biOJn8bwO4arls3AEgPb3aw56xvP0ccb36htm40Mryr2KkF5TkofOU8/dSo0UmEnhJ2Mbr2zXV4XjEqGu6QusKFTlHGfszmbbfgCGiHGjcxQkzgZrWldP1KECbGWNSQ365u4TjtR1mL7rQIiXw6+FqNToXKh2vUhtl9dMYh5hdRa645yG003yOfRSHCAAF6NA/6ct9Sq4obpENKsExk3SBsmuCxyBoSMQHvKhMJA3muixTTe2djo/9s2HuVsEdhDmb3xVpa9yzj+Omc1QwVUR6FSge+pS2/yAiKu5hmWqn9DOcO7ELe36DuB2K8AG/JFvM0CAliWqfnRM41pUjl0qvS7uEd2arOqLlenrZdcU5CHEPWMnlm8bgNkY4owB01OsBSt+HafI1UPxS8ROS82JTNrCBMhJxVIH5zkU3v2jKYhbts5hu8182W/y7iIs4Tn/+RIS7YE/ZpGSe7AC4UoFrARkSyYs/48iSbLIXB5i/m+JUtj+CijSp0GYavVxHxVlfTClvMTwEw8PnHH3um8mQ6pN68X3Z26/BhIO8h9rgY9CJvkzqqHi8a0bUiKIEFHoClsKfua8rJ519ppDnocJKqwLq8CXD3HZ4xOCEJ5UItT6MuFZhjQoDdgHUjwdK+644tmCBaGHbXob4tJV0pK1jZjXPoJRTBSSH7bSCcDRpMXTC+F+UeSmKR+HUeCA0ItGijipwuGyApEiYQnjPjGJYvM6/1aNvip22nAAaDcwEaFcXhuy+Q9 sBj8HtmR Rb74f2NiM/++X3kAryzMQcuAbLuYYWFY8vrRf1K6dqC7HzoXZ+OOo9OE/RLNWjtZa2nQLjsJYUxo8WcsyzXpdmEeuJD7KzOASlsRZzi4skI8fhZW/NceffpsUXAxozWsRon7r7On60L1SwMFXxyk8Ttksyujb4ZfSvJfQoM4Y0TMG5WxwdxajY7PW+mnTRPBvpaiY1IXPXCv7Jp8TJmsDd182tg9OPTCGiKFUqtAeMHJSGWOxEHpv9UcVehQGTTHX8v2oT0AXX+ni17Ojv2EVu5av7tj0F3kdvpuHxcutyRBhRojoM82cv3+xyfVogEIuSdEm/XdDBMm9PIsEZV0rKLda2nLjYWPq0zC9LrFAmkBcnRWt5tBnmA6Vs+WkqQgfg4HUUIBAoZRIo70AkCYX5TB46fg2Dym/M5bnPsY/mX6FfLM8BqbcII4X61hn5B5KPtlVZTT0P4zHG1Ch9NFDfjCgSefSEaeTENogl6Zz3oCKqmUGsVOh2g212vQT5QnwyeU3K7E1DFvTpd29re4RTXIFAU4/Wlhbh4IQwiEhNHKR/lXiebm4ilJ7Vg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Zi Yan It should maximize high order free page use and minimize free page splits. It might be useful before free page merging is implemented. Signed-off-by: Zi Yan --- mm/compaction.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/mm/compaction.c b/mm/compaction.c index 9c083e6b399a..91809bee5422 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -127,6 +127,37 @@ static unsigned long release_free_list(struct page_list *freepages) } #ifdef CONFIG_COMPACTION + +static void sort_folios_by_order(struct list_head *pages) +{ + struct page_list page_list[MAX_ORDER + 1]; + int order; + struct folio *folio, *next; + + for (order = 0; order <= MAX_ORDER; order++) + init_page_list(&page_list[order]); + + list_for_each_entry_safe(folio, next, pages, lru) { + order = folio_order(folio); + + if (order > MAX_ORDER) + continue; + + list_move(&folio->lru, &page_list[order].pages); + page_list[order].nr_pages++; + } + + for (order = MAX_ORDER; order >= 0; order--) { + if (page_list[order].nr_pages) { + + list_for_each_entry_safe(folio, next, + &page_list[order].pages, lru) { + list_move_tail(&folio->lru, pages); + } + } + } +} + bool PageMovable(struct page *page) { const struct movable_operations *mops; @@ -2639,6 +2670,8 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) pageblock_start_pfn(cc->migrate_pfn - 1)); } + sort_folios_by_order(&cc->migratepages); + err = migrate_pages(&cc->migratepages, compaction_alloc, compaction_free, (unsigned long)cc, cc->mode, MR_COMPACTION, &nr_succeeded);