From patchwork Wed Jan 25 13:44:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mel Gorman X-Patchwork-Id: 13115650 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB15EC27C76 for ; Wed, 25 Jan 2023 13:45:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 688BF6B0072; Wed, 25 Jan 2023 08:45:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 638EE6B0075; Wed, 25 Jan 2023 08:45:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 500946B0078; Wed, 25 Jan 2023 08:45:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 429F96B0072 for ; Wed, 25 Jan 2023 08:45:01 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 1FDF5120418 for ; Wed, 25 Jan 2023 13:45:01 +0000 (UTC) X-FDA: 80393442402.08.B101D0C Received: from outbound-smtp27.blacknight.com (outbound-smtp27.blacknight.com [81.17.249.195]) by imf16.hostedemail.com (Postfix) with ESMTP id 28BBD180018 for ; Wed, 25 Jan 2023 13:44:58 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf16.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.195 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674654299; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PKYuIAqSMWcwB9klwbP3s4uu8PZy8GlgNn39jK3DY8g=; b=s+/iT59Lh30iYPN9IfNqnhkRnmYMJ5hkPBM4Or4Aioohxxr+ogiOqJ/S0sTEmyEsjKR7Oz clSS3qh0Ebp3O1F/AUtnuilW1chAhKLfyhAX9bAkNZK92gwTk0rRr0Piovr7VnOVjZ43Gw MwQXsClYgf5m9HxkVCbwS3fze4xs1xI= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf16.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.195 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674654299; a=rsa-sha256; cv=none; b=WAqWbZIIWTLo+JXxh6urDmnP79bVdCvdpWRCmZdQnhYglDlbAC03lTGYKbmXhMcgtZQC5P KQhbt0L5M3yakWnPY1+UUFRDB666VS0XK7dxC6OwHoax3M6xaNdZMf9B2QtmG0bAjlGWnO s3bYFpY0VSBctYvhokXMoYHSPENpyF8= Received: from mail.blacknight.com (pemlinmail05.blacknight.ie [81.17.254.26]) by outbound-smtp27.blacknight.com (Postfix) with ESMTPS id D03F214A029 for ; Wed, 25 Jan 2023 13:44:57 +0000 (GMT) Received: (qmail 20634 invoked from network); 25 Jan 2023 13:44:57 -0000 Received: from unknown (HELO morpheus.112glenside.lan) (mgorman@techsingularity.net@[84.203.198.246]) by 81.17.254.9 with ESMTPA; 25 Jan 2023 13:44:57 -0000 From: Mel Gorman To: Vlastimil Babka Cc: Andrew Morton , Jiri Slaby , Maxim Levitsky , Michal Hocko , Pedro Falcato , Paolo Bonzini , Chuyi Zhou , Linux-MM , LKML , Mel Gorman Subject: [PATCH 1/4] mm, compaction: Rename compact_control->rescan to finish_pageblock Date: Wed, 25 Jan 2023 13:44:31 +0000 Message-Id: <20230125134434.18017-2-mgorman@techsingularity.net> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230125134434.18017-1-mgorman@techsingularity.net> References: <20230125134434.18017-1-mgorman@techsingularity.net> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 28BBD180018 X-Rspam-User: X-Stat-Signature: 4ag3hqbi8siuizdmy6h585fzdm6u86fx X-HE-Tag: 1674654298-853267 X-HE-Meta: U2FsdGVkX1/Lhehu1sbUnRdTRQeWkccOktV4/GFBWRr8giH+C+WkuEcKgA8nxF/dC38XMOAp0jHNPZBZwTKUwz1TGWzJqIWMSGIwa6qJTSnTpCQ7ob5Zf9OsXT6i2hfzMv1qg6N/TwSb5fwmK6B3dKXd+3+gV7LP405vAH73UZsBOjXjEBb5aTi9FkBXrHZjYMitujtPzhT5l0L0dbzBOrZqjHDKSRhnJcX90cs4ZDYE7XbRRVJIetWWJIntlQF4QQMk4+Usib2XWTNftoBgsD00mVnk2mj+xzx+KSOvF+tI2sSOX9T8SQSP9DjdcGAHMWUTanFAyHn2uhuAoHV5dWfs0l94Z132jlsnPRyLlEUwvedVAwSfkXJGv7yJwCDCZNgQ4ATg74EIY3bI8pmz30Ts4Pr4bHMBgO9feQjKRd27VpsIZWnUe8s6Nr7S4k3/1KwV9miDWNNYBN2FlKiw2qhuh/hYha72VMpEUCfTlE7t4vDxHRgyEKnpGH+mZN+62QS/AfmhnzzMrsxmU3bGs/hcnR6pHSuEIecR6UnkY+6VwEE9a9SawiwHUrXpRTyRQKYBnp0SIDlFhXnIzobC6ydTMtrbV0wzKyt1chDwVuxDJfjEpzOe+PMcQ8KFxvyuOnTuBbCikVcY8jARAOZJfP3fCDxOsbFRaphjwiiAUuUHkUhNqB5nG70RW4NtJ5Th6fSoYyuZepXpStVJ7/MCsfm4yMBUBn4gX75cuSr7jABtTr286reQKFTXfekq04dPXE14n6xrI0g30SRHxUP9O5yO0nLJEpmJyUoeMNRFmrIz8UhcNTqb3QXx9HE9ZW/tYFP0lwdS4tcluU8CaW1vQWWfng0JSxuQBXWMs3FCNnndsPG5z53GHOIphvFk3XumzC0lFOhdsl5KyhIwZzp4F4vLjkOF4p8/vhPcW+mDA2IETNjVLRg6mPLOq8Q6+tGROaSntWcmUMnH3Zb06EP A9PxqvjU ADLLbJj1Tpc1xAgwIqgSPVLOtCmuiRRzDqIQGjETocnIImdk4ZdUQ8jGMkrx2GdqUR5KmtoVIr16NHq/waTWUiogdCfwwjVMAgNfIHN/qgz6FkwT2OTmM1RETkD1Vn7oCqiK3+Ux7oWB5J73vepuezXMNj4SEt9mMEzRAoLSLSw6V+X/KSzSbKUMawQTwfhQvK1hT8q2qYdWNkObUtB2Gkfc9yYe0QVzsY2jt9b1aQXl86Hrz8QN8O4riPYq/I7uAL/utoDhW4D66fX9WHRtN85rthqASYDVlhEIp5T2RCptgCq6bUqmszvyAXJ2xMjtlE+T+DeeP21oaYrYwp8EUcjlW8/tXKV/CmulQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The rescan field was not well named albeit accurate at the time. Rename the field to finish_pageblock to indicate that the remainder of the pageblock should be scanned regardless of COMPACT_CLUSTER_MAX. The intent is that pageblocks with transient failures get marked for skipping to avoid revisiting the same pageblock. Signed-off-by: Mel Gorman Acked-by: Vlastimil Babka --- mm/compaction.c | 24 ++++++++++++------------ mm/internal.h | 6 +++++- 2 files changed, 17 insertions(+), 13 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index ca1603524bbe..c018b0e65720 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1102,12 +1102,12 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, /* * Avoid isolating too much unless this block is being - * rescanned (e.g. dirty/writeback pages, parallel allocation) + * fully scanned (e.g. dirty/writeback pages, parallel allocation) * or a lock is contended. For contention, isolate quickly to * potentially remove one source of contention. */ if (cc->nr_migratepages >= COMPACT_CLUSTER_MAX && - !cc->rescan && !cc->contended) { + !cc->finish_pageblock && !cc->contended) { ++low_pfn; break; } @@ -1172,14 +1172,14 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, } /* - * Updated the cached scanner pfn once the pageblock has been scanned + * Update the cached scanner pfn once the pageblock has been scanned. * Pages will either be migrated in which case there is no point * scanning in the near future or migration failed in which case the * failure reason may persist. The block is marked for skipping if * there were no pages isolated in the block or if the block is * rescanned twice in a row. */ - if (low_pfn == end_pfn && (!nr_isolated || cc->rescan)) { + if (low_pfn == end_pfn && (!nr_isolated || cc->finish_pageblock)) { if (valid_page && !skip_updated) set_pageblock_skip(valid_page); update_cached_migrate(cc, low_pfn); @@ -2374,17 +2374,17 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) unsigned long iteration_start_pfn = cc->migrate_pfn; /* - * Avoid multiple rescans which can happen if a page cannot be - * isolated (dirty/writeback in async mode) or if the migrated - * pages are being allocated before the pageblock is cleared. - * The first rescan will capture the entire pageblock for - * migration. If it fails, it'll be marked skip and scanning - * will proceed as normal. + * Avoid multiple rescans of the same pageblock which can + * happen if a page cannot be isolated (dirty/writeback in + * async mode) or if the migrated pages are being allocated + * before the pageblock is cleared. The first rescan will + * capture the entire pageblock for migration. If it fails, + * it'll be marked skip and scanning will proceed as normal. */ - cc->rescan = false; + cc->finish_pageblock = false; if (pageblock_start_pfn(last_migrated_pfn) == pageblock_start_pfn(iteration_start_pfn)) { - cc->rescan = true; + cc->finish_pageblock = true; } switch (isolate_migratepages(cc)) { diff --git a/mm/internal.h b/mm/internal.h index bcf75a8b032d..21466d0ab22f 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -422,7 +422,11 @@ struct compact_control { bool proactive_compaction; /* kcompactd proactive compaction */ bool whole_zone; /* Whole zone should/has been scanned */ bool contended; /* Signal lock contention */ - bool rescan; /* Rescanning the same pageblock */ + bool finish_pageblock; /* Scan the remainder of a pageblock. Used + * when there are potentially transient + * isolation or migration failures to + * ensure forward progress. + */ bool alloc_contig; /* alloc_contig_range allocation */ }; From patchwork Wed Jan 25 13:44:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mel Gorman X-Patchwork-Id: 13115651 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD2C2C27C76 for ; Wed, 25 Jan 2023 13:45:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 577A86B007D; Wed, 25 Jan 2023 08:45:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 527076B0078; Wed, 25 Jan 2023 08:45:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3EEDE6B007D; Wed, 25 Jan 2023 08:45:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 3131B6B0075 for ; Wed, 25 Jan 2023 08:45:13 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 0BD76A05F4 for ; Wed, 25 Jan 2023 13:45:13 +0000 (UTC) X-FDA: 80393442906.14.EA62EE0 Received: from outbound-smtp37.blacknight.com (outbound-smtp37.blacknight.com [46.22.139.220]) by imf28.hostedemail.com (Postfix) with ESMTP id 47EB1C0019 for ; Wed, 25 Jan 2023 13:45:09 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of mgorman@techsingularity.net designates 46.22.139.220 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674654310; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fmVhptEyIszw83QdpnyRGbIftCOVSR2GH9v6/ZAuPg0=; b=cBsACrgNZ2hN1zC+DoRPZukPZKCWUyEOW+04b5VBLLEqyE4lYKI1PHHDv5vYWLRnAzBWyb DAk4kUo45bxO4AlTBkulas8pNsNh/7c6hMnPyPGT7OBeSw+rL0+a7Ei0Q0c9fFCMmZa5nI +7pOd2KYcFsCfJ1JYZ/4HCZo32/tzPU= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of mgorman@techsingularity.net designates 46.22.139.220 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674654310; a=rsa-sha256; cv=none; b=KsE+ib/A7g2nhUC/wEyatw43db8IhWpElqxbhVuPk4lkguqWMTAhp1kv5AXSQMJHQxaLDj hVviOvtUpRvf9mys1pUbqdtUxu4a06bO6Dl0zoy8kmnk8SFr89IwxS7kYgc9KxLxfMsOoR tDnem117w+rHkr7x0+XUjqTFaHOVwq4= Received: from mail.blacknight.com (pemlinmail05.blacknight.ie [81.17.254.26]) by outbound-smtp37.blacknight.com (Postfix) with ESMTPS id 7277D1F8C for ; Wed, 25 Jan 2023 13:45:08 +0000 (GMT) Received: (qmail 21346 invoked from network); 25 Jan 2023 13:45:08 -0000 Received: from unknown (HELO morpheus.112glenside.lan) (mgorman@techsingularity.net@[84.203.198.246]) by 81.17.254.9 with ESMTPA; 25 Jan 2023 13:45:07 -0000 From: Mel Gorman To: Vlastimil Babka Cc: Andrew Morton , Jiri Slaby , Maxim Levitsky , Michal Hocko , Pedro Falcato , Paolo Bonzini , Chuyi Zhou , Linux-MM , LKML , Mel Gorman Subject: [PATCH 2/4] mm, compaction: Check if a page has been captured before draining PCP pages Date: Wed, 25 Jan 2023 13:44:32 +0000 Message-Id: <20230125134434.18017-3-mgorman@techsingularity.net> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230125134434.18017-1-mgorman@techsingularity.net> References: <20230125134434.18017-1-mgorman@techsingularity.net> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: mqwfcriqkipp385dtns6ok6qgcpze7kx X-Rspamd-Queue-Id: 47EB1C0019 X-HE-Tag: 1674654309-136549 X-HE-Meta: U2FsdGVkX18074Hhr/jJ6NqaXIXNVeFdT75Z9km1d4aSgT6NCs6TapjDxu1dpkRVpWLbL5Dm69LysixWW6kz8mG5fb2jG0bRvWdgR9xDtASekOvfI01Rhj0wVdYa1BWyMr1jFzJAwJRNRIVXTTMNqKvNomuFbozfYtpLPTu7alCUzULqRo0atrXgwYykP9WgLCsGiVABa9l+mj6fFNpzT6c1B9shiL7ljpGBotuI5LL9HrsX8+x2Nty6UrMsQKN++yb0WroDflLuWEoamsuiIqX+HWCWoiLJtIU3BlC/G/eSSgDO9C156iDFR9gp48hOaeUqJhGvqfidtlBPge0bxquYw8Qzzq5rnV7ToUGndqKrldH+Un+7fqc7XOpQiue/zlhMnmcGjWg5/4qUDi9z3R8EWu5SG4ykfu5NmwVOFoSnJFmL5/rbSJi22Lq2DLOj7J9C9OKi+w0ybO94rEcY4iAZ6L22SFUQQuBdfHY6tR6WVPS5/PBe17eVg1OOLsmZ4BwS4wo8kUVQrHiSJKtXvEETynXtQrD4/IoL/54/J/robuSre64UoxeuBztRoQhWpK/7K6B2fakhc/fmeKkhmJVhCWjIpeh91SPHj5pb+wt89kpjjPLNzCAb8AzPXUgA9BNuzTNvd2vn+/avLD/vCOZnXXS+Tg3Dn9kdg1y0aEw2XhPbbjuzeqR64MJYwRJ2l29DPYn8SnsS3sR5oD1KOrUhCO9uZCS84COw4l5FpchV1tFKH/gC7iclxczuzLSu7+pS8lJB+ZvlTBKXmh/2BWqrlLhtQTK/8JchgwMeVp8r5gjOJQ9m1rfSrfj4wKAQysnXUmX/d7CtJjArMAzaKgrjL5i4kDqxqa5UenrSUsEFC3QSkVB9arj1fD2gTPqfiygPKnztdIdZ+v4v8AsdS8wVycZcypc7p3ldIDHxbm+YEB71c8hjYTWsy8udA4D74AyR6S3n3nGeC4h6AJ7 +3A+NBQZ /2IxnllmOOUIN7u93lCtBrDGrzEG92QhLTJauyaPZzcE1jDubRf8Msi5Wi80fhbP28iYxewhO+NRf7zmxYAXhW8lIK/k0Q0V8C58NlXRPr2P5C3zyFIEha03Ac9E0vMJ2Po+ArF7UVbrkhuT2EFCia0HE4gianxNGRjndH+cusCx5JXQ4QUQ+xgmxFCp7ghLt/9jQnBM+yPOiSGO+V7Bk/vLJyoYL7cUcQ8A+7EVaPl1cMw844uFefzRdcS2KajHPMCDXRyzy/OfZpMgGzD+0hjOisqUVqrMoKFH2k8YcI/SWUAwzRkwrWuCVq4CLF98eZRHKDKgnctZgYUn5gZuBPhZmQ64qBi3/6XsGZQo7w8y7+B3q5dlo5uBtiA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If a page has been captured then draining is unnecssary so check first for a captured page. Signed-off-by: Mel Gorman Acked-by: Vlastimil Babka --- mm/compaction.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index c018b0e65720..28711a21a8a2 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -2441,6 +2441,12 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) } } + /* Stop if a page has been captured */ + if (capc && capc->page) { + ret = COMPACT_SUCCESS; + break; + } + check_drain: /* * Has the migration scanner moved away from the previous @@ -2459,12 +2465,6 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) last_migrated_pfn = 0; } } - - /* Stop if a page has been captured */ - if (capc && capc->page) { - ret = COMPACT_SUCCESS; - break; - } } out: From patchwork Wed Jan 25 13:44:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mel Gorman X-Patchwork-Id: 13115652 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8162C54E94 for ; Wed, 25 Jan 2023 13:45:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 835306B0075; Wed, 25 Jan 2023 08:45:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E6846B0078; Wed, 25 Jan 2023 08:45:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6ADC86B007E; Wed, 25 Jan 2023 08:45:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5BF986B0075 for ; Wed, 25 Jan 2023 08:45:22 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 2C62AC0746 for ; Wed, 25 Jan 2023 13:45:22 +0000 (UTC) X-FDA: 80393443284.16.5A75F93 Received: from outbound-smtp25.blacknight.com (outbound-smtp25.blacknight.com [81.17.249.193]) by imf14.hostedemail.com (Postfix) with ESMTP id 4C61810000D for ; Wed, 25 Jan 2023 13:45:20 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf14.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.193 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674654320; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bLWsjn14X33upIrHmANmDBz9DJQXfXX59tK6m2PYat4=; b=krJTm4Xe4OUnJo4R3lTRR5Mj+DR2rE74rHJqT4JFHAf4OlOvKX+uvvnp0fDMl9j3H5XznP /Z6RJqpYMp6tEn/8K4Rg1p5YfjoelFkclLgB+zVjmpEGEFl0rSQvy660hTDH9D7VybkvTv qxLa+i9aB/2mmWDke4JpyTPbMTXmvEk= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf14.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.193 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674654320; a=rsa-sha256; cv=none; b=PrPNFromKCbu90b71m0pwB+Sn2e4b2/S4EozD68/jy6xpWjRJBl6+9IkE1FRXwJhXzI6DZ zvcbcqDFisB6dNyR5m9TL43wGmNHK3M0mqqgtRs4QCQRDBlJrQ7NpjWJLn6kyTWa7/+4y3 XlB7jRoRKoFakUumHKWIdCBOioLjPec= Received: from mail.blacknight.com (pemlinmail05.blacknight.ie [81.17.254.26]) by outbound-smtp25.blacknight.com (Postfix) with ESMTPS id BF0A742020 for ; Wed, 25 Jan 2023 13:45:18 +0000 (GMT) Received: (qmail 21935 invoked from network); 25 Jan 2023 13:45:18 -0000 Received: from unknown (HELO morpheus.112glenside.lan) (mgorman@techsingularity.net@[84.203.198.246]) by 81.17.254.9 with ESMTPA; 25 Jan 2023 13:45:18 -0000 From: Mel Gorman To: Vlastimil Babka Cc: Andrew Morton , Jiri Slaby , Maxim Levitsky , Michal Hocko , Pedro Falcato , Paolo Bonzini , Chuyi Zhou , Linux-MM , LKML , Mel Gorman Subject: [PATCH 3/4] mm, compaction: Finish scanning the current pageblock if requested Date: Wed, 25 Jan 2023 13:44:33 +0000 Message-Id: <20230125134434.18017-4-mgorman@techsingularity.net> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230125134434.18017-1-mgorman@techsingularity.net> References: <20230125134434.18017-1-mgorman@techsingularity.net> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 4C61810000D X-Rspam-User: X-Stat-Signature: ya5wtmobhecanewqakzcg3gpjjj4iotw X-HE-Tag: 1674654320-327943 X-HE-Meta: U2FsdGVkX1+1HTIuFlwQYi0F43U9NEC9xDyPHvdNfjONPtcvLlNxWM9ySVzxR1zwtaBVuv4d4TWhSRcf9Lt+TUEo4ZqQQarvu8Ql90OYcIYdEgKWriwUM/VpORu0IXVZwfbD7RV3s6Q7NrwL5jFcCMOJBblWM2Bx7gLbGzIaAuuE3lDjg4GoZvf2ygoxNxf6rEZQ4Ha8GxGosD01j8GLK3KXyfdTMUHaWW8NPsUATCGCpdIqeFYSKqd5s9kGW0Y5JuuxUIW8LTE/uojfEAlmevGx3CRyDrj3VJW6EVoWZr140Q/mt4KG1rvfebQ1OkIk8xWNDevJgmeR+9pPPA3xR/uNtLgQNrzB+PaopHxuCg8cuCkRmEUFdypFI2uouHDZ5e74fNPXi3ajkz3aOGul7MEA2OTp1uo2M4UgkeHYdzkuEkZ6vKCw5cMFcZlRozEacqSD6E1SAiVQeCLf2SFv2nQumflZWu2zdVJO7hoxKUCMWt3JZ38oy38EdYRgfp/gHH1Z94uTm7AZdvxR80fHa0W9TUUqHDY5n3OtJCiW3kkZOQHP0P7T94xJnI82WEHWcnpIMMVv0NNNXFMCbpnLXZ12PEJIJaMj3J7NlRbVxjSjDnyll22u0B0sMpaxC/agBPpILDUEHZRhS770AWlDZyLWojcWEjuUeZDmAnN8tT1vp23XbdEWxDJp0Oy2+RfGLrj4y7y4G1nmB4Yef0YgfmGweg+GYvuUzDHK25ZwbXE1/yFSfjQSaR0+gv8cQu76uWePmE1kLi8KmeJnOKZyvizvsvmqLguWWVkzn5tNv+ydzkwM5ZAbv8R4tTbMjZXs7BVBIWv0dRzxFz6lZB5XgJRh0aFA44qAbL/Z/KXmAFHvCYD7Y0y9FY3I/fpqPRpsy1nWroqwyK2XPkjYCogH4bYiD54tP5wVSPRm72ZcxvUyxB8lInkrgFETdsLhJVgNgrsBsIrZ19jZCxHjsHk B+8Snfv3 EOc4tVjr4QrB0gdxjT9TSIVboif0l7KaQnUnZb4+qQTIzO3xve8fJSaTEBaKaEo31SEvIcFnv3wtppE9kaoW/rPsEcOQ5QJIiS7bh/rgbHpl9nac27ieEZcQd7MfImP2nzVOvm3xfCj9Bol0ovD4DnjOq4V7nGru7DQEfo41JeOGfYN+IO40ZEnoSkwseizz1TIAqpWwqTfY0hvLfdSHYoJGhONFLbQHyZm5OetDyxEhVifLRK5klwZwWxql3jCXQLNeVq8ll9PeXQLsM6Fcw15LzW4ZPq4Nmv9opKeMQcw9wvF0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: cc->finish_pageblock is set when the current pageblock should be rescanned but fast_find_migrateblock can select an alternative block. Disable fast_find_migrateblock when the current pageblock scan should be completed. Signed-off-by: Mel Gorman Acked-by: Vlastimil Babka --- mm/compaction.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/mm/compaction.c b/mm/compaction.c index 28711a21a8a2..4b3a0238879c 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1762,6 +1762,13 @@ static unsigned long fast_find_migrateblock(struct compact_control *cc) if (cc->ignore_skip_hint) return pfn; + /* + * If the pageblock should be finished then do not select a different + * pageblock. + */ + if (cc->finish_pageblock) + return pfn; + /* * If the migrate_pfn is not at the start of a zone or the start * of a pageblock then assume this is a continuation of a previous From patchwork Wed Jan 25 13:44:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mel Gorman X-Patchwork-Id: 13115653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F01CC27C76 for ; Wed, 25 Jan 2023 13:45:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B91C46B0078; Wed, 25 Jan 2023 08:45:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B41D66B007E; Wed, 25 Jan 2023 08:45:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A30AD6B0080; Wed, 25 Jan 2023 08:45:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9362C6B0078 for ; Wed, 25 Jan 2023 08:45:32 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6781914040D for ; Wed, 25 Jan 2023 13:45:32 +0000 (UTC) X-FDA: 80393443704.21.6834D6B Received: from outbound-smtp58.blacknight.com (outbound-smtp58.blacknight.com [46.22.136.242]) by imf23.hostedemail.com (Postfix) with ESMTP id 5807A140018 for ; Wed, 25 Jan 2023 13:45:30 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf23.hostedemail.com: domain of mgorman@techsingularity.net designates 46.22.136.242 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674654330; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dBusSDQMc1Av9eoZM4nbPmSEyQIlVUn7Q/Wo0slBnwQ=; b=uiya4Jfzh9iacfUfduNRhpJVgdVI2dRW3RxcZOdJCt3bdJe8pB8SofTvWsQ/Xk+3CSMtGp CZq9WRsara8A1OSVIvyO+4zzxQ5n0nRDD2E0zqEWn9EiR7oEXP3pezlqKWwdteJ9noDwnb ARTeIMiXVEx6qNL5g2BMG8OTeE58S1M= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf23.hostedemail.com: domain of mgorman@techsingularity.net designates 46.22.136.242 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674654330; a=rsa-sha256; cv=none; b=eLixT969VQa5dZDfD9MQobGWHln9TVmxHBDybrZAODF7OAL8C2JGmt7Na0BU/6yygdIEa1 1G8L9+RL3FaCY+2jBCDLwZIT87MgB4LhrC6ikPJIC1PAuFcHI+NT/CPLzLOtt9WIpVi76b zIQw+jomON4pY8kr/gL/WouygjF6VEg= Received: from mail.blacknight.com (pemlinmail05.blacknight.ie [81.17.254.26]) by outbound-smtp58.blacknight.com (Postfix) with ESMTPS id 05CB2FAB23 for ; Wed, 25 Jan 2023 13:45:29 +0000 (GMT) Received: (qmail 22502 invoked from network); 25 Jan 2023 13:45:28 -0000 Received: from unknown (HELO morpheus.112glenside.lan) (mgorman@techsingularity.net@[84.203.198.246]) by 81.17.254.9 with ESMTPA; 25 Jan 2023 13:45:28 -0000 From: Mel Gorman To: Vlastimil Babka Cc: Andrew Morton , Jiri Slaby , Maxim Levitsky , Michal Hocko , Pedro Falcato , Paolo Bonzini , Chuyi Zhou , Linux-MM , LKML , Mel Gorman Subject: [PATCH 4/4] mm, compaction: Finish pageblocks on complete migration failure Date: Wed, 25 Jan 2023 13:44:34 +0000 Message-Id: <20230125134434.18017-5-mgorman@techsingularity.net> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230125134434.18017-1-mgorman@techsingularity.net> References: <20230125134434.18017-1-mgorman@techsingularity.net> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 5807A140018 X-Stat-Signature: 6hwocbnnxjiwyr1k356ir9h1fn1x4j6u X-Rspam-User: X-HE-Tag: 1674654330-203337 X-HE-Meta: U2FsdGVkX18tmmndoJ1tPr9Cl2gJQ0uJ37kLNWV/zgZM3LeRZbmVKjB/ynjPtnJHkIQSKfeXnBHZCVBJdquQrsQz5//+zpIIt8EfHsoPPhtv3a3odFoXTdZpe35FeujcUvU0oaNRMiDd/wfKnR81Aqy0GzYsckRrOYKBpteJ53WA9E9PBDdQh5KD11VC/cKxF4ySsGllu57ojk27MDD2leE5Ve6TZnPUUlwoz7fTB/fQJih/7AtEiXEVUDYpyNL+eyJ9ENTdbzYzOla1h+vFkxzJbF6ddMelSYBPPpec/6mawG7SZtWCt+TRJTf+cj1ei0Nzvgpe07J7AZtnmPqNSZnj/mj5DVHVw+a6MHs7dLWv6bFAd3FMHOx//Wa7poAGfJEEJH9LvCQqOBjj6Z2/LvQc8Nvh+jKng6hEAE1hiAQyX+1fx/CcmLRO5hSFqe5Zt/YHsSbAxeB/+ZYibCB/0Ws2J+Gn8wc6j/xsu9l+nkraauAGZMQxcEkaamSrNIKPtM8/QisOFq5ZC4CiO8DyWvkuKgwhEmeLAb2Jv4cckQDYMUZK5aTjZMyH7rK90zobtjK7ixlYkNcVBON6h4HpU3L7anDHcSa4OKJnGjFJKopn1jo3ODDQJTHHMAAyBNTjlS1dKp75TjDyHZ4azDGyCNZzr0FBjgXUrfRoHGKGRL1Pghl3kjMUYcNU7I+YK0tfEkZjheuPPNoH4DO2fFFVi+VN/20cUeNnns8hgYpnG2QCpjhBvYGbzP3GsrdI2SRb1W4OYwddme5Xlww4R1Vl8O4K/x4vdr5RufFIHRR37twUfsqGEMvrncovWRBcvCN6PdTyg2bJC/S07WvfY1E0c1ZQIZguhOHPul5R/pfjCHon3rlk5Gjw38o287mXMzAqWj06XlIQerafDEY1V3BgU6YSqgIqbMHWSh2j+vZ97qY2ePEV0BfEs+66bvD9PUAv2OENV61pmR/O26o/1GK 34p9sZw6 Ntipo+3yC6n0TAmeFJxMTZvmJpsVstmnmH6lOQSi4ixSclLb09YnvF2CKliafgIovZpM8MgVO8TweRaM2fXXM6erA5J1T8s0wnOgR2KPj3JD5F/e1RmfVtZS9E5M95vnSdwnGuyV5qth+H2A7XKtzVUF3BoJtd6TG53DWMz6z+37WbiRtgYE7rEj+fLn52mQB0hF4tn/z4XGQ2g3iqJXF0wyidR5kzvL+9iGOq1K8TjxoZyNWl5nWRjOEV1ikdTayJMAcKpfJsO8F+bgdoROFWcxj502yTbyPRT1tS5MTnvIQEralCivSh5C6nQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Commit 7efc3b726103 ("mm/compaction: fix set skip in fast_find_migrateblock") address an issue where a pageblock selected by fast_find_migrateblock() was ignored. Unfortunately, the same fix resulted in numerous reports of khugepaged or kcompactd stalling for long periods of time or consuming 100% of CPU. Tracing showed that there was a lot of rescanning between a small subset of pageblocks because the conditions for marking the block skip are not met. The scan is not reaching the end of the pageblock because enough pages were isolated but none were migrated successfully. Eventually it circles back to the same block. Pageblock skip tracking tries to minimise both latency and excessive scanning but tracking exactly when a block is fully scanned requires an excessive amount of state. This patch forcibly rescans a pageblock when all isolated pages fail to migrate even though it could be for transient reasons such as page writeback or page dirty. This will sometimes migrate too many pages but pageblocks will be marked skip and forward progress will be made. "Usemen" from the mmtests configuration workload-usemem-stress-numa-compact was used to stress compaction. The compaction trace events were recorded using a 6.2-rc5 kernel that includes commit 7efc3b726103 and count of unique ranges were measured. The top 5 ranges were 3076 range=(0x10ca00-0x10cc00) 3076 range=(0x110a00-0x110c00) 3098 range=(0x13b600-0x13b800) 3104 range=(0x141c00-0x141e00) 11424 range=(0x11b600-0x11b800) While this workload is very different than what the bugs reported, the pattern of the same subset of blocks being repeatedly scanned is observed. At one point, *only* the range range=(0x11b600 ~ 0x11b800) was scanned for 2 seconds. 14 seconds passed between the first migration-related event and the last. With the series applied including this patch, the top 5 ranges were 1 range=(0x11607e-0x116200) 1 range=(0x116200-0x116278) 1 range=(0x116278-0x116400) 1 range=(0x116400-0x116424) 1 range=(0x116424-0x116600) Only unique ranges were scanned and the time between the first migration-related event was 0.11 milliseconds. Fixes: 7efc3b726103 ("mm/compaction: fix set skip in fast_find_migrateblock") Signed-off-by: Mel Gorman --- mm/compaction.c | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 4b3a0238879c..937ec2f05f2c 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -2394,6 +2394,7 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) cc->finish_pageblock = true; } +rescan: switch (isolate_migratepages(cc)) { case ISOLATE_ABORT: ret = COMPACT_CONTENDED; @@ -2436,15 +2437,28 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) goto out; } /* - * We failed to migrate at least one page in the current - * order-aligned block, so skip the rest of it. + * If an ASYNC or SYNC_LIGHT fails to migrate a page + * within the current order-aligned block, scan the + * remainder of the pageblock. This will mark the + * pageblock "skip" to avoid rescanning in the near + * future. This will isolate more pages than necessary + * for the request but avoid loops due to + * fast_find_migrateblock revisiting blocks that were + * recently partially scanned. */ - if (cc->direct_compaction && - (cc->mode == MIGRATE_ASYNC)) { - cc->migrate_pfn = block_end_pfn( - cc->migrate_pfn - 1, cc->order); - /* Draining pcplists is useless in this case */ - last_migrated_pfn = 0; + if (cc->direct_compaction && !cc->finish_pageblock && + (cc->mode < MIGRATE_SYNC)) { + cc->finish_pageblock = true; + + /* + * Draining pcplists does not help THP if + * any page failed to migrate. Even after + * drain, the pageblock will not be free. + */ + if (cc->order == COMPACTION_HPAGE_ORDER) + last_migrated_pfn = 0; + + goto rescan; } }