From patchwork Fri May 19 12:39:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13248283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83E65C7EE2E for ; Fri, 19 May 2023 12:40:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D1958900006; Fri, 19 May 2023 08:40:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CEB76900007; Fri, 19 May 2023 08:40:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BAF4A900006; Fri, 19 May 2023 08:40:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id AD7E7900003 for ; Fri, 19 May 2023 08:40:12 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 76BC41A0A4E for ; Fri, 19 May 2023 12:40:12 +0000 (UTC) X-FDA: 80806962264.25.F200ED6 Received: from mail-ed1-f47.google.com (mail-ed1-f47.google.com [209.85.208.47]) by imf07.hostedemail.com (Postfix) with ESMTP id 880EA4000F for ; Fri, 19 May 2023 12:40:09 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=UqY0Q1Qn; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf07.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.208.47 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684500009; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eLUm7WZINfPJ8EnyZppR+QB8ZeshYnpRD1lp9jOg8YM=; b=tTwJsiIM7ar5KSYYboTKpRjz22VavklEHGVz2Sf82M8qblaseCPbl9/JtmUowvhQ3fHpoy nKpeglsP2ByYvaz9h67OqDlIOEhlFt3LH2BlNzWyeVQOz4KbEFHAN9Wh/KizqhLg2efSsX FBNExD81Oak2nE33P2CthaOUNJOYtWY= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=UqY0Q1Qn; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf07.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.208.47 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684500009; a=rsa-sha256; cv=none; b=6zTYReOrYnPZFd45T226H2ajKcWf3CxnMvytija9e9ufYhWzZbIrF6ryeg1Zz9YByQEsXK eUTa5m/+Ggo+2FvREC4LXv1K1z5M+6X5SxKVYEnHmZkQEPMsqA5b3h01OWA1rvCdfaEELo YlmUAzLy4BSy83VGbDtGBxd4KbfhRig= Received: by mail-ed1-f47.google.com with SMTP id 4fb4d7f45d1cf-50bcb229adaso5802870a12.2 for ; Fri, 19 May 2023 05:40:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20221208.gappssmtp.com; s=20221208; t=1684500008; x=1687092008; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eLUm7WZINfPJ8EnyZppR+QB8ZeshYnpRD1lp9jOg8YM=; b=UqY0Q1QneRWgaChbAJgyXkbGCj5MIeuFO1Mmxqo9b0QVAxAtyzG0jFAVliXTITC41q oC79XjrJW9vlGUQVQJWPUcF9FCRAbWfWtVRthlUqELkW1+wmsNFisu0FKzt8Vuy7O/0r gyMNuX0yd/37Jh6HhJKs7jW2i8YAXmc5zUZTmN3Rlmuo6uaaRSkBGojlNeDxEigi5TXY wkeYQj8gFL6V8RbvUzItTEhn0gtfrZrDjFrQqIDzdEz+gAAc8yjZKn2/LyNCcfsdWYPq V/bJWcjzyGeL1UMuy0CC5C2DgDrohlE58IRXywwbSfuUET9z+ZmQWIUXgcTO6ixIOW9N daFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684500008; x=1687092008; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eLUm7WZINfPJ8EnyZppR+QB8ZeshYnpRD1lp9jOg8YM=; b=RQnz7bv7FYy1/qeNDKM/4dmrzOHukfTrzqUa4tqJsihIzZqQad1GHhdInt/FsDVWDN nFnnZvlQXqO7JubnpvLJhsEwWHstnF8zOjXiAC1XrkVPu5oUrL5dMlW6FfBNKb672UcY ZYkHFWwbg1ajPPpTwH/ZwMus2RKWBRieYOjZo7Sh6rkQymNtT70VS8emWzq4pQ3V4kSr Ueog1I1Z3A536B186HRdYv3mVSwAhsmoIxTR3UQG/srjelx7/9A8ZddhhEoKBYuCgukm bQMymwzVHlDPkTw430s9R8Uf1T0+o2MWmQFxZEvVj2kombtOSTplqIsDEFl8C2jaZhrl rMtA== X-Gm-Message-State: AC+VfDwsWYN6BX4l5kYPBTVvf+5kvfQrzWPlL/+II50ThxA0qdbFpULF a0n3XNsE4g9WBEkh+zc+Mm+tuw== X-Google-Smtp-Source: ACHHUZ7oj3yMtwF1chSvwKUqZhCd0vlmhEuYEkCw7mv9MYs7nSCkZ05x4A7DYg+PKersqbn5+LRfWg== X-Received: by 2002:a17:907:2cc7:b0:965:fb87:4215 with SMTP id hg7-20020a1709072cc700b00965fb874215mr1552073ejc.15.1684500008176; Fri, 19 May 2023 05:40:08 -0700 (PDT) Received: from localhost ([2a02:8070:6389:7d40:e266:3092:9afb:a7b1]) by smtp.gmail.com with ESMTPSA id m23-20020a1709061ed700b0095337c5da35sm2295229ejj.15.2023.05.19.05.40.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 May 2023 05:40:08 -0700 (PDT) From: Johannes Weiner To: Andrew Morton Cc: Mel Gorman , Vlastimil Babka , Michal Hocko , linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 1/5] mm: compaction: remove compaction result helpers Date: Fri, 19 May 2023 14:39:55 +0200 Message-Id: <20230519123959.77335-2-hannes@cmpxchg.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230519123959.77335-1-hannes@cmpxchg.org> References: <20230519123959.77335-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 880EA4000F X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 54kcje54g4smmqcet6hmasueoiek67d3 X-HE-Tag: 1684500009-773829 X-HE-Meta: U2FsdGVkX19VMg672rtS/jVSEh6MXKmHiMgUNgqxdiNZ3O49SI1i3pqpRBtoVMaoLmThTgn5JIxKPdwvPzRCsm7PudgiAQ9ohkIi3my8/+PZC25vbqi2rnasfzWbyfyc3frupUiYrmVgjGYQ3ba+9COzA6LO+gSer3OPpSMeY532C+bIV+XDxEsZ88dR2p2FbAmH637x2Fpd5YhHwxb1ZqlYksKl5Bg1f+Bcglv/PDPvO3oxt60iWIXf9haqWhjET1sicDMBvQlgLZDyxG5d07cdLmWyHV6EpU4Ypk6eXc8moXajjlFRTB9PsBn68otFUf8t+Yg2bcbj9exaNq7DQ2H/LUSq9HOPDoqoAxajkaPzED+rzKzIjNahpSoK8H3GJvNWDpm5peGX9jTc8ZTc2G0dl2Sg10BSd2zQUN+b/4fn4S14mXANqEriURTInayw35wYgrEYR2LUCgac1oCESjHn3gpK0LMU1hp5FnbzgsvP7MXQ8Ou2iglT/xuC9xYAdCwNgvE1wipfhnZ2EXMiQw8YtaJTjfGS796xC+6nMk4I+WKd/pXgu4vJj6AspjGSpTsEd8nL3S13+xR7S/NeyMcUBbMR9W920J8Gm7Pyx5QsWNyE16Yx4KFWsdY3Oq0vcztY2SrmxlDiN1UDFMhhZ/nQIQwJDqjiz7uLPjufSldJI3IRPNSLMI5QEhX1JJ3PprDdycYZHobtbd+Mp+bPu2wnE3R+uJ77tE7PrJ1IVCVZ/x67h/XUiFmSZmAWiXe6FLZ4lk/bJMKJpdbEgmPrUlsDTT0O6qjZxS0UviODqadCC583A2m/CPXoeuoOkFlOUH0XtyUcsT6lI32rcGtMZk80nAggWQvXN2NaA+Yzj9VXJRrMFPbMfjnInyMxwKLaoxgVfBz7E3MhZBocohAFpUBjImsVbaSbeBGIbNcMFj//U9cB3dGJ1eea9BnGN83TeoIqaFDzPydDKJ4AOTI wRQrKQ0k VChAchb19fHVnrwwVEz2SwFHcghuMbEmn53nbZqNHK8kT9Na/uDB97dzsc4rijcH29JKnC+dBNfPvsws5WpDwP1cOdlFlivoJJOkUtnnGd4RGaX4p/uzkrDDjH/Vn93FrRW3OMTFtWEx8p4NfiS9Qh8fRddwxhEmJDG1vVohfTPxXuVwpRBRPBUQdmma44pZnCoA+iJoYa5itc/nGoDRWawcmy6QPdA81+m3M5ngfxtDJb9Gn19XQmFE/GYGR+aSR5gV7gpQ148uvyszkcyLJHyqX5AKrBpYqlV9/tVs4Wx7j7Fb97ZLCKjh3DXhBqRGLRFZQHudxFU+L4GTmP5miOUWBjG5rtl/PgFHAXe7EXEcq/ZhmGemLbvf0vpgxCiPf8XuCMYaHHFEvibqXvK7FbVBP9wcjngymTCjlACg8AEd6yLDtz1P7H9b4yiWjck9t5IR1ZZO5nQ5/QEj3Qg+KBFmViQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The compaction result helpers encode quirks that are specific to the allocator's retry logic. E.g. COMPACT_SUCCESS and COMPACT_COMPLETE actually represent failures that should be retried upon, and so on. I frequently found myself pulling up the helper implementation in order to understand and work on the retry logic. They're not quite clean abstractions; rather they split the retry logic into two locations. Remove the helpers and inline the checks. Then comment on the result interpretations directly where the decision making happens. Signed-off-by: Johannes Weiner Acked-by: Vlastimil Babka --- include/linux/compaction.h | 92 ---------------------------------- include/trace/events/mmflags.h | 4 +- mm/page_alloc.c | 30 ++++++----- 3 files changed, 19 insertions(+), 107 deletions(-) diff --git a/include/linux/compaction.h b/include/linux/compaction.h index a6e512cfb670..1f0328a2ba48 100644 --- a/include/linux/compaction.h +++ b/include/linux/compaction.h @@ -95,78 +95,6 @@ extern enum compact_result compaction_suitable(struct zone *zone, int order, extern void compaction_defer_reset(struct zone *zone, int order, bool alloc_success); -/* Compaction has made some progress and retrying makes sense */ -static inline bool compaction_made_progress(enum compact_result result) -{ - /* - * Even though this might sound confusing this in fact tells us - * that the compaction successfully isolated and migrated some - * pageblocks. - */ - if (result == COMPACT_SUCCESS) - return true; - - return false; -} - -/* Compaction has failed and it doesn't make much sense to keep retrying. */ -static inline bool compaction_failed(enum compact_result result) -{ - /* All zones were scanned completely and still not result. */ - if (result == COMPACT_COMPLETE) - return true; - - return false; -} - -/* Compaction needs reclaim to be performed first, so it can continue. */ -static inline bool compaction_needs_reclaim(enum compact_result result) -{ - /* - * Compaction backed off due to watermark checks for order-0 - * so the regular reclaim has to try harder and reclaim something. - */ - if (result == COMPACT_SKIPPED) - return true; - - return false; -} - -/* - * Compaction has backed off for some reason after doing some work or none - * at all. It might be throttling or lock contention. Retrying might be still - * worthwhile, but with a higher priority if allowed. - */ -static inline bool compaction_withdrawn(enum compact_result result) -{ - /* - * If compaction is deferred for high-order allocations, it is - * because sync compaction recently failed. If this is the case - * and the caller requested a THP allocation, we do not want - * to heavily disrupt the system, so we fail the allocation - * instead of entering direct reclaim. - */ - if (result == COMPACT_DEFERRED) - return true; - - /* - * If compaction in async mode encounters contention or blocks higher - * priority task we back off early rather than cause stalls. - */ - if (result == COMPACT_CONTENDED) - return true; - - /* - * Page scanners have met but we haven't scanned full zones so this - * is a back off in fact. - */ - if (result == COMPACT_PARTIAL_SKIPPED) - return true; - - return false; -} - - bool compaction_zonelist_suitable(struct alloc_context *ac, int order, int alloc_flags); @@ -185,26 +113,6 @@ static inline enum compact_result compaction_suitable(struct zone *zone, int ord return COMPACT_SKIPPED; } -static inline bool compaction_made_progress(enum compact_result result) -{ - return false; -} - -static inline bool compaction_failed(enum compact_result result) -{ - return false; -} - -static inline bool compaction_needs_reclaim(enum compact_result result) -{ - return false; -} - -static inline bool compaction_withdrawn(enum compact_result result) -{ - return true; -} - static inline void kcompactd_run(int nid) { } diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index b63e7c0fbbe5..1478b9dd05fa 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -223,8 +223,8 @@ IF_HAVE_VM_SOFTDIRTY(VM_SOFTDIRTY, "softdirty" ) \ #define compact_result_to_feedback(result) \ ({ \ enum compact_result __result = result; \ - (compaction_failed(__result)) ? COMPACTION_FAILED : \ - (compaction_withdrawn(__result)) ? COMPACTION_WITHDRAWN : COMPACTION_PROGRESS; \ + (__result == COMPACT_COMPLETE) ? COMPACTION_FAILED : \ + (__result == COMPACT_SUCCESS) ? COMPACTION_PROGRESS : COMPACTION_WITHDRAWN; \ }) #define COMPACTION_FEEDBACK \ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 47421bedc12b..5a84a0bebc37 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3768,35 +3768,39 @@ should_compact_retry(struct alloc_context *ac, int order, int alloc_flags, if (fatal_signal_pending(current)) return false; - if (compaction_made_progress(compact_result)) + /* + * Compaction managed to coalesce some page blocks, but the + * allocation failed presumably due to a race. Retry some. + */ + if (compact_result == COMPACT_SUCCESS) (*compaction_retries)++; /* - * compaction considers all the zone as desperately out of memory - * so it doesn't really make much sense to retry except when the + * All zones were scanned completely and still no result. It + * doesn't really make much sense to retry except when the * failure could be caused by insufficient priority */ - if (compaction_failed(compact_result)) + if (compact_result == COMPACT_COMPLETE) goto check_priority; /* - * compaction was skipped because there are not enough order-0 pages - * to work with, so we retry only if it looks like reclaim can help. + * Compaction was skipped due to a lack of free order-0 + * migration targets. Continue if reclaim can help. */ - if (compaction_needs_reclaim(compact_result)) { + if (compact_result == COMPACT_SKIPPED) { ret = compaction_zonelist_suitable(ac, order, alloc_flags); goto out; } /* - * make sure the compaction wasn't deferred or didn't bail out early - * due to locks contention before we declare that we should give up. - * But the next retry should use a higher priority if allowed, so - * we don't just keep bailing out endlessly. + * If compaction backed due to being deferred, due to + * contended locks in async mode, or due to scanners meeting + * after a partial scan, retry with increased priority. */ - if (compaction_withdrawn(compact_result)) { + if (compact_result == COMPACT_DEFERRED || + compact_result == COMPACT_CONTENDED || + compact_result == COMPACT_PARTIAL_SKIPPED) goto check_priority; - } /* * !costly requests are much more important than __GFP_RETRY_MAYFAIL