From patchwork Tue Feb 15 14:51:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mel Gorman X-Patchwork-Id: 12747202 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D988CC433EF for ; Tue, 15 Feb 2022 14:51:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 739C26B007D; Tue, 15 Feb 2022 09:51:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6E8D86B007E; Tue, 15 Feb 2022 09:51:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5B1886B0080; Tue, 15 Feb 2022 09:51:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.25]) by kanga.kvack.org (Postfix) with ESMTP id 47E5D6B007D for ; Tue, 15 Feb 2022 09:51:46 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id ED4136181B for ; Tue, 15 Feb 2022 14:51:45 +0000 (UTC) X-FDA: 79145303412.03.D0D43C8 Received: from outbound-smtp03.blacknight.com (outbound-smtp03.blacknight.com [81.17.249.16]) by imf26.hostedemail.com (Postfix) with ESMTP id 85E8B140003 for ; Tue, 15 Feb 2022 14:51:45 +0000 (UTC) Received: from mail.blacknight.com (pemlinmail04.blacknight.ie [81.17.254.17]) by outbound-smtp03.blacknight.com (Postfix) with ESMTPS id 06529C0B2B for ; Tue, 15 Feb 2022 14:51:43 +0000 (GMT) Received: (qmail 14877 invoked from network); 15 Feb 2022 14:51:42 -0000 Received: from unknown (HELO stampy.112glenside.lan) (mgorman@techsingularity.net@[84.203.17.223]) by 81.17.254.9 with ESMTPA; 15 Feb 2022 14:51:42 -0000 From: Mel Gorman To: Andrew Morton Cc: Aaron Lu , Dave Hansen , Vlastimil Babka , Michal Hocko , Jesper Dangaard Brouer , LKML , Linux-MM , Mel Gorman Subject: [PATCH 2/5] mm/page_alloc: Track range of active PCP lists during bulk free Date: Tue, 15 Feb 2022 14:51:08 +0000 Message-Id: <20220215145111.27082-3-mgorman@techsingularity.net> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220215145111.27082-1-mgorman@techsingularity.net> References: <20220215145111.27082-1-mgorman@techsingularity.net> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 85E8B140003 X-Rspam-User: Authentication-Results: imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.16 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net; dmarc=none X-Stat-Signature: 3mauw7orgce6gqmta8yr7a3jmh3zdz9b X-HE-Tag: 1644936705-818955 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: free_pcppages_bulk() frees pages in a round-robin fashion. Originally, this was dealing only with migratetypes but storing high-order pages means that there can be many more empty lists that are uselessly checked. Track the minimum and maximum active pindex to reduce the search space. Signed-off-by: Mel Gorman --- mm/page_alloc.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 08de32cfd9bb..c5110fdeb115 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1450,6 +1450,8 @@ static void free_pcppages_bulk(struct zone *zone, int count, struct per_cpu_pages *pcp) { int pindex = 0; + int min_pindex = 0; + int max_pindex = NR_PCP_LISTS - 1; int batch_free = 0; int nr_freed = 0; unsigned int order; @@ -1478,10 +1480,17 @@ static void free_pcppages_bulk(struct zone *zone, int count, if (++pindex == NR_PCP_LISTS) pindex = 0; list = &pcp->lists[pindex]; - } while (list_empty(list)); + if (!list_empty(list)) + break; + + if (pindex == max_pindex) + max_pindex--; + if (pindex == min_pindex) + min_pindex++; + } while (1); /* This is the only non-empty list. Free them all. */ - if (batch_free == NR_PCP_LISTS) + if (batch_free >= max_pindex - min_pindex) batch_free = count; order = pindex_to_order(pindex);