From patchwork Fri Mar 4 17:02:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 12769476 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 667A3C433F5 for ; Fri, 4 Mar 2022 17:02:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CBE378D0002; Fri, 4 Mar 2022 12:02:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C45958D0001; Fri, 4 Mar 2022 12:02:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE7C08D0002; Fri, 4 Mar 2022 12:02:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.27]) by kanga.kvack.org (Postfix) with ESMTP id 9E9098D0001 for ; Fri, 4 Mar 2022 12:02:21 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 6D16F120A00 for ; Fri, 4 Mar 2022 17:02:21 +0000 (UTC) X-FDA: 79207322082.05.1A14F29 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) by imf21.hostedemail.com (Postfix) with ESMTP id E16431C001D for ; Fri, 4 Mar 2022 17:02:20 +0000 (UTC) Received: by mail-pf1-f178.google.com with SMTP id s11so8057364pfu.13 for ; Fri, 04 Mar 2022 09:02:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=m9DN8GZLnqfgQbJETPdoOENIKXarw30wjGXDiTZV/WI=; b=cXF7pznqQ+nkIi0dJDdpaB4BbNgUDe53v7wcZZhFbbLznD3OsURa5kfTqQ+T3G7yG5 gjcjozJuDDA7oAJRnhV7tiRyNMEn2JlaG1pRvk4dIp/On6jkSiSS2yoL29SOYXUll/hq j0cLZcWuOKiEUGhTk8jQ5E127PzBwO4L4GaOpLmjYQKx5XIUQ/8r0gIMK4XH+ULLWWrZ JQInpJ/j3GKVvUaLpxDsnZrfC+H6Vbdu9PcOGLpjW78RNf9fQj3FmWKZojcVLbbVd61/ HovEuaUsh+1Wk3nsDAcrQdEqMrYNeOcwajgLgwraZ1FkDYa6JUmiV/EFJioupZiKmBP2 DBvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=m9DN8GZLnqfgQbJETPdoOENIKXarw30wjGXDiTZV/WI=; b=JmOpgKEwRY+Dw2B8hdEJFSpADXYKPg1ThcqkccJh840KDJ/vPsfJjM+Gub8fi/1/wC yHXEA78d7M1mXHxRAwyxIyXSYseTahTpePHkvaXFtJIOvsnGj8fvaTxtM7OStrWob1hL z4x08CmiyBxNfCaK7nwc3WlDVwaSuqg4kU85rvOTbo//2ZnI1bBM7nBuImgCCWJAuuEW cF3jO3dtWzY+a/UUD7nf78XXzQ5R6TcdFhtf6ryOGi3whBKlIb65+PV3VfYpTc0L1p6i 2Bcb1KS0+YdurJQpgUquFCOLq/qrMINgRN7/zRJNhsilb8Sy0khMDaRtOB6qziulpUB6 6z3w== X-Gm-Message-State: AOAM530exSzrxHFO0cdIUvfmdung7bSnBUB8uAB7D2pP2lTsCtnGondF rCA+p9kJFJHCw1BHSuqK0O4= X-Google-Smtp-Source: ABdhPJw1+OU2H8gT2uzN0UKOWZeQRmUsnXEp21lDc2Wxz8YGSRZqFGpgM3kMpvTY+orF8Fq8lDlkBA== X-Received: by 2002:a05:6a00:16d3:b0:4cb:51e2:1923 with SMTP id l19-20020a056a0016d300b004cb51e21923mr43816386pfc.7.1646413339783; Fri, 04 Mar 2022 09:02:19 -0800 (PST) Received: from edumazet1.svl.corp.google.com ([2620:15c:2c4:201:5388:c313:5e37:a261]) by smtp.gmail.com with ESMTPSA id b9-20020a056a000cc900b004f3581ae086sm6947038pfv.16.2022.03.04.09.02.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Mar 2022 09:02:19 -0800 (PST) From: Eric Dumazet To: Andrew Morton Cc: linux-kernel , linux-mm , Eric Dumazet , Eric Dumazet , Mel Gorman , Vlastimil Babka , Michal Hocko , Shakeel Butt , Wei Xu , Greg Thelen , Hugh Dickins , David Rientjes Subject: [PATCH v2] mm/page_alloc: call check_new_pages() while zone spinlock is not held Date: Fri, 4 Mar 2022 09:02:15 -0800 Message-Id: <20220304170215.1868106-1-eric.dumazet@gmail.com> X-Mailer: git-send-email 2.35.1.616.g0bdcbb4464-goog MIME-Version: 1.0 X-Rspamd-Queue-Id: E16431C001D X-Stat-Signature: 1z164mo8hzm7q8g4w49f7bgdpqb3bjzi Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=cXF7pznq; spf=pass (imf21.hostedemail.com: domain of eric.dumazet@gmail.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=eric.dumazet@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1646413340-336737 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Eric Dumazet For high order pages not using pcp, rmqueue() is currently calling the costly check_new_pages() while zone spinlock is held, and hard irqs masked. This is not needed, we can release the spinlock sooner to reduce zone spinlock contention. Note that after this patch, we call __mod_zone_freepage_state() before deciding to leak the page because it is in bad state. v2: We need to keep interrupts disabled to call __mod_zone_freepage_state() Signed-off-by: Eric Dumazet Cc: Mel Gorman Cc: Vlastimil Babka Cc: Michal Hocko Cc: Shakeel Butt Cc: Wei Xu Cc: Greg Thelen Cc: Hugh Dickins Cc: David Rientjes Reviewed-by: Shakeel Butt Acked-by: David Rientjes Acked-by: Mel Gorman Reviewed-by: Vlastimil Babka Reported-by: Eric Dumazet Signed-off-by: Mel Gorman Acked-by: Eric Dumazet --- mm/page_alloc.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3589febc6d31928f850ebe5a4015ddc40e0469f3..1804287c1b792b8aa0e964b17eb002b6b1115258 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3706,10 +3706,10 @@ struct page *rmqueue(struct zone *preferred_zone, * allocate greater than order-1 page units with __GFP_NOFAIL. */ WARN_ON_ONCE((gfp_flags & __GFP_NOFAIL) && (order > 1)); - spin_lock_irqsave(&zone->lock, flags); do { page = NULL; + spin_lock_irqsave(&zone->lock, flags); /* * order-0 request can reach here when the pcplist is skipped * due to non-CMA allocation context. HIGHATOMIC area is @@ -3721,15 +3721,15 @@ struct page *rmqueue(struct zone *preferred_zone, if (page) trace_mm_page_alloc_zone_locked(page, order, migratetype); } - if (!page) + if (!page) { page = __rmqueue(zone, order, migratetype, alloc_flags); - } while (page && check_new_pages(page, order)); - if (!page) - goto failed; - - __mod_zone_freepage_state(zone, -(1 << order), - get_pcppage_migratetype(page)); - spin_unlock_irqrestore(&zone->lock, flags); + if (!page) + goto failed; + } + __mod_zone_freepage_state(zone, -(1 << order), + get_pcppage_migratetype(page)); + spin_unlock_irqrestore(&zone->lock, flags); + } while (check_new_pages(page, order)); __count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order); zone_statistics(preferred_zone, zone, 1);