From patchwork Fri Dec 4 12:56:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11951697 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6294C4361A for ; Fri, 4 Dec 2020 12:57:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EFEE622AAE for ; Fri, 4 Dec 2020 12:57:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EFEE622AAE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0792F6B0036; Fri, 4 Dec 2020 07:57:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 029026B005C; Fri, 4 Dec 2020 07:56:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E81596B0068; Fri, 4 Dec 2020 07:56:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0253.hostedemail.com [216.40.44.253]) by kanga.kvack.org (Postfix) with ESMTP id CFEFC6B0036 for ; Fri, 4 Dec 2020 07:56:59 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 984E6181AEF10 for ; Fri, 4 Dec 2020 12:56:59 +0000 (UTC) X-FDA: 77555599758.10.bird83_5d07ea6273c4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id 744B616A4AD for ; Fri, 4 Dec 2020 12:56:59 +0000 (UTC) X-HE-Tag: bird83_5d07ea6273c4 X-Filterd-Recvd-Size: 4798 Received: from mail-pl1-f193.google.com (mail-pl1-f193.google.com [209.85.214.193]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Fri, 4 Dec 2020 12:56:58 +0000 (UTC) Received: by mail-pl1-f193.google.com with SMTP id s2so3034293plr.9 for ; Fri, 04 Dec 2020 04:56:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=KfH3iRIxFYJ1fhTiZ1PPzOKH/EKjKgIPU7+T1IHn0PA=; b=ScTO4PXm9Z+w4pOvBzJ8JNPEN/ZIMlTo2uvJW5qX/+5qq3KJCdzJPC90/tIAkfwrLh eWFJR26XvnQwFU7rmOiZHTqePXUAxwJOFN+YpNdFHrMPbaL79b7jVIUNA8yDtYvQOd3Y ZUVqZwyfjweORrfN3lM2fAAtvDb7pPcTw0NySo3tX1PWC4KzX3+FMAg4kUNJyval0nUh sJSPNjbDUrpmYbRj8UdkcZzh9ihcxpAG2LYs6bvN4MRk7PJ3zVVs21JhRng1Ex54aYqS JlatRB+swyOymBnIP3hBl5pUCQ1Diu+VMB+oUJb1zVDfT3MRAE6H5Rk9aTrt/5A1VP5W SwFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=KfH3iRIxFYJ1fhTiZ1PPzOKH/EKjKgIPU7+T1IHn0PA=; b=m8+QFs5c+2lplcsFfGtewnmFgspAV4C9svLt1hxhYOM5Zdq8/4QkPBKd2+Ymc2AaQd KelL2kW3VtnOlZ6sLi34QfU9vnCySDOZ8b+ZWkIkYnJKqe8YGBHrjc0Q4nwZpc41qpsw 35ipGUExErMgSpJprYN4hfv+fUOGc90sr97+E/7zu9AMoQBKHKXXhp7xPcvaOGaAQ6jp uJm623g7+MJVhJymuG3qSbh75+jIH7Y0XAuArND76l7NBU+7ROcBZeTW5MnYY8aCO6Bc pM2U7q4+ui/IwfFBoauTMTrXTbDJpBAY36WicIHZbxKfbWzejQQNznONW/czalaacskB 8jBQ== X-Gm-Message-State: AOAM530/l82yQURtEEPZW7ZUhgp0lEV6iGeRUpaPVQiMaDfrRd79sYhy GKN00CEHNHNrvyh4yHUt0rL3cw== X-Google-Smtp-Source: ABdhPJz22BO+t4/sZnopHJiIS9RiZj23SNT2TysygLEzOwTMFHcK4mKdX9JNQSKZ2zbwifnuL0IH/g== X-Received: by 2002:a17:90a:a10c:: with SMTP id s12mr4117070pjp.86.1607086617536; Fri, 04 Dec 2020 04:56:57 -0800 (PST) Received: from localhost.bytedance.net ([103.136.220.97]) by smtp.gmail.com with ESMTPSA id a124sm4950624pfd.43.2020.12.04.04.56.54 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 04 Dec 2020 04:56:56 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, vbabka@suse.cz Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song Subject: [PATCH v2] mm/page_alloc: speeding up the iteration of max_order Date: Fri, 4 Dec 2020 20:56:40 +0800 Message-Id: <20201204125640.51804-1-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When we free a page whose order is very close to MAX_ORDER and greater than pageblock_order, it wastes some CPU cycles to increase max_order to MAX_ORDER one by one and check the pageblock migratetype of that page repeatedly especially when MAX_ORDER is much larger than pageblock_order. Signed-off-by: Muchun Song Acked-by: Vlastimil Babka --- Changes in v2: - Rework the code suggested by Vlastimil. Thanks. mm/page_alloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index f91df593bf71..56e603eea1dd 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1002,7 +1002,7 @@ static inline void __free_one_page(struct page *page, struct page *buddy; bool to_tail; - max_order = min_t(unsigned int, MAX_ORDER, pageblock_order + 1); + max_order = min_t(unsigned int, MAX_ORDER - 1, pageblock_order); VM_BUG_ON(!zone_is_initialized(zone)); VM_BUG_ON_PAGE(page->flags & PAGE_FLAGS_CHECK_AT_PREP, page); @@ -1015,7 +1015,7 @@ static inline void __free_one_page(struct page *page, VM_BUG_ON_PAGE(bad_range(zone, page), page); continue_merging: - while (order < max_order - 1) { + while (order < max_order) { if (compaction_capture(capc, page, order, migratetype)) { __mod_zone_freepage_state(zone, -(1 << order), migratetype); @@ -1041,7 +1041,7 @@ static inline void __free_one_page(struct page *page, pfn = combined_pfn; order++; } - if (max_order < MAX_ORDER) { + if (order < MAX_ORDER - 1) { /* If we are here, it means order is >= pageblock_order. * We want to prevent merge between freepages on isolate * pageblock and normal pageblock. Without this, pageblock @@ -1062,7 +1062,7 @@ static inline void __free_one_page(struct page *page, is_migrate_isolate(buddy_mt))) goto done_merging; } - max_order++; + max_order = order + 1; goto continue_merging; }