From patchwork Sun Jan 29 09:00:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peng Zhang X-Patchwork-Id: 13120051 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C37AC05027 for ; Sun, 29 Jan 2023 09:00:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0DB6B6B0074; Sun, 29 Jan 2023 04:00:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0650A6B0075; Sun, 29 Jan 2023 04:00:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E20786B0078; Sun, 29 Jan 2023 04:00:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id CF32D6B0074 for ; Sun, 29 Jan 2023 04:00:52 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id A0785C04F6 for ; Sun, 29 Jan 2023 09:00:52 +0000 (UTC) X-FDA: 80407241544.11.DD6B648 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf17.hostedemail.com (Postfix) with ESMTP id DCC7640014 for ; Sun, 29 Jan 2023 09:00:50 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=2FpCFAYP; spf=pass (imf17.hostedemail.com: domain of zhangpeng.00@bytedance.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=zhangpeng.00@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674982850; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ScnZcZbb5V5pAaOvJOkYQXxCHW6a6an6BscyyrmyHtQ=; b=2ErswRxaNm9zou07CiqNqJHFbSwiHFHRlUtIyYfGOb4QCHVQuoBx8x/GDnPqHyIh8HfH3d kdjLDOA6x1INC5PLn2vLuEkw01aJ3WNhu+SGnvsi8MGIiSIobXZTjHiBBYGT+t/wPSQjbK JrvqSVJsoUUdQMoUhZbhPMnLoz4JJ4A= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=2FpCFAYP; spf=pass (imf17.hostedemail.com: domain of zhangpeng.00@bytedance.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=zhangpeng.00@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674982850; a=rsa-sha256; cv=none; b=hHxPMwHNyUeIN0CU8FIteuKQeuP+NHywqtyXLZWG47GZwE0kfAn8TvzrIRmsSwVDMaQ0S+ iYee4KhGbfSK5CCQ3qeoa7EIA83lc98h5WROMUa/ebmPI2crGH1rbx2Ej5CSjZRYfIxh3W I1tUFuwMUECxA1CfTll1dETC/VYKC78= Received: by mail-pj1-f53.google.com with SMTP id e8-20020a17090a9a8800b0022c387f0f93so7608390pjp.3 for ; Sun, 29 Jan 2023 01:00:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ScnZcZbb5V5pAaOvJOkYQXxCHW6a6an6BscyyrmyHtQ=; b=2FpCFAYP0cmHckXG5sKIz6y4qCTRDcEXKoIhAKX34FvslF0ncTuPRaKOUtNpy1EmCh WgpZKATrUNUFKck1nV4UAqGwDcMKDlscS+Sc06+XqQU7LkSDhaWXYb/6aVZHESY77Jrb F+GFfpTsZeK4uUZ0Ayc5gZMUhYzAamJM5ggJ9iXoDkaqvL5VcISdIWYxaxW43ugmZqqO bjEum0V4dtVtwFgFOHSXHKACt+J8Snoe203X83lEusmHyJJY9atfvNzzp9d9OnEzBVXR nj5+mEplqKwzYG6kmZyMx1qdR3Q4BwL4cJfN5XwmhW2TV4fuu0bQawymKtonAcdBf4Fi RvvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ScnZcZbb5V5pAaOvJOkYQXxCHW6a6an6BscyyrmyHtQ=; b=YXDZXRrAA8EiF2kZZJXSpOGd44glq09JVndEIHCMsSN3m1v8t4op3EI6FNmMe6hFA3 NlXC+L3OIQn0Lgjp3PFg4vdjhHqIGeUyB+jLetkbTPNYPeBa7RPV9c3+nuzoOHgyfWAT zgiLE0SQfPdePic/r1jDMdUozmU9y/95yRLhpD2qPpv5ISwHfSWqzPk3potEsPj0TJNI e6NzBBP9V718E+CEHuSXteYJLSM6Fi4Vs7sEi5f6YFDuNpwDtxTQvgHyw0V620zjvPhV 4XzX2oZ6VyzMFiMeIsMgez8qdv+630K8l8RAo+UohD6KAig2E0BWuVOJEue/5tCSprku DmEQ== X-Gm-Message-State: AO0yUKW9te8x5FS67G7fclqszlqf6j/ZJYS9R07ls9aJHUAYpzM88Xhu gwSqljngEW6ccXd5s/PUlndhKg== X-Google-Smtp-Source: AK7set8OItuuxzVtM6QBoIV/KsdOqx0sis6DAF6WZwbVrgroVjmqVqyv63tM6G0OHm9J0+gld1SQOA== X-Received: by 2002:a17:903:2309:b0:196:11ad:8197 with SMTP id d9-20020a170903230900b0019611ad8197mr4558850plh.66.1674982849733; Sun, 29 Jan 2023 01:00:49 -0800 (PST) Received: from GL4FX4PXWL.bytedance.net ([139.177.225.239]) by smtp.gmail.com with ESMTPSA id h13-20020a170902f54d00b0019607984a5esm2142689plf.95.2023.01.29.01.00.47 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 29 Jan 2023 01:00:49 -0800 (PST) From: Peng Zhang To: rppt@kernel.org, akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Peng Zhang Subject: [PATCH v2 2/2] memblock: Avoid useless checks in memblock_merge_regions(). Date: Sun, 29 Jan 2023 17:00:34 +0800 Message-Id: <20230129090034.12310-3-zhangpeng.00@bytedance.com> X-Mailer: git-send-email 2.37.0 (Apple Git-136) In-Reply-To: <20230129090034.12310-1-zhangpeng.00@bytedance.com> References: <20230129090034.12310-1-zhangpeng.00@bytedance.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: tf6d1e145rfk7mpt6bw98t6min5t8653 X-Rspamd-Queue-Id: DCC7640014 X-HE-Tag: 1674982850-228076 X-HE-Meta: U2FsdGVkX19K41/cnmHiNrVZYujyncmmGCsQDJLe9gqaX2S8O8Hp0HVJyq+vf8g9mbmVpCFVWOGQbVNUqq6rPy7AnaURqjy0/zMzGbl5BA1dk5dn9lcOdJV03M98ftDcv6Ust8InSloLTlYYmlx0p4/Yilple+CyPZOGj5E4peix4RlBW9z5hluw09eNc5Xg8py98FA0HPA7mAyqFNaF6S5wht9edEMIKXLH/zX3pwXCPNERqCBBJwtrb+uoib+Ue9gpC8Lz27pBYMkavb3z0KqMJu9Mv+3abk0WFkWP3+ImjAwsg1cnRZ4RpKWJVOQYSrsZVwBWZhyY/B7kHwsc3kMg7YYC+gP2Ngj368mbi/lRSkajhqVRRmuDuI7wCzR/aO1ULq6GMO+9A10efEcy3Fm1EKs68kUSfZjGvN4bzLBj8xfDzc/p6pJCb0mSJ/JaYOeXPoMgZmHvzn/hpViKvUqIgmq5VQu6m1zfDHYhLTnreMXK1uT3EeGBPJP6YlekANSd4Hy5xGAsU6ECn6+PXTeVyD7/KEyIeRgQIWiYwzw1CVIIYnhy0HWNbHjfQAxjK14wGbarN+twlL7ddyy9pu1UcMoYJkuNEVXN0hby8XHNKA5CEzvtQqJvnEDLr6Zx/zSQTYRq9tfEsQVxu17gWqgSfPTk1W9jsE9W4rX1G8xpGxaQauub0EsQ6SUjnYTlmhS3ru65NvL3v9gcNhJluvGSzC4ZJqxtXajcqzQwOuhHysHdInn/WnFxqTLi3LDMpYQs5+KIvKjUR165Ag+KcW/wtEANYgy2VyQ5VxZ0VcI/9/ZmyFB02Y13w2rH/V0zVK8A9oOE08MHbIuWvKJs0OOxvXwPDBEFzjSAJ1a3NkOiFvAINTvaSQphtocGWaKllKHSY8OvIuvGfe1BYfQAw/8PDMx+a1iSOX7mDoUOFdL+tF0XT0vWt3NrVJpQX7EeIuNiaTv259B8RnBVfSu TFV/OUz5 o2SUImrpZYfS5A32rVUjPAj4OgbPK++R3v6zKxc0dpXO8jYaCOnC2V0oEfi4jRbCvtsmUxUJrWoao+ldWbPjs4DLg7rFLGjewtMXHxb+tgG2yVrmdD/bNOBg83jhpk9DAt0/VDybBbw0VuA479R5sBSk7VLNE5U+niIjSFMMBpxKBfkvg0UPOzyyUVNRR3ec4lwSStKVtPGChUpTykaTfy7ObFBfqghb3BA8u2RfTAo9G6rxBSCZdVXp07WRhjIwd0Bmo3IZm1xMqdq6PZ332Dnw9yYIpRXPbqtc52yVeuv8VtSmRKHfJGU/e3EtxSjpkuhfCsGGRrLMO3vSyua2/tcgzmoT4tttyYpGV/0EGJTLDn6rtBlSWjIkV8qeUZdwzcI72f1D7GE5egWDVaZMXrnPMcu8egplk4QI32tzOLuIuwTs7OuRUc/jQjroMijiIRXon3eULgzl1xH1Ss4MpzWSXVeVoN6sRUkp41WOhpimb9c8LYMNoESbWOTZYSCz0rRdqxbacXMydc5/XaoYc0uDm/A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000007, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: memblock_merge_regions() is called after regions have been modified to merge the neighboring compatible regions. That will check all regions but most checks is useless. Most of the time we only insert one or a few new regions, or modify one or a few regions. At this time, we don't need to check all regions. We only need to check the changed regions, because other not related regions cannot be merged. So this patch add two parameters to memblock_merge_regions() to indicate the lower and upper boundary to scan. Test this patch and get numbers like this: void memblock_merge_regions(struct memblock_type *type) { static int iteration_count = 0; static int max_nr_regions = 0; max_nr_regions = max(max_nr_regions, (int)type->cnt); ... while () { iteration_count++; ... } pr_info("iteration_count: %d max_nr_regions %d", iteration_count, max_nr_regions); } The folowing numbers is the last output tested on a physical machine with 1T memory. Mainline: [2.472243] iteration_count: 45410 max_nr_regions 178 Patched: [2.470869] iteration_count: 923 max_nr_regions 176 The actual startup speed seems to change little, but it does reduce the scan overhead. Signed-off-by: Peng Zhang --- mm/memblock.c | 37 +++++++++++++++++++++++++------------ 1 file changed, 25 insertions(+), 12 deletions(-) diff --git a/mm/memblock.c b/mm/memblock.c index 836eb71ea3ea..22e48b0f57ad 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -500,15 +500,19 @@ static int __init_memblock memblock_double_array(struct memblock_type *type, /** * memblock_merge_regions - merge neighboring compatible regions * @type: memblock type to scan - * - * Scan @type and merge neighboring compatible regions. + * @start_rgn: start scanning from (@start_rgn - 1) + * @end_rgn: end scanning at (@end_rgn - 1) + * Scan @type and merge neighboring compatible regions in [@start_rgn - 1, @end_rgn) */ -static void __init_memblock memblock_merge_regions(struct memblock_type *type) +static void __init_memblock memblock_merge_regions(struct memblock_type *type, + unsigned long start_rgn, + unsigned long end_rgn) { int i = 0; - - /* cnt never goes below 1 */ - while (i < type->cnt - 1) { + if (start_rgn) + i = start_rgn - 1; + end_rgn = min(end_rgn, type->cnt - 1); + while (i < end_rgn) { struct memblock_region *this = &type->regions[i]; struct memblock_region *next = &type->regions[i + 1]; @@ -525,6 +529,7 @@ static void __init_memblock memblock_merge_regions(struct memblock_type *type) /* move forward from next + 1, index of which is i + 2 */ memmove(next, next + 1, (type->cnt - (i + 2)) * sizeof(*next)); type->cnt--; + end_rgn--; } } @@ -581,7 +586,7 @@ static int __init_memblock memblock_add_range(struct memblock_type *type, bool insert = false; phys_addr_t obase = base; phys_addr_t end = base + memblock_cap_size(base, &size); - int idx, nr_new; + int idx, nr_new, start_rgn = -1, end_rgn; struct memblock_region *rgn; if (!size) @@ -635,10 +640,14 @@ static int __init_memblock memblock_add_range(struct memblock_type *type, #endif WARN_ON(flags != rgn->flags); nr_new++; - if (insert) + if (insert) { + if (start_rgn == -1) + start_rgn = idx; + end_rgn = idx + 1; memblock_insert_region(type, idx++, base, rbase - base, nid, flags); + } } /* area below @rend is dealt with, forget about it */ base = min(rend, end); @@ -647,9 +656,13 @@ static int __init_memblock memblock_add_range(struct memblock_type *type, /* insert the remaining portion */ if (base < end) { nr_new++; - if (insert) + if (insert) { + if (start_rgn == -1) + start_rgn = idx; + end_rgn = idx + 1; memblock_insert_region(type, idx, base, end - base, nid, flags); + } } if (!nr_new) @@ -666,7 +679,7 @@ static int __init_memblock memblock_add_range(struct memblock_type *type, insert = true; goto repeat; } else { - memblock_merge_regions(type); + memblock_merge_regions(type, start_rgn, end_rgn); return 0; } } @@ -902,7 +915,7 @@ static int __init_memblock memblock_setclr_flag(phys_addr_t base, r->flags &= ~flag; } - memblock_merge_regions(type); + memblock_merge_regions(type, start_rgn, end_rgn); return 0; } @@ -1275,7 +1288,7 @@ int __init_memblock memblock_set_node(phys_addr_t base, phys_addr_t size, for (i = start_rgn; i < end_rgn; i++) memblock_set_region_node(&type->regions[i], nid); - memblock_merge_regions(type); + memblock_merge_regions(type, start_rgn, end_rgn); #endif return 0; }