From patchwork Wed Aug 22 03:07:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jia He X-Patchwork-Id: 10572569 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7BEEA921 for ; Wed, 22 Aug 2018 03:08:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 72BA32B113 for ; Wed, 22 Aug 2018 03:08:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 659442B179; Wed, 22 Aug 2018 03:08:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id F1DC52B113 for ; Wed, 22 Aug 2018 03:08:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=/y1S/2gYQPOJIko0jYkwhCfxD1TemM1QfymPKdg5hjM=; b=ILgczi+mw1mLhOIZf8/cgwHhCN Z4CY5jPyEgPSusDl7ZGYpC+WpjVZ9UpIGouQ2Vv9PIWD3wvE9Kfwm7oN7NIs/nqJmpxkQhBtdZ2Fi /ZWhxPNSYcHVlriwS4vtYkfSjfZlNYpJkCvxtxqjqXKOHPk6HRMBSWeNiluajYs8yfTmt44wWYnWJ fmIl8UVkZAxlK/HBgKBGps+LSrzIUmY8zmzeU4xtNupLqx4DPsz5IzfU3aPW6Hpavq5YTkktwNjD5 +nrMjngoC50oBbUWStAvErFJmYfMqHZZ3EJSDlafDWWYoNP89tj6V1Ej1hDRHsk2nCZxj1Del07jl 8Ut4fkHQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fsJV6-0004OU-Sq; Wed, 22 Aug 2018 03:08:04 +0000 Received: from mail-pg1-x542.google.com ([2607:f8b0:4864:20::542]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1fsJV2-00046c-8M for linux-arm-kernel@lists.infradead.org; Wed, 22 Aug 2018 03:08:03 +0000 Received: by mail-pg1-x542.google.com with SMTP id v66-v6so259926pgb.10 for ; Tue, 21 Aug 2018 20:07:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=qB8wQLRm567sPd0BwIEEQEjYlqnBOl4UOSs2cGebjxQ=; b=P/PJBkwEPMZUsM7aFZSoSuMyZ+ESKprPlOZTIvj+YyTLkNr053ypWmVga5VMc06/2c CR7+mhHGvNik5HsqSTOAxkq1GeZG0sVfo+S2jxkIr91Z4gB91raZ4efl4oEKTLvxc+6T AazzPIqEiuuzu3meJfA1z183Fmn7DymWkBSlrhml//oJ8A0LytS5b3RDIth6VZizrenq fzHYl+0CvECgT+OoM97SsDKLkOpItmsbEzbU2zlnpsz9ms6SqABXr3vO92lbxAYdhtbW 9O1Gye0QNq7JVhqpgqcLppc9JRgbtbLCtC3qAky/acYd7plrEAkAM/+HSGrGyrPVEa/Q 9nHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=qB8wQLRm567sPd0BwIEEQEjYlqnBOl4UOSs2cGebjxQ=; b=Mz4+FWVXju2/TKBy2gjn+zGVzUTYL0Q/G1vrDpfslSaec4KsUIcPmfcfH2F6MtOXgi 6VoQMd6oOQeSQAUAcNieAsP1BljTGrviC/UjJbEG3isrwUprm7u+dfDPWanqD6Jl/Ofw JJ2IFYZNQi0D5iC8WE8ydk6vYCfJl2BbY6QQi80FwtJ4TKwTxhKp4EsupPwn0042QZFv SdOpC1i/fRoXzN642ow5JZOgJ+SykzNMmKpmHIdLQ31aIt6AYAZH1VxzJWSS0wkCyDRR b7Ez+J22Shf8u6MK7ue1NxZ+8uBzq2Z/cE+VFmyFUjuZCvSHPB3T6PfncufL6HjzK10g sh5A== X-Gm-Message-State: AOUpUlFK6C7vDBmIvXimgVhr2iWjjhGS35nENeVaItrrOXnvon2ry6Xl howc8GWu16GCGlsVDt5y1kI= X-Google-Smtp-Source: AA+uWPzJBdDsbSrchc52klfb1DQ0wJelNefkYjsYYabOmrVH5TFk/ERFhQ9F0GfOux4D49sKOkvVLA== X-Received: by 2002:a63:4005:: with SMTP id n5-v6mr10897866pga.221.1534907269416; Tue, 21 Aug 2018 20:07:49 -0700 (PDT) Received: from ct7host.localdomain ([38.106.11.25]) by smtp.gmail.com with ESMTPSA id q7-v6sm440165pfq.43.2018.08.21.20.07.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 21 Aug 2018 20:07:48 -0700 (PDT) From: Jia He X-Google-Original-From: Jia He To: Russell King , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Andrew Morton , Michal Hocko Subject: [PATCH v11 1/3] arm: arm64: introduce CONFIG_HAVE_MEMBLOCK_PFN_VALID Date: Wed, 22 Aug 2018 11:07:15 +0800 Message-Id: <1534907237-2982-2-git-send-email-jia.he@hxt-semitech.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1534907237-2982-1-git-send-email-jia.he@hxt-semitech.com> References: <1534907237-2982-1-git-send-email-jia.he@hxt-semitech.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180821_200800_592998_F5A4C248 X-CRM114-Status: GOOD ( 13.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Gioh Kim , Wei Yang , linux-mm@kvack.org, Eugeniu Rosca , Petr Tesarik , Nikolay Borisov , Daniel Jordan , AKASHI Takahiro , Vlastimil Babka , Andrey Ryabinin , Laura Abbott , Daniel Vacek , Mel Gorman , Vladimir Murzin , Kees Cook , Philip Derrin , YASUAKI ISHIMATSU , Jia He , Kemi Wang , linux-arm-kernel@lists.infradead.org, Steve Capper , linux-kernel@vger.kernel.org, James Morse , Johannes Weiner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Make CONFIG_HAVE_MEMBLOCK_PFN_VALID a new config option so it can move memblock_next_valid_pfn to generic code file. All the latter optimizations are based on this config. The memblock initialization time on arm/arm64 can benefit from this. Signed-off-by: Jia He Reviewed-by: Pavel Tatashin --- arch/arm/Kconfig | 1 + arch/arm64/Kconfig | 1 + mm/Kconfig | 3 +++ 3 files changed, 5 insertions(+) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 843edfd..d3c7705 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -1641,6 +1641,7 @@ config ARCH_SELECT_MEMORY_MODEL config HAVE_ARCH_PFN_VALID def_bool ARCH_HAS_HOLES_MEMORYMODEL || !SPARSEMEM + select HAVE_MEMBLOCK_PFN_VALID config HAVE_GENERIC_GUP def_bool y diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 42c090c..d4119e6 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -777,6 +777,7 @@ config ARCH_SELECT_MEMORY_MODEL config HAVE_ARCH_PFN_VALID def_bool ARCH_HAS_HOLES_MEMORYMODEL || !SPARSEMEM + select HAVE_MEMBLOCK_PFN_VALID config HW_PERF_EVENTS def_bool y diff --git a/mm/Kconfig b/mm/Kconfig index 94af022..28fcf54 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -137,6 +137,9 @@ config HAVE_MEMBLOCK_NODE_MAP config HAVE_MEMBLOCK_PHYS_MAP bool +config HAVE_MEMBLOCK_PFN_VALID + bool + config HAVE_GENERIC_GUP bool From patchwork Wed Aug 22 03:07:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jia He X-Patchwork-Id: 10572573 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5E6081579 for ; Wed, 22 Aug 2018 03:08:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 518CE2B113 for ; Wed, 22 Aug 2018 03:08:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 458412B179; Wed, 22 Aug 2018 03:08:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D78772B113 for ; Wed, 22 Aug 2018 03:08:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=TW/XCbW7dtMi+bTH5CzzcGSJpBA3fLzNDdHeKYLqiLU=; b=BB9UiVMvLYX3JCzsqEA+GiZAaF nChQSE6s9j/Txip7CjAYKjOBVPwIC+ZQOTqJ35NNXsEQSmpn5vxGjaZjUTREx5sDcR/ElkjL0CmDd GWfb7gznHRh6Gc2pXA+9VqjQ763GdiWn83QqOA2ceBsaB5IFFmCC0zoko+ZARDjCyOdKwSDdGC/VV 0G3ti2i085QPqpoc0a1tXd4b20ZYBO+jyF6bxUWzvVKMjoAyFrfIt/QyWfpwC2yxEOv/gvUBw8v2A qmxn7I8dExL1H+j+sbPUx9ctEUwN1lyUJPq3+TAdbfeDfa1qXHOGYIsvS9eIvdgjBiVPVd37lR8RM WHZtCAuw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fsJVM-0004hC-DP; Wed, 22 Aug 2018 03:08:20 +0000 Received: from mail-pl0-x242.google.com ([2607:f8b0:400e:c01::242]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1fsJVD-0004Kt-UT for linux-arm-kernel@lists.infradead.org; Wed, 22 Aug 2018 03:08:17 +0000 Received: by mail-pl0-x242.google.com with SMTP id p4-v6so243524pll.8 for ; Tue, 21 Aug 2018 20:08:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=FZQVcSjva4KqT1QnxQl07tGVuHDUFsqtAvdlpz1t47Q=; b=dUBGkRw4fQaiMecEzf8cFWCp6lhVcwMzGzSJuhJ/kDZkOS/u/pcLbU9y5Xf1q1j77r dtL/gCMhC9eqDMKxzYrXtYRlA5a94kZIwjuX+W8VHjreqVs9v6Dscjei2XBBlnncnM4p n73u1ktG72mHDkQonOkyTO1/t4u/t1qqLjVMvDj7/lvdikAWO7X+SCQYDPl7U2GSVlfJ M2ft/3Qfgwnj4P1+9mUHj4676Eku6rYRBV32xYXzPBdGb0x3G/XnuydXMwFf72akSOpp 7Krekn3z/4ghm2DDp1SZzdN5QUHqpfUvUf0gya3hnJ/6JYU5XPlrKy0IN3wXa46RWS/6 MJeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=FZQVcSjva4KqT1QnxQl07tGVuHDUFsqtAvdlpz1t47Q=; b=PgmN+HsIvPCrgGR9kl9iMrlCZwMkBedEW9kQHIj5vO9VhHV4X8j8ezB8BJX+wt2mpA ZUfw8xSH8eJ4knjp7myE+TKexsnqR+1oXPF6JAsGS1w3sn4IBJcFST5s3eO6Y4DLn/Nn 8NDM+Xu4YqcCf8fEqUfgIAG00SCBxUrx6FBtzq/XXAu3ee8GOYCIlcJYwm9GksHyo2Tn 68hgHcvbLnN7mTChiC7ND3uzOsdG3juitBZahFFMC+PikyvuRi78B4WL5uw85s1DbVPG q2uz+oPe1Q+KUOlIUpP/Rh5rEchvms4DHOrRGdtRp4zhjKdbXuxQ75xVnQ86YVi4lmt/ lBmQ== X-Gm-Message-State: AOUpUlG/EE9+v5a093Mfax3vys8DTbuQofYceJPb7iJw8ylpOpduaWHb 1QtrxcQ6hYQK48RaGHrTt0E= X-Google-Smtp-Source: AA+uWPx7O8adWCfcvaXiGqLDUgNNylojl01bhH+g/Gjqes6HLwaxhGQ5ncYX6Lz40i2yhizGHITgng== X-Received: by 2002:a17:902:42e2:: with SMTP id h89-v6mr51827967pld.69.1534907281175; Tue, 21 Aug 2018 20:08:01 -0700 (PDT) Received: from ct7host.localdomain ([38.106.11.25]) by smtp.gmail.com with ESMTPSA id q7-v6sm440165pfq.43.2018.08.21.20.07.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 21 Aug 2018 20:08:00 -0700 (PDT) From: Jia He X-Google-Original-From: Jia He To: Russell King , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Andrew Morton , Michal Hocko Subject: [PATCH v11 2/3] mm: page_alloc: remain memblock_next_valid_pfn() on arm/arm64 Date: Wed, 22 Aug 2018 11:07:16 +0800 Message-Id: <1534907237-2982-3-git-send-email-jia.he@hxt-semitech.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1534907237-2982-1-git-send-email-jia.he@hxt-semitech.com> References: <1534907237-2982-1-git-send-email-jia.he@hxt-semitech.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180821_200812_586561_C94BF53C X-CRM114-Status: GOOD ( 21.63 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Gioh Kim , Wei Yang , linux-mm@kvack.org, Eugeniu Rosca , Petr Tesarik , Nikolay Borisov , Daniel Jordan , AKASHI Takahiro , Vlastimil Babka , Andrey Ryabinin , Laura Abbott , Daniel Vacek , Mel Gorman , Vladimir Murzin , Kees Cook , Philip Derrin , YASUAKI ISHIMATSU , Jia He , Kemi Wang , linux-arm-kernel@lists.infradead.org, Steve Capper , linux-kernel@vger.kernel.org, James Morse , Johannes Weiner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns where possible") optimized the loop in memmap_init_zone(). But it causes possible panic bug. So Daniel Vacek reverted it later. But as suggested by Daniel Vacek, it is fine to using memblock to skip gaps and finding next valid frame with CONFIG_HAVE_ARCH_PFN_VALID. Daniel said: "On arm and arm64, memblock is used by default. But generic version of pfn_valid() is based on mem sections and memblock_next_valid_pfn() does not always return the next valid one but skips more resulting in some valid frames to be skipped (as if they were invalid). And that's why kernel was eventually crashing on some !arm machines." About the performance consideration: As said by James in b92df1de5, "I have tested this patch on a virtual model of a Samurai CPU with a sparse memory map. The kernel boot time drops from 109 to 62 seconds." Thus it would be better if we remain memblock_next_valid_pfn on arm/arm64. Suggested-by: Daniel Vacek Signed-off-by: Jia He --- include/linux/mmzone.h | 9 +++++++++ mm/memblock.c | 30 ++++++++++++++++++++++++++++++ mm/page_alloc.c | 5 ++++- 3 files changed, 43 insertions(+), 1 deletion(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 32699b2..8e5e20b 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -1266,6 +1266,10 @@ static inline int pfn_present(unsigned long pfn) #endif #define early_pfn_valid(pfn) pfn_valid(pfn) +#ifdef CONFIG_HAVE_MEMBLOCK_PFN_VALID +extern unsigned long memblock_next_valid_pfn(unsigned long pfn); +#define next_valid_pfn(pfn) memblock_next_valid_pfn(pfn) +#endif void sparse_init(void); #else #define sparse_init() do {} while (0) @@ -1287,6 +1291,11 @@ struct mminit_pfnnid_cache { #define early_pfn_valid(pfn) (1) #endif +/* fallback to default definitions*/ +#ifndef next_valid_pfn +#define next_valid_pfn(pfn) (pfn + 1) +#endif + void memory_present(int nid, unsigned long start, unsigned long end); /* diff --git a/mm/memblock.c b/mm/memblock.c index 3d03866..077ca62 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -1140,6 +1140,36 @@ int __init_memblock memblock_set_node(phys_addr_t base, phys_addr_t size, } #endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */ +#ifdef CONFIG_HAVE_MEMBLOCK_PFN_VALID +unsigned long __init_memblock memblock_next_valid_pfn(unsigned long pfn) +{ + struct memblock_type *type = &memblock.memory; + unsigned int right = type->cnt; + unsigned int mid, left = 0; + phys_addr_t addr = PFN_PHYS(++pfn); + + do { + mid = (right + left) / 2; + + if (addr < type->regions[mid].base) + right = mid; + else if (addr >= (type->regions[mid].base + + type->regions[mid].size)) + left = mid + 1; + else { + /* addr is within the region, so pfn is valid */ + return pfn; + } + } while (left < right); + + if (right == type->cnt) + return -1UL; + else + return PHYS_PFN(type->regions[right].base); +} +EXPORT_SYMBOL(memblock_next_valid_pfn); +#endif /*CONFIG_HAVE_MEMBLOCK_PFN_VALID*/ + static phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size, phys_addr_t align, phys_addr_t start, phys_addr_t end, int nid, ulong flags) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index cd3c7b9..607deff 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5485,8 +5485,11 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, if (context != MEMMAP_EARLY) goto not_early; - if (!early_pfn_valid(pfn)) + if (!early_pfn_valid(pfn)) { + pfn = next_valid_pfn(pfn) - 1; continue; + } + if (!early_pfn_in_nid(pfn, nid)) continue; if (!update_defer_init(pgdat, pfn, end_pfn, &nr_initialised)) From patchwork Wed Aug 22 03:07:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jia He X-Patchwork-Id: 10572575 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 906B11579 for ; Wed, 22 Aug 2018 03:08:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 841892B1D0 for ; Wed, 22 Aug 2018 03:08:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 770672B1D4; Wed, 22 Aug 2018 03:08:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EBED22B1D0 for ; Wed, 22 Aug 2018 03:08:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=p7xTmXpby1G2v2KQ7VvI1uxzrGqZbo1EAu7nV1dBuG8=; b=JpbYIJAVgO10KEtG4A5M9XL7hB ag5I7JP8JIDKx0svkbSD7hTjVmfktlyV/7ZyBObbUpjtgTzZCrSs2UuAbuTRUTHCuzR7hhtFEbKtY fDG/dnbO+OhnRsrGz3632X4YEK1nHNDljAD89LPnyib8Vk44ChT+vTvqBxwEZPluFgqR+o6ZSXpHp CHGke85LcdoW217yymKqZ92W9j37CvajRUqsqXBVBJQkXz302xue6oSEEDroLa9W5HvJ0RuMaJ+f8 HVJBYvyXR28JIrC3mz+Hy7N8DfQKel/iB31xJs/G54vUlPSbgcb97o9ayEMm4z6WJ9rksxuBvnhcD 9Qo6jKLw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fsJVf-0004xv-UG; Wed, 22 Aug 2018 03:08:39 +0000 Received: from mail-pf1-x443.google.com ([2607:f8b0:4864:20::443]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1fsJVP-0004bL-Q9 for linux-arm-kernel@lists.infradead.org; Wed, 22 Aug 2018 03:08:38 +0000 Received: by mail-pf1-x443.google.com with SMTP id j26-v6so273628pfi.10 for ; Tue, 21 Aug 2018 20:08:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=k6L7V60tgYGv+lfJrTEz7C2h3jQjmVlNf+2CpL355Uw=; b=CClWurYYhPZP1b+a7/UxW5hDMp0ivher8C5m4yrUKdrW9DeAia5nUZUldiFJtXv5u+ U51mRFG9CE0yXAjhinVdbewB1QO01ag8L6Wz0o8Qge3pwIDs7q+2C+hxvx7zo33vnJWx FlG++ASu0bEoNy1mTdNFNBYGwLSy4NGnluNaAjlZOHk5ZZ0jl+92GM1yoqoqKgcKIz2N WiWmcbtGZn4Bs1GWACMcQan/c5QDpVdBECNSdax+UKkz9N3x/PdJYUaoa0Bdxu6o6y9Z vsKFnZ7ZrfSArJq1vOw0NGWuGSfXiqzE3UsdTS4wRy41PmYfVU3zI2a6WzoodQ0iAJmO d9vQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=k6L7V60tgYGv+lfJrTEz7C2h3jQjmVlNf+2CpL355Uw=; b=VxhdELjZF18uOlFhNkf6a5Sc4GSEKkvnNkOKyD4ebfMFOZy8jVh7oF2H863Sg2EeIf LGMv0xC0fx0cM5rs2SSenUvajyuAD/ZxnNqdQ5VK8kWtNc4PY7EENB12vuXey27MDHMl z22aT8tZ/S1k6zJS8ER0Z7vLKci0rW5VgS3f2j3C3z4MshNqcKCT7jFHO621s8f93+7X esn5/SS8KNUHD9RwR6Qv1OCKkB5kuJ8QbhSNuGf9JIOylaBE/T2A3fTDP1+PgXLGKVyD EpcAfM/uYXGdvpgM2Z6G+7nYO7L+wMIq3NO0zsODvPGKxglXyUNCsvN4hN7FifitMQLx O3Hg== X-Gm-Message-State: AOUpUlEYOh+piGqypO/gITmn/PDcpT9n4kqBkNewAKRURVe5i7j/0VHd ueqtkyDyoSEFxlbLyTNd7Y0= X-Google-Smtp-Source: AA+uWPxV31z1jy7Bm6/N+EN8CZpSzp3X6Sr/3HvKeDlgpaJshLexqp3C8dErWomwKpKdjQ81IiuOyg== X-Received: by 2002:a65:6551:: with SMTP id a17-v6mr50379577pgw.132.1534907292982; Tue, 21 Aug 2018 20:08:12 -0700 (PDT) Received: from ct7host.localdomain ([38.106.11.25]) by smtp.gmail.com with ESMTPSA id q7-v6sm440165pfq.43.2018.08.21.20.08.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 21 Aug 2018 20:08:12 -0700 (PDT) From: Jia He X-Google-Original-From: Jia He To: Russell King , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Andrew Morton , Michal Hocko Subject: [PATCH v11 3/3] mm: page_alloc: reduce unnecessary binary search in memblock_next_valid_pfn Date: Wed, 22 Aug 2018 11:07:17 +0800 Message-Id: <1534907237-2982-4-git-send-email-jia.he@hxt-semitech.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1534907237-2982-1-git-send-email-jia.he@hxt-semitech.com> References: <1534907237-2982-1-git-send-email-jia.he@hxt-semitech.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180821_200824_283321_F1469152 X-CRM114-Status: GOOD ( 17.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Gioh Kim , Wei Yang , linux-mm@kvack.org, Eugeniu Rosca , Petr Tesarik , Nikolay Borisov , Daniel Jordan , AKASHI Takahiro , Vlastimil Babka , Andrey Ryabinin , Laura Abbott , Daniel Vacek , Mel Gorman , Vladimir Murzin , Kees Cook , Philip Derrin , YASUAKI ISHIMATSU , Jia He , Kemi Wang , linux-arm-kernel@lists.infradead.org, Steve Capper , linux-kernel@vger.kernel.org, James Morse , Johannes Weiner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns where possible") optimized the loop in memmap_init_zone(). But there is still some room for improvement. E.g. if pfn and pfn+1 are in the same memblock region, we can simply pfn++ instead of doing the binary search in memblock_next_valid_pfn. Furthermore, if the pfn is in a gap of two memory region, skip to next region directly if possible. Attached the memblock region information in my server. [ 0.000000] Zone ranges: [ 0.000000] DMA32 [mem 0x0000000000200000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x00000017ffffffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000200000-0x000000000021ffff] [ 0.000000] node 0: [mem 0x0000000000820000-0x000000000307ffff] [ 0.000000] node 0: [mem 0x0000000003080000-0x000000000308ffff] [ 0.000000] node 0: [mem 0x0000000003090000-0x00000000031fffff] [ 0.000000] node 0: [mem 0x0000000003200000-0x00000000033fffff] [ 0.000000] node 0: [mem 0x0000000003410000-0x00000000034fffff] [ 0.000000] node 0: [mem 0x0000000003500000-0x000000000351ffff] [ 0.000000] node 0: [mem 0x0000000003520000-0x000000000353ffff] [ 0.000000] node 0: [mem 0x0000000003540000-0x0000000003e3ffff] [ 0.000000] node 0: [mem 0x0000000003e40000-0x0000000003e7ffff] [ 0.000000] node 0: [mem 0x0000000003e80000-0x0000000003ecffff] [ 0.000000] node 0: [mem 0x0000000003ed0000-0x0000000003ed5fff] [ 0.000000] node 0: [mem 0x0000000003ed6000-0x0000000006eeafff] [ 0.000000] node 0: [mem 0x0000000006eeb000-0x000000000710ffff] [ 0.000000] node 0: [mem 0x0000000007110000-0x0000000007f0ffff] [ 0.000000] node 0: [mem 0x0000000007f10000-0x0000000007faffff] [ 0.000000] node 0: [mem 0x0000000007fb0000-0x000000000806ffff] [ 0.000000] node 0: [mem 0x0000000008070000-0x00000000080affff] [ 0.000000] node 0: [mem 0x00000000080b0000-0x000000000832ffff] [ 0.000000] node 0: [mem 0x0000000008330000-0x000000000836ffff] [ 0.000000] node 0: [mem 0x0000000008370000-0x000000000838ffff] [ 0.000000] node 0: [mem 0x0000000008390000-0x00000000083a9fff] [ 0.000000] node 0: [mem 0x00000000083aa000-0x00000000083bbfff] [ 0.000000] node 0: [mem 0x00000000083bc000-0x00000000083fffff] [ 0.000000] node 0: [mem 0x0000000008400000-0x000000000841ffff] [ 0.000000] node 0: [mem 0x0000000008420000-0x000000000843ffff] [ 0.000000] node 0: [mem 0x0000000008440000-0x000000000865ffff] [ 0.000000] node 0: [mem 0x0000000008660000-0x000000000869ffff] [ 0.000000] node 0: [mem 0x00000000086a0000-0x00000000086affff] [ 0.000000] node 0: [mem 0x00000000086b0000-0x00000000086effff] [ 0.000000] node 0: [mem 0x00000000086f0000-0x0000000008b6ffff] [ 0.000000] node 0: [mem 0x0000000008b70000-0x0000000008bbffff] [ 0.000000] node 0: [mem 0x0000000008bc0000-0x0000000008edffff] [ 0.000000] node 0: [mem 0x0000000008ee0000-0x0000000008ee0fff] [ 0.000000] node 0: [mem 0x0000000008ee1000-0x0000000008ee2fff] [ 0.000000] node 0: [mem 0x0000000008ee3000-0x000000000decffff] [ 0.000000] node 0: [mem 0x000000000ded0000-0x000000000defffff] [ 0.000000] node 0: [mem 0x000000000df00000-0x000000000fffffff] [ 0.000000] node 0: [mem 0x0000000010800000-0x0000000017feffff] [ 0.000000] node 0: [mem 0x000000001c000000-0x000000001c00ffff] [ 0.000000] node 0: [mem 0x000000001c010000-0x000000001c7fffff] [ 0.000000] node 0: [mem 0x000000001c810000-0x000000007efbffff] [ 0.000000] node 0: [mem 0x000000007efc0000-0x000000007efdffff] [ 0.000000] node 0: [mem 0x000000007efe0000-0x000000007efeffff] [ 0.000000] node 0: [mem 0x000000007eff0000-0x000000007effffff] [ 0.000000] node 0: [mem 0x000000007f000000-0x00000017ffffffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000000200000-0x00000017ffffffff] [ 0.000000] On node 0 totalpages: 25145296 [ 0.000000] DMA32 zone: 16376 pages used for memmap [ 0.000000] DMA32 zone: 0 pages reserved [ 0.000000] DMA32 zone: 1028048 pages, LIFO batch:31 [ 0.000000] Normal zone: 376832 pages used for memmap [ 0.000000] Normal zone: 24117248 pages, LIFO batch:31 Signed-off-by: Jia He Reviewed-by: Pavel Tatashin --- mm/memblock.c | 33 +++++++++++++++++++++++++++------ 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/mm/memblock.c b/mm/memblock.c index 077ca62..46cb6be 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -1144,28 +1144,49 @@ int __init_memblock memblock_set_node(phys_addr_t base, phys_addr_t size, unsigned long __init_memblock memblock_next_valid_pfn(unsigned long pfn) { struct memblock_type *type = &memblock.memory; + struct memblock_region *regions = type->regions; unsigned int right = type->cnt; unsigned int mid, left = 0; + unsigned long start_pfn, end_pfn, next_start_pfn; phys_addr_t addr = PFN_PHYS(++pfn); + static int early_region_idx __initdata_memblock = -1; + /* fast path, return pfn+1 if next pfn is in the same region */ + if (early_region_idx != -1) { + start_pfn = PFN_DOWN(regions[early_region_idx].base); + end_pfn = PFN_DOWN(regions[early_region_idx].base + + regions[early_region_idx].size); + + if (pfn >= start_pfn && pfn < end_pfn) + return pfn; + + early_region_idx++; + next_start_pfn = PFN_DOWN(regions[early_region_idx].base); + + if (pfn >= end_pfn && pfn <= next_start_pfn) + return next_start_pfn; + } + + /* slow path, do the binary searching */ do { mid = (right + left) / 2; - if (addr < type->regions[mid].base) + if (addr < regions[mid].base) right = mid; - else if (addr >= (type->regions[mid].base + - type->regions[mid].size)) + else if (addr >= (regions[mid].base + regions[mid].size)) left = mid + 1; else { - /* addr is within the region, so pfn is valid */ + early_region_idx = mid; return pfn; } } while (left < right); if (right == type->cnt) return -1UL; - else - return PHYS_PFN(type->regions[right].base); + + early_region_idx = right; + + return PHYS_PFN(regions[early_region_idx].base); } EXPORT_SYMBOL(memblock_next_valid_pfn); #endif /*CONFIG_HAVE_MEMBLOCK_PFN_VALID*/