From patchwork Tue Oct 25 02:59:17 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 9393927 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 09C4760231 for ; Tue, 25 Oct 2016 03:03:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F02B328ACD for ; Tue, 25 Oct 2016 03:03:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E49E928FA5; Tue, 25 Oct 2016 03:03:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8D31628ACD for ; Tue, 25 Oct 2016 03:03:21 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.85_2 #1 (Red Hat Linux)) id 1bys06-0003wK-1W; Tue, 25 Oct 2016 03:02:06 +0000 Received: from szxga02-in.huawei.com ([119.145.14.65]) by bombadil.infradead.org with esmtps (Exim 4.85_2 #1 (Red Hat Linux)) id 1byrzc-0003j8-O1 for linux-arm-kernel@lists.infradead.org; Tue, 25 Oct 2016 03:01:39 +0000 Received: from 172.24.1.137 (EHLO szxeml433-hub.china.huawei.com) ([172.24.1.137]) by szxrg02-dlp.huawei.com (MOS 4.3.7-GA FastPath queued) with ESMTP id DPN80591; Tue, 25 Oct 2016 11:00:24 +0800 (CST) Received: from localhost (10.177.23.164) by szxeml433-hub.china.huawei.com (10.82.67.210) with Microsoft SMTP Server id 14.3.235.1; Tue, 25 Oct 2016 11:00:17 +0800 From: Zhen Lei To: Catalin Marinas , Will Deacon , linux-arm-kernel , linux-kernel , Andrew Morton , linux-mm Subject: [PATCH 1/2] mm/memblock: prepare a capability to support memblock near alloc Date: Tue, 25 Oct 2016 10:59:17 +0800 Message-ID: <1477364358-10620-2-git-send-email-thunder.leizhen@huawei.com> X-Mailer: git-send-email 1.9.5.msysgit.1 In-Reply-To: <1477364358-10620-1-git-send-email-thunder.leizhen@huawei.com> References: <1477364358-10620-1-git-send-email-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.164] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20161024_200137_362201_44740786 X-CRM114-Status: GOOD ( 19.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xinwei Hu , Zhen Lei , Zefan Li , Hanjun Guo Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP If HAVE_MEMORYLESS_NODES is selected, and some memoryless numa nodes are actually exist. The percpu variable areas and numa control blocks of that memoryless numa nodes need to be allocated from the nearest available node to improve performance. Although memblock_alloc_try_nid and memblock_virt_alloc_try_nid try the specified nid at the first time, but if that allocation failed it will directly drop to use NUMA_NO_NODE. This mean any nodes maybe possible at the second time. To compatible the above old scene, I use a marco node_distance_ready to control it. By default, the marco node_distance_ready is not defined in any platforms, the above mentioned functions will work as normal as before. Otherwise, they will try the nearest node first. Signed-off-by: Zhen Lei --- mm/memblock.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 65 insertions(+), 11 deletions(-) -- 2.5.0 diff --git a/mm/memblock.c b/mm/memblock.c index 7608bc3..556bbd2 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -1213,9 +1213,71 @@ phys_addr_t __init memblock_alloc(phys_addr_t size, phys_addr_t align) return memblock_alloc_base(size, align, MEMBLOCK_ALLOC_ACCESSIBLE); } +#ifndef node_distance_ready +#define node_distance_ready() 0 +#endif + +static phys_addr_t __init memblock_alloc_near_nid(phys_addr_t size, + phys_addr_t align, phys_addr_t start, + phys_addr_t end, int nid, ulong flags, + int alloc_func_type) +{ + int nnid, round = 0; + u64 pa; + DECLARE_BITMAP(nodes_map, MAX_NUMNODES); + + bitmap_zero(nodes_map, MAX_NUMNODES); + +again: + /* + * There are total 4 cases: + * + * 1)2) node_distance_ready || !node_distance_ready + * Round 1, nnid = nid = NUMA_NO_NODE; + * + * 3) !node_distance_ready + * Round 1, nnid = nid; + * ::Round 2, currently only applicable for alloc_func_type = <0> + * Round 2, nnid = NUMA_NO_NODE; + * 4) node_distance_ready + * Round 1, LOCAL_DISTANCE, nnid = nid; + * Round ?, nnid = nearest nid; + */ + if (!node_distance_ready() || (nid == NUMA_NO_NODE)) { + nnid = (++round == 1) ? nid : NUMA_NO_NODE; + } else { + int i, distance = INT_MAX; + + for_each_clear_bit(i, nodes_map, MAX_NUMNODES) + if (node_distance(nid, i) < distance) { + nnid = i; + distance = node_distance(nid, i); + } + } + + switch (alloc_func_type) { + case 0: + pa = memblock_find_in_range_node(size, align, start, end, nnid, flags); + break; + + case 1: + default: + pa = memblock_alloc_nid(size, align, nnid); + if (!node_distance_ready()) + return pa; + } + + if (!pa && (nnid != NUMA_NO_NODE)) { + bitmap_set(nodes_map, nnid, 1); + goto again; + } + + return pa; +} + phys_addr_t __init memblock_alloc_try_nid(phys_addr_t size, phys_addr_t align, int nid) { - phys_addr_t res = memblock_alloc_nid(size, align, nid); + phys_addr_t res = memblock_alloc_near_nid(size, align, 0, 0, nid, 0, 1); if (res) return res; @@ -1276,19 +1338,11 @@ static void * __init memblock_virt_alloc_internal( max_addr = memblock.current_limit; again: - alloc = memblock_find_in_range_node(size, align, min_addr, max_addr, - nid, flags); + alloc = memblock_alloc_near_nid(size, align, min_addr, max_addr, + nid, flags, 0); if (alloc) goto done; - if (nid != NUMA_NO_NODE) { - alloc = memblock_find_in_range_node(size, align, min_addr, - max_addr, NUMA_NO_NODE, - flags); - if (alloc) - goto done; - } - if (min_addr) { min_addr = 0; goto again;