From patchwork Thu Jul 26 19:35:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Tatashin X-Patchwork-Id: 10546341 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D4E2414E2 for ; Thu, 26 Jul 2018 19:35:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CB9D02B685 for ; Thu, 26 Jul 2018 19:35:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BFB052BCDA; Thu, 26 Jul 2018 19:35:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 38D182B685 for ; Thu, 26 Jul 2018 19:35:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D7CB06B0010; Thu, 26 Jul 2018 15:35:34 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CE1606B0266; Thu, 26 Jul 2018 15:35:34 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE4DA6B026A; Thu, 26 Jul 2018 15:35:34 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ua0-f198.google.com (mail-ua0-f198.google.com [209.85.217.198]) by kanga.kvack.org (Postfix) with ESMTP id 72CC56B0010 for ; Thu, 26 Jul 2018 15:35:34 -0400 (EDT) Received: by mail-ua0-f198.google.com with SMTP id j12-v6so830420uaq.16 for ; Thu, 26 Jul 2018 12:35:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:subject:date:message-id :in-reply-to:references; bh=HsscRZRuyWCD3fYHB04D+mmajsGAkMaKwj+qJalzQ1o=; b=fKdFwMJD6tSi5L3uNctZkY/xWsOvSn4oPJoh2KUucc4dTlXiuy+b64yFFRZhmxbZpB OvTluweqxsg2lB+yDixvgObPpas+HGpD0CvEQwFLcIg/5O8dQrrlbFV3hYywm99cMlgL 8TwxX+GRY7efJDB6jU55RlHtLxni8bgQYYG+HygtVN85Iqrr4y2EG/LzHrljw2SKnvoW 3qqaceN+BQY5S9xSNN99qDFVMrmGnLVbFlfaQLXI3rw7FiHtb+0Rz1qGZbnix35g6ayW yVtCudMn6x/YPiCt3+X1CrYH0ZtFti3x0N2jftnoPCOwNpA0dDa0zTE5XFK5bi1u0BqT MUhA== X-Gm-Message-State: AOUpUlFbU+llP+qVjvjKPAueg/Lxh3xtiaAK3wS6jcfN0Np9KOHjUe0I Yjry1up1OqyPn2PYYilIyQpgKK2IIW+y9Z/v2qyNxM/L4qvbXNH6YNVwhYQ2Nw/UHSnoXcr2H5Z TQbfWB+M1hSQG3EudTfJwi2Uo20QTGMErH1LdCsHohu40PpQThjHIs/Kpcf6cixR57Q== X-Received: by 2002:a1f:26ce:: with SMTP id m197-v6mr2067934vkm.115.1532633734137; Thu, 26 Jul 2018 12:35:34 -0700 (PDT) X-Google-Smtp-Source: AAOMgpeGXYVWOVhs6myX6MY0lZdqNGhZ2fJzDblUFq2icOxhv/t9Yx9j+6mIJLlM8M1eOFGneLKR X-Received: by 2002:a1f:26ce:: with SMTP id m197-v6mr2067911vkm.115.1532633733443; Thu, 26 Jul 2018 12:35:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1532633733; cv=none; d=google.com; s=arc-20160816; b=0X3wCm3L9C+NxEqebk5bDr1KD3R+A7FRGQhfgke2JRb9NOfleFI3hLIpZcyrbsJySB BVH6b1yyejUR83d6LeW4O6cN3JefN8jkPBNHyjkmy4opQZvv0Qqc/KlUnvOKoCdXCJ4B wiBbhkRKGTexmaUTt9tMAethmjOF1ieshzt+EELNCAqnSNh3QrxwC4phL6/20qDiKfhY HWLVX1WjBbtD6elBg9srcZCIHh0srvJvY4OWSsYsupsbR2UQ1v44mB4beXRE3o0coXZV 9CGhPCnitEtnFBlJtRvrCqzGPiTJo9Mh7mrjdrRV6O1D7TIF+dNQ3/hdAQrWTbHHLHm0 37YA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:to:from :dkim-signature:arc-authentication-results; bh=HsscRZRuyWCD3fYHB04D+mmajsGAkMaKwj+qJalzQ1o=; b=KvOxv/qRxASNy2aOm+HdgvDzwWcEyDArmA/Pk2ugxr6A5iYX/TyAnHpwUyr1tQ5Rw6 yZIvdiPuHT/WTUbtUn0gimqwdrVQqhfQEaiicprpIkxo9OnUOLhRU60shQsRX7YV4VI8 CKaJvGEGwHV7s29BFTcx/v24dysXFfZhnZJy1YQERK/asSXq6WNjRohFj+JlGYRhGDgI Ng7qR8YtVGL0ysZuvmyYDtUwlJmAokqMXEjsnmJFOF6z1QcdxUBuv3kWBVGv8/3Mb3HH T5absh+gIHCPcBY/0awkEOIks8gXAOdUTfDs6Tqd2Zf5g9TuzjniWaR6x7NDRNEuNWAO MMwg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=OFwqRlbJ; spf=pass (google.com: domain of pasha.tatashin@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=pasha.tatashin@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2120.oracle.com (userp2120.oracle.com. [156.151.31.85]) by mx.google.com with ESMTPS id x72-v6si748474vkd.96.2018.07.26.12.35.33 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Jul 2018 12:35:33 -0700 (PDT) Received-SPF: pass (google.com: domain of pasha.tatashin@oracle.com designates 156.151.31.85 as permitted sender) client-ip=156.151.31.85; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=OFwqRlbJ; spf=pass (google.com: domain of pasha.tatashin@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=pasha.tatashin@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w6QJShvs123843; Thu, 26 Jul 2018 19:35:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=HsscRZRuyWCD3fYHB04D+mmajsGAkMaKwj+qJalzQ1o=; b=OFwqRlbJZGibVEo8iNrxqt9Hk0uyW9IHoWSVLchAflSSa6ipqJQTtrGWCjVAlRIFWuHZ IASOKaVrQAX3IL2LsD4VIIxP4dipVTIZs/FVdyZVVnXxdRwkS0+DdfVNzgqikIdTtlzm gSIbdvpP5eKy6heI/IQZQi1cFQn3c5aOBjsMogbOzbekorBJm6VrI0rRxDwpVhR6vSCh r0Sc+lp0fa1Lu8HAFahRzzOsNkQQa77WjMaLllrz0Z7aFoLVWVbVhzh6s3n43Qku0li9 gqtoSCjIeB+Yogihv2vyk4OLkUnbXrEWbpIPvy+DLrOWP3zoascrKqaTEkb0QRyJtOi0 gg== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2kbwfq4h5b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 26 Jul 2018 19:35:22 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w6QJZLoE021292 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 26 Jul 2018 19:35:21 GMT Received: from abhmp0012.oracle.com (abhmp0012.oracle.com [141.146.116.18]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w6QJZKvi002890; Thu, 26 Jul 2018 19:35:20 GMT Received: from localhost.localdomain (/73.69.118.222) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 26 Jul 2018 12:35:20 -0700 From: Pavel Tatashin To: steven.sistare@oracle.com, daniel.m.jordan@oracle.com, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, kirill.shutemov@linux.intel.com, mhocko@suse.com, linux-mm@kvack.org, dan.j.williams@intel.com, jack@suse.cz, jglisse@redhat.com, jrdr.linux@gmail.com, bhe@redhat.com, gregkh@linuxfoundation.org, vbabka@suse.cz, richard.weiyang@gmail.com, dave.hansen@intel.com, rientjes@google.com, mingo@kernel.org, osalvador@techadventures.net, pasha.tatashin@oracle.com, abdhalee@linux.vnet.ibm.com, mpe@ellerman.id.au Subject: [PATCH v2 2/3] mm: calculate deferred pages after skipping mirrored memory Date: Thu, 26 Jul 2018 15:35:08 -0400 Message-Id: <20180726193509.3326-3-pasha.tatashin@oracle.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180726193509.3326-1-pasha.tatashin@oracle.com> References: <20180726193509.3326-1-pasha.tatashin@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8966 signatures=668706 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1806210000 definitions=main-1807260200 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP update_defer_init() should be called only when struct page is about to be initialized. Because it counts number of initialized struct pages, but there we may skip struct pages if there is some mirrored memory. So move, update_defer_init() after checking for mirrored memory. Also, rename update_defer_init() to defer_init() and reverse the return boolean to emphasize that this is a boolean function, that tells that the reset of memmap initialization should be deferred. Make this function self-contained: do not pass number of already initialized pages in this zone by using static counters. Signed-off-by: Pavel Tatashin Reviewed-by: Oscar Salvador --- mm/page_alloc.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 6796dacd46ac..4946c73e549b 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -306,24 +306,33 @@ static inline bool __meminit early_page_uninitialised(unsigned long pfn) } /* - * Returns false when the remaining initialisation should be deferred until + * Returns true when the remaining initialisation should be deferred until * later in the boot cycle when it can be parallelised. */ -static inline bool update_defer_init(pg_data_t *pgdat, - unsigned long pfn, unsigned long zone_end, - unsigned long *nr_initialised) +static bool __meminit +defer_init(int nid, unsigned long pfn, unsigned long end_pfn) { + static unsigned long prev_end_pfn, nr_initialised; + + /* + * prev_end_pfn static that contains the end of previous zone + * No need to protect because called very early in boot before smp_init. + */ + if (prev_end_pfn != end_pfn) { + prev_end_pfn = end_pfn; + nr_initialised = 0; + } + /* Always populate low zones for address-constrained allocations */ - if (zone_end < pgdat_end_pfn(pgdat)) - return true; - (*nr_initialised)++; - if ((*nr_initialised > pgdat->static_init_pgcnt) && - (pfn & (PAGES_PER_SECTION - 1)) == 0) { - pgdat->first_deferred_pfn = pfn; + if (end_pfn < pgdat_end_pfn(NODE_DATA(nid))) return false; + nr_initialised++; + if ((nr_initialised > NODE_DATA(nid)->static_init_pgcnt) && + (pfn & (PAGES_PER_SECTION - 1)) == 0) { + NODE_DATA(nid)->first_deferred_pfn = pfn; + return true; } - - return true; + return false; } #else static inline bool early_page_uninitialised(unsigned long pfn) @@ -331,11 +340,9 @@ static inline bool early_page_uninitialised(unsigned long pfn) return false; } -static inline bool update_defer_init(pg_data_t *pgdat, - unsigned long pfn, unsigned long zone_end, - unsigned long *nr_initialised) +static inline bool defer_init(int nid, unsigned long pfn, unsigned long end_pfn) { - return true; + return false; } #endif @@ -5459,9 +5466,7 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, struct vmem_altmap *altmap) { unsigned long end_pfn = start_pfn + size; - pg_data_t *pgdat = NODE_DATA(nid); unsigned long pfn; - unsigned long nr_initialised = 0; struct page *page; #ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP struct memblock_region *r = NULL, *tmp; @@ -5492,8 +5497,6 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, if (!early_pfn_in_nid(pfn, nid)) continue; - if (!update_defer_init(pgdat, pfn, end_pfn, &nr_initialised)) - break; #ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP /* @@ -5516,6 +5519,8 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, } } #endif + if (defer_init(nid, pfn, end_pfn)) + break; not_early: page = pfn_to_page(pfn);