From patchwork Tue Nov 13 05:49:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sasha Levin X-Patchwork-Id: 10679569 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 41A7C13BB for ; Tue, 13 Nov 2018 05:50:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2928B2A261 for ; Tue, 13 Nov 2018 05:50:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 275A42A2A0; Tue, 13 Nov 2018 05:50:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 682B82A261 for ; Tue, 13 Nov 2018 05:50:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB1426B026A; Tue, 13 Nov 2018 00:50:46 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A344E6B026B; Tue, 13 Nov 2018 00:50:46 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D6A56B026C; Tue, 13 Nov 2018 00:50:46 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f198.google.com (mail-pl1-f198.google.com [209.85.214.198]) by kanga.kvack.org (Postfix) with ESMTP id 4428F6B026A for ; Tue, 13 Nov 2018 00:50:46 -0500 (EST) Received: by mail-pl1-f198.google.com with SMTP id b8-v6so8725266pls.11 for ; Mon, 12 Nov 2018 21:50:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+iQVxCnKMsG61tjpwUKcD3voOfFRYpNzkDmKgeptgDY=; b=FaiOtWHHmyW3EVTUOSc6M+qSejy4E8vbPso8nHdghVmhMiKBDpXVZ1n8okHkjg7rMt lJu7B4rPmnzMbFJwgd00NbPXpyRaansxuRhx8hP0cDEyN7lUQhombgGzWQap56GQLFSt V2tB2FB72RnJH8433r5arRGNO69qnB1DeDhA3gbm2NQzNyw4r/QctFBAv00YFNWbB6CW REcW5zqey4gae9KwWW979w01pRFuczOArcdw1SUaUg57jzSjqmo8eg+iIQ40ZDfFeGO+ +ERVXRnXFj1Dd4MRKVVd57ijO9eT5447LPQExPrOQAXCUT/MAAIPEW4EeccX0JmPH7z3 UuNA== X-Gm-Message-State: AGRZ1gLPgzPUdIAr1W5oi8Z8kk9a3Md6ZvXl1kbDNYO9waa2teOdrgtP mE42InaHEEUw3Lk+j8Ew+aCgQdwakj3mxdWBheHpJy0ROnNeuoVeechIEBCmW7L9Zy+YLZlAqri 0nTHAw73kz6TpCFgiMm/uPcIsyykUpHxTvprddsDHRA+oBfl3geOKqP7R4/4vl302DQ== X-Received: by 2002:a63:cf08:: with SMTP id j8mr3593761pgg.113.1542088245920; Mon, 12 Nov 2018 21:50:45 -0800 (PST) X-Google-Smtp-Source: AJdET5cZNZYF9IK6wmI7aADMGoqas/u3fVUH67M+LUhlK9umEMeMkt7wZxohia8YNQPp+od+HvYb X-Received: by 2002:a63:cf08:: with SMTP id j8mr3593723pgg.113.1542088244644; Mon, 12 Nov 2018 21:50:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542088244; cv=none; d=google.com; s=arc-20160816; b=qEhCZJ+hbGF4KTObm/xIFsepa8ztv1cPP4TL+aymp6p0nJJuodN6hIBux+J//L05n9 lV0HPV7qgFWewHm/guIces3ZmIav2NYh05NkvTh0oi+HIP5UxU1YYpLW3qnbAn70gqFF /spSCq+8tQ8ZDTMhnYH8vNlrRt7x5EhDQyw6bbLxkSUVN1POv5YHuYQYLYeWtn2Mpp4L 9rUJX+UWTqHFDDAgsYIIi7QfDjGpUjXqJCeKIQQMPU8CYYymKiahEdxftsYp2nOURF3x AomQ5jHmYawijhcE/qXcrKjbzBw5EYKP7vZHjaMz3bgVLl6Z6uXqfroOnB/qtdezi8uk 0EQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=+iQVxCnKMsG61tjpwUKcD3voOfFRYpNzkDmKgeptgDY=; b=FgBlUK40tu7Fj11GmGgY0DF9KanCd6SVQzH3CF6/ZcTnyw1wyPjjxy8XQLvHwySdXs SfudnMB9sD1VWRmihQSR698oFPQGchfon4p7ZIhL6tYbzW2eUB31Ff6MWlf+rpQpIrsp bG/uvFaZid2HIE4YQiVfXcMpbgQsuTZ42jMmsPLuyvtLFOYqZ84Y5AmuW6j7RO1wqlX6 3nV2JHmW/ykH88GXWhCBQYtf7sn1x5SXiqI4mAOottfDWk7/q0+FlAp+bF+yaXhgnG+1 6rh18ODkrrkMDAM9oYNbqneldT1XqpQ+WQrauGlx6gPbbnV+I1wo2rC0JOh95MKd4pIc licg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=WUyNx8vj; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from mail.kernel.org (mail.kernel.org. [198.145.29.99]) by mx.google.com with ESMTPS id l19si19711091pgm.432.2018.11.12.21.50.44 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Nov 2018 21:50:44 -0800 (PST) Received-SPF: pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) client-ip=198.145.29.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=WUyNx8vj; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sasha-vm.mshome.net (unknown [64.114.255.114]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 093CD22507; Tue, 13 Nov 2018 05:50:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1542088244; bh=4qzpYStjZKaOMA9LijmwsvByGEcejmbmKkqFcrbtU3Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WUyNx8vj7nv/AAUsPhl9SL1+sB2jvOEA4Oiy3f4IUzh1PTPJjjOOai4ldw1Y6WJJ8 Nn507WsuSmAQZrJ0Cj2ub/YlO1+ZlTlhHxe6yMLfj3skU55SnQaOdJZFDwEzFMtgTl BAeKPCQZIvJ9pIYj9O1wQNuDs9CfMbInK/AYbvuI= From: Sasha Levin To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Pavel Tatashin , Abdul Haleem , Baoquan He , Daniel Jordan , Dan Williams , Dave Hansen , David Rientjes , Greg Kroah-Hartman , Ingo Molnar , Jan Kara , =?utf-8?b?SsOpcsO0?= =?utf-8?b?bWUgR2xpc3Nl?= , "Kirill A . Shutemov" , Michael Ellerman , Michal Hocko , Souptick Joarder , Steven Sistare , Vlastimil Babka , Wei Yang , Pasha Tatashin , Andrew Morton , Linus Torvalds , Sasha Levin , linux-mm@kvack.org Subject: [PATCH AUTOSEL 4.19 39/44] mm: calculate deferred pages after skipping mirrored memory Date: Tue, 13 Nov 2018 00:49:45 -0500 Message-Id: <20181113054950.77898-39-sashal@kernel.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181113054950.77898-1-sashal@kernel.org> References: <20181113054950.77898-1-sashal@kernel.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Pavel Tatashin [ Upstream commit d3035be4ce2345d98633a45f93a74e526e94b802 ] update_defer_init() should be called only when struct page is about to be initialized. Because it counts number of initialized struct pages, but there we may skip struct pages if there is some mirrored memory. So move, update_defer_init() after checking for mirrored memory. Also, rename update_defer_init() to defer_init() and reverse the return boolean to emphasize that this is a boolean function, that tells that the reset of memmap initialization should be deferred. Make this function self-contained: do not pass number of already initialized pages in this zone by using static counters. I found this bug by reading the code. The effect is that fewer than expected struct pages are initialized early in boot, and it is possible that in some corner cases we may fail to boot when mirrored pages are used. The deferred on demand code should somewhat mitigate this. But this still brings some inconsistencies compared to when booting without mirrored pages, so it is better to fix. [pasha.tatashin@oracle.com: add comment about defer_init's lack of locking] Link: http://lkml.kernel.org/r/20180726193509.3326-3-pasha.tatashin@oracle.com [akpm@linux-foundation.org: make defer_init non-inline, __meminit] Link: http://lkml.kernel.org/r/20180724235520.10200-3-pasha.tatashin@oracle.com Signed-off-by: Pavel Tatashin Reviewed-by: Oscar Salvador Cc: Abdul Haleem Cc: Baoquan He Cc: Daniel Jordan Cc: Dan Williams Cc: Dave Hansen Cc: David Rientjes Cc: Greg Kroah-Hartman Cc: Ingo Molnar Cc: Jan Kara Cc: Jérôme Glisse Cc: Kirill A. Shutemov Cc: Michael Ellerman Cc: Michal Hocko Cc: Souptick Joarder Cc: Steven Sistare Cc: Vlastimil Babka Cc: Wei Yang Cc: Pasha Tatashin Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Sasha Levin --- mm/page_alloc.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e2ef1c17942f..63f990b73750 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -306,24 +306,33 @@ static inline bool __meminit early_page_uninitialised(unsigned long pfn) } /* - * Returns false when the remaining initialisation should be deferred until + * Returns true when the remaining initialisation should be deferred until * later in the boot cycle when it can be parallelised. */ -static inline bool update_defer_init(pg_data_t *pgdat, - unsigned long pfn, unsigned long zone_end, - unsigned long *nr_initialised) +static bool __meminit +defer_init(int nid, unsigned long pfn, unsigned long end_pfn) { + static unsigned long prev_end_pfn, nr_initialised; + + /* + * prev_end_pfn static that contains the end of previous zone + * No need to protect because called very early in boot before smp_init. + */ + if (prev_end_pfn != end_pfn) { + prev_end_pfn = end_pfn; + nr_initialised = 0; + } + /* Always populate low zones for address-constrained allocations */ - if (zone_end < pgdat_end_pfn(pgdat)) - return true; - (*nr_initialised)++; - if ((*nr_initialised > pgdat->static_init_pgcnt) && - (pfn & (PAGES_PER_SECTION - 1)) == 0) { - pgdat->first_deferred_pfn = pfn; + if (end_pfn < pgdat_end_pfn(NODE_DATA(nid))) return false; + nr_initialised++; + if ((nr_initialised > NODE_DATA(nid)->static_init_pgcnt) && + (pfn & (PAGES_PER_SECTION - 1)) == 0) { + NODE_DATA(nid)->first_deferred_pfn = pfn; + return true; } - - return true; + return false; } #else static inline bool early_page_uninitialised(unsigned long pfn) @@ -331,11 +340,9 @@ static inline bool early_page_uninitialised(unsigned long pfn) return false; } -static inline bool update_defer_init(pg_data_t *pgdat, - unsigned long pfn, unsigned long zone_end, - unsigned long *nr_initialised) +static inline bool defer_init(int nid, unsigned long pfn, unsigned long end_pfn) { - return true; + return false; } #endif @@ -5459,9 +5466,7 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, struct vmem_altmap *altmap) { unsigned long end_pfn = start_pfn + size; - pg_data_t *pgdat = NODE_DATA(nid); unsigned long pfn; - unsigned long nr_initialised = 0; struct page *page; #ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP struct memblock_region *r = NULL, *tmp; @@ -5489,8 +5494,6 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, continue; if (!early_pfn_in_nid(pfn, nid)) continue; - if (!update_defer_init(pgdat, pfn, end_pfn, &nr_initialised)) - break; #ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP /* @@ -5513,6 +5516,8 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, } } #endif + if (defer_init(nid, pfn, end_pfn)) + break; not_early: page = pfn_to_page(pfn);