From patchwork Mon Sep 10 23:43:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 10594911 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A7E5214E0 for ; Mon, 10 Sep 2018 23:43:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 962BD291BB for ; Mon, 10 Sep 2018 23:43:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8A751291F4; Mon, 10 Sep 2018 23:43:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1C547291BB for ; Mon, 10 Sep 2018 23:43:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 300718E0006; Mon, 10 Sep 2018 19:43:51 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2884D8E0001; Mon, 10 Sep 2018 19:43:51 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 14F928E0006; Mon, 10 Sep 2018 19:43:51 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) by kanga.kvack.org (Postfix) with ESMTP id C92A18E0001 for ; Mon, 10 Sep 2018 19:43:50 -0400 (EDT) Received: by mail-pg1-f199.google.com with SMTP id f13-v6so11374655pgs.15 for ; Mon, 10 Sep 2018 16:43:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:subject:from:to:cc:date :message-id:in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=lhFr3T5RF4ix/HN80RMUA8xymXKjrz1rlCqR/lwO+vo=; b=DzoQRO0g/M04CGo1m053bDDSLGjX+qNFNkv9OOu1EhmdUEqpDL7tmBgCe3aTxQ50JW YLiWzg+Kqpw9ffsdGiFX9+rmg/egfBQHZQ3vCCf/Uh9IUTI7eZFfdB+lh5tALZ7d1aqR tH9IE5sXWH1LDblFr+/ZZOMaO/6ur6Wqv+mlpus0Zdm1Y8z9PWdk2hxrm+E/n0BioCjI Zo0Tgt7bnTQ2+TwinsM/jrbnlsa5gZiU1yFZkl0DIs+2eMZ9ZLQLS+xF8lojOScdgxdk VrX6yzcCopl5OAjqp0vlg/OXDYdrdBZKTheyl3OnioVnnvg9R3UjAOgjNPdDdT2QFZ+H oXyw== X-Gm-Message-State: APzg51B4v5Q5JquY+ff5j7mzy068WSA3/N+62SKCh47ysc019Jd2gAf+ E8mL3CzSBWNHjwJhq7yAWm5Jwq2PrqtCqHsCpo3MhMhN2UNKj8lwcz4/APEoaXMPlcnqZ/yEN7b TQrVBG1pyrXlCzvXhUUo4nWbpTQ/SuMSeuehf0moxoVFmHu4iCv923NYxtyAwwOELLDDLBsdIf1 VhMX0LaVOWIblzM5UUxM4KWFe7S2JTWWlzSRSEGa+z7DX/6/dsOyTe9SrtWFOQPlDHF2j31q1GD SHxQjAu3ONxoHbm/iPj/src+zix58KrXQ1X+FXj3kBRDbNBH8pBQr93KCxjkjWBXrGyY5wKMrHD ij2aUkwY24+T0J4wtrZzaLaweUF9fA6rd0jAkXZa2PYWfkaeUngD/DmG0dMKF3SQQOUT6Dv0Pn+ Q X-Received: by 2002:a63:610:: with SMTP id 16-v6mr24746687pgg.96.1536623030483; Mon, 10 Sep 2018 16:43:50 -0700 (PDT) X-Received: by 2002:a63:610:: with SMTP id 16-v6mr24746655pgg.96.1536623029707; Mon, 10 Sep 2018 16:43:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536623029; cv=none; d=google.com; s=arc-20160816; b=pCn8FlYNxKvR70F9ebw9Xj391MsTKEQL7kTDCgh9Oo8VpaI2NhxlgAMlVN3HDytX92 GEgOwGTWm+dB0i3uOyPBaxyfBvFaMSG48aZRFrpDHMoidM6X3kEDqnf2AX53lijE6+Qe NO3Xa1utsQInSzjFncff3k2JgwmCF/5a/wduVhPqtOGLCV5eqDHXNbIu3LmzSK1CmU2b MNLi2mRjYefGPPCwTAsIOv+FsvJk1bOSu3WhNhntZ3a7xP7T2YTwqjZj5Ag4y9XcAlyl vHk3LZkfb3JHTuxYJZJvfJP4IFC7QDXzOPB9g+qPqlyorZGJNyY9gNkI3mzbMHBr+ARs p+LQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject:dkim-signature; bh=lhFr3T5RF4ix/HN80RMUA8xymXKjrz1rlCqR/lwO+vo=; b=ExNk0zgP7nwi5Qtvpzw1SKIPvqqPZAdh/5eJ0juU7YCYt/X3curfPNHjpm6ReWm99X SB/jkiXIbtYptzdbT5xM/p3VZvbVd6OMmHeYfQ3CKThfBVmEE+ewAbWOh7gma0UFeZKs xFEEOtjIxsQGeuCG7lyiu6wLj7HZScywkWiPTxK+MRLCKkUi74gzS599HKT+mn/3WuEK QzffSsxY39QUZ6PWERfBFfwoW3ZAlZ9/Vqb2lBx+vh/JKkkEMfrLW2PIBg6MJRbKHdUr OGsUk3xjr+HdAwCEKo7KwNZiLvEeFX3cIQiVdQNmKvJFFCL2u5skRto4qh5ypoG6ct9h 4ntw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=M5jUdHnp; spf=pass (google.com: domain of alexander.duyck@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=alexander.duyck@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id t8-v6sor3072510plz.147.2018.09.10.16.43.49 for (Google Transport Security); Mon, 10 Sep 2018 16:43:49 -0700 (PDT) Received-SPF: pass (google.com: domain of alexander.duyck@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=M5jUdHnp; spf=pass (google.com: domain of alexander.duyck@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=alexander.duyck@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=lhFr3T5RF4ix/HN80RMUA8xymXKjrz1rlCqR/lwO+vo=; b=M5jUdHnplNrgx3jtLSwG+/FiSgIS5lu66K/26ReK6zbipcdAmf2HbSKpXwL++aiju+ BBPLh+5tfixgC9jksDYNDyOzwlV9qKrTrb3wq1Avw2EtcbsmSbb5ib0kltUcSCuZkk1L tUJM4vwwtCW1eWbbOxG5/eM08A/nR81VfKIwm1GYNN9jiWxeqdtUkTee2LITfK3kV+oX bQyuH29EMKTNsd9GC774DuOq9xIPkqxxgsQSh26nIvJHbXrSG0u2q385g0pcmvaJKEfQ 9nFUXSV5uLTUIsICYY0eagbcIS8YZ9EPAqXl4PMy2TeTZ2GAVKo9jxcEUWk3UUtKBzn5 PotA== X-Google-Smtp-Source: ANB0VdZ9nofLh3IK/Vq5oODEvu42HmX3XMswWPrs5u5Htg1ryZEmW2HxTof/tUVek3Nw+LtN1MGgcw== X-Received: by 2002:a17:902:344:: with SMTP id 62-v6mr24533133pld.164.1536623029274; Mon, 10 Sep 2018 16:43:49 -0700 (PDT) Received: from localhost.localdomain (static-50-53-21-37.bvtn.or.frontiernet.net. [50.53.21.37]) by smtp.gmail.com with ESMTPSA id v20-v6sm33973962pfk.12.2018.09.10.16.43.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 10 Sep 2018 16:43:48 -0700 (PDT) Subject: [PATCH 2/4] mm: Create non-atomic version of SetPageReserved for init use From: Alexander Duyck To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org Cc: pavel.tatashin@microsoft.com, mhocko@suse.com, dave.jiang@intel.com, mingo@kernel.org, dave.hansen@intel.com, jglisse@redhat.com, akpm@linux-foundation.org, logang@deltatee.com, dan.j.williams@intel.com, kirill.shutemov@linux.intel.com Date: Mon, 10 Sep 2018 16:43:48 -0700 Message-ID: <20180910234348.4068.92164.stgit@localhost.localdomain> In-Reply-To: <20180910232615.4068.29155.stgit@localhost.localdomain> References: <20180910232615.4068.29155.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Alexander Duyck It doesn't make much sense to use the atomic SetPageReserved at init time when we are using memset to clear the memory and manipulating the page flags via simple "&=" and "|=" operations in __init_single_page. This patch adds a non-atomic version __SetPageReserved that can be used during page init and shows about a 10% improvement in initialization times on the systems I have available for testing. On those systems I saw initialization times drop from around 35 seconds to around 32 seconds to initialize a 3TB block of persistent memory. I tried adding a bit of documentation based on commit ("mm, memory_hotplug: do not associate hotadded memory to zones until online"). Ideally the reserved flag should be set earlier since there is a brief window where the page is initialization via __init_single_page and we have not set the PG_Reserved flag. I'm leaving that for a future patch set as that will require a more significant refactor. Acked-by: Michal Hocko Signed-off-by: Alexander Duyck Reviewed-by: Pavel Tatashin --- include/linux/page-flags.h | 1 + mm/page_alloc.c | 17 +++++++++++++++-- 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index d00216cf00f8..1b1f8e0378ae 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -300,6 +300,7 @@ static inline void page_init_poison(struct page *page, size_t size) PAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) __CLEARPAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) + __SETPAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) PAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) __CLEARPAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) __SETPAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 89d2a2ab3fe6..a9b095a72fd9 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1231,7 +1231,12 @@ void __meminit reserve_bootmem_region(phys_addr_t start, phys_addr_t end) /* Avoid false-positive PageTail() */ INIT_LIST_HEAD(&page->lru); - SetPageReserved(page); + /* + * no need for atomic set_bit because the struct + * page is not visible yet so nobody should + * access it yet. + */ + __SetPageReserved(page); } } } @@ -5517,8 +5522,16 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, not_early: page = pfn_to_page(pfn); __init_single_page(page, pfn, zone, nid); + + /* + * Mark page reserved as it will need to wait for onlining + * phase for it to be fully associated with a zone. + * + * We can use the non-atomic __set_bit operation for setting + * the flag as we are still initializing the pages. + */ if (context == MEMMAP_HOTPLUG) - SetPageReserved(page); + __SetPageReserved(page); /* * Mark the block movable so that blocks are reserved for