From patchwork Tue Sep 25 20:20:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 10614789 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6FD85913 for ; Tue, 25 Sep 2018 20:21:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 613682AD55 for ; Tue, 25 Sep 2018 20:21:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 552922AD65; Tue, 25 Sep 2018 20:21:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E7AC72AD55 for ; Tue, 25 Sep 2018 20:21:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0978F8E00BD; Tue, 25 Sep 2018 16:21:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 047D88E00A4; Tue, 25 Sep 2018 16:21:15 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E78378E00BD; Tue, 25 Sep 2018 16:21:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id A43B18E00A4 for ; Tue, 25 Sep 2018 16:21:14 -0400 (EDT) Received: by mail-pf1-f198.google.com with SMTP id x85-v6so13241993pfe.13 for ; Tue, 25 Sep 2018 13:21:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=E9r7KhEKHjUQ1Pm7kAQ0/qTduElaq7EW1gzJiqC69fo=; b=fSoF/PFiLSjnDoLFNpcOjVkG8fQnWurWF7HiicRSHzU2LeU36xzet368lbAPf0C9XJ Bdxfi2MNmVxg37tJeg42p6AsM8gT38/imfrAmkh7h7znFlBjIDPTxVouZBJrlm53aJhM aql3mJZD86Dy6TWRmyhN5aFUM98k5cU4zpjDo+QoPk+OYDtbpseLTVaD/Y+7NeYn74M6 LUG3J80+xqEGY01Q+eWg4nlKXUmnebzYREm9qisKweVEg9/5U7Hw+isOZCehmOGATjJd OpFu8s4Kjt4DmF3+BXTI4WnPkMhBVcQRDFUHaBA+IaLgrOUPORA4FOtubyeefawmEi4h Zkqw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: ABuFfogzRPZ1dcnZmwH8kX8SlRd9aOeiWRsCA4ei2tj0O/XPsNxbhFHQ F3wBGAS+oW8yAfZi8qNeJ40F6MkzCc4Lp58iK3HRaSRwSivUFOBDsIgF3fe6eZ+lB5uEM/trT2t pLYKEl0+Rm//wTLOYigPAfokRn1Y8pRm5e0ZV3NUcVFvJooUMRH2MIct8VLWePBjpvQ== X-Received: by 2002:a17:902:8697:: with SMTP id g23-v6mr2722647plo.292.1537906874338; Tue, 25 Sep 2018 13:21:14 -0700 (PDT) X-Google-Smtp-Source: ACcGV63e2B/NLjOecJNneWbd5wcxxugyb8IAGrrP/8zRVQO0jdm/ywvimBoDORRGOyUGabodG1sC X-Received: by 2002:a17:902:8697:: with SMTP id g23-v6mr2722583plo.292.1537906873046; Tue, 25 Sep 2018 13:21:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537906873; cv=none; d=google.com; s=arc-20160816; b=BBrI9i80gCE1fqkaPWjW+IPxk4d3lkLWMkOepjrSNLzI72lXoFDU8vKA3dP1ujqrQP oVxfWp5UPPjDI1ulMewruhRD/mfh/a04JKSut40rsjZra704lxfBMtHDF+1rzNWfMbII 5CqVYImJhQ7q66HKZsIK+8USAjj+odRltjL9IFFvZIXpHaJJCuTMvsOvxmiQpyueqa7e gWx+Uwxo4I3sytGIhpj0vU6/3ixmlv/pa8ZpYkkEUprniIW7Fyy8mFc08Ci4RaDE4tRq nUlSVudwiinHCDvt8tjkMB68pDIyl2m9SKUan2nJAPIzIlg2YfCB7IDTH3+r3cZtMS4U UvAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject; bh=E9r7KhEKHjUQ1Pm7kAQ0/qTduElaq7EW1gzJiqC69fo=; b=IXNEMHhlUwU/XVerfr/kil8AVWkAezDqtyktm0rbddayezS4EwM2dovQQklOGGh3mF BB3ewjzP8RLpZLCmPjEghFypgbCu7xtAvt8GAdlM6FloSuZ2AHMPWb+uk0zSSdWOOMg6 uyY7wfnBaRXV3gg7J9rihVb2JFha6ZpwqSSUx4y4Pjj4PL46JOY0IiP0LfLsvyv9VJvH UB7oJlsrSjDMEf+GKH1nAC5+VjZTEL3/40u+n93qRAhB5Y45hh9U8wMbHPYvJLnt7oow YmN45UTENvpQ10fwwEbTou+BrCXJhfl2VhoABnFS/gJNU7ySrVgScibqOm1uTV6103Mc Ntqg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga02.intel.com (mga02.intel.com. [134.134.136.20]) by mx.google.com with ESMTPS id c63-v6si2892404pfb.178.2018.09.25.13.21.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Sep 2018 13:21:13 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 134.134.136.20 as permitted sender) client-ip=134.134.136.20; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 25 Sep 2018 13:21:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,303,1534834800"; d="scan'208";a="77230394" Received: from ahduyck-mobl.amr.corp.intel.com (HELO localhost.localdomain) ([10.7.198.152]) by orsmga006.jf.intel.com with ESMTP; 25 Sep 2018 13:20:47 -0700 Subject: [PATCH v5 3/4] mm: Create non-atomic version of SetPageReserved for init use From: Alexander Duyck To: linux-mm@kvack.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org Cc: pavel.tatashin@microsoft.com, mhocko@suse.com, dave.jiang@intel.com, dave.hansen@intel.com, jglisse@redhat.com, rppt@linux.vnet.ibm.com, dan.j.williams@intel.com, logang@deltatee.com, mingo@kernel.org, kirill.shutemov@linux.intel.com Date: Tue, 25 Sep 2018 13:20:47 -0700 Message-ID: <20180925202018.3576.11607.stgit@localhost.localdomain> In-Reply-To: <20180925200551.3576.18755.stgit@localhost.localdomain> References: <20180925200551.3576.18755.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP It doesn't make much sense to use the atomic SetPageReserved at init time when we are using memset to clear the memory and manipulating the page flags via simple "&=" and "|=" operations in __init_single_page. This patch adds a non-atomic version __SetPageReserved that can be used during page init and shows about a 10% improvement in initialization times on the systems I have available for testing. On those systems I saw initialization times drop from around 35 seconds to around 32 seconds to initialize a 3TB block of persistent memory. I believe the main advantage of this is that it allows for more compiler optimization as the __set_bit operation can be reordered whereas the atomic version cannot. I tried adding a bit of documentation based on commit ("mm, memory_hotplug: do not associate hotadded memory to zones until online"). Ideally the reserved flag should be set earlier since there is a brief window where the page is initialization via __init_single_page and we have not set the PG_Reserved flag. I'm leaving that for a future patch set as that will require a more significant refactor. Acked-by: Michal Hocko Reviewed-by: Pavel Tatashin Signed-off-by: Alexander Duyck --- v4: Added comment about __set_bit vs set_bit to the patch description v5: No change include/linux/page-flags.h | 1 + mm/page_alloc.c | 9 +++++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 934f91ef3f54..50ce1bddaf56 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -303,6 +303,7 @@ static inline void page_init_poison(struct page *page, size_t size) PAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) __CLEARPAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) + __SETPAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) PAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) __CLEARPAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) __SETPAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 511447ac02cf..926ad3083b28 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1238,7 +1238,12 @@ void __meminit reserve_bootmem_region(phys_addr_t start, phys_addr_t end) /* Avoid false-positive PageTail() */ INIT_LIST_HEAD(&page->lru); - SetPageReserved(page); + /* + * no need for atomic set_bit because the struct + * page is not visible yet so nobody should + * access it yet. + */ + __SetPageReserved(page); } } } @@ -5512,7 +5517,7 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, page = pfn_to_page(pfn); __init_single_page(page, pfn, zone, nid); if (context == MEMMAP_HOTPLUG) - SetPageReserved(page); + __SetPageReserved(page); /* * Mark the block movable so that blocks are reserved for