From patchwork Fri Nov 30 21:52:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 10707243 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8967717D5 for ; Fri, 30 Nov 2018 21:52:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 79CC730618 for ; Fri, 30 Nov 2018 21:52:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6DA7A305F8; Fri, 30 Nov 2018 21:52:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 03DA22FFE7 for ; Fri, 30 Nov 2018 21:52:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B09486B5A6A; Fri, 30 Nov 2018 16:52:50 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A907A6B5A6B; Fri, 30 Nov 2018 16:52:50 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 97F346B5A6C; Fri, 30 Nov 2018 16:52:50 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) by kanga.kvack.org (Postfix) with ESMTP id 549AC6B5A6A for ; Fri, 30 Nov 2018 16:52:50 -0500 (EST) Received: by mail-pl1-f200.google.com with SMTP id 89so5050558ple.19 for ; Fri, 30 Nov 2018 13:52:50 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:user-agent:mime-version :content-transfer-encoding; bh=PYAwimsLhSGGH9KvHoEqVy4p6KAMwSDm54/NyUGx+08=; b=Y15GZKY4RFefr1ujaXPdBNRB/ynjq3WV/rNhis7fgcXRd8rFqUyq/W6hSymEuNg55u ET8qNQsTa9p44kfYsyK7m01xaX23wTXyJVplwlC2ZZZha8jgcloQFLQMSVOV2/lkq5nA uJ21GougPsOx5XfHcqdx4c0XzDgtFRvyYRFQqIweaTsxZGSChiMe9E/C+SlBqjExrYG0 K6yi6PoskdmqmOUmW25MqqdnE2FBTH0N2PYQutVGpXpbUZ8N60e8H9icEBKsRCy9VQtR lNEN9gYnqucouZpa6jm3uRN6Xnd0EZoBtME3hwyGBjM406GmUpEViEhr7ijgjs+Ijsdh XqZw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AA+aEWYE/jAFQz5GYMmphWer1reNnU8jJRz90pUjC6NmHUrihianf/m/ EhMeP8Z1mTskbRJGnaSgwJvFZzq1gJRXfR4aqwbBGCKIGbycRWjOdAnXzUkE9PQfbPJUVJQc95w UmcW0eTVw5xhm2gJUrLMMk65qlorly+5W0eFgpgCqBrqgE0Nr6rjgkybtKVSLbARTSA== X-Received: by 2002:aa7:8497:: with SMTP id u23mr7185763pfn.220.1543614769999; Fri, 30 Nov 2018 13:52:49 -0800 (PST) X-Google-Smtp-Source: AFSGD/WvmrLy8yHYIM+s9C1ZeuOONNRCgdBGuGwYrAt9sRBzOxW9dzf2d7CXnEzhs0gUUXnSn3bO X-Received: by 2002:aa7:8497:: with SMTP id u23mr7185718pfn.220.1543614768935; Fri, 30 Nov 2018 13:52:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543614768; cv=none; d=google.com; s=arc-20160816; b=fDJd7kJgaJMtnaSTRwzYE0fy+XQNyMOx5idAPYzZ6Hn6UR1Xc22RDwMnIwb/qr5eM4 dTd0OjA6emTTiSuC9GYD2POfrYUs+XwlXsMhvyHe+4Q7yvItPURg0/ujIrFXqnkBAtUi e30H6WSSDyx47CHjTWMSDZgeKKKG9FOjBPNCEl7MTP50aLRisO6GGZ6TMiGgQNg+98/b iwR8y7fd41ygvUiz79zxYx4maC2s7VUJRjjeYAJoBxWq+GLEsRovBUsT+2FyBZYAAGaS 66t8sxK5JqDWRYwrer8yjpYqdXV0hDe/VLMXHkyviEvdV7xOpV0OIiaIDXasl3pv+myr tnLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:message-id:date :cc:to:from:subject; bh=PYAwimsLhSGGH9KvHoEqVy4p6KAMwSDm54/NyUGx+08=; b=DNCjxc+gl3kwc5Vvqv2r+gddVJbViqnY/ngVxPA22ag6OQkTV5ob8B0YzUHgz8HYRd fcyyD8MI4dJbIVx9ep3pH8T7spxKq7qUZLJ0wxhV7HkBrwCGPTJUt+/Z+ODT55LHjT4H oE1/rIGmzM5e5yXHsuzlcU15r5pumTwLeprs7awy8a0eSwRaSdmOTOcrwgD6+3T7Hl97 OhUA8covHLYQI+2NZWC//fIg/7Uf/2eeej5FUV/tEYNzC8KJyksbOnjazEweuXqNlNCz 4RLlpMGqGbbrZjo1nNvs2n+KxGzznhyEa1HY0wX1Y1z6XElzxpzRtzePDuzFxVSC1G0G fS4A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id s11si5832718pgk.344.2018.11.30.13.52.48 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 30 Nov 2018 13:52:48 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 Nov 2018 13:52:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,300,1539673200"; d="scan'208";a="114393636" Received: from ahduyck-desk1.jf.intel.com (HELO ahduyck-desk1.amr.corp.intel.com) ([10.7.198.76]) by orsmga001.jf.intel.com with ESMTP; 30 Nov 2018 13:52:48 -0800 Subject: [mm PATCH v6 0/7] Deferred page init improvements From: Alexander Duyck To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: sparclinux@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org, davem@davemloft.net, pavel.tatashin@microsoft.com, mhocko@suse.com, mingo@kernel.org, kirill.shutemov@linux.intel.com, dan.j.williams@intel.com, dave.jiang@intel.com, alexander.h.duyck@linux.intel.com, rppt@linux.vnet.ibm.com, willy@infradead.org, vbabka@suse.cz, khalid.aziz@oracle.com, ldufour@linux.vnet.ibm.com, mgorman@techsingularity.net, yi.z.zhang@linux.intel.com, alexander.h.duyck@linux.intel.com Date: Fri, 30 Nov 2018 13:52:48 -0800 Message-ID: <154361452447.7497.1348692079883153517.stgit@ahduyck-desk1.amr.corp.intel.com> User-Agent: StGit/unknown-version MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patchset is essentially a refactor of the page initialization logic that is meant to provide for better code reuse while providing a significant improvement in deferred page initialization performance. In my testing on an x86_64 system with 384GB of RAM and 3TB of persistent memory per node I have seen the following. In the case of regular memory initialization the deferred init time was decreased from 3.75s to 1.06s on average. For the persistent memory the initialization time dropped from 24.17s to 19.12s on average. This amounts to a 253% improvement for the deferred memory initialization performance, and a 26% improvement in the persistent memory initialization performance. I have called out the improvement observed with each patch. Note: This patch set is meant as a replacment for the v5 set that is already in the MM tree. I had considered just doing incremental changes but Pavel at the time had suggested I submit it as a whole set, however that was almost 3 weeks ago so if incremental changes are preferred let me know and I can submit the changes as incremental updates. I appologize for the delay in submitting this follow-on set. I had been trying to address the DAX PageReserved bit issue at the same time but that is taking more time than I anticipated so I decided to push this before the code sits too much longer. Commit bf416078f1d83 ("mm/page_alloc.c: memory hotplug: free pages as higher order") causes issues with the revert of patch 7. It was necessary to replace all instances of __free_pages_boot_core with __free_pages_core. v1->v2: Fixed build issue on PowerPC due to page struct size being 56 Added new patch that removed __SetPageReserved call for hotplug v2->v3: Rebased on latest linux-next Removed patch that had removed __SetPageReserved call from init Added patch that folded __SetPageReserved into set_page_links Tweaked __init_pageblock to use start_pfn to get section_nr instead of pfn v3->v4: Updated patch description and comments for mm_zero_struct_page patch Replaced "default" with "case 64" Removed #ifndef mm_zero_struct_page Fixed typo in comment that ommited "_from" in kerneldoc for iterator Added Reviewed-by for patches reviewed by Pavel Added Acked-by from Michal Hocko Added deferred init times for patches that affect init performance Swapped patches 5 & 6, pulled some code/comments from 4 into 5 v4->v5: Updated Acks/Reviewed-by Rebased on latest linux-next Split core bits of zone iterator patch from MAX_ORDER_NR_PAGES init v5->v6: Rebased on linux-next with previous v5 reverted Drop the "This patch" or "This change" from patch desriptions. Cleaned up patch descriptions for patches 3 & 4 Fixed kerneldoc for __next_mem_pfn_range_in_zone Updated several Reviewed-by, and incorporated suggestions from Pavel Added __init_single_page_nolru to patch 5 to consolidate code Refactored iterator in patch 7 and fixed several issues --- Alexander Duyck (7): mm: Use mm_zero_struct_page from SPARC on all 64b architectures mm: Drop meminit_pfn_in_nid as it is redundant mm: Implement new zone specific memblock iterator mm: Initialize MAX_ORDER_NR_PAGES at a time instead of doing larger sections mm: Move hot-plug specific memory init into separate functions and optimize mm: Add reserved flag setting to set_page_links mm: Use common iterator for deferred_init_pages and deferred_free_pages arch/sparc/include/asm/pgtable_64.h | 30 -- include/linux/memblock.h | 41 +++ include/linux/mm.h | 50 +++ mm/memblock.c | 64 ++++ mm/page_alloc.c | 571 +++++++++++++++++++++-------------- 5 files changed, 498 insertions(+), 258 deletions(-) --