From patchwork Wed Oct 17 23:54:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 10646611 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 97CB8112B for ; Wed, 17 Oct 2018 23:54:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7A5BD285A2 for ; Wed, 17 Oct 2018 23:54:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6A20E285CE; Wed, 17 Oct 2018 23:54:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F3137285A2 for ; Wed, 17 Oct 2018 23:54:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE54B6B0003; Wed, 17 Oct 2018 19:54:05 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C6CE86B000A; Wed, 17 Oct 2018 19:54:05 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0E5E6B000C; Wed, 17 Oct 2018 19:54:05 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id 6ADB86B0003 for ; Wed, 17 Oct 2018 19:54:05 -0400 (EDT) Received: by mail-pf1-f200.google.com with SMTP id i81-v6so28412666pfj.1 for ; Wed, 17 Oct 2018 16:54:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:user-agent:mime-version :content-transfer-encoding; bh=CUqmKmIkECvQCNoayz3ndd9Jc55vXzeTy97m68+5aUY=; b=OfzsNAIxQ70lnguxlHSkr+8q56Ozly6N/pR2zO3nCa6n2kH/nLE+NBHt/Cw5ExZdVP HrjloMKPNPEBXoX6+a0A4aM95UfKEB8M68KuUeZJVbc+gHaRuG2L79Nql9ZrThCbpiQx 6w2u/30+fO/bhIc090oSqOofH7wHnYZd2vILGnO0wsj1fBCTK1B8kzYz/EIqe0Kk6ETK UhEUup4QOJ0cbEcfk5thFSj1LUnSifjiv4HifqmxkNGZ7s2D8SUCO5/3ctOz9qdpUDS2 K4Ck2PQE0zYaI6Nqdl/4WoTR/oR6a5wcysvkikt7G8G8c00+1/LIFRGIEgf1TXzTr/cM OhSg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: ABuFfojZ0Bih2j7XWlJRmplYuYYO5uUZl1YrPRT4vnvrZxWuu6coOERG cIuwsvf2bfDe0axwjfMmJVxn8jQonuwY3PnSWBPiopWJTfbMB+iwBZVCck4pHe8EZPwyqwio+fs AXej454uAs3DJAT+9F6KDdR5cpmeOUQdMwD+7MNur0upxH1N/BeaLEFpnaibRBIf3dA== X-Received: by 2002:a17:902:6ac7:: with SMTP id i7-v6mr28208832plt.268.1539820445058; Wed, 17 Oct 2018 16:54:05 -0700 (PDT) X-Google-Smtp-Source: ACcGV60qOKKSYU7xNyLLNQtNaIUkG2c7mwXEAaQJkA3s0VTBbGiOI0l3r85b3U5YwYZsgmDjXjnd X-Received: by 2002:a17:902:6ac7:: with SMTP id i7-v6mr28208789plt.268.1539820443982; Wed, 17 Oct 2018 16:54:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539820443; cv=none; d=google.com; s=arc-20160816; b=Ud2C5++r/QU59sRBUvbDrokwfk5a/LCt8LHKllAzVoK/hSHHfAQZ/Z/ad9HUGWs88J 1QrWkWuIO/6lMO5QwzJc/4c6jmEuGOkvJV7hCHzIwTc3sxVCNLY6ANW53WGQoWcgEHzX gL7QudmZQvkS8h2ArkYI6ommMeNIpvh33YnRbkl5OSB871rfNW+CjSZNq9n8fW9AVRuX iQDcFv6dh546EcvfuZOuqAv9q6SOqx79LSafHWWSFropRm0KGF9xaiovinkZRsQVG5ii eBrNWNMnZucev8/TYUVrXZ8gk4kVM6s5lEw23lPIOlgIwqD1h61rmCkLJ7o2dDIsuZUX jBjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:message-id:date :cc:to:from:subject; bh=CUqmKmIkECvQCNoayz3ndd9Jc55vXzeTy97m68+5aUY=; b=Zr2M0ilv6KdIYeoDcLTCDr1CewBfWQ/FyIJkRSZJc/JzNmdPKlceGNBqzqrFHgQU2P 5/upsX2+Gd71bjhw9LOyIwKOnGOGQOgT8Jv8Gf647CE0d8j33FdvbLvXS7E2eXEuupic Pv4zwOYycLKSs0pLT1AHc/e5Cehn2xoYgzsY3XmsIcgqCfVZIQtRvZkahhOWSiz1kkZD M806Sz6IzQHAgLmQNvGMQeONiJ7HF+vxSNmkF8XZiQmZTPoUGG9HNjPDOz6RTITCdgYZ DO0VhtdaInNKWhIdYiH+hIXItq9fplp/vwmnCrz+GiJQQGe+kBxuvicrV5rsUb/WARyI DULw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga05.intel.com (mga05.intel.com. [192.55.52.43]) by mx.google.com with ESMTPS id 202-v6si17961131pgb.63.2018.10.17.16.54.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 17 Oct 2018 16:54:03 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.43 as permitted sender) client-ip=192.55.52.43; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Oct 2018 16:54:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,393,1534834800"; d="scan'208";a="100366532" Received: from ahduyck-mobl.amr.corp.intel.com (HELO localhost.localdomain) ([10.7.198.154]) by orsmga001.jf.intel.com with ESMTP; 17 Oct 2018 16:54:02 -0700 Subject: [mm PATCH v4 0/6] Deferred page init improvements From: Alexander Duyck To: linux-mm@kvack.org, akpm@linux-foundation.org Cc: pavel.tatashin@microsoft.com, mhocko@suse.com, dave.jiang@intel.com, alexander.h.duyck@linux.intel.com, linux-kernel@vger.kernel.org, willy@infradead.org, davem@davemloft.net, yi.z.zhang@linux.intel.com, khalid.aziz@oracle.com, rppt@linux.vnet.ibm.com, vbabka@suse.cz, sparclinux@vger.kernel.org, dan.j.williams@intel.com, ldufour@linux.vnet.ibm.com, mgorman@techsingularity.net, mingo@kernel.org, kirill.shutemov@linux.intel.com Date: Wed, 17 Oct 2018 16:54:02 -0700 Message-ID: <20181017235043.17213.92459.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patchset is essentially a refactor of the page initialization logic that is meant to provide for better code reuse while providing a significant improvement in deferred page initialization performance. In my testing on an x86_64 system with 384GB of RAM and 3TB of persistent memory per node I have seen the following. In the case of regular memory initialization the deferred init time was decreased from 3.75s to 1.06s on average. For the persistent memory the initialization time dropped from 24.17s to 19.12s on average. This amounts to a 253% improvement for the deferred memory initialization performance, and a 26% improvement in the persistent memory initialization performance. I have called out the improvement observed with each patch. v1->v2: Fixed build issue on PowerPC due to page struct size being 56 Added new patch that removed __SetPageReserved call for hotplug v2->v3: Rebased on latest linux-next Removed patch that had removed __SetPageReserved call from init Added patch that folded __SetPageReserved into set_page_links Tweaked __init_pageblock to use start_pfn to get section_nr instead of pfn v3->v4: Updated patch description and comments for mm_zero_struct_page patch Replaced "default" with "case 64" Removed #ifndef mm_zero_struct_page Fixed typo in comment that ommited "_from" in kerneldoc for iterator Added Reviewed-by for patches reviewed by Pavel Added Acked-by from Michal Hocko Added deferred init times for patches that affect init performance Swapped patches 5 & 6, pulled some code/comments from 4 into 5 Did this as reserved bit wasn't used in deferred memory init --- Alexander Duyck (6): mm: Use mm_zero_struct_page from SPARC on all 64b architectures mm: Drop meminit_pfn_in_nid as it is redundant mm: Use memblock/zone specific iterator for handling deferred page init mm: Move hot-plug specific memory init into separate functions and optimize mm: Add reserved flag setting to set_page_links mm: Use common iterator for deferred_init_pages and deferred_free_pages arch/sparc/include/asm/pgtable_64.h | 30 -- include/linux/memblock.h | 58 ++++ include/linux/mm.h | 50 +++ mm/memblock.c | 63 ++++ mm/page_alloc.c | 569 +++++++++++++++++++++-------------- 5 files changed, 513 insertions(+), 257 deletions(-) --