From patchwork Wed Sep 12 09:26:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arun KS X-Patchwork-Id: 10597197 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6E54A920 for ; Wed, 12 Sep 2018 09:27:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5D93429B84 for ; Wed, 12 Sep 2018 09:27:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4F9A429B8D; Wed, 12 Sep 2018 09:27:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C4EC829B84 for ; Wed, 12 Sep 2018 09:27:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BBBFE8E0003; Wed, 12 Sep 2018 05:27:21 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B439D8E0001; Wed, 12 Sep 2018 05:27:21 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A0C988E0003; Wed, 12 Sep 2018 05:27:21 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) by kanga.kvack.org (Postfix) with ESMTP id 5ACBB8E0001 for ; Wed, 12 Sep 2018 05:27:21 -0400 (EDT) Received: by mail-pg1-f198.google.com with SMTP id s11-v6so788253pgv.9 for ; Wed, 12 Sep 2018 02:27:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id; bh=r4/VTKod0gV8NXfed4HWHLAS8EJF4ba+9NViZzyajPQ=; b=g5Z5e2EUsi1gIhSMr0cSLxV48Zw7wS7ZC7olT9aERVKe8gxoLH4PxRtrU2yvikDHor V9274w5k2k5g0jImQjbpRV/QdT+vZtAkYFEzkpqadYh6PN28PGLt3VU4XObUZLo6lzQ0 +wxl7LQwUqeRsbUODaleOSwEMuGeXASksgI3kordRBuKC026Hglu8wtLlVPlN0h8h12o 1upz3jsN71xm5QxhoQFtd5G9t+eawIIowCNadH/CLnSzjZ4xbZthmqmIXAnCpPOIlcJu zGhEkhcNj1rbn4sx5ZvCBIUOzT4rIqk1E+gkAeHto1jCOl2ETarYWySIyQGgML4DvA+U yykQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of arunks@qualcomm.com designates 103.229.18.197 as permitted sender) smtp.mailfrom=arunks@qualcomm.com X-Gm-Message-State: APzg51D7hgFrzfNIKKX9NHSFd7JiHxJi3+8VSmMbpR6k3e2dpYZb+Ev1 WJq3pYuVstjtxyubHqnWZnsUCI+0hbvhhE5d7JEc1ZR6cZKffSkfP6z+m7SShtx4Zd0iNhQIq3p alaebMkAzuJZbtVnOaWPnWh56eJTivr8IUTsv2wAt1wdJc4Do7VaCHVo4Y0UteD8= X-Received: by 2002:a63:e841:: with SMTP id a1-v6mr1158328pgk.126.1536744441009; Wed, 12 Sep 2018 02:27:21 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbSzwWpPAzHKS6qQuzO13piNj8bfN89z2b5jFW3mjazbvfeNYMqCBzDSTjnNOXsRtMV3v1a X-Received: by 2002:a63:e841:: with SMTP id a1-v6mr1158255pgk.126.1536744440037; Wed, 12 Sep 2018 02:27:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536744440; cv=none; d=google.com; s=arc-20160816; b=FJgCqTHqqUr3WhFfoI7B6crOaJSiC3oCx4Lk1DAdqwZQsinqjFFPi432Tqf7vBzT9m P9D+kpWcbnnxy2xfiVaNbySB+qX8x3Bs8oQ/g8txwGYfoze8/QlNXGw7yw2SZv/uk2NM WEv1NU313i1t7P37W/a3Yak59VearClP5NJs+4F8QQnAVM5fX8IANjWfQsNPQt5S7Jje Kv+agoebgATmhnpZlCcJE6oNr1IJ7dI3XDctjvTXM3Iwc7Qz0AHUbifv6Bz2mU8O5mA5 G7KDUSyn32AI/QGo5QUvXx8vweb7fhe/yEehBBS/0pjFJopLocb/v12i7+WZCIKrEg3G TsdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from; bh=r4/VTKod0gV8NXfed4HWHLAS8EJF4ba+9NViZzyajPQ=; b=fcqyjxXWycgBbfFNKTjh6ztsSjzIUx5GjXua1xUwBEOXTkxZsPC6Dq8wQXJmFdB62p 9xjUVWKciLc9m5KqtyKhyKscFVcDP4PO56r65gly2dfjh//C2bfg6+dsBKfPnnxtUQGw EbBkDySS0V2ZvM5P1Baq0ncjTAhg1xnoLIdYK/D5am45bG8i4SDBX2pInXaTpOvM0S+S GoLxq/RwskD+yFfRMvtIebF2qc0wD5Injf7lD383uqGgV3j4xyq7djqzZZMFvk28UqPx Hrl6w7hJmH6494/DsUxBaevRqSKt9xPIlVrA92Gmp/ZNsMXXcfcP3pbGgj4aZgDokgqg qkvg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of arunks@qualcomm.com designates 103.229.18.197 as permitted sender) smtp.mailfrom=arunks@qualcomm.com Received: from alexa-out-blr-01.qualcomm.com (alexa-out-blr-01.qualcomm.com. [103.229.18.197]) by mx.google.com with ESMTPS id k4-v6si469773pfc.328.2018.09.12.02.27.19 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 12 Sep 2018 02:27:19 -0700 (PDT) Received-SPF: pass (google.com: domain of arunks@qualcomm.com designates 103.229.18.197 as permitted sender) client-ip=103.229.18.197; Authentication-Results: mx.google.com; spf=pass (google.com: domain of arunks@qualcomm.com designates 103.229.18.197 as permitted sender) smtp.mailfrom=arunks@qualcomm.com X-IronPort-AV: E=Sophos;i="5.53,364,1531765800"; d="scan'208";a="188885" Received: from ironmsg02-blr.qualcomm.com ([10.86.208.131]) by alexa-out-blr-01.qualcomm.com with ESMTP/TLS/AES256-SHA; 12 Sep 2018 14:57:17 +0530 X-IronPort-AV: E=McAfee;i="5900,7806,9013"; a="1111909" Received: from blr-ubuntu-104.ap.qualcomm.com (HELO blr-ubuntu-104.qualcomm.com) ([10.79.40.64]) by ironmsg02-blr.qualcomm.com with ESMTP; 12 Sep 2018 14:57:16 +0530 Received: by blr-ubuntu-104.qualcomm.com (Postfix, from userid 346745) id F037829BF; Wed, 12 Sep 2018 14:57:15 +0530 (IST) From: Arun KS To: akpm@linux-foundation.org, dan.j.williams@intel.com, mhocko@suse.com, vbabka@suse.cz, pasha.tatashin@oracle.com, iamjoonsoo.kim@lge.com, osalvador@suse.de, arunks@codeaurora.org, malat@debian.org, gregkh@linuxfoundation.org, yasu.isimatu@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: arunks.linux@gmail.com, vinmenon@codeaurora.org Subject: [RFC] memory_hotplug: Free pages as pageblock_order Date: Wed, 12 Sep 2018 14:56:45 +0530 Message-Id: <1536744405-16752-1-git-send-email-arunks@codeaurora.org> X-Mailer: git-send-email 1.9.1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When free pages are done with pageblock_order, time spend on coalescing pages by buddy allocator can be reduced. With section size of 256MB, hot add latency of a single section shows improvement from 50-60 ms to less than 1 ms, hence improving the hot add latency by 60%. If this looks okey, I'll modify users of set_online_page_callback and resend clean patch. Signed-off-by: Arun KS --- include/linux/memory_hotplug.h | 1 + mm/memory_hotplug.c | 52 ++++++++++++++++++++++++++++++++++++------ 2 files changed, 46 insertions(+), 7 deletions(-) diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h index 34a2822..447047d 100644 --- a/include/linux/memory_hotplug.h +++ b/include/linux/memory_hotplug.h @@ -88,6 +88,7 @@ extern int test_pages_in_a_zone(unsigned long start_pfn, unsigned long end_pfn, extern void __offline_isolated_pages(unsigned long, unsigned long); typedef void (*online_page_callback_t)(struct page *page); +typedef int (*online_pages_callback_t)(struct page *page, unsigned int order); extern int set_online_page_callback(online_page_callback_t callback); extern int restore_online_page_callback(online_page_callback_t callback); diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 38d94b7..853104d 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -662,19 +662,57 @@ static void generic_online_page(struct page *page) __online_page_free(page); } +static int generic_online_pages(struct page *page, unsigned int order); +static online_pages_callback_t online_pages_callback = generic_online_pages; + +static int generic_online_pages(struct page *page, unsigned int order) +{ + unsigned long nr_pages = 1 << order; + struct page *p = page; + unsigned int loop; + + for (loop = 0 ; loop < nr_pages ; loop++, p++) { + __ClearPageReserved(p); + set_page_count(p, 0); + } + adjust_managed_page_count(page, nr_pages); + init_page_count(page); + __free_pages(page, order); + + return 0; +} + +static int online_pages_blocks(unsigned long start_pfn, unsigned long nr_pages) +{ + unsigned long pages_per_block = (1 << pageblock_order); + unsigned long nr_pageblocks = nr_pages / pages_per_block; +// unsigned long rem_pages = nr_pages % pages_per_block; + int i, ret, onlined_pages = 0; + struct page *page; + + for (i = 0 ; i < nr_pageblocks ; i++) { + page = pfn_to_page(start_pfn + (i * pages_per_block)); + ret = (*online_pages_callback)(page, pageblock_order); + if (!ret) + onlined_pages += pages_per_block; + else if (ret > 0) + onlined_pages += ret; + } +/* + if (rem_pages) + onlined_pages += online_page_single(start_pfn + i, rem_pages); +*/ + + return onlined_pages; +} + static int online_pages_range(unsigned long start_pfn, unsigned long nr_pages, void *arg) { - unsigned long i; unsigned long onlined_pages = *(unsigned long *)arg; - struct page *page; if (PageReserved(pfn_to_page(start_pfn))) - for (i = 0; i < nr_pages; i++) { - page = pfn_to_page(start_pfn + i); - (*online_page_callback)(page); - onlined_pages++; - } + onlined_pages = online_pages_blocks(start_pfn, nr_pages); online_mem_sections(start_pfn, start_pfn + nr_pages);