From patchwork Thu Jul 4 01:59:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Yang X-Patchwork-Id: 13723111 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 474CCC2BD09 for ; Thu, 4 Jul 2024 02:00:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF88F6B009C; Wed, 3 Jul 2024 22:00:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AA8346B009E; Wed, 3 Jul 2024 22:00:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 948E16B009F; Wed, 3 Jul 2024 22:00:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 76EDD6B009C for ; Wed, 3 Jul 2024 22:00:13 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A0250409D3 for ; Thu, 4 Jul 2024 02:00:12 +0000 (UTC) X-FDA: 82300415064.06.21D283C Received: from mail-ej1-f44.google.com (mail-ej1-f44.google.com [209.85.218.44]) by imf20.hostedemail.com (Postfix) with ESMTP id 00AC31C0013 for ; Thu, 4 Jul 2024 02:00:10 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EZbOvE5H; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf20.hostedemail.com: domain of richard.weiyang@gmail.com designates 209.85.218.44 as permitted sender) smtp.mailfrom=richard.weiyang@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720058383; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references:dkim-signature; bh=iBvSWsPMBovfNNKu+FcMHNPGVEksUrxj+PzgtwOaXV8=; b=e/XTPCvUUEHbyHBimkISs7T6/7iajdz9GHItYe4tbNlwwuZP5rr36BxRapPr8lRQMYoZeZ DULQta7uHP8eMFXWqM3yk24oeUgGJyv3wNY7mbXaQ4S2k1BNQXJRSVrpSoBwqxChnhTous 45MbKM59l9ePsw/cErKv0oFvlxfkRoc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720058383; a=rsa-sha256; cv=none; b=mQAVAq53eQka5IeeyqJ7MEkbxNwo/5UE56TKW7Bc0/gjl9ZnMaaPTcfi75p56wnz9RkuBb KXxct88AnxM1XqswtB6afe6mg4LRR89lCQ55tlfe8/XnWqlNDa9Tnc2cexQL5wkZm3cABA +SeiHzwlODC6czbCLlkjmJ2NrZK9l6k= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EZbOvE5H; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf20.hostedemail.com: domain of richard.weiyang@gmail.com designates 209.85.218.44 as permitted sender) smtp.mailfrom=richard.weiyang@gmail.com Received: by mail-ej1-f44.google.com with SMTP id a640c23a62f3a-a724a8097deso12138566b.1 for ; Wed, 03 Jul 2024 19:00:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720058409; x=1720663209; darn=kvack.org; h=message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iBvSWsPMBovfNNKu+FcMHNPGVEksUrxj+PzgtwOaXV8=; b=EZbOvE5HDsG1D6EkkOp1UMmeg87/BEG2/xR69dIJD1oQdevQmwUQfmi8ue3vmC8s7A oDn8QeQsx48o00kJKwE1qA4WurgJWoEMLyjz6vJvTwj3mq+1YJJKv4r4NLWPA/F2mK1w jPWvkPkQbPWex70TM5aPwaqVpvHbtaQOWHn2ic0pUvWXBtsCN5fvC0nBTy9ozZaZxhI2 JPztxHBj2b/OUTRbYqGegJqH1Qxh3o+U+9vd/9J5O8iUeF6MWjq1ZyagFdAJ2junbdxV U+vjawgwV0QmLbSnl01S2/iWz6e3EAYYAEiEz0cq+ZpRshKdTnTTRmZ3V3XkbP2zp6ER Q/PA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720058409; x=1720663209; h=message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iBvSWsPMBovfNNKu+FcMHNPGVEksUrxj+PzgtwOaXV8=; b=iuy8EcYGU3QH/ruu/UBCzxpRv3be6qWO/l2d3LO9C+Aou6VVBr2ucvUTSWVAVFOZt4 zFg68ckSJ9bKqAoi125bV6ghWDIcqYnIPiOlmh7jM1Rpjj1iSlNv5THJ7JkGQme9H3Or Nfpl/NNcOMe9rHRdvMbk1oPe2KAf80B76kRiABlZkuDESRGrbql4G0oHIZAsyAokFKht ryxu3VCGj+l7F8X6vmbZd31Vc8cTxkA1H2BwdpaaryAH9dHBypyKIHh4KucO5Do+4OKs 9Wg0zTeVpPfo8vMNJuXUrpaod2n2VdAU35q6kHDr/JN6i3B+//v2OIChh769+qsqgZpd BeTA== X-Gm-Message-State: AOJu0YwI848Z314vnDHSg9Kl9okS2N9nxI8CIUmseIZmAH4uZhCIHBPa ElVzZWWa7iZDs2d29T8zx/nLzoW9SNelGy+mUPlFI31+2WUK7xA4 X-Google-Smtp-Source: AGHT+IH91ltmDaXbfVKUDba7uqQ3g040OT1KwD2daD44uyFAo6WPdfOWlFyWr2NrfQ1DGzm64OU4ZA== X-Received: by 2002:a17:906:f287:b0:a6e:f594:a292 with SMTP id a640c23a62f3a-a77ba7210b0mr10975266b.63.1720058409099; Wed, 03 Jul 2024 19:00:09 -0700 (PDT) Received: from localhost ([185.92.221.13]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a72aaf1bd30sm562825366b.12.2024.07.03.19.00.08 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 03 Jul 2024 19:00:08 -0700 (PDT) From: Wei Yang To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, willy@infradead.org, Wei Yang , David Hildenbrand Subject: [PATCH v2] mm/page_alloc: remove prefetchw() on freeing page to buddy system Date: Thu, 4 Jul 2024 01:59:06 +0000 Message-Id: <20240704015906.18437-1-richard.weiyang@gmail.com> X-Mailer: git-send-email 2.11.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 00AC31C0013 X-Stat-Signature: 55nxdtd145fkumb5rj8urj8efemm6j19 X-Rspam-User: X-HE-Tag: 1720058410-986222 X-HE-Meta: U2FsdGVkX1+7ojdFar3l6OOf3RQXS4ZH+T2ctkww714+8aNGPGNd3bXi7CyAYp0+mwNg9tZwmPFVS1IN8CEK3rMtLaHiyYzzK9JMS4M4XBOjBZXM2/gc1SjRvXzCZYofUmYPQhKL8KVFI4uuClFKK5ftSTAJDm/s55Zv1GJCPkLACTF0TYicVO5FROhOdkpK5sZyF/FX1NTAd7p7edFGbLsFsRlyLlG4gBWsIGk/WbL4IMNZX53+TYdls+1jX3gnv3bcraaVpa6wOPX4XUoyQrxGQ19R+gsj5IholI84rCBTNxrNYG4hot5fL0I545QZ5aFSqNxt2xZ0qqcCA/CSQxzqWEph5bbEKZthMvmYNR2Gd4xEqgocCRJTT8Q9nOGn+V5QNB4fWZUHQI9p55zQTaV2XPQPg2BQaproF+j1nhZX07fJc5byv+8M2Nkh3L/8xyVy7CZaKw3bONrm4Y2xBl5bQ04zD3Dp9fD9nU9LakcshNdJ0dQ9mGlEvt99oxDXiGXr5qxjM4Gh7AZYtLFW8xiYSU2OvcyEyvPDHfVigeDQp/tTFC4vqvvno081PkYeJ+C+cUL2tOwycKRQ3V3FHyCBd+Py1LGV57Di9okhf0+0kW1VfybZ9804EOjmvLFAMTvEzGgmdJyZtgO0Dr+LxadNTLx2gktIB7BB8zte5oWyuek6MWb3lSQsxH/ZHwXl1iPqfQEzJ71UacaGGoYmlrmRme9jQ36p2pZ6Rnwg3M7hv2DUmeMFIpBwha3dVFHVHvtFJM1J0TzVU/gdrsbVbJt5+k/z8kxISMAWUIWVdc51uWvkAIw1lMJT4/f1Z4zQXWsEjY4ex2ejdM9hWYFG2onPF8Yv07EieFpREuXaSHWFxCB49o5YtWOoZZVaXa3hU5PPer031yl2bVMiI7c5iyZhlWtjkgb5dzWX9pyTwzMm971tqmWfLwq4h4ptNBBl+9Lznzqr8JKyYv5SJ+Q B2CDDxim aHI1/uvP1PkO9uSHyPqR+BRtNt1yT87W+6zHEacbCbjhsGlrBSmlBpCSBkD95KtIvskOH8F6dxrE28IWExAsm1FUrqAJoCX2XFKOT8+Btr9GkYAChx3nPNQ6I3YQPB1HpMcBJZLLaMaVb7TC5C+v4VXPIlLjLkXoK9uGq9IGjF6RyiGy0yhUbAAt+NpasxGYWItUGoxYNWNokf1vCtb7wjKjgNJ3It6y0EtRb4zPSuo7NxTvHwqRpDWY+v71YkSDySHvI0DqmimAvVEGeEgp9Wf2Xrh2ZQ6W5waaKSk0WixiOGRupBN8sLw/DIjcuLes9A55ayJygs6IyM3IlcP2EKeneTcrIhhAj9u3fAJu7vbwF/Zvydl0kXxlHQfE5bE7mRtJ3bvynoA28lgoliDvCw8Yac9Uu4eymtdxzhwHQ8Q2T+XzH/6QwENozzCt+flBhJm80wtZLS6R9fj0QJaNMQyhZFlE4WtRZhRWPC48zJiQEiagNMw1EzuuRPGuKjdRdbQfRWC/sIDjCn4lkjycBFN5mC+Bqy/KBviBR0Cey88/CFQ4cVtCFR5fieXXhi+tWeawJHy2BEUPCaI5ab+4qfHjKtPcPUqQBKbP2tcLL1zf0kJFZeuz5xEu/QgKEwZTiup1fZOP9oZVu+8TC1PNvAvg/bEXRSz/CPbX8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The prefetchw() is introduced from an ancient patch[1]. The change log says: The basic idea is to free higher order pages instead of going through every single one. Also, some unnecessary atomic operations are done away with and replaced with non-atomic equivalents, and prefetching is done where it helps the most. For a more in-depth discusion of this patch, please see the linux-ia64 archives (topic is "free bootmem feedback patch"). So there are several changes improve the bootmem freeing, in which the most basic idea is freeing higher order pages. And as Matthew says, "Itanium CPUs of this era had no prefetchers." I did 10 round bootup tests before and after this change, the data doesn't prove prefetchw() help speeding up bootmem freeing. The sum of the 10 round bootmem freeing time after prefetchw() removal even 5.2% faster than before. [1]: https://lore.kernel.org/linux-ia64/40F46962.4090604@sgi.com/ Signed-off-by: Wei Yang Suggested-by: Matthew Wilcox CC: David Hildenbrand --- v2: slightly adjust the loop based on David's comment The patch is based on mm-stable with David's change. commit 3dadec1babf9eee0c67c967df931d6f0cb124a04 mm: pass meminit_context to __free_pages_core() --- mm/page_alloc.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 116ee33fd1ce..5235015eba3d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1224,7 +1224,7 @@ void __meminit __free_pages_core(struct page *page, unsigned int order, { unsigned int nr_pages = 1 << order; struct page *p = page; - unsigned int loop; + unsigned int loop = 0; /* * When initializing the memmap, __init_single_page() sets the refcount @@ -1236,16 +1236,14 @@ void __meminit __free_pages_core(struct page *page, unsigned int order, */ if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG) && unlikely(context == MEMINIT_HOTPLUG)) { - prefetchw(p); - for (loop = 0; loop < (nr_pages - 1); loop++, p++) { - prefetchw(p + 1); + for (;;) { VM_WARN_ON_ONCE(PageReserved(p)); __ClearPageOffline(p); set_page_count(p, 0); + if (++loop >= nr_pages) + break; + p++; } - VM_WARN_ON_ONCE(PageReserved(p)); - __ClearPageOffline(p); - set_page_count(p, 0); /* * Freeing the page with debug_pagealloc enabled will try to @@ -1255,14 +1253,13 @@ void __meminit __free_pages_core(struct page *page, unsigned int order, debug_pagealloc_map_pages(page, nr_pages); adjust_managed_page_count(page, nr_pages); } else { - prefetchw(p); - for (loop = 0; loop < (nr_pages - 1); loop++, p++) { - prefetchw(p + 1); + for (;;) { __ClearPageReserved(p); set_page_count(p, 0); + if (++loop >= nr_pages) + break; + p++; } - __ClearPageReserved(p); - set_page_count(p, 0); /* memblock adjusts totalram_pages() manually. */ atomic_long_add(nr_pages, &page_zone(page)->managed_pages);