From patchwork Fri Apr 4 15:59:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 14038653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E386FC36010 for ; Fri, 4 Apr 2025 16:00:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5EC20280008; Fri, 4 Apr 2025 12:00:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 53DDA280005; Fri, 4 Apr 2025 12:00:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3420B280007; Fri, 4 Apr 2025 12:00:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 11362280001 for ; Fri, 4 Apr 2025 12:00:14 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 9E76F1C7C7F for ; Fri, 4 Apr 2025 16:00:14 +0000 (UTC) X-FDA: 83296823148.05.5E10F16 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf10.hostedemail.com (Postfix) with ESMTP id DFF1BC0013 for ; Fri, 4 Apr 2025 16:00:12 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b="GUYU/YR4"; spf=none (imf10.hostedemail.com: domain of BATV+341917c36a2da2958cd5+7894+infradead.org+dwmw2@desiato.srs.infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=BATV+341917c36a2da2958cd5+7894+infradead.org+dwmw2@desiato.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743782413; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=P5VYLYdEkeWgyeWX/EZa0WnpJ73Q8FAesiLU/ww7Cxs=; b=EMUNUCxUiXfYIpds3d9wBEWF7JpbrcsnhVrPC69rZRtv2LQDQFXI8foQFVsdHrHNBQOGFD uhk9JKA5LvnVE0+kJ3webNkiMFg+oWAoGOCafd25pp1BGrTyCees/OLpRGJWpczQf+GGC6 oa36nJBlTK/ZS1oBslNDhZBWiYDSp+I= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b="GUYU/YR4"; spf=none (imf10.hostedemail.com: domain of BATV+341917c36a2da2958cd5+7894+infradead.org+dwmw2@desiato.srs.infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=BATV+341917c36a2da2958cd5+7894+infradead.org+dwmw2@desiato.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743782413; a=rsa-sha256; cv=none; b=FByPbjzgPwegrdzwOqZkbc64ZRDxsHsbls3uTxrqNiUGEgvX0oRGmTh3w9j2zRDFao53xc 4uwzwu+pg0Mo4wZvSN7GQJ6nduycuMIY9ti1e317pdxHxS/QQVSUC6Hk/UxkdWsMsOi7ZM u4WRVJjOI2DUx/BkBCx/VpT2XpT2Dp4= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:Message-ID:Date:Subject:Cc:To:From:Reply-To: Content-ID:Content-Description:In-Reply-To:References; bh=P5VYLYdEkeWgyeWX/EZa0WnpJ73Q8FAesiLU/ww7Cxs=; b=GUYU/YR47lcF88YQE7EH7TpKB2 zN7AKKvvGE+bbZSCVfqr/oyxCsvOKug18HG2HLPn2hzCCyr28tFqyGD9WE8XnLgGAI3tZkFoNGjRO OgjbHNJH1PDByFVu8pm8ZivdCL5MEV/GNaFUfd+wAfiibiI9zWlpJtBcaljmCVRXvO2F5RxGRDFcv n+kIPUAtdP9Y82yL00kM9ObdiypoF5byTbPDzb62xDr08XwYD5r8jUm8sLxrmkORJzVZ3UuKodDTs Em+8VmA6k36CwWivAtW5UVU9ugF9e40TqXDlsMyj8sHumr6FwIMVgjq+YOd/BBlhaaZ+R0N+DDoUj 9yqvaGVQ==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0jSc-00000007QOy-0YLc; Fri, 04 Apr 2025 16:00:02 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0jSZ-0000000ERSB-2cgL; Fri, 04 Apr 2025 16:59:59 +0100 From: David Woodhouse To: Mike Rapoport Cc: Andrew Morton , "Sauerwein, David" , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Marc Zyngier , Mark Rutland , Mike Rapoport , Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v2 1/7] mm: Introduce for_each_valid_pfn() and use it from reserve_bootmem_region() Date: Fri, 4 Apr 2025 16:59:53 +0100 Message-ID: <20250404155959.3442111-1-dwmw2@infradead.org> X-Mailer: git-send-email 2.49.0 MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: DFF1BC0013 X-Stat-Signature: nck9h5ykstiz3e3ktuku1wb468ykg77q X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1743782412-329754 X-HE-Meta: U2FsdGVkX184qhR4n/WdrctrpO9r7mjm9Ay7UcgSHBmoQuYMSHt/421eo3nKu9YzcRErqM7OdlUkJ5y76ltSVpObm1fLY0CHv0DaR9XmvyaAUGy5y1ZfElPFSuT/TCs+soPYSs0HglG/Pa9vLyPVzlKYnWLqu4nVGNDTG8Nwmf1PElbRvUcjAdTbZaoPDZV+Z1CEPpBCVoamdZgjpMsLbD4C/qv+Bi3LRheCjGIGaEXEgv8e7plT65Qd51qKxgfIdfX762NFmnJz8oVwNX4F+01R8/ZDRjoTXcCwAddCg0sgMKd4NveMT5crCrhOex/PKN4fjG1k5NSJ488XXhCay2HvpAJ7mtC35O4jAEi9j19Cvf9ahCrsmestWkcHiJ8DFBgK+wklQYTFSC/ljJYw3NR82BLhFdJJ7Rt/POQ+W666idX25ZrHpkiIwZBy8JLdojMWGI+H6E98uBrTWb6+vJ2K3bUu5ptxCX173z3c/vXSl/ZrS+f4ZAsfQj4Dj3Nhra02pe2EFrYfW4gIyhFU6wbx+E+y69nI7JDVj1H4iKJPNl/VuTXp4jIsOdbBqEhCG0F/Xn3CQ+aXvFGxMSYcYTgEHBFtzKbbm0kxjJHp9Leshb0rbLnEFaTtnbEwM7JPYCAYuNUP632cNxvJiN16LieOmGrM2BJ6TomURdymME+JBT96myGa9pLw2XF0NbKBlm4DReMQgeJvUanCNaDyJzlAyPmX43O1m/uPykQI+LojLR8V8/PAnrhQ4rlz7vnWQxsw0lEf+J069tZwzcYSs0ea8wpla318ke85r0zZx7vAp7Oj6cynSkFQZtp+RpELSqtPQ/bP/7fcKV7qN+dMZ6iKuMIkWPthodHCOkWbtjB5VxeKTQB9x36ilU08JPqQ4jXKJrD8aY4zg5h8Wy/bJu0+J+Vl/4weOmtzT3NnbM24zwqEMQs7sTdggQ6mrZFyW9pyPwEZpXUDeNCWjKE iGq2n+KF cjkNnaMrcfrcspRjNDc7K2GQcK6qimzRZqmzA9YEU3170pma6zyLUFrox/WPbCZtKpwn2zvFCsLT0AckxQG94ts6xYHDFl5+8X4VoCejZJEU3bwYXUY+w832l9CvYgIW63LKFVilPt+vQqGEF9DlR1sNMqHt0Xazy2fVM2GfY1Ivy7sfCiCuw9a4Z/Lz2l411F14TsyTnJAkfTOsixrJz/FW8dsbzpQtDIoN13o63DOUlqN/xTx0DI4wIxxryuGDYBvlQapfiDTyq97lFo+wPTgp8Fhe04rW/2QfudVX3oZAG3ryg/3FvpiES7d2kVkyoL3cF54rka9ydUvwVXSnAh/uR3cl3zUGNjGUxa9c9VdegM2J2ySi//bybdKrCIz+AoYgIcjvCcsFvnwTX0Z/KPwf6WM9X7WjkFhLurGentMYeyNtOYFjirbc51hGyD/cLcBVg0ZnGwu3FsIISUWc1omaRO2xWx9lVgcQFmmeFitKrUZduh1vGbejkGS44L3LIw15RMNTqZaHszWhANpDpEoNUuuvqjhRIBS82ikgc5tgboJWdgVgNXQjntFmnXZUKCezXuqTUsTb+AXbettBWPwvbKonZb2IDiOdH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: David Woodhouse Especially since commit 9092d4f7a1f8 ("memblock: update initialization of reserved pages"), the reserve_bootmem_region() function can spend a significant amount of time iterating over every 4KiB PFN in a range, calling pfn_valid() on each one, and ultimately doing absolutely nothing. On a platform used for virtualization, with large NOMAP regions that eventually get used for guest RAM, this leads to a significant increase in steal time experienced during kexec for a live update. Introduce for_each_valid_pfn() and use it from reserve_bootmem_region(). This implementation is precisely the same naïve loop that the function used to have, but subsequent commits will provide optimised versions for FLATMEM and SPARSEMEM, and this version will remain for those architectures which provide their own pfn_valid() implementation, until/unless they also provide a matching for_each_valid_pfn(). Signed-off-by: David Woodhouse Reviewed-by: Mike Rapoport (Microsoft) --- include/linux/mmzone.h | 10 ++++++++++ mm/mm_init.c | 23 ++++++++++------------- 2 files changed, 20 insertions(+), 13 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 25e80b2ca7f4..32ecb5cadbaf 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -2176,6 +2176,16 @@ void sparse_init(void); #define subsection_map_init(_pfn, _nr_pages) do {} while (0) #endif /* CONFIG_SPARSEMEM */ +/* + * Fallback case for when the architecture provides its own pfn_valid() but + * not a corresponding for_each_valid_pfn(). + */ +#ifndef for_each_valid_pfn +#define for_each_valid_pfn(_pfn, _start_pfn, _end_pfn) \ + for ((_pfn) = (_start_pfn); (_pfn) < (_end_pfn); (_pfn)++) \ + if (pfn_valid(_pfn)) +#endif + #endif /* !__GENERATING_BOUNDS.H */ #endif /* !__ASSEMBLY__ */ #endif /* _LINUX_MMZONE_H */ diff --git a/mm/mm_init.c b/mm/mm_init.c index a38a1909b407..7c699bad42ad 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -777,22 +777,19 @@ static inline void init_deferred_page(unsigned long pfn, int nid) void __meminit reserve_bootmem_region(phys_addr_t start, phys_addr_t end, int nid) { - unsigned long start_pfn = PFN_DOWN(start); - unsigned long end_pfn = PFN_UP(end); + unsigned long pfn; - for (; start_pfn < end_pfn; start_pfn++) { - if (pfn_valid(start_pfn)) { - struct page *page = pfn_to_page(start_pfn); + for_each_valid_pfn (pfn, PFN_DOWN(start), PFN_UP(end)) { + struct page *page = pfn_to_page(pfn); - init_deferred_page(start_pfn, nid); + init_deferred_page(pfn, nid); - /* - * no need for atomic set_bit because the struct - * page is not visible yet so nobody should - * access it yet. - */ - __SetPageReserved(page); - } + /* + * no need for atomic set_bit because the struct + * page is not visible yet so nobody should + * access it yet. + */ + __SetPageReserved(page); } }