From patchwork Wed Apr 2 20:18:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 14036433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44786C36018 for ; Wed, 2 Apr 2025 20:19:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C8539280005; Wed, 2 Apr 2025 16:18:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B97BC280003; Wed, 2 Apr 2025 16:18:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9724C280005; Wed, 2 Apr 2025 16:18:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 6E8F2280003 for ; Wed, 2 Apr 2025 16:18:59 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6E7F5C075B for ; Wed, 2 Apr 2025 20:18:59 +0000 (UTC) X-FDA: 83290217598.28.6DDA642 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf23.hostedemail.com (Postfix) with ESMTP id 9935E140010 for ; Wed, 2 Apr 2025 20:18:57 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=gtFyQjWR; spf=none (imf23.hostedemail.com: domain of BATV+e93bdb537aa03e43b1bc+7892+infradead.org+dwmw2@desiato.srs.infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=BATV+e93bdb537aa03e43b1bc+7892+infradead.org+dwmw2@desiato.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743625137; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FBnAFdxWx0BAktV4ZqJguBsjZizAzyvP0/mFoBjOOso=; b=ZSamBmN+ol79TosgAJ2l3XKVMe7pQcvjSBAcUnKJOFx6dTks4dW4k6sZosJKTYXaDMw+KF SODOBO1rqqsAupTdk4DtOwmZqC//EODG6y7yR9QhaiwZBukYkWL4sWy3uWt3OroCpksnMm vIlQmDQ1NZfCUeE/llSmQPofGmUP6Ow= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=gtFyQjWR; spf=none (imf23.hostedemail.com: domain of BATV+e93bdb537aa03e43b1bc+7892+infradead.org+dwmw2@desiato.srs.infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=BATV+e93bdb537aa03e43b1bc+7892+infradead.org+dwmw2@desiato.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743625137; a=rsa-sha256; cv=none; b=WVvfXW5S4Y8cqYnIGaUEeV6KM5VfegB0Ya5aGM2a6le8MWPXRN1ioSWWrovSRg+uFolGv3 cbd6Jkv/YGq44rM6+cdKX166oANl91Tm6lK4jakyOU1lCTtV5MIL8CC85Ie7RRTllYECZN rgo8yZCIosIDBAc8JjKTim2ht4wVPU0= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc: To:From:Reply-To:Content-ID:Content-Description; bh=FBnAFdxWx0BAktV4ZqJguBsjZizAzyvP0/mFoBjOOso=; b=gtFyQjWRzw0RQoEIZ/eyYEm3uN GjONx8SwV6r+jxzwlr2NKI/vfHrTv3UTFeYMbcN7UJszuGE+oPUFapQDyf/WBNSv5Jgo6OUkXoMD8 5JkbvROq4TNwqh8IcUpqW/IkdSTQP+VYW44V3dz6q+F3gK0jtySUlCsoFS/n6C+DagNTn8PM0BPx+ 4oTiaiJP9QiVRwRe5h6Cy9JAqJoFK5XqWkK+Bx5RAvRjzrt/A/gklDZe9OTZ+WY/GjcBlz5iVMeO8 cLptprHxCVTOjPMFKnCHOPGFPWzsjXwHVSbk4URxlJKdxfR6luVVw3hM8TFvPEqLdjKawD/D9BVTT ZxOHhw5g==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98.1 #2 (Red Hat Linux)) id 1u04Xr-0000000759m-0Rof; Wed, 02 Apr 2025 20:18:43 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98.1 #2 (Red Hat Linux)) id 1u04Xp-0000000DcH1-2TfR; Wed, 02 Apr 2025 21:18:41 +0100 From: David Woodhouse To: Mike Rapoport Cc: Andrew Morton , "Sauerwein, David" , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Marc Zyngier , Mark Rutland , Mike Rapoport , Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 1/3] mm: Introduce for_each_valid_pfn() and use it from reserve_bootmem_region() Date: Wed, 2 Apr 2025 21:18:39 +0100 Message-ID: <20250402201841.3245371-1-dwmw2@infradead.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 9935E140010 X-Stat-Signature: miroo5jpqwdue34xfipgihwcwcjp9xqc X-HE-Tag: 1743625137-383515 X-HE-Meta: U2FsdGVkX1/pKOvgFCuVzmUsDYpHa5QJ6GaVQuLSsOia2t42LvXAPTrQRe/A+11vZ/c1gEgrBhg3a2nE9m5wQYyoupWsyyQJJ45x41BUifFQRGm3fMRz9AIyDTHnC5Y4GU+ArP/04LmuZYSooEXLafh3/Zdx5JTyz4MWbq49cQac1x0P2K+CTTodL0O7N9xzdTFREqGCuy89szXkpqg1U6B2/OLnPVj5bixkMo9ezyP37nCmKdJ7Jak3Tbw7eF6Zp+Ga6kshy1H9vhFbRkdZa4hk6D/b/Sfvu0tYzX2DVZYZMutFk0wChYPn5mLxiZV6kj84zoje9wHd3bNXsGGpx6D6Nms92MWSCm0NTwL0FW1St3mTxXPm3UBFT8bxWEK3yX9D4zwe1LFTIqX2gM0u8mmaO7b1f8c7aSUcTWLBYY9HMrX266p+e5WDIuomJ4/y4WsFulOkAU7Nta2AWqPawXLCXVq7GimsX0lDjvN3p3jhZj4dZZLwJKsHyz5bJWRGpAtbPBgmOvyiBaquSXv8d4Z1dUt9lgpxD60mE+59qmoewt6bmURIV1cKoDlNSuPG+WA8soqGac0Vx4Kzi0nTck8M+zC4zk/HTIinTUdQZrvYe5UnHYQhJF0LW1Z5/ilMAzqFjvS1Os4ZiMRp+kZxtt0EoUgYhef8EsjiMpZnQ94OScVBGWcBVOYXDrJ+056II3dtug3BuqS7LKJx/+Yk0zIw4dVMf1hAG5NyM5xMmwz+BVhV5ZbEqIDHMjVJftA8Tn6QW3mu7LbF2bH8cOwjy3eL/UAh9xyPNX4eHOLa9j7VwPC2bNFJiBrjOPvpVzqUSAjyuaSQ6XbtyvYM+pRYImvfoILID7lQZKUQ3eCY5zyTvVzP3cxx2MmafOloNcHIGwFoWwe006NZShN9Vf61JgQqaa1C4IOPcVaga6Xxl0CGRKEtnvHBeJwK4drWG4BRcRXkneT6GvX4PM/+zI1 3dJ4GGUM FPq9ZQR9PQQSWLxSNMlBhJBSeF1pLoBQUU6crtVdBRKpTmOhj3Djg40mAEWp0mdvEP5fJ2uSH0a2QkBocYpSeU9QBUwBbTjd8ruH5L1hfyXoj2oevO7V8UptocjDZZW86UJLzZNgCFSp1MMVH/mGo+8rT7+TnBELfjCU8l9jdKC1zOwIwiTINGc5IFqA2ciLeDg6RikokDATjvOhKS0Zw6UzgWW5v5Bs/49YpTe2mC8seQty5sGgGe2qRVtmk8KltDTdEePyJtK71GHx8KB1Pfag/YyM5D0ijJGhduz3KAmI+WEho7KuwHE0Xkou8sgHDqOzdz/0K28KaM2EgDWaqykiOT5FkUYKetsThZKa+8VdWo++4Je9Ituon7SSdMrrCJAQqgZm1JbbecrXsbmdc1w5nTL2alnOztW0/gsYheyoUIY4pE95pL0nK/zo6LqNsaUik94+qM5O4nkkZDmiTnbDb92n2/Zlu6KO0InYXLWaCWVUEgCMiM1RMz2Ic/vlNT+45JxrFkclcHcCeRTA+cL8CvmQZ4vbZsMpANpXm9uFGATcoMvFqc5gsc4liJMLgBX9LATPw1LehJfs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: David Woodhouse Especially since commit 9092d4f7a1f8 ("memblock: update initialization of reserved pages"), the reserve_bootmem_region() function can spend a significant amount of time iterating over every 4KiB PFN in a range, calling pfn_valid() on each one, and ultimately doing absolutely nothing. On a platform used for virtualization, with large NOMAP regions that eventually get used for guest RAM, this leads to a significant increase in steal time experienced during kexec for a live update. Introduce for_each_valid_pfn() and use it from reserve_bootmem_region(). This implementation is precisely the same naïve loop that the function used to have, but subsequent commits will provide optimised versions for FLATMEM and SPARSEMEM, and this version will remain for those architectures which provide their own pfn_valid() implementation, until/unless they also provide a matching for_each_valid_pfn(). Signed-off-by: David Woodhouse Reviewed-by: Mike Rapoport (Microsoft) --- include/linux/mmzone.h | 10 ++++++++++ mm/mm_init.c | 23 ++++++++++------------- 2 files changed, 20 insertions(+), 13 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 25e80b2ca7f4..32ecb5cadbaf 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -2176,6 +2176,16 @@ void sparse_init(void); #define subsection_map_init(_pfn, _nr_pages) do {} while (0) #endif /* CONFIG_SPARSEMEM */ +/* + * Fallback case for when the architecture provides its own pfn_valid() but + * not a corresponding for_each_valid_pfn(). + */ +#ifndef for_each_valid_pfn +#define for_each_valid_pfn(_pfn, _start_pfn, _end_pfn) \ + for ((_pfn) = (_start_pfn); (_pfn) < (_end_pfn); (_pfn)++) \ + if (pfn_valid(_pfn)) +#endif + #endif /* !__GENERATING_BOUNDS.H */ #endif /* !__ASSEMBLY__ */ #endif /* _LINUX_MMZONE_H */ diff --git a/mm/mm_init.c b/mm/mm_init.c index a38a1909b407..7c699bad42ad 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -777,22 +777,19 @@ static inline void init_deferred_page(unsigned long pfn, int nid) void __meminit reserve_bootmem_region(phys_addr_t start, phys_addr_t end, int nid) { - unsigned long start_pfn = PFN_DOWN(start); - unsigned long end_pfn = PFN_UP(end); + unsigned long pfn; - for (; start_pfn < end_pfn; start_pfn++) { - if (pfn_valid(start_pfn)) { - struct page *page = pfn_to_page(start_pfn); + for_each_valid_pfn (pfn, PFN_DOWN(start), PFN_UP(end)) { + struct page *page = pfn_to_page(pfn); - init_deferred_page(start_pfn, nid); + init_deferred_page(pfn, nid); - /* - * no need for atomic set_bit because the struct - * page is not visible yet so nobody should - * access it yet. - */ - __SetPageReserved(page); - } + /* + * no need for atomic set_bit because the struct + * page is not visible yet so nobody should + * access it yet. + */ + __SetPageReserved(page); } } From patchwork Wed Apr 2 20:18:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 14036431 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5108CC36018 for ; Wed, 2 Apr 2025 20:18:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 39F1A280004; Wed, 2 Apr 2025 16:18:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 328DF280003; Wed, 2 Apr 2025 16:18:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A172280004; Wed, 2 Apr 2025 16:18:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E8A75280001 for ; Wed, 2 Apr 2025 16:18:56 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id F3B2F1607EF for ; Wed, 2 Apr 2025 20:18:56 +0000 (UTC) X-FDA: 83290217472.25.5DC1214 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf27.hostedemail.com (Postfix) with ESMTP id 04EA840007 for ; Wed, 2 Apr 2025 20:18:54 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=Nfrz3ROG; spf=none (imf27.hostedemail.com: domain of BATV+e93bdb537aa03e43b1bc+7892+infradead.org+dwmw2@desiato.srs.infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=BATV+e93bdb537aa03e43b1bc+7892+infradead.org+dwmw2@desiato.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743625135; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QLbPJRlyqVSx0cIJ3mGyFLKANvzzRuV5I+YYDBn/Qb0=; b=oNCaM6SDnzwuHenzMuVtccRGVYE98nNfu9+5izKliSarXMaHEXQkduHDDCYdkQa7yVLUew s4N1a010g52nDPXqTevWumDLRq9qUyGkaukj4zFE2PlRUFih8mKZ9G+0rbDM8RnIYcjYPx JzVuMISdmM1LfPhRnCqZJl0J2EWC3E0= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=Nfrz3ROG; spf=none (imf27.hostedemail.com: domain of BATV+e93bdb537aa03e43b1bc+7892+infradead.org+dwmw2@desiato.srs.infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=BATV+e93bdb537aa03e43b1bc+7892+infradead.org+dwmw2@desiato.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743625135; a=rsa-sha256; cv=none; b=lyQkI9mbT83mqqB1rLNUA96bxD37Y7c4FEAylr1gVjkyDdOLZvLAf5Zot2h3DvKhBTogsT FaGUqFl/yJ9YedQufDB0Ll04/Et6cJoMZVAraw5Zv2li3yqCrO68N/0Vxwf6BpwGQSMIje x5TN38xnItlBYVIAePwA/PlOoqfXxh4= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=QLbPJRlyqVSx0cIJ3mGyFLKANvzzRuV5I+YYDBn/Qb0=; b=Nfrz3ROGXN0O80DToFCDxfu6XG yoyUyBkZqiwBQIjSx0CUpkS7l7FsHD3njuvDumbfUaGa2zSt7S3oisW9uQx3hTFhvpXaN/ObTk1Ow gQYkewg0TLVGgufwQ78yCM45XNjHzaEJAtcCOXjNSF0I0vubrHOQdr9/G43iAC1nHsMByQhY9dP3w XKK0ICk6AId5UHnCNKEAMKtzPshdkmLVAE4G9El40B82ioTzorcncoc3p8Uq/idZY3MdIIQxeeCtd HbfA5xi6ZzxM42IQRE/iQIBKVE7I+/KmNBg30BWFZRouqfIlIM7+NAQU01c3e+5geZbpIasnFrSaN nfD3oIvw==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98.1 #2 (Red Hat Linux)) id 1u04Xr-0000000759n-0W23; Wed, 02 Apr 2025 20:18:43 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98.1 #2 (Red Hat Linux)) id 1u04Xp-0000000DcH8-2kho; Wed, 02 Apr 2025 21:18:41 +0100 From: David Woodhouse To: Mike Rapoport Cc: Andrew Morton , "Sauerwein, David" , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Marc Zyngier , Mark Rutland , Mike Rapoport , Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 2/3] mm: Implement for_each_valid_pfn() for CONFIG_FLATMEM Date: Wed, 2 Apr 2025 21:18:40 +0100 Message-ID: <20250402201841.3245371-2-dwmw2@infradead.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250402201841.3245371-1-dwmw2@infradead.org> References: <20250402201841.3245371-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: 04EA840007 X-Stat-Signature: 1n7695z99wbb1ioqzdk68qmxz86y7zqy X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1743625134-446762 X-HE-Meta: U2FsdGVkX18446Dsj5q54UoWZar/eFZ0DMLm/5E8gMKq1rb5T9Kxn6laiVkrgf5kkPLw7mX6CuXSgHr1QKb8BDJ7iG2D0BzoU+U7UiD788ghDKdNqf4WTvNCTbZvIVZOSou+RDP27LoDy2z3Pd/GP2l+oLAwDKm1zkQgoUYPfb/Qy6+l6RBfsCx87TIJMQBSDLf1crEr4NR0o0njRK44H08ZHNVdLDLU6RrpK/vdTwAxj4YzhPJ/xYKIZyU91pANkU2ktiDKzvTmhHyXOODrHSWm4mJW3/4dT9uNCS2DsW5jpspA/ULKVFlm0OdHwpG6r5dt4ZhxiwH2NtWumuHi5f47m4JLcFpiQfCuQviVhIgxHuqjGMzBnf5fslYAARqiPPeL7VsmSM/ceZ/gBld3LQaysEbTCmKy4E/U0QAq1hf4pvwSSMEfh1VermQY08KTBSOLqjKS0Xa/sM1ZkFE13EyYsyLyuPdWq2MGvrdRx5/BqtxZfZehm/gwmbTGmJzWCsazmLCrU9ONS3tiRZrMsGJQmvAD8VRS8Gj8Ge1PZkcYFP+pAsYRzrJz7jNMRqq42s+py5qEve39S76iXCMShUKe8cWTJk13Bn4ZMNSz+LCiJDhqjOSF1mcIhseNFTvBKGSVZAAGoQziPCaiqQTrONEj5Y3k0P0nxeRe9ijRA74zSwFtx0ioJmaMQtcMOzxyPjWe24RAi4BPbs4dmG7kLiVJhdTQ/KQ/cDob0aXHk3b8B2xMDAKT+vsLuDkuSYvoemZYNeU1UyPV/Y9Drxzb1Jc0UBPi2EcPcfY4LN/+7jDmGTJNjBnF14OLrqyTy8nlxgOL135ZFTnUdyYBOSiOrHRtKsI4BpuBsieOPeDUf9MpbvSRltqGxu5Lv1/rCROr0rUHs6y/wDQjtep+bfqduklH9Xy+up+drZI40qBxiLX2o889Qc6gAKDQ3VNgROwFlVd7mzaWIdYEg/WlKwQ fY2pkDaB Qow8ueQZ9c53tVru42KZ1IR9XPzX+F/9at5OQQHM4IaM/ramJg+laZ9rEB1vkvWZN75F3fJOdS1nIKACohxQ4iCezLsb+dixsd0k3sd7Cwc9tl4I2xRoz2rF0QupfD3/bDxbWkOJBXnMdWeu35hmkjI5FtGOCBcU4+ey2/PaNwN8aME9qd4DF8D7MbcWrJ9ywNHqIB4ts4w/1KxdExIl2vNBiC9i4lVm2MDn8YpEQ4an2rvYKGibXA3GlX+Ps1Y9D1hvuPIq1jph2RaexD5PT0FvfqILlGjBaLqSFsiHSmvwueB0lrTqh1NCv5h/ACyp/AtI49WhgWOwNXLMknBRjr9/LZysdJJdYNWotoK4AtnbJv26JZ5m2muDRVkj4yzv8SW9io3L1kdjnGDebSdL7cEPifoY4YHiRl/pcpmG6xcwMN8SLGppjPnNg6/YWbzH8AhOWGJ63uDmEZVOLk1hrDf6ZNrXd6UKiLa+mhInHLyw57gcKK5PpgIWEZ9jM8NvFS26IhcF1EiQr9JBiWP+06zcjwTejCkL1ppUa X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: David Woodhouse In the FLATMEM case, the default pfn_valid() just checks that the PFN is within the range [ ARCH_PFN_OFFSET .. ARCH_PFN_OFFSET + max_mapnr ). The for_each_valid_pfn() function can therefore be a simple for() loop using those as min/max respectively. Signed-off-by: David Woodhouse Reviewed-by: Mike Rapoport (Microsoft) --- include/asm-generic/memory_model.h | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/include/asm-generic/memory_model.h b/include/asm-generic/memory_model.h index a3b5029aebbd..4fe7dd3bc09c 100644 --- a/include/asm-generic/memory_model.h +++ b/include/asm-generic/memory_model.h @@ -30,7 +30,31 @@ static inline int pfn_valid(unsigned long pfn) return pfn >= pfn_offset && (pfn - pfn_offset) < max_mapnr; } #define pfn_valid pfn_valid -#endif + +static inline bool first_valid_pfn(unsigned long *pfn) +{ + /* avoid include hell */ + extern unsigned long max_mapnr; + unsigned long pfn_offset = ARCH_PFN_OFFSET; + + if (*pfn < pfn_offset) { + *pfn = pfn_offset; + return true; + } + + if ((*pfn - pfn_offset) < max_mapnr) + return true; + + return false; +} + +#ifndef for_each_valid_pfn +#define for_each_valid_pfn(pfn, start_pfn, end_pfn) \ + for (pfn = max_t(unsigned long start_pfn, ARCH_PFN_OFFSET); \ + pfn < min_t(unsigned long, end_pfn, ARCH_PFN_OFFSET + max_mapnr); \ + pfn++) +#endif /* for_each_valid_pfn */ +#endif /* valid_pfn */ #elif defined(CONFIG_SPARSEMEM_VMEMMAP) From patchwork Wed Apr 2 20:18:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 14036432 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E19EC3601E for ; Wed, 2 Apr 2025 20:18:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 62025280001; Wed, 2 Apr 2025 16:18:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4AF4D280005; Wed, 2 Apr 2025 16:18:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2892A280001; Wed, 2 Apr 2025 16:18:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E9F8C280003 for ; Wed, 2 Apr 2025 16:18:56 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id ED54B807EC for ; Wed, 2 Apr 2025 20:18:56 +0000 (UTC) X-FDA: 83290217472.27.04C14D4 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf15.hostedemail.com (Postfix) with ESMTP id AFDBBA0016 for ; Wed, 2 Apr 2025 20:18:54 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=axLO7djj; dmarc=none; spf=none (imf15.hostedemail.com: domain of BATV+e93bdb537aa03e43b1bc+7892+infradead.org+dwmw2@desiato.srs.infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=BATV+e93bdb537aa03e43b1bc+7892+infradead.org+dwmw2@desiato.srs.infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743625135; a=rsa-sha256; cv=none; b=BEm+qcovDv/KtGkxcZDc8kbVOyNKJNm9azAZNHNtasEk9KTn4ILliojV6oIUiAay047Wjm EcYcH2JkqNcwugiks1ByqnaF5J1Xla6uarTJ9mPkxNg+UGfBDr3kaP0hZ5vVmwkIdlU+Z5 IWC2ZMIa0vMQR1znqxwOtHloVDpna0M= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=axLO7djj; dmarc=none; spf=none (imf15.hostedemail.com: domain of BATV+e93bdb537aa03e43b1bc+7892+infradead.org+dwmw2@desiato.srs.infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=BATV+e93bdb537aa03e43b1bc+7892+infradead.org+dwmw2@desiato.srs.infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743625135; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ARg7i8/7R/o0DITALlPq6tBgDHCD4rhJgI/lZjiTKt0=; b=s7jUeHTvgvaeo9NnqS+t9aDcz7KZ4mHHZtv2knM70a+zZ3eIFoaitrf3lVmRkWC3maArY3 cIXXWVStgQCF2+nvo3qe+/eKWXYEVUc0exq+79NO7X59wHSA5nDSjbr/HgSlA7TjVou5SE O9TkC3Df8r9Dx+19aHeA47EcGBqUeX0= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=ARg7i8/7R/o0DITALlPq6tBgDHCD4rhJgI/lZjiTKt0=; b=axLO7djjobtsjWQ1aIARrbP65F sOZyGGv5vn7VsWGTopSz4jE35s4eP5zwPrrImxq/C644MBHgOGC05qmUvm76DgKwDekfCVsjxlfcX 860G3h6nkqsvQRlUQlRevVJvKDMZCAtT4kMLrJirp3zuXSiJrQvgHT/dq5SKvFhhdO/vcZtfzdHko FdvcCldiguFQGQbGGTHDO1X2cWrG1vNlhXLCPw/Ce2YrNAMkN44SHkJsv0VfmFyOaOCOrgSZbniVl XzXZAEIUy8WRW9zhgDRymfDslmSgybEJWSkV67MyJDaKb1Kp8mq572qB/MgU6TydUFI6w5x6xew4L 7Hyj9v5w==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98.1 #2 (Red Hat Linux)) id 1u04Xr-0000000759o-0W4X; Wed, 02 Apr 2025 20:18:43 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98.1 #2 (Red Hat Linux)) id 1u04Xp-0000000DcHF-31hz; Wed, 02 Apr 2025 21:18:41 +0100 From: David Woodhouse To: Mike Rapoport Cc: Andrew Morton , "Sauerwein, David" , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Marc Zyngier , Mark Rutland , Mike Rapoport , Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 3/3] mm: Implement for_each_valid_pfn() for CONFIG_SPARSEMEM Date: Wed, 2 Apr 2025 21:18:41 +0100 Message-ID: <20250402201841.3245371-3-dwmw2@infradead.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250402201841.3245371-1-dwmw2@infradead.org> References: <20250402201841.3245371-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: AFDBBA0016 X-Stat-Signature: 44tyd56k1ayfwh4e73g3xmw3kngq1iy3 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1743625134-265974 X-HE-Meta: U2FsdGVkX1/t6n0/oflJUGo9vNVHP/3xyjHSxgs2VNEZli2mENPMhPgVD8iRMXRi3Bm0pJHySzwcWlPtksi9ft3u+hZGQwhDMG0ltsLenreMXQ3730C+/6fd4QkLNhAyD6Qi+Og9HM6ytE0NqFXg5jkCxBAEBCDUt0KJBP3tiKO10xvN6oJ71Zv2SNj5X95iTS4zoB7lwa9uTwF+yDB7DdEFYZ25bVP5kHA32UqopAuo7MUBiJjqpm8BEZ0Bn7ku4bPdinIGiCBlKkGTg+xrk4Bd9w8IXQGPX3Azua5lJkw5o8LrUwfO9B8YaEiEAUQBbUDVjWSMuXgsAetEZfv8mFfH5anqlpRE9aNPT3POzjXQk7McoT1s6YEkfqx18lidg82pd2gZs987jsYNEqimlir/FCiShNNOByoM8LNFeP50RVh9S4JPcHKuDPdJ+mK22DhrAxuPTjX7AfP9B8dmg26IL8TIlJG9y75Vwz2YcD6CFC8Bktx4EQF24lbNclt2faPORPXZbvoCvJBifccBnOErZhpsxdpcIiSI3CuXTV2Z+Z4RgV0rnqrZniL/ihJPz10fPnYCx38tI2g2lRb3Tfs93RFnaY9wr1eBucMIMsO1NbNiArAj2cOUMcDK/aF5ONWrq2PCnQkMltURsGg94Y6LkYVP32HCgXpiMPInCXBPHwSZcuzPlWf+y9sL2MltoU7taOW5P1ekL7Bn3TE0RELxwMnY96ujHJD8fG3JqxA0VihuWTqNBWNYSWbsP5Yn6LlPi2y3W3QUCjWA0e7G0brrqbxfeKK0Bp0ir6tEZ5O7vsrGtmKj5npNto5aVcn0cmtiK2M+GJjGLWMsnCk04S7MXynT0ZXSp/sdw1MrtY0IuCvgLQaIZk4Cubs82dELP1U8zoYV5yEaJq6VR2bLE9d4eZsQYL9Qrlj4NM6ShH4uVbZzImY2CVItbbgk9LdNrOZYMcKvPkPuzEmMx9z neH9lA3g cExbSbrivJQp6JOMAmRQ5/dPQvQqnYqxxiyoo8YaRbvrdIo7kbvcXCrPjbbJ96G1N/L7f/RkELdYkEz2kFlcbYfmX2+uQ77OpWRv4E2BoX7m3B/nz8xP5MiBLX+uNpTGke+SRZcwOqoIn9DFWCPJ+dr46DtJp/ncfHc58FHlwccdXneqSRu8h8ZPMJGOdEk156Un6KolW7sGzcqb3YwZCmOKDyKt2w590UWTnYuoHxH58LR2WF4yRMDLcRKyAX6LUhjY+i2uLxiSob/VHIs0MsnBl+xI9Nc77Wtaw6jkn9jKAC6myTB9xnRrrkW9d8dW8jAsaTRhOp/u9Xf240aLzoeLQlz6+7U6sT1BMjpM0WFI0e7u7roAamsPbUbSGCcSxPkHHq6URuLCBUGyhoP09daRql8+FoJLquXZkpFwj+gPbUGP9AgIjzrtPUycAW0nX90DMwFuT2ov316Mw+mYh9xHZXOIm0qBtq3UgkOi86YYiIoAKLx7T4xR4CmRkqXdE9+5cRwalxCv6XFlqSjFSV+gCIZODFwIl0le2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: David Woodhouse Introduce a pfn_first_valid() helper which takes a pointer to the PFN and updates it to point to the first valid PFN starting from that point, and returns true if a valid PFN was found. This largely mirrors pfn_valid(), calling into a pfn_section_first_valid() helper which is trivial for the !CONFIG_SPARSEMEM_VMEMMAP case, and in the VMEMMAP case will skip to the next subsection as needed. Signed-off-by: David Woodhouse Reviewed-by: Mike Rapoport (Microsoft) --- include/linux/mmzone.h | 65 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 32ecb5cadbaf..a389d1857b85 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -2074,11 +2074,37 @@ static inline int pfn_section_valid(struct mem_section *ms, unsigned long pfn) return usage ? test_bit(idx, usage->subsection_map) : 0; } + +static inline bool pfn_section_first_valid(struct mem_section *ms, unsigned long *pfn) +{ + struct mem_section_usage *usage = READ_ONCE(ms->usage); + int idx = subsection_map_index(*pfn); + unsigned long bit; + + if (!usage) + return false; + + if (test_bit(idx, usage->subsection_map)) + return true; + + /* Find the next subsection that exists */ + bit = find_next_bit(usage->subsection_map, SUBSECTIONS_PER_SECTION, idx); + if (bit == SUBSECTIONS_PER_SECTION) + return false; + + *pfn = (*pfn & PAGE_SECTION_MASK) + (bit * PAGES_PER_SUBSECTION); + return true; +} #else static inline int pfn_section_valid(struct mem_section *ms, unsigned long pfn) { return 1; } + +static inline bool pfn_section_first_valid(struct mem_section *ms, unsigned long *pfn) +{ + return true; +} #endif void sparse_init_early_section(int nid, struct page *map, unsigned long pnum, @@ -2127,6 +2153,45 @@ static inline int pfn_valid(unsigned long pfn) return ret; } + +static inline bool first_valid_pfn(unsigned long *p_pfn) +{ + unsigned long pfn = *p_pfn; + unsigned long nr = pfn_to_section_nr(pfn); + struct mem_section *ms; + bool ret = false; + + ms = __pfn_to_section(pfn); + + rcu_read_lock_sched(); + + while (!ret && nr <= __highest_present_section_nr) { + if (valid_section(ms) && + (early_section(ms) || pfn_section_first_valid(ms, &pfn))) { + ret = true; + break; + } + + nr++; + if (nr > __highest_present_section_nr) + break; + + pfn = section_nr_to_pfn(nr); + ms = __pfn_to_section(pfn); + } + + rcu_read_unlock_sched(); + + *p_pfn = pfn; + + return ret; +} + +#define for_each_valid_pfn(_pfn, _start_pfn, _end_pfn) \ + for ((_pfn) = (_start_pfn); \ + first_valid_pfn(&(_pfn)) && (_pfn) < (_end_pfn); \ + (_pfn)++) + #endif static inline int pfn_in_present_section(unsigned long pfn)