From patchwork Thu Aug 5 19:02:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 12421879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D455C432BE for ; Thu, 5 Aug 2021 19:04:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3C9C560EEA for ; Thu, 5 Aug 2021 19:04:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3C9C560EEA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=sent.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 504706B0088; Thu, 5 Aug 2021 15:03:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 466686B0089; Thu, 5 Aug 2021 15:03:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 29CDA6B008A; Thu, 5 Aug 2021 15:03:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0101.hostedemail.com [216.40.44.101]) by kanga.kvack.org (Postfix) with ESMTP id 046AA6B0088 for ; Thu, 5 Aug 2021 15:03:55 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id ADF9D1801C656 for ; Thu, 5 Aug 2021 19:03:55 +0000 (UTC) X-FDA: 78441951630.27.1FBFFCF Received: from new3-smtp.messagingengine.com (new3-smtp.messagingengine.com [66.111.4.229]) by imf12.hostedemail.com (Postfix) with ESMTP id 68AB01000890 for ; Thu, 5 Aug 2021 19:03:55 +0000 (UTC) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailnew.nyi.internal (Postfix) with ESMTP id 17EEE5810BB; Thu, 5 Aug 2021 15:03:55 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Thu, 05 Aug 2021 15:03:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=from :to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; s=fm1; bh=orYSo+lwOeqVc rCXP6bMLdXhBxG79CHa8pLqvu93F90=; b=JLiPNY0alezfanhcEGypspe2ruEbc +S3RVW03KEkS07YejHfFIg6iCZiCQsu9h5DwzMb0jG/b0FrkS4c9Y/NdKeacdvTK 5IEMf1NOtxdyT73Rrqkne+wzqMExrCYGUhVRLM03HCgH5m3P+z644LCalPwbCIVm pRvg3/y4tHPT7QOKLXK7FYSnNes3rxiWEY2Fen+iycw4IPFF11hOpw2BCMjH5s6z h5CWBKWlUi+I5CIkqwWeoiOd5Brg/GLBCxFsUeeC1gqF3j1D29yY23T4ZAfbI2fn 9TUNpofZvkpjxrWCLsbmytqsxyFTB838/bJYc4+xyvgFCm88NUZWSPgqw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:reply-to:subject :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=orYSo+lwOeqVcrCXP6bMLdXhBxG79CHa8pLqvu93F90=; b=tvn74sNC FppmdcuF0YkjcIsFmOlGO1oUWpkEbiGuwrgia1EXDOKlIC/zLyaHOHfhbYtbhkWN 7g8yMwgdPehka5KZaWsaChTzPoZKaUEQj1pJxR/gAF+2F20xDi5lM8YPawRtracc QWoEsFmHsl199Ji6XBbG7EfaBsygOXf8YUMjjuymzwm1BOUVI1kFPP+vjzdxsBVL yqzrJZtkow5KsmlTgNLtGxS7UPkMG8UZhiFB+QFdtaZ6hceTPyMdsIRgdk94f6lc tuA3WgTCU4tme0WjeqbcAd9Nj178lT6Q7iAFqheIt/2/RpgDVTGcRrKUOq03SUNa Ib0cAzncyRmNKQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrieelgdduvdelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffojghfrhgggfestdhqredtredttdenucfhrhhomhepkghiucgj rghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepieejue dvueduuefhgefhheeiuedvtedvuefgieegveetueeiueehtdegudehfeelnecuvehluhhs thgvrhfuihiivgepfeenucfrrghrrghmpehmrghilhhfrhhomhepiihirdihrghnsehsvg hnthdrtghomh X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 5 Aug 2021 15:03:54 -0400 (EDT) From: Zi Yan To: David Hildenbrand , linux-mm@kvack.org Cc: Matthew Wilcox , Vlastimil Babka , "Kirill A . Shutemov" , Mike Kravetz , Michal Hocko , John Hubbard , linux-kernel@vger.kernel.org, Zi Yan , Kees Cook , Peter Zijlstra , Nicholas Piggin , Thomas Gleixner Subject: [RFC PATCH 12/15] mm: Make MAX_ORDER of buddy allocator configurable via Kconfig SET_MAX_ORDER. Date: Thu, 5 Aug 2021 15:02:50 -0400 Message-Id: <20210805190253.2795604-13-zi.yan@sent.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210805190253.2795604-1-zi.yan@sent.com> References: <20210805190253.2795604-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 68AB01000890 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=sent.com header.s=fm1 header.b=JLiPNY0a; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=tvn74sNC; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf12.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.229 as permitted sender) smtp.mailfrom=zi.yan@sent.com X-Stat-Signature: zs9ghxry1cg5cmcb9eeqhfens9zk63s4 X-HE-Tag: 1628190235-668324 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zi Yan With SPARSEMEM_VMEMMAP, all struct page are virtually contigous, thus kernel can manipulate arbitrarily large pages. By checking PFN validity during buddy page merging process, all free pages in buddy allocator's free area have their PFNs contiguous even if the system has several not physically contiguous memory sections. With these two conditions, it is OK to remove the restriction of MAX_ORDER - 1 + PAGE_SHIFT < SECTION_SIZE_BITS and change MAX_ORDER freely. Add SET_MAX_ORDER to allow MAX_ORDER adjustment when arch does not set its own MAX_ORDER via ARCH_FORCE_MAX_ORDER. Make it depend on SPARSEMEM_VMEMMAP, when MAX_ORDER is not limited by SECTION_SIZE_BITS. Signed-off-by: Zi Yan Cc: Kees Cook Cc: Peter Zijlstra Cc: Nicholas Piggin Cc: Thomas Gleixner Cc: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org --- arch/Kconfig | 4 ++++ include/linux/mmzone.h | 14 +++++++++++++- mm/Kconfig | 16 ++++++++++++++++ 3 files changed, 33 insertions(+), 1 deletion(-) diff --git a/arch/Kconfig b/arch/Kconfig index 01a3f8048cb7..40bd222adeb8 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -11,6 +11,10 @@ source "arch/$(SRCARCH)/Kconfig" menu "General architecture-dependent options" +config ARCH_FORCE_MAX_ORDER + int + default "0" + config CRASH_CORE bool diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 04f790ed81b7..322b995942e5 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -25,10 +25,14 @@ /* Free memory management - zoned buddy allocator. */ #ifndef CONFIG_ARCH_FORCE_MAX_ORDER +#ifdef CONFIG_SET_MAX_ORDER +#define MAX_ORDER CONFIG_SET_MAX_ORDER +#else #define MAX_ORDER 11 +#endif /* CONFIG_SET_MAX_ORDER */ #else #define MAX_ORDER CONFIG_ARCH_FORCE_MAX_ORDER -#endif +#endif /* CONFIG_ARCH_FORCE_MAX_ORDER */ #define MAX_ORDER_NR_PAGES (1 << (MAX_ORDER - 1)) /* @@ -1245,12 +1249,20 @@ static inline struct zoneref *first_zones_zonelist(struct zonelist *zonelist, #define SECTION_BLOCKFLAGS_BITS \ ((1UL << (PFN_SECTION_SHIFT - pageblock_order)) * NR_PAGEBLOCK_BITS) +/* + * The MAX_ORDER check is not necessary when CONFIG_SET_MAX_ORDER is set, since + * it depends on CONFIG_SPARSEMEM_VMEMMAP, where all struct page are virtually + * contiguous, thus > section size pages can be allocated and manipulated + * without worrying about non-contiguous struct page. + */ +#ifndef CONFIG_SET_MAX_ORDER /* NO_MAX_ORDER_CHECK when compiling x64 32bit VDSO for 64bit system */ #ifndef NO_MAX_ORDER_CHECK #if (MAX_ORDER - 1 + PAGE_SHIFT) > SECTION_SIZE_BITS #error Allocator MAX_ORDER exceeds SECTION_SIZE #endif #endif /* NO_MAX_ORDER_CHECK */ +#endif /* CONFIG_SET_MAX_ORDER*/ static inline unsigned long pfn_to_section_nr(unsigned long pfn) { diff --git a/mm/Kconfig b/mm/Kconfig index 1f9bd3371765..3a030b439501 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -89,6 +89,22 @@ config SPARSEMEM_VMEMMAP pfn_to_page and page_to_pfn operations. This is the most efficient option when sufficient kernel resources are available. +config SET_MAX_ORDER + int "Set maximum order of buddy allocator" + depends on SPARSEMEM_VMEMMAP && (ARCH_FORCE_MAX_ORDER = 0) + range 11 255 + default "11" + help + The kernel memory allocator divides physically contiguous memory + blocks into "zones", where each zone is a power of two number of + pages. This option selects the largest power of two that the kernel + keeps in the memory allocator. If you need to allocate very large + blocks of physically contiguous memory, then you may need to + increase this value. + + This config option is actually maximum order plus one. For example, + a value of 11 means that the largest free memory block is 2^10 pages. + config HAVE_MEMBLOCK_PHYS_MAP bool