From patchwork Wed Sep 13 10:53:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13382761 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A507CA553A for ; Wed, 13 Sep 2023 10:54:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C9F616B0179; Wed, 13 Sep 2023 06:54:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 94AF46B017D; Wed, 13 Sep 2023 06:54:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68C1E6B017A; Wed, 13 Sep 2023 06:54:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 383396B017B for ; Wed, 13 Sep 2023 06:54:11 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0229780D5F for ; Wed, 13 Sep 2023 10:54:10 +0000 (UTC) X-FDA: 81231264702.13.3EEC7ED Received: from mail-lj1-f170.google.com (mail-lj1-f170.google.com [209.85.208.170]) by imf26.hostedemail.com (Postfix) with ESMTP id AACE414000A for ; Wed, 13 Sep 2023 10:54:08 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=R4YLke9u; spf=pass (imf26.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.208.170 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694602449; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=av0vrlLKsg9xBQaLOcb8CPCy93VQFu46a3TbtxpfU4U=; b=tLmw1eQdaSOaV2lKauZL5i1Nzs5vjERs/4BYYe3cZYn3guq1ppF3U2hBnlYUNdqQB8/50I e+7AZII0pS5WU8koivrEYaT5vdoZGbkl2sMgr08O2sLpr8y5tPp/sRcJ4dB155qIBvksbR FEjBxDSp0vaN2h99Y1ms3sJvL78mmks= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694602449; a=rsa-sha256; cv=none; b=zNbNXBR4SLjH0oQvKEQc7N6G+XFA5Z9jPbBNbyAn9OR/5cq2tDGbmWmKNrpayPP/H/6zV3 S6cWHN1DhETlyvLSt/SbJQbfbmg1SwtLic6408hsSfqOGdwoIwp2vR6mqfUaB4DuRCERP3 r3zUCB7ajm+frLyGpGAXWxNQm3esHqY= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=R4YLke9u; spf=pass (imf26.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.208.170 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-lj1-f170.google.com with SMTP id 38308e7fff4ca-2bf66a32f25so87873241fa.2 for ; Wed, 13 Sep 2023 03:54:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694602446; x=1695207246; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=av0vrlLKsg9xBQaLOcb8CPCy93VQFu46a3TbtxpfU4U=; b=R4YLke9uKg7N248/G2Th2cRmaN+bzvwnbZa8yhjzAgotxkRghtV4qxOGYf38Ucv4x0 CBUlHwOMV3gENssbYFE1L6EkTtP+zOIY0SV8drrVslh5RbEk16+Ehk8zQ/wSeF2T48qg DRtp1Yk4rSpPB1eiiWZ7G4+zsj/v/QcExN1f/zhadQNpscZHBS8n1veiTYiqmxK2BkEI RIgh9SFOJwUHaeb0N4vCisNupE5sYrMVtIB1mUpaWCFUkUahUgcX3S85SIsZ04DeqiiO k3zSLD+OGuTzMbwZX95K9opbCoE2cytTjCdBxgUMPWjgZdHTq7xOPzy/+jLDeSlXY1cn NZVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694602446; x=1695207246; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=av0vrlLKsg9xBQaLOcb8CPCy93VQFu46a3TbtxpfU4U=; b=aUJZ2w2fTTaCYSBcu2gq5244SeKt8hjyOcqVfOrCJJxcQKjdMY2XnB86hIWHK+ZvJm sK5rX9k8QQww09XLXK0L9e9oPPaAIC34LhxPqkzeP81zfjJMA3HsHFKwcs3y9MNugMRJ UF5Z6nY8pl+hpINC8EN2yEPdNQVuBwYu8TIv1z2egqHVegHTUXytkKXlhJi1+LwZSfCm zHBqD7Km6t5zUvApAUCIAv9YMTxWwDsoEdugPvjvf7r1fUOJwUvjttMiWzM9PMi23PdN jswMatrF2uEEUcf8R9plzPPD0eYFN+WZQi0lPkpGFgVlh03+OZ6H6F4+QkDNvtMyOuDh 4nvA== X-Gm-Message-State: AOJu0YwpgCoxH75FNUTv4bYWqoOuXga8/mZIFdzRTEDyofZUhUOrqsiw WLvA8FlbLT05Ua8tri48jL6HBDmiAmULXd4q4wo= X-Google-Smtp-Source: AGHT+IHgwCAQBLvB0z1DFSfacZLlZe7iyFgzrmCJNMai57WocOmkFwr5tPYGi6nJ7z357utYkGkfeA== X-Received: by 2002:a05:651c:8f:b0:2bf:9576:afd4 with SMTP id 15-20020a05651c008f00b002bf9576afd4mr1875974ljq.16.1694602446317; Wed, 13 Sep 2023 03:54:06 -0700 (PDT) Received: from localhost.localdomain ([2a02:6b6a:b5c7:0:59d5:28b9:7f13:cd1b]) by smtp.gmail.com with ESMTPSA id p36-20020a05600c1da400b003ff013a4fd9sm1757055wms.7.2023.09.13.03.54.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Sep 2023 03:54:05 -0700 (PDT) From: Usama Arif To: linux-mm@kvack.org, muchun.song@linux.dev, mike.kravetz@oracle.com, rppt@kernel.org Cc: linux-kernel@vger.kernel.org, songmuchun@bytedance.com, fam.zheng@bytedance.com, liangma@liangbit.com, punit.agrawal@bytedance.com, Usama Arif Subject: [v5 1/4] mm: hugetlb_vmemmap: Use nid of the head page to reallocate it Date: Wed, 13 Sep 2023 11:53:58 +0100 Message-Id: <20230913105401.519709-2-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230913105401.519709-1-usama.arif@bytedance.com> References: <20230913105401.519709-1-usama.arif@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: AACE414000A X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 56m596x3fk67r47k7nbf3a7tm4wj3xxf X-HE-Tag: 1694602448-719733 X-HE-Meta: U2FsdGVkX19jXGlSPVRFZ0O0iV+Xy3aVotzmTJmHvhqsBkeSJnMqMyZUa5Gg1NBMRrfHSg4sYYt6ielYiXdZvT+6QJ01X3Te+dgA5sFjqoC3jY3UQ9SSvk5dviRvkvGIqhvngSzZriFpemP/4OLaOqV3YO/FcnPiCTcq2PdmL32/tihbNRbMsVAHCnSMvVRo36rHK2+vyv/s6xOhlex+HP9R9Jt55QjQlxuCBTl9c5Y7b3Y9E2/sdlnKZS7YY2FQlHFkOgwHZHlPfbuXEf1KRDtYupwn4s10bsno6T8BChLXZBXoVKYyJTtnLeLNGD4SXkNrFzwKIr4hSLopT26g3MJ3WfLJvLeI9Bqfrto19y5p6QwYGY5HWv+yXT3IR8XylKMXgyFlpkDsmojG18LQW4cxKy6aQapLmhRYHxnCweli+5COCJZFautUn1i/uzWvjkJAsE1hTuTqLnxHNzGTuCsgs7DD+2FUuZ0c6/1sGgwJYW2QWx2fW4IWKMdtnCL8wDrEfchAXKMYtlUS9rfmn2+MB71dDraimfb/HTHK38b81qkFJwfDv+muvmyrHFtkuIZVHRkGwUB3UwQfIZD4xGKQ6W/iiwbyrmVUND1OVeZsru3co0+3Jdp8cKjEDSI+kS/QkG3MDS2uy0JhOS8f8RoJ+PpwiUzjBSLJbGBTHk+e0bWgr3Ro1YEhNK75grLhJ15F7yEnkU+UYd8QapAHG86cyjhRB1b6BLNCg4LQFTi3UKuxT0QAmOuFYj90URCe6tLtHWkTjQTcYuHLU0U/uWv1fS/4g53bQNLVq8TS2bEE+4RdeDnS4zpfBM/Ob0/Yx5ECD6PzEQxyOqd61mfwXVpnqWiA9Cj7Tpo9006DW0L+bSDrqsHTTiFnD1vrs527jSv0Zsq+SR8agO0VnD+MY2L/Er4EVeqcSZV8b7AFxqyAyDPUBXnZWya65ooAXFOi6APlOZcjLPmWA8pU6vL 1CtqoxPR RemUTGZHcQh0py7Gnv0bqQVCBNaZi05DfQ27vkGLLpwn2SIiQjwi2S832D89iwe2TvsClhov5Bn947wWbKgHvuajhYni/kWQy8UuA0y0MzDaT8ugxhKn5NFrx61rqbfAqTrrMjHdyfSlIfEnYOCyerdnycU4AVpbQcqMq8TUgSrbdkwKwgTBHeTWyrQWvNL3MWMs0RI7uFDZVhqEtl0TlfZpd40uSo9Lkwx38CbvLKS5VeaLiXCwylQxfpMU2W716+DN4tZsSOe5mXMkUA6+DqjHyLG3b3xsi8qI2vKq3zD6+wUDbKMvAT3DmttW+ZdQS2sN6pvay+fOKBBsO5TxwLiqI0sZlWviqIjZQ+c8rtvuXcXK16ZT7PKoofal0I1hO68wy/YMl6e6d9pmkYFmhnCBHzNeOMjuewK4yceDJDweCJqF7yaGZ2dVa5pBvfMrHyabQu4nnLgZ2sILMpjIuVdh9gC0yzyaCJd2EcaGUpNcg39M= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000003, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If tail page prep and initialization is skipped, then the "start" page will not contain the correct nid. Use the nid from first vmemap page. Signed-off-by: Usama Arif Reviewed-by: Muchun Song Reviewed-by: Mike Kravetz --- mm/hugetlb_vmemmap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index aeb7dd889eee..3cdb38d87a95 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -319,7 +319,7 @@ static int vmemmap_remap_free(unsigned long start, unsigned long end, .reuse_addr = reuse, .vmemmap_pages = &vmemmap_pages, }; - int nid = page_to_nid((struct page *)start); + int nid = page_to_nid((struct page *)reuse); gfp_t gfp_mask = GFP_KERNEL | __GFP_THISNODE | __GFP_NORETRY | __GFP_NOWARN; From patchwork Wed Sep 13 10:53:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13382759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F735CA5534 for ; Wed, 13 Sep 2023 10:54:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 79F906B017E; Wed, 13 Sep 2023 06:54:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 727686B017B; Wed, 13 Sep 2023 06:54:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 504926B017D; Wed, 13 Sep 2023 06:54:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 377956B017A for ; Wed, 13 Sep 2023 06:54:11 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 04FD080D6B for ; Wed, 13 Sep 2023 10:54:10 +0000 (UTC) X-FDA: 81231264702.16.99D6DAA Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by imf27.hostedemail.com (Postfix) with ESMTP id 018CC40027 for ; Wed, 13 Sep 2023 10:54:08 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=MrcNpBlM; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf27.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694602449; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HvAW5HglNzFFKY02dQL2QCkQguw1CPVi9WQHfYQdWVQ=; b=nwTo1oxJbGnuNeKc/eD2p90dF9oveK2Th/+4CcjNt41kx5Ea+xGc0wko7bBkkHwpPNuwme hJLpEfZiO53Iq8MrR/1p81296ioFvuetCfM4SDvwSXy8XdBvGp7AVB/DHL5gUZl7AkjFMD YTqeFPUY6sLqcfQ8Oy9vzcOmSdp8sKQ= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=MrcNpBlM; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf27.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694602449; a=rsa-sha256; cv=none; b=rWt48N/1jaYpmATcVshZxch45G4Ivj9xidfYi7iIIY0stZrdmB6mraWBQXucuoBWenXENr iea9d5ulv+3s3n+P7qGHwrnZBR09/+glYYdIvB0egNizq9+9RSph1oN/FSRbDXgInlZgxA JEgtX7AniqDwRoC31/NlMD6jDWzUKTc= Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-401d67434daso72244785e9.2 for ; Wed, 13 Sep 2023 03:54:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694602447; x=1695207247; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HvAW5HglNzFFKY02dQL2QCkQguw1CPVi9WQHfYQdWVQ=; b=MrcNpBlMO5gidEkhFhLVddJgBsbT4vvyxQZG/Hn4TbPjqhd01tH1SYbr8fylhhSF7a 3yQtCsfucfXxNqhV6sqtwqmxF0y3AmEzqvYYCV7R9YF2f9GtFFci+P+h19Hxek9GNoMC 1gyC2xAEI1M6i9enwiCjVwocNJ1asN6MVI4FW7Y1PMVIMiDN7Tq1QIchVLM2UmK2n2Iz eToU3kF8jJK3zaVUK4Pz4t9U3yrKa4mgvVVG9u4Cu3Fpj+346Rk6JT+utCA5jZw4otgP 78+enfaD9m9cYFpVTsOWBd+DtWvn2mV+mTEomMW8iRoAg5XnJL2koM1t7eeQ1pwzxTUN V8WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694602447; x=1695207247; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HvAW5HglNzFFKY02dQL2QCkQguw1CPVi9WQHfYQdWVQ=; b=ABkm69+U6H97kIxof/cevqlxYVGMORvK43CIfAEtx0tFVG6YWPNZO/GDZe3mZJ8dzo 3jrdj3X6DyR/hVCgmkbW/ACodmp5x/leNUckZi+dNiXQpIqyuMsasgovcgwL4FaGaODC EkxZg7swJmUXYrkQqvT/6cmSStcY4TBrM0Hwd2KVWQpQumgImNDptOaLOVF7mBuqMmWt LJYbzpXQhxI7da0QhnPcpH3weHYLKYJQq+Hu1QsHB2RhEtgNS7JUItcjsfUlY81m5/Uq WkZHceXZ/sX3KuC6LwQwKKx4Crb93uncM8LU0hiaYT99bQtgcoM5Te3ripcyGrYKrsC7 U8gA== X-Gm-Message-State: AOJu0YyoFPUdHyCZI/iyu3jxvKXN6zJXvB3sGUZPulWv6UtHlha4WxRs seLCJ4lzbYaldd/C/74CMAT1XKLa5GWbwScQaiE= X-Google-Smtp-Source: AGHT+IHvbrYE5CU1LLfrzLv3Yh3xia4AXw13/BHJVfTjNw8q0gSbwaI8RZroEq0GxStSNj/tvBmGxA== X-Received: by 2002:a05:600c:2196:b0:402:bcac:5773 with SMTP id e22-20020a05600c219600b00402bcac5773mr1720136wme.38.1694602447327; Wed, 13 Sep 2023 03:54:07 -0700 (PDT) Received: from localhost.localdomain ([2a02:6b6a:b5c7:0:59d5:28b9:7f13:cd1b]) by smtp.gmail.com with ESMTPSA id p36-20020a05600c1da400b003ff013a4fd9sm1757055wms.7.2023.09.13.03.54.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Sep 2023 03:54:06 -0700 (PDT) From: Usama Arif To: linux-mm@kvack.org, muchun.song@linux.dev, mike.kravetz@oracle.com, rppt@kernel.org Cc: linux-kernel@vger.kernel.org, songmuchun@bytedance.com, fam.zheng@bytedance.com, liangma@liangbit.com, punit.agrawal@bytedance.com, Usama Arif Subject: [v5 2/4] memblock: pass memblock_type to memblock_setclr_flag Date: Wed, 13 Sep 2023 11:53:59 +0100 Message-Id: <20230913105401.519709-3-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230913105401.519709-1-usama.arif@bytedance.com> References: <20230913105401.519709-1-usama.arif@bytedance.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 1b1c1msjcr1koxmnesx743f9z6h4ssz6 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 018CC40027 X-HE-Tag: 1694602448-573798 X-HE-Meta: U2FsdGVkX190NQLrhzDF2acBOg/hEGHxPJ8dbQ0v0cLjgidEEuTiAAfr4jdIKLgy0SiZS4af5foOgBm/5vlIyHuvSiZvRjGjUYu5JdFJg4z5nEJT1S7wkBlrwrYSLwrZgWdhPJ5c2NWxnj1k+mxGDBc8bOV5eV0ITYrWd/M3iukAO/oFWTQlPdH9Sswv2JfViGEnYw1/qZi7e0FzY15hVZSg+dhi+ZolO2uIEl8pf8n9LxYVWjuVwmmowR6RjT6fTBTsE+shPe2qZXF/HYDf9vKDIyZkjxJ5i15iJO8igvQ74RNPUPPUa84jY+V3dYDSPyhzsFWI36n27rB0jjKEVcoUowNRul8vDHrr+f6GGN+ZAp7ftt7oMBL02mCgECzUIqdZwr99hnfiu8v5Oco3zP28Fngfqsi2GhFlFOML+aqLCTo0rGLVO8HLEB3K7babn0FyCHGPF3m2QQ5NyWIg7mldrbVU31uiPwoYa5Cz/DpTO3Fms97ZKCAt6JEUSeeuTwUz2Dt2XrGEaCuM1QXtmQmbBkvJFbh7Twn1kiG4ZVzMJSvXUlrfxRmD1Wz1w9wcO2BscPUhv9MVCAQFMS9zC4aYHGzPLUUoJq+WxXnpsIiKtPzrCWKKbW0SFiJM8thMYihRcbDxr1uRirAAIw9BnLlhdVsRKp0c/HtbUsRZEBAcrFlp3EGOBF3I5L8W8k61R7A3+AJowz5CA4nZ0d3jI0+dWxN0mKMRnG6q3cg5z6d97NUT80xlDZPGXV4AMpaoxdMwGDmH2JCblWNIR+vYQ9NYqkTEz6U2ba5URvyiEuGY2i/S4Q1VTjsX9pnDff4LBSmONjGUEKicgtdHW/Wqt9o17rOWfnWTun2nbjMxy4gkAAIrgHi9y0dn23rcJYfcWqShrFvWwkjfezMVkATAhrb1bfUr2ESs7yyUNJ1hQYf/qu6q6Q2hf5wSYXB6oliwBSTxNqWaako6GDoxlGw QRbsqLUv x7GXXlx5+6fRfflNM/f95pLAfdzX6wYBhnQ/5IWhB4jU590emiqexeRV9yWiLjF1fAkn6ZrQaiWuvuiPCHMVj4hCbH2VuWwRP3TvS/cJg7+yyPpzc2AWMSyxjTOW6h3pyjJU9llnkVHdHgmZPYNDJM02lXCFTUpmFZ8mijhRoa5gBUzBAOMwBxAizDT2pddP2GBQcYJtJ4qvRe67A0bFpfLwbP6hBQAn+Xi2/ieAWzhv3/6H3jDaKd6OBnv6l2hu0G7cMKYT9cXiolrsBcgpxzmfglGnQe7k/MQoMx0ENnORV008q/YFlr33kxeKdrSf9rDd9xLnRVHKZ2D4CIzehNLxUwlxr86vB4uGuzIq6M3TiM3opm4GAAHZKCVnNPA6EcU10br0M65UJyWT4x0Q9OdBlivNboatlAYSs1qN9YQcYf3kFvXOhiveSdrE1nnY1iKD8NCvlis1Ht3eBd59eo2dKvM2R2s4TPme5kD21dt3D3Zq6XVuZNQUpyzyT11/ebd5AExj4eRAUIZY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This allows setting flags to both memblock types and is in preparation for setting flags (for e.g. to not initialize struct pages) on reserved memory region. Signed-off-by: Usama Arif Reviewed-by: Muchun Song Reviewed-by: Mike Rapoport (IBM) Acked-by: Mike Kravetz --- mm/memblock.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/mm/memblock.c b/mm/memblock.c index 913b2520a9a0..a49efbaee7e0 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -901,10 +901,9 @@ int __init_memblock memblock_physmem_add(phys_addr_t base, phys_addr_t size) * * Return: 0 on success, -errno on failure. */ -static int __init_memblock memblock_setclr_flag(phys_addr_t base, - phys_addr_t size, int set, int flag) +static int __init_memblock memblock_setclr_flag(struct memblock_type *type, + phys_addr_t base, phys_addr_t size, int set, int flag) { - struct memblock_type *type = &memblock.memory; int i, ret, start_rgn, end_rgn; ret = memblock_isolate_range(type, base, size, &start_rgn, &end_rgn); @@ -933,7 +932,7 @@ static int __init_memblock memblock_setclr_flag(phys_addr_t base, */ int __init_memblock memblock_mark_hotplug(phys_addr_t base, phys_addr_t size) { - return memblock_setclr_flag(base, size, 1, MEMBLOCK_HOTPLUG); + return memblock_setclr_flag(&memblock.memory, base, size, 1, MEMBLOCK_HOTPLUG); } /** @@ -945,7 +944,7 @@ int __init_memblock memblock_mark_hotplug(phys_addr_t base, phys_addr_t size) */ int __init_memblock memblock_clear_hotplug(phys_addr_t base, phys_addr_t size) { - return memblock_setclr_flag(base, size, 0, MEMBLOCK_HOTPLUG); + return memblock_setclr_flag(&memblock.memory, base, size, 0, MEMBLOCK_HOTPLUG); } /** @@ -962,7 +961,7 @@ int __init_memblock memblock_mark_mirror(phys_addr_t base, phys_addr_t size) system_has_some_mirror = true; - return memblock_setclr_flag(base, size, 1, MEMBLOCK_MIRROR); + return memblock_setclr_flag(&memblock.memory, base, size, 1, MEMBLOCK_MIRROR); } /** @@ -982,7 +981,7 @@ int __init_memblock memblock_mark_mirror(phys_addr_t base, phys_addr_t size) */ int __init_memblock memblock_mark_nomap(phys_addr_t base, phys_addr_t size) { - return memblock_setclr_flag(base, size, 1, MEMBLOCK_NOMAP); + return memblock_setclr_flag(&memblock.memory, base, size, 1, MEMBLOCK_NOMAP); } /** @@ -994,7 +993,7 @@ int __init_memblock memblock_mark_nomap(phys_addr_t base, phys_addr_t size) */ int __init_memblock memblock_clear_nomap(phys_addr_t base, phys_addr_t size) { - return memblock_setclr_flag(base, size, 0, MEMBLOCK_NOMAP); + return memblock_setclr_flag(&memblock.memory, base, size, 0, MEMBLOCK_NOMAP); } static bool should_skip_region(struct memblock_type *type, From patchwork Wed Sep 13 10:54:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13382762 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4EC7DCA5534 for ; Wed, 13 Sep 2023 10:54:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CD0086B017B; Wed, 13 Sep 2023 06:54:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C59286B017C; Wed, 13 Sep 2023 06:54:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E7EA6B017D; Wed, 13 Sep 2023 06:54:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 7ECE66B017B for ; Wed, 13 Sep 2023 06:54:12 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 51C55C03B9 for ; Wed, 13 Sep 2023 10:54:12 +0000 (UTC) X-FDA: 81231264744.04.837292D Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) by imf02.hostedemail.com (Postfix) with ESMTP id 5812D8001C for ; Wed, 13 Sep 2023 10:54:10 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=EQg1IVhA; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf02.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.128.54 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694602450; a=rsa-sha256; cv=none; b=qs8W03RdX1g6nmupvR/mcMunSLUL2FBE7s3EtyeGMRhgt/bfVN6ZpqUOZ9bla6wPdhDsY1 ivEXRpN/kblNG9cgDmnLP+IUxnI+Gvp05WG2BulwH3TVI9XBZtNhp50wMggJcHrLWjmsAT hvqfbEpItgO79DjWq1YyDuE4SF8Sbrc= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=EQg1IVhA; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf02.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.128.54 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694602450; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Xz27K9YR9w0fQiJ9Te2x4N9NzQgI00KGt4T8XMBUFg0=; b=ttcJYYnlQqoPblK/A1Amk+S9q21KCNWlDhzacPzZjHkoqA665ldin+7BkQrZ6Egs6JFVyF JjBLxsYciW90XMqHk3vAHE9wc3h56rgzEBXzANzBcOAxJWJp4OMtSE/R9hGs0f09Oi6m3/ ewORppWHqOuW+FGJdayr7QwGIhRinL0= Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-402cc6b8bedso75445435e9.1 for ; Wed, 13 Sep 2023 03:54:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694602448; x=1695207248; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Xz27K9YR9w0fQiJ9Te2x4N9NzQgI00KGt4T8XMBUFg0=; b=EQg1IVhADuXDfFXrF+0VBDuNSoybY85KNBlcvUFDvx6yLd3Z9VrOZckFsRn84GSOR0 bqGXhvt5lLyac31QQDUFsXw+YKufsOo/vwRw/TiKlB0TmqD5+o6hmkLMvmXhqcJ6lpJq auWSc2fznqBIpQvrsTzcWRE6TjlnSBwzUxd2i+kfRsooeqMHSOsWKNXQpEPC+5oU3+Z1 GMeR6dLZqTZQZ7TwpjArq3boXCAz/ZH5v+uXwQeLvEmaazmCeFYv3XJDnmxK1h0U4tnC eysPFy1Sb4mur1CT5Ve1y9aXyigu2oC1hWye5kI+JEJ6K/2z/wLqJ2bz8iK/donca3HI fFzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694602448; x=1695207248; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Xz27K9YR9w0fQiJ9Te2x4N9NzQgI00KGt4T8XMBUFg0=; b=IlfM5v8BtWDUFvVlfVQB+6YRbCdcUS5von3acCBS03NFBoOM0oCMvA9JsIrpb6rlmF I2P4DpWLErm0utJ94eP/8Z8bwQTdiRDAD4axHvyS8PTc7JYorgR6fbbwMAJjlyFazWfj haXmBUf2BPLM5cjaHrrhClFNlkJBjm5XiCoJ4QgWyJvvqt67KbYbtMW37KffamgIxf23 +jo97tU38mu2rr6uIfD34rXrYSUXpUH1u+QisQiQFpaYLvMQQjRb4qRrNdejGjS6eeiV XhKY5oBA6uiQmd1GqOyBZZE1OXjBgN7aOADfPUY9F77KF02FZy5vieZ+78x19ohpDigV nYLw== X-Gm-Message-State: AOJu0YxTxV1g2f27SsU6RZ6nsO2BoSeo+uyI2Ow3dg1+9tWuNNxetS3J sEX9fpdBVlH3enNrH6OXgVnQa5L0+XHDhtWRiKo= X-Google-Smtp-Source: AGHT+IELq4IpOoGAAOVl1ZdlRYGeDT1zMvMdNhwEFXBsF/9A4wBjX2ebeHr30qj0D/lpVvMZsIc+Jw== X-Received: by 2002:a05:600c:ac5:b0:401:d803:624f with SMTP id c5-20020a05600c0ac500b00401d803624fmr1620823wmr.4.1694602448422; Wed, 13 Sep 2023 03:54:08 -0700 (PDT) Received: from localhost.localdomain ([2a02:6b6a:b5c7:0:59d5:28b9:7f13:cd1b]) by smtp.gmail.com with ESMTPSA id p36-20020a05600c1da400b003ff013a4fd9sm1757055wms.7.2023.09.13.03.54.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Sep 2023 03:54:07 -0700 (PDT) From: Usama Arif To: linux-mm@kvack.org, muchun.song@linux.dev, mike.kravetz@oracle.com, rppt@kernel.org Cc: linux-kernel@vger.kernel.org, songmuchun@bytedance.com, fam.zheng@bytedance.com, liangma@liangbit.com, punit.agrawal@bytedance.com, Usama Arif Subject: [v5 3/4] memblock: introduce MEMBLOCK_RSRV_NOINIT flag Date: Wed, 13 Sep 2023 11:54:00 +0100 Message-Id: <20230913105401.519709-4-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230913105401.519709-1-usama.arif@bytedance.com> References: <20230913105401.519709-1-usama.arif@bytedance.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 5812D8001C X-Stat-Signature: z434zxutyhwt16eae8d7z9izxfgb45r8 X-HE-Tag: 1694602450-971195 X-HE-Meta: U2FsdGVkX18Em2z09Mw0rAxkzUquGcy94lHV7cv1apxEiv/7KG22b8cICXXSWpehflzazBnqHm27X3Ez2+39BW7LsLdMUxK83HTVDIOvz31astrPtUawjvZnZsRmgqzhkOIBDOTclQhr/WzcNIBcVfY/PegjdIBY9DrWgrM89s9OeW+vTXPlIeDUwNtRCB3OvimgwOAsc8X+Ri0WkdJIKJQYq4YOFN7S2QNYdmxFifZL/qQrO2EtxfJ9tyWiTNHz+xTLy0Y/CxTb4RwyrM5Wf8SXKu0LYewpL97AXgbmZnyV0SKmiXzc7MWMzY912AnCUif3m3iCh+37TP98mpBBMXhGY/hzmfvfmScMCaq7C1eaD10L0eJ8aWX4poNGS8om9uryu+cQxRWtB3GnqvAgpZDeAbwXImd5lPCqi+5M5mMSVV/e5mXEQ/x197+PYgyqr3b0jvCkeAiYEQqyv6IjPjd8EDgdoXshpdpwt5S5KtK9Oc26Q70XDh88uX3BalZCstF23kHS2D0WzVANwZGNZe03sy6e0idJ84csHIj81g3be9+efsfKH+ROCRl5tRQFJ/rbgFstC/dZUHNuN83IkkWHEnXR2pj9EYJUp2f2DpNATGg8VOctjfEE2eSO3jTAt8CqXtrnpjxLpbptHNYw5hwFdOSwi3H+tsWzIAylG+XJXLKuYZZRPoyNh6cWuzHI2dLNhwtNoEsg6bWve1pnddV8hCiIjJaPHF14nPnoF+FTJh/v8Vxr2wfeWId40be66ln00IbTSyo+J4uWCpipqbCjIY87249HiG0O8hDXEvClQUCVqnWL7Sb3HBtL2L80Y9mYY5M0hMpLkOdJhjMXv6SEDe12tZi+5D2HC4ZjacvSdNQtgVf27DsKM15DfaRS93X6LyqzwsRwJOPsb7YaYAg6l5RplmYngRAECGvMuZEGOOBxMNkTq2FLQtcC7e+uUlt3Ln1UXMyfmgPIlcW eohk9+kM 0HOdV23mmkoCSovZeM04JEmCjD6ooEoB0XsetSpXm5zmzmada03IkeTTl9AMO4H8Rxm4vcjgMgm0Jl3OX5GQg+HWkpdpfLJcFefZIq6gZDRizBDCQ4b0E5lHYLvUdSLsedmgqRY9jajb32WA6/B9k8QGkGk9xQ07zUzB4tLZSCUSmcrAGjnjy/6zCTIQvI+UzEB597pwOCbOK2tUoR0vqbc/firFgB+2iSb8/v2FIgJjhE6ImG9UqR1ccAtBL1TOGoPz/h/LW4OzG7jPD5qd7DKaXLnqCo5xVzHxpF1Dy5MTkOTJeZ7qdkgI6o31xXz2IdZHIHEj8E5xD8JgiXv4fqPKPnqxBl8tSVIOapVx85wZwc9JUEIzOXTypor7uNBWiYXOja/uIAPb3CYNIRZbDc5F168rS1jf3NmOi2pQwcZf5hMCyplmrfbdMzMzYxmtvCzik3INlr805kNmVRtgDGBpNka9zilTQFe4I/9x74BntCRB2JAfdQKcL92jpg0i+z6+e0/zSdoAcRcFg4X04HLNEIw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For reserved memory regions marked with this flag, reserve_bootmem_region is not called during memmap_init_reserved_pages. This can be used to avoid struct page initialization for regions which won't need them, for e.g. hugepages with Hugepage Vmemmap Optimization enabled. Signed-off-by: Usama Arif Acked-by: Muchun Song Reviewed-by: Mike Rapoport (IBM) --- include/linux/memblock.h | 9 +++++++++ mm/memblock.c | 33 ++++++++++++++++++++++++++++----- 2 files changed, 37 insertions(+), 5 deletions(-) diff --git a/include/linux/memblock.h b/include/linux/memblock.h index 1c1072e3ca06..ae3bde302f70 100644 --- a/include/linux/memblock.h +++ b/include/linux/memblock.h @@ -40,6 +40,8 @@ extern unsigned long long max_possible_pfn; * via a driver, and never indicated in the firmware-provided memory map as * system RAM. This corresponds to IORESOURCE_SYSRAM_DRIVER_MANAGED in the * kernel resource tree. + * @MEMBLOCK_RSRV_NOINIT: memory region for which struct pages are + * not initialized (only for reserved regions). */ enum memblock_flags { MEMBLOCK_NONE = 0x0, /* No special request */ @@ -47,6 +49,7 @@ enum memblock_flags { MEMBLOCK_MIRROR = 0x2, /* mirrored region */ MEMBLOCK_NOMAP = 0x4, /* don't add to kernel direct mapping */ MEMBLOCK_DRIVER_MANAGED = 0x8, /* always detected via a driver */ + MEMBLOCK_RSRV_NOINIT = 0x10, /* don't initialize struct pages */ }; /** @@ -125,6 +128,7 @@ int memblock_clear_hotplug(phys_addr_t base, phys_addr_t size); int memblock_mark_mirror(phys_addr_t base, phys_addr_t size); int memblock_mark_nomap(phys_addr_t base, phys_addr_t size); int memblock_clear_nomap(phys_addr_t base, phys_addr_t size); +int memblock_reserved_mark_noinit(phys_addr_t base, phys_addr_t size); void memblock_free_all(void); void memblock_free(void *ptr, size_t size); @@ -259,6 +263,11 @@ static inline bool memblock_is_nomap(struct memblock_region *m) return m->flags & MEMBLOCK_NOMAP; } +static inline bool memblock_is_reserved_noinit(struct memblock_region *m) +{ + return m->flags & MEMBLOCK_RSRV_NOINIT; +} + static inline bool memblock_is_driver_managed(struct memblock_region *m) { return m->flags & MEMBLOCK_DRIVER_MANAGED; diff --git a/mm/memblock.c b/mm/memblock.c index a49efbaee7e0..8f7a0cb668d4 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -996,6 +996,24 @@ int __init_memblock memblock_clear_nomap(phys_addr_t base, phys_addr_t size) return memblock_setclr_flag(&memblock.memory, base, size, 0, MEMBLOCK_NOMAP); } +/** + * memblock_reserved_mark_noinit - Mark a reserved memory region with flag + * MEMBLOCK_RSRV_NOINIT which results in the struct pages not being initialized + * for this region. + * @base: the base phys addr of the region + * @size: the size of the region + * + * struct pages will not be initialized for reserved memory regions marked with + * %MEMBLOCK_RSRV_NOINIT. + * + * Return: 0 on success, -errno on failure. + */ +int __init_memblock memblock_reserved_mark_noinit(phys_addr_t base, phys_addr_t size) +{ + return memblock_setclr_flag(&memblock.reserved, base, size, 1, + MEMBLOCK_RSRV_NOINIT); +} + static bool should_skip_region(struct memblock_type *type, struct memblock_region *m, int nid, int flags) @@ -2112,13 +2130,18 @@ static void __init memmap_init_reserved_pages(void) memblock_set_node(start, end, &memblock.reserved, nid); } - /* initialize struct pages for the reserved regions */ + /* + * initialize struct pages for reserved regions that don't have + * the MEMBLOCK_RSRV_NOINIT flag set + */ for_each_reserved_mem_region(region) { - nid = memblock_get_region_node(region); - start = region->base; - end = start + region->size; + if (!memblock_is_reserved_noinit(region)) { + nid = memblock_get_region_node(region); + start = region->base; + end = start + region->size; - reserve_bootmem_region(start, end, nid); + reserve_bootmem_region(start, end, nid); + } } } From patchwork Wed Sep 13 10:54:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13382763 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDAC1CA5538 for ; Wed, 13 Sep 2023 10:54:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 814D06B017C; Wed, 13 Sep 2023 06:54:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 776A16B017D; Wed, 13 Sep 2023 06:54:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 52C6E6B017F; Wed, 13 Sep 2023 06:54:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 34C056B017C for ; Wed, 13 Sep 2023 06:54:13 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 0B8BEA0E3C for ; Wed, 13 Sep 2023 10:54:13 +0000 (UTC) X-FDA: 81231264786.28.A068EE5 Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) by imf24.hostedemail.com (Postfix) with ESMTP id 2533B18001D for ; Wed, 13 Sep 2023 10:54:10 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=cjw9Yxv0; spf=pass (imf24.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.128.41 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694602451; a=rsa-sha256; cv=none; b=7wJJR7nGU93H/6yw7u3JBaUpTHsfkxYT+DIxavsU54i7f8R3iHWr49HhvvnjUSemwTLhLG zzEKiIkF5oPRitolN6P1egFxZsxSZ08G5zEG7gVSnM4kqXBGD8xDG8AiNBNFBJE4A0GK0K /IYmfiyUG8GZDGw5Jlo/scHH/zFAGsQ= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=cjw9Yxv0; spf=pass (imf24.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.128.41 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694602451; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=a6h7k6NjGFuqJLpb7jnCYfQxn4IX1VZuVPP7jz0KAXo=; b=Yy495b6EreF1IzGqcPJEEYE5TXdAFbE7SPCZbACs8vF8nkZGhjUMz+uWwoqVrswItCoXb7 YDnfZXAWJdNZfzYyjFDp9ndX9qIrxCYUtaPiKYYC7I2xMPLRFdLZcBumx5Msr8av0ce3lh tJFkf6PLLqh2so7hsgxljuRqgBMPs5A= Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-401b5516104so70506015e9.2 for ; Wed, 13 Sep 2023 03:54:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694602449; x=1695207249; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=a6h7k6NjGFuqJLpb7jnCYfQxn4IX1VZuVPP7jz0KAXo=; b=cjw9Yxv08xe7VFVjB9XsS1u9P7noFlUIj97VaEM8gJAHNmfbuljtt3sPctcuGRZwhF vgY/ThXhJ9av0GfHCUwCM5b/5bgfxmGIpu+kT9biyLZtal7wDaqv2kcVG2BKI5Rqr20a 0aXgmi3QexDpH7Qcw4PTjyxn48Oh06PomFnOLuYyHwlmp+E4XdilDX9pKD8X6g9jKwpl F+Tep3HboC8IhtSgvedE5n9JzpRwvJSJKQWO0ypNsLLM2fJsA51g+y3U4OlpHTOIMdBy JFK6ulgLsNbZ2HDXn5epgHCtkvfJcnn5ITGX+15v6pRGvU5A1Ic87xn31/n6HaUdBD7t OEnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694602449; x=1695207249; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a6h7k6NjGFuqJLpb7jnCYfQxn4IX1VZuVPP7jz0KAXo=; b=ijbF+TogZkreULOsY9jK2KrGsQ2/eYQOpeUtQsASI2T6vXdsde/ZQMSVAEOSJAjn7J +IJ4JUirTNKrD2dWZFdYkMh7g2Np0tLUU0tgsQMBEWshplIORDC7Z4vnhxe7BF3dF6Ju l8vUhVQDqB26J3Mg5lrvOQ+6yfjbs/+rtpKIngR7g/olii+gVWLgq7bEiSW6OBheivzH 4XhmHSPBDW4ssNPatW5zUU6gTR6cayjB3xbDSHCA+qStwaDjdSPaHe28aLlVq9qFmbdS BKQsqNyekoAMhuQLgsKWyV/0y3kcMTcfy8E+t4ttmT3ZRD/2uiD4LkGzUMUiEvoSm54D SKdA== X-Gm-Message-State: AOJu0YypdptzrGhgbyr2bZUxuiX8F00nihkMUiFBS+stJUEJ7QKa1vui WODQsD82mEx4zropyNm0PhEDHI0uUB2uQjfHItk= X-Google-Smtp-Source: AGHT+IGeRnxvvuKSKnq0o8Epoa2KLRxqMqvvhw8b/xo192TKibj8bohHyp3e0nydPYtGTqXT2wrAMg== X-Received: by 2002:a1c:4c0d:0:b0:3fd:3006:410b with SMTP id z13-20020a1c4c0d000000b003fd3006410bmr1731104wmf.34.1694602449407; Wed, 13 Sep 2023 03:54:09 -0700 (PDT) Received: from localhost.localdomain ([2a02:6b6a:b5c7:0:59d5:28b9:7f13:cd1b]) by smtp.gmail.com with ESMTPSA id p36-20020a05600c1da400b003ff013a4fd9sm1757055wms.7.2023.09.13.03.54.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Sep 2023 03:54:08 -0700 (PDT) From: Usama Arif To: linux-mm@kvack.org, muchun.song@linux.dev, mike.kravetz@oracle.com, rppt@kernel.org Cc: linux-kernel@vger.kernel.org, songmuchun@bytedance.com, fam.zheng@bytedance.com, liangma@liangbit.com, punit.agrawal@bytedance.com, Usama Arif Subject: [v5 4/4] mm: hugetlb: Skip initialization of gigantic tail struct pages if freed by HVO Date: Wed, 13 Sep 2023 11:54:01 +0100 Message-Id: <20230913105401.519709-5-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230913105401.519709-1-usama.arif@bytedance.com> References: <20230913105401.519709-1-usama.arif@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 2533B18001D X-Stat-Signature: q4m7g8qacbbxmg9qofag5gc7zjec5x46 X-Rspam-User: X-HE-Tag: 1694602450-359573 X-HE-Meta: U2FsdGVkX1/y0jRGH9U0yyFLz7Qs/GLkFxiP+Mu5+FgWq+GxnlKqKqgKdNDzi+QB2eJlMsRp4SQ3tGoFvi4T7edVpJpRn7ApdjSdd0wElNy45AQIn8OH6pqbZ47M35vn4f3Ugf6WyTd98/jZq0aawJCAjrX9NsoK68G9yCEBlfb6Py7RDpl05dJMk9i4G+Hw0qPF9Ksq8T3U6S1NBLSar43HQ+RL4dtJukd6ha99ACndJKhvyW7l5V5Gre2lNKTF6ZAz9IUiD6GWbyA7JHLleKf/Z86Pq2eTo6Ie3IJQ4zij2d8rqaeWzTIW0yEzFifCoOyRHA0smKK65QjaxDnE8wUIkT0QUTZjE2DNE9qERu/kUW8FSRo459wHqSf2ZPeuZ2OZsI5ZuzYZoHjlHIBjOlgeAuEJpcWsIXL200Mb2hm3Cbm59oo2Ii4KAGcVcjsMwkFkyZWANbrGRT4/HTxJK3xzC5+sYeR711iCj4ijM0rKg1toZU0S4oLRvYYCHQmvrsoUKdmTv/GJxddQf08h15b6utWqHPOlOXdGUoYoyqMQFjRL6kI9ZoHw1DQx3kRWVYvgpRYgKc7PS1O6zQLC2SpirQC+3jWLGzi78BUnVQL1OFMr2xGJxgzi0p9X8PfAmnH3l1i6Twb/lNGvDPMlOgrmejfI7iD++PxZCfpLQXc9A18FuibzmAw+uTLDMS8SbV9JtDoHJMBGq4t71eebkMlxLeGJ29bYhU2DGhorMkjy/iofExNZpE9kddKxtX2QrjrNqJNCX33Mj6S+AuHekTHGkAQXKSXJwo9o/Yx4wJ4nNetYX7jYv2y+pI791WlzQJr0PIGbEQ8uZbqBpNOhJAhymgd5qMXhPYqykap1B45QGLN1GHgdNobe77ThEXjPnC6C4zoFtLW6iWffKil3hwxtRlY0BFEqPpFObXtafoPr3zbQm1OB+FeCMG5QNwGMz0kQJixrieyZ0xLQSMD EEXePS7c JSQYLNxhnqJ59i3Q3W1E3iDfaVEKioVAh/KGp3t67qWeHWD5MW8eFSYcpatwk5bgWQtGNzUaM5r2UFYpj8/5p9paaCBO2JVSC8OmDEA76wuiIjPWeWlIqeVHfPEsT9XT3Dk6u3f8A5fXmqWYRq4TwfUPTreqP0vRkz/EdtKUwiATb6nlqk6QVDBrRaLfzcwiV/tQ4Xnm/BPCJpQJ/QOzMVYmbe6R/Fuyjtn7FEZoFMHwe2CQ/9bMA2wOgEEd+afchBB4Ady+AnUJhcW6kzWlsIxyu/SzWrw9gB6oDOQRCJ1MptAu/hZK+/1ZeVulg06fv4u3aM1IlQFVRF7ACuz2mAyp+IyyivZWLZ47XrXa7q7IW2c/tUZHnDQpUsyX5krrBH6p7SU9GaQen6ymSOetk3iSi9gpkuX6yVS13pYYnr6AtoVY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The new boot flow when it comes to initialization of gigantic pages is as follows: - At boot time, for a gigantic page during __alloc_bootmem_hugepage, the region after the first struct page is marked as noinit. - This results in only the first struct page to be initialized in reserve_bootmem_region. As the tail struct pages are not initialized at this point, there can be a significant saving in boot time if HVO succeeds later on. - Later on in the boot, the head page is prepped and the first HUGETLB_VMEMMAP_RESERVE_SIZE / sizeof(struct page) - 1 tail struct pages are initialized. - HVO is attempted. If it is not successful, then the rest of the tail struct pages are initialized. If it is successful, no more tail struct pages need to be initialized saving significant boot time. The WARN_ON for increased ref count in gather_bootmem_prealloc was changed to a VM_BUG_ON. This is OK as there should be no speculative references this early in boot process. The VM_BUG_ON's are there just in case such code is introduced. Signed-off-by: Usama Arif Reviewed-by: Muchun Song Reviewed-by: Mike Kravetz --- mm/hugetlb.c | 63 +++++++++++++++++++++++++++++++++++++------- mm/hugetlb_vmemmap.c | 2 +- mm/hugetlb_vmemmap.h | 9 ++++--- mm/internal.h | 3 +++ mm/mm_init.c | 2 +- 5 files changed, 64 insertions(+), 15 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index c32ca241df4b..ed37c6e4e952 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3169,6 +3169,15 @@ int __alloc_bootmem_huge_page(struct hstate *h, int nid) } found: + + /* + * Only initialize the head struct page in memmap_init_reserved_pages, + * rest of the struct pages will be initialized by the HugeTLB subsystem itself. + * The head struct page is used to get folio information by the HugeTLB + * subsystem like zone id and node id. + */ + memblock_reserved_mark_noinit(virt_to_phys((void *)m + PAGE_SIZE), + huge_page_size(h) - PAGE_SIZE); /* Put them into a private list first because mem_map is not up yet */ INIT_LIST_HEAD(&m->list); list_add(&m->list, &huge_boot_pages); @@ -3176,6 +3185,42 @@ int __alloc_bootmem_huge_page(struct hstate *h, int nid) return 1; } +/* Initialize [start_page:end_page_number] tail struct pages of a hugepage */ +static void __init hugetlb_folio_init_tail_vmemmap(struct folio *folio, + unsigned long start_page_number, + unsigned long end_page_number) +{ + enum zone_type zone = zone_idx(folio_zone(folio)); + int nid = folio_nid(folio); + unsigned long head_pfn = folio_pfn(folio); + unsigned long pfn, end_pfn = head_pfn + end_page_number; + int ret; + + for (pfn = head_pfn + start_page_number; pfn < end_pfn; pfn++) { + struct page *page = pfn_to_page(pfn); + + __init_single_page(page, pfn, zone, nid); + prep_compound_tail((struct page *)folio, pfn - head_pfn); + ret = page_ref_freeze(page, 1); + VM_BUG_ON(!ret); + } +} + +static void __init hugetlb_folio_init_vmemmap(struct folio *folio, struct hstate *h, + unsigned long nr_pages) +{ + int ret; + + /* Prepare folio head */ + __folio_clear_reserved(folio); + __folio_set_head(folio); + ret = page_ref_freeze(&folio->page, 1); + VM_BUG_ON(!ret); + /* Initialize the necessary tail struct pages */ + hugetlb_folio_init_tail_vmemmap(folio, 1, nr_pages); + prep_compound_head((struct page *)folio, huge_page_order(h)); +} + /* * Put bootmem huge pages into the standard lists after mem_map is up. * Note: This only applies to gigantic (order > MAX_ORDER) pages. @@ -3186,19 +3231,19 @@ static void __init gather_bootmem_prealloc(void) list_for_each_entry(m, &huge_boot_pages, list) { struct page *page = virt_to_page(m); - struct folio *folio = page_folio(page); + struct folio *folio = (void *)page; struct hstate *h = m->hstate; VM_BUG_ON(!hstate_is_gigantic(h)); WARN_ON(folio_ref_count(folio) != 1); - if (prep_compound_gigantic_folio(folio, huge_page_order(h))) { - WARN_ON(folio_test_reserved(folio)); - prep_new_hugetlb_folio(h, folio, folio_nid(folio)); - free_huge_folio(folio); /* add to the hugepage allocator */ - } else { - /* VERY unlikely inflated ref count on a tail page */ - free_gigantic_folio(folio, huge_page_order(h)); - } + + hugetlb_folio_init_vmemmap(folio, h, HUGETLB_VMEMMAP_RESERVE_PAGES); + prep_new_hugetlb_folio(h, folio, folio_nid(folio)); + /* If HVO fails, initialize all tail struct pages */ + if (!HPageVmemmapOptimized(&folio->page)) + hugetlb_folio_init_tail_vmemmap(folio, HUGETLB_VMEMMAP_RESERVE_PAGES, + pages_per_huge_page(h)); + free_huge_folio(folio); /* add to the hugepage allocator */ /* * We need to restore the 'stolen' pages to totalram_pages diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 3cdb38d87a95..772a877918d7 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -589,7 +589,7 @@ static int __init hugetlb_vmemmap_init(void) const struct hstate *h; /* HUGETLB_VMEMMAP_RESERVE_SIZE should cover all used struct pages */ - BUILD_BUG_ON(__NR_USED_SUBPAGE * sizeof(struct page) > HUGETLB_VMEMMAP_RESERVE_SIZE); + BUILD_BUG_ON(__NR_USED_SUBPAGE > HUGETLB_VMEMMAP_RESERVE_PAGES); for_each_hstate(h) { if (hugetlb_vmemmap_optimizable(h)) { diff --git a/mm/hugetlb_vmemmap.h b/mm/hugetlb_vmemmap.h index 25bd0e002431..4573899855d7 100644 --- a/mm/hugetlb_vmemmap.h +++ b/mm/hugetlb_vmemmap.h @@ -10,15 +10,16 @@ #define _LINUX_HUGETLB_VMEMMAP_H #include -#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP -int hugetlb_vmemmap_restore(const struct hstate *h, struct page *head); -void hugetlb_vmemmap_optimize(const struct hstate *h, struct page *head); - /* * Reserve one vmemmap page, all vmemmap addresses are mapped to it. See * Documentation/vm/vmemmap_dedup.rst. */ #define HUGETLB_VMEMMAP_RESERVE_SIZE PAGE_SIZE +#define HUGETLB_VMEMMAP_RESERVE_PAGES (HUGETLB_VMEMMAP_RESERVE_SIZE / sizeof(struct page)) + +#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP +int hugetlb_vmemmap_restore(const struct hstate *h, struct page *head); +void hugetlb_vmemmap_optimize(const struct hstate *h, struct page *head); static inline unsigned int hugetlb_vmemmap_size(const struct hstate *h) { diff --git a/mm/internal.h b/mm/internal.h index d1d4bf4e63c0..d74061aa6de7 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1154,4 +1154,7 @@ struct vma_prepare { struct vm_area_struct *remove; struct vm_area_struct *remove2; }; + +void __meminit __init_single_page(struct page *page, unsigned long pfn, + unsigned long zone, int nid); #endif /* __MM_INTERNAL_H */ diff --git a/mm/mm_init.c b/mm/mm_init.c index 50f2f34745af..fed4370b02e1 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -555,7 +555,7 @@ static void __init find_zone_movable_pfns_for_nodes(void) node_states[N_MEMORY] = saved_node_state; } -static void __meminit __init_single_page(struct page *page, unsigned long pfn, +void __meminit __init_single_page(struct page *page, unsigned long pfn, unsigned long zone, int nid) { mm_zero_struct_page(page);