From patchwork Fri Aug 25 11:18:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13365444 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8D0EC3DA66 for ; Fri, 25 Aug 2023 11:18:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8904928005F; Fri, 25 Aug 2023 07:18:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 824918E0011; Fri, 25 Aug 2023 07:18:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5D367280091; Fri, 25 Aug 2023 07:18:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 3BF3628005F for ; Fri, 25 Aug 2023 07:18:53 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 116F48062F for ; Fri, 25 Aug 2023 11:18:53 +0000 (UTC) X-FDA: 81162379746.09.2D14D46 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) by imf06.hostedemail.com (Postfix) with ESMTP id DAD4F18002A for ; Fri, 25 Aug 2023 11:18:50 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="K/Jam9ne"; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf06.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.221.48 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692962331; a=rsa-sha256; cv=none; b=iX363L9lV8+aj5ETFZMScxDYqQnHTR2BNqjQNguBxNdJMLFXKV1kpt/3L9si6dCypvVHxs sWzLGOu6pzdaXySQOjeuxR+pglSRj9qkbqVYz8NosQCcFcfBw972/D/gOi8ncHORB2Zy+R tpRuONrxhOZzqEogA4t/bIqkWlcVZYQ= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="K/Jam9ne"; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf06.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.221.48 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692962331; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nxvZHKtYENKC1N901ur/ZbT/Aj89mU0uDVwxqxk+868=; b=ilR3e87253qRqv98l1YBn+yBJklDqLts9nPJqXivPHbs/h7jvuntNQTLMXQJz9h2noq0D1 4JjBj7yUdyepCyhzwMPK2Y1EKPNbCYsPD7xUJjcLaSGdaUdcN5PUtuQD/DxhYHwAmB/Sxb y+XWOSueQJ9vRo9KVFhjDoL2xBNonyM= Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-31c5cac3ae2so618013f8f.3 for ; Fri, 25 Aug 2023 04:18:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1692962329; x=1693567129; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nxvZHKtYENKC1N901ur/ZbT/Aj89mU0uDVwxqxk+868=; b=K/Jam9neS+gbs4EpfQDhGOSkHOH1JbnmbxU85K89H2k7ENKtddEhSBMHutTYc2ujhw ZMp0c1Adl/pniAsythqyZLNw4NXnEULOCFp0MuLwG2lxrPEoKnblKqitSh0fpcWNhXkz C4MYw4KlJP7ZB7HCzQeS49dldXs559hTCc1db7Kq3xFpD1EY8DCo7sLfxOqmJl6PxG/S wbJjBOIR3B2vAcL9tu3KuUiquHb/e+5RaoNbkUK/TBgFdvAo0zeqvkuVeYFqFBSB6MUa tZwke+2F5iihc4kb3eDabnmS298FS/DaifERsJLqGQC2ThyLlXPsFG2aaNJGbHq00ppL y1JQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692962329; x=1693567129; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nxvZHKtYENKC1N901ur/ZbT/Aj89mU0uDVwxqxk+868=; b=iDJ2tH9ZOLHTvnp43G/Enn6ttyv6DsdAlFQlgX6c56db76NNS7jWBB797RIYdOkpGB LekIdDS5f0HDqbvvcMi2iEBt1QLc2akmcCRHarPYV+U/P4kUQjRjp4M/Yu9yBTK23hkK QGcTdy8/DVY+UkoSx97E7TBPQo2BmAba3zpAJmSyoxRdjh8wqzc8xQctQHkZV/4X04EI cMsIJESY5vzRAyJb5GvwiC8Vjo4EW0GCe5FpRqj9Y5+JbxvrjY2SB/i5hYDdaP+O7TOf T1Q4qE9W4F2p8Mipxd2GAK4oCeSOmBonqYSGcpOP3yOFr+HEodTdkqCFQ2rJFPqQoFCK D+BQ== X-Gm-Message-State: AOJu0YyYZvDju7qChtcm73KopEtbV6ujFz5JkZyuk0DA1XWJ1x4ClPzE eOlufYSeyOreTgutD1l6qYHup1PKNw5F3Zg5x+4= X-Google-Smtp-Source: AGHT+IG7fidFBnhPCCm7z2OPyt2Wleqo2J6p8ROG+32Spl+z/EhkhXc8nY2Kixdo4gyAJlJbfWGltA== X-Received: by 2002:a5d:568f:0:b0:317:e5ec:8767 with SMTP id f15-20020a5d568f000000b00317e5ec8767mr13413417wrv.21.1692962329092; Fri, 25 Aug 2023 04:18:49 -0700 (PDT) Received: from localhost.localdomain ([2a02:6b6a:b5c7:0:9000:8d13:d0bb:ba7d]) by smtp.gmail.com with ESMTPSA id g9-20020a056000118900b0031ad5fb5a0fsm1939502wrx.58.2023.08.25.04.18.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Aug 2023 04:18:48 -0700 (PDT) From: Usama Arif To: linux-mm@kvack.org, muchun.song@linux.dev, mike.kravetz@oracle.com, rppt@kernel.org Cc: linux-kernel@vger.kernel.org, songmuchun@bytedance.com, fam.zheng@bytedance.com, liangma@liangbit.com, punit.agrawal@bytedance.com, Usama Arif Subject: [v3 1/4] mm: hugetlb_vmemmap: Use nid of the head page to reallocate it Date: Fri, 25 Aug 2023 12:18:33 +0100 Message-Id: <20230825111836.1715308-2-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230825111836.1715308-1-usama.arif@bytedance.com> References: <20230825111836.1715308-1-usama.arif@bytedance.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: DAD4F18002A X-Stat-Signature: gf3jyjunbqo6wwyyc1ixxx8wab6yj4py X-HE-Tag: 1692962330-818077 X-HE-Meta: U2FsdGVkX1+OAHnRMFl0z6YC68zD8DT0RN0ZRJODaax8+qzOAk9cxjl2ploxNGECw6BwKR766Rdn5RkWf4ruCLxgdqv95nTah9w4fc6K5p+Wl7MBuV8oSY8Eh6AiHOXYPju4xisqKa45gzDK6zQtefKt2EDlvDjgj3rG/YcYl6c79RZgFuMQBDbLRbQXMYa/ECPgsoHARiN6/qRtIAi/B3zszJ0d26zK64b0JYBR+4BXIJ81xPVlWDRBUz0KgXQOvPgY6PboAFO/F4GbIIppud7kge1yoDJiFvLcibu0Mcyxwcw+boxkk/tociNfFMN+PW7jK7XtnnnlzMm3Jnq48p5GAV0m6fqPXnWD1E9voDSkRbflNfuKIuDCZFtmH7nHeVQWL7IAW1pfjYMtUW3d+U5kTbgq09m2HsKQaGcSwUDT//eAn3Fj87t5AIrbMWWjQscZb6n4tkS3No7nqrzMrjkJKSpTVwgMCeOs9Ba9ahAd03sVtvR2CbAWuN8MK035eCwCCtogxjvDZ/QSENltqzH5+D1HCGVyPR8PPIB4C7OlIbhVk8Zp2VMjHc9eDM8MgIA8S6Qe8ZRzppFnW/e7pps7c/Tmgr/Z6LIh6cdxAYnd5QO8gAiQx3w+qFD/O7kDKoh+hWTqcAQG+IJ3Jr3WpASS5tnrcg3I3OMGSZZ5b4yT7H3txLd5lVMgxOl5ejWvrsCk12b6MxSSJHQ6d43vAuj54nOwgf14le9Mu+83HsazONKbFfiN/qTA41aa4wxum5FsLVUiWgVi2UqG4PWuI6QsemPzAHbR0VSqMRifDWsOiH+oFdYWias8kXJDaPIVK2vBStLA62CyYlGsGwfKvjWNOFxdebqFYBYeqBM9wOQZHj2uWE7+iyLy87a/IBV8CRvUqNI8cEZwHLO5Fk40Rh3QxcYY/XkCLlMN3DJS1maw9bORlKzs0AesTGrntawIJ00FcN/AN2ksLH1CoTp I5GjIPV1 UwV0lwv8G1nCAbiJWdcUlebhHFrEzRXAYBw0cekRkVp50SnQukYypwUifx3dLTwYa6kpGM9Ezbuk6NmogpMabMvGO9a500BkWLyyWP5Auqc2r8dpPOM1YelNemGCqfLCRLqBG7FcjLMqIdLqAhU8yZtZ+RNFAaIGImkzWf/HnTW2VD/nyOcZ6Ra38cU3f2REfRga5kSDb0zG7dCLD8uFWvLjznU4kzjrVXGbY2y6v6/iMP4GHev/Yy18ft+3O1H/ngFHsVdtwYjNwZ4OjVemKyKuW8hgXTgaRZ4ZjnnvuUeJAAePMs6Az6Pl/pXsvtKGvVfF/wwjTE/8wRlBthQp9wfdmoQqZtQBKuou+P9reNFK9I5mEbRPcNIDGTGTDJtetx0A9T9Sg96Ie+DdODatRumfUecpz5O3s36G6gMm+uvvV3EUmV8si3AqGfFsKl8j9yyeEVD65szXwdy+95rPF+x3zcA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If tail page prep and initialization is skipped, then the "start" page will not contain the correct nid. Use the nid from first vmemap page. Signed-off-by: Usama Arif Reviewed-by: Muchun Song Reviewed-by: Mike Kravetz --- mm/hugetlb_vmemmap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index c2007ef5e9b0..208907f2c5e1 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -324,7 +324,7 @@ static int vmemmap_remap_free(unsigned long start, unsigned long end, .reuse_addr = reuse, .vmemmap_pages = &vmemmap_pages, }; - int nid = page_to_nid((struct page *)start); + int nid = page_to_nid((struct page *)reuse); gfp_t gfp_mask = GFP_KERNEL | __GFP_THISNODE | __GFP_NORETRY | __GFP_NOWARN; From patchwork Fri Aug 25 11:18:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13365445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42B71C88CB2 for ; Fri, 25 Aug 2023 11:18:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 387748E0011; Fri, 25 Aug 2023 07:18:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2EA2E280091; Fri, 25 Aug 2023 07:18:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13C588E0016; Fri, 25 Aug 2023 07:18:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id EE8168E0011 for ; Fri, 25 Aug 2023 07:18:53 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C8ECDA0614 for ; Fri, 25 Aug 2023 11:18:53 +0000 (UTC) X-FDA: 81162379746.04.12F668B Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by imf30.hostedemail.com (Postfix) with ESMTP id E567B8001A for ; Fri, 25 Aug 2023 11:18:51 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=arFqt1RT; spf=pass (imf30.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.221.41 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692962332; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LjF+zN6TJ1BvEvRrI5C4CD8rIkkkthqR+fjsUt4Q8CQ=; b=dGtinzwsLQHityj5Z0jX2Qs/7+vAQkzXf/lyJMfyIVTF1+fbUVVULF++oZgT8jEbzaaWhi L/REQLdVmrIwKE7fhG90r99gNo1SHgFVtGHU75sfY0IMd1/WmA8jpeBqKIl+cp02SUiNYS KPRFiuPPNlMnumSxJHNeQbvr8Sgo9PE= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=arFqt1RT; spf=pass (imf30.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.221.41 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692962332; a=rsa-sha256; cv=none; b=2R0NALmgHEQheETdCEKpCt5I9AApIGACexCkDiMODgj5FykMgmihXovlLKFfHrW9uKXTzd Zr9Zb8caTkQE3reUaAJacYlMnB00yJPeaNbwCrx0OWmkdlhqguNjqYA1eQ8sRgVZsZLThi EKtAtT+zUTnBWhaK/IJ05vU+LFkf0Gs= Received: by mail-wr1-f41.google.com with SMTP id ffacd0b85a97d-31aec0a1a8bso554414f8f.0 for ; Fri, 25 Aug 2023 04:18:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1692962330; x=1693567130; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LjF+zN6TJ1BvEvRrI5C4CD8rIkkkthqR+fjsUt4Q8CQ=; b=arFqt1RTyp2yJ+qBHYnjbmsTvf9yikTfxTS2ZNvYs2M11Hlpj8DhIjlISwrDnL6HJG WEAKtjAc0kFEN8lxFwik4/1OUesCxPtqjE8+g0XBxBCrsd8wW2t+0KR83Ei7FuHlf3+N q44ZOVmcpl/SM5UrImy/4Zwyb3S4VKAI9MvDv5kHG3Wl49UpTUbepRZ8KnvEAvA+tbHw 6iuQYc0p0a1s7nX0im3kgSKFNlKt2Dpu3gp59UyVHEqsh7BFth7+gIqsiHpq9VZvb1ar YrKNpyAQzfy61sEIwUFBts1gdBuOR47DT+cM+daulfr4gUp3ttCceUtshX1CRUx3IgGV vfdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692962330; x=1693567130; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LjF+zN6TJ1BvEvRrI5C4CD8rIkkkthqR+fjsUt4Q8CQ=; b=PVOmCJvQNKQ1vwzbbtSMkYjefnM/e+ERkPGKcu+UQdYBDTOzOF5P8WCGYdS8c4Pr5u khkEViOaBGPc8i1ZQGL6kWEkGPHOK7OofZZh1dGN9ubje3VwgSqoZ7rSCP1gXU5u4sec AdJDiNV65jY6DnoBRm6bEaRmGfgfyQ5F6i5b/cZDqpZ4onnM16Z91UqMzUb9zONYrIlY /AAiHFu0hLNfCWV7qVp+ChPnCQoUo6MiPbNzmxGudFyA9FplUE/tIcfwoSAFs6pSG7Sb y+8361IABhYC+cj2/zKOc0yJfzk0VV8PuQ7Mw6wKDz44GdzRxG+lkz2Id2/0N1J05a05 Zlhg== X-Gm-Message-State: AOJu0Ywyk4YVNJg2WYeJ1jChvp7VCY4inGsqEvEmZ7pEYEcdQnrBndP3 /r8nARJfIc+o5GXxqpuM+RX1uSETiYh0n3JESq8= X-Google-Smtp-Source: AGHT+IFlq4uXcqCHuwCivQQQkMphZFErOKTGJdelk1WSHrPd5grjavTs+RZd4i9P7/+NjTNfRJorLw== X-Received: by 2002:a5d:510f:0:b0:313:df09:acfb with SMTP id s15-20020a5d510f000000b00313df09acfbmr16500274wrt.11.1692962330072; Fri, 25 Aug 2023 04:18:50 -0700 (PDT) Received: from localhost.localdomain ([2a02:6b6a:b5c7:0:9000:8d13:d0bb:ba7d]) by smtp.gmail.com with ESMTPSA id g9-20020a056000118900b0031ad5fb5a0fsm1939502wrx.58.2023.08.25.04.18.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Aug 2023 04:18:49 -0700 (PDT) From: Usama Arif To: linux-mm@kvack.org, muchun.song@linux.dev, mike.kravetz@oracle.com, rppt@kernel.org Cc: linux-kernel@vger.kernel.org, songmuchun@bytedance.com, fam.zheng@bytedance.com, liangma@liangbit.com, punit.agrawal@bytedance.com, Usama Arif Subject: [v3 2/4] memblock: pass memblock_type to memblock_setclr_flag Date: Fri, 25 Aug 2023 12:18:34 +0100 Message-Id: <20230825111836.1715308-3-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230825111836.1715308-1-usama.arif@bytedance.com> References: <20230825111836.1715308-1-usama.arif@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: E567B8001A X-Rspam-User: X-Stat-Signature: r85ssx5yg6ehpmka5xj7zfgfxey13fnx X-Rspamd-Server: rspam01 X-HE-Tag: 1692962331-744719 X-HE-Meta: U2FsdGVkX19VBA7r67DlKCM1ag8j6oEg88imdg9w4rreVhJvgb3DxSB8S6LmPtnR4WAYW2ZhCIcaazvvpOXBbI6I8V//pRWFP0BESEb18BodkYUgNno74qym7rdna3Kc0SMILDGHXi/Befy8dmvvDVkoiCX0NqfzjpfR1FUHpYyusuQ0saAYGDY37OlNRK0n0bomJEesrY4ddpFfZoQReYgSZt+5b7Rc+yLADUnDLeQRAmBEv+EAr3tQNI4kER4NWwkXdkEGrdmXV0CXNjfjU+pReRWnmsNvT9G04ZVcCsYiydngWBDraOxeqkTG2vTuituTahDXoL5M8L+0jME8TutXG5INIwn+QEG0WFTr2xTr9m4aNsNtmW7ofm4vhtALSop4XKDY+NbJvsA5NkYw6apJZhimYfzaJyh30CKTSVMF+SkPgmDlVQu8jxlJKUG8iwoDMkoGzrzzMH7tPnXtslnZVVUSXgWUubR1eFnJHNAMFO2f4Bo6sCoWSpOg3KYcZCgGjTwC2Zz0gvBugiW9+p0e9xQM9EqdmwLYpf9o18CU0Zis0XRhkpJ7H1k9KuI+sbJsq4rbX6auAAaiECFNXxOO5pmfxSew2VUGNwyrZZwgmPdEooHSrX/4R8x9awUiq5a40PB826gzekvuidRejuJ59DUXHfHE6t3CoooAHbfRgBkQeroQU7YGN2YIXRdV7Fs6kMRM5nmflYGmkdoEOBupIbKrSLUlt9yxbmkKdQLcd4ovoRP1q5UmQUZgggzb8o+//kv3UL5WRjYXartPYvjvp78d4zIXv8S4T+0HexLPNyzK8ZFvjvJYgUz892KXXwZDB21aCsnTEMDOTx5Eqpitox9wDMjxkGKxjCe7glNp9l+9nCj07bOYG0EQCIiY6YR7P8fgVec3EwlmpNjr9VKoiCVIQeEE0xhOdGJLB4UVPXltmj0egbBRE8JasddwgynX5FZJIci9BlQpl9h XMr2S2ef JuRMD58YSyr90CeEF6QFce3t/qHQEQhb8Hrjo3SkZZD6dGuHxYgpFns0+8ePWMOurkIF9T2CQXiCWlXo6LM1crWE3+xS5qAezfPxDS3pAAIBOlBtbL0vNJDxizyPcFD8xp1QCLgfHihSFkpl4iO5QZiWtBajsnn61N+iTZAMtGXwo3ZyW/rNNacOWiVyHBlmlnrSYim3gUPtKBeGcOy2NKvQfKiY49ZW54YuNr3UXqvQLONdgxHEAsoY54wfa8SyWqCMbK5zaFSZz9PuuCbdori27+q+dvih9FewKEfQ0/3mmXlgU5a0qXVhi6nm8mQPjEsUHrmCGb2SNqe5IJqshKyPc27hb5DvYrdlZ0FApS1JXpWvZjUre28YRJeZZ4Iip8H2dkie06peyw67BV3+TOisY0LL6rIC12V+2YNRcSuYKI25GnbDBuYM+1sMb/0glR4lkltGgdQU9JocPeCyoMflvEw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This allows setting flags to both memblock types and is in preparation for setting flags (for e.g. to not initialize struct pages) on reserved memory region. Signed-off-by: Usama Arif Reviewed-by: Muchun Song Reviewed-by: Mike Rapoport (IBM) Acked-by: Mike Kravetz --- mm/memblock.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/mm/memblock.c b/mm/memblock.c index f9e61e565a53..43cb4404d94c 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -896,10 +896,9 @@ int __init_memblock memblock_physmem_add(phys_addr_t base, phys_addr_t size) * * Return: 0 on success, -errno on failure. */ -static int __init_memblock memblock_setclr_flag(phys_addr_t base, - phys_addr_t size, int set, int flag) +static int __init_memblock memblock_setclr_flag(struct memblock_type *type, + phys_addr_t base, phys_addr_t size, int set, int flag) { - struct memblock_type *type = &memblock.memory; int i, ret, start_rgn, end_rgn; ret = memblock_isolate_range(type, base, size, &start_rgn, &end_rgn); @@ -928,7 +927,7 @@ static int __init_memblock memblock_setclr_flag(phys_addr_t base, */ int __init_memblock memblock_mark_hotplug(phys_addr_t base, phys_addr_t size) { - return memblock_setclr_flag(base, size, 1, MEMBLOCK_HOTPLUG); + return memblock_setclr_flag(&memblock.memory, base, size, 1, MEMBLOCK_HOTPLUG); } /** @@ -940,7 +939,7 @@ int __init_memblock memblock_mark_hotplug(phys_addr_t base, phys_addr_t size) */ int __init_memblock memblock_clear_hotplug(phys_addr_t base, phys_addr_t size) { - return memblock_setclr_flag(base, size, 0, MEMBLOCK_HOTPLUG); + return memblock_setclr_flag(&memblock.memory, base, size, 0, MEMBLOCK_HOTPLUG); } /** @@ -957,7 +956,7 @@ int __init_memblock memblock_mark_mirror(phys_addr_t base, phys_addr_t size) system_has_some_mirror = true; - return memblock_setclr_flag(base, size, 1, MEMBLOCK_MIRROR); + return memblock_setclr_flag(&memblock.memory, base, size, 1, MEMBLOCK_MIRROR); } /** @@ -977,7 +976,7 @@ int __init_memblock memblock_mark_mirror(phys_addr_t base, phys_addr_t size) */ int __init_memblock memblock_mark_nomap(phys_addr_t base, phys_addr_t size) { - return memblock_setclr_flag(base, size, 1, MEMBLOCK_NOMAP); + return memblock_setclr_flag(&memblock.memory, base, size, 1, MEMBLOCK_NOMAP); } /** @@ -989,7 +988,7 @@ int __init_memblock memblock_mark_nomap(phys_addr_t base, phys_addr_t size) */ int __init_memblock memblock_clear_nomap(phys_addr_t base, phys_addr_t size) { - return memblock_setclr_flag(base, size, 0, MEMBLOCK_NOMAP); + return memblock_setclr_flag(&memblock.memory, base, size, 0, MEMBLOCK_NOMAP); } static bool should_skip_region(struct memblock_type *type, From patchwork Fri Aug 25 11:18:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13365446 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00C7CC71133 for ; Fri, 25 Aug 2023 11:18:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E36BC280093; Fri, 25 Aug 2023 07:18:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DBFF8280091; Fri, 25 Aug 2023 07:18:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5F48280093; Fri, 25 Aug 2023 07:18:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B19E0280091 for ; Fri, 25 Aug 2023 07:18:54 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 748ED8061B for ; Fri, 25 Aug 2023 11:18:54 +0000 (UTC) X-FDA: 81162379788.30.4185E74 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) by imf30.hostedemail.com (Postfix) with ESMTP id 8976680021 for ; Fri, 25 Aug 2023 11:18:52 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=EsMNC1tz; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf30.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.221.48 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692962332; a=rsa-sha256; cv=none; b=5Qdu2d8vV8ebphofYI96LPgEFTuJBLcIetJPyQDCjomB3E1/PSFfgQoQcMNMK0XLqwQcAK DweDR1rhGAseVaNwZgEAf1rzkoA+oRsx7BOKav8gUj/Us6zxaHKxjuygcC65W0mw/TPrUz P1JTlQCFLg9tjAnfyWm4Csu6zVJ2C1g= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=EsMNC1tz; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf30.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.221.48 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692962332; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rt7VA+fn11wPN2WZOsDR2kSrdsBwWADUH8ZcQSjaiPs=; b=zR00Fr4ff23Yo1DHGeoDV5Zxuw9kZeTtdP8QktJYZuh4W+ZYhKtCKwTNQDpwNN0fW6yvp2 rPyISYJteRu450bNj7RK1V8CHzneRbWrXh1zRDhVVU39t001WTtx8D5FJT0fsOAn/u+Sag +SE9AVKjd6Uky64LEJpEPRPCT5nnPnw= Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-31c7912416bso689407f8f.1 for ; Fri, 25 Aug 2023 04:18:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1692962331; x=1693567131; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rt7VA+fn11wPN2WZOsDR2kSrdsBwWADUH8ZcQSjaiPs=; b=EsMNC1tzWSxQdc599l5SHDEEa98rqlp7iDryuFhBYk8fHJ511NFQyTznh0mCDBUIn8 VL8K3tloTbGtKUrcH4jjAAsVmaUTZ2RNBUAs8YhCr5NG+vVevNZdq3uQ1G1uBY6LhJVF rTwmWgEz1MvQA9fuhdFC9bXz4LoNfymZjqSABGUPV23JzQFZnk9thziUhDk9YadzQzpK 0Y4SguwVDubIC/esFXv70EfncJva9GyavT5l0kJrs+9ZjMhqgofSo0kMiSVl8P6uaV89 V29hVlv/XbysBximjie5EPo24O7osCBf+oDoT7rAPNGdseqXFPTWzYczRyNlM8M7odR8 9gkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692962331; x=1693567131; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rt7VA+fn11wPN2WZOsDR2kSrdsBwWADUH8ZcQSjaiPs=; b=C36JjLsvcpObFIWL0HZ6O1HcBjyq6Lg6EcyJUZcn/WhXCcv5SUCp0UtWOhOZmIUI0W Gs6GK6YKOTjkyD4L0kUMysEud+pFFGlQBKJ4pdD/GkSmcdKsU7iJJBfr4ijRRWb6pqTw SH3ryFYvQWNs4DyUNA3zCeHOohrz4MWcwNt0Sqh3C9SvypvHyEPXHPR6Be/xaDJV4qQ1 +h/8JvFgniRqiK2vmNuIcCoYyRGhHNKqqXucB8XrvlUoTHmWZTQBl/ysRnP4gczVVg+Q MuIo+vDfOl2d468rfpHRFvgUzylDFDj1evYogyEmoAm3KZVTY42y2to9NjuZ8Bkesf+D frfg== X-Gm-Message-State: AOJu0YyWz+dIqi5O0GwKbIYfnDzK2QyobPgqQJQtTBE2osQLyBBLq2ZZ oTmJuBkLP2c5ve6LKPVjLMsGvS/jjU65DHMcrO0= X-Google-Smtp-Source: AGHT+IEec5EPpcaKObbex9bUWyp4GWlAmHPClDlt9NrVYi+5iTZNS2Jj3ebXr6R8Gt7v2gUbQKOW1A== X-Received: by 2002:a5d:4403:0:b0:31a:d49a:38d with SMTP id z3-20020a5d4403000000b0031ad49a038dmr13545294wrq.54.1692962331003; Fri, 25 Aug 2023 04:18:51 -0700 (PDT) Received: from localhost.localdomain ([2a02:6b6a:b5c7:0:9000:8d13:d0bb:ba7d]) by smtp.gmail.com with ESMTPSA id g9-20020a056000118900b0031ad5fb5a0fsm1939502wrx.58.2023.08.25.04.18.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Aug 2023 04:18:50 -0700 (PDT) From: Usama Arif To: linux-mm@kvack.org, muchun.song@linux.dev, mike.kravetz@oracle.com, rppt@kernel.org Cc: linux-kernel@vger.kernel.org, songmuchun@bytedance.com, fam.zheng@bytedance.com, liangma@liangbit.com, punit.agrawal@bytedance.com, Usama Arif Subject: [v3 3/4] memblock: introduce MEMBLOCK_RSRV_NOINIT_VMEMMAP flag Date: Fri, 25 Aug 2023 12:18:35 +0100 Message-Id: <20230825111836.1715308-4-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230825111836.1715308-1-usama.arif@bytedance.com> References: <20230825111836.1715308-1-usama.arif@bytedance.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 8976680021 X-Stat-Signature: qbwhuhth6b7sr3w7dffgwt984s94m3ff X-HE-Tag: 1692962332-508322 X-HE-Meta: U2FsdGVkX18Xua8B6NWgLiGPnxdyw7x2lYTKbefda1NHLxhrkuYTRfhfczotP1/Z2cnAozaCc0khNCeXz+WmPCRMM7WPLJXzzF/kQcxvR004UeTZ7D9XNVU48Bx/QXlzLDNCWswrf8f+6+wTyfOlOkoqejVA6nX3FwxjP4Q4CrKeV+tVwGqZu6XItMSx38T9F3e6RbZrNNYWx8KcMyc6MQXNSNB9+ZpIxHE6d2VfAnjikT8E3i8yabUP1CcAeO9HEEHLSzlMxTM5zODlhWrjbilDed1TP+pzaTSdDNWY538GdleZWIOD1QkjDjhWzmNHIFWXRIXTss1Tqqh1XzJ1+fTB2lO5+oxWKqpfJa1XJQG3cEhcFcTskQ1QYZixyxy8B5EB31OktO2g06juogPcsm1wnld6z+sMElWttGY54GFIVSE3OFWlLe6iCPiMGJiJgDIgIzSDFS6q/7HbVBR/GdjleERnMsriN6X/iU+BtxgTCet+zS8SEBvKU5oXyLT2Z6J+RiZH4eDOFlTkIYJoLkg76wrVol2NBXuRwXsoE4lpdRfA+ALJyP8I/W8EIOfzNtTIGnVoaxCmZfA4zKcDILQ3AgkJKYgXJqua12Ojhjb+4VgLjqQKJ9l5oxUEC297JGnL7CZoI554FM+ComiOePxnj1q1EczV+pYzXHaDBDWhuNckXP8odCInKu+Z0Q421+2CVvfGF7mNi2NtY449TubRICbuAWr2gPjeZtpI+gD5cL+EON3vXvYeosEbAq5xNwVgYpcp5HI7iAmFd1GLK1R6YW6CIFfN0YHFmIX+Qdo8nM2DXrZSwvfZLausKmtnHO3lsLQse57aHWevh7/TxE+JQcJHwS7lOLFLrxunqVirE7HWS2ibTboaylEv9rQc2PWaYp8LxLd3V2VIuHW3TNdrHjaBrKmo3dw85sdL+4QWBt5U1QUPThb036yxPc/jX5qSD74at3ReY0qRqcU qUkEU128 GD+pOmDFXz3vq4XB9OsOZICKLYMvMJ7jViIPta0VfZBjRlYnxmd/rNyoc340hy1RV71ZX3wtIRGu5a+v486xOv8vLDWitJEJKu0ZC8BVpj82QStAn22WYs+PBOgOcVdSvptJU2p3smfY0zz0NaO63EL+4+nRbxQ2RVVoJPlxhsoUZecMYpPA9d5KQCW7rYa450pOC/urSAqVdA6O8rRSzAveilnXWBiwDNIwhyDYOWQTulT42BK+WYfxtNGjtverPQjuWNb1QLgG5StWP8YJuVB4aOo9HlKXCh87SLE1K0jqJkg4cTzk7M0q3jYQcxxojfz8sjoP9aee4MuaXB5KBygHJWzr+iPVOXNfuN3njUOsEByug6j2uZHfyzajoPmLue6/2/E0CrhxqEEjsGRhf0pI9Ac8aiUzJRcaFP+Abju0QB8DR5n4uA+ZMMKKlQnZnpNN5pTuK6jI0TjReBqEK3wly9LjD6aZOaI97 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For reserved memory regions marked with this flag, reserve_bootmem_region is not called during memmap_init_reserved_pages. This can be used to avoid struct page initialization for regions which won't need them, for e.g. hugepages with HVO enabled. Signed-off-by: Usama Arif Reviewed-by: Muchun Song --- include/linux/memblock.h | 10 ++++++++++ mm/memblock.c | 32 +++++++++++++++++++++++++++----- 2 files changed, 37 insertions(+), 5 deletions(-) diff --git a/include/linux/memblock.h b/include/linux/memblock.h index f71ff9f0ec81..6d681d053880 100644 --- a/include/linux/memblock.h +++ b/include/linux/memblock.h @@ -40,6 +40,8 @@ extern unsigned long long max_possible_pfn; * via a driver, and never indicated in the firmware-provided memory map as * system RAM. This corresponds to IORESOURCE_SYSRAM_DRIVER_MANAGED in the * kernel resource tree. + * @MEMBLOCK_RSRV_NOINIT_VMEMMAP: memory region for which struct pages are + * not initialized (only for reserved regions). */ enum memblock_flags { MEMBLOCK_NONE = 0x0, /* No special request */ @@ -47,6 +49,8 @@ enum memblock_flags { MEMBLOCK_MIRROR = 0x2, /* mirrored region */ MEMBLOCK_NOMAP = 0x4, /* don't add to kernel direct mapping */ MEMBLOCK_DRIVER_MANAGED = 0x8, /* always detected via a driver */ + /* don't initialize struct pages associated with this reserver memory block */ + MEMBLOCK_RSRV_NOINIT_VMEMMAP = 0x10, }; /** @@ -125,6 +129,7 @@ int memblock_clear_hotplug(phys_addr_t base, phys_addr_t size); int memblock_mark_mirror(phys_addr_t base, phys_addr_t size); int memblock_mark_nomap(phys_addr_t base, phys_addr_t size); int memblock_clear_nomap(phys_addr_t base, phys_addr_t size); +int memblock_reserved_mark_noinit_vmemmap(phys_addr_t base, phys_addr_t size); void memblock_free_all(void); void memblock_free(void *ptr, size_t size); @@ -259,6 +264,11 @@ static inline bool memblock_is_nomap(struct memblock_region *m) return m->flags & MEMBLOCK_NOMAP; } +static inline bool memblock_is_noinit_vmemmap(struct memblock_region *m) +{ + return m->flags & MEMBLOCK_RSRV_NOINIT_VMEMMAP; +} + static inline bool memblock_is_driver_managed(struct memblock_region *m) { return m->flags & MEMBLOCK_DRIVER_MANAGED; diff --git a/mm/memblock.c b/mm/memblock.c index 43cb4404d94c..a9782228c840 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -991,6 +991,23 @@ int __init_memblock memblock_clear_nomap(phys_addr_t base, phys_addr_t size) return memblock_setclr_flag(&memblock.memory, base, size, 0, MEMBLOCK_NOMAP); } +/** + * memblock_reserved_mark_noinit_vmemmap - Mark a reserved memory region with flag + * MEMBLOCK_RSRV_NOINIT_VMEMMAP. + * @base: the base phys addr of the region + * @size: the size of the region + * + * struct pages will not be initialized for reserved memory regions marked with + * %MEMBLOCK_RSRV_NOINIT_VMEMMAP. + * + * Return: 0 on success, -errno on failure. + */ +int __init_memblock memblock_reserved_mark_noinit_vmemmap(phys_addr_t base, phys_addr_t size) +{ + return memblock_setclr_flag(&memblock.reserved, base, size, 1, + MEMBLOCK_RSRV_NOINIT_VMEMMAP); +} + static bool should_skip_region(struct memblock_type *type, struct memblock_region *m, int nid, int flags) @@ -2107,13 +2124,18 @@ static void __init memmap_init_reserved_pages(void) memblock_set_node(start, end, &memblock.reserved, nid); } - /* initialize struct pages for the reserved regions */ + /* + * initialize struct pages for reserved regions that don't have + * the MEMBLOCK_RSRV_NOINIT_VMEMMAP flag set + */ for_each_reserved_mem_region(region) { - nid = memblock_get_region_node(region); - start = region->base; - end = start + region->size; + if (!memblock_is_noinit_vmemmap(region)) { + nid = memblock_get_region_node(region); + start = region->base; + end = start + region->size; - reserve_bootmem_region(start, end, nid); + reserve_bootmem_region(start, end, nid); + } } } From patchwork Fri Aug 25 11:18:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13365447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9A25C7EE2C for ; Fri, 25 Aug 2023 11:19:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 50EDC280095; Fri, 25 Aug 2023 07:18:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 49767280091; Fri, 25 Aug 2023 07:18:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 274B7280095; Fri, 25 Aug 2023 07:18:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 07350280091 for ; Fri, 25 Aug 2023 07:18:56 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 8E4521C9832 for ; Fri, 25 Aug 2023 11:18:55 +0000 (UTC) X-FDA: 81162379830.17.4CD0FCC Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) by imf03.hostedemail.com (Postfix) with ESMTP id AE74120016 for ; Fri, 25 Aug 2023 11:18:53 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=KyUsO0r4; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf03.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.221.49 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692962333; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=r6ewY66oYf3vDH9aPoj7pb9ZlWRxktJaiCYaiSDx5Sk=; b=w/g52t08bRcULrkDlVmQ+0cldxUOMt4tRzwTpWZKnbx4MmxPwt0eNOMHHlt7oCNNxw+dHj wjS4uvjbXzhPQ5xCSWEEKdMg2zYoWVLi48Jbb1OcjVa5lkvRrSO8EWSeXyiZ0M/au/Zoss e8DLmbLaXI0EFH1orkkqVyCxCWwcP+w= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=KyUsO0r4; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf03.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.221.49 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692962333; a=rsa-sha256; cv=none; b=dEskc0zOQhrO8mfaHTc6Xl9BoBYrNFOaQiUum2AOVdM9nAOvEgbTZJDZ3wuMkNW3ZiURx8 FoR2ldt5Zwx18W/Y+P+kxstjD7yekWPwO8roAYq1lO9yGtOafnZdugBkA/fR3LrCON53r6 qgWll784Ed/pgadrx7Ap9tNsvAO9gwk= Received: by mail-wr1-f49.google.com with SMTP id ffacd0b85a97d-31c3726cc45so664235f8f.0 for ; Fri, 25 Aug 2023 04:18:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1692962332; x=1693567132; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=r6ewY66oYf3vDH9aPoj7pb9ZlWRxktJaiCYaiSDx5Sk=; b=KyUsO0r4rRvScvgaql7Mq5sBM59AgdiLlku31XVLehfo/OyOKwTeyZBHqhENtKIEl9 P3ymLBkS3If/LAKRNANtlRWYrskCp2K3RSSWfZcBfwGiK9kYFXjQDHz7qE2ymRYfFUMn uUbWss9XFgzITSidEKkZUWaJh7avtNC9lP7769KoK82tO41UWCXcLCs+4Xm8A5SkBzXK RYBbCFiYgMht2pLOGOKshr6GOT8+pWg3D47dUqEho1eTmsge1mAirZ0LvqexlT/0jC5M TYPbzcJPLn+3vaif0HDBwx0dC08rv+ZWDWMU5TML4/SojKT5VJpBFSDMRI3vpn/dt11/ wBLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692962332; x=1693567132; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=r6ewY66oYf3vDH9aPoj7pb9ZlWRxktJaiCYaiSDx5Sk=; b=BiITSgAn/GWU4sg2DQEupRC2ItuGGMgVLFytkSG/+tOIpicuGTa2NODAI0o0staBQt GCWPuUjaNiHqui9dLC/g1Yjyp69XIxiNvQFg/cG0ZRwcNu8AK1lYfME9sZjVLD9rt/JL KyM4aznSFJ/bcFgx1qLkNeoZF2y1M8fBWV+19bk6M+iIt2ftqeL8atFnG57H5GCq/If3 LlWeIBl+mSxlaVVshUUkLAJsWJoWyGfzRbueG7ADVwGJBEqG3xesj4uXWmtKWA1t4Q0W 4IF9cYvmhUUVdilLsQpPZMbXCde8IcgetiNJsp3SbHlwtnSs/+tbLkvTDBb38X5BCfxr 336A== X-Gm-Message-State: AOJu0YxoLr0FotX2qnJTZgAyAzbZTu61s2MINp1g2lj9KC57qY0zlVBx LjNAUS6tpDURs7IlIhmXTwyLiNa3HQAq6BSoauc= X-Google-Smtp-Source: AGHT+IGJuVnbgEdL7B1yqcNDta9SW8PrCZLooH6K+4U6WsXn8SKs1tWicPmF0cFJa/KnOnpJGZAshw== X-Received: by 2002:adf:f08b:0:b0:315:a32d:311f with SMTP id n11-20020adff08b000000b00315a32d311fmr14949179wro.14.1692962331890; Fri, 25 Aug 2023 04:18:51 -0700 (PDT) Received: from localhost.localdomain ([2a02:6b6a:b5c7:0:9000:8d13:d0bb:ba7d]) by smtp.gmail.com with ESMTPSA id g9-20020a056000118900b0031ad5fb5a0fsm1939502wrx.58.2023.08.25.04.18.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Aug 2023 04:18:51 -0700 (PDT) From: Usama Arif To: linux-mm@kvack.org, muchun.song@linux.dev, mike.kravetz@oracle.com, rppt@kernel.org Cc: linux-kernel@vger.kernel.org, songmuchun@bytedance.com, fam.zheng@bytedance.com, liangma@liangbit.com, punit.agrawal@bytedance.com, Usama Arif Subject: [v3 4/4] mm: hugetlb: Skip initialization of gigantic tail struct pages if freed by HVO Date: Fri, 25 Aug 2023 12:18:36 +0100 Message-Id: <20230825111836.1715308-5-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230825111836.1715308-1-usama.arif@bytedance.com> References: <20230825111836.1715308-1-usama.arif@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: AE74120016 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: yktyw54d6ixxb41gjo9acpud4mwrwdpw X-HE-Tag: 1692962333-871634 X-HE-Meta: U2FsdGVkX1/8yE5+o+GOIabkNYV+fDCuJYxQ1l6Dh/IXvlgeQYoq22ZkAmY/uDmHPGm0L2Mdwx1qJcf3lU4UaLvDsebDsVMlbgaMbZUTmd0u3+qfFJxGYTl0WJnctq0FGCdTBtJCket5iHuP2/sqt2vwmxeAZriJwZhdZ393AXfqVXRAooBMygTC9N3mTbrz39PrWGRHTfKIglu3pfpWHqSBfq2VLYs/ifwO483FjzSHwnuxyaq+iPJlRWUMN0zJfja5MhVobEbyhPqf+mnCqaYoEFObiExA74jlcW4EEO5hYhyrmLRADX4ooMh3EPe43TQl0Jr4IWaMLZtR3suReSXYvCC2Y9pL3egl4sKpbujTNbzAFlHgaM2y90530UfLMsPPV779siqZR6gIQnjQqM8+n7gd1SbspIgiUO805AA6vYGoOI0v/vikB5ebz1tCBV+FYNmVk5T3JlBwyoqwrQfTPHXWNABKcHaLNmfMkkxsAlDJ+3AJibu7qAqmQdqOG8zLCfKk7tD6vOOItNK2a9IwLsow0xIApTq2I7ZOFKZIsuKmZqT9s3zSRXQU1qjlm7+x6cTukjEZYwY6JCLJLZs4/Syz1jnjE5LAq4y8k3g8jNdfnQDgoMHWDt3g/mVrJpcqbQiCw+9yv/ZIrMJG+cMJnohs0NHMrFGbL5z5sv6k9kF7gNX3INJrxYn6V9jmWswAP7HFl2wI9BXrjiLHKNGF75W8eX0juNWufDaZ7Z/xxvCSqhimm0AguFdBPM8OP9tEM18fJVnoTe4VPSU93fDV9LS18TxpvU5nVZEo0fyehrjEW7D12Y0mm3kqt9OUdW3ncyG0v649UQw081v21F49KDqNGwoojA/klbxsY2N5bqkQ8Khsm4+wdX+AxWzv5e20LbwwelRebrIVbGEPflVvBA9HO8ZmybKoczIZhWBtLISryhxrijcYT6GHHSOmKH4J9QCle2JFHsN+NFl 55y9yHwK TAVtnPTYoOxKKslndEBZemsb2VcujyLsdeynNF5dCOAtTTJ2tL54CU8tBhlkwc8Ox3JKaUkdrJGCWdBCbSCjMQKBFgaHTdhIiLw8pQcZS9zsyKwXjbpm9yqqAiV7ILl8srA0uJBrOqQY/TXyIPeB8ycGIQDcu4ItXXkFB+7cfTUl0DqsoWGd5p9ibiKwtuKHZjoM3EtZurVtP1svM74sZ0vo1XfyIELH1lQuNGyDEm8OtVHDgeUCVVl5wpO7q6N97ZrowhfoHL2LMpBaVi80kzcz4OfRZt/E1zklTXG1mr/ZT+does/REk4T0FHKkNumR55jVIkVW0uTK3vgqNqf9nDMbA4Z5pZluw069bfOrhEqAkVuyEg0YOd4/VVjNOXfxmsUPcHQTvkScdMHU6aVgb1D4R8eWxLVzohWRdul04d1sHR4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The new boot flow when it comes to initialization of gigantic pages is as follows: - At boot time, for a gigantic page during __alloc_bootmem_hugepage, the region after the first struct page is marked as noinit. - This results in only the first struct page to be initialized in reserve_bootmem_region. As the tail struct pages are not initialized at this point, there can be a significant saving in boot time if HVO succeeds later on. - Later on in the boot, HVO is attempted. If its successful, only the first HUGETLB_VMEMMAP_RESERVE_SIZE / sizeof(struct page) - 1 tail struct pages after the head struct page are initialized. If it is not successful, then all of the tail struct pages are initialized. Signed-off-by: Usama Arif --- mm/hugetlb.c | 52 +++++++++++++++++++++++++++++++++++--------- mm/hugetlb_vmemmap.h | 8 +++---- mm/internal.h | 3 +++ mm/mm_init.c | 2 +- 4 files changed, 50 insertions(+), 15 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 6da626bfb52e..964f7a2b693e 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1953,7 +1953,6 @@ static void __prep_account_new_huge_page(struct hstate *h, int nid) static void __prep_new_hugetlb_folio(struct hstate *h, struct folio *folio) { - hugetlb_vmemmap_optimize(h, &folio->page); INIT_LIST_HEAD(&folio->lru); folio_set_compound_dtor(folio, HUGETLB_PAGE_DTOR); hugetlb_set_folio_subpool(folio, NULL); @@ -2225,6 +2224,7 @@ static struct folio *alloc_fresh_hugetlb_folio(struct hstate *h, return NULL; } } + hugetlb_vmemmap_optimize(h, &folio->page); prep_new_hugetlb_folio(h, folio, folio_nid(folio)); return folio; @@ -2943,6 +2943,7 @@ static int alloc_and_dissolve_hugetlb_folio(struct hstate *h, new_folio = alloc_buddy_hugetlb_folio(h, gfp_mask, nid, NULL, NULL); if (!new_folio) return -ENOMEM; + hugetlb_vmemmap_optimize(h, &new_folio->page); __prep_new_hugetlb_folio(h, new_folio); retry: @@ -3206,6 +3207,15 @@ int __alloc_bootmem_huge_page(struct hstate *h, int nid) } found: + + /* + * Only initialize the head struct page in memmap_init_reserved_pages, + * rest of the struct pages will be initialized by the HugeTLB subsystem itself. + * The head struct page is used to get folio information by the HugeTLB + * subsystem like zone id and node id. + */ + memblock_reserved_mark_noinit_vmemmap(virt_to_phys((void *)m + PAGE_SIZE), + huge_page_size(h) - PAGE_SIZE); /* Put them into a private list first because mem_map is not up yet */ INIT_LIST_HEAD(&m->list); list_add(&m->list, &huge_boot_pages); @@ -3213,6 +3223,27 @@ int __alloc_bootmem_huge_page(struct hstate *h, int nid) return 1; } +static void __init hugetlb_folio_init_vmemmap(struct hstate *h, struct folio *folio, + unsigned long nr_pages) +{ + enum zone_type zone = zone_idx(folio_zone(folio)); + int nid = folio_nid(folio); + unsigned long head_pfn = folio_pfn(folio); + unsigned long pfn, end_pfn = head_pfn + nr_pages; + + __folio_clear_reserved(folio); + __folio_set_head(folio); + + for (pfn = head_pfn + 1; pfn < end_pfn; pfn++) { + struct page *page = pfn_to_page(pfn); + + __init_single_page(page, pfn, zone, nid); + prep_compound_tail((struct page *)folio, pfn - head_pfn); + set_page_count(page, 0); + } + prep_compound_head((struct page *)folio, huge_page_order(h)); +} + /* * Put bootmem huge pages into the standard lists after mem_map is up. * Note: This only applies to gigantic (order > MAX_ORDER) pages. @@ -3223,19 +3254,19 @@ static void __init gather_bootmem_prealloc(void) list_for_each_entry(m, &huge_boot_pages, list) { struct page *page = virt_to_page(m); - struct folio *folio = page_folio(page); + struct folio *folio = (void *)page; struct hstate *h = m->hstate; + unsigned long nr_pages = pages_per_huge_page(h); VM_BUG_ON(!hstate_is_gigantic(h)); WARN_ON(folio_ref_count(folio) != 1); - if (prep_compound_gigantic_folio(folio, huge_page_order(h))) { - WARN_ON(folio_test_reserved(folio)); - prep_new_hugetlb_folio(h, folio, folio_nid(folio)); - free_huge_page(page); /* add to the hugepage allocator */ - } else { - /* VERY unlikely inflated ref count on a tail page */ - free_gigantic_folio(folio, huge_page_order(h)); - } + + hugetlb_vmemmap_optimize(h, &folio->page); + if (HPageVmemmapOptimized(&folio->page)) + nr_pages = HUGETLB_VMEMMAP_RESERVE_SIZE / sizeof(struct page); + hugetlb_folio_init_vmemmap(h, folio, nr_pages); + prep_new_hugetlb_folio(h, folio, folio_nid(folio)); + free_huge_page(page); /* add to the hugepage allocator */ /* * We need to restore the 'stolen' pages to totalram_pages @@ -3656,6 +3687,7 @@ static int demote_free_hugetlb_folio(struct hstate *h, struct folio *folio) else prep_compound_page(subpage, target_hstate->order); folio_change_private(inner_folio, NULL); + hugetlb_vmemmap_optimize(h, &folio->page); prep_new_hugetlb_folio(target_hstate, inner_folio, nid); free_huge_page(subpage); } diff --git a/mm/hugetlb_vmemmap.h b/mm/hugetlb_vmemmap.h index 25bd0e002431..d30aff8f3573 100644 --- a/mm/hugetlb_vmemmap.h +++ b/mm/hugetlb_vmemmap.h @@ -10,16 +10,16 @@ #define _LINUX_HUGETLB_VMEMMAP_H #include -#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP -int hugetlb_vmemmap_restore(const struct hstate *h, struct page *head); -void hugetlb_vmemmap_optimize(const struct hstate *h, struct page *head); - /* * Reserve one vmemmap page, all vmemmap addresses are mapped to it. See * Documentation/vm/vmemmap_dedup.rst. */ #define HUGETLB_VMEMMAP_RESERVE_SIZE PAGE_SIZE +#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP +int hugetlb_vmemmap_restore(const struct hstate *h, struct page *head); +void hugetlb_vmemmap_optimize(const struct hstate *h, struct page *head); + static inline unsigned int hugetlb_vmemmap_size(const struct hstate *h) { return pages_per_huge_page(h) * sizeof(struct page); diff --git a/mm/internal.h b/mm/internal.h index a7d9e980429a..31b3d45f4609 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1102,4 +1102,7 @@ struct vma_prepare { struct vm_area_struct *remove; struct vm_area_struct *remove2; }; + +void __meminit __init_single_page(struct page *page, unsigned long pfn, + unsigned long zone, int nid); #endif /* __MM_INTERNAL_H */ diff --git a/mm/mm_init.c b/mm/mm_init.c index a1963c3322af..3d4ab595ca7d 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -551,7 +551,7 @@ static void __init find_zone_movable_pfns_for_nodes(void) node_states[N_MEMORY] = saved_node_state; } -static void __meminit __init_single_page(struct page *page, unsigned long pfn, +void __meminit __init_single_page(struct page *page, unsigned long pfn, unsigned long zone, int nid) { mm_zero_struct_page(page);