From patchwork Fri May 12 14:57:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 13239440 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25216C77B7F for ; Fri, 12 May 2023 14:58:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B71AA6B0080; Fri, 12 May 2023 10:58:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B22D46B0081; Fri, 12 May 2023 10:58:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E9C06B0082; Fri, 12 May 2023 10:58:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 8F0D06B0080 for ; Fri, 12 May 2023 10:58:22 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E1DFF140F51 for ; Fri, 12 May 2023 14:58:21 +0000 (UTC) X-FDA: 80781908802.25.760A1C7 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf18.hostedemail.com (Postfix) with ESMTP id 1EA821C0019 for ; Fri, 12 May 2023 14:58:19 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=lFJE9ofA; spf=pass (imf18.hostedemail.com: domain of bjorn@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=bjorn@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1683903500; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aj6GQIswVFDjikg1aHibrD6lxBUr92Ih68mWiYBsEIA=; b=tHWyR8t3kd9I27XqBdhd7DgvgqlrrlLhWuQe2kQKu1dSs9H0jtUJxylOxSrTVLH8fDOBKJ sxOOHOMFWyCGe9NMNugc8OQ0APu/4H8S5Sv5/PJiCrE88YJYqkA7tgNymilfaqOoKrFcZB gxpyxVUPS3qd3L6raGyhI5BYkHSkEFk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1683903500; a=rsa-sha256; cv=none; b=bOr0TfSYNqgH6Vslbhy/YsnriQgd+6OfUKAsQfYJbizCnrers4tVuFSFWTqREnH8ZUK5N2 84Ct2Jbzxg/6AEdnhBJp7q6NV9sGJA50kDrGtR9Fg7eaopwHEnidWarkSjy/pdyonVzazD HKdN5mgX5fkI4nb//aF8IagjcosAEdM= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=lFJE9ofA; spf=pass (imf18.hostedemail.com: domain of bjorn@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=bjorn@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4735E655F5; Fri, 12 May 2023 14:58:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E9BF1C4339C; Fri, 12 May 2023 14:58:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1683903498; bh=lRwO3YfIMF9fmZSTE+/WO7BqWpiO2x+8QNF4La5hiEE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lFJE9ofAvx/W7GMthxymGs+b9O5oFO1BwdKIGcHaAN62YeWePDQ0N/sK71knTdSoF FVBIIqU4AmmF6IG79I2XtGg0JavzeJwFrHvwtyzSV76oVsE2HRceOMMnAX1c65IAP/ wzRAeUAfR4ibQ4s6pSnjOcUPEJrI0Uwkbd762zC/EYqhScoqAKaEuy84NEiMva3S8O cUjmxdLyvrJYLCFOB3H2n8gsVJcGT3aBKjaVsXZO/B0iymi2R+/sIoE+4yZ4U/slfd k0/FpzS+nNiLNCLb4BGq3Qd2jSem/r0rZlEnE2QEYIKn4OVGCS3P4sJXIb6FK2umT0 TTdE0MNOfw90w== From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-riscv@lists.infradead.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-kernel@vger.kernel.org, linux-mm@kvack.org, David Hildenbrand , Oscar Salvador , virtualization@lists.linux-foundation.org, linux@rivosinc.com, Alexandre Ghiti Subject: [PATCH 7/7] riscv: mm: Pre-allocate vmalloc PGD leaves Date: Fri, 12 May 2023 16:57:37 +0200 Message-Id: <20230512145737.985671-8-bjorn@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230512145737.985671-1-bjorn@kernel.org> References: <20230512145737.985671-1-bjorn@kernel.org> MIME-Version: 1.0 X-Stat-Signature: djrmidhdkr7nn5nfhrw5d9po7dxtewst X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1EA821C0019 X-Rspam-User: X-HE-Tag: 1683903499-295954 X-HE-Meta: U2FsdGVkX1+gZCZz5BSDjCuKE7sbV29cniQOFshCX4xhn5nQiQOBp84MwaIJRbWNtgcRhT6MFnv3JDLQv87aT4AZY5Kg+daHP45Ahvi8MuBpzmuUPG1OzLHCMrYpqH9jB06ili+fzlk95Qw9Rh3Dx/x/ngk0hSetH4+Xk4gNnQralzWuQmRTtP+hITtCHlf++K3PB5cd6fZzcYb0jeodM5HJBDktd5RySSVJRSPnG+4ttdcra038xDGlzedpwRojFEyEw5iwR7C+EaYuPU6BNzSSbdELshKCjTXh99Mpvotb216HyxGXcVzcYg2zrpt70K7S0yICxrzFAj84wLhidNkzV0G0OdJcY44YcO4g3UbXNDoF8PaTuuiqXrYB16uhcoSNFQem/kFyKv3Nncf/qq3mdT60iD07z/a2yHRGKu69mxo8VIj1MPAZxE4o3cfX1vBVlJeZYqmPavpla+7infL0/Faagz+5Cpr/GZ7QQ3JTyEUgsIG98UHi+XzSyOzBpPaPfMmyUN7s95HdKIBtTiGvW/OjwZ4ww+bnxZcMqt61rgwbcdZP7LOqju8Bml5iUnsiF7kRqwGBuf09TuQUiM2ykPzLbO4VEf+SKx1HI+AObfBHwWSrwrE00O3+ZUDcXp092x8/bY6xn2LZCoujXJusNthC/UICvvGhF96Sbi5ZtTokyUf7XLpH6Dq52/vJdvlofoeWOWIsaHvuRPqEv0ebGFgbDugfMhINuIAlTBh1iSDeHjBfIUSVn/gZZvgwu1ZlqCLGNw/yp95KgNbWiNj82d0aEQZ33MVGYJ9zc/z3b86YQrTzBofsNvOYdnIinU6LSACEFY0pwCjpY+BF0dqlVOhn+UJiL5s4NclFt4vsF/q+6xz6MsBqkE+md5PLF4ps3dB55uEmPTmvYWcCMunryzUdSnWKte7c5oL4eZ91+k2cHDPrUaFYdvx6s3MJrnmVyZRCLC8hQ1G+bA+ VVrvVm0i NRxTT0hsQz7G7rMcGl+KL06jBe8Zl+F4MXGkhAYJh7lIFuaEtZ60JyjX6emdtMSfjOXucVU20aQfLnRlyJ9k2gbUwRfpkGf7wOLpp7TJ+6upea2vpJj5VxG6oEQeSW34nQbSxPIu6/jvcxDfJj1cRQ4Hu4+wwGslq2hNUWawlquUzb+YE/SJqy2f4HbWESwBJWXVo/uIyLTiGj7hXsiEUgM+v2wn6+MZKrEr2UUO6ldPiPEip9WE0TqQ6dvKf3cBA+6IP/ERHqnNdKOouIGi7hd70ub5+i/9QW0+wKie+D4IHLiy9weMyHPKPRu6g4bhpDIWhXqjjsbPo0TWWv4oyCEFiRp4fM9AadI29uPzF7zrgMN8ydbqUP1DHA6I74qR/HWU1u5gcOCiHKIPCW9tqh4A6YqGT8ESNr6IC X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Björn Töpel Instead of relying on vmalloc_fault() to synchronize the page-tables, pre-allocate the PGD leaves of the vmalloc area. This is only enabled if memory hot/add is enabled by the build. Signed-off-by: Björn Töpel --- arch/riscv/mm/fault.c | 7 ++++++- arch/riscv/mm/init.c | 1 + 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c index 8685f85a7474..b61e279acd50 100644 --- a/arch/riscv/mm/fault.c +++ b/arch/riscv/mm/fault.c @@ -233,12 +233,17 @@ void handle_page_fault(struct pt_regs *regs) * Fault-in kernel-space virtual memory on-demand. * The 'reference' page table is init_mm.pgd. * + * For memory hotplug enabled systems, the PGD entries are + * pre-allocated, which avoids the need to synchronize + * pgd/fault-in. + * * NOTE! We MUST NOT take any locks for this case. We may * be in an interrupt or a critical region, and should * only copy the information from the master page table, * nothing more. */ - if (unlikely((addr >= VMALLOC_START) && (addr < VMALLOC_END))) { + if (unlikely(!IS_ENABLED(CONFIG_MEMORY_HOTPLUG) && + (addr >= VMALLOC_START) && (addr < VMALLOC_END))) { vmalloc_fault(regs, code, addr); return; } diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index a468708d1e1c..fd5a6d3fe182 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -236,6 +236,7 @@ static void __init preallocate_pgd_pages_range(unsigned long start, unsigned lon static void __init prepare_memory_hotplug(void) { #ifdef CONFIG_MEMORY_HOTPLUG + preallocate_pgd_pages_range(VMALLOC_START, VMALLOC_END, "vmalloc"); preallocate_pgd_pages_range(VMEMMAP_START, VMEMMAP_END, "vmemmap"); preallocate_pgd_pages_range(PAGE_OFFSET, PAGE_END, "direct map"); #endif