From patchwork Wed Dec 13 08:40:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13490547 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A6C92C4332F for ; Wed, 13 Dec 2023 08:41:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=7HJA62YIgww3Mn0QQHWSQvWGfIOWN0fxOZ0okvwYbSU=; b=xTNgd6tetGkbrOos5buT2pz8RD ENtcTG4hGgW4gr5MW2PcgQ/duOZo0GCIY3wj+PgQMMUB+hNqLm+m4QmPDAvVktnZ4stTtpnrd2cJI 7/+vdNmokC1WH/H7wlDCjo7A7Xy9fofrpkmt4v6cMXlgEq+R4pQWJurpSZYHqskJiyxB8SygsVJEP zsXF+1YiNbVWr5sCPwXyJerl7ZjMdYERxruHUO+vWKeEJUafD/qkrchMG7bgfD8loYVpsj5lf5uhC VAiLeuVpC9Bv3UQwi+ZMa8lvjC+IKYhSvrYH+E8gz4FWTj9Ud8sOL8K/2xBnHVQLS7cu1zL43PXCr 6OkIzH0Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rDKnM-00E3GM-2D; Wed, 13 Dec 2023 08:40:44 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rDKnJ-00E3Fi-1z for linux-arm-kernel@lists.infradead.org; Wed, 13 Dec 2023 08:40:42 +0000 Received: by mail-wr1-x449.google.com with SMTP id ffacd0b85a97d-33331e69698so5499543f8f.1 for ; Wed, 13 Dec 2023 00:40:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702456840; x=1703061640; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=GoyUtCF75ewAMKWXP+u8oG8Jk8yBlmCfkzmrWMQ4pXg=; b=ulPvmifj7kRJpNIepqcFo0f68YTYi2MYw7CCUErtrZn7Z21pMVKNW53hEJyIBHarfN yZ22ilPhVsRMZtb3ITvHG6/r4Ubox7e2AX2sdrpsAjm0SktOmIMb3/N9MybP8MrvKLjP n6CauLwSWzz+hPlrSleo5gFPHnBtqDAnvxxwFtcbHh4GgFqioOr8oF4bE3HMzXAYaZnb pZerMsVnUU/p9CHnJ309Bj1aL7cJKTlMfApEix1Mo2gIQUFrt8RJiIiZbCAxPNjWnRmn b2d9R/TANk+qKkIxjLo2MBmW76nmZSFDZsuPYrRQ3MfrnKDDWx0Vzpty1V/GKjQ0irHQ TZKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702456840; x=1703061640; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GoyUtCF75ewAMKWXP+u8oG8Jk8yBlmCfkzmrWMQ4pXg=; b=oAUVtFFmhs06i8EhZSh2qaxzzEorQmkFlnIKw+2tGkKHCklsLuFGK0f25RokSZsvAD 3PpL2vguKugunWAWCRFJIEJ6e5fBrMFPpdTYXV3obkRe3YUhc2O6PLEMSDcQrdvszEr6 w/AhrAcEOPoOXSpXL9f6mqEXvNuKeccZzX+Bi8urSE60xZ0sa9zwjJKyG7Msnf8U6eLw gO7ic6i7MpXmQQELSfh+ZT20DvAdwxMbNHPjHanbflVg1z67oAVhdYKpuE+rDRZv9iXs uqA2xtFCjVp+LAunVXeupLQHcdVeLUX5PLhRAKkR7MeTtH3obRnXnBOlnnRgiESXeHad OUjw== X-Gm-Message-State: AOJu0YzpSG1fz8S1NUYO3JEBxkMu/YFBXDmSaaCy6yQ5u4EPsRXIJNvp e0SJ/R8NCN943FNSB1TWoltfFdH+hKh8frsK6PFdh9Rf5/7x4XUCuFbT0NvoRyBvHO9qsxvjyP/ fzFRPn7H5gTfMmF4FTJQSlwfRxU3zvJ2bM/uvnJikZqvAU9JYwSzihHyoC5y1DTb6ABHBHKz9fr g= X-Google-Smtp-Source: AGHT+IEI4wBQwUVJv/h1vCMCJjPbZK8Lx3DuyOCWvjG6iWIa81ZKk58UDLgl77qBBsK0Zd/3cV5ce6Nr X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a5d:6741:0:b0:336:117a:4c9b with SMTP id l1-20020a5d6741000000b00336117a4c9bmr37237wrw.3.1702456839566; Wed, 13 Dec 2023 00:40:39 -0800 (PST) Date: Wed, 13 Dec 2023 09:40:26 +0100 In-Reply-To: <20231213084024.2367360-9-ardb@google.com> Mime-Version: 1.0 References: <20231213084024.2367360-9-ardb@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1960; i=ardb@kernel.org; h=from:subject; bh=XYlN9A4Hc2oFGV0tPBY2kCASm/vXrFv3ZkkR4QiL/FA=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIbUy95ec2fJgnu67Mh3sFjEt1e0i55crZi4ofJUnvfX2h Myz7fwdpSwMYhwMsmKKLAKz/77beXqiVK3zLFmYOaxMIEMYuDgFYCJZvxkZdnku3uLrIim6b1OZ 0nrbsCidbYbXLfTDj3lwvp6nknv2FsM/nReyfhbS8+M/eTi0/Mh70MN54mzz3xexdw/9kRLl6Tr HCgA= X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231213084024.2367360-10-ardb@google.com> Subject: [PATCH v7 1/7] arm64: mm: Move PCI I/O emulation region above the vmemmap region From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231213_004041_654972_CE5243FB X-CRM114-Status: GOOD ( 12.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel Move the PCI I/O region above the vmemmap region in the kernel's VA space. This will permit us to reclaim the lower part of the vmemmap region for vmalloc/vmap allocations when running a 52-bit VA capable build on a 48-bit VA capable system. Also, given that PCI_IO_START is derived from VMEMMAP_END, use that symbolic constant directly in ptdump rather than deriving it from VMEMMAP_START and VMEMMAP_SIZE, as those definitions will change in subsequent patches. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/memory.h | 4 ++-- arch/arm64/mm/ptdump.c | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index b8d726f951ae..99caeff78e1a 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -49,8 +49,8 @@ #define MODULES_VSIZE (SZ_2G) #define VMEMMAP_START (-(UL(1) << (VA_BITS - VMEMMAP_SHIFT))) #define VMEMMAP_END (VMEMMAP_START + VMEMMAP_SIZE) -#define PCI_IO_END (VMEMMAP_START - SZ_8M) -#define PCI_IO_START (PCI_IO_END - PCI_IO_SIZE) +#define PCI_IO_START (VMEMMAP_END + SZ_8M) +#define PCI_IO_END (PCI_IO_START + PCI_IO_SIZE) #define FIXADDR_TOP (VMEMMAP_START - SZ_32M) #if VA_BITS > 48 diff --git a/arch/arm64/mm/ptdump.c b/arch/arm64/mm/ptdump.c index e305b6593c4e..46acb2a24da0 100644 --- a/arch/arm64/mm/ptdump.c +++ b/arch/arm64/mm/ptdump.c @@ -47,10 +47,10 @@ static struct addr_marker address_markers[] = { { VMALLOC_END, "vmalloc() end" }, { FIXADDR_TOT_START, "Fixmap start" }, { FIXADDR_TOP, "Fixmap end" }, + { VMEMMAP_START, "vmemmap start" }, + { VMEMMAP_END, "vmemmap end" }, { PCI_IO_START, "PCI I/O start" }, { PCI_IO_END, "PCI I/O end" }, - { VMEMMAP_START, "vmemmap start" }, - { VMEMMAP_START + VMEMMAP_SIZE, "vmemmap end" }, { -1, NULL }, }; From patchwork Wed Dec 13 08:40:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13490546 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 390B3C4167B for ; Wed, 13 Dec 2023 08:41:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=shyN2ArQPnPC/ABWliBu9+y0sUQMFmJwWJ80oW8vyE8=; b=ImDUrgz5051Pgsl1k0vH6HhiGk ojLY81IUOSatTLRLaqGn+8Xz38y8OLhyQfJneCu2RzvC64yyQPCd1IjtA6K1RrpguEMPr5Rh0Q9OM +dLci1/L+ZVMPXPvPGIPud8x+DBG5rZOOefgYW6jrIwDgTYLPeq7q49zBBnTsWRowPoqEbwfgot5q vxDmZ6xSipX1Yns/F5tOgQlWtsrzDMtfsV2c5wWkoseenPP4vPgxiJLi8yohIzAhhUCk/3nwvW6RD f4dQ/WwqmXf0w+XI67JXJf3KJCPJ5E7bfNmnGEqk/cmCx3ke3w3gyVWSqAreDtl73y+ofLvZ1MCAD pR0Ne0PA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rDKnT-00E3Hx-2t; Wed, 13 Dec 2023 08:40:51 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rDKnO-00E3GJ-1g for linux-arm-kernel@lists.infradead.org; Wed, 13 Dec 2023 08:40:47 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-5e282ec7531so9063737b3.0 for ; Wed, 13 Dec 2023 00:40:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702456843; x=1703061643; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=vCl+HPeBrh+IA/A7jnlZ9z32gTluMzfFG/j8n/9wsTE=; b=t/beFxrlQf5n+jgzJYgxCuhU6Ni2ZhnFpdrGfScs2itx4GPz8rdLxVX/a+c2ivKQz8 2QaXLlKhUFAaNlZNkeu7l3epJ6brLOtkoTg46so/HZhMlFhtdRJdMpQdBBiSPGMU44TU dGJ/H/ZalU0GPbLU7nTe9OAVdTOAHiMMvKnl81e9OWOxpYKMyVAkkQfORMWG2m5ZpgDT ZymR3JlMMHoqh7YOTl9x//FhUKiv6LgzMemhjHx/tVYTRi4c9hGvYiJmWSd+luu9Ovt1 V/dmkU2jBQWF9gXN28E0EE87832RThgmjXp8b0XmXkhhrXRLSFPBjOjf9TOl6HWni//f 5twQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702456843; x=1703061643; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=vCl+HPeBrh+IA/A7jnlZ9z32gTluMzfFG/j8n/9wsTE=; b=GTnkCpZqs2Ff8/dSBfQlA3zLFa2w6vTLi42ZKJ0bgP50tOhTWGxfVFmSQHbjC/t2d1 s4JNAlvq8NXJmI+Y0kwWFysvsSXdEEE69bjJ6tRv0J6bHHenWkjT7FXhZzLQjbxuQ8OA zmuZWmaPRyBm1UUuMmRdrCdDksm6Egw5mU+H2F86xskerlPwch1bKEfraTYKwsvzBhEp T/5YPxhuXSGSpdobaTNEOBqMTuToqJiqIp5OJQ3qHhh3HmWjXzGCvczisId3AR28yQpv HIrz7k2rkgf9wDACUBeHlFlXvXNCjy+aAvLlRw1WJZWxG/cKLekXQEtu64YMyNt1f4+m r1fw== X-Gm-Message-State: AOJu0YwpgvLyTfApkSWLl8DVu133K7PvUJMTd7Pzucw//gPIyMMv98AO jORYNsXpuFoX+/FoCUc1/PCJM4aZrh128WSmripmJspTZUaOs0R6WSPaaiCaanHBx+IzBiTwhgV wfd4nBrNKrgBNYxAB/6zuG2+i3JsCU1LdlQCj6wbgWDV31qNWrGrNK0vxiO2ExJX6DcAZoSmtZ8 s= X-Google-Smtp-Source: AGHT+IEN7BoKO2CwOcJ+pKu4HTId8p/PS8Q/NpYJgbyFuC8MK3x3v8lPiPZy0icGTv7DPJubd1+yDySc X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:690c:d08:b0:5cd:c47d:d89a with SMTP id cn8-20020a05690c0d0800b005cdc47dd89amr70002ywb.2.1702456842562; Wed, 13 Dec 2023 00:40:42 -0800 (PST) Date: Wed, 13 Dec 2023 09:40:27 +0100 In-Reply-To: <20231213084024.2367360-9-ardb@google.com> Mime-Version: 1.0 References: <20231213084024.2367360-9-ardb@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3045; i=ardb@kernel.org; h=from:subject; bh=XkIz2zihTZXI27u4VK6G8hX9aM3QbLXFrzvr/rgD7eo=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIbUy93eQz8Sw5SFJGfNOvbh16SEH2233fsGS22ZXBOYet 88UadnSUcrCIMbBICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACZi/5ORYcF0i5XT79Q4+Obc m6J1jDthSka0ezsrd0PW4YQNB6TOaTAynHx0YEHGGQn2lMKE56r7/nIubNU+u7nwaWOs2VmLB9I /OQE= X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231213084024.2367360-11-ardb@google.com> Subject: [PATCH v7 2/7] arm64: mm: Move fixmap region above vmemmap region From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231213_004046_559394_7876AA99 X-CRM114-Status: GOOD ( 16.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel Move the fixmap region above the vmemmap region, so that the start of the vmemmap delineates the end of the region available for vmalloc and vmap allocations and the randomized placement of the kernel and modules. In a subsequent patch, we will take advantage of this to reclaim most of the vmemmap area when running a 52-bit VA capable build with 52-bit virtual addressing disabled at runtime. Note that the existing guard region of 256 MiB covers the fixmap and PCI I/O regions as well, so we can reduce it 8 MiB, which is what we use in other places too. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/memory.h | 2 +- arch/arm64/include/asm/pgtable.h | 2 +- arch/arm64/mm/fixmap.c | 3 +++ arch/arm64/mm/ptdump.c | 4 ++-- 4 files changed, 7 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 99caeff78e1a..2745bed8ae5b 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -51,7 +51,7 @@ #define VMEMMAP_END (VMEMMAP_START + VMEMMAP_SIZE) #define PCI_IO_START (VMEMMAP_END + SZ_8M) #define PCI_IO_END (PCI_IO_START + PCI_IO_SIZE) -#define FIXADDR_TOP (VMEMMAP_START - SZ_32M) +#define FIXADDR_TOP (-UL(SZ_8M)) #if VA_BITS > 48 #define VA_BITS_MIN (48) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index b19a8aee684c..8d30e2787b1f 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -22,7 +22,7 @@ * and fixed mappings */ #define VMALLOC_START (MODULES_END) -#define VMALLOC_END (VMEMMAP_START - SZ_256M) +#define VMALLOC_END (VMEMMAP_START - SZ_8M) #define vmemmap ((struct page *)VMEMMAP_START - (memstart_addr >> PAGE_SHIFT)) diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index c0a3301203bd..6fc17b2e1714 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -16,6 +16,9 @@ #include #include +/* ensure that the fixmap region does not grow down into the PCI I/O region */ +static_assert(FIXADDR_TOT_START > PCI_IO_END); + #define NR_BM_PTE_TABLES \ SPAN_NR_ENTRIES(FIXADDR_TOT_START, FIXADDR_TOP, PMD_SHIFT) #define NR_BM_PMD_TABLES \ diff --git a/arch/arm64/mm/ptdump.c b/arch/arm64/mm/ptdump.c index 46acb2a24da0..a929b5a321db 100644 --- a/arch/arm64/mm/ptdump.c +++ b/arch/arm64/mm/ptdump.c @@ -45,12 +45,12 @@ static struct addr_marker address_markers[] = { { MODULES_END, "Modules end" }, { VMALLOC_START, "vmalloc() area" }, { VMALLOC_END, "vmalloc() end" }, - { FIXADDR_TOT_START, "Fixmap start" }, - { FIXADDR_TOP, "Fixmap end" }, { VMEMMAP_START, "vmemmap start" }, { VMEMMAP_END, "vmemmap end" }, { PCI_IO_START, "PCI I/O start" }, { PCI_IO_END, "PCI I/O end" }, + { FIXADDR_TOT_START, "Fixmap start" }, + { FIXADDR_TOP, "Fixmap end" }, { -1, NULL }, }; From patchwork Wed Dec 13 08:40:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13490548 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2CBAC4167D for ; Wed, 13 Dec 2023 08:41:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=C2SKgiBy8Qesoia4gbvMKQPRjO/rJemW/XIizGdxBRU=; b=LJLG/ttXUdoC9Bwgi/Cgtg4rE5 B0DMAUIqsC81X1R0iQVr5ri8InLDA5EAGtADlrjkydNqtqvdzJ/zSW23EcRbV1mqDY/4aZpoKjO/W zhl8wx1NwoUR+ASkkhxRhh/wMeF2l22amB1H3Y9/H5mB0p9uS5xRA3IDBql2ksr2nbAjnvjwPFall OmNJGa+Sm+0i4fcFf9rQ8vS9/cC0WDFkrBw72MruFUkQhYGWqQqkyzPZyC7/7hPKP8Ho+BmrwJaJ5 R4YUQ4eCCvjqsm/X+pFSkvI9LdHIXy/+QYAUEtmW6gEJlv2xlgsTQsLfrz9bXHCb6v1SHYXgx5dwo GmkKuQ/w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rDKnU-00E3IF-1L; Wed, 13 Dec 2023 08:40:52 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rDKnO-00E3GV-1t for linux-arm-kernel@lists.infradead.org; Wed, 13 Dec 2023 08:40:48 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-5e19414f714so29311477b3.0 for ; Wed, 13 Dec 2023 00:40:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702456845; x=1703061645; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=WN6+IbHUwsDj2Mp0byBMbFLcPXljf01zSOzdPY+DU7c=; b=x01SnZEMrvW2oC2l5BcjTIljYakfa/T1h+FzhgyB03IxFlkZUiHTW8sLZaoCOf+m4h WEdCoMntb8oMKUKxl1CSiSDiW0EUApkBhN5JAhGDOEDWfEQEmURtaROqcI4TXv0mSret f8j+wHKq/TXo4v/JaPooIlLFKijoP3qQrdPPNW/KEEGta277jPwFjtPsK9fZALS762dd Xdf/FLvEc4z9+Lnv1SgeeUP2tIYVky6K37Ori0gkf5pnnLi19K9RkSqCIHm7bR05jhJG Jije8cYKcNEi/N3OLfLDib1ULTVFKveHFEy/25j+RrW25uZQIGjGxZZHGniqiJ1tq6gT lFwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702456845; x=1703061645; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WN6+IbHUwsDj2Mp0byBMbFLcPXljf01zSOzdPY+DU7c=; b=Qtp7HOV5sIIBCrkTDxt3I50GeeY6F1Y55rWaYYTTl3o40iwhlpdSWf43eMyDww0b2L WF8ckAsFIqN/yhmwoGR4lZGmaNWA9qCVcD6PQeuXbRQYwFHMOy53N2SGbnYFzUMLOvbz ppxodfZfxAVjkGS2wj2UW5esar8jyu/bvcVdW2XbGd+2v+fxSirlaY4zVvmUX2rYYlK1 Qor9/l2Z+Z2j0ijT7rpeuKZ3YKV46ZMwcR0m0qOKilNj2dq6bMGob58LfoWpqB9n6rcx MEisuAqV+2zKDtHTbbfAnlmPWIUY9ZlFxUr/5QuVEINPrM+klbdvp/vuX6KyEcFPRBA5 IOTA== X-Gm-Message-State: AOJu0YyDUDO2R6LKCT3pIqF0WcSgLaXP2cALtiuYZFhQar/RTMkaaY7y tgEmm14gYDhe6cmdOmoGa/6UoencWOss+cmnZl6h/BFYWSS9UjSfSxRYaNzN0zYdwm19XDITFjZ +KQJeARYRCKDLeH4AMUTOYYl/0oCEDcltAvwbTDkIrqezjC1utGO9DTXEyx5AHF2b2vzPKZg9Pd 0= X-Google-Smtp-Source: AGHT+IFpMEe1QleOIzYlmYbSHqhNIh05XJ6XpgT4sJ+3DPfuCuPFnd7Pcxt1uJoIQCCBm5n+ozuTWoGl X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a81:fe04:0:b0:5d9:712e:202b with SMTP id j4-20020a81fe04000000b005d9712e202bmr63020ywn.6.1702456844985; Wed, 13 Dec 2023 00:40:44 -0800 (PST) Date: Wed, 13 Dec 2023 09:40:28 +0100 In-Reply-To: <20231213084024.2367360-9-ardb@google.com> Mime-Version: 1.0 References: <20231213084024.2367360-9-ardb@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3343; i=ardb@kernel.org; h=from:subject; bh=5iAh2YabWlLS7lJYeNG7SghkjjhrswiK2tPdvB6wvio=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIbUy9/dcc+abs9V+nN7jr7lgQl7Tl12bW3fekbyiwb3f6 u60ep2kjlIWBjEOBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjCR/6cYGVbOsGdnWfX87Ysj y0Msitvlu76d/6twSkrscXBieGLW6iiGf0p+9y56FPdxdHCenRUeOWeS25RVn/UtmfuOdl94GLa 7hhsA X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231213084024.2367360-12-ardb@google.com> Subject: [PATCH v7 3/7] arm64: ptdump: Allow all region boundaries to be defined at boot time From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231213_004046_624382_DA817C15 X-CRM114-Status: GOOD ( 13.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel Rework the way the address_markers array is populated so that we can tolerate values that are not compile time constants generally, rather than keeping track manually of the array indexes in question, and poking new values into them manually. This will be needed for VMALLOC_END, which will cease to be a compile time constant after a subsequent patch. Acked-by: Mark Rutland Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/ptdump.c | 54 ++++++++------------ 1 file changed, 22 insertions(+), 32 deletions(-) diff --git a/arch/arm64/mm/ptdump.c b/arch/arm64/mm/ptdump.c index a929b5a321db..66ccb8d6997e 100644 --- a/arch/arm64/mm/ptdump.c +++ b/arch/arm64/mm/ptdump.c @@ -26,34 +26,6 @@ #include -enum address_markers_idx { - PAGE_OFFSET_NR = 0, - PAGE_END_NR, -#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) - KASAN_START_NR, -#endif -}; - -static struct addr_marker address_markers[] = { - { PAGE_OFFSET, "Linear Mapping start" }, - { 0 /* PAGE_END */, "Linear Mapping end" }, -#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) - { 0 /* KASAN_SHADOW_START */, "Kasan shadow start" }, - { KASAN_SHADOW_END, "Kasan shadow end" }, -#endif - { MODULES_VADDR, "Modules start" }, - { MODULES_END, "Modules end" }, - { VMALLOC_START, "vmalloc() area" }, - { VMALLOC_END, "vmalloc() end" }, - { VMEMMAP_START, "vmemmap start" }, - { VMEMMAP_END, "vmemmap end" }, - { PCI_IO_START, "PCI I/O start" }, - { PCI_IO_END, "PCI I/O end" }, - { FIXADDR_TOT_START, "Fixmap start" }, - { FIXADDR_TOP, "Fixmap end" }, - { -1, NULL }, -}; - #define pt_dump_seq_printf(m, fmt, args...) \ ({ \ if (m) \ @@ -339,9 +311,8 @@ static void __init ptdump_initialize(void) pg_level[i].mask |= pg_level[i].bits[j].mask; } -static struct ptdump_info kernel_ptdump_info = { +static struct ptdump_info kernel_ptdump_info __ro_after_init = { .mm = &init_mm, - .markers = address_markers, .base_addr = PAGE_OFFSET, }; @@ -375,10 +346,29 @@ void ptdump_check_wx(void) static int __init ptdump_init(void) { - address_markers[PAGE_END_NR].start_address = PAGE_END; + struct addr_marker m[] = { + { PAGE_OFFSET, "Linear Mapping start" }, + { PAGE_END, "Linear Mapping end" }, #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) - address_markers[KASAN_START_NR].start_address = KASAN_SHADOW_START; + { KASAN_SHADOW_START, "Kasan shadow start" }, + { KASAN_SHADOW_END, "Kasan shadow end" }, #endif + { MODULES_VADDR, "Modules start" }, + { MODULES_END, "Modules end" }, + { VMALLOC_START, "vmalloc() area" }, + { VMALLOC_END, "vmalloc() end" }, + { VMEMMAP_START, "vmemmap start" }, + { VMEMMAP_END, "vmemmap end" }, + { PCI_IO_START, "PCI I/O start" }, + { PCI_IO_END, "PCI I/O end" }, + { FIXADDR_TOT_START, "Fixmap start" }, + { FIXADDR_TOP, "Fixmap end" }, + { -1, NULL }, + }; + static struct addr_marker address_markers[ARRAY_SIZE(m)] __ro_after_init; + + kernel_ptdump_info.markers = memcpy(address_markers, m, sizeof(m)); + ptdump_initialize(); ptdump_debugfs_register(&kernel_ptdump_info, "kernel_page_tables"); return 0; From patchwork Wed Dec 13 08:40:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13490549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1C809C4332F for ; Wed, 13 Dec 2023 08:41:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=c7xnywwQ1BcC+ihcC975KCXsuVyuavbhIkHY/WpQfNc=; b=piikfrpd+EvfIS3qsO8f7WhrA6 Ue7o42R2e9RVJKjQD4l4Zd3BOqqMwCitBSjK/+6PAms2jpjcb7N96b+PkzBNtfrtjeP23M+d8tOFX ZxLuAZg86cWCaJAytvPXQ8BrXVlj/Yzx7W7cf7ekcY7Nz7Ri597MBH278Ral5fMpBuXCDfvop2PVC 3VwyRPC8Am2D3/4QWzRQiVetUIIwc5DGmZZ7f5u3HuoOMO1yXd+Cciy9FThjYTqssvUKrSfhsgQRp VN11KND0QRYXEeLua31Sowk95kuc/vD/aBWR5J/Jjm/qmlyudUW/jG6XJxeB4Dd/PZxj3zgx9y5Fj lHdiqXlQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rDKnd-00E3LZ-32; Wed, 13 Dec 2023 08:41:01 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rDKnT-00E3HV-1g for linux-arm-kernel@lists.infradead.org; Wed, 13 Dec 2023 08:40:52 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-5caf86963ecso75155567b3.3 for ; Wed, 13 Dec 2023 00:40:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702456848; x=1703061648; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=rJ0nJzPo2EGbXOK934MZ4luWn83Mpf1GIL6qGBAWogI=; b=NoM4g65Gogd49Mua3SZgpdxBfbg93ln5uqTk/s7vEBBsx31jz26xguaETzKc1TuVAn OGK7E8dUhX8pfiYkgpEvdFxpMmrUGiXI5AE2d7Y1TZ/sSkhkNZj863S0XTj5AvWxyRPV KmCnqj+VmxPShWBeQFV3CrW3MgllAN7/g/etqstFPY6ykmfUceyOXRFaGy14FgkF5lZt eu82kFZWG/XuMdpdEbCih4GI2t/9cyG4du6DsKZy/ANM2l6bo16W37i2j/rUCvsEc2OR JAM0f1sUdcua1uqMfv2Pq+UkErOTdS14BrWBY+ditKh7bJys1oG/EAnjIplVuI/tEnbH pAgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702456848; x=1703061648; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rJ0nJzPo2EGbXOK934MZ4luWn83Mpf1GIL6qGBAWogI=; b=sVfz0BtjOsaNVQTt9pxwm8hDBAO9auVSOG/eaqjn4nu4+1M/4XgKEpk+HoapkkjQGj vhClWUZSg0eROl8vbeZdORWf0lPFOwJTDiGSsyWh8/kpUiOTgPx4uNQhcoyWeLyqpz6R o4UeyioFedeInxmKj5I5Mv8tQmOCH02Y+IpmJZxgP1ACglXzYe7adTP3rTTEKp725KLm /SM1qbveHttUPb//xzaQuFHICP1smCuScHAQ6U2lsjeFtt2MDSblcVfwZPvOxVTCwXqX DNMOGEJwr5uslC+OIUkDD8NtSOZg1ZpG0f95nhSM4OFqF2hKlXxkryoig+fFKkiu+gl1 k9ew== X-Gm-Message-State: AOJu0Yxp/o9LEsJeSw8t9ykDQ7yQ2zM2kMknVWhjcnQaUqCvjR8zZL41 iGvLS3WqyAQyPtYUQ8oTBIh5KgdBTuGml9+rczcP0yWc4WCZJCf22fgnVkMShS7E8L4co3kJlZW bJ3BFg78jTRkVnHMzkNnuNPIdbPEgp6Wl1RuhdjsPa8wbFpA784VZIE9Nk8tmPY1+ncXDdSJSa/ 4= X-Google-Smtp-Source: AGHT+IHhco7XOP+WjT4LMER8iT0G0W1pZxRZPdmeqqE16/VqR6EZUSTJJE16VcRfatmRb/J6RzOWfEm9 X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a25:7489:0:b0:db7:dce9:76d3 with SMTP id p131-20020a257489000000b00db7dce976d3mr53547ybc.9.1702456847625; Wed, 13 Dec 2023 00:40:47 -0800 (PST) Date: Wed, 13 Dec 2023 09:40:29 +0100 In-Reply-To: <20231213084024.2367360-9-ardb@google.com> Mime-Version: 1.0 References: <20231213084024.2367360-9-ardb@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1366; i=ardb@kernel.org; h=from:subject; bh=PXk5B5kMCgj0oJs3f8j+ajJ+kqxIqxIVvOIDnmygz4M=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIbUy988CNbOczS4vFqw8UeLAsjtrmr5r5ZsdF+YZdvD9m OJQ3u/YUcrCIMbBICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACZy4z/D/9j/X017SieIxDnw XYyu3vzXUnkRx5Ejd6+2RezIOcCytIbhD++jd8Yucy8lOr3jsDUR5CveqqhT/aFIqMz7qW5w57x p7AA= X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231213084024.2367360-13-ardb@google.com> Subject: [PATCH v7 4/7] arm64: ptdump: Discover start of vmemmap region at runtime From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231213_004051_558048_7479871C X-CRM114-Status: GOOD ( 13.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel We will soon reclaim the part of the vmemmap region that covers VA space that is not addressable by the hardware. To avoid confusion, ensure that the 'vmemmap start' marker points at the start of the region that is actually being used for the struct page array, rather than the start of the region we set aside for it at build time. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/ptdump.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/arm64/mm/ptdump.c b/arch/arm64/mm/ptdump.c index 66ccb8d6997e..5f0849528ccf 100644 --- a/arch/arm64/mm/ptdump.c +++ b/arch/arm64/mm/ptdump.c @@ -346,6 +346,8 @@ void ptdump_check_wx(void) static int __init ptdump_init(void) { + u64 page_offset = _PAGE_OFFSET(vabits_actual); + u64 vmemmap_start = (u64)virt_to_page((void *)page_offset); struct addr_marker m[] = { { PAGE_OFFSET, "Linear Mapping start" }, { PAGE_END, "Linear Mapping end" }, @@ -357,7 +359,7 @@ static int __init ptdump_init(void) { MODULES_END, "Modules end" }, { VMALLOC_START, "vmalloc() area" }, { VMALLOC_END, "vmalloc() end" }, - { VMEMMAP_START, "vmemmap start" }, + { vmemmap_start, "vmemmap start" }, { VMEMMAP_END, "vmemmap end" }, { PCI_IO_START, "PCI I/O start" }, { PCI_IO_END, "PCI I/O end" }, From patchwork Wed Dec 13 08:40:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13490550 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 79A72C4332F for ; Wed, 13 Dec 2023 08:41:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=rIKqReHc+/WEQNbryCmp44jBOWzAZKUW9eBiFRTU11k=; b=wGL7mRIi+3gm/0Zjj3m1lYx6Ge YNtNKbawQ4hSoxbglgsLYKpImx7REor29tfB8qp6doMi6jOd8VWukH+O3rgAPvw8qh+oc+9G1f7bT bj2cwC26bRUPsEjRgKocqwe+Xkel2NEJ389oZtnKbzMEBg/HAWX5LXvAIB++eTR56YJbhWq/i802T ICMTxr4bvwX0lDBjUNpTu5py7+HS34l9E3yiRg0QKqoxp4YBSnSsIj850WlgWFzYJaaVRU57gChiK N+fTIZMlIVyhn+DkdJU2KNiUaFkMs9CUjd0qg0jintEaD6IsM8xdv6vB5gjppa1gepvomh/YS2Z2a mUGIiD5A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rDKnf-00E3MK-0D; Wed, 13 Dec 2023 08:41:03 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rDKnW-00E3Hh-1w for linux-arm-kernel@lists.infradead.org; Wed, 13 Dec 2023 08:40:55 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-dbc68661060so4276492276.2 for ; Wed, 13 Dec 2023 00:40:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702456851; x=1703061651; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=oycniDVE52sU1JSV8+a0iSsIio+ypPfsg2M+Cjxiv+A=; b=YerarOCSviELmH0WfXTh9yp8LjsbpzZjFkEKXVf6yRX5nAmL2umpUbomQ58WgBFDUB qwZVTObmVFt3Fyxmpk7IBegKtjhxQJmxNShk0KH0O5hIpEvvSLOYYUClUxixNpIfx73/ dJc5AKkRRpgn6fu5y2zvjLZRjuE0L3c8+LXA3KBj2RghWmVAtVeJJ+NYZ14jIg6vUbP+ 1W447SWGCdF3AIg2HuJgmcZVwZuTQh/UQUR6BljFNas1/xbWcCUMbAcjOK9/IZy9fhdK TuLERSpyl771oNqJPFXIq2JJqUTeMODo7cotfzYm21PvyfXLb5H40lG9JM6C3k68GPkc CJPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702456851; x=1703061651; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=oycniDVE52sU1JSV8+a0iSsIio+ypPfsg2M+Cjxiv+A=; b=btl5X+sH17Vr4SXfKq9x+RBAqyk6hU9/PCXWZqcEwGur4IqoXHdTwtw/Gbd201j7MC MXQ05z3OL0JJZC3TLltxAqYepNLchsQdDIfEt+yYDFPJC8dkYJiYoq1YzugxN4dtbOOk p8pO8C531DpuAaKJ6J2cBhYQvFfqWg84lic1cHqK92R1Vn2SuYz4q9STWcDj7R4LUf2o WjZEYySEKLrLLxlM/WrRf9A2s2/9+Ei6iqIFHEQo1E02rChAY6UK5m4JqdxdJry3R2OV 8FiAHmuZH6i/eWrO8ggJBJqRHaJL0JomjFILI9JCKDURpbUugOo5Jmm5raxZkpithJyV bQRQ== X-Gm-Message-State: AOJu0YwMMfiJWZMwKzJYxLDOgsYAd/zyHyHK8sBj0a5KYcpDoYzbYHB0 BmVvlaEVcYKBplHb9JWN2jF5pmHYGZv8gEAAuzE1OlX3NHjm+94EcWjkatD+/r6oS/bCbuh2D+B krt+HlQp/EWG9OLvMgBMvdyt38CmNiiDGlfq8A+Bv3zw0sUdf8QmJfi4Jq+7shi8XGd6dsEMNbb g= X-Google-Smtp-Source: AGHT+IEcpSKZJseD72Iy4bWd61HCfzNtSMViwyxlgovX8MSoDrPzKOaYSJm46tGd9qNzzZ+03TXE3B0s X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a25:aca7:0:b0:dbc:b717:5e39 with SMTP id x39-20020a25aca7000000b00dbcb7175e39mr20671ybi.13.1702456850302; Wed, 13 Dec 2023 00:40:50 -0800 (PST) Date: Wed, 13 Dec 2023 09:40:30 +0100 In-Reply-To: <20231213084024.2367360-9-ardb@google.com> Mime-Version: 1.0 References: <20231213084024.2367360-9-ardb@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2503; i=ardb@kernel.org; h=from:subject; bh=+I5alsz6DqjYnaptAOO6mtqNRIftFAO6fBvjLV7D6dg=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIbUy96/NQYkUlZSrqn/mbhI0FhU4NrmYWSTyStYT43rLH 2ohv4s6SlkYxDgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwESuf2BkOKUe6pSzsu/nRI0f 3bu2cT5J+FG/+ULiNeusE5Gpj9fonmJkmNnO/az1yL7bUhwXrUQSyqz1tCcf3LS4/nWU+8m1Qiu TmQA= X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231213084024.2367360-14-ardb@google.com> Subject: [PATCH v7 5/7] arm64: vmemmap: Avoid base2 order of struct page size to dimension region From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231213_004054_664990_4D946797 X-CRM114-Status: GOOD ( 16.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel The placement and size of the vmemmap region in the kernel virtual address space is currently derived from the base2 order of the size of a struct page. This makes for nicely aligned constants with lots of leading 0xf and trailing 0x0 digits, but given that the actual struct pages are indexed as an ordinary array, this resulting region is severely overdimensioned when the size of a struct page is just over a power of 2. This doesn't matter today, but once we enable 52-bit virtual addressing for 4k pages configurations, the vmemmap region may take up almost half of the upper VA region with the current struct page upper bound at 64 bytes. And once we enable KMSAN or other features that push the size of a struct page over 64 bytes, we will run out of VMALLOC space entirely. So instead, let's derive the region size from the actual size of a struct page, and place the entire region 1 GB from the top of the VA space, where it still doesn't share any lower level translation table entries with the fixmap. Acked-by: Mark Rutland Signed-off-by: Ard Biesheuvel Acked-by: Mark Rutland --- arch/arm64/include/asm/memory.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 2745bed8ae5b..b49575a92afc 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -30,8 +30,8 @@ * keep a constant PAGE_OFFSET and "fallback" to using the higher end * of the VMEMMAP where 52-bit support is not available in hardware. */ -#define VMEMMAP_SHIFT (PAGE_SHIFT - STRUCT_PAGE_MAX_SHIFT) -#define VMEMMAP_SIZE ((_PAGE_END(VA_BITS_MIN) - PAGE_OFFSET) >> VMEMMAP_SHIFT) +#define VMEMMAP_RANGE (_PAGE_END(VA_BITS_MIN) - PAGE_OFFSET) +#define VMEMMAP_SIZE ((VMEMMAP_RANGE >> PAGE_SHIFT) * sizeof(struct page)) /* * PAGE_OFFSET - the virtual address of the start of the linear map, at the @@ -47,8 +47,8 @@ #define MODULES_END (MODULES_VADDR + MODULES_VSIZE) #define MODULES_VADDR (_PAGE_END(VA_BITS_MIN)) #define MODULES_VSIZE (SZ_2G) -#define VMEMMAP_START (-(UL(1) << (VA_BITS - VMEMMAP_SHIFT))) -#define VMEMMAP_END (VMEMMAP_START + VMEMMAP_SIZE) +#define VMEMMAP_START (VMEMMAP_END - VMEMMAP_SIZE) +#define VMEMMAP_END (-UL(SZ_1G)) #define PCI_IO_START (VMEMMAP_END + SZ_8M) #define PCI_IO_END (PCI_IO_START + PCI_IO_SIZE) #define FIXADDR_TOP (-UL(SZ_8M)) From patchwork Wed Dec 13 08:40:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13490552 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D2BA6C4332F for ; Wed, 13 Dec 2023 08:41:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=SXcLtZ0OgTxj8B8lebxRnyaF8Pi1q7NayBF+qRpeNzo=; b=3NHbhf+0zZ8KV0JMZWc7vQakVy G6IeDcCEu5SlB8mYBY77dCKVlu4j2ykMqjNNP3G6XS/okO8KU0vj4sYFMfHcG1YwHwJBySoikZwh2 ostE+pzNliXfzfIvhGL2xhdwW0QtBDyzNuqOtaAwlqoRDh7SNdR/yM3xtu0ZoYMdufIyr8/mEa9Kx RO0h6hIYpFPyG6DNeKOnttswAIPEGxRTBS9kQzxGJ1ybMPwGoyryubFbWi7UEAVoF0Rofgh4dAiys LE0D3n1Hp8Thl5uZoyPU21orfEU9U+i/9umtcFIkNzwLFnqM8H4GuNvfZOz4UmfmLZb3Y3w20sMME QOYkeumw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rDKne-00E3Lu-1Z; Wed, 13 Dec 2023 08:41:02 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rDKnW-00E3In-11 for linux-arm-kernel@lists.infradead.org; Wed, 13 Dec 2023 08:40:55 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-5d12853cb89so75288727b3.3 for ; Wed, 13 Dec 2023 00:40:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702456853; x=1703061653; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=SyyAeMosOrBSkbl1SQsOUBScLakw7in0tEhwFQt6I28=; b=OQ95MfTEzq2+fbMldl1BbIEdw0F6cmV1XXtUdk+umJp1idJ7MoHbVOSjGgYiWkwgHW q8ibcL2rUVvKx1i5ZtUe4Qlr5dCQXoPUAfkS52t78WHM8VVC0stVyTq/ueAwJjw9vV0N kH/ijYYqezuuz6S5oL4Bs9qm5QR0N46Nw/tZZV4PycD5GaFg9pTsLc9cgTuVWjZ+WYSO k3fv6P+7jgHt5VB5ZoclRXE5Aad9BINWlVbYDYprhsEzUNEzQnorqY1iVQWLLkHEn+n2 2dJ6Z4ssbAWQS109tvLCTvUeMwtXtsKGsNJBqp85SNEyB8YNDmQW9Qk0VVMZFMRXHtMV ZBtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702456853; x=1703061653; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=SyyAeMosOrBSkbl1SQsOUBScLakw7in0tEhwFQt6I28=; b=nbcrnq0cL4OqzMnvMSBsczLJDXQKE/BD+m2vRRbkkjyWkgGanTocsF722vhFqfJC04 wTZMY0nSYIXczvCFa71X76rP0M8MoYTnrTfyr8muYY4cWYfJBvK3FEqs/spdiOxycPHN Cqjh9IS7k+ia2oj/j7XSRhtCgjDlRskQWyNF/yvJqHXY2oO0F2UDtlWntW1QfY9yYZqx 6w/rDrwGnUFlTdGhgKbb20OdS+Hu4deqRbiESrd21tCTHpvZoXhN+9MVa0BOylXqLenN qC54S2lRxItiOsVXUqXW4UNGbx9ae8eo3T+rZHGu4VHVVhgId3ZM5ov6vL1ujTkTcbk/ iKGA== X-Gm-Message-State: AOJu0Yyv4BAtpb+OpzI/vaQgHb+D7V43xbi7KfVAb9iMZrG7YcdHRpdo K3crafabmCgtvtJgD29f1Ow6FJ/FCgUFSjRektZvMjKqxkiP/p9EuzpIiZBUSNXFgjsTIAuPoyw jWcJ8n5Q0bM4RaKai6mkoipaQVl+GNhwcsQs/cZjRhHNE4An3u65PSQEaagA8xvjHTFPV7yWV/+ E= X-Google-Smtp-Source: AGHT+IH1nZ91uvYxypIv/QE1OKKsPyeXRwla1LpNu60BTzPWP12TOJtGt8foTnbQIjE6e0FAXR6sFRWm X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:6902:4d0:b0:dbc:b226:bd23 with SMTP id v16-20020a05690204d000b00dbcb226bd23mr27602ybs.0.1702456852591; Wed, 13 Dec 2023 00:40:52 -0800 (PST) Date: Wed, 13 Dec 2023 09:40:31 +0100 In-Reply-To: <20231213084024.2367360-9-ardb@google.com> Mime-Version: 1.0 References: <20231213084024.2367360-9-ardb@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1790; i=ardb@kernel.org; h=from:subject; bh=EprKnS4h+kj2zZNqC12tc0oCTHrUD5uJVzU1RyQ/vR4=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIbUy99+ev10cMxVW1X6YwG+s5TKnr/4F14vts86rL0+5b iX082FlRykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZjI2k+MDHdljKorPscttZ4q w2F+8MJFlfh74SqX2Tjz3m7k4ur2c2f47/hOIz/TSt9gz93Vau8yt21/+vNQ3oEn9881aH7Wn3J AiQkA X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231213084024.2367360-15-ardb@google.com> Subject: [PATCH v7 6/7] arm64: mm: Reclaim unused vmemmap region for vmalloc use From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231213_004054_356711_3B840894 X-CRM114-Status: GOOD ( 14.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel The vmemmap array is statically sized based on the maximum supported size of the virtual address space, but it is located inside the upper VA region, which is statically sized based on the *minimum* supported size of the VA space. This doesn't matter much when using 64k pages, which is the only configuration that currently supports 52-bit virtual addressing. However, upcoming LPA2 support will change this picture somewhat, as in that case, the vmemmap array will take up more than 25% of the upper VA region when using 4k pages. Given that most of this space is never used when running on a system that does not support 52-bit virtual addressing, let's reclaim the unused vmemmap area in that case. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/pgtable.h | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 8d30e2787b1f..5b5bedfa320e 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -18,11 +18,15 @@ * VMALLOC range. * * VMALLOC_START: beginning of the kernel vmalloc space - * VMALLOC_END: extends to the available space below vmemmap, PCI I/O space - * and fixed mappings + * VMALLOC_END: extends to the available space below vmemmap */ #define VMALLOC_START (MODULES_END) +#if VA_BITS == VA_BITS_MIN #define VMALLOC_END (VMEMMAP_START - SZ_8M) +#else +#define VMEMMAP_UNUSED_NPAGES ((_PAGE_OFFSET(vabits_actual) - PAGE_OFFSET) >> PAGE_SHIFT) +#define VMALLOC_END (VMEMMAP_START + VMEMMAP_UNUSED_NPAGES * sizeof(struct page) - SZ_8M) +#endif #define vmemmap ((struct page *)VMEMMAP_START - (memstart_addr >> PAGE_SHIFT)) From patchwork Wed Dec 13 08:40:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13490551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 40C55C4167D for ; Wed, 13 Dec 2023 08:41:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=wGBe6E9cIqSA1COGSgoGTBL0oKQoTYvGfL+d69kdGaM=; b=it5XB9z4AGphU8e/OBPy5URhnB 4wKVLIJeLu0y3NwVsbwGA3OnpQ4HmS+D3eVNdgnmGoaziJx8rNiV3syJjyGYbiGObMZGVnpGS6BuH xcRgLsuzEyM2mdId0zWydwSvQNw3JqStDdJQLpjUeKK23j9eq1i3uy/Hz66oQlALicRpKIpHtKu8p bhvsuoY3bhuZaw7VzGW8rTyTD/HNTG14v91IYa6bv60X+upYSjwP4clg+t9nt9MWWaI9ClfVNYgbm QVAzd+zV6f7o+2fw7Kb7AZhdSNCqyeEodjUuO/eeeBA5mXDalhfNnRMA+v6Z8/iKT3OnN/ePWuD6d qV/VjjVQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rDKnf-00E3Mk-21; Wed, 13 Dec 2023 08:41:03 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rDKnY-00E3Ju-2P for linux-arm-kernel@lists.infradead.org; Wed, 13 Dec 2023 08:40:58 +0000 Received: by mail-wr1-x449.google.com with SMTP id ffacd0b85a97d-333501e22caso5287305f8f.2 for ; Wed, 13 Dec 2023 00:40:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702456855; x=1703061655; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=BX+wtcx32mlMaYxBO5qCEKCbHJOmCWQWGX70DB2Gh3c=; b=USiYLW2jI+PArGrCi4uN52PCQ/yWWR/ctpFqeV5jK/jUpVcOj61dMZWRc8nIiTXlTV /AXlSaUnB3A9/fu+GbcAU2SRnbLeSA8b16h0ih9lFrYzcqOZmcaZsTagCU9SKP1qCdTg jxTvBOM/Xk2CgHo5RQLa8mV2TPQC9TKeNOFFH73P3+awYiEBbx7HV6WULFw29zIanI8p n9OQd93OIHQ64BuRX/5zh/nGHCsVfWhc8J3QVAb3p9Kf5KV0egjwAfEy59bmJUZ34Xwm ZPMNiGvSfSVX3yg4vS0tLaDPKnM8rl99U2yIj/pl7N7OtEUEuH5MCvv6w2wRi7ppNgH0 vGWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702456855; x=1703061655; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=BX+wtcx32mlMaYxBO5qCEKCbHJOmCWQWGX70DB2Gh3c=; b=YMVr9hlJ6Tl6EXVPNLmpAPbZzdZT1t9IxGHgpL1kyIHMJwW8RdIaS35BYN3HVB3MAG c0jE0YqjCpvHnWaWcmF51NVgw1U0nLn/aKwMdY8rynNmg9qKbSNZeLkTm3K55dTcHChO 9JKFN+erb9DAAoLnyKxIXE3M7jrmT8Pj/y8YEGPK56B06stM0os1diP7Seg9Qh1Ef3B2 TRrc8fKCdUEHSkxemzYLelDrQuFoUvp+YCFQauTY9bI0E4qHtCwNKBl45XgBl9yzF6u7 zS1FLFqpvrxMpfZp3uFmLJvIIO6bHjnaDMVaAkau4FyPQxbC7I5Biu78/12ovNhXWV0A VlkQ== X-Gm-Message-State: AOJu0Yzwx9C/zJYwJ1qthlczTG7B3/T2hY+muDj1BaQaZ0LTBYM7RySd RqTTiH+a+KqpR9jio/lW32V9qN3QOxOsMN3Gdfu7qbvvOGV7CUSIz2LKvSzVETbZ8wqh2Vg30Tn xfRut9+JctmZZ0iUKj4kt0HpecX7VqRlRf7KaiOhK3e37RcaRTFu8WErn/nKXE8SF5suDnhEvfW 8= X-Google-Smtp-Source: AGHT+IGR0z+EnBVucZm+4vYqP5kGE6FZswCqbjTP9a187glIu6MkPzFznNvTt/xaSdr1N7wKWKiln7rY X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a5d:6daf:0:b0:333:543a:9f9b with SMTP id u15-20020a5d6daf000000b00333543a9f9bmr38509wrs.11.1702456854719; Wed, 13 Dec 2023 00:40:54 -0800 (PST) Date: Wed, 13 Dec 2023 09:40:32 +0100 In-Reply-To: <20231213084024.2367360-9-ardb@google.com> Mime-Version: 1.0 References: <20231213084024.2367360-9-ardb@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2763; i=ardb@kernel.org; h=from:subject; bh=XJvCPbUowF3LdiBGPhgFAl7bd7Pk9sOwkOq/ytxs+3M=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIbUy9/9CBTFmhQn/t7efybXiksnacul0b0/q34RpFyOu3 HWfuFiso5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAExE8Ccjw9KGP9FnCw+Z62Xm XLi7iHF9iXG3+Jv1JedEDoduVKs3y2P4Xxt9Rm9b/Ge3NKtiieshmred54e8NjPrmCLxkTcgMpC ZBwA= X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231213084024.2367360-16-ardb@google.com> Subject: [PATCH v7 7/7] arm64: kaslr: Adjust randomization range dynamically From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231213_004056_782169_1F504CE8 X-CRM114-Status: GOOD ( 15.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel Currently, we base the KASLR randomization range on a rough estimate of the available space in the upper VA region: the lower 1/4th has the module region and the upper 1/4th has the fixmap, vmemmap and PCI I/O ranges, and so we pick a random location in the remaining space in the middle. Once we enable support for 5-level paging with 4k pages, this no longer works: the vmemmap region, being dimensioned to cover a 52-bit linear region, takes up so much space in the upper VA region (the size of which is based on a 48-bit VA space for compatibility with non-LVA hardware) that the region above the vmalloc region takes up more than a quarter of the available space. So instead of a heuristic, let's derive the randomization range from the actual boundaries of the vmalloc region. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/image-vars.h | 2 ++ arch/arm64/kernel/pi/kaslr_early.c | 11 ++++++----- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 5e4dc72ab1bd..e931ce078a00 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -36,6 +36,8 @@ PROVIDE(__pi___memcpy = __pi_memcpy); PROVIDE(__pi___memmove = __pi_memmove); PROVIDE(__pi___memset = __pi_memset); +PROVIDE(__pi_vabits_actual = vabits_actual); + #ifdef CONFIG_KVM /* diff --git a/arch/arm64/kernel/pi/kaslr_early.c b/arch/arm64/kernel/pi/kaslr_early.c index 17bff6e399e4..b9e0bb4bc6a9 100644 --- a/arch/arm64/kernel/pi/kaslr_early.c +++ b/arch/arm64/kernel/pi/kaslr_early.c @@ -14,6 +14,7 @@ #include #include +#include /* taken from lib/string.c */ static char *__strstr(const char *s1, const char *s2) @@ -87,7 +88,7 @@ static u64 get_kaslr_seed(void *fdt) asmlinkage u64 kaslr_early_init(void *fdt) { - u64 seed; + u64 seed, range; if (is_kaslr_disabled_cmdline(fdt)) return 0; @@ -102,9 +103,9 @@ asmlinkage u64 kaslr_early_init(void *fdt) /* * OK, so we are proceeding with KASLR enabled. Calculate a suitable * kernel image offset from the seed. Let's place the kernel in the - * middle half of the VMALLOC area (VA_BITS_MIN - 2), and stay clear of - * the lower and upper quarters to avoid colliding with other - * allocations. + * 'middle' half of the VMALLOC area, and stay clear of the lower and + * upper quarters to avoid colliding with other allocations. */ - return BIT(VA_BITS_MIN - 3) + (seed & GENMASK(VA_BITS_MIN - 3, 0)); + range = (VMALLOC_END - KIMAGE_VADDR) / 2; + return range / 2 + (((__uint128_t)range * seed) >> 64); }