From patchwork Tue Jan 23 14:53:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13527544 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9E99AC47DDF for ; Tue, 23 Jan 2024 14:57:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=mlStx1g+wdLkIuUHVXPME0QCvqUJJi7rYZ+CV00EXVU=; b=yKiiZx3Al7wbxiET62lADWQ6UD y2U89DiyPBoYbZ8snpETEgNl/tF/9TscEiHELZgXV3VD53hZl5sno4gOX2A0MjwlYr5uv4BI7WAVY CpG3mFnnfoNPPGel0xOiP6A7lrXTNwhDDjkORMtqC3ClYkguP7fARIVuGE1jryFPfRa1JB8FQFEp7 fEv+fvMrq9ShvnP4DHloF+24bQ9rptDVpQlPfjqGb7LvphblgcBrIs+0H6QSFtwYKrTsXBytbQVKR Hu78v4U3dkf3/P8Cz/hG4NCkNiKrR4hznAxQMSbR0WCg9H9eQqq6Odzr6Am2rLopOWL4rKscAHU75 KPBuQ6Fw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rSICQ-00GucK-08; Tue, 23 Jan 2024 14:56:26 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rSIBb-00GuDx-15 for linux-arm-kernel@bombadil.infradead.org; Tue, 23 Jan 2024 14:55:35 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:Cc:To:From:Subject: Message-ID:References:Mime-Version:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=Kn7P6Oq1LFp+DUl5GfvtGAwXOTQGXV6AGlrtT3ROFPE=; b=kmzDiuJkmsEWz7whhg45SI76/a mcos/Wen+zi1igRs7JsTiNscikyjN46qiObpqwsb9Nbkwx/3979btxtRJu1yT6qfG4Ycc/YRwtyfk R4TvQ4VaEnaXDAQkKEQiWqgAuOFcE9EuTxtcRhm1OqrknOokS+XcF3u65dJs9PQtlb9aRKC2uIZZX b+yIhJEcNFCQKcZbY8+Opgbp8uAgo2sN6lyct0SecSPuciSZQl0eBQoBkRsqBVFTPGClXK345/teJ sGBmt+izK1MC0HV8FNW4XAag7EFSPjo3F/QDrHF0smBiFcO/mSXIyaNsjBek7piKMeytriJk+GM7c sKocvdOg==; Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by desiato.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rSIBT-00000003vQw-288f for linux-arm-kernel@lists.infradead.org; Tue, 23 Jan 2024 14:55:34 +0000 Received: by mail-wr1-x44a.google.com with SMTP id ffacd0b85a97d-3392c9e8001so1157867f8f.0 for ; Tue, 23 Jan 2024 06:55:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706021725; x=1706626525; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Kn7P6Oq1LFp+DUl5GfvtGAwXOTQGXV6AGlrtT3ROFPE=; b=TrwN74pHKM+ayAmJsh2xJDuSc2ipP9oWN/283RXDFo2SJnP9IPr5Y9XvjsfqEDbhAs NNaeQwrQlQmrIW+ADjjj1JpBK/efVx6ywqeGzFY6MX28T/3tBIMBNR6nXhKagC/YWmEm RyMZ6q6XHjPFFVs4tduf8wPIPkcrhGw3S1X8aZDa7p9axTaQuj2AUqA2iPOdFH7FogI3 XTWSScNL78onH8XgJqlf7jzw+bhVobljvjDYyHj8Yv9vHJmWpEuL70ahj27AUsjALViA TVPaTK6QKVuhbTv3SwN7goPqVDFwKb2uq1lkUvsXzsZWowXzoGJA2AuHsJ13Yn3dwI/2 4LdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706021725; x=1706626525; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Kn7P6Oq1LFp+DUl5GfvtGAwXOTQGXV6AGlrtT3ROFPE=; b=Tyy6IINws//Mqphq2pUmnL4I5kwRbjPg6i2Di6knTSHyp/FT82ALJxh5qxzd6hGjUl VsvhwSb3/WYwCLXJD6xDjSnOONJrWWkDv/dX8bVAQsKHFVdguOCXdqqrSE8Sp9dQ1Abp 1+wywQSb3aFMDgXw9bOq1uyw08yA4+eAPqwjOKu1OhfrV7s3VM/fSvHkX9ztAok+L7x8 qTV5LSayqYCZOisoVMXGKo3b7/fXYs0dplOQRMuZsmz0eter50VbKTMVXf0B2hma4tED sjrYTUlmMmDGzRjAOm2K4aRKVAlj1FwZ8bJYUw10Ifpoz4gXeV52K7WMltNQ3ttZmzr5 5BMA== X-Gm-Message-State: AOJu0YwVuLh36aFg1EHHR3v7m8ZJwXlx7RH44Ox8B5dFlr2eqCwPrwoT SSdiNjYORCTqV6DmdNX/7tRtFzOeukuZLOETrTyV3LyKBLB3l0ySZVLDGFvatHIkmFCCTefCiZ2 qLN84Kl50LRoYUvi32iqSmBpxuz8dFoGY5EkCFSDvNYdC4V/fWXO0XPBNlOM2R/5zko9qnBcOme 07cOxqI0lvGbXPuEIUUJr0riW0hh9RaFoo1SxzW3mM X-Google-Smtp-Source: AGHT+IFSGlfzVFO1SEhS+T9cw6KdL5rmEayfzpP52Mr8aUfn3VQFjxlhpXWNDZ60LXWFYCHwM6UiiRDZ X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a5d:6a42:0:b0:337:bce4:7f3b with SMTP id t2-20020a5d6a42000000b00337bce47f3bmr29930wrw.7.1706021724969; Tue, 23 Jan 2024 06:55:24 -0800 (PST) Date: Tue, 23 Jan 2024 15:53:26 +0100 In-Reply-To: <20240123145258.1462979-52-ardb+git@google.com> Mime-Version: 1.0 References: <20240123145258.1462979-52-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=5168; i=ardb@kernel.org; h=from:subject; bh=3d8OYNvEb2VUdK20lgpBOirlo30qq1GDYd9RE4l7dFo=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXX9pRdBDk0NgScj781KvTKn7b/rlfANCp2CnfuNdRY39 deUTrnfUcrCIMbBICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACbyvIyRoU1lsfRGidm/7wqe dzh/6+4pTd4pU9qPCa8q5jugWbQ8ZxfD/6LOvRLbeyYIr1Z4tf2Nf9v1a+wn24uP5yw5y79l+7Z 55/gA X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240123145258.1462979-79-ardb+git@google.com> Subject: [PATCH v7 27/50] arm64: pgtable: Decouple PGDIR size macros from PGD/PUD/PMD levels From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland , Ryan Roberts , Anshuman Khandual , Kees Cook X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240123_145527_743912_19125897 X-CRM114-Status: GOOD ( 18.96 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel The mapping from PGD/PUD/PMD to levels and shifts is very confusing, given that, due to folding, the shifts may be equal for different levels, if the macros are even #define'd to begin with. In a subsequent patch, we will modify the ID mapping code to decouple the number of levels from the kernel's view of how these types are folded, so prepare for this by reformulating the macros without the use of these types. Instead, use SWAPPER_BLOCK_SHIFT as the base quantity, and derive it from either PAGE_SHIFT or PMD_SHIFT, which -if defined at all- are defined unambiguously for a given page size, regardless of the number of configured levels. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/kernel-pgtable.h | 65 ++++++-------------- 1 file changed, 19 insertions(+), 46 deletions(-) diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h index 742a4b2778f7..f1fc98a233d5 100644 --- a/arch/arm64/include/asm/kernel-pgtable.h +++ b/arch/arm64/include/asm/kernel-pgtable.h @@ -13,27 +13,22 @@ #include /* - * The linear mapping and the start of memory are both 2M aligned (per - * the arm64 booting.txt requirements). Hence we can use section mapping - * with 4K (section size = 2M) but not with 16K (section size = 32M) or - * 64K (section size = 512M). + * The physical and virtual addresses of the start of the kernel image are + * equal modulo 2 MiB (per the arm64 booting.txt requirements). Hence we can + * use section mapping with 4K (section size = 2M) but not with 16K (section + * size = 32M) or 64K (section size = 512M). */ - -/* - * The idmap and swapper page tables need some space reserved in the kernel - * image. Both require pgd, pud (4 levels only) and pmd tables to (section) - * map the kernel. With the 64K page configuration, swapper and idmap need to - * map to pte level. The swapper also maps the FDT (see __create_page_tables - * for more information). Note that the number of ID map translation levels - * could be increased on the fly if system RAM is out of reach for the default - * VA range, so pages required to map highest possible PA are reserved in all - * cases. - */ -#ifdef CONFIG_ARM64_4K_PAGES -#define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS - 1) +#if defined(PMD_SIZE) && PMD_SIZE <= MIN_KIMG_ALIGN +#define SWAPPER_BLOCK_SHIFT PMD_SHIFT +#define SWAPPER_SKIP_LEVEL 1 #else -#define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS) +#define SWAPPER_BLOCK_SHIFT PAGE_SHIFT +#define SWAPPER_SKIP_LEVEL 0 #endif +#define SWAPPER_BLOCK_SIZE (UL(1) << SWAPPER_BLOCK_SHIFT) +#define SWAPPER_TABLE_SHIFT (SWAPPER_BLOCK_SHIFT + PAGE_SHIFT - 3) + +#define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS - SWAPPER_SKIP_LEVEL) #define IDMAP_VA_BITS 48 #define IDMAP_LEVELS ARM64_HW_PGTABLE_LEVELS(IDMAP_VA_BITS) @@ -53,24 +48,13 @@ #define EARLY_ENTRIES(vstart, vend, shift, add) \ (SPAN_NR_ENTRIES(vstart, vend, shift) + (add)) -#define EARLY_PGDS(vstart, vend, add) (EARLY_ENTRIES(vstart, vend, PGDIR_SHIFT, add)) - -#if SWAPPER_PGTABLE_LEVELS > 3 -#define EARLY_PUDS(vstart, vend, add) (EARLY_ENTRIES(vstart, vend, PUD_SHIFT, add)) -#else -#define EARLY_PUDS(vstart, vend, add) (0) -#endif +#define EARLY_LEVEL(lvl, vstart, vend, add) \ + (SWAPPER_PGTABLE_LEVELS > lvl ? EARLY_ENTRIES(vstart, vend, SWAPPER_BLOCK_SHIFT + lvl * (PAGE_SHIFT - 3), add) : 0) -#if SWAPPER_PGTABLE_LEVELS > 2 -#define EARLY_PMDS(vstart, vend, add) (EARLY_ENTRIES(vstart, vend, SWAPPER_TABLE_SHIFT, add)) -#else -#define EARLY_PMDS(vstart, vend, add) (0) -#endif - -#define EARLY_PAGES(vstart, vend, add) ( 1 /* PGDIR page */ \ - + EARLY_PGDS((vstart), (vend), add) /* each PGDIR needs a next level page table */ \ - + EARLY_PUDS((vstart), (vend), add) /* each PUD needs a next level page table */ \ - + EARLY_PMDS((vstart), (vend), add)) /* each PMD needs a next level page table */ +#define EARLY_PAGES(vstart, vend, add) (1 /* PGDIR page */ \ + + EARLY_LEVEL(3, (vstart), (vend), add) /* each entry needs a next level page table */ \ + + EARLY_LEVEL(2, (vstart), (vend), add) /* each entry needs a next level page table */ \ + + EARLY_LEVEL(1, (vstart), (vend), add))/* each entry needs a next level page table */ #define INIT_DIR_SIZE (PAGE_SIZE * (EARLY_PAGES(KIMAGE_VADDR, _end, EXTRA_PAGE) + EARLY_SEGMENT_EXTRA_PAGES)) /* the initial ID map may need two extra pages if it needs to be extended */ @@ -81,17 +65,6 @@ #endif #define INIT_IDMAP_DIR_PAGES EARLY_PAGES(KIMAGE_VADDR, _end + MAX_FDT_SIZE + SWAPPER_BLOCK_SIZE, 1) -/* Initial memory map size */ -#ifdef CONFIG_ARM64_4K_PAGES -#define SWAPPER_BLOCK_SHIFT PMD_SHIFT -#define SWAPPER_BLOCK_SIZE PMD_SIZE -#define SWAPPER_TABLE_SHIFT PUD_SHIFT -#else -#define SWAPPER_BLOCK_SHIFT PAGE_SHIFT -#define SWAPPER_BLOCK_SIZE PAGE_SIZE -#define SWAPPER_TABLE_SHIFT PMD_SHIFT -#endif - /* The number of segments in the kernel image (text, rodata, inittext, initdata, data+bss) */ #define KERNEL_SEGMENT_COUNT 5