From patchwork Tue Sep 12 14:16:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13381796 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1C6D7CA0EEC for ; Tue, 12 Sep 2023 14:20:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=G7RgftsZu0sa9jLZvNOu3eQlDi0SGKjyYr1vC5o5SeY=; b=3k7EXpRHRRqsgCMRkyaDKbarq2 Q4tlk5a5JILbymGXMDe/+dsUOcw7Num9u9HlV93KFGPvLTF2d5fn2qXqDZP09Sgn3NZd4f74MZomn a72/i40PzLViPiQAWUIKDNTjjF/XkZPrN9zO3D+NIIDBxB1sR0iP+2FXXdZPlj4zOYitR3we6pdxz Bk/7P6zr3tkETJuVzd6oJ3aenJ+7+Y47EliGMH//R/TsGwrzjtkoDzxlsXzlCxFwbmIiJyUJoHyQy 7dwwV258cAO7V8//My/6F1dUFzENRZgKy8ktCUS8LAmTHvaDtfGP8TjfEIwpDDUN8LyxSF08tfQbZ 3v48gyQA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qg4Ej-003XWC-05; Tue, 12 Sep 2023 14:19:29 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qg4E4-003WwF-1b for linux-arm-kernel@bombadil.infradead.org; Tue, 12 Sep 2023 14:18:48 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:Cc:To:From:Subject: Message-ID:References:Mime-Version:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=7rSPmb28NDHeLvOcEanQajPSW8hxzl+nLXq6JrjyI9s=; b=kecX8OLjV/3yo3g/YCIXDW3/LV PvQ9AE2LjzAaz5fJ/5IQMMNfPOT+c65OxPvpqV8GcZfosw0hnUrMnnz18puOV5GQ86PjkALAu5B8R QXgC4hXn/66KfVFdd+FFNkt8xxKykLMeCB/uFWJdGCMaJL3SKsoHmShar21tBKRlzzHbdkC6L+pCY wmA9qOo6Hf2S+gihDvXUxAwcC6y/mcE+QP7t3SpSAuCwpevSQpCv/OmQodu+EeCXN9QNHEwelSQze 2UwRUTPbG4Wf464FFdMLFl3ucc1CijfMI3IUe2TCCNPKvEJ91tRKLeSnUtdD34Ur09AMy9luVoDe8 NIXRK/tw==; Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by desiato.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qg4Dy-0069zW-0g for linux-arm-kernel@lists.infradead.org; Tue, 12 Sep 2023 14:18:46 +0000 Received: by mail-wr1-x44a.google.com with SMTP id ffacd0b85a97d-31c5cc3b512so3704908f8f.1 for ; Tue, 12 Sep 2023 07:18:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1694528321; x=1695133121; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=7rSPmb28NDHeLvOcEanQajPSW8hxzl+nLXq6JrjyI9s=; b=Ia21qUluD9oiKO86SqenezasnEjn473/XEMRfNRD9AZvFUlIKAxX+dV8+kUqeKOHsh W6WxUA3uoKhv+pzc/K5ZwUh4HWbigOEw4i9+ge6VW0+Uw/RMvkhsCRDhhac4jR6djT5N JAVPcvsLSWq5WSNcLoStO2V72mz77XiNAOZbaL7RZPfCSnyCFqo2KD69XB8xNvkmX+cK ybQL8BeMBDz6Pjm24aGWdyTqhSopn8gqM+iWz6cKZXbwbP7Iyslcy4cB3J9MiczO4PPG jVHC2Thb+q9rHOgbKJjDR2SsOyPqkQ7qaPq+yJwb0GtHb/zMlG/lVYMIyZhkZw6S9nM0 jLMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694528321; x=1695133121; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7rSPmb28NDHeLvOcEanQajPSW8hxzl+nLXq6JrjyI9s=; b=k4gkkFcN3o86TQp5a9XrYSE+5bwm1on40MLp5SGEm80lqwPWfHRXINtlk3I0h2HjKu XfD5bI8haL+Cb1Ck1HIipvFcaxXzohDYuU7/rEoFUFGFLDNrqGB4dsj2f/hwjJFDVCnh +LlR4TuSBaKkc8lsj2lff+7bHbCHKqdH4vPOYBcsf3NrW0gZpBFiaEugDvpNDTqt07FA Mzvq8gJpSVql39Gt01tO1agMj1dv42IkEOKIfzYy6/uGprpIH0JmiPGjL4qd69oSxXoY mRtzsAChVgrwah2tn/mWSAhIg2O1CfPhw30aZKniszomy1Xgm/d1Zej/wZ4VmeFFZHzI 8kWw== X-Gm-Message-State: AOJu0Yz3MhpHmEqh+Gdorjze5YdqALHYyE2iSgt2ShGXKy1Vsu52sCwN 7Ysujqz65agwP3JRlqDtGh521W1jWr3ARRPwvaCt4jpsYbqlKBo8NIlawm07KVGMXq66S3fcPHy ps1DBdSfAGtwKIdHJEgEXcf32t01rhAGp32WafN/aLxMPQgTdC2DUlzON1WWmZzhyeTMPUb+WjB I= X-Google-Smtp-Source: AGHT+IG0zayvnQdH/RqoUrJoPRXJEIJmOddCZXS7OJVSGPZyfMKc4D+VRvRtz5OhI7lJ6PX/DA6pZ3h7 X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a5d:60cb:0:b0:317:690e:7247 with SMTP id x11-20020a5d60cb000000b00317690e7247mr166538wrt.0.1694528321037; Tue, 12 Sep 2023 07:18:41 -0700 (PDT) Date: Tue, 12 Sep 2023 14:16:25 +0000 In-Reply-To: <20230912141549.278777-63-ardb@google.com> Mime-Version: 1.0 References: <20230912141549.278777-63-ardb@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=5162; i=ardb@kernel.org; h=from:subject; bh=yFqAuJnKNAsURVmbFl/LbDGXxKPczokDU1lCacgcPzM=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIZWhaFda8JckW17PZR1zeDgku/uOJ9Roh5w94ytlftH4l MazD/c6SlkYxDgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwERkTRj+R/96Kbpxu/WhRbeP 1aow6U+RceH/ayShmRmRztw8M+ZdDcM/W+44GdvF+UzHsyJOtq72a5V7N/tQwrRuH88PR+beO8j CAAA= X-Mailer: git-send-email 2.42.0.283.g2d96d420d3-goog Message-ID: <20230912141549.278777-98-ardb@google.com> Subject: [PATCH v4 35/61] arm64: pgtable: Decouple PGDIR size macros from PGD/PUD/PMD levels From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland , Ryan Roberts , Anshuman Khandual , Kees Cook , Joey Gouly X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230912_151843_776117_6B220D73 X-CRM114-Status: GOOD ( 19.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel The mapping from PGD/PUD/PMD to levels and shifts is very confusing, given that, due to folding, the shifts may be equal for different levels, if the macros are even #define'd to begin with. In a subsequent patch, we will modify the ID mapping code to decouple the number of levels from the kernel's view of how these types are folded, so prepare for this by reformulating the macros without the use of these types. Instead, use SWAPPER_BLOCK_SHIFT as the base quantity, and derive it from either PAGE_SHIFT or PMD_SHIFT, which -if defined at all- are defined unambiguously for a given page size, regardless of the number of configured levels. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/kernel-pgtable.h | 65 ++++++-------------- 1 file changed, 19 insertions(+), 46 deletions(-) diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h index 742a4b2778f7..5000f38ae0c6 100644 --- a/arch/arm64/include/asm/kernel-pgtable.h +++ b/arch/arm64/include/asm/kernel-pgtable.h @@ -13,27 +13,22 @@ #include /* - * The linear mapping and the start of memory are both 2M aligned (per - * the arm64 booting.txt requirements). Hence we can use section mapping - * with 4K (section size = 2M) but not with 16K (section size = 32M) or - * 64K (section size = 512M). + * The physical and virtual addresses of the start of the kernel image are + * equal modulo 2 MiB (per the arm64 booting.txt requirements). Hence we can + * use section mapping with 4K (section size = 2M) but not with 16K (section + * size = 32M) or 64K (section size = 512M). */ - -/* - * The idmap and swapper page tables need some space reserved in the kernel - * image. Both require pgd, pud (4 levels only) and pmd tables to (section) - * map the kernel. With the 64K page configuration, swapper and idmap need to - * map to pte level. The swapper also maps the FDT (see __create_page_tables - * for more information). Note that the number of ID map translation levels - * could be increased on the fly if system RAM is out of reach for the default - * VA range, so pages required to map highest possible PA are reserved in all - * cases. - */ -#ifdef CONFIG_ARM64_4K_PAGES -#define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS - 1) +#if defined(PMD_SIZE) && PMD_SIZE <= MIN_KIMG_ALIGN +#define SWAPPER_BLOCK_SHIFT PMD_SHIFT +#define SWAPPER_SKIP_LEVEL 1 #else -#define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS) +#define SWAPPER_BLOCK_SHIFT PAGE_SHIFT +#define SWAPPER_SKIP_LEVEL 0 #endif +#define SWAPPER_BLOCK_SIZE (UL(1) << SWAPPER_BLOCK_SHIFT) +#define SWAPPER_TABLE_SHIFT (SWAPPER_BLOCK_SHIFT + PAGE_SHIFT - 3) + +#define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS - SWAPPER_SKIP_LEVEL) #define IDMAP_VA_BITS 48 #define IDMAP_LEVELS ARM64_HW_PGTABLE_LEVELS(IDMAP_VA_BITS) @@ -53,24 +48,13 @@ #define EARLY_ENTRIES(vstart, vend, shift, add) \ (SPAN_NR_ENTRIES(vstart, vend, shift) + (add)) -#define EARLY_PGDS(vstart, vend, add) (EARLY_ENTRIES(vstart, vend, PGDIR_SHIFT, add)) - -#if SWAPPER_PGTABLE_LEVELS > 3 -#define EARLY_PUDS(vstart, vend, add) (EARLY_ENTRIES(vstart, vend, PUD_SHIFT, add)) -#else -#define EARLY_PUDS(vstart, vend, add) (0) -#endif +#define EARLY_LEVEL(l, vstart, vend, add) \ + (SWAPPER_PGTABLE_LEVELS > l ? EARLY_ENTRIES(vstart, vend, SWAPPER_BLOCK_SHIFT + l * (PAGE_SHIFT - 3), add) : 0) -#if SWAPPER_PGTABLE_LEVELS > 2 -#define EARLY_PMDS(vstart, vend, add) (EARLY_ENTRIES(vstart, vend, SWAPPER_TABLE_SHIFT, add)) -#else -#define EARLY_PMDS(vstart, vend, add) (0) -#endif - -#define EARLY_PAGES(vstart, vend, add) ( 1 /* PGDIR page */ \ - + EARLY_PGDS((vstart), (vend), add) /* each PGDIR needs a next level page table */ \ - + EARLY_PUDS((vstart), (vend), add) /* each PUD needs a next level page table */ \ - + EARLY_PMDS((vstart), (vend), add)) /* each PMD needs a next level page table */ +#define EARLY_PAGES(vstart, vend, add) (1 /* PGDIR page */ \ + + EARLY_LEVEL(3, (vstart), (vend), add) /* each entry needs a next level page table */ \ + + EARLY_LEVEL(2, (vstart), (vend), add) /* each entry needs a next level page table */ \ + + EARLY_LEVEL(1, (vstart), (vend), add))/* each entry needs a next level page table */ #define INIT_DIR_SIZE (PAGE_SIZE * (EARLY_PAGES(KIMAGE_VADDR, _end, EXTRA_PAGE) + EARLY_SEGMENT_EXTRA_PAGES)) /* the initial ID map may need two extra pages if it needs to be extended */ @@ -81,17 +65,6 @@ #endif #define INIT_IDMAP_DIR_PAGES EARLY_PAGES(KIMAGE_VADDR, _end + MAX_FDT_SIZE + SWAPPER_BLOCK_SIZE, 1) -/* Initial memory map size */ -#ifdef CONFIG_ARM64_4K_PAGES -#define SWAPPER_BLOCK_SHIFT PMD_SHIFT -#define SWAPPER_BLOCK_SIZE PMD_SIZE -#define SWAPPER_TABLE_SHIFT PUD_SHIFT -#else -#define SWAPPER_BLOCK_SHIFT PAGE_SHIFT -#define SWAPPER_BLOCK_SIZE PAGE_SIZE -#define SWAPPER_TABLE_SHIFT PMD_SHIFT -#endif - /* The number of segments in the kernel image (text, rodata, inittext, initdata, data+bss) */ #define KERNEL_SEGMENT_COUNT 5