From patchwork Thu Nov 24 12:39:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13054934 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4DAF5C433FE for ; Thu, 24 Nov 2022 12:44:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=W/9xjie7XD2oFVVX5Ojp3N8NZSnuONAsF/sfucPMmhQ=; b=NyGF6xBUOH4sBc W4O9aXNqgjAPMCQk4gnSQYhCG15UtKoTDfWcrHHidNV1VZWU/BZL4Cy5Gz8b+SOSHHlJlqvEIpc8W y/8iWjGYIqHlJNmjhvDf/B3Mm07XbYKfJoeL83VKD/TjHj/kR1YLqfJ4u7cJ1JcRD4FEMWU20wIKN v1HcC5/n8xJdAeg7XpeW3+fPK1AeETsTCl74qpTBhKlrH9EiEECsXeeuOzQagyJiEbpxSpF80UmYd NDTq/NoSrBzWtJQhUFiQjdpY7A2HBo+3RnkXjZ/lmEPPfkZevD5hlBFqQdVMmB3TvSnIjqQdFl45v Y+VbuOQKcN/A3Wxs+L6Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oyBZP-008PxY-0G; Thu, 24 Nov 2022 12:43:11 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oyBWa-008OTy-Li for linux-arm-kernel@lists.infradead.org; Thu, 24 Nov 2022 12:40:18 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 3E550620F8; Thu, 24 Nov 2022 12:40:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 17A59C433D7; Thu, 24 Nov 2022 12:40:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669293615; bh=qcj/VXBa9HsYQa/yB03lZWjYFItXlx2dXWMhWosGmag=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Mq/pGKwAJ6dt1KpWZ0oj/kjTqxq5tO3atNnpxj0axdG5SRsfaLRgo+Dph3VpDFY7v djh4rrgMdLPo6z9qzOWEZqeeByMVSFa//bHN+ignfFD23badjjEFryhz7RtKAz945e GXJY5Sms2FaEQKhB8U94sWhdmxujIj5z2IpLIBPsZfKHthRJmrUbhQqv1BKkL4Qqwt P3QtdXXphip6lAWBSlnZyf7KeEpX0Z+X+8JTcpbteC62WFmCZFD5RbHu05bnAcIhHI r10genkqMltgpfhYg3AHLK52q8KDo6BHOUYuM0XI6WIFHvIDmfDEu9atS4BdeC+TwP LFZEX5RG3jMmQ== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Mark Rutland , Kees Cook , Catalin Marinas , Mark Brown , Anshuman Khandual , Richard Henderson , Ryan Roberts Subject: [PATCH v2 08/19] arm64: mm: Deal with potential ID map extension if VA_BITS > VA_BITS_MIN Date: Thu, 24 Nov 2022 13:39:21 +0100 Message-Id: <20221124123932.2648991-9-ardb@kernel.org> X-Mailer: git-send-email 2.38.1.584.g0f3c55d4c2-goog In-Reply-To: <20221124123932.2648991-1-ardb@kernel.org> References: <20221124123932.2648991-1-ardb@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221124_044016_839242_4B12BCA1 X-CRM114-Status: GOOD ( 25.62 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 16k pages with LPA2, we will fall back to 47 bits of VA space in case LPA2 is not implemented. Since we support loading the kernel anywhere in the 48-bit addressable PA space, this means we may have to extend the ID map like we normally do in such cases, even when VA_BITS >= 48. Since VA_BITS_MIN will equal 47 in that case, use that symbolic constant instead to determine whether ID map extension is required. Also, use vabits_actual to determine whether create_idmap() needs to add an extra level as well, as this is never needed if LPA2 is enabled at runtime. Note that VA_BITS, VA_BITS_MIN and vabits_actual all collapse into the same compile time constant on configurations that support ID map extension currently, so there this change should have no effect whatsoever. Note that the use of PGDIR_SHIFT in the calculation of EXTRA_SHIFT is no longer appropriate, as it is derived from VA_BITS not VA_BITS_MIN. So rephrase the check whether VA_BITS_MIN is consistent with the number of levels. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/kernel-pgtable.h | 2 +- arch/arm64/kernel/head.S | 40 ++++++++++---------- arch/arm64/mm/mmu.c | 4 +- arch/arm64/mm/proc.S | 5 ++- 4 files changed, 26 insertions(+), 25 deletions(-) diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h index 4278cd088347..faa11e8b4a0e 100644 --- a/arch/arm64/include/asm/kernel-pgtable.h +++ b/arch/arm64/include/asm/kernel-pgtable.h @@ -73,7 +73,7 @@ #define INIT_DIR_SIZE (PAGE_SIZE * (EARLY_PAGES(KIMAGE_VADDR, _end, EARLY_KASLR) + EARLY_SEGMENT_EXTRA_PAGES)) /* the initial ID map may need two extra pages if it needs to be extended */ -#if VA_BITS < 48 +#if VA_BITS_MIN < 48 #define INIT_IDMAP_DIR_SIZE ((INIT_IDMAP_DIR_PAGES + 2) * PAGE_SIZE) #else #define INIT_IDMAP_DIR_SIZE (INIT_IDMAP_DIR_PAGES * PAGE_SIZE) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index d423ff78474e..94de42dfe97d 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -266,40 +266,40 @@ SYM_FUNC_START_LOCAL(create_idmap) * space for ordinary kernel and user space mappings. * * There are three cases to consider here: - * - 39 <= VA_BITS < 48, and the ID map needs up to 48 VA bits to cover - * the placement of the image. In this case, we configure one extra - * level of translation on the fly for the ID map only. (This case - * also covers 42-bit VA/52-bit PA on 64k pages). + * - 39 <= VA_BITS_MIN < 48, and the ID map needs up to 48 VA bits to + * cover the placement of the image. In this case, we configure one + * extra level of translation on the fly for the ID map only. (This + * case also covers 42-bit VA/52-bit PA on 64k pages). * - * - VA_BITS == 48, and the ID map needs more than 48 VA bits. This can - * only happen when using 64k pages, in which case we need to extend - * the root level table rather than add a level. Note that we can - * treat this case as 'always extended' as long as we take care not - * to program an unsupported T0SZ value into the TCR register. + * - VA_BITS_MIN == 48, and the ID map needs more than 48 VA bits. This + * can only happen when using 64k pages, in which case we need to + * extend the root level table rather than add a level. Note that we + * can treat this case as 'always extended' as long as we take care + * not to program an unsupported T0SZ value into the TCR register. * * - Combinations that would require two additional levels of * translation are not supported, e.g., VA_BITS==36 on 16k pages, or * VA_BITS==39/4k pages with 5-level paging, where the input address * requires more than 47 or 48 bits, respectively. */ -#if (VA_BITS < 48) -#define EXTRA_SHIFT (PGDIR_SHIFT + PAGE_SHIFT - 3) +#if VA_BITS_MIN < 48 +#define EXTRA_SHIFT VA_BITS_MIN /* - * If VA_BITS < 48, we have to configure an additional table level. - * First, we have to verify our assumption that the current value of - * VA_BITS was chosen such that all translation levels are fully - * utilised, and that lowering T0SZ will always result in an additional - * translation level to be configured. + * If VA_BITS_MIN < 48, we may have to configure an additional table + * level. First, we have to verify our assumption that the current + * value of VA_BITS_MIN was chosen such that all translation levels are + * fully utilised, and that lowering T0SZ will always result in an + * additional translation level to be configured. */ -#if VA_BITS != EXTRA_SHIFT -#error "Mismatch between VA_BITS and page size/number of translation levels" +#if ((VA_BITS_MIN - PAGE_SHIFT) % (PAGE_SHIFT - 3)) != 0 +#error "Mismatch between VA_BITS_MIN and page size/number of translation levels" #endif #else #define EXTRA_SHIFT /* - * If VA_BITS == 48, we don't have to configure an additional - * translation level, but the top-level table has more entries. + * If VA_BITS_MIN == 48, we don't have to configure an additional + * translation level, but the top-level table may have more entries. */ #endif adrp x0, init_idmap_pg_dir diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index b0c702e5bf66..bcf617f956cb 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -702,9 +702,9 @@ static void __init create_idmap(void) u64 pgd_phys; /* check if we need an additional level of translation */ - if (VA_BITS < 48 && idmap_t0sz < (64 - VA_BITS_MIN)) { + if (vabits_actual < 48 && idmap_t0sz < (64 - VA_BITS_MIN)) { pgd_phys = early_pgtable_alloc(PAGE_SHIFT); - set_pgd(&idmap_pg_dir[start >> VA_BITS], + set_pgd(&idmap_pg_dir[start >> VA_BITS_MIN], __pgd(pgd_phys | P4D_TYPE_TABLE)); pgd = __va(pgd_phys); } diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index e14648ca820b..873adaccb12f 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -430,10 +430,11 @@ SYM_FUNC_START(__cpu_setup) tcr_clear_errata_bits tcr, x9, x5 -#if VA_BITS < 48 +#if VA_BITS_MIN < 48 idmap_get_t0sz x9 tcr_set_t0sz tcr, x9 -#elif VA_BITS > VA_BITS_MIN +#endif +#if VA_BITS > VA_BITS_MIN mov x9, #64 - VA_BITS alternative_if ARM64_HAS_LVA tcr_set_t1sz tcr, x9