From patchwork Mon Oct 14 10:58:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834713 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16139D1A45A for ; Mon, 14 Oct 2024 11:01:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 953CF6B00D8; Mon, 14 Oct 2024 07:01:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 902F76B00DA; Mon, 14 Oct 2024 07:01:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A20D6B00DB; Mon, 14 Oct 2024 07:01:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5951C6B00D8 for ; Mon, 14 Oct 2024 07:01:48 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 820D780D96 for ; Mon, 14 Oct 2024 11:01:41 +0000 (UTC) X-FDA: 82671917328.12.7078B93 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf07.hostedemail.com (Postfix) with ESMTP id B3DFA4001F for ; Mon, 14 Oct 2024 11:01:37 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903564; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OACUuu7EkK/bVx0oPSFpoAc1oPx/wt8iZOtMNIFSzjQ=; b=tRLhfdfIBPsahxvB08TXwQc95xbWz7bzem0/LOx+idZ8LRRNe3vn7fpYZgeAv4oytrxgp1 EgQR2+fljVUHVWrDiFp3C/3PLH/ctPkyWw7974+zLbISjrM6xbFtfVdjABteTS5nuQJA3q 0hGVmD2ARl2xzPWcfn9Qqf5Qcixjd0U= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903564; a=rsa-sha256; cv=none; b=jmUwi65Lqdkhs02Zru0GcLRQRKSF94IA+mc7xVsCmS9WCvToh/sAu6x8J6jWu6hR6qam+S Ypjw1J9IoYpkCgCWkADWPq0CWqeJCM95bOpYgvKKcpmzNqSPFA2z6JSNfENJnecgs/HOUq 3msDQKCgq7P9pKbUakyQ7Zh0eqRzR+I= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5E6BA1764; Mon, 14 Oct 2024 04:02:15 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5FBE13F51B; Mon, 14 Oct 2024 04:01:43 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 45/57] arm64: Rework trampoline rodata mapping Date: Mon, 14 Oct 2024 11:58:52 +0100 Message-ID: <20241014105912.3207374-45-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B3DFA4001F X-Stat-Signature: xsy6gzjuyr6at81qqdtqa7n9tdkyik7x X-HE-Tag: 1728903697-529767 X-HE-Meta: U2FsdGVkX1+8KHz8oIVN+rM1CspMuHHvnLANQup23QWEpxMK+zTzs6StcOSqi1TCfE2jbBgEz0k01dltOSw6ISlvaeSumE+K/ICPjOYENHO0nHYzO4S5LC5S6i8xBxJcup6ackRIHmpURIUxEh/UQA4gSdwQRYTvsNNBZ/iJWtH9zEG2eX4slUhF63KFR4x4ZXbx4m57sN8jc4VVBz4nlebrtIPqjmZwobd4DkhW55Eys3UmzyOhzqqBtLKYUhtB17BfVTNYyy+PNg+w5/0pRkmSMLvu1qTrWLrJk2YhHJsimhRFsUsTVDvaMtmPlbtF40AtjeHDDnb1xXYkj8p7nVcIYYCvzMBI2JQnPVwSYQa+WO5Nm2KgmqT0AOwoncPpCQEG05VRvd5OWa+ZumEatSih47I7EVMX80Ww+tqOmsFrIZ1VJXJJvPWxjwLMdrWuXFJs6roSvye2RC2m4gRV81aj5JEDzUmpiraz1ZQea3jO1z1koxfcgwELtU33sWVGH3OvpG2bUNhZqzoBwDyxOnEfYvDlt4/b3SgCq991Cs4K6yXgkVnC403e4IFMnGztYo3m5N+a3Nlkndhj0UC8hbe+ucIhQGihQM40m7q8u9HBRBzwq6JhZWXDRAQGCkpbnvPXapW9HElTAaPNRO7PeaDDxN4513sTSqzIfpzG3wxQFCUfn40xUaq7x/5PXAxXXR7XSBMLN1kb/34B4lswepK8GW0lp+ZrQ0MRvS7MppPiWOnJHnfv8YDL4i/oDa9UC7hzJknMc1xKKGMyYV80vEzoe7scH/ZrS/FjTL+zfzsown0vo/9Egbg3TXoiLlPuJbx3M7y6ukfYLjUFGufMs15mCJI1hw8RQfH0u7T+sxJTwBOf6yqcc71Fj95iD/eAw98i2/4YExMB0KVAP2NcYV+A2QUPAOEJc4t3BAmX+z6dUCavRruxnNiEYnWhs2/wGh7KzxFJB4l8OBWtTiB ltgE9Wio Vsqy6n5X1U33cHsKBwvQCQJgok2hF8ItvDAtV92BdU7JpIcAZmTur7kqIfpLViIYhMUziSsDJmbe3S8nO2ABKoiRfqox3L3xLDZnj3Ze/kUbOjXoGMHDTVWWOvXSKzquW3R4JwL5+6CxdzVTNJ7Sjm0TCuRefzQ9bS2gAPlaCGny4cz3GtF6RzEJHQA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now that the trampoline rodata is aligned to the next PAGE_SIZE_MAX boundary after the end of the trampoline text, the code that maps it in the fixmap is incorrect, because it still assumes the rodata is in the next page immediately after the text. Of course it still works for now with compile-time page size but for boot-time page size when selecting a page size less than PAGE_SIZE_MAX, it will fail. So let's fix that by allocating sufficient fixmap slots to cover the extra alignment padding in the worst case (PAGE_SIZE == PAGE_SIZE_MIN) and explicitly mapping the rodata to the slot offset correctly from the text. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/fixmap.h | 16 +++++++++++----- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/vmlinux.lds.S | 4 +++- arch/arm64/mm/mmu.c | 22 ++++++++++++++-------- 4 files changed, 29 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/fixmap.h b/arch/arm64/include/asm/fixmap.h index 87e307804b99c..9a496d54dfe6e 100644 --- a/arch/arm64/include/asm/fixmap.h +++ b/arch/arm64/include/asm/fixmap.h @@ -59,13 +59,19 @@ enum fixed_addresses { #endif /* CONFIG_ACPI_APEI_GHES */ #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +#define TRAMP_TEXT_SIZE (PAGE_SIZE_MIN * 3) #ifdef CONFIG_RELOCATABLE - FIX_ENTRY_TRAMP_TEXT4, /* one extra slot for the data page */ +#define TRAMP_DATA_SIZE PAGE_SIZE_MIN +#define TRAMP_PAD_SIZE (PAGE_SIZE_MAX - PAGE_SIZE_MIN) +#else +#define TRAMP_DATA_SIZE 0 +#define TRAMP_PAD_SIZE 0 #endif - FIX_ENTRY_TRAMP_TEXT3, - FIX_ENTRY_TRAMP_TEXT2, - FIX_ENTRY_TRAMP_TEXT1, -#define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_TEXT1)) +#define TRAMP_SIZE (TRAMP_TEXT_SIZE + TRAMP_DATA_SIZE + TRAMP_PAD_SIZE) + FIX_ENTRY_TRAMP_END, + FIX_ENTRY_TRAMP_BEGIN = FIX_ENTRY_TRAMP_END + + DIV_ROUND_UP(TRAMP_SIZE, PAGE_SIZE_MIN) - 1, +#define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_BEGIN)) #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ __end_of_permanent_fixed_addresses, diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 40971ac1303f9..252ec58963093 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -21,6 +21,7 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __entry_tramp_rodata_start[], __entry_tramp_rodata_end[]; extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; static inline size_t entry_tramp_text_size(void) diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 1ef6dea13b57c..09fcc234c0f77 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -118,7 +118,9 @@ jiffies = jiffies_64; *(.entry.tramp.text) \ __entry_tramp_text_end = .; \ . = ALIGN(PAGE_SIZE_MAX); \ - *(.entry.tramp.rodata) + __entry_tramp_rodata_start = .; \ + *(.entry.tramp.rodata) \ + __entry_tramp_rodata_end = .; #else #define TRAMP_TEXT #endif diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index a528787c1e550..84df9f278d24d 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -734,25 +734,31 @@ static int __init map_entry_trampoline(void) return 0; pgprot_t prot = kernel_exec_prot(); - phys_addr_t pa_start = __pa_symbol(__entry_tramp_text_start); + phys_addr_t pa_text = __pa_symbol(__entry_tramp_text_start); + phys_addr_t pa_data = __pa_symbol(__entry_tramp_rodata_start); + int slot = FIX_ENTRY_TRAMP_BEGIN; /* The trampoline is always mapped and can therefore be global */ pgprot_val(prot) &= ~PTE_NG; /* Map only the text into the trampoline page table */ memset(tramp_pg_dir, 0, PGD_SIZE); - __create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS, + __create_pgd_mapping(tramp_pg_dir, pa_text, TRAMP_VALIAS, entry_tramp_text_size(), prot, __pgd_pgtable_alloc, NO_BLOCK_MAPPINGS); /* Map both the text and data into the kernel page table */ - for (i = 0; i < DIV_ROUND_UP(entry_tramp_text_size(), PAGE_SIZE); i++) - __set_fixmap(FIX_ENTRY_TRAMP_TEXT1 - i, - pa_start + i * PAGE_SIZE, prot); + for (i = 0; i < DIV_ROUND_UP(entry_tramp_text_size(), PAGE_SIZE); i++) { + __set_fixmap(slot, pa_text, prot); + pa_text += PAGE_SIZE; + slot--; + } - if (IS_ENABLED(CONFIG_RELOCATABLE)) - __set_fixmap(FIX_ENTRY_TRAMP_TEXT1 - i, - pa_start + i * PAGE_SIZE, PAGE_KERNEL_RO); + if (IS_ENABLED(CONFIG_RELOCATABLE)) { + slot -= (pa_data - pa_text) / PAGE_SIZE; + VM_BUG_ON(slot < FIX_ENTRY_TRAMP_END); + __set_fixmap(slot, pa_data, PAGE_KERNEL_RO); + } return 0; }